Thursday, July 18, 2019

Problem with Copy-PnPFile cmdlet and File Not Found error

Sharepoint PnP PowerShell library has many useful commands which simplify scripting against Sharepoint Online. One of them is Copy-PnPFile which allows to copy file from one document library to another. Source and target doclibs may be even located in different site collections in the same tenant.

One of example of Copy-PnPFile (basically 1st example) says that it is possible to copy single file like that:

Copy-PnPFile -SourceUrl Documents/company.docx -TargetUrl /sites/otherproject/Documents/company.docx

Unfortunately currently it gives File not found error. Looks like there is a bug in Copy-PnPFile which prevents it from working correctly – it is also described in this StackOverflow thread which was created couple of days ago: Copy-PnPFile returns File Not Found.

The only working way which I’ve found so far is to copy whole root folder:

Copy-PnPFile -SourceUrl Documents -TargetUrl /sites/otherproject/Documents -SkipSourceFolderName

It will copy all files from Documents doclib to Documents doclib on another site collection /sites/otherproject. However it will also try to copy OTB list view AllItems.aspx as last file and it will give error that AllItems.aspx already exists in target doclib. In order to ignore this error I used the following solution:

$error = $null
Copy-PnPFile -SourceUrl Documents -TargetUrl /sites/otherproject/Documents -SkipSourceFolderName -ErrorAction SilentlyContinue -ErrorVariable error
if ($error -and !$error.Exception.Message.ToLower().Contains("allitems.aspx")) {
    throw $error
}

I.e. it will throw error only if message doesn’t contain allitems.aspx occurrence. Hope that it will help someone.

Monday, July 15, 2019

Problem with not editable host.json file for Azure function app

Some time we need to manually modify content of host.json file of Azure function app (e.g. to change logging settings). In order to do that you need to go to Azure function app > Platform features > Function app settings > host.json. However you may face with situation that textarea which stores content of host.json will be readonly:

In this case change Function app edit mode setting on the same page from “Read only” to “Read/Write”. After that you will be able to edit content of host.json file for you Azure function app:

Friday, July 12, 2019

Get all Azure AD groups where current user is a member transitively via Graph API


As you probably know we may get all groups where user is member using memberOf endpoint:

GET /users/{id | userPrincipalName}/memberOf

This endpoint returns only those groups where user was added as direct member. I.e. if user was added to GroupA and this GroupA was then added to GroupB – it will return only GroupA but not GruopB. However we often need to get all groups where user is member transitively. Fortunately it is also possible with another endpoint getMemberGroups:

POST /users/{id | userPrincipalName}/getMemberGroups

Until recently it was available only in Graph API itself – not in .Net Graph client library. Fortunately starting with 1.16 version Microsoft.Graph.User class got new property TransitiveMemberOf propery:

Using this property we may get all groups where user is member transitively. It supports paging so in order to get all groups we also need to iterate through pages. Here is the code example which does that:

private static List<Guid> GetUserGroupsTrasitively(string userPrincipalName)
{
 try
 {
  var graph = new GraphServiceClient(new AzureAuthenticationProvider());
  var groups = graph.Users[userPrincipalName].TransitiveMemberOf.Request().GetAsync().Result;
  if (groups == null)
  {
   return new List<Guid>();
  }

  var result = new List<Guid>();
  while (groups.Count > 0)
  {
   foreach (var group in groups)
   {
    result.Add(new Guid(group.Id));
   }

   if (groups.NextPageRequest != null)
   {
    groups = groups.NextPageRequest.GetAsync().Result;
   }
   else
   {
    break;
   }
  }

  return result;
 }
 catch (Exception x)
 {
  // error handling
 }
}

Monday, July 1, 2019

Sharepoint MVP 2019

I got very exciting email from MS today that I’ve got MVP award in Office Apps & Services category. Although it is 9th award for me it definitely has own place in my professional life and I’m very glad that MS recognizes my contribution to community life with this award. Last year a lot of work was done related with MS Graph API and OfficeDevPnP. Many issues, workarounds and solutions were discussed during last year on forums, in blog posts comments, github issues, etc. Also I continue maintenance of Camlex library which simplifies creation of dynamic CAML queries for developers. Nowadays new technologies appear very often and they allow us to do such things which were not possible before. This is great but developers’ life don’t become easier because of that – we get new and new challenges in our work every day. From this perspective community role is crucial – I can’t say how many times I by myself found solutions for technical challenges in blog posts, forums, code samples, etc. Knowing on practice how important this work is I also try to share my findings, ideas and solutions with community. Thank you MS and thank you dear readers of my blog for being with me this year. Looking forward for the new year with it’s own interesting challenges and inventive solutions.

Tuesday, June 25, 2019

How to get localized field titles via CSOM in Sharepoint

Some time ago I wrote article which shows how to localize web part titles via CSOM. If you are not familiar with it I recommend to read it before continue as it has useful information about Sharepoint MUI feature in general: Localize web part titles via client object model in Sharepoint. In current post I will show how to get localized field titles via CSOM in Sharepoint. This technique can be used both in on-prem and online versions.

Currently CSOM has Field.TitleResource property and it looks suitable when you want to get localized title of some field. However this is not the case. In order to get localized fields’ titles you still have to use Field.Title property but with little tuning of ClientContext – similar to those which is mentioned in the article above. More specifically you need to specify target language via “Accept-Language” HTTP header associated with ClientContext and then request field titles (of course assuming that fields have these localized titles provisioned. See e.g. Provision multilingual sites with PnP templates to see how to provision multilingual fields’ titles using PnP templates). Here is the code which shows this concept:

string siteUrl = "...";
string clientId = "...";
string clientSecret = "...";
int lcid = ...;
using (var ctx = new OfficeDevPnP.Core.AuthenticationManager().GetAppOnlyAuthenticatedContext(siteUrl, clientId, clientSecret))
{
 ctx.PendingRequest.RequestExecutor.WebRequest.Headers["Accept-Language"] = new CultureInfo(lcid).Name;
 ctx.Load(ctx.Web);
 ctx.Load(ctx.Web.Fields, f => f.Include(c => c.Id, c => c.Title));
 ctx.ExecuteQueryRetry();
 ...
}

As result when you will iterate through site columns retrieved this way they will contain titles localized for language specified with lcid parameter.

Thursday, June 13, 2019

Use pagination with Sharepoint search API

Often search request return a lot of data. It may be insufficient to show all this data at once on the page – performance may suffer, page may be overloaded with data, etc. In order to address these issues we may use pagination i.e. get data by chunks. But how it often happens with Sharepoint there are own considerations related with pagination in search API.

If we will check documentation of SharePoint Search REST API we will find several properties which affect pagination logic:

  • StartRow: The first row that is included in the search results that are returned. You use this parameter when you want to implement paging for search results.
  • RowLimit: The maximum number of rows overall that are returned in the search results. Compared to RowsPerPage, RowLimit is the maximum number of rows returned overall.
  • RowsPerPage: The maximum number of rows to return per page. Compared to RowLimit, RowsPerPage refers to the maximum number of rows to return per page, and is used primarily when you want to implement paging for search results.

So based on this description we may assume that most obvious way to get paginated data is to use StartRow and RowsPerPage:

var searchUrl = "http://{tenant}.sharepoint.com/_api/search/query?querytext='" + query + "'&selectproperties='Title'&startRow=" + startRow + "&rowsPerPage=" + pageSize;

It will work but with one condition: if page size is less than default page size which is 10 items per page. If page size is greater than default page size it won’t work: in this case you have to use RowLimit which will work as page size even though documentation says different:

var searchUrl = "http://{tenant}.sharepoint.com/_api/search/query?querytext='" + query + "'&selectproperties='Title'&startRow=" + startRow + "&rowLimit=" + pageSize;

This approach will allow to implement pagination with page size bigger than default search page size (10 items per page).

Thursday, June 6, 2019

Grant permissions and trust SharePoint app automatically via PowerShell

In Sharepoint app model we may need to grant permissions to Sharepoint app on AppInv.aspx page by providing appropriate permissions request xml. If permissions are granted on Tenant level you need to open AppInv.aspx in context of Central admin i.e. https://{tenant}-admin.sharepoint.com:

It was historically quite painful to automate this process as automatic permissions grant is not currently possible. There were attempts to automate O365 login and automate trust process using COM automation in PowerShell (using New-Object -com internetexplorer.application): https://github.com/wulfland/ScriptRepository/blob/master/Apps/Apps/Deploy-SPApp.ps1. With this approach script opens AppInv.aspx page and simulates user’s input.

However O365 login experience was changed since this script was implemented and there is no guarantee that it won’t be changed further. Also there may be several login scenarios:

  • user may be already logged in if chose Remember credentials during previous login
  • user may use MFA with SMS, authenticator app or something else which will make login automation even more complicated

Keeping that in mind I implemented the following semi-automatic way of granting app permissions and trust the app:

1. app is registered in Azure AD via PowerShell (in Sharepoint Online it is not necessary to register app which will be used for communicating with Sharepoint via AppRegNew.aspx. You may also register it in Azure Portal > App Registrations). See e.g. Create an Azure Active Directory Application and Key using PowerShell for example

2. Then script opens AppInv.aspx page in IE (using Start-Process cmdlet) and asks user to authenticate him/herself manually. After that user returns to the script and clicks Enter – all other steps (grant permissions and trust the app) are performed by the following PowerShell script:

function Trust-SPAddIn {
    [CmdletBinding(SupportsShouldProcess=$true)]
    [OutputType([int])]
    Param
    (
        [Parameter(Mandatory=$true, Position=0)]
        [string]$AppInstanceId,

        [Parameter(Mandatory=$true, Position=1)]
        [string]$WebUrl,

        [parameter(Mandatory=$true, Position=2)] 
        [string]$UserName, 

        [parameter(Mandatory=$true, Position=3)] 
        [string]$Password
    )

    $ie = New-Object -com internetexplorer.application
    try {
  Log-Warn ("Script will now open $WebUrl. Please authenticate yourself and wait until Admin Center home page will be loaded.")
  Log-Warn ("After that leave Admin Center window opened (don't close it), return to the script and follow provided instructions.")
  Log-Warn ("In case you are already signed in Admin Center window will be opened without asking to login. In this case wait until Admin Center window will be loaded, leave it opened and return to the script.")
  if (-not $silently) {
   Log-Warn ("Press Enter to open $WebUrl...")
   Read-Host
  }
 
        $ie.Visible = $true
        $ie.Navigate2($WebUrl)
  
  if (-not $silently) {
   Log-Warn ("Wait until Admin Center window will be fully loaded and press Enter to continue installation")
   Log-Warn ("Don't close Admin Center window - script will close it automatically")
   Read-Host
  }
  
  $authorizeURL = "$($WebUrl.TrimEnd('/'))/_layouts/15/appinv.aspx"
  Log-Info ("Open $authorizeURL...")
  $ie.Visible = $false
  $ie.Navigate2($authorizeURL)
  WaitFor-IEReady $ie -initialWaitInSeconds 3

  Log-Info ("Grant permissions to the app...")
  $appIdInput = $ie.Document.getElementById("ctl00_ctl00_PlaceHolderContentArea_PlaceHolderMain_IdTitleEditableInputFormSection_ctl01_TxtAppId")
  $appIdInput.value = $AppInstanceId
  $lookupBtn = $ie.Document.getElementById("ctl00_ctl00_PlaceHolderContentArea_PlaceHolderMain_IdTitleEditableInputFormSection_ctl01_BtnLookup")
  $lookupBtn.Click()
  WaitFor-IEReady $ie -initialWaitInSeconds 3
  Log-Info ("Step 1 of 2 done")
  $appIdInput = $ie.Document.getElementById("ctl00_ctl00_PlaceHolderContentArea_PlaceHolderMain_TitleDescSection_ctl01_TxtPerm")
  $appIdInput.value = '<AppPermissionRequests AllowAppOnlyPolicy="true"><AppPermissionRequest Scope="http://sharepoint/content/tenant" Right="FullControl" /></AppPermissionRequests>'
  $createBtn = $ie.Document.getElementById("ctl00_ctl00_PlaceHolderContentArea_PlaceHolderMain_ctl01_RptControls_BtnCreate")
  $createBtn.Click()
  WaitFor-IEReady $ie -initialWaitInSeconds 3
  Log-Info ("Step 2 of 2 done")

  Log-Info ("Trust the app...")
  $trustBtn = $ie.Document.getElementById("ctl00_ctl00_PlaceHolderContentArea_PlaceHolderMain_BtnAllow")
  $trustBtn.Click()
  WaitFor-IEReady $ie -initialWaitInSeconds 3

  Log-Info ("All steps are done")
    }
    finally {
        $ie.Quit()
    } 
}

WaitFor-IEReady helper method is given from original script mentioned above so credits go to it’s author:

function WaitFor-IEReady {
    [CmdletBinding()]
    Param
    (
        [Parameter(Mandatory=$true, ValueFromPipelineByPropertyName=$true, Position=0)]
        $ie,

        [Parameter(Mandatory=$false, Position=1)]
        $initialWaitInSeconds = 1
    )

    sleep -Seconds $initialWaitInSeconds

    while ($ie.Busy) {

        sleep -milliseconds 50
    }
}

Log-Info and Log-Warn are basic logger methods and you may implement them as needed for your scenario. Since we delegated login to the end user we don’t need to handle different O365 login scenarios and script is greatly simplified, e.g. there is no need to perform javascript activities which work not very stable via COM automation.

Monday, May 20, 2019

One reason why SPFarm.CurrentUserIsAdministrator may return false when current user is farm administrator

Sharepoint farm administrators are powerful users who may perform administrative actions in Sharepoint farm (see SharePoint Farm Administrator account for more details). Sometime you may need to check whether or not current user is farm admin in order to allow or disallow specific actions. You may do that using SPFarm.CurrentUserIsAdministrator method. The problem however that it will work as expected only in context of Central administration web application (i.e. if page which calls this method is running in Central administration). If you will try to call this method from regular Sharepoint web application it will always return false even if current user is added to Farm administrations group on your farm - you may add users to Farm administrators using Central administration > Manage the farm administrators group:

In this case (when you need to call SPFarm.CurrentUserIsAdministrator from regular Sharepoint web application) you have to use overridden version of this method with boolean parameter:

allowContentApplicationAccess
true to make the check work in the content port of a Web application; otherwise, false.

i.e. like this:

bool isFarmAdmin = SPFarm.Local.CurrentUserIsAdministrator(true);

In this case method will work as expected i.e. return true if user is member of Farm administrators group and false otherwise.

Tuesday, May 7, 2019

Maximum alerts limit per user in Sharepoint

When you try to subscribe for alerts to specific user account in Sharepoint and get the following error:

You have created the maximum number of alerts allowed for this site

it may be so that you have reached alerts limit for this user in current site. This per-web application setting and it is possible to extend this limit in Central administration > Manage web applications > web app > General settings > Alerts:

By default it is set to 500 alerts per user. After you will increase this limit error should disappear.

Friday, April 26, 2019

Provision multilingual sites with PnP templates

Sharepoint PnP Powershell library (which is also available on Nuget) has own provisioning engine. It is quite powerful engine which allows to provision MUI sites (although it has own problems with stability). In order to do that you need to use several basic elements:

1. Have all literals which should be localized in resx file

2. Specify supported languages inside <pnp:SupportedUILanguages>…</pnp:SupportedUILanguages> section of PnP template. After provisioning these languages will be set in Site settings > Language settings > Alternate languages. You need to have resx file for each supported language in standard resources.xx-XX.resx format

3. Resx files with translations should be specified inside <pnp:Localizations>…</pnp:Localizations> section of PnP template. Note that PnP will provision all these languages to your site regardless of what alternate languages are set in Site settings > Language settings on the moment when this template is applied to target site

4. For localized strings use {resource:ResourceKey} format inside template

Here is example of how PnP template for MUI site provisioning may look like:

<?xml version="1.0"?>
<pnp:Provisioning xmlns:pnp="http://schemas.dev.office.com/PnP/2018/01/ProvisioningSchema">
  <pnp:Localizations>
 <pnp:Localization LCID="1033" Name="English" ResourceFile="resources.en-US.resx"/>
    <pnp:Localization LCID="1031" Name="German" ResourceFile="resources.de-DE.resx"/>
  </pnp:Localizations>
  <pnp:Templates ID="Test-Container">
    <pnp:ProvisioningTemplate ID="Test" Version="1">

      <pnp:SupportedUILanguages>
        <pnp:SupportedUILanguage LCID="1033" />
        <pnp:SupportedUILanguage LCID="1031" />
      </pnp:SupportedUILanguages>

   <pnp:SiteFields xmlns:pnp="http://schemas.dev.office.com/PnP/2018/01/ProvisioningSchema">
  <Field ID="..." Name="MyField" DisplayName="{resource:MyFieldTitle}" Type="Text" Group="Custom" SourceID="..." StaticName="MyField"></Field>
   </pnp:SiteFields>
   
    </pnp:ProvisioningTemplate>
  </pnp:Templates>
</pnp:Provisioning>

It will provision file with English and German UI languages.

Thursday, April 18, 2019

Can’t get groups created from MS Teams from Graph endpoint /beta/me/joinedGroups

With MS Graph API you may use /beta/me/joinedGroups endpoint for getting list of groups where current user is a member. With the same endpoint you may also get isFavorite attribute for the group which shows whether or not user added group to favorites. However this endpoint has own issues: recently we found that it doesn’t return groups which were created from MS Teams: when you create new Team there also related Group is created. It is possible to get details of this group using basic groups endpoint

https://graph.microsoft.com/v1.0/groups/{id}

But if you will try to get list of user’s groups via beta endpoint such groups created from MS Teams won’t be returned:

https://graph.microsoft.com/beta/me/joinedgroups/?$select=id,isfavorite,displayName&$top=200

One possible explanation could be that internally /me/joinedgroups end point is routed to Outlook services which is not integrated with Teams well enough yet: when I tried to add createdDateTime attribute to the REST url (this attribute is returned for groups from basic endpoint - see above)

https://graph.microsoft.com/beta/me/joinedgroups/?$select=id,isfavorite,displayName,createdDateTime &$top=200

it returned error saying that returned entities have Microsoft.OutlookServices.Group type:

May be this is a bug or such functionality is not implemented in beta endpoint yet. For now I asked this question in StackOverflow – hope that somebody from MS Graph product team will answer it.

Friday, April 5, 2019

Use Sharepoint search API using HTTP POST requests

As you probably know it is possible to use Sharepoint search API programmatically by calling /_api/search/query endpoint with HTTP GET and provide KQL query and details (like selected managed properties) in query string. Popular Search Query Tool uses the same technique. However query string approach has own limitation, e.g. max 4kb length limit which is common for ASP.Net applications. If your query is built dynamically and you don’t know the actual length on compile time you may use search API with HTTP POST and provide KQL query in request body. If you want to use search API with HTTP POST you need to use slightly different endpoint: /_api/search/postquery. Let’s see how it works in Postman tool.

The first thing which we need to get is to obtain access token. It can be done by sending another HTTP POST request to the following address: https://accounts.accesscontrol.windows.net/{tenant_name}.onmicrosoft.com/tokens/OAuth/2. In request body we need to provide several parameters which are described in the following list:

  • grant_type = client_credentials
  • client_id – client id of your Sharepoint app (you should register it in advance using /_layouts/15/appregnew.aspx and then grant appropriate permissions using /_layouts/15/appinv.aspx) in the following form {cliend_id}@{tenant_id}. You may check tenant id in Azure portal > Azure Active Directory > Properties > Directory ID
  • client_secret – client secret of your Sharepoint app
  • resource – should have value in the form 00000003-0000-0ff1-ce00-000000000000/{tenant_name}.onmicrosoft.com@{tenant_id}

so request should look like this in Postman:

If everything was configured properly you should get success response which should contain access_token in the response body. Copy it’s value – it will be needed on the next step.

Now we are ready to send search POST requests to search API endpoint. We will use https://{tenant_name}.sharepoint.com/_api/search/postquery address for that. Let’s get list of all sites using the following KQL query:

contentclass:STS_Site

Request body should look like this:

{
   "request":{
      "Querytext":"contentclass:STS_Site",
      "RowLimit":100,
      "SelectProperties":{
         "results":[
            "Title",
            "SiteID",
            "OriginalPath"
         ]
      }
   }
}

After that switch to Authorization type and select Type = Bearer Token and specify value of access_token which was obtained on the previous step:

On the Headers tab add Accept and Content-type headers with “application/json;odata=verbose” (note that it is important to specify these headers exactly like this: if you will specify different odata version request may return error because parameters schema may be different.):

If everything was done properly when you will execute POST request in Postman you will get list of Sharepoint sites in your tenant.

Wednesday, March 13, 2019

Get login name of special group “Everyone except external users” programmatically in Sharepoint

In Sharepoint Online you may assign permissions to all employees of your organization using special group “Everyone except external users”. In order to add permissions to this group programmatically we need to know login name of the appropriate object in Sharepoint object model. In this article I will show how to get login name of this special group programmatically.

The main difficulty is that login name of “Everyone except external users” group is different per tenant. But the good thing is that it is built using known rule:

c:0-.f|rolemanager|spo-grid-all-users/{realm}

where instead of {realm} placeholder you need to use realm for your tenant. We can get realm using TokenHelper.GetRealmFromTargetUrl() method. So code will look like this:

protected virtual string GetEveryoneExceptExternalsLoginName(string siteUrl)
{
 var realm = TokenHelper.GetRealmFromTargetUrl(new Uri(siteUrl));
 return string.Format("c:0-.f|rolemanager|spo-grid-all-users/{0}", realm);
}

public static string GetRealmFromTargetUrl(Uri targetApplicationUri)
{
 #if ONPREMISES
 if (targetApplicationUri.Scheme.ToLower() == "https")
  ServicePointManager.ServerCertificateValidationCallback = new RemoteCertificateValidationCallback(delegate { return true; });
 #endif

 WebRequest request = WebRequest.Create(targetApplicationUri + "/_vti_bin/client.svc");
 request.Headers.Add("Authorization: Bearer ");

 try
 {
  using (request.GetResponse())
  {
  }
 }
 catch (WebException e)
 {
  if (e.Response == null)
  {
   return null;
  }

  string bearerResponseHeader = e.Response.Headers["WWW-Authenticate"];
  if (string.IsNullOrEmpty(bearerResponseHeader))
  {
   return null;
  }

  const string bearer = "Bearer realm=\"";
  int bearerIndex = bearerResponseHeader.IndexOf(bearer, StringComparison.Ordinal);
  if (bearerIndex < 0)
  {
   return null;
  }

  int realmIndex = bearerIndex + bearer.Length;

  if (bearerResponseHeader.Length >= realmIndex + 36)
  {
   string targetRealm = bearerResponseHeader.Substring(realmIndex, 36);

   Guid realmGuid;

   if (Guid.TryParse(targetRealm, out realmGuid))
   {
    return targetRealm;
   }
  }
 }
 return null;
}

After that you will be able to grant permissions to “Everyone except external users” programmatically.

Wednesday, March 6, 2019

Strange behavior of Sharepoint Online when specify different users as primary site collection administrator

When you create new site collection (doesn’t matter modern or classic) using Tenant.CreateSite() method you need to specify primary site collection administrator in SiteCreationProperties.Owner property. So if you would check site collection administrators of the newly created site you would expect to see specified user there. However it is not always the case. Sometimes Sharepoint really shows specified user there:

But sometimes instead of actual user account there will be Company Administrator – special user which covers all users in the directory with Global Administrator rights (see e.g. Special SharePoint Groups):

It happens regardless of specified user directory role: it may have Global Administrator role and may not have it:

And even more – if user have more directory roles there may be both Company Administrator and Sharepoint Service Administrator:

I.e. looks like Sharepoint get’s user’s roles and assigns permissions to these roles instead of actual user account. But again – for some user accounts it just adds actual user account to Site collection administrators. Logic behind this behavior is not yet clear so if you have any thoughts on that please share it in comments.

Sunday, February 24, 2019

Camlex 5.1 and Camlex.Client 3.2 have been released: support for Includes/NotIncludes operations

Good news for Sharepoint developers which use Camlex and Camlex.Client in their work (Camlex is free open source library for simplifying creation of CAML queries in Sharepoint by using C# lambda expressions): today new versions Camlex 5.1 and Camlex.Client 3.2 have been released. In this version support of Include/NotInclude CAML operations was added to the library. Here is what MSDN says about Includes element:

If the specified field is a Lookup field that allows multiple values, specifies that the Value element is included in the list item for the field that is specified by the FieldRef element.

and about NotIncludes:

If the specified field is a Lookup field that allows multiple values, specifies that the Value element is excluded from the list item for the field that is specified by the FieldRef element.

I.e. these operations are used for multi lookup fields. And although there are known workarounds which allows to use another operations to make similar queries (see e.g. CAML query for field of type “Person or Group” which allows multiple selections) I’ve decided to add support of official operations for multiple lookup field types in order to have complete set of CAML operations in Camlex. Let’s see how it works.

In order to create CAML query with Includes operation with Camlex use the following syntax:

string caml = Camlex.Query().Where(x => ((int)x["Foo"]).Includes(1)).ToString();

Here we used Includes extension method defined in Camlex. It will produce the following CAML:

<Where>
  <Includes>
    <FieldRef Name="Foo" />
    <Value Type="Integer">1</Value>
  </Includes>
</Where>

If we need to perform query using lookup id (i.e. add LookupId=”TRUE” attribute to FieldRef element) – use overloaded version of Includes method with boolean parameter and pass true there:

string caml = Camlex.Query().Where(x => ((int)x["Foo"]).Includes(1, true)).ToString();

which will produce:

<Where>
  <Includes>
    <FieldRef Name="Foo" LookupId="True" />
    <Value Type="Integer">1</Value>
  </Includes>
</Where>

The same functionality is also added to Camlex.Client – Camlex version built for CSOM. Both Camlex and Camlex.Client are available in Nuget. You may add them to your projects by using the following commands:

Install-Package Camlex.NET.dll

or for CSOM version:

Install-Package Camlex.Client.dll

Hope that new feature will help in your work.

Saturday, February 9, 2019

Workaround for render error of SPFx web part

If on your Sharepoint site you use SPFx web part (ClientSideWebPart) you may face with the following error:

Failed to render client side web part. Web part … failed to render as manifest is not found

Web Part framework is not loaded

In order to  avoid this problem try the following workaround: go to App catalog, delete web part’s app package from there and upload it again.

Thursday, February 7, 2019

Set DenyAddAndCustomizePages to Disabled for modern Sharepoint sites via PowerShell

Recently we faced with the following problem: when tried to set DenyAddAndCustomizePages property of modern Sharepoint site to Disabled (which means that customizations will be enabled for that site) using PnP PowerShell:

Set-PnPTenantSite -Url $url -NoScriptSite:$false

the following error was thrown:

Error in proc_GetSitesAllowDenyList, no rowset returned

In order to fix it you have to call Connect-PnPOnline for tenat’s admin center (https://{tenant}-admin.sharepoint.com) not to target site itself. Also using CSOM version works more stable than shown Set-PnPTenantSite. Here is the final code:

$adminUrl = $tenantUrl.Replace(".sharepoint", "-admin.sharepoint")
$adminConnection = Ensure-PnPConnection $adminUrl
$pnpSite = Get-PnPTenantSite -Url $url -Detailed -Connection $adminConnection
$pnpSite.DenyAddAndCustomizePages = "Disabled"
$pnpSite.Update()
$pnpSite.Context.ExecuteQuery()

After that error should gone.

Friday, February 1, 2019

How to fix Sharepoint search crawler when it stucks on Crawling full

If you use search on your Sharepoint site you may face with the issue that search crawler stuck in Crawling full status. In the logs you may find the following error:

Failed to create session with indexer ---> Microsoft.Ceres.SearchCore.Services.ContentRouter.ContentException: Unable to connect to index system

In this case try to execute the following PowerShell cmdlets:

$ssa = Get-SPEnterpriseSearchServiceApplication
Get-SPEnterpriseSearchStatus -Text -SearchApplication $ssa

It should show all components of search topology in Active state:

If there will be components which state is Degraded – your topology is in wrong state. In this case solution is to recreation of Search service application. After that run full crawl and it should work and stop successfully this time.

Tuesday, January 29, 2019

Sync delay between O365 Group owners and members with Team owners and members

When you create O365 group and add team to this group you may face with the following issue: group owners and members which are added via Azure AD portal (https://portal.azure.com) are synced to team owners and members with delay. Let’s say we created group and added team to the group. During creation we specified 1 user as group owner:

If we will check now Team for this group > Manage team > Owners we will see that Owners list will be initially empty:

After some time group owner will appear in team owners:

(Note that there is still 0 owners count in parenthesis – but this is another issue. Probably bug in current version of Teams web app). Same happens also for group members and team members. This sync delay may take several hours.

In this forum thread there is suggests to use beta API for adding owners to group, i.e.:

https://graph.microsoft.com/beta/groups/{groupId}/owners/$ref

instead of

https://graph.microsoft.com/v1.0/groups/{groupId}/owners/$ref

But the point is that even with old API owners/members will be synchronized between group and team but after some time.

Saturday, January 26, 2019

Save tenant storage entity from PowerShell using app permissions in Sharepoint Online

Tenant storage entities (or tenant properties) allow to store tenant-wide properties which will be accessible from all site collections across tenant. It is often needed in installation PowerShell scripts to save some common settings to tenant storage entities. Also we may need to do that using app permissions i.e. using app id and secret instead of credentials of specific user. In this post I will show the method which may be used for that.

At first let’s try the most obvious way using PnP PowerShell cmdlets. There is overload of Connect-PnPOnline cmdlet which connects to Shrepoint Online using app id and secret instead of user credentials:

PS C:\> Connect-PnPOnline -Url https://{tenant}.sharepoint.com/sites/AppCatalog -AppId $appId -AppSecret $appSecret
PS C:\> Set-PnPStorageEntity -Key "foo" -Value "bar"

However it will give Access denied error even if app has FullControl permissions on tenant level:

Note that in this example I connected to the app catalog site for setting storage entity (below it is described why). But there will be also the same error if you will try to connect to the tenant’s root site or to tenant admin center.

Now let’s try use CSOM and OfficeDevPnP assemblies directly in PowerShell:

$csomDir = ...
$pnpCoreDir = ...
$url = "https://{tenant}.sharepoint.com/sites/test"
$appId = ...
$appSecret = ...
[System.Reflection.Assembly]::LoadFile([System.IO.Path]::Combine($csomDir, "Microsoft.SharePoint.Client.Runtime.dll"))
[System.Reflection.Assembly]::LoadFile([System.IO.Path]::Combine($csomDir, "Microsoft.SharePoint.Client.dll"))
[System.Reflection.Assembly]::LoadFile([System.IO.Path]::Combine($csomDir, "Microsoft.Online.SharePoint.Client.Tenant.dll"))
[System.Reflection.Assembly]::LoadFile([System.IO.Path]::Combine($pnpCoreDir, "OfficeDevPnP.Core.dll"))

function Get-App-Catalog-Url {
    param (
        $ctx
    )
    $tenantSettings = [Microsoft.SharePoint.Client.TenantSettings]::GetCurrent($ctx)
    Load-CSOMProperties -object $tenantSettings -propertyNames @("CorporateCatalogUrl")
    $ctx.ExecuteQuery()
    return $tenantSettings.CorporateCatalogUrl
}

$am = New-Object OfficeDevPnP.Core.AuthenticationManager
$ctx = $am.GetAppOnlyAuthenticatedContext($url, $appId, $appSecret)
$appCatalogUrl = Get-App-Catalog-Url $ctx

$appCtx = $am.GetAppOnlyAuthenticatedContext($appCatalogUrl, $appId, $appSecret)
$appCtx.Web.SetStorageEntity("foo", "bar", "", "");
$appCtx.ExecuteQuery()

At first we load CSOM and OfficeDevPnP.Core assemblies. Then we need to connect to some existing site in our tenant and get app catalog url programmatically using CSOM. After that we create ClientContext using app catalog url and call Web.SetStorageEntity method. This is the trick because if you will try to call this method with any other web site you will get the same Access denied error. Also in this example we used great Load-CSOMProperties helper method from Gary Lapointe which allows to load object with properties using CSOM in PowerShell. It can be found here: https://gist.github.com/glapointe/cc75574a1d4a225f401b.

After you will run this code it will update tenant storage entity using app permissions.


Friday, January 4, 2019

Difference between fields InternalName and StaticName in Sharepoint

If you work with Sharepoint you probably faced already with fields xml definitions and probably know that field has 3 different names:

  • Title/TitleResource (display name)
  • InternalName
  • StaticName

Title is quite obvious: this is the user friendly display name. But what about InternalName and StaticName? In most cases they are the same but there are situations when they may be different. In this post I will show such situation.

Let’s create new site content type TestContenType and add new Text site column with long name. Length of the field name should be greater than 32 symbols. We won’t use spaces and non-latin symbols for simplicity. In my test I used name TextFieldWithVeryVeryLongInternalName which has 37 symbols:

image

If we will check schema xml of this content type and this field we will see something like that (I used Sharepoint Online client browser for checking xml schema’s):

image

(below it will be explained why field link TextFieldWithVeryVeryLongInternalName is highlighted)

Here is content type’s xml definition:

<ContentType ID="0x0100CD88C368C93C7D4D89E7D60F478C2BF0" Name="TestContentType" Group="Custom Content Types" Version="2">
    <Folder TargetName="_cts/TestContentType" />
    <FieldRefs>
        <FieldRef ID="{c042a256-787d-4a6f-8a8a-cf6ab767f12d}" Name="ContentType" />
        <FieldRef ID="{fa564e0f-0c70-4ab9-b863-0177e6ddd247}" Name="Title" Required="TRUE" ShowInNewForm="TRUE" ShowInEditForm="TRUE" />
        <FieldRef ID="{947d0169-b8c1-417b-aba9-891357b7d4d5}" Name="TextFieldWithVeryVeryLongInternalName" />
    </FieldRefs>
</ContentType>

and here is field’s xml definition:

<Field Type="Text"
       DisplayName="TextFieldWithVeryVeryLongInternalName"
	   Required="FALSE"
	   EnforceUniqueValues="FALSE"
	   Indexed="FALSE"
	   MaxLength="255"
	   Group="Custom Columns"
	   ID="{947d0169-b8c1-417b-aba9-891357b7d4d5}"
	   SourceID="{d754b6da-6897-4a5a-8d73-49abb52cb20a}"
	   StaticName="TextFieldWithVeryVeryLongInternalName"
	   Name="TextFieldWithVeryVeryLongInternalName"
	   Version="1"
	   Customization="" />

As you can see both in content type and field xml definition all names (Title, InternalName, StaticName) are set to original value “TextFieldWithVeryVeryLongInternalName”. It gives us first important finding: site columns may have InternalNames which length is greater than 32 symbols.

Now let’s create new list and add our TestContentType to this list. As you probably know in this case Sharepoint creates new List content type which has the same name as original site content type but is not equal to it – it inherits original site content type instead. First of all content type id will be longer: it will inherit original content type id and add guid after “00”:

{parent content typeid}00{guid without dashes}

i.e. in our case content type id of list content type will be:

0x0100CD88C368C93C7D4D89E7D60F478C2BF000{guid without dashes}

(see below). But this is not the only difference. Let’s see what happened with FieldLink on our long name field:

image

See the difference with the site content type? List content type has field link with exactly 32 symbols: TextFieldWithVeryVeryLongInterna. Let’s check xml definitions of list content type and list field:

<ContentType ID="0x0100CD88C368C93C7D4D89E7D60F478C2BF0006A178989905DF348B5D2BD81EF28374A" Name="TestContentType" Group="Custom Content Types" Version="1">
    <Folder TargetName="TestContentType" />
    <Fields>
        <Field ID="{c042a256-787d-4a6f-8a8a-cf6ab767f12d}"
			Type="Computed"
			DisplayName="Content Type"
			Name="ContentType"
			DisplaceOnUpgrade="TRUE"
			RenderXMLUsingPattern="TRUE"
			Sortable="FALSE"
			SourceID="http://schemas.microsoft.com/sharepoint/v3"
			StaticName="ContentType"
			Group="_Hidden"
			PITarget="MicrosoftWindowsSharePointServices"
			PIAttribute="ContentTypeID"
			FromBaseType="TRUE">
            <FieldRefs>
                <FieldRef Name="ContentTypeId" />
            </FieldRefs>
            <DisplayPattern>
                <MapToContentType>
                    <Column Name="ContentTypeId" />
                </MapToContentType>
            </DisplayPattern>
        </Field>
        <Field ID="{fa564e0f-0c70-4ab9-b863-0177e6ddd247}"
			Type="Text"
			Name="Title"
			DisplayName="Title"
			Required="TRUE"
			SourceID="http://schemas.microsoft.com/sharepoint/v3"
			StaticName="Title"
			FromBaseType="TRUE"
			ColName="nvarchar1"
			ShowInNewForm="TRUE"
			ShowInEditForm="TRUE" />
        <Field Type="Text"
			DisplayName="TextFieldWithVeryVeryLongInternalName"
			Required="FALSE"
			EnforceUniqueValues="FALSE"
			Indexed="FALSE"
			MaxLength="255"
			Group="Custom Columns"
			ID="{947d0169-b8c1-417b-aba9-891357b7d4d5}"
			SourceID="{d754b6da-6897-4a5a-8d73-49abb52cb20a}"
			StaticName="TextFieldWithVeryVeryLongInternalName"
			Name="TextFieldWithVeryVeryLongInterna"
			Version="1"
			Customization=""
			ColName="nvarchar16"
			RowOrdinal="0" />
    </Fields>
</ContentType>

Pay attention on TextFieldWithVeryVeryLongInternalName field. Title and StaticName are still set to original value (TextFieldWithVeryVeryLongInternalName) but InternalName is now trimmed to 32 symbols TextFieldWithVeryVeryLongInterna. Also notice that field ID is the same as in original site column:

{947d0169-b8c1-417b-aba9-891357b7d4d5}

In list field’s xml definition we will see the same:

<Field Type="Text"
       DisplayName="TextFieldWithVeryVeryLongInternalName"
	   Required="FALSE"
	   EnforceUniqueValues="FALSE"
	   Indexed="FALSE"
	   MaxLength="255"
	   Group="Custom Columns"
	   ID="{947d0169-b8c1-417b-aba9-891357b7d4d5}"
	   SourceID="{d754b6da-6897-4a5a-8d73-49abb52cb20a}"
	   StaticName="TextFieldWithVeryVeryLongInternalName"
	   Name="TextFieldWithVeryVeryLongInterna"
	   Version="1"
	   Customization=""
	   ColName="nvarchar16"
	   RowOrdinal="0" />

So this is the scenario when field’s InternalName and StaticName may be different: when you try to add site column which InternalName’s length is greater than 32 symbols to the list.

In addition to that I would also mention that if you would try to add more fields which have equal first 32 symbols in InternalName Sharepoint will add integer suffix to ensure that InternalName is still unique: TextFieldWithVeryVeryLongIntern0, TextFieldWithVeryVeryLongIntern1, etc.