Thursday, June 13, 2019

Use pagination with Sharepoint search API

Often search request return a lot of data. It may be insufficient to show all this data at once on the page – performance may suffer, page may be overloaded with data, etc. In order to address these issues we may use pagination i.e. get data by chunks. But how it often happens with Sharepoint there are own considerations related with pagination in search API.

If we will check documentation of SharePoint Search REST API we will find several properties which affect pagination logic:

  • StartRow: The first row that is included in the search results that are returned. You use this parameter when you want to implement paging for search results.
  • RowLimit: The maximum number of rows overall that are returned in the search results. Compared to RowsPerPage, RowLimit is the maximum number of rows returned overall.
  • RowsPerPage: The maximum number of rows to return per page. Compared to RowLimit, RowsPerPage refers to the maximum number of rows to return per page, and is used primarily when you want to implement paging for search results.

So based on this description we may assume that most obvious way to get paginated data is to use StartRow and RowsPerPage:

var searchUrl = "http://{tenant}.sharepoint.com/_api/search/query?querytext='" + query + "'&selectproperties='Title'&startRow=" + startRow + "&rowsPerPage=" + pageSize;

It will work but with one condition: if page size is less than default page size which is 10 items per page. If page size is greater than default page size it won’t work: in this case you have to use RowLimit which will work as page size even though documentation says different:

var searchUrl = "http://{tenant}.sharepoint.com/_api/search/query?querytext='" + query + "'&selectproperties='Title'&startRow=" + startRow + "&rowLimit=" + pageSize;

This approach will allow to implement pagination with page size bigger than default search page size (10 items per page).

Thursday, June 6, 2019

Grant permissions and trust SharePoint app automatically via PowerShell

In Sharepoint app model we may need to grant permissions to Sharepoint app on AppInv.aspx page by providing appropriate permissions request xml. If permissions are granted on Tenant level you need to open AppInv.aspx in context of Central admin i.e. https://{tenant}-admin.sharepoint.com:

It was historically quite painful to automate this process as automatic permissions grant is not currently possible. There were attempts to automate O365 login and automate trust process using COM automation in PowerShell (using New-Object -com internetexplorer.application): https://github.com/wulfland/ScriptRepository/blob/master/Apps/Apps/Deploy-SPApp.ps1. With this approach script opens AppInv.aspx page and simulates user’s input.

However O365 login experience was changed since this script was implemented and there is no guarantee that it won’t be changed further. Also there may be several login scenarios:

  • user may be already logged in if chose Remember credentials during previous login
  • user may use MFA with SMS, authenticator app or something else which will make login automation even more complicated

Keeping that in mind I implemented the following semi-automatic way of granting app permissions and trust the app:

1. app is registered in Azure AD via PowerShell (in Sharepoint Online it is not necessary to register app which will be used for communicating with Sharepoint via AppRegNew.aspx. You may also register it in Azure Portal > App Registrations). See e.g. Create an Azure Active Directory Application and Key using PowerShell for example

2. Then script opens AppInv.aspx page in IE (using Start-Process cmdlet) and asks user to authenticate him/herself manually. After that user returns to the script and clicks Enter – all other steps (grant permissions and trust the app) are performed by the following PowerShell script:

function Trust-SPAddIn {
    [CmdletBinding(SupportsShouldProcess=$true)]
    [OutputType([int])]
    Param
    (
        [Parameter(Mandatory=$true, Position=0)]
        [string]$AppInstanceId,

        [Parameter(Mandatory=$true, Position=1)]
        [string]$WebUrl,

        [parameter(Mandatory=$true, Position=2)] 
        [string]$UserName, 

        [parameter(Mandatory=$true, Position=3)] 
        [string]$Password
    )

    $ie = New-Object -com internetexplorer.application
    try {
  Log-Warn ("Script will now open $WebUrl. Please authenticate yourself and wait until Admin Center home page will be loaded.")
  Log-Warn ("After that leave Admin Center window opened (don't close it), return to the script and follow provided instructions.")
  Log-Warn ("In case you are already signed in Admin Center window will be opened without asking to login. In this case wait until Admin Center window will be loaded, leave it opened and return to the script.")
  if (-not $silently) {
   Log-Warn ("Press Enter to open $WebUrl...")
   Read-Host
  }
 
        $ie.Visible = $true
        $ie.Navigate2($WebUrl)
  
  if (-not $silently) {
   Log-Warn ("Wait until Admin Center window will be fully loaded and press Enter to continue installation")
   Log-Warn ("Don't close Admin Center window - script will close it automatically")
   Read-Host
  }
  
  $authorizeURL = "$($WebUrl.TrimEnd('/'))/_layouts/15/appinv.aspx"
  Log-Info ("Open $authorizeURL...")
  $ie.Visible = $false
  $ie.Navigate2($authorizeURL)
  WaitFor-IEReady $ie -initialWaitInSeconds 3

  Log-Info ("Grant permissions to the app...")
  $appIdInput = $ie.Document.getElementById("ctl00_ctl00_PlaceHolderContentArea_PlaceHolderMain_IdTitleEditableInputFormSection_ctl01_TxtAppId")
  $appIdInput.value = $AppInstanceId
  $lookupBtn = $ie.Document.getElementById("ctl00_ctl00_PlaceHolderContentArea_PlaceHolderMain_IdTitleEditableInputFormSection_ctl01_BtnLookup")
  $lookupBtn.Click()
  WaitFor-IEReady $ie -initialWaitInSeconds 3
  Log-Info ("Step 1 of 2 done")
  $appIdInput = $ie.Document.getElementById("ctl00_ctl00_PlaceHolderContentArea_PlaceHolderMain_TitleDescSection_ctl01_TxtPerm")
  $appIdInput.value = '<AppPermissionRequests AllowAppOnlyPolicy="true"><AppPermissionRequest Scope="http://sharepoint/content/tenant" Right="FullControl" /></AppPermissionRequests>'
  $createBtn = $ie.Document.getElementById("ctl00_ctl00_PlaceHolderContentArea_PlaceHolderMain_ctl01_RptControls_BtnCreate")
  $createBtn.Click()
  WaitFor-IEReady $ie -initialWaitInSeconds 3
  Log-Info ("Step 2 of 2 done")

  Log-Info ("Trust the app...")
  $trustBtn = $ie.Document.getElementById("ctl00_ctl00_PlaceHolderContentArea_PlaceHolderMain_BtnAllow")
  $trustBtn.Click()
  WaitFor-IEReady $ie -initialWaitInSeconds 3

  Log-Info ("All steps are done")
    }
    finally {
        $ie.Quit()
    } 
}

WaitFor-IEReady helper method is given from original script mentioned above so credits go to it’s author:

function WaitFor-IEReady {
    [CmdletBinding()]
    Param
    (
        [Parameter(Mandatory=$true, ValueFromPipelineByPropertyName=$true, Position=0)]
        $ie,

        [Parameter(Mandatory=$false, Position=1)]
        $initialWaitInSeconds = 1
    )

    sleep -Seconds $initialWaitInSeconds

    while ($ie.Busy) {

        sleep -milliseconds 50
    }
}

Log-Info and Log-Warn are basic logger methods and you may implement them as needed for your scenario. Since we delegated login to the end user we don’t need to handle different O365 login scenarios and script is greatly simplified, e.g. there is no need to perform javascript activities which work not very stable via COM automation.

Monday, May 20, 2019

One reason why SPFarm.CurrentUserIsAdministrator may return false when current user is farm administrator

Sharepoint farm administrators are powerful users who may perform administrative actions in Sharepoint farm (see SharePoint Farm Administrator account for more details). Sometime you may need to check whether or not current user is farm admin in order to allow or disallow specific actions. You may do that using SPFarm.CurrentUserIsAdministrator method. The problem however that it will work as expected only in context of Central administration web application (i.e. if page which calls this method is running in Central administration). If you will try to call this method from regular Sharepoint web application it will always return false even if current user is added to Farm administrations group on your farm - you may add users to Farm administrators using Central administration > Manage the farm administrators group:

In this case (when you need to call SPFarm.CurrentUserIsAdministrator from regular Sharepoint web application) you have to use overridden version of this method with boolean parameter:

allowContentApplicationAccess
true to make the check work in the content port of a Web application; otherwise, false.

i.e. like this:

bool isFarmAdmin = SPFarm.Local.CurrentUserIsAdministrator(true);

In this case method will work as expected i.e. return true if user is member of Farm administrators group and false otherwise.

Tuesday, May 7, 2019

Maximum alerts limit per user in Sharepoint

When you try to subscribe for alerts to specific user account in Sharepoint and get the following error:

You have created the maximum number of alerts allowed for this site

it may be so that you have reached alerts limit for this user in current site. This per-web application setting and it is possible to extend this limit in Central administration > Manage web applications > web app > General settings > Alerts:

By default it is set to 500 alerts per user. After you will increase this limit error should disappear.

Friday, April 26, 2019

Provision multilingual sites with PnP templates

Sharepoint PnP Powershell library (which is also available on Nuget) has own provisioning engine. It is quite powerful engine which allows to provision MUI sites (although it has own problems with stability). In order to do that you need to use several basic elements:

1. Have all literals which should be localized in resx file

2. Specify supported languages inside <pnp:SupportedUILanguages>…</pnp:SupportedUILanguages> section of PnP template. After provisioning these languages will be set in Site settings > Language settings > Alternate languages. You need to have resx file for each supported language in standard resources.xx-XX.resx format

3. Resx files with translations should be specified inside <pnp:Localizations>…</pnp:Localizations> section of PnP template. Note that PnP will provision all these languages to your site regardless of what alternate languages are set in Site settings > Language settings on the moment when this template is applied to target site

4. For localized strings use {resource:ResourceKey} format inside template

Here is example of how PnP template for MUI site provisioning may look like:

<?xml version="1.0"?>
<pnp:Provisioning xmlns:pnp="http://schemas.dev.office.com/PnP/2018/01/ProvisioningSchema">
  <pnp:Localizations>
 <pnp:Localization LCID="1033" Name="English" ResourceFile="resources.en-US.resx"/>
    <pnp:Localization LCID="1031" Name="German" ResourceFile="resources.de-DE.resx"/>
  </pnp:Localizations>
  <pnp:Templates ID="Test-Container">
    <pnp:ProvisioningTemplate ID="Test" Version="1">

      <pnp:SupportedUILanguages>
        <pnp:SupportedUILanguage LCID="1033" />
        <pnp:SupportedUILanguage LCID="1031" />
      </pnp:SupportedUILanguages>

   <pnp:SiteFields xmlns:pnp="http://schemas.dev.office.com/PnP/2018/01/ProvisioningSchema">
  <Field ID="..." Name="MyField" DisplayName="{resource:MyFieldTitle}" Type="Text" Group="Custom" SourceID="..." StaticName="MyField"></Field>
   </pnp:SiteFields>
   
    </pnp:ProvisioningTemplate>
  </pnp:Templates>
</pnp:Provisioning>

It will provision file with English and German UI languages.

Thursday, April 18, 2019

Can’t get groups created from MS Teams from Graph endpoint /beta/me/joinedGroups

With MS Graph API you may use /beta/me/joinedGroups endpoint for getting list of groups where current user is a member. With the same endpoint you may also get isFavorite attribute for the group which shows whether or not user added group to favorites. However this endpoint has own issues: recently we found that it doesn’t return groups which were created from MS Teams: when you create new Team there also related Group is created. It is possible to get details of this group using basic groups endpoint

https://graph.microsoft.com/v1.0/groups/{id}

But if you will try to get list of user’s groups via beta endpoint such groups created from MS Teams won’t be returned:

https://graph.microsoft.com/beta/me/joinedgroups/?$select=id,isfavorite,displayName&$top=200

One possible explanation could be that internally /me/joinedgroups end point is routed to Outlook services which is not integrated with Teams well enough yet: when I tried to add createdDateTime attribute to the REST url (this attribute is returned for groups from basic endpoint - see above)

https://graph.microsoft.com/beta/me/joinedgroups/?$select=id,isfavorite,displayName,createdDateTime &$top=200

it returned error saying that returned entities have Microsoft.OutlookServices.Group type:

May be this is a bug or such functionality is not implemented in beta endpoint yet. For now I asked this question in StackOverflow – hope that somebody from MS Graph product team will answer it.

Friday, April 5, 2019

Use Sharepoint search API using HTTP POST requests

As you probably know it is possible to use Sharepoint search API programmatically by calling /_api/search/query endpoint with HTTP GET and provide KQL query and details (like selected managed properties) in query string. Popular Search Query Tool uses the same technique. However query string approach has own limitation, e.g. max 4kb length limit which is common for ASP.Net applications. If your query is built dynamically and you don’t know the actual length on compile time you may use search API with HTTP POST and provide KQL query in request body. If you want to use search API with HTTP POST you need to use slightly different endpoint: /_api/search/postquery. Let’s see how it works in Postman tool.

The first thing which we need to get is to obtain access token. It can be done by sending another HTTP POST request to the following address: https://accounts.accesscontrol.windows.net/{tenant_name}.onmicrosoft.com/tokens/OAuth/2. In request body we need to provide several parameters which are described in the following list:

  • grant_type = client_credentials
  • client_id – client id of your Sharepoint app (you should register it in advance using /_layouts/15/appregnew.aspx and then grant appropriate permissions using /_layouts/15/appinv.aspx) in the following form {cliend_id}@{tenant_id}. You may check tenant id in Azure portal > Azure Active Directory > Properties > Directory ID
  • client_secret – client secret of your Sharepoint app
  • resource – should have value in the form 00000003-0000-0ff1-ce00-000000000000/{tenant_name}.onmicrosoft.com@{tenant_id}

so request should look like this in Postman:

If everything was configured properly you should get success response which should contain access_token in the response body. Copy it’s value – it will be needed on the next step.

Now we are ready to send search POST requests to search API endpoint. We will use https://{tenant_name}.sharepoint.com/_api/search/postquery address for that. Let’s get list of all sites using the following KQL query:

contentclass:STS_Site

Request body should look like this:

{
   "request":{
      "Querytext":"contentclass:STS_Site",
      "RowLimit":100,
      "SelectProperties":{
         "results":[
            "Title",
            "SiteID",
            "OriginalPath"
         ]
      }
   }
}

After that switch to Authorization type and select Type = Bearer Token and specify value of access_token which was obtained on the previous step:

On the Headers tab add Accept and Content-type headers with “application/json;odata=verbose” (note that it is important to specify these headers exactly like this: if you will specify different odata version request may return error because parameters schema may be different.):

If everything was done properly when you will execute POST request in Postman you will get list of Sharepoint sites in your tenant.