Thursday, July 18, 2019

Problem with Copy-PnPFile cmdlet and File Not Found error

Sharepoint PnP PowerShell library has many useful commands which simplify scripting against Sharepoint Online. One of them is Copy-PnPFile which allows to copy file from one document library to another. Source and target doclibs may be even located in different site collections in the same tenant.

One of example of Copy-PnPFile (basically 1st example) says that it is possible to copy single file like that:

Copy-PnPFile -SourceUrl Documents/company.docx -TargetUrl /sites/otherproject/Documents/company.docx

Unfortunately currently it gives File not found error. Looks like there is a bug in Copy-PnPFile which prevents it from working correctly – it is also described in this StackOverflow thread which was created couple of days ago: Copy-PnPFile returns File Not Found.

The only working way which I’ve found so far is to copy whole root folder:

Copy-PnPFile -SourceUrl Documents -TargetUrl /sites/otherproject/Documents -SkipSourceFolderName

It will copy all files from Documents doclib to Documents doclib on another site collection /sites/otherproject. However it will also try to copy OTB list view AllItems.aspx as last file and it will give error that AllItems.aspx already exists in target doclib. In order to ignore this error I used the following solution:

$error = $null
Copy-PnPFile -SourceUrl Documents -TargetUrl /sites/otherproject/Documents -SkipSourceFolderName -ErrorAction SilentlyContinue -ErrorVariable error
if ($error -and !$error.Exception.Message.ToLower().Contains("allitems.aspx")) {
    throw $error
}

I.e. it will throw error only if message doesn’t contain allitems.aspx occurrence. Hope that it will help someone.

Monday, July 15, 2019

Problem with not editable host.json file for Azure function app

Some time we need to manually modify content of host.json file of Azure function app (e.g. to change logging settings). In order to do that you need to go to Azure function app > Platform features > Function app settings > host.json. However you may face with situation that textarea which stores content of host.json will be readonly:

In this case change Function app edit mode setting on the same page from “Read only” to “Read/Write”. After that you will be able to edit content of host.json file for you Azure function app:

Friday, July 12, 2019

Get all Azure AD groups where current user is a member transitively via Graph API


As you probably know we may get all groups where user is member using memberOf endpoint:

GET /users/{id | userPrincipalName}/memberOf

This endpoint returns only those groups where user was added as direct member. I.e. if user was added to GroupA and this GroupA was then added to GroupB – it will return only GroupA but not GruopB. However we often need to get all groups where user is member transitively. Fortunately it is also possible with another endpoint getMemberGroups:

POST /users/{id | userPrincipalName}/getMemberGroups

Until recently it was available only in Graph API itself – not in .Net Graph client library. Fortunately starting with 1.16 version Microsoft.Graph.User class got new property TransitiveMemberOf propery:

Using this property we may get all groups where user is member transitively. It supports paging so in order to get all groups we also need to iterate through pages. Here is the code example which does that:

private static List<Guid> GetUserGroupsTrasitively(string userPrincipalName)
{
 try
 {
  var graph = new GraphServiceClient(new AzureAuthenticationProvider());
  var groups = graph.Users[userPrincipalName].TransitiveMemberOf.Request().GetAsync().Result;
  if (groups == null)
  {
   return new List<Guid>();
  }

  var result = new List<Guid>();
  while (groups.Count > 0)
  {
   foreach (var group in groups)
   {
    result.Add(new Guid(group.Id));
   }

   if (groups.NextPageRequest != null)
   {
    groups = groups.NextPageRequest.GetAsync().Result;
   }
   else
   {
    break;
   }
  }

  return result;
 }
 catch (Exception x)
 {
  // error handling
 }
}

Monday, July 1, 2019

Sharepoint MVP 2019

I got very exciting email from MS today that I’ve got MVP award in Office Apps & Services category. Although it is 9th award for me it definitely has own place in my professional life and I’m very glad that MS recognizes my contribution to community life with this award. Last year a lot of work was done related with MS Graph API and OfficeDevPnP. Many issues, workarounds and solutions were discussed during last year on forums, in blog posts comments, github issues, etc. Also I continue maintenance of Camlex library which simplifies creation of dynamic CAML queries for developers. Nowadays new technologies appear very often and they allow us to do such things which were not possible before. This is great but developers’ life don’t become easier because of that – we get new and new challenges in our work every day. From this perspective community role is crucial – I can’t say how many times I by myself found solutions for technical challenges in blog posts, forums, code samples, etc. Knowing on practice how important this work is I also try to share my findings, ideas and solutions with community. Thank you MS and thank you dear readers of my blog for being with me this year. Looking forward for the new year with it’s own interesting challenges and inventive solutions.

Tuesday, June 25, 2019

How to get localized field titles via CSOM in Sharepoint

Some time ago I wrote article which shows how to localize web part titles via CSOM. If you are not familiar with it I recommend to read it before continue as it has useful information about Sharepoint MUI feature in general: Localize web part titles via client object model in Sharepoint. In current post I will show how to get localized field titles via CSOM in Sharepoint. This technique can be used both in on-prem and online versions.

Currently CSOM has Field.TitleResource property and it looks suitable when you want to get localized title of some field. However this is not the case. In order to get localized fields’ titles you still have to use Field.Title property but with little tuning of ClientContext – similar to those which is mentioned in the article above. More specifically you need to specify target language via “Accept-Language” HTTP header associated with ClientContext and then request field titles (of course assuming that fields have these localized titles provisioned. See e.g. Provision multilingual sites with PnP templates to see how to provision multilingual fields’ titles using PnP templates). Here is the code which shows this concept:

string siteUrl = "...";
string clientId = "...";
string clientSecret = "...";
int lcid = ...;
using (var ctx = new OfficeDevPnP.Core.AuthenticationManager().GetAppOnlyAuthenticatedContext(siteUrl, clientId, clientSecret))
{
 ctx.PendingRequest.RequestExecutor.WebRequest.Headers["Accept-Language"] = new CultureInfo(lcid).Name;
 ctx.Load(ctx.Web);
 ctx.Load(ctx.Web.Fields, f => f.Include(c => c.Id, c => c.Title));
 ctx.ExecuteQueryRetry();
 ...
}

As result when you will iterate through site columns retrieved this way they will contain titles localized for language specified with lcid parameter.

Thursday, June 13, 2019

Use pagination with Sharepoint search API

Often search request return a lot of data. It may be insufficient to show all this data at once on the page – performance may suffer, page may be overloaded with data, etc. In order to address these issues we may use pagination i.e. get data by chunks. But how it often happens with Sharepoint there are own considerations related with pagination in search API.

If we will check documentation of SharePoint Search REST API we will find several properties which affect pagination logic:

  • StartRow: The first row that is included in the search results that are returned. You use this parameter when you want to implement paging for search results.
  • RowLimit: The maximum number of rows overall that are returned in the search results. Compared to RowsPerPage, RowLimit is the maximum number of rows returned overall.
  • RowsPerPage: The maximum number of rows to return per page. Compared to RowLimit, RowsPerPage refers to the maximum number of rows to return per page, and is used primarily when you want to implement paging for search results.

So based on this description we may assume that most obvious way to get paginated data is to use StartRow and RowsPerPage:

var searchUrl = "http://{tenant}.sharepoint.com/_api/search/query?querytext='" + query + "'&selectproperties='Title'&startRow=" + startRow + "&rowsPerPage=" + pageSize;

It will work but with one condition: if page size is less than default page size which is 10 items per page. If page size is greater than default page size it won’t work: in this case you have to use RowLimit which will work as page size even though documentation says different:

var searchUrl = "http://{tenant}.sharepoint.com/_api/search/query?querytext='" + query + "'&selectproperties='Title'&startRow=" + startRow + "&rowLimit=" + pageSize;

This approach will allow to implement pagination with page size bigger than default search page size (10 items per page).

Thursday, June 6, 2019

Grant permissions and trust SharePoint app automatically via PowerShell

In Sharepoint app model we may need to grant permissions to Sharepoint app on AppInv.aspx page by providing appropriate permissions request xml. If permissions are granted on Tenant level you need to open AppInv.aspx in context of Central admin i.e. https://{tenant}-admin.sharepoint.com:

It was historically quite painful to automate this process as automatic permissions grant is not currently possible. There were attempts to automate O365 login and automate trust process using COM automation in PowerShell (using New-Object -com internetexplorer.application): https://github.com/wulfland/ScriptRepository/blob/master/Apps/Apps/Deploy-SPApp.ps1. With this approach script opens AppInv.aspx page and simulates user’s input.

However O365 login experience was changed since this script was implemented and there is no guarantee that it won’t be changed further. Also there may be several login scenarios:

  • user may be already logged in if chose Remember credentials during previous login
  • user may use MFA with SMS, authenticator app or something else which will make login automation even more complicated

Keeping that in mind I implemented the following semi-automatic way of granting app permissions and trust the app:

1. app is registered in Azure AD via PowerShell (in Sharepoint Online it is not necessary to register app which will be used for communicating with Sharepoint via AppRegNew.aspx. You may also register it in Azure Portal > App Registrations). See e.g. Create an Azure Active Directory Application and Key using PowerShell for example

2. Then script opens AppInv.aspx page in IE (using Start-Process cmdlet) and asks user to authenticate him/herself manually. After that user returns to the script and clicks Enter – all other steps (grant permissions and trust the app) are performed by the following PowerShell script:

function Trust-SPAddIn {
    [CmdletBinding(SupportsShouldProcess=$true)]
    [OutputType([int])]
    Param
    (
        [Parameter(Mandatory=$true, Position=0)]
        [string]$AppInstanceId,

        [Parameter(Mandatory=$true, Position=1)]
        [string]$WebUrl,

        [parameter(Mandatory=$true, Position=2)] 
        [string]$UserName, 

        [parameter(Mandatory=$true, Position=3)] 
        [string]$Password
    )

    $ie = New-Object -com internetexplorer.application
    try {
  Log-Warn ("Script will now open $WebUrl. Please authenticate yourself and wait until Admin Center home page will be loaded.")
  Log-Warn ("After that leave Admin Center window opened (don't close it), return to the script and follow provided instructions.")
  Log-Warn ("In case you are already signed in Admin Center window will be opened without asking to login. In this case wait until Admin Center window will be loaded, leave it opened and return to the script.")
  if (-not $silently) {
   Log-Warn ("Press Enter to open $WebUrl...")
   Read-Host
  }
 
        $ie.Visible = $true
        $ie.Navigate2($WebUrl)
  
  if (-not $silently) {
   Log-Warn ("Wait until Admin Center window will be fully loaded and press Enter to continue installation")
   Log-Warn ("Don't close Admin Center window - script will close it automatically")
   Read-Host
  }
  
  $authorizeURL = "$($WebUrl.TrimEnd('/'))/_layouts/15/appinv.aspx"
  Log-Info ("Open $authorizeURL...")
  $ie.Visible = $false
  $ie.Navigate2($authorizeURL)
  WaitFor-IEReady $ie -initialWaitInSeconds 3

  Log-Info ("Grant permissions to the app...")
  $appIdInput = $ie.Document.getElementById("ctl00_ctl00_PlaceHolderContentArea_PlaceHolderMain_IdTitleEditableInputFormSection_ctl01_TxtAppId")
  $appIdInput.value = $AppInstanceId
  $lookupBtn = $ie.Document.getElementById("ctl00_ctl00_PlaceHolderContentArea_PlaceHolderMain_IdTitleEditableInputFormSection_ctl01_BtnLookup")
  $lookupBtn.Click()
  WaitFor-IEReady $ie -initialWaitInSeconds 3
  Log-Info ("Step 1 of 2 done")
  $appIdInput = $ie.Document.getElementById("ctl00_ctl00_PlaceHolderContentArea_PlaceHolderMain_TitleDescSection_ctl01_TxtPerm")
  $appIdInput.value = '<AppPermissionRequests AllowAppOnlyPolicy="true"><AppPermissionRequest Scope="http://sharepoint/content/tenant" Right="FullControl" /></AppPermissionRequests>'
  $createBtn = $ie.Document.getElementById("ctl00_ctl00_PlaceHolderContentArea_PlaceHolderMain_ctl01_RptControls_BtnCreate")
  $createBtn.Click()
  WaitFor-IEReady $ie -initialWaitInSeconds 3
  Log-Info ("Step 2 of 2 done")

  Log-Info ("Trust the app...")
  $trustBtn = $ie.Document.getElementById("ctl00_ctl00_PlaceHolderContentArea_PlaceHolderMain_BtnAllow")
  $trustBtn.Click()
  WaitFor-IEReady $ie -initialWaitInSeconds 3

  Log-Info ("All steps are done")
    }
    finally {
        $ie.Quit()
    } 
}

WaitFor-IEReady helper method is given from original script mentioned above so credits go to it’s author:

function WaitFor-IEReady {
    [CmdletBinding()]
    Param
    (
        [Parameter(Mandatory=$true, ValueFromPipelineByPropertyName=$true, Position=0)]
        $ie,

        [Parameter(Mandatory=$false, Position=1)]
        $initialWaitInSeconds = 1
    )

    sleep -Seconds $initialWaitInSeconds

    while ($ie.Busy) {

        sleep -milliseconds 50
    }
}

Log-Info and Log-Warn are basic logger methods and you may implement them as needed for your scenario. Since we delegated login to the end user we don’t need to handle different O365 login scenarios and script is greatly simplified, e.g. there is no need to perform javascript activities which work not very stable via COM automation.