Wednesday, October 5, 2016

Work with Azure AD via Microsoft Graph API

In this post I will show example of working application (we will use simple console application for simplicity) which will read Azure AD groups using Microsoft Graph API.

First of all we should have MS account and MSDN subscription for accessing Azure portal: https://manage.windowsazure.com. Note that we will use old Azure portal in our example, when I tried to use new portal and made the same steps there, application didn’t work. Instead it threw exception:

Code: Authorization_IdentityNotFound
Message: The identity of the calling application could not be established

May be I missed some steps there, but with steps described below application worked successfully.

Once we logged to Azure portal we need to create Azure AD directory with which we will work. Alternatively you may use existing Azure AD. In order to create it go to Active Directory > New:

Notice domain name – it should be globally unique and it will be used later in our console application. For testing let’s create several groups in newly created AD:

Now we have working AD and groups for testing, next step is to register application. Some tutorials say that you may create it in https://apps.dev.microsoft.com, but after I tried to do it and ran application I also got same exception:

Code: Authorization_IdentityNotFound
Message: The identity of the calling application could not be established

May be after you add application from https://apps.dev.microsoft.com you should go to Azure portal > Active Directory > choose your AD > Applications and add your application here explicitly. But for me the following method works: go to Azure portal > Active Directory > choose your AD > Applications > Add > Add an application my organization is developing. On the 1st wizard step specify application name and leave Web application and/or Web API selected by default:

On 2nd step we need to specify application urls. Once we develop application which won’t require user sign-on and delegating permissions from users, leave https://example.com there:

After application has been created go to its Configuration page, create new Key (or Secret. Actual value will be shown after changes will be saved). Under Permissions to other applications uncheck all Delegated permissions as we don’t need them and in Application permissions list check Read directory data. Save changes and copy Client Id and generated Key value – they will be needed in console application.

When configuration described above is completed we may implement our console application for reading Azure AD groups via Microsoft Graph. Create new console project in VS and add the following Nuget packages:

  • Microsoft Graph Core Client Library
  • Active Directory Authentication Library
  • Json.NET

In App.config add the following settings:

   1: <appSettings>
   2:   <add key="ClientId" value="..." />
   3:   <add key="ClientSecret" value="..." />
   4:   <add key="AuthorityUrl"
   5: value="https://login.windows.net/alexeydemo.onmicrosoft.com/oauth2/token" />
   6: </appSettings>

ClientId and ClientSecret should contain values copied from application registration page and AuthorityUrl should be in format https://login.windows.net/{AzureDomain}/oauth2/token, where {AzureDomain} is the domain of our Azure AD specified during creation of AD (alexeydemo.onmicrosoft.com in our example). In Program.cs add the following code:

   1: class Program
   2: {
   3:     static void Main(string[] args)
   4:     {
   5:         Console.WriteLine("before: enumGroups");
   6:         enumGroups();
   7:         Console.WriteLine("after: enumGroups");
   8:         Console.ReadKey();
   9:     }
  10:  
  11:     static async void enumGroups()
  12:     {
  13:         var graph = new GraphServiceClient(new AzureAuthenticationProvider());
  14:         try
  15:         {
  16:             Console.WriteLine("before: graph.Groups.Request().GetAsync()");
  17:             var groups = await graph.Groups.Request().GetAsync();
  18:             Console.WriteLine("after: graph.Groups.Request().GetAsync()");
  19:  
  20:             foreach (var g in groups)
  21:             {
  22:                 Console.WriteLine(g.DisplayName);
  23:             }
  24:         }
  25:         catch (ServiceException x)
  26:         {
  27:             Console.WriteLine("Exception occured: {0}", x.Error);
  28:         }
  29:     }
  30: }
  31:  
  32: public class AzureAuthenticationProvider : IAuthenticationProvider
  33: {
  34:     public async Task AuthenticateRequestAsync(HttpRequestMessage request)
  35:     {
  36:         string clientId = ConfigurationManager.AppSettings["ClientId"];
  37:         string clientSecret = ConfigurationManager.AppSettings["ClientSecret"];
  38:         string authority = ConfigurationManager.AppSettings["AuthorityUrl"];
  39:  
  40:         var authContext = new AuthenticationContext(authority);
  41:  
  42:         var creds = new ClientCredential(clientId, clientSecret);
  43:  
  44:         Console.WriteLine("before: get token");
  45:         var authResult =
  46: await authContext.AcquireTokenAsync("https://graph.microsoft.com/", creds);
  47:         Console.WriteLine("after: get token {0}", authResult.AccessToken);
  48:  
  49:         request.Headers.Add("Authorization", "Bearer " + authResult.AccessToken);
  50:     }
  51: }

There are several interesting moments. First of all note AzureAuthenticationProvider class (lines 32-50) – it is passed to constructor of GraphServiceClient (line 13) and used for getting authentication token for our app. In order to get it we retrieve client id, client secret and authority url from app.config (lines 36-38). We used AuthenticationContext.AcquireTokenAsync() methods for acquiring auth token because in our case app-only permissions are used. For applications which require users’ sign-on with their MS accounts and delegeting permissions DelegateAuthenticationProvider class should be used (see examples on Microsoft Graph site). Once authentication token is received we can query groups via GraphServiceClient.Groups.Request().GetAsync() (line 17).

Also notice that in our example async programming model is used (async and await keywords). If you are not familiar with it yet, read this MSDN article: Asynchronous Programming with async and await (C#). They key thing is that methods which are marked with async may be called asynchronously (i.e. control is returned to the caller immediately after calling this method without waiting when its execution will be completed). This is happen when we call such method with await keyword (lines 17 and 45-46). After such call control is returned to the caller of parent method immediately, and code which is located after await call will be resumed in separate thread when execution of this await call will be completed. I.e. after we call

var authResult = await authContext.AcquireTokenAsync("https://graph.microsoft.com/", creds)

in AuthenticateRequestAsync(HttpRequestMessage request) control is returned to the caller of AuthenticateRequestAsync() method and resumed in own thread after call to authContext.AcquireTokenAsync() will be completed starting from the next line after it:

Console.WriteLine("after: get token {0}", authResult.AccessToken)

I added console traces for better demo how execution flow goes. Also as tasks are completed in separate threads we need to add Console.ReadKey() to the main thread (line 8) in order to avoid its completing (and as result closing of application) before tasks are completed.

After we run the program we will get the following result:

before: enumGroups
before: graph.Groups.Request().GetAsync()
before: get token
after: enumGroups
after: get token eyJ0eXAiOiJKV1Q...
after: graph.Groups.Request().GetAsync()
Group3
Group2
Group1

As you can see calls are made asynchronously (e.g. we left enumGroups method before got token and queried groups from AD) and at the end we have list of AD groups we created earlier. Hope this tutorial will save your time and help to better understand Microsoft Graph API.

Friday, September 16, 2016

Copy publishing page with web parts in Sharepoint Online via PowerShell

As you probably know for copying publishing pages with all web parts in Sharepoint we may use Sharepoint Designer. It is convenient way when you need to copy one page, but what if there are a lot of pages and copying them manually will take a lot of time? In this case we may use PowerShell. The tricky moment here is that when you copy page in Sharepoint Designer it uses author.dll RPC call (you may check it Fiddler). So we will do the same: construct http request with necessary parameters and authenticate ourselves using Sharepoint Online cookies. In our example we will copy standard search results page results.aspx on search sub to the same Pages doclib with different name:

   1: function Get-Authentication-Cookie($context, $siteCollectionUrl)
   2: {
   3:     $sharePointUri = New-Object System.Uri($siteCollectionUrl)
   4:     $authCookie = $context.Credentials.GetAuthenticationCookie($sharePointUri)
   5:     if( $? -eq $false )
   6:     {
   7:         return $null
   8:     }
   9:     $fedAuthString = $authCookie.TrimStart("SPOIDCRL=".ToCharArray())
  10:     $cookieContainer = New-Object System.Net.CookieContainer
  11:     $cookieContainer.Add($sharePointUri,
  12:     (New-Object System.Net.Cookie("SPOIDCRL", $fedAuthString)))
  13:     return $cookieContainer
  14: }
  15:  
  16: function Copy-Page($ctx, $site, $web, $serviceName, $sourcePage, $destPage) 
  17: { 
  18:     Write-Host "Copy page $sourcePage to $destPage on" $web.Url
  19: -foregroundcolor green
  20:  
  21:     $requestUrl = $web.Url + "/_vti_bin/_vti_aut/author.dll"
  22:     $request = $ctx.WebRequestExecutorFactory.
  23: CreateWebRequestExecutor($ctx, $requestUrl).WebRequest
  24:     $request.Method = "POST"
  25:     $request.Headers.Add("Content", "application/x-vermeer-urlencoded")
  26:     $request.Headers.Add("X-Vermeer-Content-Type",
  27: "application/x-vermeer-urlencoded")
  28:     $request.UserAgent = "FrontPage"
  29:     $request.UseDefaultCredentials = $false
  30:     
  31:     $cookiesContainer = Get-Authentication-Cookie $ctx $site.Url
  32:     $request.CookieContainer = $cookiesContainer
  33:     #$request.ContentLength = 0
  34:     
  35:     $rpcCallString =
  36:         "method=move+document%3a15%2e0%2e0%2e4569&service%5fname=" +
  37:         "$([System.Web.HttpUtility]::UrlEncode($serviceName))&oldUrl=" +
  38:         "$([System.Web.HttpUtility]::UrlEncode($sourcePage))&newUrl=" +
  39:         "$([System.Web.HttpUtility]::UrlEncode($destPage))&url%5flist=" +
  40:         "%5b%5d&rename%5foption=nochangeall&put%5foption=edit&docopy=true"
  41:  
  42:     $requestStream = $request.GetRequestStream()
  43:     $rpcHeader = [System.Text.Encoding]::UTF8.GetBytes($rpcCallString)
  44:     $requestStream.Write($rpcHeader, 0, $rpcHeader.Length);
  45:     $requestStream.Close();
  46:     
  47:     #$request.ContentLength = 0
  48:      
  49:     $response = $request.GetResponse()
  50:     $stream = $response.GetResponseStream()
  51:          
  52:     $reader = New-Object System.IO.StreamReader($stream)
  53:     $content = $reader.ReadToEnd()
  54:     $reader.Close()
  55:     $reader.Dispose()
  56:     
  57:     Write-Host "Page is successfully copied" -foregroundcolor green 
  58: }
  59:  
  60: function Add-New-Search-Results-Page($ctx, $site, $webRelativeUrl,
  61: $sourcePageName, $targetPageName)
  62: {
  63:     Write-Host "Process site $webRelativeUrl" -foregroundcolor green
  64:     
  65:     $web = $site.OpenWeb($webRelativeUrl)
  66:     $ctx.Load($web)
  67:     $ctx.ExecuteQuery()
  68:  
  69:     Write-Host "Server relative url" $web.ServerRelativeUrl -foregroundcolor green
  70:     
  71:     $lists = $web.Lists
  72:     $ctx.Load($lists)
  73:     $ctx.ExecuteQuery()
  74:     $pagesList = $null
  75:     foreach($l in $lists)
  76:     {
  77:         Load-CSOMProperties -object $l -propertyNames @("Title", "BaseTemplate")
  78: -executeQuery
  79:         #Write-Host $l.Title $l.BaseTemplate
  80:         if ($l.BaseTemplate -eq 850)
  81:         {
  82:             $pagesList = $l
  83:             break
  84:         }
  85:     }
  86:     
  87:     if ($pagesList -eq $null)
  88:     {
  89:         Write-Host "Pages doclib not found" -foregroundcolor red
  90:         return
  91:     }
  92:     
  93:     $pages = $pagesList.GetItems(
  94: [Microsoft.SharePoint.Client.CamlQuery]::CreateAllItemsQuery())
  95:     $ctx.Load($pages)
  96:     $ctx.ExecuteQuery()
  97:     $sourcePage = $null
  98:     foreach($p in $pages)
  99:     {
 100:         Load-CSOMProperties -object $p -propertyNames @("File") -executeQuery
 101:         Write-Host $p.File.ServerRelativeUrl
 102:         
 103:         if ($p.File.ServerRelativeUrl.ToLower().EndsWith("/" +
 104: $sourcePageName.ToLower()))
 105:         {
 106:             Write-Host "Source page found" -foregroundcolor green
 107:             $sourcePage = $p.File.ServerRelativeUrl
 108:             continue
 109:         }
 110:         if ($p.File.ServerRelativeUrl.ToLower().EndsWith("/" +
 111: $targetPageName.ToLower()))
 112:         {
 113:             Write-Host "Page $targetPageName already exists"
 114: -foregroundcolor yellow
 115:             return
 116:         }
 117:     }
 118:     
 119:     if ($sourcePage -eq $null)
 120:     {
 121:         Write-Host "Source page not found" -foregroundcolor red
 122:         return
 123:     }
 124:  
 125:     $serviceName = $webRelativeUrl.SubString($webRelativeUrl.LastIndexOf("/"))
 126:     $sourcePage = $sourcePage.SubString($web.ServerRelativeUrl.Length + 1)
 127:     $targetPage =
 128: $sourcePage.SubString(0, $sourcePage.IndexOf($sourcePageName)) + $targetPageName
 129:  
 130:     Write-Host "Service name $serviceName" -foregroundcolor green
 131:     Write-Host "Source page $sourcePage" -foregroundcolor green
 132:     Write-Host "Target page $targetPage" -foregroundcolor green
 133:     
 134:     Copy-Page $ctx $site $web $serviceName $sourcePage $targetPage
 135: }
 136:  
 137: $siteUrl = "https://mytenant.sharepoint.com/sites/foo"
 138: $username = Read-Host -Prompt "Enter username"
 139: $password = Read-Host -Prompt "Enter password"
 140:  
 141: $ctx = New-Object Microsoft.SharePoint.Client.ClientContext($siteUrl)    
 142: $ctx.RequestTimeOut = 1000 * 60 * 10;
 143: $ctx.AuthenticationMode =
 144: [Microsoft.SharePoint.Client.ClientAuthenticationMode]::Default
 145: $securePassword = ConvertTo-SecureString $password -AsPlainText -Force
 146: $credentials = New-Object Microsoft.SharePoint.Client.
 147: SharePointOnlineCredentials($username, $securePassword)
 148: $ctx.Credentials = $credentials
 149: $web = $ctx.Web
 150: $site = $ctx.Site
 151: $ctx.Load($web)
 152: $ctx.Load($site)
 153: $ctx.ExecuteQuery()
 154:  
 155: Add-New-Search-Results-Page $ctx $site "en/search" "results.aspx" "fooresults.aspx"

Here we used additional helper function Load-CSOMProperties from Gary Lapointe which allows to load object with specified properties in PowerShell (in C# we may use ClientContext.Load() for that with properties defined via lambda expressions). I won’t copy it here, you may just copy it from github to your ps1 file as is.

Lets check what script does. At first we specify username and password and initialize ClientContext (lines 137-153). The we pass context and site objects to Add-New-Search-Results-Page, also pass sub site url “en/search”, source page “results.aspx” and target page “fooresults.aspx” (line 155). Inside function we initialize sub site and get Pages doclib (lines 63-91). After that we iterate through all pages and find specified source page and at the same time check that target page doesn’t exist yet (lines 93-132). Having source page url and target page urls in form “Pages/results.aspx” and “Pages/fooresults.aspx” and additional parameter called service name which is needed for RPC post value in form “/search” we make call to Copy-Page function which makes actual copy (lines 130-134). In Copy-Page we construct http request object (lines 21-33). In order to authenticate http request we set HttpWebRequest.CookieContainer property using another helper function Get-Authentication-Cookie (lines 31-32). Then we construct RPC call post parameter (lines 35-40) and perform http request (42-55). In result we will have new fooresults.aspx page with the same web parts as original result.aspx page. Hope this information will help you in your work.