Wednesday, July 30, 2014

Upload sandbox solutions to Sharepoint Online via PowerShell

Most of Sharepoint developers often deployed farm solutions to Sharepoint web application using Add-SPSolution and Install-SPSolution PowerShell cmdlets. In order to deploy sandbox solutions to on-premise Sharepoint sites we can use appropriate sandbox versions of mentioned cmdlets: Add-SPUserSolution and Install-SPUserSolution. Unfortunately these cmdlets work only on-premise. But how to upload sandbox solution to Sharepoint Online site?

In order to do it we may use Sharepoint Online helper open source project which is hosted on Codeplex. It makes changes in solutions gallery by making http requests to Sharepoint Online site: as far as I understand author checked what parameters are sent when solutions are managed from UI and then built the same requests programmatically. For authentication you have to provide your user name and password for Sharepoint Online site. They will be used for getting authentication cookies which then will be used for making authenticated requests.

At the moment of writing this post it had the following methods for managing sandbox solutions in Sharepoint Online:

  • Upload solution
  • Activate solution
  • Deactivate solution

Unfortunately for complete set of methods it didn’t contain Delete solution method. So first of all we need to add this method to the library. Code is hosted on codeplex with compiled library, so it is quite easy to do:

   1: public static void DeleteSolution(String siteCollectionUrl,
   2:     CookieContainer cookies, String filePath)
   3: {
   4:     siteCollectionUrl = siteCollectionUrl.TrimEnd('/');
   5:     var fileInfo = new FileInfo(filePath);
   6:     var ctx = Authenticator.GetClientContext(siteCollectionUrl, cookies);
   7:  
   8:     var fileUrl = String.Format("{0}/_catalogs/solutions/{1}",
   9:         siteCollectionUrl, fileInfo.Name);
  10:     var fileUri = new Uri(fileUrl);
  11:  
  12:     Microsoft.SharePoint.Client.File file =
  13:         ctx.Web.GetFileByServerRelativeUrl(fileUri.AbsolutePath);
  14:     ctx.Load(file);
  15:     file.DeleteObject();
  16:     ctx.ExecuteQuery();
  17: }

I also provided patch with this method on project’s site. After that we have all methods we need for managing sandbox solutions on Sharepoint Online site.

Next step is to write PowerShell script for uploading, activating, deactivating and deleting solutions remotely:

   1: param(
   2:     [switch]$uploadSolution,
   3:     [switch]$deactivateSolution,
   4:     [switch]$activateSolution,
   5:     [switch]$deleteSolution,
   6:     [switch]$updateSolution
   7: )
   8:  
   9: . ./Params.ps1
  10:  
  11: [System.Reflection.Assembly]::LoadFile([System.IO.Path]
  12: ::Combine($externalDllsDirectory, "Microsoft.SharePoint.Client.dll"))
  13: [System.Reflection.Assembly]::LoadFile([System.IO.Path]
  14: ::Combine($externalDllsDirectory, "Microsoft.SharePoint.Client.Runtime.dll"))
  15: [System.Reflection.Assembly]::LoadFile([System.IO.Path]
  16: ::Combine($externalDllsDirectory, "SharePointOnline.Helper.dll"))
  17:  
  18: function Get-Auth-Cookies()
  19: {
  20:     Write-Host "Get authentication cookies from O365-foregroundcolor Green
  21:     $cookies = [SharePointOnline.Helper.Authenticator]
  22: ::GetAuthenticatedCookies($siteCollUrl, $login, $password)
  23:     return $cookies
  24: }
  25:  
  26:  
  27: function Upload-Solution-Impl($cookies)
  28: {
  29:     Write-Host "Upload solution $solutionName to $siteCollUrl"
  30: -foregroundcolor Green
  31:     [SharePointOnline.Helper.SandboxSolutions]
  32: ::UploadSolution($siteCollUrl, $cookies, $solutionFullPath)
  33:     Write-Host "Solution was successfully uploaded" -foregroundcolor Green
  34: }
  35:  
  36: function Upload-Solution()
  37: {
  38:     $cookies = Get-Auth-Cookies
  39:     Upload-Solution-Impl $cookies
  40: }
  41:  
  42: function Activate-Solution-Impl($cookies)
  43: {
  44:     Write-Host "Activate solution $solutionName on $siteCollUrl"
  45: -foregroundcolor Green
  46:     [SharePointOnline.Helper.SandboxSolutions]
  47: ::ActivateSolution($siteCollUrl, $cookies, $solutionName)
  48:     Write-Host "Solution was successfully activated" -foregroundcolor Green
  49: }
  50:  
  51: function Activate-Solution()
  52: {
  53:     $cookies = Get-Auth-Cookies
  54:     Activate-Solution-Impl $cookies
  55: }
  56:  
  57: function Deactivate-Solution-Impl($cookies)
  58: {
  59:     Write-Host "Deactivate solution $solutionName on $siteCollUrl"
  60: -foregroundcolor Green
  61:     [SharePointOnline.Helper.SandboxSolutions]
  62: ::DeactivateSolution($siteCollUrl, $cookies, $solutionName)
  63:     Write-Host "Solution was successfully deactivated" -foregroundcolor Green
  64: }
  65:  
  66: function Deactivate-Solution()
  67: {
  68:     $cookies = Get-Auth-Cookies
  69:     Deactivate-Solution-Impl $cookies
  70: }
  71:  
  72: function Delete-Solution-Impl($cookies)
  73: {
  74:     Write-Host "Delete solution $solutionName from $siteCollUrl"
  75: -foregroundcolor Green
  76:     [SharePointOnline.Helper.SandboxSolutions]
  77: ::DeleteSolution($siteCollUrl, $cookies, $solutionName)
  78:     Write-Host "Solution was successfully deleted" -foregroundcolor Green
  79: }
  80:  
  81: function Delete-Solution()
  82: {
  83:     $cookies = Get-Auth-Cookies
  84:     Delete-Solution-Impl $cookies
  85: }
  86:  
  87: function Update-Solution()
  88: {
  89:     $cookies = Get-Auth-Cookies
  90:     Deactivate-Solution-Impl $cookies
  91:     Delete-Solution-Impl $cookies
  92:     Upload-Solution-Impl $cookies
  93:     Activate-Solution-Impl $cookies
  94: }
  95:  
  96: # solutions management
  97: if ($uploadSolution)
  98: {
  99:     Upload-Solution
 100: }
 101:  
 102: if ($activateSolution)
 103: {
 104:     Activate-Solution
 105: }
 106:  
 107: if ($deactivateSolution)
 108: {
 109:     Deactivate-Solution
 110: }
 111:  
 112: if ($deleteSolution)
 113: {
 114:     Delete-Solution
 115: }
 116:  
 117: if ($updateSolution)
 118: {
 119:     Update-Solution
 120: }

In Params.ps1 script we define path to solution on local file system, site url, user name and password for Sharepoint Online:

   1: # Configure environment specific variables here.
   2: # Use absolute paths if needed.
   3: $currentPath = Convert-Path(Get-Location)
   4: $externalDllsDirectory = resolve-path($currentPath + "\ExternalDll\")
   5: $packageDirectory = resolve-path($currentPath + "\Packages\")
   6: $solutionName = "Test.wsp"
   7: $solutionFullPath = [System.IO.Path]::Combine($packageDirectory, $solutionName)
   8:  
   9: $siteCollUrl = "http://example.com"
  10: $login = "login"
  11: $password = "password"

After that in order to update our solution (deactivate, delete, upload and activate) we may use the following command:

   1: .\install.ps1 -updateSolution

Also we may execute mentioned tasks separately if needed. I hope that this article will be useful and will help you in your work.

Saturday, July 26, 2014

Provision managed metadata term sets and fields for Sharepoint Online using client object model

Starting from Sharepoint 2010 we used to work with managed metadata. But Sharepoint Online introduced new challenge: we have to provision managed metadata fields and term sets via client object model now. I searched for existing solutions first and they didn’t satisfy me. The most often solution I found was to hardcode term store id and provision managed metadata fields declaratively (with optional sugar like automatic replace of this id during publishing of wsp package). I don’t like this approach. What I needed is the same way which we use for regular on-premise Sharepoint installations:

  1. create term sets from xml file;
  2. provision managed metadata fields;
  3. bind fields to term sets.

I wrote PowerShell script which automates these tasks for Sharepoint Online using client object model. Let’s start from creating term sets in local site collection’s term store:

   1: function Create-Term($ctx, $termSet, $label, $lcid)
   2: {
   3:     $term = $termSet.CreateTerm($label, $lcid, [System.Guid]::NewGuid())
   4:     $ctx.ExecuteQuery()
   5: }
   6:  
   7: function Create-TermSet($ctx, $group, $termSetXml, $lcid)
   8: {
   9:     Write-Host "Creating term set" $termSetXml.Name -foregroundcolor Green
  10:     $termSet = $group.CreateTermSet($termSetXml.Name, $termSetXml.Id, $lcid)
  11:     $ctx.ExecuteQuery()
  12:  
  13:     $termSetXml.Term | ForEach-Object { Create-Term $ctx $termSet $_.Name $lcid }
  14: }
  15:  
  16: function Get-TermStore($ctx)
  17: {
  18:     Write-Host "Loading taxonomy session" -foregroundcolor Green
  19:     $session =
  20: [Microsoft.SharePoint.Client.Taxonomy.TaxonomySession]::GetTaxonomySession($ctx)
  21:     $session.UpdateCache();
  22:     $ctx.Load($session)
  23:     $ctx.ExecuteQuery()
  24:  
  25:     Write-Host "Loading term stores" -foregroundcolor Green
  26:     $termStores = $session.TermStores
  27:     $ctx.Load($termStores)
  28:     $ctx.ExecuteQuery()
  29:     $termStore = $termStores[0]
  30:     $ctx.Load($termStore)
  31:     Write-Host "Term store with the following id is loaded:"
  32: $termStore.Id -foregroundcolor Green
  33:     return $termStore
  34: }
  35:  
  36: function Provision-TermSets($ctx, $xmlFilePath)
  37: {
  38:     Write-Host "Load term sets from xml" -foregroundcolor Green
  39:     [xml]$xmlContent = (Get-Content $xmlFilePath)
  40:     if (-not $xmlContent)
  41:     {
  42:         Write-Host "Xml was not loaded successfully. Term sets won't be created"
  43: -foregroundcolor Red
  44:         return
  45:     }
  46:  
  47:     $termStore = Get-TermStore $ctx
  48:  
  49:     Write-Host "Creating group" $xmlContent.Id -foregroundcolor Green
  50:     $groups = $termStore.Groups
  51:     $ctx.Load($groups)
  52:     $ctx.ExecuteQuery()
  53:  
  54:     $group = $groups | Where-Object {$_.Name -eq $xmlContent.Group.Name}
  55:     if ($group)
  56:     {
  57:         Write-Host "Group" $xmlContent.Group.Name
  58: "already exists. If you want to recreate it, delete existing group first"
  59: -foregroundcolor Yellow
  60:         return
  61:     }
  62:  
  63:     $group = $termStore.CreateGroup($xmlContent.Group.Name, $xmlContent.Group.Id)
  64:     $ctx.ExecuteQuery()
  65:     $xmlContent.Group.TermSet |
  66: ForEach-Object { Create-TermSet $ctx $group $_ $termStore.DefaultLanguage }
  67: }

The entry point is Provision-TermSets() method (line 36). Before to call it we need to define term sets in the xml. I used the following structure:

   1: <?xml version="1.0" encoding="utf-8" ?>
   2: <Group Name="TestGroup" Id="...">
   3:   <TermSet Name="TermSet1" Id="...">
   4:     <Term Name="Term11" />
   5:     <Term Name="Term12" />
   6:     <Term Name="Term13" />
   7:   </TermSet>
   8:   <TermSet Name="TermSet2" Id="...">
   9:     <Term Name="Term21" />
  10:     <Term Name="Term22" />
  11:     <Term Name="Term23" />
  12:   </TermSet>
  13:   <TermSet Name="TermSet3" Id="...">
  14:     <Term Name="Term31" />
  15:     <Term Name="Term32" />
  16:     <Term Name="Term33" />
  17:   </TermSet>
  18: </Group>

One thing we should notice here is that we explicitly define term set ids with names. It will help us when we will bind managed metadata fields to them (see below). Having term sets structure in xml file we can now create them using the following command:

   1: $context = New-Object Microsoft.SharePoint.Client.ClientContext($siteURL)
   2: $context.AuthenticationMode =
   3: [Microsoft.SharePoint.Client.ClientAuthenticationMode]::Default
   4: $securePassword = ConvertTo-SecureString $password -AsPlainText -Force
   5: $credentials =
   6: New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($username,
   7: $securePassword)
   8: $context.Credentials = $credentials
   9:  
  10: Provision-TermSets $context $xmlPath

On lines 1-8 we prepare client context with user’s credentials and then call Provision-TermSets() function defined above with created context and path to xml file. After that we will have our term sets created in the term store.

Second step is to create managed metadata fields. We will do it declaratively, which is supported in sandbox solutions, so it will be the same as it would be for on-premise version. In our example we have 3 term sets, so lets provision 3 managed metadata fields:

   1: <?xml version="1.0" encoding="utf-8"?>
   2: <Elements xmlns="http://schemas.microsoft.com/sharepoint/">
   3:   <Field Type="Note"
   4:     DisplayName="Field1TaxHTField"
   5:     MaxLength="255"
   6:     Group="Test"
   7:     ID="..."
   8:     StaticName="Field1TaxHTField"
   9:     Name="Field1TaxHTField"
  10:     Hidden="TRUE"
  11:     ShowInViewForms="FALSE"
  12:     Description="" />
  13:   <Field ID="..."
  14:     SourceID="http://schemas.microsoft.com/sharepoint/v3"
  15:     Type="TaxonomyFieldType"
  16:     DisplayName="Field1"
  17:     ShowField="Term1033"
  18:     Required="FALSE"
  19:     EnforceUniqueValues="FALSE"
  20:     Group="Test"
  21:     StaticName="Field1"
  22:     Name="Field1"
  23:     Hidden="FALSE"
  24:     Mult="FALSE">
  25:     <Default></Default>
  26:     <Customization>
  27:       <ArrayOfProperty>
  28:         <Property>
  29:           <Name>IsPathRendered</Name>
  30:           <Value xmlns:q7="http://www.w3.org/2001/XMLSchema"
  31: p4:type="q7:boolean" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">
  32:             true
  33:           </Value>
  34:         </Property>
  35:         <Property>
  36:           <Name>TextField</Name>
  37:           <Value xmlns:q6="http://www.w3.org/2001/XMLSchema"
  38: p4:type="q6:string" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">
  39:             {... - use id of note field defined above}
  40:           </Value>
  41:         </Property>
  42:       </ArrayOfProperty>
  43:     </Customization>
  44:   </Field>
  45:   <Field Type="Note"
  46:     DisplayName="Field2TaxHTField"
  47:     MaxLength="255"
  48:     Group="Test"
  49:     ID="..."
  50:     StaticName="Field2TaxHTField"
  51:     Name="Field2TaxHTField"
  52:     Hidden="TRUE"
  53:     ShowInViewForms="FALSE"
  54:     Description="" />
  55:   <Field ID="..."
  56:     SourceID="http://schemas.microsoft.com/sharepoint/v3"
  57:     Type="TaxonomyFieldType"
  58:     DisplayName="Field2"
  59:     ShowField="Term1033"
  60:     Required="FALSE"
  61:     EnforceUniqueValues="FALSE"
  62:     Group="Test"
  63:     StaticName="Field2"
  64:     Name="Field2"
  65:     Hidden="FALSE"
  66:     Mult="TRUE">
  67:     <Default></Default>
  68:     <Customization>
  69:       <ArrayOfProperty>
  70:         <Property>
  71:           <Name>IsPathRendered</Name>
  72:           <Value xmlns:q7="http://www.w3.org/2001/XMLSchema"
  73: p4:type="q7:boolean" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">
  74:             true
  75:           </Value>
  76:         </Property>
  77:         <Property>
  78:           <Name>TextField</Name>
  79:           <Value xmlns:q6="http://www.w3.org/2001/XMLSchema"
  80: p4:type="q6:string" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">
  81:             {...- use id of note field defined above}
  82:           </Value>
  83:         </Property>
  84:       </ArrayOfProperty>
  85:     </Customization>
  86:   </Field>
  87:   <Field Type="Note"
  88:     DisplayName="Field3TaxHTField"
  89:     MaxLength="255"
  90:     Group="Test"
  91:     ID="..."
  92:     StaticName="Field3TaxHTField"
  93:     Name="Field3TaxHTField"
  94:     Hidden="TRUE"
  95:     ShowInViewForms="FALSE"
  96:     Description="" />
  97:   <Field ID="..."
  98:     SourceID="http://schemas.microsoft.com/sharepoint/v3"
  99:     Type="TaxonomyFieldType"
 100:     DisplayName="Field3"
 101:     ShowField="Term1033"
 102:     Required="FALSE"
 103:     EnforceUniqueValues="FALSE"
 104:     Group="Test"
 105:     StaticName="Field3"
 106:     Name="Field3"
 107:     Hidden="FALSE"
 108:     Mult="TRUE">
 109:     <Default></Default>
 110:     <Customization>
 111:       <ArrayOfProperty>
 112:         <Property>
 113:           <Name>IsPathRendered</Name>
 114:           <Value xmlns:q7="http://www.w3.org/2001/XMLSchema"
 115: p4:type="q7:boolean" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">
 116:             false
 117:           </Value>
 118:         </Property>
 119:         <Property>
 120:           <Name>TextField</Name>
 121:           <Value xmlns:q6="http://www.w3.org/2001/XMLSchema"
 122: p4:type="q6:string" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">
 123:             {... - use id of note field defined above}
 124:           </Value>
 125:         </Property>
 126:       </ArrayOfProperty>
 127:     </Customization>
 128:   </Field>
 129: </Elements>

As I wrote above this part is the same as for on-premise Sharepoint, so I won’t comment it.

The last step is to bind created managed metadata fields to the term sets. It can be done via the following PowerShell script:

   1: function Bind-Managed-Metadata-Field($ctx, $termStoreId, $fieldId, $termSetId)
   2: {
   3:     $rootWeb = $ctx.Web
   4:     $fields = $rootWeb.Fields
   5:     $ctx.Load($fields)
   6:     $ctx.ExecuteQuery()
   7:  
   8:     try
   9:     {
  10:         $field = $fields.GetById($fieldId)
  11:     }
  12:     catch
  13:     {
  14:         Write-Host "Field" $fieldId "not found in site columns collection."
  15: "It won't be bound to the term set" -foregroundcolor red
  16:         return
  17:     }
  18:  
  19:     $taxField = [Microsoft.SharePoint.Client.ClientContext].GetMethod("CastTo").
  20: MakeGenericMethod([Microsoft.SharePoint.Client.Taxonomy.TaxonomyField]).
  21: Invoke($ctx, $field)
  22:     $taxField.SspId = $termStoreId
  23:     $taxField.TermSetId = $termSetId
  24:     $taxField.TargetTemplate = ""
  25:     $taxField.AnchorId = [System.Guid]::Empty
  26:     $taxField.UpdateAndPushChanges($true)
  27:     $ctx.ExecuteQuery()
  28:     Write-Host "Field" $fieldId "was successfully bound to termset"
  29: $termSetId -foregroundcolor green
  30: }
  31:  
  32: function Bind-Managed-Metadata-Fields($ctx, $xmlFilePath)
  33: {
  34:     Write-Host "Binding managed metadata fields to term sets"
  35: -foregroundcolor green
  36:     [xml]$xmlContent = (Get-Content $xmlFilePath)
  37:     if (-not $xmlContent)
  38:     {
  39:         Write-Host "Xml was not loaded successfully. "
  40: "Fields won't be bound to term sets" -foregroundcolor Red
  41:         return
  42:     }
  43:     $termStore = Get-TermStore $ctx
  44:     $groups = $termStore.Groups
  45:     $ctx.Load($groups)
  46:     $ctx.ExecuteQuery()
  47:     $group = $groups | Where-Object {$_.Name -eq $xmlContent.Group.Name}
  48:     if (-not $group)
  49:     {
  50:         Write-Host "Group" $xmlContent.Group.Name "not found. "
  51: "Fields won't be bound to term sets" -foregroundcolor Red
  52:         return
  53:     }
  54:  
  55:     Bind-Managed-Metadata-Field $ctx $termStore.Id "{field1 id}" "{term set1 id}"
  56:     Bind-Managed-Metadata-Field $ctx $termStore.Id "{field2 id}" "{term set2 id}"
  57:     Bind-Managed-Metadata-Field $ctx $termStore.Id "{field2 id}" "{term set3 id}"
  58: }

Method Bind-Managed-Metadata-Field() which is shown on lines 1-30 makes the actual binding. Its code is quite obvious, the only interesting thing is how to call clientContext.CastTo<TaxonomyField>() generic method in PowerShell. It is shown on lines 19-21. As we know ids of the term sets (see above) we may just specify these ids when call Bind-Managed-Metadata-Field() for our fields (lines 55-57). After that managed metadata fields will be bound to the term sets and you will be able to define values for these fields when create or update content.

As you can see for Sharepoint Online regular tasks are implemented in different way, but if you work with on-premise Sharepoint installations, there should not be a lot of problems to move to client object model. Hope that this information will help you in your work.