Saturday, September 13, 2014

Using WatiN for avoiding limitations of sandbox solutions and client object model in Sharepoint

If you developed applications for Sharepoint Online using sandbox solutions or client object model you probably know that not all things which are possible to do with farm solutions are possible in the cloud world. In such cases we need to use compromises and workarounds, e.g. instead of fully automated processes write instructions for administrators for finishing installation process manually. In this article I will write about one creative way for avoiding limitations of sandbox solutions and client object model: WatiN web application testing and automation framework.

WatiN is open source project and it is free for using in your projects. Like many other good projects in .Net stack it was migrated from WatiR framework from Ruby world. It allows to simulate actions which administrator may do in the web browser via UI, e.g. open particular pages, type text into textboxes, check/uncheck checkboxes, click buttons and others. The idea of using it in Sharepoint came to me when I faced with one task for which didn’t find solution neither via sandbox code nor by client object model. Suppose that we have custom content type Project Meeting inherited from Document Set. As you probably know for such content types in Sharepoint there is special Document Set settings page accessible on content type details page:

image

On this page we may specify additional settings of the document set (technically document set is advanced folder. Internally it inherits OTB Folder content type and adds additional functionality):

  • Allowed content types – list of available content types which can be added into document set;
  • Default content – default content type shown first in New document button in the ribbon when user is inside document set. Also it is used when user clicks New document from the list view inside document set;
  • Shared columns – columns of the current document set which will be automatically inherited to the documents inside this document set (if content types of these documents have the same columns);
  • Welcome page columns – fields of the current document set’s content type which will be shown on the document set’s welcome page which is displayed when user clicks on the document set inside the list (this is one of the big differences between document set and folder: when user clicks on the folder regular list view is opened).

The page looks like this:

image

In our example we will use custom content type for document set Project Meeting which extends parent CT with the following fields:

  • Meeting Name
  • Meeting Date

Inside this document set we will allow to create only documents which have another custom content type Project Meeting Document which inherits OTB Document content type and among with other metadata also have Meeting Name and Meeting Date fields. As document set and document content types have the same common fields (Meeting Name/Date) we may inherit them from document set to documents.

From UI everything looks quite straightforward, problems begin when we will try to automate the configuration of document sets and do it e.g. during provisioning. If we will check codebehind of the mentioned Document Set settings page we will see that configuration is mostly done via DocumentSetTemplate class from Microsoft.Office.DocumentManagement assembly and its properties. Compilation of the same code won’t cause errors in sandbox code, but in runtime we will get exception that this assembly doesn’t allow partially trusted callers. In Sharepoint 2013 there is also client version of this assembly Microsoft.SharePoint.Client.DocumentManagement, but investigation shows that it is very limited: has 4 classes including DocumentSet which contains only method Create(). This is not enough for working with document set settings in Sharepoint Online programmatically. And this is where I thought about WatiN library (to be true my first thought was to simulate raw HTTP POST request programmatically, but after I checked in Fiddler that it uses content-disposition: form-data for multi-part files upload I quickly switched to less extreme approach).

One of the problem with sandbox solutions is that when you deactivate it Web-scoped features from this solution are automatically deactivated as well on all sub sites in the current site collection (see Reactivate Web-scoped features from PowerShell using client object model in Sharepoint Online). When you activate solution back you need to reactivate these features on sub sites and in mentioned article I showed how to do it. But sometimes it is not enough: related with document sets after reactivation of sandbox solution settings of custom document sets defined in this solution are reset to default. I.e. in our example they will look like this:

image

As you can see instead of custom allowed content type it uses OTB Document now and custom document set columns are not inherited anymore. Of course for solving this problem we may write instructions for administrators and rely on them that during each solution update they will change settings of all documents sets back to the correct values manually. But we may go by different way and even without having appropriate mechanisms in sandbox code and client object mode automate configuration of document sets by using WatiN.

Here is the working script written on C# with WatiN which makes the same actions which administrator need to do for configuring documents sets after solution update:

   1: public struct ThreadParams
   2: {
   3:     public string SiteUrl;
   4:     public string DocSetCTId;
   5:     public string DocCTId;
   6: }
   7:  
   8: public static void Run(string siteUrl, string docSetCTId,
   9:     string docCTId)
  10: {
  11:     var t = new Thread(runImpl);
  12:     t.SetApartmentState(ApartmentState.STA);
  13:     t.Start(new ThreadParams {SiteUrl = siteUrl, DocSetCTId = docSetCTId,
  14:         DocCTId = docCTId});
  15:     t.Join();
  16: }
  17:  
  18: private static void runImpl(object arg)
  19: {
  20:     var p = (ThreadParams)arg;
  21:     string url = SPUrlUtility.CombineUrl(p.SiteUrl,
  22:         "/_layouts/15/docsetsettings.aspx?ctype=" + p.DocSetCTId);
  23:     using (var b = new IE(url))
  24:     {
  25:         // Check whether custom content type was already configured
  26:         // as allowed content type for document set
  27:         var option = b.SelectList(l => l.Id.EndsWith("SelectResult"))
  28:             .Option(o => o.Value == p.DocCTId);
  29:         if (option != null && option.Exists)
  30:         {
  31:             return;
  32:         }
  33:  
  34:         // Select custom Workspaces group in the list. It will limit
  35:         // number of content types in the list below
  36:         b.SelectList(l => l.Id.EndsWith("SelectGroup"))
  37:             .Select("Workspaces");
  38:  
  39:         // Remove selection from left list of allowed content types.
  40:         // This code works not for all lists because of some reason
  41:         b.SelectList(l => l.Id.EndsWith("SelectCandidate"))
  42:             .Option(o => true).Clear();
  43:  
  44:         // Select custom content type Project Meeting Document
  45:         b.SelectList(l => l.Id.EndsWith("SelectCandidate"))
  46:             .SelectByValue(p.DocCTId);
  47:  
  48:         // Click on Add button and move custom CT to the right list.
  49:         // After that CT appears in Default Content section
  50:         b.Button(btn => btn.Id.EndsWith("AddButton")).Click();
  51:  
  52:         // Set custom CT in Default Content section
  53:         b.SelectList(l => l.Id.EndsWith("selAllowedCTs"))
  54:             .SelectByValue(p.DocCTId);
  55:  
  56:         // Remove OTB Document CT from the right list in Allowed
  57:         // content types. It became possible to do after we changed
  58:         // Default content type. Otherwise it would show
  59:         // Content type is in use error
  60:         b.SelectList(l => l.Id.EndsWith("SelectResult"))
  61:             .SelectByValue("0x0101");
  62:  
  63:         // Removes both content types from the right list
  64:         // (I didn't find a quick way to deselect items from there)
  65:         b.Button(btn => btn.Id.EndsWith("RemoveButton")).Click();
  66:  
  67:         // Adds single custom CT as custom group is selected
  68:         b.Button(btn => btn.Id.EndsWith("AddButton")).Click();
  69:  
  70:         // Check Meeting Name and Meeting Date shared columns
  71:         b.CheckBox(chk => chk.Id.EndsWith("chkSyncField") &&
  72:             chk.Parent.GetAttributeValue("Title") == "Meeting Name")
  73:                 .Click();
  74:         b.CheckBox(chk => chk.Id.EndsWith("chkSyncField") &&
  75:             chk.Parent.GetAttributeValue("Title") == "Meeting Date")
  76:                 .Click();
  77:  
  78:         // Optionally show these fields on Document set's
  79:         // welcome page: select it from the left list and add to right
  80:         b.SelectList(l => l.Id.EndsWith("FldsSelectCandidate"))
  81:             .Option(o => true).Clear();
  82:         b.SelectList(l => l.Id.EndsWith("FldsSelectCandidate"))
  83:             .Select("Meeting Name");
  84:         b.Button(btn => btn.Id.EndsWith("AddFldButton")).Click();
  85:         b.SelectList(l => l.Id.EndsWith("FldsSelectCandidate"))
  86:             .Select("Meeting Date");
  87:         b.Button(btn => btn.Id.EndsWith("AddFldButton")).Click();
  88:  
  89:         // Click on Ok button
  90:         b.Button(btn => btn.Id.EndsWith("btnOK")).Click();
  91:         b.WaitForComplete();
  92:     }
  93: }

I added a lot of comments to the program so I think it is quite self explanatory. Notice the way how the program is running: separate thread is created with STA apartment state (lines 11-15). If you will run this automation code in primary app thread you will get the following error:

The CurrentThread needs to have it's ApartmentState set to ApartmentState.STA to be able to automate Internet Explorer.

Having this code you will just need to run it (it is also quite easy to call it from PowerShell) and see how things happen on opened IE window by themselves without your interaction. Looks very impressive.

And few words about Sharepoint Online case. As you probably know before to do something there you need to authenticate yourself on login page https://login.microsoftonline.com with nice picture:

image

It adds one additional step to our automation for authenticating ourselves:

   1: string url = SPUrlUtility.CombineUrl(p.SiteUrl,
   2:     "/_layouts/15/docsetsettings.aspx?ctype=" + p.DocSetCTId);
   3: using (var b = new IE(url))
   4: {
   5:     // Type user name
   6:     b.ElementOfType<TextFieldExtended>("cred_userid_inputtext")
   7:         .TypeTextQuickly("username");
   8:     // Type password
   9:     b.TextField("cred_password_inputtext").Focus();
  10:     b.TextField("cred_password_inputtext").TypeTextQuickly("password");
  11:     // Because of some reason 1 Enter click simulation is not enough
  12:     b.TextField("cred_password_inputtext").PressEnter();
  13:     b.TextField("cred_password_inputtext").PressEnter();
  14:     b.WaitForComplete();
  15:     ...
  16: }

As O365 login page uses HTML5 input element with type=”email” we need to use extended text field class for working with it in WatiN (natively it doesn’t support HTML5 fields). I found this trick on one of the StackOverflow discussions. Also we will use extension for simulating Enter key press on password textbox (WatiN don’t have native methods for that as well, or more correctly they don’t work like they work in original Ruby WatiR parent of WatiN). Here are all extensions needed for running this code:

   1: [ElementTag("input", InputType = "text", Index = 0)]
   2: [ElementTag("input", InputType = "password", Index = 1)]
   3: [ElementTag("input", InputType = "textarea", Index = 2)]
   4: [ElementTag("input", InputType = "hidden", Index = 3)]
   5: [ElementTag("textarea", Index = 4)]
   6: [ElementTag("input", InputType = "email", Index = 5)]
   7: [ElementTag("input", InputType = "url", Index = 6)]
   8: [ElementTag("input", InputType = "number", Index = 7)]
   9: [ElementTag("input", InputType = "range", Index = 8)]
  10: [ElementTag("input", InputType = "search", Index = 9)]
  11: [ElementTag("input", InputType = "color", Index = 10)]
  12: public class TextFieldExtended : TextField
  13: {
  14:     public TextFieldExtended(DomContainer domContainer,
  15:         INativeElement element)
  16:         : base(domContainer, element)
  17:     {
  18:     }
  19:  
  20:     public TextFieldExtended(DomContainer domContainer,
  21:         ElementFinder finder)
  22:         : base(domContainer, finder)
  23:     {
  24:     }
  25:  
  26:     public static void Register()
  27:     {
  28:         Type typeToRegister = typeof(TextFieldExtended);
  29:         ElementFactory.RegisterElementType(typeToRegister);
  30:     }
  31: }
  32:  
  33: public static class MyWatiNExtensions
  34: {
  35:     [DllImport("user32.dll")]
  36:     private static extern IntPtr SetFocus(IntPtr hWnd);
  37:  
  38:     [DllImport("user32.dll")]
  39:     [return: MarshalAs(UnmanagedType.Bool)]
  40:     private static extern bool SetForegroundWindow(IntPtr hWnd);
  41:  
  42:     public static void TypeTextQuickly(this TextField textField,
  43:         string text)
  44:     {
  45:         textField.SetAttributeValue("value", text);
  46:     }
  47:  
  48:     public static void TypeTextQuickly(this TextFieldExtended textField,
  49:         string text)
  50:     {
  51:         textField.SetAttributeValue("value", text);
  52:     }
  53:  
  54:     public static void PressEnter(this TextField textField)
  55:     {
  56:         SetForegroundWindow(textField.DomContainer.hWnd);
  57:         SetFocus(textField.DomContainer.hWnd);
  58:         textField.Focus();
  59:         System.Windows.Forms.SendKeys.SendWait("{ENTER}");
  60:         Thread.Sleep(1000);
  61:     }
  62: }

With all of that you will be able to run your web automation code for configuring document set settings (or any other scenario) also in Sharepoint Online.

Even so there may be more correct ways to implement particular example used in this article in Sharepoint (while I didn’t find them), approach shown here is quite creative and may help you to automate very rich set of tasks and will make your set of Sharepoint experience wider.