Tuesday, October 30, 2012

Fix NullReferenceException when use user profiles service application without Sharepoint context

In some cases you may need to use Sharepoint object model outside of the Sharepoint context (i.e. when SPContext.Current is null) and without http context (HttpContext.Current is null), e.g. from console application. If internal methods of the API require not nullable SPContext.Current, you may fake it using the following method (see my another post Override SPContext.Current.Web and Site with sites opened with elevated privileges for explanations why it works like this):

   1: using (var site = new SPSite(args[0]))
   2: {
   3:     using (var web = site.OpenWeb())
   4:     {
   5:         HttpRequest request = new HttpRequest("", web.Url, "");
   6:         HttpContext.Current = new HttpContext(request,
   7:             new HttpResponse(TextWriter.Null));
   8:         HttpContext.Current.Items["HttpHandlerSPWeb"] = web;
   9:         ...
  10:     }
  11: }

After this SPContext.Current won’t be null, and code which requires it will work. However it brings one problem. If you will try to use User profiles service application with such fake context, you will get NullReferenceException. For example, suppose that we need to enumerate all imported user profiles using the following code:

   1: using (var site = new SPSite(url))
   2: {
   3:     var serviceContext = SPServiceContext.GetContext(site);
   4:     var upm = new UserProfileManager(serviceContext);
   5:     foreach (UserProfile profile in upm)
   6:     {
   7:         ...
   8:     }
   9: }

When you will run it in console application you will get NullReferenceException with the following stack trace:

at Microsoft.Office.Server.UserProfiles.UserProfileGlobal.IsWindowsAuth()
at Microsoft.Office.Server.UserProfiles.UserProfileManager..ctor(SPServiceContext serviceContext, Boolean IgnoreUserPrivacy, Boolean backwardCompatible)
at Microsoft.Office.Server.UserProfiles.UserProfileManager..ctor(SPServiceContext serviceContext)

Let’s check implementation of UserProfileGlobal.IsWindowsAuth() method which causes exception:

   1: internal static bool IsWindowsAuth()
   2: {
   3:     HttpContext current = HttpContext.Current;
   4:     if (current != null)
   5:     {
   6:         return (current.User.Identity is WindowsIdentity);
   7:     }
   8:     return true;
   9: }

As you replace context with fake instance, condition on line 4 will be true and execution will go to the line 6 where it will fail, because context.User is null. In order to fix it you have to specify user when you fake HttpContext. The way how you should do that depends on the authentication type used in your web application: standard or claims based.

For standard windows authentication you should use the following code:

   1: AppDomain.CurrentDomain.SetPrincipalPolicy(PrincipalPolicy.WindowsPrincipal);
   2: ...
   3: HttpRequest request = new HttpRequest("", web.Url, "");
   4: HttpContext.Current = new HttpContext(request, new HttpResponse(TextWriter.Null));
   5: HttpContext.Current.Items["HttpHandlerSPWeb"] = web;
   6: HttpContext.Current.User = Thread.CurrentPrincipal as WindowsPrincipal;

For claims based use the following code:

   1: HttpRequest request = new HttpRequest("", web.Url, "");
   2: HttpContext.Current = new HttpContext(request, new HttpResponse(TextWriter.Null));
   3: HttpContext.Current.Items["HttpHandlerSPWeb"] = web;
   4: HttpContext.Current.User = new ClaimsPrincipal(Thread.CurrentPrincipal);

After that you will be able to use UserProfileManager and work with user profiles in the console application.

Saturday, October 20, 2012

Hide content type fields in Orchard CMS

In Orchard CMS there is a concept of content types. From high vision it is similar to the content types idea in Sharepoint. It has less features of course, but the general idea is the same: you may create your own content types, add new content parts or fields and then create content items based on created content type. Content parts can be found from Dashboard > Content > Content Parts:


You can add fields of the following types (available in 1.5.1 version):

  • Boolean
  • Content picker (allows you to select content items like pages and display them on page)
  • Date time
  • Enumeration (shows list of predefined values. You can choose how you want to display values: using listbox, dropdown list, radio button list, checkboxes list)
  • Input (allows to add input fields to your content type, add validation for phones and emails, define max length, watermark, custom css and other features)
  • Link
  • Media picker (allows to show items from Media folder, e.g. images)
  • Numeric
  • Text (can be plain text, multiline text, html)

In order to add new field you should go to the Dashboard > Content > Content types > Edit content type > Add field:


What is missing in the current implementation is the ability to hide added fields. E.g. if you need to add metadata for displaying in liftups (or projections in terms of Orchard), but not in Detail view. In Sharepoint you can define fields visibility for each form separately using the following properties of SPField class: ShowInDisplayForm, ShowInEditForm, ShowInNewForm, ShowInViewForms. So how to hide field in e.g. detail view (default view which is used when you open e.g. the page)? I found the following way.

First of all you need to install Designer Tools module and enable Shape Tracing feature in it:


At the moment of writing Designer Tools module also has Url Alternates and Widget Alternates features. Once you enabled Shape Tracing, you will see the following icon in the right bottom corner of the page:


If you will click on it you will see panel similar to the firebug or IE developer tools:


Now you can check all shapes on your page using convenient visual representation like you do in firebug. In order to hide the field at first find its shape on the page. Here you can create alternate template for displaying appropriate field. Click Create button near the template name which is more suitable for you. After that panel will look like that:


Active template shows the path to the View which is currently used for rendering the field:

Active Template: ~/Themes/Terra/Views/Fields.Common.Text-Logo.cshtml

Now go to that folder and check the code of the view. For text fields it looks like this:

   1: @using Orchard.Utility.Extensions;
   2: @{
   3:     string name = Model.ContentField.DisplayName;
   4: }
   6: if (HasText(name) && HasText(Model.Value)) {
   7:     <p class="text-field"><span class="name">@name:</span>
   8:     <span class="value">@(new MvcHtmlString(Html.Encode((HtmlString) Model.Value)
   9:         .ReplaceNewLinesWith("<br />$1")))</span></p>
  10: }

In order to hide the field we need to modify the view a bit:

   1: @using Orchard.Utility.Extensions;
   2: @{
   3:     string name = Model.ContentField.DisplayName;
   4: }
   6: @if (Model.Metadata.DisplayType != "Detail") {
   7:     if (HasText(name) && HasText(Model.Value)) {
   8:         <p class="text-field"><span class="name">@name:</span>
   9:         <span class="value">@(new MvcHtmlString(Html.Encode((HtmlString) Model.Value)
  10:             .ReplaceNewLinesWith("<br />$1")))</span></p>
  11:     }
  12: }

Here on line 6 I added additional condition: @if (Model.Metadata.DisplayType != "Detail"). So view will be rendered only in other than Detail display types. After that you will have content items with metadata which you can use e.g. only in projections, but which won’t be shown in default view.

May be there is better and faster way to do that in Orchard, but I didn’t find it yet. If you know other ways to achieve the same result please share them in comments.

Saturday, October 13, 2012

Camlex.Net for client object model is released

I’m glad to announce that today we’ve released Camlex version for client object model (managed clients). Special credits go to Stef Heyenrath who joined Camlex project and did the main work for releasing of client object model. He will maintain client object model from now. Also I would like to thanks user with nickname Scriv, who created this feature request on Camlex site on Codeplex and showed example of how basic Camlex can be transformed to the client object model.

You can download client object model from codeplex or using nuget:

   1: Install-Package Camlex.Client.dll

Client version is based on basic Camlex.Net 3.2. It contains all features which basic Camlex has (except those which are mentioned below) and is also fully reversible. The following changes were done:

  • References are changed Microsoft.SharePoint.dll to Microsoft.SharePoint.Client.dll and Microsoft.SharePoint.Client.Runtime.dll
  • Use ListItem instead of SPListItem
  • Remove guid and int indexers support as they don't exist in ListItem
  • Modify .ToCaml() to only output xml with a surrounding View and Query elements
  • Remove IQueryEx interface which contained all ViewFields methods. In SPQuery there is separate property ViewFields, so you can’t specify both Query and ViewFields in the same fluent methods chain. But in client version everything is specified in single property CamlQuery.ViewXml, so this separate interface is not needed there
  • Method ToCamlQuery() was added to IQuery interface which returns ready for use CamlQuery instance.

Then all unit tests were fixed. Here is example of the usage:

   1: var camlQuery = Camlex.Query()
   2:     .Where(x => (string)x["Title"] == "test")
   3:     .ToCamlQuery();

It will return instance of CamlQuery class which ViewXml property will be set to the following CAML query:

   1: <View>
   2:     <Query>
   3:         <Where>
   4:             <Eq>
   5:                 <FieldRef Name=\"Title\" />
   6:                 <Value Type=\"Text\">test</Value>
   7:             </Eq>
   8:         </Where>
   9:     </Query>
  10: </View>

We created separate branch for client object model. Starting from now all new features in basic Camlex will be reviewed to check whether they can be added to the client version and vise versa. I.e. they will be maintained in sync with each other.

Saturday, October 6, 2012

Problems with upgrading claims based site from Sharepoint 2010 to Sharepoint 2013 Preview

Recently we upgraded several sites which were implemented for Sharepoint 2010 to Sharepoint 2013 Preview. Upgrade was done according to MS guides. Existing SP2010 sites use claims based authentication. So we made backup of existing content database from SP2010 farm, copied them to SP2013 farm and restored on database server. After that we created web applications on SP2013 farm also using claims based authentication (we created them from UI and as you probably know in SP2013 claims based authentication is the only option when you create web application from Central administration. If you want to create web application with classic Windows authentication you have to use PowerShell).

After that we used Test-SPContentDatabase cmdlet in order to check restored SP2010 content database against SP2013 web application (at that moment SP2010 database was not mounted yet to SP2013 web application. The last one used its own content database which was created with web application). Test-SPContentDatabase gives several messages about missing assemblies which was Ok because custom wsp packages were not deployed to SP2013 farm yet. The most interesting was the last message:

Category Configuration
Error False
UpgradeBlocking False

The [web_app_name] web application is configured with claims authentication mode however the content database you are trying to attach is intended to be used against a windows classic authentication mode.


There is an inconsistency between the authentication mode of target web application and the source web application. Ensure that the authentication mode setting in upgraded web application is the same as what you had in previous SharePoint 2010 web application. Refer to the link "http://go.microsoft.com/fwlink/?LinkId=236865" for more information.


So it says that target web app on SP2013 farm is configured to use claims based authentication, but source content database from SP2010 supposed to be used with classic Windows authentication. It is not correct, because as I said migrated SP2010 web sites use claims based authentication as well.

Ok, but as you can see UpgradeBlocking property was false for this message, we ignored this message and mounted content database to the web application using Mount-SPContentDatabase cmdlet. As you probably know this command also performs actual upgrade of content database schema from SP2010 to SP2013. Command ran successfully, but number of site collections for mounted content database was shown as 0 (while actually there should be 1 site collection) and site didn’t work.

After that I thought that if it says that content database should be used with classic Windows authentication, we should try to create such web application in SP2013 farm and run Test-SPContentDatabase again. So we created web application with classic Windows authentication using PowerShell, restored SP2010 content database again (it was upgraded to SP2013 schema during previous run of Mount-SPContentDatabase cmdlet) and ran Test-SPContentDatabase. Now it showed opposite error:

Category Configuration
Error False
UpgradeBlocking False

The [web_app_name] web application is configured with windows classic authentication mode however the content database you are trying to attach is intended to be used against a claims authentication mode.


There is an inconsistency between the authentication mode of target web application and the source web application. Ensure that the authentication mode setting in upgraded web application is the same as what you had in previous SharePoint 2010 web application. Refer to the link "http://go.microsoft.com/fwlink/?LinkId=236865" for more information.


This message is actually correct, but the fact that it is opposite to the message which we got for the same database first time makes me think that this is a bug in Test-SPContentDatabase.

We ignored this message again and ran Mount-SPContentDatabase for claims based database from SP2010 farm and Windows classic authentication web application from SP2013 farm. This time number of site collections was set to 1 which was correct. But web application used Windows classic mode, so the last step was migrating it to use claims based authentication as described in the article which link was in the messages shown above: Migrate from classic-mode to claims-based authentication. After that upgraded sites were opened successfully. This is so-called hybrid mode when sites work in SP2010 mode on SP2013 farm (for this purpose folder “14” is included into SP2013 installation).

This is how we performed upgrade of claims based authentication sites from SP2010 to SP2013. It may be so that these problems are caused by Preview version of Sharepoint 2013. If you will face with similar problems in would be great if you will share them in comments here.

Friday, October 5, 2012

Analyze code in Sharepoint projects with NDepend

In this post I will show how you may use NDepend tool for analyzing the code in your Sharepoint projects. When refer to Sharepoint I will mean Sharepoint 2010 mostly, but of course NDepend can be used with any version of Sharepoint, because it can analyze .Net assemblies v1.0 v1.1 v2.0 v3.0 v3.5 v4.0 v4.5 and Silverlight. I wanted to try it on some real task for a long time and recently had a chance to do that. Especially it is worth to mention that authors of NDepend give free license if you are MVP awarded, so I had everything I needed for starting using it :).

NDepend is very powerful tool, it contains many nice looking graphic reports and analysis statistics, and one article is not enough for showing all its features. I won’t describe all features here, instead I will show one particular use case in this article which however shows how useful it can be. There is one great feature called CQLinq (Code Query LINQ) which allows you to write LINQ queries to the codebase by the same way as you do when write LINQ query to objects, Sql or other data. The only difference is that codebase plays role of database.

Returning to the use case. On one of my projects I needed to analyze the code and find all custom classes which inherit standard Page and Control classes from System.Web.UI namespace and which contains static fields (no matter public or private). Static variables in codebehind class of the pages or controls may lead to potential problems with concurrent access, because they are shared across all instances of the class. Especially it will be hard to catch such errors in Sharepoint web applications where we often have several WFEs and code may be executed on different servers (it may hide problem even deeper). But of course it is just a warning. It is not necessary that static variables will cause such errors. Anyway we needed to review all of them, so I decided to use NDepend for that.

Let’s start with launching NDepend and creating new project:


It shows Project properties window where we can select VS project, solution or separate assembly which we want to analyze. For example if we want to analyze whole solution we should select “Add assemblies of VS solutions” button:


Convenient thing that NDepend will show you list of recently used solution in VS, so you don’t have to browse for them if you worked with them recently. It supports the following VS versions: 2005, 2008, 2010 and 2012.

After that all found assemblies are shown in the Project properties window (so solution should be compiled first). Important thing: by default NDepend uses target framework set to 4.0 or 4.5 versions. It means that if e.g. you are opening solution for Sharepoint 2010 (which is built on top of .Net 3.5), standard Microsoft assemblies like Microsoft.SharePoint.dll won’t be loaded and analyzed. You will see the following error:


In turn it means that when you will use CQLinq, types defined in this assemblies won’t be shown. E.g. if you have custom application layouts page which class inherits LayoutsPageBase class from Microsoft.SharePoint.ApplicationPages.dll, then its base class will be shown as null in NDepend.

In order to fix it at first you need to change target framework to 3.5. You can do it if will click “View folders that contain .Net assemblies to analyze” button:


Unfortunately after that Microsoft.SharePoint.* assemblies are still remain unresolved. I used the following workaround: copy necessary assemblies from C:\Windows\Assembly\GAC_MSIL to temp folder, and then select it in “Add .net assemblies in folder” window which is opened from the same Project properties window. After I specified my temp folder all Sharepoint assemblies became resolved. The last step was selecting necessary Microsoft.SharePoint.* assemblies for analysis by clicking “Move selected to application assemblies” on each of them.

At this point you have everything ready for analysis. Click “Run analysis on current project” button on top of Project properties window or press F5. NDepend will analyze codebase and will show you a lot of statistics. For us it is important that after analysis it will allow you to write CQLinq queries to the codebase. When analysis will be done it will show you the following window where you can choose the next steps:


In our case we need to select “Browse NDepend interactive code rules”. After that in the left doc window choose “Queries and rules edit” and write the following query:

   1: let page = ThirdParty.Types.WithFullName("System.Web.UI.Page").FirstOrDefault()
   2: let control = ThirdParty.Types.WithFullName("System.Web.UI.Control").FirstOrDefault()
   4: from t in JustMyCode.Types
   5: where (t.DeriveFrom(page) || t.DeriveFrom(control)) &&
   6:     t.FullNameLike("MyNamespace") &&
   7:     t.StaticFields.Any(f => !f.IsGeneratedByCompiler)
   8: orderby t.Name
   9: select new { t, t.FullName, StaticFields =
  10:     string.Join(";", t.StaticFields.Where(f => !f.IsGeneratedByCompiler)
  11:         .Select(f => f.Name).ToArray()) }

Syntax of CQLinq expressions is described here. On lines 1 and 2 we define variables for page and control types which we will use in query. Then on lines 4-9 we write actual query to the codebase. So we select all custom types (from t in JustMyCode.Types) which inherit Page or Control class from specified namespace (t.FullNameLike("MyNamespace")) which has at least one non-compiler generated static field (t.StaticFields.Any(f => !f.IsGeneratedByCompiler)). Then we just select new anonymous object with convenient properties representation: we join names of all static properties using semicolon (select new { t, t.FullName, StaticFields = string.Join(";", t.StaticFields.Where(f => !f.IsGeneratedByCompiler).Select(f => f.Name).ToArray()) }). As result we will have list of all classes which inherit Page or Control with list of static variables for each class. It will show you also classes which have const strings because it is treated as static field, I didn’t find a quick way to get rid of it, because it was not crucial. Already this report contained almost everything I needed. Report can be exported to html, excel, text or even graphics representation.

As you can see NDepend may be very helpful in such task where you need to analyze codebase for some specific usages. It uses LINQ queries familiar to most .Net developers, so you won’t have big problems with writing them if you worked with LINQ before. Hope this post will help you if you will want to use NDepend with Sharepoint and will be good starting point for using NDepend in your work.