Wednesday, January 29, 2014

Fix problem with not saving navigation settings in Sharepoint or working with navigation on content database level

Some time when you work with publishing sites in Sharepoint you may encounter with strange problem: when you change navigation settings (e.g. hide some navigation node or rearrange them for global or current navigation) in Site settings > Navigation and click Ok, changes aren’t saved, i.e. when you will go to navigation settings page again you will see the same parameters as before. Interesting that problem exist only for navigation nodes in global and current navigation. Other navigation settings, like “Show sub sites” or “Show pages” are saved correctly. Another interesting moment is that if you will try to access navigation nodes programmatically (via PublishingWeb.Navigation.GlobalNavigationNodes or PublishingWeb.Navigation.CurrentNavigationNodes) they will be null even after you explicitly clicked Ok. If you worked with Sharepoint navigation a lot this fact will surprise you, because this is the known Sharepoint behavior when navigation nodes are populated only when you first time click Ok button in navigation settings page (see e.g. Change order of newly created publishing page or publishing site in navigation programmatically in Sharepoint).

This problem often happens when you export/import the site with stsadm utility (see Export: Stsadm operation and Import: Stsadm operation). This problem will lead us to the lower content database level and workarounds which will be described in this article will require manual change of content database. As you know MS doesn’t support such way of working with Sharepoint, so you may use it on your own risk. However information described here will help to understand internal Sharepoint mechanisms better, so I suggest to read it anyway.

Ok, if we will check the logs after we tried to save navigation settings, we may find the following errors:

Unable to retrieve TopNavigationBar SPNavigationNodeCollection from Web at: The SPNavigation store is likely corrupt.


Unable to retrieve QuickLaunch SPNavigationNodeCollection from Web at: The SPNavigation store is likely corrupt.

(first one about global navigation, second about current). These errors come from codebehind of AreaNavigationSettings.aspx applications layout page which is opened when you go to Site settings > Navigation: its AreaNavigationSettingsPage.OKButton_Click() method:

   1:  protected void OKButton_Click(object sender, EventArgs e)
   2:  {
   3:          ...
   4:          SPNavigationNodeCollection collection = null;
   5:          if (!publishingWeb.Navigation.InheritGlobal)
   6:          {
   7:              collection = publishingWeb.Navigation.GlobalNavigationNodes;
   8:              if (collection != null)
   9:              {
  10:                  PopulateNavigationNodeDictionary(dictionary, collection, 0, 1);
  11:              }
  12:              else
  13:              {
  14:                  ULS.SendTraceTag(0x38747331, ULSCat.msoulscat_CMS_Publishing,
  15:  ULSTraceLevel.Unexpected,
  16:  "Unable to retrieve TopNavigationBar SPNavigationNodeCollection from Web at: {0}." +
  17:  "The SPNavigation store is likely corrupt.",
  18:  new object[] { publishingWeb.Url });
  19:              }
  20:          ...
  21:          collection = publishingWeb.Navigation.CurrentNavigationNodes;
  22:          if (collection != null)
  23:          {
  24:              PopulateNavigationNodeDictionary(dictionary, collection, 0, 1);
  25:          }
  26:          else
  27:          {
  28:              ULS.SendTraceTag(0x38747332, ULSCat.msoulscat_CMS_Publishing,
  29:  ULSTraceLevel.Unexpected,
  30:  "Unable to retrieve QuickLaunch SPNavigationNodeCollection from Web at: {0}." +
  31:  "The SPNavigation store is likely corrupt.",
  32:  new object[] { publishingWeb.Url });
  33:          }
  34:          ...
  35:  }

As you can see it happens when PublishingWeb.Navigation.GlobalNavigationNodes or PublishingWeb.Navigation.CurrentNavigationNodes is null. If you will search for mentioned error messages, you will find that this problem may be caused by missing NavBar element in onet.xml:

   1:  <NavBars>
   2:      <NavBar Name="SharePoint Top Navbar" ID="1002">
   3:      </NavBar>
   4:   </NavBars>

If you exported and imported the site this information won’t help a lot because you don’t have access to onet.xml and in the source site navigation works properly.

Also you may find the following workaround for this problem: you need to execute the following SQL query in the content database:

   1:  INSERT INTO [NavNodes]
   2:  ([SiteId], [WebId], [Eid], [EidParent], [NumChildren], [RankChild],[ElementType],
   3:  [Url], [DocId], [Name], [DateLastModified], [NodeMetainfo], [NonNavPage],
   4:  [NavSequence], [ChildOfSequence])
   5:  SELECT DISTINCT SiteId, WebId ,1002 ,0 ,0 ,1 ,1 ,'', NULL, 'SharePoint Top Navbar',
   6:  getdate() ,NULL ,0 ,1 ,0
   7:  FROM NavNodes
   8:  WHERE WebId NOT IN (SELECT WebId FROM NavNodes WHERE Eid = 1002)

without explanation of what it does and why it will fix the problem. Also this method may fix only global navigation problem, while current navigation requires another query. I will try to fill in this space and will describe safer method which will fix both global and current navigation.

In order to continue we need to use some example. Let’s suppose that we have SPWeb with working navigation settings, which have several navigation nodes in global navigation and in current navigation:

Global navigation:

  • Site 1
  • Page 1

Current navigation:

  • Site 2
  • Page 2

Navigation nodes are stored in NavNodes table in content database, which has the following structure (it may be slightly different in different Sharepoint versions. This example uses schema from Sharepoint 2007 as most simple. But the same information may be used also in Sharepoint 2010 and 2013. They just have more columns in this table):


In order to query navigation nodes for particular web we need to know ID of parent SPSite and ID of SPWeb itself. They can be retrieved e.g. by PowerShell:

   1:  $w = Get-SPWeb
   2:  $w.Site.ID
   3:  $w.ID

It will print the guids:

   1:  Guid
   2:  ----
   3:  {42514464-2EB2-4BB9-8A57-A5A6D80E3F67}
   5:  Guid
   6:  ----
   7:  {92524380-DAE3-4CD6-B2C1-5A30C52ABC28}

(in our example SPSite.ID = {42514464-2EB2-4BB9-8A57-A5A6D80E3F67} and SPWeb.ID = {92524380-DAE3-4CD6-B2C1-5A30C52ABC28}). Now we can read the navigation items from NavNodes table:

   1:  select * from dbo.NavNodes
   2:  where WebId = '92524380-DAE3-4CD6-B2C1-5A30C52ABC28'


SiteId WebId Eid EidParent NumChildren RankChildren ElementType Url DocId Name DateLastModified NodeMetaInfo NonNavPage NavSequence ChildOfSequence
{Site ID}
{Web ID}
0 -1 1 0 0   NULL   {DateTime} NULL 0 0 0
{Site ID} {Web ID} 1025 0 2 0 1   NULL Quick launch {DateTime} NULL 1 1 0
{Site ID} {Web ID} 1002 0 2 1 1   NULL SharePoint Top Navbar {DateTime} NULL 0 1 0
{Site ID} {Web ID} 1000 0 0 2 0 NULL {Guid} default.aspx {DateTime} NULL 0 0 0
{Site ID} {Web ID} 2002 1002 0 0 0 NULL {Guid} Site 1 {DateTime} {binary} 0 0 1
{Site ID} {Web ID} 2003 1002 0 1 0 NULL {Guid} Page 1 {DateTime} {binary} 0 0 1
{Site ID} {Web ID} 2009 1025 0 0 0 NULL {Guid} Site 2 {DateTime} {binary} 0 0 1
{Site ID} {Web ID} 2017 1025 0 1 0 NULL {Guid} Page 2 {DateTime} {binary} 0 0 1

It is classical representation of hierarchical data in relational database table: there are 2 columns for creating parent-child relationship: Eid and EidParent, plus NumOfChildren which contains number of children for each node for improving performance I guess (in our example both global and local navigation contains 2 nodes). Root node has Eid = 0 and EidParent = –1. After that we have 2 special root nodes for current and global navigation: with Eid = 1025 and Eid = 1002 respectively. This is the key moment for understanding the workaround. These 2 special navigation nodes always should exist in NavNodes table for SPWeb (at least for publishing web). If they don’t exist, navigation won’t be saved like described in the beginning of the article. And this is what may happen when you export and import your site with stsadm. If you will try to get navigation nodes using the query above, you will get only root level node with Eid = 0 (and may be node for default.aspx with Eid = 1000), but without root nodes for global and current navigation with Eid = 1002 and Eid = 1025.

Now if you will get back to workarounds mentioned earlier you will see why it is important to have NavBar with ID = 1002 in onet.xml and why mentioned SQL insert works: it finds all WebId which don’t have node with Eid = 1002 and inserts it manually. However doing it for all SPWebs in one operation is not safe. First of all remember that any manual change in content database is not supported, and second – it may not be needed depending on web template used for web site. So it is better to minimize affected sites and make changes only for single problematic SPWeb. Also mentioned workarounds fix problem only for global navigation. Let’s address these problems and create more safer fix (if this word is relevant in case when we want to do manual changes in content database):

   1:  -- global navigation
   2:  INSERT INTO [NavNodes]
   3:  ([SiteId], [WebId], [Eid], [EidParent], [NumChildren], [RankChild],[ElementType],
   4:  [Url], [DocId], [Name], [DateLastModified], [NodeMetainfo], [NonNavPage],
   5:  [NavSequence], [ChildOfSequence])
   6:  values ('42514464-2EB2-4BB9-8A57-A5A6D80E3F67',
   7:  '92524380-DAE3-4CD6-B2C1-5A30C52ABC28' ,1002 ,0 ,0 ,1 ,1 ,'', NULL,
   8:  'SharePoint Top Navbar',getdate() ,NULL ,0 ,1 ,0)
  10:  -- current navigation
  11:  INSERT INTO [NavNodes]
  12:  ([SiteId], [WebId], [Eid], [EidParent], [NumChildren], [RankChild],[ElementType],
  13:  [Url], [DocId], [Name], [DateLastModified], [NodeMetainfo], [NonNavPage],
  14:  [NavSequence], [ChildOfSequence])
  15:  values ('42514464-2EB2-4BB9-8A57-A5A6D80E3F67',
  16:  '92524380-DAE3-4CD6-B2C1-5A30C52ABC28',1025 ,0 ,0 ,0 ,1 ,'', NULL,
  17:  'Quick launch',getdate() ,NULL ,1 ,1 ,0)

After that go to Site settings > Navigation, made changes in global or current navigation nodes and click save. This time changes should be preserved. If you will select all navigation nodes from NavNodes table, you will see that now it contains all structure, and if you will try to work with navigation programmatically via PublishingWeb.Navigation.GlobalNavigationNodes or PublishingWeb.Navigation.CurrentNavigationNodes, you will be able to do it. BTW it also explains the fact how internally navigation nodes are filled when you click Ok on navigation settings page first time.

This is all for the problem with not saved navigation in Sharepoint. It was interesting investigation and I hope that even you won’t use it on practice, it will help to understand Sharepoint navigation better.

Thursday, January 23, 2014

Default model binder deserializes json array with single element as null in ASP.Net MVC

Some time ago I faced with strange problem. Suppose that we have POCO which is used as action post parameter:

   1: public class RequestWrapper
   2: {
   3:     public Request request { get; set; }
   4: }
   6: public class Request
   7: {
   8:     public string id { get; set; }
   9:     public string status { get; set; }
  10:     public string notes { get; set; }
  11:     ... // a lot of other integer and string fields
  12:     public RequestItem[] items { get; set; }
  13: }
  15: public class RequestItem
  16: {
  17:     public int id { get; set; }
  18:     public string title { get; set; }
  19:     ... // a lot of other integer and string fields
  20: }

Class structure is quite straightforward, all properties are public and have getter and setter. As you can see Request contains array of RequestItems objects. And it is used as parameter of controller’s post action:

   1: [HttpPost]
   2: public ActionResult Foo(RequestWrapper request)
   3: {
   4:     ...
   5: }

The following json object is successfully passed to this action and deserialized with default model binder:

   1: {
   2:    "request":{
   3:       "id":"1",
   4:       "status":"TEST",
   5:       "notes":"",
   6:       ...
   7:       "items":[
   8:          {
   9:             "id":"2",
  10:             "title":"Test1",
  11:             ...
  12:          },
  13:          {
  14:             "id":"3",
  15:             "title":"Test2",
  16:             ...
  17:          }
  18:       ]
  19:    }
  20: }

The problems however begin when items array contains only 1 element:

   1: {
   2:    "request":{
   3:       "id":"1",
   4:       "status":"TEST",
   5:       "notes":"",
   6:       ...
   7:       "items":[
   8:          {
   9:             "id":"2",
  10:             "title":"Test1",
  11:             ...
  12:          }
  13:       ]
  14:    }
  15: }

In this case default model binder successfully deserialized all properties of Request object except items, which was set to null. Adding more items caused deserialization to work properly, i.e. items array was not empty in this case. I didn’t find mentions of this problem on forums and blogs. The only thing which I noticed that many people faced with unstable work of default model binder when json is deserialized. One of the solution was to create custom model binder and replace default deserializer with JSON.Net library. I tried first standard JavaScriptSerializer from System.Web.Extensions assembly. I.e. create custom model binder:

   2: public class RequestWrapperModelBinder : DefaultModelBinder
   3: {
   4:     public override object BindModel(ControllerContext controllerContext,
   5:         ModelBindingContext bindingContext)
   6:     {
   7:         try
   8:         {
   9:             var httpRequest = controllerContext.HttpContext.Request;
  10:             httpRequest.InputStream.Position = 0;
  11:             using (var sr = new StreamReader(httpRequest.InputStream))
  12:             {
  13:                 var str = sr.ReadToEnd();
  14:                 var request =
  15:                     new JavaScriptSerializer().Deserialize<RequestWrapper>(str);
  16:                 return request;
  17:             }
  18:         }
  19:         catch
  20:         {
  21:             return base.BindModel(controllerContext, bindingContext)
  22:                 as RequestWrapper;
  23:         }
  24:     }
  25: }

As you can see it first reads the content of post request from InputStream and tries to deserialize it via JavaScriptSerializer. If it fails and exception is thrown, it still uses default model binder. And this approach worked: i.e. array with single element became deserialized properly.

And the last thing which is needed is to register custom model binder:

   1: ModelBinders.Binders.Add(typeof(RequestWrapper), new RequestWrapperModelBinder());

However the original problem is still unclear. Need to mention that it is specific to this class only. There were another similar classes with arrays (they were more simple, i.e. contained less properties) and they were deserialized by default model binder successfully when array contained single element. So if you faced with this problem and know another solution, please share it in comments.