Build dynamic expressions for CAML queries based on list of values

Recently we had an interesting feedback related with Camlex.NET project – see http://habrahabr.ru/blogs/sharepoint/82787 (on Russian language). Community member with nick Olegas asked:

“I had an array with integer values. I need to create query which will return all items which have ID which presents in this array of integers. I.e. I need to join several <FieldRef> with different <Value> via Or operation”.

It is interesting feature and we didn’t address it in the 1st release of Camlex. Nevertheless there is a workaround for this problem which requires additional coding and is not very convenient. But it works.

The general idea is to build expression dynamically and pass it into Camlex. So we have an array of integers:

   1: var ids = new[] { 1, 2, 3 };

At first we need to create single expression which will be translated into <Eq> operation in CAML for each id in list, i.e. we need the following CAML for each element of array:

   1: <Eq>

   2:   <FieldRef Name="ID" />

   3:   <Value Type="Integer">1</Value>

   4: </Eq>

The following code shows how it can be done:

   1: var exprs = new List<Expression>();

   2: ids.ToList().ForEach(i =>

   3:                          {

   4: var expr =

   5:     Expression.Equal(

   6:         Expression.Convert(

   7:             Expression.Call(

   8:                 Expression.Parameter(typeof (SPItem), "x"),

   9:                 typeof (SPItem).GetMethod("get_Item", new[] {typeof (string)}),

  10:                 new[] {Expression.Constant("ID")}),

  11:             typeof (int)),

  12:         Expression.Constant(i));

  13: exprs.Add(expr);

  14: });

Here we construct expression “(int)x[“ID”] == i” for each i in array of integers. Now we need to combine the list of these expressions using || (OrElse) operation:

   1: var result = generateResultOrElseExpression(1, exprs, exprs[0]);

   2: var lambda = Expression.Lambda<Func<SPItem, bool>>(result, Expression.Parameter(typeof(SPItem), "x"));

   3: var caml = Camlex.Query().Where(lambda).ToString();

Function generateResultOrElseExpression() has the following code:

   1: static BinaryExpression generateResultOrElseExpression(

   2: int currentExpressionToAdd, List<Expression> allExprs, Expression prevExpr)

   3: {

   4:     if (currentExpressionToAdd >= allExprs.Count)

   5:     {

   6:         return (BinaryExpression)prevExpr;

   7:     }

   8:     var resultExpr =

   9:         Expression.OrElse(prevExpr, allExprs[currentExpressionToAdd]);

  10:     return generateResultOrElseExpression(currentExpressionToAdd + 1, allExprs, resultExpr);

  11: }

I.e. it accumulates expressions into prevExpr parameter and recursively calls itself. After that you will have the following CAML query:

   1: <Where>

   2:   <Or>

   3:     <Or>

   4:       <Eq>

   5:         <FieldRef Name="ID" />

   6:         <Value Type="Integer">1</Value>

   7:       </Eq>

   8:       <Eq>

   9:         <FieldRef Name="ID" />

  10:         <Value Type="Integer">2</Value>

  11:       </Eq>

  12:     </Or>

  13:     <Eq>

  14:       <FieldRef Name="ID" />

  15:       <Value Type="Integer">3</Value>

  16:     </Eq>

  17:   </Or>

  18: </Where>

As you see it is not so simple to achieve the result. We will try to help developers to solve such problems with less efforts in the next releases of Camlex. I’m not sure about exact view of solution, may be something like this:

   1: var caml =

   2:     Camlex.Query()

   3:         .Where(x => Camlex.ForEach(ids, i => (int)x["ID"] == i)

   4:             .JoinWithOr());

What do you think?

And thanks to Olegas for feedback.

How Sharepoint sets CurrentThread.CurrentUICulture depending on Language of SPWeb

There are 2 properties in SPWeb class which can be used for localization purposes:

1. Language – used for UI localization
2. Locale – affects regional settings like date time, currency formatting, etc.

Sharepoint uses these properties in order to set CurrentUICulture and CurrentCulture of current thread when request is executed in context of Sharepoint site. SPWeb.Language property of current site is used to set CurrentThread.CurrentUICulture and SPWeb.Locale is used to set CurrentThread.CurrentCulture. In this post I’m going to describe internal mechanisms used by Sharepoint when it initializes localization properties of current thread.

The main class which should be considered here is SPRequestModule class which represents http module. This module exists in web.config of each web application running under Sharepoint:

   1:  

   2:  

   3: <httpModules>

   4:   <clear />

   5:   <add name="SPRequest"

   6: type="Microsoft.SharePoint.ApplicationRuntime.SPRequestModule,

   7: Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral,

   8: PublicKeyToken=71e9bce111e9429c" />

   9:   ...

  10: </httpModules>

Also important point is that SPRequestModule is located on 1st place in list of http modules so it will be executed 1st in pipeline. SPRequestModule makes many important things (e.g. it registers SPVirtualPathProvider) and exactly this module initializes CurrentUICulture and CurrentCulture properties of current thread. Lets look at SPRequest.Init() method:

   1: void IHttpModule.Init(HttpApplication app)

   2: {

   3:     if (app is SPHttpApplication)

   4:     {

   5:        ...

   6:     }

   7:     else

   8:     {

   9:         return;

  10:     }

  11:     app.BeginRequest +=

  12:         new EventHandler(this.BeginRequestHandler);

  13:     app.PostResolveRequestCache +=

  14:         new EventHandler(this.PostResolveRequestCacheHandler);

  15:     app.PostMapRequestHandler +=

  16:         new EventHandler(this.PostMapRequestHandler);

  17:     app.ReleaseRequestState +=

  18:         new EventHandler(this.ReleaseRequestStateHandler);

  19:     app.PreRequestHandlerExecute +=

  20:         new EventHandler(this.PreRequestExecuteAppHandler);

  21:     app.PostRequestHandlerExecute +=

  22:         new EventHandler(this.PostRequestExecuteHandler);

  23:     app.AuthenticateRequest +=

  24:         new EventHandler(this.AuthenticateRequestHandler);

  25:     app.PostAuthenticateRequest +=

  26:         new EventHandler(this.PostAuthenticateRequestHandler);

  27:     app.Error +=

  28:         new EventHandler(this.ErrorAppHandler);

  29:     app.EndRequest +=

  30:         new EventHandler(this.EndRequestHandler);

  31: }

As you can see it checks that application is Sharepoint http application (SPHttpApplication), makes some initialization and subscribes to number of application events. We are interesting currently only in PreRequestHandlerExecute event as exactly in handler of this event Sharepoint initializes localization properties of current thread. Lets looks inside PreRequestExecuteAppHandler:

   1:  

   2: private void PreRequestExecuteAppHandler(object oSender, EventArgs ea)

   3: {

   4:     if (this.RequestPathIndex != PathIndex._layouts)

   5:     {

   6:         if (s_isAdminWebApp &&

   7:             (this.RequestPathIndex == PathIndex._admin))

   8:         {

   9:             this._adminApp.Application_PreRequestHandlerExecute(oSender, ea);

  10:         }

  11:     }

  12:     else

  13:     {

  14:         this._layoutsApp.Application_PreRequestHandlerExecute(oSender, ea);

  15:     }

  16: }

So Sharepoint executes PreRequestHandlerExecute only if request goes to application _layout pages or if current request goes to Central Administration. This is interesting moment that for publishing pages (also known as site or content pages) PreRequestHandlerExecute is not called and as result CurrentUICulture and CurrentCulture properties of current thread are not set here (for publishing pages these properties are initialized in another place - I will describe it in another post. Update 2010-03-20: see http://sadomovalex.blogspot.com/2010/03/set-locale-of-current-thread-for.html).

Finally lets look inside _layoutsApp.Application_PreRequestHandlerExecute() method which is called for _layouts pages:

   1: internal void Application_PreRequestHandlerExecute(object sender, EventArgs e)

   2: {

   3:     SPWeb contextWeb;

   4:     ...

   5:     contextWeb = SPControl.GetContextWeb(application.Context);

   6:     ...

   7:     SPUtility.SetThreadCulture(contextWeb);

   8: }

Here Sharepoint retrieves SPWeb contextWeb for current request and calls SPUtility.SetThreadCulture(contextWeb) method which in turn calls the following method:

   1: internal static void SetThreadCulture(SPWeb spWeb, bool force)

   2: {

   3:     if (spWeb != null)

   4:     {

   5:         CultureInfo locale = spWeb.Locale;

   6:         CultureInfo uiCulture = new CultureInfo((int) spWeb.Language);

   7:         SetThreadCulture(locale, uiCulture, force);

   8:     }

   9: }

  10:  

And finally we come to the last method which initializes CurrentUICulture and CurrentCulture properties of current thread:

   1: internal static void SetThreadCulture(CultureInfo culture, CultureInfo uiCulture, bool force)

   2: {

   3:     HttpContext current = HttpContext.Current;

   4:     if ((force || (current == null)) || (current.Items[IsThreadCultureSet] == null))

   5:     {

   6:         if (!Thread.CurrentThread.CurrentCulture.Equals(culture))

   7:         {

   8:             Thread.CurrentThread.CurrentCulture = culture;

   9:         }

  10:         if (!Thread.CurrentThread.CurrentUICulture.Equals(uiCulture))

  11:         {

  12:             Thread.CurrentThread.CurrentUICulture = uiCulture;

  13:         }

  14:         if (!force && (current != null))

  15:         {

  16:             current.Items[IsThreadCultureSet] = true;

  17:         }

  18:     }

  19: }

This is the way Sharepoint uses to initialize localization properties of current thread depending on localization properties of current SPWeb.

Open SPSite under RunWithElevatedPrivileges in proper zone

Sometimes in Sharepoint development we need to run some code under SPSecurity.RunWithElevatedPrivileges() in order to execute it under System account. As you know if we use SPSecurity.RunWithElevatedPrivileges() we should reopen SPSite object in delegate passed to this method:

   1: SPSecurity.RunWithElevatedPrivileges(() =>

   2: {

   3:     using (var site = new SPSite(SPContext.Current.Site.ID))

   4:     {

   5:         ...

   6:     }

   7: });

This code works well if you have single zone configured for your web application (you can see zones for web applications in Central Administration -> Application Management -> Authentication Providers). But if you have several authentication zones for single web application it can lead to troubles: Sharepoint will always open SPSite in Default zone. I.e. even if code in example above is executed under Internet zone with FBA (e.g. under http://www.example.com) it will open SPSite object for Default zone which may use windows authentication (e.g. http://example). It in turn may lead to unclear bugs like Access denied, Operation is not valid due to the current state of object, etc.

In order to avoid this trouble use another constructor of SPSite class which receives additional parameter of type SPUrlZone:

   1: SPSecurity.RunWithElevatedPrivileges(() =>

   2: {

   3:     using (var site = new SPSite(SPContext.Current.Site.ID,

   4: SPContext.Current.Site.Zone))

   5:     {

   6:         ...

   7:     }

   8: });

With this code SPSite will be opened in proper zone.

Runtime evaluation of lambda expressions

During working over Camlex.NET project in order to provide support of non-constant expressions we needed evaluate sub-expressions in runtime as results of these expression are required to construct CAML query. I.e. suppose that we have expression of the following general schema:

   1: x => (Type)x[expr1] == expr2

Here x is a parameter of lambda expression, Type – type allowed for casting in Camlex, and expr1 and expr2 – some expressions which should be evaluated in runtime. Expr1 and expr2 can be for example variables, method calls, ternary operator, etc.. For example:

   1: Func<object> expr1 = () => "Title";

   2: Func<object> expr2 = () => "Camlex";

   3:  

   4: var caml =

   5:     Camlex.Query()

   6:         .Where(x => (string)x[(string)expr1()] == expr2()).ToString();

The expression above will be translated in the following CAML:

   1: <Where>

   2:   <Eq>

   3:     <FieldRef Name="Title" />

   4:     <Value Type="Text">Camlex</Value>

   5:   </Eq>

   6: </Where>

As shown here results of expr1 and expr2 (“Title” and “Camlex”) are used for CAML query creation. So how do we evaluate exp1 and expr2:

   1:  

   2: private object evaluateExpression(Expression expr)

   3: {

   4:     // need to add Expression.Convert(..) in order to define Func<object>

   5:     var lambda =

   6: Expression.Lambda<Func<object>>(Expression.Convert(expr, typeof(object)));

   7:     return lambda.Compile().Invoke();

   8: }

First of all we need to represent expr1 and expr2 as Expression object using expression trees. In order to evaluate this expression we need to convert it to lambda expression () => expr which returns “object” type. Here little trick is made: we construct new unary expression which represents casting to object:

   1: Expression.Convert(expr, typeof(object))

Whatever actual expression was in expr1 or expr2, now the final operation for these expressions will be casting to object: (object)expr1 (of course it is not so efficiency if expressions return value type because of boxing/unboxing, but it will allow us to invoke expression in runtime). I.e. we added new operation to existing expressions.

Now it is not so hard to evaluate this expression. We create lambda expression: () => (object)expr1 with the following code:

   1:  

   2: var lambda =

   3: Expression.Lambda<Func<object>>(Expression.Convert(expr, typeof(object)));

And then we call Compile()  and Invoke() methods to get results. After that these results are used to create CAML query.