Sunday, April 12, 2015

Create search crawl rules for Sharepoint search service application via PowerShell

In one of my previous articles I showed how we may exclude system pages like AllItems.aspx from search results: Exclude AllItems.aspx from search results in Sharepoint 2013. In this post I will show how to create search crawl rules via PowerShell. It may be useful when you need to exclude a lot of contents from search crawling and doing it manually would mean a lot of work (e.g. when you restored large content database from production, but don’t need to crawl all sites). Here is the script:

   1:  
   2: # Ensure SharePoint PowerShell Snapin
   3: if ((Get-PSSnapin "Microsoft.SharePoint.PowerShell" -ErrorAction SilentlyContinue) -eq $null) 
   4: {
   5:     Add-PSSnapin "Microsoft.SharePoint.PowerShell"
   6: }
   7:  
   8: [xml]$xmlinput=(Get-Content "CrawlRules.xml")
   9:  
  10: foreach($WebApplication in $xmlinput.SelectNodes("Build/WebApplication"))
  11: {
  12:     foreach($SearchService in $WebApplication.SelectNodes("SearchService"))
  13:     {
  14:         #Get search service
  15:         $strServiceName=$SearchService.Name;
  16:         $spService=Get-SPEnterpriseSearchServiceApplication -Identity $strServiceName;
  17:         
  18:         #Clear rules if needed
  19:         $Rules=$SearchService.SelectNodes("Rules");
  20:         $strClearRules=$Rules.ItemOf(0).Clear;
  21:         if ($strClearRules -eq "True")
  22:         {
  23:             $spRules=Get-SPEnterpriseSearchCrawlRule -SearchApplication $spService;
  24:             foreach ($spRule in $spRules)
  25:             {
  26:                 if ($spRule -ne $null)
  27:                 {
  28:                     Write-Host "Deleting rule:" $spRule.Path -ForegroundColor Yellow
  29:                     $spRule.Delete();
  30:                 }
  31:             }
  32:         }
  33:  
  34:         #Add new rules
  35:         foreach($CrawlRule in $SearchService.SelectNodes("Rules/Rule"))
  36:         {
  37:             $FollowComplexUrls=$false;
  38:             if($CrawlRule.FollowComplexUrls -eq "True")
  39:             {
  40:                 $FollowComplexUrls=$true;
  41:             }
  42:             
  43:             if ($CrawlRule.Type -eq "ExclusionRule")
  44:             {
  45:                 #In exclusion FollowComplexUrls actually means "Exclude complex URLs"
  46:                 $FollowComplexUrls=!$FollowComplexUrls;
  47:                 New-SPEnterpriseSearchCrawlRule -Path $CrawlRule.URL -SearchApplication
  48: $spService -Type $CrawlRule.Type -FollowComplexUrls $FollowComplexUrls
  49:             }
  50:             else
  51:             {
  52:                 $CrawlAsHttp=$false;
  53:                 if($CrawlRule.CrawlAsHttp -eq "True")
  54:                 {
  55:                     $CrawlAsHttp=$true;
  56:                 }
  57:                 
  58:                 $SuppressIndexing=$false;
  59:                 if($CrawlRule.SuppressIndexing -eq "True")
  60:                 {
  61:                     $SuppressIndexing=$true;
  62:                 }
  63:                 
  64:                 New-SPEnterpriseSearchCrawlRule -Path $CrawlRule.URL -SearchApplication
  65: $spService -Type $CrawlRule.Type -FollowComplexUrls $FollowComplexUrls -CrawlAsHttp
  66: $CrawlAsHttp -SuppressIndexing $SuppressIndexing
  67:             }
  68:         }
  69:     }
  70: }

Rules are defined in CrawlRules.xml file which has the following structure:

   1:  
   2: <?xml version="1.0" encoding="utf-8"?>
   3: <Build>
   4:   <WebApplication>
   5:     <SearchService Name="Search Service Application">
   6:       <Rules Clear="True">
   7:         <Rule URL="*://*/_layouts/*" Type="ExclusionRule" FollowComplexUrls="False" />
   8:         <Rule URL="*://*/_catalogs/*" Type="ExclusionRule" />
   9:         <Rule URL="*://*/_vti_bin/*" Type="ExclusionRule" />
  10:         <Rule URL="*://*/forms/AllItems.aspx*" Type="ExclusionRule" />
  11:         <Rule URL="*://*/forms/DispForm.aspx*" Type="ExclusionRule" />
  12:         <Rule URL="*://*/forms/EditForm.aspx*" Type="ExclusionRule" />
  13:         <Rule URL="*://*/forms/NewForm.aspx*" Type="ExclusionRule" />
  14:       </Rules>
  15:     </SearchService>
  16:   </WebApplication>
  17: </Build>

As result it will create exclusion rules for layouts pages, also for pages from _catalogs and _bti_bin and for list forms AllItems.aspx, DispForm.aspx, EditForm.aspx and NewForm.aspx. You may generate this xml file programmatically if you have a lot of sites which should be excluded and then pass it to the script above. It will simplify administrative work, which is not needed to be done manually.

Saturday, April 4, 2015

Fix 401 Unauthorized error for anonymous users when use owssvr.dll

Library owssvr.dll is used for performing various operations against Sharepoint content: URL Protocol. E.g. you may retrieve data from particular list using this dll like this:

http://example.com/[sites/][Site_Name/]_vti_bin/owssvr.dll?Cmd=Display&List=GUID&XMLDATA=TRUE

There is however problem with using owssvr.dll by anonymous users which may get HTTP 401 Unauthorized error. In order to make owssvr.dll work for anonymous users few things should be done. At first we need to enable anonymous state on particular SPWeb and add SPBasePermissions.ViewFormPages and SPBasePermissions.UseRemoteAPIs to Limited access role definition (SPRoleType.Guest) which is used for anonymous users:

   1: web.RoleDefinitions.BreakInheritance(true, true);
   2: var rd = web.RoleDefinitions.GetByType(SPRoleType.Guest);
   3: rd.BasePermissions |= SPBasePermissions.ViewFormPages |
   4:     SPBasePermissions.UseRemoteAPIs;
   5: rd.Update();
   6: web.AnonymousState = SPWeb.WebAnonymousState.On;
   7: web.Update();

This is however not enough. We also need to add SPBasePermissions.UseRemoteAPIs permission to the list from which we will retrieve data via owssvr.dll. The main problem is that when you enable anonymous access (grant read permissions for anonymous users) for the list from UI List Settings > List Permissions > Anonymous Access:

image

only the following permissions are added to SPList.AnonymousPermMask64:

SPBasePermissions.ViewListItems | SPBasePermissions.ViewVersions | SPBasePermissions.ViewFormPages | SPBasePermissions.Open | SPBasePermissions.ViewPages | SPBasePermissions.UseClientIntegration

and SPBasePermissions.UseRemoteAPIs is not there as you can see. Even if we added it on SPWeb level, without that anonymous users will still get 401 Unauthorized error when will try to use owssvr.dll library. So in order to make it work we need to do grant SPBasePermissions.UseRemoteAPIs on list level:

   1: SPList list = ...;
   2: list.BreakRoleInheritance(true);
   3: list.AnonymousPermMask64 = SPBasePermissions.ViewListItems |
   4:         SPBasePermissions.ViewVersions | SPBasePermissions.ViewFormPages |
   5:         SPBasePermissions.Open | SPBasePermissions.ViewPages |
   6:         SPBasePermissions.UseClientIntegration | SPBasePermissions.UseRemoteAPIs;
   7: list.Update();

After that owssvr.dll should work for anonymous users.

Wednesday, April 1, 2015

Sharepoint MVP 2015

Hello dear readers of my blog. I’m pleasant to announce that I’ve became Sharepoint MVP 2015: 5th time in sequence from 2011 year. So this award is little anniversary for me and I’m glad to be with community all these years. Hope that my humble contribution helps people in their work and make the world a little bit better. Community work became part of my life, and it is not kind of real work for me, but is more kind of pleasure and a way to give something to people regardless of the countries, cultures, religions and any other differences. In our not simple time I think it is very important to keep hearts and minds open and help other people without waiting any compensations. Good that MS recognizes it, but I would continue to do that even without MVP award. Often people ask how to become MVP: my answer is teach to give, not only take.

Saturday, March 21, 2015

Retrieve user profile properties using javascript object model in Sharepoint

With new programming model for Sharepoint Online we need to use client and javascript object models more often than regular server-side object model. And we need to perform basic operations using client API. One of such operations is working with user profile properties. In this post I will show how to retrieve user profile properties using javascript object model. For example let’s use the script which retrieves language and country properties from user profile:

   1: SP.SOD.executeFunc('userprofile', 'SP.UserProfiles.PeopleManager', function () {
   2:     var pmPeopleManager = new SP.UserProfiles.PeopleManager(spContext);
   3:  
   4:     var spSite = spContext.get_site();
   5:     var spWeb = spSite.get_rootWeb();
   6:  
   7:     spContext.load(spSite);
   8:     spContext.load(spWeb);
   9:  
  10:     var currentUser = spWeb.get_currentUser();
  11:     spContext.load(currentUser);
  12:  
  13:     spContext.executeQueryAsync
  14:     (
  15:         Function.createDelegate
  16:         (
  17:            this, function () {
  18:                var strUsername = currentUser.get_loginName();
  19:                var arrProperties = ["DefaultLanguage", "DefaultCountry"];
  20:                var userProperties = new
  21:                  SP.UserProfiles.UserProfilePropertiesForUser(spContext,
  22:                    strUsername, arrProperties);
  23:                var uppProperties =
  24:                    pmPeopleManager.getUserProfilePropertiesFor(userProperties);
  25:  
  26:                spContext.load(userProperties);
  27:                spContext.executeQueryAsync
  28:                (
  29:                    Function.createDelegate
  30:                    (
  31:                       this, function () {
  32:                           var strLanguage = uppProperties[0];
  33:                           var strCountry = uppProperties[1];
  34:                       }
  35:                    ),
  36:                    Function.createDelegate
  37:                    (
  38:                        this, function (sender, args) {
  39:                        }
  40:                    )
  41:                );
  42:            }
  43:         ),
  44:         Function.createDelegate
  45:         (
  46:             this, function (sender, args) {
  47:             }
  48:         )
  49:     );
  50: });

First of all whole script is executed under SP.SOD.executeFunc() in order to ensure that SP.UserProfiles.PeopleManager type is loaded. In script we first asynchronously load current user (lines 5-19). After that we prepare parameters for loading user profile properties (lines 14-23) and retrieve properties by calling SP.UserProfiles.PeopleManager.getUserProfilePropertiesFor() method (lines 24-27). When query is executed we store DefaultLanguage and DefaultCountry user profile properties into local variables for further use (lines 34-35).

This is how you may retrieve use profile properties using javascript object model in Sharepoint. Hope that this information will help someone.

Saturday, March 14, 2015

Set search settings in all site collections of Sharepoint web application via PowerShell

In site collection’s search settings page (Site settings > Search settings) we may set the following parameters:

  • Search center url: when you've specified a search center, the search system displays a message to all users offering them the ability to try their search again from that Search Center;
  • Search results page: page where search queries should be sent to.

Page looks like that:

image

If we have a lot of site collections where search settings should be set it will have sense to use PowerShell script which will do the job. Here is the script which iterates through all site collections in web applications and changes search settings of each site collection:

   1: param(
   2:     [string]$webAppUrl,
   3:     [string]$searchCenterUrl,
   4:     [string]$resultsPage
   5: )
   6:  
   7: function Set-Search-Settings($site)
   8: {
   9:     Write-Host "Set search settings on site" $site.Url
  10: -foregroundcolor green
  11:     $web = Get-SPWeb $site.Url        
  12:     $web.AllProperties["SRCH_SB_SET_SITE"] = 
  13: '{"Inherit":false,"ResultsPageAddress":"' + $resultsPage + '","ShowNavigation":false}' 
  14:     $web.AllProperties["SRCH_ENH_FTR_URL_SITE"] = $searchCenterUrl
  15:     $web.Update()
  16:     Write-Host "    Search settings were successfully set"
  17: -foregroundcolor green
  18: }
  19:  
  20: if (-not $webAppUrl)
  21: {
  22:     Write-Host "Specify web app url in webAppUrl parameter"
  23: -foregroundcolor red
  24:     return
  25: }
  26:  
  27: if (-not $searchCenterUrl)
  28: {
  29:     Write-Host "Specify web app url in searchCenterUrl parameter"
  30: -foregroundcolor red
  31:     return
  32: }
  33:  
  34: if (-not $resultsPage)
  35: {
  36:     Write-Host "Specify web app url in resultsPage parameter"
  37: -foregroundcolor red
  38:     return
  39: }
  40:  
  41: Start-Transcript -Path "output.log" -Append -Force -Confirm:$false
  42:  
  43: $wa = Get-SPWebApplication $webAppUrl
  44: $wa.Sites | ForEach-Object { Set-Search-Settings $_ }
  45:  
  46: Stop-Transcript

Script has 3 parameters:

  • webAppUrl - url of web application where search settings should be changed;
  • searchCenterUrl - url of search center where search query will be redirected to;
  • resultsPage - results page where search query will be redirected to.

Here is example of usage:

.\SetSearchSettings.ps1 -webAppUrl http://example1.com -searchCenterUrl http://example2.com/search/Pages -resultsPage "{SearchCenterURL}/results.aspx"

Note that if {SearchCenterURL} token is used in resultPage parameter it is necessary to enclose parameter’s value into double quotes, otherwise instead of “{SearchCenterURL}/results.aspx” it will be set to “SearchCenterURL” value which is incorrect.