Wednesday, March 28, 2018

Avoid problem with redirecting Sharepoint sites which use .dev sub domains to https in Chrome and Firefox

As you probably heard some time ago Chrome and FF started to redirect sites which use .dev sub domains in URL to https. Issue is described e.g. here: Chrome & Firefox now force .dev domains to HTTPS via preloaded HSTS. Briefly .dev domain is owned by Google and they added HSTS rule for it which forces redirection to https. You may check it in Chrome: open chrome://net-internals#hsts and make query for dev domain:

2018-03-28_16-38-32

As you can see there is FORCE_HTTPS rule.

This problem may occur in local dev Sharepoint environments which often use .dev domain. So if you have e.g. site http://intranet.sp.dev and will try to open it in Chrome or Firefox it won’t open it with unclear error:

2018-03-28_16-37-56

But if you notice url address bar of the browser you will find that it actually tries to open https://intranet.sp.dev instead of http://intranet.sp.dev and it causes issue. You may try to add https binding to the site using self-signed certificate, but browser will still show error that connection is not secure.

The following solution may be used for avoiding this problem with Sharepoint site. It requires using new site address http://intranet.sp.local for your Sharepoint site instead of http://intranet.sp.dev:

  • add intranet.sp.local binding for your Sharepoint site in IIS Manager
  • in C:/Windows/System32/drivers/etc/host add
      127.0.0.1 intranet.sp.local
  • in Central administration > Alternate access mappings > Add internal url for not used zone: http://intranet.sp.local

After that site can be opened in Chrome and FF using http://intranet.sp.local url.

Monday, March 26, 2018

Avoiding StackOverflowException when use assembly binding redirect in PowerShell

Recently I faced with interesting problem with assembly binding redirect in PowerShell: in my script I needed to use the following versions of the assemblies (these exact versions were used in other components and I couldn’t add another versions because of number of reasons):

OfficeDevPnP.Core 2.22.1801.0
Microsoft.Graph 1.7.0.0
Microsoft.Graph.Core 1.7.0.0

The problem is that OfficeDevPnP.Core 2.22.1801.0 references different versions of Microsoft.Graph and Microsoft.Grap.Core:

Microsoft.Graph 1.1.1.0
Microsoft.Graph.Core 1.2.1.0

So I needed to use assembly binding redirect. At first I tried approach described in the following post: Use specific version of Sharepoint client object model in PowerShell via assembly binding redirection:

$currentDir = Convert-Path(Get-Location)
$pnpCoreDir = resolve-path($currentDir + "\..\packages\SharePointPnPCoreOnline.2.22.1801.0\lib\net45\")
$graphDir = resolve-path($currentDir + "\..\packages\Microsoft.Graph.1.7.0\lib\net45\")
$graphCoreDir = resolve-path($currentDir + "\..\packages\Microsoft.Graph.Core.1.7.0\lib\net45\")

$AssemblyGraphCore = [System.Reflection.Assembly]::LoadFile([System.IO.Path]::Combine($graphCoreDir, "Microsoft.Graph.Core.dll"))
$AssemblyGraph = [System.Reflection.Assembly]::LoadFile([System.IO.Path]::Combine($graphDir, "Microsoft.Graph.dll"))
[System.Reflection.Assembly]::LoadFile([System.IO.Path]::Combine($pnpCoreDir, "OfficeDevPnP.Core.dll"))


<#$OnAssemblyResolve = [System.ResolveEventHandler] {
	param($sender, $e)
	
	if ($e.Name.StartsWith("Microsoft.Graph,"))
	{
		$AssemblyGraph = [System.Reflection.Assembly]::LoadFile([System.IO.Path]::Combine($graphDir, "Microsoft.Graph.dll"))
		return $AssemblyGraph
	}
	if ($e.Name.StartsWith("Microsoft.Graph.Core,"))
	{
		$AssemblyGraphCore = [System.Reflection.Assembly]::LoadFile([System.IO.Path]::Combine($graphCoreDir, "Microsoft.Graph.Core.dll"))
		return $AssemblyGraphCore
	}

	foreach($a in [System.AppDomain]::CurrentDomain.GetAssemblies())
	{
		if ($a.FullName -eq $e.Name)
		{
		  return $a
		}
	}
	Write-Host "Return null" -foregroundcolor red
	return $null
}
[System.AppDomain]::CurrentDomain.add_AssemblyResolve($OnAssemblyResolve)

However attempt to call method from loaded assemblies caused StackOverflowException and closing PowerShell session.

After that I tried C#-based assembly redirector (found it in the following forum thread: Powershell - Assembly binding redirect NOT found in application configuration file):

if (!("Redirector" -as [type]))
{
$source = 
@'
using System;
using System.Linq;
using System.Reflection;
using System.Text.RegularExpressions;

public class Redirector
{
    public readonly string[] ExcludeList;

    public Redirector(string[] ExcludeList = null)
    {
        this.ExcludeList  = ExcludeList;
        this.EventHandler = new ResolveEventHandler(AssemblyResolve);
    }

    public readonly ResolveEventHandler EventHandler;

    protected Assembly AssemblyResolve(object sender, ResolveEventArgs resolveEventArgs)
    {
        Console.WriteLine("Attempting to resolve: " + resolveEventArgs.Name); // remove this after its verified to work
        foreach (var assembly in AppDomain.CurrentDomain.GetAssemblies())
        {
            var pattern  = "PublicKeyToken=(.*)$";
            var info     = assembly.GetName();
            var included = ExcludeList == null || !ExcludeList.Contains(resolveEventArgs.Name.Split(',')[0], StringComparer.InvariantCultureIgnoreCase);

            if (included && resolveEventArgs.Name.StartsWith(info.Name, StringComparison.InvariantCultureIgnoreCase))
            {
                if (Regex.IsMatch(info.FullName, pattern))
                {
                    var Matches        = Regex.Matches(info.FullName, pattern);
                    var publicKeyToken = Matches[0].Groups[1];

                    if (resolveEventArgs.Name.EndsWith("PublicKeyToken=" + publicKeyToken, StringComparison.InvariantCultureIgnoreCase))
                    {
                        Console.WriteLine("Redirecting lib to: " + info.FullName); // remove this after its verified to work
                        return assembly;
                    }
                }
            }
        }

        return null;
    }
}
'@

    $type = Add-Type -TypeDefinition $source -PassThru 
}

try
{
    $redirector = [Redirector]::new($null)
    [System.AppDomain]::CurrentDomain.add_AssemblyResolve($redirector.EventHandler)
}
catch
{
    #.net core uses a different redirect method
    Write-Warning "Unable to register assembly redirect(s). Are you on ARM (.Net Core)?"
}

And surprisingly this approach worked. It looks like a bug in PowerShell script-based assembly binding redirect. Hope it will help someone.

Wednesday, March 21, 2018

How to reset credentials for Sharepoint Designer

Here are instructions of how to reset credentials for Sharepoint Designer:

1. Go to Windows control panel and select User accounts:

01

2. In opened window click Manage your credentials:

02

3. Then choose Windows credentials:

03

4. On credentials window under Generic credentials there will be account which start with “MicrosoftOffice15_Data:” prefix (this is a prefix for Sharepoint Designer 2013. For other versions most probably prefix will be different):

04

Find those which was cached for your site and remove it from the list.

After that if you will open this site again in Sharepoint Designer it will ask to enter credentials.

Thursday, March 15, 2018

How to renew expired app in Sharepoint Online

As you probably know when you register new app in Sharepoint (using /_layouts/15/AppRegNew.aspx) it’s expiration date is set to 1 year from moment of registration. If your app is expired perform the following steps in order to renew it on 3 years:

1. Run Windows PowerShell and connect to Msol:

Connect-MsolService

2. Get list of all apps:

Get-MsolServicePrincipal -all | Where-Object -FilterScript { ($_.DisplayName -notlike "*Microsoft*") -and ($_.DisplayName -notlike "autohost*") -and  ($_.ServicePrincipalNames -notlike "*localhost*") } | Out-File log_apps.txt -Append

3. From generated log_apps.txt copy AppPrincipalId for expired app

4. Get list of all principals:

Get-MsolServicePrincipalCredential -AppPrincipalId {copied_app_principal_id} -ReturnKeyValues $true | Out-File log_principals.txt -Append
5. Check end dates for app principals. If they are expired run the following script which will generate new client secret and renew principals on 3 years:
# start script
$clientId = "{copied_app_principal_id}"
$bytes = New-Object Byte[] 32
$rand = [System.Security.Cryptography.RandomNumberGenerator]::Create()
$rand.GetBytes($bytes)
$rand.Dispose()
$newClientSecret = [System.Convert]::ToBase64String($bytes)
New-MsolServicePrincipalCredential -AppPrincipalId $clientId -Type Symmetric -Usage Sign -Value $newClientSecret -StartDate (Get-Date) -EndDate (Get-Date).AddYears(3)
New-MsolServicePrincipalCredential -AppPrincipalId $clientId -Type Symmetric -Usage Verify -Value $newClientSecret -StartDate (Get-Date) -EndDate (Get-Date).AddYears(3)
New-MsolServicePrincipalCredential -AppPrincipalId $clientId -Type Password -Usage Verify -Value $newClientSecret -StartDate (Get-Date) -EndDate (Get-Date).AddYears(3)
Write-Host "New client secret:"
$newClientSecret
# end script

Depending on permissions which are required for your app you may need to run the last script under tenant admin account (if app requires tenant full access).

Tuesday, March 13, 2018

Fix missing usage analytics logs in Sharepoint 2013 event store

Sharepoint 2013 usage analytics reports give to administrators view of site usage statistics. But it is not so simple to configure them to work. Most frequent issue is that Excel reports contain only zeros. There may be many reasons for this problem. One of them is missing log files in EventStore. EventStore is located in "C:\Program Files\Microsoft Office Servers\15.0\Data\Office Server\Analytics_{GUID}\EventStore\" folder on the server. If it is empty check the following in PowerShell:

$aud = Get-SPUsageDefinition | where {$_.Name -like "Analytics*"}
$aud | fl

It will show something like this:

01

Pay attention on EnableReceivers and Receivers – first should be set to True and second to Microsoft.Office.Server.Search.Analytics.Internal.AnalyticsCustomRequestUsageReceiver like shown on the picture. If EnableReceivers is set to False and Receivers is empty execute the following code also in PowerShell:

$aud.Receivers.Add("Microsoft.Office.Server.Search.Applications, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c","Microsoft.Office.Server.Search.Analytics.Internal.AnalyticsCustomRequestUsageReceiver")
$aud.EnableReceivers = $true
$aud.Update()

After that check also requests usage:

$prud = Get-SPUsageDefinition | where {$_.Name -like "Page Requests"}
$prud | fl

Which should look like this:

02

Here also if EnableReceivers is set to False and Receivers is empty execute the following code:

$prud.Receivers.Add("Microsoft.Office.Server.Search.Applications, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c", "Microsoft.Office.Server.Search.Analytics.Internal.ViewRequestUsageReceiver")
$prud.EnableReceivers = $true
$prud.Update()

After that go to the site and click some links to generate traffic. Log files should be created in EventStore folder.

Thursday, March 1, 2018

How to check does site collection exist by absolute url using CSOM in Sharepoint

Suppose that we need to check whether site collection with specified absolute url exists or not using client side object model. In OfficeDevPnP library there is convenient extension method for ClientContext WebExtensions.WebExistsFullUrl:

        public static bool WebExistsFullUrl(this ClientRuntimeContext context, string webFullUrl)
        {
            bool exists = false;
            try
            {
                using (ClientContext testContext = context.Clone(webFullUrl))
                {
                    testContext.Load(testContext.Web, w => w.Title);
                    testContext.ExecuteQueryRetry();
                    exists = true;
                }
            }
            catch (Exception ex)
            {
                if (IsUnableToAccessSiteException(ex) || IsCannotGetSiteException(ex))
                {
                    exists = true;
                }
            }
            return exists;
}

Unfortunately this method works properly only within single site collection i.e. when client context (which is extended with this extension method) and url to check belong to the same managed path.

Example:
context is created from the root site http://example.com. This site has http://example.com/test sub site. In this case WebExistsFullUrl returns true for http://example.com/test url and false for some non-existent sub site's url like http://example.com/test123. I.e. behavior is correct.

But if context and url to check belong to different managed path then it always returns true.

Example:
context is created from the root site http://example.com and we call WebExistsFullUrl for some url which may belong to other site collection (e.g. which uses different managed path like http://example.com/teams/some-not-real-url. Suppose that at the moment of call we don't know whether this collection exists or not - we want to determine it by WebExistsFullUrl call). In this case WebExistsFullUrl returns true even if site collection with specified url doesn't exists.

When I analyzed the code I found the following. When we call this method with context and url which belong to the same managed path it throws exception like expected. But when it is called with context and url which belong to different managed paths exception is not thrown. Instead context is created for the root site http://example.com and method returns true and caller thinks that site exists. In order to fix this problem for different managed paths I applied the following fix in our local version:

        public static bool WebExistsFullUrl(this ClientRuntimeContext context, string webFullUrl)
        {
            bool exists = false;
            try
            {
                using (ClientContext testContext = context.Clone(webFullUrl))
                {
                    testContext.Load(testContext.Web, w => w.Title, w => w.Url);
                    testContext.ExecuteQueryRetry();
                    exists = (string.Compare(testContext.Web.Url, webFullUrl, true) == 0);
                }
            }
            catch (Exception ex)
            {
                if (IsUnableToAccessSiteException(ex) || IsCannotGetSiteException(ex))
                {
                    // Site exists, but you don't have access .. not sure if this is really valid
                    // (I guess if checking if URL is already taken, e.g. want to create a new site
                    // then this makes sense).
                    exists = true;
                }
            }
            return exists;
}

I.e. before to return true method also checks whether loaded url is the same as url to be checked. If they are different (which happens in scenario with different managed paths) it will return false like it should.

Described behavior was found when root site collection (http://example.com) was host-named site collection, but most probably it will also work same way for regular site collections.

This problem is also submitted to OfficeDevPnP Core issues section on GitHub: WebExtensions.WebExistsFullUrl returns true for non-existent sites from different managed path in Sharepoint 2013 on-premise.

Update 2020-11-05: this problem is fixed now in CSOM. See Call WebExtensions.WebExistsFullUrl method from OfficeDevPnP library for different site collection.