Fetch Sharepoint Online sites which are not associated with O365 groups via Sharepoint Search KQL

In the previous article I showed how we can use Search API in order to fetch sites which are associated with O365 groups:

(contentclass:STS_Site OR contentclass:STS_Web) GroupId<>""

But what if we need to get opposite result: fetch only those sites which are not associated with O365 groups (sites without groups). Our knowledge about GroupId indexed property bag property will help also here since we need to fetch those sites which don't have any value in GroupId property.

First (naive) attempt to do that would be trying something like that:

(contentclass:STS_Site OR contentclass:STS_Web) GroupId:""

However it won't work because KQL supports not equal operation with empty string but doesn't support equal to empty string operation. My colleague Juha Alhojoki pointed me to the original trick which allows to do that: Check Not Null condition in Keyword Query Language for SharePoint Search. So idea is that if we know possible start letters for managed property (remember that KQL supports prefix matching but doesn't support suffix matching, i.e. we may search by "StartsWith" operator but not with "EndsWith") we may enumerate them all and then apply NOT operator to the final condition.

In our example GroupId contains guid i.e. it may start with Latin alphabet symbols and digits. So we will have the following condition to fetching sites which are not associated with O365 groups:

(contentclass:STS_Site OR contentclass:STS_Web) NOT(GroupId:a* OR GroupId:b* OR GroupId:c* OR GroupId:d* OR GroupId:e* OR GroupId:f* OR GroupId:g* OR GroupId:h* OR GroupId:i* OR GroupId:j* OR GroupId:k* OR GroupId:l* OR GroupId:m* OR GroupId:n* OR GroupId:o* OR GroupId:p* OR GroupId:q* OR GroupId:r* OR GroupId:s* OR GroupId:t* OR GroupId:u* OR GroupId:v* OR GroupId:w* OR GroupId:x* OR GroupId:y* OR GroupId:z* OR GroupId:0* OR GroupId:1* OR GroupId:2* OR GroupId:3* OR GroupId:4* OR GroupId:5* OR GroupId:6* OR GroupId:7* OR GroupId:8* OR GroupId:9*)

What we did here is enumerated all Latin symbols and digits and applied NOT operator to exclude sites which match enumerated criterias.

Fetch Sharepoint Online sites associated with O365 groups via Sharepoint Search KQL

As you probably know O365 groups have associated Sharepoint Online site behind. It may be needed to fetch only those sites which have associated O365 group via Sharepoint Search and KQL. In this post I will show how to do that and in the future post I will describe how to fetch only those sites which are not associated with O365 groups.

Modern Team sites which are associated with O365 group are created using Groups template. I.e. in order to get all sites associated with groups we may use the following KQL query:

(contentclass:STS_Site OR contentclass:STS_Web) WebTemplate:GROUP

It will work however there may be problem with old sites which were "groupified". I.e. regular site could be connected to O365 site - this procedure is called groupifying (see Connect to a Microsoft 365 group). Above query won't return such sites.

Another approach is based on the fact that groupified sites contain special property bag properties which contain information about connected group:

• GroupAlias
• GroupDocumentsListId
• GroupDocumentsUrl
• GroupId
• GroupType
  

By default custom property bag properties are not searchable but they can be made searchable by adding them to Indexed Property Bag which can be done via PowerShell:

Connect-PnPOnline -Url "http://{tenant}.sharepoint.com"
Set-PnPPropertyBagValue -Key "customProperty" -Value "foo" -Indexed  

Here is good article which explains this topic: What Are Indexed Property Bags And How To Use Them For SharePoint Site.

The good thing is that at least GroupId looks like indexed by default. This gives us possibility to use another KQL for getting sites connected with O365 groups:

(contentclass:STS_Site OR contentclass:STS_Web) GroupId<>""

This query will work also with groupified sites.

Update 2021-02-25: see also article which shows how to fetch sites which are not associated with O365 groups: Fetch Sharepoint Online sites which are not associated with O365 groups via Sharepoint Search KQL.

How to check what sensitivity label is applied to O365 group via Graph API

In order to check what sensitivity label is applied to O365 group you may go to Azure portal > Azure Active Directory > Groups > select group. Sensitivity label will be displayed in overview tab of the group:

In order to get sensitivity label applied to O365 group programmatically via Graph API you may use the following endpoint:

https://graph.microsoft.com/v1.0/groups/%7BgroupId%7D?$select=assignedLabels

It will return applied label id and display name like this:

{
    "@odata.context": "https://graph.microsoft.com/v1.0/$metadata#groups(assignedLabels)/$entity",
    "assignedLabels": [
        {
            "labelId": "...",
            "displayName": "Private"
        }
    ]

You may test this endpoint e.g. in Graph Explorer:

Create new Team UX when use sensitive labels

In this article I will show how different sensitive labels affect UX on the standard Create new Team form. For testing I created 6 different sensitive labels with different Privacy and external sharing settings:

	Name	Privacy	External user access
1	Private	Private	No
2	Public	Public	No
3	Private or public	None	No
4	Private with external users	Private	Yes
5	Public with external users	Public	Yes
6	Private or public with external users	None	Yes

Let’s see how teams creation form looks like for each sensitivity label from table above.

1. Private

2. Public

3. Private or public

4. Private with external users

5. Public with external users

3. Private or public with external users

Applying O365 sensitivity labels for different containers

In my previous post I showed how to enable sensitivity labels for different containers (SPO sites, Teams, O365 groups) in the tenant . Let’s see in more details how exactly applying of sensitivity labels looks like and to which exact containers it can be applied.

First of all let’s go to Sharepoint Online and try to create new site there. We will see that sensitivity labels are available for Modern Sharepoint Online sites: both Modern Team site

and Modern Communication sites

But if we will try to create Classic site (e.g. using Publishing Portal site template) we will see that sensitivity labels are not available:

So for Classic Sharepoint sites sensitivity labels are not available.

Next let’s go to Teams and try to create new Team there: we will see that Sensitivity field appeared on the Teams creation form:

Finally if we will go to Azure portal and will try to create new O365/M365 group we will also see that Sensitivity field will be available:

In the future posts we will check other details of sensitivity labels functionality in O365.

Enable sensitivity labels for Sharepoint sites, Teams and O365 groups

Sensitivity labels help to maintain content in your organization. In opposite to classification labels which are more like additional metadata for O365 groups/SP sites where custom policies have to be enforced by internal tools or custom PowerShell scripts (i.e. don’t have O365 policies assigned to them) sensitivity labels have policies behind and allow to use O365 infrastructure to maintain sensitive data in your organization.

Sensitivity labels may be enabled from several places:

• Security and compliance center: https://protection.office.com
• Microsoft 365 security center: https://security.microsoft.com
• Microsoft 365 compliance center: https://compliance.microsoft.com

By default they can be used for files in emails but in order to enable them for “container” (SP online sites, Teams and O365 groups) several additional steps should be done:

1. First of all enable sensitivity labels from PowerShell using the following script:

Import-Module AzureADPreview
Connect-AzureAD
$Setting = Get-AzureADDirectorySetting -Id (Get-AzureADDirectorySetting | where -Property DisplayName -Value "Group.Unified" -EQ).id
$Setting["EnableMIPLabels"] = "True"
Set-AzureADDirectorySetting -Id $Setting.Id -DirectorySetting $Setting

2. After that we need to sync them to AzureAD using the following script:

Install-Module -Name ExchangeOnlineManagement
Import-Module ExchangeOnlineManagement
$UserCredential = Get-Credential
Connect-IPPSSession -Credential $UserCredential
Execute-AzureADLabelSync
Disconnect-ExchangeOnline

If you will have error "It is about Unable to resolve package source https://www.powershellgallery.com/api/v2” then start new PowerShell session as administrator and run the following command as 1st command in the session:

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

and then run above script again.

After these steps you will be able to create sensitivity labels for SP sites, Teams and O365 groups. Let’s see how it looks like in Security and compliance center > Classification > Sensitivity labels. Pay attention that there is now the following note:

You can now create sensitivity labels with privacy and access control settings for Teams, SharePoint sites, and Microsoft 365 Groups.

Click Create a label – after that New label wizard will be opened. On the first step we need tp specify name and description and on 2nd step it will be possible to choose both Files & emails and Groups & sites:

Here we are interested in Groups and sites so let’s keep only this option checked. Skip next step for Files and emails and open next step “Define protection settings for groups and sites”. On this step we may set “Privacy and external user access settings” and “Device access and external sharing settings”:

E.g. if we will check “Privacy and external user access settings” then on the next step we will be able to set privacy and external users settings for sites/teams/groups for which this label will be applied:

On the final step we will need to publish our new label (this will open own wizard).

After label has been published it will take up to 10 minutes before it will appear in O365 UI:

This is how you may enable sensitive labels for SP sites, Teams and O365 groups. Hope this information will help someone.

Problem with fetching photos of private O365 groups

As you probably know in O365 we may create groups which has one of the following visibilities:

• Public
• Private

Everybody in your organization may join/leave public groups while for private groups only owners of this group may add you to the group. In this article I will describe one problem related with fetching private group photos via Graph (see also my previous article where I mentioned another problem related with groups images: Why you should be careful with /groups/{id}/photo and /users/{id}/photo endpoints in MS Graph or unintentional getting photos of big sizes in Graph).

In order to fetch group photo the following Graph endpoint should be used:

https://graph.microsoft.com/v1.0/groups/{id}/photo

First of all we need to mention that this endpoint is available only via user delegated permissions (it doesn’t work with app-only permissions). If we will try to fetch photo of some group in Graph explorer using service account which is not member of this group we will get 404 Not found error:

After we will add the same service account to members of the same group image will be retrieved successfully:

Even if we will try to fetch photos of private group under global admin account which is not member of this group – we will still get 404 Not found. So the only way to fetch photo of the private group is to add user account to members or owners of this group. Be aware of this problem when will plan groups images fetching functionality.

One problem with AllowToAddGuests and AllowGuestsToAccessGroups O365 groups tenant settings

As you probably know AllowToAddGuests and AllowGuestsToAccessGroups tenant settings determine whether or not external users are able to access O365 groups in your tenant. You may view or change them via PowerShell:

AzureADPreview\Connect-AzureAD
Get-AzureADDirectorySetting

Which will show something like that:

If you will try to update them then you may face with different behavior on different tenants. E.g. if we will try to change them to true like this:

$groupsConfig = Get-AzureADDirectorySetting -Id {settingId}
$groupsConfig["AllowToAddGuests"] = $true
$groupsConfig["AllowGuestsToAccessGroups"] = $true
Set-AzureADDirectorySetting -Id {settingId} -DirectorySetting $groupsConfig

we may get the following result:

Note that in script we used lowercase $true while in result we got True with capital T. But if you will try to run the same script on another tenant it may save values using the same letters registry as used in script. I.e. $true will be saved as true and $True will be saved as True. And if software uses case-sensitive comparison it will cause problems. So be aware of this problem – hope it will be help someone.

Problems with Teams creation via beta teams Graph endpoint with owner without O365 license

When you create Team by sending HTTP POST request to beta Graph endpoint /beta/teams (see Create team) you need to specify exactly 1 user as an owner of the new team:

POST https://graph.microsoft.com/beta/teams
Content-Type: application/json
{
  "displayName": "Test",
  "owners@odata.bind": [
    "https://graph.microsoft.com/beta/users('userId')"
  ]
}

where userId is login name of the user which will be owner of the group. However this request may fail with the following error:

Invoking endpoint 'https://graph.microsoft.com/beta/teams/' didn't succeed
Response status code 'Forbidden', reason phrase 'Forbidden'
Response content '
"code": "AccessDenied",
”message": "Failed to execute Templates backend request CreateTeamFromTemplateRequest

It may happen if user which is specified as owner of the team doesn’t have O365 license. In order to avoid this error use users with O365 license as team owners.

How to test and debug Timer triggered Azure functions running locally on localhost

With Azure function project type we may create many types of Azure functions. One of them is Timer triggered Azure function which is triggered by scheduler (CRON expression is specified during function creation and may be modified later in Azure function attribute):

When you will run this function locally (on locahost) you will notice thatt it won’t be listed under

http://localhost:7071/api/*

address together with Http triggered Azure function. Runtime will call your function automatically when schedule time will come. But this is not very convenient since during development you would probably like to have possibility to call it synchronously on demand.

In order to call Timer triggered Azure function you may use Postman tool. You have to construct URL little bit different from Http triggered functions: instead of /api you should send POST request to http://localhost:7071/admin/functions/* base url and send the following request body:

{
    "input": ""
}

Using this technique you may debug your Timer triggered Azure functions synchronously.

Use Office 365 connected groups in Yammer

Yammer platform allows you to integrate your Yammer groups to Office 365. Technically it means that each time when new Yammer group will be created it will be also created in Azure AD of the tenant of this Yammer network. More information about Yammer and O365 groups can be found in this article: Yammer and Office 365 Groups. This article also contains information how to enable Office 365 connected groups in Yammer and which conditions should be met before you will be able to enable it:

• You must enforce Office 365 identity for Yammer users. When you first enforce Office 365 identity there is a seven-day trial period, after which the Status of your Office 365 Identity Enforcement changes to Committed.
• Your Yammer network must be in a 1:1 network configuration. This means you have one Yammer network that is associated with one Office 365 tenant.

Note that currently it is possible to enforce Office 365 identity for Yammer users without 7 days trial i.e. right away.

In order to enable Office 365 connected groups in Yammer it you need to login to your Yammer network with administrator account and go to Network settings (gear icon in top left) > Network Admin > Security setting. On this page at first we need to enforce Office 365 identity for Yammer users:

Once it’s status will be changed to Committed after some time “Office 365 Connected Yammer Groups” setting will be changed to Enabled:

Documentation above says that it takes up to 24h to change status to Enabled. But in my case it was changed quite fast: almost immediately after enforcing of Office 365 identity for Yammer users.

Let’s see what happens when “Office 365 Connected Yammer Groups” setting is disabled for your Yammer network. When you create new Yammer group:

this group won’t be found in Azure AD of the Yammer network’s tenant:

After Office 365 Connected Yammer Groups have been enabled Yammer groups will appear in Yammer:

So you will be able to use all advantages of O365 connected groups. Note that for those Yammer groups which already exist when O365 connected groups had been enabled also appropriate O365 groups will be created after some time. Documentation says that it may take up to 1 week:

After about 1 week, existing eligible groups will be converted to Office 365 groups.

but for me it also happened quite fast within 1 hour or so.

Sync delay between O365 Group owners and members with Team owners and members

When you create O365 group and add team to this group you may face with the following issue: group owners and members which are added via Azure AD portal (https://portal.azure.com) are synced to team owners and members with delay. Let’s say we created group and added team to the group. During creation we specified 1 user as group owner:

If we will check now Team for this group > Manage team > Owners we will see that Owners list will be initially empty:

After some time group owner will appear in team owners:

(Note that there is still 0 owners count in parenthesis – but this is another issue. Probably bug in current version of Teams web app). Same happens also for group members and team members. This sync delay may take several hours.

In this forum thread there is suggests to use beta API for adding owners to group, i.e.:

https://graph.microsoft.com/beta/groups/{groupId}/owners/$ref

instead of

https://graph.microsoft.com/v1.0/groups/{groupId}/owners/$ref

But the point is that even with old API owners/members will be synchronized between group and team but after some time.

Update Azure AD group via MS Graph client library

In my previous posts I showed several scenarios of using MS Graph client library, see:

Work with Azure AD via Microsoft Graph API
Create Azure AD group and set group owner using Microsoft Graph Client library
Retrieve paginated data from Azure AD via Microsoft Graph Client library

In this article we will continue to get familiar with the MS Graph client library and see how to update Azure AD group programmatically. Examples in this post will use the same AzureAuthenticationProvider class for authenticating against Azure AD as in examples provided above so I won’t duplicate it here.

Here is how we can rename Azure AD group programmatically using MS Graph client library:

   1: renameGroup("oldGroupName", "newGroupName");

   2:  

   3: ...

   4:  

   5: static void renameGroup(string oldName, string newName)

   6: {

   7:     var graph = new GraphServiceClient(new AzureAuthenticationProvider());

   8:     var group = getGroup(oldName);

   9:     group.DisplayName = newName;

  10:  

  11:     var groupReq = new GroupRequest(graph.Groups[group.Id].Request().RequestUrl,

  12:         graph, new List<Option>());

  13:     var result = groupReq.UpdateAsync(group);

  14:  

  15:     do

  16:     {

  17:         Console.WriteLine("Result status: {0}", result.Status);

  18:         Thread.Sleep(5000);

  19:     } while (result.Status == TaskStatus.WaitingForActivation);

  20: }

  21:  

  22: static Group getGroup(string name)

  23: {

  24:     var graph = new GraphServiceClient(new AzureAuthenticationProvider());

  25:     try

  26:     {

  27:         var groups = graph.Groups.Request().GetAsync();

  28:         int requestNumber = 1;

  29:         while (groups.Result.Count > 0)

  30:         {

  31:             foreach (var g in groups.Result)

  32:             {

  33:                 if (string.Compare(g.DisplayName, name, true) == 0)

  34:                 {

  35:                     return g;

  36:                 }

  37:             }

  38:  

  39:             if (groups.Result.NextPageRequest != null)

  40:             {

  41:                 groups = groups.Result.NextPageRequest.GetAsync();

  42:             }

  43:             else

  44:             {

  45:                 break;

  46:             }

  47:         }

  48:         return null;

  49:     }

  50:     catch (ServiceException x)

  51:     {

  52:         Console.WriteLine("Exception occured: {0}", x.Error);

  53:         return null;

  54:     }

  55: }

At first in method retrieveGroup() we get reference on the Group object and update group’s DisplayName property (lines 7-9). Then we create GroupRequest object and call it’s UpdateAsync method (lines 11-13) and wait until request will be processed (lines 15-19). After that group will appear in Azure portal with new name. But note that if group was already used in Sharepoint Online site (e.g. for granting permissions on some site) changes won’t be synced here automatically – you will need to sync user profiles and then update user data in User information list.

Problem with delayed propagation of Azure AD groups to Sharepoint Online

As you probably know in Sharepoint Online it is possible to use Azure AD groups which belong to your tenant for configuring permissions. The problem is that Azure AD security groups are not become available in Sharepoint Online immediately after creation, there is some delay between the moment when group was created and the moment when it can be resolved in Sharepoint people picker. If you have automation process which creates Sharepoint Online site, Azure groups and then grants them permissions on created site, you need to handle this delay.

One of the solutions is to make several attempts to try to resolve Azure AD group in Sharepoint. If group is not available yet, wait some time and repeat attempt. And you need to decide maximum number of attempts and delay between them. Here is possible solution:

   1: private static void AddAzureGroupToSharepointGroup(ClientContext ctx,

   2:     Microsoft.SharePoint.Client.Group spGroup, string azureGroupName)

   3: {

   4:     if (spGroup == null)

   5:     {

   6:         Console.WriteLine("Sharepoint group is null (Azure group name '{0}')",

   7:             azureGroupName);

   8:         return;

   9:     }

  10:     if (string.IsNullOrEmpty(azureGroupName))

  11:     {

  12:         Console.WriteLine("Azure group name '{0}' is empty", azureGroupName);

  13:         return;

  14:     }

  15:  

  16:     Console.WriteLine("Add Azure group '{0}' to Sharepoint group '{1}'",

  17:         azureGroupName, spGroup.LoginName);

  18:  

  19:     int num = 0, numOfAttempts = 30;

  20:     do

  21:     {

  22:         try

  23:         {

  24:             var user = ctx.Web.EnsureUser(azureGroupName);

  25:             ctx.Load(user);

  26:             ctx.ExecuteQueryRetry();

  27:             break;

  28:         }

  29:         catch

  30:         {

  31:             Console.WriteLine("Group '{0}' is not available yet (attempt #{1})",

  32:                 azureGroupName, num + 1);

  33:         }

  34:         Thread.Sleep(60000);

  35:         num++;

  36:     } while (num < numOfAttempts);

  37:  

  38:     ctx.Web.AddUserToGroup(spGroup.LoginName, azureGroupName);

  39: }

This methods add specified Azure AD group to Sharepoint group. In order to make it work Azure group should be available in Sharepoint Online (should be resolvable in people picker). It makes max 30 attempts and waits 1 minute between each attempts (lines 19-36). If group is not available yet, then call to Web.EnsureUser() will throw exception (line 24). We catch this exception and increment attempts counter (lines 29-35). If group was resolved it means that it is propagated to Sharepoint Online (Azure AD groups are represented in Sharepoint Online as User object – in the same way as for regular AD groups) and we can add it to the Sharepoint group (line 38). Method Web.AddUserToGroup() is extension method which is implemented in OfficeDevPnP.Core.