Get Sharepoint data in browser console via Rest API without additional tools

Sometimes during troubleshooting you need to quickly get some data from Sharepoint, e.g. id of current site collection. There are many ways how to do that with additional tools e.g. from PowerShell and PnP, SPEditor Chrome extension and it's pnpjs console, etc. But it requires installation of these tools and their knowledge (of course if you work with Sharepoint it will be better if you will know these tools :) ).

One way how you may get this data without extra tools is to use SP Rest API directly from browser console. E.g. for getting site collection details we may fetch /_api/site endpoint and output JSON response to console:

fetch("https://{mytenant}.sharepoint.com/sites/test/_api/site", {headers: {"accept": "application/json; odata=verbose"}}).then(response => response.json().then(txt => console.log(JSON.stringify(txt))))

(here instead of {mytenant} you should use your tenant name. Note however that this approach will also work in on-prem)

It will output a lot of information about current site collection to the console:

{
    "d": {
        "__metadata": {
            "id": "https://{mytenant}.sharepoint.com/sites/test/_api/site",
            "uri": "https://{mytenant}.sharepoint.com/sites/test/_api/site",
            "type": "SP.Site"
        },
        "Audit": {
            "__deferred": {
                "uri": "https://{mytenant}.sharepoint.com/sites/test/_api/site/Audit"
            }
        },
        "CustomScriptSafeDomains": {
            "__deferred": {
                "uri": "https://{mytenant}.sharepoint.com/sites/test/_api/site/CustomScriptSafeDomains"
            }
        },
        "EventReceivers": {
            "__deferred": {
                "uri": "https://{mytenant}.sharepoint.com/sites/test/_api/site/EventReceivers"
            }
        },
        "Features": {
            "__deferred": {
                "uri": "https://{mytenant}.sharepoint.com/sites/test/_api/site/Features"
            }
        },
        "HubSiteSynchronizableVisitorGroup": {
            "__deferred": {
                "uri": "https://{mytenant}.sharepoint.com/sites/test/_api/site/HubSiteSynchronizableVisitorGroup"
            }
        },
        "Owner": {
            "__deferred": {
                "uri": "https://{mytenant}.sharepoint.com/sites/test/_api/site/Owner"
            }
        },
        "RecycleBin": {
            "__deferred": {
                "uri": "https://{mytenant}.sharepoint.com/sites/test/_api/site/RecycleBin"
            }
        },
        "RootWeb": {
            "__deferred": {
                "uri": "https://{mytenant}.sharepoint.com/sites/test/_api/site/RootWeb"
            }
        },
        "SecondaryContact": {
            "__deferred": {
                "uri": "https://{mytenant}.sharepoint.com/sites/test/_api/site/SecondaryContact"
            }
        },
        "UserCustomActions": {
            "__deferred": {
                "uri": "https://{mytenant}.sharepoint.com/sites/test/_api/site/UserCustomActions"
            }
        },
        "AllowCreateDeclarativeWorkflow": false,
        "AllowDesigner": true,
        "AllowMasterPageEditing": false,
        "AllowRevertFromTemplate": false,
        "AllowSaveDeclarativeWorkflowAsTemplate": false,
        "AllowSavePublishDeclarativeWorkflow": false,
        "AllowSelfServiceUpgrade": true,
        "AllowSelfServiceUpgradeEvaluation": true,
        "AuditLogTrimmingRetention": 90,
        "ChannelGroupId": "00000000-0000-0000-0000-000000000000",
        "Classification": "",
        "CompatibilityLevel": 15,
        "CurrentChangeToken": {
            "__metadata": {
                "type": "SP.ChangeToken"
            },
            "StringValue": "..."
        },
        "DisableAppViews": false,
        "DisableCompanyWideSharingLinks": false,
        "DisableFlows": false,
        "ExternalSharingTipsEnabled": false,
        "GeoLocation": "EUR",
        "GroupId": "00000000-0000-0000-0000-000000000000",
        "HubSiteId": "00000000-0000-0000-0000-000000000000",
        "Id": "32d406dc-dc97-46dd-b01c-e6346419ceb7",
        "SensitivityLabelId": null,
        "SensitivityLabel": "00000000-0000-0000-0000-000000000000",
        "IsHubSite": false,
        "LockIssue": null,
        "MaxItemsPerThrottledOperation": 5000,
        "MediaTranscriptionDisabled": false,
        "NeedsB2BUpgrade": false,
        "ResourcePath": {
            "__metadata": {
                "type": "SP.ResourcePath"
            },
            "DecodedUrl": "https://{mytenant}.sharepoint.com/sites/test"
        },
        "PrimaryUri": "https://{mytenant}.sharepoint.com/sites/test",
        "ReadOnly": false,
        "RequiredDesignerVersion": "15.0.0.0",
        "SandboxedCodeActivationCapability": 2,
        "ServerRelativeUrl": "/sites/test",
        "ShareByEmailEnabled": false,
        "ShareByLinkEnabled": false,
        "ShowUrlStructure": false,
        "TrimAuditLog": true,
        "UIVersionConfigurationEnabled": false,
        "UpgradeReminderDate": "1899-12-30T00:00:00",
        "UpgradeScheduled": false,
        "UpgradeScheduledDate": "1753-01-01T00:00:00",
        "Upgrading": false,
        "Url": "https://{mytenant}.sharepoint.com/sites/test",
        "WriteLocked": false
    }
}

sUsing the same approach you may call another Rest API end points directly from browser console. It may save your time during troubleshooting. Hope this information will help someone.

Get log of triggered Azure web job via REST API programmatically in PowerShell

Continue series of posts about Azure web jobs with this post how to fetch log of triggered Azure web job programmatically via REST API. Previous articles from the series can be found here:

• Get status of timer triggered Azure web job via REST API in PowerShell
• Trigger web job via web jobs REST API
  

In order to get log of triggered Azure web job programmatically we need to use the following endpoint:

https://{webAppName}.scm.azurewebsites.net/api/triggeredwebjobs/{webJobName}/history

It returns array of runs - information about last run is stored in first element of array. One of the properties there is output_url which contains url of the log of this run. Having this information we may write the following PowerShell function which will fetch log of Azure web job run:

function Get-Web-Job-Most-Recent-Log {
    param (
        $webAppName,
        $webJobName
    )
	$token = Get-AzAccessToken
	$headers = @{ "Authorization" = "Bearer $($token.Token)" }
	$userAgent = "powershell/1.0"
	$history = Invoke-RestMethod -Uri "https://$($webAppName).scm.azurewebsites.net/api/triggeredwebjobs/$($webJobName)/history" -Headers $headers -UserAgent $userAgent -Method GET
	if ($history -eq $null -or $history.runs -eq $null -or $history.runs.Length -eq 0) {
		return $null
	}
	$log = Invoke-RestMethod -Uri $history.runs[0].output_url -Headers $headers -UserAgent $userAgent -Method GET
	return $log
}

With this method we may fetch log of Azure web job programmatically.

Trigger web job via web jobs REST API

In my previous post I showed how to get status of triggered Azure web job via web jobs REST API in PowerShell (see Get status of timer triggered Azure web job via REST API in PowerShell). In this post I will show how to trigger web job programmatically also via REST API.

For triggering web job via REST API we need to send HTTP POST request to the following endpoint:

https://{webAppName}.scm.azurewebsites.net/api/triggeredwebjobs/{webJobName}/run

PowerShell method which triggers web job looks like that:

function Run-Web-Job {
    param (
        $webAppName,
	$webJobName
    )
	$token = Get-AzAccessToken
	$headers = @{ "Authorization" = "Bearer $($token.Token)" }
	$userAgent = "powershell/1.0"
	Invoke-RestMethod -Uri "https://$($webAppName).scm.azurewebsites.net/api/triggeredwebjobs/$($webJobName)/run" -Headers $headers -UserAgent $userAgent -Method POST
}

Here we first get access token via Get-AzAccessToken cmdlet and sent POST request with received bearer token. As result it will trigger specified web job to run now.

Get status of timer triggered Azure web job via REST API in PowerShell

In order to get current status of timer triggered Azure web job we may use the following web jobs REST API endpoint:

https://{webAppName}.scm.azurewebsites.net/api/triggeredwebjobs/{webJobName}/history

It will return object with "runs" property which is array containing information about recent runs:

id         : 202206101445293407
name       : 202206101445293407
status     : Running
start_time : 2022-06-10T14:45:29.3564164Z
end_time   : 0001-01-01T00:00:00
duration   : 00:00:04.0337019
output_url : https://{webAppName}.scm.azurewe.scm.azurewebsites.net/vfs/data/jobs/triggered/{webJobName}/202206101445293407/output_log.txt
error_url  :
url        : https://{webAppName}.scm.azurewe.scm.azurewebsites.net/api/triggeredwebjobs/{webJobName}/history/202206101445293407
job_name   : test
trigger    : External -

First element of this array corresponds to the most recent run. If web job is currently running status property will be set to Running. Here is function which we can use for fetching web job status in PowerShell:

function Get-Web-Job-Status {
    param (
    	$webAppName,
	$webJobName
    )
	$token = Get-AzAccessToken
	$headers = @{ "Authorization" = "Bearer $($token.Token)" }
	$userAgent = "powershell/1.0"
	$history = Invoke-RestMethod -Uri "https://$($webAppName).scm.azurewebsites.net/api/triggeredwebjobs/$($webJobName)/history" -Headers $headers -UserAgent $userAgent -Method GET
	if ($history -eq $null -or $history.runs -eq $null -or $history.runs.Length -eq 0) {
		return $null
	}
	return $history.runs[0].status
}

It makes HTTP GET request to mentioned endpoint and gets list of runs for specified web job.

Get functions invocations list for continuous Azure web job via REST API

This post is continuation of series of articles about continuous Azure web jobs. You may find previous articles about continuous web jobs here:

• How to run continuous Azure web job as singleton
• How to configure number of retries for processing failed Azure queue message before it will be moved to poison queue
• How to set limit on simultaneously processed queue messages with continuous Azure web job
  

In this post I will show how to get list of functions' invocations for continuous web job via REST API.

As you probably know (and as it was shown in one of above articles) when you work with continuous web job we need to implement function which will be triggered e.g. using Azure queue trigger. Log which is shown in Azure portal looks bit different for continuous web jobs comparing with timer triggered web jobs: every time when this handler function got triggered new invocation is added to the log. It contains details of this particular function's invocation:

In order to get this list of functions' invocations programmatically we may use WebJobs REST API and more specifically the following endpoint:

https://{app-service-name}.scm.azurewebsites.net/azurejobs/api/jobs/continuous/{webJob}/functions?limit={limit}

Response will look like that:

{
    "entries": [{
            "executingJobRunId": null,
            "id": "0399f7d5-a7aa-43c9-8853-70886174fda3",
            "functionId": null,
            "functionName": null,
            "functionFullName": null,
            "functionDisplayTitle": "ProcessQueueMessage (5540, )",
            "status": "CompletedSuccess",
            "whenUtc": "2022-05-03T12:12:05.3054119Z",
            "duration": 101650.2167,
            "exceptionMessage": null,
            "exceptionType": null,
            "hostInstanceId": "00000000-0000-0000-0000-000000000000",
            "instanceQueueName": null
        }, {
            "executingJobRunId": null,
            "id": "d581d6fa-8ba0-4c4d-ad39-c8505d21b851",
            "functionId": null,
            "functionName": null,
            "functionFullName": null,
            "functionDisplayTitle": "ProcessQueueMessage (5539, )",
            "status": "CompletedSuccess",
            "whenUtc": "2022-05-03T12:12:04.4842786Z",
            "duration": 100893.4565,
            "exceptionMessage": null,
            "exceptionType": null,
            "hostInstanceId": "00000000-0000-0000-0000-000000000000",
            "instanceQueueName": null
        }
    ],
    "continuationToken": null,
    "isOldHost": false
}

It contains array of objects which represent invocation details. Id property contains actual invocationId which may be used e.g. for getting invocation log by the following url:

https://{app-service-name}.scm.azurewebsites.net/azurejobs/#/functions/invocations/{invocationId}

Property functionDisplayTitle contains name of invocation together with passed parameter. It may hep if you need to map input parameters with invocation log. Also pay attention on continuationToken - if there are many invocations (more than passed in limit query string parameter) it will contain continuation token using which you may get whole list of invocations by pages.

List and delete Azure web jobs using KUDU Web Jobs REST API

Some time ago I wrote how to remove Azure web jobs using AzureRM PowerShell module: How to remove Azure web job via Azure RM PowerShell. However since that time AzureRM was discontinued. MS recommended to switch to new Az module instead. The problem that if you will follow AzureRM migration guide then the same code won't work anymore. The main problem is that new Get-AzResource cmdlet doesn't return resources with type microsoft.web/sites/triggeredwebjobs (although it still returns  other Azure resources like App services, Azure functions, Storage accounts, KeyVaults, SSL certificates). It means that you also can't use Remove-AzResource cmdlet because there won't be resource name which you may pass there. For now I submitted issue to Azure Powershell github about that: Get-AzResource doesn't return web jobs. But let's see how we may workaround that until MS will fix it.

There is another open issue in Azure PowerShell github ARM :: WebApp Web Job Cmdlets which reports about lack of cmdlets for managing web jobs. It was created in 2015 and is still opened. One suggestion there is to use az cli tool. However it is not always possible (e.g. if you run your PowerShell in runbook or can't add this dependency to your script).

Fortunately there is KUDU Web Job REST API which may be utilized for this purpos. In order to list available web jobs we may use the following code:

$token = Get-AzAccessToken
$headers = @{ "Authorization" = "Bearer $($token.Token)" }
$userAgent = "powershell/1.0"
Invoke-RestMethod -Uri "https://{appServiceName}.scm.azurewebsites.net/api/triggeredwebjobs" -Headers $headers -UserAgent $userAgent -Method GET

It will return list of all existing (in our example triggered) web jobs in specific App service. In order to delete web job we need to send HTTP DELETE request to web job's endpoint:

$token = Get-AzAccessToken
Invoke-RestMethod -Uri "https://{appServiceName}.scm.azurewebsites.net/api/triggeredwebjobs/{webjobName}" -Headers $headers -UserAgent $userAgent -Method DELETE

There are other commands as well (like run job, list running history and others). You may check them in REST API documentation using link posted above.

Issue with PnP JS sp.site.exists() call when page url contains hashes

Today we faced with interesting problem when tried to use sp.site.exists(url) method from PnP JS for determining whether site collection with specified url exists or not: all calls returned 403 with System.UnauthorizedAccessException:

During analysis I've found that it tried to make HTTP POST calls to the following url for all sites: https://{tenant}.sharepoint.com/sites/_api/contextinfo which is definitely not correct because https://{tenant}.sharepoint.com/sites is not correct url but managed path. Then we noticed that page where this code was executed contains #/ in url:

https://{tenant}.sharepoint.com/sites/mysitecol#/favorites

and it seems like PnP JS parses it incorrectly in this case. As workaround we checked which API is used inside sp.site.exists(url) call (/_api/SP.Site.Exists) and rewrote code with explicit API call instead:

let promise = context.spHttpClient.post(context.pageContext.site.absoluteUrl + "/_api/SP.Site.Exists",
  SPHttpClient.configurations.v1, {
  headers: {
	"accept": "application/json;"
  },
  body: JSON.stringify({
	url: url
  })
}).then((response: SPHttpClientResponse) => {
  response.json().then((exists: any) => {
	if (exists.value) {
	  // site exists
	} else {
	  // site doesn't exist
	}
  });
});

Hope that PnP JS will fix this issue at some point.

Update 2021-07-05: issue inside pnp js was fixed by adding initialization call:

sp.setup({
  spfxContext: this.context
});

See this discussion for more details.

Get Sharepoint site collection id (SPSite.ID) and web id (SPWeb.ID) from followed sites returned from REST API

In Sharepoint we can fetch all followed sites for the current user by the following REST API endpoint:

http://example.com/_api/social.following/my/Followed(types=4)

(instead of http://example.com you should use url of your SP site). It will return collection of site objects which will contain such properties as name, url, etc. However often we need to know also site collection id (SPSite.ID) and web id (SPWeb.ID). We can of course go through all returned sites and fetch their ids by separate JSOM/CSOM calls but it will affect performance (it is classic n+1 problem when we at first get list of items (1st call) and then for each item in the list make separate API call (n calls)).

Fortunately it is possible to get these ids right from REST API response. There is one strange field called "id" which looks like this:

"Id": "8.b4af2aa5fb834daa87aa9fb4155abd7d.b14bd3b3d6084dadb0fc7b79679fc767.
b4af2aa5fb834daa87aa9fb4155abd7d.00000000000000000000000000000000",

So there are several strings divided by dot. If we will check site and web id we will see that second string looks like SPSite.ID and 3rd string like SPWeb.ID:

The only difference is that they don't contain dashes. But it is quite easy do add them by ourselves in the code:

id.substr(0, 8) + "-" + id.substr(8, 4) + "-" + id.substr(12, 4) + "-" + id.substr(16, 4) + "-" + id.substr(20)

Using this approach we can get ids of Sharepoint site collections and web sites directly from REST API response.

Several problems when follow/unfollow Sharepoint site via REST API

If you need to implement follow/unfollow site functionality via javascript you most probably will find the following MS article Follow documents, sites, and tags by using the REST service in SharePoint which contains the following example which uses REST API:

$.ajax( {
	url: followingManagerEndpoint + "/isfollowed",
	type: "POST",
	data: JSON.stringify( { 
		"actor": {
			"__metadata": {
				"type":"SP.Social.SocialActorInfo"
			},
			"ActorType":2,
			"ContentUri":siteUrl,
			"Id":null
		} 
	}),
	headers: { 
		"accept":"application/json;odata=verbose",
		"content-type":"application/json;odata=verbose",
		"X-RequestDigest":$("#__REQUESTDIGEST").val()
	},
	success: function (responseData) { 
		...
	},
	error: requestFailed
});

However currently if you will try to run it you will get the following 400 Bad request error:

{"error":{"code":"-1, Microsoft.OData.Core.ODataException","message":"The property '__metadata' does not exist on type 'SP.Social.SocialActorInfo'. Make sure to only use property names that are defined by the type."}}

The problem is with __metadata property of the actor which seems to be outdated nowadays. In order to avoid this error just remove or comment __metadata from actor object.

Another problem is related with odata=verbose specified in Accept header. If you will try to follow site with it API will return the following 406 Not acceptable error:

{"error":{"code":"-1, Microsoft.SharePoint.Client.ClientServiceException","message":"The HTTP header ACCEPT is missing or its value is invalid."}}

In order to resolve it change odata=verbose to odata.metadata=minimal in Accept header. Here is the working code written with Typescript and SPFx:

context.spHttpClient.post(context.pageContext.web.absoluteUrl + "/_api/social.following/follow",
  SPHttpClient.configurations.v1, {
	headers: {
	  'Accept': 'application/json;odata.metadata=minimal',
	  'Content-type': 'application/json;odata=verbose',
	},
	body: JSON.stringify({
	  "actor": {
		  /*"__metadata": {
			  "type": "SP.Social.SocialActorInfo"
		  },*/
		  "ActorType": 2,
		  "ContentUri": site.url,
		  "Id": null
	  }
	})
  });

Hope that it will help someone.

How to explore Sharepoint REST API endpoints

Sometimes you need to get list of available operations in the Sharepoint REST API endpoint. Let’s say we want to check operations available for /_api/SitePages endpoint.

First of all we need to get authentication cookies. In order to get them lunch Fiddler and open e.g. Sharepoint landing page (/_layouts/15/Sharepoint.aspx) which is opened from App launcher > Sharepoint. On this page there will be several REST API calls which will contain Cookies header. E.g. /_api/GroupSiteManager/CanUserCreateGroup:

From this Fiddler view copy value of Cookie header.

After that launch Postman and create request on endpoint in question: /_api/SitePages. In Headers section add Cookie, put value copied from Fiddler and click Send:

In the response it will return list of relative endpoint operations available under selected endpoint. In this example they are:

• /_api/SitePages/CommunicationSite
• /_api/SitePages/Pages
• /_api/SitePages/PublishingSite

Note that this method doesn’t return POST/PUT endpoints unfortunately.

Problem with sync delay between Azure AD and Sharepoint Online when Rest API is used

When you create user or group in Azure AD it is not immediately available in Sharepoint Online. I wrote about this problem here: Problem with delayed propagation of Azure AD groups to Sharepoint Online. In this post I will describe another interesting problem which may occur because of this delay.

Azure AD group members and owners may be retrieved with Graph API and with Rest API:

Graph AP endpoint:

https://graph.microsoft.com/v1.0/groups/{groupId}/members

Rest API endpoint

http://example.com/_api/SP.Directory.DirectorySession/Group('{groupId}')/members?$select=displayName,id

where instead of http://example.com you need to use url of your Sharepoint site.

The problem is that until Azure AD data won’t be fully synced to Sharepoint Online Rest API may return not correct data. E.g. /members endpoint may return actually owners, while /owners endpoint may not return users at all. Depending on how fast MS data center will propagate changes it may take up to several hours. So be aware about this problem.

The main advantage of Rest API endpoint is that it returns members count. While in Gtaph API $count query string parameter is not supported for users and groups: Use query parameters to customize responses:

Note: $count is not supported for collections of resources that derive from directoryObject like collections of users or groups.

So you may want to use Rest but notice that it may work incorrectly first several hours.