Thursday, July 16, 2020

One problem with caching old web job binaries after Azure web job update with WAWSDeploy

WAWSDeploy is convenient console utility which allows to deploy a folder or a zip file to an Azure Website using WebDeploy (see Using this tool it is possible to automate tasks in your Azure deployment. E.g. you may upload Azure web job binaries (they should be packaged to zip file for that). However you should be aware of one potential problem.

Triggered web job binaries are uploaded to "app_data/jobs/triggered/{JobName}" path (full path will be /site/wwwroot/app_data/jobs/triggered/{JobName}). You may connect to your Azure App Service via FTP (connection settings may be copied from Azure portal > App service > Deployment center > FTP) and check that your web jobs folders are there

Note that you need to use FTP protocol (not FTPS) if you will connect e.g. via WinSCP tool.

The problem is the following: you may create zip package with web job binaries just by selecting all files for web jobs and adding them to archive – in this case there won’t be root folder in archive and all files will be located inside archive root. Alternatively you may select their parent folder (e.g. bin/Release) and add whole folder to archive – in this case archive will contain Release root folder.

So if 1st time you created archive using 1st method (by archiving files), then created another archive using 2nd method (by archiving folder) and uploaded this 2nd zip archive via WAWSDeploy – it will just create subfolder Release inside app_data/jobs/triggered/{JobName}. I.e. there will be 2 versions of web job: one in the app_data/jobs/triggered/{JobName} and another in app_data/jobs/triggered/{JobName}/Release. Azure runtime will use 1st found exe file which will be exe file from the root (not from Release sub folder) and will use that for running web job.

One way to avoid this problem is to always delete existing web job before to update it (you may check how to delete web job via PowerShell here: How to remove Azure web job via Azure RM PowerShell). In this case old binaries will be deleted so selected archiving method won’t cause problems during update.

Wednesday, July 15, 2020

How to remove Azure web job via Azure RM PowerShell

Azure web jobs allow to run your code by scheduler. Schedule is configured via CRON expression. This is convinient tool to perform repeated actions. Sometimes we need to remove them via PowerShell in automation process. In this article I will show how to remove Azure web job using Azure RM PowerShell cmdlets.

If you will try to google this topic you will most probably find Remove-AzureWebsiteJob cmdlet. The problem however is that this cmdlet is used for classic resources and you will most probably have problems using it (e.g. because of error Select-AzureSubscription -Default). So we need some other way to remove Azure web jobs – preferably by using Azure RM cmdlets. Fortunately it is possible – we may remove web job by using another cmdlet Remove-AzureRmResource. Here is syntax for removing triggered web job:

$resourceName = $appServiceName + "/" + $webJobName
Remove-AzureRmResource -ResourceGroupName $resourceGroupName -ResourceName $resourceName -ResourceType microsoft.web/sites/triggeredwebjobs

The important thing to note here is how ResourceName param is specified for web job (app service name/web job name) and ResourceType which is set to microsoft.web/sites/triggeredwebjobs. If you need to delete continuous web job you need to use microsoft.web/sites/continuouswebjobs resourceType.

Using this command you will be able to delete Azure web jobs via Azure RM PowerShell.

Monday, July 6, 2020

Run Azure function as Web job without code change

Sometime it may be needed to test existing Azure function as Web job (e.g. if you decide to move some long-running process from AF to web job). In order to do that you need to create App service first. After that create new console app project in Visual Studio, reference your Azure functions project as it would be regular class library and from Program.cs call necessary Azure function (it is possible since AFs are created as static methods of the class).

The problem however is that if you read some app settings in Azure function code – you most probably do it like that:

string foo = Environment.GetEnvironmentVariable("Foo", EnvironmentVariableTarget.Process);

But if you will just copy AF app settings to appSettings section of app.config of the new console project then above code will return empty string for any app setting. So in order to call our Azure function from console application and run it as web job we need to perform additional step: iterate through all app settings and fill environment variables for the process:

var appSettings = ConfigurationManager.AppSettings;
foreach (var key in appSettings.AllKeys)
 Environment.SetEnvironmentVariable(key, appSettings[key],

After that app settings will be successfully read and Azure function may run as web job without changing its code.

Thursday, July 2, 2020

Sharepoint MVP 2020

Yesterday I got email from MS about my MVP status renewal. This is 10th award for me in a row and I would like to say thanks to MS and to whole community which gives me motivation and energy to continue sharing my experience and solutions from every-day work:

Sharepoint and MS ecosystem in general evolves: now we switched to Modern Sharepoint, tight integration with Azure which becomes more and more powerful. New technologies require new skills and experience so I think it is crucial that we will continue sharing our knowledge with colleagues all over the world. Global pandemic which we faced with this year introduced new challenges for IT world: tools which we build became as crucial as never before since people switched to remote work. Performance and stability went to the 1st plan. From this perspective it is very important to take part in IT community life by answering forum questions, contributing to open source software, speaking on events, writing technical articles and so on. Hope that my contributing helped someone and would like to say thanks one more time to MS for recognizing it.

Monday, June 29, 2020

Problem with ReSharper unit test runner with NUnit 3.12: unit test is inconclusive

Some time ago we switched to NUnit 3.12 and NUnit3TestAdapter in Camlex library and noticed the following problem with ReSharper unit test runner: all unit tests were marked as “inconclusive” and didn’t run. At the same time built-in Visual Studio test runner worked. I used 2019 version of ReSharper when found this problem.

By searching the web I found similar problems from other people in JetBrains support site. In order to fix it I uninstalled 2019 version and installed latest available version of ReSharper JetBrains.ReSharperUltimate.2020.1.3. After that test become working again:

So if you will face with the same problem try to install last ReSharper version.

Friday, June 12, 2020

Problem with threads count grow when use OfficeDevPnP AuthenticationManager

Recently we faced with the following problem in our Azure function app: after some time of functioning Azure functions got stuck and the only way to make them work again was to restart them manually from Azure portal UI. Research showed that problem was related with threads count: because of some reason threads count permanently grew to 6K thread after which Azure function app became unresponsive:

We reviewed our code, made some optimizations, change some async calls to sync equivalents, but it didn’t fix the problem. Then we continued troubleshooting and found the following: in the code we used OfficeDevPnP AuthenticationManager in order to get access token for communicating with Sharepoint Online like that:

using (var ctx = new OfficeDevPnP.Core.AuthenticationManager().GetAppOnlyAuthenticatedContext(url, clientId, clientSecret))

Note that in this example new instance of AuthenticationManager is created each time when above code was executed. In our case it was inside queue-triggered Azure function which was called quite frequently. Elio Struyf (we worked over this problem together so credits for this finding go to him Smile) tried to make AuthenticationManager static – i.e. only one instance was created for the whole Azure function worker process:

public static class Helper
 private static OfficeDevPnP.Core.AuthenticationManager authMngr = new OfficeDevPnP.Core.AuthenticationManager();

 public static void QueueTriggeredFunc()
  using (var ctx = authMngr.GetAppOnlyAuthenticatedContext(url, clientId, clientSecret))

After that threads count got stabilized with avg 60 threads:

We reported this problem to OfficeDevPnP team and they confirmed that there is problem in AuthenticationManager which internally creates thread for monitoring access token’s lifetime, but this thread is not released when AuthenticationManager got recycled by garbage collector (also need to mention that AuthenticationManager itself was not disposable at the moment when this problem was found. We used OfficeDevPnP 3.20.2004.0 when found this problem). For now this workaround with making AuthenticationManager static was Ok and as far as I know currently OfficeDevPnP team works over correct fix for this problem. So hopefully it will be available soon globally.

Update 2020-06-15: from latest news I've got (thanks to Yannick Plenevaux :) ) OfficeDevPnP team addressed this issue in 202006.2 version and made AuthenticationManager disposable. So you may need to review your code in order to add using() around it.

Monday, June 8, 2020

Fix AccessToKeyVaultDenied error in Azure App service app settings which use KeyVault reference

Azure KeyVault is convenience safe storage for secrets (passwords, keys, etc.) which can be used in your apps instead of storing them in plain text in app settings. It adds number of advantages like access control, expiration policies, versioning, access history and others. However it has some gotchas which you should be aware. In this post I will describe one such gotcha.

First of all you need to configure access to key vault secret so your App service or Azure function will be able to read values from there. You may check how to do that e.g. here: Provide Key Vault authentication with an access control policy. After that go to App service > Configration and click Edit icon for the app setting which uses reference on KeyVault secret and check that there are no errors. Because even if you did everything correctly you may see Status = AccessToKeyVaultDenied and the following error description:

Key Vault reference was not able to be resolved because site was denied access to Key Vault reference's vault

In order to fix it try the following workaround:

  1. Delete app setting from UI
  2. Save changes
  3. Add the same app setting with KeyVault reference (i.e. with @Microsoft.KeyVault(SecretUri=…))
  4. Save changes again

After that if permissions are configured properly Status should be changed to Resolved:

and your app should be able to successfully resolve secret from KeyVault reference.