Monday, October 18, 2021

Obfuscar + NVelocity + anonymous types = issue

Some time ago I faced with interesting issue related with Obfuscar and anonymous types used with NVelocity template engine. If you use NVelocity for populating templates then your code may look like this:

string Merge(string templateText, params Tuple<string, object>[] args)
{
    var context = new VelocityContext();
    foreach (var t in args)
    {
        context.Put(t.Item1, t.Item2);
    }

    var velocity = new VelocityEngine();
    var props = new ExtendedProperties();
    velocity.Init(props);
    using (var sw = new StringWriter())
    {
        velocity.Evaluate(context, sw, "", templateText);
        sw.Flush();
        return sw.ToString();
    }
}

Here we pass templateText itself and tuple(s) of (key, value) pairs where key is string name of parameter used inside template (like $p) and value is object which has to be used for expanding this parameter to the real string value in resulting text (e.g. if you have "$p.Foo" inside template then pass object which has "Foo" string property set to "bar" value - then "$p.Foo" in template will be replaced by "bar").

For objects passed as parameters into this method anonymous types can be used - it is very convenient:

string text = Merge("My object $p.Foo", Tuple.Create("p", new { Foo = "bar" }));

If you will run this code as is it will return "My object foo" which is correct. But if you will try to obfuscate your code with Obfuscar then you will get unexpected result: instead of expanded template you will get original template text "My object $p.Foo" i.e. template won't be expanded to string.

The reason of this problem is that Obfuscar by default removes properties of anonymous types - there are issues about that: Anonymous class obfuscation removes all the properties or Anonymous Types. In order to solve the problem you need to instruct Obfuscar to not obfuscate anonymous types:

<Module file="MyAssembly.dll">
  <SkipType name="*AnonymousType*" skipProperties="true" skipMethods="true" skipFields="true" skipEvents="true" skipStringHiding="true" />
</Module>

After that anonymous types won't be changed and NVelocity templates will be populated to string correctly.

Friday, October 15, 2021

Convert self-signed SSL certificate's private key from .pfx to .key and .crt

Some time ago I wrote a post how to use self-signed SSL certificate in Azure App service: Use self-signed SSL certificate for web API hosted in Azure App service. In this article we generated self-signed certificate, exported private key to pfx file and used it in Azure. But what it you will use another hosting provider which require private key in .key format and SSL certificate itself in .crt file? Good news is that it is possible to create .key/.crt files from .pfx and in this article I will show how to do that.

We will need openssl.exe tool which will make actual work. If you will try to Google it there will be plenty of 3rt party sites where you may download that. The problem however is that there is no guarantee that these are safe download links (i.e. that there won't be malwares and viruses). Safer option is to use openssl.exe which is shipped with git client for Windows: I found my in the following location:

C:\Program Files\Git\usr\bin\openssl.exe

With this tool in order to export SSL certificate from .pfx to .crt we may use the following command:

openssl pkcs12 -in myCert.pfx -clcerts -nokeys -out myCert.crt

and in order to export private key to .key format from .pfx the following:

openssl pkcs12 -in myCert.pfx -nocerts -out myCert-encrypted.key

Here you will need to specify pfx password and also provide so called PEM pass phrase which will proteck .key file. I.e. private key in .key format will be still encrypted. If you will need it in unencrypted format (ensure that it will be stored in safe location in this case) use the following command:

openssl rsa -in myCert-encrypted.key -out myCert-decrypted.key

After that you will be able to use self-signed SSL certificate in .key/.crt format.

Monday, October 11, 2021

One reason for Connect-SPOService error: The sign-in name or password does not match one in the Microsoft account system

Today we have faced with interesting issue. When tried to use Connect-SPOService cmdlet on one tenant we got the following error:

The sign-in name or password does not match one in the Microsoft account system

Interesting that on other tenants this cmdlet worked properly. Troubleshooting and searching showed that one of the reason of this error can be enabled MFA (more specifically, when Connect-SPOService is used with -Credentials param). However we double checked that for those accounts for which this error was shown MFA was disabled.

Then we tried to login to O365 site with this account in browser and my attention was attracted by the following message which was shown after successful login:

Microsoft has enabled security defaults to keep your account secure:

 


As it turned out on this tenant AAD Security defaults were enabled which forced MFA for all users. In turn it caused mentioned error with Connect-SPOService. Solution was to disable security defaults in AAD properties:

After that error disappeared and we were able to use SPO cmdlets.

Friday, October 8, 2021

Fix Azure web job startup error "Storage account connection string 'AzureWebJobsStorage' does not exist"

If you develop continuous Azure web job which monitors e.g. Azure storage queue messages then your code may look like this (in our example we will use latest v3 version of Microsoft.Azure.WebJobs):

var builder = new HostBuilder();
static void Main(string[] args)
{
    var builder = new HostBuilder();
    builder.UseEnvironment("development");
    builder.ConfigureWebJobs(b =>
    {
        b.AddAzureStorageCoreServices();
        b.AddAzureStorage();
    });
    builder.ConfigureLogging((context, b) =>
    {
        b.AddConsole();
    });
    var host = builder.Build();
    using (host)
    {
        host.Run();
    }
}

public static void ProcessQueueMessage([QueueTrigger("myqueue")] string message, TextWriter log)
{
    ...
}

Here in Main method (which is entry point for continuous web job) we create host which subscribes on specified queue. When new message will be added to the queue it should trigger our ProcessQueueMessage handler. However on startup you may get the following error:

Storage account connection string 'AzureWebJobsStorage' does not exist

This error is thrown when host can't find connection string AzureWebJobsStorage of Azure storage where specified queue is located. In order to avoid this error we need to create appsettings.json file in our project which looks like this:

{
  "ConnectionStrings": {
    "AzureWebJobsStorage": "..."
  }
}

Also in file's properties we need to set Copy to output directory = Copy always so it will be always copied in output directory with exe file. After that error should disappear and web job should be successfully started.

Friday, October 1, 2021

Use self-signed SSL certificate for web API hosted in Azure App service

Suppose that we develop web API (e.g. https://api.example.com) and want to use self-signed certificate for it's domain name. Since it is web API we usually don't worry a lot about errors or warning which browser may show when you try to access it by url there. At the same time we don't want to compromise security and thus want to allow only our SSL certificate with known hash/thumbprint.

First of all we need to create self-signed certificate for domain name which will be used for web API (it should contain CN=api.example.com otherwise it won't be possible to bind it to custom domain name in Azure App service). Also certificate should be created with exportable private key (pfx) so we can use it in Azure. It can be done by the following PowerShell:

$start = [System.DateTime]::Now.AddDays(-1)
$end = $start.AddYears(2)
New-SelfSignedCertificate `
    -FriendlyName "api.example.com" `
    -KeyFriendlyName "api.example.com" `
    -KeyAlgorithm "RSA" `
    -DnsName "api.example.com" `
    -NotBefore $start `
    -NotAfter $end `
    -KeyUsage CertSign, CRLSign, DataEncipherment, DigitalSignature, NonRepudiation `
    -KeyUsageProperty All `
    -KeyLength 2048 `
    -CertStoreLocation cert:\LocalMachine\My `
    -KeyExportPolicy Exportable
$cert = Get-ChildItem -Path Cert:\LocalMachine\my | where-object { $_.Subject -eq "CN=api.example.com" }
$password = Read-Host -Prompt "Enter Password to protect private key" -AsSecureString
Export-PfxCertificate -Cert $cert -Password $password -FilePath api.example.com.pfx

Here we first create certificate itself and then export it's private key to file system.

Next step is to add custom domain name api.example.com to Azure app service where we will host our API. First of all we need to ensure that current pricing tier supports "Custom domains / SSL" feature. Currently minimal pricing tier with this option is B1 (it is not free):

Then we go to App service > Custom domains > Add custom domain and specify desired domain name for our web API:

Before Azure will allow to use custom domain name we will need to prove hostname ownership by adding TXT and CNAME DNS records for specified domain name - it is done in hosting provider control panel (here are detailed instructions of the whole process: Map an existing custom DNS name to Azure App Service).

Last part is related with client which call our web API. Since web API uses self-signed certificate attempt to call it from C# using HttpClient will fail. We need to configure it to allow usage of our SSL certificate but at the same time don't allow to use other self-signed SSL certificates. It can be done with the following C# code:

// thumprint of self-signed SSL certificate
private const string CERT_THUMBRINT = "...";

ServicePointManager.ServerCertificateValidationCallback = validateServerCertficate;
...
private static bool validateServerCertficate(
    object sender,
    X509Certificate cert,
    X509Chain chain,
    SslPolicyErrors sslPolicyErrors)
{
    if (sslPolicyErrors == SslPolicyErrors.None)
    {
        // Good certificate.
        return true;
    }

    return string.Compare(cert.GetCertHashString(), CERT_THUMBRINT, true) == 0;
}

In this code we instruct ServicePointManager to allow only our self-signed certificate (if error occurred during checking of SSL certificate when connection to remote host is established). Those it won't allow connections to hosts which use other self-signed SSL certificates.

Update 2021-10-15: see also post Convert self-signed SSL certificate's private key from .pfx to .key and .crt which describes how to export private key of self-signed certificate from pfx to key/crt.

Thursday, September 30, 2021

Speaking on online IT community event

Today I was speaking on online IT community event about development and debugging of Azure functions: https://tulaitcommunity.ru. On this event I wanted to show how to efficiently develop Azure functions and use them in Sharepoint Online. Development of Azure functions for Sharepoint Online require skills and experience of usage of many tools like Visual Studio, Postman, ngrok, PnP PowerShell, SPO browser. They were used in my demo. Presentation from speech is available from Slideshare here: https://www.slideshare.net/sadomovalex/azure-sharepoint-online. Recording from online meeting is available on youtube (on Russian language):


 

Hope that it was interesting :)

Wednesday, September 1, 2021

List and delete Azure web jobs using KUDU Web Jobs REST API

Some time ago I wrote how to remove Azure web jobs using AzureRM PowerShell module: How to remove Azure web job via Azure RM PowerShell. However since that time AzureRM was discontinued. MS recommended to switch to new Az module instead. The problem that if you will follow AzureRM migration guide then the same code won't work anymore. The main problem is that new Get-AzResource cmdlet doesn't return resources with type microsoft.web/sites/triggeredwebjobs (although it still returns  other Azure resources like App services, Azure functions, Storage accounts, KeyVaults, SSL certificates). It means that you also can't use Remove-AzResource cmdlet because there won't be resource name which you may pass there. For now I submitted issue to Azure Powershell github about that: Get-AzResource doesn't return web jobs. But let's see how we may workaround that until MS will fix it.

There is another open issue in Azure PowerShell github ARM :: WebApp Web Job Cmdlets which reports about lack of cmdlets for managing web jobs. It was created in 2015 and is still opened. One suggestion there is to use az cli tool. However it is not always possible (e.g. if you run your PowerShell in runbook or can't add this dependency to your script).

Fortunately there is KUDU Web Job REST API which may be utilized for this purpos. In order to list available web jobs we may use the following code:

$token = Get-AzAccessToken
$headers = @{ "Authorization" = "Bearer $($token.Token)" }
$userAgent = "powershell/1.0"
Invoke-RestMethod -Uri "https://{appServiceName}.scm.azurewebsites.net/api/triggeredwebjobs" -Headers $headers -UserAgent $userAgent -Method GET

It will return list of all existing (in our example triggered) web jobs in specific App service. In order to delete web job we need to send HTTP DELETE request to web job's endpoint:

$token = Get-AzAccessToken
Invoke-RestMethod -Uri "https://{appServiceName}.scm.azurewebsites.net/api/triggeredwebjobs/{webjobName}" -Headers $headers -UserAgent $userAgent -Method DELETE

There are other commands as well (like run job, list running history and others). You may check them in REST API documentation using link posted above.