Tuesday, November 2, 2021

How to set limit on simultaneously processed queue messages with continuous Azure web job

With continuous Azure web job we may create web job host which will listen to specified Azure storage queue. When new message will be added to queue Azure will trigger handler method found in assembly of continuous web job (public method which has parameter with [QueueTrigger] attribute):

public virtual void RunContinuous()
{
    var storageConn = ConfigurationManager.ConnectionStrings["AzureWebJobsDashboard"].ConnectionString;
    var config = new JobHostConfiguration();
    config.StorageConnectionString = storageConn;
    var host = new JobHost(config);
    host.RunAndBlock();
}

public void ProcessQueueMessage([QueueTrigger("orders")]string message, TextWriter log)
{
    log.WriteLine("New message has arrived from queue");
	...
}

If several messages were added simultaneously Azure will trigger several instances of handler method in parallel. Number of messages which can be processed in parallel can be configured via JobHostConfiguration.Queues.BatchSize property. By default it is set to 16 i.e. by default 16 messages can be processed simultaneously in one batch:

 

If we want e.g. configure continuous web job so it will process only 1 message at time we can set BatchSize to 1:

public virtual void RunContinuous()
{
    var storageConn = ConfigurationManager.ConnectionStrings["AzureWebJobsDashboard"].ConnectionString;
    var config = new JobHostConfiguration();
    config.StorageConnectionString = storageConn;
	
    // set batch size to 1
    config.Queues.BatchSize = 1;
	
    var host = new JobHost(config);
    host.RunAndBlock();
}

public void ProcessQueueMessage([QueueTrigger("orders")]string message, TextWriter log)
{
    log.WriteLine("New message has arrived from queue");
	...
}

Using the same approach we can increase BatchSize from default 16 to bigger value. However as you may notice from above screenshot MaxBatchSize is set to 32 to you may increase BatchSize only up to 32.

No comments:

Post a Comment