Azure Container Apps Jobs

Azure Container Apps is a fairly new service, but it is maturing at blazing speed. The product team puts in a lot of effort to push out more and more features. One of these major features in my opinion is Azure Container Apps Jobs. The feature does exactly what you’d expect by its name, execute a job. You could compare jobs with Azure Functions as both need a trigger to execute. However, the bindings you get with Azure Functions are not available in Jobs. Instead, Jobs are very volatile containers that are designed to spin up, execute a job and be torn down again.

Triggers

Jobs can be triggered in three different ways:

Manual: As easy as it sounds. You can fire the job by hand, for example from within the Azure Portal.
Timer: You specify a timer schedule CRON style, and the job will execute periodically
Event Driven: This allows your job to execute whenever an event occurs. For example, a message ends up in a queue.

In this blog, I have replaced one of my Azure Functions that ran in a container, with a Job. First a little background. The Application is a demo application that I use to show Azure Container Apps at conferences. The app is a polling app where you can group one or more polls in a session. The audience can vote (multiple choice) on these polls. This app is pretty over-engineered for the purpose of the demo. There are three (micro) services.

One Users service to be able to identify individual users in the audience so when they button bash votes, it will actually change their original vote instead of creating new ones.

The second is the Sessions service to group one or more poll questions. Users can join sessions and then start voting on the active poll.

The Polls service allows me to define questions and possible answer options, and push them using a real-time mechanism (Azure Web PubSub) to the devices in the audience.

The Votes service handles incoming votes. It will publish a message per vote on a service bus. One Azure Function handles this vote message and stores the result in a Storage Account Table. It then publishes another message to a different queue, that will trigger a calculation to draw a pie chart in a second Azure Function.

From function to job

I removed the first Azure Function, from the Votes service, to handle the incoming vote, store it and publish a message to a different Service Bus queue. This function is not transformed in an Azure Container Apps Job that looks like so:

using Azure.Messaging.ServiceBus;
using Azure;
using System.Diagnostics;
using System.Text;
using Azure.Data.Tables;
using Newtonsoft.Json;
using PollStar.Core.ExtensionMethods;
using PollStar.Votes.Abstractions.Commands;
using PollStar.Votes.Abstractions.DataTransferObjects;
using PollStar.Votes.Repositories.Entities;

const string sourceQueueName = "votes";
const string targetQueueName = "charts";
const string storageTableName = "votes";

async static Task Main()
{
    Console.WriteLine("Starting the process job");

    var serviceBusConnectionString = Environment.GetEnvironmentVariable("ServiceBusConnection");
    var storageAccountConnection = Environment.GetEnvironmentVariable("StorageAccountConnection");

    var serviceBusClient = new ServiceBusClient(serviceBusConnectionString);
    var receiver = serviceBusClient.CreateReceiver(sourceQueueName);

    Console.WriteLine("Receiving message from service bus");
    var receivedMessage = await receiver.ReceiveMessageAsync();

    if (receivedMessage != null)
    {
        Console.WriteLine("Got a message from the service bus");
        var payloadString = Encoding.UTF8.GetString(receivedMessage.Body);
        var payload = JsonConvert.DeserializeObject<CastVoteDto>(payloadString);
        if (payload != null)
        {
            Console.WriteLine("Deserialized to a descent payload");

            Activity.Current?.AddTag("PollId", payload.PollId.ToString());
            Activity.Current?.AddTag("UserId", payload.UserId.ToString());

            var voteEntity = new VoteTableEntity
            {
                PartitionKey = payload.PollId.ToString(),
                RowKey = payload.UserId.ToString(),
                OptionId = payload.OptionId.ToString(),
                Timestamp = DateTimeOffset.UtcNow,
                ETag = ETag.All
            };

            Console.WriteLine("Created entity instance");
            var client = new TableClient(storageAccountConnection, storageTableName);
            Console.WriteLine("Saving entity in table storage");
            await client.UpsertEntityAsync(voteEntity);

            var calcCommand = new ChartCalculationCommand
            {
                PollId = payload.PollId,
                SessionId = payload.SessionId
            }.ToServiceBusMessage();
            Console.WriteLine("Constructed service bus command");


            Console.WriteLine("Sending message to chart calculation queue");
            var sender = serviceBusClient.CreateSender(targetQueueName);
            await sender.SendMessageAsync(calcCommand);
            Console.WriteLine("Completing original message in service bus");
            await receiver.CompleteMessageAsync(receivedMessage);
            Console.WriteLine("All good, process complete");
        }
        else
        {
            Console.WriteLine("No service bus message received, terminating container");
        }
    }
}

await Main();

You can see the code is pretty straightforward. It uses the connection strings of a Service Bus instance and a Storage Account, to fetch messages from a queue, publish messages to a (different queue) and store the vote result in Table Storage. I think I can do some work to organize the code a little better, but for now, it will do.

The project is a Console App. This console app is published as a container to my Container Registry.

Adding the job

The job appears as a separate resource in your Azure Portal. And to create it, I used a magical Azure CLI command.

az containerapp job create \
    --name "process-incoming-votes" \
    --resource-group "pollstar-votes-api-prod-neu" \
    --trigger-type "Event" \
    --replica-timeout "60" \
    --replica-retry-limit "1" \
    --replica-completion-count "1" \
    --parallelism "1" \
    --min-executions "0" \
    --max-executions "10" \
    --polling-interval "30" \
    --image "container-registry-name.azurecr.io/pollstar-votes-job:1.0.17" \
    --cpu "0.5" \
    --memory "1Gi" \
    --secrets "servicebus-connection-string=sb-connstr"
              "storage-account-connection-string=sa-connstr" \
    --env-vars "ServiceBusConnection=secretref:service-bus-connection-string"
               "StorageAccountConnection=secretref:storage-account-connection-string" \
    --environment "resource-id-of-container-app-environment" \
    --registry-server "container-registry-name.azurecr.io" \
    --registry-username "container-registry-name" \
    --registry-password "container-registry-password" \
    --scale-rule-name "azure-servicebus-queue-rule" \
    --scale-rule-type "azure-servicebus" \
    --scale-rule-metadata "queueName=votes"
                          "namespace=pollstar-int-prod-neu-bus"
                          "messageCount=5" \
    --scale-rule-auth "connection=servicebus-connection-string"

The Azure Container Apps Job in the Azure Portal

So yeah, that is quite a command. Let’s go through it step by step. First, the command itself, az containerapp job create obviously creates an Azure Container Apps Job.

Paramater	Purpose
name	The name of the Container Apps Job resource as it will appear in the Azure Portal
resource-group	The name of the resource group where to deploy the Job in
trigger-type	The type of trigger that fires your Job
replica-timeout	Timeout for each replica created
replica-retry-limit	The max amount of retries after a job execution fails
replica-completion-count	The number of replicas to complete successfully before a job execution is considered successful
parallelism	The number of replicas to start per job
min-executions	The minimum number of job executions to run per polling interval
max-executions	The maximum number of job executions to run per polling interval
polling-interval	The polling interval at which to evaluate the scale rule
image	Container registry, container image name, and version of the container image to pull
cpu	Amount of compute power reserved for a job instance
memory	Amount of memory reserved for a job instance
secrets	Key / Value pair that allows you to store secrets in a fairly safe way
env-vars	Allows you to set environment variables for your container. Easy for configuration values
environment	The full Azure Resource ID of your Azure Container Apps Environment
registry-server	Name of your container registry
registry-username	Username to log in to your container registry
registry-password	Password to log in to your container registry
scale-rule-name	Name of the rule that defines scaling
scale-rule-type	Type of scaling, in my case an Azure Service Bus queue
scale-rule-metadata	Additional information that my scale rule needs to scale properly
scale-rule-auth	Authorization information, in my case to allow the scale rule to connect to the Azure Service Bus

Time to test my application. As said, I remove the Azure Function from my functions app and verified that messages on the Service Bus Queue were not processed.

Then I published the Container Apps job by executing the command above and to my surprise, messages were being processed and I could see a pie chart changing.

The Azure Container Apps Job in the Azure Portal

When I opened the properties of my Container Apps Job in the Azure Portal, I saw some executions in the Execution history, that all succeeded. Now, to be honest, this wasn’t exactly my first shot. I had some struggles getting the az CLI command right and obviously the app itself was not perfect at the beginning. But I think it’s fair to say I’m pretty surprised how easy it is to create a job and I am eager to see how this service will mature.

The complete code repository for the votes service is on GitHub. Feel free to peek around here and there and grab some stuff you need.