Distributed Software With Service Bus

Today, everyone is moving to the cloud with their software system. Personally I’m pretty much fan of Microsoft Azure. My job is to support companies migrating software systems to the cloud. What I see, is that a lot of companies and developers don’t really know how cloud solutions work and how you can make them work for you.

One system in Microsoft Azure is the Service Bus. It’s a messaging system that is designed for software systems, or software components to communicate with each other.

Now when you have a ASP.NET website running somewhere in a data center of choice and you want to move to the cloud (Azure), you simply create a Web App and just host the website as is. However, when your system is getting more and more load, you need to scale (up or out), which is fairly expensive. You can save lot of money by investigating why your system demands these resources and why scaling up or out is a requirement.

Often, there is just one single part of the website demanding these resources, while all the other parts are running just fine. A bank for example, the services for creating a new account, changing an address, or request a new debit card demand way less resources then for example the transactions service allowing money transfers. In such a case, it could be valuable to try and get the pressure off the transactions service, by distributing the work load. The Service Bus is an excellent native cloud service that will definitely help you and I’m going to explain how.

The basics of the Service Bus

So what is this Service Bus thing? Well, basically a very simple messaging mechanism. It contains Queues and Topics. The difference is that a Queue is, like it’s name assumes, a queue of messages. Each message will be delivered only once to any system reading from that queue. For example, when you make a bank transfer, you want that transfers to take place only once. So when multiple systems read from the queue, and a new message arrives, only one of those systems will receive the message. A topic can be compared to a newspaper, or your favorite magazine. Whoever has a subscription, gets the message as soon as it comes out. So if multiple systems have a subscription, to a certain message, the message will be delivered multiple times.

A tiny side-step to Microservices

In case you’re developing Microservices, you may need a messaging system to make sure you meet the eventual consistency requirement. Only one microservice will be responsible of manipulating a certain entity, but more services may need to receive an update of the changed entity. The Service Bus would be an excellent solution here, because you can easily broadcast the updated entity through a topic. All services that may need this update can subscribe to that certain message.

A practical example

So here we go, an example that makes sense. Let’s take the bank example, having a transactions service that demands a lot of resources because it’s drawing a lot of traffic, and a lot of validations are going on during each request. Therefore a good subject for change.

[HttpPost]
public async Task Post([FromBody] CreateTransactionDto dto)
{
    if (ModelState.IsValid)
    {
        dto.TransactionOn = DateTimeOffset.UtcNow;
        var messageBody = JsonConvert.SerializeObject(dto);
        var message = new Message(Encoding.UTF8.GetBytes(messageBody));
        await _queueClient.SendAsync(message);
        return Accepted(dto);
    }
    return BadRequest();
}

In the previous block of code, I removed all validations and ‘heavy’ stuff that demand a lot of resources. Usually when you create a bank transaction, a large amount of validations are required to make sure the transaction can actually take place. The only validation done here, is the ModelState validation. Next thing is creating a Service Bus message which is sent to a queue client. In this example I return an excepted HTTP response to indicate that I ‘accepted the request of creating a bank transaction’. The process of creating a bank transaction is now officially distributed, YESSSS!

Now, handling the message

Now I need a mechanism that handles the queue message and will actually create the bank transaction for me. I decided to create an Azure Function, because they’re fast, cheap and scale like a maniac. So this solution not only takes the pressure off the old web solution, but is also distributed in a system that behaves depending on the load and is thus pretty future proof.

[FunctionName("CreateTransaction")]
public static async void CreateTransaction(
    [ServiceBusTrigger("transactions", Connection = "AzureServiceBus")] string message,
    [ServiceBus("bank", Connection = "AzureServiceBus", EntityType = EntityType.Topic)] IAsyncCollector serviceBusTopic,
    [Table("transactions")] IAsyncCollector table,
    ILogger log)
{

    var transaction = JsonConvert.DeserializeObject(message);
    if (transaction != null)
    {
        if (transaction.Amount > 100)
        {
            var integrationEvent = new TransactionCreateFailedIntegrationEvent
            {
                Amount = transaction.Amount,
                FromAccountName = transaction.FromAccountHolder,
                ToAccountName = transaction.ToAccountHolder,
                Reason = "Maximum transaction amount is 100"
            };
            await SendServicebusMessage(integrationEvent, serviceBusTopic);
        }
        else
        {
            var transactionEntity = new TransactionEntity
            {
                PartitionKey = "transaction",
                RowKey = Guid.NewGuid().ToString(),
                FromAccountNumber = transaction.FromAccountNumber,
                FromAccountHolder = transaction.FromAccountHolder,
                ToAccountNumber = transaction.ToAccountNumber,
                ToAccountHolder = transaction.ToAccountHolder,
                Amount = transaction.Amount,
                Description = transaction.Description,
                TransactionOn = transaction.TransactionOn,
                Timestamp = DateTimeOffset.UtcNow
            };
            await table.AddAsync(transactionEntity);
            var integrationEvent = new TransactionCreatedIntegrationEvent
            {
                TransactionId = Guid.Parse( transactionEntity.RowKey),
                FromAccountName= transaction.FromAccountHolder,
                ToAccountName= transaction.ToAccountHolder,
                NewBalance = 3581.53M
            };
            await SendServicebusMessage(integrationEvent, serviceBusTopic);
        }
        await serviceBusTopic.FlushAsync();
    }
}

I know, it’s a large method which may need some refactoring in a production environment (or not), but for this demo works pretty fine. You can see I use the Service Bus Queue Trigger to fire the Azure Function. This way, each and every transaction is executed only once, by an instance of the Azure Function. I implemented a validation rule for demo purpose. The amount of the bank transaction cannot be greater than 100. If the transaction meets this validation rule it will be stored in table storage. When the validation fails, or succeeds, I create an integration event which will be sent to a Service Bus Topic. This mechanism allows me to notify the user what actually happened with the create bank transaction request.

Oh and by the way, the SendServiceBusMessage() function looks like this:

private static async Task SendServicebusMessage(T message, IAsyncCollector serviceBusTopic)
{
    var eventName = message.GetType().Name.Replace(IntegrationEventSufix, "");
    var jsonMessage = JsonConvert.SerializeObject(message);
    var body = Encoding.UTF8.GetBytes(jsonMessage);

    var serviceBusErrorMessage = new Message
    {
        MessageId = Guid.NewGuid().ToString(),
        Body = body,
        Label = eventName,
    };
    await serviceBusTopic.AddAsync(serviceBusErrorMessage);
}

Finally, pushing the outcome to the client

I created a Service Bus Topic Subscription in the ASP project that allows me to notify the user what happened with the ‘create bank transaction’ request. For the subscription on the Service Bus topic, I used some helper methods from the eShopOnContainers project. I removed the RabbitMQ stuff leaving me with only a Service Bus connection and the ability to subscribe to certain messages.

I also added SignalR to my project and created a hub so I’m able to send the confirmation message and/or error message to the client (web browser). Then I added a handler for both the error message and the confirmation message. The handlers create an instance of the SignalR Hub and invoke the corresponding method on that SignalR hub.

public class TransactionCreatedIntegrationEventHandler : IIntegrationEventHandler
{
    private readonly IHubContext _hubContext;
    public async Task Handle(TransactionCreatedIntegrationEvent @event)
    {
        var hub = new TransactionsHub(_hubContext);
        await hub.TransactionCreated(new TransactionCreatedDto
        {
            ToAccountName = @event.ToAccountName,
            FromAccountName = @event.FromAccountName,
            NewBalance = @event.NewBalance,
            TransactionId = @event.TransactionId
        });
    }
    public TransactionCreatedIntegrationEventHandler(IHubContext hubContext)
    {
        _hubContext = hubContext;
    }
}

Pretty awesome right? The full demo source code it available on my GitHub page. I added an Angular client which enables you to post your transactions to the backend. The readme file of the project will explain how to get the project running.

Let me know what you thing in the comments below!