Consuming message queues using .net core background workers – part 3: the code, finally
Tags: rabbitmq, dotnet, microservices, architecture
Published: 2019-11-06 12:42:33

<p>And here we go with the last part of this Series! This time we’re going to take a look at the code and see how a microservice can consume messages from a queue using a background worker.</p>



<p><a rel="noreferrer noopener" aria-label="Last time (opens in a new tab)" href="https://www.davideguida.com/consuming-message-queues-using-net-core-background-workers-part-2-background-workers/" target="_blank">Last time</a> I introduced <strong>Background Workers</strong> and some possible use cases. The possibilities are limitless and today we’re going to see how we can integrate this idea in a microservice.</p>



<p>As usual, all the code is<a rel="noreferrer noopener" aria-label=" available on GitHub (opens in a new tab)" href="https://github.com/mizrael/WebApiWithBackgroundWorker" target="_blank"> available on GitHub</a>, so feel free to take a look.</p>



<p>We  can find 3 projects in the solution:</p>



<ol><li><strong>WebApiWithBackgroundWorker.Common</strong> contains the common code to open a persistent connection (more on this later) and the <a rel="noreferrer noopener" aria-label="Message  (opens in a new tab)" href="https://github.com/mizrael/WebApiWithBackgroundWorker/blob/master/WebApiWithBackgroundWorker.Common/Messaging/Message.cs" target="_blank">Message </a>class. This one is just a simple DTO shared between the publisher and the subscriber and represents the “contract” of what will be sent over the network.</li><li><strong>WebApiWithBackgroundWorker.Publisher</strong> is the message producer. It opens a connection to RabbitMQ and sends messages.</li><li><strong>WebApiWithBackgroundWorker.Subscriber</strong> is our subscribing microservice. It hosts the background worker and exposes a single endpoint to display all the received messages.</li></ol>



<p>Let’s get into the details now.</p>



<p>The <a href="https://github.com/mizrael/WebApiWithBackgroundWorker/blob/master/WebApiWithBackgroundWorker.Common/Messaging/RabbitPersistentConnection.cs">RabbitPersistentConnection </a>class is responsible of holding our TCP connection to RabbitMQ and generating channels. It <a rel="noreferrer noopener" aria-label="is recommended (opens in a new tab)" href="https://www.cloudamqp.com/blog/2017-12-29-part1-rabbitmq-best-practice.html#connections-and-channels" target="_blank">is recommended</a> to have a single connection per application and multiple channels, one per thread. Channels are not thread-safe, so sharing them is not a great idea.</p>



<p>Once a connection is acquired, we register to the <strong>ConnectionShutdown</strong>, <strong>CallbackException </strong>and <strong>ConnectionBlocked </strong>events and try to connect again if something bad occurs.</p>



<h4>Beware that this is just example code: I have left out some security checks and exception handling so don’t use this on production.</h4>



<p>The <a href="https://github.com/mizrael/WebApiWithBackgroundWorker/blob/master/WebApiWithBackgroundWorker.Publisher/RabbitPublisher.cs">RabbitPublisher </a>class is the core of the Publisher project. Nothing particular fancy here: it gets a channel from the connection and uses it to send messages to an exchange. We’re using a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.rabbitmq.com/tutorials/amqp-concepts.html#exchange-fanout" target="_blank">Fanout Exchange</a> in this example so all the subscribers will receive the messages, discarding the routing key.</p>



<p>Let’s dig into the subscriber now.</p>



<p>The microservice exposes a single GET endpoint, /<strong>messages</strong> , that returns a list of all the received messages. The <a rel="noreferrer noopener" aria-label="API Controller (opens in a new tab)" href="https://github.com/mizrael/WebApiWithBackgroundWorker/blob/master/WebApiWithBackgroundWorker.Subscriber/Controllers/MessagesController.cs" target="_blank">API Controller</a> has a single dependency on the Message Repository. Easy-peasy.</p>



<p>For the sake of the example, the persistence is handled in-memory.</p>



<p>The same repository is also used by our <a href="https://github.com/mizrael/WebApiWithBackgroundWorker/blob/master/WebApiWithBackgroundWorker.Subscriber/Messaging/BackgroundSubscriberWorker.cs">BackgroundSubscriberWorker</a>. This one is registered in the ConfigureServices() method of the Startup class with this call:</p>



<pre class="wp-block-preformatted">services.AddHostedService&lt;BackgroundSubscriberWorker&gt;();</pre>



<p>That’s all we need to host a background worker. Well actually that’s not entirely true: as you may have noticed, the worker class inherits from BackgroundService which helps us encapsulating the logic and handling the lifetime. There’s a good article on <a href="https://docs.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services?view=aspnetcore-3.0&amp;tabs=visual-studio" target="_blank" rel="noreferrer noopener" aria-label=" (opens in a new tab)">the Microsoft website</a>, make sure to check it out.</p>



<p>Our worker class is responsible of starting the <a rel="noreferrer noopener" aria-label="RabbitSubscriber (opens in a new tab)" href="https://github.com/mizrael/WebApiWithBackgroundWorker/blob/master/WebApiWithBackgroundWorker.Subscriber/Messaging/RabbitSubscriber.cs" target="_blank">RabbitSubscriber</a> and listening to incoming messages. When one arrives, it calls the Add() method on the repository and that’s it.</p>



<p>As you may have noticed, the connection string is missing from the projects. I am using <a href="https://www.cloudamqp.com/" target="_blank" rel="noreferrer noopener" aria-label="CloudAMQP  (opens in a new tab)">CloudAMQP </a>to host RabbitMQ, they have a nice free tier, extremely useful for experiments like this one.</p>



<p>This concludes our Series about Queues and Background Workers. It took me a little bit more than expected to write this last article, I got distracted by other topics and also by everything else is happening to me these weeks. I’m about to move to Canada, packing an entire house is never easy.</p>



<p>Don’t miss <a rel="noreferrer noopener" aria-label="Part 1 (opens in a new tab)" href="https://www.davideguida.com/consuming-message-queues-using-net-core-background-workers-part-1-message-queues/" target="_blank">Part 1</a> and <a href="https://www.davideguida.com/consuming-message-queues-using-net-core-background-workers-part-2-background-workers/" target="_blank" rel="noreferrer noopener" aria-label="Part 2 (opens in a new tab)">Part 2</a>!</p>