Messaging and Madness: Sending Messages with AMQP and Amazon MQ
Tags: python, aws, amqp, amazonmq
Published: 2021-10-30 17:23:47

*Originally posted on [my personal blog](https://georgeoffley.com/blog/messaging-and-madness-sending-messages-with-amqp-and-amazon-mq.html)*

## Table Of Contents
* [Introduction](#introduction)
* [AMQP](#amqp)
* [AMQP and Amazon MQ](#amqp_amazon_mq)
* [Serialization](#serialization)
* [Conclusion](#conclusion)

### Introduction <a name="introduction"></a>

How do software systems talk to each other? Back-end systems can scale into giant melted together [Cronenberg monsters](https://youtu.be/NH-8L1iZq20), often making up different tools and languages. So, communicating between these services can become an untenable challenge without some shared vocabulary. We can communicate in many ways, but today I wanted to talk about asynchronous messaging protocols and figure out how AWS can help.

<!--more-->

### AMQP <a name="amqp"></a>

AMQP stands for Advanced Message Queuing Protocol. I’ve been working to implement it for some back-end software suites I’m building out to enable them to talk to each other. AMQP utilizes these things called *brokers* to publish messages on, then on the other end, a receiving service subscribed to the same “channel” that we posted to can pick up that message. 

![Hello World Visualization](https://georgeoffley-blog-images.s3.amazonaws.com/2021-10-30-messaging-and-madness/hello-world-example-routing.png)
via [Rabbit MQ Tutorials](https://www.rabbitmq.com/tutorials/amqp-concepts.html)

Let’s dive a little further down; the *publisher* service publishes a message to an *exchange* on a *broker*. This exchange has *routes* that lead to *queues*, or “channels,” where the payload is published. We make sure to include the sending information with our message to be routed to the correct queue. The broker cannot see the message, although it might look into any metadata attached to the message from the publisher. This workflow asynchronously sends messages. Imagine a server version of a mail sorting machine shooting letters into the correct mail slot based on the address.

![Mail Sorting Gif from MIB II](https://georgeoffley-blog-images.s3.amazonaws.com/2021-10-30-messaging-and-madness/mail_sorting.gif)

When referring to a publisher, I mean some code that we utilize to connect and send a message. AMQP is programmable, so I can shape it to fit most situations. In this case, we need to send messages to our different software suites to trigger actions to happen. Learning this took some time, but it’s been simple to implement. 

There are different types of exchanges that we can use to make these services fit our needs. I’m going to explain what we use briefly.

We use a *direct exchange* utilizing *routing keys* to bind queues to exchanges. Our code can use direct exchanges to distribute tasks to many different endpoints, but we used these direct exchanges to make direct routes between our services. Other types of exchanges can be used to broadcast messages. More information can be found [here](https://www.rabbitmq.com/tutorials/amqp-concepts.html). For now, we’re going to focus on direct exchanges. 

### AMQP and Amazon MQ <a name="amqp_amazon_mq"></a>

We touched on all that because I wanted to talk about [Amazon MQ](https://aws.amazon.com/amazon-mq/?amazon-mq.sort-by=item.additionalFields.postDateTime&amazon-mq.sort-order=desc). Amazon MQ is a fully managed platform for setting up message brokers. Amazon MQ utilizes both RabbitMQ and Apache Active MQ for creating brokers. We’re sticking with Rabbit MQ for the time being.

![Amazon MQ Dashboard](https://georgeoffley-blog-images.s3.amazonaws.com/2021-10-30-messaging-and-madness/amazon_mq_dash.png)

Here above, you can see you can easily set up a broker in just a few clicks. I left most of the settings on default, except for choosing “RabbitMQ” for our broker engine and setting some security up for accessing our management console. 

![Rabbit MQ Management Console](https://georgeoffley-blog-images.s3.amazonaws.com/2021-10-30-messaging-and-madness/rabbit_mq_dash.png)

Once we get that, we have access to the RabbitMQ dashboard Amazon MQ created and is managing. Now that we have a broker set up, we can play with some code. 

![Code](https://georgeoffley-blog-images.s3.amazonaws.com/2021-10-30-messaging-and-madness/code.png)

Above I use the library [Kombu](https://github.com/celery/kombu) to create some connections and send some stuff. I started by setting up our environment variables. Then created exchange and queue objects. Finally, I made our connection object and the producer object, and then we sent a simple “Hello” message.

### Serialization <a name="serialization"></a>

Serialization is [another blog post](https://www.tutorialspoint.com/object_oriented_python/object_oriented_python_serialization.htm), but I chose to use JSON to serialize the payload. In the production software, I use a combination of JSON and [Pickle](https://docs.python.org/3/library/pickle.html) to serialize things like image data.

Now we can see our message published on the queue I declared in our publisher service. An identical receiving service would be set up on the other side to read out messages sent to that queue.

![Results](https://georgeoffley-blog-images.s3.amazonaws.com/2021-10-30-messaging-and-madness/results.png)

### Conclusion <a name="conclusion"></a>

In conclusion, using Amazon MQ allows us to set up managed brokers for us to send messages. With AMQP as the broker engine, we have a lightweight message-sending workflow. Thanks for reading.

-George