Yggdrasil and RabbitMQ Subscriptions
Tags: elixir, pubsub, rabbitmq, tutorial
Published: 2020-03-05 21:39:47

---
cover_image: "https://images.unsplash.com/photo-1583301286816-f4f05e1e8b25"
canonical_url: "https://thebroken.link/yggdrasil-easy-pub-sub-in-elixir/#III"
series: "Yggdrasil: Easy Pub-Sub in Elixir"
---

One of the features I really like about RabbitMQ is its queue routing. Its flexibility allows you to do interesting things without much of a hassle. But before I dig deep into RabbitMQ's routing capabilities, I would like to mention some concepts.

### Connections and Channels

RabbitMQ uses not only **connections**, but virtual connections called **channels**. The idea of channels is to introduce multiplexing in a single connection. A small system could establish only one connection with RabbitMQ while opening a channel for every single execution thread e.g:

![RabbitMQ connection multiplexing](http://thebroken.link/assets/img/yggdrasil-easy-pub-sub-in-elixir/rabbitmq-connection.png)

The rule of thumb would be to use:

- One connection per application.
- One channel per process in the application.

> **Note**: Once our connection starts to be overloaded, we can start adding more connections to our connection pool.

With a normal RabbitMQ setup, we need to deal with:

- **Connection pools**: avoiding over consuming resources.
- **Channel cleaning**: avoiding channel memory leaks when they are not closed properly.
- **Fault-tolerant connections**: supporting re-connections in case of failure or disconnection.
- **Re-connection back-off time**: avoiding overloading the database on multiple re-connections.

### Exchanges and Queues

An **exchange** is a message router. Every **queue** attached to it will be identified by a **routing key**. Typically, routing keys are words separated by dots e.g. `spain.barcelona.gracia`.

Additionally, routing keys support wildcards, for example: `spain.barcelona.*` will match messages with routing keys like `spain.barcelona.gracia` and `spain.barcelona.raval`.

It's easier to see these concepts with an image example:


![RabbitMQ message routing](http://thebroken.link/assets/img/yggdrasil-easy-pub-sub-in-elixir/rabbitmq-exchange.png)

In the previous image:

- **__Publisher X__** and **__Publisher Y__** are sending messages to **_Exchange logs_**.
- **__Subscriber A__** is subscribed to `logs.*`.
- **__Subscriber B__** is subscribed to `logs.error`.

Then:

- **__Publisher X__** message will end up in **_Queue_** `logs.info`.
- **__Publisher Y__** message will end up in **_Queue_** `logs.error`.
- **__Subscriber A__** will receive **__Publisher X__** and **__Publisher Y__**'s messages.
- **__Subscriber B__** will receive **__Publisher Y__**'s message.

![](https://media.giphy.com/media/3o6gDSdED1B5wjC2Gc/giphy.gif)


### Handling Subscriptions in Yggdrasil

Handling RabbitMQ's complexity might be intimidating. Fortunately, [Yggdrasil for RabbitMQ](https://github.com/gmtprime/yggdrasil_rabbitmq) generalizes the complexity in order to have a simpler API.

The biggest difference with previous adapters is the channel name. Instead of being a string, it's a tuple with the exchange name and the routing key e.g:

A subscriber would connect to the exchange `amq.topic` using the routing key `logs.*` as follows:

```elixir
iex(subscriber)> Yggdrasil.subscribe(name: {"amq.topic", "logs.*"}, adapter: :rabbitmq)
iex(subscriber)> flush()
{:Y_CONNECTED, %Yggdrasil.Channel{...}}
```

> **Note**: The exchange must exist and its type should be `topic`. The exchange `amq.topic` is created by default in RabbitMQ.

Then a publisher could send a message to the exchange `amq.topic` using `logs.info` as routing key:

```elixir
iex(publisher)> Yggdrasil.publish([name: {"amq.topic", "logs.info"}, adapter: :rabbitmq], "Some message")
:ok
```

Finally, the subscriber would receive the message:

```elixir
iex(subscriber)> flush()
{:Y_EVENT, %Yggdrasil.Channel{...}, "Some message"}
```

Additionally, the subscriber can be written using the `Yggdrasil` behaviour:

```elixir
defmodule Logs.Subscriber do
  use Yggdrasil

  def start_link(options \\ []) do
    channel = [
      name: {"amq.topic", "logs.*"},
      adapter: :rabbitmq
    ]

    Yggdrasil.start_link(__MODULE__, [channel], options)
  end

  @impl true
  def handle_event(_channel, message, _state) do
    ... handle event ...
    {:ok, nil}
  end
end
```

### Lost Messages

Yggdrasil will acknowledge the messages as soon as they arrive to the adapter, then it will broadcast them to all the subscribers. If the adapter is alive while the subscribers are restarting/failing, some messages might be lost.

Though it's possible to overcome this problem with exclusive queues, this feature is not implemented yet.

### Conclusion

[Yggdrasil for RabbitMQ](https://github.com/gmtprime/yggdrasil_rabbitmq) handles RabbitMQ complexity let's you focus in what really matters: **messages**.

_Cover image by [Aswathy N](https://unsplash.com/@abnair)_