Exploring Redis III. Reliable queue.
Tags: redis
Published: 2021-07-03 01:11:32

### Introduction

In this article, I'll be using Redis as a "reliable queue". As the articles before, I will try to keep it simple, is better this way.

## Prerequisites

* Same as previous article.

##  A reliable queue in Redis :thinking:, hugh?

Well yes, it can be done and actually is pretty good. Let's show you a queue first, I won't enter in too much technical detail, because a queue is pretty intuitive. Take a look at this pretty sketch:

![Queue](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jaigcl89re2w1feuos2g.png)

The idea is to have a FIFO(First in, first out) behavior with the queue. But having just this is not enough, because network problems could happen, or one of the parts that consumes from the queue just crashed, what happen with the element that has being processed? 

With Redis we could manage this using **LPUSH** and **BRPOPLPUSH**, you could use also [RPOPLPUSH](https://redis.io/commands/rpoplpush) as the non-blocking alternative. Let's see how to use these two commands. Having a `redis-server`, and two instances of `redis-cli` running,

On the first instance of the `redis-cli`, we push elements into the queue like this:

```bash
LPUSH tasks file1 file2 file3
```

**tasks** is the queue, and **file1, file2 and file3** are the elements that we are pushing into the queue. Until now, we just have a simple **queue**, nothing reliable with this. Let's see how to pop elements from the **queue**.

```bash	
BRPOPLPUSH tasks tasks_backup 10
```

Now, comes the reliable part. In this command we see two new things, one is the **tasks_backup**. The particularity of this command is that it pops an element from the queue **tasks**, but also push that same element to a backup queue, so in case the consumer crash, there's still a copy of the element in the backup queue. This, of course, adds more space consumption, after you have a successful pop you could just destroy this list since you don't need it anymore.

The **10**, represents the timeout in seconds that you'll be waiting for an element in the queue, given that **BRPOPLPUSH** is a blocking operation you may not want to wait forever.

## Example use case

Suppose that you have an application, that needs to download videos, given a request from a user. You could handle the download in the same application, but this is not so smart, because the time you start getting more traffic this could be a significant load for the application alone. Instead of managing the download by the application, you, as a pretty smart engineer, decide to have some **workers** to do all these heavy tasks.

Now, we need a way to communicate between the **application** and the **workers**. You could just establish a direct connection between the application and every single worker, 🤨, no no we want to add more workers as we need without bothering the main application with some nasty configuration. Instead, we are going to use a middleman, a broker, a good guy, we are going to use Redis, because this is an article about Redis of course.

General behavior, of our application, user ask for some **video**, application process request and push this as a task into **Redis**. Then the amazing **workers**, at the other end, pop this tasks, one or more remember we don't have just one user, that would be sad :pensive:.  

![Application flow](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/orll8mx13m8wcda9dpa5.png)

## Conclusion

I hope you get an idea, and a concrete one, of how to make use of Redis in this way. Don't know what to do in the next article, :thinking: mmm...

## Bibliography

1. [RPOPLPUSH command](https://redis.io/commands/RPOPLPUSH)
2. [BRPOPLPUSH command](https://redis.io/commands/brpoplpush)