Redis Streams + NestJS: Part 1 | Setup
Tags: redis, node, typescript, tutorial
Published: 2023-02-22 16:58:39

## Intro

This is part 1 of a 3-part series, where we will explore how to use Redis streams with NestJS.

It is structured in 3 parts:
- Part 1 - Setting up the NestJS application and connecting to Redis
- [Part 2](https://dev.to/magickriss/redis-streams-nestjs-part-2-reading-from-stream-1643) -Populating Redis streams and reading from in **fan-out mode**
- [Part 3](https://dev.to/magickriss/redis-streams-nestjs-part-3-consumer-groups-3am7) - Using **consumer groups** to handle one stream from multiple actors in a way that one message is sent to and processed only by a single actor (consumer)

By the end of this series, you will have the knowledge and tools necessary to create your own NestJS app that utilizes Redis streams to handle real-time data.

Full code is available on [the github](https://github.com/MagicKriss/Redis-Streams-NestJS-Example)

## Assumed prior knowledge
Before diving into this post, it is assumed that you have a basic understanding of **NestJS** and **Redis**.
Familiarity with **JavaScript generators** is also recommended, as they will be used in the examples to listen to the Redis stream continuously.
We will also use **Docker** and **Docker Compose** to create and run our Redis server and application.
NestJS is a typescript framework, so you should be familiar with it too.
To familiarize yourself with these things, I recommend:

- [NestJS docs](https://docs.nestjs.com/)
- [TypeScript](https://www.typescriptlang.org/)
- Redis:
  - [Redis streams intro](https://redis.io/docs/data-types/streams/)
  - [The tutorial](https://redis.io/docs/data-types/streams-tutorial/)
- Generators:
  - [JavaScript Generators reference](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Generator)
  - Dr. Axel Rauschmayer's book's "JavaScript for impatient programmers" sections on [synchronous](https://exploringjs.com/impatient-js/ch_sync-generators.html) and [asynchronous ](https://exploringjs.com/impatient-js/ch_async-iteration.html#async-generators) generators (available free online)
- Docker:
  - [Docker](https://www.docker.com/get-started/)
  - [Docker Compose](https://docs.docker.com/get-started/08_using_compose/)

  

### Side note

> It's also important to note that while this post explicitly covers the integration of Redis streams with NestJS, the concepts and techniques discussed here can also be applied to other frameworks and languages. Redis streams is a versatile tool that can be used in a wide range of applications and architectures. Therefore, even if you are not using NestJS, the general concepts and approaches outlined in this post can still apply to your projects and stack.


## Setup
### Creating a new NestJS application

To get started, we will need to create a new NestJS application. This can be done using the NestJS CLI, which can be installed via npm by running `npm i -g @nestjs/cli`. Once installed, you can create a new application by running `nest new redis-streams`, where "redis-streams" is the name of your application.
This will create a new directory with the same name, containing the basic structure of a NestJS application.
### Docker setup
To simplify things, we are going to use `docker-compose.yml` file to include both our app and Redis server:
```yaml
# docker-compose.yml

version: '3'

services:
  app:
    container_name: app
    image: node:18
    environment:
      HTTPS_METHOD: noredirect
    ports:
      - 8081:3000
    volumes:
      - ./:/usr/src/app/
    working_dir: /usr/src/app
    command: npm run start:dev

  redis:
    container_name: redis
    image: redis:7
    restart: always
```

Now you can run `docker-compose up -d` to build and run your services. To see console output, use `docker-compose logs`
You should see the following message:
```
 LOG [NestApplication] Nest application successfully started +7ms
```
## Connecting to Redis
### Setting up `node-redis` client library

To call Redis from we are going to use the official NodeJS Redis client library [node-redis](https://github.com/redis/node-redis)
```
$ npm i redis
```
There are also other libraries, e.g. `ioredis` is noteworthy alternative. You can see the list of clients [on the Redis website](https://redis.io/resources/clients/#nodejs)

#### Redis module

Finally, we can start working with Redis from our application.
First, will create a new module for Redis-related services.
```
$ nest g module redis
```
You should see this: 
``` typescript
// redis.module.ts

import { Module } from '@nestjs/common';

@Module({
  providers: [],
  exports: [],
})
export class RedisModule {}
```
#### RedisClient factory
To use `RedisClient` from `node-redis` library we are going to create a factory provider for it:
``` typescript
// redis-client.factory.ts

import { FactoryProvider } from '@nestjs/common';
import { createClient } from 'redis';
import { RedisClient, REDIS_CLIENT } from './redis-client.type';

export const redisClientFactory: FactoryProvider<Promise<RedisClient>> = {
  provide: REDIS_CLIENT,
  useFactory: async () => {
    const client = createClient({ url: 'redis://redis:6379/0' });
    await client.connect();
    return client;
  },
};
```
Let's break it down:
We create a `FactoryProvider` that will call async function provided in `useFactory`:
``` typescript
// redis-client.factory.ts
// --snip--
  useFactory: async () => {
    const client = createClient({ url: 'redis://redis:6379/0' });
    await client.connect();
    return client;
  },
// --snip--
```
1. We call `createClient` function from `redis` library, and pass it the URL consisting of `{protocol}://{host}:{port}/{database}`, where:
  - protocol= `redis`
  - host = `redis` - this is specified in `docker-compose.yml` with `container_host: redis`. Usually, you would create an environment variable with your Redis instance IP and use it here.
  - port = `6379` - default Redis port
  - database = `0` default database
2. We connect to the Redis server `await client.connect();`
3. Return the created & connected client.

You may have noticed that we did not provide an instance of `RedisClient` type but `REDIS_CLIENT`, which is our injection token. Also, `RedisClient` is our custom type, not `redis`.
This is due to `node-redis` on v4 not exporting the `RedisClient` type, so we need to create our own in `/redis-client.type.ts` file:
``` typescript
export type RedisClient = ReturnType<typeof createClient>;
export const REDIS_CLIENT = Symbol('REDIS_CLIENT');
```

All that is left is to add this to our module:

``` typescript
// redis.module.ts

//  --snip--
@Module({
  providers: [redisClientFactory],
})
export class RedisModule {}
```

#### Creating RedisService

Next, let's add a `RedisService` that will create a layer of abstraction from `RedisClient`.
```
$ nest g service redis
```
In there we will inject our `RedisClient`
``` typescript
// redis.service.ts

// --snip--
@Injectable()
export class RedisService implements OnModuleDestroy {
  public constructor(
    @Inject(REDIS_CLIENT) private readonly redis: RedisClient,
  ) {}

  onModuleDestroy() {
    this.redis.quit();
  }
}
```
Not to leave hanging connections, we are going to close the connection to the Redis by calling `this.redis.quit()` on `OnModuleDestroy` lifecycle event.

#### Ping for Redis server

To check that we have successfully connected to Redis, let's add an API endpoint that calls call Redis `ping`
``` typescript
// redis.service.ts

  ping() {
    return this.redis.ping();
  }
```

Let's export RedisService so that we can use it in other modules:
``` typescript
// redis.module.ts

// --snip--
  exports: [RedisService],
// --snip--
```
Now we will import it into `AppService` and pass through our call to ping redis:
``` typescript
// app.service.ts

@Injectable()
export class AppService {
// --snip--
  constructor(
    private readonly redisService: RedisService,
  ) {}

  redisPing() {
    return this.redisService.ping();
  }
// --snip--
```

Finally, we can add a new endpoint to AppController that will execute a ping to the Redis server and send the response to the user:
``` typescript
// app.controller.ts

// --snip--

@Controller()
export class AppController {
  constructor(private readonly appService: AppService) {}

  @Get('redis-ping')
  redisPing() {
    return this.appService.redisPing();
  }
}
```

Now, with our app and Redis server running, we can open our `/redis-ping` endpoint [http://localhost:8081/redis-ping](http://localhost:8081/redis-ping), and you should see the response:


![Pong message displayed in browser](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ih3bt59mzkezab654ki2.png)

Congratulations! We have finished part 1 of the 3-part series and created a NestJS application with a working connection to our Redis server! In [part 2](https://dev.to/magickriss/redis-streams-nestjs-part-2-reading-from-stream-1643) we are going to create, populate and read from Redis streams.