How to rate limit a login route in Express using node-rate-limiter-flexible and Redis
Tags: node, redis, security, tutorial
Published: 2021-05-31 20:09:37

## Introduction

Rate limiting is a method used for controlling network traffic. It limits the number of actions a user can make per unit of time <sup>[1](https://www.cloudflare.com/learning/bots/what-is-rate-limiting/)</sup>. In this tutorial, we will rate limit a login route to help protect it from [brute force attacks](https://www.kaspersky.com/resource-center/definitions/brute-force-attack). This limits the number of password guesses that can be made by an attacker. We'll use the npm package [node-rate-limiter-flexible](https://www.npmjs.com/package/rate-limiter-flexible) to count and limit the number of login attempts by key. Each key will have a points value that will count the number of failed login attempts. The keys will expire after a set amount of time. The key-value pairs will be stored in [Redis](https://redis.io/), which is an open-source in-memory data structure store. It has many different use cases. We will use it as a simple database. Redis is simple to use and very fast. We'll create an online instance of Redis, connect it to an express application, and then use the Redis command-line interface (redis-cli) to view the database. A prerequisite for this tutorial is an ExpressJS application with a login route and user authentication. 



![Overview of tutorial: Express application with rate limiting on the log in route.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0duyg6kymp7yu4g0jz2i.png)

<figcaption>Express application with rate limiting on the login route. The key for a particular user stores the number of failed logins. If the number of failed logins exceeds the set maximum number of failed logins, then the route will be blocked. The keys expire after a set amount of time.</figcaption>  

&nbsp;

We will use two types of keys to count the number of failed logins. One will be a string made using the user's IP address. The other will be a string made by joining the user's email address and IP address. When a user attempts to log in, if the user exists and the password is not correct, the two keys will be created for the user. 

For example, the keys stored in Redis may look like this after a failed login attempt where the password was incorrect:

**key 1:** `"login_fail_ip-192.168.1.1" : 1`
**key 2:** `"login_fail_username_and_ip-example@example.com_192.168.1.1" : 1` 


---

## Prerequisites

- Express app with login route and login authentication (login with username or email)

- Registered users stored in a database

  

---
&nbsp;
## Set up the rate-limiting middleware

### Middleware used that is not necessary for rate-limiting

This example is from an Express application that uses MongoDB as a database to store the users' data. The following libraries, which will be used in this example, are not necessarily required to set up login rate limiting. 

- [passport](http://www.passportjs.org/) -  authentication middleware
- [util.promisify()](https://nodejs.org/dist/latest-v8.x/docs/api/util.html#util_util_promisify_original) - a method defined in the utilities module of the Node.js standard library. It converts methods that return responses using a callback function to instead return responses in a promise object. The syntax is much cleaner.
- [connect-flash](https://www.npmjs.com/package/connect-flash) - middleware for flash messages notifying a user if the login was successful or not

Submitted data on the `request.body` is parsed as a JSON object by the built-in middleware function in Express: `Express.json()`. The data is stored in JSON format as it is a commonly used, organized, and easily accessible text-based format <sup>[2](https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Objects/JSON)</sup>.  

These were added as application-level middleware in `app.js` using `app.use()`.

### Rate-limiting middleware

The rate-limiting middleware used is a modification of the [node-rate-limiter-flexible](https://www.npmjs.com/package/rate-limiter-flexible) library example of how to protect a [login endpoint](https://github.com/animir/node-rate-limiter-flexible/wiki/Overall-example#login-endpoint-protection).  This rate-limiting middleware is written for an Express application using a Redis store, but the same idea can be applied to rate-limiting middleware with other Node.js frameworks such as Koa, Hapi, and Nest or a pure NodeJS application <sup>[3](https://github.com/animir/node-rate-limiter-flexible/wiki/Overall-example#login-endpoint-protection)</sup>. We'll create 2 rate limiters. The first blocks the login route, for one hour, after 10 consecutive failed login attempts.  The failed login counts are reset after a successful login.  Rate limiting is based on the user's email address and IP address. The second blocks the login route, for one day, after 100 failed login attempts. Rate limiting is based on the user's IP address. After this middleware is set up, we will set up the Redis database.

You can simply rate limit based on IP address only, the problem with this is that IP addresses are not always unique <sup>[4](https://en.wikipedia.org/wiki/Network_address_translation)</sup>. A user in a network that shares a public IP address could block other users in that network. If you limit based on email address only, then a malicious user could block someone's access to the application by simply sending many requests to log in. Blocking by email address and IP address adds some flexibility. A user may be blocked using one IP address but could try login from another device. It is important to note that most devices use a dynamic IP address that changes over time and that IP addresses can be modified <sup>[5](https://support.google.com/fiber/answer/3547208?hl=en</sup>)</sup><sup>, </sup><sup>[6]( https://www.cloudflare.com/learning/ddos/glossary/ip-spoofing )</sup>.  Rate-limiting aims to minimize brute force attacks to guess a user's password. When rate limiting, user experience also needs to be considered. Being too strict by blocking users after only a few attempts is not good for the user experience. You need to make a trade-off between security and user experience.

#### npm packages required for Redis connection and rate-limiting

- [redis](https://www.npmjs.com/package/redis)

- [node-rate-limiter-flexible](https://www.npmjs.com/package/rate-limiter-flexible)

&nbsp;

#### Rate limit controller

Create a file for the rate-limiting middleware. For example, `rateLimitController.js`.

In this [controller](https://developer.mozilla.org/en-US/docs/Learn/Server-side/Express_Nodejs/routes) that will handle the login route POST request, a connection to Redis will be set up. Then a rate limiter instance that counts and limits the number of failed logins by key will be set up. The `storeClient` property of the rate limiter instance will link the rate limiter instance to a Redis database (redisClient) that will be set up later. A points property on the rate limiter instance determines how many login attempts can be made. Keys are created on the instance by using the IP address of the login request or the IP address and email address. When a user fails to log in, points are consumed. This means the count for the key increases. When this count exceeds the points property value, which is the maximum number of failed login attempts allowed, then a message is sent to the user that states that too many login attempts have been made. The keys only exist for a defined amount of time, after this time the rate-limiting is reset. A variable, retrySecs, will be created to determine when a user can try to log in again. The time remaining until another login can be attempted is determined by using the `msBeforeNext()` method on the rate limiter instance. 

If the login route is not being rate-limited, then we will authenticate the user. In this tutorial, [Passport](http://www.passportjs.org/) is used. If the authentication fails and the user's email exists, then a point will be consumed from each rate limiter instance. If authentication is successful the key for the current user, based on IP address and email address, will be deleted and the user will be logged in. A login session is established using the Passport.js method `logIn()`.

```javascript
const redis = require('redis');
const { RateLimiterRedis } = require('rate-limiter-flexible');
const passport = require('passport');

// create a Redis client - connect to Redis (will be done later in this tutorial)
const redisClient = redis.createClient(process.env.REDIS_URL, {
  enable_offline_queue: false
});

// if no connection, an error will be emitted
// handle connection errors
redisClient.on('error', err => {
  console.log(err);
  // this error is handled by an error handling function that will be explained later in this tutorial
  return new Error();
});

const maxWrongAttemptsByIPperDay = 100;
const maxConsecutiveFailsByEmailAndIP = 10; 

// the rate limiter instance counts and limits the number of failed logins by key
const limiterSlowBruteByIP = new RateLimiterRedis({
  storeClient: redisClient,
  keyPrefix: 'login_fail_ip_per_day',
  // maximum number of failed logins allowed. 1 fail = 1 point
  // each failed login consumes a point
  points: maxWrongAttemptsByIPperDay,
  // delete key after 24 hours
  duration: 60 * 60 * 24,
  // number of seconds to block route if consumed points > points
  blockDuration: 60 * 60 * 24 // Block for 1 day, if 100 wrong attempts per day
});

const limiterConsecutiveFailsByEmailAndIP = new RateLimiterRedis({
  storeClient: redisClient,
  keyPrefix: 'login_fail_consecutive_email_and_ip',
  points: maxConsecutiveFailsByEmailAndIP,
  duration: 60 * 60, // Delete key after 1 hour
  blockDuration: 60 * 60 // Block for 1 hour
});

// create key string
const getEmailIPkey = (email, ip) => `${email}_${ip}`;

// rate-limiting middleware controller
exports.loginRouteRateLimit = async (req, res, next) => {
  const ipAddr = req.ip;
  const emailIPkey = getEmailIPkey(req.body.email, ipAddr);
	
  // get keys for attempted login
  const [resEmailAndIP, resSlowByIP] = await Promise.all([
    limiterConsecutiveFailsByEmailAndIP.get(emailIPkey),
    limiterSlowBruteByIP.get(ipAddr)
  ]);

  let retrySecs = 0;
  // Check if IP or email + IP is already blocked
  if (
    resSlowByIP !== null &&
    resSlowByIP.consumedPoints > maxWrongAttemptsByIPperDay
  ) {
    retrySecs = Math.round(resSlowByIP.msBeforeNext / 1000) || 1;
  } else if (
    resEmailAndIP !== null &&
    resEmailAndIP.consumedPoints > maxConsecutiveFailsByEmailAndIP
  ) {
    retrySecs = Math.round(resEmailAndIP.msBeforeNext / 1000) || 1;
  }
   
  // the IP and email + ip are not rate limited  
  if (retrySecs > 0) {
    // sets the response’s HTTP header field
    res.set('Retry-After', String(retrySecs));
    res
      .status(429)
      .send(`Too many requests. Retry after ${retrySecs} seconds.`);
  } else {
    passport.authenticate('local', async function(err, user) {
      if (err) {
        return next(err);
      }
           if (!user) {
        // Consume 1 point from limiters on wrong attempt and block if limits reached
        try {
          const promises = [limiterSlowBruteByIP.consume(ipAddr)];
          // check if user exists by checking if authentication failed because of an incorrect password
          if (info.name === 'IncorrectPasswordError') {
            console.log('failed login: not authorized');
            // Count failed attempts by Email + IP only for registered users
            promises.push(
              limiterConsecutiveFailsByEmailAndIP.consume(emailIPkey)
            );
          }
          // if user does not exist (not registered)
          if (info.name === 'IncorrectUsernameError') {
            console.log('failed login: user does not exist');
          }

          await Promise.all(promises);
          req.flash('error', 'Email or password is wrong.');
          res.redirect('/login');
        } catch (rlRejected) {
          if (rlRejected instanceof Error) {
            throw rlRejected;
          } else {
            const timeOut =
              String(Math.round(rlRejected.msBeforeNext / 1000)) || 1;
            res.set('Retry-After', timeOut);
            res
              .status(429)
              .send(`Too many login attempts. Retry after ${timeOut} seconds`);
          }
        }
      }
      // If passport authentication successful
      if (user) {
        console.log('successful login');
        if (resEmailAndIP !== null && resEmailAndIP.consumedPoints > 0) {
          // Reset limiter based on IP + email on successful authorisation
          await limiterConsecutiveFailsByEmailAndIP.delete(emailIPkey);
        }
        // login (Passport.js method)
        req.logIn(user, function(err) {
          if (err) {
            return next(err);
          }
          return res.redirect('/');
        });
      }
    })(req, res, next);
  }
};

```


&nbsp;
##### Extra notes 

Within the RedisClient, the property `enable_offline_queue` is set to false. This is done to prevent issues such as slowing down servers if many requests are queued due to a Redis connection failure. The author of node-rate-limiter-flexible recommends this setting unless you have reasons to change it <sup>[7](https://github.com/animir/node-rate-limiter-flexible/issues/37)</sup>.

`req.ip` contains the remote IP address of the request <sup>[8](https://expressjs.com/en/api.html#req.ip "Express docs")</sup>.  If you are using the Express app behind a [reverse proxy](https://www.cloudflare.com/learning/cdn/glossary/reverse-proxy/), such as Cloudflare CDN, then you should set the Express apps trust proxy setting to true and provide the IP address, subnet, or an array of these that can be trusted as a reverse proxy. If you do not do this, the value of `req.ip`  will be the IP address of the reverse proxy <sup>[9](http://expressjs.com/en/guide/behind-proxies.html)</sup>.  Also note that running your application locally during development, `req.ip` will return 127.0.0.1 if you are using IPv4 or ::1, ::fff:127.0.0.1 if you are using IPv6 <sup>[10](https://www.npmjs.com/package/request-ip)</sup>.  These describe the local computer address.



In `index.js`, the file with all of your routes. The following route is defined:

```javascript
router.post('/login', catchErrors(rateLimitController.loginRouteRateLimit));
```

`catchErrors` is an error handling function that is used to catch any async-await errors in the controller. This error handling method is from the Wes Bos course [Learn Node](https://learnnode.com/). 

The errors for a Redis connection failure are handled as follows: Node Redis returns a `NR_CLOSED` error code if the client's connection dropped. `ECONNRESET` is a connection error. You can also set up a retry strategy for Node Redis to try and reconnect if the connection fails <sup>[11](https://github.com/NodeRedis/node-redis#options-object-properties)</sup>. 

```javascript
  if (err.code === 'NR_CLOSED' || err.code === 'ECONNRESET') {
    req.flash('error', 'There was a connection error');
    res.redirect('back');
```



---
&nbsp;
## Set up Redis

 The code above will not work yet as there is no Redis database set up. We will create a Redis database in the cloud using [Redis Labs](https://redislabs.com/). We will use the free plan. Then we will connect to this database through our Express app. To view the database, we will download Redis locally so that we can use the built-in client redis-cli (command-line interface). We will download and use Redis using Windows Subsystem for Linux (WSL), which allows you to use a Linux terminal in Windows. Other methods are described on the [Redis website download page](https://redis.io/download).



### Create an account with Redis Labs

Create an account on the [Redis Labs website](https://redislabs.com/try-free/). Follow the instructions in the documentation to learn how to [create a database](https://docs.redislabs.com/latest/rc/rc-quickstart/).

### Connect the Redis instance on Redis Labs with your Express application

In your express application `variables.env` add the REDIS_URL:

`REDIS_URL=redis://<password>@<Endpoint>`

Your Endpoint and password can be found in the database in the **Configuration** details of the **View Database** screen:

- The **Endpoint** setting shows the URL for your database and the port number.
- The **Access Control & Security** setting shows the password. 

In the rate limit controller from the previous section, the following code connects the cloud Redis instance, hosted on Redis Labs, to the Express application:

```javascript
const redisClient = redis.createClient(process.env.REDIS_URL, {
  // if no connection, an error will be emitted
  enable_offline_queue: false
});
```



The rate limiter instances connect to the  cloud Redis instance as follows (also from the rate limit controller): 

```javascript
const limiterSlowBruteByIP = new RateLimiterRedis({
  storeClient: redisClient,
   
...
    
const limiterConsecutiveFailsByUsernameAndIP = new RateLimiterRedis({
  storeClient: redisClient,
    
...
```


&nbsp;
### Set up WSL and download Redis

You will be able to rate limit your login route now, the next step is to set up Redis locally so that we can view the Redis database using the Redis command-line interface (redis-cli). Redis works best with Linux. Linux and OS X are the two operating systems where Redis is developed and tested the most. Linux is recommended for deployment <sup>[12](https://redis.io/topics/introduction), [13](https://medium.com/@RedisLabs/windows-subsystem-for-linux-wsl-10e3ca4d434e)</sup>. 

You can follow [this article](https://medium.com/@RedisLabs/windows-subsystem-for-linux-wsl-10e3ca4d434e) on how to set up WSL, download and install a supported Linux distro and install Redis locally. Install Redis somewhere outside of your application. The Linux distro used in this tutorial is Ubuntu 18.04.



### Connect the redis-cli to the Redis instance on Redis Labs

We will use the redis-cli locally to see the key-value pairs created. Run your Express application and in a WSL terminal run the redis-cli:

- cd into the Redis folder that you downloaded

  `cd redis-6.2.3`

- make sure the server is running

  `sudo service redis-server start`

  > to stop the server: `sudo service redis-server stop`

If you run `redis-cli`, you will connect to the local instance of Redis and will run locally on the Localhost (127.0.0.1:6379). To exit, run `quit`. To connect the redis-cli to the cloud instance of the Redis Labs database that we created, we'll use the URL-based connection method from the [Redis Labs docs](https://docs.redislabs.com/latest/rs/administering/creating-databases/). This connects to the Redis database using an endpoint URL and port number. Check the database **Configuration** details in the **View Database** screen to find the endpoint url and password.

```
$ redis-cli -h redis-19836.c9.us-east-1-2.ec2.cloud.redislabs.com
-p 19836 -a astrongpassword
```

- h is the host: add your endpoint, without the port number

- p is the port, which is shown at the end of the endpoint url

- a is access control. Add your password



You can test if the connection worked by typing `PING`. If the connection worked redis-cli will return `PONG`.

if the response is `NOAUTH Authentication required` - check that you typed the password correctly. You can run `quit` to exit the redis-cli so that you can try again.

&nbsp;


### Basic Redis commands

There are many [commands](https://redis.io/commands) available as shown in the docs. For our use case, we only need to know a few simple commands. You can try them in the redis-cli that is connected to your Redis Labs Redis instance. Note that the commands are all uppercase in the Redis docs, but the commands are not case-sensitive. However, key names are case-sensitive.

#### PING

Checks the connection to the Redis database. If there is a connection, `PONG` will be returned.

#### SET 

Set the string value of a key. It is used to create a key-value pair or change the value of an existing key.

```
> SET job teacher
OK
```

This sets the key "job" to the value "teacher". The response `OK` means that the command was successful.

#### MSET

Like SET, but its sets the values of multiple keys.

```
> MSET job "teacher" AGE "50" TITLE "Mr."
OK
```

#### GET

Get the value for a key.

```
> GET job
"teacher"
```

#### MGET

Get the value of multiple keys.

```
> MGET job age title
1) "teacher"
2) "50"
3) "Mr."
```

#### DEL

Deletes a specific key.

```
> DEL job
(integer) 1 -> this means that it found a key with the name "job" and deleted it. 
```

If you try :

```
> GET job
(nil) -> this means that no key with the name "job" exists.
```

#### SCAN

View all keys. It iterates over a collection of keys. It is a cursor-based iterator. If you want to view all entries then run 

```
> SCAN 0
1) "0"
2) "age"
3) "title"
```

The first value returned is "0", which indicates that a full iteration occurred. This means that all of the keys in the database were scanned. For more details, you can read up the description of the [SCAN command](https://redis.io/commands/scan) in the docs. 

If you want to view all keys, excluding the first key then run `SCAN 1`.

#### FLUSHALL

This deletes all of the keys in the database.

#### CLEAR

Clears the terminal.


---
&nbsp;
## Test the rate-limiting

We are going to test one of the rate limiters. Run your application locally and connect to Redis labs via the redis-cli in a WSL terminal. Before starting, make sure all of the keys in your database are deleted by running the command  `FLUSHALL`. In your rate limit controller middleware (`rateLimitController.js`.), set `maxConsecutiveFailsByEmailAndIP` to 3. Set the options `duration` and `blockDuration`  of `limiterConsecutiveFailsByEmailAndIP` to 60. This will allow us to test the rate-limiting quickly. 

```javascript
...

const maxConsecutiveFailsByEmailAndIP = 3; 

...

const limiterConsecutiveFailsByEmailAndIP = new RateLimiterRedis({
  storeClient: redisClient,
  keyPrefix: 'login_fail_consecutive_email_and_ip',
  points: maxConsecutiveFailsByEmailAndIP,
  duration: 60 
  blockDuration: 60  
});

...
```



### Failed login with an account that does not exist

Try login using an email (or another user identifier, such as user name, used in your app) that does not exist (not registered).

After this, in the redis-cli, that is connected to your cloud Redis instance hosted on Redis Labs, view all of the keys. 

```
yourRedisLabsEndpoint> SCAN 0
1)"0"
2) "login_fail_ip_per_day:::1"
```

> On localhost, `req.ip` will return 127.0.0.1 if you are using IPv4 or ::1,  ::fff:127.0.0.1 if you are using IPv6.

You can now check the number of consumed points (number of failed logins) of the `limiterSlowBruteByIP` rate limiter for the IP that tried to log in.

```
yourRedisLabsEndpoint> GET login_fail_ip_per_day:::1
"1"
```



### Failed login with an account that does exist

Now try log in with an existing account and use the wrong password. Then view all of the keys in your Redis database.

```
yourRedisLabsEndpoint> SCAN 0
1)"0"
2) "login_fail_ip_per_day:::1"
3) "login_fail_consecutive_username_and_ip:realuser@example.com_::1"
```



You can now check the number of points consumed for the IP that tried to log in for the `limiterSlowBruteByIP` rate limiter key.

```
yourRedisLabsEndpoint> GET login_fail_ip_per_day:::1
"2"
```



Check the number of consumed points for the `limiterConsecutiveFailsByEmailAndIP` rate limiter key.

```
yourRedisLabsEndpoint> GET login_fail_consecutive_username_and_ip:realuser@example.com_::1
"1"
```



Try logging in more than 3 times within 1 minute. After this, you will get this message displayed in your browser:

`Too many requests. Retry after 60 seconds.`



The login route for the given IP and username pair will be blocked for 60 seconds. This is because the `blockDuration` that we set  for the `limiterConsecutiveFailsByEmailAndIP` rate limiter is 60 seconds. After 60 seconds, check the number of consumed points for the key again: 

```
yourRedisLabsEndpoint> GET login_fail_ip_per_day:::1
(nil)
```

It does not exist anymore as we set the `duration` property to 60. The key is deleted after 60 seconds. 



Now try login using an existing account with the wrong password. After this, log in with the correct password. This will delete the `limiterConsecutiveFailsByEmailAndIP` rate limiter key for the given user and IP pair. This occurs once the login is successful, as can be seen in the rate limit controller:

```javascript
      ... 
       
        if (resEmailAndIP !== null && resEmailAndIP.consumedPoints > 0) {
          // Reset on successful authorisation
          await limiterConsecutiveFailsByEmailAndIP.delete(emailIPkey);
        }
      ...
```



>  You can do more thorough testing of POST requests using services such as [Postman](https://www.postman.com/), which is a tool used to build and test APIs.



---
&nbsp;
## Conclusion

This is a basic example of how to rate limit a login route in an Express app using node-rate-limiter-flexible and Redis. node-rate-limiter-flexible was used to count and limit the number of login attempts by key. Redis was used to store the keys. We created a rate limiter middleware in an existing application with a login route and authentication. Two rate limiters were created. The first rate limiter rate-limited based on IP. The second rate-limited based on IP and the user's email address. Redis Labs was set up to create an online instance of Redis. The Redis Labs instance was connected to the Express app using an endpoint URL. Redis was installed locally and was connected to the online instance of Redis. Rate-limiting was tested by viewing the database keys, using the redis-cli, after attempted logins. 



**Here are some useful links for further study:**

 1) [Redis Crash Course Tutorial](https://www.youtube.com/watch?v=Hbt56gFj998) - Learn the basics of Redis

 2) [Redis Caching in Node.js](https://www.youtube.com/watch?v=oaJq1mQ3dFI) - Learn how to cache API calls using Redis.

 3) [API Rate Limiting with Node and Redis](https://codeburst.io/api-rate-limiting-with-node-and-redis-95354259c768)

 4) [node-rate-limiter-flexible: rate-limiting examples](https://github.com/animir/node-rate-limiter-flexible/wiki/Overall-example)

 5) [Redis documentation](https://redis.io/documentation) 

 6) [Redis Labs documentation](https://docs.redislabs.com/latest/)

 7) [Redis Labs YouTube channel](https://www.youtube.com/c/Redislabs)