How to Cache Database Results with Redis in Node.js
Tags: webdev, node, api, caching
Published: 2023-11-19 05:00:09

The speed with which an application responds to user queries is an important performance metric that the application can be rated by. A major factor that has been proven to improve the response speed of an application is caching. Caching helps to enhance the performance of distributed systems.

When dealing with data that does not change too often, it is recommended that the results of the request be cached; this helps to reduce the number of API calls made to the database.

There are several tools for caching; however, in this article, we will be using [Redis] (https://redis.io/), an in-memory database that stores data in the server memory.

This article will be focused on caching and how to perform caching on database results with Redis in Node.

**Why should you perform caching?**
There are several reasons why developers should employ caching in their applications, amongst which are

- To reduce latency, thus reducing response time.
- To save or reduce costs. This is possible because caching reduces the load on the backend database(s), especially if the database charges per throughput.
- Caching can greatly improve the performance of your application and, in the long run, lead to user satisfaction.
- By implementing caching, developers can anticipate the performance of their applications, particularly during periods of high network requests.
- When data is stored in the cache, users can still access this data even when there is a network outage.

###Prerequisite
To follow through with this tutorial, a basic understanding of Node.js and how to build REST APIs' is required. You should have the following installed on your computer;

- [Node.js](https://nodejs.org/en/download).
- Redis: You can either have a local instance of Redis running on your local machine or use the Redis cloud service. For the purpose of this tutorial, we will be using a local instance of Redis.
For Windows users, you can follow through on [how to download and install Redis on your local computer] (https://www.youtube.com/watch?v=qTN0GqF_Gq8&t=87s). For other operating systems, check out [install Redis] (https://redis.io/docs/install/install-stack/).

- RedisInsight: is a Redis Graphical User Interface (GUI) tool that helps with visualizing data stored in the Redis database. It also helps in monitoring real-time changes in the database. [Download and install RedisInsight](https://redis.com/redis-enterprise/redis-insight/?_gl=1*1j606fs*_gcl_au*Mzk0OTcxNjI4LjE2OTkyNjYwMzA.*_ga*NjQxNTIzMzI0LjE2OTkyNjYwMzE.*_ga_8BKGRQKRPV*MTY5OTM4NTQwNC4yLjEuMTY5OTM5MDQ2MS41MC4wLjA).

###Getting Started

#####Step 1: Project Setup
We will start with installing all the dependencies needed for this project and start an express server
i. create the directory for the project using the `mkdir` command
```
$ mkdir node_cache
```
ii. Navigate into the project directory using the `cd` command
```
$ cd node_cache
```
iii. Initialize your project and create a `package.json` file in the root folder of the project directory using the `npm init` command
```
$ npm init -y
```
The `-y` flag accepts all default suggestions automatically.
iv. Install the following dependencies from:
- `express`, 
- `ioredis`: a Redis client for Node.js which can be used to connect your application to the Redis server
- `fetch` If you are using the latest version of Node (> version 17.5), then you don't need to install the fetch API, as it comes inbuilt with the latest Node version. You can also use the `axios` API instead of the node fetch API, but to use `axios` you will need to install the package from the npm store. For the purpose of this project, we will be using the inbuilt node `fetch` API.
To install the required packages all at once

```
$ npm i express ioredis
```
v. Now that the dependencies have all been installed, create an `app.js` file in the root directory of your project.
```
touch app.js
```
For Windows OS users, you can achieve the same method of file creation through **Windows Powershell**, using the command `new-item`. Thus to create an `app.js` file in windows
```
new-item app.js
```
You can also create the `app.js` file in your preferred text editor.
vi. Open the file in your preferred text editor. I will be using [VS Code] (https://code.visualstudio.com/) as my preferred text editor. In the `app.js` file, we are going to create a simple Express server

```app.js
const express = require('express');

const app = express();
const PORT = 3300;

app.listen(PORT, () => {
  console.log(`App listening on port ${PORT}`);
});
```

#####Step 2: Retrieve data from an API without caching
We will make an API call to a third-party service, which, in turn, fetches data from a database, in order to retrieve the data we need. We will be making the API call from the Express server we built in step 1 above. Just so you know, the API call will be made without caching.
To begin, still, inside your `app.js` file, we will be making an API call to the [REST COUNTRIES] (https://restcountries.com/v3.1/name/) API to retrieve some data.

In your `app.js`, create a `GET` route. The `GET` method accepts two parameters, which are:

- route parameter: It defines what should happen when a request is made to that specific route, in this case, `'/country/:countryName'`
- callback function
`app.get('/country/:countryName', async (req, res) => {})`

Note: The usage of the `async` keyword in the callback function of the `GET` route above is necessary due to the asynchronous nature of the `fetch` API. As the `fetch` function is asynchronous, we use `async` to declare the callback function as asynchronous. This allows us to use the `await` keyword within the function to wait for the result of the `fetch` API, ensuring that the data is fully retrieved before proceeding with further logic.

In the body of the callback function for our `GET` route, we will include a `try{} catch{}` block to effectively manage and handle any runtime errors that may occur

```
app.get('/country/:countryName', async (req, res) => {  
  try {
    
  } catch (error) {
    
  }
});
```

In the `try{} catch{}` block, we will call the node `fetch` API, which we will use to make an API call to [REST COUNTRIES] (https://restcountries.com/v3.1/name/).

```
try {
   const data = await fetch(`https://restcountries.com/v3.1/name/${countryName}`)
  } catch (error) {
    console.log(error)
  }
```

When making a `fetch` request, it's necessary to invoke the `.json()` method on the response object to obtain the returned JSON data. Remember to include the `await` keyword, as you might get unparsed JSON data without it.

```
let result = await data.json();
    return res.status(200).send(result);
```
Our final code output will look like this:
```
const express = require('express');

const app = express();
const PORT = 3300;

app.get('/country/:countryName', async (req, res) => {
  const {countryName} = req.params;

  try {
    const data = await fetch(`https://restcountries.com/v3.1/name/${countryName}`);

let result = await data.json();
    return res.status(200).send(result);
  } catch (error) {
    console.log(error);
  }
});

app.listen(PORT, () => {
  console.log(`App listening on port ${PORT}`);
});
```
Now we can start our server by running `$ node app.js` and then open any API test tool of your choice. I will be using [Postman](https://www.postman.com/downloads/) for this. You can also download the [Postman VS code extension](https://marketplace.visualstudio.com/items?itemName=Postman.postman-for-vscode).
The REST COUNTRIES API accepts many country's name initials, but we will use only the `eest` as a route parameter on the endpoint we will be testing throughout this tutorial.
In your postman, make a request to the country endpoint.
`http://localhost:3300/country/eest`

![postman_api_testing_without_caching](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k2mpl7rrfpvasvrbmaq9.png)

In the image displayed above, you'll notice that there's a highlighted section indicating that it takes approximately `1281 milliseconds` for the request to be completed. This delay can be better optimized using caching, considering that the data being requested doesn't change frequently. Additionally, as your user base expands, the number of requests to this particular endpoint will increase, potentially leading to excessive strain on the database. But this issue can be resolved with caching

#####Step 3: Implementing Caching using Redis
Here, we will begin by starting our Redis server. In your terminal (wsl for windows users or redis-cli), type `redis-server`; this script will start the local redis server. 
**PS**: You must have Redis installed in your computer.
To confirm if your local redis instance is up and running, open another terminal window and type `redis-cli ping`, and you will get back a response `PONG`.

Now that our redis server is up, we are going to connect our Express application to the local Redis server.

Firstly, inside your `app.js` file, import the `ioredis` module that was installed as a depency in step 1
`const Redis = require('ioredis');`

At the top of your `GET` route in the `app.js` file, create a function that connects to the Redis server.

```
let client;
(() => {  
    client = new Redis({
      port: 6379,
      host: '127.0.0.1',
    });

    client.on('connect', () => {
      console.log('Connected to Redis yo!');
    });

    client.on('error', (error) => {
      console.error(error);
    });
})();
```
In the above code, we declared a variable and then created an anonymous IIFE (Immediately Invoked Function Expression) function to connect our Express application to redis using `ioredis`. The variable `client` is declared outside of the IIFE function, so that it can be globally accessible outside of the function.

Inside the function, we called the `new Redis()` class to create a Redis instance and passed the connection parameters into the object. This method will come in handy if you are also trying to connect to the Redis cloud service. The new Redis instance is assigned to the `client` variable.

Also, call on the Node.js `on()` method which registers events on whichever object it is called on (in this case, the ioredis object). The `on()` method accepts two arguments; the name of the event and a callback function.

The `client.on('connect', () => {})` function checks if the new Redis instance has successfully connected.

The `client.on('error', () => {})` function checks if there is an error while trying to connect to the Redis server.

Next, we are going to implement **cache hit** logic. A cache hit is when data is successfully served from the cache memory. This code will look like:

```
let cachedData = await client.get(countryName);
    if (cachedData ) {
      return res.status(200).send(JSON.parse(cachedData));
    }
```
The ioredis `get()` method is used to get data from the redis server. We pass the `countryName` as an argument into the `get()` method.  
The `if(cachedData){}` conditional statement checks if the cachedData variable has data, if it returns true, this is known as a **cache hit**. The cacheData variable is then converted to a JavaScript object using the `JSON.parse()` method, and the result is returned to the user.

After implementing the cache hit logic, next we will implement the **cache miss** logic. This is what happens when the cache hit fails, meaning there is no data with that key name in the redis server. The code logic for the cache miss would look like:

```
client.set(countryName, JSON.stringify(result));
```
The ioredis `set()` method plays a crucial role in storing data on our Redis cache server. This method requires two arguments: the key and data (in this case, `countryName` and `result`).

To elaborate further, the first argument, `countryName`, serves as the key under which the data is stored on the Redis server. It's essential to recall that `countryName` is a dynamic value, initially parsed in our endpoint `'/country/:countryName'`. Therefore, when users access the endpoint `'/country/eest'`, the `countryName` dynamically becomes and is stored as `eest` on the Redis server.

The second argument holds the result retrieved either directly from the database or through a third-party API call. To ensure compatibility, we employ the `JSON.stringify()` method on the result. This step converts the obtained data into a JSON string. Also, recall that in our cache hit logic, when we retrieve data using the `get()` method, we subsequently use `JSON.parse()` on the `cachedData`. This action converts the data back into a JavaScript object, ensuring seamless integration into our application's logic.

Your updated code should look like:

```
const express = require('express');
const Redis = require('ioredis');

const app = express();
const PORT = 3300;

// Connect to the redis server
let client;
(() => {
  client = new Redis({
    port: 6379,
    host: '127.0.0.1',
  });

  client.on('connect', () => {
    console.log('Connected to Redis yo!!');
  });

  client.on('error', (error) => {
    console.error(error);
  });
})();


app.get('/country/:countryName', async (req, res) => {
  const {countryName} = req.params;

  try {
    //cache hit
    let cachedData = await client.get(countryName);

    if (cachedData) {
      return res.status(200).send(JSON.parse(cachedData));
    }

const data = await fetch(`https://restcountries.com/v3.1/name/${countryName}`);

    let result = await data.json();

    //cache miss
    client.set(countryName, JSON.stringify(result));

    return res.status(200).send(result);
  } catch (error) {
    console.log(error);
  }
});

app.listen(PORT, () => {
  console.log(`App listening on port ${PORT}`);
});

```
Save your code and navigate to Postman to test the endpoint


![API call withought caching](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7ufgxc29ajmxt38t46bl.png)

From the above image, we can see that the response time was high (1789ms). When we requested the data from the endpoint, the API call was made directly to the Rest Countries API, the `cachedData` variable returns null because there is no data with the key `countryName` stored in our Redis cache server yet. Thus there is a cache miss, then the rest codes after the `if(cachedData){}` condition would run, and eventually, the data is saved to the Redis cache server. 

If you make the same request to the same API endpoint as shown below, you will see that the response time would significantly reduce, as shown below:

![caching implemented](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/45ln0p4x6jpdt4ydkwv8.png)

From the image above, we can see that it took 311ms to return data to the user, this is because, the data was already saved in the Redis server, thus faster retrieval. 

###Conclusion
The significance of caching in software development cannot be overstated. Its impact extends beyond just enhancing our application; it also plays a crucial role in cost savings, particularly when our service/application relies on data from paid API services or databases.

The complete code can be found [Here](https://github.com/Muyixone/Caching-with-Redis-Tutorial/tree/main)