Node.js and Redis deployed in Docker containers, using Docker Compose - then load-balancing the Node.js servers with Nginx
Tags: docker, node, redis, nginx
Published: 2021-12-24 21:35:16

This article contains two main stages:
**(1) Containerizing a Node.js server application and a Redis database instance into two separate Docker containers, using Dockerfile and Docker Compose, and showing how these two applications communicate with each other.**

**(2) Load-balancing the Node.js server, using a containerized Nginx reverse-proxy.** 

**Let’s start with Stage 1:
(1) Containerizing a Node.js server application and a Redis instance into two separate Docker containers, using Dockerfile and Docker Compose, and showing how these two applications communicate with each other**

Starting with a simple **Node.js server** application (we’ll call it “_test-webapp_”) that responds to an _HTTP GET_ request by displaying the “numbers of visits”. The numbering scheme below (i.e. (1.1), (1.2), (1.3) etc.), matches the numbering in the diagram below:


![Figure 1.a – Schematic diagram of the components](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3yp6moj945jli4n6vfl4.png)
_Figure 1.a - Schematic diagram of the components_

In "**Figure 1.a - Schematic diagram of the components**" above we have the following components:
(**1.1**) "_Docker Container 1_" - container running the **Node.js server** called "_test-webapp_" that communicates with the browser on the left. Each time we refresh the URL `localhost:80` i.e. we send a `GET `command to the **Node.js server** "_test-webapp_", the server code increments the number of visits, then saves this value into the **_Redis_** database instance that runs on "_Docker Container 2_", and also displays the value back in the browser.

(**1.2**) “_Dockerfile_” - defines and controls the **Node.js server** process in “_Docker Container 1_”. 

(**1.3**, **1.3.1**, **1.3.2**) “_[docker-compose.yml](https://github.com/marcelkatz/test-docker-nodejs-redis-nginx/blob/main/docker-compose.yml)_” – the **Docker Compose** config file defines and controls both “_Docker Container 1_” and “_Docker Container 2_”. “_Docker Container 1_” runs the **Node.js server** process “_test-webap_p”. “_Docker Container 2_” runs the **_Redis_** database instance.

(**1.3.3**) **Docker Compose** establishes by default a communication network between “_Docker Container 1_” and “_Docker Container 2_” which allow the **Node.js server** process “_test-webapp_” to communicate with the **_Redis_** database instance, and exchange between them the “number of visits to the app/web server” (`numVisits`) value.

(**1.3.4**) **Docker Compose** maps local hosting machine Port 80 to “_Docker Container 1_” Port 5000. Port 5000 is the port on which the **Node.js server** “_test-webapp_” listens and reacts to the `GET `commands sent by the browser.

(**1.4**) Connecting to the shell of “_Docker Container 2_” and then to the client command line of the **_Redis_** database instance via “`redis-cli`” we can see that the value of `numVisits` (which represents the number of times the browser issued a `GET `command to the **Node.js server**) is in sync with the value displayed in the browser by the **Node.js server** – thus showing that inter-process communication occurs between the processes “_test-webapp_” in “_Docker Container 1_” and the **_Redis_** process in “_Docker Container 2_”.

(**1.5**) This step illustrates the `restart` directive and capability in **Docker Compose** (specified in config file “_docker-compose.yml_”) – when connecting to the Linux shell of “_Docker Container 1_”, we can `kill -9` the **Node.js server** process, but the **Node.js server** process will be restarted automatically by **Docker Compose** – illustrating the automatic recovery provided by **Docker Compose**.

And now let’s describe the steps and the flow of this scenario. The numbering scheme in the description below (i.e. (1.1), (1.2), (1.3) etc.), matches the numbering in “**Figure 1.a – Schematic diagram of the components**”.

**(1.1) File structure:**

![Figure 1.b – File structure for Stage 1](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rm92ysovtnnq9azlunop.png)
_Figure 1.b – File structure for Stage 1_

**Node.js files for process ‘test-webapp’:**

The contents of directory “_test-webapp_”, where the source code for the **Node.js server** “_test-webapp_” resides:

![Node.js files for process 'test-webapp'](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jqed4cw6m1mpm1upws9m.png)

(**1.2**) The _Dockerfile _containerizes and controls the **Node.js application** by downloading the “_node:alpine_” image from [Docker Hub](https://hub.docker.com/), installing **Node.js** on the container, copying to the container the [source files](https://github.com/marcelkatz/test-docker-nodejs-redis-nginx/tree/main/test-webapp) – then launching the **Node.js server** web app (see source code in file “[_server.js_](https://github.com/marcelkatz/test-docker-nodejs-redis-nginx/blob/main/test-webapp/server.js)”).

(**1.3**) Going one directory above, we see the "_[docker-compose.yml](https://github.com/marcelkatz/test-docker-nodejs-redis-nginx/blob/main/docker-compose.yml)_" file that organizes the containerization and sets up the architecture of all the components. (File 
“_[docker-composer-nginx.yml](https://github.com/marcelkatz/test-docker-nodejs-redis-nginx/blob/main/docker-compose-nginx.yml)_” will be presented and explained in **Stage 2** of this article)

!['docker-compose.yml' file for process 'test-webapp'](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tqtlkv6v7vams8x3xq9d.png)

**Purge all images and containers:**

We run command `docker system prune -a` to clear all Docker images and containers and start with a clean slate.

```
C:\test-docker\test-redis>docker system prune -a                                          
WARNING! This will remove:                                                                                                
- all stopped containers                                                                                                
- all networks not used by at least one container                                                                       
- all images without at least one container associated to them                                                          
- all build cache                                                                                                                                                                                                                             
Are you sure you want to continue? [y/N] y
```

**(1.3) Build and run the 'test-webapp' image with Docker Compose**

Use command `docker-compose -f <config-filename> build` to build containers and the applications that will be running in each container:

`C:\test-docker\test-redis>docker-compose -f docker-compose.yml build`

See the results below of the built Docker image:

```
C:\test-docker\test-redis>docker images                                                                                                                     
REPOSITORY               TAG       IMAGE ID       CREATED         SIZE                                                                                                                     
test-redis_test-webapp   latest    e8145bea0fec   4 minutes ago   175MB
```

**Run the 'test-webapp' and 'redis' containers with 'docker-compose':**

Let’s launch both “_test-webapp_” and “_redis_” services, as described in config file 
“_[docker-compose.yml](https://github.com/marcelkatz/test-docker-nodejs-redis-nginx/blob/main/docker-compose.yml)_”, using the `docker-compose -f <config-filename> up` command.

![docker-compose -f docker-compose.yml up](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/66ukw6kmm4djhk2nzxsa.PNG)

We can see from the output above, that both the “_redis_” container (“_test-redis_1_” – corresponding to “_Docker Container 2_” in **Figure 1.a**) and the “_test-webapp_” container (“_test-webapp_1_” corresponding to “_Docker Container 1_” in **Figure 1.a**) are running and printing to stdout in the command line window where we launched **Docker Compose** to run these two containers.

**View the 'test-webapp' and 'redis' running containers:**


```
C:\test-docker\test-redis\test-webapp>docker ps                                                                                        
CONTAINER ID   IMAGE                    PORTS                
NAMES                                         
928b8b07415d   test-redis_test-webapp   0.0.0.0:80->5000/tcp   test-redis_test-webapp_1                      
a8756127bff5   redis:alpine             6379/tcp               test-redis_test-redis_1
```

(**1.3.1**, **1.3.2**) The two containers above match the containers “_Docker Container 1_” and “_Docker Container 2_” in the **Figure 1.a** above. Note the “_CONTAINER ID_” column whose values we will use below to perform operation on each individual running container. 

(**1.3.4**) Port 5000 in the **Node.js server** "_test-webapp_" container is mapped to local (hosting) Port 80, so when one connects in the local (hosting) browser to URL **http://localhost:80**, for each refresh, the **Node.js process** in the “_test-webapp_” container increments the number of visits in variable `numVisits` which is set and saved in the **_Redis_** in variable `numVisits` -- and this value is also send back and displayed in the browser. 

“Docker-compose” sets-up by default a network with both “_test-webapp_” container (“_Docker Container 1_” in **Figure 1.a**) and “_redis_” container (“_Docker Container 2_” in **Figure 1.a**) within this network, and both containers are reachable by each other via this network. 

The local browser communicates with the **Node.js server** container. When refreshing the connection in the browser, the server callback is invoked which responds to the browser with the updated number of visits.

![1.3.4 Browser - localhost - number of visits 8](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/euscx0msi6am2kz9tk7j.png)

(**1.4**) We are using the `docker exec -it` command that allows us to connect to a running container while the `-it` option allows us to capture the stdin/stdout of that container. Then we specify the CONTAINER ID _a8756127bff5_ obtained from `docker ps` command above, followed by the shell (**_sh_**) that we want to launch as we enter the container. 

`C:\test-redis\test-webapp>docker exec -it a8756127bff5 sh`

Then, once we are inside the container’s shell, we connect to the **_Redis_** database using the `redis-cli` command. At the **_Redis_** prompt we use `get numVisits` to obtain the value of the variable “**numVisits**” inside “_redis_”. We can see that the “_redis_” instance communicates with the “_test-webapp_” process in its respective container and the variable “_numVisits_” in the **_Redis_** database instance is in sync with its value in the browser. In this case both have the value “**8**”, because we refreshed 8 times the “**localhost:80**” URL thus issuing a `GET` command in the browser that is intercepted by the **Node.js server ** which increments the “number of visits” (`numVisits`) variable. The “number of visits” value is sent back to the browser by the “_test-webapp_” process which also saves the value in the “_redis_” database in variable `numVisits`).

```
/data # redis-cli                                                                                                                                                     
127.0.0.1:6379> get numVisits                                                                                                                                      
"8"                                                                                                                                                                    
127.0.0.1:6379> 
```

From within the “_redis-cli_” in the “_redis_” container (“_Docker Container 2_”) we can also set in **_Redis_** manually the “_numVisits_” variable to a random value of let’s say “**342**”…

![1.4 - redis - set num visits 342](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3snsztp8x4gcgco7t7ec.png)

…the `numVisits` variable is updated in the “test-webapp” **Node.js server** (running in “_Docker Container 1_”), and therefore in the browser (due to the fact that in order to invoke the _callback_ in the **[Node.js server](https://github.com/marcelkatz/test-docker-nodejs-redis-nginx/blob/main/test-webapp/server.js)**, one needs to refresh the connection to “**localhost:80**”, the number of visits increases by **1**, thus **342** + 1 = **343**. This shows that we have two-way inter-process communications between the processes running in “_Docker Container 1_” and “_Docker Container 2_”.

![1.4 Browser - localhost - number of visits 343](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/oapryvdzrnppyzztuxfn.png)

(**1.5**) A useful feature provided by **Docker Compose** is the capability to specify in “_docker-compose.yml_” a “[restart](https://docs.docker.com/config/containers/start-containers-automatically/#use-a-restart-policy)” option. 
This will allow us when connecting to the shell of “_Docker Container 1_”, to “kill” the **Node.js server** process, but the **Node.js server** process will be restarted automatically by the **Docker Compose** “_restart_” directive.

```
C:\test-docker\test-redis>docker ps
CONTAINER ID   IMAGE                      PORTS                    NAMES
c675ff6c0464   test-redis_nginx           0.0.0.0:80->80/tcp       test-redis_nginx_1
3137d1468ec7   test-redis_test-webapp-2   0.0.0.0:3009->5000/tcp   test-redis_test-webapp-2_1 
57d399295421   redis:alpine                                        test-redis_test-redis_1
b30635f44151   test-redis_test-webapp-1   0.0.0.0:3008->5000/tcp   test-redis_test-webapp-1_1
```

Connect to the Docker container whose ID is _928b8b07415d_ and invoke the shell (**_sh_**). 

`C:\test-redis\test-webapp>docker exec -it 928b8b07415d sh`

Inside the container, at the shell prompt, show all process id’s using `ps -al`.

```
/usr/src/app # ps -al
PID   USER     TIME  COMMAND
1     root     0:00  npm start
19    root     0:00  node server.js
30    root     0:00  sh
36    root     0:00  ps -al
```

Proceed with “killing” the “_node server.js_” process by issuing a `kill -9 <process-id>` command:

`/usr/src/app # kill -9 19`
      
In the command line window that is running **Docker Compose** we can see how the “_test-webapp_” receives a “kill signal” (`SIGKILL`), exited with code ‘**1**’, and then restarted automatically.

![npm ERR command sh -c node server.js](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7m25v5kmnu9zlbobgqgf.PNG)

**Conclusion**

In **Stage 1** of this example we showed how **Docker Compose** allows us to easily establish independent environments that communicate with each other, and also the automatic fault-tolerance (restart on failure) capability of Docker Compose.

**Let’s continue with Stage 2:
(2) Load-balancing the Node.js server, with the help of a containerized Nginx reverse-proxy**

The diagram in “**Figure 2.a – Schematic diagram of the components for Stage 2**” describes an architecture similar to the one described earlier in “**Figure 1.a – Schematic diagram of the components**” but with the changes described below.


![Figure 2.a – Schematic diagram of the components for Stage 2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/02s6wkco89eaeu00ezxu.png)
_Figure 2.a – Schematic diagram of the components for Stage 2_

In “**Figure 2.a – Schematic diagram of the components for Stage 2**” we have the following components:

(**2.1.1**, **2.1.2**) “_Docker Container 1_” and “_Docker Container 2_” – two identical containers whose source code reside in directories “_[test-webapp-1](https://github.com/marcelkatz/test-docker-nodejs-redis-nginx/tree/main/test-webapp-1)_” and “_[test-webapp-2](https://github.com/marcelkatz/test-docker-nodejs-redis-nginx/tree/main/test-webapp-2)_” (as shown in “**Figure 2.b – File structure for Stage 2**” below), that are almost identical copies of the application “_test-webapp_” that was described earlier in **Stage 1**. This time we are using two **Node.js server** processes that will serve the client browser from the local host machine, scaling up and load-balancing the original one-server configuration from **Stage 1**. These two containers are defined and controlled each by their respective “_Dockerfile_” (**2.1.1.1**) and (**2.1.1.2**). Each **Node.js server** “_Docker Container 1_” and “_Docker Container 2_” counts the number of visits coming from the local host browser. Then it saves the number of visits into the **_Redis_** database, and it also responds back to the browser with the number of visits and with which specific **Node.js server** served each individual _HTTP GET_ request coming from the browser, by sending back to the browser a message of type: 
“**test-webapp-1**: Number of visits is: <numVisits>”, or 
“**test-webapp-2**: Number of visits is: <numVisits>”
…thus highlighting the load-leveling nature of this stage.

(**2.1.3**) “Docker Container 3” – the container running the **_Redis_** database instance, identical to the one described in **Stage 1**, storing the “number of visits” performed by the localhost machine browser to “**localhost:80**”. The number of visits is stored by the **Node.js server** processes “_test-webapp-1_” and “_test-webapp-2_” in the **_Redis_** variable `numVisits` whose value is transmitted by each **Node.js server** to the **_Redis_** database on each refresh on the local host browser.

(**2.2**) “_[docker-compose-nginx.yml](https://github.com/marcelkatz/test-docker-nodejs-redis-nginx/blob/main/docker-compose-nginx.yml)_” – the main **Docker Compose** config file defines and controls: (I) “_Docker Container 1_” running **Node.js server** “_test-webapp-1_”,  (II) “_Docker Container 2_” running **Node.js server** “_test-webapp-2_”, (III) “_Docker Container 3_” running **_Redis_**, and (IV) “_Docker Container 4_” running **_Nginx_**.

(**2.3**) “_Docker Container 4_” running “_Nginx_” – This is an additional container introduced in **Stage 2**, defined and controlled by its own _Dockerfile_ (**2.3.1**), that runs an “_nginx_” instance, and acts a as reverse-proxy that routes the _HTTP GET_ requests coming from the local host browser. The “**_Nginx_**” process in “_Docker Container 4_” routes the _HTTP GET_ requests coming from local host browser “**localhost:80**”, in a **round-robin** manner (**(2.3.3)** and **(2.3.4)**), to either the “_test-webapp-1_” **Node.js server** in “_Docker Container 1_” or to “_test-webapp-2_” **Node.js** server in “_Docker Container 2_”. The “**_nginx_**” process in “_Docker Container 4_” is defined and controlled by the **_Nginx _**config file “_[nginx.conf](https://github.com/marcelkatz/test-docker-nodejs-redis-nginx/blob/main/nginx/nginx.conf)_” which is copied by **_Nginx_** container’s _Dockerfile_ to the “_Docker Container 4_” environment file “_/etc/nginx/conf.d./**default.conf**_” (this is a standard **_Nginx_** setup). The “_nginx_” instance distributes the incoming traffic from the local host browser, thus scaling up and load- balancing the single-container web/app server architecture presented in **Stage 1**. 

And now let’s describe the steps and the flow of this scenario. The numbering scheme in the description below (i.e. (2.1), (2.2), (2.3) etc.), matches the numbering in “**Figure 2.a – Schematic diagram of the components for Stage 2**”.

**(2.1) File structure:**

![Figure 2.b – File structure for Stage 2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8v7qndptehh545sknesh.png)

The file structure described in “**Figure 2.b – File structure for Stage 2**” is almost identical to the files structure described earlier in “**Figure 1.b – File structure for Stage 1**” with the following changes:

(**2.1.1**, **2.1.2**) The files from directory “_test-webapp_” from **Stage 1** were copied into directories “_test-webapp-1_” and “_test-webapp-2_”.

(**2.2**) Going one directory above, we see the "_[docker-compose-nginx.yml](https://github.com/marcelkatz/test-docker-nodejs-redis-nginx/blob/main/docker-compose-nginx.yml)_" config file that organizes the containerization and sets up the architecture of all the components:

!['docker-compose-nginx.yml' file](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/23ztsgagx0fthaktnn3n.png)

**Purge all images and containers:**

As in **Stage 1**, we run command `docker system prune -a` to clear all Docker images and containers and start with a clean slate.

**(2.3) Build and run the 'test-webapp-1',  'test-webapp-2', ‘redis’, and ‘nginx’ images with Docker Compose**

Build with Docker Compose:

`C:\test-docker\test-redis>docker-compose -f docker-compose-nginx.yml build`

Run with Docker Compose:

`C:\test-docker\test-redis>docker-compose -f docker-compose-nginx.yml up`

In the command line window where we issue the `docker-compose -f docker-compose-nginx.yml up` command, **Docker Compose** replies with:

![4 containers running](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xi20q9vn6kp7m67olqst.PNG)

...showing that all 4 Docker containers have started successfully and are up and running: “_test-redis_1_” corresponds to the **_Redis_** process running in “_Docker Container 3_”, “_test-webapp-2_1_” corresponds to the **Node.js server** process running in “_Docker Container 2_”, “_test-webapp-1_1_” corresponds to the **Node.js server** process running in “_Docker Container 1_”, and “_nginx_1_” corresponds to the **_Nginx_** server running in “_Docker Container 4_”.

**View the 'test-webapp-1', ‘test-webapp-2’, 'redis', and ‘nginx’ running containers:**

```
C:\test-docker\test-redis>docker ps 
CONTAINER ID   IMAGE                       PORTS       NAMES                                            c675ff6c0464   test-redis_nginx            0.0.0.0:80->80/tcp        test-redis_nginx_1                               
3137d1468ec7   test-redis_test-webapp-2    0.0.0.0:3009->5000/tcp   
test-redis_test-webapp-2_1                       
57d399295421   redis:alpine                                                                         test-redis_test-redis_1                          
b30635f44151   test-redis_test-webapp-1    0.0.0.0:3008->5000/tcp   test-redis_test-webapp-1_1
```

The four containers above match containers “_Docker Container 1_” through “_Docker Container 4_” in “**Figure 2.a – Schematic diagram of the components for Stage 2**”
above. Note the “_CONTAINER ID_” column whose values we will use below to potentially perform operations on each individual running container.

Let’s run first two instances of the browser on the hosting machine, and point them to URL “**localhost:80**”:

![2.3 Browser - test-webapp-2 - number of visits 7](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/42dwmonlt8jn07ifovp0.png)

![2.3 Browser - test-webapp-1 - number of visits 8](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/d06t4jutlz9c47pd5po8.png)

Notice how due to the **round-robin** routing mechanism employed by the **_Nginx_** reverse-proxy, the “_GET localhost:80_” request is routed once to “_test-webapp-1_” **Node.js server**, and once to the “_test-webapp-2_” **Node.js server**, achieving the scaling-up and load balancing that we intended to demonstrate. 

Let’s connect to the container that is running **_Redis_**, to its **_sh_** (shell) environment:

`C:\test-docker\test-redis>docker exec -it 57d399295421 sh  `

Then, inside the container, let’s connect to **_Redis_** itself using “_redis-cli_”:

```
/data #
/data # redis-cli
127.0.0.1:6379> 
127.0.0.1:6379> get numVisits
"8"
127.0.0.1:6379>
```

Note how the `get numVisits` command in **_Redis_** returns the expected value of “number of visits” that is communicated to the “**_redis_**” container from the containers that are running the **Node.js app servers**.

**Conclusion**

In **Stage 2** of this example we showed how **Docker Compose** allows us to easily establish multiple containers with their independent environments that communicate with each other, and also how scaling and load-balancing achieved with Nginx.

**Source code:**
[https://github.com/marcelkatz/test-docker-nodejs-redis-nginx](https://github.com/marcelkatz/test-docker-nodejs-redis-nginx)