From PM2 to Docker: Cluster Mode
Tags: node, docker, webdev, devops
Published: 2020-04-25 14:40:37

*This article was originally published a day earlier at https://maximorlov.com/from-pm2-to-docker-cluster-mode/*

After publishing my [previous article](https://maximorlov.com/pm2-to-docker-automatic-restarts/) in the *From PM2 to Docker* series, I've received a few questions:

*"What about clustering? That's very easy with PM2, but how would you do that with Docker?"*
*"Can Docker utilise all cores available?"*
*"is Docker also easily scalable?"*

These are very legitimate questions. After all, cluster mode in PM2 is a commonly used feature in Node.js applications.

This article answers these questions. By the end of it, you will be able to **scale an application with Docker** in a way that **utilises all available CPU cores** for *maximum* performance.

You will also learn the architectural *differences* between scaling with PM2 and scaling with Docker, and the *benefits* the latter brings to the table.

## Horizontal scaling
For your application to be able to scale horizontally, it has to be *stateless* and [share nothing](https://en.wikipedia.org/wiki/Shared-nothing_architecture). Any data that needs to persist *must* be stored in a stateful backing, typically a database.

To **scale an application with Docker**, you simply start multiple container instances. Because [containers are just processes](https://twitter.com/b0rk/status/1235754793253171200), you end up with multiple processes of an application. This is somewhat similar to what you get when you use cluster mode in PM2.

The **difference with PM2** is that it uses the Node.js [cluster module](https://nodejs.org/api/cluster.html#cluster_how_it_works). PM2 creates multiple processes and the cluster module is responsible for distributing incoming traffic to each process. With Docker, distribution of traffic is handled by a *load balancer*, which we'll talk about in a bit.

A benefit of this approach is that you are not only able to scale on a single server but *across* multiple servers as well. The cluster module can only distribute traffic on a single machine, whereas a load balancer can distribute traffic to other servers.

To get the maximum server performance and **use all available CPU cores (vCPUs)**, you want to have one container per core. Starting multiple containers of the same application is simple. You just have to give them different names each time you use the `docker run` command:

```bash
# Start four container instances of the same application
docker run -d --name app_1 app
docker run -d --name app_2 app
docker run -d --name app_3 app
docker run -d --name app_4 app
```

We'll run into an issue if we want to use the same port for all containers:

```shell
$ docker run -d -p 80:3000 --name app_1 app
06fbad4394aefeb45ad2fda6007b0cdb1caf15856a2c800fb9c002dba7304896
$ docker run -d -p 80:3000 --name app_2 app
d5e3959defa0d4571de304d6b09498567da8a6a38ac6247adb96911a302172c8
docker: Error response from daemon: driver failed programming external connectivity on endpoint app_2 (d408c39433627b00183bb27897fb5b3ddc05e189d2a94db8096cfd5105364e6b): Bind for 0.0.0.0:80 failed: port is already allocated.
```

The clue is at the end: `Bind for 0.0.0.0:80 failed: port is already allocated.`. **A port can be assigned to only *one* container/process at a time.** If web traffic comes in on port 80, how do we spread it across all instances?

We would need a process that receives incoming traffic and *distributes* it among several other processes, that's what a load balancer does.

## Load balancing
**A load balancer sits in front of your application and routes client requests to all instances of that application.** A load balancing algorithm determines how to distribute traffic. The most common load balancing algorithm is *round-robin* — requests are distributed sequentially among a group of instances. That's the default for most load balancers and it's what the cluster module in Node.js uses for the distribution of traffic.

From all the load balancers out there, Nginx is the most popular in the Node.js community. **Nginx can do more than load balancing traffic** — it can also terminate *SSL encryption* and serve *static files*. Nginx is more efficient at those than Node.js. Shifting that responsibility away from the application frees up resources for handling more client requests.

Nginx configuration goes in a file named `nginx.conf`. Let's look at an example specific to load balancing. If you want to learn more about Nginx, the [official documentation](https://nginx.org/en/docs/) is a great place to start.

```nginx
# General configuration
user nginx;
worker_processes 1;

error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
  worker_connections 1024;
}

# Load balancing configuration starts here
http {
  # Define a group of servers named "app" and use the default round-robin distribution algorithm
  upstream app {
    server app_1:3000;
    server app_2:3000;
    server app_3:3000;
    server app_4:3000;
  }

  # Start a proxy server listening on port 80 that proxies traffic to the "app" server group 
  server {
    listen 80;

    location / {
      proxy_pass http://app;
    }
  }
}
```

We define a server group named `app` using the `upstream` directive. Inside the directive, we have a `server` definition for *each* container instance of our application. The addresses *match* the names we gave the containers and the port is the same port the Node.js server is listening on.

Below that, we define a proxy `server` that listens on port 80 and proxies all incoming traffic to the `app` server group.

While it's not inherently wrong to install Nginx directly on the host system, it's much **easier to communicate with other containers if we use Nginx inside a container**. Having the entire application stack inside containers also makes it easier to manage collectively using Docker Compose. You'll see how that works in the next section.

Let's use the [official Nginx image](https://hub.docker.com/_/nginx) from Docker Hub to start an Nginx container that will handle the load balancing for your application.

```bash
# Start an Nginx container configured for load balancing
docker run -d --name nginx -p 80:80 -v $(pwd)/nginx.conf:/etc/nginx/nginx.conf:ro nginx
```

We mount our configuration file inside the container using the `-v` flag. Additionally, we map port 80 on the host to port 80 inside the container. Port 80 on the host is where internet traffic arrives, and port 80 inside the container is what the Nginx proxy server listens to.

Note: The load balancer needs to [share a user-defined network with the application containers](https://maximorlov.com/4-reasons-why-your-docker-containers-cant-talk-to-each-other/#do-the-containers-share-a-network) to be able to communicate with them. Use the `--network` flag to place a container inside an existing network at start up time.

Let's confirm all containers are up and running using `docker ps`:

```shell
$ docker ps
CONTAINER ID        IMAGE        COMMAND                  CREATED             STATUS              PORTS                NAMES
0dc2055e0195        app          "docker-entrypoint.s…"   25 hours ago        Up 25 hours                              app_4
dea61045c74e        app          "docker-entrypoint.s…"   25 hours ago        Up 25 hours                              app_3
827a2a7e429b        app          "docker-entrypoint.s…"   25 hours ago        Up 25 hours                              app_2
eb2bd86b0b59        app          "docker-entrypoint.s…"   25 hours ago        Up 25 hours                              app_1
ba33b8db60d7        nginx        "nginx -g 'daemon of…"   25 hours ago        Up 32 minutes       0.0.0.0:80->80/tcp   nginx
```

That's *four* `app` servers and *one* `nginx` load balancer listening on port 80. We resolved the port conflict, and traffic is now distributed across all our application instances in a round-robin fashion. Perfect!

## Bringing it all together with Docker Compose
Instead of manually starting four containers and one load balancer, you can do it much quicker with a single command:

```shell
$ docker-compose up -d --scale app=4
Creating network "playground_default" with the default driver
Creating playground_app_1 ... done
Creating playground_app_2 ... done
Creating playground_app_3 ... done
Creating playground_app_4 ... done
Creating playground_nginx_1 ... done
```

Docker Compose brings the *entire* application stack together in one `docker-compose.yml` configuration file. You define all the services you need — a database, a backend, a frontend, a load balancer, networks, volumes, etc. — and control them as *a single unit*. Start everything up with `docker-compose up`, and bring everything down with `docker-compose down`. That's how easy it is.

**Head over to this [Github repository](https://github.com/Maximization/scaling-nodejs-with-docker)** to see the `docker-compose.yml` used in the example above along with a Node.js sample project. Compare with your project to figure out what's missing.

## Become a skilled Node.js developer

Every other Tuesday I send an email with tips on building solid Node.js applications. If you want to grow as a web developer and move your career forward with me, [drop your email here 💌](https://maximorlov.ck.page/0076aaf499).