How to setup a MongoDB Replica set for development using Docker
Tags: docker, mongodb, tutorial
Published: 2021-02-15 15:41:19

In this post I will be sharing how I configure and run a MongoDB [replica set](https://docs.mongodb.com/manual/replication/) on my local machine, using Docker.  
While there are ample examples on the internet about running MongoDB containers, not many of them focus on how to bring up a MongoDB replica set.  

Doing development against single node MongoDB is fine for starting, but as one starts using features like [transactions](https://docs.mongodb.com/manual/core/transactions), it becomes imperative that the MongoDB deployment has replica set enabled (or sharded cluster, a topic for future discussion).  
Creating and developing against a MongoDB replica set in local development setup helps you learn the topic, and be more confident that your application code is already adapted to run against replica sets in production.  

To follow along, you will need to have **Docker**, and **Docker Compose** installed in your machine.  
To install Docker, I suggest following the official documentation [at this link](https://docs.docker.com/get-docker/).  
Once Docker is installed, install Docker Compose by following the links from [this link](https://docs.docker.com/compose/install/).  

I have created [this Github repository](https://github.com/sntnupl/devcontainers-mongodb-replica-set-with-docker), which contains all the required artifacts to follow along with this tutorial.  

## The Docker Compose file  

Once Docker and Docker Compose are both installed on your machine, we can start writing the Docker Compose file, that contains directives to bring up the MongoDB replica set. I will show the full `docker-compose.yml` file first, and then explain the individual details.  

```yml
version: '3.9'  # Docker Engine release 19.03.0+ [https://docs.docker.com/compose/compose-file/]

services: 

    # setup MongoDB cluster for production
    mongo-replica-setup:
        container_name: mongo-setup
        image: 'mongo:4.2'
        restart: on-failure
        networks:
            - netApplication
        volumes:
        - ./.docker/mongodb/scripts/mongosetup.sh:/scripts/mongosetup.sh
        entrypoint: ["bash", "/scripts/mongosetup.sh" ]
        env_file:
            - .env
        environment:
            MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
            MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
        depends_on:
        - mongo1
        - mongo2
        - mongo3

    mongo1:
        hostname: 'mongo1'
        container_name: 'mongo1'
        image: 'mongo:4.2'
        restart: 'on-failure'
        command: ["-f", "/etc/mongod.conf", "--keyFile", "/auth/file.key", "--replSet", "${MONGO_REPLICA_SET_NAME}", "--bind_ip_all"]
        expose: 
            - 27017
        ports: 
            - 30001:27017 
        networks: 
            - netApplication
        volumes:
            - dataMongo1:/data/db
            - logMongo1:/var/log/mongodb
            - ./.docker/mongodb/initdb.d/:/docker-entrypoint-initdb.d/
            - ./.docker/mongodb/mongod.conf:/etc/mongod.conf
            - ./.docker/mongodb/file.key:/auth/file.key
        healthcheck:
            test: test $$(echo "rs.status().ok" | mongo -u $${MONGO_INITDB_ROOT_USERNAME} -p $${MONGO_INITDB_ROOT_PASSWORD} --quiet) -eq 1
            interval: 30s
            start_period: 60s
        env_file:
            - .env
        environment:
            MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
            MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
            MONGO_INITDB_DATABASE: ${MONGO_INITDB_DATABASE}

    mongo2:
        hostname: 'mongo2'
        container_name: 'mongo2'
        image: 'mongo:4.2'
        command: ["-f", "/etc/mongod.conf", "--keyFile", "/auth/file.key", "--replSet", "${MONGO_REPLICA_SET_NAME}", "--bind_ip_all"]
        restart: 'on-failure'
        expose: 
            - 27017
        ports: 
            - 30002:27017  
        networks: 
            - netApplication
        volumes:
            - dataMongo2:/data/db
            - logMongo2:/var/log/mongodb
            - ./.docker/mongodb/mongod.conf:/etc/mongod.conf
            - ./.docker/mongodb/file.key:/auth/file.key
        env_file:
            - .env
        environment:
            MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
            MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
            MONGO_INITDB_DATABASE: ${MONGO_INITDB_DATABASE}
        depends_on: 
            - mongo1

    mongo3:
        hostname: 'mongo3'
        container_name: 'mongo3'
        image: 'mongo:4.2'
        command: ["-f", "/etc/mongod.conf", "--keyFile", "/auth/file.key", "--replSet", "${MONGO_REPLICA_SET_NAME}", "--bind_ip_all"]
        restart: 'on-failure'
        expose: 
            - 27017
        ports: 
            - 30003:27017  
        networks: 
            - netApplication
        volumes:
            - dataMongo3:/data/db
            - logMongo3:/var/log/mongodb
            - ./.docker/mongodb/mongod.conf:/etc/mongod.conf
            - ./.docker/mongodb/file.key:/auth/file.key
        env_file:
            - .env
        environment:
            MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
            MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
            MONGO_INITDB_DATABASE: ${MONGO_INITDB_DATABASE}
        depends_on: 
            - mongo1

volumes: 
    dataMongo1:
    dataMongo2:
    dataMongo3:
    logMongo1:
    logMongo2:
    logMongo3:

networks: 
    netApplication:
```

## The Setup

![Replica set nodes](https://dev-to-uploads.s3.amazonaws.com/i/wibnhlmmmfqi7gfd0atf.jpg)

Here we are trying to setup a MongoDB replica set with 3 nodes.  
One of these nodes will be the **Primary** node - all writes to the database will happen via this node.  
The remaining two, are **Secondary** nodes, providing two-levels of data replication in the setup.  
A solid explanation of how data replication works in MongoDB can be found [at this link](https://docs.mongodb.com/manual/replication/#replication-in-mongodb).  

## Exploring the docker-compose file  

Coming back to the `yaml` file, let's go through the service declaration of `mongo1` and see what we are trying to achieve.

### The basic stuff

```yml
mongo1:
    hostname: 'mongo1'
    container_name: 'mongo1'
    image: 'mongo:4.2'
    restart: 'on-failure'
    :
    expose: 
        - 27017
    ports: 
        - 30001:27017 
    networks: 
        - netApplication
```  

Nothing fancy here, we are targetting MongoDB server version 4.2 to run these containers.   
We have declared hostname and container name, and set the container [restart policy](https://docs.docker.com/config/containers/start-containers-automatically/) to `on-failure`.  
On the ports front, we have exposed `27017` to other docker containers running on the same docker network. Also, this port is mapped to host port `30001`.  
Lastly, we will be running our replica set on a named network.  

### Starting the mongod instance

```yml
command: ["-f", "/etc/mongod.conf", "--keyFile", "/auth/file.key", "--replSet", "${MONGO_REPLICA_SET_NAME}", "--bind_ip_all"]
```

This will start the `mongod` service with the following:
+ `-f /etc/mongod.conf`  
   - specifies that the runtime configuration options should be picked up from the provided `mongod.conf` file.  
   - we will see the contents of this file later
+ `-keyFile /auth/file.key`  
   - In our replica set, the nodes will use the contents of a shared keyfile to authenticate to each other.  
   - Here we are specifying the path to the keyfile
+ `--replSet ${MONGO_REPLICA_SET_NAME}`  
   - This configures the node to run in a replica set.  
   - We are also specifying the name of the replica set, which is to be picked up from the environment variable `${MONGO_REPLICA_SET_NAME}`  
+ `--bind_ip_all`  
   - This specifies the `mongod` instance to bind to all IPv4 addresses (0.0.0.0).  

### Container volumes

Before proceeding it is worth mentioning again, that I have created [this Github repository](https://github.com/sntnupl/devcontainers-mongodb-replica-set-with-docker) to house all the artifacts mentioned here.

```yml
volumes:
    - mongoData1:/data/db
    - mongoLog1:/var/log/mongodb
    - ./.docker/mongodb/initdb.d/:/docker-entrypoint-initdb.d/
    - ./.docker/mongodb/mongod.conf:/etc/mongod.conf
    - ./.docker/mongodb/file.key:/auth/file.key
```

Here we have setup a few [Docker managed volumes](https://docs.docker.com/storage/volumes/) to store the container's data in host's file system.  
+ `mongoData1`: This will act as the peristent store for MongoDB's data.  
+ `mongoLog1`: All of MongoDB's logs for this container will be stored here.  
+ `./.docker/mongodb/initdb.d/`
   - When `mongod` container starts for the *very first time*, it will look for a directory named `/docker-entrypoint-initdb.d`, and execute all files with extensions `.sh` and `.js`.  
   - We map this directory to `./.docker/mongodb/initdb.d/` in host. 
   - As we will see later, we will use this directory to store a bash script that allows us to create a user.  
+ `./.docker/mongodb/mongod.conf`  
   - We are essentially providing the path (in host) from where MongoDB will pick up its config file.  
+ `./.docker/mongodb/file.key`  
   - Like above volume, here we are specifying the keyfile that this MongoDB node will be using to authenticate itself to other nodes in the replica set.  
   - Steps to create this keyfile is provided in [the Github repository](https://github.com/sntnupl/devcontainers-mongodb-replica-set-with-docker) that I have created.
  


```yml
env_file:
    - .env
environment:
    MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
    MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
    MONGO_INITDB_DATABASE: ${MONGO_INITDB_DATABASE}
```

Here we have specified that the docker compose will be using the `.env` file to read environment variables and inject them into the container.  
This allows us to specify that the database's root username/password and initial database name to be picked up from specific environment variables.  
In my [github repository](https://github.com/sntnupl/devcontainers-mongodb-replica-set-with-docker), I have provided a sample env file that you can use to create your own `.env` file.  
Contents of the sample env file looks like this:  
```
# MongoDB
MONGO_URL=mongodb://mongodb:27017
MONGO_INITDB_ROOT_USERNAME=<root_username>
MONGO_INITDB_ROOT_PASSWORD=<root_password>
MONGO_INITDB_DATABASE=<app_db-name>
MONGO_INITDB_USERNAME=<app_username>
MONGO_INITDB_PASSWORD=<app_password>
MONGO_REPLICA_SET_NAME=rs0
```

One must make sure not to commit this file in github, as these contains secrests like root username/password.  

![Tree view of files and directories](https://dev-to-uploads.s3.amazonaws.com/i/219a3dpztqttnvlhsakg.jpg)

The above image provides a tree view of the files and directories discussed here.  

### Health Check

```yml
healthcheck:
    test: test $$(echo "rs.status().ok" | mongo -u $${MONGO_INITDB_ROOT_USERNAME} -p $${MONGO_INITDB_ROOT_PASSWORD} --quiet) -eq 1
    interval: 30s
    start_period: 60s
```

Here we are specifying the healthcheck command that docker will use to check if the container is up and running.  

To summarize, the `mongo1` service's specification looks like this:

```yml
mongo1:
    hostname: 'mongo1'
    container_name: 'mongo1'
    image: 'mongo:4.2'
    restart: 'on-failure'
    command: ["-f", "/etc/mongod.conf", "--keyFile", "/auth/file.key", "--replSet", "${MONGO_REPLICA_SET_NAME}", "--bind_ip_all"]
    expose: 
        - 27017
    ports: 
        - 30001:27017 
    networks: 
        - netApplication
    volumes:
        - mongoData1:/data/db
        - mongoLog1:/var/log/mongodb
        - ./.docker/mongodb/initdb.d/:/docker-entrypoint-initdb.d/
        - ./.docker/mongodb/mongod.conf:/etc/mongod.conf
        - ./.docker/mongodb/file.key:/auth/file.key
    healthcheck:
        test: test $$(echo "rs.status().ok" | mongo -u $${MONGO_INITDB_ROOT_USERNAME} -p $${MONGO_INITDB_ROOT_PASSWORD} --quiet) -eq 1
        interval: 30s
        start_period: 60s
    env_file:
        - .env
    environment:
        MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
        MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
        MONGO_INITDB_DATABASE: ${MONGO_INITDB_DATABASE}
```

The other two nodes - `mongo1` and `mongo2` is a subset of this specification. The only things missing are the healthcheck section, and a volume that maps to `docker-entrypoint-initdb.d`. This is because we have designed this setup such that `mongo1` will become the primary and the rest secondary.  
Being secondary nodes, they dont need the `initdb` part to work.  


## Create the Mongo Cluster

Once all the nodes in the replica set is up, we will need to initiate Replica set configuration on these nodes.  
We will be using a 4th container - `mongo-replica-setup` to do this.  

```yml
mongo-replica-setup:
    container_name: mongo-setup
    image: 'mongo:4.2'
    restart: on-failure
    networks:
        - netApplication
    volumes:
    - ./.docker/mongodb/scripts/mongosetup.sh:/scripts/mongosetup.sh
    entrypoint: ["bash", "/scripts/mongosetup.sh" ]
    env_file:
        - .env
    environment:
        MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
        MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
    depends_on:
        - mongo1
        - mongo2
        - mongo3
```

Once this container starts, it will connect to `mongo1`, and execute a script that will initiate a replica set over `mongo1`, `mongo2` and `mongo3`.  
Once the script is executed this container exits.  

Its important to note that we have set the `depends_on` field of this container to `mongo1`, `mongo2` and `mongo3`, which ensures that this container will start only after all those 3 containers are up. This is important, as you wont want to run the script without all the nodes of the replica set being up.  

The following is the script that this container runs, to initiate the replica set:  
```sh
#!/bin/bash

MONGODB1=mongo1
MONGODB2=mongo2
MONGODB3=mongo3

echo "**********************************************" ${MONGODB1}
echo "Waiting for startup.."
sleep 30
echo "done"

echo SETUP.sh time now: `date +"%T" `
mongo --host ${MONGODB1}:27017 -u ${MONGO_INITDB_ROOT_USERNAME} -p ${MONGO_INITDB_ROOT_PASSWORD} <<EOF
var cfg = {
    "_id": "rs0",
    "protocolVersion": 1,
    "version": 1,
    "members": [
        {
            "_id": 0,
            "host": "${MONGODB1}:27017",
            "priority": 2
        },
        {
            "_id": 1,
            "host": "${MONGODB2}:27017",
            "priority": 0
        },
        {
            "_id": 2,
            "host": "${MONGODB3}:27017",
            "priority": 0,
        }
    ]
};
rs.initiate(cfg, { force: true });
rs.secondaryOk();
db.getMongo().setReadPref('primary');
rs.status();
EOF
```

If you look closely, we have set the `priority` for `mongo1` higher than the other two nodes, which ensures that initially, `mongo1` will be acting as the Primary node.  

## Final Result

To start the replica set, run `docker-compose up -d` from command line.  
Once everything comes up, you can run `docker-compose status`, and see something like this:  

![Replica set status](https://dev-to-uploads.s3.amazonaws.com/i/wn7vr5y9k1ftxqzjlwez.jpg)

To connect to the replica set, you can use mongo client like so:   
```sh
$ mongo "mongodb://localhost:30001,localhost:30002,localhost:30003/<MONGO_INITDB_DATABASE>" -u <MONGO_INITDB_USERNAME>
```  

![Connect to Replica set](https://dev-to-uploads.s3.amazonaws.com/i/j7sca0rmvukrxpz4so9f.jpg)