MongoDB Cluster Setup Centos
Tags: mongodb, centos, opensource, linux
Published: 2020-08-10 14:19:12

We will be setting up Mongo DB for high availability using 3 virtual machines 

1) Mongo Primary (t3a.medium)
2) Mongo Secondary (t3a.medium)
3) Mongo Arbiter (t3a.small)

##Let's begin the system preparations

###The following setup has to be done in all 3 machines

* login as sudo in each machine

```
sudo --login 
```

*  Update centos packages to the latest version
Centos 7
```
yum update -y
```
Centos 8
```
dnf update -y
```

* install Nano editor (I am a Nano fan)
Centos 7
```
yum install nano -y
```
Centos 8
```
dnf install nano -y
```
* add hosts values 
edit the `nano /etc/hosts ` file to add the following entries

```
172.31.28.16 mongo1
172.31.31.194 mongo2
172.31.25.65 mongo3
```
## important Step
Disable SELinux as it has breaking effects  on internode connection
 edit ` /etc/selinux/config`
change 
```
SELINUX=disabled
```
do the above step in all 3 machines and make sure to do a reboot by running 
```
reboot
```

## Now Install Mongo

* Configure the package management system

Create a `nano /etc/yum.repos.d/mongodb-org-4.4.repo` file so that you can install MongoDB directly using yum

Add the following content to it.

```
[mongodb-org-4.4]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/4.4/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-4.4.asc
```

* Install Mongo DB latest Stable
Centos 7

```
yum install -y mongodb-org
```
Centos 8
```
dnf install -y mongodb-org
```
Before we can begin cluster Setup lets prepare mongo

### Enable Mongo Authentication

To add authentication to mongo add the following lines to 
` nano /etc/mongod.conf `
```
security:
    authorization: enabled
```
Then do a restart 

```
service mongod restart
```

* let's create mongodb user
Open mongo shell

` mongo`

the following commands
```
use admin
```
```
db.createUser({
    user: "tomahawk",
    pwd: "tomahawkPilot",
    roles: [ 
        { 
            role: "userAdminAnyDatabase", 
            db: "admin" 
        },
        {
            role: "clusterAdmin",
             db: "admin"
        },
        {
            role: "root,
            db: "admin
        }
    ]
})
```


## Now let's Begin the Replication Configuration

* Add replSet to mongod.conf in all 3 machines
edit your `nano /etc/mongod.conf` file 
```
replication:
  replSetName: rs0
```

## Keyfile Access Control on Replica Set 
navigate to your mongo path
```
/var/lib/mongo in my case
```
Use the following command to generate a keyfile
```
openssl rand -base64 756 > keyfile
```
Copy the key file to all 3 machines
add Appropriate permissions
```
chmod 400 keyfile
chown mongod:mongod keyfile
```
Now add the keyfile to your `nano /etc/mongod.conf` file

```
security:
    authorization: enabled
    keyFile: /var/lib/mongo/keyfile
```
* do a restart of all mongod services
```
systemctl restart mongod
```
for the changes in replication to take effect before initializing the replica

* On mongo1
```
use admin
db.auth("tomahawk","tomahawkPilot");
rs.initiate(
   {
      _id: "rs0",
      version: 1,
      members: [
         { _id: 0, host : "mongo1:27017" }
      ]
   }
)
```

You can always run 
```
mongo --port 27017 -u tomahawk --authenticationDatabase 'admin' -p tomahawkPilot
```
to directly login to the cluster


Now let's create the replica set
```
rs.add("mongo2:27017");
```
Check connection status by running
```
rs.status()
```

On Successful connection, you will get a response 
```
rs.status()
{
        "set" : "rs0",
        "date" : ISODate("2020-08-10T14:03:42.405Z"),
        "myState" : 1,
        "term" : NumberLong(2),
        "syncSourceHost" : "",
        "syncSourceId" : -1,
        "heartbeatIntervalMillis" : NumberLong(2000),
        "majorityVoteCount" : 2,
        "writeMajorityCount" : 2,
        "votingMembersCount" : 2,
        "writableVotingMembersCount" : 2,
        "optimes" : {
                "lastCommittedOpTime" : {
                        "ts" : Timestamp(1597068221, 2),
                        "t" : NumberLong(2)
                },
                "lastCommittedWallTime" : ISODate("2020-08-10T14:03:41.727Z"),
                "readConcernMajorityOpTime" : {
                        "ts" : Timestamp(1597068221, 2),
                        "t" : NumberLong(2)
                },
                "readConcernMajorityWallTime" : ISODate("2020-08-10T14:03:41.727Z"),
                "appliedOpTime" : {
                        "ts" : Timestamp(1597068221, 2),
                        "t" : NumberLong(2)
                },
                "durableOpTime" : {
                        "ts" : Timestamp(1597068221, 2),
                        "t" : NumberLong(2)
                },
                "lastAppliedWallTime" : ISODate("2020-08-10T14:03:41.727Z"),
                "lastDurableWallTime" : ISODate("2020-08-10T14:03:41.727Z")
        },
        "lastStableRecoveryTimestamp" : Timestamp(1597066170, 1),
        "electionCandidateMetrics" : {
                "lastElectionReason" : "electionTimeout",
                "lastElectionDate" : ISODate("2020-08-10T14:03:41.721Z"),
                "electionTerm" : NumberLong(2),
                "lastCommittedOpTimeAtElection" : {
                        "ts" : Timestamp(0, 0),
                        "t" : NumberLong(-1)
                },
                "lastSeenOpTimeAtElection" : {
                        "ts" : Timestamp(1597066987, 1),
                        "t" : NumberLong(1)
                },
                "numVotesNeeded" : 2,
                "priorityAtElection" : 1,
                "electionTimeoutMillis" : NumberLong(10000),
                "numCatchUpOps" : NumberLong(0),
                "newTermStartDate" : ISODate("2020-08-10T14:03:41.727Z"),
                "wMajorityWriteAvailabilityDate" : ISODate("2020-08-10T14:03:41.898Z")
        },
        "members" : [
                {
                        "_id" : 0,
                        "name" : "mongo1:27017",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 454,
                        "optime" : {
                                "ts" : Timestamp(1597068221, 2),
                                "t" : NumberLong(2)
                        },
                        "optimeDate" : ISODate("2020-08-10T14:03:41Z"),
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "electionTime" : Timestamp(1597068221, 1),
                        "electionDate" : ISODate("2020-08-10T14:03:41Z"),
                        "configVersion" : 2,
                        "configTerm" : 2,
                        "self" : true,
                        "lastHeartbeatMessage" : ""
                },
                {
                        "_id" : 1,
                        "name" : "mongo2:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 7,
                        "optime" : {
                                "ts" : Timestamp(1597066987, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1597066987, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2020-08-10T13:43:07Z"),
                        "optimeDurableDate" : ISODate("2020-08-10T13:43:07Z"),
                        "lastHeartbeat" : ISODate("2020-08-10T14:03:41.727Z"),
                        "lastHeartbeatRecv" : ISODate("2020-08-10T14:03:42.261Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "configVersion" : 2,
                        "configTerm" : 1
                }
        ],
        "ok" : 1,
        "$clusterTime" : {
                "clusterTime" : Timestamp(1597068221, 2),
                "signature" : {
                        "hash" : BinData(0,"17CdXYPWQ6I24TCLhxpQt8nGGPk="),
                        "keyId" : NumberLong("6859346398467325956")
                }
        },
        "operationTime" : Timestamp(1597068221, 2)
}
rs0:PRIMARY> 
```

# Setting up the Arbiter
```
rs.addArb("mongo3:27017");
```
```
{
        "ok" : 1,
        "$clusterTime" : {
                "clusterTime" : Timestamp(1597068567, 1),
                "signature" : {
                        "hash" : BinData(0,"q5F6v883q8+1gBfmEJtwINbXAYY="),
                        "keyId" : NumberLong("6859346398467325956")
                }
        },
        "operationTime" : Timestamp(1597068567, 1)
}
```

Now check connection status by running
```
rs.status()
```
rs.status()
```
rs0:PRIMARY> rs.status()
{
        "set" : "rs0",
        "date" : ISODate("2020-08-10T14:10:18.328Z"),
        "myState" : 1,
        "term" : NumberLong(2),
        "syncSourceHost" : "",
        "syncSourceId" : -1,
        "heartbeatIntervalMillis" : NumberLong(2000),
        "majorityVoteCount" : 2,
        "writeMajorityCount" : 2,
        "votingMembersCount" : 3,
        "writableVotingMembersCount" : 2,
        "optimes" : {
                "lastCommittedOpTime" : {
                        "ts" : Timestamp(1597068611, 1),
                        "t" : NumberLong(2)
                },
                "lastCommittedWallTime" : ISODate("2020-08-10T14:10:11.737Z"),
                "readConcernMajorityOpTime" : {
                        "ts" : Timestamp(1597068611, 1),
                        "t" : NumberLong(2)
                },
                "readConcernMajorityWallTime" : ISODate("2020-08-10T14:10:11.737Z"),
                "appliedOpTime" : {
                        "ts" : Timestamp(1597068611, 1),
                        "t" : NumberLong(2)
                },
                "durableOpTime" : {
                        "ts" : Timestamp(1597068611, 1),
                        "t" : NumberLong(2)
                },
                "lastAppliedWallTime" : ISODate("2020-08-10T14:10:11.737Z"),
                "lastDurableWallTime" : ISODate("2020-08-10T14:10:11.737Z")
        },
        "lastStableRecoveryTimestamp" : Timestamp(1597068601, 1),
        "electionCandidateMetrics" : {
                "lastElectionReason" : "electionTimeout",
                "lastElectionDate" : ISODate("2020-08-10T14:03:41.721Z"),
                "electionTerm" : NumberLong(2),
                "lastCommittedOpTimeAtElection" : {
                        "ts" : Timestamp(0, 0),
                        "t" : NumberLong(-1)
                },
                "lastSeenOpTimeAtElection" : {
                        "ts" : Timestamp(1597066987, 1),
                        "t" : NumberLong(1)
                },
                "numVotesNeeded" : 2,
                "priorityAtElection" : 1,
                "electionTimeoutMillis" : NumberLong(10000),
                "numCatchUpOps" : NumberLong(0),
                "newTermStartDate" : ISODate("2020-08-10T14:03:41.727Z"),
                "wMajorityWriteAvailabilityDate" : ISODate("2020-08-10T14:03:41.898Z")
        },
        "members" : [
                {
                        "_id" : 0,
                        "name" : "mongo1:27017",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 850,
                        "optime" : {
                                "ts" : Timestamp(1597068611, 1),
                                "t" : NumberLong(2)
                        },
                        "optimeDate" : ISODate("2020-08-10T14:10:11Z"),
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "electionTime" : Timestamp(1597068221, 1),
                        "electionDate" : ISODate("2020-08-10T14:03:41Z"),
                        "configVersion" : 3,
                        "configTerm" : 2,
                        "self" : true,
                        "lastHeartbeatMessage" : ""
                },
                {
                        "_id" : 1,
                        "name" : "mongo2:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 403,
                        "optime" : {
                                "ts" : Timestamp(1597068611, 1),
                                "t" : NumberLong(2)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1597068611, 1),
                                "t" : NumberLong(2)
                        },
                        "optimeDate" : ISODate("2020-08-10T14:10:11Z"),
                        "optimeDurableDate" : ISODate("2020-08-10T14:10:11Z"),
                        "lastHeartbeat" : ISODate("2020-08-10T14:10:17.346Z"),
                        "lastHeartbeatRecv" : ISODate("2020-08-10T14:10:17.389Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncSourceHost" : "mongo1:27017",
                        "syncSourceId" : 0,
                        "infoMessage" : "",
                        "configVersion" : 3,
                        "configTerm" : 2
                },
                {
                        "_id" : 2,
                        "name" : "mongo3:27017",
                        "health" : 1,
                        "state" : 7,
                        "stateStr" : "ARBITER",
                        "uptime" : 50,
                        "lastHeartbeat" : ISODate("2020-08-10T14:10:17.351Z"),
                        "lastHeartbeatRecv" : ISODate("2020-08-10T14:10:17.471Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "configVersion" : 3,
                        "configTerm" : 2
                }
        ],
        "ok" : 1,
        "$clusterTime" : {
                "clusterTime" : Timestamp(1597068611, 1),
                "signature" : {
                        "hash" : BinData(0,"Pir6qAUNxIu6/AbFv6fCxpVVwOs="),
                        "keyId" : NumberLong("6859346398467325956")
                }
        },
        "operationTime" : Timestamp(1597068611, 1)
}
```

now run 
``` 
rs.conf()
```
to see your config
```
rs0:PRIMARY> rs.conf()
{
        "_id" : "rs0",
        "version" : 3,
        "term" : 2,
        "protocolVersion" : NumberLong(1),
        "writeConcernMajorityJournalDefault" : true,
        "members" : [
                {
                        "_id" : 0,
                        "host" : "mongo1:27017",
                        "arbiterOnly" : false,
                        "buildIndexes" : true,
                        "hidden" : false,
                        "priority" : 1,
                        "tags" : {

                        },
                        "slaveDelay" : NumberLong(0),
                        "votes" : 1
                },
                {
                        "_id" : 1,
                        "host" : "mongo2:27017",
                        "arbiterOnly" : false,
                        "buildIndexes" : true,
                        "hidden" : false,
                        "priority" : 1,
                        "tags" : {

                        },
                        "slaveDelay" : NumberLong(0),
                        "votes" : 1
                },
                {
                        "_id" : 2,
                        "host" : "mongo3:27017",
                        "arbiterOnly" : true,
                        "buildIndexes" : true,
                        "hidden" : false,
                        "priority" : 0,
                        "tags" : {

                        },
                        "slaveDelay" : NumberLong(0),
                        "votes" : 1
                }
        ],
        "settings" : {
                "chainingAllowed" : true,
                "heartbeatIntervalMillis" : 2000,
                "heartbeatTimeoutSecs" : 10,
                "electionTimeoutMillis" : 10000,
                "catchUpTimeoutMillis" : -1,
                "catchUpTakeoverDelayMillis" : 30000,
                "getLastErrorModes" : {

                },
                "getLastErrorDefaults" : {
                        "w" : 1,
                        "wtimeout" : 0
                },
                "replicaSetId" : ObjectId("5f314b35458f8460ab22cda1")
        }
}
```
Have an awesome day ahead