How to easily create a cluster with Docker Swarm to deploy applications on a large scale ?
By: Emmanuel Hodonou
Tags: Engineering, Orchestration, Docker, DevOps, Swarm

Docker container technology was launched in 2013 as an open source Docker Engine. It leveraged existing computing concepts around containers and specifically in the Linux world, primitives known as cgroups and namespaces. Docker’s technology is unique because it focuses on the requirements of developers and systems operators to separate application dependencies from infrastructure.

Docker swarm is a quite addition to Docker. It’s designed to easily manage container scheduling over multiple hosts.

Main point: It allows to connect multiple hosts with Docker together .

. It’s relatively simple . Compared with others solutions, starting with Docker Swarm is really easy.

. Compared with others solutions, starting with Docker Swarm is really easy. High availability — there are two node types in cluster: master and worker. One of masters is the leader. If current leader fails, other master will become leader. If worker host fails, all containers will be rescheduled to other nodes.

— there are two node types in cluster: master and worker. One of masters is the leader. If current leader fails, other master will become leader. If worker host fails, all containers will be rescheduled to other nodes. Declarative configuration . You tell what you want, how many replicas, and they’ll be automatically scheduled with respect to given constraints.

. You tell what you want, how many replicas, and they’ll be automatically scheduled with respect to given constraints. Rolling updates — Swarm stores configuration for containers. If you update configuration, containers are updated in batches, so service by default will be available all the time.

— Swarm stores configuration for containers. If you update configuration, containers are updated in batches, so service by default will be available all the time. Build-in Service discovery and Load balancing — similar to load balancing done by Docker-Compose. You can reference other services using their names, it doesn’t matter where containers are stored, they will receive requests in a round-robin fashion.

— similar to load balancing done by Docker-Compose. You can reference other services using their names, it doesn’t matter where containers are stored, they will receive requests in a round-robin fashion. Overlay network — if you expose a port from a service, it’ll be available on any node in cluster. It really helps with external load balancing.

What will we see in this article ?

We will form a cluster of three machines with Docker Swarm

Pre-conditions:

— Machines with linus installed. In the case of this example. I used the debian distribution.

— Docker installed on each machine.

— Machines are seeing between them and can send and received packets.

What you should know before starting to setup a swarm ?

— In you cluster, you can have one, two, three or several servers.

— Each machine is called a node in your cluster.

— To take advantage of swarm mode’s fault-tolerance features, Docker recommends you implement an odd number of nodes according to your organization’s high-availability requirements

— To set up a swarm, you need to have managers nodes/machines and workers nodes/machines.

The manager nodes handle cluster management tasks:

maintaining cluster state

scheduling services

serving swarm mode HTTP API endpoints

The Worker nodes don’t participate in the Raft distributed state, make scheduling decisions, or serve the swarm mode HTTP API.

— An N manager cluster tolerates the loss of at most (N-1)/2 managers.

Example: A three-manager swarm tolerates a maximum loss of one manager.

— You can create a swarm of one manager node, but you cannot have a worker node without at least one manager node. By default, all managers are also workers. In a single manager node cluster, you can run commands like docker service create and the scheduler places all tasks on the local Engine.

Form a cluster of machines.

For this example, i am using 3 machines.

See bellow my machines roles and IP(All the three machines will be managers):

My manager one has the IP 10.10.25.165 with the hostname serveur17

My manager two has the IP 10.10.25.166 with the hostname serveur18

My manager three has the IP 10.10.25.167 with the hostname serveur19

a- Open three terminal sessions and ssh on the three servers.

ssh root@machine_IP and enter password.

On terminal 1

$ ssh root@10.10.25.165

On terminal 2

$ ssh root@10.10.25.166

On terminal 3

$ ssh root@10.10.25.167

b- Set up your machines hosts

Repeat this operation on 3 machines:

— Edit host file on each server

$ vim /etc/hosts

— Add these 3 lines in the host file of each machine and save it

10.10.25.165 dockernode1

10.10.25.166 dockernode2

10.10.25.167 dockernode3

c- Ping machines between them to see packets sent and well received

From dockernode1 machine (10.10.25.165), ping others managers dockernode2 and dockernode3

$ ping dockernode2

$ ping dockernode3

From dockernode2 machine (10.10.25.166), ping others managers dockernode1 and dockernode3

$ ping dockernode1

$ ping dockernode3

From dockernode3 machine (10.10.25.167), ping others managers dockernode1 and dockernode2

$ ping dockernode1

$ ping dockernode3

Possible issues: If packets are not received, please check your firewall settings.

d- Initialize docker swarm and let’s form the cluster

Go on the dockernode1 machine (10.10.25.165) to initialize our swarm

$ docker swarm init --advertise-addr 10.10.25.165

You will see some sort of following output:

Swarm initialized: current node (0knoujuvtkoq1pg3mhjhxsbhn) is now a manager. To add a worker to this swarm, run the following command: docker swarm join — token SWMTKN-1–3g85y47bq6w7frwqq7gnpxqg85klk02escd5n5i180eo6yiwb1-e46nzdj2oh4yc62p66rogdj7h 10.10.25.165:2377 To add a manager to this swarm, run ‘docker swarm join-token manager’ and follow the instructions.

After initializing of our swarm, always on dockernode1 machine, run this command:

$ docker swarm join-token manager

You will see some sort of following output: To add a manager to this swarm, run the following command: docker swarm join — token SWMTKN-1–3g85y47bq6w7frwqq7gnpxqg85klk02escd5n5i180eo6yiwb1–4wes668cwnawczst9v32736g6 10.10.25.165:2377

Please copy the output command from this confirmation text. The command will be used to add our other servers to the swarm.

Go on the dockernode2 machine (10.10.25.166) to add it to our swarm

To add the dockernode2 machine to the swarm as manager, follow the instructions after you run this command:

$ docker swarm join --token SWMTKN-1-3g85y47bq6w7frwqq7gnpxqg85klk02escd5n5i180eo6yiwb1-4wes668cwnawczst9v32736g6 10.10.25.165:2377

You will see some sort of following output: This node joined a swarm as a manager.

The output tells you the dockernode2 is now a manager.

Go on the dockernode3 machine (10.10.25.167) to add it to our swarm

To add the dockernode3 machine to the swarm as manager, follow the instructions after you run this command:

$ docker swarm join --token SWMTKN-1-3g85y47bq6w7frwqq7gnpxqg85klk02escd5n5i180eo6yiwb1-4wes668cwnawczst9v32736g6 10.10.25.165:2377

The output tells you the node is now a manager.

You will see some sort of following output: This node joined a swarm as a manager.

The output tells you the dockernode3 is now a manager.

f- Check nodes status

$ docker node ls

You will see some sort of following output:

Our cluster is ready…

We could have added workers instead of managers after the initialization of the swarm.To do this, you must copy from the confirmation text of the swarm initialization the command indicating how to add workers. In my case, the confirmation text says:

To add a worker to this swarm, run the following command: docker swarm join — token SWMTKN-1–3g85y47bq6w7frwqq7gnpxqg85klk02escd5n5i180eo6yiwb1-e46nzdj2oh4yc62p66rogdj7h 10.10.25.165:2377

Thanks for reading. You can add comments if necessary to leave your appreciation. I will take your comments into consideration to improve this post.