Setup kubernetes cluster using kubeadm
Tags: kubernetes, kubeadm, virtualbox
Published: 2018-11-03 06:56:11

---
title: Setup kubernetes cluster using kubeadm
published: true
description: Opinionated kubeadm setup on VMs
tags: #kubernetes,#kubeadm,#virtualbox
cover_image: https://thepracticaldev.s3.amazonaws.com/i/wyx3gl538scmgc5vqe93.jpg
---
[root]: https://thepracticaldev.s3.amazonaws.com/i/kwomwjpa4pay1thg5zqd.png "Switch to root user"
[dockerps]: https://thepracticaldev.s3.amazonaws.com/i/xy6w99f0qe9jp97jtyt5.png "Docker ps"
[dockerpsall]: https://thepracticaldev.s3.amazonaws.com/i/8swpt8056k7eek65qyco.png "Docker ps"
[swap]: https://thepracticaldev.s3.amazonaws.com/i/bisgoywyxdq5gg0w6gal.png "Disable swap"
[init]: https://thepracticaldev.s3.amazonaws.com/i/n0hu2jrhy2cmkjz8yo9p.png "kubeadm init"
[workerjoin]: https://thepracticaldev.s3.amazonaws.com/i/e097gdwx4i58rssq8x3p.png "Worker join result"
[podnetwork]: https://thepracticaldev.s3.amazonaws.com/i/gb116tqc17jfsw9vwy4a.png "Pod network installed"
[getnodes]: https://thepracticaldev.s3.amazonaws.com/i/j51gqqdto5a8wb0j8go6.png "Get nodes result"
[dashboardlogin]: https://thepracticaldev.s3.amazonaws.com/i/qwqtpnfl244pnb5svlpw.png "Dashboard login"
[dashboardoverview]: https://thepracticaldev.s3.amazonaws.com/i/uukarje7wh7y56vijbxk.png "Dashboard overview"


From [previous post](https://dev.to/pongsatt/prepare-virtual-machine-using-virtualbox-and-ubuntu-558j), you prepare a number of VMs machine, in this post, we will install kubeadm and setup it to work together as a cluster.

This is part of [Setup your own kubernetes cluster on VMs](https://dev.to/pongsatt/setup-your-own-kubernetes-cluster-on-vms-5ln) series.

**Note**
> This is an opinionated kubernetes cluster setup so it is easy enough for beginners. Meaning, something have already been chosen for you but you can change it if you want.
> This post is based on [this kubernetes official document](https://kubernetes.io/docs/setup/independent/install-kubeadm/)

##Prerequisite:##
- At least 2 Virtual machines with Ubuntu installed and connected network


#Switch to root user (All machines)#

Run below command and enter root password (password when you setup VM)

```
sudo su
```

![][root]

#Install Docker (All machines)#

Follow these steps to install docker to all machines.

###1. Install Docker###
```
# Install Docker from Ubuntu's repositories:
apt-get update
apt-get install -y docker.io
```

###2. Configure docker###

In case you want to use your own docker private registry, follow below, otherwise follow "Without private registry below"

*Note:* "192.168.1.105:8082" is my docker private registry.

```
# Setup daemon.
cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  
  "insecure-registries": [
    "192.168.1.105:8082"
  ],
  "disable-legacy-registry": true
}
EOF

mkdir -p /etc/systemd/system/docker.service.d

# Restart docker.
systemctl daemon-reload
systemctl restart docker
```


Without private registry.

```
# Setup daemon.
cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF

mkdir -p /etc/systemd/system/docker.service.d

# Restart docker.
systemctl daemon-reload
systemctl restart docker
```

###3. (Optional) Install image cleanup job###

I recommend to install this job to clean up unused docker images otherwise they will consume all your disk.

```
docker run -d --restart=always \
  -v /var/run/docker.sock:/var/run/docker.sock:rw \
  -v /var/lib/docker:/var/lib/docker:rw \
  -e "CLEAN_PERIOD=86400" \
  meltwater/docker-cleanup:latest
```

###4. Check if everything alright###

Run below command to see if everything is ok.

```
docker ps
```

You should see something like below on all machines and you're good to proceed.

![][dockerps]
![][dockerpsall]


#Install Kubeadm (All machines)#

Next step is to install kubernetes cluster tool.

###1. Disable swap###

This is a prerequisite for kubeadm.

```
# disable temporarily
swapoff -a
```

To disable permanently, edit /etc/fstab and command swap line out and save.

```
vi /etc/fstab

```

![][swap]

**Note for vi newby:**
- move cursor to the beginning of last line
- type 'i' to insert
- type '#' then 'esc'
- type ':wq' then enter to save

###2. Run below commands on all machines###

```
apt-get update && apt-get install -y apt-transport-https curl
```

Then
```
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet kubeadm kubectl nfs-common
```

And
```
apt-mark hold kubelet kubeadm kubectl
```

These commands will install:
- kubelet : kubernetes node runtime
- kubeadm : kubernetes cluster setup tool
- kubectl : kubernetes cluster command line interface
- nfs-common: nfs client library to be able to connect to NFS server


#Setup cluster (Master Node only)#

This step, we will initialize master node by running command.

```
# init master
kubeadm init
```

*Note:* Run on master node only

You will need to wait for sometimes depending on your internet connection.

If everything is ok, you will see.

![][init]

**Note:**
Please copy the last line. It will be used in the next step to join worker node.


#Setup cluster (Worker Node only)#

Run join command copied from previous step on all the worker nodes.

Here is an example (Do not use this command, it won't work for you)

```
kubeadm join 192.168.1.109:6443 --token srzyez.wjnqsmt2gcohxtp4 --discovery-token-ca-cert-hash sha256:ac8d21e46aeaba3664c4f6060072de03d906d0032f9731b021eeb8d54a876e35
```

On worker node, if join command succeed, you will see.

![][workerjoin]


#Post install (Master node only)#

###1. Setup cluster connection configuration###

Switch back to normal user.
```
exit
```

Run below command and enter root password.
```
# setup kube config
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
```

###2. Install Pod network (weavenet)###

A kubernetes cluster needs pod network. To install weavenet (one of several pod network providers), run command on master node as below.

```
sudo sysctl net.bridge.bridge-nf-call-iptables=1
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
```

![][podnetwork]


###3. Check the result###

On master node, run command below.

```
kubectl get nodes
```

If you see something like below, congratulation! you have your cluster.

![][getnodes]


#Install Dashboard (Optional)#

To be able to see the overall picture of your cluster, you need a dashboard.


###1. Install dashboard###

On master node, run below command.

```
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml
```

###2. Setup permission to access dashboard###

For simplify thing, we will grant admin permission to anyone access dashboard. You won't do this in the actual cluster.

```
echo "apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: kubernetes-dashboard
  labels:
    k8s-app: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: kubernetes-dashboard
  namespace: kube-system" | kubectl apply -f -
```

###3. Run proxy command###

Open a proxy so that the cluster can be accessed from outside.

```
kubectl proxy --address='0.0.0.0' --accept-hosts='.*'
```

###4. Open dashboard UI###

Open url `https://<your master node ip>:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy`.

You should see login page.

![][dashboardlogin]

Click "Skip" button and you will see overview page.

![][dashboardoverview] 

Congratulation! You've got a dashboard to see and manage your workload.

#Summary#

We've got a kubernetes cluster running on VMs. [Next step](https://dev.to/pongsatt/setup-dynamic-storage-class-provisioner-based-on-nfs-1d4i) will setup cluster storage so we can build application that store something.