Building a Kubernetes cluster out of 3 Raspberry Pis
Tags: kubernetes, raspberrypi, flannel
Published: 2020-02-10 04:57:36

Based off of the great the post https://kubecloud.io/setting-up-a-kubernetes-1-11-raspberry-pi-cluster-using-kubeadm-952bbda329c8
and the YouTube video https://www.youtube.com/watch?v=2YoK4bBy3CM, for networking.

## Hardware
For the hardware
1. [3 RaspberryPi 4B, with 4GB RAM](https://www.microcenter.com/product/609038/raspberry-pi-4-model-b---4gb-ddr4)
2. [Dorhea Raspberry Pi 4 B Case](https://www.amazon.com/gp/product/B07XRF4GRS/ref=ppx_yo_dt_b_asin_title_o02_s00?ie=UTF8&psc=1)
3. [A set of aluminum heatsinks](https://www.amazon.com/gp/product/B07YR6M6F6/ref=ppx_yo_dt_b_asin_title_o00_s00?ie=UTF8&psc=1)
4. [TP-LINK 8-port Gigabit Desktop Switch](https://www.microcenter.com/product/414582/tp-link-tl-sg108-8-port-10-100-1000-gigabit-desktop-switch)
4. [A set of Cat6 Ethernet Cables](https://www.amazon.com/gp/product/B0721RFHT8/ref=ppx_yo_dt_b_asin_title_o01_s00?ie=UTF8&psc=1)
5. [Set of USB C charging cables](https://www.amazon.com/gp/product/B01N32UM0Q/ref=ppx_yo_dt_b_asin_title_o01_s00?ie=UTF8&psc=1)
6. [Anker 60W 6-Port USB wall charger](https://www.amazon.com/gp/product/B00P933OJC/ref=ppx_yo_dt_b_asin_title_o01_s01?ie=UTF8&psc=1) 
7. [Set of Samsung 32GB micro SDHC cards](https://www.amazon.com/gp/product/B07NP96DX5/ref=ppx_yo_dt_b_asin_title_o01_s02?ie=UTF8&psc=1)


## Setting up the Raspberry Pis

In this setup I am using 3 Raspberry Pi 4s, but other versions of the Pis should work as well.  To set them up:
1. Flash 3 SD Cards with the latest Raspian images.
2. After flashing, remount the SD Card and in the /boot directory of the SD Card add an empty file named `ssh`.  This will enable ssh on the Pi running off of this image.  
3. Unmount the SD Card and insert it into a Pi.

## Network and Kubeadm Installs

Connect the Pis to a switch and the switch to your local network.  When the Pi boots, it can be reached at `ssh pi@raspberrypi.local` with the password `raspberry`.  Now since all three Pis start with the same DNS name, I would power on one, updated its network settings, and rebooted it.  After it is up and running with it's own DNS name, I could then plugin the next Pi and repeated the process.  Stepping through Pis this way keeps only one Pi with the domain name raspberrypi.local on the network at a time.


### Network Setup

We start by updating the Pi's `/etc/hosts` and `/etc/dhcpcd.conf`.  To do this, I used the script below, passing in the hostname and last octet of the IP address as the parameters.


```
#!/bin/sh
#
# set_host.sh
#
# Usage: ./set_host.sh <hostname> <last octet>
#  ./set_host.sh k8master 223
# for example
#

hostname=$1
ip=$2
dns=192.168.1.1

sudo -- sh -c "cat <<EOT >> /etc/hosts
192.168.1.223   k8master
192.168.1.224   k8worker1
192.168.1.225   k8worker2
EOT"


sudo cat <<EOT >> /etc/dhcpcd.conf
interface eth0
static ip_address=192.168.1.${ip}/24
static routers=${dns}
static domain_name_servers=${dns}
EOT

sudo sed -i s/raspberrypi/${hostname}/g /etc/hosts
sudo hostnamectl --transient set-hostname ${hostname}
sudo hostnamectl --static set-hostname ${hostname}
sudo hostnamectl --pretty set-hostname ${hostname}

```

After running the script, be sure to **reboot** the Pi, for the network changes to take effect.

### Software Install

After the network on the Pi setup, installation of docker and Kubernetes software is next.  Again a shell script was used for installation.

```
#!/bin/sh
#
# Usage: ./set_kube.sh
# InstalLs the docker and kubernetes software
#

# Install Docker
curl -sSL get.docker.com | sh && \
sudo usermod pi -aG docker

# Disable Swap
sudo dphys-swapfile swapoff && \
sudo dphys-swapfile uninstall && \
sudo update-rc.d dphys-swapfile remove
echo Adding " cgroup_enable=cpuset cgroup_enable=memory" to /boot/cmdline.txt
sudo cp /boot/cmdline.txt /boot/cmdline_backup.txt
orig="$(head -n1 /boot/cmdline.txt) cgroup_enable=cpuset cgroup_enable=memory"
echo $orig | sudo tee /boot/cmdline.txt

# Add repo list and install kubeadm
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - && \
echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list && \
sudo apt-get update -q && \
sudo apt-get install -qy kubeadm
```

## Start up

Only on your master node, run kubeadm init, and include the pod-network-cdir parameter, needed for the flannel networking.  The pod-network-cidr needs to be different then the one for your node's network.

```
sudo kubeadm init --pod-network-cidr=10.244.0.0/16
```
As kubeadm runs, it will print out needed information.

Setup for kubectl:

```
  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config
```

Connection command for worker nodes

```
  kubeadm join 192.168.1.223:6443 --token k3mpju.xxxxxxxloccv878b \
    --discovery-token-ca-cert-hash sha256:63225cc63b7189c3cc4c091fff07cad4bxxxxxxxxxx2846f36cd00543ad6f54c
```

*Before adding worker nodes* install the Flannel network.

## Install Flannel network
On the master node, run the command below to install networking.

```
kubectl apply -f https://github.com/coreos/flannel/raw/master/Documentation/kube-flannel.yml
```

## Adding Worker Nodes

Log into each worker node, and using the connection command displayed when the master's kubeadm was executed, add the workers to the cluster.

```
sudo kubeadm join 192.168.1.223:6443 --token k3mpju.xxxxxxxloccv878b \
    --discovery-token-ca-cert-hash sha256:63225cc63b7189c3cc4c091fff07cad4bxxxxxxxxxx2846f36cd00543ad6f54c
```

If you lose the join token, you can recreate it with these commands.
```
kubeadm token generate
kubeadm token create <generated token> --print-join-command --ttl=0

```

## Conclusion
At the end of this process, you should have three nodes Ready along with set of pods ready as well.