Install Rancher K3s on Raspberry Pi Cluster
Tags: kubernetes, rancher, k3s, raspberrypi
Published: 2021-04-05 19:55:23

<img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lmqakct3mglc9iqwj2wt.png" />

<sup><b>Credits:</b> Logo by [cncf-branding](https://cncf-branding.netlify.app/projects/k3s/)</sup>

Install a Rancher Labs Kubernetes distribution (k3s) on a Raspberry Pi cluster.

- [Master Server](#k3s-master-server)
   - [Install](#k3s-master-install) 
   - [Uninstall](#k3s-master-uninstall) 
- [Worker Node](#k3s-worker-node)
   - [Join a Cluster](#k3s-worker-join) 
   - [Uninstall](#k3s-worker-uninstall) 
- [Utilities](#k3s-utilities)
   - [`kubectl`](#k3s-kubectl)
   - [`k9s`](#k3s-k9s)
- [Summary](#summary)

<p>&nbsp;</p>

> **Note:** This post refers to laptop / desktop as ***client machines***. These are the clients used to connect to the Raspberry Pi master / worker nodes remotely. 

<p>&nbsp;</p>

<h3 id="prerequisites">Prerequisites</h3>

- [Setting Up a Raspberry Pi Cluster](https://dev.to/zachinachshon/setting-up-a-raspberry-pi-cluster-1oa0)

<p>&nbsp;</p>

<div style="text-align: center;"><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6irzgkhqmi4jym5rhs6q.png" alt="rpi-cluster-numbered" height="400"/></div>

<p>&nbsp;</p>

<h1 id="k3s-master-server">Master Server</h1>

**What is a Kubernetes master node?** A master node is a server that controls and manages a set of worker nodes, in our case it is the Raspberry Pi that controls the rest of the Raspberry Pi(s) on our cluster.

<h2 id="k3s-master-install">Install</h2>

1. SSH into the Raspberry Pi server that is intended to operate as the Kubernetes master. It should be the one named `kmaster` as instructed by this [post](https://dev.to/zachinachshon/setting-up-a-raspberry-pi-cluster-1oa0#rpi-configuration)

    ```bash
    # Connect to RPi server that operates as the k8s master
    ssh pi@kmaster
    ```
    <p>&nbsp;</p>

1. Run the following command to install a plain version of `k3s` without `traefik` load balancer and `k8s-dashboard`

    ```bash
    curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC=" --no-deploy traefik --no-deploy kubernetes-dashboard"  sh -
    ```
      <p>&nbsp;</p>

      > **Note:** We will install a plain version of `k3s` without Traefik load balancer and/or Kubernetes dashboard. These should be covered by other dedicated blog posts.

1. Verify that `k3s` was installed successfully. Run the following commands from within the RPi master server

    ```bash
    # Check for status - active (running)
    sudo systemctl status k3s
   
    # Check for status - Ready
    sudo kubectl get nodes
   
    # Optional - check that there are no error logs
    tail -f /var/log/syslog
    ```
    <p>&nbsp;</p>

1. **(Optional):** Run when in need to restart `k3s`

    ```bash
    # Restart k3s
    sudo systemctl restart k3s
    ```
    <p>&nbsp;</p>

    > **Note:** The `k3s` service is automatically started and restarted during installation. The install script will install `k3s` and additional utilities, such as `kubectl`, `crictl`, `k3s-killall.sh`, and `k3s-uninstall.sh`.

    > **Note:** During installation `kubectl` on the master server will be aliased to the command `k3s kubectl` so that we can use the pre-packaged version of `kubectl`. `k3s` uses a container runtime called `containerd` directly (no `docker`), interact using `crictl`.

<p>&nbsp;</p>

<h2 id="k3s-master-uninstall">Uninstall</h2>

1. SSH into the `k3s` master server 

    ```bash
    ssh pi@kmaster
    ```
    <p>&nbsp;</p>

1. Uninstall `k3s` by executing the following script:

    ```bash
    /usr/local/bin/k3s-uninstall.sh
    ```
    <p>&nbsp;</p>

    > **Note:** Rancher `k3s` service configuration can be found at `/etc/rancher/k3s/k3s.yaml`.

    > **Note:** Container runtime `containerd` configuration can be found at `/var/lib/rancher/k3s/agent/etc/containerd/config.toml`.

---

<h1 id="k3s-worker-node">Worker Node</h1>

**What is a Kubernetes worker node?** These are Raspberry Pi servers that act as workload runtimes i.e. run our applications, jobs, whatever we require them to run but they aren't the ones that manage the cluster, just the ones that _"get the job done"_.

<h2 id="k3s-worker-join">Join a Cluster</h2>

1. Extract the `k3s` join cluster token
	
    ```bash
    # SSH to the RPi master server
    ssh pi@kmaster
   
    # Extract the join token
    sudo cat /var/lib/rancher/k3s/server/node-token
   
    # Alternatively, you can run this on-liner directly from a client machine
    ssh pi@kmaster "sudo cat /var/lib/rancher/k3s/server/node-token"
    ```
    <p>&nbsp;</p>

1. Find the `k3s` master server IP address that is assigned to `kmaster`, either from the server itself or from a client machine if you've followed this [post](https://dev.to/zachinachshon/setting-up-a-raspberry-pi-cluster-1oa0#rpi-configuration)

    ```bash
    # From the RPi master server
    ip addr show eth0 | grep "inet\b" | awk '{print $2}' | cut -d/ -f1
   
    # Alternatively, from a client machine
    cat /etc/hosts | grep kmaster | awk '{print $1}'
    ```
    <p>&nbsp;</p>

1. SSH into a Raspberry Pi server intended to be used as a Kubernetes worker node. It would be the one named `knode<number>` as instructed on this [post](https://dev.to/zachinachshon/setting-up-a-raspberry-pi-cluster-1oa0#rpi-configuration)

    ```bash
    # Connect to a k3s worker node
    ssh pi@knode<number>  
    ```
    <p>&nbsp;</p>

1. Run the following command to install `k3s-agent` and join the worker node to an existing cluster

    ```bash
    # Replace MASTER-IP-ADDRESS with the master server IP address from previous step
    # Replace JOIN-TOKEN with the join token from previous step
    curl -sfL http://get.k3s.io | K3S_URL=https://<MASTER-IP-ADDRESS>:6443 \
   K3S_TOKEN=<JOIN-TOKEN> sh -
    ```
    <p>&nbsp;</p>

1. Verify that `k3s-agent` was installed successfully. Run the following commands from within the RPi worker server

    ```bash
    # Check for status - active (running)
    sudo systemctl status k3s-agent
   
    # Optional - check that there are no error logs
    tail -f /var/log/syslog
    ```
    <p>&nbsp;</p>

1. Repeat the above steps for every Raspberry Pi board intended to be used as a Kubernetes worker node

<p>&nbsp;</p>

<h2 id="k3s-worker-uninstall">Uninstall</h2>

1. SSH into the `k3s` worker node 

    ```bash
    ssh pi@knode<number>
    ```
    <p>&nbsp;</p>

1. Uninstall `k3s-agent` by executing the following script:

    ```bash
    /usr/local/bin/k3s-agent-uninstall.sh
    ```
    <p>&nbsp;</p>

---

<h1 id="k3s-utilities">Utilities</h1>

These are common utilities that should get installed on client machines to interacts with `k3s` master server.

**Why do I need to install them?**
You'll want to interacts with Kubernetes in order to deploy services, execute Helm charts and/or use utilities that grant you cluster visibility.

> **Note:** If you are planning to interact with `k3s` on a CI environment, make sure that the agent image you are using in the pipeline includes utilities such as `kubectl`.

<p>&nbsp;</p>

<h2 id="k3s-kubectl">kubectl</h2>

Install `kubectl`, a command-line-interface tool that allows you to run commands against a remote `k3s` cluster.

1. On a client machine, create a new empty `k3s` config file

    ```bash
    mkdir -p $HOME/.kube/k3s 
    touch $HOME/.kube/k3s/config
    chmod 600 $HOME/.kube/k3s/config  # Set limited user permissions
    ```
    <p>&nbsp;</p>

1. Copy the `k3s` cluster configuration from the RPi master server

    ```bash
    ssh pi@kmaster "sudo cat /etc/rancher/k3s/k3s.yaml" > $HOME/.kube/k3s/config
    ```
    <p>&nbsp;</p>

1. Edit the `k3s` config file on the client machine and change the remote IP address of the `k3s` master from `localhost/127.0.0.1` to `kmaster`

    ```bash
    # Edit master config
    vim $HOME/.kube/k3s/config
   
    # Search for the 'server' attribute located in - 
    # clusters:
    # - cluster:
    #   server: https://127.0.0.1:6443 or https://localhost:6443
    #
    # Change 'server' value to https://kmaster:6443
    ```
    <p>&nbsp;</p>

    > **Note:** Make sure `kmaster` is properly defined as a host name in `/etc/hosts`, otherwise - use the RPi master server IP address.

1. Install `kubectl` as described in the [official docs](https://kubernetes.io/docs/tasks/tools/install-kubectl/)

    ```bash
    # tl;dr - macOS only
    brew install kubectl
   
    # Verify client version
    kubectl version --client
    ```
    <p>&nbsp;</p>

1. Export `k3s` config file path as `KUBECONFIG` environment variable and by doing that set the `kubectl` context to use the RPi `k3s` cluster 

    ```bash
    export KUBECONFIG=$HOME/.kube/k3s/config
    ```
    <p>&nbsp;</p>

    > **Note:** Add the export command into a `.bash_profile` / `.bashrc` file. This way every new shell session would have the `k3s` cluster config set as the `kubectl` active context. _**Optional:** check [this post](https://dev.to/zachinachshon/configure-local-environment-using-dotfiles-1km0) to manage your dotfiles in style._

1. Verify that `kubectl` was installed properly and can communicate with the RPi master server

    ```bash
    kubectl get nodes
   
    # Expect the following respones as success:
    #
    # NAME      STATUS   ROLES                  AGE   VERSION
    # knodeX    Ready    <none>                 10m   v1.20.4+k3s1
    # ...
    # knode1    Ready    <none>                 23m   v1.20.4+k3s1
    # kmaster   Ready    control-plane,master   52m   v1.20.4+k3s1
    ```
    <p>&nbsp;</p>

1. **(Optional)**: read [here](https://kubernetes.io/docs/tasks/tools/install-kubectl/) for additional information about `kubectl`

<p>&nbsp;</p>

<h2 id="k3s-k9s">k9s</h2>

Install `k9s`, a terminal UI that interacts with the `k3s` cluster, increase velocity by saving you from typing repetitive commands and/or the need to alias common ones. It allows easy navigation, observation and management - all in one package.

1. Install as instructed on the official repository [docs](https://github.com/derailed/k9s)

    ```bash
    # tl;dr for macOS
    brew install k9s
    ```
    <p>&nbsp;</p>

    > **Note:** Make sure `$KUBECONFIG` is properly defined and set to `k3s` config path.

1. In case you are working on a single cluster and it is the `default` one, move to the next step, otherwise, if you are working on multiple clusters and/or using a cluster which isn't named `default`, change the `currentContext` and `currentCluster` attributes on the `k9s` config file to the proper cluster values.

    > **Note:** `k9s` configuration file can be found at `$HOME/.k9s/config.yml`

1. Run `k9s` on a fresh shell session and verify that you can connect the `k3s` cluster successfully 

1. **(Optional)**: read [here](https://k9scli.io/) for additional information about `k9s`

<p>&nbsp;</p>

---

<h3 id="summary">Summary</h3>

Well done for successfully installing a Kubernetes cluster on top of your Raspberry Pi cluster ! :clap:

**What now?** Check back for future posts explaining how to install a load balancer, certificate manager and a private docker registry on that cluster.

Please leave your comment, suggestion or any other input you think is relevant to this post in the discussion below.

---

Like this post?
You can find more by:

Checking out my blog: https://blog.zachinachshon.com
Following me on twitter: @zachinachshon

Thanks for reading! ❤️