Build your own 3 nodes Kubernetes (1.20) cluster with Ubuntu (20), CRI-O (1.17) and kubeadm - Easily practice for CKA/CKAD
Tags: Ubuntu, Cluster, Kubernetes, Mac

(For video tutorial of this blog click here)

Kubernetes has became vital tool for managing containers. Kubernetes is widely used as containers orchestration framework for all size load.

Although you can find many tools, websites and practice LABS where you can learn about Kubernetes, having your own cluster makes it so easy and convineint compare to cloud based tools. If you have your own cluster you can deploy your container applications and test it out, learn more about how Kubernets works and even practice CKA (Certified Kubernetes Administrator) scenarios, questions and many more.

In this tutorial I will help you install 3 nodes Kubernetes cluster on your own computer so you don’t have to sign-up for AWS or Azure account, spin up computer servers and make it available when you need it.

Following instructions will work out of the box on latest Mac OS X however similar steps may work on any computer with virtualization software installed. We will be using few tools and commands to achieve cluster creation.

Let’s start with requirements:

Base computer used: MacBook Pro with most recent OS X installed. (You can use Windows 8 or any later version of Windows Desktop).

Virtualization Software: Parallels Desktop 11 for Mac. — https://www.parallels.com/parallelsdesktop11/ (You can install Oracle VirtualBox on Mac OS X or Windows — https://www.virtualbox.org

Guest Virtual Machine: Ubuntu 20.04.1 LTS (Focal Fossa)

Kubernetes Cluster Installation tool: kubeadm

At high level Kubernetes cluster has master nodes and worker nodes.

Master: A Kubernetes Master nodes or Control Plane Components are:

kube-api-server

etcd

kube-scheduler

kube-controller-manager

The minimum requirements for master server(s) are:

Memory: 2 GiB or more of RAM per machine

or more of RAM per machine CPUs: At least 2 CPUs on the control plane machine.

Worker Node: A Node is a system that provides the run-time environments for the containers. A set of container pods can span multiple nodes.

kubelet

kube-proxy

The minimum requirements for worker nodes are:

Memory: 1GiB or more of RAM per machine

or more of RAM per machine CPUs: At least 1CPU on the control plane machine.

on the control plane machine. Outbound Internet connection for pulling containers required (Private registry can also be used)

Full network connectivity between machines in the cluster — This is private or public

Install Kubernetes Cluster on Ubuntu 20.04

Our setup has three servers. One master node or control plane + etcd and two worker nodes. Each servers are Ubuntu VMs.

Here’s the table with detailed specs for each VM.

First provision 3 virtual machines with Ubuntu 20.04 (You can download the server ISO from here). Once the servers are built and ready follow these steps.

Note: Once VM(s) are built login using ssh(protocol) command using the username and password setup during the VM creation. Example:

ssh administrator@<IP_ADDRESS> (you will get DHCP IP address of each VM from the Host OS (Virtualization software) e.g. ssh administrator@10.211.55.1

Step 1: Update Ubuntu servers

sudo apt update

sudo apt -y upgrade && sudo systemctl reboot

Step 2: Install kubelet, kubeadm and kubectl

Once the servers are rebooted, add Kubernetes repository for Ubuntu 20.04 to all the servers.

sudo apt -y install curl apt-transport-https net-tools curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list

Then install required packages.

sudo apt update

sudo apt -y install vim git curl wget kubelet kubeadm kubectl

sudo apt-mark hold kubelet kubeadm kubectl

Confirm installation by checking the version of kubectl.

kubeadm version

Step 3: Disable Swap

Turn off swap.

sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

sudo swapoff -a

Step 4: Install Container runtime

To run containers in Pods, Kubernetes uses a container runtime. Supported container runtimes are:

Runtime: since Docker will be deprecated from Kubernetes (late 2021 — https://kubernetes.io/blog/2020/12/02/dockershim-faq/) we will be using cri-o as runtime environment for our installation.

Installing CRI-O:

# Load necessary modules.

sudo modprobe overlay

sudo modprobe br_netfilter

# Set up required sysctl params

sudo tee /etc/sysctl.d/kubernetes.conf<<EOF

net.bridge.bridge-nf-call-ip6tables = 1

net.bridge.bridge-nf-call-iptables = 1

net.ipv4.ip_forward = 1

EOF

echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables

# Reload sysctl

sudo sysctl --system

# Add cri-o repo

. /etc/os-release sudo sh -c "echo 'deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/x${NAME}_${VERSION_ID}/ /' > /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list" wget -nv https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable/x${NAME}_${VERSION_ID}/Release.key -O- | sudo apt-key add - sudo apt update

# Install CRI-O

sudo apt install cri-o-1.17 -y

# Enable Services

sudo systemctl daemon-reload

sudo systemctl enable crio Kubelet

Modify crio config file /etc/crio/crio.conf

# Cgroup management implementation used for the runtime.

1). update this line and change systemd to cgroupfs -cgroup_manager = “cgroupfs” 2). comment out this line -> seccomp_profile (secure computing mode) is kernel level security in Linux (https://en.wikipedia.org/wiki/Seccomp) 3). uncomment these lines ->registries registries = [

“quay.io”,

“docker.io”

]

Add a default kubelet file under /etc/default and add following content.

vi /etc/default/kubelet

KUBELET_EXTRA_ARGS=--feature-gates="AllAlpha=false,RunAsGroup=true" --container-runtime=remote --container-runtime-endpoint='unix:///var/run/crio/crio.sock' --runtime-request-timeout=5m

# Restart crio service

systemctl restart crio kubelet

Pull necessary container images:

sudo kubeadm config images pull

Step 5: Initialize master node

Following command (kubeadm init) initializes and boostrap a Kubernetes control-plane node.

sudo kubeadm init --pod-network-cidr=10.244.0.00/16 --cri-socket=unix:///var/run/crio/crio.sock

Note for CKA/CKAD readers: During exam pay attention to the output of above command and check for any error(s). Based on output you may have to modify the command, as an example: you may have to add additional parameter like “ --skip-preflight-checks" or provide the config file with --config with the path to config file.

Note: If 10.244.0.00/16 is already in use within your network you must select a different pod network CIDR, replacing 10.244.0.00/16 in the above command.

Setup environment for kubectl command.

mkdir -p $HOME/.kube

sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

sudo chown $(id -u):$(id -g) $HOME/.kube/config

Step 6: Install network plugin (CNI) configuration on Master

Check cluster status:

kubectl cluster-info

Additional Master nodes can be added using the command in installation Check all pods are running in kube-system:

watch kubectl get pods -n kube-system

Confirm master node is ready:

kubectl get nodes

Step 7: Add worker nodes

Build two worker nodes and follow steps 1 through 4 on each servers.

Once the master node (control plane) ready we will add worker nodes to the cluster by running “kubeadm join” command. We will perform this command on both worker node servers.

“kubeadm join” will bootstrap a Kubernetes worker node and join it to the cluster

kubeadm join 10.211.55.11:6443 — token 3cfmdk.8stnadsdkbw5zk94 \ — discovery-token-ca-cert-hash sha256:56aa50d03f5a6189e0c81b5ea528a92bf8a9d78a21e647f90349f12ab51a6ca2

Run below command on the control-plane (master) node to see if the node joined the cluster.

kubectl get nodes -o wide (for more details)

Before we install our demo pod and Kubernetes dashboard we will label our worker nodes.

kubectl label node kube-worker-01 kubernetes.io/role=worker1 kubectl label node kube-worker-02 kubernetes.io/role=worker2

Step 8: Deploy test application on cluster

It's time to deploy test application and see how the worker nodes gets the load and pods are being created.

First create manifests folder on worker nodes:

mkdir /etc/kubernetes/manifests

kubectl run nginx --image=nginx --dry-run=client --restart=Never -o yaml > nginx.yaml kubectl apply -f nginx.yaml # modify the nginx yaml to create pod on first worker node. Or you can copy and create manifest from below snippet

Once the pod is up you can expose it by creating a service.

kubectl expose pod nginx --name=nginx-svc --port 80 --type NodePort

kubectl get svc

Get the IP address of the service and then curl.

example: curl http:// 10.107.42.253

Step 9: Install Kubernetes Dashboard (Optional)

Optionally you can install Kubernetes dashboard on a local cluster by following this guideline here — https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/

Install

To deploy Dashboard, execute following command:

Alternatively, you can install Dashboard using Helm as described at https://artifacthub.io/packages/helm/k8s-dashboard/kubernetes-dashboard.

Access

To access Dashboard from your local workstation you must create a secure channel to your Kubernetes cluster. Run the following command:

kubectl proxy &

For Mac users, open a tunnel to the master server to expose the port 8001. Replace IP_ADDRESS_MASTER to the IP address of your master server.

ssh -A administrator@<IP_ADDRESS_MASTER> -L 8001:127.0.0.1:8001

# Create admin user

vi admin-dashboard.yaml

kubectl apply -f admin-dashboard.yaml

# Get the access token

echo "" ; kubectl get secret -n kubernetes-dashboard $(kubectl get serviceaccount admin-user -n kubernetes-dashboard -o jsonpath="{.secrets[0].name}") -o jsonpath="{.data.token}" | base64 --decode ; echo ""

# You can now access Dashboard at:

http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/overview?namespace=default (Use the token you grabbed from previous command)

For faster completion of long kubectl command follow these steps which enabled autocompletion for kubectl

Enable kubectl autocompletion

During the exam and daily operation of Kubernetes cluster you have to execute “kubectl” command many time. To make it easy it allows bash completion. follow these easy steps to autocomplete each kubeclt sub-commands via tab — my favorite feature.

Source the completion script in your ~/.bashrc file:

echo 'source <(kubectl completion bash)' >>~/.bashrc

Add the completion script to the /etc/bash_completion.d directory:

kubectl completion bash >/etc/bash_completion.d/kubectl

If you have an alias for kubectl, you can extend shell completion to work with that alias:

echo 'alias k=kubectl' >>~/.bashrc

echo 'complete -F __start_kubectl k' >>~/.bashrc

Congratulations!!! you now have your own 3 nodes Kubernetes cluster with dashboard. Let me know if what you think and if you have questions or comments.

In next tutorial I will be posting more topics about CKA exams and Kubernetes. If you have enjoyed this tutorial and video please 👏