Provisioning Kubernetes Clusters with Kubespray
Tags: kubernetes, kubespray, devops, ansible
Published: 2024-05-12 12:50:59

Kubernetes, an open-source orchestration system, automates the deployment and management of containerized applications. For beginners, the journey into Kubernetes can often start with the daunting question: "Where do I begin?"

In the early days, setting up and managing a Kubernetes cluster was a challenging and time-consuming task. However, with the evolution of Kubernetes, user-friendly solutions have emerged to simplify this process. Among these solutions, Kubespray shines as an invaluable tool.

**Kubespray**, an open-source solution, facilitates the automated deployment of Kubernetes clusters across nodes. Engineered to be highly customizable, efficient, and lightweight, Kubespray caters to a wide range of requirements, making Kubernetes cluster deployment accessible to all.

**Overview of Kubespray**
Kubespray is a composition of Ansible playbooks, inventory, provisioning tools, and generic Kubernetes cluster configuration management tasks. In this writeup, I'll demonstrate how to deploy a Kubernetes cluster on 3 nodes (1master and 2 worker nodes) using Kubespray. 
While a basic understanding of Ansible and Kubernetes terminologies is assumed, the steps are simple enough for beginners to follow along.

**Prerequisites**
Before proceeding, ensure the following prerequisites are in place:

- **Provision Infrastructure:** Set up computing resources, such as 3 servers, for the cluster.
- **Install Dependencies:** Install the following dependencies on your Ansible server (can be one of your nodes or a dedicated server {recommended} for ansible):
- [Git](https://github.com/git-guides/install-git)
- [Python3](https://docs.python-guide.org/starting/install3/linux/)
- [Pip3](https://docs.python-guide.org/starting/install3/linux/)
- [Ansible](https://github.com/kubernetes-sigs/kubespray/blob/master/docs/ansible.md#installing-ansible)

**Setting Up the Cluster**
Follow these steps to set up your Kubernetes cluster with Kubespray:
**Step 1: Set Up SSH Keys**
Generate SSH keys on the Ansible node and copy the key to all your cluster nodes:

```
ssh-keygen # Go with the defaults
ssh-copy-id <user>@<node-IP>
```
**Step 2: Download and Configure Kubespray**
Download the Kubespray GitHub repository and checkout the latest version:

```
git clone git@github.com:Kubernetes-sigs/Kubespray.git
cd Kubespray
git checkout release-2.xx #replace 'xx' with release number
```
**Step 3: Install Python Dependencies**
Install the required Python dependencies using pip:

```
pip3 install -r ./requirements.txt
```
**Step 4: Update Ansible Inventory**
Update the Ansible inventory file with the IP addresses of your nodes:

```
cp -rfp inventory/sample inventory/mycluster
declare -a IPS=(<node1-IP> <node2-IP> <node3-IP>)
CONFIG_FILE=inventory/mycluster/hosts.yaml python3 contrib/inventory_builder/inventory.py ${IPS[@]}
```
Further customize inventory/mycluster/hosts.yaml to specify your master, worker, and etcd nodes.

**Step 5: Review and Customize Configuration**
Review and customize parameters under inventory/mycluster/group_vars:

```
cat inventory/mycluster/group_vars/all/all.yml
cat inventory/mycluster/group_vars/k8s_cluster/k8s-cluster.yml
```
**Step 6: Allow Kubernetes Ports**
If behind a firewall, ensure all necessary Kubernetes ports are allowed. These ports include; api-server, etcd, DNS, kubelet, kube-controller-manager, calico ports, kube-scheduler e.t.c.

**Step 7: Clean Up Old Kubernetes Cluster**
Run the playbook to clean up any old Kubernetes cluster:

```
ansible-playbook -i inventory/mycluster/hosts.yaml --user=<your-user-with-sudo-access> --ask-become-pass --become reset.yml
```
**Step 8: Deploy Kubernetes with Kubespray**
Run the playbook to deploy Kubernetes:

```
ansible-playbook -i inventory/my-cluster/hosts.yml --user=<your-user-with-sudo-access> --ask-become-pass --become cluster.yml
```
**Step 9: Access the Cluster**
To access the cluster using kubectl commands, update your kubeconfig file to be in the ~/.kube directory:

```
mkdir .kube
cd .kube/
sudo cp /etc/kubernetes/admin.conf config
kubectl get nodes
```

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a05hx5h6y7cwiempdcsx.png)

**Note:** The playbook will take some time to complete, but once finished, you'll have a highly available and self-managed Kubernetes cluster at your disposal. 
Kubespray enables you to manage Kubernetes clusters, you can add/remove nodes (playbooks/scale.yml|remove_node.yml), delete the entire cluster (playbooks/reset.yml), and performing other Kubernetes management tasks.

**Troubleshooting**

1. Issue with Python Packages Installation: When Ansible is already installed via system packages on the control node, Python packages installed using sudo pip install -r requirements.txt may end up in a different directory tree (e.g., /usr/local/lib/python2.7/dist-packages on Ubuntu) compared to Ansible's directory (e.g., /usr/lib/python2.7/dist-packages/ansible on Ubuntu). Consequently, the ansible-playbook command may fail with the following error:

```
ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.
```
This likely indicates that a task depends on a module present in requirements.txt.
2.Ensure Firewall Rules Allow Necessary Ports: Make sure that all necessary ports are allowed through the firewall to ensure proper communication between components.
3.Failure to Run Playbook without --become: The playbook will fail to run if the --become flag is not used. Ensure that you include --become to grant necessary privileges for the playbook to execute successfully.
4.For further troubleshooting on any encountered issue, please refer to the [official Kubespray repository](https://github.com/kubernetes-sigs/kubespray) for comprehensive troubleshooting steps.