Building a CI/CD Pipeline with Nodeless Jenkins On Top of Kubernetes
By: Idan Shahar
Tags: DevOps, Jenkins, Ci, Cd, Kubernetes

Well, one of the coolest Kubernetes scenarios is for running a CI/CD pipeline. Imagine that inside your production cluster, alongside your production application, there is a small Jenkins Pod that is looking forward to deploy your next release. Yes yes, one Pod… you know what, two Pods. Probably a question now is rising in your head — what about the Jenkins Slaves?! That’s right — they all will be dynamically created when necessary! (as Kubernetes’ Pods of course)

Curious? So am I, let’s start to hack this out!

K8S Cluster Setup and Jenkins Deployment

First, you need to have a running cluster right? I created my cluster with AKS (The Azure Managed Kubernetes Service). So please go ahead and install Azure CLI or use your existing cluster if you already have one.\

Now we need to have the Helm Tiller installed in your cluster. If you don’t have Helm installed already, this is the time to do so. Take a look here.

To see the if Tiller is running, run this kubectl get pods — namespace kube-system

Now install Jenkins on your cluster using Helm:

To find the Jenkins IP address you can run kubectl get svc and look for the Jenkins master external IP (It takes a few minutes until the pending status is replaced with the external IP)

Nodeless Jenkins Configuration

Now we have a few things to configure in Jenkins:

Go to “Manage Jenkins” -> “Configure System”. Under Container Template, change the Docker Image to: idanshahar/jenkins-slave:latest (or your own docker image), then add the following host path volumes: /var/run/docker.sock, /usr/bin/docker, and change the Kubernetes URL to 10.0.0.1

Go to “Credentials” -> “System” -> “Global credentials” -> “Add Credentials”, choose “Global” Scope, add your Dockerhub credentials and insert ‘docker-hub-credentials’ as the credentials ID.

To run Docker commands inside Jenkins Slaves’ containers I added these volumes: /var/run/docker.sock , /usr/bin/docker, but it’s not enough. To be able to run DinD (Docker in Docker) the libltdl7-dev library is a necessary dependency and needs to be installed in the Dockerfile.

Now let’s have a look at the Slave’s Dockerfile:

Line 1: setting the base image to be jenkins/jnlp-slave:3.16–1

Lines 9–11: installing the libltdl7-dev library.

Lines 13–20: installing Helm & Draft.

I already built this Docker Image, you can find this in my Dockerhub repo idanshahar/jenkins-slave

So far, we have made all the necessary configurations to be able to run Jenkins Slaves as Kubernetes Pods.

Now, we are ready to build our CI/CD job. Generally, I prefer to manage the Jenkinsfile using git instead of using the Jenkins UI.

First, we want to setup the Docker credentials.

Writing Your Jenkins Job

We will create five stages — Login, Preparation, Build, Push and Deploy.

The Login stage will login to your Docker registry, the Preparation stage will clone your code, the Build stage will build your Docker Image, the Push stage will Push the Docker Image to your registry and the Deploy stage will deploy your pod.

Login Stage

Make sure that the DOCKER_USER and DOCKER_PASS variables were created in the withCredentials section.

Preparation Stage

Clone the project’s code.

Build Stage

Build the Docker Image.

Push Stage

Push the Docker Image to your Docker Registry.

And the final one, Deploy Stage

As you can see, I used Helm to deploy the app to the cluster. helm upgrade — install means that if the app is already deployed, helm will just upgrade your Pods, but if this is the first time you are deploying this app, helm will create the whole deployment. I used the — set parameter to replace the Docker Image and Docker tag and also to change the Kubernetes Service type to LoadBalancer in the Deployment file (to allow external access to thePods).

This is how the final Jenkinsfile looks:

Now we are ready to create the Pipeline Job.

Go to “New Item” enter the Job name, choose Pipeline job and hit the OK button. Now, choose the Jenkinsfile from git, or paste it into the Pipeline script. When you run this job, you will see that a new Pod is dynamically created. To watch this in actions, run kubectl get pods — watch and click on build job, then you will see that a new Pod is being created.

You can find this project in my github repository.

*Imported from my TechThisOut.io blog

Follow us on Twitter 🐦 and Facebook 👥 and join our Facebook Group 💬.

To join our community Slack 🗣️ and read our weekly Faun topics 🗞️, click here⬇