Production Ready Kubernetes CI CD Pipeline
Tags: Jenkins, Microservices, Cicd Pipeline, Pipeline, Kubernetes

Container-based microservices architectures have changed the way development and operations teams test and deploy modern application/services. Containers help companies modernize by making it easier to scale and deploy applications, but containers have also introduced new challenges and more complexity by creating an entirely new infrastructure ecosystem.

Large and small software companies are now deploying thousands of container instances daily, and that’s a complexity of scale they have to manage. So how do they do it?

The magic is kubernetes.

Originally developed by Google, Kubernetes is an open-source container orchestration platform designed to automate the deployment, scaling, and management of containerized applications.

In this tutorial, let’s see how we can include kubernetes in our existing traditional CI/CD pipelines and achieve high availability of our services and get the ability to do the code changes in production at anytime(yes, anytime without affecting the services).

Tools Used

This tutorial believes you have basic/good understanding of below topics.

Kubernetes (Our service runs on kubernetes) Jenkins & Jenkins shared library (CI/CD tool) GIT (SCM tool) HAProxy (Network Load balancer) Ansible (Configuration management tool)

CI/CD setup

Jenkins Server — Installed with Docker, Ansible and Kubectl (Copy kube admin configuration .kube from K8s master to Jenkins server home directory)

K8s cluster with one master and two nodes

HAProxy Server

CI/CD Pipeline

CI/CD using K8s

GIT & Jenkins

Let’s go in detail of how we configure all these various tools and make it work perfectly.

Our example app called as shoppingapp has three repositories in GitHub and each repo is for a particular microservice.

shoppingapp-home for home page microservice. shoppingapp-kids for kids page microservice. shoppingapp-mens for mens page microservice.

Similarly we have three Jenkins Pipeline job for three repos and GitHub webhook is set up in the repositories to automatically start the build when there is a new commit.

Jenkins pipeline job

Let’s look at the configuration of shoppingapp-home Job.

Jenkins Pipeline configuration

The repo url in the pipeline job shoppingapp-home points to shoppingapp-home GitHub repositority. Similarly the other two pipeline job points to their respective repositories.

Below is the Jenkinsfile in the shoppingapp-home repository.

Jenkinsfile for shoppingapp-home repository

Below are the different steps in the pipeline job. The job declares a couple of variables, app — refers to our application name, service — refers to our service name, registryCredentials — is our dockerhub username and password saved in jenkins, imageid — the docker container image name with a tag, here tag will be the jenkins build number.

In the above Jenkinsfile, I have used shared library (Incase if you are not familiar with shared library refer Jenkins website).

In the stage ‘Build’ I call my function dockerbuild in the shared library and passing imageid as an argument. Below is the definition of the dockerbuild function.

Build stage

It just calls docker.build with the image id we passed as an argument which basically triggers ‘docker build -t imageid .’ and that creates a docker image based on the Dockerfile. We will look the Dockerfile little later.

Next step is ‘Test’ where you can run test cases. I current don’t execute any test cases as this is very basic app.

Test stage

Next we have ‘Publish’ stage which again calls a function imagepush with imageid as an argument in shared library. The function basically login into Docker Hub and pushes the image to Docker Hub.

Publish stage

Next is the ‘Pull Playbook Repo’ just cloning the repo which has all our playbooks.

Pull Playbook Repo stage

Next is the ‘Deploy’ stage which triggers the playbook to deploy the container into Kubernetes cluster.

Deploy stage

Now we have fair amount of clarity on how we develop and maintain repos for each microservices and configure Jenkins Pipeline to create Docker image, push image to Docker Hub, trigger Ansible Playbook to deploy to K8s cluster. Let’s see how our Dockerfile looks like.

Dockerize application

Dockerfile

The Dockerfile uses python:3.7.3-alpine3.9 as the base image and we install a flask app to just print out some text. Refer the shoppingapp-home GitHub repo for the code. The app runs on the port 5000.

Ansible Playbook to takeover further

Below is the Ansible Playbook deploy_k8s.yml which get triggered from Jenkins Pipeline job.