Devops Project - CI/CD Jenkins Ansible Kubernetes
Published: 2022-04-05 14:13:50

**Kubernetes on AWS**

```


      

                                               Dockerhub 
                                              /        \
                                            /            \                          
                                      Push/Image       Pull\Image        
                                        /                    \
      Pull Code       Copy Artifacts  /     Deploy Container   \
Github--------->Jenkin-------------->Ansible--------------->Kubernetes         |
                    |  
                    | Build Code
                   \|/
                  Maven  


```

**Setup Kubernetes on Amazon EKS**

You can follow same procedure in the official  AWS document [Getting started with Amazon EKS – eksctl](https://docs.aws.amazon.com/eks/latest/userguide/getting-started-eksctl.html)   

**Pre-requisites:** 
  - an EC2 Instance 
  - Install AWSCLI latest verison 


Setup kubectl   
   - Download kubectl version 1.21  
   - Grant execution permissions to kubectl executable   
   - Move kubectl onto /usr/local/bin   
   - Test that your kubectl installation was successful    

   ```sh
   curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.21.2/2021-07-05/bin/linux/amd64/kubectl
   chmod +x ./kubectl
   mv ./kubectl /usr/local/bin 
   kubectl version --short --client      
   ```
 
Setup eksctl 
   - Download and extract the latest release   
   - Move the extracted binary to /usr/local/bin   
   - Test that your eksclt installation was successful   

   ```sh
   curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
   sudo mv /tmp/eksctl /usr/local/bin
   eksctl version
   ```
  
Create an IAM Role and attache it to EC2 instance  
   `Note: create IAM user with programmatic access if your bootstrap system is outside of AWS`   
   IAM user should have access to   
   IAM   
   EC2   
   CloudFormation  
   Note: Check eksctl documentaiton for [Minimum IAM policies](https://eksctl.io/usage/minimum-iam-policies/)
   
Create your cluster and nodes
   ```sh
   eksctl create cluster --name cluster-name  \
   --region region-name \
   --node-type instance-type \
   --nodes-min 2 \
   --nodes-max 2 \ 
   --zones <AZ-1>,<AZ-2>
   
   example:
   eksctl create cluster --name valaxy-cluster \
   --region ap-south-1 \
   --node-type t2.small \
   ```

To delete the EKS clsuter 
   ```sh 
   eksctl delete cluster cluster-name --region ap-south-1
   ```
   
Validate your cluster using by creating by checking nodes and by creating a pod 
   ```sh 
   kubectl get nodes
   kubectl run tomcat --image=tomcat 
   ```
   
**Deploying Nginx pods on Kubernetes**

Deploying Nginx Container
   ```sh
    kubectl create deployment  demo-nginx --image=nginx --replicas=2 --port=80
    # kubectl deployment regapp --image=valaxy/regapp --replicas=2 --port=8080
    kubectl get all
    kubectl get pod
   ```

Expose the deployment as service. This will create an ELB in front of those 2 containers and allow us to publicly access them.

   ```sh
   kubectl expose deployment demo-nginx --port=80 --type=LoadBalancer
   # kubectl expose deployment regapp --port=8080 --type=LoadBalancer
   kubectl get services -o wide
   ```

**Create Pod and Service using manifest file**

Create pod manifest file

```
sudo su -
vim pod.yml

apiVersion: v1
kind: Pod
metadata:
  name: demo-pod
  labels: 
    app: demo-app

spec:
  containers: 
    - name: demo-nginx
      image: nginx
      ports:
        - name: demo-nginx
          containerPort: 80
```

Create service manifest file

```
vim service.yml

apiVersion: v1
kind: Service
metadata:
  name: demo-service
  labels: 
    app: demo-app

spec:
  ports:
  - name: nginx-port
    port: 80
    targetPort: 80
  selector:
    app: demo-app
  type: LoadBalancer
```

Run pod and service

```
kubectl apply -f pod.yml
kubectl get pod

kubectl apply -f service.yml
kubectl get service

kubectl get all
kubectl get pods -o wide
kubectl get services -o wide
kubectl describe pod podname
kubectl describe service servicename
```
**Setup Pod and Service**

```
                          selector:                labels:
                            app: demo-app            app: demo-app
External network-------------->Service------------------->Pod
```

As we know there will be large number of pods,When we send a request from External network to Service it looks for selector app: demo-app and forward the requset to the pods having Labels app: demo-app.That is why we use selector in service manifest file and labels in pod manifest file.

####Integrating Kubernetes in CI/CD pipeline

Go to ansible machine

create a user and allow password authentication.

```
sudo su -
adduser ansadmin
id ansadmin

vim /etc/ssh/sshd_config

PasswordAuthentication yes
#PasswordAuthentication no

systemctl restart ssh
```
Modify visudo for created user.

```
vim /etc/sudoers

# User privilege specification
root    ALL=(ALL:ALL) ALL
ansadmin ALL=(ALL:ALL) NOPASSWD: ALL
```


**User Privilege Lines**
```
root ALL=(ALL:ALL) ALL The first field indicates the username that the rule will apply to (root).

root ALL=(ALL:ALL) ALL The first “ALL” indicates that this rule applies to all hosts.

root ALL=(ALL:ALL) ALL This “ALL” indicates that the root user can run commands as all users.

root ALL=(ALL:ALL) ALL This “ALL” indicates that the root user can 
run commands as all groups.

root ALL=(ALL:ALL) ALL The last “ALL” indicates these rules apply to all commands.
```
Generate ssh-key.

```
su - ansadmin
ssh-keygen
```

Go to Kubernetes machine

Create Deployment and Service using manifest file

Create deployment manifest file

```
sudo su -
vim deployment.yml

apiVersion: v1
kind: Deployment
metadata:                        --->Deployment name and label
  name: demo-webapp 
  labels: 
    app: webapp

spec:
  replicas: 2
  selector:                      --->Create 2 pods from pod template
    matchLabels:
       app: webapp
  
template:                                                    |
    metadata:                    ---> pod definition         |        
      labels:                                                |       
        app: webapp                                          |
                                                             |           
    spec: containers                                         |---> Template to create a pod                                     |
    - name: webapp                                           |
      image: nithinalias/mytomcat ---> container definition  |
      imagePullPolicy: Always                                |
      ports:                                                 |
      - containerPort: 8080                                  |   

strategy:
  type: RollingUpdate
  rollingUpdate:                 --->make sure only one pod updated at a time
    maxSugre: 1
    maxUnavialable: 1
```

Create service manifest file

```
vim service.yml

apiVersion: v1
kind: Service                    ---> Resource Type

metadata:
  name: demo-service             ---> service name and label
  labels: 
    app: webapp

spec:
  ports:
  - name: nginx-port
    port: 8080                   ---> port number exposed at cluster level
    targetPort: 8080             ---> port that container listening
  
  selector:                      ---> To which deployment it can send traffic
    app: webapp
  type: LoadBalancer             ---> service type          
```
`labels in deployment manifest file need to match with selector in service manifest file.containerPort in deployment manifest file need to match with targetPort in service manifest file.`

```
sudo su -
passwd
adduser ansadmin
id ansadmin
docker login
```
Allow password authentication.
```
vim /etc/ssh/sshd_config

PermitRootLogin yes
PasswordAuthentication yes
#PasswordAuthentication no

systemctl restart ssh
```
Modify visudo for created user.

```
vim /etc/sudoers

# User privilege specification
root    ALL=(ALL:ALL) ALL
ansadmin ALL=(ALL:ALL) NOPASSWD: ALL
```
Go to ansible machine and add ip address of kubernetes machine and ansible machine(then only jenkin server can access ansible and kubernetes machine) inside ansible host file.

```
vim /etc/ansible/hosts

[ansible]
192.168.33.25
[kubernetes]
192.168.33.30
```
copy the ssh public key to kubernetes machine and ansible machine itself(localhost).

```
su - ansadmin
ssh-copy-id ansadmin@ip-of-ansiblemachine
ssh-copy-id ansadmin@ip-of-kubernetesmachine
ssh-copy-id root@ip-of-kubernetesmachine
ansible all -m ping
```
Now authorized key will be generated inside /home/ansadmin/.ssh/authorized_keys of kubernetes,ansible machine and /root/.ssh/authorized_keys of kubernetes machine.

create a directory,give ownership of ansadmin,add ansadmin to docker group and docker.sock permissions to all users.

```
sudo su -
mkdir /opt/docker
chown ansadmin:ansadmin /opt/docker
usermod -aG docker ansadmin
id ansadmin
chmod 777 /var/run/docker.sock
systemctl restart docker
su - ansadmin
docker login
```
Create dockerfile for tomcat image creation.

```
vim /opt/docker/dockerfile

FROM tomcat
RUN cp -R /usr/local/tomcat/webapps.dist/* /usr/local/tomcat/webapps
COPY ./*.war /usr/local/tomcat/webapps
```
Generate ansible-playbook for building tomcat image.

```
vim /opt/docker/build_tomcatimage.yml

---

- hosts: ansible

  tasks:
  - name: create docker image
    command: docker build -t mytomcat .
    args:
     chdir: /opt/docker

  - name: create tag to push image onto dockerhub
    command: docker tag mytomcat nithinalias/mytomcat

  - name: push docker image
    command: docker push nithinalias/mytomcat

```
Generate ansible-playbook to run deployment and service manifest file.

```
vim /opt/docker/run_kube_manifestfile.yml

---

- hosts: kubernetes
  user: root
  
  tasks:
  - name: run deployment manifestfile on kubernetes
    command: kubectl apply -f deployment.yml
    
  - name: run service manifestfile on kubernetes
    command: kubectl apply -f service.yml

  - name: update deployment with new pods if image updated in dockerhub
    command: kubectl rollout restart deployment.apps/demo-webapp

# we don't want to mention exact path like /root/deployment.yml or /root/service.yml, since we are performing the task as root user.

```


**Integrate Ansible with Jenkins**

we need to add ansadmin ssh connection of ansible machine.

Manage Jenkins  - Configure System - SSH servers - Add - Add name(ansible-server),Hostname(ip-of-ansible machine),username
(ansadmin) - Advanced - Enable Use password authentication or use a different key - Add password of user ansadmin(instead you can use ssh-key/path of ssh-key if present) - Test configuration - Apply - save.

Continuous Integration using Jenkins 

New Item - Add item name(CI webapp),select Maven project - OK - Add description - Select Git and add Repositories URL,Branch - Build Triggers(Poll SCM=* * * * * means every minute check the repository and if there any update it will trigger build - Build periodically means it will trigger build periodically even if repository not updated) - Build(Add Root POM=pom.xml,Goals and options=clean install) - Add post-build Action - Send build artifacts over SSH - Select SSH server name(ansible-server),Add sourcefile(webapp/target/*.war),Add Remove prefix(webapp/target),Add Remote directory="//opt//docker"(if it is blank webapp.war will be in /home/ansadmin),
Exec command ="ansible-playbook /opt/docker/build_tomcatimage.yml" - Add post-build action - Build other projects - Go to post-build action(top) - Add Projects to build="CD webapp" - Enable Trigger only if build is stable - Apply - Save - Build Now - Console output.You can see all build outcomes in /var/lib/jenkins/workspace/CI webapp/webapp/target.surefire contain the reports of build and webapp.war is called artifacts.This webapp.war will be in ansible machine /opt/docker directory and copy this to tomcat image.The image created inside ansible machine is pushed to dockerhub.

Continuous Deployment using Jenkins

New Item - Add item name(CD webapp) - OK - Add description - Add post-build Action - Send build artifacts over SSH - Select SSH server name(ansible-server),Exec command =
"ansible-playbook /opt/docker/run_kube_manifestfile.yml" - Apply - Save.

Go to Kubernetes machine 

```
kubectl get service
```
copy the EXTERNAL-IP of your service and paste it in browser.Now you get the webapp.
                            OR

You can go to LoadBalancer in AWS - Description - Copy DNS name - paste it in browser.Now you get the webapp.

`Source:` https://github.com/yankils