DevSecOps: Orchestrating Secure and Observable 3-Tier Deployments on AWS with Terraform, EKS, Jenkins, Prometheus etc.
Tags: aws, kubernetes, monitoring, devsecops
Published: 2024-02-07 08:27:34


![architecture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7hxlw85735m6zdy00jbj.png)

**Project Introduction:**
This End-to-End DevSecOps Project involves setting up a robust Three-Tier architecture on AWS. It aims to demonstrate the deployment of a production ready infrastructure that is secure and scalable with monitoring measures in place for application observability. It was inspired by a similar project from [Aman-Pathak](https://github.com/AmanPathak-DevOps).

**Project Overview:**

1. IAM User Setup: Create an IAM user on AWS with the necessary permissions to facilitate deployment and management activities.
2. Infrastructure as Code (IaC): Use Terraform and AWS CLI to set up the Jenkins server (EC2 instance) on AWS.
3. Jenkins Server Configuration: Install and configure essential tools on the Jenkins server, including Jenkins itself, Docker, Sonarqube, Terraform, Kubectl, AWS CLI, and Trivy.
4. EKS Cluster Deployment: Utilize eksctl commands to create an Amazon EKS cluster, a managed Kubernetes service on AWS.
5. Load Balancer Configuration: Configure AWS Application Load Balancer (ALB) for the EKS cluster.
6. Amazon ECR Repositories: Create private repositories for both frontend and backend Docker images on Amazon Elastic Container Registry (ECR).
7. ArgoCD Installation: Install and set up ArgoCD for continuous delivery and GitOps.
8. Sonarqube Integration: Integrate Sonarqube for code quality analysis in the DevSecOps pipeline.
9. Jenkins Pipelines: Create Jenkins pipelines for deploying backend and frontend code to the EKS cluster.
10. Monitoring Setup: Implement monitoring for the EKS cluster using Helm, Prometheus, and Grafana.
11. ArgoCD Application Deployment: Use ArgoCD to deploy the Three-Tier application, including database, backend, frontend, and ingress components.
12. DNS Configuration: Configure DNS settings to make the application accessible via custom subdomains.
13. Data Persistence: Implement persistent volume and persistent volume claims for database pods to ensure data persistence.
14. Conclusion and Monitoring: Key achievements and monitoring the EKS cluster’s performance using Grafana.

**Prerequisites:**

An AWS account with the necessary permissions to create resources.
Terraform and AWS CLI installed on the local machine.
Basic familiarity with Kubernetes, Docker, Jenkins, and DevOps principles.

**Step 1: To create an IAM user and generate the AWS Access key**
Create a new IAM User on AWS and give it the AdministratorAccess for the purposes of this project (not recommended for Production Projects, always grant least privilege)

Go to the AWS IAM Service and click on Users => Create Users
- Provide the name then click next
- Select the 'Attach policies directly' option and search for 'AdministratorAccess' then select it.


![IAM user creation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bre4rfpl7p3q2urucrw6.png)

Click on Create user

Now, Select your created user then click on Security credentials and generate access key by clicking on Create access key.

- Select Command Line Interface (CLI) as the Use case 
- select the checkmark for the confirmation and click on Next

Add the Description tag of your choice and click on the Create access key.

![Access Key Created](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4t6xsj2loh0m98ml51fi.png)

Download the .csv file containing the credentials and keep in a secure location

**Step 2: Install Terraform & AWS CLI to deploy the Jenkins Server(EC2) on AWS.**
Install & Configure Terraform and AWS CLI on your local machine to create Jenkins Server on AWS Cloud

Terraform Installation Script
```
wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg - dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update
sudo apt install terraform -y
```
AWSCLI Installation Script

```
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
sudo apt install unzip -y
unzip awscliv2.zip
sudo ./aws/install
Now, Configure both the tools
```

Configure Terraform

- Edit the file /etc/environment using the below command 

```sudo vim /etc/environment```

- add the below lines and replace the placeholders credentials with the actual your keys.

```
export AWS_ACCESS_KEY_ID={placeholder_key_id}
export AWS_SECRET_ACCESS_KEY={placeholder_access_key}
export AWS_DEFAULT_REGION=eu-north-1
export AWS_CONFIG_FILE="/root/.aws/config"
```

Save the changes and restart your machine to reflect the changes of your environment variables.

Configure AWS CLI

Run the below command, and add your keys

```aws configure```


![aws cli configuration](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jw94egbsm05dxetr0m9h.png)

**Step 3: Deploy the Jenkins Server(EC2) using Terraform**

Clone the Git repository - https://github.com/abdulxs/DevSecOps-Project

Navigate to the Jenkins-Server-TF

Do some modifications to the backend.tf file such as changing the bucket name and dynamodb table(make sure you have created both manually on AWS Cloud - Ensure the 'Lock-files' table has a partition key named 'LockID' of type string).

![backend.tf file](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jvgus3f8bxkc3s25uk15.png)

On the variables.tfvars file, replace the key name with the Pem File name you have created on AWS. (If you have not, create a new one)

![edit key name](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f9qb3di3cc0s2f08w0m6.png)

Validate that the provider.tf and vpc.tf files also have the correct aws region set.

Initialize the backend by running the below command in the `/Jenkins-Server-TF` directory

```terraform init```

![terraform init](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nasdfo2j3hrseqkeks6u.png)

Run the below command to check the syntax error

```terraform validate```

![terraform validate](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rgo6fcbgxxk8v0g23nzi.png)

Run the below command to get the blueprint of what AWS services will be created.

```terraform plan -var-file=variables.tfvars```

![Terraform plan](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8n8xeesz6sntcot4tjvo.png)

Now, run the below command to create the infrastructure on AWS Cloud which will take 3 to 4 minutes maximum

```terraform apply -var-file=variables.tfvars --auto-approve```

![terraform apply](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8wtk8fsewd6jd70t3iv4.png)

Now, connect to the Jenkins-Server from AWS console by clicking on Connect.

![jenkins server](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pf0ty17639jqls9yvr6q.png)

You can use ec2 instance connect or copy the ssh command to connect from your local machine (don't forget to add your key name)

**Step 4: Configure the Jenkins**

Now, we log into the Jenkins server.

We have installed services such as Jenkins, Docker, Sonarqube, Terraform, Kubectl, AWS CLI, and Trivy from the tools-install.sh file.

Let’s validate their installation.

```
jenkins --version
docker --version
docker ps
terraform --version
kubectl version
aws --version
trivy --version
eksctl version
```

![Installation validation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mm93avla7g4l6eben7kc.png)

Now, to configure Jenkins. Copy the public IP of the Jenkins Server and paste it on the browser with an 8080 port.

![Jenkins](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mbo1tmnfzw4iu931145u.png)

Locate the password as directed. Enter it and proceed
Click on Install suggested plugins

After installing the plugins, continue as admin.

![jenkins url](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/thcrbv340i2hgoj8udt6.png)

Click on Save and Finish. Click on 'Start using Jenkins'.

Then you should see the Jenkins Dashboard

![Jenkins dashboard](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s9h78hupyzcp2boc45s4.png)

**Step 5: Deploy the EKS Cluster using eksctl commands**

Now, go back to your Jenkins Server terminal and configure AWS with the command `aws configure`.

Go to Manage Jenkins. Click on Plugins.

Select the 'Available plugins', install the following plugins and click on Install

AWS Credentials

Pipeline: AWS Steps

![Plugin installation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/juhaop0sk8ch3vd37dep.png)

Once, both the plugins are installed, restart your Jenkins service by checking the Restart Jenkins option.

![Jenkins restart](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ipat6e0cvmybs0f8jhi8.png)

Login to the Jenkins Server Again

![Jenkins sign in](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/n9r8zwjwglopyh4f5r1n.png)

Now, we have to set the AWS credentials on Jenkins

Go to Manage Plugins and click on Credentials

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tukiyl570g7l8uojpany.png)

Click on global.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ncp411qe7ea78elri95b.png)

Click on 'add credentials'
Select AWS Credentials as Kind and add the ID same as shown in the below snippet except for your AWS Access Key & Secret Access key and click on Create.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k9t2s5t7bmqb1t6zx6yz.png)

Now, We need to add GitHub credentials as well.

So, add the username and password/access token of your GitHub account.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/n60z2uasbr6p9iehavo6.png)

Create an eks cluster using the below commands.

```
eksctl create cluster --name Three-Tier-K8s-EKS-Cluster --region eu-north-1 --node-type t3.medium --nodes-min 2 --nodes-max 2
aws eks update-kubeconfig --region eu-north-1 --name Three-Tier-K8s-EKS-Cluster
```

![cluster creation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ryqgxlqanqfrdpavhzx9.png)

Once your cluster is created, you can validate whether your nodes are ready or not by the below command

```kubectl get nodes```

![Kube nodes](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g9pxqfo853z8nvb2van7.png)

**Step 6: Configure the Load Balancer on the EKS because the application will have an ingress controller.**

Download the policy for the LoadBalancer prerequisite.

```curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.5.4/docs/install/iam_policy.json```

![download policy](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k9a4n02fxw3afu8zohr3.png)

Create the IAM policy using the below command

```aws iam create-policy --policy-name AWSLoadBalancerControllerIAMPolicy --policy-document file://iam_policy.json
```

![policy creation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/28n740o70k7wlnb4n2pt.png)

Create OIDC Provider

```eksctl utils associate-iam-oidc-provider --region=eu-north-1 --cluster=Three-Tier-K8s-EKS-Cluster --approve```

![oidc](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xnht7y3o852ztsgexshm.png)

Create Service Account (replace the account ID with yours)

```eksctl create iamserviceaccount --cluster=Three-Tier-K8s-EKS-Cluster --namespace=kube-system --name=aws-load-balancer-controller --role-name AmazonEKSLoadBalancerControllerRole --attach-policy-arn=arn:aws:iam::{account ID}:policy/AWSLoadBalancerControllerIAMPolicy --approve --region=eu-north-1```

![service account](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v9vr151ioozz7t8qab8k.png)

Run the below command to deploy the AWS Load Balancer Controller

```sudo snap install helm --classic
helm repo add eks https://aws.github.io/eks-charts
helm repo update eks
helm install aws-load-balancer-controller eks/aws-load-balancer-controller -n kube-system --set clusterName=my-cluster --set serviceAccount.create=false --set serviceAccount.name=aws-load-balancer-controller```

After 2 minutes, run the command below to check whether the pods are running or not.

```kubectl get deployment -n kube-system aws-load-balancer-controller
```
![LB controller](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ap3ob1g5ibrqblvfjjg2.png)

Step 7: **Create Amazon ECR Private Repositories for both Tiers (Frontend & Backend)**
Go to Amazon Elastic Container Registry on the AWS console and click on Create repository

![ECR](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/euhlfqws3k0ujja9y2ba.png)

Select the Private option and provide the repository name 'frontend'. click Save.

Do the same for the backend repository and click on Save

![FE and BE repos](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e3kgvr6ov2co0dh61e4x.png)

We have set up our ECR Private Repositories

Now, we need to configure ECR locally because we have to upload our images to Amazon ECR.

Open the repository, view push command and copy the first push command for login

Now, run the copied command on the Jenkins Server.

![ECR login](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/glwbv5mz6eevupjxwe09.png)

**Step 8: Install & Configure ArgoCD**

We will be deploying the application in a namespace called 'three-tier'. To do that, we will create the namespace on EKS

```kubectl create namespace three-tier```

![create namespace](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sfak1d5qzx5e2h6j4es2.png)

The two ECR repositories are private. So, when we try to push images to the ECR Repos it will give an 'Imagepullerror' error.

To avoid this error, we will create a secret for our ECR Repo with the below command and then, we will add this secret to the deployment file.

Note: The Secrets are coming from the .docker/config.json file which is created while logging in the ECR in the earlier steps

```
kubectl create secret generic ecr-registry-secret \
 --from-file=.dockerconfigjson=${HOME}/.docker/config.json \
 --type=kubernetes.io/dockerconfigjson --namespace three-tier
kubectl get secrets -n three-tier
```

![create secret](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ge12vwtbxz04jzxpmgif.png)

Now, we will install argoCD.

To do that, create a separate namespace for it and apply the argocd configuration for installation.

```
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/v2.4.7/manifests/install.yaml
```

![deploy argocd](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t03zwjw6h939qdf25rfo.png)

All pods must be running, to validate run the below command

```kubectl get pods -n argocd```

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3iewjevcwxehgxvxjy79.png)

Now, expose the argoCD server as a LoadBalancer using the below command

```
kubectl patch svc argocd-server -n argocd --type='json' -p='[{"op":"replace","path":"/spec/type","value":"LoadBalancer"}]'’
```

![argocd patch](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hu2e6ur0u4287djtazyb.png)

You can validate whether the Load Balancer is created or not by checking on the AWS Console => EC2 => Load balancers

![ec2 LB](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lliwmwf6bd6njj0igbsu.png)

To access the argoCD, copy the LoadBalancer DNS and access the url on the browser

You will get a warning like the below snippet.

Click on Advanced.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/413g32tb1e8lvybzecye.png)

Click on the link that appeared below it to proceed

![argocd dashboard](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fi77ca6runl1clnt58gw.png)

Now, we need to get the password for our argoCD server to perform the deployment.

To do that, we have a pre-requisite which is **jq.** Install it with the command below.

`sudo apt install jq -y`

![jq](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bdafgt7vnhl7gvzll3r4.png)

```
export ARGOCD_SERVER=$(kubectl get svc argocd-server -n argocd -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
export ARGO_PWD=$(kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d)
echo $ARGO_PWD
```

![argo password](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ex4affn3nnkunxyrl514.png)

Enter the username and password in argoCD and click on SIGN IN.

![argo sign in](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fbusbigxoz9s5egxmxni.png)

Here is the ArgoCD Dashboard.

![argo dashboard](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/63dh699t5iruoa5l0yjo.png)

**Step 9: To configure Sonarqube for the DevSecOps Pipeline**

Copy the Jenkins Server public IP and paste it in the browser with a 9000 port

The username and password will be 'admin'

Click on Log In.

![sonar login](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/azwtnijktzud3vzziqvc.png)

Update the password and login

![sonar dashboard](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/of38cs4pq2aoipgkokf4.png)

Click on Administration then Security => select Users => Update tokens

![update token](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8385i967q8do4cy1j24r.png)

Click on Generate

![gen token](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wku6j3a2m6s5jyfusful.png)

Copy the token keep it somewhere safe and click Done.

![sonar token](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fuy6jg9gfzb3f5a48a87.png)

Now, we have to configure webhooks for quality checks.

Click on 'Administration' then, 'Configuration' and select 'Webhooks'

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/90gkesv9gbujd1qwuype.png)

Click on Create
Provide the name of the server and in the URL, provide the Jenkins server public IP with port 8080. Add sonarqube-webhook in the suffix, and click on Create.

http://<jenkins-server-public-ip>:8080/sonarqube-webhook/

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dfv2i352cxxurxee5ach.png)

Now, we have to create a Project for frontend code.

Click on 'Manually'.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/giqffmq0vcxdj8mhg5xu.png)

Provide the display name to the Project and click on Setup

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gsit5zpxsy4yfi8wtjo0.png)

Click on 'Locally'.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/i0pwdg2sbdisllq9jyao.png)
Select the 'Use existing token' and click on Continue.
Put in the token generated earlier and continue

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/irpik00ox8ugvp32o5ak.png)

Select 'Other' and 'Linux' as OS.

After performing the above steps, you will get the command which you can see in the below snippet.

Now, use the command in the Jenkins Frontend Pipeline where Code Quality Analysis will be performed.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p50tuqtcl740g7xddi4i.png)
Now, we have to create a Project for backend code.

Click on 'Create Project' and repeat similar steps as done for the frontend project to create the backend equivalent.

Once done, we have to store the sonar credentials.

Go to Dashboard -> Manage Jenkins -> Credentials

Select the kind as 'Secret text', paste your token in 'Secret' and keep other things as is.

Click on Create

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hjvdr1x5efgj9r8u5qfb.png)

Now, we have to store the GitHub Personal access token to push the deployment file which will be modified in the pipeline itself for the ECR image.

Add GitHub credentials on Jenkins

Select the kind as 'Secret text' and paste your GitHub Personal access token(not password) in 'Secret', and keep other things as is.

Click on Create

Note: If you haven’t generated your token then, you have it generated first then paste it into the Jenkins credentials.

Now, according to the Pipeline, we need to add an Account ID in the Jenkins credentials because of the ECR repo URI.

Select the kind as Secret text paste your AWS Account ID in Secret and keep other things as it is.

Click on Create.

Now, we need to provide the ECR image name for frontend which is 'frontend' only.

Select the kind as 'Secret text', paste the frontend repo name in 'Secret' and keep other things as is.

Click on Create

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2xxgpv0tauouf544rgur.png)

Now, we need to provide the ECR image name for the backend which is 'backend' only.

Select the kind as 'Secret text', paste the backend repo name in 'Secret', and keep other things as is.

Click on Create

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0e2i4exzwjbdo5owdl62.png)

Final Snippet of all Credentials needed.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w9ty11wgqzkqohi1lpe5.png)

**Step 10: Install the required plugins and configure the plugins to deploy the Three-Tier Application**

Install the following plugins by going to Dashboard -> Manage Jenkins -> Plugins -> Available Plugins

```
Docker
Docker Commons
Docker Pipeline
Docker API
docker-build-step
Eclipse Temurin installer
NodeJS
OWASP Dependency-Check
SonarQube Scanner
```
To configure the installed plugins.

Go to Dashboard -> Manage Jenkins -> Tools

- To configure jdk

Search for 'jdk' and provide the configuration as in the below snippet.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ibnsm5zxxkmdlu8ragp7.png)

- To configure the sonarqube-scanner

Search for the sonarqube scanner and provide the configuration like the below snippet.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/duy3afycqua4mciwarzr.png)

- Next is to configure nodejs

Search for node and provide the configuration like the below snippet.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/y7s8cjer36uifcdqa5t2.png)

- Next is to configure the OWASP Dependency check

Search for Dependency-Check and provide the configuration like the below snippet.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3woxadnkevyjjfhq57uy.png)

- To configure docker

Search for docker and provide the configuration like the below snippet.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gd271j7cqd3ar5sopeof.png)

Next is to set the path for Sonarqube in Jenkins

Go to Dashboard -> Manage Jenkins -> System

Search for SonarQube installations

Provide the name as it is, then in the Server URL copy the sonarqube public IP (same as Jenkins) with port 9000 select the sonar token that we have added recently, and click on Apply & Save.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ujw2224ogihr5oa5pxf0.png)

Now, we are ready to create our Jenkins Pipeline to deploy the Backend Code.

Go to Jenkins Dashboard

Click on 'New Item'
Provide the name of the Pipeline and click on OK.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fq04gs25nr5p571tqqq4.png)

This is the Jenkins file to deploy the Backend Code on EKS.

Copy and paste it into the Jenkins

https://github.com/abdulxs/DevSecOps-Project/blob/master/Jenkins-Pipeline-Code/Jenkinsfile-Backend

Click Apply & Save.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qtsylle7bczd5xqic9yg.png)

Now, click on the build.

The pipeline was successful after troubleshooting some errors.

Note: Make changes in the Pipeline according to your project.

![backend pipeline successful](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7cwk0mp677mfxzzelg5y.png)

Now, we create the Jenkins Pipeline to deploy the Frontend Code.

Go to Jenkins Dashboard

Click on 'New Item'

Provide the name of the Pipeline and click on OK.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7jq1ui010rqyijcj274b.png)

This is the Jenkins file to deploy the Frontend Code on EKS.

Copy and paste it into the Jenkins

https://github.com/Abdulxs/DevSecOps-Project/blob/master/Jenkins-Pipeline-Code/Jenkinsfile-Frontend

Click Apply & Save.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kphvriqdvgkk9amru2kf.png)

Now, click on the build.

The pipeline was successfully run.

Note: Make changes in the Pipeline according to your project.

![FE pipeline successful](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/968v8xjcc2ok420crkfx.png)

**Step 10: Set up Monitoring for the EKS Cluster for observability**

We will achieve the monitoring using Helm.
Add the prometheus repo by using the below command

`helm repo add stable https://charts.helm.sh/stable`
`helm repo add prometheus-community https://prometheus-community.github.io/helm-charts`

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9m4lf37de8qao4soa4ns.png)

Install the prometheus

```helm install stable prometheus-community/kube-prometheus-stack```

Now, check the service with the below command

`kubectl get svc`

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6dicpcc81vw2msfex3u9.png)

Next is to access the Prometheus and Grafana consoles from outside of the cluster.

For that, we need to change the Service type from ClusterType to LoadBalancer

Edit the stable-kube-prometheus-sta-prometheus service

```kubectl edit svc stable-kube-prometheus-sta-prometheus```

Modify the 48th line from ClusterType to `LoadBalancer`

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3d0v3xop1ogbfq8d4sis.png)

Edit the stable-grafana service

`kubectl edit svc stable-grafana`

Modify the 39th line from ClusterType to `LoadBalancer`

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ehsen9bs2pgmqx3t02xs.png)

Now, if you list again the service then, you will see the LoadBalancers DNS names

`kubectl get svc`

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l7e0btfqu4ejut0evwtb.png)

You can also validate from the console.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s9bdzx8rmlk50l1xzzar.png)

Now, to access the Prometheus Dashboard

Paste the <Prometheus-LB-DNS>:9090 in the browser and you will see like this

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1j6lfzr8brcrip40k2tz.png)

Click on Status and select Target.

A lot of Targets will show up on the page.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hev9nxvv7am6xg9uibnv.png)

To access the Grafana Dashboard

Copy the ALB DNS of Grafana and paste it into the browser.

The username will be 'admin' and the password will be 'prom-operator' for the Grafana LogIn.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jutmbtmf9sg51rmzen9c.png)

Next, click on Data Source

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yh0j96lusohrlvtiwkh7.png)
Select Prometheus
In the 'Connection', paste your <Prometheus-LB-DNS>:9090.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v7obnhiljv9a3rr8sabc.png)

A correct URL will see result in a green notification.

Click on Save & test.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wx8x6bwpyl7zchr3w1l2.png)

Now, we will create a dashboard to visualize our Kubernetes Cluster Logs.

Click on Dashboard.
Once you click on Dashboard. You will see a lot of Kubernetes components monitoring.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kdpu1qqxqyv354j6irpu.png)

A certain type of Kubernetes Dashboard will be imported.

Click on New and select Import

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/veyqdhk3le3f1rujqy8e.png)

Provide '6417' ID and click on Load

Note: 6417 is a unique ID from Grafana which is used to Monitor and visualize Kubernetes Data

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hk14bjoi2g9e45dkbf8i.png)

Select the data source that you have created earlier and click on Import.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8xmyy7lj9buquq5nlne1.png)

Now we can view the visualized Kubernetes Cluster Data.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/c127snp8jbtb5x560a8l.png)

**Step 11: Deploy the Three-Tier Application using ArgoCD.**

As the repository is private. So, we need to configure the Private Repository in ArgoCD.

Click on Settings and select Repositories

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7kf5eybm2rvdbg1ofkhv.png)

Click on 'CONNECT REPO USING HTTPS'

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/o0zsmmxe2ns1u0l1uvew.png)

Next, provide the repository name where the Manifest files are present.

Provide the username and GitHub Personal Access token and click on CONNECT.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nuy03485hmbbjt1wznwq.png)

The Connection Status is Successful.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f6asfpscjuq60p0xzgz1.png)

Next, create the first application which will be a database.

Click on 'CREATE APPLICATION'.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/o31lznvxuuuhtfyl0lwn.png)

Provide the details as it is provided in the below snippet and scroll down.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4m0mloi0tmgs98krjitz.png)

Select the same repository that was configured in the earlier step.

In the Path, provide the location where the Manifest files are present and provide other details as shown in the below screenshot.

Click on CREATE.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k3jgctdmbrg0dmkzcouy.png)

While the database Application is starting to deploy, We will create an application for the backend.

Provide the details as provided in the below snippet and scroll down.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rvht77g6mqw7q86j17m4.png)

Select the same repository as configured in the earlier step.

In the Path, provide the location where the Manifest files are present and provide other details as shown in the below screenshot.

Click on CREATE.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mpox6dhzhzl0o2kq045o.png)

While the backend Application is starting to deploy, We will create an application for the frontend.

Provide the details as provided in the below snippet and scroll down.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/npx9vfrpe5tl1uebi5zy.png)

Select the same repository as configured in the earlier step.

In the Path, provide the location where the Manifest files are present and provide other details as shown in the below screenshot.

Click on CREATE.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/edwjz4723w0t3vjes510.png)

While the frontend Application is starting to deploy, We will create an application for the ingress.

Provide the details as it is provided in the below snippet and scroll down.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x4zohn2glu9oegkg73mw.png)

Select the same repository as configured in the earlier step.

In the Path, provide the location where the Manifest files are present and provide other details as shown in the below screenshot.

Click on CREATE.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l260njpxis11j2ifglit.png)

Once the Ingress application is deployed, it will create an Application Load Balancer.

You can validate this as the load balancer named k8s-three on the console.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8m4nzreh80p7omhpn2iq.png)

Now, Copy the ALB-DNS and go to the Domain Provider; in my case Hostinger is the domain provider.

Create a CNAME record for the ALB-DNS.

Note: The subdomain in use here is backend.lagbaja.tech

All 4 application deployments are now successful

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g663w3qi631fkatgunit.png)

Now we can try to access the application from the browser

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/oj0d9myjyob8mefoy0zg.png)

We explore the Grafana Dashboard to view the EKS data such as pods, namespace, deployments, etc

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ofidnagyt81irb6dtwmo.png)

This is the Ingress Application Deployment in ArgoCD

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pbu9e4r6g4pkl2oc6twm.png)

This is the Frontend Application Deployment in ArgoCD

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gun0xaiy2r6vy65y9uv1.png)

This is the Backend Application Deployment in ArgoCD

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fojc77xblt7lby0rutgh.png)

This is the Database Application Deployment in ArgoCD

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b6jvb8cz8x6jkjgr19xf.png)

As observed, Persistent Volume & Persistent Volume Claim have been configured. So, if the pods get deleted, the data won’t be lost. The Data will be persisted on the host machine.

Conclusion:
In this comprehensive DevSecOps Kubernetes project, the following have been achieved:

* Established IAM user and Terraform for AWS setup.
* Deployed Jenkins on AWS, configured tools, and integrated it with Sonarqube.
* Set up an EKS cluster, configured a Load Balancer, and established private ECR repositories.
* Implemented monitoring with Helm, Prometheus, and Grafana.
* Installed and configured ArgoCD for GitOps practices.
* Created Jenkins pipelines for CI/CD, deploying a Three-Tier application.
* Ensured data persistence with persistent volumes and claims.

Clean up Resources
Run `Terraform destroy` and go to the AWS console to delete other resources created to avoid incurring extra costs. 
Delete ECR repo, dynamodb table, the EKS nodegroup, eks cluster, nat gateway. Detach and delete network interfaces,
delete vpc, security groups, load balancers and other resources.