Deploying an AWS EKS with StatefulSet and EBS Volume using Terraform. Kubernetes Series - Episode 2
Tags: terraform, eks, aws, storage
Published: 2023-12-11 16:46:55

Cloud people!

In the last Episode covered steps to set up a VPC, subnets, route tables, NAT gateways, and deploying an EKS cluster with managed node groups. 

{% embed https://dev.to/segoja7/deploying-an-aws-eks-using-terraform-kubernetes-series-episode-1-2fph %}

In this episode, the focus shifts to deploying the first application within Kubernetes while ensuring persistent storage for the application's data even when pods are ephemeral.

## Requirements
* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html)
* [Kubectl](https://kubernetes.io/docs/tasks/tools/)
* [Terraform](https://www.terraform.io/downloads.html)
* [k9s](https://k9scli.io/topics/install/)

Let's see how we can do this using terraform and raw manifest.

##Reference Architecture
The reference architecture operates within a node.

![Statefulset architecture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8jkdu28h2jk3uo9txjqp.png)

Currently, the focus is on adding add-ons to the EKS cluster, considering that the EKS cluster was deployed in the previous episode.

## Step 1.
Adding the aws-ebs-csi-driver allows the utilization of the EBS driver for the application.

This is the link for all code terraform.{% embed https://github.com/segoja7/EKS.git %} 

```
  cluster_addons = {
    coredns = {
      most_recent = true
    }
    kube-proxy = {
      most_recent = true
    }
    vpc-cni = {
      most_recent = true
    }
    aws-ebs-csi-driver = {
      most_recent = true
    }
  }
```
## Step 2.

It is very important that the next step is to add the permissions policy in the node configuration so that the driver can create the EBS Volume, for this it is necessary to add the following code that allows to use a managed policy for the ebs csi driver.

```
  iam_role_additional_policies = {
      AmazonEBSCSIDriverPolicy  = "arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy"
  }
```

When the eks driver addon is installed, it is possible to list the ebs pods that are running in the kube-system namespace.

```
kubectl get pods -n kube-system
```
![Get Pods](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mnd0hkswba4yfez1qm4l.png)

```
kubectl get storageclass -n kube-system
```

![Get StorageClass](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uuf8d5g4ztr263ru5j6u.png)

Now, that the EBS driver is installed. It is possible to build an application using the raw manifests.

##Step 3.
First apply the service. Remember, you must create the namespace manually.

```
kubectl create ns episode2
```

```
kubectl apply -f manifests/service.yaml
```

```
apiVersion: v1
kind: Service
metadata:
  name: mysql-svc
  namespace: episode2
  labels:
    app: mysql-db
spec:
  selector:
    app: mysql-db
  ports:
    - name: mysql-db
      port: 3306
      targetPort: 3306
  clusterIP: None
```
Output:
```
service/mysql-svc created
```

##Step 4. 
Now, it is necessary to use a stateful set, there are some reasons to use a stateful set and not a deployment and the first reason is that the app must have persistent storage.

Deployments are excellent for apps stateless, and this is not the case.

```
kubectl apply -f manifests/statefulset.yaml
```

```
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mysql-db
  namespace: episode2
spec:
  serviceName: mysql-svc
  replicas: 2
  selector:
    matchLabels:
      app: mysql-db
  template:
    metadata:
      labels:
        app: mysql-db
    spec:
      containers:
      - name: mysql-db
        image: mysql:latest
        ports:
          - containerPort: 3306
        volumeMounts:
          - name: statefulset-storage
            mountPath: /var/lib/mysql
        env:
          - name: MYSQL_ROOT_PASSWORD
            value: "segoja7secure!" #This is not recommend, use secrets!
          - name: MYSQL_USER
            value: "user" #This is not recommend, use secrets!
          - name: MYSQL_PASSWORD
            value: "episode2" #This is not recommend, use secrets!
  volumeClaimTemplates:
  - metadata:
      name: statefulset-storage
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "gp2"
      resources:
        requests:
          storage: 2Gi

```
Output:
```
statefulset.apps/mysql-db created
```
> Note: Now, the storage class was automatically configured when the cluster was created.

In the aws console, these are EBS volumes that were created.


![AWS console EBS](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jsdnkyxwptjxrnbssll9.png)


##Step 5.

StorageClass:

![K9s storageclass](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dd452hdsgf2bbqekpdh5.png)

pvc:

![k9s pvc](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dn2o9nd1sf6d0i45dqkt.png)

##Step 6.
Now you can perform a storage validation, create a database and put some data there.

First it is necessary to perform a port forwarding to connect to the database from a client, using k9s is easy.

![k9s port forward](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/m5i6vy9xg0av3w28p42m.png)

Now, you can access from the client, in this case the client is heidi, you can use any client.


![Heidi client](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/o7l347bic9xpw1n56q1c.png)

From customer, a database named **episode2** has been created, additional a table named **mytable** with two rows **FirstName and LastName**.

![Heidi table](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wynho7mnfo4qmo897bhp.png)

Adding some data.

![Heidi table with data](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/so3a09js4i7volfhgbk1.png)


##Step 7.

Recreating pods of statefulset.

In this step, the statefulset pods have been removed so that they are recreated, the age is only seconds. 


![Recreating pods statefulset](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v7uzd3rs4762hcw3e14c.png)


By connecting back on a Heidi client to the mysql pod, you can see that the data is persistent and the database, table and data are stored in the ebs volume, which shows that the data is being persisted beyond the lifecycle of the pod.

![Validating data persistent](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l72udv2d9retiqkktflr.png)


In this phase, a statefulset database pod was created using the ebs volume.

The following addons have been installed: 
• ebs-csi

Successful!!