Install & Manage Amazon EKS Add-ons with Terraform
Tags: aws, terraform, kubernetes
Published: 2022-01-30 22:47:09


### What is an Add-on in Amazon EKS ?
An add-on is software that provides supporting operational capabilities to Kubernetes applications, but is not specific to the application.
This includes software like observability agents or Kubernetes drivers that allow the cluster to interact with underlying AWS resources for networking, compute, and storage.

> **Reference:** https://docs.aws.amazon.com/eks/latest/userguide/eks-add-ons.html

---

### Amazon EKS Add-ons
**Amazon VPC CNI** 
The Amazon VPC CNI add-on for Kubernetes is the networking plugin for pod networking in Amazon EKS clusters. The plugin is responsible for allocating VPC IP addresses to Kubernetes nodes and configuring the necessary networking for pods on each node.

> Reference: https://docs.aws.amazon.com/eks/latest/userguide/cni-iam-role.html

**CoreDNS**
CoreDNS is a flexible, extensible DNS server that can serve as the Kubernetes cluster DNS. When you launch an Amazon EKS cluster with at least one node, two replicas of the CoreDNS image are deployed by default, regardless of the number of nodes deployed in your cluster. The CoreDNS pods provide name resolution for all pods in the cluster. 

> Reference: https://docs.aws.amazon.com/eks/latest/userguide/managing-coredns.html

**kube-proxy**
Kube-proxy maintains network rules on each Amazon EC2 node. It enables network communication to your pods.

> Reference: https://docs.aws.amazon.com/eks/latest/userguide/managing-kube-proxy.html

**Amazon EBS CSI**
The Amazon Elastic Block Store (Amazon EBS) Container Storage Interface (CSI) driver allows Amazon Elastic Kubernetes Service (Amazon EKS) clusters to manage the lifecycle of Amazon EBS volumes for persistent volumes.

> Reference: https://docs.aws.amazon.com/eks/latest/userguide/ebs-csi.html
https://docs.aws.amazon.com/eks/latest/userguide/managing-ebs-csi.html

Amazon EKS automatically installs self-managed add-ons such as the **Amazon VPC CNI**, **kube-proxy**, and **CoreDNS** for every cluster, but you can change the default configuration of the add-ons and update them when desired.

> **Amazon EBS CSI Driver** is now available as an Add-on. This Add-on is in preview version with some limitations and inconsistencies. For this reason, its use in Production is not recommended.

This post will show the step by step how to install the Add-ons with terraform


### Prerequisites

- Amazon EKS cluster running Kubernetes version 1.18 and later

> To create an EKS cluster you can check this post https://dev.to/aws-builders/creating-an-eks-cluster-and-node-group-with-terraform-1lf6

- AWS CLI installed 

> https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html

- An IAM role with the **AmazonEKS_CNI_Policy**  attached to it

---
### STEPS

#### Step 01 - Get the version of each Add-on
To know the all version of each Add-on we can use AWS CLI to describe all information about the Add-ons

```javascript
aws eks describe-addon-versions > addons.json
```
This command allows to describe all add-on versions and for more confort we can save the output in a JSON file

The resulting JSON file will be like this

```javascript
{
    "addons": [
        {
            "addonName": "kube-proxy",
            "type": "networking",
            "addonVersions": [
                {
                    "addonVersion": "v1.21.2-eksbuild.2",
                    "architecture": [
                        "amd64",
                        "arm64"
                    ],
                    "compatibilities": [
                        {
                            "clusterVersion": "1.21",
                            "platformVersions": [
                                "*"
                            ],
                            "defaultVersion": true
                        }
                    ]
                },
...
```
This JSON file contains all information about the EKS cluster Add-ons, like **name**, **type**, **compatibilities**, etc.

Take a note of each add-on version to specify them in the terraform code, according to the cluster version


#### Step 02 - Create the terraform code

First, We can create a variable of the type **list(object())** to specify all names and versions of each Add-ons, based on the information that we get from the JSON file obtained in the last step

```javascript
variable "addons" {
  type = list(object({
    name    = string
    version = string
  }))

  default = [
    {
      name    = "kube-proxy"
      version = "v1.21.2-eksbuild.2"
    },
    {
      name    = "vpc-cni"
      version = "v1.10.1-eksbuild.1"
    },
    {
      name    = "coredns"
      version = "v1.8.4-eksbuild.1"
    },
    {
      name    = "aws-ebs-csi-driver"
      version = "v1.4.0-eksbuild.preview"
    }
  ]
}
```

The AWS Provider for Terraform has a specific resource to install and manage the Add-ons.

Using a `for_each` statement we can iterate all information in the `addons` variable

```javascript
resource "aws_eks_addon" "addons" {
  for_each          = { for addon in var.addons : addon.name => addon }
  cluster_name      = aws_eks_cluster.eks-cluster.id
  addon_name        = each.value.name
  addon_version     = each.value.version
  resolve_conflicts = "OVERWRITE"
}
```


#### Step 03 - Apply the terraform code
To apply the new resource we can use 

```
terraform apply -auto-approve
```

Once the installation is finished, we can check the add-ons using the UI

![EKS Add-ons](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mkgsutssmxf70x5rqgoz.PNG)

The installation was successful so we can see the `Status` of each Add-on is `Active`

Another way to check that everything is can be using `kubectl`

```
kubectl get po -n kube-system
```
With this command we can check if the pods of each Add-on are running