Access data in S3 securely from Pods running in AWS EKS
Published: 2023-03-19 15:35:38

The **AWS IAM (Identity & Access Management)** service allows AWS services to interact with each other based on the policies given to its attached role(s).


![AWS IAM Roles association with EKS](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9ba9itmjp9onddig7euv.png)


We can also use the IAM role with a Kubernetes (k8s) native Service Account (SA) which will allow the Pods running in the Kubernetes cluster or AWS Elastic Kubernetes Service (EKS) to talk to AWS service(s).

In this blog, we will see how we can allow a Pod running in AWS EKS to list the objects in the AWS S3 bucket by using the IAM role with Kubernetes native Service Account.

Go to AWS S3 service and create a bucket and then add few objects to it.


![AWS S3 Bucket Setup](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gecwxcu0aw5hkqf8lbqo.png)


![AWS S3 Bucket Setup](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/90k9pmtq584duy1bdew2.png)


![AWS S3 Bucket Setup](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wq7p7g693a0xe8irjsl0.png)


**Create an Identity Provider:-**
a) Copy the OIDC (OpenID Connect) provider URL from the existing AWS EKS cluster, for instance, in my case this is the URL https://oidc.eks.us-east-1.amazonaws.com/id/8B7D06AD395F38CE1EE8EF0AF2922255


![OIDC setup](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/78ukx8e1bti1fbs5z1w2.png)



b) Go to IAM -> Identity Providers and click on the ‘Add providers’ button and then select ‘OpenID Connect’.


![OIDC Setup](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8uefkhz6l37kq4m26fce.png)

Paste the copied OIDC URL (from EKS) under the ‘Provider URL’ option and click the ‘Get thumbprint’ button and put sts.amazonaws.com under the ‘Audience’ option as shown below. Give tags as necessary and click the ‘Add provider’ button at the end to add the identity provider.

![OIDC Setup](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qs4astvmfyxvbhio07yp.png)

**Create an IAM Policy to read objects in the S3 bucket:-**
Go to IAM Policy and create a custom policy with the following JSON to read the objects from the S3 bucket:-

`{
 “Version”: “2012–10–17”,
 “Statement”: [
    {
      “Effect”: “Allow”,
      “Action”: [
               “s3:ListBucket”,
               “s3:GetObject”
       ],
       “Resource”: “arn:aws:s3:::k8s-nest”
    }
 ]
}`
**Create an IAM Role with Trust Relationship with Identity Provider:-**
Go to the IAM and create a new role with ‘Web identity’ trust type and select the right Identity Provider (from the dropdown) that we created earlier and Audience as sts.amazonaws.com and click on the ‘Next: Permissions’ button as shown below:-

![IAM Role](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5wgsjvjpsepi22m2qjn5.png)

Attach the previously created. custom policy to read the S3 bucket objects:-

![IAM Role](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ygm1rkagn7zj5m210247.png)

![IAM Role](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wzsyfiglhsofigfiirz5.png)

Give a name to the role and create it.

Then open the same role again and edit its trust relationship to make sure that only a specific Kubernetes Service Account can assume this Role.

![IAM Role](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p2zzmh77xqsthquwcz70.png)

![IAM Role](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7otmqjf2ibr9k8c981ws.png)

![IAM Role](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ejqfr60z06lxbc4dbtpw.png)

Following policy, will only allow the Kubernetes service account named ‘my-sa’ (in the namespace ‘dev’) to assume the role ‘custom-read-s3-bucket-objects’ using the AWS STS AssumeRoleWithWebIdentity.

`{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::195725532069:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/8B7D06AD395F38CE1EE8EF0AF2922255"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "oidc.eks.us-east-1.amazonaws.com/id/8B7D06AD395F38CE1EE8EF0AF2922255:sub": "system:serviceaccount:dev:my-sa"
        }
      }
    }
  ]
}`
All configurations at the AWS side are done. Now head towards, your CLI (Command Line Interface) to create a Service Account and Pod in the same AWS EKS Cluster.

**Create a namespace ‘dev’ in Kubernetes:-**
`kubectl create ns dev`

**Create a Service Account named ‘my-sa’:-**
Create a service account annotating the IAM Role ARN created before. as shown below.

![IAM Role](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1bfu5jdwmjqnwjtlaxci.png)

`apiVersion: v1
kind: ServiceAccount
metadata:
 name: my-sa
 namespace: dev
 annotations:
   eks.amazonaws.com/role-arn:  arn:aws:iam::195725532069:role/custom-read-s3-bucket-objects`

`kubectl create -f my-sa.yaml`

![k8s Pods](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jxfspul44t2p2cwubrf1.png)

**Schedule a new Pod using this Service Account:-**
Instead of the default service account, use the one which was created above i.e. my-sa, and attach it to the Pod as shown below.

`
apiVersion: v1
kind: Pod
metadata:
  labels:
    run: my-pod
  name: my-pod
  namespace: dev
spec:
  serviceAccountName: my-sa
  initContainers:
  - image: amazon/aws-cli
    name: my-aws-cli
    command: ['aws', 's3', 'ls', 's3://k8s-nest/']
  containers:
  - image: nginx
    name: my-pod
    ports:
    - containerPort: 80
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
`

`kubectl create -f my-pod.yaml`


![k8spods](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bj7ljo6qd3iylk70ia7u.png)


Now, check the logs of the init container, you will notice that the Pod was successfully able to assume the role and communicate with AWS S3 to list all the objects in it securely with the principle of least privilege reducing the blast radius.

`kubectl logs my-pod my-aws-cli -n dev`


![k8s pods](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xw9c76261ebgszg5jdl7.png)


![k8s pods](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wfuitg93s0i4zl8pue5r.png)


Hope you liked this article :)


**Summary:**
In this blog, we learned that how we can use the AWS IAM Role and create Trust Relationship with the Kubernetes Service Account and use that service account with the Pods running inside the AWS EKS to make calls to AWS S3 with the principle of least privilege.

Behind the scene, the OIDC federation allows the Pod to assume the IAM role via the Kubernetes Service Account with AWS STS to receive a JSON Web Token (JWT). In Kubernetes, we then use the projected service account tokens which are valid OIDC tokens, giving each Pod encrypted signed tokens that can be verified by STS against the OIDC provider for establishing its identity by exchanging the OIDC tokens for IAM Role credentials using AssumeRoleWithWebIdentity of AWS STS.

As usual, you will find the complete source code here at my GitHub repository. Please feel free to fork it and add more IaC (Infrastructure as Code) to it:-

https://github.com/vinod827/k8s-nest/tree/main/iac/k8s/iam-roles-with-k8s-service-account