Schedule Access Key Rotation Using CodeCatalyst
Tags: aws, cicd, codecatalyst, iam
Published: 2024-03-03 08:16:49

To interact with services/resources in Amazon Web Services (AWS), commonly We need security credentials. Security credentials are needed for authentication and authorization. There are two types of security credentials, long term and temporary. Both have its own use cases, but temporary credentials is more recommended.

Despite using temporary security credentials is more recommended, maybe in some use cases We still need or have to use long term credentials such as IAM user access key and secret key. To prevent security hole, We can and encouraged to rotate access key and secret key periodically.

More details about AWS security credentials: [[+]](https://docs.aws.amazon.com/IAM/latest/UserGuide/security-creds.html)

## Table of Contents

1. [Scenario](#scenario)
2. [Prerequisites](#prerequisites)
3. [Configuration Steps](#configuration-steps)
    - [Directory Structure](#directory-structure)
    - [Access Key Rotation Script Configurations](#access-key-rotation-script-configurations)
    - [CodeCatalyst Workflow File Configurations](#codecatalyst-workflow-file-configurations)
    - [Example of Automate Access Key Rotation in CI/CD CodeCatalyst](#example-of-automate-access-key-rotation-in-cicd-codecatalyst)
4. [Summary](#summary)

## Scenario

In this blog, I will create CI/CD pipeline in Amazon CodeCatalyst or in CodeCatalyst also called as workflow, where this workflow will run scheduled automated IAM user access key rotation. First, CodeCatalyst will delete old access key, then it will create new access key. In the process, CodeCatalyst workflow will run shell scripts that has been configured to run aws cli commands that needed for each steps.

## Prerequisites

Here are some prerequisites that needed to follow configurations discussed in this blog:

- CodeCatalyst space and project that has been connected with AWS account and source repository. In this blog, I use GitHub as source repository. For configuration reference, please check my other blog “[Build and Release Container Image to Amazon Elastic Container Registry (ECR) via Amazon CodeCatalyst](https://aws.plainenglish.io/build-and-release-container-image-to-amazon-elastic-container-registry-ecr-via-amazon-ae7f9f07a135)”
- IAM user

## Configuration Steps

### Directory Structure

Below is directory structure used in this blog:

```
.
├── .codecatalyst/
│   └── workflows/
│       └── rotate-access-key.yml
└── sandbox2/
    └── iam/
        ├── create-access-key.sh
        └── delete-access-key.sh
```

### Access Key Rotation Script Configurations

Configuration started with create shell scripts that will be used for automate access key rotation.

1. Create shell script **delete-access-key.sh** that will be used to check whether IAM user has access key and if user has, script will delete that access key. In this script, We need to export environment variable value `IAM_USER` that used to define which IAM user will be checked

    ```bash
    #!/usr/bin/env bash

    set -euo pipefail
    
    IAM_USER=$IAM_USER
    
    ACCESS_KEY=$(aws iam list-access-keys \
      --user-name $IAM_USER \
      --query 'AccessKeyMetadata[].AccessKeyId' \
      --output text \
      --no-cli-pager)
    
    if [ -z "$ACCESS_KEY" ]; then
      echo "No access key found on user $IAM_USER"
    else  
      echo "Found access key $ACCESS_KEY on user $IAM_USER"
    
      DELETE_KEY=$(aws iam delete-access-key \
        --user-name $IAM_USER \
        --access-key-id $ACCESS_KEY)
    
      echo "Access key $ACCESS_KEY deleted"
    fi
    ```

2. Create shell script **create-access-key.sh** that will be used to create/generate new access key. Similar with **delete-access-key.sh**, We need to export enviroment variable value `IAM_USER`. Output of access key creation will be saved into file called **access-key.csv**

    ```bash
    #!/usr/bin/env bash
     
    set -euo pipefail
     
    IAM_USER=$IAM_USER
     
    CREATE_KEY=$(aws iam create-access-key \
      --user-name $IAM_USER\
      --output text)
     
    echo "Creating access key for user $IAM_USER"
    echo "$CREATE_KEY" > ./access-key.csv
    echo "Access key created. Please find the key values on access-key.csv"
    
    ```

### CodeCatalyst Workflow File Configurations

After created shell script for automate access key rotation, then We need to define workflow file that will be used to schedule automation of access key rotation.

I configured that workflow in file **.codecatalyst/workflows/rotate-access-key.yml**. Here is the full content of workflow file:

```yaml
Name: rotate-access-key
SchemaVersion: "1.0"

Triggers:
  - Type: SCHEDULE
    Expression: "30 12 ? * THU *"
    Branches:
      - master

Actions:
  delete-access-key:
    Identifier: aws/build@v1
    Inputs:
      Sources:
        - WorkflowSource
      Variables:
        - Name: IAM_USER
          Value: "tf-admin"
    Environment:
      Name: sandbox
      Connections:
        - Name: lanandra-sandbox
          Role: codecatalyst-admin
    Configuration:
      Steps:
      - Run: cd sandbox2/iam
      - Run: ./delete-access-key.sh
    Compute:
      Type: EC2
  create-access-key:
    DependsOn:
      - delete-access-key
    Identifier: aws/build@v1
    Inputs:
      Sources:
        - WorkflowSource
      Variables:
        - Name: IAM_USER
          Value: "tf-admin"
    Outputs:
      Artifacts:
        - Name: ACCESSKEY
          Files:
            - sandbox2/iam/access-key.csv
    Environment:
      Name: sandbox
      Connections:
        - Name: lanandra-sandbox
          Role: codecatalyst-admin
    Configuration:
      Steps:
      - Run: cd sandbox2/iam
      - Run: ./create-access-key.sh
    Compute:
      Type: EC2
```

Next I will explain more details each section inside workflow file.

1. Define workflow name. Then define how workflow run. In this blog, as an example, workflow will be ran as scheduled on Thursday at 12:30 UTC and only will be ran on `master` branch. Please adjust schedule following cron expression guide that used by CodeCatalyst. CodeCatalyst use six-field syntax and separated with space, more details please check this documentation: [CodeCatalyst workflow triggers expression](https://docs.aws.amazon.com/codecatalyst/latest/userguide/workflow.top.level.html#workflow.triggers.expression) & [EventBridge cron expressions reference](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-cron-expressions.html)

    ```yaml
    Name: rotate-access-key
    SchemaVersion: "1.0"
    
    Triggers:
      - Type: SCHEDULE
        Expression: "30 12 ? * THU *"
        Branches:
          - master
    ```

2. Define actions. Action will be divided into 2 sections. First section will run shell script **delete-access-key.sh** in `action delete-access-key`. As mentioned above, we need to export environment variable value `IAM_USER`. In this blog, I will use example user called `tf-admin`. Action will be ran in sandbox environment that has been connected with AWS account and also has IAM role associated with that account. In configuration section, I defined steps how to run shell script **delete-access-key.sh** from directory where the file located

    ```yaml
    Actions:
      delete-access-key:
        Identifier: aws/build@v1
        Inputs:
          Sources:
            - WorkflowSource
          Variables:
            - Name: IAM_USER
              Value: "tf-admin"
        Environment:
          Name: sandbox
          Connections:
            - Name: lanandra-sandbox
              Role: codecatalyst-admin
        Configuration:
          Steps:
          - Run: cd sandbox2/iam
          - Run: ./delete-access-key.sh
        Compute:
          Type: EC2
    ```

3. Next section will run shell script **create-access-key.sh** in action `create-access-key`. This action will depends on previous action which is `delete-access-key` and this action will not run if previous action is failed. More or less this action will be the same with action `delete-access-key`. We need to export environment variable value `IAM_USER`. Similar with previous action, i will input example user called `tf-admin`. Action will be ran in sandbox environment that has been connected with AWS account and also has IAM role associated with that account. In configuration section, there will be different steps, I defined steps to run shell script **create-access-key.sh** from directory where the file located. Beside that, this action will generate output artifact called `ACCESSKEY`, this artifact consist of file `access-key.csv` where information releted access key and secret key stored

    ```yaml
    create-access-key:
        DependsOn:
          - delete-access-key
        Identifier: aws/build@v1
        Inputs:
          Sources:
            - WorkflowSource
          Variables:
            - Name: IAM_USER
              Value: "tf-admin"
        Outputs:
          Artifacts:
            - Name: ACCESSKEY
              Files:
                - sandbox2/iam/access-key.csv
        Environment:
          Name: sandbox
          Connections:
            - Name: lanandra-sandbox
              Role: codecatalyst-admin
        Configuration:
          Steps:
          - Run: cd sandbox2/iam
          - Run: ./create-access-key.sh
        Compute:
          Type: EC2
    ```

### Example of Automate Access Key Rotation in CI/CD CodeCatalyst

After configure CodeCatalyst workflow file, next we verify scheduled automation access key rotation is working as expected.

1. Go to CodeCatalyst web console, choose space and project where workflow resided. Go to menu CI/CD — Workflows. Find workflow name. In section Recent runs, verify there is a successful scheduled run

    ![rotate-access-key-workflow](https://ludesdeveloper.files.wordpress.com/2024/02/1.-rotate-access-key-workflow.png)

2. Click action `delete-access-key`. In the right will appear window to check `Logs` related to that action. Verify shell script **delete-access-key.sh** is run and successfully found existing access key in user and delete that key

    ![delete-access-key-action](https://ludesdeveloper.files.wordpress.com/2024/02/2.-delete-access-key-action.png)

3. Click action `create-access-key`. In the right will appear window to check `Logs` related to that action. Verify shell script **create-access-key.sh** is run and successfully create access key

    ![create-access-key-action](https://ludesdeveloper.files.wordpress.com/2024/02/3.-create-access-key-action.png)

4. Go to `Artifacts` menu in related run. Verify artifact is successfully created and verify file can be downloaded

    ![rotate-access-key-artifact](https://ludesdeveloper.files.wordpress.com/2024/02/4.-rotate-access-key-artifact.png)

5. Open csv file inside the artifact, verify information related access and secret key. Verify user is match with what defined in workflow environment variable

    ![verify-access-key-in-csv](https://ludesdeveloper.files.wordpress.com/2024/02/5.-verify-access-key-in-csv.png)

6. For validation, go to AWS web console. Go to IAM user menu and find related user. Verify access-key is match with what defined inside csv file

    ![verify-access-key-in-web](https://ludesdeveloper.files.wordpress.com/2024/02/6.-verify-access-key-in-web.png)

## Summary

We have reached the last section of this blog. Here are some key takeaways that can be summarized:

- There are two types of security credentials in AWS, long term and temporary
- Temporary credentials is more recommended. But in some cases or environments, we may need or have to use long term credentials such as access key
- To prevent security hole, please ensure to rotate access key periodically
- CodeCatalyst can be used as tool to schedule automation of access key rotation

Please check out my GitHub repository to see source code example that used in this blog:

- Shell scripts used for access key rotation: [rotation scripts](https://github.com/lanandra/mock-aws-infrastructure/tree/master/sandbox2/iam)
- Amazon CodeCatalyst workflow file: [codecatalyst workflow](https://github.com/lanandra/mock-aws-infrastructure/tree/master/.codecatalyst/workflows)

Please comment if you have any suggestions, critiques, or thoughts.

Hope this article will benefit you. Thank you!