Seamlessly connect Gitlab CI/CD with AWS: Gitlab AWS credential helper
Tags: aws, cicd, gitlab
Published: 2023-10-19 12:11:29

---
title: Seamlessly connect Gitlab CI/CD with AWS: Gitlab AWS credential helper
published: true
date: 2023-10-01 22:08:00 UTC
tags: AWS,CICD,GITLAB
canonical_url: https://xebia.com/blog/seamlessly-connect-gitlab-cicd-with-aws-gitlab-aws-credential-helper/
---

In this blog we will show you how our Gitlab AWS credential helper will make it very easy to connect to an AWS account using the gitlab pipeline [id token](https://docs.gitlab.com/ee/ci/secrets/id_token_authentication.html) as your identity.

## Introduction

Since June 2022, Gitlab provides JWT identity tokens for each project pipeline, which can be exchanged for AWS access credentials using the [assume-role-with-web-identity](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRoleWithWebIdentity.html) API call. This is great, because it allows you to configure the AWS permissions for each individual pipeline. However the problem is that exchanging the token for credentials requires quite a bit of shell programming. Something along the following lines:

```
ROLE_ARN=$1
WEB_IDENTITY_TOKEN=$2
ROLE_SESSION_NAME=gitlab-${CI_PROJECT_PATH_SLUG}-${CI_PIPELINE_ID}

CREDENTIALS=($(aws sts assume-role-with-web-identity \
        --role-arn ${ROLE_ARN} \
        --role-session-name ${ROLE_SESSION_NAME} \
        --web-identity-token "${WEB_IDENTITY_TOKEN}" \
        --duration-seconds 3600 \
        --query 'Credentials.[AccessKeyId,SecretAccessKey,SessionToken]' \
        --output text))

aws configure set aws_access_key_id "${CREDENTIALS[0]}"
aws configure set aws_secret_access_key "${CREDENTIALS[1]}"
aws configure set aws_session_token "${CREDENTIALS[2]}"

```

As you can see that is quite a complex shell script. This will not look pretty in your `.gitlab-ci.yml`, and you will would probably add this as a separate script to your project. For a single project this is ok-ish, but to apply that to all your projects, is not going to scale well: As this script is not suitable for all situations, you probably end up with some variants as well.

### Gitlab AWS credential helper to the rescue

This is where the gitlab aws credential helper comes in! The utility is flexible and very easy to use in all sorts of situations. It is simple to configure, as it only needs the AWS account number and Gitlab id token. The Gitlab project path slug and pipeline id will be used to determine the IAM role- and session name. The general configuration of the CI/CD pipeline consists of the following five steps:

- [Add the Gitlab ID token](#add-gitlab-id-token)
- [Set the AWS variables](#set-aws-variables)
- [Configure the credential helper](#configure-the-credential-helper)
- [Create the IAM role for the pipeline](#create-the-pipeline-iam-role) 
- [Install the credential helper](#install-the-credential-helper)

Let’s us take you through these five steps in the following paragraphs.

### Add Gitlab ID token 

To obtain an Gitlab ID token in your pipeline jobs, add the following definition to your pipeline definition:

```
default:
  id_tokens:
    GITLAB_AWS_IDENTITY_TOKEN:
      aud: https://gitlab.com

```

This will cause each job to have an environment variable name `GITLAB_AWS_IDENTITY_TOKEN` with the JWT identity token signed by Gitlab as a value. The credential helper expects the token in this variable.

### Set AWS variables 

To know which AWS account to connect to, add the environment variable `GITLAB_AWS_ACCOUNT_ID` with your AWS account id:

```
variables:
  GITLAB_AWS_ACCOUNT_ID: '1234567898012'

```

If you have many gitlab projects, you can also set this variable as a CI/CD variable [on the group level](https://docs.gitlab.com/ee/ci/variables/#for-a-group). We recommend to add the following variables as well:

```
variables:
  - AWS_CONFIG_FILE: ${CI_PROJECT_DIR}/.aws/config
  - AWS_SHARED_CREDENTIALS_FILE: ${CI_PROJECT_DIR}/.aws/credentials
  - AWS_PROFILE: default

```

These will make sure the credential configurations are kept in scope of the pipeline build and that the obtained credentials are actually used.

### Using the credential helper

Now you are ready to use the credential helper, in one of three different ways:

- [as credential process](#as-credential-proces)
- [stored as shared credentials](#stored-shared-credentials)
- [as environment variables](#as-env-variables)

Let’s look at each of these options.

#### As credential process 

The most elegant use, is to configure the credential helper as [AWS credential process](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sourcing-external.html), like shown below:

```
before_script:
  - >
    aws config --profile default 
    set credential_process 
    "gitlab-aws-credential-helper process"

script:
  - aws --profile default sts get-caller-identity

```

The AWS SDK will call the credential helper whenever it requires to access to AWS!

#### Stored as shared credentials 

Alternatively, the helper can store the AWS credentials directly in the AWS shared credentials file. You can do this as follows:

```
before_script:
  - >
    gitlab-aws-credential-helper aws-profile 

script:
  - aws --profile default sts get-caller-identity

```

This will store the credentials in shared credentials file, normally ~/.aws/credentials.

#### Get as environment variables 

Finally, you can use the helper to export the environment variables AWS\_ACCESS\_KEY\_ID, AWS\_SECRET\_ACCESS\_KEY  
 and AWS\_SESSION\_TOKEN, as shown below:

```
script:
  - eval $(gitlab-aws-credential-helper env --export)
  - aws sts get-caller-identity

```

Alternatively, you can directly execute the command:

```
script:
  - gitlab-aws-credential-helper env -- aws sts get-caller-identity

```

### Create the pipeline IAM role

Once the pipeline is prepared, you need to add the IAM role for the CI/CD pipeline. The name of the role should be equal to the name of the gitlab project path slug as defined by the variable CI\_PROJECT\_PATH\_SLUG, prefixed with `gitlab-`. The following CloudFormation template shows an example IAM role definition:

```
AWSTemplateFormatVersion: '2010-09-09'
Description: Role for gitlab pipeline with limited AWS access
Parameters:
  RepositoryPath:
    Description: the gitlab repository path
    Type: String
  RepositoryPathSlug:
    Description: the gitlab repository path slug
    Type: String
Resources:
  GitlabPipeline:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub 'gitlab-${RepositoryPathSlug}'
      MaxSessionDuration: 3600
      AssumeRolePolicyDocument:
        Statement:
          - Action: sts:AssumeRoleWithWebIdentity
            Principal:
              Federated: !Sub 'arn:aws:iam::${AWS::AccountId}:oidc-provider/gitlab.com'
            Effect: Allow
            Condition:
              ForAnyValue:StringLike:
                "gitlab.com:sub":
                  - !Sub "project_path:${RepositoryPath}:ref_type:branch:ref:*"
      Policies:
        - PolicyName: MetaInformationAccess
          PolicyDocument:
            Statement:
              - Effect: Allow
                Action:
                  - sts:GetCallerIdentity
                Resource:
                  - '*'

```

The template assumes you have already created the OIDC provider for gitlab.com in your account. If not, checkout [this blog](https://xebia.com/blog/how-to-update-the-thumbprint-for-an-openid-connect-identity-provider-in-cloudformation/).

To create the role you need, type:

```
CI_PROJECT_PATH=mvanholsteijn/aws-credential-helper-demo
CI_PROJECT_PATH_SLUG=mvanholsteijn-aws-credential-helper
aws cloudformation deploy \
  --stack-name gitlab-$CI_PROJECT_PATH_SLUG \
  --template-file gitlab-pipeline-limited-access.yaml \
  --capabilities CAPABILITY_NAMED_IAM \
   --parameter-overrides \
     "RepositoryPath=$CI_PROJECT_PATH" \
     "RepositoryPathSlug=$CI_PROJECT_PATH_SLUG"

```

This all you need for one pipeline. The following two paragraphs will explain how to install the credential helper and show you how to override defaults.

### Install the credential helper 

Of course, your job will need to have the [gitlab-aws-credential-helper](https://github.com/binxio/gitlab-aws-credential-helper) installed. We recommend to add it to the image that runs your job using a multi-stage docker build file, as show below:

```
FROM ghcr.io/binxio/gitlab-aws-credential-helper:0.3.2 as gitlab-aws-credential-helper

FROM <yourimage>

COPY --from=gitlab-aws-credential-helper /usr/local/bin/gitlab-aws-credential-helper /usr/local/bin

```

If you do not manage your own runner image, you can add the following command to the before\_script section.

```
NAME=gitlab-aws-credential-helper
RELEASE=0.3.2
URL=https://github.com/binxio/$NAME/releases/download/${RELEASE}/${NAME}_${RELEASE}_linux_amd64.tar.gz
CHECKSUM=b8c84f1ca5622f4b0f529eca74e7689ea20418eab31a96666366ae1e1531b41f
ARCHIVE="$(basename $URL)"

cd /tmp && \
    curl -L -O -sS $URL && \
    (echo "$CHECKSUM $ARCHIVE" | sha256sum -c -) && \
    tar -C /usr/local/bin/ -xzf $ARCHIVE && \
    rm -f $ARCHIVE

```

### Overriding defaults

The credential helper uses sensible defaults to make it as easy as possible. Of course, you can override all options via environment variables or via the command line. The following example shows how to create different profiles for a named role in different accounts:

```
before_script:
  ->
    gitlab-aws-credential-helper
    aws-profile --name development
    --aws-account 111111111111 
    --role Deployer
  ->
    gitlab-aws-credential-helper
    aws-profile --name test
    --aws-account 222222222222 
    --role Deployer
  ->
    gitlab-aws-credential-helper
    aws-profile --name production
    --aws-account 333333333333 
    --role Deployer

```

Use the `--help` flag to get an overview of all possible options for each of the command usages.

## Conclusion

With the Gitlab AWS credential helper, it becomes really easy to provide each Gitlab CI/CD pipeline with the least amount of privileges required to perform their task on AWS. Just configure the AWS account number and add the credential helper to the runner and create an IAM role for each gitlab project.

The post [Seamlessly connect Gitlab CI/CD with AWS: Gitlab AWS credential helper](https://xebia.com/blog/seamlessly-connect-gitlab-cicd-with-aws-gitlab-aws-credential-helper/) appeared first on [Xebia](https://xebia.com).