Harnessing Managed GitHub Action Runners on AWS CodeBuild for Efficient DevOps Workflows
Tags: aws, devops, github, codebuild
Published: 2024-05-10 12:37:48

Few weeks back, AWS announced a [new](https://aws.amazon.com/about-aws/whats-new/2024/04/aws-codebuild-managed-github-action-runners/) feature involving AWS CodeBuild, that allows you to configure self-hosted GitHub action runners in CodeBuild containers to process GitHub Action workflow jobs. This feature allows CodeBuild projects to receive GitHub Actions workflow job events and run them on CodeBuild ephemeral hosts. 

**What is AWS CodeBuild?**
AWS CodeBuild is a robust, managed continuous integration service that automates code compilation, testing, and artifact production without requiring the management of underlying servers. 

Traditionally, our approach involved using EC2 Spot Instances with custom AMIs in a scheduled auto-scaling setup to accommodate the fluctuating demands of GitHub Action runners. This method, while effective, often led to bottlenecks due to time-zone variances across our DevOps teams, resulting in delayed CI/CD pipelines.

The introduction of managed GitHub Action runners by AWS offers a promising alternative, integrating seamlessly with AWS services like IAM, Secrets Manager, CloudTrail, and VPC for enhanced security and operational efficiency.

Let us explore this feature step by step, by connecting one of my GitHub [repositories](https://github.com/shekar-ym/codebuild-github-action-runner) to a CodeBuild project, see how it can pick up the queued workflow job and performs the GitHub actions configured in the workflow job. The workflow is quite simple one - it builds a container image based on a Docker file, tags it and pushes this to an existing Amazon Elastic Container Registry (ECR).

Step 1- CodeBuild project:

Let us navigate to CodeBuild console and create a project with name **github-action-runners**

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/j5tyoahc2aycwd8ktqua.png)

Select the source provider as GitHub and Connect using OAuth. We can also us PAC (personal access tokens) to connect to GitHub. To keep things simple, let us stick to OAuth:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7bm1mlg5800i6ikovzl9.png)

Authorize aws-codesuite to access your GitHub repositories:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kclaxsghd4tni5by2wzt.png).

Once the connection is successful, you should be able to select your repository from the list:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ju5h8yzrczukvh8wm72y.png)

For **Webhook**, let us select _Rebuild every time a code change is pushed to this repository_ and **Event type** as _WORKFLOW_JOB_QUEUED_

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qy9pegqnu1y6hnfq2bnb.png)

There are following event types available - PUSH, PULL_REQUEST_CREATED, PULL_REQUEST_UPDATED, PULL_REQUEST_REOPENED, PULL_REQUEST_MERGED, PULL_REQUEST_CLOSED, WORKFLOW_JOB_QUEUED,RELEASED, PRERELEASED.

You can also add additional conditions for "Start build" and "Don't start build", if needed.

Next, let us choose the Compute Environment for the CodeBuild. CodeBuild offers On-demand and Reserved Capacity options to choose from. For image options you can choose CodeBuild managed images or a custom Docker image. In case of custom image option, you can choose an image from your Amazon ECR (in your account or from another account). Or you can choose an image hosted in an external Docker registry. 

Let us stick to default values for our use case. 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ieh8iemtrzo1ex7q688v.png)

Note that, we can always override these options by using a label in our GitHub Actions workflow file. More on this in later sections.

CodeBuild can create a service role with required permissions for you or you can create and choose your own custom role for the project.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dpq43uo1s6geg1yvj4wk.png)

Next is Buildspec yaml file. In our case, Buildspec will be ignored when we use CodeBuild to run GitHub Actions workflow jobs. Instead, CodeBuild will override it to use commands that will setup the self-hosted runner.

Let us go-ahead and create the CodeBuild project. Once the creation is successful, you can see that a webhook has been created on your GitHub repository. Navigate to Settings --> Webhooks section to see this: 


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9zd9x4ognlehe4kax8ri.png)

Step 2: GitHub Actions Workflow Configuration 

Below is the workflow file:

```
name: HelloWorld app
on:
  repository_dispatch:
    types: [webhook_triggered]
  pull_request:
    branches:
      - main
  push:
    branches:
      - main
env:
  AWS_REGION: 'us-east-1'

jobs:
  build:
    name: Build Docker Image
    runs-on: codebuild-github-action-runner-${{ github.run_id }}-${{ github.run_attempt }}-al2-5.0-small
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      
      - name: Set outputs
        id: vars
        run: echo "short_sha=$(git rev-parse --short HEAD)" >> $GITHUB_OUTPUT
      
      - name: Setup AWS ECR Details
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{env.AWS_REGION}} 

      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v2

      - name: Build, tag, and push image to Amazon ECR
        id: build
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          ECR_REPOSITORY: ${{ secrets.AWS_ECR_REPO }}
          IMAGE_TAG: v1.0.0.${{ steps.vars.outputs.short_sha }}
        run: |
          docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
          echo "image_tag=$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG" >> $GITHUB_OUTPUT
          echo "${{ github.event.action }}"
    outputs:
      image_tag: ${{ steps.build.outputs.image_tag }}
```

From above, you can see that I have AWS credentials and ECR private repository name, configured as repository secrets and referred in the workflow. Refer to my repository [here](https://github.com/shekar-ym/codebuild-github-action-runner) for Docker file and other scripts. 

Note that `runs-on` label has the value in format 

```
codebuild-<project-name>-${{ github.run_id }}-${{ github.run_attempt }}-<image>-<image-version>-<instance-size>
```
where 

- **project-name** = Name of the CodeBuild project we created in step 1 above. 
- **image-image-version-instance-size** = al2-5.0-small, which indicates I am overriding the values configured in Environment section of the CodeBuild project. 

Refer to the _Supported compute images_ table in this [page](https://docs.aws.amazon.com/codebuild/latest/userguide/action-runner.html#action-runner-questions) for the list of compute images that CodeBuild provides.

When you push any changes or create a PR to the `main` branch, workflow will be triggered. 


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yw668vtxxjfywo9x7k2v.png)

The webhook associated with this repository will notify the CodeBuild project, which is now our GitHub actions runner picks up this job, as shown below:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ccezs0p7zmedh6x4myxr.png)


If you navigate to CodeBuild project on AWS console, you can see that CodeBuild run is in progress, executing the GitHub actions:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/oxx5x7hxnph0bz2xpauy.png)


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5g9m7qw7a94ds4no3n1e.png)

Also, on GitHub Settings page, you can see an **ephemeral** GitHub actions runner powered by CodeBuild is in works.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/y98w1wvgwcirfr4s4vc1.png)

Below is snapshot of the container image that was built and pushed to an existing ECR repository, as part of the GitHub workflow.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/n3w0xvzhjzs7sxokhyo1.png)

**Looking Ahead**
This integration not only simplifies our operations but also potentially reduces costs and improves performance compared to our previous setups. I plan to further explore and optimize this feature, considering even custom runner images to align with our security standards.

I'm excited to see how this feature evolves and look forward to sharing more insights, including a detailed cost comparison among various runner options.

Please let me know what you think by adding your comments.