Running AWS CodeBuild locally on a MacBook-M1
Tags: macbook, aws, codebuild, docker
Published: 2024-01-08 04:35:58

by Udaybhaskar Sarma Seetamraju
ToSarma@gmail.com
Dec 31 2023

# Highest-level Context

If you are into “**Shift-Left**” (whether re: Testing, Security, or Replicating-problems-on-developer-laptop, etc ..), then this article is for you.

For the very first time that you switch to an M1-chipset based MacBooks (from intel-chip based MacBooks) .. Productivity is significantly impacted when doing development/testing/troubleshooting “locally” on your laptop.  Out-of-scope of this article is supporting those switching from Windoze.

Towards enabling up to 5x developer-productivity by allowing developers to robustly SIMULATE the Cloud-environment on a laptop — I have the following series of articles re: M1-chipset based MacBooks:

1.  (This) Running AWS CodeBuild locally on MacBook-M1.
    *  Running Containers based on older `Ubuntu 20.04` (released in the year 2020) as well as on the newer `Ubuntu 22.04` (released in the year 2022)
    *  Running Containers based on `arm64`-based Linux
2.  [Running `AWS Glue` locally](https://dev.to/seetamraju/running-aws-glue-locally-on-a-macbook-m1-3635) on MacBook-M1.
Various scenarios covered like: you do Not have “aws credentials” on your Laptop (forcing you to mock all the AWS API calls like `S3 GET`, Glue-Catalog queries, etc..)
3.  [New Security-related Best-Practices](https://dev.to/seetamraju/creating-arm64aarch64-docker-images-on-a-macbook-m1-new-security-related-best-practices-4jn9) when creating `arm64/aarch64` Docker-Images on a MacBook-M1.

# Summary
Aiming for very simple set of commands, based on bash-shell scripts --> to start a CodeBuild’s build locally on your MacBook-M1 laptop.




**You may ask**: Why even bother with `arm64/aarch64` docker images, especially when we can set the following ENV-Variables and successfully emulate `x86/amd64` chipsets on MacBook-M1?
```
export BUILDPLATFORM="linux/amd64"
export DOCKER_DEFAULT_PLATFORM="${BUILDPLATFORM}"
export TARGETPLATFORM="${DOCKER_DEFAULT_PLATFORM}"
```
**My response**: You should aim to have your software work on `linux/arm64` containers, which invariably is cheapest compute on cloud.

Next, I have tips on how to significantly raise your productivity, in debugging/developing your `buildspec.yaml`, even if your company denies your AWS CLI credentials.

Yes, this article is based on [AWS own official Documentation](https://docs.aws.amazon.com/codebuild/latest/userguide/use-codebuild-agent.html).

But .. ..

those AWS instructions (link above) are a very complex for anything other than `AmazonLinux2/2023` on x86, with “gotchas” and challenges of using Ubuntu-vs-AmazonLinux on a MacBook-M1 Laptop.

Based on very simple CLI-arguments, the bash-scripts offered in this article will help automatically download these images (if missing) and execute a build using the buildspec file in current-working directory.

# Very-Advanced User: Short summary

1. You need 2 container-images are required to start a CodeBuild’s build locally on your laptop.
   1.  `amazon/aws-codebuild-local:latest` (This is the “_Engine_” a.k.a. AWS-CodeBuild’s Agent/Platform on your laptop) 
   2. If `AmazonLinux2/2023` is your Build O/S: pull the image  `public.ecr.aws/codebuild/amazonlinux2-x86_64-standard:5.0` 
   3. If Ubuntu `20.04` or `22.04` is your Build-env O/S, then you’ll need locally custom-build the following Docker-Images from AWS official source-code:
       *  `aws/codebuild/standard:5.0` 
       *  `aws/codebuild/standard:7.0`
       *  No, you can NOT find these images anywhere! 
1. Optionally, instead of image #2 above, if you want `aarch64`-chipset:
   1. then download the image:  `public.ecr.aws/codebuild/local-builds:aarch64` 
   2. For a full list of all possible images (with Go/Java/Python/..):   `aws codebuild list-curated-environment-images` 

# Problem Statements

1. Want to use Ubuntu as the O/S for CodeBuild projects, and use it to test/debug locally on my MacBook-M1.
2. Rather than wait 10+ minutes to find a stupid-mistake in the `buildspec.yaml` file, can I run `buildspec.yaml` file locally on my laptop, to test everything in it?
3. How to proactively ensure the CodeBuild project’s build will work on all chip-architectures - without having to scramble later? How to explicitly utilize `x86_64/amd64/arm64/aarch64` architectures locally on laptop?
4. Company does Not allow Laptops to have AWS-Credentials (in `~/.aws/credentials` file); My `buildspec.yaml` file uses Secrets, Parameter-Store, etc.. as well as AWS CLI commands. Even so, how can I **EFFICIENTLY** test/debug the `buildspec.yaml` file locally on my laptop?

# Ready to use scripts
Please note: For **Ubuntu** based CodeBuild, read the **sub-section** (**below**) titled “_CodeBuild projects using Ubuntu_”.

I have bash-shell scripts that require just one CLI-argument (either `AmazonLinux2` or `UBUNTU`).

To get started .. ..

1. `git clone https://gitlab.com/tosarma/macbook-m1`  
   *  You’ll notice 2 bash-scripts in AWS-CodeBuild/  sub-folder:
      * “`LOCAL-aws-codebuild-runner.sh`” and
      * “`LOCAL-create-aws-codebuild-standard-image.sh`” 
1. On your `bash`-terminal, run the 1st script above using the full-path to it.
   * FYI: The 1st script will automatically run the 2nd one as needed.

FYI — Running the **1st** script assumes you have a  `buildspec.yaml` file in the current-folder.  Or a “`.yml`” file.
Note: If you have a **`LOCAL`**`-buildspec.yaml` file in the current-folder, then it is used instead.
Re: this “**`LOCAL`**-” file, immediately please read the following **sub-section** “_No AWS Credentials on your laptop?_”.

Note: Only for “`AmazonLinux2`” cli-argument, the __ FIRST __ time you run the above script it’ll take **up to 5-minutes** to download about 30+/- images from AWS ECR-Repo.

## No Bash?  Want Python instead?

Just replace the “`.sh`” with “`.py`” — in the script names (above).
And, of course, you must insert “`python3`” at the very beginning of the CLI (this is a platform-independent advice).

**WARNING**: Without the benefit of “`docker` cli”, you get **ZERO** visibility into the progress of docker-activity. This is due to use of un-friendly Docker’s Python APIs, because of which the python-code _ WILL _  _ HANG _ for very long time!

To repeat:
*  “`LOCAL-aws-codebuild-runner.py`” will hang with NO output, for roughly 2-to-5 minutes (depending on how much CPU and MEMORY you have allocated to the Docker-Desktop, as well as speed of your internet connection)
*  “`LOCAL-create-aws-codebuild-standard-image.py`” will hang with NO output .. ..
   *  .. for **AmazonLinux2** for roughly 2-to-5 minutes (depending on how much CPU and MEMORY you have allocated to the Docker-Desktop, as well as speed of your internet connection)
   *  .. for **Ubuntu** (see full details below) for a minimum of 2+  <u>**H O U R S**</u> !! (If you have minimal CPU allocated for Docker-Desktop, it will as long as 4+ **HOURS**)

Important - Note these:
1.  I only tested using `Python3.11`; No other Python version tested.
1.  PRE-REQUISITES:
    * `pip3 install docker`
    * `pip3 install GitPython`

<HR/>
<HR/>

# Using Ubuntu 20.04

**Warning**: As of 2024-January, only ‘`x86_64/amd64`’ chipset-architecture supported (on MacBook-M1) for “_Ubuntu_” O/S.

**Question**: Who would use Ubuntu (instead of AmazonLinux) - for their CodeBuild projects?

**Answer**: Ubuntu has **great trouble-free** support for installing EXACT versions of software, whether Google-Chrome (for headless testing) or older versions of NodeJS or Python, etc.. ..

**Warning**: Only for “_Ubuntu_” O/S CodeBuild builds, you _ MUST _ turn-**OFF** (that is, **un**-check the checkbox) as shown in screenshot below.
Note: For fix many __ OTHER __ issues while running Docker-containers on MacBook-M1, you are REQUIRED to turn-**ON** this checkbox.
So, please pay attention to conflicting configurations (within Docker-Desktop on MacBook-M1)!

**Warning**: Only for “_Ubuntu_” O/S, the __ FIRST __ time you run the above script it’ll **takes minimum 2 hours** — to re-create Ubuntu Image runtime from scratch (after downloading the AWS Source-code).  
Why? Because, unfortunately, AWS does Not offer these ready-to-use images for Ubuntu O/S to download.

### To get started, here are the simple commands to run!

```
cd <Your-own-project>

export BUILDPLATFORM=linux/amd64
export DOCKER_DEFAULT_PLATFORM="${BUILDPLATFORM}"
export TARGETPLATFORM="${DOCKER_DEFAULT_PLATFORM}"

<Path-to-Git-Cloned-folder>/AWS-CodeBuild/LOCAL-aws-codebuild-runner.sh  UBUNTU
```

Let me know of any issues with the above.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4lgwj27y8v73lav8pwx7.png)

# Issues & Errors?
See Appendix, for resolving the errors.
Example: See Appendix sub-section titled “_Docker-Desktop settings for **Ubuntu**-on-x86 images_”.

## Installing Chromium Headless on Ubuntu
See `install-Chromium-latest-on-ubuntu20.04.sh` 
under https://gitlab.com/tosarma/macbook-m1/-/tree/main/AWS-CodeBuild/software-install-scripts?ref_type=heads 

## Installing Old Node.JS on Ubuntu
Edit and run `install-NodeJS-latest-on-ubuntu20.04.sh` 
under https://gitlab.com/tosarma/macbook-m1/-/tree/main/AWS-CodeBuild/software-install-scripts?ref_type=heads 

## Installing Old Python on Ubuntu
Edit and run `install-python-latest-on-ubuntu20.04.sh` 
under https://gitlab.com/tosarma/macbook-m1/-/tree/main/AWS-CodeBuild/software-install-scripts?ref_type=heads 

## Ubuntu 22.04
To switch to the **newer** Ubuntu `22.04` (released in the year 2022) ..

1. Edit script “`./AWS-CodeBuild/LOCAL-create-aws-codebuild-standard-image.sh`”.
2. UN-comment the **line # 21** (to use “`aws/codebuild/standard:`**`7.0`**”)
3. Comment out the next **line # 22**. (Disable “~~`5.0`~~”)
4. Must run “`docker system prune --all --force —volumes`”.
5. Finally, follow the instructions in **above** section titled “_CodeBuild projects using Ubuntu 20.04_”

# No AWS Credentials on your laptop?
For security-reasons, many companies are denying developers the AWS-credentials for AWS-CLI use.
If your `buildspec.yaml` file has **AWS-Secrets** (quite common!) or if it runs **AWS-CLI** inside (example: to get Stack-outputs), etc .. ..
.. then, you have a **showstopper** in locally testing/debugging your `buildspec.yaml` locally on your laptop.

My best-practice is to:

- Create a 2nd (new) file named “`LOCAL-buildspec.yaml`” (as described below) + 
- Create a 3rd (new) file called “`.env`” file to along with it.

**WARNING: Do _ NOT _ git-commit the “.env” file, as per global practices.
Immediately add the “`.env`” file to your “`.gitignore`” file.**

This **new** “`LOCAL-buildspec.yaml`” file will:

1. Not have entries for Secrets and Parameter-Store entries
2. Not have all AWS-CLI commands
3. Not have any CDK or other commands.
4. Must “source” the .env file, in the “Install or Pre-Build” phases as:

>    `.  .env`

NOTE: Re: the `.env` file:-  

- You will ensure the `.env` file sets all `bash`-variables that’ll contain the values for the Secrets, Parameter-Store entries, etc. ..
- You will ensure `.env` file also provides all the values expected from the **AWS-CLI** commands (that were removed).

Finally, just run:

```
cd <your-own-project>

<Path-to-Git-Cloned-folder>/AWS-CodeBuild/LOCAL-aws-codebuild-runner.sh
```

# Open Questions, Concerns and Challenges
**WARNING: Do _ NOT _ `git commit` the `.env` file, as per global practices.
Immediately add the `.env` file to your `.gitignore` file.**

If you mistakenly commit it, confess honestly and immediately to your corporate security team, and work on fixing the security vulnerability.

FYI - CodeBuild for Ubuntu O/S on `aarch64` & `amd64` chipsets are Not yet supported.
No timetables available.

Note: If you mistakenly or consciously run “`docker system prune`”, the Ubuntu O/S option will take 2+ hrs again (to re-create images from AWS official source-code)!
Warning: Do __ Not __ rely on “save” and “import” of the docker-image.
The Following __failed__ for me.

```
 docker save                   --output ~/aws_codebuild_standard_5.0.tar   aws/codebuild/standard:5.0  
 docker import --platform linux/x86_64  ~/aws_codebuild_standard_5.0.tar   aws/codebuild/standard:5.0 
```

FYI: Above “.tar” file is typically 10GB in size and takes about 2-3 minutes to be “saved”.

# APPENDIX
## Docker-Desktop settings for Ubuntu-on-x86 images


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/93oi48blk0j6uioc27vh.png)

Why?
As of 2023, per https://github.com/moby/moby/issues/44578 Docker-Desktop’s _containerd-integration_ can NOT interact with images that don't have the default platform.
And .. `aarch64` is the default-platform on MacBook-M1, while we’re seeking `amd64` for Ubuntu.

# Docker-Desktop settings for aarch64-chipset 
See screenshot in previous sub-section.
Turn _ ON _ the setting titled “_Use containerd for pulling and storing images_”!
Yup!  Doing **<u>just the opposite</u>**!
I can’t help explain this crazy conflicting instructions.
As of 2023, this is a Docker-on-MacBook issue.

# Running out of Disk-space or Memory?
Screenshot below shows the recommended “high” settings.
After building images, you can reduce:
-  “CPU” can be lowered to “2”.
- “Memory” can be lowered to “4GB”.

FYI only - To run on a MacBook-M1, many `amd64` emulated containers like `Neo4j v4.x` will frequently fail, unless you provide Docker with a minimum of 5+ cpus and 8GB of RAM!


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gw83ftjg90tb61gc07ub.png)