Running a git binary in AWS Lambda
Tags: aws, binary, lambda, run
Published: 2022-12-22 07:55:36

In some scenarios it can be easier to use a binary in a lambda function. In this blog post I will walk you through how you can run a git binary in an [AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/welcome.html "Learn more about AWS Lambda").

For this blog post I will use the [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/what-is-sam.html "Learn more about AWS Serverless Application Model")(AWS SAM). AWS SAM makes it easy to build, package and deploy lambda functions. The used template can be found on [GitHub](https://github.com/binxio/aws-lambda-git/). AWS SAM will look at the `Runtime` of the `AWS::Serverless::Function` resource. In this example I used python so it will look at the [requirements.txt](https://github.com/binxio/aws-lambda-git/blob/main/git_client/requirements.txt). It will look at the packages defined in the file. AWS SAM installs them in `.aws-sam/build/<ResourceName>`. This folder is compressed and uploaded to [Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html "Learn more about Amazon S3") when you package and deploy the template.

You can also invoke the function on your own computer. This makes it easier to develop and troubleshoot. To invoke the function we first need to build it. Afterwards you need to invoke it. I created a [Makefile](https://github.com/binxio/aws-lambda-git/blob/main/Makefile), for more information on this you can read one of my previous [blogs](https://binx.io/blog/2021/09/27/make-your-life-easier-using-makefiles/).

```bash
make invoke
```

You will see that you will see the following error:

```json
{
    "errorMessage": "[Errno 2] No such file or directory: 'git'",
    "errorType": "FileNotFoundError"
}
```

And that makes sense right? We do not have the git client installed. You cannot install it using the `requirements.txt` file. This is because it is not in the PyPi package manager.

Install the git client
----------------------

If you look at the [Makefile](https://github.com/binxio/aws-lambda-git/blob/main/Makefile) you can see that it will:

* Build a docker container.
	+ Install git on the container.
	+ Move all the dependencies into the `/var/task` structure.
	+ Zip the content of the `/var/task` folder.
* Extract the zip file from the container.
* Unzip the contents into the `git_client` folder.
* `sam build` will copy the content to `.aws-sam/build/GitClient`.
* `sam build` installs the dependencies from the `requirements.txt`.
* `sam invoke` will use the `.aws-sam/build/GitClient` content to invoke the function.

The Git binary is located in `/var/task/bin`. Because when we invoke the git command we also supply the `/var/task/bin` as `PATH`. This allows us to call `git` and the container knows it needs to invoke `/var/task/bin/git`. The libraries are located in `/var/task/lib`. Because we have set the `LD_LIBRARY_PATH` to `/var/task/lib` the libraries can be found.

I created a [Dockerfile](https://github.com/binxio/aws-lambda-git/blob/main/Dockerfile) that uses `public.ecr.aws/lambda/python:3.8` as the base. This is the same runtime of the Lambda function. I prepare 2 folders `/var/task/bin` and `/var/task/lib`. Then I install the `git` and `zip` packages within the docker context.

I then copy the `/usr/bin/git` and the `/usr/libexec/git-core/git-remote-http` executables to the `/var/task/bin` folder.
When you try to build the Lambda function and run it it fails on a missing library.

```json
{
    "errorMessage": "git: error while loading shared libraries: libpcre2-8.so.0: cannot open shared object file: No such file or directory",
    "errorType": "Exception"
}
```

By using the `ldd` command you can find all the shared libraries. We will use this to identify the needed libraries and copy them into the `/var/task/lib/` folder.
Once you have all dependencies in place. You can fetch the `git --version` running inside of the Lambda function.
 
> Git version: git version 2.32.0

If you combine this with the [awscli](https://pypi.org/project/awscli/) and [git-remote-codecommit](https://pypi.org/project/git-remote-codecommit/) packages. You can clone a CodeCommit repository and perform your actions and commit it again.

Conclusion
----------

It is possible to embed binaries in AWS Lambda and use them. It might take you some effort. But it opens new possibilities that you might not considered before.
In my next [blog](https://binx.io/blog/2022/04/12/sync-aws-codecommit-repositories/) you will read how you can use this to synchronize two CodeCommit repositories.

Photo by [Emile Perron](https://unsplash.com/@emilep?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText) on [Unsplash](https://unsplash.com/s/photos/infrastructure-as-code?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText)