Container Images for AWS Lambda with Python
Tags: serverless, aws, python, docker
Published: 2021-05-10 18:32:09

One of the best things about AWS Lambda is the variety of ways that you can create a serverless function. For example, you can dive right into the console, or use approaches like [Chalice](https://chalice.readthedocs.io/en/stable/) or the [Serverless Framework](https://www.serverless.com/), to name a few. For me, the latest way, announced at re:Invent 2020, is the most efficient way of testing your serverless function locally and dealing with large or awkward dependencies in your code. As you'd expect, this works perfectly with [AWS SAM](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/what-is-sam.html). This post will give you a quickstart into what you need to do to build a container-based function using containers & Python. 

In my case, I was building a function that would read data from Firestore in Google Cloud, run a data transformation and store the result in S3. I used to swear by Chalice, and built my application but found that it couldn't bundle the GRPC dependency - I would have to build it myself. While the solution to this is just to build the dependency and package it up in the `vendor` directory of my Chalice application, I used the opportunity to finally try out container support. 

### Prerequisites 
If you're reading this, you're probably using Python already. As well as having Python installed, you'll need to have [Docker](https://www.docker.com/get-started) too. 

### Building a Docker container with dependencies 
As always, the [AWS documentation](https://docs.aws.amazon.com/lambda/latest/dg/python-image.html) will guide you through the basics. 

First you'll need to install the Python runtime interface client using `pip install awslamdaric` 

Next, create a `Dockerfile` that references the base image you are using. In the case below, I was using Python 3.8
```dockerfile
FROM public.ecr.aws/lambda/python:3.8
```
Presuming your code is in app.py with a `handler` as the function, you'll add these two lines too 

```dockerfile
COPY app.py   ./
CMD ["app.handler"]      
```

Dealing with dependencies is pretty straightforward. As with all Python application, all the modules you use should be defined in `requirements.txt` like this: 

```plaintext
google-cloud-firestore
google-auth
```

To make sure all dependencies are correctly bundled, just add these two lines to your Dockerfile 

```dockerfile
COPY requirements.txt ./
RUN pip install -r requirements.txt 
```
It's as simple as that.

### Building & Testing Locally 
The huge benefit to working with containers is the ability to completely test your Lambda function locally before deploying to your AWS account. Let's take a look into how to do that: 

#### Build Your Container
The standard docker build syntax applies here.  

```bash
docker build -t my-container-image .
``` 

The commmand above creates 'my-container-image' in the local directory

#### Running Locally 
Typically you would run your container as follows 

```bash
docker run -p 9000:8080  my-continer-image:latest
```

However, it's very likely that you're interacting with AWS services (in my case S3), so you will need to ensure the access credentials exist for your function [documentation here](https://docs.aws.amazon.com/lambda/latest/dg/images-test.html). These credentials can be set using the following environment variables: 
- `AWS_ACCESS_KEY_ID`
- `AWS_SECRET_ACCESS_KEY`
- `AWS_SESSION_TOKEN`
- `AWS_REGION`

Provided you've got the credentials set on a user (which should be using the same policy as your Lambda would) you can pass them through as following when running locally: 

~~~dockerfile 
docker run -p 9000:8080  
-e AWS_ACCESS_KEY_ID='ACCESS_KEY_ID_GOES_HERE'
-e AWS_SECRET_ACCESS_KEY= 'SECRET ACCESS KEY GOES HERE'
-e AWS_REGION= 'eu-west-1'
my-container-image:latest
~~~

Finally, to invoke events, you can pass a payload through while the container is running as follows: 

```bash
curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" 
-d '{"param1":"parameter 1 value"}'  
```

Note that you can also use the `AWS_LAMBDA_FUNCTION_TIMEOUT` environment variable to set a timeout for your function in milliseconds.  


### Running your Lamba 

Now that you have a working container, you need to deploy it to your AWS account
#### Deploy Your Container 
Authenticate the Docker CLI to your ECR registry  

```bash
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin <AWS ACCOUNT ID>.dkr.ecr.<REGION>.amazonaws.com    
```

The first time that you are going through this process, you will need to create a repository on ECR with the name that you want to use 

```bash
aws ecr create-repository —repository-name my-container-image
```

After this you just need to tag your image before pushing it to ECR 


```bash
docker tag  my-container-image:latest <AWS ACCOUNT ID>.dkr.ecr.<REGION>.amazonaws.com/my-container-image:latest
```

```bash
docker push <AWS ACCOUNT ID>.dkr.ecr.<REGION>.amazonaws.com/my-container-image:latest        
```

Your container is now ready to be used in a Lambda function. 

#### Creating Your Lambda Function  

Just open up the AWS console, go to AWS Lambda and click on Create Function 

![Creating a container based function on the AWS console](https://posts.jamessugrue.ie/images/articles/aws-console-lambda-container.png)


From there you'll be able to create a function from a container image, and simply select the image you need from ECR through Browse Images. 

Having taken this console based approach, you are obviously going toneed to attach a role to your Lambda at a minimum. 
If you want to be more disciplined, and less console dependent, on how you create your container based Lambda functions, check out [https://twitter.com/edjgeek](Eric Johnson's) post on [Using Container Image Support for AWS Lambda with AWS SAM](https://aws.amazon.com/blogs/compute/using-container-image-support-for-aws-lambda-with-aws-sam/)



### Extra Resources 
As well as the documentation resources linked in this article, make sure to check [Chris Munns'](https://twitter.com/chrismunns) excellent talk on [Container Image Support in AWS Lambda](https://www.youtube.com/watch?v=X-1xf-DbCBk) from re:Invent 2020