Should you use Lambda containers?
Tags: docker, lambda, cloud, serverless
Published: 2023-06-05 08:20:26

_Written by Samuel Durand_

**When to use and when to avoid Lambda based on containers instead of the standard Lambda packaging**

# Assumptions

This article assumes that you have some knowledge about:

- AWS Lambda functions
- Docker

# First impressions

A while back, when AWS announced [Lambda support for containers](https://aws.amazon.com/blogs/aws/new-for-aws-lambda-container-image-support/), my first reaction was to dismiss it as something to be used only as a last resort, typically if you really wanted to use a language not supported out of the box, such as Rust or PHP. I also assumed that there would be some disadvantages, like slower start time and some additional work to create your own container, but I didn't look further into it.

It has now been 3 years since this feature was released and I decided it was time I revisited the subject to see how much of my assumptions were true and if containers for Lambda had other usages or limitations that I hadn't considered at the time.

So let's get to it and see what we can learn!

# Standard Lambda deployment
Let's start with reviewing how a standard Lambda packaging and deployment normally works.

The usual steps are as follows:

1. __package__ your code, [usually as a Zip file](https://docs.aws.amazon.com/lambda/latest/dg/java-package.html) 
2. __deploy__ your code, via one of the many tools available (SAM, Cloudformation, Serverless, Terraform...)
3. configure one or more triggers to __execute__ your Lambda function (SQS, API Gateway...)

This simple process has several advantages:
- very little management overhead
- offers Lambda Layers to provide common dependencies and extensions
- no need for any Docker experience

Downsides
- limited runtime choices
- no control over what is eventually present alongside your executable (in the Firecracker MicroVm used by AWS)
- limited package and dependencies size (50 Mb for the package, or 250Mb when decompressed for the Lambda + Layers) 

Currently, the [supported runtimes ](https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html) are:
- Node.js: 12 to 18
- Python: 3.7 to 3.10
- Java: 8, 11 or 17
- Ruby: 2.7
- Go
- .NET 7 and .Net Core 3.1

Let's now have a look at what Lambda containers offer.

# Deployment via Docker container
### Why use it?

Before looking at what this type of deployment entails, let's consider why it might be worth considering an alternative Lambda deployment method. Here are the most common use cases for which a standard Lambda deployment would be inadequate, at least in its base form:
- __Runtime:__ Unsupported runtime (PHP, Rust): Unsupported runtime version, either too recent (Java 19...) or too old (legacy code using Node.js 10...) 
- __Custom binaries:__ The need for specific binaries not included inside the default lambda Microvm, for instance pdf processing libraries like `pdf2image`
- __Performance:__ A special container setup that allows the code to load and run much faster than the default Lambda (barebone Go image...)
- __Dependency size:__ Very large set of dependencies, for instance for Data Processing via Panda (it can easily exceed the maximum Lambda layer size of 250Mb)
- __Deployment package size:__ Very large deployment package (exceeds the maximum 50Mb)
- __Dataset size:__ A very large dataset needs to be included and used in every Lambda
- __Legal:__ Some regulatory or legal requirements, which impose to limit Lambda to a simple orchestrator, and to control what goes into the container
## How does it work?
The only step that changes compared to a standard lambda deployment is the __packaging__ step. But the change is drastic and adds a lot of complexity.

Here are the steps:
1. __package__ the code
2. choose a base image among the following options (more details [in the docs)](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-images.html#runtimes-images-lp)
a. base image with runtimes included
b. base linux image with Lambda components but no runtime
c. any docker image (Debian, Alpine...) as long as they support [the Lambda Runtime API](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-api.html)
3. write a Dockerfile based on that image
4. build your container
5. push it to [ECR](https://aws.amazon.com/ecr/)
6. provide the ECR image URL in the Lambda config
7. __deploy__ the code
8. configure the __execution triggers__

As expected, the operational overhead is a lot more significant when using a Docker image. However, __with complexity comes flexibility__, since any OS base and Docker image can be used, or even start completely from scratch (literally if the [Scratch Docker image](https://hub.docker.com/_/scratch) is used).

Note that the container will be limited to __10GB__ in size, much more than what is allowed with standard Lambdas.

This additional operational overhead is however not the only disadvantage when using containers for Lambdas.

## What are the downsides?
Using containers instead of the standard Zip archive has several negative implications, which affect both the management and execution performances of Lambdas.

- __Operational overhead__: As mentioned previously, the build process and the maintenance of the Lambda will be much more complex than the standard process. 
- __Extra costs__: ECR is the only container registry allowed, and using it will result in additional storage costs (VPC transfer inside the same region is free, so no additional network charges).
- __Lambda Layers__: [Lambda Layers](https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html) are not usable with containers, so no shared libraries, and it also means that the potential __cold start speed boost__ it can provide via caching will not apply. 
- __Lambda Extensions__: [Lambda Extensions](https://docs.aws.amazon.com/lambda/latest/dg/lambda-extensions.html) are a type of Lambda Layers and thus not usable either. This also means that AWS managed extensions such as the __Parameters and Secrets__ extension cannot be used. 
- __Slow cold start__: If the container is relatively large, the cold start performance will be significantly impacted, just as large Zip files affect standard Lambda cold start.

Note that it is however possible to find solutions to compensate or mitigate some of those issues.

For instance the slow __cold start for large container__ can be improved by using one of the [AWS amazon base images](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-images.html#runtimes-images-lp). This seems counter-intuitive at first, since they are larger than most base Linux Docker images, but those images are cached __pro-actively__ by the Lambda service, which means they will not need to be downloaded from ECR.

# Consider alternatives
As we have seen, in some cases the container for Lambdas option looks viable and can even seem necessary. But some of the advantages it provides can be obtained by other means that will not preclude the usage of the standard Lambda packaging and will avoid most of the operational overhead.

__Runtime__
If the main reason for using containers is to use a __runtime not officially supported__, then consider instead using a __Lambda Layer__, since it can be used to include a custom runtime. This can solve the problem without any of the previously described downsides.

__Custom binaries__
Similarly to runtimes, custom binaries can be integrated via Lambda Layers. See [this article](https://medium.com/@mayanknayyar95/aws-lambda-and-custom-binaries-ead12ca5c62e) for an example on how this can be achieved.

__Dataset__
If the goal is instead to include a __common dataset__ consider loading the dataset in S3 and downloading it from there (using an [S3 VPC Gateway endpoint](https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-s3.html) will remove the network fees at no extra cost) 

__Deployment package__
If the problem is that the deployment package is too big (> 50Mb), consider uploading the overly large package to S3 and [using it in your Lambda configuration](https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-package.html#gettingstarted-package-awsother). 

# Conclusion
Using containers can fit the bill for specific edge cases, but it will imply significant operational overhead and disadvantages. It is worth considering that the difficulties that make us consider this option can instead often be overcome by using the __standard Lambda deployment strategy, combined with some simple techniques.__ So it is best to make sure to consider all the alternatives, as well as to compare cost and maintenance complexity before settling on using this feature.

If you eventually decide that the extra complexity and downsides are worth it for your use case, make sure to test your containers to verify that they will function properly in the AWS Lambda environment via the [Lambda RIE](https://github.com/aws/aws-lambda-runtime-interface-emulator/).

Thank you for reading! And I hope this helps you make the best decision possible.

# References
- [AWS Lambda with Docker documentation](https://docs.aws.amazon.com/lambda/latest/dg/images-create.html)
- [AWS Lambda with Docker and Layers blog post](https://aws.amazon.com/blogs/compute/working-with-lambda-layers-and-extensions-in-container-images/) 
- [Optimize Lambda Docker Containers](https://aws.amazon.com/blogs/compute/optimizing-lambda-functions-packaged-as-container-images/) 

---
Still questions? Get in touch: aws@kreuzwerker.de