How to publish and use AWS Lambda Layers with the Serverless Framework
Tags: aws, lambda, serverless, tutorial
Published: 2022-03-24 18:55:26

Originally posted at [Serverless](https://www.serverless.com/blog/publish-aws-lambda-layers-serverless-framework/)

[AWS re:Invent](https://serverless.com/blog/reinvent-2018-serverless-announcements) is in full swing, with AWS announcing a slew of new features. Most notably, we’re pretty excited about AWS Lambda’s support for Layers.

[Layers](https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html) allows you to include additional files or data for your functions. This could be binaries such as [FFmpeg](https://www.ffmpeg.org/) or [ImageMagick](https://imagemagick.org/script/index.php), or it could be difficult-to-package dependencies, such as [NumPy](http://www.numpy.org/) for Python. These layers are added to your function’s zip file when published. In a way, they are comparable to [EC2 AMIs](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html), but for functions.

The killer feature of Lambda’s Layers is that they can be shared between Lambda functions, accounts, and even publicly!

There are two aspects to using Lambda Layers:

 1. Publishing a layer that can be used by other functions

 2. Using a layer in your function when you publish a new function version.

We’re excited to say that the Serverless Framework has day 1 support for both publishing and using Lambda Layers with your functions with [Version 1.34.0](https://github.com/serverless/serverless/releases/tag/v1.34.0)!

See how you can publish and use Lambda Layers with the Serverless Framework below.

![](https://cdn-images-1.medium.com/max/5200/0*5ba1vFiVOvxWGqi9.png)

### Example use case: Creating GIFs with FFmpeg

For a walkthrough, let’s make a service that takes an uploaded video and converts it to a GIF.

We’ll use [FFmpeg](https://www.ffmpeg.org/), a open source tool for manipulating video and audio. FFmpeg is a binary program and a great example use case for a layer as managing the binary falls outside the responsibility of your runtime’s packaging system.

In this example, we’ll build and publish a layer that contains FFmpeg. Then, we’ll create a Lambda function that uses the FFmpeg layer to convert videos to GIFs.

To get started, create a serverless project for your layer & service:
{% gist https://gist.github.com/richardgrantserverless/e31c8661bec99e769f17cc5bf6add42f.js %}
Then at the bottom of your serverless.yml add the following to define your layer that will contain FFmpeg. The path property is a path to a directory that will be zipped up and published as your layer:
{% gist https://gist.github.com/richardgrantserverless/ec3b6841becc48be3dc33961add7967b.js %}
Run the following commands to download the contents of your layer:
{% gist https://gist.github.com/richardgrantserverless/8761dc72b299450237950c45f36982fe.js %}
You’re ready to test deployment of your layer. Deploy and you’ll see the layer’s ARN in the output info:
{% gist https://gist.github.com/richardgrantserverless/70d751af79653ca21f8cbfeebd457e86.js %}
Next, we’ll add a custom section to serverless.yml to specify the S3 bucket name (choose your own unique bucket name):
{% gist https://gist.github.com/richardgrantserverless/9ab876f83e2e791c3aa2d17045d1da0f.js %}
Now rename your function from hello to mkgif, specify that your function uses the layer you’re publishing, and add an S3 event configuration:
{% gist https://gist.github.com/richardgrantserverless/7100ec278bee35352c90ccbb63e528d5.js %}
You’ll also need to give your service permission to read & write your S3 bucket, add the following in the provider section of your serverless.yml file:
{% gist https://gist.github.com/richardgrantserverless/612a12c3b28cebe8952f8b57ecd2c684.js %}
Your serverless.yml should now [look like this](https://github.com/serverless/examples/blob/master/aws-ffmpeg-layer/serverless.yml).

We need to make our handler. Replace the contents of handler.js with the following code, which gets the file from S3, downloads it to disk, runs ffmpeg on it, reads the GIF, and finally puts it back to S3:
{% gist https://gist.github.com/richardgrantserverless/8c8ba01f622b60993d5f624d0738ad3b.js %}
Now you can deploy both the layer & updated function with sls deploy. Let’s test it out by uploading a video to our S3 bucket:
{% gist https://gist.github.com/richardgrantserverless/31e5b10897bd7240eb9b6885731f768e.js %}
You now have a GIF copy of the mp4 you uploaded!

For the full source of this example, check it out in our [examples repo](https://github.com/serverless/examples/tree/master/aws-ffmpeg-layer).

### Some tips on working with layers

In the example above, instead of specifying an ARN for the layer that the function is using, we used {Ref: FfmpegLambdaLayer}. This is a [CloudFormation Reference](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-ref.html).

The name is derived from your layer’s name, e.g., ffmpeg becomes FfmpegLambdaLayer. If you're not sure what your layer's name will be, you can find it by running sls package then searching for LambdaLayer in .serverless/cloudformation-template-update-stack.json.

You may have noticed that every time you deploy your stack, a new layer version is created. This is due to limitations with CloudFormation. The best way to deal with this is by keeping your layer and your function in separate stacks.

Let’s try that with the example we just made.

First, create a new folder and move the layers directory into it:
{% gist https://gist.github.com/richardgrantserverless/587566f5917aaffeb7280fb6082c1b35.js %}
Remove the top-level layers section in gifmaker/serverless.yml, then create a new serverless.yml in the ffmpeg-layer folder containing:
{% gist https://gist.github.com/richardgrantserverless/9f7641b2c1d4cd5c029c36bc8a5f7511.js %}
Now you can run sls deploy to publish your layer!

Go back to the gifmaker service directory and change {Ref: FfmpegLambdaLayer} in the serverless.yml to ${cf:ffmpeg-layer-dev.FfmpegLayerExport}. You can now run sls deploy and it'll use the layer from the other service. Note that the dev in the variable above is the [stage](https://serverless.com/framework/docs/providers/aws/guide/workflow#using-stages) of your layer service.

### More Examples

You can see the following projects for some examples of using this plugin to build a layer. They all leverage Docker and the [docker-lambda images](https://github.com/lambci/docker-lambda) to compile for AWS’s Lambda environment on any operating system:

* [geoip-lambda-layer](https://github.com/dschep/geoip-lambda-layer) — A layer containing [MaxMind](https://maxmind.com/)’s GeoIP libraries

* [sqlite-lambda-layer](https://github.com/dschep/sqlite-lambda-layer) — A layer to fix SQLite support in Python 3.6 runtimes

Awesome layers

Also check out this repository of awesome layers: [https://github.com/mthenw/awesome-layers](https://github.com/mthenw/awesome-layers)

### Custom runtime support: even better!

Along with layers support, AWS also just announced support for building your own runtime using the [Runtime API](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html).

This allows you to build, use, and share runtime support for Lambda outside of what AWS officially supports.

Custom runtimes with the Serverless Framework

To utilize custom runtimes with Serverless, specify the runtime as provided in your serverless.yml and include a layer that provides a custom runtime. For documentation on building your own runtime, see AWS’s documentation [here](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html)

More re:Invent news

* [All the Serverless announcements at re:Invent 2018](https://serverless.com/blog/reinvent-2018-serverless-announcements/)

* [DynamoDB On-Demand: When, why and how to use it in your serverless applications](https://serverless.com/blog/dynamodb-on-demand-serverless/)

* [Real-time applications with API Gateway WebSockets and AWS Lambda](https://serverless.com/blog/api-gateway-websockets-support/)

* [What Firecracker open-source means for the serverless community](https://serverless.com/blog/firecracker-what-means-serverless/)