Deploying AWS Lamba Function & Layer with Serverless Framework | by Fernando López | Towards Data Science

Serverless environments have emerged as a disruptive alternative for the deployment of applications in cloud instances without the need to configure, manage, operate and maintain a traditional server.
In this tutorial, we will see an example about how to develop, configure, integrate and deploy a Lambda Function [1] that extends the functionalities of a Lambda Layer [2] through the Serverless Framework [3].
This blog will be divided as follows:
AWS Lambda Function & Layer
Serverless Framework
What we are going to do
Installing Serverless Framework
Deploying AWS Lambda Layer
Deploying AWS Lambda Function
If you’re already familiar with AWS Lambda and the Serverless Framework, feel free to skip the next two sections and go straight to the examples.
Let’s get our cup of coffee and enjoy ☕️!
AWS Lambda is a computing service that allows you to run code without the need to configure and manage a server. To use AWS Lambda, you just need to package your code as well as the dependencies you will need (as a .zip file or a docker image) and organize them in a Lambda function.
In Figure 2 we see a representative description of the organization of the elements within a Lambda function.
As we can see in the figure above, each Lambda function is an isolated environment, that is, by nature they cannot share or extend functionality directly with another Lambda function. And this is one of the reasons why Lambda Layer emerged.
Lambda Layer is an AWS Lambda extension that works directly with Lambda functions. The aim of Lambda Layer is to contain applications, requirements, dependencies, and even custom runtime components that can be extended from a Lambda Function.
In figure 3 we see a description of the integration of a Lambda layer with a pair of Lambda functions.
As we can see from the previous figure, the implementation of a Lambda layer favors the organization of components that can be shared and extended through Lambda functions. Also, the use of Lambda layer helps on reducing the size of Lambda functions because it would not be necessary to pack dependencies multiple times (as seen in Figure 2).
However, in practice, defining and configuring one or more Lambda functions, as well as integrating with one or more Lambda layers, can be a laborious and tedious process, since AWS Lambda is commonly integrated with different services such as Amazon API Gateway, Amazon S3, Amazon SQS, Amazon SNS and many more. This is where Serverless Framework takes center stage.
Serverless Framework is a tool that facilitates the definition and configuration of AWS Lambda as well as its integration with other AWS services. In a nutshell, Serverless Framework streamlines your AWS Lambda development, integration, and deployment process.
Serverless is a cloud-agnostic framework, that is, it allows the development and deployment of services through the definition of a YAML file and the use of a CLI, to different cloud providers such as AWS, Azure, Google Cloud, Knative, and more.
Once we are clear on the relationship between AWS Lambda Function & Layer as well as the relevance of Serverless Framework in the development and deployment of cloud services, let’s go straight to the tutorial.
Don’t forget to fill up your cup of coffee ☕️!
We are going to create a Lambda function and a Lambda Layer. The Lambda layer will contain only the numpy library. The lambda function will extend the numpy library from the Lambda layer and implement it in its main method. The entire development and deployment process will be through the Serverless Framework.
Once the context is defined, let’s go for it!
To install Serverless, we first need to have node.js. To install node.js on macOS:
$ brew install node
For installing node.js on Ubuntu:
$ sudo apt install nodejs$ sudo apt install npm
Or if you prefer, you can go directly to the official node site to download it. Once node.js is installed and the npm manager is enabled, we proceed to install Serverless:
$ npm install -g serverless
If you have any problems with the installation through the npm manager, you can follow the official Serverless installation guide for installing the standalone binary.
Since we are going to develop and deploy to AWS in this example, you need to add the credentials of an AWS account, as follows:
$ serverless config credentials --provider aws \                                --key <access_key> \                                --secret <secret_key>
Once Serverless is installed and the credentials configured, we are ready to start with the fun part, let’s go for it!
First, let’s create a template for our Lambda layer with the serverless CLI, which we will call lambda-layer.
(venv) $ serverless create --template aws-python3 \                           --name layer-numpy \                           --path layer-numpy
The above command will generate a directory called lambda-layer which will contain the handler.py and serverless.yml files.
(venv) $ tree.└── layer-numpy    ├── handler.py    └── serverless.yml
In essence, the handler.py file contains the code that will run on AWS Lambda. The serverless.yml file contains the definition of the Lambda function or layer as well as all the necessary configurations for the deployment of the service.
Nervertheless, since in this case we want to deploy a Lambda layer that only contains a library, the handler.py file is not necessary. On the contrary, the file we require is requirements.txt which will contain the library that Lambda layer will host.
So, removing the handler.py file and adding the requirements.txt file, our directory looks like this:
(venv) $ tree.└── layer-numpy    ├── requirements.txt    └── serverless.yml
where the requirements.txt file only contains a reference to the numpy library:
(venv) $ cat requirements.txt numpy==1.21.0
To deploy the content of the requirements.txt file via serverless, we are going to require the serverless-python-requirements plugin which allows serverless to access the content of the requirements.txt file and prepare it for deployment.
To install the serverless-python-requirements plugin we type:
(venv) $ sls plugin install -n serverless-python-requirements
When installing the plugin, the serverless.yml file is updated by adding the definition of the installed plugin. The default content of our serverless.yml is like the one shown in the following code snippet.
As we can see, in line 10 we have the definition for a Lambda function created by default, however for the Lambda layer deployment we do not need it. On the other hand, in line 14 we have the reference to the plugin installed in the previous step.
Our serverless.yml file, after the modifications for the deployment of our Lambda layer, is as follows:
As we can see, from line 13 the definition of our Lambda layer begins, which is of type pythonRequirements whose content will be dockerized . Our Lambda layer is called python-numpy and is compatible with python 3.8 runtime.
It is important to mention that, since we enable the dockerization of the requirements, it is necessary to have docker installed on our local machine from where we will perform the deployment.
Great, once we have the requirements.txt file, customized the serverless.yml file and installed the plugin , we are ready to deploy. For the deployment, we only type:
(venv) $ sls deploy
And that’s it! our Lambda layer should be visible from our AWS console as shown in the following figure:
Once our Lambda layer is deployed, it is necessary to identify the associated ARN since it will be necessary to reference from the definition of our Lambda function.
Great, we have deployed our Lambda layer. Now let’s see how to deploy a Lambda function that extends dependencies of the Lambda layer.
Let’s go for it!
First, we are going to generate an aws-python3 template for our Lambda function which we will name test-function.
(venv) $ serverless create --template aws-python3 \                           --name test-function \                           --path test-function
The above command will generate a directory called test-function which contains two files, handler.py and serverless.yml. The handler.py file will contain the code that AWS Lambda will run. The serverless.yml file will contain the definition of the deployment.
(venv) $ tree.└── test-function    ├── handler.py    └── serverless.yml
First, let’s modify the handler.py file. Since we assume that we will extend the numpy library from the Lambda layer created in the previous section, all we have to do here is import numpy and declare some basic functionality. Finally, our handler.py file would look like this:
Now we will update the serverless.yml file. Basically, we will define 3 main things: the Lambda function, the Lambda layer extension and an API Gateway method (which we will use to make requests to our Lambda function). Finally, the serverless.yml file looks like this:
In line 12 we define the name of our function ( hello ), in line 13 we define the method that will be executed from the handler.py file. In line 15 we define the extension to Lambda layer (this is where we use the ARN of our Lambda layer created in the previous section) and finally in line 19 we define a GET method to make requests to our Lambda function.
And that’s it, we are ready to deploy!
(venv) $ sls deploy
Once our Lambda function is deployed, we could already see it in our AWS console. In the following figure we see the characteristics of our AWS Lambda function deployed from the console.
As we can see, our Lambda function has a Lambda layer attached and it has an API Gateway as a trigger.
Finally, to test our Lambda function, we use the serverless CLI as follows:
(venv) $ sls invoke -f hello {    "statusCode": 200,    "body": "\"Random number: 17. Numpy version: 1.21.0\""}
And that’s it! our AWS Lambda function extends the numpy library from a Lambda layer and the entire development and deployment process has been through the Serverless Framework.
[1] AWS Lambda Function
[2] AWS Lambda Layer
[3] Serverless Framework