How to make an AWS Lambda custom layer for python
Tags: aws, lambda, cloud, server
Published: 2023-03-25 13:40:51

AWS Lambda isn't truly "serverless", your code must run somewhere, right? Don't answer that, it's a rhetorical question! 😅
Yes! AWS lambda runs on servers!  
![Joe](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gtcsf5oo0044ma590mwd.gif)
But the word "*serverless*" means "I don't really care where it runs as long as it runs! And the actual servers aren't my concern therefore *I won't have access to them*".
Pay close attention to the last part as sometimes you would need some packages/libraries depending on your language of choice to use in your lambda function (actually most of the time 😂).
For me I use Python, and there is no way to "pip install ..." into the lambda function the packages I need as *I don't have access to the servers that runs the function*. See the issue here?! 😕

## Enter: AWS Lambda Custom Layers
Custom layers are AWS' solution to that problem ☝️. With custom layers you can include the extra "Umph" your amazing Lambda function needs to do its magic. You can read all about them form the [AWS Docs](https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html).
But I'm not here to direct you to the docs. I wanted to write this to all the busy cloud enthusiasts out there that are too short on time to read the docs and share my experience creating a python custom layer in a short and clear set of steps. (Actually, you must read the related docs always! Even if you are remotely interested in developing cloud workloads 🤷‍♂️)

(I'm on a windows machine and I use Ubuntu WSL for this. AWS recommends the use of Cloud9 to do that as your custom layer must be compatible with Amazon Linux to function correctly on your lambda. But I find WSL satisfactory in MOST cases)

For the demo, I'll just be following the example listed in the [docs](https://docs.aws.amazon.com/lambda/latest/dg/python-package.html) as it's very clear but I'll remove the clutter and share my own experience 😉.

1- I'll make one directory to group everything I need for this task as follows:

my-lambda
  |_ venv-python3.9
  |_ python
  |_ lambda-layer.zip

This is how it WILL look like but first I'll create my root directory and "cd" into it.
```console
mkdir my-lambda
cd my-lambda
```
2- You must use the same python version in your local environment as in your lambda function. My Lambda uses Python 3.9 and I'll use that too. You can use whatever Python environment management solution that makes sense for you. I use [Conda](https://docs.conda.io/en/latest/) but [pyenv](https://github.com/pyenv/pyenv) is good. 
This is how you do it using Conda.
```bash
conda create -p ./venv-python3.9 python=3.9 -y
```
This will install python in *venv-python3.9* directory in *my-lambda*. After the installation, you must activate this environment as follows:
```bash
conda activate ./venv-python3.9
```
3- I'll be implementing this functions that uses *requests* package and that's what I'm making the custom layer for
```python
import requests
def lambda_handler(event, context):   
    response = requests.get("https://www.example.com/")
    print(response.text)
    return response.text
```
This function just requests the "https://www.example.com" website and prints the response. To see what we would expect, use the following command:
```bash
curl https://www.example.com/ 
```
And we should get something like that.
![curl result](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xx8vwqblrq8qj1sop1sa.png)
4- I'll *pip install* the *requests* package but I'll change its download path using the *--target* param so that I don't download it in the default *site-packages* directory and just download it in the *my-lambda* directory
```bash
pip install --target=./python requests
```
After executing this command, you will see the *packages* directory containing all the dependencies for the *requests* package to work.

5- I'll zip the *python* directory and that's what I'll use as my custom layer. AWS Lambda extracts whatever is in the zip file in the /opt directory and the lambda python runtime automatically supports */opt/python* in the PATH env var. 
```bash
zip -r lambda-layer.zip python
```
This will produce the *lambda-layer.zip* file that I'll upload.

6- Now for the AWS stuff, in order to upload your custom layer, we will do the following:
    1- Navigate to Layers form the left navigation bar then press create layer

![create layer](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sw6m75glxqywrj42kq7y.png)
    2- Fill the *Create Layer* form and upload the zip file we have created.
    3- Then press *create*.

![form](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/y1a2x9vpmipo0r1g5j4l.png)
    4- After your zip file is uploaded and if everything goes well, you will get the following message.

![upload layer](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6b0893hb05u6vz7pnk5l.png)
    5- Navigate again to your lambda function, scroll down and click *Add a lawer*.

![Add layer](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uhjm90z1v4di3pwix3uv.png)
    6- Select a *Custom Layer*.
    7- Choose your recently uploaded layer.
    8- Select your version (generally it will be version 1 unless you have previously made changes to it)
    9- Press *Add*

![Add](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x5qp6rdkevqvpzsc8g7v.png)
    10- To test the new layer, press *Test*.

![test](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/25mee6jzwpaxu1sinjmp.png)
    11- Configure your test event, the default values will be fine.

![test event](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/j5258duvhnpprubgiqlr.png)
    12- Press *Test* again and this time you will see similar results as the *curl* command

![final result](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b9vfyowbo7i1bpwc9hn9.png)


### [Optional] Clean up our local environment
In order to clean up, we will have to do the follwing:
1- Deactivate the python3.9 env
```bash
conda deactivate
```
2- Remove the environement
```bash
conda env remove -p ./venv-python3.9
```
3- Delete the *my-lambda* directory
```bash
cd ..
rm -fr my-lambda
```

And that's it! Now you know how to create custom AWS Lambda Layers 😉.