Dockerising a lambda – Tweeting from a lambda
Tags: python, aws, lambda, docker
Published: 2022-02-14 11:36:13

---
title: Dockerising a lambda – Tweeting from a lambda
published: true
description: We will improve the appearance of the map by leveraging Matplotlib and seaborn
tags: python, aws, lambda, docker
cover_image: https://ik.imagekit.io/thatcsharpguy/posts/python-lambdas/cycles-part-4_ck6phbCDbl.jpg
series: Tweeting from an AWS Lambda
---

The reasoning behind using *lambdas* in AWS is that it is code that is not constantly running on a server (hence the paradigm is known as *serverless*); *lambdas* are executed on demand and for a short time.

Creating a *lambda* is trivial when our code has no dependencies on third-party packages; but we already know that this is not our case – we have dependencies on various packages: *pandas*, *matplotlib*, *seaborn*, *geopandas* y *twython*, plus a few files that contain the map.

You may ask yourself: if "there is no server", where do I install these dependencies? and the answer that AWS gives us is made up of 3 solutions, I am going to list them and I will tell you why I choose one of them:

- You can include your [*dependencies inside a .zip](https://docs.aws.amazon.com/lambda/latest/dg/python-package.html) file along with your code – in our case, the dependencies and files needed exceed the maximum supported size.
- You can create something known as *layers*, which can contain dependencies and other files necessary for the execution of the functions – I did not opt ​​for this because of the size of the dependencies (which is the same as the .zip from the previous point), in fact the *layers* they are a good option for sharing dependencies among multiple *lambdas*.
- You can create a container and run your lambda using it – it's the option I went for, as well as making the deployment easier because the size limitation is no longer a problem for me, it allows me to test the lambda locally (you know I I love being able to run things locally)

## Dependencies

My idea is to make the container as light as possible, so before packaging the python dependencies I need to export the pipenv dependencies to the popular *requirement.txt*, I don't want to install *pipenv* in the container (as I said at the beginning of these posts, it's not necessary to use *pipenv* to manage your dependencies).

I created an instruction in the *Makefile* to generate the *requirements.txt* file:

```makefile
requirements.txt:
	pipenv lock -r > requirements.txt
```

You also have to download the *shapefiles* – just remember that I made an instruction in the *Makefile* to download them:

```makefile
shapefiles:
	wget https://data.london.gov.uk/download/statistical-gis-boundary-files-london/9ba8c833-6370-4b11-abdc-314aa020d5e0/statistical-gis-boundaries-london.zip
	unzip statistical-gis-boundaries-london.zip
	mkdir -p shapefiles
	mv statistical-gis-boundaries-london/ESRI/London_Borough_Excluding_MHW* shapefiles/
	rm -rf statistical-gis-boundaries-london statistical-gis-boundaries-london.zip
```

## Apps entry point

We previously created a file called *app.py* that contains a method that aggregates the functions that we previously created. It is in this file that we will add the entry point of our *lambda* – it can have any name but it must always take two arguments. In this case, I go with the naming convention and I will name it `handler`:

```python
def handler(event, context):
    execute()
    return {"success": True}
```

The return value must be a searialisable object, in this case, we return a dictionary.

## Dockerfile

To create the lambda container I will use Docker, and as you know, Dockerfiles are the recipes we use to create containers. This is the file I will use:

```docker
FROM public.ecr.aws/lambda/python:3.8

COPY requirements.txt .

COPY shapefiles/ ./shapefiles/

RUN pip3 install -r requirements.txt

COPY *.py ./

CMD ["app.handler"]
```

Let's go step by step:

1. `FROM ...`:  Although we can use almost any image as a base, AWS offers us some with guaranteed level of support, `public.ecr.aws/lambda/python:3.8` is an example of them.
2. `COPY requirements.txt ...`: We copy the file with our dependencies.
3. `COPY shapefiles/ ...`: We copy our *shapefiles* folder.
4. `RUN pip3 insta...`: Installs the dependencies outlined in the requirements file.
5. `COPY *.py ...`: Copy the applications file to the container.
6. `CMD ["app...`: Specifies the container what is the entrypoint for the *lambda*.

To build the image we just need to execute a command such as the following:

```bash
docker build -t lambda-cycles .
```

Likewise, I created an insturction in the *Makefile*:

```makefile
container: shapefiles requirements.txt
	docker build -t lambda-cycles .
```

## Testing locally

The big advantage of using containers is that we can run the lambda locally, for example once you create an image with the above *Dockerfile* you can run it as follows.

It is important that you use the `-p` and `-e` flags, the first one to specify a port mapping between the *8080* inside the container and the host's *9000*. The second one specifies the secrets we previously got from Twitter as environment variables.

```bash
docker run \
    -p 9000:8080 
    -e API_KEY="ACTUAL VALUE FOR API_KEY" \
    -e API_SECRET="ACTUAL VALUE FOR API_SECRET" \
    -e ACCESS_TOKEN="ACTUAL VALUE FOR ACCESS_TOKEN" \
    -e ACCESS_TOKEN_SECRET="ACTUAL VALUE FOR ACCESS_TOKEN_SECRET" \
        lambda-cycles
```
From a separate terminal you can execute the *lambda* using *curl*:

```bash
curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" \
		-d '{}'
```

This is how the [repository looks like](https://github.com/fferegrino/tweeting-cycles-lambda/tree/part-3-dockerise) by the end of this post.

Remember that you can find me on Twitter [at @feregri_no](https://twitter.com/feregri_no) to ask me about this post – if something is not so clear or you found a typo. The final code for this series is on [GitHub](https://github.com/fferegrino/tweeting-cycles-lambda) and the account tweeting the status of the bike network is [@CyclesLondon](https://twitter.com/CyclesLondon).