Provisioning Lambda Docker Images with AWS CDK (Python)
Tags: aws, python, tutorial, docker
Published: 2023-02-10 04:43:51

This is Part 1 of a two-part post. In this part, I will cover provisioning the lambda docker image, and in the next part, I will cover interacting with it.

Covered:

-   Declare Lambda Docker Image in CDK
-   Importing local modules to main docker script
-   Writing the Dockerfile
-   Reading/Writing to Docker Image filesystem
-   Testing locally


### Prerequisites
- [Docker](https://docs.docker.com/get-docker/) installed
- CDK installed and set up correctly

Prior experience with Docker would be helpful, although I managed to figure this out and it was my first time dealing with Docker so you should be fine.

### Directory structure
```json
cdk-project
|-- lambdas/
    |-- docker_func/
	    |-- app/
		   |-- __init__.py
		   |-- components/
			  |-- __init__.py
			  |-- com1.py
			  |-- com2.py
		   |-- utils/
		      |-- __init__.py
			  |-- ut1.py
		   |-- main.py
		   |-- output.json # Optional
		|-- Dockerfile
		|-- README.md
		|-- requirements.txt
|-- project/
   |-- project_stack.py
```

---

### Process
Step by step process based on the above project structure. The file names I’ll be using in these steps directly relate to the files in the above Directory Structure.

Note: Starting in the root directory of your CDK project `../#/cdk-project>`

1 - Declare the docker using CDK

```python
# ../project/project_stack.py
docker_lambda = _lambda.DockerImageFunction(self, 'DockerLambda', 
    code=_lambda.DockerImageCode.from_image_asset(
             'lambdas/docker_func'
    ),  
    timeout=Duration.seconds(30), # Default is only 3 seconds  
    memory_size=1000, # If your docker code is pretty complex  
    environment={  
        "BUCKET_NAME": bucket.bucket_name # Optional  
    }  
)
```

2 - In a new terminal navigate to top-level of docker function
```bash
cd lambdas/docker_func
```

3 - Provision some basic components/util classes
```python
# app/components/com1.pyclass Com1():  
def __init__(self):  
  pass
###

# app/components/com2.pyclass Com2():  
def __init__(self):  
  pass
###

# app/utils/ut1.pyclass Ut1():  
def __init__(self):  
  pass
```

4 - Set `components` and `utils` to be treated as modules by adding `__init__.py` to each folder and importing the classes specified in #3.

```python
# app/components/__init__.py  
from components.com1 import *  
from componented.com2 import *

###
# app/utils/__init__.py  
from utils.ut1 import *
```
> **Warning**
> If you skip this step you will not be able to import these modules in the way that you think. When docker packages up your files it moves them all around which breaks relative imports (even though it may work locally)


5 -  Write the dockerfile
```go
FROM public.ecr.aws/lambda/python:3.8

# Copy function code  
COPY app/main.py ${LAMBDA_TASK_ROOT}

# Add local dependencies  
ADD app/components components  
ADD app/utils utils  
ADD app/template.docx template.docx  

# Install the function's dependencies using file requirements.txt  
# from your project folder.COPY requirements.txt  .  
RUN  pip3 install -r requirements.txt --target "${LAMBDA_TASK_ROOT}"

# Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)  
CMD [ "main.handler" ]
```

> **Info**
> I'm using `main.py` in this example, note all occurances and change if needed

In the _Add local dependencies_ section of the Dockerfile there’s some important details.  
These `ADD` commands tell docker that when it is packaging up your files it needs to grab the contents of `source` and place it in `target` when done.

eg. `ADD source target` > `ADD app/components components`

This means that once our app is built, we can access the files inside `components` via
```python
from components.com1 import Com1
```

The last line `CMD ["main.handler"]` sets the docker entry point to the function `handler` in `main.py`:
```python
# main.py
def handler(event, context):  
	pass
```

If you don’t want to call your file `main.py` then just change all instances where I use it to your preferred name.

6 - Deploy your changes. It should take a while to upload initially. I mean a long while. Like 7–10 minutes.

### Writing to the local file system (optional)
If you try to write a file you will get:
```
[ERROR] OSError: [Errno 30] Read-only file ststel: '/path'
```

This is because the only writable directory with any AWS Lambda function is the `/tmp` folder.

If you need to write a document, do it like below:
```python
document.save('/tmp/report.docx') # example using python-docx
```

## Testing locally
Uploading the docker image every time you make a change is possibly the fastest way to transform your brain from goo to soup. Don’t do that. Do this instead.

Say you’ve got a Docker Image that uses the `python-docx` library (not important, it’s just a good example). Your script loads a `template.docx` file and saves the resulting `output.docx` file.

To test locally we use python’s `if __name__ == "__main__":` magic to provide dummy data to our handler function. This method will **_ONLY_** run when we manually run the script.

```python
# main.py
def handler(payload, context, prefix='/tmp/'):  
   # ... code  
   document.save(prefix+"output.json")  
   # ... more code  
   
   return  
   
if __name__ == "__main__":  
    with open("input.json") as f: # input.json sample event input  
        handler(json.load(f), None, prefix='')
```

Note in the above code I’m using a `prefix` argument to my `handler()` function. This is to control where the output file is written. For my local testing, I don’t want the file to be written to the `/tmp` directory because that will be a reserved folder once I deploy.

To run the file locally you will need to be in the `app` folder (referencing the directory structure from the start).
```powershell
C:\<path>\cdk-project\lambdas\docker_func\app> python main.py
```

---
That should pretty much cover it. The above steps are all I needed to do to get my lambda working. For reference what it does is:

1.  One lambda function preps a json object and sends it to the Docker Image.
2.  Docker Image parses the json data, imports 3rd party libraries, reads and populates template documents, saves the output to docker filesystem, uploads the result to S3, and finally returns the S3 object URI to the Lambda that invoked it.

Hopefully this is helpful to anyone who reads. Thanks.