How to write a great Dockerfile for Python apps
Tags: containers, docker, dockerfile, python
Published: 2020-10-26 12:11:47

---
title: How to write a great Dockerfile for Python apps
description: Learn 6 things to really improve your Dockerfiles
published: true
author: Christian Barra
tags: containers, docker, dockerfile, python
canonical_url: https://www.pybootcamp.com/blog/how-to-write-dockerfile-python-apps/
---

In the previous article, <a href="/blog/how-to-containerize-python-application/" title="How to containerize a Python Application">how to containerize a Python application</a>, we created a Dockerfile and containerized our application from scratch.

Now we really want our Dockerfile to stand out, **make it more production-ready**, that's the goal of this article.

We will cover 6 different ways to improve our `Dockerfile`:

- setting env variables and a working directory
- avoiding invalidating the Docker cache
- **changing the default user to non-root**
- taking care of zombie processes
- correctly forwarding signals to our application
- updating `pip`, `setuptools` and `wheel`

### Intro

This is the `Dockerfile` we created last time:

```Dockerfile
# 1. Base image
FROM python:3.8.3-slim-buster

# 2. Copy files
COPY . /src

# 3. Install our deps
RUN pip install -r /src/requirements.txt
```

While fully functional, there are a few things we can improve regarding usability, security and performance.

You can clone this [repository](https://github.com/py-bootcamp/movie-recommender/tree/part-1) if you want to follow along.

### Passing the git commit hash

We want to _mark_ each Docker image and container with a tag, this tag is the `git commit hash`.

**At runtime we should be able to determine which version of our software we are running.**

> The idea is that every artifact we generate is traceable, we can go back and check which commit generated it.

The [ARG](https://docs.docker.com/engine/reference/builder/#arg) and [ENV](https://docs.docker.com/engine/reference/builder/#env) instructions can help us achieving it.

`ARG` specifies arguments that we can pass to the `docker build` command, `ENV` are env variables set inside the `Dockerfile` and accessible at runtime, from within the container.

This is the new `Dockerfile`, with `ARG` and `ENV`:

```Dockerfile
FROM python:3.8.3-slim-buster

# 👇
ARG GIT_HASH
ENV GIT_HASH=${GIT_HASH:-dev}
# 👆

COPY . /src

RUN pip install -r /src/requirements.txt
```

The `-dev` is a way to specify defaults. If the `GIT_HASH` argument is omitted then `GIT_HASH` will be set to `dev`.

Let's build our Docker image and check the `GIT_HASH` env variable:

```bash
> docker build -t movie-app .
> docker run --rm movie-app env | grep GIT_HASH
GIT_HASH=dev
```

How do we pass the git commit hash to our Docker image?

We use the [`--build-arg`](https://docs.docker.com/engine/reference/commandline/build/#set-build-time-variables---build-arg) flag from the Docker cli:

```bash
# this will work on bash but not on 🐟
> export GIT_HASH=$(git rev-parse HEAD)
> docker build --build-arg GIT_HASH=${GIT_HASH::7} -t movie-app .
> docker run --rm movie-app env | grep GIT_HASH
GIT_HASH=6a78e6b
```

We don't need the whole commit hash, the first 7 characters are enough.

> Why are we not passing the base image using `ARG`?
> Because we don't want to change the base Docker image from the Docker cli, but only through a new commit.

### Adding a working directory

Right now we are copying our files inside a `/src` folder and then we specify all the other paths relative to `/src`.

Wouldn't be nicer if we could specify a working directory and run commands from that folder?

That would be neat, and [WORKDIR](https://docs.Docker.com/engine/reference/builder/#workdir) is exactly what we need.

```dockerfile
FROM python:3.8.3-slim-buster

ARG GIT_HASH
ENV GIT_HASH=${GIT_HASH:-dev}

# 👇
WORKDIR /project

# 👇
COPY . .

# 👇
RUN pip install -r requirements.txt
```

After we specify a `WORKDIR`, any `RUN`, `CMD`, `ENTRYPOINT`, `COPY` and `ADD` instructions that follow will use that working directory.

Note how the path of `COPY` and `pip install` changed.

Let's test our application:

```
> docker build -t movie-app .
# 💥 it's not python /src/app.py anymore 💥
> docker run --rm -p 8888:8888 movie-app python app.py
> curl localhost:8888
```

### Caching dependencies

Our application has a small number of external dependencies, the `requirements.txt` contains only a few dependencies, so the `pip install` command is fast, just a couple of seconds.

What if it were taking minutes instead of seconds?

Wouldn't be better to cache our dependencies until something changes?

If you try to modify any file inside our application's folder and try to run the Docker build command you will see how Docker builds the image starting from zero.

If you check the console output you should see something like this:

```bash
Step 6/7 : RUN pip install -r requirements.txt
 ---> Running in 2233484e3f72
 ```

Basically any change to our codebase, even if it's not related to `requirements.txt` will invalidate the Docker cache.

We can be smarter and save some time, **we just need to install our dependencies first**.

```dockerfile
FROM python:3.8.3-slim-buster

ARG GIT_HASH
ENV GIT_HASH=${GIT_HASH:-dev}

WORKDIR /project

# 👇
COPY requirements.txt ./
RUN pip install -r requirements.txt
# 👆

COPY . .
```

We added a new `COPY`, just for `requirements.txt`, and moved the `pip install` right after.

If you now try to build the Docker image again, then change the `main.py` and rerun the `docker build` command again that shouldn't invalidate the cache.

This is the output you should see, `Using cache`:
```
> docker build -t movie-app .
Step 6/7 : RUN pip install -r requirements.txt
 ---> Using cache
 ---> cbe7b2865e10
```

### Running your container as non-root user

By default the user running your command inside a Docker container is `root`.

```
> docker run --rm movie-app whoami
root
```

> Long story short, Docker containers should not run as root and is highly recommended to change the default user to a non-root user.

How do we change the user?

We create a new one and we set the new user with the [USER](https://docs.Docker.com/engine/reference/builder/#user) instruction.

```dockerfile
FROM python:3.8.3-slim-buster

ARG GIT_HASH
ENV GIT_HASH=${GIT_HASH:-dev}

WORKDIR /project

# here we create a new user
# note how the commands are using &&
# this helps with caching
RUN useradd -m -r user && \
    chown user /project
# 👆

COPY requirements.txt ./
RUN pip install -r requirements.txt

COPY . .

# 👇 here we set the user
USER user
```

```
> docker build -t movie-app .
> docker run --rm movie-app whoami
user
```

Our `user` can't create new files outside of the `/project` folder (user is the owner of the folder):

```
> docker run --rm touch /hello
touch: cannot touch '/hello': Permission denied
# 👇 but this command would work
> docker run --rm touch hello
```

Let's test our application to be sure it has all the necessary permissions:

```bash
> docker run --rm -p 8888:8888 movie-app python app.py
> curl localhost:8888
```

### Taking care of zombie processes and signals

Each Docker container is a PID namespace, and A PID namespace is a tree, which starts at PID 1, commonly called init.

The entire process of starting the system and shutting it down is maintained by init, when you run a Docker container, PID 1 is what you set inside your `ENTRYPOINT`.

If you don't set it by default Docker will use `/bin/sh -c`, which does not pass signals, making almost impossible to gracefully stop your application.

This is why we need a better init, [Tini](https://github.com/krallin/tini).

`Tini` doesn't only take care of reaping zombie processes but also of forwarding any signals we send to the Docker container to our application process.

> Forwarding signals correctly is really important. Kubernetes relies on signals during the lifecycle of a pod.

[More about Kubernetes and signals here](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination).

```dockerfile
FROM python:3.8.3-slim-buster

ARG GIT_HASH
ENV GIT_HASH=${GIT_HASH:-dev}
# 👇 you can use env variables to pin library versions
ENV TINI_VERSION="v0.19.0"

# 👇
ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /tini
RUN chmod +x /tini
# 👆

WORKDIR /project

RUN useradd -m -r user && \
    chown user /project

COPY requirements.txt ./
RUN pip install -r requirements.txt

COPY . .

USER user

# 👇
ENTRYPOINT ["/tini", "--"]
```

We have two new instructions here, [ADD](https://docs.docker.com/engine/reference/builder/#entrypoint) and [ENTRYPOINT](https://docs.docker.com/engine/reference/builder/#entrypoint).

`ADD` is a really useful instruction, it can add remote files to you Docker image.

The `ENTRYPOINT` specifies the entry point for any command, in our case `python app.py`, pretty much like running `/tini -- python app.py`

### Updating pip, setuptools and wheel

One last thing, it's important to keep `pip`, `setuptools` and `wheel` updated, so it's wise to bump them directly inside our Docker image.

```dockerfile
FROM python:3.8.3-slim-buster

ARG GIT_HASH
ENV GIT_HASH=${GIT_HASH:-dev}
ENV TINI_VERSION="v0.19.0"

ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /tini
RUN chmod +x /tini

# 👇 STYLE YOUR DOCKERFILE LIKE A PRO
RUN pip install -U \
    pip \
    setuptools \
    wheel

WORKDIR /project

RUN useradd -m -r user && \
    chown user /project

COPY requirements.txt ./
RUN pip install -r requirements.txt

COPY . .

USER user

ENTRYPOINT ["/tini", "--"]
```

Let's test our application once again:

```
> docker build -t movie-app .
> docker run --rm -p 8888:8888 movie-app python app.py
curl localhost:8888
```

And with this last step we are done!

### Quick recap

- `ARG` and `ENV` are neat, use them
- Copy and install your dependencies before copying your application
- Don't run containers as root, set a new user with `USER`
- Try to prettify your dockerfiles
- Always use `Tini`
- Defining a `WORKDIR` helps

### Useful resources

- [Github repo for movie-recommender](https://github.com/py-bootcamp/movie-recommender/tree/part-2)
- [ARG](https://docs.docker.com/engine/reference/builder/#arg)
- [ENV](https://docs.docker.com/engine/reference/builder/#env)
- [WORKDIR](https://docs.Docker.com/engine/reference/builder/#workdir)
- [USER](https://docs.Docker.com/engine/reference/builder/#user)
- [ENTRYPOINT](https://docs.docker.com/engine/reference/builder/#entrypoint)