Python HOW: Starting with Docker
By: Gabriel Harris Ph.D.
Tags: Machine Learning, Python, Docker, Data Science

Photo by Kaique Rocha from Pexels

Docker helps you to package up your project with all of the dependencies needed to run it from anywhere “Build, share and run any application, anywhere!”

Is it a Docker image or a container? 😕

Let’s clear this out straight away. You first build a Docker image by reading a set of instructions from a Dockerfile . Once you run this image, it’s called a container

Docker Engine 🚒

To do any thing Docker, you first need to install the Docker Engine. Docker Engine is available on a variety of Linux platforms, Mac and Windows through Docker Desktop, Windows Server, and as a static binary installation. You can have a look here and choose the one that works for your OS

To check that you have the engine installed, open a terminal and run docker version

Project Structure 📂

Assume your project has the following structure:

app.py is a dummy scripts that only prints out pandas version when executed:

requirements.txt has all the required dependencies:

To learn how to create requirements.txt using pipenv check my article 👉

Creating a Dockerfile 📁

Create a file and name it Dockerfile (without an extension) and add the following instructions to it:

Let’s explain these instructions one by one

Line 1: a valid Dockerfile must start with a FROM instruction that sets the parent image. Parent images are hosted on Docker Hub, so you don’t need to store them locally. Parent images are based on an OS (e.g. Debian, Alpine Linux, etc.). Each subsequent instruction in the Dockerfile modifies this parent image

In our case, the parent image is the slim variant of the python 3.7 image. This image is recommended to start with as it contains the minimal packages needed to run Python so it’s small. There are few other variants that you can choose from. Have a look here for more description

Line 3: COPY the requirements.txt into the app folder in the parent image (first the folder app will be created as it doesn’t exists)

Line 4: set the app folder as the working directory for any instructions that follow (i.e. we can now simply refer to it as . )

Lines 6&7: RUN 2 pip commands, the first to upgrade pip in the parent image and the second to install all the dependencies found in requirements.txt in the parent image ( && to separate commands and \ to use a new line for clarity). We could’ve used 2 RUN instructions but this will create 2 layers instead of one (later in Tips)

Line 9: COPY the app.py to the app folder in the parent image

Line 11: define what command gets executed when running the image (i.e. python app.py )

Tips and Best Practices 🙇

A Docker image consists of read-only layers (each of which consists of the files generated from running an instruction*). The layers are stacked and each one is a delta of the changes from the previous layer

You should always try to:

Keep the number of layers in your image to a minimum as much as possible. This can be done by using one instruction to execute multiple instructions separated with && *Only the instructions RUN , COPY , and ADD create layers (other instructions create temporary intermediate images, and do not increase the size) Start by building the layers that you know won’t be changed during development, such as copying the requirements.txt and installing the dependencies. This is useful because every time a layer instruction has changed, that cashed layer is invalidated and every layer after it, so they’ve to be rebuilt. This is really time saving! so always keep your scripts under development in the last layer You can run app.py with any number of arguments by using CMD["python", "app.py", "param1", "param2", …]

For much more in depth best practices check here

Build Dockerfile as a local Docker image

Your project structure should now looks like:

To build you Docker image, open a terminal and navigating to the project folder, then run the build command:

Don’t forget the . at the end

This will create a local image with a name <image-name> (e.g. starting-with-docker ) and a tag <tag> (e.g. v.1.0 ). If you don’t add a tag (it’s optional), the tag will be latest by default. However, it’s advised to always give a tag. A common practice is to tag with your image version

To check that the image was built, you can list all local images and their IDs by running:

You can see 2 images in fact, one is the parent image python:3.7-slim, and the other is starting-with-docker

Run local Docker image 🏃

Now you’ve built a local image, you can run it from the terminal using the run command:

The --rm flag automatically cleans up the container and removes the file system when the container exits

You’ve built and ran your first Docker image/container!

Push local image to a Container Registry 👆

So you’ve built a local image, and you can run it locally. But how can you share it with others? or even use it yourself from another machine? To be able to do that, you need to host it as a repository on a remote Container Registry

Docker offers a Container Registry at Docker Hub (similar to GitHub but for hosting Docker images instead). However, you will only get 1 free private repository, so unless you don’t mind the rest of your images being public (i.e. anyone can access them), then you need to pay a premium for private repositories (Microsoft Azure, Google Cloud, and AWS all offer private Registries for a premium as well)

Docker Hub Repository Plan

For now, we will use the free private Repository offered by Docker Hub. Head to Docker Hub and create an account (if you haven’t already when downloading the Docker Engine). Navigate to repositories > Create Repository > and name the repository starting-with-docker and set the Visibility to Private > Create

You will then see the Docker command to push a local image to this repository, mine is:

To push your image however, it needs to be first associated with your Docker Hub repository. To do that, you tag the same local image with a new name using the tag command:

This tags the image starting-with-docker:v.1.0 into the starting-with-docker repository in drgabrielharris container registry with the v.1.0 tag

If you list all your local images, you can now see an extra image with the same ID as the source image

Your local images are exactly the same but with different names

Finally, you push to the repository by using the push command:

If you look at your Docker Hub, you will see your image there with the command to pull it

You can push different versions of your image (using different tags) and they will all be here

Run your remote image

You can now run the image directly from the Registry from anywhere using the run command:

You have to be logged into the docker engine with your account

If you would rather have a local copy of the remote image in a new machine, you can pull the remote image:

You have to be logged into the docker engine with your account

To allow others pull/push access to your repository, you need to add them as collaborators. Navigate to your repository > Collaborators > and add users by their Docker IDs (collaborators can’t delete the repository or change its status from private to public). If you want to assign more granular collaborator rights (“Read”, “Write”, or “Admin”), you need to use teams & organizations

What is next?

You can read more about advanced uses of Docker to deploy:

Azure Web Apps (coming soon)

Azure Function Apps (coming soon)

Dash Web Apps (coming soon)

Prodigy annotation tool from Explosion AI (coming soon)

Happy coding!