Build a Docker Container with Your Machine Learning Model | by Tina Bu | Towards Data Science

As a data scientist, I don’t have a lot of software engineering experience but I have certainly heard a lot of great comments about containers. I have heard about how lightweight they are compared to traditional VMs, how good they are at ensuring a safe consistent environment for your code, and how my devops effort it saves so you can focus on your code.
However, when I tried to Dockerize my own model, I soon realized it is not that intuitive. It is not at all as simple as putting RUN in front of your EC2 bootstrap script. I found that inconsistencies and unpredictable behaviors happen quite a lot and it can be frustrating to learn to debug a new tool.
All of these motivated me to create this post with all the code snippets you need to factorize your ML model in Python to a Docker container. I will guide you through installing all the pip packages you need and build your first container image. And in the second part of this post, we will be setting up all the necessary AWS environments and kicking off the container as a Batch job in the third and last part of this series.
Disclaimer 1: The model I am talking about here is a batch job on a single instance, NOT a web service with API endpoints, NOT distributed parallel jobs. If you follow this tutorial, the whole process to put your code to a container should not take more than 25 minutes.
Disclaimer 2: If this is your first time reading about containers, this post is probably not going to provide the necessary information for understanding how containers work and I would highly recommend you check out some tutorials online before you proceed.
an AWS account
AWS CLI installed
Docker installed, and account setup at DockerHub
Python 3 installed
To get your code to a container, you need to create a Dockerfile, which tells Docker what you need in your application.
A Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image.
(You can build a Docker image from either Dockerfile or docker-compose.yml. If your code can be refactored as a multi-container Docker application, you may want to look into docker compose but now a Dockerfile should suffice.)
A Docker image starts with a base image and is built up by read-only layers, with each of them adding some dependencies. In the end, you tell the container how to trigger your model.
In the Dockerfile above, I started with the base Python 3.6 stretch image, apt-get updated the system libraries, installed some make and build stuff, checked my python and pip version to make sure they are good, set up my work directory, copied requirements.txt to the container and pip installed all the libraries in it, and finally copied all the other code files to the container, listed all the files to make sure all I need is there and triggered my entrypoint main.py file.
This Dockerfile should work for you if your code folder structure is like this.
- app-name     |-- src          |-- main.py          |-- other_module.py     |-- requirements.txt     |-- Dockerfile
If your code doesn’t have a main.py or shell script to trigger the model training/inferencing yet, then you may want to refactor your code first. Also remember to freeze the library dependencies into a requirements.txt file with pipreqs path/to/project . I recommend using pipreqs rather than pip freeze because when you run pip freeze > requirements.txt it outputs all the installed packages in that environment whereas pipreqs gives you only the ones actually imported by this project. (Install pipreqs with pip(3) install pipreqs if you don’t have it already)
If your code already has an entrypoint, all you need to do is to change the <app- name> to your application’s name and we are ready to build it into an image.
There are a lot of best practices to make a Dockerfile smaller and more efficient but most of them are out of the scope for this post. However, a few things you may want to be mindful about are:
People say instead of starting with a generic Ubuntu image, use an official base image like Alpine Python instead. Alpine Python is a really small Python Docker image based on Alpine Linux, much smaller than the default docker python images but still has everything needed for the most common python projects. But I have found it extremely difficult to work with especially for installing packages.
A Ubuntu base image, on the other hand, will provide more predictable behavior but you need to install all the Python stuff yourself.
So I suggest you start with Python 3.6 stretch, which is the official Python image based on Debian 9 (aka stretch). Python stretch comes with the Python environment and pip installed and up to date, all of which you need to figure out how to install if you choose Ubuntu.
It’s also very tempting to copy and paste some Dockerfile template posted online especially if this is your first Docker project. But it’s suggested to only install the things you actually need to control the size of the image. If you see a whole bunch of make and build stuff other people installed, try to not include them first and see if your container will work. A smaller image generally means it’s faster to build and deploy. (Another reason you should try my minimalism template above!)
Also to keep the image as lean as possible, use .dockerignore which works exactly like .gitignore to ignore files that won’t impact the model.
In your Dockerfile, always add your requirements.txt file before copying the source code. That way, when you change your code and re-build the container, Docker will re-use the cached layer up until the installed packages instead of executing the pip install command on every build even if the library dependencies never changed. No one wants to wait a couple of extra minutes just because you added an empty line in your code.
If you are interested to learn more about Dockerfile, in appendix I there is a quick summary of a few basic commands we used. Or you can check the Dockerfile documentation here.
Now feel free to jump to Step 2 to build a container with the Dockerfile you just created.
After you have a Dockerfile ready, it’s time to build a container image. docker build creates an image according to the instructions given in the Dockerfile. All you need to do is to give your image a name (an an optional version tag).
$ docker build -t IMAGE_NAME:TAG .$ # or$ docker build -t USERNAME/IMAGE_NAME:TAG .
The only difference between the 2 commands is if there is a USERNAME/ before the image name. An image name is made up of slash-separated name components, usually the prefix is the registry hostname. USERNAME/IMAGE_NAME is not a mandatory format for specifying the name of the image in general. But images in Amazon ECR repositories do follow the full REGISTRY/REPOSITORY:TAG naming convention. For example, aws_account_id.dkr.ecr.region.amazonaws.com/my-web-app:latest.
A few things are happening in the commands above.
First, we told the Docker daemon to fetch the Dockerfile present in the current directory (that’s what the . at the end does).
Next, we told the Docker daemon to build the image and give it the specified tag. Tagging has always been very confusing for me. Generally tagging an image is like giving it an alias. It’s the same as assigning an existing image another name to refer to it. It helps with distinguishing versions of your Docker images.
It’s not mandatory to specify a tag name. The :latest tag is a default tag when build is run without a specific tag specified. So it’s suggested that you explicitly tag your image after each build if you want to maintain a good version history.
$ docker build -t USERNAME/IMAGE_NAME .$ docker tag USERNAME/IMAGE_NAME USERNAME/IMAGE_NAME:1.0$ ...$ docker build -t USERNAME/IMAGE_NAME .$ docker tag USERNAME/IMAGE_NAME USERNAME/IMAGE_NAME:2.0$ ...
You can tag an image while building your image with docker build -t USERNAME/IMAGE_NAME:TAG . or explicit tag it like docker tag SOURCE_IMAGE:TAG TARGET_IMAGE:TAG after it was built.
Now if you run docker images, you should see an image exists locally whose repository is USERNAME/IMAGE_NAME and tag is TAG .
$ docker images
I would also recommend you to test your container on your local machine at this point to make sure everything works fine.
$ docker run USERNAEM/IMAGE_NAME:TAG
Feel free to check appendix II for a quick summary of some basic Docker CLI commands or the official document here.
Congratulation! You just baked your model into a container that can be run anywhere Docker is installed. Join me for the second part of this post to configure your AWS environment to meet the prerequisite for scheduling a batch job.
FROM starts the Dockerfile. It is a requirement that the Dockerfile must start with the FROM command. Images are created in layers, which means you can use another image as the base image for your own. The FROM command defines your base layer. As arguments, it takes the name of the image. Optionally, you can add the Docker Cloud username of the maintainer and image version, in the format username/imagename:version.
RUN is used to build up the image you’re creating. For each RUN command, Docker will run the command then create a new layer of the image. This way you can roll back your image to previous states easily. The syntax for a RUN instruction is to place the full text of the shell command after the RUN(e.g., RUN mkdir /user/local/foo). This will automatically run in a /bin/sh shell. You can define a different shell like this: RUN /bin/bash -c 'mkdir /user/local/foo'.
COPY copies local files into the container.
CMD defines the commands that will run on the Image at start-up. Unlike a RUN, this does not create a new layer for the Image, but simply runs the command. There can only be one CMD per a Dockerfile/Image. If you need to run multiple commands, the best way to do that is to have the CMD run a script. CMD requires that you tell it where to run the command, unlike RUN.
EXPOSE creates a hint for users of an image which ports provide services. It is included in the information which can be retrieved via docker inspect <container-id>.
Note: The EXPOSE command does not actually make any ports accessible to the host! Instead, this requires publishing ports by means of the -p flag when using docker run.
PUSH pushes your image to a private or cloud registry.
Some basic Docker CLI commands include:
docker build builds an image from a Dockerfile
docker images displays all Docker images on the machine
docker run starts a container and runs commands in it
docker run options:
-p specify ports in host and Docker container
-it opens an interactive console
-d starts the container in daemon mode (runs in the background)
-e sets environment variables
docker ps displays all running containers
docker rmi removes one or more images
docker rm removes one or more containers
docker kill kills one or more running containers
docker tag tags an image with an alias which can be referenced later
docker login logs in to your Docker registry
I am a machine learning engineer and I write about productivity and self-development besides tech tutorials. I visited Yellowstone on 4th of July thus the cover image.