Title: Getting Started with Docker for Machine Learning - PyImageSearch

URL Source: https://pyimagesearch.com/2023/09/04/getting-started-with-docker-for-machine-learning/

Published Time: 2023-09-04T13:00:00+00:00

Markdown Content:
* * *

**Table of Contents**
---------------------

*   [Getting Started with Docker for Machine Learning](#h2BPTitle)

*   [Overview: Why the Need?](#h3Overview)
*   [What Are Containers?](#h3Containers)
*   [How Do Containers Differ from Virtual Machines?](#h3VMs)
*   [Most Popular Container Images](#h3Popular)

*   [TensorFlow](#h4TensorFlow)
*   [PyTorch](#h4PyTorch)
*   [Python](#h4Python)
*   [NGINX](#h4NGINX)

*   [Install Docker](#h3Install)
*   [Install Docker Manually](#h3Manually)
*   [Install Docker Desktop](#h3Desktop)
*   [Install NVIDIA Container Toolkit](#h3NVIDIA)

*   [Summary](#h2Summary)

*   [Citation Information](#h3Citation)

* * *

In this tutorial, you will learn what Containers are, how they differ from Virtual Machines (VMs), and what Docker is. Finally, we will top it off by installing Docker on our local machine with simple and easy-to-follow steps.

[![Image 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20393'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2023/09/docker-1-featured.png)

This lesson is the 1st of a 3-part series on **Docker for Machine Learning**:

1.  [**_Getting Started with Docker for Machine Learning_**](https://pyimg.co/k3xw6) **(this tutorial)**
2.  _Lesson 2_
3.  _Lesson 3_

* * *

### [**Overview: Why the Need?**](#TOCh3Overview)

Envision yourself as an **ML Engineer** at one of the world’s largest companies. You make a Machine Learning (ML) pipeline that does everything, from gathering and preparing data to making predictions. All the individual blocks are in the proper order, and you ensure everything works like a well-oiled machine.

After verifying everything, you hand over this pipeline to a colleague to get a review before your pipeline is deployed to perform _real-time streaming predictions_. But, alas! The pipeline broke down when moved from your computer to your colleague’s computer.

Switching gears, imagine yourself being part of a high-tech research lab working with Machine Learning algorithms. You quickly iterate with several different things daily, each with additional requirements and dependencies. One day, you are working with TPUs to run JAX code. Next, you’re trying to make your app available to the world as an API.

In these cases, you’d also encounter annoying times when your entire environment is polluted by one dirty install. Repairing such issues becomes **time-consuming**, especially in research iteration where time is **crucial**.

* * *

### [**What Are Containers?**](#TOCh3Containers)

In the cases we just saw, the problem patterns stem from the _tight coupling_ between the **software** and **hardware** we develop.

This can be a **major impediment** to developing software that “just works” on all systems. When we look at this problem in the Machine Learning context, this can bring quick iteration to a halt.

What if we **decouple** the dependencies of the software we write from the hardware it works on? Could our software development environment be wrapped into a neat and simple package that can be shifted to any machine we want, irrespective of hardware or other dependency issues?

Enter the concept of **Containers**. Containers are **_lightweight packages_** that act as portable, sandboxed environments for use on any hardware that supports them. This would include code, dependencies, files, operating systems, runtime, and system settings.

Let’s break that down. Containers are lightweight since they do not store Operating System (OS) images within them while using a **snapshot** instead. They virtualize the resources available to the OS for a specific process concerning CPU usage, memory, and storage. This allows your process to be portable and sandboxed from all other processes running on the host.

Container runtimes are consistent, meaning they would work precisely the same whether you’re on a _Dell_ laptop with an _AMD_ CPU, a top-notch _MacBook Pro_, or an old _Intel Lenovo ThinkPad_ from 2015.

One of the most popular methods of creating and working with containers is by using [**Docker**](https://www.docker.com/). It is a platform that provides a _batteries-included_ experience for **creating**, **deploying**, and **working** with containers.

Throughout this tutorial, we will use Docker as the software for manipulating and working with containers. Some other alternatives to Docker include [LXC (Linux Container Runtime)](https://linuxcontainers.org/) and [Podman](https://podman.io/).

**_Note:_** _Something of this sort would be useful to an ML Engineer such as yourself because it allows you to expose services from isolated runtimes and ensure horizontal scalability without any hassle._

* * *

### [**How Do Containers Differ from Virtual Machines?**](#TOCh3VMs)

A question that may have popped up by now should be how containers differ from traditional [Virtual Machines](https://pyimagesearch.com/?s=virtual+machines). Let’s see how they **diverge**.

The most notable difference between a virtual machine (VM) and a container is the level of virtualization. This refers to the introduction of an abstraction layer over the hardware at a level where the resources are allocated and utilized separately from the original OS on the machine. This abstraction layer is also referred to commonly as a [hypervisor](https://www.redhat.com/en/topics/virtualization/what-is-a-hypervisor). They tend to have several functions (e.g., creating and deleting VMs, managing, and scheduling them). VMs work to virtualize everything up from the hardware layers on a machine with this layer on top.

On the other hand, containers only virtualize up to the software layers of the container engine that handles all the containers present in a system. It does not go down to the hypervisor layer and instead hands over its own control to the original OS on the machine, which is controlled by the container runtime. The container runtime is like a hypervisor but made only on a software level for containers. It does not have direct access to the hardware of the machine.

Virtual machines typically provide higher isolation and security by working directly with the hardware (depending on the type of hypervisor used). In contrast, Containers tend to simply isolate the specific process and filesystem while delegating the grunt work of allocating resources to the host OS itself via the container runtime.

When working with different Operating Systems vis-à-vis Containers, they tend only to have a snapshot of an OS instead of the full OS image, making them significantly lighter than traditional VMs. But, this brings about a problem wherein we must ensure that the container’s OS matches the same kernel as the original OS. That means the container’s OS must have the same format for requesting machine resources as that of the host OS. For that reason, one would see that a Windows container cannot be used on Linux machines. In comparison, Linux containers may be created on Windows machines with the advent of [WSL2](https://learn.microsoft.com/en-us/windows/wsl/about).

* * *

### [**Most Popular Container Images**](#TOCh3Popular)

Today, popular software applications have container images available for direct use on any hardware that the user desires. Some notable examples include the following.

* * *

#### [**TensorFlow**](#TOCh4TensorFlow)

The container runtime for [TensorFlow](https://hub.docker.com/r/tensorflow/tensorflow/) acts as a simple method to quickly experiment or iterate with TensorFlow in a throwaway fashion. These images also support interfacing with the GPU, meaning you can leverage it for training your Deep Learning networks written in TensorFlow. Note that, using a GPU typically refers to using NVIDIA CUDA-enabled GPUs ([Alternative NVIDIA NGC Container Image here](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/tensorflow)).

[![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20343'%3E%3C/svg%3E)](https://lh4.googleusercontent.com/dzlLMy4ELN3CyhtXJeBOwsL_fnq6tjriJzxmdLiglT_JZk6M5ea8B31ngKIugzFIekDDsTSQc87iN4z5MbJj9q5OY6Na5dZGtvJMAVsC0-Q6TJ8Elcwig4osCRidQ4-0QFuap9aFNMNcgiCsTKvE4QI)

Docker and TensorFlow (source: image created by the author)

* * *

#### [**PyTorch**](#TOCh4PyTorch)

The container runtime for [PyTorch](https://hub.docker.com/r/pytorch/pytorch/) brings torch and torchvision to your runtime, along with all dependencies pre-installed. This allows you to begin your development directly and provides a platform for quick iteration. Similar to TensorFlow, this container image also provides a GPU-compatible version that uses Compute Unified Device Architecture (CUDA) to speed up Tensor processing. ([Alternative NVIDIA NGC Container Image here](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch))

* * *

#### [**Python**](#TOCh4Python)

The container runtime for [Python](https://hub.docker.com/_/python) sets up a Debian Linux instance with Python pre-installed. This is suitable for making a variety of Python applications with other dependencies being added to it at the user’s convenience. This container is commonly used for simpler tasks (e.g., developing web services, APIs, running tests, etc.). Depending on their host platform, one can choose between using Debian Linux, Alpine Linux, or Windows Server Core versions of this image.

* * *

#### [**NGINX**](#TOCh4NGINX)

The container runtime for [NGINX](https://hub.docker.com/_/nginx) sets up a Debian Linux instance with NGINX pre-installed. This allows you to use the popular reverse proxy server with load-balancers and HTTP caches without the hassle of setting it up across different machines. This image is also available in various styles (e.g., the standard Debian Linux base or with an Alpine Linux base).

* * *

### [**Install Docker**](#TOCh3Install)

With this information on why Docker containers are so popular, it’s time we install it on our local machine and start making our containers for Machine Learning.

To later work with all the required components of Docker, we install Docker Desktop. It includes a standalone Desktop GUI (graphical user interface) application that simplifies your experience with Docker.

If you are using Ubuntu, Debian, Fedora, Red Hat Enterprise Linux (RHEL), CentOS, or other common Linux Distributions, you can use the official convenience script from Docker by executing the command below.

\`\`\`bash
curl https://get.docker.com | sh \\
  && sudo systemctl --now enable docker
\`\`\`

This should set up Docker completely on your system! Skip over the next section to see how you can install Docker Desktop for a GUI-based interface.

If you want to improve your Docker experience for later, you can [follow some convenience-enhancing steps](https://docs.docker.com/engine/install/linux-postinstall/) for Linux systems. Note that these are optional and are geared toward superusers.

* * *

### [**Install Docker Manually**](#TOCh3Manually)

Now, let’s look at how to install Docker on a Linux system manually. For this example, we will look at a Fedora Linux setup with GNOME. Note, if you are using Ubuntu or any other Linux distribution, you can simply replace most of these commands with the distribution’s respective package manager (“apt” for Ubuntu, “pacman” for Arch Linux, “brew” for Mac, etc.).

*   Ensure you have `gnome-terminal` installed (or the appropriate terminal as required). To do so, use the following command:

\`\`\`bash 
>> sudo dnf install gnome-terminal
\`\`\`

*   Set up the Docker package repositories

\`\`\`bash
>> sudo dnf -y install dnf-plugins-core
>> sudo dnf config-manager --add-repo https://download.docker.com/linux/fedora/docker-ce.repo
\`\`\`

*   Install Docker Engine, containerd, Docker build kit, Docker Compose, and related command line interfaces (CLIs)

\`\`\`bash
>> sudo dnf install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
\`\`\`

Once this process is complete, you will have a working installation of Docker on your system!

* * *

### [**Install Docker Desktop**](#TOCh3Desktop)

Now, let’s look at how to install Docker Desktop on your system. Follow along!

*   Download the RPM (Red Hat Package Management system) file for [Docker Desktop](https://desktop.docker.com/linux/main/amd64/docker-desktop-4.20.1-x86_64.rpm) (**_Note:_** This link may change in the future. To get the latest link, please [look here](https://www.docker.com/products/docker-desktop/).)
*   Use the following command to install it:

\`\`\`bash
>> sudo dnf install ./docker-desktop-<version>-<arch>.rpm
\`\`\`

*   Start Docker Desktop

\`\`\`bash
systemctl --user start docker-desktop
\`\`\`

The last command should launch a GUI window for Docker Desktop. You should now have a full installation of all the components required to work with Docker and a GUI application to handle it too!

Installation of Docker Desktop can be performed for other Linux distributions or Windows by following the instructions given on [their official website](https://docs.docker.com/desktop/) as well.

* * *

### [**Install NVIDIA Container Toolkit**](#TOCh3NVIDIA)

If you have a CUDA-enabled NVIDIA GPU, you can also install the NVIDIA Container Toolkit to use the GPU for your workflows. To do so, follow the [instructions here](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html).

Note that to install the toolkit and have it work correctly, you must have previously installed the appropriate CUDA drivers for your GPU.

*   Set up the package repository and the GPG (GNU Privacy Guard) Key

\`\`\`bash
$ distribution=$(. /etc/os-release;echo $ID$VERSION\_ID) \\
      && curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \\
      && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \\
            sed 's#deb https://#deb \[signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg\] https://#g' | \\
            sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

\`\`\`

*   Update your system

\`\`\`bash
sudo apt-get update
\`\`\`

*   Install `nvidia-container-toolkit` as a package

\`\`\`bash
sudo apt-get install -y nvidia-container-toolkit-base nvidia-container-toolkit
\`\`\`

*   Configure Docker such that it can recognize `nvidia-container-toolkit` and work with it

\`\`\`bash
sudo nvidia-ctk runtime configure --runtime=docker
\`\`\`

*   Restart the daemon to activate changes

\`\`\`bash
sudo systemctl restart docker
\`\`\`

*   Now, let’s try running a base CUDA container. Enter the following command:

\`\`\`bash
sudo docker run --rm --runtime=nvidia --gpus all nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi
\`\`\`

*   You should see a table giving information about the GPU you have on your device, along with information on the CUDA driver version.

[![Image 3](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20328'%3E%3C/svg%3E)](https://lh4.googleusercontent.com/rsH5fPcW7CerGvW68-liXjh7KneKRQ_hbNiyPZ7Z72V_JJ-rc5r4rB2a2imPpIEkic2kvCdm23f1IavCwbZFeeGccDYnu8Jq8EHSn4LZ3Otk-EkLgfdNqrEWSxGvjaHvLT-u1xYZnDWPzUoMtIdG2T0)

Example output of `nvidia-smi` (source: image created by the author).

* * *

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

* * *

[**Summary**](#TOCh2Summary)
----------------------------

In this blog post, we demystify Docker, highlighting its role in streamlining Machine Learning (ML) workflows. We introduce Docker as a lightweight, efficient tool that packs, ships, and runs applications as portable, isolated containers, contrasting this with the bulkier design of traditional Virtual Machines.

We guide you through setting up Docker on your system, explaining its significance in ML. Docker ensures consistent environments from development to deployment, eliminating compatibility issues and facilitating reproducible research.

Our post underlines Docker’s use cases (e.g., enhancing collaboration, easing cloud deployments, and scaling ML applications), thus demonstrating its value in simplifying and enhancing ML projects.

* * *

### [**Citation Information**](#TOCh3Citation)

**Mukherjee, S.** “Getting Started with Docker for Machine Learning,” _PyImageSearch_, P. Chugh, A. R. Gosthipaty, S. Huot, K. Kidriavsteva, and R. Raha, eds., 2023, [https://pyimg.co/k3xw6](https://pyimg.co/k3xw6)

@incollection{Mukherjee\_2023\_Docker4ML,
  author = {Suvaditya Mukherjee},
  title = {Getting Started with Docker for Machine Learning},
  booktitle = {PyImageSearch},
  editor = {Puneet Chugh and Aritra Roy Gosthipaty and Susan Huot and Kseniia Kidriavsteva and Ritwik Raha},
  year = {2023},
  url = {https://pyimg.co/k3xw6},
}

* * *

![Image 4: Featured Image](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20500%20276'%3E%3C/svg%3E)

### Unleash the potential of computer vision with Roboflow - Free!

*   Step into the realm of the future by [signing up or logging into your Roboflow account](https://roboflow.com/?ref=pyimagesearch). Unlock a wealth of innovative dataset libraries and revolutionize your computer vision operations.
*   Jumpstart your journey by choosing from our broad array of datasets, or benefit from PyimageSearch’s comprehensive library, crafted to cater to a wide range of requirements.
*   Transfer your data to Roboflow in any of the 40+ compatible formats. Leverage cutting-edge model architectures for training, and deploy seamlessly across diverse platforms, including API, NVIDIA, browser, iOS, and beyond. Integrate our platform effortlessly with your applications or your favorite third-party tools.
*   Equip yourself with the ability to train a potent computer vision model in a mere afternoon. With a few images, you can import data from any source via API, annotate images using our superior cloud-hosted tool, kickstart model training with a single click, and deploy the model via a hosted API endpoint. Tailor your process by opting for a code-centric approach, leveraging our intuitive, cloud-based UI, or combining both to fit your unique needs.
*   Embark on your journey today with absolutely no credit card required. Step into the future with Roboflow.

[Join Roboflow Now](https://roboflow.com/?ref=pyimagesearch)

* * *

![Image 5](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Join the PyImageSearch Newsletter and Grab My FREE 17-page Resource Guide PDF

Enter your email address below to **join the PyImageSearch Newsletter** and **download my FREE 17-page Resource Guide PDF** on Computer Vision, OpenCV, and Deep Learning.