DOCKER + NVIDIA + CONDA + H204GPU — Make an Machine Learning Docker Image
By: Didier Itembe
Tags: Conda, Ubuntu, Nvidia Docker, Machine Learning, H2o4gpu

In this article, Docker is used to create an image that will help to solve Machine Learning (ML) problems and develop advanced ML models on GPU. Docker has been conquering the IT world since 2013 and is still mentioned everywhere. Just as virtualization revolutionized IT departments a few years ago, this technology is revolutionizing the way IT departments work in their application areas.

To achieve our goal, we will use the wrapper Nvidia-docker and the open sources Conda, H2O4GPU.

To do this we will also use the wrapper Nvidia-docker and the open sources Conda, H2O4GPU.

Before realizing the project, a brief description of the why, what and how around Docker will be given.

Why docker?

Let’s say for example that we have different technologies like Redis, Ansible, Jenkins and want to set up all these together to develop an application. It would be very difficult, because of their compatibility with the underlying operating system (OS). Sometimes, there can be versions of these technologies, that are not compatible with the OS. In that case, we have to get another OS that is compatible with the software. The dependencies, libraries on the OS should also be compatible with these software. With the time, the architecture of the application would be very difficult to maintain.

This issue brings us to Docker, and with docker, we can easily solve it. With docker, we have the possibility to change and modify a technology without affecting the components of the other technologies, and also modify the underlying OS.

For machine learning projects, as said above, time can be saved by the installation of libraries or packages. Projects can be run on different servers. People can follow and help by maintaining of Machine Learning projects. In the same (or different) environment, different models can be trained on individual or multiple containers and served.

What is docker?

Docker can refer to many different things, such as the company that supports the project, which is also called Docker Inc. There is also the Docker IT software which is a containerization technology. All these names can be sometimes confusing.

Docker can be referenced as a platform of different software, tools, projects. That means that if a person say that she/he uses docker for their project, it can be a reference to docker image or docker compose or both. All these tools form together the platform docker. With docker, we can easily install and start software. It also solves the problem of dependencies and libraries by installing and running a software in separate container. We only need to make sure that docker is installed on the system.

After the installation of docker, a terminal can be opened. After this, the command docker version can be used to check the version of the docker installation.

A new container is started with the command run as docker run hello-world

Behind the command, a series of actions take place. We will not go to deep in details here, for this, the docker run reference can be checked.

When the command docker run hello-world is run, the docker client (docker CLI) take that command and passes it to the docker server (docker daemon). The docker daemon is responsible for creating, uploading images, maintaining container…. The docker server will check if there is a local copy of the hello-world image on the local machine in the image cache. If the copy is not on the machine, the server will contact the docker hub and ask about the image called hello-world.

The docker hub or docker store is a repository that contains a lot of containerized versions of applications. Most companies have their products containerized and available in that public docker registry. The docker will answer, “yes I have it”. The server will then download that image from the docker hub locally into the image cache, such that if we run that command again, the server will directly get the image in the cache without having to download it again and again.

And then the default start up command of the image is run. When we run an image, we actually create a container and start it. Docker run command can be replaced by the commands create and start, like:

docker run = docker create + docker start

We can override the default start up command by adding a command (e.g. here ls ) after the name of the image:

If the command or program which is used to override the default command doesn’t exist in the container file system image, then an error message is returned.

An image is a single file containing all the configuration, libraries, dependencies to run a specific program. The image is downloaded on the hard drive and can be used to create a container. To remove all the images container, and clean the image cache, the command docker system prune can be used.

Some other useful commands:

What is a container?

A container is an instance of an image. It is an isolated environment. Docker technology uses methods such as namespacing and cgroups (control groups), to isolate processes so they can run independently of each other. Namespacing is a method that is used to isolate resources of a process or group of processes and cgroups limit the amount of resources of a process. This independence is the goal of the container. It is the capability to run separately multiple processes. This will make better use of the infrastructure while ensuring the security of working with disparate systems.

The container processes their own little space of memory and hard drive, their own services, mounts, networking interfaces. The operating systems like Ubuntu, Debian, CentOS consist of the same OS kernel which is Linux and a set of software that make these OS different. The software here can be different drivers, tools, file managers, compilers etc. Sharing the kernel here means that docker can run different types of OS (like Ubuntu, Debian…) as long they are based on Linux which is the same OS kernel of these different OS. If the OS kernel is Ubuntu, the docker can run different distributions of OS like Debian, Suse… based on that OS kernel.

Dockerfile

It was said above in paragraph what is docker? that there is a lot of containerized versions of applications available in the docker hub. In the hub, images of most common OS, tools, services can be found.

A custom image can be created, if we don’t find what we are looking for in the docker hub. With the creation of the custom image, some applications can be run inside of the personalized container. After the creation of the custom image, that custom image can be pushed to the docker hub, making it available for the public.

For this, a dockerfile is created. A dockerfile is a file written in a specific format without extension that docker can process.

The dockerfile will contain the configuration for the image to build. The configuration will define what different programs the container will contain. After the creation of the dockerfile, we pass it to the docker client, then the docker CLI will present the file to the docker server and the docker server will look at the configuration in the dockerfile to build the image. That image can be used to run container.

Let’s now create the dockerfile, which can be used to build an image and run a container to solve machine learning problems on GPU. In that dockerfile, an image from Nvidia-docker is used as base image. Conda and H2O4GPU are installed in it. With H204GPU, ML problems can be solved and advanced models can be built on GPU. A model can run much faster on GPU than on CPU. This is possible with the usage of Nvidia CUDA. That being said, check if Nvidia driver are compatible with the GPU of the host.

So let’s start. We are on Ubuntu 16.04 and have installed docker. Unfortunately, nvidia-docker is not supported on windows and mac OS, which is not a good news for windows and mac OS users.

First, a working directory is created and called “forDocker” with the command mkdir forDocker . Then switch into it. A small script is created with the command touch gpucode.py and the following lines of code:

import h2o4gpu

import numpy as np X = np.array([[1.,1.], [1.,4.], [1.,0.]])

model = h2o4gpu.KMeans(n_clusters=2,random_state=1234).fit(X)

model.cluster_centers_

are added in the last created file to test if the installation of h2o4gpu was successful or not (refer to H204GPU).

Then, the “Dockerfile” is created with touch Dockerfile without extension. The dockerfile will contain the following configuration:

# Use a docker image as base image

FROM nvidia/cuda:9.0-cudnn7-runtime # Declare some ARGuments

ARG PYTHON_VERSION=3.6

ARG CONDA_VERSION=3

ARG CONDA_PY_VERSION=4.5.11 # Installation of some libraries / RUN some commands on the base image

RUN apt-get update && \

DEBIAN_FRONTEND=noninteractive apt-get install -y — no-install-recommends python3-pip python3-dev wget \

bzip2 libopenblas-dev pbzip2 libgl1-mesa-glx && \

apt-get clean && \

rm -rf /var/lib/apt/lists/* # INSTALLATION OF CONDA

ENV PATH /opt/conda/bin:$PATH RUN wget — quiet https://repo.anaconda.com/miniconda/Miniconda$ CONDA_VERSION-$ CONDA_PY_VERSION-Linux-x86_64.sh -O ~/miniconda.sh && \

/bin/bash ~/miniconda.sh -b -p /opt/conda && \

rm ~/miniconda.sh && \

/opt/conda/bin/conda clean -tipsy && \

ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh && \

echo “. /opt/conda/etc/profile.d/conda.sh” >> ~/.bashrc && \

echo “conda activate base” >> ~/.bashrc ENV TINI_VERSION v0.16.1

ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /usr/bin/tini

RUN chmod +x /usr/bin/tini # Create a conda environment to use the h2o4gpu

RUN conda update -n base -c defaults conda && \

conda create -y -n gpuenvs -c h2oai -c conda-forge h2o4gpu-cuda9 # You can add the new created environment to the path

#ENV PATH /opt/conda/envs/gpuenvs/bin:$PATH # Copy the files in the actual directory from the directory forDocker on our host into the container in the directory /testenv

COPY . /testenv # Set the working directory to be /testenv

WORKDIR /testenv

On the left side in caps, there are instructions like FROM , RUN , COPY , WORKDIR … and on the right side the arguments of those instructions like nvidia/cuda:9.0-cudnn7-runtime , CONDA_VERSION=3 . The dockerfile can be now built with command:

docker build –t ongpu:1 .

This will take some time. After the build is done. A successful message will be returned like on the next image:

A container can be run with the command:

docker run -it --runtime=nvidia ongpu:1

It allows to run commands directly into the container. After the container is started, the created environment is activated with:

conda activate gpuenvs or source activate gpuenvs

And in the environment, the script can be executed with python gpucode.py

The result [[1. 1.] [1. 4.]] is then returned

The environment can be exited with conda deactivate and the container with the command ctrl+c , ctrl+d or exit depending of the OS.

Now we are good to go. The environment can now be used to solve machine learning problems. The dockerfile can be modified by adding or removing some libraries based on different tasks.

Conclusion

In this article, we have briefly seen the basis of docker, what is about and why it has become so viral in the IT technologies. We created an image based on h2o4gpu, an open source that can be used to develop advanced ML models on GPU. There are a lot more to discover about docker like docker swarm or docker compose. These topics are not covered here. But for now, we have some good basis to create our own custom image and share them with the public. With that, I wish you all much fun in the world of docker and machine learning.

Some references

[1] Docker Inc (Get Started). URL https://docs.docker.com/get-started/, https://docs.docker.com/reference/ .

[2] Docker Cheat Sheet. URL https://github.com/wsargent/docker-cheat-sheet .

[3] P. Srivastav. Learn to build and deploy your distributed applications easily to the cloud with Docker. URL https://docker-curriculum.com/ .

[4] J. Calmels, F. Abecassis and P. Ramarao. Enabling GPUs in the Container Runtime Ecosystem. URL https://devblogs.nvidia.com/gpu-containers-runtime/

[5] R. Olson, J. Calmels, F. Abecassis and P. Rogers. NVIDIA Docker: GPU Server Application Deployment Made Easy. URL https://devblogs.nvidia.com/gpu-containers-runtime/