Cloud GPU instance with PyTorch and TensorFlow easy setup in 10 minutes
Tags: linode, gpu, pytorch, tensorflow
Published: 2022-12-22 09:42:25

PyTorch from Meta (ex-Facebook) and TensorFlow from Google are highly popular deep learning frameworks. While GPU is essential to the development and training process of deep learning models, it is time-consuming to build an environment to make GPU available for these frameworks and to make both PyTorch and TensorFlow usable in a single environment. This article shows how to set up an environment with GPU-enabled PyTorch and TensorFlow installed on Linode in 10 minutes. The procedures in this article make it easy to set up a dedicated deep learning environment in the cloud, even for those unfamiliar with setting up a Linux server.

# What is Linode?

[Linode](https://www.linode.com/) is an IaaS acquired by Akamai in February 2022. Linode offers simple and predictable pricing, which includes SSD storage and a certain amount of network transfers within the price, which are often expensive in other cloud computing providers. There are no price differences by region, making spending on cloud providers more predictable. For example, a virtual machine with 32GB of memory with NVIDIA Quadro RTX 6000 costs $1000 per month (as of December 2022), including 640GB of SSD storage and 16TB of network transfer. If you use a service for just part of the month, hourly billing enables you to only be charged for the time the machine is present on the account.

You can use the [Cloud Estimator](https://www.linode.com/ja/estimator/) tool provided by Linode to compare prices with other cloud providers.

# The goal of this article

In this article, we will set up Docker Engine Utility for NVIDIA (nvidia-docker), a container virtualization platform that supports NVIDIA's GPUs, on a GPU instance on Linode, where we will deploy [NGC Container](https://catalog.ngc.nvidia.com/containers), containers officially provided by Facebook and Google for deep learning. You can set up the environment in about 10 minutes with almost no prior knowledge of Linode, Docker, or NGC Container by using StackScripts, Linode's deployment automation function.

[Tools - StackScripts](https://www.linode.com/docs/products/tools/stackscripts/)

![NVIDIA Container Toolkit](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w3iaa94qf61r6p64ymxl.png)
https://github.com/NVIDIA/nvidia-docker/

The environment built with this procedure includes a sample Jupyter Notebook that can be used with [OpenAI Whisper](https://openai.com/blog/whisper/), a speech recognition model that has been widely praised for its extremely high recognition accuracy so that even those who do not develop deep learning models themselves can experience the benefits of GPU instances.

![Voice Recognition with OpenAI Whisper](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4lbsucziqzqbp7700irw.png)

If you provide the Linode Object Storage credentials to the StackScript, the PyTorch and TensorFlow containers will automatically mount the external Linode Object Storage, which can be used to retrieve training data from or to store your deep learning models. Using Linode Object Storage is optional. You can skip it.

# Setup a GPU instance with PyTorch and TensorFlow

First, open the StackScript I have prepared for you from the following link. This StackScript will automatically install nvidia-docker, PyTorch, and TensorFlow. (You must be logged into your Linode account to access the link.) If you can't open this StackScript for some reason, I have uploaded [the contents of this StackScript to GitHub](https://github.com/hokamoto/stackscripts/blob/main/deeplearning.sh) for you.

**deeplearning-gpu**
https://cloud.linode.com/stackscripts/1102035

Click "Deploy New Linode"
![Deploy New Linode](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xpk231js8o1v95f3c18j.png)

StackScript has a feature called UDF (User Defined Fields) that automatically creates an input form with the parameters required for deployment. This StackScript requires you to set the login credential of a non-root user who can SSH into the virtual machine, Access Key to mount Linode Object Storage as external storage (optional). If you want to mount Linode Object Storage, create a bucket and obtain an Access Key in advance.

- [Guides - Create and Manage Buckets](https://www.linode.com/docs/products/storage/object-storage/guides/manage-buckets/)
- [Guides - Manage Access Keys](https://www.linode.com/docs/products/storage/object-storage/guides/access-keys/)

The regions where both Linode Object Storage and GPU instances are available are as follows as of December 2022.

- Atlanta, GA
- Newark, NJ
- Frankfurt, DE
- Singapore, SG

![Linode configuration](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/oxi5ixjbfiwcoffxgvp6.png)

Since GPU instances are available only in limited regions, select a virtual machine type first, then the region. Here I have selected Dedicated 32 GB + RTX6000 GPU x1 in the Singapore region for example.
![Singapore region](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nep2bna7cwe0x5js6b8l.png)
Name the virtual machine, enter the root password, and click "Create Linode".
![Name the virtual machine](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4aqlbnnsn7kf4pwon7g3.png)
The screen will transition to the virtual machine management dashboard. Wait a few minutes until the virtual machine status changes from PROVISIONING to RUNNING. The IP address of the virtual machine you just created is displayed on the same screen, so take note of it.
![IP address](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nwm4smur2s58jed5q5ef.png)
The virtual machine is now booted. The installation process of nvidia-docker and NGC Containers will proceed automatically in the background. Wait 10 minutes for the installation to complete before proceeding to the next step.

# Starting a container
Now let's log in to the virtual machine via SSH. If the setup process performed by StackScript is complete, the following message will appear when you log in. If you do not see this message, log out and wait a few minutes before logging in again. If you have inadvertently started a virtual machine that does not have a GPU, you will get the message "GPU is not available. This StackScript should be used for GPU instances." In that case, please start a GPU instance and redo the procedure from the beginning.
```console
% ssh root@45.118.XX.XX
root@45.118.XX.XX's password:

(snip)

##############################################################################
You can launch a Docker container with each of the following commands:

pytorch: Log into an interactive shell of a container with Python and PyTorch.
tensorflow: Log into an interactive shell of a container with Python and TensorFlow.
pytorch-notebook: Start Jupyter Notebook with PyTorch as a daemon. You can access it at http://[Instance IP address]/
tensorflow-notebookm: Start Jupyter Notebook with TensorFlow as a daemon. You can access it at http://[Instance IP address]/

Other commands:
stop-all-containers: Stop all running containers.
##############################################################################
```

The following five commands are available on the machine created by this StackScript.

| Command | Usage |
|:--|:--|
| pytorch | Start a container with PyTorch installed and enter its interactive shell |
| tensorflow | Start a container with TensorFlow installed and enter its interactive shell |
| pytorch-notebook | Start Jupyter Notebook with PyTorch installed as a daemon |
| tensorflow-notebook | Start Jupyter Notebook with TensorFlow installed as a daemon |
| stop-all-containers | Stop all running containers |

Each container has the directories `/workspace/HOST-VOLUME/` and `/workspace/OBJECT-STORAGE/` to mount the host machine directory and external Linode Object Storage. The container created by the above command is configured to remove the container when it is stopped (`--rm` option of `docker run` is set), so place the files you want to keep in `/workspace/HOST-VOLUME/` or `/workspace/OBJECT-STORAGE/`.
![Directory structure](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p8jqtnn7s88txzaia2e0.png)
Let's spin up Jupyter Notebook with PyTorch as a daemon and run a speech recognition model [OpenAI Whisper](https://openai.com/blog/whisper/). Run the `pytorch-notebook` command from the console.
```console
root@45-118-XX-XXX:~# pytorch-notebook
[I 04:36:22.823 NotebookApp] http://hostname:8888/?token=0ee3290287b3bd90f2e8e3ab447965d3e074267f0d60420b
        http://hostname:8888/?token=0ee3290287b3bd90f2e8e3ab447965d3e074267f0d60420b
```

Jupyter Notebook should now be started. If you get the error "Bind for 0.0.0.0:80 failed: port is already allocated." Stop the existing container first with the `stop-all-containers` command. If you get the above result without any problem, replace *hostname* of the URL with the IP address of the virtual machine that you noted when creating the virtual machine, delete `:8888`, and access the virtual machine from a web browser. The token will change each time the container is started. 

| URL displayed in console | URL to be entered into a web browser |
|:--|:--|
| http://hostname:8888/?token=0ee3290287b3bd90f2e8e3ab447965d3e074267f0d60420b | http://45.118.XX.XX/?token=0ee3290287b3bd90f2e8e3ab447965d3e074267f0d60420b |

Click on `Voice Recognition with OpenAI Whisper.ipynb` in `HOST-VOLUME` to open it.
![Jupyter Notebook](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/m7mmz7901xj5lv2ovg1c.png)
Click `Cell`->`Run All` in the menu to run OpenAI Whisper. The first time you run it, it will take a few minutes to download dependent software and deep learning models.
![OpenAI Whisper](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h8qsxm5e2rpk8dfxouvf.png)
If the execution completes without problems, the last cell will show the result of the speech recognition: "I'm getting them for $12 a night."

**Congratulations! Now you have GPU-enabled PyTorch and TensorFlow**

# Deleting the instance

You can delete the virtual machine that you have finished by clicking "Delete" in the Linode Management Console. The contents of `/workspace/HOST-VOLUME/` (`/root/shared/` from the host OS) will be deleted, so move any files you want to keep to another location.
![Delete instance](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hwiqi3eksxr44e8t88xs.png)

**You are charged even for powered-off virtual machines. Delete virtual machines that you do not want to be charged for.**

[Platform - Billing](https://www.linode.com/docs/products/platform/billing/#if-my-linode-is-powered-off-will-i-be-billed)

# Access control for the instance

Access to the virtual machines created in the above procedure via SSH requires password authentication or public key authentication, and access to Jupyter Notebook requires token authentication. If you want to add access control based on the IP address of the client, refer to the following articles to apply firewalls to port 22 (SSH) and port 80 (HTTP).

[Cloud Firewall - Get Started](https://www.linode.com/docs/products/networking/cloud-firewall/get-started/)

For more advanced access control, Akamai's zero-trust solution, Enterprise Application Access, can be used for integration with external Identity Providers and SSO support.

- [Enterprise Application Access](https://www.akamai.com/resources/product-brief/enterprise-application-access-product-brief)
- [Deploying a Zero Trust Architecture on Linode Cloud in Minutes](https://www.linkedin.com/pulse/deploying-zero-trust-architecture-linode-cloud-minutes-luca-moglia/)

# Enabling HTTPS 

Follow the steps below to enable HTTPS in Jupyter Notebook for production use.

[Running a public notebook server](https://jupyter-notebook.readthedocs.io/en/stable/public_server.html#notebook-public-server)

The five commands listed above are defined as aliases for docker commands in `/root/.bash_profile`. When HTTPS is enabled, the argument of the `-p` option of the docker command used by the `pytorch-notebook` and `tensorflow-notebook` commands should also be modified to the appropriate port such as `443`. And finally, execute `ufw allow 443/tcp` so that the firewall allows port 443.