Installing Multiple CUDA & cuDNN Versions in Ubuntu
By: Mathanraj Sharma
Tags: Ubuntu, Cuda, AI, Gpu, Deep Learning

PC: Lucas Kepner (unsplash.com)

Nowadays most of us use the CUDA toolkit to train deep learning models. But things get messy when we grow from PROJECT to PROJECTS. Yes, soon as we start to work on two or three deep learning projects we may end up with the need for different environments. Sometimes we may need different versions of CUDA and cuDNN for different projects.

If you are a person who only uses Python the Anaconda (conda package manager) will come in handy in such situations. But there are situations you need to be relay on C++ as well. One good example is when you try to use TensorRT to optimize your inference time. Recently I ended up in such a situation where I need to use a specific version of TensorRT which is not compatible with the CUDA version I have in my Ubuntu system. I struggled a bit and found a way to have multiple CUDA installation on the system.

Step 01: Check whether your system is CUDA capable

First of all, you need to check whether your laptop/desktop has a NVIDIA GPU. Open your terminal and run the below command.

sudo lshw -C display

PC Author

You will get a similar output and notice that VGA from NVIDIA is available.

Step 02: Install proper Nvidia Driver

Then we need to ensure that we have installed proper Nvidia Driver

sudo ubuntu-drivers devices

PC: Author

Listed are the available drivers that we can install. Some suggest it is good to install the latest “distro non-free” (proprietary) version. But I choose to install the recommended version by ubuntu. It is totally up to you.

# To install recommended

sudo ubuntu-drivers autoinstall OR # To install specific distro

sudo apt install nvidia-driver-[version number]

Once the driver installation finished, reboot your system and run this command to check whether the driver installed properly.

nvidia-smi

PC: Author

You can see the Nvidia configurations printed. Note, the CUDA version displayed on the top right corner is the driver version. Not the CUDA-toolkit version itself.

NOTE: You can repeat Step 03, 04 & 05 as many times as you need to install different versions of CUDA. But first, install the latest compatible version of CUDA Toolkit for the Nvidia driver you installed. If you start to install the CUDA Toolkit from the smaller version it will replace the driver.

Step 03: Installing CUDA Toolkit

The main intention of this article starts here. CUDA is a parallel computing platform developed by NVIDIA it will help us to train our models much faster than in CPUs. We can install a cuda-toolkit suitable for the Nvidia-driver through the apt manager. Since we are intended to install multiple versions it is good to install through deb (Debian) or tar installation method.

Go to the cuda-toolkit archive and click on the version you need to download. It will redirect you to the download page. Select the preferences as shown in the below image

PC: Author (Captured from NVIDIA Website)

Go to your Download directory then run the commands popped on the Cuda download page.

cd ~/Downloads

PC: Author (Captured from NVIDIA Website)

It will take a few minutes to download and install the cuda-toolkit.

cd /usr/local

ls

You will see cuda-11.2 directory listed there.

Step 04: Installing cuDNN

After installing the cuda-toolkit we need to install a suitable version of cuDNN libraries in order to use GPU for deep learning tasks. Actually, cuDNN stands for CUDA Deep Neural Network library is a GPU-accelerated library of primitives for deep neural networks. He will optimize our training and make it run in GPU using CUDA.

Since we installing multiple versions it is recommended to follow tar installation. If you install using deb or run file it will replace the existing cudnn installation.

So to install cudnn go to the download page and select the cudnn suitable for your CUDA version.

Download cuDNN v8.0.4 (September 28th, 2020), for CUDA 11.1 Download cuDNN v8.0.4 (September 28th, 2020), for CUDA 11.0 Download cuDNN v8.0.4 (September 28th, 2020), for CUDA 10.2 Download cuDNN v8.0.4 (September 28th, 2020), for CUDA 10.1

Note: Since there is no cuDNN for CUDA 11.2 is listed we can use cuDNN for CUDA 11.1. Caution don’t go for any further down versions.

Download the tar file of the selected version

Once you download go to your Download directory and run the below commands,

Before issuing the following commands, you’ll need to replace x.x and v8.x.x.x with your specific CUDA and cuDNN versions and the package date you downloaded.

And M.m with the CUDA version you installed in the example it is 11.2

cd ~/Downloads [OR where you downloaded]

tar -xzvf cudnn-x.x-linux-x64-v8.x.x.x.tgz

tar -xzvf cudnn-x.x-linux-aarch64sbsa-v8.x.x.x.tgz sudo cp cuda/include/cudnn*.h /usr/local/cuda-M.m/include

sudo cp cuda/lib64/libcudnn* /usr/local/cuda-M.m/lib64



sudo chmod a+r /usr/local/cuda-M.m/include/cudnn*.h /usr/local/cuda-M.m/lib64/libcudnn*

Step 05: Path Setting

Once you have installed your CUDA versions, you will get something similar to this.

cd /usr/local

ls

PC: Author

As you can see I have installed “cuda-10.1”, “cuda-11”, “cuda-11.1”, “cuda-11.2”. And there is also another directory “cuda” which is installed default when I installed multiple CUDA versions. Now,

sudo gedit ~./bashrc

Copy and paste these lines

export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}



export LD_LIBRARY_PATH="/usr/local/cuda/lib64:/usr/local/cuda-10.1/lib64:/usr/local/cuda-11/lib64:/usr/local/cuda-11.1/lib64:/usr/local/cuda-11.2/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}" #make sure you add your cuda-X.x lib64 paths

Save and close the editor. What it does is, when a specific version of CUDA library is required by a task it will simply search through all the paths listed here one by one and finds it.

Now all set reboot your computer and you can enjoy your multiple versions of CUDA & cuDNN.