Installing Multiple CUDA & cuDNN Versions in Ubuntu | by Mathanraj Sharma | Towards Data Science

Nowadays most of us use the CUDA toolkit to train deep learning models. But things get messy when we grow from PROJECT to PROJECTS. Yes, soon as we start to work on two or three deep learning projects we may end up with the need for different environments. Sometimes we may need different versions of CUDA and cuDNN for different projects.
If you are a person who only uses Python the Anaconda (conda package manager) will come in handy in such situations. But there are situations you need to be relay on C++ as well. One good example is when you try to use TensorRT to optimize your inference time. Recently I ended up in such a situation where I need to use a specific version of TensorRT which is not compatible with the CUDA version I have in my Ubuntu system. I struggled a bit and found a way to have multiple CUDA installation on the system.
First of all, you need to check whether your laptop/desktop has a NVIDIA GPU. Open your terminal and run the below command.
sudo lshw -C display
You will get a similar output and notice that VGA from NVIDIA is available.
Then we need to ensure that we have installed proper Nvidia Driver
sudo ubuntu-drivers devices
Listed are the available drivers that we can install. Some suggest it is good to install the latest “distro non-free” (proprietary) version. But I choose to install the recommended version by ubuntu. It is totally up to you.
# To install recommended sudo ubuntu-drivers autoinstall OR# To install specific distrosudo apt install nvidia-driver-[version number]
Once the driver installation finished, reboot your system and run this command to check whether the driver installed properly.
nvidia-smi
You can see the Nvidia configurations printed. Note, the CUDA version displayed on the top right corner is the driver version. Not the CUDA-toolkit version itself.
NOTE: You can repeat Step 03, 04 & 05 as many times as you need to install different versions of CUDA. But first, install the latest compatible version of CUDA Toolkit for the Nvidia driver you installed. If you start to install the CUDA Toolkit from the smaller version it will replace the driver.
The main intention of this article starts here. CUDA is a parallel computing platform developed by NVIDIA it will help us to train our models much faster than in CPUs. We can install a cuda-toolkit suitable for the Nvidia-driver through the apt manager. Since we are intended to install multiple versions it is good to install through deb (Debian) or tar installation method.
Go to the cuda-toolkit archive and click on the version you need to download. It will redirect you to the download page. Select the preferences as shown in the below image
Go to your Download directory then run the commands popped on the Cuda download page.
cd ~/Downloads
Instead of the last command to install Cuda,
sudo apt install cuda-[version]Eg: If you have downloaded cuda 11.2sudo apt install cuda-11.2
It will take a few minutes to download and install the cuda-toolkit.
cd /usr/localls
You will see cuda-11.2 directory listed there.
After installing the cuda-toolkit we need to install a suitable version of cuDNN libraries in order to use GPU for deep learning tasks. Actually, cuDNN stands for CUDA Deep Neural Network library is a GPU-accelerated library of primitives for deep neural networks. He will optimize our training and make it run in GPU using CUDA.
Since we installing multiple versions it is recommended to follow tar installation. If you install using deb or run file it will replace the existing cudnn installation.
So to install cudnn go to the download page and select the cudnn suitable for your CUDA version.
Download cuDNN v8.0.4 (September 28th, 2020), for CUDA 11.1Download cuDNN v8.0.4 (September 28th, 2020), for CUDA 11.0Download cuDNN v8.0.4 (September 28th, 2020), for CUDA 10.2Download cuDNN v8.0.4 (September 28th, 2020), for CUDA 10.1
Note: Since there is no cuDNN for CUDA 11.2 is listed we can use cuDNN for CUDA 11.1. Caution don’t go for any further down versions.
Download the tar file of the selected version
Once you download go to your Download directory and run the below commands,
Before issuing the following commands, you’ll need to replace x.x and v8.x.x.x with your specific CUDA and cuDNN versions and the package date you downloaded.
And M.m with the CUDA version you installed in the example it is 11.2
cd ~/Downloads [OR where you downloaded]tar -xzvf cudnn-x.x-linux-x64-v8.x.x.x.tgztar -xzvf cudnn-x.x-linux-aarch64sbsa-v8.x.x.x.tgzsudo cp cuda/include/cudnn*.h /usr/local/cuda-M.m/includesudo cp cuda/lib64/libcudnn* /usr/local/cuda-M.m/lib64sudo chmod a+r /usr/local/cuda-M.m/include/cudnn*.h /usr/local/cuda-M.m/lib64/libcudnn*
Once you have installed your CUDA versions, you will get something similar to this.
cd /usr/localls
As you can see I have installed “cuda-10.1”, “cuda-11”, “cuda-11.1”, “cuda-11.2”. And there is also another directory “cuda” which is installed default when I installed multiple CUDA versions. Now,
sudo gedit ~./bashrc
Copy and paste these lines
export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}export LD_LIBRARY_PATH="/usr/local/cuda/lib64:/usr/local/cuda-10.1/lib64:/usr/local/cuda-11/lib64:/usr/local/cuda-11.1/lib64:/usr/local/cuda-11.2/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}"#make sure you add your cuda-X.x lib64 paths
Save and close the editor. What it does is, when a specific version of CUDA library is required by a task it will simply search through all the paths listed here one by one and finds it.
Now all set reboot your computer and you can enjoy your multiple versions of CUDA & cuDNN.