Title: How to install CUDA Toolkit and cuDNN for deep learning - PyImageSearch

URL Source: https://pyimagesearch.com/2016/07/04/how-to-install-cuda-toolkit-and-cudnn-for-deep-learning/

Published Time: 2016-07-04T14:00:52+00:00

Markdown Content:
![Image 1: installing_cuda_terminal](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20378'%3E%3C/svg%3E)

If you’re serious about doing any type of deep learning, you should be utilizing your _GPU_ rather than your _CPU_. **And the more GPUs you have, the better off you are.**

If you already have an NVIDIA supported GPU, then the next logical step is to install two important libraries:

1.  **The NVIDIA CUDA Toolkit:** A development environment for building GPU-accelerated applications. This toolkit includes a compiler _specifically designed_ for NVIDIA GPUs and associated math libraries + optimization routines.
2.  **The cuDNN library**: A GPU-accelerated library of primitives for deep neural networks. Using the cuDNN package, you can increase training speeds by upwards of 44%, with over _6x speedups_ in Torch and Caffe.

In the remainder of this blog post, I’ll demonstrate how to install both the NVIDIA CUDA Toolkit and the cuDNN library for deep learning.

Specifically, I’ll be using an Amazon EC2 g2.2xlarge machine running Ubuntu 14.04. Feel free to spin up an instance of your own and follow along.

**By the time you’re finished this tutorial, you’ll have a _brand new system ready for deep learning._**

As I mentioned in an [earlier blog post](https://pyimagesearch.com/2014/10/13/deep-learning-amazon-ec2-gpu-python-nolearn/), Amazon offers an EC2 instance that provides access to the GPU for computation purposes.

This instance is named the **g2.2xlarge instance** and costs approximately _$0.65 per hour_. The GPU included on the system is a K520 with 4GB of memory and 1,536 cores.

You can also upgrade to the **g2.8xlarge instance** (_$2.60 per hour_) to obtain _four_ K520 GPUs (for a grand total of 16GB of memory).

For most of us, the g2.8xlarge is a bit expensive, especially if you’re only doing deep learning as a hobby. On the other hand, the g2.2xlarge instance is a totally reasonable option, allowing you to forgo your afternoon Starbucks coffee and trade a caffeine jolt for a bit of deep learning fun and education.

Insider the remainder of this blog post, I’ll detail how to install the NVIDIA CUDA Toolkit v7.5 along with cuDNN v5 in a g2.2xlarge GPU instance on Amazon EC2.

If you’re interested in deep learning, I _highly encourage you_ to setup your own EC2 system using the instructions detailed in this blog post — you’ll be able to use your GPU instance to follow along with future deep learning tutorials on the PyImageSearch blog (and trust me, there will be a lot of them).

_**Note:** Are you new to Amazon AWS and EC2? You might want to read [Deep learning on Amazon EC2 GPU with Python and nolearn](https://pyimagesearch.com/2014/10/13/deep-learning-amazon-ec2-gpu-python-nolearn/) before continuing. This blog post provides step-by-step instructions (with tons of screenshots) on how to spin up your first EC2 instance and use it for deep learning._

### Installing the CUDA Toolkit

Assuming you have either (1) an EC2 system spun up with GPU support or (2) your own NVIDIA-enabled GPU hardware, the next step is to install the [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit).

But before we can do that, we need to install a few required packages first:

$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get install build-essential cmake git unzip pkg-config
$ sudo apt-get install libopenblas-dev liblapack-dev
$ sudo apt-get install linux-image-generic linux-image-extra-virtual
$ sudo apt-get install linux-source linux-headers-generic

One issue that I’ve encountered on Amazon EC2 GPU instances is that we need to _disable the Nouveau kernel driver_ since it _conflicts with the NVIDIA kernel module_ that we’re about to install.

_**Note:** I’ve only had to disable the Nouveau kernel driver on Amazon EC2 GPU instances — I’m not sure if this needs to be done on standard, desktop installations of Ubuntu. Depending on your own hardware and setup, you can_ potentially _skip this step._

To disable the Nouveau kernel driver, first create a new file:

$ sudo nano /etc/modprobe.d/blacklist-nouveau.conf

And then add the following lines to the file:

blacklist nouveau
blacklist lbm-nouveau
options nouveau modeset=0
alias nouveau off
alias lbm-nouveau off

Save this file, exit your editor, and then update the initial RAM filesystem, followed by rebooting your machine:

$ echo options nouveau modeset=0 | sudo tee -a /etc/modprobe.d/nouveau-kms.conf
$ sudo update-initramfs -u
$ sudo reboot

After reboot, the Nouveau kernel driver should be disabled.

The next step is to install the CUDA Toolkit. We’ll be installing [CUDA Toolkit v7.5](https://developer.nvidia.com/cuda-downloads) for Ubuntu 14.04. Installing CUDA is actually a fairly simple process:

1.  Download the installation archive and unpack it.
2.  Run the associated scripts.
3.  Select the default options/install directories when prompted.

To start, let’s first download the `.run` file for CUDA 7.5:

$ wget http://developer.download.nvidia.com/compute/cuda/7.5/Prod/local\_installers/cuda\_7.5.18\_linux.run

With the super fast EC2 connection, I was able to download the entire 1.1GB file in less than 30 seconds:

![Image 2: Figure 1: Downloading the CUDA Toolkit from NVIDIA's official website.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20377'%3E%3C/svg%3E)

**Figure 1:** Downloading the CUDA Toolkit from NVIDIA’s official website.

Next, we need to make the `.run` file executable:

$ chmod +x cuda\_7.5.18\_linux.run

Followed by extracting the individual installation scripts into an `installers` directory:

$ mkdir installers
$ sudo ./cuda\_7.5.18\_linux.run -extract=\`pwd\`/installers

Your `installers` directory should now look like this:

![Image 3: Figure 2: Extracting the set of .run files into the 'installers' directory.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20374'%3E%3C/svg%3E)

**Figure 2:** Extracting the set of .run files into the ‘installers’ directory.

Notice how we have three separate `.run` files — we’ll need to execute each of these individually and in the correct order:

*   `NVIDIA-Linux-x86_64-352.39.run`
*   `cuda-linux64-rel-7.5.18-19867135.run`
*   `cuda-samples-linux-7.5.18-19867135.run`

The following set of commands will take care of actually installing the CUDA Toolkit:

$ sudo ./NVIDIA-Linux-x86\_64-352.39.run
$ modprobe nvidia
$ sudo ./cuda-linux64-rel-7.5.18-19867135.run
$ sudo ./cuda-samples-linux-7.5.18-19867135.run

**Again, make sure you select the default options and directories when prompted.**

To verify that the CUDA Toolkit is installed, you should examine your `/usr/local` directory which should contain a sub-directory named `cuda-7.5` , followed by a sym-link named `cuda` which points to it:

![Image 4: Figure 3: Verifying that the CUDA Toolkit has been installed.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20599%20295'%3E%3C/svg%3E)

**Figure 3:** Verifying that the CUDA Toolkit has been installed.

Now that the CUDA Toolkit is installed, we need to update our `~/.bashrc` configuration:

$ nano ~/.bashrc

And then append the following lines to define the CUDA Toolkit `PATH` variables:

\# CUDA Toolkit
export CUDA\_HOME=/usr/local/cuda-7.5
export LD\_LIBRARY\_PATH=${CUDA\_HOME}/lib64:$LD\_LIBRARY\_PATH
export PATH=${CUDA\_HOME}/bin:${PATH}

Your `.bashrc` file is automatically `source` ‘d each time you login or open up a new terminal, but since we _just_ modified it, we need to manually `source` it:

$ source ~/.bashrc

Next, let’s install cuDNN!

### Installing cuDNN

We are now ready to install the [NVIDIA CUDA Deep Neural Network library](https://developer.nvidia.com/cudnn), a GPU-accelerated library for deep neural networks. Packages such as Caffe and Keras (and at a lower level, Theano) use cuDNN to _dramatically_ speedup the networking training process.

To obtain the cuDNN library, you first need to [create a (free) account with NVIDIA](https://developer.nvidia.com/accelerated-computing-developer-program). From there, you can [download cuDNN](https://developer.nvidia.com/cudnn).

For this tutorial, **we’ll be using cuDNN v5:**

![Image 5: Figure 4: We'll be installing the cuDNN v5 library for deep learning.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20337'%3E%3C/svg%3E)

**Figure 4:** We’ll be installing the cuDNN v5 library for deep learning.

Make sure you download the _cuDNN v5 Library for Linux:_

![Image 6: Figure 5: Since we're installing the cuDNN on Ubuntu, we download the library for Linux.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20337'%3E%3C/svg%3E)

**Figure 5:** Since we’re installing the cuDNN on Ubuntu, we download the library for Linux.

This is a small, 75MB download which you should save to your _local machine_ (i.e., the laptop/desktop you are using to read this tutorial) and then _upload_ to your EC2 instance. To accomplish this, simply use `scp` , replacing the paths and IP address as necessary:

$ scp -i EC2KeyPair.pem ~/Downloads/cudnn-7.5-linux-x64-v5.0-ga.tgz ubuntu@<ip\_address>:~

Installing cuDNN is quite simple — all we need to do is copy the files in the `lib64` and `include` directories to their appropriate locations on our EC2 machine:

$ cd ~
$ tar -zxf cudnn-7.5-linux-x64-v5.0-ga.tgz
$ cd cuda
$ sudo cp lib64/\* /usr/local/cuda/lib64/
$ sudo cp include/\* /usr/local/cuda/include/

Congratulations, cuDNN is now installed!

### Doing a bit of cleanup

Now that we have (1) installed the NVIDIA CUDA Toolkit and (2) installed cuDNN, let’s do a bit of cleanup to reclaim disk space:

$ cd ~
$ rm -rf cuda installers
$ rm -f cuda\_7.5.18\_linux.run cudnn-7.5-linux-x64-v5.0-ga.tgz

In future tutorials, I’ll be demonstrating how to use both CUDA and cuDNN to facilitate faster training of deep neural networks.

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

Summary
-------

In today’s blog post, I demonstrated how to install the CUDA Toolkit and the cuDNN library for deep learning. If you’re interested in working with deep learning, I _highly recommend_ that you setup a GPU-enabled machine.

If you don’t already have an NVIDIA-compatible GPU, no worries — Amazon EC2 offers the g2.2xlarge ($0.65/hour) and the g2.8xlarge ($2.60/hour) instances, both of which can be used for deep learning.

The steps detailed in this blog post will work on _both_ the g2.2xlarge and g2.8xlarge instances for Ubuntu 14.04 — feel free to choose an instance and setup your own deep learning development environment (in fact, I encourage you to do just that!)

The entire process should only take you 1-2 hours to complete if you are familiar with the command line and Linux systems (and have a small amount of experience in the EC2 ecosystem).

Best of all, you can use this EC2 instance to follow along with future deep learning tutorials on the PyImageSearch blog.

**Be sure to signup for the PyImageSearch Newsletter using the form below to be notified when new deep learning articles are published!**

![Image 7](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Join the PyImageSearch Newsletter and Grab My FREE 17-page Resource Guide PDF

Enter your email address below to **join the PyImageSearch Newsletter** and **download my FREE 17-page Resource Guide PDF** on Computer Vision, OpenCV, and Deep Learning.