[BTY] Day 1: Install NVIDA Driver, CUDA, and CUDNN on Ubuntu 20.04
Tags: betterthanyesterday, nvidia, cuda
Published: 2022-01-24 16:22:14


Updated: 23.01.2022

## References

[How to install Cuda 11.4 on ubuntu 18.04(or 20.04](https://medium.com/@anarmammadli/how-to-install-cuda-11-4-on-ubuntu-18-04-or-20-04-63f3dee2099)

[How to Install CUDA on Ubuntu 20.04](https://linoxide.com/how-to-install-cuda-on-ubuntu/)

[Installing the NVIDIA driver, CUDA and cuDNN on Linux](https://gist.github.com/kmhofmann/cee7c0053da8cc09d62d74a6a4c1c5e4)

## **Section 0 — Gather information**

- **Perform Pre-installation actions**
    
    <aside>
    ⚠️ View the detail in the following web bookmark
    
    </aside>
    
    [Installation Guide Linux :: CUDA Toolkit Documentation](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#pre-installation-actions)
    
    ### [2.1. Verify You Have a CUDA-Capable GPU](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#verify-you-have-cuda-enabled-system)
    
    ```
    lspci | grep -i nvidia
    ```
    
    ### [2.2. Verify You Have a Supported Version of Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#verify-you-have-supported-version-of-linux)
    
    ```
    uname -m && cat /etc/*release
    ```
    
    ### [2.3. Verify the System Has gcc Installed](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#verify-that-gcc-is-installed)
    
    ```
    gcc --version
    ```
    
    ### [2.4. Verify the System has the Correct Kernel Headers and Development Packages Installed](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#verify-kernel-packages)
    
    ```bash
    uname -r
    ```
    
- **Find the right NVIDIA driver version**
    
    <aside>
    ⚠️ The CUDA runfile also includes a version of the NVIDIA graphics driver, but I like to separate installing either, as the version supplied with CUDA Is not necessarily the latest version of the driver.
    
    </aside>
    
    - If you know your graphic card:
        
        [Download Drivers](https://www.nvidia.com/Download/index.aspx)
        
        
        - If you don’t know, detect the model of your nvidia graphic card and the recommended driver by execute the following command:
            
            ```
            $ ubuntu-drivers devices
            == /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0 ==
            modalias : pci:v000010DEd00001C03sv00001043sd000085ABbc03sc00i00
            vendor   : NVIDIA Corporation
            model    : **GP106 [GeForce GTX 1060 6GB]**
            driver   : nvidia-driver-390 - distro non-free
            driver   : nvidia-driver-435 - distro non-free
            **driver   : nvidia-driver-440 - distro non-free recommended**
            driver   : xserver-xorg-video-nouveau - distro free builtin
            ```
            
        - You can also find latest driver version here: [https://www.nvidia.com/en-us/drivers/unix/](https://www.nvidia.com/en-us/drivers/unix/)
            
    
- **Find the right CUDA version by checking CUDA Compatibility**
    
    Access [NVIDIA Driver Documentation](https://docs.nvidia.com/deploy/cuda-compatibility/index.html#binary-compatibility__table-toolkit-driver) to check the compatibility and find the proper download link [here](https://developer.nvidia.com/cuda-toolkit-archive).
    
    Or you can find the latest version of CUDA from here: [https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=Ubuntu](https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=Ubuntu)
    

## **Section 1 — Cleaning remaining files**

- **Find and delete files**
    
    <aside>
    ⚠️ BECAREFUL THIS SECTION!
    
    </aside>
    
    Deleting any NVIDIA/CUDA packages you may already have installed
    
    ```
    sudo rm /etc/apt/sources.list.d/cuda*
    sudo apt remove --autoremove nvidia-cuda-toolkit
    sudo apt remove --autoremove nvidia-*
    ```
    
    Deleting any remaining Cuda files on **/usr/local/**
    
    ```
    sudo rm -rf /usr/local/cuda*
    ```
    
    Purge any remaining NVIDIA configuration files
    
    ```
    sudo apt-get purge nvidia*
    ```
    
    updating and deleting unnecessary dependencies.
    
    ```
    sudo apt-get update
    sudo apt-get autoremove
    sudo apt-get autoclean
    ```
    

## **Section 2 — Installing Cuda**

- **Installing the NVIDIA driver**
    
    Execute these command (you can get the link by right click and Copy link from Download button from [this block](https://www.notion.so/NVIDIA-7db026b58a3d4fce9347de190006a069)):
    
    ```bash
    wget https://us.download.nvidia.com/XFree86/Linux-x86_64/470.94/NVIDIA-Linux-x86_64-470.94.run
    sudo sh NVIDIA-Linux-x86_64-450.57.run
    nvidia-smi 
    ```
   
    
- **Installing CUDA repository pin**
    
    First, download the CUDA repository pin:
    
    ```
    $ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin
    ```
    
    Next, move the pin to the **`/etc/preferences.d`** directory and rename it as **`cuda-repository-pin-600`**.
    
    ```
    $ sudo mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600
    ```
    
- **Installing CUDA**
    
    You may need to confirm that the display driver is already installed, and de-select installation of the display driver.
    
    ```bash
    wget https://developer.download.nvidia.com/compute/cuda/11.4.2/local_installers/cuda_11.4.2_470.57.02_linux.run
    sudo sh cuda_11.4.2_470.57.02_linux.run
    ```
   
    
    Unselect the NVIDIA driver because we’ve installed in the previous step.
   
    

## **Section 3 — Adding Cuda to Path**

- **For a specific user**
    
    Open .`profile` file
    
    ```
    sudo nano ~/.profile
    ```
    
    and add these lines
    
    ```bash
    # set PATH for cuda 11.4 installation
    if [ -d "/usr/local/cuda-11.4/bin/" ]; then
        export PATH=/usr/local/cuda-11.4/bin${PATH:+:${PATH}}
        export LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
    fi
    ```
    
    Or you can add the given directories to your `PATH` and `LD_LIBRARY_PATH` by adding the following lines to your `.bashrc`, `.zshrc`, or whatever shell you are using:
    
    ```bash
    export PATH=/usr/local/cuda-11.4/bin:$PATH
    export LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64:$LD_LIBRARY_PATH
    
    ```
    
- **For all users (not tested yet)**
    
    If you want to permanently add to all users, following this (The `profile` file works for all users. If you want it to be valid only for the active user, change the ".bashrc" file):
    
    ```bash
    # Modify the "/etc/profile" file
    vi /etc/profile
    
    # Press the I key to enter editing mode and move the cursor to the end of the file. Additional entries：
    
    export PATH=$PATH:/path/to/dir;
    
    # Press the Esc key to exit edit mode, and :wq to save the file.
    
    # Make the configuration effective
    source /etc/profile
    
    ```
    
    Ref: [https://stackoverflow.com/a/53443815/6563277](https://stackoverflow.com/a/53443815/6563277)
    

## **Section 4 — Checking**

Now reboot your computer and check **Nvidia driver** and **Cuda.**

- **For checking Nvidia driver**
    
    ```
    nvidia-smi
    ```
    

    
- **For checking Cuda version**
    
    ```
    nvcc --version
    ```
   

## **Section 5 — Testing**

- **Make CUDA file and run it**
    
    ```
    sudo nano kernel.cu
    ```
    
    and add these lines to this file.
    
    ```cpp
    #include <stdio.h>
    
    __global__
    void saxpy(int n, float a, float *x, float *y)
    {
      int i = blockIdx.x*blockDim.x + threadIdx.x;
      if (i < n) y[i] = a*x[i] + y[i];
    }
    
    int main(void)
    {
      int N = 1<<20;
      float *x, *y, *d_x, *d_y;
      x = (float*)malloc(N*sizeof(float));
      y = (float*)malloc(N*sizeof(float));
    
      cudaMalloc(&d_x, N*sizeof(float));
      cudaMalloc(&d_y, N*sizeof(float));
    
      for (int i = 0; i < N; i++) {
        x[i] = 1.0f;
        y[i] = 2.0f;
      }
    
      cudaMemcpy(d_x, x, N*sizeof(float), cudaMemcpyHostToDevice);
      cudaMemcpy(d_y, y, N*sizeof(float), cudaMemcpyHostToDevice);
    
      // Perform SAXPY on 1M elements
      saxpy<<<(N+255)/256, 256>>>(N, 2.0f, d_x, d_y);
    
      cudaMemcpy(y, d_y, N*sizeof(float), cudaMemcpyDeviceToHost);
    
      float maxError = 0.0f;
      for (int i = 0; i < N; i++)
        maxError = max(maxError, abs(y[i]-4.0f));
      printf("Max error: %f\n", maxError);
    
      cudaFree(d_x);
      cudaFree(d_y);
      free(x);
      free(y);
    }
    ```
    
    Next, use `nvcc` the Nvidia CUDA compiler to compile the code and run the newly compiled binary:
    
    ```bash
    nvcc -o kernel kernel.cu
    ./kernel
    ```
    
    **Result** :
    
    ```
    Max error: 0.000000
    ```
    

## Section 6 **— Install CUDNN**

- **Find the right version**
    
    CuDNN doesn't come with Cuda. To [download CuDNN](https://linoxide.com/how-to-install-cuda-on-ubuntu/) you need to register to become a member of the NVIDIA Developer Program which is free.
    
   
    For me I choose 8.2.4 due to the current CUDA version and Triton 21.10 which I’m experimenting.
    
- **Download the cuDNN Library for Linux**
    
    <aside>
    ⚠️ Download [file](https://developer.nvidia.com/compute/machine-learning/cudnn/secure/8.2.4/11.4_20210831/cudnn-11.4-linux-x64-v8.2.4.15.tgz) to local and then upload to the server (use SCP) due to the download link requires login.
    
    </aside>
    
    Extract the cuDNN package:
    
    ```bash
     tar -xzvf cudnn-11.4-linux-x64-v8.2.4.15.tgz
    ```
    
    Then copy the following files to the CUDA directory:
    
    ```
    $ sudo cp cuda/include/cudnn*.h /usr/local/cuda/include
    $ sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64
    $ sudo chmod a+r /usr/local/cuda/include/cudnn*.h /usr/local/cuda/lib64/libcudnn*
    ```
    
    

## Useful command

### Export the CUDA device with the lowest GPU ultilization

[How do I select which GPU to run a job on?](https://stackoverflow.com/questions/39649102/how-do-i-select-which-gpu-to-run-a-job-on)

```jsx
export CUDA_VISIBLE_DEVICES=$(nvidia-smi --query-gpu=memory.free,index --format=csv,nounits,noheader | sort -nr | head -1 | awk '{ print $NF }')
```

### Find CUDA version (in case of `nvcc` command not found)

```bash
/usr/local/cuda/bin/nvcc --version
```

### Find CuDNN version

- For CUDA 8.1 and above

```bash
cat /usr/local/cuda/include/cudnn_version.h | grep CUDNN_MAJOR -A 2
```