[TensorFlow] Resolve CUDA DLL missing warning
Tags: tensorflow
Published: 2021-09-14 08:10:47

*This article is written on Sep 14th 2021. It will be old in the future.

## Summary
I got a CUDA DLL missing warning when I ran my TensorFlow program and tried to use GPU on Windows 11. Adding DLL import code in my program solved the issue.
```python
import os
os.add_dll_directory("C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.2/bin")
```
*The CUDA toolkit version may be different for you.

## What happened to me
I'm a beginner of TensorFlow. While setting up GPU binding, I encountered the following warning message. It means that GPU feature is not enabled because a CUDA DLL is missing.

```
W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
```

Then I realized I should install [CUDA toolkit](https://developer.nvidia.com/cuda-toolkit) and [cuDNN](https://developer.nvidia.com/cudnn) to use GPU to run ML code. I've read instructions provided by TensorFlow, NVIDIA and other developers and found that the versions of tools are very important.

- [TensowFlow GPU support manual](https://www.tensorflow.org/install/gpu)
- [cuDNN installation guide v8.1.0](https://docs.nvidia.com/deeplearning/cudnn/archives/cudnn-810/install-guide/index.html)

Even by installing required version on the doc, it did not solve the problem.

Finally I found a key comment in [a Github issue](https://github.com/tensorflow/tensorflow/issues/48868#issuecomment-841396124).
```
I was able to get past this error by doing os.add_dll_directory("C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.2/bin") before importing tensorflow
```
*Even required version of tools are installed and PATH env variable is set as required. There is a case that DLLs are NOT imported as expected !!*

My setup is like the below.
```
GPU: GeForce RTX 2080
Windows Edition: Windows 11 Pro
Windows Version: 21H2
Python: 3.9.7
TensorFlow: 2.6.0
CUDA Toolkit: 11.2
cuDNN: v8.1.0 (January 26th, 2021), for CUDA 11.0,11.1 and 11.2 
```

## Tips
### Message example when GPU feature is enabled
```
I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 5967 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 2080, pci bus id: 0000:01:00.0, compute capability: 7.5
```

### Message example when cuDNN library it not appropriate
```
E tensorflow/stream_executor/cuda/cuda_dnn.cc:362] Loaded runtime CuDNN library: 8.0.5 but source was compiled with: 8.1.0.  CuDNN library needs to have matching major version and equal or higher minor version. If using a binary install, upgrade your CuDNN library.  If building from sources, make sure the library loaded at runtime is compatible with the version specified during compile configuration.
```

### How to avoid Already installed error
If CUDA toolkit installer shows the following error and you cannot install. You should uninstall a program pointed out in the error message. When I was installing several versions of CUDA Toolkit, it occurred.
![already_installed](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xilwl9ia5gbg9u853xg1.png)

### PATH is important but you do not have to do anything
CUDA Toolkit installer cares the PATH setting so you do not have to do anything. Just in case, I show my setup.

CUDA_PATH
```
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2
```

CUDA_PATH_V11_2
```
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2
```

*The version number may be different for your case.

### How to check if PATH is set correctly for DLLs.
Run one of following commands if your error message says `cudart64_110.dll` is missing. If the command returns the absolute path to the file, your PATH setting is fine. The cause is another.

Command Prompt
```
where cudart64_110.dll
```

PowerShell
```
Get-Command cudart64_110.dll
```

## Traps
### cuDNN architecture support description is wrong
[cuDNN archive v8.1.0 for 11.0 for windows](https://developer.nvidia.com/rdp/cudnn-archive#:~:text=Download%20cuDNN%20v8.1.0%20(January%2026th%2C%202021)%2C%20for%20CUDA%2011.0%2C11.1%20and%2011.2) shows download link for x86 but actually it's for x64.