Repository: pytorch
Issue: Failed to static link latest cuDNN while compiling
Labels: module: build,module: cudnn,triaged

### 🐛 Describe the bug

I tried to compile 1.12 with `TORCH_CUDA_ARCH_LIST="6.0;6.1;7.0;7.5;7.5+PTX" CUDNN_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/ CUDNN_INCLUDE_DIR=/usr/include/ USE_CUDNN=1 USE_STATIC_CUDNN=1 USE_STATIC_NCCL=1 CAFFE2_STATIC_LINK_CUDA=1 python setup.py bdist_wheel`.

It cannot find the cuDNN installed on my system:
<img width="587" alt="截屏2022-07-19 下午7 58 23" src="https://user-images.githubusercontent.com/73142299/179744748-92517f55-a1ee-49cb-bdf8-49d94c41fd7b.png">

I located the problem in file `cmake/Modules_CUDA_fix/FindCUDNN.cmake`
```
option(CUDNN_STATIC "Look for static CUDNN" OFF)
if (CUDNN_STATIC)
  set(CUDNN_LIBNAME "libcudnn_static.a")
else()
  set(CUDNN_LIBNAME "cudnn")
endif()

set(CUDNN_LIBRARY $ENV{CUDNN_LIBRARY} CACHE PATH "Path to the cudnn library file (e.g., libcudnn.so)")
if (CUDNN_LIBRARY MATCHES ".*cudnn_static.a" AND NOT CUDNN_STATIC)
  message(WARNING "CUDNN_LIBRARY points to a static library (${CUDNN_LIBRARY}) but CUDNN_STATIC is OFF.")
endif()

find_library(CUDNN_LIBRARY_PATH ${CUDNN_LIBNAME}
  PATHS ${CUDNN_LIBRARY}
  PATH_SUFFIXES lib lib64 cuda/lib cuda/lib64 lib/x64)
```
This code would look for `libcudnn_static.a` if `USE_STATIC_CUDNN` was set.
However, as follow:
<img width="1422" alt="截屏2022-07-19 下午8 02 37" src="https://user-images.githubusercontent.com/73142299/179745670-1d27d21a-eca7-402d-b94e-c2c0701b3757.png">

Newer version of cuDNN does not have the `libcudnn_static.a`.


### Versions

Collecting environment information...
PyTorch version: N/A
Is debug build: N/A
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.6 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: version 3.22.1
Libc version: glibc-2.17

Python version: 3.7.13 (default, Mar 29 2022, 02:18:16)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-4.19.91-011.ali4000.alios7.x86_64-x86_64-with-debian-buster-sid
Is CUDA available: N/A
CUDA runtime version: 10.2.89
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.4.1
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: N/A

Versions of relevant libraries:
[pip3] numpy==1.21.5
[pip3] torch==1.12.0
[conda] mkl                       2022.0.1           h06a4308_117    defaults
[conda] mkl-include               2022.0.1           h06a4308_117    defaults
[conda] numpy                     1.21.5           py37hf838250_3    defaults
[conda] numpy-base                1.21.5           py37h1e6e340_3    defaults
[conda] torch                     1.12.0                   pypi_0    pypi

cc @malfet @seemethere @csarofeen @ptrblck @xwang233