Repository: pytorch
Issue: RuntimeError: CUDA error: CUBLAS_STATUS_INTERNAL_ERROR when calling `cublasCreate(handle)`
Labels: module: cuda,triaged

## 🐛 Bug

Hello,

I have an RTX 2070 GPU and I'm struggling for about 4 weeks to try and use it with pytorch.
I have tried different combinations of the offered binaries (pytorch nighlty + cuda10.2, pytorch nigthly + cuda11.1, pytorch installed from source + cuda 10.2, pytorch + cuda pip packages  , pytorch 1.9 installed from source + cuda 11.2) but ended up with the same result which I will describe in the next section.

Environment details:
1. Windows 10 19042.867
2. Nvidia Driver: 461.92
3. Cuda version: tried with both cuda 10.2 and cuda 11.2
4. cuDNN version: 8.1.133 (for cuda 11.2 attempt, i used the appropriate cuDNN)
5. Conda version:  4.10.0
6. Python versions: 3.8.8, 3.8.5
7. Torch versions: 1.8.1, 1.9 
## To Reproduce

Steps to reproduce the behavior:

1. Install pytorch with cuda support as described [here](https://pytorch.org/)
2.
```
>>> import torch
>>> x=torch.randn(1024, 1024).cuda() <---- this takes around 15-20 minutes to execute
>>> y = torch.matmult(x, x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: CUDA error: CUBLAS_STATUS_INTERNAL_ERROR when calling `cublasCreate(handle)`
```

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

## Expected behavior

Expected behavior is for it to work.

## Environment

Collecting environment information...
PyTorch version: 1.9.0.dev20210409
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 10 Pro
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect

Python version: 3.8 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: 10.2.89
GPU models and configuration: GPU 0: GeForce RTX 2070
Nvidia driver version: 461.92
cuDNN version: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\bin\cudnn_ops_train64_8.dll
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.19.2
[pip3] torch==1.9.0.dev20210409
[pip3] torchaudio==0.9.0.dev20210409
[pip3] torchvision==0.10.0.dev20210409
[conda] blas                      1.0                         mkl
[conda] cudatoolkit               10.2.89              h74a9793_1
[conda] mkl                       2020.2                      256
[conda] mkl-service               2.3.0            py38h196d8e1_0
[conda] mkl_fft                   1.3.0            py38h46781fe_0
[conda] mkl_random                1.1.1            py38h47e9c7a_0
[conda] numpy                     1.19.2           py38hadc3359_0
[conda] numpy-base                1.19.2           py38ha3acd2a_0
[conda] pytorch                   1.9.0.dev20210409 py3.8_cuda10.2_cudnn7_0    pytorch-nightly
[conda] torchaudio                0.9.0.dev20210409            py38    pytorch-nightly
[conda] torchvision               0.10.0.dev20210409      py38_cu102    pytorch-nightly
## Additional context

I also tried building from source.  A desciption of the problems I faced is [here](https://discuss.pytorch.org/t/pytorch-cuda-11-2-build-from-source-runtimeerror-cuda-error-no-kernel-image-is-available-for-execution-on-the-device/116392)

Thank you in advance,


cc @ngimel