Repository: pytorch
Issue: CUDA error `CUBLAS_STATUS_NOT_INITIALIZED` 
Labels: module: cuda,triaged,module: cublas

### 🐛 Describe the bug

I am using PyTorch through another library ([TabNet](https://github.com/dreamquark-ai/tabnet)).

When I run the fit of a model, this library calls [this](https://github.com/pytorch/pytorch/blob/1892c75a45f599bd7ea652b10c379325abca8648/torch/nn/modules/linear.py#L114) function which causes the following error:
```
  RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling `cublasCreate(handle)`

 // my calls stack (omitted)

  File "/app/.venv/lib/python3.10/site-packages/pytorch_tabnet/abstract_model.py", line 241, in fit
    self._train_epoch(train_dataloader)
  File "/app/.venv/lib/python3.10/site-packages/pytorch_tabnet/abstract_model.py", line 457, in _train_epoch
    batch_logs = self._train_batch(X, y)
  File "/app/.venv/lib/python3.10/site-packages/pytorch_tabnet/abstract_model.py", line 495, in _train_batch
    output, M_loss = self.network(X)
  File "torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/app/.venv/lib/python3.10/site-packages/pytorch_tabnet/tab_network.py", line 586, in forward
    return self.tabnet(x)
  File "torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/app/.venv/lib/python3.10/site-packages/pytorch_tabnet/tab_network.py", line 471, in forward
    steps_output, M_loss = self.encoder(x)
  File "torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/app/.venv/lib/python3.10/site-packages/pytorch_tabnet/tab_network.py", line 156, in forward
    att = self.initial_splitter(x)[:, self.n_d :]
  File "torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/app/.venv/lib/python3.10/site-packages/pytorch_tabnet/tab_network.py", line 706, in forward
    x = self.shared(x)
  File "torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/app/.venv/lib/python3.10/site-packages/pytorch_tabnet/tab_network.py", line 743, in forward
    x = self.glu_layers[0](x)
  File "torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/app/.venv/lib/python3.10/site-packages/pytorch_tabnet/tab_network.py", line 772, in forward
    x = self.fc(x)
  File "torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "torch/nn/modules/linear.py", line 114, in forward
    return F.linear(input, self.weight, self.bias)
```

Can you help me solve the problem? Unfortunately I cannot provide a script to reproduce the problem, but if there is any other information that can help I can provide it

### Versions

```
PyTorch version: 1.13.1+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0] (64-bit runtime)
Python platform: Linux-5.10.133+-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration:
GPU 0: Tesla T4
GPU 1: Tesla T4

Nvidia driver version: 470.141.03
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.6.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.24.1
[pip3] pytorch-tabnet==4.0
[pip3] torch==1.13.1
[conda] Could not collect
```

cc @ngimel @csarofeen @ptrblck @xwang233