Repository: pytorch
Issue: I cannot use x.to(GPU) or x.cuda(GPU)
Labels: module: cuda,triaged,module: cuda graphs

## 🐛 Bug

<!-- A clear and concise description of what the bug is. -->

## To Reproduce

Steps to reproduce the behavior:
```
>>> import torch
>>> x=torch.rand(2)
>>> x
tensor([0.8933, 0.5460])
>>> x.to(0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: CUDA error: operation would make the legacy stream depend on a capturing blocking stream
>>> x.cuda(0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: CUDA error: operation would make the legacy stream depend on a capturing blocking stream


```

![image](https://user-images.githubusercontent.com/21344631/134495818-75a703aa-e87d-4a0d-8d75-a17415c972c1.png)




<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

## Expected behavior
```
>>> import torch
>>> x=torch.rand(2)
>>> x
tensor([0.9744, 0.4073])
>>> print(torch.cuda.is_available())
True
>>> x.to(0)
tensor([0.9744, 0.4073], device='cuda:0')
>>> quit();
```
<!-- A clear and concise description of what you expected to happen. -->

## Environment
1.pytorch version:1.9.1
2.cuda 11.1 or 10.2
3.GPU:A100

![image](https://user-images.githubusercontent.com/21344631/134495616-bc51d100-f4f6-4fbd-8a07-eee6939ccc9f.png)
Please copy and paste the output from our
[environment collection script](https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py)
(or fill out the checklist below manually).

You can get the script and run it with:
```
wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py
```
Collecting environment information...
PyTorch version: 1.9.1
Is debug build: False
CUDA used to build PyTorch: 11.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.3 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.17

Python version: 3.7.10 (default, Jun  4 2021, 14:48:32)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.11.0-34-generic-x86_64-with-debian-bullseye-sid
Is CUDA available: False
CUDA runtime version: 10.1.243
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.2.4
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.2.4
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.2.4
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.2.4
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.2.4
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.2.4
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.2.4
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.20.3
[pip3] torch==1.9.1
[pip3] torchaudio==0.9.0a0+a85b239
[pip3] torchvision==0.10.1
[conda] blas                      1.0                         mkl
[conda] cudatoolkit               11.1.1               h6406543_8    conda-forge
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] mkl                       2021.3.0           h06a4308_520
[conda] mkl-service               2.4.0            py37h5e8e339_0    conda-forge
[conda] mkl_fft                   1.3.0            py37h42c9631_2
[conda] mkl_random                1.2.2            py37h219a48f_0    conda-forge
[conda] numpy                     1.20.3           py37hf144106_0
[conda] numpy-base                1.20.3           py37h74d4b33_0
[conda] pytorch                   1.9.1           py3.7_cuda11.1_cudnn8.0.5_0    pytorch
[conda] torchaudio                0.9.1                      py37    pytorch
[conda] torchvision               0.10.1               py37_cu111    pytorch



 - PyTorch Version (e.g., 1.0):1.9.1
 - OS (e.g., Linux):Ubuntu 20.04.3 LTS (x86_64)
 - How you installed PyTorch (`conda`, `pip`, source):conda
 - Build command you used (if compiling from source):conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c conda-forge

 - Python version:3.7.10
 - CUDA/cuDNN version: both 10.2 and 11.1
 - GPU models and configuration: A100
 - Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->


cc @ngimel