Repository: pytorch
Issue: why single pytorch process is displayed on each GPUs with memory 0MB
Labels: module: cuda,triaged

When I create a tensor in GPU with pytorch, the same process is displayed on each other GPUs with memory 0MB, even though the memory of only one GPU is allocated. 

For example, if I run these:
```python
import torch
a = torch.rand(100,100)
b = a.cuda()
```
then the result of `nvidia-smi` will be:

 (just ignore process 72497, which is run by other user)
```
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 455.23.04    Driver Version: 455.23.04    CUDA Version: 11.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce RTX 3090    On   | 00000000:3D:00.0 Off |                  N/A |
| 30%   19C    P8    24W / 350W |   1001MiB / 24268MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  GeForce RTX 3090    On   | 00000000:3E:00.0 Off |                  N/A |
| 30%   20C    P8    27W / 350W |      2MiB / 24268MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   2  GeForce RTX 3090    On   | 00000000:40:00.0 Off |                  N/A |
| 30%   32C    P2   108W / 350W |   7361MiB / 24268MiB |      2%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   3  GeForce RTX 3090    On   | 00000000:41:00.0 Off |                  N/A |
| 30%   21C    P8    17W / 350W |      2MiB / 24268MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   4  GeForce RTX 3090    On   | 00000000:B1:00.0 Off |                  N/A |
| 30%   18C    P8    21W / 350W |      2MiB / 24268MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   5  GeForce RTX 3090    On   | 00000000:B2:00.0 Off |                  N/A |
| 30%   20C    P8    17W / 350W |      2MiB / 24268MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   6  GeForce RTX 3090    On   | 00000000:B4:00.0 Off |                  N/A |
| 30%   21C    P8    21W / 350W |      2MiB / 24268MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   7  GeForce RTX 3090    On   | 00000000:B5:00.0 Off |                  N/A |
| 30%   21C    P8    26W / 350W |      2MiB / 24268MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     22060      C   python                            999MiB |
|    1   N/A  N/A     22060      C   python                              0MiB |
|    2   N/A  N/A     22060      C   python                              0MiB |
|    2   N/A  N/A     72497      C   python                           7359MiB |
|    3   N/A  N/A     22060      C   python                              0MiB |
|    4   N/A  N/A     22060      C   python                              0MiB |
|    5   N/A  N/A     22060      C   python                              0MiB |
|    6   N/A  N/A     22060      C   python                              0MiB |
|    7   N/A  N/A     22060      C   python                              0MiB |
+-----------------------------------------------------------------------------+
```

cc @ngimel