Strange issue with tensor asyncio and RPC
Labels: oncall: distributed

### 🐛 Describe the bug

Hi Folks,

I have a bizarre issue and can't narrow down a problem.  I'll update a case if you can provide more debug commands for this issue.

## platform and system
   * Latest torch version, CUDA 11.7, Python 3.10 and 3.11 ( test both)
   * Test CPU , CUDA , Linux , MacOs, WSL2 , Docker.

Consider on the same host, communicating over RPC, mainly to address the issue of torch.mp.
i.e., I need one process to detach the tensor from CUDA to the CPU, send 
it to the agent and upload it back to GPU.

Thus, 

Agent <--- > Observer.  
(Observer takes the tensors and sends via async RPC on request from Agent, it async call 
and Observer responds as torch.future).

```
Observer:
   # agent will issue this call, and foobar_rpc put tensor take a tensor from cuda 
   # to CPU and send.
   foobar_rpc():
      tensor.clone().cpu()
```

The agent takes the torch future and puts the asyncio queue.
Now in the asyncio dequeue phase (i.e, asyncio wait on queue),  does 

```
Agent
   async def the_rpc_call() 
         fu = rpc_async().foobar() # args etc. removed to make it clear
         await THE_QUEUE.put(the_fu)
```

```
async def foo_consumer()
        # skip code for a loop, not relevant. 
        the_future = THE_QUEUE.get()  # <- asyncio queue
	the_future.wait()       # wait for torch future
	val = future.value()   # read value
	
	x, y = val # ( let assume and x and y two named tuple)
	i.e., 

        x.rewards <-- is tensor
        x.observation.  <-- is tensor
        x.someint <-- native python type

	# If I do all good
	xsum=torch.sum(x.rewards, dim=0)
        print(xsum)   # <-- no issue

        some_dict , list etc = xsum  # <--- error in RPC , or blocking>
        left_hand = xsum.clone()     # <-- error
        left_hand = copy_ same error etc.

        # if I try to do anything else, it either 
        # block or crash, or I see an error in RPC 

       await some_foo(x, y)
       no errors. at all
 ```

It looks like somewhere in downstream code, it either has a lock, but on read-only, it works; on left-hand assignment, it fails for tensors. But if that is the case, the last call some_foo (x, y) should produce the same behavior. i.e., if I pass x and y to downstream code, I have an issue now.

Does RPC logic in torch, do anything specific with the tensor it receives somewhere deep in a C library with the tensors it receives?

Thank you,

### Versions

Linux

```
Collecting environment information...
PyTorch version: 1.13.0
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.10 (x86_64)
GCC version: (conda-forge gcc 10.4.0-19) 10.4.0
Clang version: Could not collect
CMake version: version 3.22.1
Libc version: glibc-2.36

Python version: 3.10.8 (main, Nov 24 2022, 14:13:03) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.19.0-26-generic-x86_64-with-glibc2.36
Is CUDA available: True
CUDA runtime version: 11.7.99
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3090
Nvidia driver version: 520.61.05
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.4
[pip3] torch==1.13.0
[pip3] torchvision==0.14.0
[pip3] torchviz==0.0.2
[conda] blas                      1.0                         mkl
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] mkl                       2021.4.0           h06a4308_640
[conda] mkl-service               2.4.0           py310h7f8727e_0
[conda] mkl_fft                   1.3.1           py310hd6ae3a3_0
[conda] mkl_random                1.2.2           py310h00e6091_0
[conda] numpy                     1.23.4          py310hd5efca6_0
[conda] numpy-base                1.23.4          py310h8e6c178_0
[conda] pytorch                   1.13.0          py3.10_cuda11.7_cudnn8.5.0_0    pytorch
[conda] pytorch-cuda              11.7                 h67b0de4_0    pytorch
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] torchvision               0.14.0              py310_cu117    pytorch
[conda] torchviz                  0.0.2                    pypi_0    pypi
```

This is on mac 1.12

```
Collecting environment information...
PyTorch version: 1.12.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.0.1 (x86_64)
GCC version: Could not collect
Clang version: 14.0.0 (clang-1400.0.29.202)
CMake version: version 3.24.3
Libc version: N/A

Python version: 3.8.13 (default, Mar 28 2022, 06:16:26)  [Clang 12.0.0 ] (64-bit runtime)
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.4
[pip3] torch==1.12.0
[pip3] torchaudio==0.12.0
[pip3] torchvision==0.13.0
[conda] blas                      1.0                         mkl
[conda] mkl                       2021.4.0           hecd8cb5_637
[conda] mkl-service               2.4.0            py38h9ed2024_0
[conda] mkl_fft                   1.3.1            py38h4ab4a9b_0
[conda] mkl_random                1.2.2            py38hb2f4e1b_0
[conda] numpy                     1.22.4                   pypi_0    pypi
[conda] numpy-base                1.22.3           py38h3b1a694_0
[conda] torch                     1.12.0                   pypi_0    pypi
[conda] torchaudio                0.12.0                   pypi_0    pypi
[conda] torchvision               0.13.0                   pypi_0    pypi
```

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu