Repository: pytorch
Issue: Launching two processes causes hanging
Labels: module: multiprocessing,triaged

## 🐛 Bug

Launching two processes causes hanging.

## To Reproduce

Steps to reproduce the behavior:

1. Run the following Python code:
```python
from torch import multiprocessing as mp
from torchvision.models import resnet50


def main():
    model = resnet50()

    copy1 = [p.clone() for p in model.parameters()]
    copy2 = [p.clone() for p in model.parameters()]

    processes = []
    for rank in range(2):
        process = mp.Process(target=run_worker, args=(rank,))
        process.start()
        processes.append(process)

    for p in processes:
        p.join()


def run_worker(rank):
    print(f'Started Worker {rank}')
    model = resnet50()
    print('Does get here')
    local_copy = [p.clone() for p in model.parameters()]
    print('And should also get here')


if __name__ == "__main__":
    main()
```
2. Wait a while and see that it hangs
3. Press `ctrl-c` to stop execution once you're convinced it hung

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

## Expected behavior

Each process should print `f'Started Worker {rank}'`, `'Does get here'`, `'And should also get here'`, and finally quit. The program should end successfully.

## Actual behavior
Each process prints `f'Started Worker {rank}'` and `'Does get here'`, but does not print `'And should also get here'`. After `ctrl-c`, the two subprocesses remain alive.

## Environment

Collecting environment information...
PyTorch version: 1.7.1
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.5 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: version 3.10.2

Python version: 3.8 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: Could not collect (CUDA Version 10.2.89, checked manually using the `/usr/local/cuda/version.txt` file)
GPU models and configuration:
GPU 0: GeForce RTX 2080 Ti
GPU 1: GeForce RTX 2080 Ti
GPU 2: GeForce RTX 2080 Ti
GPU 3: GeForce RTX 2080 Ti
GPU 4: GeForce RTX 2080 Ti
GPU 5: GeForce RTX 2080 Ti
GPU 6: GeForce RTX 2080 Ti
GPU 7: GeForce RTX 2080 Ti

Nvidia driver version: 440.33.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.19.2
[pip3] torch==1.7.1
[pip3] torchaudio==0.7.0a0+a853dff
[pip3] torchvision==0.8.2
[conda] blas                      1.0                         mkl
[conda] cudatoolkit               10.2.89              hfd86e86_1
[conda] mkl                       2020.2                      256
[conda] mkl-service               2.3.0            py38he904b0f_0
[conda] mkl_fft                   1.2.0            py38h23d657b_0
[conda] mkl_random                1.1.1            py38h0573a6f_0
[conda] numpy                     1.19.2           py38h54aff64_0
[conda] numpy-base                1.19.2           py38hfa32c7d_0
[conda] pytorch                   1.7.1           py3.8_cuda10.2.89_cudnn7.6.5_0    pytorch
[conda] torchaudio                0.7.2                      py38    pytorch
[conda] torchvision               0.8.2                py38_cu102    pytorch

## Additional context

One thing I don't quite understand is that omitting lines 8-9, where `copy1` and `copy2` are defined, somehow prevents the bug from occurring. This is despite the fact that they aren't actually used anywhere in the code. However, in my real scenario, `copy1` and `copy2` are used before launching the two processes, and their memory is shared between them. Each process reads from and writes to this shared memory. So removing those lines isn't a feasible solution.

Also, this code works on my Windows computer (same Python and PyTorch versions), and it also worked on a friends Linux computer (not sure about their environment). This led me to think I might have some environment issues, but the issue persists even after completely reinstalling Conda, Python, and PyTorch (and nothing else). Another thing I tried is checking different versions of PyTorch. The issue persisted on versions 1.8.0, 1.6.0, 1.5.1, but did not occur on version 1.4.0.