Repository: pytorch
Issue: deadlock when using mp.spawn multiprocessing
Labels: module: multiprocessing,triaged

## 🐛 Bug

PyTorch deadlocks when using queues and events with `mp.spawn`.

## To Reproduce
Minimal example:
```
import torch.multiprocessing as mp

def main_worker(gpu, queue, event):
    print(f'gpu {gpu} putting into queue')
    queue.put({'gpu': gpu})
    
    print(f'gpu {gpu} waiting')
    event.wait()

def main():
    num_gpus = 4

    queue = mp.Queue()
    event = mp.Event()

    context = mp.spawn(main_worker, nprocs=num_gpus, args=(queue, event), join=False)
    print('started processes')

    for i in range(num_gpus):
        print(f'getting {i}th queue value')
        d = queue.get()
        print('popped', d)

    event.set()
    context.join()

if __name__ == '__main__':
    main()
```
Output:
```
started processes
getting 0th queue value
gpu 1 putting into queue
gpu 0 putting into queue
gpu 3 putting into queue
gpu 2 putting into queue
```
And then it hangs

## Expected behavior
Using `Process` instead of `spawn` works fine:
```
import torch.multiprocessing as mp

def main_worker(gpu, queue, event):
    print(f'gpu {gpu} putting into queue')
    queue.put({'gpu': gpu})
    
    print(f'gpu {gpu} waiting')
    event.wait()

def main():
    num_gpus = 4

    queue = mp.Queue()
    event = mp.Event()

    jobs = []
    for i in range(num_gpus):
        p = mp.Process(target=main_worker, args=(i, queue, event))
        p.start()
        jobs.append(p)
    print('started processes')

    for i in range(num_gpus):
        print(f'getting {i}th queue value')
        d = queue.get()
        print('popped', d)

    event.set()
    for p in jobs:
        p.join()

if __name__ == '__main__':
    main()
```
Output:
```
gpu 0 putting into queue
gpu 0 waiting
started processes
getting 0th queue value
gpu 1 putting into queue
popped {'gpu': 0}
getting 1th queue value
gpu 2 putting into queue
gpu 1 waiting
popped {'gpu': 1}
getting 2th queue value
gpu 2 waiting
gpu 3 putting into queue
popped {'gpu': 2}
getting 3th queue value
gpu 3 waiting
popped {'gpu': 3}
```
## Environment

PyTorch version: 1.4.0
Is debug build: No
CUDA used to build PyTorch: 10.1

OS: Ubuntu 18.04.3 LTS
GCC version: (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
CMake version: version 3.10.2

Python version: 3.7
Is CUDA available: Yes
CUDA runtime version: Could not collect
GPU models and configuration:
GPU 0: GeForce RTX 2080 Ti
GPU 1: GeForce RTX 2080 Ti
GPU 2: GeForce RTX 2080 Ti
GPU 3: GeForce RTX 2080 Ti
GPU 4: GeForce RTX 2080 Ti
GPU 5: GeForce RTX 2080 Ti
GPU 6: GeForce RTX 2080 Ti
GPU 7: GeForce RTX 2080 Ti

Nvidia driver version: 440.33.01
cuDNN version: Could not collect

Versions of relevant libraries:
[pip3] numpy==1.16.3
[pip3] torch==1.1.0
[pip3] torchvision==0.2.2.post3
[conda] blas 1.0 mkl
[conda] efficientnet-pytorch 0.4.0 pypi_0 pypi
[conda] mkl 2019.3 199
[conda] mkl-service 1.1.2 py37he904b0f_5
[conda] mkl_fft 1.0.10 py37ha843d7b_0
[conda] mkl_random 1.0.2 py37hd81dba3_0
[conda] torch 1.4.0 pypi_0 pypi
[conda] torchvision 0.5.0 pypi_0 pypi