Issue 1:
🐛 Bug

I am trying to have a generator load objects in the background, as in the following example. It hangs when trying to call torch.zeros in split_loader_creator, but if I remove the seemingly irrelevant line torch.zeros(152*4, 168*4).float() near the end, it seemingly can make progress. It also seems fine if I change 152*4 and 168*4 to much smaller numbers. This is on PyTorch 1.5.1, and I do not encounter the issue on 1.4.0. Am I somehow doing this multiprocessing incorrectly?

To Reproduce

Run the following code:

import torch
import multiprocessing
import atexit

def split_loader_creator():
for i in range(20):
yield torch.zeros(10, 170, 70)

def background_generator_helper(gen_creator):
def _bg_gen(gen_creator, conn):
gen = gen_creator()
while conn.recv():
try:
conn.send(next(gen))
except StopIteration:
conn.send(StopIteration)
return
except Exception:
import traceback
traceback.print_exc()

parent_conn, child_conn = multiprocessing.Pipe()
p = multiprocessing.Process(target=_bg_gen, args=(gen_creator, child_conn))
p.start()
atexit.register(p.terminate)

parent_conn.send(True)
while True:
parent_conn.send(True)
x = parent_conn.recv()
if x is StopIteration:
return
else:
yield x

def background_generator(gen_creator): # get several processes in the background fetching batches in parallel to keep up with gpu
generator = background_generator_helper(gen_creator)
while True:
batch = next(generator)
if batch is StopIteration:
return
yield batch

torch.zeros(152*4, 168*4).float()

data_loader = background_generator(split_loader_creator)
for i, batch in enumerate(data_loader):
print(i)

Expected behavior

I expect this script to print the first few integers, but it just hangs at torch.zeros in split_loader_creator.

Environment

PyTorch Version (e.g., 1.0): 1.5.1

OS (e.g., Linux): Linux

How you installed PyTorch (conda, pip, source): conda

Build command you used (if compiling from source): n/a

Python version: 3.6.10

CUDA/cuDNN version: 10.2 (though I think I got the issue on 10.1 as well)

GPU models and configuration: Quadro RTX 6000 though the same bug happened for me on other GPUs as well

Any other relevant information: I suspect it's related to PyTorch, since as mentioned before I didn't encounter the issue on PyTorch 1.4.0. The issue also occurs on 1.5.0.

Additional context

Someone else also confirmed they could reproduce the issue: https://discuss.pytorch.org/t/pytorch-hangs-in-thread-after-large-torch-zeros-call-in-main-process/88484/3

cc @ezyang @gchanan @zou3519

Issue 2:
🐛 Bug

The following program never terminates.

import torch
import torch.multiprocessing as mp

def foo():
x = torch.ones((2, 50, 10))
return torch.einsum('ijl,ikl->ijk', x, x)

if __name__ == '__main__':
foo()
p = mp.Process(target=foo)
p.start()
p.join()

The behavior persists if one changes the einsum inside foo with an equivalent operation (e.g., bmm(y, y.transpose(1,2)), or (y.unsqueeze(2) * y.unsqueeze(1)).sum(3).

It doesn't reproduce, however, if one doesn't call foo inside the main block.

Environment

PyTorch version: 1.0.1.post2

Is debug build: No

CUDA used to build PyTorch: 9.0.176

OS: Arch Linux

GCC version: (GCC) 8.2.1 20181127

CMake version: version 3.13.4

Python version: 3.7

Is CUDA available: No

CUDA runtime version: No CUDA

GPU models and configuration: No CUDA

Nvidia driver version: No CUDA

cuDNN version: No CUDA

Additional context

Perhaps related to #2245.