Repository: pytorch
Issue: Multiprocessing model evaluation gets stuck
Labels: module: multiprocessing,triaged

## 🐛 Bug

I try to parallelize model evaluation by using multiple processes. The application gets stuck when the sub processes get to the model prediction part.

## To Reproduce

This minimum example reproduces the problem:

```
"""
bug.py
"""
from PIL import Image
import torch
import torch.multiprocessing as mp
import torchvision
from torchvision.transforms import functional as F
from torch.utils.data import Dataset
from torchvision.models.detection import fasterrcnn_resnet50_fpn
from torchvision.models.detection.faster_rcnn import FastRCNNPredictor

import logger
logging=logger.get_logger(name='example')


class MyDataset(Dataset):

    def __init__(self):
        self.imgs = [
            '/datasets/1.png',
            '/datasets/2.png',
            '/datasets/3.png',
            '/datasets/4.png',
            '/datasets/5.png',
            '/datasets/6.png',
            '/datasets/7.png',
            '/datasets/8.png',
            ]

    def __getitem__(self, idx):
        img_path = self.imgs[idx]
        img = Image.open(img_path).convert("RGB")
        img = F.to_tensor(img)        
        return img

    def __len__(self):
        return len(self.imgs)


def get_model(num_classes):
    logging.debug("Set up FasterRcnn Resnet50 Fpn model")
    model = fasterrcnn_resnet50_fpn(pretrained=True)
    in_features = model.roi_heads.box_predictor.cls_score.in_features
    model.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)
    return model


def predict(q, model, image_tensor):
    """
    Generate model's prediction (bounding boxes, scores and labels) for an 
    image tensor
    """
    model.eval()
    with torch.no_grad():
        logging.debug("before prediction")
        prediction = model([image_tensor])
        logging.debug("after prediction")
    boxes = prediction[0]["boxes"]
    scores = prediction[0]["scores"]
    labels = prediction[0]["labels"]
    q.put(scores)
    

def main():
    
    num_classes = 5

    logging.debug(torch.__version__)
    logging.debug(torchvision.__version__)

    logging.info("Set up dataset")
    dataset = MyDataset()
    
    logging.info("Set up model and training algorithm")
    model = get_model(num_classes)

    logging.info("Predict one sample")
    q = mp.Queue()
    predict(q, model, dataset[0])
    logging.debug(q.get())

    logging.info("Predict 8 samples (parallel)")
    num_processes = 8
    model.share_memory()
    processes = []
    for i in range(num_processes):
        p = mp.Process(target=predict, args=(q, model, dataset[i]))
        p.start()
        processes.append(p)
    for p in processes:
        p.join()
    
    while not q.empty():
        logging.debug(q.get())
    

if __name__ == '__main__':
    main()

```

The program gets stuck before the prediction, and I need to interrupt it manually (ctrl+c, twice). The output log:
```
2021-06-07 08:55:35,394 - example - DEBUG - 1.8.1+cu102
2021-06-07 08:55:35,394 - example - DEBUG - 0.9.1+cu102
2021-06-07 08:55:35,394 - example - INFO - Set up dataset
2021-06-07 08:55:35,394 - example - INFO - Set up model and training algorithm
2021-06-07 08:55:35,395 - example - DEBUG - Set up FasterRcnn Resnet50 Fpn model
2021-06-07 08:55:36,177 - example - INFO - Predict one sample
2021-06-07 08:55:36,240 - example - DEBUG - before prediction
2021-06-07 08:55:38,786 - example - DEBUG - after prediction
2021-06-07 08:55:38,789 - example - DEBUG - tensor([0.4587, 0.4534, 0.4255, 0.4232, 0.4104, 0.4075, 0.4057, 0.3953, 0.3940,
        0.3932, 0.3931, 0.3848, 0.3832, 0.3800, 0.3768, 0.3752, 0.3738, 0.3724,
        0.3687, 0.3685, 0.3683, 0.3673, 0.3644, 0.3630, 0.3620, 0.3610, 0.3577,
        0.3546, 0.3520, 0.3498, 0.3489, 0.3480, 0.3464, 0.3461, 0.3436, 0.3432,
        0.3423, 0.3419, 0.3398, 0.3387, 0.3385, 0.3378, 0.3373, 0.3349, 0.3347,
        0.3336, 0.3334, 0.3325, 0.3313, 0.3281, 0.3275, 0.3272, 0.3257, 0.3253,
        0.3244, 0.3242, 0.3233, 0.3220, 0.3211, 0.3195, 0.3175, 0.3174, 0.3171,
        0.3163, 0.3157, 0.3156, 0.3146, 0.3144, 0.3137, 0.3132, 0.3130, 0.3127,
        0.3123, 0.3122, 0.3119, 0.3115, 0.3103, 0.3100, 0.3097, 0.3094, 0.3092,
        0.3091, 0.3088, 0.3087, 0.3062, 0.3058, 0.3053, 0.3052, 0.3046, 0.3041,
        0.3033, 0.3029, 0.3022, 0.3020, 0.3010, 0.3005, 0.3004, 0.2994, 0.2990,
        0.2990])
2021-06-07 08:55:38,790 - example - INFO - Predict 8 samples (parallel)
2021-06-07 08:55:38,994 - example - DEBUG - before prediction
2021-06-07 08:55:39,054 - example - DEBUG - before prediction
2021-06-07 08:55:39,120 - example - DEBUG - before prediction
2021-06-07 08:55:39,199 - example - DEBUG - before prediction
2021-06-07 08:55:39,254 - example - DEBUG - before prediction
2021-06-07 08:55:39,309 - example - DEBUG - before prediction
2021-06-07 08:55:39,365 - example - DEBUG - before prediction
2021-06-07 08:55:39,419 - example - DEBUG - before prediction
^CTraceback (most recent call last):
  File "bug.py", line 99, in <module>
    main()
  File "bug.py", line 92, in main
    p.join()
  File "/opt/conda/lib/python3.7/multiprocessing/process.py", line 140, in join
    res = self._popen.wait(timeout)
  File "/opt/conda/lib/python3.7/multiprocessing/popen_fork.py", line 48, in wait
    return self.poll(os.WNOHANG if timeout == 0.0 else 0)
  File "/opt/conda/lib/python3.7/multiprocessing/popen_fork.py", line 28, in poll
    pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt
^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/multiprocessing/popen_fork.py", line 28, in poll
    pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt

```

## Expected behavior

I would expect the 8 processes to put the predicted score in the queue and complete. The main process would then get the prediction from the queue and log them.

## Environment

```
python -m torch.utils.collect_env
Collecting environment information...
PyTorch version: 1.8.1+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Debian GNU/Linux 10 (buster) (x86_64)
GCC version: (Debian 8.3.0-6) 8.3.0
Clang version: Could not collect
CMake version: version 3.13.4

Python version: 3.7 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration:
GPU 0: Tesla K80
GPU 1: Tesla K80

Nvidia driver version: 450.80.02
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.19.5
[pip3] torch==1.8.1
[pip3] torchaudio==0.8.1
[pip3] torchvision==0.9.1
[conda] numpy                     1.19.5           py37haa41c4c_1    conda-forge
[conda] torch                     1.8.1                    pypi_0    pypi
[conda] torchaudio                0.8.1                    pypi_0    pypi
[conda] torchvision               0.9.1                    pypi_0    pypi
```

## Additional context

<!-- Add any other context about the problem here. -->