Repository: pytorch
Issue: Dataloader._shutdown_workers hangs
Labels: module: multiprocessing,module: dataloader,triaged

## 🐛 Bug

Dataloader._shutdown_workers hangs unexpectedly. The program has to be killed with Ctrl-C. 

```
  File "iterative_clustering.py", line 80, in calculate_features
    for batch in tqdm(dataloader, desc=f"Features (pass {i})"):
  File "/data1/mschroeder/miniconda3/envs/20-ssdc/lib/python3.7/site-packages/tqdm/std.py", line 1119, in __iter__
    for obj in iterable:
  File "/data1/mschroeder/miniconda3/envs/20-ssdc/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 571, in __next__
    self._shutdown_workers()
  File "/data1/mschroeder/miniconda3/envs/20-ssdc/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 659, in _shutdown_workers
    w.join()
  File "/data1/mschroeder/miniconda3/envs/20-ssdc/lib/python3.7/multiprocessing/process.py", line 140, in join
    res = self._popen.wait(timeout)
  File "/data1/mschroeder/miniconda3/envs/20-ssdc/lib/python3.7/multiprocessing/popen_fork.py", line 48, in wait
    return self.poll(os.WNOHANG if timeout == 0.0 else 0)
  File "/data1/mschroeder/miniconda3/envs/20-ssdc/lib/python3.7/multiprocessing/popen_fork.py", line 28, in poll
    pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt
```

## To Reproduce

```python
    features = np.zeros((n_samples, n_features))
    model.eval()
    with torch.no_grad():
        for i in range(n_passes):
            offset = 0
            for batch in tqdm(dataloader, desc=f"Features (pass {i})"):
                batch_images = torch.rot90(batch["image"].cuda(), i, (-2, -1))
                assert (
                    batch_images.size() == batch["image"].size()
                ), f"{batch_images.size()} vs. {batch['image'].size()}"
                batch_features = model.forward(batch_images)
                batch_features = torch.flatten(batch_features, 1).cpu().numpy()
                batch_size = batch_features.shape[0]
                features[offset : offset + batch_size] += batch_features
                offset += batch_size
```

The error happens in the forth (last) iteration of `for i in range(n_passes)`. `num_workers` is 8. 

## Expected behavior

I would expect the loop to terminate without problems.

## Environment

PyTorch version: 1.1.0
Is debug build: No
CUDA used to build PyTorch: 9.0.176

OS: Linux Mint 18.1 Serena
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609
CMake version: version 3.5.1

Python version: 3.7
Is CUDA available: Yes
CUDA runtime version: 8.0.61
GPU models and configuration: GPU 0: GeForce GTX TITAN X
Nvidia driver version: 384.98
cuDNN version: Could not collect

Versions of relevant libraries:
[pip] numpy==1.18.1
[pip] pytorch-lightning==0.7.2.dev0
[pip] torch==1.1.0
[pip] torchvision==0.3.0
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               9.0                  h13b8566_0  
[conda] mkl                       2020.1                      217  
[conda] mkl-service               2.3.0            py37he904b0f_0  
[conda] mkl_fft                   1.0.15           py37ha843d7b_0  
[conda] mkl_random                1.1.1            py37h0573a6f_0  
[conda] numpy                     1.18.1           py37h4f9e942_0  
[conda] numpy-base                1.18.1           py37hde5b4d6_1  
[conda] pytorch                   1.1.0           py3.7_cuda9.0.176_cudnn7.5.1_0    pytorch
[conda] pytorch-lightning         0.7.2.dev0               pypi_0    pypi
[conda] torchvision               0.3.0           py37_cu9.0.176_1    pytorch

## Additional info

When examining the code, I see no reason why this wouldn't work. A work-around may be passing a timeout to `w.join()` at the expense of leaving zombie processes around.

cc @SsnL