Repository: pytorch
Issue: ConnectionResetError when using dataLoader with pin_memory=True
Labels: needs reproduction,module: dataloader,triaged

## 🐛 Bug

ConnectionResetError  when 'pin_memory=True' for dataLoader.  
A simple test case is provided.
( I found the issue in a more complex training code with larger dataset. I made a simple code to pinpoint the problem) 

## To Reproduce
```

from torch.utils.data import Dataset, DataLoader

class Set_ex(Dataset):
	def __init__(self):
		self.lst = [1, 2, 3, 4, 5, 6, 7, 8]

	def __getitem__(self, index):
		return self.lst[index]

	def __len__(self):
		return len(self.lst)

if __name__ == "__main__":
	## ex_call1 loader
	ds_ex = Set_ex()
	dsLoader = DataLoader(ds_ex, 3, True, num_workers=20,  pin_memory=True)
	iterator = iter(dsLoader)
	dtIn =next(iterator)

	print(dtIn)
```
output + error infor: 
> tensor([5, 8, 1])
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/home/jun/miniconda3/envs/py37/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/home/jun/miniconda3/envs/py37/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/jun/miniconda3/envs/py37/lib/python3.7/site-packages/torch/utils/data/_utils/pin_memory.py", line 21, in _pin_memory_loop
    r = in_queue.get(timeout=MP_STATUS_CHECK_INTERVAL)
  File "/home/jun/miniconda3/envs/py37/lib/python3.7/multiprocessing/queues.py", line 113, in get
    return _ForkingPickler.loads(res)
  File "/home/jun/miniconda3/envs/py37/lib/python3.7/site-packages/torch/multiprocessing/reductions.py", line 276, in rebuild_storage_fd
    fd = df.detach()
  File "/home/jun/miniconda3/envs/py37/lib/python3.7/multiprocessing/resource_sharer.py", line 57, in detach
    with _resource_sharer.get_connection(self._id) as conn:
  File "/home/jun/miniconda3/envs/py37/lib/python3.7/multiprocessing/resource_sharer.py", line 87, in get_connection
    c = Client(address, authkey=process.current_process().authkey)
  File "/home/jun/miniconda3/envs/py37/lib/python3.7/multiprocessing/connection.py", line 498, in Client
    answer_challenge(c, authkey)
  File "/home/jun/miniconda3/envs/py37/lib/python3.7/multiprocessing/connection.py", line 741, in answer_challenge
    message = connection.recv_bytes(256)         # reject large message
  File "/home/jun/miniconda3/envs/py37/lib/python3.7/multiprocessing/connection.py", line 216, in recv_bytes
    buf = self._recv_bytes(maxlength)
  File "/home/jun/miniconda3/envs/py37/lib/python3.7/multiprocessing/connection.py", line 407, in _recv_bytes
    buf = self._recv(4)
  File "/home/jun/miniconda3/envs/py37/lib/python3.7/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer


<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

## Expected behavior

This simple code suppose to finish without error. 
The code can be executed fully. Yet in the end, such error will raise randomly. 

## Additional test 

- If 'pin_memory=False',  no problem 
- If change the batch_size or num_workers, can work sometimes.  batch_size=2,  workers=20 will not work. 
- Test on another Centos OS cluster node, no such problem.  Same pytorch install via conda, same CUDA, same python3.7. 
- Run same code several times, sometimes it worked, sometimes failed. 

pin_memory seems to cause the trouble.  
Problem seems random.  

## Environment
PyTorch version: 1.1.0
Is debug build: No
CUDA used to build PyTorch: 9.0.176

OS: Ubuntu 16.04.6 LTS
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609
CMake version: version 3.12.0

Python version: 3.7
Is CUDA available: Yes
CUDA runtime version: 9.0.176
GPU models and configuration: GPU 0: GeForce GTX 1080 Ti
Nvidia driver version: 418.56
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.2.1

Versions of relevant libraries:
[pip] numpy==1.17.2
[pip] torch==1.1.0
[pip] torchvision==0.3.0
[conda] blas                      1.0                         mkl
[conda] mkl                       2019.4                      243
[conda] mkl-service               2.3.0            py37he904b0f_0
[conda] mkl_fft                   1.0.14           py37ha843d7b_0
[conda] mkl_random                1.1.0            py37hd6b4f25_0
[conda] pytorch                   1.1.0           py3.7_cuda9.0.176_cudnn7.5.1_0    pytorch
[conda] torchvision               0.3.0           py37_cu9.0.176_1    pytorch

## Additional context

<!-- Add any other context about the problem here. -->


cc @SsnL