Repository: pytorch
Issue: Reference the randomness issue in DataLoader & Dataset documentation.
Labels: module: docs,module: dataloader,triaged

Old title: Random seed of numpy is reset to the same value inside DataLoader workers

## 🐛 Bug

<!-- A clear and concise description of what the bug is. -->
I’m using an IterableDataset inside a DataLoader (with multiple workers).  My IterableDataset code calls numpy.random functions. In each epoch and for each worker, I noticed the sequence of values returned by numpy.random functions is exactly the same.  This is unexpected since numpy would normally seed the random generator when it is initialized.

Example use case: the IterableDataset uses numpy.random to choose image crops - expected to be, well, *random*, not exactly the same for each image for each epoch.

Note: this happens without calling torch.manual_seed() or numpy.random.seed(), and without doing anything to make things deterministic.  Even if determinism is the default PyTorch policy, the workers should not get re-seeded each epoch.

## To Reproduce

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

```
import torch
import numpy as np
from torch.utils.data import DataLoader, Dataset

class TestIterableDataset(torch.utils.data.IterableDataset):
    def __init__(self):
        super(TestIterableDataset).__init__()

    def __iter__(self):
        worker_info = torch.utils.data.get_worker_info()
        for n in range(10):
            yield(worker_info.id, np.random.randint(1000000))
            
ds = TestIterableDataset()

for worker_id, number in DataLoader(ds, batch_size=4, num_workers=2):
    print(worker_id, number)

for worker_id, number in DataLoader(ds, batch_size=4, num_workers=2):
    print(worker_id, number)
```

This prints the same result every time it is run, and the same sequence of random numbers from each worker:
```
# tensor([0, 0, 0, 0]) tensor([ 68669, 230721, 801136, 274196])
# tensor([1, 1, 1, 1]) tensor([ 68669, 230721, 801136, 274196])
# tensor([0, 0, 0, 0]) tensor([617084, 429589, 436968, 718987])
# tensor([1, 1, 1, 1]) tensor([617084, 429589, 436968, 718987])
# tensor([0, 0]) tensor([150977,  59469])
# tensor([1, 1]) tensor([150977,  59469])
```

## Expected behavior

<!-- A clear and concise description of what you expected to happen. -->

Numpy random generator would seed itself from /dev/urandom when RandomState is initialized (different in each worker).

## Environment

PyTorch version: 1.5.0
Is debug build: No
CUDA used to build PyTorch: 10.1

OS: Ubuntu 18.04.4 LTS
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
CMake version: version 3.13.3

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 10.0.130
GPU models and configuration: GPU 0: Tesla T4
Nvidia driver version: 440.33.01
cuDNN version: Probably one of the following:
/usr/local/cuda-10.1/targets/x86_64-linux/lib/libcudnn.so.7.6.5
/usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn.so.7.6.5

Versions of relevant libraries:
[pip3] numpy==1.15.4
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               10.1.243             h6bb024c_0  
[conda] mkl                       2020.0                      166  
[conda] mkl-service               2.3.0            py36he904b0f_0  
[conda] mkl_fft                   1.0.15           py36ha843d7b_0  
[conda] mkl_random                1.1.0            py36hd6b4f25_0  
[conda] numpy                     1.18.1           py36h4f9e942_0  
[conda] numpy-base                1.18.1           py36hde5b4d6_1  
[conda] numpydoc                  0.9.2                      py_0  
[conda] pytorch                   1.5.0           py3.6_cuda10.1.243_cudnn7.6.3_0    pytorch
[conda] torchvision               0.6.0                py36_cu101    pytorch
[conda] torchviz                  0.0.1                    pypi_0    pypi




cc @SsnL @VitalyFedyunin @jlin27