Repository: pytorch/pytorch

Issue:
[Pytorch] DataLoader and python random module
Even with seeding, the following script print different ouputs for `random.uniform` at the different runs. `random` module is even reseeded [here](https://github.com/pytorch/pytorch/blob/master/torch/utils/data/dataloader.py#L86).
Outputs for `torch.rand` are the same though.
```python
import torch
import random

from torch.utils.data import Dataset, DataLoader

class Data(Dataset):
    def __len__(self):
        return 10000
    def __getitem__(self, index):
        print(index, torch.rand(2, 2).sum().item(), random.uniform(0, 1))
        return 1

seed = 2018
random.seed(seed)
torch.manual_seed(seed)
loader = DataLoader(Data(), num_workers=4, shuffle=True)

for x in loader:
    print('-'*10)
    break
```
First run
```
4717 2.202341079711914 0.9952153654478976
4607 2.3166141510009766 0.6813692345925851
4194 1.9806793928146362 0.6281118075687344
2595 2.95841383934021 0.8414756141240453
4691 0.9809015393257141 0.7622458327788627
9868 2.521920680999756 0.5253262288522356
7367 2.333574056625366 0.35079311205192487
9490 3.02830171585083 0.16235006783937567
----------
6759 3.1252167224884033 0.4424384676992986
```
Next run
```
4607 2.3166141510009766 0.15198273935290807
4194 1.9806793928146362 0.36414129463658884
4691 0.9809015393257141 0.027569260048619926
4717 2.202341079711914 0.5512619092026773
7367 2.333574056625366 0.7932627754589792
9490 3.02830171585083 0.19395324967791994
9868 2.521920680999756 0.5497794735158222
2595 2.95841383934021 0.782779934368899
----------
6759 3.1252167224884033 0.7098308465010348
```
- Ubuntu 16.04
- Python 3.6
- PyTorch version: 0.4




Discussion:
This is because you used workers. Seed in `worker_init_fn`.

Relevant doc (even if it says when worker return **identical** numbers): https://pytorch.org/docs/master/notes/faq.html#my-data-loader-workers-return-identical-random-numbers



Resolving patch:
diff --git a/torch/utils/data/dataloader.py b/torch/utils/data/dataloader.py
--- a/torch/utils/data/dataloader.py
+++ b/torch/utils/data/dataloader.py
@@ -246,7 +246,7 @@ def __init__(self, loader):
 
         self.sample_iter = iter(self.batch_sampler)
 
-        base_seed = torch.LongTensor(1).random_()[0]
+        base_seed = torch.LongTensor(1).random_().item()
 
         if self.num_workers > 0:
             self.worker_init_fn = loader.worker_init_fn


Test patch:
diff --git a/test/test_utils.py b/test/test_utils.py
--- a/test/test_utils.py
+++ b/test/test_utils.py
@@ -116,6 +116,15 @@ def __len__(self):
         return 10
 
 
+class RandomDatasetMock(object):
+
+    def __getitem__(self, index):
+        return torch.tensor([torch.rand(1).item(), random.uniform(0, 1)])
+
+    def __len__(self):
+        return 1000
+
+
 class TestCheckpoint(TestCase):
 
     # Test whether checkpoint is being triggered or not. For this, we check
@@ -233,6 +242,20 @@ def setUp(self):
         self.dataset = torch.randn(5, 3, 3, 2)
         self.batch_size = 3
 
+    def test_random_seed(self):
+        def run():
+            dataloader = torch.utils.data.DataLoader(RandomDatasetMock(),
+                                                     batch_size=2,
+                                                     num_workers=4,
+                                                     shuffle=True)
+            return next(iter(dataloader))
+
+        torch.manual_seed(2018)
+        x1 = run()
+        torch.manual_seed(2018)
+        x2 = run()
+        self.assertEqual(x1, x2)
+
     def test_single_keep(self):
         dataloader = torch.utils.data.DataLoader(self.dataset,
                                                  batch_size=self.batch_size,