Dataloader loading the same indexes in a batch

shadow: <p>I’m building a dataloader, I wrote it as follows:</p>
<p>train_sampler = RandomSampler(train_dataset,num_samples=100)<br>
valid_sampler = RandomSampler(valid_dataset,num_samples=50)<br>
train_data_loader = DataLoader(train_dataset,<br>
shuffle = False,<br>
batch_size = self.train_config.train_batch_size,<br>
sampler = train_sampler,<br>
)<br>
valid_data_loader = DataLoader(valid_dataset,<br>
shuffle = False,<br>
batch_size = self.train_config.valid_batch_size,<br>
sampler = valid_sampler,<br>
)</p>
<p>The problem is that the dataloader is getting a batch with the same index, for example i want to get a batch of 32 elements, all the indexes are the same index, i double checked with the <strong>getitem</strong> and iterated with next(iter(trainsampler)) the indexes change but with the loader it’s the same index, so basically iterating over the same point for each epoch.</p>
<p>Could you help me to sort this out?</p>
<p>Thanks</p>

ptrblck: <p>I cannot reproduce the issue and get the expected results:</p>
<pre data-code-wrap="python"><code class="lang-python">class MyDataset(Dataset):
    def __init__(self, length=10):
        self.data = torch.arange(length).unsqueeze(1)
        
    def __getitem__(self, index):
        print("calling __getitem__ with index {}".format(index))
        x = self.data[index]
        return x
    
    def __len__(self):
        return len(self.data)


train_dataset = MyDataset()
for x in train_dataset:
    print(x)
# calling __getitem__ with index 0
# tensor([0])
# calling __getitem__ with index 1
# tensor([1])
# calling __getitem__ with index 2
# tensor([2])
# calling __getitem__ with index 3
# tensor([3])
# calling __getitem__ with index 4
# tensor([4])
# calling __getitem__ with index 5
# tensor([5])
# calling __getitem__ with index 6
# tensor([6])
# calling __getitem__ with index 7
# tensor([7])
# calling __getitem__ with index 8
# tensor([8])
# calling __getitem__ with index 9
# tensor([9])
# calling __getitem__ with index 10


train_sampler = RandomSampler(train_dataset,num_samples=100)

train_data_loader = DataLoader(train_dataset, shuffle=False, batch_size=2, sampler=train_sampler)
for x in train_data_loader:
    print(x)
# calling __getitem__ with index 5
# calling __getitem__ with index 2
# tensor([[5],
#         [2]])
# calling __getitem__ with index 0
# calling __getitem__ with index 6
# tensor([[0],
#         [6]])
# calling __getitem__ with index 1
# calling __getitem__ with index 8
# tensor([[1],
#         [8]])
# calling __getitem__ with index 4
# calling __getitem__ with index 3
# tensor([[4],
#         [3]])
# calling __getitem__ with index 7
# calling __getitem__ with index 9
# tensor([[7],
#         [9]])
# calling __getitem__ with index 1
# calling __getitem__ with index 4
# tensor([[1],
#         [4]])
# calling __getitem__ with index 9
# calling __getitem__ with index 2
# tensor([[9],
#         [2]])
# calling __getitem__ with index 5
# calling __getitem__ with index 6
# tensor([[5],
#         [6]])
# ...
</code></pre>
<p>You can see that the <code>DataLoader</code> shuffles the indices and returns the desired 100 samples.</p>
<aside class="quote no-group" data-username="shadow" data-post="1" data-topic="168836">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/5fc32e/48.png" class="avatar"> shadow:</div>
<blockquote>
<p>iterated with next(iter(trainsampler))</p>
</blockquote>
</aside>
<p>If you are calling this code directly as posted above, note that it will recreate a new <code>iterator</code> in each call, so you might want to split the <code>iter</code> and <code>next</code> calls.</p>

shadow: <p>I tried to reproduce the same results but still the same issue.</p>
<p>train_sampler = RandomSampler(TrainData,num_samples=10)</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/a/5a58820571bfa046162d5216142f30f751f4f059.png" data-download-href="https://discuss.pytorch.org/uploads/default/5a58820571bfa046162d5216142f30f751f4f059" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/a/5a58820571bfa046162d5216142f30f751f4f059_2_690x148.png" alt="image" data-base62-sha1="cTeBaZ7PwRGVsNcAbaI3bF3iMVb" width="690" height="148" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/a/5a58820571bfa046162d5216142f30f751f4f059_2_690x148.png, https://discuss.pytorch.org/uploads/default/original/3X/5/a/5a58820571bfa046162d5216142f30f751f4f059.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/5/a/5a58820571bfa046162d5216142f30f751f4f059.png 2x" data-dominant-color="252627"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">787×169 12 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>i tried to test to see if the data is loaded properly and here is what i got</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/8/a882041074329840e32b17f17417c549de170fea.png" data-download-href="https://discuss.pytorch.org/uploads/default/a882041074329840e32b17f17417c549de170fea" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/a/8/a882041074329840e32b17f17417c549de170fea_2_556x500.png" alt="image" data-base62-sha1="o2GLUvAwxDiXBRZIr5arofXQrnY" width="556" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/a/8/a882041074329840e32b17f17417c549de170fea_2_556x500.png, https://discuss.pytorch.org/uploads/default/original/3X/a/8/a882041074329840e32b17f17417c549de170fea.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/a/8/a882041074329840e32b17f17417c549de170fea.png 2x" data-dominant-color="262626"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">710×638 63.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>and tested on the loaded batch but the results differ. Noting that if I compute x[‘data’][1][11] i will get the same tensor as indx 0</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/1/b1a1b7c8cc5d4a1810e099d1eee546d92fbf5f7e.png" data-download-href="https://discuss.pytorch.org/uploads/default/b1a1b7c8cc5d4a1810e099d1eee546d92fbf5f7e" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/b/1/b1a1b7c8cc5d4a1810e099d1eee546d92fbf5f7e.png" alt="image" data-base62-sha1="ploZsvROA2ImiTtVFjPFdGSblng" width="555" height="500" data-dominant-color="282828"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">645×581 60.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>I don’t know where the issue might be coming from, so feel free to post a minimal, executable code snippet reproducing it. Your previous code snippet did not show the issue as mentioned before.</p>

shadow: <p>I realized from where the problem is coming.<br>
It was from the getitem, I wrote it as follows at the beginning.</p>
<p>def <strong>getitem</strong>(self,item):</p>
<pre><code>  .....
  data = dict()
  data['features'] = ...
  data['target'] = ....
  return data
</code></pre>
<p>So i was returning a dict not (x,y) which confused the dataloader. After modifying the output format it worked as a charm. Many thanks for your time and help in this regard.</p>