Dataloader shuffles at every epoch

akgulozlem: <p>Hello everyone,</p>
<p>We have some problems with the shuffling property of the dataloader. It seems that dataloader shuffles the whole data and forms new batches at the beginning of every epoch.</p>
<p>However, we are performing semi supervised training and we have to make sure that at every epoch the same images are sent to the model.<br>
For example let’s say our batches are as the following:<br>
Batch 1 consists of images [a,b,c,…]<br>
Batch 2 consists of images [ f,g,h,…]</p>
<p>Batch n consists of images [x,y,z,…]</p>
<p>So after one epoch we need the exact same batches at the other epochs as well. Because at every epoch we are using the images a,f,… and x from the example  above. The model needs other images, that’s why we cannot eliminate them, however there is also a decent amount of necessity to obtain these specific first images.</p>
<p>Our training method necessitates that we should shuffle the data in the very beginning, form batches from that shuffled data, and use the same exact batches in the rest of the training.</p>
<p>In the beginning using a dataloader wouldn’t cause any problems but as I’ve mentioned before we’ve seen that new batches are formed at each epoch. We have also tried using the SubsetRandomSampler, but couldn’t accomplish anything.</p>

Programmer-RD-AI: <p>try passing shuffle=False * as a parameter <code>DataLoader(dataset,shuffle=False)</code><br>
in the DataLoader</p>
<p>Hope this helps</p>

ZimoNitrome: <p>If you would still like one initial shuffling you could maybe shuffle your dataset at the start somehow or make a custom sampler:</p>
<pre><code class="lang-python">import torch
from torch.utils.data.sampler import Sampler
from typing import Iterator, Sized

class ConstantRandomSampler(Sampler[int]):
    def __init__(self, data_source: Sized) -&gt; None:
        self.num_samples = len(self.data_source)
        generator = torch.Generator()

        self.shuffled_list = torch.randperm(self.num_samples, generator=generator).tolist()

    def __iter__(self) -&gt; Iterator[int]:
        yield from self.shuffled_list

    def __len__(self) -&gt; int:
        return self.num_samples

</code></pre>

akgulozlem: <p>Thank you so much for your responses :))<br>
However, I am not familiar enough with the concept of samplers, can you propose a way to shuffle the dataset in the beginning?<br>
Maybe then we can disable the shuffle option of the dataloader and obtain what we want.</p>

Suy: <p>I have the same problem in my project. I want to shuffle the dataset in the beginning of the training, just once. Although I use SubsetRandomSampler, the dataset is shuffled every epoch. In my research through the internet, I found that every iteration of complete dataset is considered as one epoch by the DataLoader. Do you have a solution for this issue? Thank you <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=10" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

my3bikaht: <p>Use dataloader with shuffle=False and shuffle your data manually before adding into dataset</p>

akgulozlem: <p>Hello everyone,<br>
Thank you so much for all of your responses and yes we did it. Apparently scikit-library has a tool named shuffle <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=10" title=":slight_smile:" class="emoji" alt=":slight_smile:"> after importing that we’ve created a new variable for the shuffled version of the data. Giving that to the dataloader and disabling shuffling solved our problem.</p>

Ren_Pang: <p>pass the provided sampler as argument of <code>sampler=</code> to dataloader <code>__init__</code> method.</p>
<p>See docs at<br>
<a href="https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader</a></p>
<p>Sampler docs:<br>
<a href="https://pytorch.org/docs/stable/data.html#torch.utils.data.Sampler" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/data.html#torch.utils.data.Sampler</a></p>
<p>The sampler solution shall be more efficient than using scikit library.</p>
<h2>
<a name="another-simple-solution-1" class="anchor" href="#another-simple-solution-1"></a>Another simple solution</h2>
<p>If you want to get a shuffled Dataset, you can use <code>Subset</code> instead of using scikit.</p>
<pre><code class="lang-auto">ShuffledDataset = torch.utils.data.Subset(YourDataset, torch.randperm(len(YourDataset)))
</code></pre>