How dataloader shuffled with enumerate()?

GB_K: <p>Hello.</p>
<p>I am coding for training loop with dataloader which is flagged shuffle=True.</p>
<p>to my knowledge, people usually coding epoch and iteration for dataloader as follows:</p>
<pre><code class="lang-python">for epoch in epochs:
   for iter, (input, target) in enumerate(dataloader):
     """Do Training"""
</code></pre>
<p>With this commonsense, I got question.<br>
how <strong>enumerate</strong> function <em>shuffle</em> the data order in the <strong>dataloader</strong>?<br>
what is in the dataloader called by the enumerate function?</p>
<p>Please let me solve this question.</p>
<p>Thank you in advance.</p>

RaLo4: <p>If you specify your dataloader with <code>shuffle=True</code> and a specific batch size the dataloader will shuffle your data and put the shuffled data into batches of your specified size.<br>
If you then call <code>enumerate()</code> on the dataloader you will be able to loop over the shuffled batches of your predefined size and get a counter as well.</p>

RaLo4: <aside class="quote no-group" data-username="GB_K" data-post="1" data-topic="90264">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/gb_k/48/24999_2.png" class="avatar"> GB_K:</div>
<blockquote>
<p>how <strong>enumerate</strong> function <em>shuffle</em> the data order in the <strong>dataloader</strong> ?</p>
</blockquote>
</aside>
<p><code>enumerate()</code> does not shuffle your data. It just loops over the already shuffled batches in your dataloader</p>

GB_K: <p>Thank <a class="mention" href="/u/ralo4">@RaLo4</a>  so much for your reply.</p>
<p>That means dataloader, with <code>shuffle=True</code> and given ‘batch_size’, calculate the number of batches and the loader will shuffle if it reaches to the end by enumerate.</p>
<p>Am I correct?</p>

RaLo4: <p>I’m not sure I understand what you mean by</p>
<aside class="quote no-group" data-username="GB_K" data-post="4" data-topic="90264">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/gb_k/48/24999_2.png" class="avatar"> GB_K:</div>
<blockquote>
<p>and the loader will shuffle if it reaches to the end by enumerate.</p>
</blockquote>
</aside>
<p>but the data is already shuffled at the time you call <code>enumerate()</code> on the dataloader.</p>
<p>The dataloader first shuffles the data and puts it into batches. When you call <code>enumerate(dataloader)</code> you iterate over these shuffled batches.<br>
The batches themselves get not shuffled by calling <code>enumerate()</code> but the data contained in these batches is shuffled by the dataloader (and also gets reshuffled every epoch according to <a href="https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader" rel="noopener nofollow ugc">the pytorch doc</a>).</p>

GB_K: <p>Oh now I understand from your teaching.</p>
<p>enumerate call dataloader.</p>
<p>since the dataloader is called, it shuffles before enumerate works start.</p>
<p>right?</p>

RaLo4: <p>yes. basically.</p>
<aside class="quote no-group" data-username="GB_K" data-post="6" data-topic="90264">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/gb_k/48/24999_2.png" class="avatar"> GB_K:</div>
<blockquote>
<p>right?</p>
</blockquote>
</aside>
<p>right!</p>

GB_K: <p>Thank you!!!<br>
my question has been solved.</p>
<p>Another question is that how did you refer part of my reply in your reply??</p>

RaLo4: <aside class="quote no-group" data-username="GB_K" data-post="8" data-topic="90264">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/gb_k/48/24999_2.png" class="avatar"> GB_K:</div>
<blockquote>
<p>Another question is that how did you refer part of my reply in your reply??</p>
</blockquote>
</aside>
<p>mark the text you want to reply to and then a <strong>"Quote</strong> button pops up <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"></p>

GB_K: <p>Thanks ~ XD<br>
Have a good day</p>