Understanding DataLoader Iterator

Torcione: <p>Hi,<br>
I have a doubt about how batches are selected in some situations.<br>
Let say I defined a data loader with:</p>
<pre><code class="lang-auto">train_sampler = SubsetRandomSampler(indeces)
...
train_loader = torch.utils.data.DataLoader(train_data, batch_size = bs , 
                                                   sampler = train_sampler, num_workers = nw)
</code></pre>
<p><strong>1. Dataloader Iterables</strong></p>
<p>If I well understood at this point with Dataloader I wrap an iterable around the <code>Dataset</code> to enable easy access to the samples; in particular due to <code>SubsetRandomSampler</code> every time the iterable is called, the elements of <code>train_sampler</code> are reshuffled and a new sequence of batches is defined.<br>
If I’d have used <code>SequentialSampler</code> to define the sampler, every time I’d get the same sequence:</p>
<ol>
<li>Have I understood well this?<br>
.<br>
<strong>2. Iter() and next(): sequence of batches</strong><br>
Now If I write:</li>
</ol>
<pre><code class="lang-auto">for data,label in train_loader:
    #do stuff
</code></pre>
<p>It calls the <code>__iter__()</code> method on the iterable, and then calls <code>__next__()</code> on the returned iterator until it reaches the end of the iterator. It then raises a <code>stopIteration</code> and the loop stops. (see <a href="https://stackoverflow.com/questions/62549990/what-does-next-and-iter-do-in-pytorchs-dataloader" rel="noopener nofollow ugc">source</a>)</p>
<p>In other words it should be equivalent to:</p>
<pre><code class="lang-auto">iterable = iter(train_loader)
try:
    img, lab = next(iterable)
                

#do stuff
</code></pre>
<p>where the line <code>iterable = iter(train_loader)</code> fixes the sequence of batches for the iterable and with <code>next()</code> we call them one by one.</p>
<p>If instead I write:</p>
<pre><code class="lang-auto">            try:
                img, lab = next(iter(train_loader))
            except StopIteration:
                img, lab = next(iter(train_loader))
</code></pre>
<p>By calling <code>iter()</code> each time I create a new list of batches over which iterate time by time taking always only the first element of that and in particular:</p>
<ul>
<li>
<p>If I defined a DataLoader by <code>SequentialSampler</code> the sequence of batches will be always the same and so I will pick with <code> img, lab = next(iter(train_loader))</code> always the same batch</p>
</li>
<li>
<p>If I defined a DataLoader by <code>SubsetRandomSampler</code> the sequence of batches will be reshuffled every time I call <code> img, lab = next(iter(train_loader))</code> and a new composed batch (in principle with same element chosen multiple times before every one has been selected ) will be picked each time</p>
</li>
</ul>
<ol start="2">
<li>Have I well understood the <code>next()</code>, <code>iter()</code> mechanism?</li>
</ol>
<p><strong>3. Nested iteration</strong><br>
If what said above is right I expect that I can define multiple batches’ sequence and scroll over them in parallel, for example:</p>
<pre><code class="lang-auto">iterable1 = iter(train_loader)
iterable2 = iter(train_loader)
try:
    img, lab = next(iterable1) #select the first batch of iterable1
try:
    img, lab = next(iterable2) #select the first batch of iterable2
</code></pre>
<p>I expect that with ‘iterable1’ and  ‘iterable2’  I init 2 different sequence of batches and that    <code>    img, lab = next(iterable1)</code>, <code>    img, lab = next(iterable2)</code> will follow independently the 2 iterable</p>
<ol start="3">
<li>Is it correct?</li>
</ol>