Help in understanding how Dataloader works internally

a_d: <p>Hello all, I am having trouble understanding how the dataloader works internally, especially when we define the number of workers. I noticed a weird behavior and made a minimal code snippet replicating the issue. Here is the dataset class.</p>
<pre><code class="lang-auto">
class testClass(Dataset):
    def __init__(self):
        pass

    def __len__(self):
        return 1000

    def __getitem__(self, item):
        print("Accessing the __getitem__ method")
        return torch.rand(10)
</code></pre>
<p>I am calling the class as   and testing as follows -</p>
<pre><code class="lang-auto">
dataset = testClass()
dataloader = DataLoader(dataset, batch_size=5, num_workers=5)
for _, data in enumerate(dataloader):
    print(data.shape)
    print('--------------------------------------------------')
    break
</code></pre>
<p>when my batch_size is 5, the output is as follows -</p>
<pre><code class="lang-auto">Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
torch.Size([5, 10])
--------------------------------------------------
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
</code></pre>
<p>There are 2 things I am not able to understand. firstly, why is the <code>__getitem__</code> method being called so many times, since my batch is 5, I expect it to be called only 5 times. Secondly, as you can see the why is it being called after I have printed the dashed lines, I have already received my first batch of data and added a <code>break</code> . Nothing should be printed after the dashed lines I suppose. Also the behaviour is not always the same, it sometimes prints or not prints after the dashed lines.</p>
<p>This is also the issue when I specify <code>num_workers=1</code> and <code>batch_size=1</code>. The output for this combination is as follows-</p>
<pre><code class="lang-auto">Accessing the __getitem__ method
Accessing the __getitem__ method
torch.Size([1, 10])
--------------------------------------------------
</code></pre>
<p>Again it is being called twice.<br>
The only time I notice the expected behavior is when I do not pass the <code>num_workers</code> argument. For example for <code>batch_size=1</code> and not passing <code>num_workers</code> the output is as follows -</p>
<pre><code class="lang-auto">Accessing the __getitem__ method
torch.Size([1, 10])
--------------------------------------------------
</code></pre>
<p>and for <code>batch_size=5</code> the output is as follows(Again not passing <code>num_workers</code>  argument when calling Dataloader).</p>
<pre><code class="lang-auto">Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
Accessing the __getitem__ method
torch.Size([5, 10])
--------------------------------------------------
</code></pre>
<p>When dealing with my original issue, I realized that I was loading gibberish data all along, that is wrong labels corresponding to input data…<br>
What am I doing wrong here?</p>

albanD: <p>Hi,</p>
<blockquote>
<p>firstly, why is the <code>__getitem__</code> method being called so many times, since my batch is 5, I expect it to be called only 5 times</p>
</blockquote>
<p>This is because the worker processes are loading batch in advance to be able to provide them to the training as quickly as possible when it asks for a new one.</p>
<blockquote>
<p>Secondly, as you can see the why is it being called after I have printed the dashed lines, I have already received my first batch of data and added a <code>break</code> . Nothing should be printed after the dashed lines I suppose.</p>
</blockquote>
<p>This most likely happens because the worker that load the data work asynchronously and load the data before the main process tells them to stop doing stuff.</p>
<blockquote>
<p>Again it is being called twice.</p>
</blockquote>
<p>Again because it preloads 2 batchs in advance. You can check the doc for dataloader on master and in particular the <code>prefetch_factor</code> argument that allows you to control how many batch are loaded in advance: <a href="https://pytorch.org/docs/master/data.html#torch.utils.data.DataLoader" class="inline-onebox">torch.utils.data — PyTorch master documentation</a></p>

a_d: <p>Thanks a lot for your reply <a class="mention" href="/u/alband">@albanD</a>.</p>
<aside class="quote no-group" data-username="a_d" data-post="1" data-topic="99573">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/e19b73/48.png" class="avatar"> a_d:</div>
<blockquote>
<pre><code class="lang-auto">Accessing the __getitem__ method
Accessing the __getitem__ method
torch.Size([1, 10])
--------------------------------------------------
</code></pre>
</blockquote>
</aside>
<p>Here, I noticed that It loaded the data of the first and label of the second.  This is actually the reason I started digging into the Dataloader…<br>
I am still not sure why was that happening.</p>

albanD: <aside class="quote no-group" data-username="a_d" data-post="3" data-topic="99573">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/e19b73/48.png" class="avatar"> a_d:</div>
<blockquote>
<p>Here, I noticed that It loaded the data of the first and label of the second.</p>
</blockquote>
</aside>
<p>I’m not sure what you mean by that? There is not concept of data and label here. The Dataset should return the right pair of them for a given item.</p>

a_d: <p>Thanks a lot for replying, I will look into it.</p>