Iterating through a Dataloader object

toganico: <p>Hello! I saw the following codes today in a LSTM/MNIST example:</p>
<pre><code class="lang-auto">train_loader = Data.DataLoader(dataset=train_data, batch_size=BATCH_SIZE, shuffle=True)

for epoch in range(EPOCH):
    for step, (x, y) in enumerate(train_loader):   # gives batch data
         ......
</code></pre>
<p>For these codes, how does the second for loop work (for plus a tuple)? What’s the structure of the DataLoader object here? The for loop also normalizes the MNIST data according to the tutorial? (reference: <a href="https://morvanzhou.github.io/tutorials/machine-learning/torch/4-02-RNN-classification/" rel="nofollow noopener">https://morvanzhou.github.io/tutorials/machine-learning/torch/4-02-RNN-classification/</a>) You may just provide a link here, thanks.</p>

tom: <p>The normalization is usually done in the <a href="https://pytorch.org/docs/master/torchvision/datasets.html#mnist" rel="nofollow noopener">dataset</a> via the <code>transform</code> argument.</p>
<p>The dataloader provides a Python iterator returning tuples and the enumerate will add the step. You can experience this manually (in Python3):</p>
<pre><code class="lang-python">it = iter(train_loader)
first = next(it)
second = next(it)
</code></pre>
<p>will give you the first two things from the train_loader that the for loop would get.<br>
Python Iterators are a concept many people ask and write about in various forums, I don’t know a canonical reference to link to, but searching for “python iterators” you’ll find many things on it.</p>
<p>Finally the <code>step, (x, y)</code> works due to “tuple unpacking”, again a general Python thing.</p>
<p>Best regards</p>
<p>Thomas</p>

kelam_goutam: <p>The snippet basically tells that, for every epoch the <em>train_loader</em> is invoked which returns <em>x</em> and <em>y</em> say input and its corresponding label. The second <em>for loop</em> is iterating over the entire dataset and the <em>enumerate</em> is simply assigning the <em>i</em> th value to the variable <em>step</em> which corresponds to the <em>i</em> th training example that is loaded. When the <em>train_loader</em> is invoked the training examples that are being loaded are normalized by the <em>DataLoader</em> via the <em>transform</em> argument which allows a combination of data preprocessing steps like resize, normalizing to [0,1], etc.</p>

saurav.prakash16: <p>In cifar10 tutorial at <a href="https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html" rel="noopener nofollow ugc">https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html</a>, the following approach is used for obtaining the data and labels:</p>
<pre><code class="lang-auto">trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,
                                          shuffle=True, num_workers=2)

def imshow(img):
    img = img / 2 + 0.5     # unnormalize
    npimg = img.numpy()
    plt.imshow(np.transpose(npimg, (1, 2, 0)))


# get some random training images
dataiter = iter(trainloader)

images, labels = dataiter.next()
</code></pre>
<p>If we keep calling dataiter.next() repeatedly, will it give error at the point it reaches the end of the dataset, or will the iterator reset to start when the end of the dataset is reached? Also, how will shuffle work in this approach?</p>

InnovArul: <p>Dataloader <code>iter()</code> behaves like any other iterator in python. It raises <code>StopIteration</code> exception when the end is reached.</p>
<p>In pytorch tutorial, after <a href="https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html#loading-and-normalizing-cifar10">loading the data</a>, <code>iter()</code> followed by <code>next()</code> is used just to get some images and display them in the notebook. In the <a href="https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html#train-the-network">training loop</a>, a <code>for</code> loop ()  is used to loop over the training data.</p>