What is len(dataloader) equal to?

username: Shisho_Sama
cooked: <p>Hello all.<br>
I recently noticed the <code>len(dataloader)</code> is not the same as <code>len(dataloader.dataset)</code><br>
based on Udacity Pytorch course, I tried to calculate accuracy with the following lines of codes :</p>
<pre><code class="lang-auto">accuracy=0
for imgs, labels in dataloader_test: 
    preds = model(imgs)
    values, indexes = preds.topk(k=1, dim=1) 
    result = (indexes == labels).float()
    accuracy += torch.mean(result)
print(f'acc_val = {accuracy / len(dataloader_test)}'
</code></pre>
<p>For the record, Udacity wrote this :</p>
<pre><code class="lang-auto">        test_loss = 0
        accuracy = 0
        
        # Turn off gradients for validation, saves memory and computations
        with torch.no_grad():
            for images, labels in testloader:
                log_ps = model(images)
                test_loss += criterion(log_ps, labels)
                
                ps = torch.exp(log_ps)
                top_p, top_class = ps.topk(1, dim=1)
                equals = top_class == labels.view(*top_class.shape)
                accuracy += torch.mean(equals.type(torch.FloatTensor))
                
        train_losses.append(running_loss/len(trainloader))
        test_losses.append(test_loss/len(testloader))

        print("Epoch: {}/{}.. ".format(e+1, epochs),
              "Training Loss: {:.3f}.. ".format(running_loss/len(trainloader)),
              "Test Loss: {:.3f}.. ".format(test_loss/len(testloader)),
              "Test Accuracy: {:.3f}".format(accuracy/len(testloader)))
</code></pre>
<p>and as you can see below, the validation accuracy is reported like this:<br>
<code>"Test Accuracy: {:.3f}".format(accuracy/len(testloader)))</code></p>
<p>So<code>len(testloader)</code> must match the whole <code>testset</code>. also, two lines above it in :<br>
<code>test_losses.append(test_loss/len(testloader))</code><br>
its dividing the loss by the <code>len(testloader),</code> so it should be equal to the whole test set size otherwise it doesnt make sense!</p>
<p>In my case, it only prints 313 as my <code>dataloader_test</code>.<br>
and my <code>dataloader_test</code> is defined as follows :</p>
<pre><code class="lang-auto">dataset_train = datasets.MNIST(root='MNIST', train=True, transform=transformations, download=True)
dataset_test = datasets.MNIST(root='MNIST', train=False, transform=transformations, download=True)
import torch.utils.data as data
dataloader_train = data.DataLoader(dataset_train, batch_size=32, shuffle=True, num_workers=2)
dataloader_test = data.DataLoader(dataset_test, batch_size=32,shuffle=False,num_workers=2)

print(f'test dataloader size: {len(dataloader_test)}')
</code></pre>
<p>So what am I missing here? why am I getting 313 for <code>len(dataloader_test)</code> while I shoud be getting 10K for MNIST test set?</p>

username: SimonW
cooked: <p>it equals number of batches</p>

username: Brando_Miranda
cooked: <p><a class="mention" href="/u/simonw">@SimonW</a> or the sake of sanity check, is:</p>
<pre><code class="lang-auto">len(dataloader) = dataset size / batch size
</code></pre>
<p>?<br>
I am getting these values as I print which are confusing me:</p>
<pre><code class="lang-auto">len(dataloaders['train'].dataset)=236436
len(dataloaders['train'])=59109
len(dataloaders['train'])/opts.batch_size=14777.25
</code></pre>
<p>related: <a href="https://discuss.pytorch.org/t/how-can-i-know-the-size-of-data-loader-when-i-use-torchvision-datasets-imagefolder/7800" class="inline-onebox">How can I know the size of data_loader when i use: torchvision.datasets.ImageFolder</a></p>

username: Brando_Miranda
cooked: <p>ok here is the code:</p>
<pre><code class="lang-auto">    def __len__(self) -&gt; int:
        if self._dataset_kind == _DatasetKind.Iterable:
            # NOTE [ IterableDataset and __len__ ]
            #
            # For `IterableDataset`, `__len__` could be inaccurate when one naively
            # does multi-processing data loading, since the samples will be duplicated.
            # However, no real use case should be actually using that behavior, so
            # it should count as a user error. We should generally trust user
            # code to do the proper thing (e.g., configure each replica differently
            # in `__iter__`), and give us the correct `__len__` if they choose to
            # implement it (this will still throw if the dataset does not implement
            # a `__len__`).
            #
            # To provide a further warning, we track if `__len__` was called on the
            # `DataLoader`, save the returned value in `self._len_called`, and warn
            # if the iterator ends up yielding more than this number of samples.

            # Cannot statically verify that dataset is Sized
            length = self._IterableDataset_len_called = len(self.dataset)  # type: ignore
            if self.batch_size is not None:  # IterableDataset doesn't allow custom sampler or batch_sampler
                from math import ceil
                if self.drop_last:
                    length = length // self.batch_size
                else:
                    length = ceil(length / self.batch_size)
            return length
        else:
            return len(self._index_sampler)
</code></pre>

username: ptrblck
cooked: <p>Your calculation is a bit wrong, since you are dividing the number of batches by the batch size:</p>
<pre><code class="lang-python">len(dataloaders['train'].dataset)=236436
len(dataloaders['train'])=59109
len(dataloaders['train'])/opts.batch_size=14777.25 # this is wrong
len(dataloaders['train'].dataset) / opts.batch_size = 59109 # 236436 / 4 = 59109 
</code></pre>

username: Brando_Miranda
cooked: <p>I had just figured it out…darn it. That was a subtle typo…Thanks! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: Flock1
cooked: <p>I believe it gives the batch size. If you want to get the total number of datapoints to find accuracy, you’ll have to divide by <code>trainloader.__len__()</code>. This gives the number of datapoints.</p>