Training with replacement

username: Simone256
cooked: <p>Hello everyone!<br>
I’m trying to train a deep neural network sampling my images with replacement. In other words, if I have for example a batch size of 10, and a data-set made of (let’s say) 1000 images, I would like to create 100 batches where each sample is randomly sampled from the whole data-set.</p>
<p>What I’m doing is the following:</p>
<pre><code class="lang-auto">random_train_loader = torch.utils.data.DataLoader(
        torchvision.datasets.MNIST('./datasets/', train=True, transform=torchvision.transforms.ToTensor()),
        batch_size=batch_size,
        sampler=torch.utils.data.sampler.RandomSampler(torchvision.datasets.MNIST, replacement=True)
    )
for input_images, labels in iter(random_train_loader):
    # I do stuff
</code></pre>
<p>Sadly this build a</p>
<p><code>object of type 'type' has no len()</code></p>
<p>error.</p>
<p>I’ve probably misunderstood the documentation of RandomSampler, any suggestion?</p>
<p>Many thanks!</p>

username: vfdev-5
cooked: <p><a class="mention" href="/u/simone256">@Simone256</a> try like the following:</p>
<pre><code class="lang-python">dataset = torchvision.datasets.MNIST('./datasets/', train=True, transform=torchvision.transforms.ToTensor())
random_train_loader = torch.utils.data.DataLoader(
        dataset,
        batch_size=batch_size,
        sampler=torch.utils.data.sampler.RandomSampler(dataset, replacement=True)
    )
for input_images, labels in iter(random_train_loader):
</code></pre>
<p>You should pass the instance of the dataset to the Random Sampler and not the class.</p>

username: Simone256
cooked: <p>Ok sorry! Now it works, but I suppose using the full</p>
<p><code>torchvision.datasets.MNIST('./datasets/', train=True, transform=torchvision.transforms.ToTensor())</code></p>
<p>in the RandomSampler would have worked too, I’m sorry that was really a stupid error.</p>
<p>Just to ask one more thing, if I write something like this</p>
<pre><code class="lang-auto">data = torch.randn(10, 1, 28, 28)
target = torch.arange(10)

dataset = torch.utils.data.TensorDataset(data, target)
</code></pre>
<p><code>dataset</code> “the same thing” as the dataset you define in your example? Where with “the same thing” I mean “another instance of the same class” (I suppose). So I can treat them equivalently?</p>

username: vfdev-5
cooked: <p>Yes, a dataset instance should be an instance of a class derived from <code>torch.utils.data.Dataset</code>. So, <code>torchvision.datasets.MNIST</code> and <code>torch.utils.data.TensorDataset</code> are both derived from <code>torch.utils.data.Dataset</code>.</p>
<p>Please, see this docs for more details:<br>
</p><aside class="onebox allowlistedgeneric">
  <header class="source">
      <a href="https://pytorch.org/docs/stable/data.html?highlight=dataset#torch.utils.data.Dataset" target="_blank" rel="noopener nofollow ugc">pytorch.org</a>
  </header>
  <article class="onebox-body">
    <img src="" class="thumbnail" width="" height="">

<h3><a href="https://pytorch.org/docs/stable/data.html?highlight=dataset#torch.utils.data.Dataset" target="_blank" rel="noopener nofollow ugc">torch.utils.data — PyTorch 1.7.0 documentation</a></h3>



  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>