WeightedRandomSampler does not oversample when calling dataloader.new()

anlytix: <p>I have a dataloader that wraps 100 samples of data.<br>
I have a WeightedRandomSampler that returns 1000 samples.<br>
However, when I create a new dataloader with this sampler, the new dataloader also has 100 samples, and not 1000 as I expect - why is this?</p>
<pre><code class="lang-auto">from random import randint
from torch.utils.data import WeightedRandomSampler

# dl1 is a dataloader with 100 samples
weights = [randint(1, 10) for _ in range(100)]
nsamples = 1000
sampler = WeightedRandomSampler(weights, nsamples)
dl2 = dl1.new(shuffle=False, sampler=sampler)
print((len(dl1.y), len(dl2.y)))
# outputs 100, 100 (I expect this to be (100, 1000))
</code></pre>

ptrblck: <p><code>torch.utils.data.DataLoader</code> doesn’t implement the <code>new()</code> method (or I’m not aware of it).<br>
Which <code>DataLoader</code> implementation are you using?</p>

anlytix: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> I am using the fastai implementation.</p>

anlytix: <p>You are right, <a class="mention" href="/u/ptrblck">@ptrblck</a> - the class I am working with is the <a href="https://github.com/fastai/fastai/blob/5986f1e5887038fb85f4d345985e65a1757a07f2/fastai/basic_data.py#L27" rel="nofollow noopener">fastai DeviceDataLoader</a> - that binds a DataLoader to a torch.device.</p>

ptrblck: <p>Thanks for the information.<br>
I’m unfortunately not familiar with this <code>DeviceDataLoader</code>, but do you know, why you would like to bind a <code>DataLoader</code> to a specific device?</p>

anlytix: <p>From the <a href="https://docs.fast.ai/basic_data.html#DeviceDataLoader" rel="noopener nofollow ugc">documentation</a>:</p>
<p>Bind a <a href="https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader" rel="noopener nofollow ugc"> <code>DataLoader</code> </a> to a <a href="https://pytorch.org/docs/stable/tensor_attributes.html#torch-device" rel="noopener nofollow ugc"> <code>torch.device</code> </a>.</p>
<blockquote>
<p>Put the batches of  <code>dl</code>  on  <code>device</code>  after applying an optional list of  <code>tfms</code> .  <code>collate_fn</code>  will replace the one of  <code>dl</code> . All dataloaders of a <a href="https://docs.fast.ai/basic_data.html#DataBunch" rel="noopener nofollow ugc"> <code>DataBunch</code> </a> are of this type.</p>
</blockquote>
<p>I noticed that <a href="https://github.com/fastai/fastai/blob/5986f1e5887038fb85f4d345985e65a1757a07f2/fastai/basic_data.py#L61#L65" rel="noopener nofollow ugc">the <code>new</code> method</a> is just overwriting the kwargs of the DeviceDataLoader object with the ones passed in with new and constructing a new DeviceDataLoader object with it.</p>
<pre><code class="lang-auto">def new(self, **kwargs):
        "Create a new copy of `self` with `kwargs` replacing current values."
        new_kwargs = {**self.dl.init_kwargs, **kwargs}
        return DeviceDataLoader(self.dl.__class__(self.dl.dataset, **new_kwargs), self.device, self.tfms,
                                self.collate_fn)

</code></pre>
<p>I don’t see why the returned object is not oversampling as expected.</p>

ptrblck: <p>I’m unfortunately not familiar enough with this wrapper, but <a class="mention" href="/u/jphoward">@jphoward</a> should be able to help out. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>