How does WeightedRandomSampler work?

username: mderakhshani
cooked: <p>Hi,<br>
I have wrote below code for understanding how <code>WeightedRandomSampler</code> works.</p>
<pre><code>import torch
from torch.utils.data.sampler import Sampler
from torch.utils.data import TensorDataset as dset

inputs = torch.randn(100,1,10)
target = torch.floor(3*torch.rand(100))
trainData = dset(inputs, target)

num_sample = 3
weight = [0.2, 0.3, 0.7]
sampler = torch.utils.data.sampler.WeightedRandomSampler(weight, batch_size)
trainLoader = torch.utils.data.DataLoader(trainData, num_sample , shuffle=False, sampler=sampler)

for i, (inp, tar) in enumerate(trainLoader):
    print(inp.size())
</code></pre>
<p>I have got 100 instances in my fake dataset. When I run above code, only 10 of them sampled from dataset and also the number of iteration is 4 for my run! Could you please help me how does it work?</p>
<p>Best</p>

username: smth
cooked: <p>it;'s a very small function, have a look at how it works: <a href="https://github.com/pytorch/pytorch/blob/master/torch/utils/data/sampler.py#L73-L90">https://github.com/pytorch/pytorch/blob/master/torch/utils/data/sampler.py#L73-L90</a></p>

username: mderakhshani
cooked: <p>Hey <a class="mention" href="/u/smth">@smth</a>. Thanks for your response. I have understood <code>WeightedRandomSampler</code>, But I did not understand what is the logic of the <strong>enumerate(trainLoader)</strong>. How many times does <strong>for</strong> loop execute?</p>

username: quazi
cooked: <p>Been looking at the code in DataLoader and WeightedRandomSampler, I can’t see how it takes class labels into account. From the code comment “weights (sequence)   : a sequence of weights, not necessary summing up to one”. Not very helpful really for someone who’s trying to learn torch. It looks like weights is a list of weights per data point in the data set we are drawing from, NOT a weight per class (which I initially, maybe carelessly, assumed). And if that’s the case, you’d have to write code that computes this weight per data point and somehow “attach” that weight to the data point, e.g. a text data point becomes (sentence, label, weight for sampling) UNLESS some order is implied on the data set before we can use WeightedRandomSampler.</p>
<p>While the code is fairly straight forward, the semantics around WeightedRandomSampler are not clear at all.</p>

username: lbugnon
cooked: <p>Totally agreed, i’d found it today and it is counter-intuitive</p>

username: cuixing158_1
cooked: <p>your understanding may be correct,pls reference here ,perhps it helps.</p><aside class="quote" data-post="2" data-topic="2703">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/smth/48/13_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/balanced-sampling-between-classes-with-torchvision-dataloader/2703/2">Balanced Sampling between classes with torchvision DataLoader</a> <a class="badge-category__wrapper " href="/c/vision/5"><span data-category-id="5" data-drop-close="true" class="badge-category " title="Topics related to either pytorch/vision or vision research related topics"><span class="badge-category__name">vision</span></span></a>
  </div>
  <blockquote>
    you should look at <a href="https://discuss.pytorch.org/t/how-to-prevent-overfitting/1902/6">How to Prevent Overfitting</a>
  </blockquote>
</aside>


username: Albert_Christianto
cooked: <p>hi, i want to train a network using 3 dataset. i have created my custom dataset class, then i need to load 3 datasets with different ratio in 1 batch. does anyone have sample code for this application?</p>

username: Alexander_Soare
cooked: <p>I agree that the docs are not specific enough here. I just spent the last 30 minutes figuring out that the weights are meant to be specified at a data point level, not a class level. In fact, I spent some time trying to figure out how <code>WeightedRandomSampler</code> knows what my class labels are.</p>

username: Pcamellon
cooked: <p>Hi! Any update on this?</p>

username: ptrblck
cooked: <p>The documentation claims:</p>
<blockquote>
<p>Samples elements from <code>[0,..,len(weights)-1]</code> with given probabilities (weights).</p>
</blockquote>
<p>I agree that this might not be a sufficiently detailed explanations for some users and I’m sure PRs are welcome to improve the docs <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>