How to give more importance to one class

username: alx
cooked: <p>I have five classes to predict and would like one of my models to prioritize class 3, how could I code that?</p>

username: SU801T
cooked: <p>Is this what you mean?</p>
<aside class="quote quote-modified" data-post="4" data-topic="80514">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/weightedrandomsampler-imbalanced-classes-multiclass-classification/80514/4">WeightedRandomSampler - Imbalanced classes Multiclass classification</a> 
  </div>
  <blockquote>
    WeightedRandomSampler is used to provide weights to each sample, which is used during the sampling process of selecting the data samples for each batch. 
The higher the weight assigned to a particular index, the more likely this data sample will be used in a batch. 
To created batches of balanced classes, you would therefore assign a higher weight to samples from the minority class, and a lower weight to samples from the majority class. 

Yes, the reduction is independent from the sampling strat…
  </blockquote>
</aside>


username: alx
cooked: <p>I think so, how does it work exactly? I just read about it.</p>
<p>Since I have five classes and I want to prioritize class 3 would it be something like:</p>
<pre><code class="lang-auto">num_sample = 5
weight = [0.5, 0.5, 0.5, 0.9, 0.5]
sampler = torch.utils.data.sampler.WeightedRandomSampler(weight, batch_size)
trainLoader = torch.utils.data.DataLoader(trainData, num_sample , shuffle=False, sampler=sampler)
</code></pre>
<p>Also found this comment “It looks like weights is a list of weights per data point in the data set we are drawing from, NOT a weight per class”</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="alx" data-post="3" data-topic="83588">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alx/48/17306_2.png" class="avatar"> alx:</div>
<blockquote>
<p>Also found this comment “It looks like weights is a list of weights per data point in the data set we are drawing from, NOT a weight per class”</p>
</blockquote>
</aside>
<p>Yes, <code>WeightedRandomSampler</code> expects a weight value for each sample in the dataset.<br>
<a href="https://discuss.pytorch.org/t/how-to-handle-imbalanced-classes/11264/2">Here</a> is a complete example.</p>
<p>Alternatively, you could also use a class weight and pass it to your criterion, if it fits your use case.<br>
E.g. <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.CrossEntropyLoss"><code>nn.CrossEntropyLoss</code></a> accepts a <code>weight</code> tensor, which assigns a specific weight to each class.</p>

username: Shanto_Islam
cooked: <p>How about for <code>nn.BCEWithLogitsLoss</code> or for <code>nn.BCELoss</code>? How can we pass the weights? I mean I know I will be passing <code>weight=weight</code> but the weight tensor should contain only one element and how could we do that?</p>

username: ptrblck
cooked: <p>I would generally recommend to use <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.BCEWithLogitsLoss"><code>nn.BCEWithLogitsLoss</code></a> instead of <code>nn.BCELoss</code> for increased numerical stability and the availability of the <code>pos_weight</code> argument.</p>
<p>Note that both loss funcitons are used for a binary of multi-label classification/segmentation, so I’m not sure, if it’ll fit your use case.</p>
<p>That being said, you could specify the <code>pos_weight</code> argument as described in the docs to balance a class imbalance in your dataset. You could also use the <code>weight</code> argument to rescale each sample in the batch. In the latter case, I would personally use the functional API, as you would probably recreate the criterion in each iteration based on the current batch.</p>

username: Shanto_Islam
cooked: <p>Okay, going back to the multi-class issue. Say I’ve 3 classes each having following the number of labels</p>
<p>Class 1 : 3711<br>
Class 2 : 7107<br>
Class 3 : 2942</p>
<p>I was wondering if there’s any other way to assign class weights rather than just inversion like this</p>
<p><code>weight = 1.0 / torch.Tensor([3711,7107,2942], dtype = torch.float)</code><br>
<code>criterion = nn.CrossEntropyLoss(weight=weight)</code></p>
<p>And also do we have to follow any intuitive rule like all the weights have to equal to something?</p>

username: alx
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="4" data-topic="83588">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>Alternatively, you could also use a class weight and pass it to your criterion, if it fits your use case.<br>
E.g. <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.CrossEntropyLoss" rel="noopener nofollow ugc"> <code>nn.CrossEntropyLoss</code> </a> accepts a <code>weight</code> tensor, which assigns a specific weight to each class.</p>
</blockquote>
</aside>
<p>Thanks Patrick, what does the tensor consists of? 5 values between zero and one?</p>

username: chetan_patil
cooked: <p>The weights need not necessarily be values between 0 and 1.<br>
Follow this link <a href="https://scikit-learn.org/stable/modules/generated/sklearn.utils.class_weight.compute_class_weight.html" rel="nofollow noopener">https://scikit-learn.org/stable/modules/generated/sklearn.utils.class_weight.compute_class_weight.html</a></p>