How to set weights for 38 number of classes individually in Cross Entropy Loss, via pytorch

username: Fikrat
cooked: <p>Hi to Everyone<br>
I have 38 number of multiple classes, which is in turn 37 colourful pixels and 1 black pixel. Then, my target is load more weights for colourful pixels,but less weight for black pixel.Then, concern is how to do this?</p>
<p>Thansk in advance</p>

username: KFrank
cooked: <p>Hello Fikrat!</p>
<aside class="quote no-group" data-username="Fikrat" data-post="1" data-topic="74127" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/fikrat/48/21597_2.png" class="avatar"> Fikrat:</div>
<blockquote>
<p>my target is load more weights for colourful pixels,but less weight for black pixel.</p>
</blockquote>
</aside>
<p>I answered this exact question in my reply to your previous post:</p>
<aside class="quote quote-modified" data-post="2" data-topic="74021">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/how-to-implement-weighted-cross-entropy-loss-for-lane-marking-segmentation/74021/2">How to implement weighted Cross Entropy Loss for lane marking segmentation</a> 
  </div>
  <blockquote>
    Hi Fikrat! 

If your problem is one a binary segmentation, presumably 
black = background, color = foreground, then you should use 
<a href="https://pytorch.org/docs/stable/nn.html#torch.nn.BCEWithLogitsLoss" rel="noopener nofollow ugc">BCEWithLogitsLoss</a> and the pos_weight argument that you 
pass to its constructor. 
If it’s multi-class segmentation, e.g., black = background, some 
colors or patterns = lane 1, some other colors or patterns = lane 2, 
etc., then you should use <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.CrossEntropyLoss" rel="noopener nofollow ugc">CrossEntropyLoss</a> and the weight 
argument passed to its constructor. 
Best. 
K. Frank
  </blockquote>
</aside>

<p>If this doesn’t work for you or your have further questions, it would<br>
probably make the most sense to move the discussion back to that<br>
earlier thread.</p>
<p>Best.</p>
<p>K. Frank</p>

username: Fikrat
cooked: <p>Dear Frank,</p>
<p>Thanks for your kind request and valuable time shared with me. But I want to know the principle of how to set wieght for each individual color pixel in which I am really confused.<br>
In order to be much more clear, I provided constructor as I did:</p>
<p>class SegmentationLosses(object):<br>
def <strong>init</strong>(self, weight=None, size_average=True, batch_average=True, ignore_index=255, cuda=False):<br>
self.ignore_index = ignore_index<br>
self.weight = weight<br>
self.size_average = size_average<br>
self.batch_average = batch_average<br>
self.cuda = cuda</p>

username: KFrank
cooked: <p>Hi Fikrat!</p>
<aside class="quote no-group" data-username="Fikrat" data-post="3" data-topic="74127" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/fikrat/48/21597_2.png" class="avatar"> Fikrat:</div>
<blockquote>
<p>But I want to know the principle of how to set wieght for each individual color pixel</p>
</blockquote>
</aside>
<p>I’m not entirely sure what you are asking.</p>
<p>But let’s say you have 38 classes represented by (labelled by) colored<br>
pixels – class 0 = black, class 1 = color-1, class 2 = color-2, and so on.</p>
<p>You would count – or estimate the count of – each kind of pixel in your<br>
dataset, for example:<br>
<code>count[0]  = 1000, count[1] = 10, ..., count[37] = 10</code></p>
<p>This would be an <em>unbalanced</em> dataset, that is, their are 100 times as<br>
many black pixels (examples of class 0) as pixels of any other single<br>
color (examples of any other single class).</p>
<p>You would then set <code>weight = 1.0 / count</code> and pass it as the <code>weight</code><br>
argument to <code>torch.nn.CrossEntropyLoss</code> when you instantiate it.<br>
That is, you reduce the weight (by a factor of 1 / 100) of class 0, for<br>
which you have many examples, relative to the other 37 classes, for<br>
which, individually, you have significantly fewer examples.</p>
<p>(<code>weight</code>, in this example, is a tensor of shape <code>[38]</code>, where 38 is the<br>
number of classes you have.)</p>
<aside class="quote no-group">
<blockquote>
<p>class SegmentationLosses(object):<br>
def <strong>init</strong>(self, weight=None, size_average=True, batch_average=True, ignore_index=255, cuda=False):<br>
self.ignore_index = ignore_index<br>
self.weight = weight<br>
self.size_average = size_average<br>
self.batch_average = batch_average<br>
self.cuda = cuda</p>
</blockquote>
</aside>
<p>This looks like you are defining your own loss-function class,<br>
<code>SegmentationLosses</code>.</p>
<p>There is no need to do this.  If I understand your use case you should<br>
simply use pytorch’s built-in <code>torch.nn.CrossEntropyLoss</code>, as it<br>
already supports reweighting an unbalanced dataset by using its<br>
constructor’s <code>weight</code> argument, as outlined above.</p>
<p>Good luck!</p>
<p>K. Frank</p>

username: Fikrat
cooked: <p>Dear Mr. Frank</p>
<p>Thanks for your kind support and valuable time</p>