Choosing weights for Weighted Cross Entropy Loss

Entropy: <p>I am trying to train a U-Net for image segmentation. My dataset consists of 80x80 pixel images (so 6400 pixels per image), and each image can be segmented into 3 parts: primary background, secondary background, and a third class that can be any one of 9 separately defined classes. So I have 11 classes in total.</p>
<p>My goal is to use image segmentation to determine what that third class is for each image. I decided to sample roughly 1000 images from the dataset and this is what I observed: on average ~5587 pixels belonged to the primary background class, ~802 pixels belonged to the secondary background class, and ~11 pixels belonged to the third class (which again, can be any one of the nine special classes).</p>
<p>Obviously, this is a very unbalanced dataset, so I was thinking about using weighted cross-entropy loss as my criterion. How should I go about choosing these weights? Any input will be appreciated.</p>

KFrank: <p>Hi Entropy!</p>
<aside class="quote no-group" data-username="Entropy" data-post="1" data-topic="102240" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/e/5f9b8f/48.png" class="avatar"> Entropy:</div>
<blockquote>
<p>on average ~5587 pixels belonged to the primary background class, ~802 pixels belonged to the secondary background class, and ~11 pixels belonged to the third class (which again, can be any one of the nine special classes).</p>
<p>Obviously, this is a very unbalanced dataset, so I was thinking about using weighted cross-entropy loss as my criterion. How should I go about choosing these weights?</p>
</blockquote>
</aside>
<p>The conventional wisdom suggests that you should weight each<br>
class inversely to the number of samples (pixels) in that class.</p>
<p>Now your “third class” is actually nine “special classes.”  Let’s<br>
assume (since you don’t give the numbers) that those “special<br>
classes” all appear about equally often in your data.</p>
<p>Then I would weight your eleven actual classes as follows:</p>
<p>primary background: 6400 / 5587<br>
secondary background:  6400 / 802<br>
third class - 1:  9 * 6400 / 11<br>
third class - 2:  9 * 6400 / 11<br>
…<br>
third class - 9:  9 * 6400 / 11</p>
<p>(Of course, if your nine “special classes” don’t occur about equally<br>
often, you would not want to weight them all the same, but rather in<br>
inverse proportion to how frequently they occur.)</p>
<p>Lastly, the <em>exact</em> values of these weights shouldn’t really matter.  You<br>
just need them to be approximately right to smooth out the significant<br>
imbalance among classes in your dataset.</p>
<p>Best.</p>
<p>K. Frank</p>

Entropy: <p>Thank you for replying! They do occur equally often so that’s nice! Since those values don’t need to be exact and I need to be somewhere in that ballpark, is there an estimate as to how much I can be off and still have a decent loss function.</p>

moreshud: <p>Hi <a class="mention" href="/u/entropy">@Entropy</a>,</p>
<p>Kind share more light in getting average class count for each class. Did you summed up the 1’s in each pixel and takes it average for a particular class?</p>