Categorical Cross Entropy for sparsely labelled data

username: aszmul
cooked: <p>Hello Everyone,</p>
<p>I am facing a problem of semantic segmentation of 2D data. I would like to apply 5 classes.<br>
The dataset is sparsely labelled, in each image just a small portion of it is labelled, usually with just one class. It is common that in the same image more regions should have that label, and other parts of the image should have one of the other labels as well (which is not done).</p>
<p>I have tried naively applying CrossEntropy loss with 5 classes but without success. It seems that I should probably use weighting for every class depending on the amount of data that is annotated. Simple weight per class would not do the job here. Is there a way that I could do that with the loss functions that are already implemented in PyTorch, or should I rather look at implementing my own loss function?</p>

username: KFrank
cooked: <p>Hi Adam!</p>
<aside class="quote no-group quote-modified" data-username="aszmul" data-post="1" data-topic="59500" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/ba8739/48.png" class="avatar"> aszmul:</div>
<blockquote>
<p>The dataset is sparsely labelled, …</p>
<p>I have tried naively applying CrossEntropy loss with 5 classes but without success. It seems that I should probably use weighting for every class depending on the amount of data that is annotated. Simple weight per class would not do the job here.</p>
</blockquote>
</aside>
<p>Why wouldn’t using class weights (as supported, for example, by<br>
<code>CrossEntropyLoss</code>) work for your use case?</p>
<p>What specifically do you understand your problem to be?</p>
<p>If your dataset is so sparsely annotated that you don’t really have<br>
enough annotated data to train with, you’re kind of stuck.</p>
<p>If your problem is just that your annotated data is unbalanced with<br>
respect to how often your various classes are, in fact, labelled,<br>
then compensating for the imbalance by using class weight is a<br>
legitimate approach.</p>
<p>Have you tried it, and, if so, what failed?</p>
<p>Good luck.</p>
<p>K. Frank</p>

username: aszmul
cooked: <p>Hi K. Frank,</p>
<p>Thanks for getting back to me.<br>
I think the problem which I have is twofold and class inbance which (as far as I understand) can be addressed by weighting the classes is just one of them, and probably the smaller one. I have tried adding such weights and my network dies (gives me all zeros outputs) after a few epochs. Just like without adding weights. I was thinking if there is a simple way providing a binary mask, where I have labels, which would be included in the loss function?</p>

username: KFrank
cooked: <p>Hello Adam!</p>
<aside class="quote no-group" data-username="aszmul" data-post="3" data-topic="59500" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/ba8739/48.png" class="avatar"> aszmul:</div>
<blockquote>
<p>I was thinking if there is a simple way providing a binary mask, where I have labels, which would be included in the loss function?</p>
</blockquote>
</aside>
<p>Let me speak as if you have images that are labelled on a per-pixel<br>
basis, and that have only a small fraction of their pixels actually<br>
labelled.  Each pixel is either unlabelled, or is labelled with one<br>
of five classes.</p>
<p>(My comments apply more generally to unlabelled / labelled data,<br>
but it’s easier for me to speak concretely.)</p>
<p>I would treat this a a <em>six</em>-class classification problem, but use<br>
class weights that assign a weight of zero to the “unlabelled”<br>
class.  (The five weights for the five real classes could either be<br>
all one, or could be non-trivial weights that compensate for an<br>
unbalanced dataset.)</p>
<p>Now (because they get multiplied by zero) your unlabelled pixels<br>
won’t contribute to your loss function – your loss function and<br>
training won’t care what your network predicted for those<br>
unlabelled pixels.</p>
<p>Does this make sense (and would it apply to / work for your use<br>
case)?</p>
<p>Good luck.</p>
<p>K. Frank</p>

username: aszmul
cooked: <p>Thank you K. Frank!</p>
<p>When I set zero as a weight for the ‘0’ (backgroud) label, my network stopped dying after a few epochs and gives at least some results.<br>
This is a huge step forward. Now I should probably carefully adjust the weights for the remaining classes, based on the frequency of their appearance.</p>
<p>Thanks so much for clarifying that.</p>