In multiclass+multilabel segmentation, is single channel target must be one-hot encoding?

username: Alex_Luya
cooked: <p>Hello,I have<br>
outputs like [B,C,H,W]<br>
and<br>
targets like [B,H,W] ,where each pixel may be the one of [0,1,2,3,4] for background+4 classes</p>
<p>and BCEWithLogitsLoss(outputs,targets) gives me</p>
<p><code>ValueError: Target size (torch.Size([32, 224, 416])) must be the same as input size (torch.Size([32, 4, 224, 416]))</code></p>
<p>Is one-hot encoding target the only way,or any loss support this kind of targets directly?</p>

username: KFrank
cooked: <p>Hello Alex!</p>
<aside class="quote no-group" data-username="Alex_Luya" data-post="1" data-topic="65497" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alex_luya/48/6004_2.png" class="avatar"> Alex_Luya:</div>
<blockquote>
<p>Hello,I have<br>
outputs like [B,C,H,W]<br>
and<br>
targets like [B,H,W] ,where each pixel may be the one of [0,1,2,3,4] for background+4 classes</p>
</blockquote>
</aside>
<p>You say in your title “multiclass+multilabel,” but you say in your post<br>
itself “where each pixel may be the one of [0,1,2,3,4].”</p>
<p>To clarify the terminology:</p>
<p>Yes, I would call this “multiclass” because you have five different<br>
classes (one background plus four non-background).  (If you had<br>
only two classes (“yes” vs. “no” or “0” vs. “1”), I would call it binary,<br>
in contrast to multiclass.)</p>
<p>However, “multilabel” conventionally means that each “thing”<br>
(sample, pixel, whatever) can be assigned to more than one<br>
class at a time – that is, it can be “labelled” with multiple labels.</p>
<p>I read your phrase “where each pixel may be the one of [0,1,2,3,4]"<br>
as meaning that each pixel will be (labelled with) <em>exactly</em> one of<br>
[0, 1, 2, 3, 4].  In this case I would call your problem a conventional<br>
(that is, single-label) multiclass classification problem.  For this you<br>
would want to use <code>CrossEntropy</code> (<code>cross_entropy_loss()</code>), for<br>
which your “outputs” (predictions) and targets will be the shapes<br>
you give, the <code>C</code> dimension (where <code>C</code> = 5) of your outputs will be the<br>
raw scores (<em>logits</em>) predicted for the pixel in question to be in each<br>
of your five classes, and targets will be categorical class labels<br>
(integers in [0, 1, 2, 3, 4]), and not one-hot encoded.</p>
<p>On the other hand, if you really mean “multilabel,” then I would read<br>
your phrase as meaning that each pixel can be labelled with any<br>
number of the labels [0, 1, 2, 3, 4], including none of them.</p>
<p>In this case you would indeed want to use</p>
<aside class="quote no-group">
<blockquote>
<p>BCEWithLogitsLoss(outputs,targets)</p>
</blockquote>
</aside>
<p>but now your targets would be the same shape as your outputs, that<br>
is, [B, C, H, W].  But your <code>C</code> dimension would not be one-hot encoded<br>
(one value of 1, the other four 0), because more than one label, or<br>
none, could be active at the same time.  (So your <code>C</code> dimension might<br>
have value <code>(0, 1, 0, 1, 0)</code> meaning that the pixel in question is<br>
both in class “1” and “3” at the same time.)</p>
<aside class="quote no-group">
<blockquote>
<p>Is one-hot encoding target the only way,or any loss support this kind of targets directly?</p>
</blockquote>
</aside>
<p>If I’m right that you don’t mean “multilabel”, then no, one-hot<br>
encoding is not the only way, and yes, the <code>CrossEntropy</code> loss<br>
supports your kind of targets directly.</p>
<p>That is:</p>
<p>I think that you <em>don’t</em> mean “multilabel,” that the shapes of your<br>
outputs and targets are correct, that your targets are categorical<br>
class labels (as is consistent with the shape you give of [B, H, W])<br>
(and hence not one-hot encoded), but that you will want to use<br>
<code>CrossEntropy</code> (which really means “categorical cross-entropy<br>
with logits”) as your loss function.</p>
<p>Best.</p>
<p>K. Frank</p>