Loss for each label in multi-label classification

Anshul_Tomar: <p>The problem is that of text classification where each document can be identified with one or more labels (104). So given a prediction <code>\hat{y}</code> of size (batch_size, 104) and a multi-one hot vector <code>y</code> of size  (batch_size, 104) which consists of value 1 for the labels it contains and values 0 for the labels it does not contain, I would want a vector <code>loss_labels</code> of size (104, 1) where the value at position <code>l</code> in this vector is given by<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/e/f/ef73a74b7b12a0a53be2be6ca772ebb84a2610e9.png" alt="image" data-base62-sha1="yahTUW58QEmXbDe7IjNofrQBeGd" width="350" height="51"><br>
Here y_i^l is the true <code>l</code>th label of <code>i</code>th sample and \hat{y}_i^l is the same thing for the predicted output.</p>

KFrank: <p>Hi Anshul!</p>
<aside class="quote no-group" data-username="Anshul_Tomar" data-post="1" data-topic="115375" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/anshul_tomar/48/12946_2.png" class="avatar"> Anshul_Tomar:</div>
<blockquote>
<p>So given a prediction <code>\hat{y}</code> of size (batch_size, 104) and a multi-one hot vector <code>y</code> of size  (batch_size, 104) which consists of value 1 for the labels it contains and values 0 for the labels it does not contain, I would want a vector <code>loss_labels</code> of size (104, 1)</p>
</blockquote>
</aside>
<p>You should use <code>binary_cross_entropy_with_logits()</code> with no<br>
“reduction.”  Thus:</p>
<pre data-code-wrap="python"><code class="lang-python">torch.nn.functional.binary_cross_entropy_with_logits (prediction, loss_labels, reduction = 'none')
</code></pre>
<p>This will return a tensor of loss values of shape <code>[batch_size, 104]</code><br>
(neither summed, nor averaged, nor otherwise “reduced” over any of its<br>
dimensions) that you can then sum over the batch dimension.  (If you<br>
really want the trailing singleton dimension, use <code>unsqueeze()</code>.)</p>
<p>Note that <code>binary_cross_entropy_with_logits()</code> takes a <code>prediction</code><br>
consisting of raw-score <em>logits,</em> rather than the probabilities that would<br>
be passed to <code>binary_cross_entropy()</code>.  This is preferred for reasons<br>
of numerical stability.</p>
<p>(You could also use <code>BCEWithLogitsLoss</code> if you prefer the class over the<br>
function version.)</p>
<p>Best.</p>
<p>K. Frank</p>

moreshud: <p>Hi <a class="mention" href="/u/kfrank">@KFrank</a>,</p>
<p>Your clarification will be appreciated on a similar topic which you had answered in the past.</p><aside class="quote quote-modified" data-post="8" data-topic="104158">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/f17d59/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/same-output-when-using-multilabel-bce/104158/8">Same output when using Multilabel BCE</a> 
  </div>
  <blockquote>
    Hi <a class="mention" href="/u/kfrank">@KFrank</a> 
Firstly, I would like to appreciate your explanations and contributions. However, I would be glad for the affirmation of gained knowledge from the thread and likewise some clarifications. 

The difference between single label and multi-label multi-class segmentation is the presence overlapping - a case where particular corresponding pixel value(s) for each target classes is/are annotated
The go loss function for single label multi-class is CrossEntropyLoss while for multi-label multi…
  </blockquote>
</aside>