Problem about torch.nn.BCELoss for soft labels

James_Lee: <p>Hi there. I found this statement (<a href="https://pytorch.org/docs/1.6.0/generated/torch.nn.BCELoss.html?highlight=bce#:~:text=Note%20that%20the,0%20and%201" class="inline-onebox" rel="noopener nofollow ugc">BCELoss — PyTorch 1.6.0 documentation</a>. ) in  PyTorch doc. Does this mean that I could use the <code>torch.nn.</code> <code>BCELoss</code>  for soft labels without transforming the soft labels into hard labels with a threshold when calling BCE loss?  Thanks.</p>

KFrank: <p>Hi James!</p>
<aside class="quote no-group" data-username="James_Lee" data-post="1" data-topic="133358" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/james_lee/48/29487_2.png" class="avatar"> James_Lee:</div>
<blockquote>
<p>Does this mean that I could use the <code>torch.nn.</code> <code>BCELoss</code>  for soft labels</p>
</blockquote>
</aside>
<p>Yes.  <code>BCELoss</code> accepts a <code>target</code> (“labels”) consisting of probabilities<br>
that run over 0.0 to 1.0 (inclusive) (so, “soft labels”).  They do not have<br>
to be exactly 0.0 or 1.0 (“hard labels”), although they can be.</p>
<p>As a aside, for reasons of numerical stability, you should use<br>
<code>BCEWithLogitsLoss</code> in preference to <code>BCELoss</code>.</p>
<p>Best.</p>
<p>K. Frank</p>

James_Lee: <p>Thanks for your prompt reply! When calling BCE loss, is there a way to change the soft labels into hard ones by customizing the nn.BCELoss with a specific threshold?</p>

KFrank: <p>Hi James!</p>
<aside class="quote no-group" data-username="James_Lee" data-post="3" data-topic="133358" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/james_lee/48/29487_2.png" class="avatar"> James_Lee:</div>
<blockquote>
<p>is there a way to change the soft labels into hard ones by customizing the nn.BCELoss with a specific threshold?</p>
</blockquote>
</aside>
<p>Neither <code>BCEWithLogitsLoss</code> (which you should use), nor <code>BCELoss</code><br>
(which you should not use) has a built-in thresholding feature.</p>
<p>But it’s easy enough to threshold <code>target</code> (the labels) before you pass<br>
it in:</p>
<pre data-code-wrap="python"><code class="lang-python">my_threshold = 0.65
thresholded_target = (target &gt; my_threshold).float()
loss = torch.nn.BCEWithLogitsLoss() (input, thresholded_target)
</code></pre>
<p>However, it might not be a good idea to threshold your <code>target</code> – your<br>
ground-truth labels – if they are “soft” (probabilistic).  In the soft case<br>
their unthresholded values represent information that you might not<br>
want to discard.</p>
<p>Let’s say that your threshold is 0.65.  If your <code>target</code> is 0.649, you<br>
will reward greatly a prediction of 0.0, penalize some a prediction<br>
of 0.65, and penalize greatly a prediction of 1.0.  But if, instead,<br>
your target is 0.651, you will penalize greatly a prediction of 0.0,<br>
penalize mildly a prediction of 0.65, and reward greatly a prediction<br>
of 1.0.</p>
<p>Would that really make sense for your use case?</p>
<p>Best.</p>
<p>K. Frank</p>

James_Lee: <p>Got it.Thank you very much.</p>