Loss is low, but all the prediction near 0.5 for binary classification

satinder147: <p>I am having a binary classification task.<br>
My network</p>
<pre><code class="lang-auto">def set_parameter_requires_grad(model, feature_extracting):
    if feature_extracting:
        for param in model.parameters():
            param.requires_grad = False

model = models.resnet18(pretrained=True)

set_parameter_requires_grad(model, True)

model.fc = nn.Sequential(nn.Linear(in_features=512, out_features=1, bias=True))
</code></pre>
<p>loss vs epochs curve(orange is validation loss)<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/0/a/0a3c316bbc25ade51d37e0f1f5a861975300f51a.png" alt="download" data-base62-sha1="1sxJJNfOhbqVJmepyb0Qw3sqihA" width="372" height="248"></p>
<p>But when I get the predictions, all the outputs are near 0.5 (which is like the model was not trained at all)<br>
Since the loss function used was nn.BCEWithLogitsLoss(), I added an explicit sigmoid while generating the predictions.<br>
The way I generate the predictions</p>
<pre><code class="lang-auto">transformed = data_transforms(pil_img)
label = model(transformed.float().to(device).unsqueeze(0))
label = sigmoid(label.cpu().detach().numpy())
</code></pre>
<p>Can you please help me.</p>

KFrank: <p>Hi Satinder!</p>
<aside class="quote no-group" data-username="satinder147" data-post="1" data-topic="106070" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/satinder147/48/20263_2.png" class="avatar"> satinder147:</div>
<blockquote>
<p>But when I get the predictions, all the outputs are near 0.5 (which is like the model was not trained at all)<br>
Since the loss function used was nn.BCEWithLogitsLoss(), I added an explicit sigmoid while generating the predictions.</p>
</blockquote>
</aside>
<p>As an aside, if you train with <code>BCEWithLogitsLoss</code>, you will be training<br>
your model to produce <em>logits</em>.  A model making “neutral” predictions<br>
will produce logits near <code>0.0</code> (that correspond to probabilities near <code>0.5</code>).</p>
<p>You haven’t given us much concrete information about your loss function<br>
and predictions, but, making simple assumptions, your results don’t add<br>
up.</p>
<p>The graph you posted shows your loss function falling below <code>0.1</code>.</p>
<p>A neutral prediction (a probability of <code>0.5</code> with <code>BCELoss</code> or a logit of<br>
<code>0.0</code> with <code>BCEWithLogitsLoss</code>) will give a loss of <code>log (2)</code>, regardless<br>
of the target value.</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; tone = torch.ones (1)
&gt;&gt;&gt; torch.nn.functional.binary_cross_entropy (0.5 * tone, tone)
tensor(0.6931)
&gt;&gt;&gt; torch.nn.functional.binary_cross_entropy_with_logits (0.0 * tone, tone)
tensor(0.6931)
</code></pre>
<p>(To get a loss of <code>0.1</code> you have to be pretty far away from neutral.)</p>
<p>So either the predictions underlying the training and validations losses<br>
shown in the chart you posted aren’t neutral, or your loss calculation is<br>
something different than the simple:</p>
<pre data-code-wrap="python"><code class="lang-python">loss = torch.nn.BCEWithLogitsLoss() (predictions, targets)
</code></pre>
<p>Best.</p>
<p>K. Frank</p>