How to get the accuracy without softmax layer?

shirui-japina: <p>I’m trying to fine-tuning vgg16. Then I got the classifier of it:<br>
(I have changed the last output layer.)</p>
<pre><code class="lang-auto">(classifier): Sequential(
  (0): Linear(in_features=25088, out_features=4096, bias=True)
  (1): ReLU(inplace)
  (2): Dropout(p=0.5)
  (3): Linear(in_features=4096, out_features=4096, bias=True)
  (4): ReLU(inplace)
  (5): Dropout(p=0.5)
  (6): Linear(in_features=4096, out_features=1, bias=True)
)
</code></pre>
<p>My question is that, when use the model to predict, its output is <strong>with no range</strong>. Actually, I got</p>
<pre><code class="lang-auto">tensor([[ 0.9261],
        [ 0.6800],
        [ 0.5750],
        [ 0.5498],
        [ 0.6597],
        [ 0.7453],
        [ 0.5137],
        [ 0.6788],
        [ 1.0495],
        [ 0.7216],
        [-0.2671],
...
</code></pre>
<p>And I use <em><strong>nn.BCEWithLogitsLoss()</strong></em> ,<br>
(<em>nn.BCEWithLogitsLoss()</em> is better than <em>nn.BCELoss()</em> ? )<br>
so I can’t (shouldn’t) use <code>output = torch.sigmoid(output)</code>  and there is no softmax layer in the model. What is the correct way I get the <em>accuracy</em>? (The label is 0 or 1.)</p>

shirui-japina: <p>The way I think of is,</p>
<pre><code class="lang-auto">output = torch.sigmoid(output)
if 0 =&lt; output &lt; 0.5:
    # prediction is label 0
else:
    # prediction is label 1
</code></pre>
<p>But this makes</p>
<ul>
<li>get <strong>value of loss</strong> by <em><strong>output data</strong></em>
</li>
<li>get <strong>value of accuracy</strong> by <strong>torch.sigmoid(<em>output data</em>)</strong>
</li>
</ul>
<p>Can I do like this? Does it mean get value of loss and accuracy by <strong>different data</strong>, so it’s not in line with mathematical logic?</p>

phan_phan: <p>Yes, your own answer makes sense <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"><br>
But you can do simpler : comparing the output of the sigmoid to <code>0.5</code> is equivalent to comparing the input of the sigmoid to <code>0</code> ! (see <a href="https://en.wikipedia.org/wiki/Sigmoid_function" rel="nofollow noopener">wikipedia</a>)<br>
So, you don’t need to call <code>.sigmoid()</code> ; just see where <code>output &lt; 0</code>.</p>

shirui-japina: <p>Oh…</p>
<blockquote>
<p>comparing the output of the sigmoid to  <code>0.5</code>  is equivalent to comparing the input of the sigmoid to  <code>0</code></p>
</blockquote>
<p>Yes, thank you for your suggestion <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=12" title=":blush:" class="emoji" alt=":blush:" loading="lazy" width="20" height="20"><br>
But why</p>
<blockquote>
<ul>
<li>get  <strong>value of loss</strong>  by  <em><strong>output data</strong></em></li>
<li>get  <strong>value of accuracy</strong>  by  <strong>torch.sigmoid( <em>output data</em> )</strong></li>
</ul>
</blockquote>
<p>is the right way for getting value of loss and accuracy? <img src="https://discuss.pytorch.org/images/emoji/apple/flushed.png?v=12" title=":flushed:" class="emoji" alt=":flushed:" loading="lazy" width="20" height="20"><br>
We should get both <strong>loss</strong> and <strong>accuracy</strong> by <em><strong>the same data</strong></em>, isn’t it right?</p>

phan_phan: <p>Because calling <code>nn.BCEWithLogitsLoss</code> is the equivalent of calling first <code>nn.Sigmoid</code>, and then <code>nn.BCELoss</code>.<br>
It’s just that those two are often called one after the other, so they designed <code>nn.BCEWithLogitsLoss</code> that does both, and better.<br>
To cite the <a href="https://pytorch.org/docs/stable/nn.html#bcewithlogitsloss" rel="nofollow noopener">docs</a> :<br>
<em>This loss combines a Sigmoid layer and the BCELoss in one single class. This version is more numerically stable than using a plain Sigmoid followed by a BCELoss as, by combining the operations into one layer, we take advantage of the log-sum-exp trick for numerical stability.</em><br>
(no idea what it means)</p>
<p>So in both cases (for the loss and for the accuracy) it’s the data resulting of the sigmoid which is taken into account. It’s just hidden beneath a trick for the loss, and for the accuracy you don’t really need to compute it since you can just compare the raw logits with <code>0</code>.</p>

shirui-japina: <p>Oh…I got it…<img src="https://discuss.pytorch.org/images/emoji/apple/flushed.png?v=9" title=":flushed:" class="emoji" alt=":flushed:"><br>
Truly appreciate your timely help <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=9" title=":smile:" class="emoji" alt=":smile:"></p>