Get probabilities from Resnet output

username: over9k
cooked: <p>I am using the Resnet18 model (<a href="https://pytorch.org/docs/stable/torchvision/models.html#id10" rel="nofollow noopener">https://pytorch.org/docs/stable/torchvision/models.html#id10</a>) for 2D image classification, with 2 output classes. The final layer is the fully connected (fc) layer according to the model descriptions.</p>
<p>How would I go about getting the probabilities of the output classes? For example, now I get a 0 or 1, but I would want something like 0.75 (for 0) and 0.25 (for 1). Do I have to use a softmax layer somehow? And would this only need to be done for the testing portion or will I also need to make some change for the training as well?</p>
<p>Thank you for the help.</p>

username: ptrblck
cooked: <p>If you are using 2 output units for the binary classification use case and thus use <code>torch.argmax</code> to calculate the predicitons, you could use <code>F.softmax</code> to get the probabilities.<br>
Note that you won’t need these probabilities to calculate the loss etc. if you are using <code>nn.CrossEntropyLoss</code>, but can of course inspect them.</p>

username: over9k
cooked: <p>Just to be clear, do you mean to just add a last F.softmax layer when getting my test predictions?</p>
<p>And yes, I am using nn.CrossEntropyLoss, so I won’t need it to calculate the loss, but I want to be formal here and use softmax for another operation.</p>
<p>Thanks!</p>

username: ptrblck
cooked: <p>Yes, just use <code>F.softmax</code> outside of the model:</p>
<pre><code class="lang-python">output = model(data) # output contains logits
# you can calculate the loss using `nn.CrossEntropyLoss` and the logits output
loss = criterion(output, target) 

# and you can calculate the probabilities, but don't pass them to `nn.CrossEntropyLoss`
probs = F.softmax(output, dim=1)
</code></pre>

username: keshav-b
cooked: <p>Thank you so much for this, since all the pre-trained models are majorly used for classification, why can’t we have the final layer as the softmax so that we can get the probabilities with just <code>output = model(data)</code> won’t that be easier? And we can take argmax of it if we need labels.</p>
<p>Im not sure if im right, but i think the current pre-trained models output logits. Maybe we can have a flag or something to output both, or either probabilities or logits.</p>

username: ptrblck
cooked: <p>I think the main reason to output logits is because commonly used loss functions such as <code>nn.CrossEntropyLoss</code> (multi-class classification) and <code>nn.BCEWithLogitsLoss</code> (multi-label classification) expect logits, not probabilities. Applying the <code>softmax</code> manually would reduce the numerical precision without much benefit besides being able to “see” the probabilities for debugging purposes.<br>
Adding the <code>F.softmax</code> activation is trivial and (for these classification models) doesn’t yield any purpose besides printing the probabilities (you should not pass the probabilities to the mentioned loss functions).</p>

username: keshav-b
cooked: <p>alright, understood. Thank you <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>