Multi classification activation function for last layer

username: Mohamed_Farag
cooked: <p>I started to learn about <strong>pytorch l</strong>ately after using <strong>tensorflow</strong> for almost 1 year, i am confused about something:</p>
<p>In <strong>Tensorflow</strong> when we have multiclassification problem we set at the last activation layer the number of classes and the <em><strong>type of activation function which is "Softmax</strong></em>" and using <em>“Cross-entropy loss”</em></p>
<p>so in <strong>Pytorch</strong> when building a network we set last layer to nn.linear().</p>
<p>can anyone clarify the concept?</p>
<p>Thanks a lot. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: Dwight_Foster
cooked: <p>Basically in pytorch the cross entropy loss function combines a softmax and nllloss function into one. So for training you end up not needing to add a softmax function to the model because it is just computed in the loss function. However for predicting values with your model you would have to use a softmax function. You can also include a softmax function in your model and just use the nllloss function and it would do the same thing.</p>

username: KFrank
cooked: <p>Hi Dwight and Mohamed!</p>
<p>I would like to offer two clarifications:</p>
<aside class="quote no-group" data-username="Dwight_Foster" data-post="2" data-topic="112445" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dwight_foster/48/18182_2.png" class="avatar"> Dwight_Foster:</div>
<blockquote>
<p>However for predicting values with your model you would have to use a softmax function.</p>
</blockquote>
</aside>
<p>If by “predicting values” you mean predicting class labels, this is not<br>
true.  To get a predicted class label from the probabilities returned by<br>
<code>softamx()</code> you take <code>argmax()</code>.  But <code>softmax()</code> does not change the<br>
order of the values, so you can just as well take <code>argmax()</code> of the logits<br>
you would have input to <code>softmax()</code>.  That is, you can skip <code>softmax()</code><br>
altogether.</p>
<aside class="quote no-group">
<blockquote>
<p>You can also include a softmax function in your model and just use the nllloss function and it would do the same thing.</p>
</blockquote>
</aside>
<p>This is not quite right.  You would need to use pytorch’s <code>log_softmax()</code><br>
followed by <code>nll_loss()</code> to reproduce pytorch’s <code>cross_entropy()</code>.</p>
<p>Best.</p>
<p>K. Frank</p>

username: Mohamed_Farag
cooked: <p>Thank you a lot!<br>
i will try both ideas <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=9" title=":blush:" class="emoji" alt=":blush:"></p>

username: Mohamed_Farag
cooked: <p>Thank you a lot K.Frank <img src="https://discuss.pytorch.org/images/emoji/apple/smiling_face_with_three_hearts.png?v=9" title=":smiling_face_with_three_hearts:" class="emoji" alt=":smiling_face_with_three_hearts:"><br>
I will try today.</p>

username: Mohamed_Farag
cooked: <p>I have another question what about using nn.softmax() instead of log_softmax() with nll_loss()?</p>
<p>i saw documentation about the first fuchtion, will it work ?</p>