Cross Entropy Loss get predicted class

username: Er_Hall
cooked: <p>Hi all,</p>
<p>I am using in my multiclass text classification problem the cross entropy loss. But I have been confused. My targets are in [0, c-1] format. How can I obtain the predicted class? An example will be  helpful, since cross entropy loss is using softmax why I don’t take probabilities as output with sum =1?</p>

username: ptrblck
cooked: <p><code>nn.CrossEntropyLoss</code> expects logits, as internally <code>F.log_softmax</code> and <code>nn.NLLLoss</code> will be used.<br>
If you want to get the predicted class, you could simply use <code>torch.argmax</code>:</p>
<pre><code class="lang-python">output = model(input)
pred = torch.argmax(output, dim=1)
</code></pre>
<p>I assume dim1 is representing the classes. If not, you should change the <code>dim</code> argument.</p>

username: Er_Hall
cooked: <p>Can you explain me why this implementation for multiclass text classification doesn’t use sigmoid  since expects logits?</p>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/keishinkickback/Pytorch-RNN-text-classification/blob/master/model.py" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/keishinkickback/Pytorch-RNN-text-classification/blob/master/model.py" target="_blank" rel="nofollow noopener">keishinkickback/Pytorch-RNN-text-classification/blob/master/model.py</a></h4>
<pre><code class="lang-py">import torch
import torch.nn as nn
from torch.nn.utils.rnn import pack_padded_sequence, pad_packed_sequence


class RNN(nn.Module):

    def __init__(self, vocab_size, embed_size, num_output, rnn_model='LSTM', use_last=True, embedding_tensor=None,
                 padding_index=0, hidden_size=64, num_layers=1, batch_first=True):
        """

        Args:
            vocab_size: vocab size
            embed_size: embedding size
            num_output: number of output (classes)
            rnn_model:  LSTM or GRU
            use_last:  bool
            embedding_tensor:
            padding_index:
            hidden_size: hidden size of rnn module
</code></pre>

  This file has been truncated. <a href="https://github.com/keishinkickback/Pytorch-RNN-text-classification/blob/master/model.py" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


username: ptrblck
cooked: <p><code>sigmoid</code> would convert each output to a probability in the range <code>[0, 1]</code>.<br>
Logits on the other side are unbound (<code>[-inf, inf]</code>), so you should not apply any activation function on your model outputs.</p>

username: Er_Hall
cooked: <p>So if I have 5 output classes and 3 test instances and for example I will take the below output:<br>
tensor([[ 0.4657, -0.7640, -1.4268, -0.5012,  1.2167],<br>
[-0.4578, -0.5621, -0.5652, -0.4056,  0.2509],<br>
[-0.5617,  0.8141, -0.1722, -0.1264,  0.2285]]</p>
<p>this means that for the first instance the right class is the 5th? for the second also the 5h and of the last instance the 2nd?  (the biggest ones) ??</p>

username: ptrblck
cooked: <p>The mentioned classes indices might not be the right ones (this is determined by the target), but the predicted ones (with the highest probability).</p>
<p>If you want to get probability values, you could use <code>F.softmax</code> to get values in the range <code>[0, 1]</code>.<br>
However do not pass these values to the criterion. Use them just for debugging/printing purposes.</p>

username: Er_Hall
cooked: <p>So the positions I mentioned above is the “predicted classes”, right?</p>
<p>Thank you very much for your replies, I really appreciate it!</p>

username: ptrblck
cooked: <p>Yes, that’s correct. The highest logit (in this setup) gives you the predicted class.<br>
That’s also why you can call <code>torch.argmax</code> directly on the logits without applying softmax, as this won’t change the predicted classes (max in logits will still be the max value after softmax).</p>

username: R_S
cooked: <p>i used torch.argmax before training the model to convert one hot encoded data ( b, class, h, w) =(16, 12, 256, 256) to (16, 256, 256). Now i can use the cross entropy loss function. Is it the right way to train a model for segmentation task?</p>

username: ptrblck
cooked: <p>Yes, as explained <a href="https://discuss.pytorch.org/t/training-semantic-segmentation/49275/17">here</a>.</p>