Why the torch.max() of predictions and F.softmax(pred) are equal? and What is the logic behind this?

ziba: <p>I’m new in pytorch. Sorry if my question is stupid.</p>
<p>I have a multiclass classification problem and for it I have a convolutional neural network that has <code>Linear</code> layer in its last layer.</p>
<p>I get predictions from this model so it gives me a tensor that has n_class elements. For example:<br>
<code>tensor([class_1, class_2, class_3])</code></p>
<pre><code class="lang-auto">class CNN(nn.Module):
    
    # Constructor
    def __init__(self):
        
        super(CNN, self).__init__()
        
        
        # cnn and maxpool layers
        self.cnn_ = nn.Conv2d(in_channels=1, 
                         out_channels=conv_kernel_output_channel, 
                         kernel_size=(filter_size, embedding_size), 
                         stride=1, 
                         padding=0)
        self.maxpool_ = nn.MaxPool2d(kernel_size=(longest_sentence_length - filter_size + 1, 1), 
                                stride=1, 
                                padding=0)
        
        # droput layer
        self.drop = nn.Dropout(p = dropout_keep_prob)
        
        # fully connected layer
        self.fc1 = nn.Linear(conv_kernel_output_channel * len(filter_sizes), num_classes)
    
    
    # Prediction
    def forward(self, x):
        # convolution layer
        x1 = self.cnn_(x)

        # apply activation function
        x1 = torch.relu(x1)

        # maxpooling layer
        x1 = self.maxpool_(x1)

        # flatten output
        x1 = x1.view(x1.size(0), -1)
        
        # dropout
        x1 = self.drop(x1)
        
        # fully connected layer
        x1 = self.fc1(x1)
        
        # no softmax at the end, because cross entropy loss have it implicitly
        
        return x1

pred = CNN(x)
_, label_1 = torch.max(pred)

pred_soft = F.softmax(pred)
_, label_2 = torch.max(pred_soft )
</code></pre>
<p><strong>Questions:</strong><br>
1- Why getting the <code>torch.max()</code> from this prediction will give us the label, I mean why for desired label our model produce bigger values? What is the logic behind this?</p>
<p>2- why getting the <code>torch.max()</code> from this prediction and from <code>F.softmax()</code> will give use same results and why we can interpret them as same and is enough to use one of them for getting the predicted label?</p>

KFrank: <p>Hi Ziba!</p>
<aside class="quote no-group" data-username="ziba" data-post="1" data-topic="141019" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/e56c9b/48.png" class="avatar"> ziba:</div>
<blockquote>
<p>I have a multiclass classification problem and for it I have a convolutional neural network that has <code>Linear</code> layer in its last layer.</p>
</blockquote>
</aside>
<p>Your final <code>Linear</code> layer will produce* a set of raw-score <em>logits</em><br>
(unnormalized log-odds-ratios), one for each of the classes.  These<br>
are related to the probabilities that the network predicts for the sample<br>
in question being in each of the classes, and, specifically, the class<br>
probabilities are given by <code>softmax()</code> of the predicted logits.</p>
<p>*)  Your network produces such values in essence because you train<br>
it to produce such values.</p>
<aside class="quote no-group">
<blockquote>
<pre><code class="lang-auto">pred = CNN(x)
_, label_1 = torch.max(pred)

pred_soft = F.softmax(pred)
_, label_2 = torch.max(pred_soft )
</code></pre>
</blockquote>
</aside>
<p>Here you want <code>_, label_1 = torch.max (pred, dim = 1)</code> (assuming<br>
that <code>pred</code> has shape <code>[nBatch, nClass]</code>).  As written, your code will<br>
produce an error.  (Similarly, you want <code>torch.max (pred_soft, dim = 1)</code>.)</p>
<aside class="quote no-group">
<blockquote>
<p>1- Why getting the <code>torch.max()</code> from this prediction will give us the label, I mean why for desired label our model produce bigger values?</p>
</blockquote>
</aside>
<p>With the corrected expression, <code>torch.max()</code> will return both the <code>max()</code>,<br>
which gets assigned to the variable <code>_</code> (used stylistically in python as a<br>
“throw-away” variable), and the <code>argmax()</code> (the index of the maximum<br>
element), which gets assigned to <code>label_1</code>.  The largest logit corresponds<br>
to the largest probability, and the <em>index</em> of the largest logit is the class<br>
label for what the network is predicting as the most probable class.</p>
<aside class="quote no-group">
<blockquote>
<p>2- why getting the <code>torch.max()</code> from this prediction and from <code>F.softmax()</code> will give use same results?</p>
</blockquote>
</aside>
<p>The logits, <code>pred</code>, and the probabilities, <code>F.softmax (pred)</code>, are different<br>
numbers, but the largest logit and the largest probability correspond to<br>
one another (as do the second largest, and so on).  So the index of the<br>
largest logit (<code>argmax (pred)</code>) and the index of the largest probability<br>
(<code>argmax (F.softmax (pred))</code>) are the same – that is, they both give<br>
you the same predicted class label.</p>
<p>Best.</p>
<p>K. Frank</p>