Loss decreasing but accuracy is still the same

username: bing
cooked: <p>Hi Guys,</p>
<p>I am trying to do multi-class image classification.<br>
I am trying to debug my network for potential bugs so training and validating are on the same subset of data. Logically, the training and validation loss should decrease and then saturate which is happening but also, it should give 100% or a very large accuracy on the valid set( As it is same as of training set), but it is giving 0% accuracy.<br>
So I am wondering whether my calculation of accuracy is correct or not? Below is my code, could you please check and point me if there are any bugs which I am not able to figure it out.</p>
<pre><code>    model.eval()
    for batch_idx, (data, target) in enumerate(loader['valid']):
        # move to GPU
        if torch.cuda.is_available():
            data, target = data.cuda(), target.cuda()
        # update the average validation loss
        output = model(data).squeeze()
        output = torch.unsqueeze(output, 0)
        loss = criterion(output, target)   # changes
        valid_loss += ((1 / (batch_idx + 1)) * ((loss.data) - valid_loss))
        pred = output.data.max(1, keepdim=True)[0]
        correct += np.sum(np.squeeze(pred.eq(target.data.view_as(pred)).cpu().numpy()))
        total += data.size(0)
    accuracy = 100. * (correct/total)
    print('Epoch: {} \tTraining Loss: {:.6f} \tValidation Loss: {:.6f} \t Validation Accuracy: {:.6f}'.format(
        epoch,
        train_loss,
        valid_loss,
        accuracy
        ))</code></pre>

username: Caruso
cooked: <p>Hi <a class="mention" href="/u/bing">@bing</a>,</p>
<p>When using <code>pred = output.data.max(1, keepdim=True)[0]</code> the 0 index is the maximum value and the first index the argmax. You probably want to compare the argmax of your predictions with your target labels, so  <code>pred = output.data.max(1, keepdim=True)[1]</code> should work.</p>

username: bing
cooked: <p>Thanks <a class="mention" href="/u/caruso">@Caruso</a> for answering.<br>
It works but there is a blind spot in my understanding which I am able to catch slightly but I want to clear it up more.<br>
Suppose model will predict the following probabilities - <code>0.14, 1.2, -0.3</code><br>
how the index’s are mapped with the labels, like how do we know that the first index is mapping to the first label and similarly the other indexes too</p>

username: KFrank
cooked: <p>Hi Bing!</p>
<aside class="quote no-group" data-username="bing" data-post="3" data-topic="97954" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/b/839c29/48.png" class="avatar"> bing:</div>
<blockquote>
<p>Suppose model will predict the following probabilities - <code>0.14, 1.2, -0.3</code></p>
</blockquote>
</aside>
<p>Just to be precise, these are not “probabilities.”  (If they were, they<br>
would be values between 0 and 1 that sum to 1.)  They are most<br>
likely the output of a final <code>Linear</code> layer (with no subsequent non-linear<br>
“activation”), and should be understood as raw-score <em>logits.</em></p>
<aside class="quote no-group">
<blockquote>
<p>how the index’s are mapped with the labels, like how do we know that the first index is mapping to the first label and similarly the other indexes too</p>
</blockquote>
</aside>
<p>The output of a model means whatever you train it to mean.</p>
<p>If your <code>target</code> values are “integer categorical labels,” and your<br>
<code>criterion</code> (loss function) is <code>CrossEntropyLoss</code>, then you will<br>
be training your model so that the<code>[0]</code> element of its <code>output</code> maps<br>
to the class whose integer class label is <code>0</code>, element <code>[1]</code> maps to<br>
class label <code>1</code>, and so on.</p>
<p>To reiterate, the relationship between the output of your model and<br>
the class labels you use to train your model depends entirely on what<br>
you train your model to do.</p>
<p>Best.</p>
<p>K. Frank</p>

username: bing
cooked: <p>Thank You KFrank.<br>
That resolved my doubt and was really helpful.<br>
I have one more thing - In this case, I have integer categorical labels but they are starting from 1 not 0, so do I need to convert them to start from 0 to get them mapped to the <code>[0]</code> element.</p>

username: Caruso
cooked: <p>Hi <a class="mention" href="/u/bing">@bing</a>,</p>
<p>sorry for not answering. If you look at the documentation for <a href="https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html#torch.nn.CrossEntropyLoss" rel="noopener nofollow ugc">CrossEntropyLoss</a>, it says</p>
<blockquote>
<p>This criterion expects a class index in the range [0,C−1] as the target for each value of a 1D tensor of size minibatch.</p>
</blockquote>
<p>Greetings.</p>