Class-wise accuacy

username: oasjd7
cooked: <pre><code class="lang-auto">[1]
for batch_idx, (inputs, labels) in enumerate(test_loader):
     inputs, labels = inputs.cuda(), labels.cuda()
     with torch.no_grad():
         outputs = model(inputs)
     _, predicted = outputs.max(1)
     total += labels.size(0)
     correct += predicted.eq(labels).sum().item()
 print(correct / total)
</code></pre>
<pre><code class="lang-auto">[2]

nb_classes = 10
net.eval()
confusion_matrix = torch.zeros(nb_classes, nb_classes)
for batch_idx, (inputs, labels) in enumerate(test_loader):
    inputs = inputs.cuda()
    labels = labels.cuda()
    outputs = net(inputs)
    _, preds = torch.max(outputs, 1)
    for t, p in zip(labels.view(-1), preds.view(-1)):
        confusion_matrix[t.long(), p.long()] += 1
ele_wise_acc = confusion_matrix.diag() / confusion_matrix.sum(1) # Class-wise acc
print(ele_wise_acc.mean() * 100) # Total acc


</code></pre>
<p>[1]: Calculate accuracy code<br>
[2]: Calculate class-wise accuracy from <a href="https://discuss.pytorch.org/t/how-to-find-individual-class-accuracy/6348/2" class="inline-onebox">How to find individual class Accuracy</a>.</p>
<p>[1] and [2] have different accuracy.<br>
I think both of them are looks fine, Anyone can find problems?</p>

username: KFrank
cooked: <p>Hi oasjd7!</p>
<aside class="quote no-group quote-modified" data-username="oasjd7" data-post="1" data-topic="88141" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/oasjd7/48/5053_2.png" class="avatar"> oasjd7:</div>
<blockquote>
<pre><code class="lang-auto">[1]
     total += labels.size(0)
     correct += predicted.eq(labels).sum().item()
 print(correct / total)
</code></pre>
<pre><code class="lang-auto">[2]
    for t, p in zip(labels.view(-1), preds.view(-1)):
        confusion_matrix[t.long(), p.long()] += 1
ele_wise_acc = confusion_matrix.diag() / confusion_matrix.sum(1) # Class-wise acc
print(ele_wise_acc.mean() * 100) # Total acc
</code></pre>
<p>[1] and [2] have different accuracy.<br>
I think both of them are looks fine, Anyone can find problems?</p>
</blockquote>
</aside>
<p>In short, the two results will differ when the classes don’t all have<br>
the same number of samples (and some other conditions that<br>
aren’t the main point).</p>
<p>In [1] you are calculating the number of correct predictions<br>
(regardless of the specific class) divided by the total number<br>
of predictions.  This is what I would typically call “accuracy.”</p>
<p>In [2] you are calculating the accuracy for each class separately,<br>
and then taking the <em>mean</em> of those class accuracies.  This is<br>
not the same.</p>
<p>To see this, consider a case where you have two classes, but<br>
class A has 1000 samples and class B has 10 samples.  Let’s<br>
say you get all 1000 class A predictions wrong and get all 10<br>
class B predictions right.  Your overall accuracy ([1]) will be<br>
<code>10 / 1010</code>, which is about 1%.  But your class A accuracy is<br>
0% and your class B accuracy is 100%, so averaging those<br>
accuracies over the two classes will give you 50%.</p>
<p>Best.</p>
<p>K. Frank</p>