Challenges when calculating accuracy for cifar10

blackbirdbarber: <p>Help me with the reasoning because I am at the moment a bit lost since I saw many different approaches.</p>
<p>I need to calculate accuracy the best possible way, let’s use some of the predefined datasets in PyTorch like cifar10, cifar100, … so single label classification problem.</p>
<p>I saw many different approaches. What is the best approach.</p>
<h1>approach 1 (keep track of correct and total labels on cpu)</h1>
<p>I think this approach is not optimal because you need to convert to cpu to some numpy array possible the number of correct examples. Why not keeping it on gpu.</p>
<h1>approach 2 (using the formula for a single batch)</h1>
<p>It may be just the correct labels we need. The totals we may get based on the batch enumerator and batch size.</p>
<p>However, this is a single batch accuracy formula.</p>
<pre><code class="lang-auto">def accuracy(preds, true): 
   preds = preds.argmax(dim=-1) 
   return (preds==true).float().mean()
</code></pre>
<p>This can work for all full dataset  if we concatentate preds and true arrays.<br>
I also think this is a wrong approach for cifar10, or ciffar100 because just keeping the single number of <code>correct</code> is enough.</p>
<p>You are welcome to comment.</p>

KaiHoo: <aside class="quote no-group" data-username="blackbirdbarber" data-post="1" data-topic="96646">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/b/6de8d8/48.png" class="avatar"> blackbirdbarber:</div>
<blockquote>
<p>Why not keeping it on gpu.</p>
</blockquote>
</aside>
<p>I cannot see any benefits to keep just a number on GPU. There’s no heavy computation. So approach 1 is already good.</p>

blackbirdbarber: <p>Thanks <a class="mention" href="/u/kaihoo">@KaiHoo</a>, just keeping correct and total labels on CPU is OK, but you do it for all batches like that. On the other hand the formula in approach 2 I saw in <a href="https://github.com/fastai/fastai/blob/dba806c926499b3380b1a1c27c456a1897015913/fastai/metrics.py#L98" rel="nofollow noopener">here</a> and I slightly modified it. It is good if you need per batch accuracy, but more important is to have per epoch accuracy. Is there any nice example for cifar10 that uses nice accuracy metrics practice, I would be happy if someone share.</p>
<p>For the epoch accuracy do you just average the per batch accuracy with <code>drop_last=False</code> or you keep the correct count?</p>

KaiHoo: <p>What about this one:</p>
<pre><code class="lang-auto">correct = total = 0
for batch_idx, (inputs, targets) in enumerate(trainloader):
    inputs, targets = inputs.to(device), targets.to(device)
    optimizer.zero_grad()
    outputs = model(inputs)
    loss = criterion(outputs, targets)
    loss.backward()
    optimizer.step()
    train_loss += loss.item()
    _, predicted = outputs.max(1)
    """
    if batch accuracy required
    total_batch = targets.size(0)
    correct_batch = predicted.eq(targets).sum().item()
    batch_acc = total_batch / correct_batch
    """
    correct += predicted.eq(targets).sum().item()
    total +=  targets.size(0)
</code></pre>