Training epoch accuracy greater than 100% at the end of each epoch

upasana_siva: <p>This is my code for traning and validation loop. Please help me with this issue.</p>
<pre><code class="lang-auto">model = get_pretrained_model('inceptionv3')
#print(model.forward)
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = model.to(device)
criterion = torch.nn.CrossEntropyLoss()
n_class=5
optimizer = torch.optim.Adam(model.parameters())
scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer, 'min' if n_class &gt; 1 else 'max', patience=2)
#training and validation
totEpochs = 5
for ep in range(totEpochs):

    # Training loop
    epoch_loss = 0
    train_acc=0
    valid_acc=0
    with tqdm(total=len(trainset), desc=f'Epoch {ep + 1}/{totEpochs}', unit='img') as pbar:
        for batch in trainLoader:
            img = batch['image'].to(device)
            label = batch['label'].to(device)
            #depth = batch['depth']
            net_pred = model(img)#,depth.unsqueeze(1))
            
            loss = criterion(net_pred[0], label)
            epoch_loss += loss.item()

            pbar.set_postfix(**{'Loss (Batch)': loss.item()})

            optimizer.zero_grad()
            loss.backward()
            nn.utils.clip_grad_value_(model.parameters(), 0.1)
            optimizer.step() 

            # Calculate training accuracy 
            pred = torch.argmax(net_pred[0], dim=1)
            correct_tensor = pred.eq(label)
            # Need to convert correct tensor from int to float to average
            accuracy = torch.mean(correct_tensor.type(torch.FloatTensor))
            # Multiply average accuracy times the number of examples in batch
            train_acc += accuracy.item() * img.size(0)
            pbar.update(img.shape[0])
        pbar.set_postfix(**{'Average Loss': epoch_loss/len(trainset), 'training Accuracy': f'{train_acc:.2f}%'})
        #pbar.set_postfix(**{'training Accuracy': f'{train_acc:.2f}%'})
   # validation loop
    val_loss = 0
    with tqdm(total=len(valset), desc=f'Validation', unit='img') as pbar:
        for batch in validLoader:
            with torch.no_grad():
                img = batch['image'].to(device)
                label = batch['label'].to(device)
                #depth = batch['depth']
                net_pred = model(img)#,depth.unsqueeze(1))
                
                loss = criterion(net_pred[0], label)
                val_loss += loss.item()

                pbar.set_postfix(**{'Loss (validation)': loss.item()})

                #calculate validation aaccuracy
                pred = torch.argmax(net_pred[0], dim=1)
                correct_tensor = pred.eq(label)
                accuracy = torch.mean(correct_tensor.type(torch.FloatTensor))
                # Multiply average accuracy times the number of examples
                valid_acc += accuracy.item() * img.size(0)
                pbar.update(img.shape[0])
        scheduler.step(val_loss / len(valset))
        pbar.set_postfix(**{'Average Loss': val_loss / len(valset), 'Validation Accuracy': f'{valid_acc:.2f}%'})
     #pbar.set_postfix(**{'Validation Accuracy': f'{valid_acc:.2f}%'})
    #print(f'\t\tTraining Accuracy: {train_acc:.2f}%\t Validation Accuracy: {valid_acc:.2f}%')
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/e/7e4053d5888bea7da5bf818a2f2ddd6ed46311dc.png" data-download-href="https://discuss.pytorch.org/uploads/default/7e4053d5888bea7da5bf818a2f2ddd6ed46311dc" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/7/e/7e4053d5888bea7da5bf818a2f2ddd6ed46311dc.png" alt="image" data-base62-sha1="i0RYTOauueFxOEr8TQLw2vdGSok" width="690" height="239" data-dominant-color="CEE3CF"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1134×394 22.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>It seems you are calculating the accuracy of each batch but are also scaling it with the batch size, which could yield values &gt; 1. E.g. if your batch size is 10 and all samples are correctly predicted, <code>accuracy</code> would be <code>1.</code> and <code>train_acc</code> would then accumulate <code>1 * 10 = 10</code>. Here is a small example:</p>
<pre><code class="lang-python">train_acc = 0
output = torch.randn(10, 10)
label = torch.randint(0, 10, (10,))
print(label)
# &gt; tensor([3, 8, 1, 6, 8, 4, 9, 8, 1, 1])
pred = torch.argmax(output, dim=1)
print(pred)
# &gt; tensor([9, 3, 7, 6, 0, 9, 6, 1, 6, 2])

correct_tensor = pred.eq(label)
print(correct_tensor)
# &gt; tensor([False, False, False,  True, False, False, False, False, False, False])

accuracy = torch.mean(correct_tensor.type(torch.FloatTensor))
print(accuracy)
# &gt; tensor(0.1000)

train_acc += accuracy.item() * output.size(0)
print(train_acc)
# &gt; 1.0000000149011612
</code></pre>
<p>I might be missing something in your code so feel free to correct me.</p>

upasana_siva: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Thank you for replying.<br>
Yes, It worked when I removed *img.size(0)</p>
<p>I have another doubt regarding this matter. When the training accuracy is low, does that mean the model is not training?</p>

ptrblck: <p>A low accuracy shows that the model is (currently) creating predictions which are not very accurate, i.e. it’s mostly wrong.</p>