Batch_size and validation accuracy

elynn: <p>I follow the tutorial to train a cnn model on CIFAR10, and when I use this model to validate on test_data, I got different accuracy when I use different batch_size on test_data, is it normal? As you can see below, as the batch_size increased to 280, the accuracy of this model has declined. I use batch size of 64 to train this model, I don’t know why just changing the batch size of test data will get different accuracy.</p>
<pre><code class="lang-auto"># Validate on test_data
model.eval()
def test_accu(batch_size):
    test_loader = DataLoader(test_data, batch_size=batch_size, shuffle=False)
    correct_count = 0.0
    for i, data in enumerate(test_loader, 0):
        img, labels = data
        x = Variable(img)
        y = Variable(labels)
        if torch.cuda.is_available():
            x = x.cuda()
            y = y.cuda()
        outs = model(x)
        _, pred = torch.max(outs, -1)
        correct_count += (pred == y).sum().data[0]
    return correct_count
for i in range(200, 500, 10):
    correct_count = test_accu(i)
    print('BatchSize: {}, Accu: {}'.format(i, correct_count/len(test_data)))
</code></pre>
<p>BatchSize: 200, Accu: 0.90894<br>
BatchSize: 210, Accu: 0.90894<br>
BatchSize: 220, Accu: 0.90894<br>
BatchSize: 230, Accu: 0.90894<br>
BatchSize: 240, Accu: 0.90894<br>
BatchSize: 250, Accu: 0.90894<br>
BatchSize: 260, Accu: 0.90894<br>
BatchSize: 270, Accu: 0.90382<br>
BatchSize: 280, Accu: 0.4891<br>
BatchSize: 290, Accu: 0.08974<br>
BatchSize: 300, Accu: 0.06414<br>
BatchSize: 310, Accu: 0.08462<br>
BatchSize: 320, Accu: 0.11022<br>
BatchSize: 330, Accu: 0.13582<br>
BatchSize: 340, Accu: 0.1563<br>
BatchSize: 350, Accu: 0.17678<br>
BatchSize: 360, Accu: 0.19726<br>
BatchSize: 370, Accu: 0.21774<br>
BatchSize: 380, Accu: 0.23822<br>
BatchSize: 390, Accu: 0.25358<br>
BatchSize: 400, Accu: 0.26894<br>
BatchSize: 410, Accu: 0.2843<br>
BatchSize: 420, Accu: 0.29966<br>
BatchSize: 430, Accu: 0.31502<br>
BatchSize: 440, Accu: 0.32526<br>
BatchSize: 450, Accu: 0.34062<br>
BatchSize: 460, Accu: 0.35086<br>
BatchSize: 470, Accu: 0.36622<br>
BatchSize: 480, Accu: 0.37646<br>
BatchSize: 490, Accu: 0.3867</p>

smth: <p>once the model is in <code>.eval()</code> mode, the BatchSize should not matter. This is weird.<br>
If you give a script that I can run and reproduce this, I’m happy to look into it. I think it’s just a user-error somewhere.</p>

elynn: <p>Hi smth,<br>
I use this script to train the model, <a href="https://github.com/lynic/notebooks/blob/master/ml/pytorch/CIFAR10_CNN.ipynb" rel="nofollow noopener">https://github.com/lynic/notebooks/blob/master/ml/pytorch/CIFAR10_CNN.ipynb</a><br>
EPOCHES I set to 20, and I use pytorch 0.1.12 with cuda 8.0 .<br>
Then I use the code in question description to test the model, seems batch_size really impact the results.</p>

smth: <p>This was a subtle issue.</p>
<p>Change this line:</p>
<pre><code class="lang-python">correct_count += (pred == y).sum().data[0]
</code></pre>
<p>to</p>
<pre><code class="lang-python">correct_count += (pred == y).double().sum().data[0]
</code></pre>
<p>The problem is that <code>pred == y</code> returns a ByteTensor, which has only an 8-bit range. Hence, after a particular batch-size, the sum was overflowing, and hence the wrong results.</p>

elynn: <p>Thanks smth, that works!</p>

deepcode: <p>wow, that is a subtle bug, anyway to prevent these kinds of issues from happening better? I imagine this kind of issue will pop up often.</p>

Annus_Zulfiqar: <p>Hi! I am also having a similar issue with my code. Testing with different batch sizes is giving me different results. I am doing it like this…</p>
<p>(inside my test loader code, after prediction…)</p>
<p>pred = pred.view(-1)<br>
label = label.view(-1)<br>
correct_count += (pred == label).double().sum().item()<br>
total_count += pred.size(0)</p>
<p>and at the end of the loop over test loader I do this…</p>
<p>mean_accuracy = correct_count * 100 / total_count</p>
<p>I have tried so many different test sizes and found out that test accuracy is max, 96% with a test batch size of 512 and above and keeps declining with decrease in batch size until batch size 1 gives me 11% accuracy and I have a total of 10 classes, which means I could achieve this with random weights as well. I can’t figure out what I am doing wrong?</p>

Anishkumar_Iyer: <p>Hi did you figure out your issue ?<br>
Coz I’m having the same problem.<br>
Testing with batch size 1 gives lowest score, but with higher gives higher results</p>