The testing data accuracy is higher than training data

li0218: <p>I met the situation that the testing data accuracy is higher than the training data, I have checked the dataset and I am sure that the training data is the training data while the testing is testing… What is this indicate to and how to understand it?</p>
<p>I used the following code to calculate the accuracy in certain epochs,</p>
<blockquote>
<pre><code>        cnn = cnn.eval()
        train_correct = 0
        for _, (images, labels) in enumerate(train2_loader):
            images = Variable(images).cuda()
            labels = Variable(labels).long().cuda()
            #test
            outputs = cnn(images)
            train_loss = criterion(outputs, labels)
            train_pred = outputs.data.max(1, keepdim=True)[1] # get the index of the max log-probability
            train_correct += train_pred.eq(labels.data.view_as(train_pred)).long().sum().item()

        test_correct = 0
        for _, (images, labels) in enumerate(test_loader):
            images = Variable(images).cuda()
            labels = Variable(labels).long().cuda()
            #test
            outputs = cnn(images)
            test_loss = criterion(outputs, labels)
            test_pred = outputs.data.max(1, keepdim=True)[1] # get the index of the max log-probability
            test_correct += test_pred.eq(labels.data.view_as(test_pred)).long().sum().item()
        
        cnn = cnn.train()
</code></pre>
</blockquote>
<p>while here the</p>
<blockquote>
<p>train2_loader</p>
</blockquote>
<p>is the subset of my train data and the test_loader is my test data.</p>
<p>For some example, one of the output is:</p>
<blockquote>
<p>CNN Epoch [10/10], Iter [100/1563] Loss: 0.3904, Batch correct: 0.8516, Train Correct: 0.7488, Train Loss: 0.2358, Test Correct: 0.7503, Test Loss: 0.1708, At time: 4.08CNN<br>
Epoch [10/10], Iter [200/1563] Loss: 0.4498, Batch correct: 0.8125, Train Correct: 0.7690, Train Loss: 0.2809, Test Correct: 0.7677, Test Loss: 0.2149, At time: 4.11CNN<br>
Epoch [10/10], Iter [300/1563] Loss: 0.4276, Batch correct: 0.8125, Train Correct: 0.7468, Train Loss: 0.2255, Test Correct: 0.7510, Test Loss: 0.1658, At time: 4.14CNN<br>
Epoch [10/10], Iter [400/1563] Loss: 0.4731, Batch correct: 0.7852, Train Correct: 0.7264, Train Loss: 0.2473, Test Correct: 0.7294, Test Loss: 0.2133, At time: 4.17CNN<br>
Epoch [10/10], Iter [500/1563] Loss: 0.4603, Batch correct: 0.7891, Train Correct: 0.7352, Train Loss: 0.2176, Test Correct: 0.7389, Test Loss: 0.1578, At time: 4.20CNN<br>
Epoch [10/10], Iter [600/1563] Loss: 0.4361, Batch correct: 0.8047, Train Correct: 0.7471, Train Loss: 0.2297, Test Correct: 0.7493, Test Loss: 0.1737, At time: 4.23CNN<br>
Epoch [10/10], Iter [700/1563] Loss: 0.4625, Batch correct: 0.8125, Train Correct: 0.7709, Train Loss: 0.2767, Test Correct: 0.7704, Test Loss: 0.2082, At time: 4.26CNN<br>
Epoch [10/10], Iter [800/1563] Loss: 0.4046, Batch correct: 0.8359, Train Correct: 0.7140, Train Loss: 0.1969, Test Correct: 0.7171, Test Loss: 0.1459, At time: 4.29CNN<br>
Epoch [10/10], Iter [900/1563] Loss: 0.4715, Batch correct: 0.7812, Train Correct: 0.7494, Train Loss: 0.2575, Test Correct: 0.7533, Test Loss: 0.2058, At time: 4.32CNN<br>
Epoch [10/10], Iter [1000/1563] Loss: 0.3714, Batch correct: 0.8516, Train Correct: 0.7497, Train Loss: 0.2552, Test Correct: 0.7497, Test Loss: 0.1978, At time: 4.35CNN<br>
Epoch [10/10], Iter [1100/1563] Loss: 0.4417, Batch correct: 0.8125, Train Correct: 0.7710, Train Loss: 0.2845, Test Correct: 0.7725, Test Loss: 0.2007, At time: 4.38CNN<br>
Epoch [10/10], Iter [1200/1563] Loss: 0.4808, Batch correct: 0.7891, Train Correct: 0.7563, Train Loss: 0.2644, Test Correct: 0.7565, Test Loss: 0.2081, At time: 4.41<br>
CNN Epoch [10/10], Iter [1300/1563] Loss: 0.4471, Batch correct: 0.8125, Train Correct: 0.7532, Train Loss: 0.2323, Test Correct: 0.7512, Test Loss: 0.1790, At time: 4.44<br>
CNN Epoch [10/10], Iter [1400/1563] Loss: 0.4374, Batch correct: 0.8281, Train Correct: 0.7438, Train Loss: 0.2278, Test Correct: 0.7448, Test Loss: 0.1688, At time: 4.46<br>
CNN Epoch [10/10], Iter [1500/1563] Loss: 0.4210, Batch correct: 0.8125, Train Correct: 0.7587, Train Loss: 0.2712, Test Correct: 0.7592, Test Loss: 0.2311, At time: 4.49</p>
</blockquote>
<p>Some of the “Train correct” are higher and some are lower than the “Test correct”, and the “Test Loss” is smaller than “Train Loss”.  Does this mean there is potential to increase the neural network complexity?</p>

Mazhar_Shaikh: <p>Hi li0218,<br>
This means that your model may have generalized well. You could try increasing the network size to try to get a better fit.<br>
Precaution : Have a look at the confusion matrix of your model for the training and test data. And compare them with the ratio of positive and negative classes in your data. It may be happening that your model is mostly predicting one class, and the accuracy simply reflects the proportion of that class in your data.<br>
Hope this helps!<br>
Regards</p>

li0218: <p>Thank you Mazhar,<br>
Good! I used equal ratios data so I think it’s not that problem. I will try increase the layers…</p>