Why I only change "shuffle=False" to "shuffle=True" in my DataLoader when I test my model, my test result will be different so much?

username: khtao
cooked: <p>I use pytorch 0.4 and python 3.6<br>
my test code is:</p>
<pre><code class="lang-auto">def test_model(dataloader, model, test_num=3000):
    device = torch.device("cuda" if opt.use_cuda else "cpu")
    total = 0
    correct_pos = 0
    correct_num = 0
    predicted_pos = 0
    pos_num = 0
    with torch.no_grad():
        for ii, (img, label) in tqdm(enumerate(dataloader)):
            label = label.view(len(label)).numpy()
            img = img.to(device)
            output = model(img)  # 1*3*299*299
            _, predicted = torch.max(output.data, dim=1)
            predicted = predicted.cpu().numpy()
            total += len(label)
            predicted_pos += predicted.sum()
            correct = predicted == label
            correct_num += correct.sum()
            correct_pos += np.logical_and(label, correct).sum()
            pos_num += label.sum()
            if ii &gt; test_num:
                break
    recall = correct_pos / pos_num
    precision = correct_pos / predicted_pos
    accuracy = correct_num / total
    neg_precision = (correct_num - correct_pos) / (total - pos_num)
    f_num = 2 * recall * precision / (recall + precision)
    # print('Accuracy of the network :' + str(accuracy))
    # print('Recall of the network :' + str(recall))
    # print('Precision of the network :' + str(precision))
    # print('Neg_precision of the network :' + str(neg_precision))
    return {"Recall": recall,
            "Accuracy": accuracy,
            "Precision": precision,
            "Neg_precision": neg_precision,
            "F": f_num}
</code></pre>
<p>when shuffle=True</p>
<p>{‘Recall’: 0.7745604963805585, ‘Accuracy’: 0.9315490043961727, ‘Precision’: 0.9413095387708935, ‘Neg_precision’: 0.9838965517241379, ‘F’: 0.8498326431043286}</p>
<p>when shuffle=False</p>
<p>{‘Recall’: 0.19451913133402274, ‘Accuracy’: 0.639384535815878, ‘Precision’: 0.23404255319148937, ‘Neg_precision’: 0.7877241379310345, ‘F’: 0.2124583498051618}</p>

username: lili.ece.gwu
cooked: <p>You probably don’t want to shuffle your test data loader … it is even safer to test the samples one by one, I think.</p>

username: khtao
cooked: <p>but I want know what cause the differences. Do you know?</p>

username: SimonW
cooked: <aside class="quote no-group" data-username="khtao" data-post="1" data-topic="18047">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/c77e96/48.png" class="avatar"> khtao:</div>
<blockquote>
<p>test_num</p>
</blockquote>
</aside>
<p>It’s probably because that you don’t enumerate over all test data</p>

username: khtao
cooked: <p>I should use model.eval()，before I test it</p>

username: Lee_Daron
cooked: <p>but why  model.eval  influence that ？</p>

username: Vidit_Agarwal
cooked: <p>By using shuffle=True you shuffle up the dataset which makes the training batches more generalized which in turn makes your model more generalized  thus you model performs much better on the unseen data</p>