Why batch size will affect my validation set accuracy？

username: shifei
cooked: <p>I tried to train Lprnet, an ocr network for Recognition license plates I used <a href="https://github.com/xuexingyu24/License_Plate_Detection_Pytorch/blob/master/LPRNet/LPRNet_Train.py" rel="nofollow noopener">https://github.com/xuexingyu24/License_Plate_Detection_Pytorch/blob/master/LPRNet/LPRNet_Train.py</a> to train this model<br>
When I tested my validation set with batch size = 128 I got 95% accuracy rate but when I put batch size = 1 the model is very poor with only 73% accuracy rate which means that I predict single image with only 73% accuracy rate for me I was very surprised that I set model.eval () to ensure the model is in validation mode but it seems that the BN layer still affects my model Who has experience dealing with similar things can help me solve this problem Thank you</p>

username: shifei
cooked: <p>I found that other people have the same confusion as me. I tried some of the suggestions but they didn’t help me solve the problem, including<br>
for child in model.children():<br>
for ii in range(len(child)):<br>
if type(child[ii]) == nn.BatchNorm2d:<br>
child[ii].track_running_stats = False</p>

username: vmirly1
cooked: <p>I looked at the code for computing acc, and I found the evaluation a bit odd, in these lines: <a href="https://github.com/xuexingyu24/License_Plate_Detection_Pytorch/blob/master/LPRNet/Evaluation.py#L89-L96" rel="nofollow noopener">https://github.com/xuexingyu24/License_Plate_Detection_Pytorch/blob/master/LPRNet/Evaluation.py#L89-L96</a></p>
<p>If you can print some values for a single batch (use <code>break</code> to get out of the loop after one single iteration and avoid printing too much) and compare them across different batch-sizes. Then, you may see some weird or unexpected things happen when you change the batch-size.</p>

username: shifei
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/1/e17e1d0b6f57e4e9e15299dfb36578065eca721e.png" data-download-href="https://discuss.pytorch.org/uploads/default/e17e1d0b6f57e4e9e15299dfb36578065eca721e" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/1/e17e1d0b6f57e4e9e15299dfb36578065eca721e_2_690x173.png" alt="image" data-base62-sha1="waNCUOCTm5PYRNdOP5wDMmf3Trw" width="690" height="173" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/1/e17e1d0b6f57e4e9e15299dfb36578065eca721e_2_690x173.png, https://discuss.pytorch.org/uploads/default/optimized/3X/e/1/e17e1d0b6f57e4e9e15299dfb36578065eca721e_2_1035x259.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/1/e17e1d0b6f57e4e9e15299dfb36578065eca721e_2_1380x346.png 2x" data-dominant-color="31343B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1553×390 398 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div> Yes, the left side of this picture is the result when batch size = 1, and the right side is the result when batch size = 128. It can be found that when batch size = 1, the model will always predict one more bit in front.  In fact, I found that when the same picture (batch size = 1) and this picture and other 127 pictures (batch size = 128) forward results are different, I think it may be that the BN layer parameters are not fixed, but I used model.eval ()</p>

username: shifei
cooked: <p>This problem has been bothering me for more than a day,  who has encountered similar problems can help me .Thanks</p>

username: BarCodeReader
cooked: <p>bro, here are my suggestions:</p>
<p>firstly you should post this question to github repo instead of here…</p>
<p>secondly…you can try to write the evaluation dataloader by your own…and see if it works</p>
<p>you spend maybe 2 days to debug but maybe if you use 10min re-write the dataloader the problem solved.</p>

username: Fri
cooked: <p>Hi to everyone,<br>
I have a question related to training and validation datasets. I have different value between training loss and validation loss for the same datasets (training = validation), I think that it’s not normal behavior.<br>
Thanks for your answer.</p>

username: aditya_gupta
cooked: <p>Similar issue, my accuracy increases when I increase my batch size form 1 to 16. Any updates on this issue ?<br>
Thanks.</p>

username: ptrblck
cooked: <p>In case you are seeing a bad validation performance when using a training batch size of 1: this could happen, if the running stats are not representing the underlying dataset stats and a known limitation of batchnorm layers. You could try to change the <code>momentum</code> to smooth the updates and also might want to read e.g. the <a href="https://arxiv.org/abs/1803.08494">GroupNorm paper</a> which also explains these limitations.</p>