Using and not using model.eval()

shrutishrestha: <p>When I use model.eval() with torch.no_grad() during my evaluation, I get 36% accuracy. But, if I do not use model.eval() and only use torch.no_grad() then my accuracy is 87%. Why is this hapenning?<br>
with_model_eval<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/8/68a243508a3a8033d838f0e23c2f6d05c131f959.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/68a243508a3a8033d838f0e23c2f6d05c131f959" title="with_model_eval"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/8/68a243508a3a8033d838f0e23c2f6d05c131f959_2_345x97.jpeg" alt="with_model_eval" data-base62-sha1="eVDiPnxMxybQEAmz4NgZClm7DtL" width="345" height="97" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/8/68a243508a3a8033d838f0e23c2f6d05c131f959_2_345x97.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/6/8/68a243508a3a8033d838f0e23c2f6d05c131f959_2_517x145.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/6/8/68a243508a3a8033d838f0e23c2f6d05c131f959_2_690x194.jpeg 2x" data-dominant-color="3A281F"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">with_model_eval</span><span class="informations">1883×530 53.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>without_model_eval<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/2/126aa1d7ec1b019ab68813446e0da6a31aef7ac0.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/126aa1d7ec1b019ab68813446e0da6a31aef7ac0" title="without_model_eval"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/2/126aa1d7ec1b019ab68813446e0da6a31aef7ac0_2_345x97.jpeg" alt="without_model_eval" data-base62-sha1="2CV3hmK9kpdOd5ggOMdA3QS1ujm" width="345" height="97" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/2/126aa1d7ec1b019ab68813446e0da6a31aef7ac0_2_345x97.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/1/2/126aa1d7ec1b019ab68813446e0da6a31aef7ac0_2_517x145.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/1/2/126aa1d7ec1b019ab68813446e0da6a31aef7ac0_2_690x194.jpeg 2x" data-dominant-color="342118"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">without_model_eval</span><span class="informations">1883×530 47.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>This might be happening if you are using e.g. batchnorm layers, with “bad” running stats for the mean and var. If you use the model in training mode during evaluation, the batch statistics will be used and the running stats updated (which could be seen as a data leak for future <code>model.eval()</code> validation runs).<br>
You could play around with the <code>momentum</code> to smooth the update of the running stats and check other posts in this forum, as other users are also facing this issue.</p>