Bad Inference result during testing even when using training data

luminoussin: <p>Hi, I have encounter a weird behavior from my model, during training phase, I printed out some of my validation result from my model and the result is very satisfying but after I stop my training and try to do testing with it, I got a very bad result which is very different from the validation result, even when I am using data used for training to do the testing. Here is some of my validation result:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/7/677432577890af8e2a547acaf23c99a39387aba7.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/677432577890af8e2a547acaf23c99a39387aba7" title="Test_8_0_recon"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/7/677432577890af8e2a547acaf23c99a39387aba7_2_500x500.jpeg" alt="Test_8_0_recon" data-base62-sha1="eLc82hcD7gjDdga1y1OrXmDkMYf" width="500" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/7/677432577890af8e2a547acaf23c99a39387aba7_2_500x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/6/7/677432577890af8e2a547acaf23c99a39387aba7_2_750x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/6/7/677432577890af8e2a547acaf23c99a39387aba7_2_1000x1000.jpeg 2x" data-dominant-color="39281F"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Test_8_0_recon</span><span class="informations">2066×2066 810 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div> .<br>
While this is some of the result I got from testing, only pay attention to the color distribution: <div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/e/ee5747ad7ade00b43fc8a960d91d611be314569e.png" data-download-href="https://discuss.pytorch.org/uploads/default/ee5747ad7ade00b43fc8a960d91d611be314569e" title="Screenshot from 2020-03-21 08-53-22"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/e/ee5747ad7ade00b43fc8a960d91d611be314569e_2_489x500.png" alt="Screenshot from 2020-03-21 08-53-22" data-base62-sha1="y0sDkXvauCbROHjQXytuFpGJk3Y" width="489" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/e/ee5747ad7ade00b43fc8a960d91d611be314569e_2_489x500.png, https://discuss.pytorch.org/uploads/default/original/3X/e/e/ee5747ad7ade00b43fc8a960d91d611be314569e.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/e/e/ee5747ad7ade00b43fc8a960d91d611be314569e.png 2x" data-dominant-color="252928"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2020-03-21 08-53-22</span><span class="informations">498×509 141 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
as you can see, the result is grayish not even close to how the validation result looks like. So I am hoping someone who knows what is the problem with my model can give me some clue to fix this.</p>

ptrblck: <p>Are you calling <code>model.eval()</code> for the validation and test case?<br>
Also, make sure to use the same preprocessing for both use cases.</p>
<p>If that doesn’t help, could you post your training and validation/testing loop, so that we could have a look?</p>

luminoussin: <p>During validation I use with torch.no.grad() before starting , while for test case I use model.eval() and yes I use the same pre-processing steps, the only difference would be the validation use batch size of 64 while during testing I use batch size of 1 since I use different PC for them. Would that be a problem?</p>

ptrblck: <p>Could you use <code>model.eval()</code> for both, the validation and testing phase?<br>
This will make sure to disable dropout as well as use the running stats of batchnorm layers instead of the current batch stats.</p>
<p>If you validation results are bad after using <code>model.eval()</code>, then the batchnorm stats might be off.</p>
<p>PS: <code>torch.no_grad()</code> is still useful for both phases, as it will save memory.</p>

luminoussin: <p>Hi, thanks for the reply. Apparently after discussing with my friend, we found out that the problem lies on my model. But thank you for your advice on model_eval(), I will use it on my future work</p>

Amiya_Maity: <p>Hi luminoussin, how did you solve this issue, because i am also facing same issue, and regarding model.evaI() i pretty sure about that, I am calling this eval()  from my prediction function but prediction result is not good.</p>

luminoussin: <p>Hi Amiya, as I said on my previous post the problem turn out to be on my model itself. So if you already use model.eval() before doing inference with your trained model then that means there is a possibility that your model has overfitted. At least that was what happened on mine</p>

Amiya_Maity: <p>Hi luminoussin, can you check my <a href="https://drive.google.com/open?id=163uXkzFkfjoJvBjtWZoOA7MqD2oe3QWm" rel="nofollow noopener">code</a>?  i am in beginner level.</p>

luminoussin: <p>I think it is better if you create your own post about it. Therefore more people can help you and it is better if you post your code instead of giving link</p>

H_MP: <p>I had same problem and in my case it was due to loading images, I needed to convert my images and rescale them to (32x32x1)</p>
<pre><code class="lang-auto">img = cv2.imread(path)
img = cv2.cvtColor(img, cv2_COLORBGR2GRAY)`
img = cv2.resize( img, (32,32) )
</code></pre>
<p>but when I change it to</p>
<pre><code class="lang-auto">transform_valid = transforms.Compose([
    transforms.ToPILImage(), 
    transforms.Resize((32, 32)),
    transforms.ToTensor()])

img = cv2.imread(path, 0)
img = custom_transfrom(img).unsqueeze(0)

</code></pre>
<p>my problem solved</p>