Visuals not correlating to values in Project

teohongwei898: <p>My current project involves deblurring images using CNNs.<br>
Upon plotting the 3 images, which are:</p>
<ol>
<li>Original sharp image</li>
<li>Blurred image using OpenCV</li>
<li>Deblured image of (2) using the model I have trained.</li>
</ol>
<p>Upon plotting these 3, I find out that (3) is a bit clearer than (2), which kinds of achieving what I am doing here.<br>
But calculating the MSE/SIM/PSNR values shows that (2) is better than (3) in terms of quality. I have normalized the values of all 3 images and this is what i get:</p>
<p>Degraded Image:<br>
PSNR: 25.266282651000232<br>
MSE: 580.1941565688776<br>
SSIM: 0.8825525369859532</p>
<p>Reconstructed Image:<br>
PSNR: 24.81144082688032<br>
MSE: 644.2546071027518<br>
SSIM: 0.8196236328699172</p>
<p>I was wondering what are the possible reasons why this will happen despite the reconstructed image looking better than the degraded image. Thanks!</p>

KFrank: <p>Hi Hong!</p>
<aside class="quote no-group" data-username="teohongwei898" data-post="1" data-topic="162609" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/teohongwei898/48/53587_2.png" class="avatar"> teohongwei898:</div>
<blockquote>
<p>Deblured image of (2) using the model I have trained.</p>
</blockquote>
</aside>
<p>How did you train your model?</p>
<p>For example, you might train a model to maximize some sort of “sharpness<br>
criterion” of the images it produces.</p>
<p>On the other hand, you might train a model to best match a ground-truth<br>
“original sharp image.”</p>
<p>In the former case you could well get deblurred images that look sharp but<br>
do differ somewhat from the “original sharp image.”</p>
<p>Best.</p>
<p>K. Frank</p>

teohongwei898: <p>I trained my model by putting (2) as my data and (1) as my ‘labels’, using the SRCNN model.<br>
Hence I was expecting it to be sharper <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=12" title=":confused:" class="emoji" alt=":confused:" loading="lazy" width="20" height="20"></p>

KFrank: <p>Hi Hong!</p>
<aside class="quote no-group" data-username="teohongwei898" data-post="3" data-topic="162609" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/teohongwei898/48/53587_2.png" class="avatar"> teohongwei898:</div>
<blockquote>
<p>I trained my model by putting (2) as my data and (1) as my ‘labels’, using the SRCNN model.</p>
</blockquote>
</aside>
<p>You haven’t told us what loss criterion you have used for training (nor<br>
the relationship of your training data to the “original sharp image” and<br>
“blurred image” of your original post).</p>
<p>If you trained, for example, with <code>MSELoss</code>, then you would expect<br>
your model to predict “deblurred” images that are closer to your<br>
“original sharp image,” as measured by <code>MSELoss</code>, than is the “blurred<br>
image” you input to your model.</p>
<p>But you say that your “deblurred image” has a greater <code>MSELoss</code> than<br>
the “blurred image”.</p>
<p>If your “original sharp image” / “blurred image” pair are not in your<br>
training set but are in a validation set that is of the same character<br>
as your training set, then you could be overfitting so that you get<br>
good results for your training set (lower <code>MSELoss</code>) but worse results<br>
(higher <code>MSELoss</code>) for samples not in your training set.</p>
<p>If your “original sharp image” / “blurred image” pair differs in character<br>
from your training data (as distinct from merely not being in your training<br>
set), then you might not be overfitting, <em>per se,</em> but your model might<br>
not be generalizing to data samples with somewhat different character.</p>
<p>Or, you could just have a bug somewhere.</p>
<p>(If you’re not using a loss criterion that teaches your model to predict<br>
deblurred images that match the unblurred images, <em>as measured by<br>
<code>MSELoss</code>,</em> then there is no real discrepancy here.)</p>
<p>One comment:  Even if the deblurring problem is hard, it would not<br>
be hard (unless you have a lossy model) for your model to learn to<br>
output the blurry image that was input to the model as its prediction,<br>
rather than output a deblurred image that has a higher <code>MSELoss</code> with<br>
respect to the unblurred target image, because outputting the input<br>
image would lower the loss.</p>
<p>If you are using <code>MSELoss</code> for your training, you might take a smallish<br>
subset of your training data and verify that you can overfit your data,<br>
which would mean that your model would predict deblurred images<br>
with low <code>MSELoss</code>s, at least for samples in the training subset you<br>
used to overfit.</p>
<p>Note, sharpness is, so to speak, a perceptual characteristic.  Producing<br>
an image that looks sharp is not the same thing as recovering the<br>
original unblurred image.  One could easily imagine a deblurring model<br>
(that was trained using some sort of sharpness criterion) that produces<br>
nice, sharp images that deviate more from the ground-truth original<br>
sharp image (as measured, say, by <code>MSELoss</code>) than does the blurred<br>
image input to the model.</p>
<p>Best.</p>
<p>K. Frank</p>

teohongwei898: <aside class="quote no-group" data-username="KFrank" data-post="4" data-topic="162609">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p>deblurring problem is hard, it would not<br>
be hard (unless you have a lossy model) for your model to learn to<br>
output the blurry image that was input to the model as its prediction,<br>
rather than output a deblurred image that has a higher <code>MSELoss</code> with<br>
respect to the unblurred target image, because outputting the input<br>
image would lower the loss.</p>
<p>If you are using <code>MSELoss</code> for your training, you might take a smallish<br>
subset of your training data and verify that you can overfit your data,<br>
which would mean that your model would predict deblurred images<br>
with low <code>MSELoss</code></p>
</blockquote>
</aside>
<p>Hi Frank!<br>
Thanks for the reply.<br>
I’ve used MSELoss, and the image that I am testing is actually in the training set.<br>
Hence, in theory, shouldn’t the MSE Loss be actually smaller for the Deblurred image compared to the blurred image?</p>
<p>Thank you!</p>

KFrank: <p>Hi Hong!</p>
<aside class="quote no-group" data-username="teohongwei898" data-post="5" data-topic="162609" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/teohongwei898/48/53587_2.png" class="avatar"> teohongwei898:</div>
<blockquote>
<p>Hence, in theory, shouldn’t the MSE Loss be actually smaller for the Deblurred image compared to the blurred image?</p>
</blockquote>
</aside>
<p>Yes, your reasoning is correct – the deblurred image should give you a<br>
lower <code>MSELoss</code> (because that’s what you’ve trained your model to do).</p>
<p>That’s why I suggest trying to overfit part of your training set.  If you take<br>
an adequately small subset of your training data and train your model with<br>
an adequately large number of epochs, your model should be able to<br>
“memorize” the “original sharp images” that you are using for your targets<br>
and then simply reproduce them close to exactly (which will then give an<br>
<code>MSELoss</code> very close to zero).</p>
<p>If you can’t overfit in this way, you either have a bug somewhere or you have<br>
a <em>very</em> weird model (or you aren’t training enough).</p>
<p>First make sure you can overfit and then go on to the next step of training<br>
a useful model (that <em>isn’t</em> overfit) that can produce deblurred images that<br>
do have a lower <code>MSELoss</code> than the blurred-image inputs (and that also<br>
hopefully look sharp to the eye) for data in a validation set that isn’t used<br>
for training.</p>
<p>Best.</p>
<p>K. Frank</p>

teohongwei898: <p>To over fit the model, should I simply increase the number of epochs? Or are there any other methods I can use to overfit my model?<br>
Thank you!</p>