VGG-16 and ResNet-9 loss values not corresponding to accuracy on test set

username: alyeko
cooked: <p>I have two models whose performance I am comparing, a ResNet-9 model and a VGG-16 model. They are being used for image classification. Their accuracies on the same test set are:</p>
<p>ResNet-9 = 99.25% accuracy<br>
VGG-16 = 97.90% accuracy</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/d/cde4eaad1739e93b9b443f25f5a03393bdaf7675.png" data-download-href="https://discuss.pytorch.org/uploads/default/cde4eaad1739e93b9b443f25f5a03393bdaf7675" title="resnet9-tv-losses"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/d/cde4eaad1739e93b9b443f25f5a03393bdaf7675_2_381x375.png" alt="resnet9-tv-losses" data-base62-sha1="tnqkET6Eymlg4dnGB8gOBlPfW97" width="381" height="375" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/d/cde4eaad1739e93b9b443f25f5a03393bdaf7675_2_381x375.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/d/cde4eaad1739e93b9b443f25f5a03393bdaf7675_2_571x562.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/d/cde4eaad1739e93b9b443f25f5a03393bdaf7675_2_762x750.png 2x" data-dominant-color="F7F8F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">resnet9-tv-losses</span><span class="informations">4223×4155 341 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/0/70d4d9dc54bcfab158a4a8a4976f79d44dbe7e11.png" data-download-href="https://discuss.pytorch.org/uploads/default/70d4d9dc54bcfab158a4a8a4976f79d44dbe7e11" title="VGG16-tv-losses"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/0/70d4d9dc54bcfab158a4a8a4976f79d44dbe7e11_2_378x375.png" alt="VGG16-tv-losses" data-base62-sha1="g69vulhubMiPwxAV96JWMOyN9n3" width="378" height="375" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/0/70d4d9dc54bcfab158a4a8a4976f79d44dbe7e11_2_378x375.png, https://discuss.pytorch.org/uploads/default/optimized/3X/7/0/70d4d9dc54bcfab158a4a8a4976f79d44dbe7e11_2_567x562.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/7/0/70d4d9dc54bcfab158a4a8a4976f79d44dbe7e11_2_756x750.png 2x" data-dominant-color="F5F6F7"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">VGG16-tv-losses</span><span class="informations">4223×4180 395 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>However from their loss curves during training, shown in the images, I see that the VGG-16 has lower losses as compared to ResNet-9 which has higher losses.</p>
<p>I am using torch.nn.CrossEntropyLoss() for both VGG-16 and ResNet-9. I would have expected the ResNet-9 to have lower losses (because it performs better on the test set) but this is not the case.</p>
<p>Is this observation normal?</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="alyeko" data-post="1" data-topic="160736">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alyeko/48/49123_2.png" class="avatar"> alyeko:</div>
<blockquote>
<p>I am using torch.nn.CrossEntropyLoss() for both VGG-16 and ResNet-9. I would have expected the ResNet-9 to have lower losses (because it performs better on the test set) but this is not the case.</p>
<p>Is this observation normal?</p>
</blockquote>
</aside>
<p>It could be expected as the loss is not only reflecting the number of correctly classifier examples but also the “confidence” of these predictions. You could thus compare the logits (or probabilities) of both models and see if the ResNet’s predictions are “smoother” but still correct.</p>