Weird evaluation results for facial landmark prediction using ResNet18

Mona_Jalal: <p>Running this evaluation code, yields very weird results. I understand that I only have 69 images in faces directory but aren’t these results very weird?</p>
<pre><code class="lang-auto"># Test the model
model.eval()  # eval mode (batchnorm uses moving mean/variance instead of mini-batch mean/variance)
with torch.no_grad():
    correct = 0
    total = 0
    for i, sample_batched in enumerate(train_loader):
        print(i, sample_batched['image'].size(),
        sample_batched['landmarks'].size())
        
        images_batch, landmarks_batch = \
            sample_batched['image'], sample_batched['landmarks']
        
        images = images_batch
        labels = landmarks_batch.reshape(-1, 68 * 2)
        
        images = Variable(images.float())
        labels = Variable(labels)
        
        images = images.to(device)
        labels = labels.to(device)
        
        outputs = model(images)
        #_, predicted = torch.max(outputs.data, 1)
        #_, predicted = outputs.data
        print("Predicted", outputs.data.shape)
        
        outputs = outputs.cpu()
        images = images.cpu()
        
        if i_batch == 3:
          plt.figure()
          show_landmarks_batch({'image': images, 'landmarks': outputs.data.reshape(-1, 68, 2) })
          plt.axis('off')
          plt.ioff()
          plt.show()
          show_landmarks_batch({'image': images, 'landmarks': labels.reshape(-1, 68, 2) })
          plt.axis('off')
          plt.ioff()
          plt.show()
          break
</code></pre>
<pre><code class="lang-auto">
(base) mona@mona:~/research/facial_landmark$ ls faces/*.jpg | wc -l
69
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/1/b1b7e46378cf41a15fe11ea6902ab4b646434430.png" data-download-href="https://discuss.pytorch.org/uploads/default/b1b7e46378cf41a15fe11ea6902ab4b646434430" title="Screenshot_2020-10-06_20-56-51"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/1/b1b7e46378cf41a15fe11ea6902ab4b646434430_2_690x304.png" alt="Screenshot_2020-10-06_20-56-51" data-base62-sha1="pmauXztFq77icjkEXGSC2zmFhrG" width="690" height="304" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/1/b1b7e46378cf41a15fe11ea6902ab4b646434430_2_690x304.png, https://discuss.pytorch.org/uploads/default/optimized/3X/b/1/b1b7e46378cf41a15fe11ea6902ab4b646434430_2_1035x456.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/b/1/b1b7e46378cf41a15fe11ea6902ab4b646434430_2_1380x608.png 2x" data-dominant-color="F4EDED"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot_2020-10-06_20-56-51</span><span class="informations">1436×634 128 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I also have to mention I have a very terrible test loss:</p>
<pre><code class="lang-auto">size of train loader is:  1
type(loss_train_step) is:  &lt;class 'torch.Tensor'&gt;
loss_train_step.dtype is:  torch.float32
Test Steps: 1/1  Loss: 15511.6732 
--------------------------------------------------
Epoch: 1  Train Loss: 15590.3896  Test Loss: 15511.6732
--------------------------------------------------

Minimum Test Loss of 15511.6732 at epoch 1/10
Model Saved

size of train loader is:  1
type(loss_train_step) is:  &lt;class 'torch.Tensor'&gt;
loss_train_step.dtype is:  torch.float32
Test Steps: 1/1  Loss: 16363.6969 
--------------------------------------------------
Epoch: 2  Train Loss: 15584.9707  Test Loss: 16363.6969
--------------------------------------------------
size of train loader is:  1
type(loss_train_step) is:  &lt;class 'torch.Tensor'&gt;
loss_train_step.dtype is:  torch.float32
Test Steps: 1/1  Loss: 16458.4072 
--------------------------------------------------
Epoch: 3  Train Loss: 15594.6885  Test Loss: 16458.4072
--------------------------------------------------
size of train loader is:  1
type(loss_train_step) is:  &lt;class 'torch.Tensor'&gt;
loss_train_step.dtype is:  torch.float32
Test Steps: 1/1  Loss: 15063.5689 
--------------------------------------------------
Epoch: 4  Train Loss: 16293.1094  Test Loss: 15063.5689
--------------------------------------------------

Minimum Test Loss of 15063.5689 at epoch 4/10
Model Saved

size of train loader is:  1
type(loss_train_step) is:  &lt;class 'torch.Tensor'&gt;
loss_train_step.dtype is:  torch.float32
Test Steps: 1/1  Loss: 16673.0292 
--------------------------------------------------
Epoch: 5  Train Loss: 14949.3047  Test Loss: 16673.0292
--------------------------------------------------
size of train loader is:  1
type(loss_train_step) is:  &lt;class 'torch.Tensor'&gt;
loss_train_step.dtype is:  torch.float32
Test Steps: 1/1  Loss: 15699.0407 
--------------------------------------------------
Epoch: 6  Train Loss: 16076.5234  Test Loss: 15699.0407
--------------------------------------------------
size of train loader is:  1
type(loss_train_step) is:  &lt;class 'torch.Tensor'&gt;
loss_train_step.dtype is:  torch.float32
Test Steps: 1/1  Loss: 15512.4619 
--------------------------------------------------
Epoch: 7  Train Loss: 15775.4736  Test Loss: 15512.4619
--------------------------------------------------
size of train loader is:  1
type(loss_train_step) is:  &lt;class 'torch.Tensor'&gt;
loss_train_step.dtype is:  torch.float32
Test Steps: 1/1  Loss: 16625.0978 
--------------------------------------------------
Epoch: 8  Train Loss: 14958.9971  Test Loss: 16625.0978
--------------------------------------------------
size of train loader is:  1
type(loss_train_step) is:  &lt;class 'torch.Tensor'&gt;
loss_train_step.dtype is:  torch.float32
Test Steps: 1/1  Loss: 15330.6030 
--------------------------------------------------
Epoch: 9  Train Loss: 15395.0596  Test Loss: 15330.6030
--------------------------------------------------
size of train loader is:  1
type(loss_train_step) is:  &lt;class 'torch.Tensor'&gt;
loss_train_step.dtype is:  torch.float32
Test Steps: 1/1  Loss: 15949.3246 
--------------------------------------------------
Epoch: 10  Train Loss: 15716.1689  Test Loss: 15949.3246
--------------------------------------------------
Training Complete
Total Elapsed Time : 19.73250699043274 s
</code></pre>
<p>This is how the evaluation results look in the original file here: <a href="https://colab.research.google.com/drive/1-28T5nIAevrDo6MwN0Qi_Cgdy9TEiSP_?usp=sharing#scrollTo=LatJ9Cb39so-" class="inline-onebox" rel="noopener nofollow ugc">Google Colab</a></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/2/b2d98f1dd58e6153cac39c43c930e8b7ef7ec62f.png" data-download-href="https://discuss.pytorch.org/uploads/default/b2d98f1dd58e6153cac39c43c930e8b7ef7ec62f" title="Screenshot_2020-10-06_22-07-12"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/2/b2d98f1dd58e6153cac39c43c930e8b7ef7ec62f_2_492x499.png" alt="Screenshot_2020-10-06_22-07-12" data-base62-sha1="pwb6GAjdXuR7TMYfaDOgTF5QiPZ" width="492" height="499" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/2/b2d98f1dd58e6153cac39c43c930e8b7ef7ec62f_2_492x499.png, https://discuss.pytorch.org/uploads/default/optimized/3X/b/2/b2d98f1dd58e6153cac39c43c930e8b7ef7ec62f_2_738x748.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/b/2/b2d98f1dd58e6153cac39c43c930e8b7ef7ec62f.png 2x" data-dominant-color="CACAC9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot_2020-10-06_22-07-12</span><span class="informations">782×794 156 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>What are the two images showing?<br>
Is the first one plotting the labels and the second one the predictions?<br>
If so, it seems the plot function or a reshaping is wrong, since the target labels are already visualized out of bounds.</p>

Mona_Jalal: <p>yeah one is gt the other one predictions</p>