Training not working with Dataset

Jae0143: <p><a href="https://jovian.ml/jae0143/car-classification" rel="nofollow noopener">Link to my notebook</a>. Hey guys I am trying to predict car model using Stanford Car Dataset. However, as they did not have labels on their dataset, I have connected the image with label. And I tried to replicate Image loader by creating another custom Dataloader. However, my training is not working. Can you guys please check it out??</p>

ptrblck: <p>What exactly is not working?<br>
Are you getting any errors using your new dataset or is the model accuracy bad?<br>
In the latter case, I would recommend to try to overfit a small data sample (e.g. just 10 samples) and make sure your model is able to do so by playing around with the hyper-parameters.</p>

Jae0143: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>. Here I want to predict the car label using the training data. Therefore, I am using one-hot encoding to represent the total of 196 classes. However, when I am training, then the F score and other losses shows some weird numbers. Ex) F score not going up at all or even decrease when training loss and valid loss is decreasing. Can you take a look at it and what is going wrong?</p>

ptrblck: <p>It seems you are using <code>F.cross_entropy</code> as your loss function, which will not accept one-hot encoded target. Instead the target should contain the class indices and have the shape <code>[batch_size]</code> for a multi-class classification.</p>
<p>Did you also check the loss curves and the accuracy? Is the accuracy going up?</p>

Jae0143: <p>I should check the accuracy. However. what do you mean by target should contain the class indices and have the shape [batch_size]? Sorry, but I am beginner. To my understanding, the class indices sounds like it should have integer value which matches up with the class name. Could you please clarify for me?</p>

Jae0143: <p>the accuracy is also not increasing <a class="mention" href="/u/ptrblck">@ptrblck</a>. Can you please take a look at it if you don’t mind? Thanks for your help.</p>

ptrblck: <p>As described in the <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.CrossEntropyLoss">docs</a> for <code>nn.CrossEntropyLoss</code>, the target should use class indices and be a <code>LongTensor</code>.<br>
Here is a small example for 5 classes:</p>
<pre><code class="lang-python">batch_size = 10
nb_classes = 5

output = torch.randn(batch_size, nb_classes, requires_grad=True) # your model output
target = torch.randint(0, nb_classes, (batch_size,))
print(target)
&gt; tensor([1, 4, 2, 2, 2, 0, 2, 1, 3, 3])

criterion = nn.CrossEntropyLoss()
loss = criterion(output, target)
</code></pre>
<p>If you are passing one-hot encoded targets, the criterion should raise an error, since your target tensor should have two dimensions in the one-hot version.<br>
Could you check your current target and make sure it contains the expected values in the expected shape?</p>

Jae0143: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/3/33841dab9dc7a5afa28b8575b131978a0f684ce3.png" data-download-href="https://discuss.pytorch.org/uploads/default/33841dab9dc7a5afa28b8575b131978a0f684ce3" title="Screen Shot 2020-06-29 at 12.59.18 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/3/33841dab9dc7a5afa28b8575b131978a0f684ce3_2_690x334.png" alt="Screen Shot 2020-06-29 at 12.59.18 PM" data-base62-sha1="7lJpimxxpZftGMsgeN6win9Gawr" width="690" height="334" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/3/33841dab9dc7a5afa28b8575b131978a0f684ce3_2_690x334.png, https://discuss.pytorch.org/uploads/default/optimized/3X/3/3/33841dab9dc7a5afa28b8575b131978a0f684ce3_2_1035x501.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/3/33841dab9dc7a5afa28b8575b131978a0f684ce3.png 2x" data-dominant-color="F2F2F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2020-06-29 at 12.59.18 PM</span><span class="informations">1236×600 105 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>If you see the picture, it is visible that both validation and training set loss Is working. I have changed the structure of the code also. Now I separated the test and train image first, then I loaded using custom datasset. Batch size is 50, so I don’t know what is going on. Thank you for your help.</p>

Jae0143: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>, hey it seems like I have used cross entropy and sigmoid, which lead to the breakdown. I totally forgot that sigmoid was used only for 2 cases and softmax for multi class classification.</p>

ptrblck: <p>Note that <code>F.cross_entropy</code> does not need a softmax, but expects the raw logits. You should therefore not use any activation function after your last linear layer. (Unsure, if you are going to swap the sigmoid for a softmax)</p>