Model not getting better after each epoch

username: SurajSubramanian
cooked: <p>Using transfer learning for a multi-label image classification task (changed the classifier layer of resnet50).</p>
<p>I have changed the fully connected layer of resnet50 as follows :</p>
<pre><code class="lang-auto">model = models.resnet50(pretrained=True)
n_inputs = model.fc.in_features
model.fc = nn.Sequential(
                      nn.Linear(n_inputs, 256), 
                      nn.ReLU(), 
                      nn.Dropout(0.4),
                      nn.Linear(256, n_classes),                   
                      nn.LogSoftmax(dim=1))
model = model.to(device)
</code></pre>
<pre><code class="lang-auto">def fit(n_epochs):
  for epoch in range(n_epochs):

      running_loss = 0.0
      for i, (inputs,labels) in enumerate(Bar(train_loader)):
          inputs, labels = inputs.to(device), labels.to(device)
          optimizer.zero_grad()

          outputs = model(inputs)
          loss = criterion(outputs, labels)
          loss.backward()
          optimizer.step()

          running_loss += loss.item()
      print("Finished epoch {}; running_loss =", running_loss)

  print('Finished Training' )
</code></pre>
<pre><code class="lang-auto">fit(1)
</code></pre>
<pre><code class="lang-auto">4794/4794: [===============================&gt;] - ETA 0.1s
Finished epoch {}; running_loss = 1571.782483279705
Finished Training
</code></pre>
<pre><code class="lang-auto">fit(1)
</code></pre>
<pre><code class="lang-auto">4794/4794: [===============================&gt;] - ETA 0.1s
Finished epoch {}; running_loss = 1571.5039553046227
Finished Training
</code></pre>
<p>My loss functions and optimizers are</p>
<pre><code class="lang-auto">criterion = nn.NLLLoss()
optimizer = optim.Adam(model.parameters(), lr=0.00001)
</code></pre>
<p>After each epoch, my model is not getting any better. Is there any mistake in my training loop</p>

username: ptrblck
cooked: <p>The training loop looks alright.</p>
<p>Are you working on a multi-label (each sample belongs to 0, 1 or more classes) or multi-class (each sample belongs to a single class only) classification?<br>
In the former case <code>nn.NLLLoss</code> is most likely the wrong criterion to use.<br>
Also, what did you change in the resnet? Since you are using <code>nn.NLLLoss</code>, you would have to add <code>F.log_softmax</code> as the last activation function.</p>

username: SurajSubramanian
cooked: <p>Thanks,<br>
I am working on a multi-label classification problem, there are 4 classes in total.<br>
And I changed <code>nn.NLLLoss</code> to <code>nn.CrossEntropyLoss</code> and my model is training <img src="https://discuss.pytorch.org/images/emoji/apple/grinning.png?v=9" title=":grinning:" class="emoji" alt=":grinning:"></p>
<p>Should I use Adam or SGD optimizers.<br>
Thanks a lot !!</p>

username: ptrblck
cooked: <p>Good to hear it’s working now!</p>
<aside class="quote no-group" data-username="SurajSubramanian" data-post="3" data-topic="73997">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/surajsubramanian/48/19757_2.png" class="avatar"> SurajSubramanian:</div>
<blockquote>
<p>Should I use Adam or SGD optimizers.</p>
</blockquote>
</aside>
<p>I don’t have a general answer, so you would have to try both for your use case. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"></p>

username: SurajSubramanian
cooked: <p>Sure, thanks a lot <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=9" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:"></p>