My CNN Model is not Converging

username: aatiibutt
cooked: <p>I am beginner in PyTorch; with the help of a YouTube channel “Deep Lizard” i learned about the tensors along with the operations. Now, i am working on a vehicle classification model, having 10 different classes of 1000 images each. But my model is not converging the training loss. I have attached the some chunks of code below. Please let me know, where i did mistake. Thanks.</p>
<p><span class="hashtag">#Resize</span> Image<br>
transform_in = transforms.Compose([<br>
transforms.Resize([128, 128]),<br>
transforms.ToTensor(),<br>
transforms.Normalize((0.4915, 0.4823, 0.4468),<br>
(0.2470, 0.2435, 0.2616))<br>
])</p>
<p><span class="hashtag">#Model</span> Class<br>
class Net(nn.Module):<br>
def <strong>init</strong>(self):<br>
super().<strong>init</strong>()<br>
self.conv1 = nn.Conv2d(3, 64, kernel_size=3, padding=1)<br>
self.pool1 = nn.MaxPool2d(2,2)</p>
<pre><code>    self.conv2 = nn.Conv2d(64, 32, kernel_size=3, padding=1)
    self.pool2 = nn.MaxPool2d(2,2)
    
    self.conv3 = nn.Conv2d(32, 16, kernel_size=3, padding=1)
    self.pool3 = nn.MaxPool2d(2,2)
    
    self.conv4 = nn.Conv2d(16, 8, kernel_size=3, padding=1)
    self.pool4 = nn.MaxPool2d(2,2)
    
    self.fc1 = nn.Linear(8 * 8 * 8, 288)
    self.fc2 = nn.Linear(288, 144)
    self.fc3 = nn.Linear(144, 72)
    self.fc4 = nn.Linear(72, 2)
    
def forward(self, x):
    out = F.max_pool2d(torch.relu(self.conv1(x)), 2)
    out = F.max_pool2d(torch.relu(self.conv2(out)), 2)
    out = F.max_pool2d(torch.relu(self.conv3(out)), 2)
    out = F.max_pool2d(torch.relu(self.conv4(out)), 2)
    out = out.view(-1, 8 * 8 * 8)
    out = torch.relu(self.fc1(out))
    out = torch.relu(self.fc2(out))
    out = torch.relu(self.fc3(out))
    out = torch.relu(self.fc4(out))        
    out = F.softmax(out, dim=1)
    return out
</code></pre>
<p><span class="hashtag">#Training</span><br>
def training_loop(n_epochs, optimizer, model, loss_fn, train_loader):<br>
for epoch in range(1, n_epochs + 1):<br>
loss_train = 0.0<br>
for imgs, labels in train_loader:<br>
imgs = imgs.to(device=device)  # &lt;1&gt;<br>
labels = labels.to(device=device)<br>
outputs = model(imgs)<br>
loss = loss_fn(outputs, labels)</p>
<pre><code>        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        loss_train += loss.item()

    if epoch == 1 or epoch % 2 == 0:
        print('Epoch {}, Training loss {}' epoch, loss_train / len(train_loader)))
</code></pre>
<p><span class="hashtag">#Test</span><br>
model = Net().to(device=device)  # &lt;1&gt;<br>
optimizer = optim.SGD(model.parameters(), lr=0.001, momentum=0.9)<br>
loss_fn = nn.CrossEntropyLoss()</p>
<p>training_loop(<br>
n_epochs = num_epochs,<br>
optimizer = optimizer,<br>
model = model,<br>
loss_fn = loss_fn,<br>
train_loader = train_loader,<br>
)</p>

username: futscdav
cooked: <p>From nn.CrossEntropyLoss docs:</p>
<pre><code class="lang-auto">The input is expected to contain raw, unnormalized scores for each class.
</code></pre>
<p>do not include the softmax in your forward.</p>

username: aatiibutt
cooked: <p>Thanks for your kind response. I updated the softmax with “self.fc4(out)”, and here is the output.</p>
<p>Epoch 1, Training loss 0.2394336342997849<br>
Epoch 2, Training loss 0.41787283269688485<br>
Epoch 4, Training loss 0.4808643462508917<br>
Epoch 6, Training loss 0.4917321742884815<br>
Epoch 8, Training loss 0.49367881812155245<br>
Epoch 10, Training loss 0.4937179864756763<br>
Epoch 12, Training loss 0.4934278305619955<br>
Epoch 14, Training loss 0.4925368168763816<br>
Epoch 16, Training loss 0.489936351031065<br>
Epoch 18, Training loss 0.48225364279001953<br>
Epoch 20, Training loss 0.41559089563786983<br>
Epoch 22, Training loss 0.47623008105903863<br>
Epoch 24, Training loss 0.47684788253158333<br>
Epoch 26, Training loss 0.48494275493547323<br>
Epoch 28, Training loss 0.46326695576310156<br>
Epoch 30, Training loss 0.39736311135813596<br>
Epoch 32, Training loss 0.14281618097797036<br>
Epoch 34, Training loss 0.06763775760307908<br>
Epoch 36, Training loss 0.03323963325470686<br>
Epoch 38, Training loss 0.03329418990761042<br>
Epoch 40, Training loss 0.04322414346039295<br>
Epoch 42, Training loss 0.029511863365769386<br>
Epoch 44, Training loss 0.022096360959112646<br>
Epoch 46, Training loss 0.16103133413940668<br>
Epoch 48, Training loss 0.38556999355554583<br>
Epoch 50, Training loss 0.18612061651423573<br>
Epoch 52, Training loss 0.026320594605058433<br>
Epoch 54, Training loss 0.0228287323564291<br>
Epoch 56, Training loss 0.032784194946289064<br>
Epoch 58, Training loss 0.03217705018818379<br>
Epoch 60, Training loss 0.10582428770139814<br>
Epoch 62, Training loss 0.34886730622500184<br>
Epoch 64, Training loss 0.3977765748649836<br>
Epoch 66, Training loss 0.45065462989732624<br>
Epoch 68, Training loss 0.22659578237682582<br>
Epoch 70, Training loss 0.4815091462433338<br>
Epoch 72, Training loss 0.4800715918652713<br>
Epoch 74, Training loss 0.3046822458691895<br>
Epoch 76, Training loss 0.13410430723801256<br>
Epoch 78, Training loss 0.026426276881247757<br>
Epoch 80, Training loss 0.018655495941638945</p>

username: SANTOSH_S
cooked: <p>Hi <a class="mention" href="/u/aatiibutt">@aatiibutt</a><br>
Its not good to see train loss increase. You can add <code>Dropout()</code> and <code>batchnorm1d</code> to your linear layers.<br>
Make sure to give <code>shuffle=true</code> in the train dataloaders. Try these changes.<br>
Its always a good practice to have a validation set which evaluates after each epoch.<br>
Let me know if this worked.</p>

username: ptrblck
cooked: <p>Additionally to <a class="mention" href="/u/santosh_s">@SANTOSH_S</a>’ answer, the learning rate might also be too high, as the loss is quite shaky.</p>

username: chhaya_kumar_das
cooked: <p><a class="mention" href="/u/aatiibutt">@aatiibutt</a> If the problem still persists, try to use an LR scheduler</p>