Unexpectedly bad performance on EMNIST Letters dataset

username: Danibahnani
cooked: <p>I am making a model to classify images on a subset of the EMNIST letters dataset. I have 26 classes. I have tried modifying the learning rate and the size of the dataset, but crossentropyloss is still converging around 3.25, and the probabilities for each class are all around 1/26, which essentially means that the model is guessing. I have also tried adding another convolution-pooling block, but I am still getting the same performance.<br>
Data transform:</p>
<pre><code class="lang-auto">image_transform = v2.Compose([
    v2.Resize(28),
    v2.Compose([v2.ToImage(), v2.ToDtype(torch.float32, scale=True)]), # to tensor
    v2.RandomHorizontalFlip(p=1),#100% probability
    v2.RandomRotation(degrees=(90,90)), #flip 90 degrees
    v2.Normalize((0.1736,),(0.3248,))
])
</code></pre>
<p>This is my model:</p>
<pre><code class="lang-auto">        self.conv1 = nn.Conv2d(1,256,5,padding='same')
        self.le_relu1 = nn.LeakyReLU()
        self.mpool1 = nn.MaxPool2d(2)
        self.bn1 = nn.BatchNorm2d(256)

        #Convolution-Pooling block 2:
        self.conv2 = nn.Conv2d(256,128,3,padding='same')
        self.le_relu2 = nn.LeakyReLU()
        self.mpool2 = nn.MaxPool2d(2)

        #output block
        self.flatten = nn.Flatten()
        flattened_dim = int((mpool2_HW**2)*conv2) #flattened dims of height and width
        self.fc1 = nn.Linear(in_features=flattened_dim,out_features=128)
        self.le_relu3 = nn.LeakyReLU()
        self.dropout1 = nn.Dropout()
        self.fco = nn.Linear(in_features=128,out_features=26) #26 categories
</code></pre>
<p>This is my training loop:</p>
<pre><code class="lang-auto">model.train(True) #training mode
running_loss = 0
loss_fn = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(params=model.parameters(),lr=1e-3,momentum=0.9) #have tried lr 1e-4 aswell, and AdamW 

 for i, (X, y) in enumerate(train_dataloader):
            optimizer.zero_grad() 
            X = X.to(device=device)
            y = y.to(device=device)
            X = X.requires_grad_(True)
            preds = model(X) # make predictions
            loss = loss_fn(preds,y) #calculate loss with loss function
            current_loss = loss.item()
            loss.backward()
            optimizer.step() #update params
            #accuracy:
            pred_probabilities = torch.softmax(preds,1)
            most_probable = torch.argmax(pred_probabilities,1) #select highest probability of each class 
            num_correct = (most_probable == y).sum().item()
            accuracy = num_correct/(y.size(0))

</code></pre>
<p>The loss looks like this, reported each batch:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/f/af9cfcad38d91aef903976c3179dbf4597ae2cb0.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/af9cfcad38d91aef903976c3179dbf4597ae2cb0" title="Train_CO_loss"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/a/f/af9cfcad38d91aef903976c3179dbf4597ae2cb0_2_517x271.jpeg" alt="Train_CO_loss" data-base62-sha1="p3xTPnJxJcCM0lilkxQohTcOuUo" width="517" height="271" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/a/f/af9cfcad38d91aef903976c3179dbf4597ae2cb0_2_517x271.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/a/f/af9cfcad38d91aef903976c3179dbf4597ae2cb0_2_775x406.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/a/f/af9cfcad38d91aef903976c3179dbf4597ae2cb0_2_1034x542.jpeg 2x" data-dominant-color="F8F8F7"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Train_CO_loss</span><span class="informations">1920×1009 68.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: Tony-Y
cooked: <aside class="onebox githubblob" data-onebox-src="https://github.com/Tony-Y/pytorch_warmup/blob/master/examples/emnist/main.py">
  <header class="source">

      <a href="https://github.com/Tony-Y/pytorch_warmup/blob/master/examples/emnist/main.py" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/Tony-Y/pytorch_warmup/blob/master/examples/emnist/main.py" target="_blank" rel="noopener nofollow ugc">Tony-Y/pytorch_warmup/blob/master/examples/emnist/main.py</a></h4>


      <pre><code class="lang-py">import argparse
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torchvision import datasets, transforms

import pytorch_warmup as warmup
import os


class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 20, 5, 1)
        self.conv2 = nn.Conv2d(20, 50, 5, 1)
        self.fc1 = nn.Linear(4*4*50, 500)
        self.fc2 = nn.Linear(500, 47)

    def forward(self, x):
</code></pre>



  This file has been truncated. <a href="https://github.com/Tony-Y/pytorch_warmup/blob/master/examples/emnist/main.py" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>This code might help you.</p>