I am getting the same loss and accuracy for all epochs.

Lana: <p>I a beginner in Deep Learning, especially CNN. I have a dataset of the handwritten words - “TRUE”, “FALSE” and “NONE”.I want my model to recognize the words and put them in the correct class. I created a simple CNN model copying the TinyVGG architecture. However, the loss and accuracy values are the same for all epochs. I tried changing the batch size, epochs and learning rate and it’s not working. Since I am new to this concept and this is being my first project I do not know what changes to be made to the model.</p>
<pre><code class="lang-auto">class CUSTOMDATASETV2(nn.Module):
  
    def __init__(self, input_shape: int, hidden_units: int, output_shape: int):
        super().__init__()
        self.block_1 = nn.Sequential(
            nn.Conv2d(in_channels=input_shape, 
                      out_channels=hidden_units, 
                      kernel_size=3, # how big is the square that's going over the image?
                      stride=1, # default
                      padding=1),# options = "valid" (no padding) or "same" (output has same shape as input) or int for specific number 
            nn.ReLU(),
            nn.Conv2d(in_channels=hidden_units, 
                      out_channels=hidden_units,
                      kernel_size=3,
                      stride=1,
                      padding=1),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2,
                         stride=2) # default stride value is same as kernel_size
        )
        self.block_2 = nn.Sequential(
            nn.Conv2d(hidden_units, hidden_units, 3, padding=1),
            nn.ReLU(),
            nn.Conv2d(hidden_units, hidden_units, 3, padding=1),
            nn.ReLU(),
            nn.MaxPool2d(2)
        )
        self.classifier = nn.Sequential(
            nn.Flatten(),
            # Where did this in_features shape come from? 
            # It's because each layer of our network compresses and changes the shape of our inputs data.
            nn.Linear(in_features=hidden_units*16*16, 
                      out_features=output_shape)
        )
    
    def forward(self, x: torch.Tensor):
        x = self.block_1(x)
        # print(x.shape)
        x = self.block_2(x)
        # print(x.shape)
        x = self.classifier(x)
        # print(x.shape)
        return x

torch.manual_seed(42)
model_2 = CUSTOMDATASETV2(input_shape=3, 
    hidden_units=10, 
    output_shape=len(class_names)).to(device)
model_2


x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x

from tqdm.auto import tqdm
torch.manual_seed(42)

# Measure time
from timeit import default_timer as timer
train_time_start_model_2 = timer()

# Train and test model 
epochs = 10
for epoch in tqdm(range(epochs)):
    print(f"Epoch: {epoch}\n---------")
    train_step(data_loader=train_dataloader, 
        model=model_2, 
        loss_fn=loss_fn,
        optimizer=optimizer,
        accuracy_fn=accuracy_fn,
        device=device
    )
    test_step(data_loader=test_dataloader,
        model=model_2,
        loss_fn=loss_fn,
        accuracy_fn=accuracy_fn,
        device=device
    )

train_time_end_model_2 = timer()
total_train_time_model_2 = print_train_time(start=train_time_start_model_2,
                                           end=train_time_end_model_2,
                                           device=device)

</code></pre>

Aniruth_Sundararaja1: <p>These are the potential issues you could have</p>
<ol>
<li>
<p>Your learning rate is too low and so model is not learning any more .  Since you said you have tried this already , leave this. however , do try initializing it and then running a scheduler to update its value over epochs.</p>
</li>
<li>
<p>Dataset : Check your data pre processing , this error might occur if you failed to normalize your data.</p>
</li>
<li>
<p>Model Complexity : Your model is not complex enough to handle the given dataset.<br>
In that case , try adding batch normalization layers after every conv block.</p>
</li>
</ol>
<p>class CustomDatasetV2(nn.Module):</p>
<pre><code>def __init__(self, input_shape: int, hidden_units: int, output_shape: int):
    super().__init__()
    self.block_1 = nn.Sequential(
        nn.Conv2d(in_channels=input_shape, 
                  out_channels=hidden_units, 
                  kernel_size=3,
                  stride=1,
                  padding=1),
        nn.BatchNorm2d(hidden_units),
        nn.ReLU(),
        nn.Conv2d(in_channels=hidden_units, 
                  out_channels=hidden_units,
                  kernel_size=3,
                  stride=1,
                  padding=1),
        nn.BatchNorm2d(hidden_units),
        nn.ReLU(),
        nn.MaxPool2d(kernel_size=2,
                     stride=2)
    )
    self.block_2 = nn.Sequential(
        nn.Conv2d(hidden_units, hidden_units, 3, padding=1),
        nn.BatchNorm2d(hidden_units),
        nn.ReLU(),
        nn.Conv2d(hidden_units, hidden_units, 3, padding=1),
        nn.BatchNorm2d(hidden_units),
        nn.ReLU(),
        nn.MaxPool2d(2)
    )
    self.classifier = nn.Sequential(
        nn.Flatten(),
        nn.Dropout(0.5),
        nn.Linear(in_features=hidden_units*16*16, 
                  out_features=output_shape)
    )

def forward(self, x: torch.Tensor):
    x = self.block_1(x)
    x = self.block_2(x)
    x = self.classifier(x)
    return x
</code></pre>
<ol start="3">
<li>Its also possible that your data is too simple for this model .In that case try simplifying this model (although it does not look like that with your model)</li>
</ol>

Lana: <p>I tried 1 and 3 of your suggestions and it made some good results, so thank you for that.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/0/d05069e4ef2225878b5a1ef747061819a617c313.png" data-download-href="https://discuss.pytorch.org/uploads/default/d05069e4ef2225878b5a1ef747061819a617c313" title="4-OUTPUT"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/d/0/d05069e4ef2225878b5a1ef747061819a617c313_2_690x354.png" alt="4-OUTPUT" data-base62-sha1="tIPB9IF1w6CZphEkbKdAvX6zDEv" width="690" height="354" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/d/0/d05069e4ef2225878b5a1ef747061819a617c313_2_690x354.png, https://discuss.pytorch.org/uploads/default/optimized/3X/d/0/d05069e4ef2225878b5a1ef747061819a617c313_2_1035x531.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/d/0/d05069e4ef2225878b5a1ef747061819a617c313.png 2x" data-dominant-color="F8F8F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">4-OUTPUT</span><span class="informations">1214×624 80.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
The current best test accuracy is 78.12 %. How do I improve ?</p>