Loss decrease but test accuracy still about 50%

Gab404: <p>Hello, I’ve tried to create a CNN that classifies musical genres with GTZAN dataset. So my training loss decrease correctly but when I test my CNN with test data, the accuracy still about 50%-60%.</p>
<p>I tried to put dropout at each layer, change hyper-parameter like batch size, learning rate or number of epoch, and I tried so delete some layer, but accuracy still the same. I’ve no more idea.<br>
I don’t think that overfit…</p>
<p>Thank you in advance.</p>
<p>My Network:</p>
<pre><code class="lang-auto">class Net(nn.Module):
  def __init__(self):
    """Intitalize neural net layers"""
    super(Net, self).__init__()
    self.conv1 = nn.Conv2d(in_channels=3, out_channels=8, kernel_size=3, stride=1, padding=0)
    self.conv2 = nn.Conv2d(in_channels=8, out_channels=16, kernel_size=3, stride=1, padding=0)
    self.conv3 = nn.Conv2d(in_channels=16, out_channels=32, kernel_size=3, stride=1, padding=0)
    self.conv4 = nn.Conv2d(in_channels=32, out_channels=64, kernel_size=3, stride=1, padding=0)
    self.conv5 = nn.Conv2d(in_channels=64, out_channels=128, kernel_size=3, stride=1, padding=0)
    self.conv6 = nn.Conv2d(in_channels=128, out_channels=256, kernel_size=3, stride=1, padding=0)
    self.fc1 = nn.Linear(in_features=256, out_features=128)
    self.fc2 = nn.Linear(in_features=128, out_features=10)

    self.batchnorm1 = nn.BatchNorm2d(num_features=8)
    self.batchnorm2 = nn.BatchNorm2d(num_features=16)
    self.batchnorm3 = nn.BatchNorm2d(num_features=32)
    self.batchnorm4 = nn.BatchNorm2d(num_features=64)
    self.batchnorm5 = nn.BatchNorm2d(num_features=128)
    self.batchnorm6 = nn.BatchNorm2d(num_features=256)

    self.dropout = nn.Dropout(p=0.3, inplace=False)

  def forward(self, x):
    # Conv layer 1.
    x = self.conv1(x)
    x = self.batchnorm1(x)
    x = F.relu(x)
    x = F.max_pool2d(x, kernel_size=2)

    # Conv layer 2.
    x = self.conv2(x)
    x = self.batchnorm2(x)
    x = F.relu(x)
    x = F.max_pool2d(x, kernel_size=2)

    # Conv layer 3.
    x = self.conv3(x)
    x = self.batchnorm3(x)
    x = F.relu(x)
    x = F.max_pool2d(x, kernel_size=2)

    # Conv layer 4.
    x = self.conv4(x)
    x = self.batchnorm4(x)
    x = F.relu(x)
    x = F.max_pool2d(x, kernel_size=2)

    # Conv layer 5.
    x = self.conv5(x)
    x = self.batchnorm5(x)
    x = F.relu(x)
    x = F.max_pool2d(x, kernel_size=2)
    
    # Conv layer 6.
    x = self.conv6(x)
    x = self.batchnorm6(x)
    x = F.relu(x)
    x = F.max_pool2d(x, kernel_size=2)

    # Fully connected layer 1.
    x = torch.flatten(x, 1)
    #x = self.dropout(x)
    x = F.relu(self.fc1(x))
    x = self.fc2(x)

    return x
</code></pre>
<p>My train loop:</p>
<pre><code class="lang-auto">model = Net()

# loss function
criterion = nn.CrossEntropyLoss(reduction='mean')
optimizer = torch.optim.SGD(model.parameters(), lr=lr)

#training loop
for epoch in range(num_epochs):
    for i, batch in enumerate(train_loader):
        #forward
        outputs = model(batch)
        optimizer.zero_grad()
        loss = criterion(outputs, train_label[i])
        #backwards
        loss.backward()
        optimizer.step()
    print(f'epoch {epoch + 1} / {num_epochs}, loss = {loss.item():.4f}')
print("End of training")
</code></pre>
<p>And my test loop:</p>
<pre><code class="lang-auto">with torch.no_grad():
    n_correct = 0
    n_samples = 0
    n_class_correct = [0 for i in range(10)]
    n_class_samples = [0 for i in range(10)]
    for i, images in enumerate(test_loader):
        outputs = model(images)
        
        _, predicted = torch.max(outputs.data, 1)
        n_samples += test_label[i].size(0)
        n_correct += (predicted == test_label[i]).sum().item()

        for j in range(batch_size):
            label = test_label[i][j]
            pred = predicted[j]
            if (label == pred):
                n_class_correct[label] += 1
            n_class_samples[label] += 1
            
    acc = 100.0 * n_correct / n_samples
    print(f'Accuracy of the network on the {n_samples} test images: {acc} %')
</code></pre>

carbocation: <p>Before iterating over your test loop, did you remember to put your model into <code>eval</code> mode (by calling <code>model.eval()</code>)?</p>

Gab404: <p>Ok thanks, I forgot, but It decreases accuracy…<br>
By the way, I improve my model and now loss still same and not decrease.</p>
<pre><code class="lang-auto">class Net(nn.Module):
  def __init__(self):
    """Intitalize neural net layers"""
    super(Net, self).__init__()
    self.conv1 = nn.Conv2d(in_channels=3, out_channels=8, kernel_size=3, stride=1, padding=0)
    self.conv2 = nn.Conv2d(in_channels=8, out_channels=16, kernel_size=3, stride=1, padding=0)
    self.conv3 = nn.Conv2d(in_channels=16, out_channels=32, kernel_size=3, stride=1, padding=0)
    self.conv4 = nn.Conv2d(in_channels=32, out_channels=64, kernel_size=3, stride=1, padding=0)
    self.conv5 = nn.Conv2d(in_channels=64, out_channels=128, kernel_size=3, stride=1, padding=0)
    self.conv6 = nn.Conv2d(in_channels=128, out_channels=256, kernel_size=3, stride=1, padding=0)
    self.fc1 = nn.Linear(in_features=256, out_features=128)
    self.fc2 = nn.Linear(in_features=128, out_features=64)
    self.fc3 = nn.Linear(in_features=64, out_features=32)
    self.fc4 = nn.Linear(in_features=32, out_features=10)

    self.batchnorm1 = nn.BatchNorm2d(num_features=8)
    self.batchnorm2 = nn.BatchNorm2d(num_features=16)
    self.batchnorm3 = nn.BatchNorm2d(num_features=32)
    self.batchnorm4 = nn.BatchNorm2d(num_features=64)
    self.batchnorm5 = nn.BatchNorm2d(num_features=128)
    self.batchnorm6 = nn.BatchNorm2d(num_features=256)

    self.dropout = nn.Dropout(p=0.3, inplace=False)

  def forward(self, x):
    # Conv layer 1.
    x = self.conv1(x)
    x = F.relu(x)
    x = self.batchnorm1(x)
    x = F.max_pool2d(x, kernel_size=2)

    # Conv layer 2.
    x = self.conv2(x)
    x = F.relu(x)
    x = self.batchnorm2(x)
    x = F.max_pool2d(x, kernel_size=2)

    # Conv layer 3.
    x = self.conv3(x)
    x = self.batchnorm3(x)
    x = F.relu(x)
    x = F.max_pool2d(x, kernel_size=2)

    # Conv layer 4.
    x = self.conv4(x)
    x = F.relu(x)
    x = self.batchnorm4(x)
    x = F.max_pool2d(x, kernel_size=2)
    
    # Conv layer 5.
    x = self.conv5(x)
    x = F.relu(x)
    x = self.batchnorm5(x)
    x = F.max_pool2d(x, kernel_size=2)
    
    # Conv layer 6.
    x = self.conv6(x)
    x = F.relu(x)
    x = self.batchnorm6(x)
    x = F.max_pool2d(x, kernel_size=2)

    # Fully connected layer 1.
    x = torch.flatten(x, 1)
    x = self.dropout(x)
    x = F.relu(self.fc1(x))
    x = F.relu(self.fc2(x))
    x = F.relu(self.fc3(x))
    x = self.fc4(x)
    x = F.softmax(x)

    return x
</code></pre>

carbocation: <p>Just to double check - now that you are putting your model into <code>.eval()</code> mode when looking at your test data, are you putting it back into <code>.train()</code> mode before you run your next iteration of the training loop?</p>

Gab404: <p>So I train my model then I put my model into .eval and I test my model, and that’s all.<br>
Is it not good ?</p>

carbocation: <p>Yes, that is correct. I am just checking because if you were testing a validation set during training, you’d want to put your model into .eval when looking at the validation set but then be sure to put it back into .train before the next epoch of training.</p>

Gab404: <p>I‘ve not validation set, what is it for ?</p>