3D CNN accuracy and Loss function are almost stable

Anna_yah: <p>I used Pytorch to create 3D CNN with 2 conv layer<br>
I used 1000 epochs as shown in the curve but the accuracy and the loss values are almost stable.<br>
can you explain the reason to me please ?</p>
<pre><code class="lang-auto">class CNNModel(nn.Module):
    def __init__(self): 
        super(CNNModel, self).__init__() # héritage
        
        self.conv_layer1 = self._conv_layer_set(3, 32)
        self.conv_layer2 = self._conv_layer_set(32, 64) 
        self.fc1 = nn.Linear(64*28*28*28, 2) 
        self.fc2 = nn.Linear(1404928, num_classes) 
        self.relu = nn.LeakyReLU()
        self.batch=nn.BatchNorm1d(2)
        self.drop=nn.Dropout(p=0.15, inplace = True)   
        
    def _conv_layer_set(self, in_c, out_c):
        conv_layer = nn.Sequential(
        nn.Conv3d(in_c, out_c, kernel_size=(3, 3, 3), padding=0),
        nn.LeakyReLU(),
        nn.MaxPool3d((2, 2, 2)),
        )
        return conv_layer
    

    def forward(self, x):
        # Set 1
        out = self.conv_layer1(x)
        out = self.conv_layer2(out)
        out = out.view(out.size(0), -1) 
        out = self.fc1(out)
        out = self.relu(out)
        out = self.batch(out)
        out = self.drop(out)
        out = F.softmax(out, dim=1) 
        return out

# Create CNN
model = CNNModel()
model.cuda() #pour utiliser   GPU
print(model)
# Cross Entropy Loss 
for param in model.parameters():
    param.requires_grad = True
    error = nn.CrossEntropyLoss()
# SGD Optimizer
learning_rate = 0.001 
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate) 
###################################################accuracy function ##################################
def accuracyCalc (predicted, targets):
    correct = 0
    p = predicted.tolist()
    t = targets.flatten().tolist() 
    for i in range(len(p)):
        if (p[i] == t[i]):
            correct +=1
    accuracy = 100 * correct / targets.shape[0]
    return(accuracy)
#######################################################################################################
print(" build model --- %s seconds ---" % (time.time() - start_time))
#######################################################{{{{{{{training}}}}}}}##################################
print('data preparation ')
training_data = np.load("/content/drive/My Drive/brats6G/train/training_data.npy", allow_pickle=True)
training_data = training_data[:2]
targets = np.load("/content/drive/My Drive/brats6G/train/targets.npy", allow_pickle=True)
targets = targets[:2]
from sklearn.utils import shuffle
training_data, targets = shuffle(training_data, targets)

training_data = changechannel(training_data, 1, 5) #Channels ordering : first channel to ==&gt; last channel'
training_data  = resize3Dimages(training_data) #resize images
training_data = channel1to3(training_data,)#1 channel to 3 channel ===&gt; RGB
training_data = changechannel(training_data, 4, 1)# last to first

#Definition of hyperparameters
num_epochs = 5
loss_list_train = []
accuracy_list_train = []
for epoch in range(num_epochs): 
    outputs = []
    outputs= torch.tensor(outputs).cuda()
    for fold in range(0, len(training_data), 4):
        xtrain = training_data[fold : fold+4]
        xtrain =torch.tensor(xtrain).float().cuda() 
        xtrain = xtrain.view(2, 3, 120, 120, 120)
        # Clear gradients
        optimizer.zero_grad()
        # Forward propagation
        v = model(xtrain)
        outputs = torch.cat((outputs,v.detach()),dim=0)
      
    targets = torch.Tensor(targets)
    labels = targets.cuda()
    outputs = torch.tensor(outputs,  requires_grad=True) 
    _, predicted = torch.max(outputs, 1)
    accuracy = accuracyCalc(predicted, targets)
    labels = labels.long() 
    labels=labels.view(-1) 
    loss = nn.CrossEntropyLoss()
    loss = loss(outputs, labels)    
    # Calculating gradients
    loss.backward()
    # Update parameters
    optimizer.step()
    loss_list_train.append(loss.data) #loss values loss
    accuracy_list_train.append(accuracy/100)
    np.save('/content/drive/My Drive/brats6G/accuracy_list_train.npy', np.array(accuracy_list_train))
    np.save('/content/drive/My Drive/brats6G/loss_list_train.npy', np.array(loss_list_train))
    print('Iteration: {}/{}  Loss: {}  Accuracy: {} %'.format(epoch+1,  num_epochs, loss.data, accuracy))
print('Model training  : Finished')
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/c/bc225af9d5222e019c34f6788fc99ed3fd806ca4.png" data-download-href="https://discuss.pytorch.org/uploads/default/bc225af9d5222e019c34f6788fc99ed3fd806ca4" title="122862268_647997725913779_12410710553506702_n"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/c/bc225af9d5222e019c34f6788fc99ed3fd806ca4_2_690x405.png" alt="122862268_647997725913779_12410710553506702_n" data-base62-sha1="qQjmYbmsvHKed7AWoqbp7OWnZbu" width="690" height="405" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/c/bc225af9d5222e019c34f6788fc99ed3fd806ca4_2_690x405.png, https://discuss.pytorch.org/uploads/default/original/3X/b/c/bc225af9d5222e019c34f6788fc99ed3fd806ca4.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/b/c/bc225af9d5222e019c34f6788fc99ed3fd806ca4.png 2x" data-dominant-color="CCD4CB"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">122862268_647997725913779_12410710553506702_n</span><span class="informations">753×442 77.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Remove the <code>nn.Softmax</code> at the end of the model, since <code>nn.CrossEntropyLoss</code> will internally apply <code>F.log_softmax</code> and <code>nn.NLLLoss</code>.</p>
<p>Also, probably unrelated to the training issue, but your linear layers are quite big.<br>
I’m also a bit skeptical about using a dropout layer at the end of the model as it would mask logits.</p>

Anna_yah: <p>Thank you for your reply <a class="mention" href="/u/ptrblck">@ptrblck</a><br>
I set the linear layer in_features to 64 * 28 * 2 * 28 because the input image size is 120 * 120 * 120 and I have two conv layers. So how can I reduce the linear layer size and add another linear layer after the dopout layer?<br>
Than you in advance.</p>

ptrblck: <p>You could reduce the spatial size of the activation with pooling layers or a generally deeper architecture.<br>
However, I would focus on the other two points, i.e. the softmax layer and dropout at the end of the model.<br>
Did you remove them and did anything change?</p>

Anna_yah: <p>I deleted them but the result = 0.5 for all epochs (200)</p>
<pre><code class="lang-auto">class CNNModel(nn.Module):
    def __init__(self):
        super(CNNModel, self).__init__() 
        
        self.conv_layer1 = self._conv_layer_set(3, 32)
                                                      
        self.conv_layer2 = self._conv_layer_set(32, 64) 
        self.fc1 = nn.Linear(64*28*28*28, 2)
        self.relu = nn.LeakyReLU()
        self.batch=nn.BatchNorm1d(2)
  
        
    def _conv_layer_set(self, in_c, out_c):
        conv_layer = nn.Sequential(
        nn.Conv3d(in_c, out_c, kernel_size=(3, 3, 3), padding=0),
        nn.LeakyReLU(),
        nn.MaxPool3d((2, 2, 2)),
        )
        return conv_layer
    

    def forward(self, x):
        # Set 1
        out = self.conv_layer1(x)
        out = self.conv_layer2(out)
        out = out.view(out.size(0), -1)
        out = self.fc1(out)
        out = self.relu(out)
        out = self.batch(out) 
        return out

</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/5/d563eda08cace318d6bcbcda7a4fbaefeccf97d8.png" data-download-href="https://discuss.pytorch.org/uploads/default/d563eda08cace318d6bcbcda7a4fbaefeccf97d8" title="ammm"><img src="https://discuss.pytorch.org/uploads/default/original/3X/d/5/d563eda08cace318d6bcbcda7a4fbaefeccf97d8.png" alt="ammm" data-base62-sha1="urJNnz6W7bIhGVRyQIA6lL7vpvq" width="690" height="398" data-dominant-color="EAEBF1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">ammm</span><span class="informations">730×422 9.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>In that case you could try to overfit a small dataset, e.g. just 10 samples, and make sure your model is able to do so by playing around with hyperparameters.<br>
If that’s still not working, there might be other issues I haven’t seen yet.</p>

Anna_yah: <aside class="quote no-group" data-username="ptrblck" data-post="6" data-topic="100909">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>In that case you could try to overfit a small dataset, e.g. just 10 samples, and make sure your model is able to do so by playing around with hyperparameters.</p>
</blockquote>
</aside>
<p>I am only using 4 images ( 2 images as training set and 2 images as validation set)</p>

Anna_yah: <p>I did not find a solution to this problem so far</p>

ptrblck: <p>Your model works fine with 10 random samples and achieves a perfect accuracy after a few steps:</p>
<pre><code class="lang-python">device = 'cuda'
model = CNNModel().to(device)
data = torch.randn(10, 3, 120, 120, 120).to(device)
target = torch.randint(0, 2, (10,)).to(device)

optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
criterion = nn.CrossEntropyLoss()

for epoch in range(100):
    optimizer.zero_grad()
    output = model(data)
    loss = criterion(output, target)
    loss.backward()
    optimizer.step()
    acc = (torch.argmax(output, 1) == target).float().mean()
    print('epoch {}, loss {}, acc {}'.format(epoch, loss.item(), acc))

&gt; epoch 0, loss 0.707537829875946, acc 0.5
epoch 1, loss 0.25594037771224976, acc 0.9000000357627869
epoch 2, loss 0.18612369894981384, acc 1.0
epoch 3, loss 0.18332494795322418, acc 1.0
epoch 4, loss 0.18075349926948547, acc 1.0
epoch 5, loss 0.17838451266288757, acc 1.0
</code></pre>
<p>so I would still recommend to play around with some hyperparameters and make sure your model can overfit the tiny dataset.</p>

Anna_yah: <p>Thank you very much <a class="mention" href="/u/ptrblck">@ptrblck</a></p>