CNN always outputs the same values whatever the input image

Gerasimos_Delivorias: <p>So my problem is that I try a CNN to learn to classify images of skin cancer as benign or malignant. I feed the images, and whatever the image, I get the same outputs always. I tracked it down and it seems that after the first conv layer, the outputs are the same with every image. And it is not getting any better with more epochs. Is it a problem of the input to the neural network? Because every image i feed it is different, i checked. Or is it something else? Any ideas?</p>
<p>class ConvNet(nn.Module):</p>
<pre><code>def __init__(self):
    super(ConvNet, self).__init__()
    self.conv1 = nn.Conv2d(3, 32, kernel_size=5)
    self.conv2 = nn.Conv2d(32,64, kernel_size=5)
    self.conv3 = nn.Conv2d(64,128, kernel_size=5)
    self.pool = nn.MaxPool2d(3,3)
    self.fc1 = nn.Linear(128*5*5, 512)
    self.fc2 = nn.Linear(512, 256)
    self.fc3 = nn.Linear(256,1)
    self.check = []

def forward(self, x):        
    x = self.pool(F.relu(self.conv1(x)))        
    x = self.pool(F.relu(self.conv2(x)))  
    self.check.append(x)
    if len(self.check) &gt; 1:
        print((self.check[-1] == self.check[-2]).float().sum())
    x = self.pool(F.relu(self.conv3(x)))       
    x = x.view(-1, 128*5*5)
    x = F.relu(self.fc1(x))        
    x = F.relu(self.fc2(x))        
    x = torch.sigmoid(self.fc3(x))
    return x
</code></pre>
<p>model = ConvNet().to(device)<br>
PATH = “CNN_model.pt”<br>
try:<br>
model.load_state_dict(torch.load(PATH))<br>
except:<br>
print(‘Model not found!’)<br>
criterion = nn.BCELoss()<br>
optimizer = optim.Adam(model.parameters(), lr=0.001)</p>
<p>epochs = 50<br>
accuracy = []<br>
batch_size = 1</p>
<p>for epoch in range(epochs):</p>
<pre><code>model.train()
permutation = torch.randperm(len(train_data))

for i in range(0, len(train_imgs[0]), batch_size):
    optimizer.zero_grad()    
    indices = permutation[i:i+batch_size]
    batch_x, batch_y = train_imgs[0, indices], y_train[indices].unsqueeze(0).transpose(0,1).float()
    outputs = model.forward(batch_x)
    loss = criterion(outputs,batch_y)
    loss.backward()
    optimizer.step()       
          
torch.save(model.state_dict(), PATH)
model.eval()
output = model(val_imgs[0])
print(output[0:5])
val_loss = criterion(output, y_val.unsqueeze(0).transpose(0,1).float())
y_pred = (output&gt;0.5).float()
correct = (torch.squeeze(y_pred) == y_val.float()).float().sum()
accuracy += [100 * correct / len(val_data)]
print('Epoch:', epoch, 'Accuracy', accuracy[-1], 'loss', val_loss, '\n')</code></pre>

Dwight_Foster: <p>It may be because you are using the same validation images for every epoch. You might want to put your validation run in a for loop to just like your training run.</p>

Gerasimos_Delivorias: <p>Thanks for your answer. I don’t think that is the reason because the training and val sets are supposed to be the same always and i also tried what you are proposing before and it didn’t work</p>

Dwight_Foster: <p>So what is the accuracy is it always 0 or something else?</p>

ptrblck: <p>For a better numerical stability you could remove the <code>sigmoid</code> and use <code>nn.BCEWithLogitsLoss</code> instead of <code>nn.BCELoss</code>. If this doesn’t change anything, try to overfit a small dataset (e.g. just 10 samples), as your model training might be stuck (and the outputs might just be the <code>bias</code> parameters from your model).</p>

Gerasimos_Delivorias: <p>It’s always like 0.5345. 1 value for all epochs</p>

Gerasimos_Delivorias: <p>I tried BCEwithLogitsLoss it didn’t work. I can’t overfit a dataset because my model is not even training</p>

Gerasimos_Delivorias: <p>dataset = pd.read_csv(‘augmented_data.csv’)</p>
<h1>randomize the dataset</h1>
<p>dataset = dataset.sample(frac=1)</p>
<h1>Only use 20% of the dataset for ease</h1>
<p>mask = np.random.rand(len(dataset)) &lt; 0.02<br>
dataset = dataset[mask]</p>
<h1>split the dataset into training, verification and validation</h1>
<p>mask = np.random.rand(len(dataset)) &lt; 0.8<br>
train_val_data = dataset[mask]<br>
test_data = dataset[~mask]<br>
print(‘test data’, len(test_data))</p>
<p>mask = np.random.rand(len(train_val_data)) &lt; 0.8<br>
train_data = train_val_data[mask]<br>
val_data = train_val_data[~mask]<br>
print(‘train data’, len(train_data))<br>
print(‘validation data’, len(val_data))</p>
<h1>Load all the images for each dataset</h1>
<p>IMG_DIM = (200,200)<br>
train_imgs = []<br>
files_read = 0<br>
start = time.time()<br>
for index, row in train_data.iterrows():<br>
try:<br>
train_imgs.append(img_to_array(load_img(‘padded_images/’ + row[‘image_name’] +’.jpg’, target_size=IMG_DIM)))<br>
except:<br>
train_imgs.append(img_to_array(load_img(‘augmented_images/’ + row[‘image_name’] +’.jpeg’, target_size=IMG_DIM)))<br>
files_read += 1<br>
so_far = time.time()<br>
print_time_remaining(so_far, start, files_read, len(train_data))</p>
<p>val_imgs = []<br>
files_read = 0<br>
start = time.time()<br>
for index, row in val_data.iterrows():<br>
try:<br>
val_imgs.append(img_to_array(load_img(‘padded_images/’ + row[‘image_name’] +’.jpg’, target_size=IMG_DIM)))<br>
except:<br>
val_imgs.append(img_to_array(load_img(‘augmented_images/’ + row[‘image_name’] +’.jpeg’, target_size=IMG_DIM)))<br>
files_read += 1<br>
so_far = time.time()<br>
print_time_remaining(so_far, start, files_read, len(val_data))</p>
<p>test_imgs = []<br>
files_read = 0<br>
start = time.time()<br>
for index, row in test_data.iterrows():<br>
try:<br>
test_imgs.append(img_to_array(load_img(‘padded_images/’ + row[‘image_name’] +’.jpg’, target_size=IMG_DIM)))<br>
except:<br>
test_imgs.append(img_to_array(load_img(‘augmented_images/’ + row[‘image_name’] +’.jpeg’, target_size=IMG_DIM)))<br>
files_read += 1<br>
so_far = time.time()<br>
print_time_remaining(so_far, start, files_read, len(test_data))<br>
y_train = torch.tensor(train_data[‘target’].values)<br>
y_val = torch.tensor(val_data[‘target’].values)<br>
y_test = torch.tensor(test_data[‘target’].values)</p>
<h1>Convert to float</h1>
<p>train_imgs = torch.from_numpy(np.array(train_imgs, dtype=np.float32)).float()<br>
print(‘Train images scaled!’)<br>
val_imgs  = torch.from_numpy(np.array(val_imgs, dtype=np.float32)).float()<br>
print(‘Validation images scaled!’)<br>
test_imgs  = torch.from_numpy(np.array(test_imgs, dtype=np.float32)).float()<br>
print(‘Test images scaled!’)</p>
<h1>divide the pixels by 255 to scale the pixels between 0 and 1</h1>
<p>train_imgs /= 255<br>
val_imgs /= 255<br>
test_imgs /= 255<br>
print(train_imgs.size())<br>
train_imgs = train_imgs.transpose(2,4)<br>
val_imgs = val_imgs.transpose(2,4)<br>
test_imgs = test_imgs.transpose(2,4)<br>
print(train_imgs.size())</p>
<h1>Unsqueeze images</h1>
<p>train_imgs = train_imgs.unsqueeze(0).to(device)<br>
val_imgs = val_imgs.unsqueeze(0).to(device)<br>
test_imgs = test_imgs.unsqueeze(0).to(device)</p>
<p>This is how i load the images btw. I don’t know if there is something wrong with this</p>

ptrblck: <p>Your model is able to overfit random data perfectly:</p>
<pre><code class="lang-python">class ConvNet(nn.Module):
    def __init__(self):
        super(ConvNet, self).__init__()
        self.conv1 = nn.Conv2d(3, 32, kernel_size=5)
        self.conv2 = nn.Conv2d(32,64, kernel_size=5)
        self.conv3 = nn.Conv2d(64,128, kernel_size=5)
        self.pool = nn.MaxPool2d(3,3)
        self.fc1 = nn.Linear(128*5*5, 512)
        self.fc2 = nn.Linear(512, 256)
        self.fc3 = nn.Linear(256,1)
        self.check = []
    
    def forward(self, x):        
        x = self.pool(F.relu(self.conv1(x)))        
        x = self.pool(F.relu(self.conv2(x)))  
        self.check.append(x)
        if len(self.check) &gt; 1:
            print((self.check[-1] == self.check[-2]).float().sum())
        x = self.pool(F.relu(self.conv3(x)))       
        x = x.view(x.size(0), -1)
        x = F.relu(self.fc1(x))        
        x = F.relu(self.fc2(x))        
        x = self.fc3(x)
        return x

model = ConvNet()
data = torch.randn(10, 3, 200, 200)
target = torch.randint(0, 2, (10, 1)).float()
criterion = nn.BCEWithLogitsLoss()    
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)

for epoch in range(100):
    optimizer.zero_grad()    
    outputs = model(data)
    loss = criterion(outputs, target)
    loss.backward()
    optimizer.step()
    print('epoch {}, loss {}'.format(epoch, loss.item()))

acc = ((outputs &gt; 0.0) == target).float().mean()
print(acc)
&gt; tensor(1.)
</code></pre>
<p>so I’m unsure why overfitting a small dataset from your training set doesn’t work.<br>
Maybe try to overfit random data first as well and then compare it to your real data and see if this is still not working. If so, then try to normalize the input data etc.</p>
<p>PS: you can post code snippets by wrapping them into three backticks ```, which makes debugging easier. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>