Slight mistake in CNN model

Ayush_Singhal: <p>I was training a simple <code>CNN Model</code> for <code>Binary Image Classification</code>. But I am getting issues, here is the code</p>
<pre><code class="lang-auto">import tqdm
import torch 
import torch.nn as nn 
import os 
import numpy as np 
import tqdm
import seaborn as sns
import pandas as pd 

cats = np.load('/kaggle/input/notebook98410ea555/cats.npy')
dogs = np.load('/kaggle/input/notebook98410ea555/dogs.npy')

cats_tars = np.array([0 for _ in cats])
dogs_tars = np.array([1 for _ in dogs])

r = np.concatenate([cats , dogs])

data = pd.DataFrame({
    'feats' : [val for val in r] , 
    'targs' : np.concatenate([cats_tars , dogs_tars])
})

data = data.sample(frac = 1).reset_index(drop = True)

class mod(nn.Module):
    
    def __init__(self):
        
        super().__init__()
        
        self.conv_1 = nn.Conv2d(3 , 3 , (3 , 3))
        self.conv_2 = nn.Conv2d(3 , 3 , (3 , 3))
        self.conv_3 = nn.Conv2d(3 , 3 , (3 , 3))
        self.conv_4 = nn.Conv2d(3 , 3 , (3 , 3))
        self.max_pool = nn.MaxPool2d(3 , 3)
        
        self.linear_1 = nn.Linear(12288 , 1400)
        self.linear_2 = nn.Linear(1400 , 1)
        
    def forward(self , inps , middle = False):
        
        embs = self.conv_1(inps)
        embs = self.conv_2(embs)
        embs = self.conv_3(embs)
        embs = self.conv_4(embs)
        embs = self.max_pool(embs)
        embs = embs.flatten()
        
        if middle : 
            return embs
        
        embs = self.linear_1(embs)
        embs = torch.sigmoid(embs)
        embs = self.linear_2(embs)
        
        embs = torch.sigmoid(embs)
        
        return embs

model = mod().to('cuda')

optim = torch.optim.SGD(model.parameters() , lr = 0.001)

los = nn.BCELoss()

loss_ = []
o = []

for index in tqdm.tqdm(range(data.shape[0])) , total = data.shape[0]):
    
    inps = torch.tensor(data['feats'][index] , dtype = torch.float32).to('cuda')
    
    out = model(inps)
    o.append(out)
    loss = los(
        out , 
        torch.tensor([data['targs'][index]] , dtype = torch.float32).to('cuda'))
    
    loss.backward()
    optim.step()
    loss_.append(loss)
    
    torch.cuda.empty_cache()    
</code></pre>
<p>The model is not able to learn something at all. The loss is sometimes <code>0</code> and sometimes <code>100</code>. I am not able to understand what slight thing I am doing wrong here</p>
<div class="md-table">
<table>
<thead>
<tr>
<th>What I have tried</th>
<th>Effect</th>
</tr>
</thead>
<tbody>
<tr>
<td>Sending <code>cats/dogs</code> alternative</td>
<td>None</td>
</tr>
<tr>
<td>More shuffling of the data</td>
<td>None</td>
</tr>
<tr>
<td>Different Learning Rate</td>
<td>None</td>
</tr>
<tr>
<td>Different Optimizers → <code>Adam/Adagrad/SGD</code></td>
<td>None</td>
</tr>
<tr>
<td>Using MSE</td>
<td>None</td>
</tr>
<tr>
<td>Using <code>(preds/targs) - (targs/preds)</code></td>
<td>None</td>
</tr>
<tr>
<td>Changing <code>([0 , 2]/[1 , 2])</code> and corresponding changes to BCE formula</td>
<td>None</td>
</tr>
</tbody>
</table>
</div><p>I know that I am doing something slightly wrong, but what it is</p>
<p>The model consists of extra parameter <code>middle</code>, I will be using the <code>CNN Layers</code> as an embedder for a project. <strong><a href="https://www.kaggle.com/datasets/ayushs9020/dogscats-npy-files" rel="noopener nofollow ugc">Here is the Dataset</a></strong></p>

akt42: <p>Observation: your convolution layers do not have activation functions.</p>

Ayush_Singhal: <p>Thanks Akt42 for pointing that out <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>