Binary classification in CNN

Tornike: <p>Hello, maybe it’s easy but it is very confusing to me.</p>
<p>So doing binary classification with BCEWithlogitsloss.</p>
<p>this is my Model:</p>
<pre><code class="lang-auto">class BreastCancerModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1=nn.Conv2d(3,6,3,1)
        self.conv2=nn.Conv2d(6,10,3,1)
        self.conv3=nn.Conv2d(10,13,3,1)
        self.fc1=nn.Linear(13*4*4,84)
        self.fc2=nn.Linear(84,10)
        self.fc3=nn.Linear(10,1)
    
    def forward(self,X):
        X=X.view(-1,3,50,50)
        X=F.relu(self.conv1(X))
        X=F.max_pool2d(X,2,2)
        X=F.relu(self.conv2(X))
        X=F.max_pool2d(X,2,2)
        X=F.relu(self.conv3(X))
        X=F.max_pool2d(X,2,2)
        X=X.view(-1,13*4*4)
        X=F.relu(self.fc1(X))
        X=F.relu(self.fc2(X))
        X=self.fc3(X)
        return X
        
           
</code></pre>
<p>This is Training loop :</p>
<pre><code class="lang-auto">def train(optim,criterion,num_epochs):
    train_correct=[]
    train_losses=[]
    test_correct=[]
    test_losses=[]
    for i in range(num_epochs):
        trn_corr=0
        tst_corr=0

        for b,(X_train,y_train) in enumerate(train_loader):
            b+=1

            y_pred=model(X_train)
            print('ypred')
            print(y_pred)
            loss=criterion(y_pred,y_train.unsqueeze(1).float())
            print('ytrain')
            print(y_train.unsqueeze(1))
            #get the number of correct predictions
            predicted = torch.round((y_pred.data)[1])
            print('predicted')
            print(predicted)
            batch_corr = (predicted == y_train).sum()
            trn_corr += batch_corr

            #update Parameters
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()

#             if b%1000 == 0:
        print(f'epoch: {i:2}/{num_epochs} loss: {loss.item():10.8f}  \
        accuracy: {trn_corr.item()*100/(64*b):7.3f}%')

        train_losses.append(loss)
        train_correct.append(trn_corr)

    with torch.no_grad():
        for b,(X_test,y_test) in enumerate(val_loader):
            y_val=torch.sigmoid(model(X_test))
            loss=criterion(y_val,y_test.unsqueeze(1).float())
            predicted=round(y_val.data)[1]
            tst_corr+=(predicted==y_test).sum()
        test_losses.append(loss)
        test_correct.append(tst_corr)

   
    
    
    
</code></pre>
<p>This is THe output of first batch and its very confusing to me. Should i be doing training loop differently ?</p>
<pre><code class="lang-auto">ypred
tensor([[0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236],
        [0.0236]], grad_fn=&lt;AddmmBackward&gt;)
ytrain
tensor([[1.],
        [0.],
        [1.],
        [1.],
        [0.],
        [1.],
        [1.],
        [0.],
        [0.],
        [1.],
        [0.],
        [0.],
        [1.],
        [0.],
        [1.],
        [0.],
        [1.],
        [0.],
        [1.],
        [1.],
        [0.],
        [0.],
        [1.],
        [1.],
        [0.],
        [0.],
        [1.],
        [1.],
        [0.],
        [0.],
        [1.],
        [0.],
        [1.],
        [0.],
        [0.],
        [0.],
        [1.],
        [1.],
        [0.],
        [0.],
        [0.],
        [0.],
        [1.],
        [0.],
        [0.],
        [0.],
        [0.],
        [0.],
        [1.],
        [1.],
        [0.],
        [1.],
        [0.],
        [0.],
        [1.],
        [1.],
        [1.],
        [1.],
        [1.],
        [0.],
        [1.],
        [0.],
        [1.],
        [1.]])
predicted
tensor([0.]
</code></pre>
<p>What should i change in training loop for accuracy to be correct?<br>
Now my accuracy is 50 %.</p>

eqy: <p>I think there may be an issue with the training data here as I cannot reproduce this output with random input:</p>
<pre><code class="lang-auto">net = BreastCancerModel().cuda()
x = torch.randn((8, 3, 50, 50), device='cuda')
print(net(x))
</code></pre>
<pre><code class="lang-auto">tensor([[0.0419],
            [0.0497],
            [0.0391],
            [0.0421],
            [0.0439],
            [0.0445],
            [0.0450],
            [0.0524]], device='cuda:0', grad_fn=&lt;AddmmBackward&gt;) 
</code></pre>
<p>Can you print an example of the input to the model (e.g., the first batch)?</p>

Tornike: <p>Thanks for your reply.</p>
<p>Here is first batch:</p>
<pre><code class="lang-auto">tensor([[0.2174],
        [0.3320],
        [0.2788],
        [0.4845],
        [0.1838],
        [0.2834],
        [0.2081],
        [0.2673],
        [0.1727],
        [0.2490],
        [0.4976],
        [0.2831],
        [0.2340],
        [0.3234],
        [0.2539],
        [0.2469],
        [0.1592],
        [0.3147],
        [0.4204],
        [0.3544],
        [0.3064],
        [0.1808],
        [0.2562],
        [0.3065],
        [0.0838],
        [0.2785],
        [0.3098],
        [0.2009],
        [0.2919],
        [0.2356],
        [0.3093],
        [0.2793],
        [0.2551],
        [0.3788],
        [0.2043],
        [0.3354],
        [0.2083],
        [0.3548],
        [0.3419],
        [0.4007],
        [0.2365],
        [0.3014],
        [0.2157],
        [0.2639],
        [0.4706],
        [0.2764],
        [0.3063],
        [0.3154],
        [0.3702],
        [0.2395],
        [0.2813],
        [0.3398],
        [0.3706],
        [0.3421],
        [0.2573],
        [0.2464],
        [0.3041],
        [0.2284],
        [0.3122],
        [0.2555],:
        [0.2161],:
        [0.2907],
        [0.1727],
        [0.3898]]
</code></pre>
<p>All of them are 0 class but i did many things while processing data. Data is well balanced and every image is readable.</p>
<p>Should i do more in processing?</p>

eqy: <p>That seems like the model output rather than the model input. However, it looks different from what you posted before (the same value repeated). If you can verify that the starting loss is about 0.693 then it suggests the model itself is probably fine.</p>

Tornike: <p>It is maybe because of model parameters.</p>
<pre><code class="lang-auto">def count_parameters(model):
    params = [p.numel() for p in model.parameters() if p.requires_grad]
    for item in params:
        print(f'{item:&gt;6}')
    print(f'______\n{sum(params):&gt;6}')

count_parameters(model)
</code></pre>
<blockquote>
<p>162<br>
6<br>
540<br>
10<br>
1170<br>
13<br>
17472<br>
84<br>
840<br>
10<br>
20<br>
2<br>
–<br>
20329</p>
</blockquote>
<hr>
<p>20000 parameters might be very low for this Task.</p>
<p>I think i just need to raise these parameters.</p>
<p>Thanks for your Reply.</p>