Calculation of ConfusionMetrics for binary classification

username: Fathima
cooked: <p>This is my CM class</p>
<pre><code class="lang-auto">class ConfusionMetrics():
 
    def __init__(self, threshold=0.5, apply_sigmoid=False, device='cpu'):
        self.threshold=threshold
        self.__matrix = torch.zeros(2,2).to(device)#rows are output
        self.__apply_sigmoid = apply_sigmoid

    def __label(self,y):
        """
        convert the output into a binary label
        """
        pred_label = y.sigmoid() if self.__apply_sigmoid else y

        output = (pred_label&gt;self.threshold).type(y.type())
        
        return output
    
    def matrix(self):
        return self.__matrix

    def accuracy(self):
        return self.__matrix.diagonal().sum()/self.__matrix.sum()

    def se(self):
        print('to implement')
        return 
    def ppv(self):
        print('to implement')
        return
    def f1(self):
        print('to implement')
        return 

    #accumulation and reset#
    def accumulate(self, targets, outputs):
        """
        accumulate the confusion matrix over the 
        """
        #from output to label
        pred_label = self.__label(outputs)
        for i in range(targets.size(0)):
            row, col = int(targets[i]), int(pred_label[i])
            #print(row,col)
            self.__matrix[row,col]+=1
    

    def reset(self):
        """
        reset the confusion matrix to 0 
        """
        self.__matrix *= 0
</code></pre>
<p>This the my training loop</p>
<pre><code class="lang-auto">import numpy as np
batch_size = 32
epochs = 5
min_valid_loss = np.inf
min_valid_loss = np.inf # track change in validation loss

min_valid_loss = np.Inf # track change in validation loss
train_losses, valid_losses = [], []


M = ConfusionMetrics()

for e in range(epochs):
    train_loss = 0.0
    model.train()     # Optional when not using Model Specific layer
    for data, target in train_loader:
         # Transfer Data to GPU if available
        if torch.cuda.is_available():
            data, target = data.cuda(), target.cuda()
            #print("targets.data", targets.item())
        
        
         # Clear the gradients
        optimizer.zero_grad()
        
        # Forward Pass
        output = (model(data))
        squeezed_output = torch.squeeze(output)
        #print(squeezed_output, target)
        #print(squeezed_output.size(), target.size())
        
        
        # Find the Loss
        loss = criterion(squeezed_output.view(-1,1),target.view(-1,1))
        
        # Calculate gradients 
        loss.backward()
        # Update Weights
        optimizer.step()
        # Calculate Loss
        train_loss += loss.item()
    
    valid_loss = 0.0
    model.eval()     # Optional when not using Model Specific layer
    for data, target in valid_loader:
        if torch.cuda.is_available():
            data, target = data.cuda(), target.cuda()
        
        output = model(data)
        squeezed_output = torch.squeeze(output)
        print("=====LOSS===========",target)
        
        #loss = criterion(squeezed_output,target)
        loss = criterion(squeezed_output.view(-1,1),target.view(-1,1))
        valid_loss = loss.item() * data.size(0)
        
        # calculate average losses
        #train_loss = train_loss/len(train_loader.dataset)
        train_losses.append(train_loss)
        
        #valid_loss = valid_loss/len(valid_loader.dataset)
        valid_losses.append(valid_loss)
        
        print('total confusion matrix')
        matrix = M.matrix()
        print('Accuracy', M.matrix())
        accuracy = M.accuracy()
        print('Accuracy', M.accuracy())
        accumulate = M.accumulate()
        print('accumulate',M.accumulate())
        reset = M.reset()
        print('confusion matrix',M.reset)

    print(f'Epoch {e+1} \t\t Training Loss: {train_loss / len(train_loader)} \t\t Validation Loss: {valid_loss / len(valid_loader)}')
   
   

    if min_valid_loss &gt; valid_loss:
        print(f'Validation Loss Decreased({min_valid_loss:.6f}---&gt;{valid_loss:.6f}) \t Saving The Model')
        min_valid_loss = valid_loss
        
</code></pre>
<p>This the error I get. I am training a 1D signal with binary classification. can someone help to fix?</p>
<pre><code class="lang-auto">=====LOSS=========== tensor([1., 0., 1., 0., 1., 0., 1., 0., 1., 0., 1., 0., 1., 0., 1., 0., 1., 0.,
        1., 1., 1., 0., 1., 0., 1., 0., 1., 0., 1., 0., 1., 0.])
total confusion matrix
Accuracy tensor([[0., 0.],
        [0., 0.]])
Accuracy tensor(nan)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
&lt;ipython-input-100-ce5e897c698c&gt; in &lt;module&gt;
     67         accuracy = M.accuracy()
     68         print('Accuracy', M.accuracy())
---&gt; 69         accumulate = M.accumulate()
     70         print('accumulate',M.accumulate())
     71         reset = M.reset()

TypeError: accumulate() missing 2 required positional arguments: 'targets' and 'outputs'
</code></pre>

username: ptrblck
cooked: <p>The error is raised in:</p>
<pre><code class="lang-python">---&gt; 69         accumulate = M.accumulate()

TypeError: accumulate() missing 2 required positional arguments: 'targets' and 'outputs'
</code></pre>
<p>as <code>accumulate</code> expects to get two input arguments: <code>targets</code> and <code>outputs</code> as the error message claims:</p>
<pre><code class="lang-python">def accumulate(self, targets, outputs):
</code></pre>
<p>so make sure to pass these tensors to this method.</p>

username: Fathima
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="153087">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>pass these tensors to this method.</p>
</blockquote>
</aside>
<pre><code class="lang-auto">---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
&lt;ipython-input-44-5830167d59f9&gt; in &lt;module&gt;
     67         print('confusion matrix', M.matrix())
     68         print('Accuracy', M.accuracy())
---&gt; 69         targets ,output  = M.accumulate()
     70         print('accumulate',M.accumulate(targets ,output))
     71         reset = M.reset()

TypeError: accumulate() missing 2 required positional arguments: 'targets' and 'outputs'

</code></pre>
<p>I still get this error. what I am trying is to calculate the performance matric. I want to call the function here but I really don’t understand how to. Could you provide the right code I should use?</p>

username: ptrblck
cooked: <p>You have to pass the two needed arguments as:</p>
<pre><code class="lang-python">M.accumulate(targets, outputs)
</code></pre>

username: Fathima
cooked: <p>It says target not defined. The confusion matrix class has target already.</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
&lt;ipython-input-46-f772ac90dea1&gt; in &lt;module&gt;
     66         print('confusion matrix', M.matrix())
     67         print('Accuracy', M.accuracy())
---&gt; 68         targets ,output  = M.accumulate(targets,output)
     69         #reset = M.reset()
     70         #print('confusion matrix',M.reset)

NameError: name 'targets' is not defined

</code></pre>