Evaluation of a CNN?

username: SoulHokib
cooked: <p>Hey guys !</p>
<p>I am currently working on a CNN using a public dataset in order to make classification between two type of images : images containing brain hemorraghies, and images that doesn’t.</p>
<p>I am trying to evaluate my model, but was wondering how to implement various things, like accuracy, precision, recall, specificity, etc.<br>
So two things : are these methods the best way to evaluate CNN ? How to implement it correctly ? I was wondering if I could use sklearn metrics or if it would be easier to write functions in PyTorch.</p>
<p>My way to calculate accuracy at the moment seems like not working, as it doesn’t update at all.<br>
Here’s my code (definition of model and training).</p>
<pre><code class="lang-auto">class Net(Module):   
    def __init__(self):
        super(Net, self).__init__()

        self.cnn_layers = Sequential(
            # Defining a 2D convolution layer
            Conv2d(3, 4, kernel_size=3, stride=1, padding=1),
            BatchNorm2d(4),
            ReLU(inplace=True),
            MaxPool2d(kernel_size=2),
            # Defining another 2D convolution layer
            Conv2d(4, 4, kernel_size=4, stride=1, padding=1),
            BatchNorm2d(4),
            ReLU(inplace=True),
            MaxPool2d(kernel_size=2),
        )

        self.linear_layers = Sequential(
            Linear(4 * 49 * 49, 1)
        )

    # Defining the forward pass    
    def forward(self, x):
        x = self.cnn_layers(x)
        x = x.view(x.size(0), -1)
        x = self.linear_layers(x)
        return x
        
# defining the model
model = Net()
model = model.to('cuda')
# defining the optimizer
optimizer = Adam(model.parameters(), lr=0.07)
# defining the loss function
criterion = BCEWithLogitsLoss()
# checking if GPU is available
if torch.cuda.is_available():
    model = model.cuda()
    criterion = criterion.cuda()
    
from torchsummary import summary

#vgg = models.vgg16()
#summary(model.cuda(), (3, 200, 200))
print(model)

</code></pre>
<pre><code class="lang-auto">#Fonction d'entraînement actuelle.
def train2(epoch):
    model.train()
    tr_loss = 0
    total = 0
    correct = 0
    target_true = 0
    predicted_true = 0
    correct_true = 0
    X_train = X
    y_train = y
    # converting the data into GPU format
    if torch.cuda.is_available():
        X_train = X_train.cuda()
        y_train = y_train.cuda()
    
    # clearing the Gradients of the model parameters
    optimizer.zero_grad()
    # prediction for training and validation set
    output_train = model(X_train)
    # computing the training and validation loss
    loss_train = criterion(output_train, y_train.unsqueeze(1).type_as(output_train))
    train_losses.append(loss_train)
    total += y_train.size(0)
    correct += (output_train.argmax(-1) == y_train).sum().item()
    # computing the updated weights of all the model parameters
    loss_train.backward()
    optimizer.step()
    tr_loss = loss_train.item()
    if epoch%2 == 0:
        # printing the validation loss
        print('Epoch : ',epoch+1, '\t', 'loss : ', tr_loss, 'accuracy : ', (correct / total)*100)
        #print('Accuracy : ', accuracy_score(output_train.cpu().detach().numpy(), y_train.cpu().detach().numpy()))
</code></pre>

username: SANTOSH_S
cooked: <p>hi <a class="mention" href="/u/soulhokib">@SoulHokib</a></p>
<p>You can draw the confusion matrix as given here:<br>
</p><aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://deeplizard.com/images/deeplizard-1.png" class="site-icon" width="16" height="16">
      <a href="https://deeplizard.com/learn/video/0LhiS6yu2qQ" target="_blank" rel="nofollow noopener">deeplizard.com</a>
  </header>
  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:128/72;"><img src="https://i.ytimg.com/vi/0LhiS6yu2qQ/maxresdefault.jpg" class="thumbnail" width="128" height="72"></div>

<h3><a href="https://deeplizard.com/learn/video/0LhiS6yu2qQ" target="_blank" rel="nofollow noopener">CNN Confusion Matrix with PyTorch - Neural Network Programming</a></h3>

<p>In this episode, we learn how to build, plot, and interpret a confusion matrix using PyTorch. We also talk about locally disabling PyTorch gradient tracking or computational graph generation. This is</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>You can get the tn,tp,fn,fp as follows:<br>
</p><aside class="onebox stackexchange">
  <header class="source">
      <a href="https://stackoverflow.com/questions/31324218/scikit-learn-how-to-obtain-true-positive-true-negative-false-positive-and-fal" target="_blank" rel="nofollow noopener">stackoverflow.com</a>
  </header>
  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/4355092/euskalduna" target="_blank" rel="nofollow noopener">
    <img alt="Euskalduna" src="https://www.gravatar.com/avatar/8273213c8ea7ece7b8c73b927ea34a1c?s=128&amp;d=identicon&amp;r=PG&amp;f=1" class="thumbnail onebox-avatar" width="60" height="60">
  </a>
<h4>
  <a href="https://stackoverflow.com/questions/31324218/scikit-learn-how-to-obtain-true-positive-true-negative-false-positive-and-fal" target="_blank" rel="nofollow noopener">Scikit-learn: How to obtain True Positive, True Negative, False Positive and False Negative</a>
</h4>

<div class="tags">
  <strong>python, machine-learning, scikit-learn, classification</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/4355092/euskalduna" target="_blank" rel="nofollow noopener">
    Euskalduna
  </a>
  on <a href="https://stackoverflow.com/questions/31324218/scikit-learn-how-to-obtain-true-positive-true-negative-false-positive-and-fal" target="_blank" rel="nofollow noopener">05:19PM - 09 Jul 15 UTC</a>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


username: SoulHokib
cooked: <p>Thanks for your help, I’ll have an eye on it !</p>