Implementation of IOU for multiclass semantic segmentation

MA_CASANDRA_QUILANG: <p>Hello! I would want to compute the training accuracy of my neural network, but I want it to be IOU. I found something on the internet but mostly are for binary classification only, can anyone help me?</p>
<p>by the way this is my script so far:</p>
<pre><code class="lang-auto">def display_segmentation():

  model = ESNet(classes=6)
  path = F"/content/drive/MyDrive/Thesis_QuilangCastillano/Models/ESNET_5250.37-checkpoint.pt"  
  checkpoint = torch.load(path)
  model.load_state_dict(checkpoint['state_dict'])

  total = 0
  correct = 0

  torch.set_printoptions(edgeitems=128)

  with torch.no_grad():
    model.eval()
    for i in tqdm(range(1)):
      net_out = model(img[100].view(-1, 3, 384, 512))
      segmentation_label = torch.argmax(net_out.squeeze(), dim= 0)
      segmented_image = decode_segmap(segmentation_label)

     
      label = np.unique(segmentation_label)
      groundtruth = np.unique(mask[100])


      alpha = 0.3 # how much transparency to apply
      beta = 1 - alpha # alpha + beta should equal 1
      gamma = 0 # scalar added to each sum

      image = np.array(img[100].view(384, 512, 3))
      prediction = cv2.addWeighted(segmented_image, alpha, image, beta, gamma, image,  dtype=cv2.CV_64F)

      Sub = f"Prediction: {label}, Groundtruth: {groundtruth}"

      plt.imshow(prediction)
      plt.title(Sub)
      plt.show()
   

  
display_segmentation()

</code></pre>
<p>this script outputs this one:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/1/d185a6d6eb145d3635c2f6bdc55e7aa232a1685c.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/d185a6d6eb145d3635c2f6bdc55e7aa232a1685c" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/d/1/d185a6d6eb145d3635c2f6bdc55e7aa232a1685c_2_690x291.jpeg" alt="image" data-base62-sha1="tTw8Bzw3yuaQNpL8UE5jTqA0zHC" width="690" height="291" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/d/1/d185a6d6eb145d3635c2f6bdc55e7aa232a1685c_2_690x291.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/d/1/d185a6d6eb145d3635c2f6bdc55e7aa232a1685c_2_1035x436.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/d/1/d185a6d6eb145d3635c2f6bdc55e7aa232a1685c.jpeg 2x" data-dominant-color="ECEDEE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1303×551 58.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
the groundtruth array contains the class number of what class should be present in an image, the prediction array contains the class number of what the neural net thinks the image is composed of</p>

wolterlw: <p>I would recommend you don’t start from vanilla PyTorch as there are multiple decent libraries built on top of it that simplify building models.<br>
You could try <a href="https://lightning-bolts.readthedocs.io/en/latest/convolutional.html#semantic-segmentation" rel="noopener nofollow ugc">lightning bolts</a> for baseline model architecture + training procedure (you just need to replace the provided dataset with your own. For IoU implementation you can use <a href="https://torchmetrics.readthedocs.io/en/latest/references/modules.html#iou" rel="noopener nofollow ugc">torchmetrics</a>.</p>