Intersection over union (IoU) calculation for evaluating an image segmentation model | by Oleksii Sheremet | Towards Data Science

In the simplest case, segmentation is the process of dividing a digital image into several segments. The result of instance segmentation using Mask R-CNN is a mask applied to the desired object and a bounding box around this object.
In a practical task that I was solving, it was necessary to determine the buildings in the Google Earth photos. This task was successfully accomplished using Mask R-CNN for instance segmentation. After training the model, it was necessary to evaluate it quality for prediction the bounding boxes around buildings. I would like to acquaint readers with the approach (with code) I have used to evaluate the quality of predicting the coordinates of bounding boxes.
Intersection over union (IoU) is known to be a good metric for measuring overlap between two bounding boxes or masks.
If the prediction is completely correct, IoU = 1. The lower the IoU, the worse the prediction result.
All code was implemented in Google Colab. Let’s take a closer look. Import libraries and load source image:
import cv2import numpyimport matplotlib.pyplot as pltimg = cv2.imread(“./some_image.jpg”)plt.imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))
Import libraries and load source image:
# An example of first bounding boxfirst_bb_points = [[250, 210], [440, 210], [440, 390], [250, 390]]stencil = numpy.zeros(img.shape).astype(img.dtype)contours = [numpy.array(first_bb_points)]color = [255, 255, 255]cv2.fillPoly(stencil, contours, color)result1 = cv2.bitwise_and(img, stencil)result1 = cv2.cvtColor(result1, cv2.COLOR_BGR2RGB)plt.imshow(result1)
Let’s set the coordinates of the original bounding box and fill the entire content of the photo outside the bounding box with black:
The coordinates of the second bounding box are obtained as a result of prediction:
# An example of second bounding boxsecond_bb_points = [[280, 190], [438, 190], [438, 390], [280, 390]]stencil = numpy.zeros(img.shape).astype(img.dtype)contours = [numpy.array(second_bb_points)]color = [255, 255, 255]cv2.fillPoly(stencil, contours, color)result2 = cv2.bitwise_and(img, stencil)result2 = cv2.cvtColor(result2, cv2.COLOR_BGR2RGB)plt.imshow(result2)
Like in the previous case, fill all the content outside the bounding box with black:
By filling images with black outside of the bounding boxes, pixels outside the area of interest do not affect the IoU calculation:
# IoU calculationintersection = numpy.logical_and(result1, result2)union = numpy.logical_or(result1, result2)iou_score = numpy.sum(intersection) / numpy.sum(union)print(‘IoU is %s’ % iou_score)
Thus, we get the following result:
IoU is 0.7625239952938262
Conclusion
The proposed approach has worked well in practice. It is possible to define overlap between image areas, not necessarily rectangular. You can calculate the coordinates of the contour around the masked object, for example, using OpenCV. Then using these points and the approach described above, you can calculate IoU between the masks.
References
Mask R-CNN for Object Detection and Segmentation
Open Computer Vision Library
Intersection over Union (IoU) for object detection