  We present Boundary IoU (Intersection-over-Union), a new segmentation
evaluation measure focused on boundary quality. We perform an extensive
analysis across different error types and object sizes and show that Boundary
IoU is significantly more sensitive than the standard Mask IoU measure to
boundary errors for large objects and does not over-penalize errors on smaller
objects. The new quality measure displays several desirable characteristics
like symmetry w.r.t. prediction/ground truth pairs and balanced responsiveness
across scales, which makes it more suitable for segmentation evaluation than
other boundary-focused measures like Trimap IoU and F-measure. Based on
Boundary IoU, we update the standard evaluation protocols for instance and
panoptic segmentation tasks by proposing the Boundary AP (Average Precision)
and Boundary PQ (Panoptic Quality) metrics, respectively. Our experiments show
that the new evaluation metrics track boundary quality improvements that are
generally overlooked by current Mask IoU-based evaluation metrics. We hope that
the adoption of the new boundary-sensitive evaluation metrics will lead to
rapid progress in segmentation methods that improve boundary quality.
