# How to Correctly Detect Face-Masks for COVID-19 from Visual Information?
## 4 Evaluation Methodology
### First Stage--Face Detection
The model is publicly available as part of the _InsightFace: open-source 2D and 3D deep face analysis toolbox_ ([[https://github.com/deepinsight/insightface/tree/master/detection/RetinaFaceAntiCov](https://github.com/deepinsight/insightface/tree/master/detection/RetinaFaceAntiCov)]([https://github.com/deepinsight/insightface/tree/master/detection/RetinaFaceAntiCov](https://github.com/deepinsight/insightface/tree/master/detection/RetinaFaceAntiCov)), accessed on 25 February 2021). It uses a RetinaFace MobileNet as the backbone network. * The Mask detector by AIZooTech--The AIZoo face-mask detector is an open-source detector again based on the SSD framework [39] ([[https://github.com/AIZOOTech/FaceMaskDetection](https://github.com/AIZOOTech/FaceMaskDetection)]([https://github.com/AIZOOTech/FaceMaskDetection](https://github.com/AIZOOTech/FaceMaskDetection)), accessed on 25 February 2021). It uses a lightweight backbone network to ensure low computational complexity. The models were selected for our experiments because of their competitive detection performance and the fact that pretrained models are publicly available. Some of the models (Baidu and AIZooTech, for example) are also part of face-mask detection frameworks and, therefore, designed specifically for the studied detection problem. **Performance measures. ** To evaluate performance of the detection models we use _average precision_ (AP), which is a standard performance measure also used in visible competitions, such as PASCAL VOC [40], ImageNet [41] and COCO [42]. When computing performance scores, we consider all detections with a confidence score greater than 0.02. To match the detections with ground-truth annotations, we define a criterion that states: two bounding boxes represent the same face if their IoU (Intersection over Union) is equal or greater than a specific threshold (typically 0.5). This \"match\" is considered a true positive (TP) if that ground-truth annotation has not been already used (to avoid multiple detections of the same face). IoU is defined as:  
\[IoU=\frac{A\cap B}{A\cup B} \tag{1}\]  
where A is the area of the predicted bounding box and B is the area of the ground-truth bounding box. A good way to characterize the performance of face detectors (also used in this paper) is to look at how precision and recall change with respect to changes of the confidence threshold. Using this procedure, we calculate precision/recall curves, which visualize the trade-off between the precision \(p\) and the recall \(r\) of the evaluated detection model. Here, precision tells us how many of the detected faces are relevant (i.e., present in the ground-truth) and is defined as:  
\[p=\frac{TP}{TP+FP} \tag{2}\]  
where \(TP\) stands for the number of true positives and \(FP\) stands for the number of false positives. Recall, on the other hand, tells us how many relevant faces are detected and is defined as:  
\[r=\frac{TP}{TP+FN} \tag{3}\]  
where \(FN\) stands for the number of false negatives. _AP_ is precision averaged across all recall values between 0 and 1 and represent an overall performance measure often used to characterize the performance of detection models. _AP_ corresponds to the area under the precision--recall curve and is defined as:  
\[AP=\int_{0}^{1}p(r)dr, \tag{4}\]  
where \(p(r)\) is precision at recall \(r\). To compute the above integral, we use in this paper the methodology from VOC2012 (Visual Object Classes Challenge 2012), which can be computed from the corresponding Matlab Development Kit or Carutcho's open-source Python implementation [43].