# A Nested U-Shaped Residual Codec Network for Strip Steel Defect Detection
## 4 Evaluation Metrics  
Evaluation metrics are essential for evaluating the effectiveness of algorithms and, traditionally, the mean absolute error (MAE) [40] is one of the most frequently used metrics for the saliency object detection. The MAE aims to measure the dissimilarity between the predicted saliency map \(y^{pred}\) and the ground truth \(y^{gt}\), defined as follows:  
\[MAE=\frac{1}{N}\sum\limits_{i=1}^{N}\left|y^{gt}_{i}-y^{pred}_{i}\right| \tag{5}\]  
where \(N\) is the number of test saliency defect images.  
However, MAE is inadequate for the strip steel defect detection problem due to the sparsity of defect objects. In lieu of MAE, other assessment metrics, including the weighted F-measure (WF) [41] score and structure-measure (SM) [42] are frequently adopted in the research community to evaluate the performance of models for strip steel defect detection. The WF is designed based on the precision (the proportion of true foreground pixels in the predicted foreground pixels) and recall (the proportion of predicted foreground pixels in the true foreground pixels). Specifically, the precision and recall are defined as follows:  
\[\text{Recall}=\frac{TF}{TF+FB}, \tag{6}\]  
\[\text{Recall}=\frac{TF}{TF+FB}, \tag{7}\]  
where \(TF\), \(FF\), and \(FB\) indicate the number of foreground pixels predicted as foreground pixels (true foreground), background pixels predicted as foreground pixels (false foreground),and foreground pixels as background pixels, respectively. Based on precision and recall, WF is defined as follows:  
\[WF=\frac{(1+\beta^{2})\cdot\text{Precision}^{w}}{(\beta^{2}\cdot\text{Precision}^{ w}+\text{Recall}^{w})} \tag{8}\]  
where \(\beta\), often set to be 1, is a coefficient for adjusting the relative importance of precision with respect to recall. Therefore, f-measure is a harmonic mean between recall and precision.  
The SM [42] is another often used measure to evaluate the performance of models for the strip steel defect detection, defined as follows:  
\[S=\alpha*s_{o}+(1-\alpha)*s_{r} \tag{9}\]  
Where \(S_{o}\) is an object-aware structural similarity evaluation definition, and \(S_{r}\) is a region-aware structural similarity evaluation definition. Therefore, SM is a structural similarity evaluation metric, which simultaneously considers the object-aware and region-aware structural similarity between the predicted saliency map and ground truth. We set \(\alpha=0.5\) in our implementation.  
The boundary quality is an important indicator for evaluating the detection effect of the model. The Pratt's figure of merit (PFOM [43]) is often used to evaluate the boundary quality of predicted saliency map, defined by the combination of three factors, namely the missed detection of the real edge, the misunderstanding of the pseudo-edge, and the positioning error of the edge. Formally, PFOM is defined as follows:  
\[\textit{PFOM}=\frac{1}{\text{max}(N_{e},N_{d})}\text{$\sum_{k=1}^{N_{d}}$}\frac {1}{1+\beta d_{k}^{2}} \tag{10}\]  
where \(N_{e}\) and \(N_{d}\) are the number of ideal and real edge points, respectively, \(d_{k}\) is the pixel miss distance between the \(k\)-th ideal edge point and the corresponding detected edge point, and \(\beta\) is a scaling constant chosen to 1/9 to provide a relative penalty between smeared edges and isolated, but offset, edges.