# A Survey on Surface Defect Inspection Based on Generative Models in Manufacturing
## 4 Datasets and Metrics
### Metrics  
Metrics are very important and necessary which can guide model design and demonstrate the effectiveness and superiority of the proposed methods. For surface inspection, it is desirable first to evaluate the accuracy of the inspection methods. The most basic indicators are accuracy (ACC), precision, recall, and F1-score, where the definitions of these indicators are defined in Equations (4)-(7). \[ACC=\frac{TP+TN}{TP+TN+FP+FN} \tag{4}\]  
\[Precision=\frac{TP}{TP+FP} \tag{5}\]  
\[Recall=\frac{TP}{TP+FN} \tag{6}\]  
\[F1-score=\frac{2\times Recall\times Presion}{Recall+Presion} \tag{7}\]  
where TP, TN, FP, and FN indicate a situation in which the model correctly predicts a positive sample as a positive case, the model correctly predicts a negative sample as a negative case, the model incorrectly predicts a negative sample as a positive case, and the model incorrectly predicts a positive sample as a negative case, respectively. Additionally, in order to reasonably evaluate the prediction effect of methods, the indicators mAP (mean Average Precision) and mIoU (mean Intersection over Union) are always \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Dataset** & **Objective** & \multicolumn{2}{c}{**Description**} \\ \hline AITEX [114] & Fabric & Number of defect-free images: 140 & Number of defect types: 12 \\  & & Number of defect images: 105 & Size of image: 4096 \(\times\) 256 \\ \cline{3-4} DAGM [115] & Artificial defect & Number of defect-free images: 6000 & Size of image: 512 \(\times\) 512 \\  & & Number of defect images: 1500 & With oval annotations \\  & & Number of defect types: 10 & \\ KolektorSDD1 [116] & Metal Surface & Number of defect-free images: 347 & Number of defect types: 1 \\  & & Number of defect images: 52 & Size of image: 1240–1270 \(\times\) 500 \\  & & Number of defect images: 2979 & Size of image: 230 \(\times\) 630 \\  & & Number of defect images: 356 & With segmentation annotations \\ MT [118] & Magnetic tile & Number of defect images: 1344 & Size of image: 128 \(\times\) 128 \\  & & Number of defect types: 6 & With saliency annotations \\ \cline{3-4} MVTec [119] & Multi-type & Number of defect images: 5354 & Size of image: 700 \(\times\) 700–1024 \(\times\) 1024 \\  & & Number of defect types: 15 & Size of image: 700 \(\times\) 700–1024 \(\times\) 1024 \\  & & Number of defect images: 1800 & Size of image: 200 \(\times\) 200 \\  & & Number of defect types: 6 & Size of image: 200 \(\times\) 200 \\  & & Number of defect images: 1800 & Size of image: 200 \(\times\) 200 \\  & & Number of defect types: 6 & With bounding-box annotations \\ NEU-CLS-64 [72] & Steel plate & Number of defect images: 7000 & Size of image: 64 \(\times\) 64 \\  & & Number of defect types: 9 & Size of image: 64 \(\times\) 64 \\ RSDDs [121] & Rail surface & Number of defect images: 195 & Size of image: 1500 \(\times\) 2180 \\  & & Number of defect types: 2 & Size of image: 1500 \(\times\) 2180 \\ X-SDD [122] & Hot-rolled strip & Number of defect images: 1360 & Size of image: 128 \(\times\) 128 \\  & & Number of defect types: 7 & Size of image: 128 \(\times\) 128 \\ YDFID-1 [123] & Fabric & Number of defect-free images: 3189 & Number of defect types: 3 \\  & & Number of defect images: 312 & Size of image: 512 \(\times\) 512 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Summary of public surface defect datasets. used for surface defect detection and segmentation tasks. For k-class defect inspection, they are calculated as in Equations (8) and (9). \[mAP=\frac{1}{k}\sum_{i=1}^{k}\frac{Recall+Pesion}{2} \tag{8}\]  
\[mIoI=\frac{1}{k}\sum_{i=1}^{k}\frac{|X\cap Y|}{|X\cup Y|} \tag{9}\]  
where X denotes the predictions and Y denotes the ground-truths. Similar to mAP, AUROC is also a common indicator for surface defect inspection. The AUROC means the area under the receiver operating characteristic (ROC) curve, where the calculation formula is as Equation (10).