## 4 Experiment and Results
### Results
#### 4.3.2 Stage 2: Surface Pixel Defect Mapping

Table 5: Comparison of AP and mAP for each defect type using different backbone networks in faster R-CNN. Figure 12: Bounding boxes for defect types detected through Faster RCNN with ResNet-50. The left column of each result displays the ground truth, while the right column shows the predictions. (a) Crazing, (b) inclusion, (c) patches, (d) pitted surface, (e) rolled-in scale, (f) scratches. Table 7 presents a comparison of Precision, Recall, and AP for steel surface defects using the faster R-CNN model with the ResNet-50 backbone, which showed the best results among the three models. The AP value for the crazing defect was the lowest at 0.6528, reflecting the difficulty in identifying this defect, which often closely resembles normal pixels. Consequently, the model struggles to distinguish this defect clearly, resulting in lower precision and recall. Future research could improve detection by implementing augmentation techniques to enhance defect patterns or using multi-scale approaches to better discern the shape, size, and brightness of defects. The bounding boxes detected by faster R-CNN were refined for weakly supervised segmentation by excluding certain regions within the boxes and the GrabCut algorithm was applied. The GrabCut algorithm is one of the image segmentation techniques that separates the foreground and background based on minimal input provided by the user. To create an initial mask, the bounding box is first reduced to 80% of its original size, with the center of the box serving as the reference, to make the boundary closer to the object. Then, using the GrabCut algorithm, a binary mask is generated by marking the interior of the shrunken bounding box as the foreground and the exterior as the background. As a result, an initial mask is generated, as shown in Figure 13c, which is then fed into the DeepLabv3+ model for inference. The DeepLabv3+ model used in this process was pre-trained on large-scale segmentation datasets such as PASCAL VOC and Cityscapes, which are specifically designed for semantic segmentation tasks. Subsequently, the initial mask is input into the DeepLabv3+ model, which uses ResNet-101 as the backbone network. Recursive learning was used to train the segmentation model across 10 rounds, with 5 epochs per round. The model is trained on both the shrunken bounding box masks and the original image. In each round, the predicted mask is compared with the ground truth from the previous round. If the predicted mask covers a larger area, the previous ground truth is retained. This approach starts from the second round, with the mask from the shrunken bounding box used as the initial ground truth in the first round. This approach ensures that the model progressively refines its predictions and prevents over-prediction by mitigating the accumulation of incorrect predictions. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Method** & **Backbone** & **mAP\%** & **FPS** & **GFLOPs** \\ \hline Faster R-CNN & ResNet-50 & 0.7666 & 16 & 182.7 \\ SSD & VGG-16 & 0.6102 & 51 & 42.4 \\ YOLOv4 & CSPDarkNet & 0.6433 & 55 & 34.2 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Model comparison results on the NEU-DET dataset. \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Defect Category** & **Precision\%** & **Recall\%** & **AP\%** \\ \hline Crazing & 0.7259 & 0.6759 & 0.6528 \\ Inclusion & 0.875 & 0.8792 & 0.8208 \\ Patches & 0.9583 & 0.8639 & 0.8723 \\ Pitted surface & 0.8333 & 0.7756 & 0.7409 \\ Rolled-in scale & 0.8529 & 0.7941 & 0.768 \\ Scratches & 0.8534 & 0.7822 & 0.7448 \\ \hline
**Average** & **0.8498** & **0.7636** & **0.7666** \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Precision, recall, and average precision results for each defect type using faster R-CNN with ResNet-50.