## 3 \(D^{2}\)-SPDM: Faster R-CNN-Based Defect Detection and Classification with Label Enhancement
### Surface Pixel Defect Mapping  
Figure 6 depicts the structure of the SPDM framework, which comprises two main components as shown: object detection and classification, as well as label enhancement. Faster R-CNN is a representative two-stage object detection model comprising two primary modules: the region proposal network (RPN) and the classifier. These modules share convolutional layers in an end-to-end style. Initially, the input image is resized to a fixed dimension of 600 \(\times\) 800 and fed into faster R-CNN. The backbone network, ResNet-50, extracts feature maps from the image, which are then transferred to the RPN. The RPN generates anchor boxes in various sizes and aspect ratios at each feature map position. The network assigns each anchor box an objectness score to indicate whether it contains an object and performs bounding box regression to refine the anchor box coordinates to better fit the object's location. The following equations describe the operations for calculating objectness score and bounding box regression in faster R-CNN. Figure 5: Network structure of ResNet-50. \[p_{\text{obj}}=\frac{e^{s_{\text{obj}}}}{e^{s_{\text{obj}}}+e^{s_{\text{bg}}}} \tag{3}\]  
Here, \(p_{\text{obj}}\) denotes the probability that an object is present within the anchor box, \(s_{\text{obj}}\) is the score for the object class, and \(s_{\text{bg}}\) is the score for the background class. \[t_{x} =\frac{x-x_{a}}{w_{a}} \tag{4}\] \[t_{y} =\frac{y-y_{a}}{h_{a}}\] \[t_{w} =\log\biggl{(}\frac{w}{w_{a}}\biggr{)}\] \[t_{h} =\log\biggl{(}\frac{h}{h_{a}}\biggr{)}\]  
Bounding box regression learns the transformation relationship between the anchor box and the ground truth box. Here, \(t_{x}\) and \(t_{y}\) represent the offset transformations for the x and y coordinates, whereas \(t_{w}\) and \(t_{h}\) represent the offset transformations for the width and height, respectively. The coordinates (x, y) indicate the center of the actual bounding  
Figure 6: Steel pixel defect mapping. box, and (w, h) represent the width and height of the bounding box. The coordinates (\(x_{a},y_{a}\)) refer to the center of the anchor box, and (\(w_{a}\), \(h_{a}\)) to its width and height. After generating multiple potential bounding box predictions, the highest-scoring proposal boxes are selected. Non-maximum suppression (NMS) is then applied to remove overlapping boxes. The chosen proposal boxes are resized using ROI pooling to produce a uniform feature map. These feature vectors are input into fully connected layers, and ultimately, the softmax probabilities for each defect object class and background class, along with the refined bounding box locations, are output using the following equations. \[ \begin{split}\hat{x}&=t_{x}\cdot w_{a}+x_{a}\\ \hat{y}&=t_{y}\cdot h_{a}+y_{a}\\ \hat{w}&=\exp(t_{w})\cdot w_{a}\\ \hat{h}&=\exp(t_{h})\cdot h_{a}\end{split}  \tag{5}\]  
(\(\hat{x},\hat{y}\)) represent the center coordinates of the transformed bounding box, while (\(\hat{w}\) and \(\hat{h}\)) denote the width and height of the transformed bounding box. The classification loss is calculated using cross-entropy loss, and the regression loss is computed through Smooth L1 Loss. The model is updated via backpropagation using the total loss, which is the sum of these two losses, as depicted in the equation below:  
\[L_{\text{total}}=L_{\text{cls}}+\lambda\sum_{i\in\{x,y,w,h\}}L_{\text{reg}}(t_ {i},t_{i}^{*}) \tag{6}\]  
\(L_{\text{reg}}(t_{i},t_{i}^{*})\) represents the regression loss, where \(t_{i}\) denotes the predicted transformation parameter and \(t_{i}^{*}\) indicates the ground truth transformation parameter. This section outlines the conversion of the bounding box generated by faster R-CNN into refined segmentation labels. The 'bounding box to segmentation' process employs the GrabCut technique to isolate the object from the background. All pixels within the given bounding box are classified as belonging to a single class, while pixels outside the box are considered background.