# Few-Shot Steel Defect Detection Based on a Fine-Tuned Network with Serial Multi-Scale Attention
## 4 Experimental Results and Analysis  
### Comparative Analysis of Backbone Networks  
All experiments were conducted using an NVIDIA GeForce RTX 2070 GPU, including training, validation, and testing phases. In the context of fine-tuning-based few-shot object detection networks, our study explored four distinct architectures for defect detection: ResNet50, ResNet101, and VGG16 [40] within the Faster R-CNN framework, as well as YOLOv4 [41], whose backbone is CSPDarknet53. Each network was trained using the stochastic gradient descent (SGD), with a batch size of 4, momentum set at 0.9, and a weight decay coefficient of 0.0001. In the first stage of training for the base classes, the learning rate was set to 0.02, while in the second stage of training for the new classes, the learning rate was adjusted to 0.001. It is important to recognize that SGD initially converges rapidly, but as it approaches the optimal solution, its convergence rate slows, potentially leading to local minima. Momentum can assist in navigating past these minima. Furthermore, careful adjustments to the learning rate and weight decay are essential to prevent overfitting and maintain the stability of the training process. In this research, we performed the fine-tuning-based few-shot object detection based on the methodology put forth by [PERSON] et al. [42]. This approach is based on the widely employed Faster R-CNN, a two-stage object detector, which served as the foundation [43]. Comprising a backbone network (e.g., ResNet [44], VGG16), a RPN, and dual fully connected sub-networks, Faster R-CNN functions as a feature extractor at the proposal level. Furthermore, it incorporates a classifier and a regressor to perform object classification and bounding box coordinate prediction. The authors introduce new benchmarks on PASCAL VOC, COCO, and LVIS datasets, achieving stable accuracy estimations and outperforming previous meta-learning methods by notable margins. This simple yet powerful approach establishes new state-of-the-art results, demonstrating substantial gains in average precision (AP) for rare classes with minimal impact on frequent classes. During the initial phase of training, Faster R-CNN underwent an intensive training process using a large number of samples from the base class, analogous to other object detection frameworks. The loss function utilized in this stage is represented in Equation (5):  
\[L=L_{rpn}+L_{cls}+L_{loc} \tag{5}\]  
where \(L_{rpn}\) is applied to RPN's output to discriminate between the foreground and background and to refine anchors. \(L_{cls}\) represents the cross-entropy loss for the bounding box classifier, used for object classification, and \(L_{loc}\) indicates the smooth L1 loss for the bounding box regressor, used for bounding box coordinate prediction. The Smooth L1 loss is a fusion of the L1 and L2 losses. While the L1 loss is not differentiable at the zero point, the L2 loss can result in gradient explosion when the prediction significantly deviates from the target. The Smooth L1 loss ameliorates the drawbacks of both by incorporating the advantages of each component. The first piecewise function corresponds to the L2 loss, and the second piecewise function corresponds to the L1 loss, as shown in Equation (6):  
\[L_{1}(x)=\left\{ \begin{array}{l}0.5x^{2},\text{ if }|x|<1\\ |x|-0.5,\text{ other }\end{array} \right. \tag{6}\]  
where \(x\) is equal to \(f(x)-y\), which is the difference between the true value and the predicted value. Transitioning to the second phase, a curated dataset, containing a limited number of base class and new class samples, was introduced for the fine-tuning phase. The network initially allocated randomly initialized weights to the new classes. This was followed by fine-tuning confined to the bounding box classifier and regressor, while the feature extractor remained unchanged. In the context of few-shot learning, the traditional softmax classifier is sub-optimal for learning classification features. In light of this, [PERSON] et al.