# Few-Shot Steel Defect Detection Based on a Fine-Tuned Network with Serial Multi-Scale Attention
## 2 Related Work
### Research Progress Related to Attention Mechanisms  
In academic research, attention mechanisms have yielded significant advancements, exerting profound influence on fields such as natural language processing and computer vision (see Table 3). Inspired by style transfer, [PERSON] et al. [26] introduced the style-based recalibration module (SRM) to adaptively recalibrate intermediate feature maps. The SRM model correlates among input data channels and strengthens the network's representational capacity through recalibration. [PERSON] et al. [27] proposed the spatial transformer network (STN) architecture, integrating a spatial transformation network that enables neural networks to autonomously learn spatial transformations of input data. [PERSON] et al. [28] introduced the dynamic capacity network (DCN), comprising secondary networks for both global and detailed processing. DCN employs a coarse model to compute feature vectors, extracting a few saturated target regions. This is accomplished by assessing the significance of regions using information from the coarse model's output and selecting areas that most significantly influence model predictions. [PERSON] et al. [29] introduced a dilated weighted across-stages feature pyramid network to adapt the receptive fields and attention weight preferences of output feature maps at various scales, thereby enhancing the algorithm's utilization of abnormal-sized defect features and improving the detection of unusual defects. These studies led us to recognize the challenge posed by significant variations in defect spans within our dataset. It became promising that attention mechanisms could serve as highly effective modules to address this issue. Subsequently, tailored modifications to the attention mechanisms were made in accordance with our specific application requirements, ultimately achieving the desired results. \begin{table}
\begin{tabular}{c c c c} \hline
**Reference** & **Methods** & **Performance** & **Disadvantages** \\ \hline \multirow{4}{*}{[PERSON] et al. [22]} & This paper proposed a & \multirow{2}{*}{This method achieves 95.7\%} & The effectiveness of transfer \\  & transfer-based CNN method for & & \\  & the recognition and detection of & & \\  & ground objects in small-sample & & \\  & SAR images. & & \\ \hline \multirow{4}{*}{[PERSON] et al. [23]} & The study proposed a novel & \multirow{2}{*}{The method achieves a 97.51\%} & The method requires more \\  & transitive transfer learning CNN & & accuracy rate with an average & computational resources and \\  & ensemble framework for & & detection time of 155 ms & storage space due to training \\  & classifying bearing surface defects. & & per bearing. & across multiple domains. \\ \hline \multirow{4}{*}{[PERSON] et al. [24]} & The research proposed a & \multirow{2}{*}{MPSR achieves 42.3\% in} & MPSR’s performance in some \\  & Multi-scale Positive Sample & & cross-dataset experiments, & \\  & Refinement (MPSR) approach to & & \\  & enrich object scales in FSOD. & & \\ \cline{1-1}  & & & \\ \cline{1-1}  & & & \\ \cline{1-1}  & & & \\ \cline{1-1}  & & & \\ \cline{1-1}  & & & \\ \cline{1-1}  & & & \\ \cline{1-1}  & & & \\ \cline{1-1}  & & & \\ \cline{1-1}  & & & \\ \cline{1-1}  & & & \\ \cline{1-1}  & & & \\ \hline \end{tabular}
\end{table} 
Table 2: Existing methods in small-sample object detection. Considering existing studies on surface defect detection and contributions from researchers in the field of few-shot learning, it is observed that conventional surface defect detection methods predominantly rely on a large number of data samples. However, obtaining labeled steel surface defect samples of high quality presents considerable challenges, necessitating a pressing shift towards utilizing few-shot object detection techniques for steel surface analysis. Among the numerous few-shot learning strategies, those employing fine-tuning demonstrate superior efficacy [30]. Consequently, the research incorporates this approach in the investigation of steel surface defect detection.