# Few-Shot Steel Defect Detection Based on a Fine-Tuned Network with Serial Multi-Scale Attention
## 3 Methodology
### Enhanced Semi-Supervised Learning Algorithm  
SSL and data augmentation are effective techniques for handling small data sample sizes [38]. By using smoothing estimation, these methods can significantly improve model performance under conditions of data scarcity. The lack of labeled data may be widespread across all categories or concentrated in specific ones, leading to classification bias. SSL distinguishes itself by exploiting unlabeled data in addition to labeled data, thereby enabling the improvement of model performance even with relatively fewer labeled instances. The principal aim of this approach is to exploit the information in unlabeled data to enhance the model's generalization, ultimately achieving superior performance on new datasets. This technique finds particular resonance in contexts where extensive datasets are at play, but labeled samples are few, such as in image classification, speech recognition, and natural language processing. The SSL algorithm based on pseudo-labeling involves feeding unlabeled data into the model and employing the class with the highest probability as the pseudo-label to facilitate the training process. SSL based on pseudo-labels is divided into a real label part and a pseudo-label part, as detailed in Equation (3). \[L=\frac{1}{n}\sum_{m=1}^{n}\sum_{i=1}^{c}L\bigl{(}y_{i}^{m},f_{i}^{m}\bigr{)}+ \alpha(t)\frac{1}{n^{\prime}}\sum_{m=1}^{n^{\prime}}\sum_{i=1}^{c}L\bigl{(}y_{ i}^{m},f_{i}^{m}\bigr{)} \tag{3}\]where \(L\left(y_{i}^{m},f_{i}^{m}\right)\) represents the loss of the real labels, \(y_{i}^{m}\) is the real label of the labeled samples, and \(f_{i}^{m}\) is the output of the network. For the pseudo section, \(L\left(y_{i}^{m},f_{i}^{m}\right)\) denotes the pseudo-label loss, \(y_{i}^{m}\) refers to the label of the unlabeled samples, and \(f_{i}^{m}\) is the network's output. Notably, the pseudo-label component incorporates a weight factor, \(\alpha(t)\), which modulates the influence of pseudo-labels. \(\alpha(t)\) follows a deterministic simulated annealing process, computed as described in Equation (4), and the pseudo code follows. \[\alpha(t)=\left\{ \begin{array}{c}0,(t<T_{1})\\ \frac{t-T_{1}}{t_{2}-T_{1}},(T_{1}\leq t\leq T_{2})\\ \alpha_{f},(T_{2}\leq t)\end{array} \right. \tag{4}\]  
The tuning of \(T_{1}\) and \(T_{2}\) is conducted through a phased approach. During the initial phase \(t<T_{1}\), the weight factor \(\alpha(t)\) is set to 0, meaning the influence of pseudo-labels is completely ignored, and only real labels are used for training. This phase ensures that the model learns effectively from real labels in the early stages. In the transition phase, \(T_{1}\leq t\leq T_{2}\), \(\alpha(t)\) increases linearly from 0 to the final value \(\alpha_{f}\), allowing the model to gradually adapt to the inclusion of pseudo-labels in the training process. The duration of this phase is critical as it dictates the smooth introduction of pseudo-labels. Finally, in the stable phase, \(T_{2}<t\), \(\alpha(t)\) maintains its final value \(\alpha_{f}\), maximizing the influence of pseudo-labels and enabling the model to fully leverage both pseudo-labels and real labels for joint training. The specific values of \(T_{1}\) and \(T_{2}\) are fine-tuned experimentally based on the task and dataset to ensure the model achieves a balance between initial reliance on real labels and the gradual incorporation of pseudo-labels, ultimately optimizing performance, as shown in Algorithm 1. ```
1:while\(t<T_{1}\)do
2:\(\alpha(t)=0\)
3:endwhile
4:while\(T_{1}<t<T_{2}\)do
5:\(\alpha(t)=\frac{t-T_{1}}{t_{2}-T_{1}}\)
6:endwhile
7:while\(T_{2}<t\)do
8:\(\alpha(t)=\alpha_{f}\)
9:endwhile
10:endfunction
```  
**Algorithm 1** Calculation method of weight factor  
Pseudo-label-based semi-supervised learning emerges as a prominent method within the SSL framework [39]. This strategy involves using a model to make predictions on unlabeled data and then applying those predictions as pseudo-labels to train the model together with the labeled data.