## 2 Methodologies
### Proposed Method for Identifying Arc Magnet Surface Defects  
Combining the advantages of YOLOv5s, transfer learning, network pruning, and knowledge distillation, we designed a framework to quickly and accurately identify the surface defects from visible light images of arc magnets. As shown in Figure 2, our model is composed of three stages as follows:  
(1) **Stage 1**: Construct a transferred model from a pre-trained YOLOv5s via transfer learning. _Step 1-1_: Prepare the YOLOv5s network that has been trained by the public COCO2017 image dataset as a pre-trained model. Figure 2: Proposed framework for identifying arc magnet surface defects. _Step 1-2_: Retrain the pre-trained model with a relatively small sample of the specific arc magnet image dataset to obtain a transferred YOLOv5s network under a transfer learning mechanism that freezes the layers up to the first CSP1-X module in the backbone network of YOLOv5s. (2) **Stage 2**: Compress the model by knowledge distillation from the transferred YOLOv5s network to its pruned network. _Step 2-1_: Prune the transferred YOLOv5s network by removing unimportant output channels in each layer of the network according to a custom rule, which specifies that the removable channels (not including those in the input and output layers of YOLOv5s) are those corresponding to the smaller norm within a fixed proportion (that is, 75%) after calculating the \(L_{2}\) norm of each channel and sorting them. The \(L_{2}\) norm can be defined as below:  
\[L_{2}=\sqrt{\Sigma_{i=1}^{n}\left(a_{i}\right)^{2}}\, \tag{6}\]  
where \(a_{i}\) is the \(i\)-th weight parameter of a convolution kernel corresponding to a channel in a layer. _Step 2-2_: Treat the transferred YOLOv5s network as the teacher network of knowledge distillation, and its pruned network as the corresponding student network. _Step 2-3_: Use the previous arc magnet image dataset to retrain the student network iteratively until the total loss of distillation learning is stabilized to a minimum. The total loss, which is also regarded as a customized loss function, refers to the sum of two parts: the first one is the distillation loss between the outputs of the teacher network and those of the student network, while the second one is the student loss (a total of the confidence loss, the class loss, and the location loss) that is produced by the training process of the student network. Assuming that \(L_{KD}\), \(L_{d}\), and \(L_{s}\) are used to describe the aforementioned total loss, distillation loss, and student loss, respectively, these three losses can be formulated as  
\[\left\{ \begin{array}{l}L_{KD}=L_{d}+L_{s}\\ L_{d}=\alpha\times T^{2}\times f_{KLD}\left(O_{s}^{T},O_{t}^{T}\right)\\ L_{s}=(1-\alpha)\Big{(}L_{confidence}+L_{class}+L_{location}\ \Big{)}\\ L_{confidence}=-\frac{1}{n}\Sigma_{i=1}^{n}\big{[}\lambda*y_{true}\times\ln O_ {sc}^{i}+(1-y_{true})\times\ln\!\big{(}1-O_{sc}^{i}\big{)}\big{]}\\ L_{class}=-\frac{1}{n}\Sigma_{i=1}^{n}\big{[}y_{true}\times\ln O_{sp}^{i}+(1 -y_{true})\times\ln(1-O_{sp}^{i})\big{]}\\ L_{location}=\frac{1}{n}\Sigma_{i=1}^{n}\bigg{(}1-\big{|}\frac{A_{i}\cap B_{ i}}{\big{|}A_{i}\cup B_{i}}\big{|}+\frac{\big{|}A_{i}^{n}-A_{i}\cup B_{i} \big{|}}{\big{|}A_{i}^{n}\big{|}}\bigg{)}\end{array} \right.