# Real-Time Detection of Insulator Defects with Channel Pruning and Channel Distillation
## 4 Experiment and Results
### Experiment
The forward inference time is reduced by 0.53 ms, while the [EMAIL_ADDRESS] decreased by 4.8%. However, there is a significant reduction in the number of parameters and GFLOPs in the pruned model, and the accuracy of the pruned model is closely comparable to that of the pre-pruning insulator defect detection model. These findings suggest that the aforementioned improved model can be precisely simplified through channel pruning. The changes in the \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Speed\_up** & **Sparsity Learning** & **Fine-Tune** & **Sparsity Ratio** & **P** & **[EMAIL_ADDRESS]** & **Inference Time** \\
**Ratio** & **Epoch** & **Training Epoch** & **Sparsity Ratio** & **P** & **[EMAIL_ADDRESS]** & **(Batch Size 32)** \\ \hline
2.0 & 300 & 300 & 32.7\% & 73.3\% & 64.6\% & 0.78 ms \\
3.0 & 300 & 300 & 46.5\% & 50.6\% & 39.2\% & 0.61 ms \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Main parameter and results of different speed up experiments. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & **P** & **R** & **[EMAIL_ADDRESS]** & **[EMAIL_ADDRESS]** & **Parameter** & **GFLOPs** & **Inference Time** \\
**Before pruning** & 78.4\% & 65.4\% & 71.2\% & 32.8\% & 5.5 & 10.7 & 1.31 ms \\ After channel & 73.3\% & 58.5\% & 64.6\% & 28.0\% & 2.8 & 5.3 & 0.78 ms \\ pruning & 70.9\% & 58.6\% & 61.7\% & 26.3\% & 2.8 & 5.3 & 0.81 ms \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The comparative results of model performance before and after two pruning methods. number of channels per convolutional layer in the model after channel pruning are shown in Figure 15. The orange segments represent the number of channels in all convolutional layers of the model before pruning, while the red segments represent the number after pruning. It is evident that the number of channels in most convolutional layers has been substantially reduced, demonstrating the effectiveness of the channel pruning algorithm in decreasing the model's parameter count. Sparse training introduces sparsity constraints, such as L1 regularization, during the training process, encouraging a significant portion of the weights in the network to converge to zero [40]. As illustrated in Figure 16, after undergoing sparse training, most of the model's weights will be close to zero. This means that compression technologies can be employed to further reduce space consumption when storing and transmitting the model. 3 Channel distillation  
The student network designed in this study is a channel-pruned model, while the teacher network is based on YOLOv7, and after channel distillation, we call it a distilled network. Their detection results and the comparative results of the student network (prune model) with the distilled network are shown in Figure 17. The results indicate that, without altering the number of parameters, the accuracy of the pruned model is 75.5%, improved by 2.2%, and the [EMAIL_ADDRESS] is 28.4%, increased by 0.4%, following channel distillation. This  
Figure 16: The distribution of model weights after sparse training. Figure 15: The changes in the number of channels per convolutional layer in the model after channel pruning. demonstrates that channel distillation effectively enhances the precision of the lightweight detection model for insulator defect detection. 4. Comparative experiment  
At the same time, we also conducted experiments with SSD-lite [41], YOLOV8n, YOLOx-tiny, and YOLOv5s on our dataset. As shown in Table 4, the improved YOLOv7-tiny achieves the highest [EMAIL_ADDRESS] value among these algorithms while having the least number of parameters, effectively balancing model compression and accuracy loss. 5. Generalization Ability Verification  
To verify the robustness and generalization capability of our algorithm, we conducted defect detection using our method on the publicly available PCB Defects dataset from the Kaggle website [42]. The PCB Defects dataset contains 1386 images, with six types of defects: missing hole, mouse bite, open circuit, short, spur, and spurious copper.