# Real-Time Detection of Insulator Defects with Channel Pruning and Channel Distillation
## 2 New Feature Fusion Network: OD-Slimneck
The scalar \(\alpha_{w_{i}}\in\mathbb{R}\) weights the \(W^{i}\) with an attention mechanism; \(\alpha_{f_{i}}\in\mathbb{R}^{\ c_{out}}\), \(\alpha_{c_{i}}\in\mathbb{R}^{\ c_{in}}\), and \(\alpha_{s_{i}}\in\mathbb{R}^{\ k\times k}\) are newly introduced attention scales. The operation \(\odot\) represents element-wise multiplication across different dimensions of the kernel space. The operation \(*\) denotes convolution. Building on the concepts introduced for SlimNeck and ODConv, we aimed to both reduce the inference speed and complexity of the network's neck section, while also enhancing the model's accuracy in detecting insulator defects. Consequently, we propose a novel structure: OD-Slimneck, as illustrated in Figure 8. By integrating the ODConv module following the GSConv module, this structure increases the model's parameter count minimally, yet significantly boosts the accuracy in detecting small-scale targets such as insulator defects. Replacing the neck network of YOLOv7-tiny with our OD-SlimNeck structure not only utilizes the lightweight SlimNeck design, but also significantly  
Figure 6: The structures of the (**a**) GS bottleneck module and the (**b**) VoV-GSCSP modules. enhancement boosts the model's generalization capabilities and robustness, significantly improving the performance of object detection. 4 Channel pruning of the improved YOLOv7-tiny network  
Although the improved YOLOv7-tiny model has fewer parameters, deploying it on drones for real-time insulator inspections remains a significant challenge due to the limited memory and computational capabilities of embedded devices. To address this issue, we applied a channel pruning algorithm to the trained YOLOv7-tiny insulator defect detection model to streamline it, facilitating easier deployment on lightweight devices with constrained processing power. The most common method of model pruning is channel pruning, which requires minimal software and hardware resources and effectively balances fine-grained and coarse-grained approaches. As illustrated in Figure 10, channel pruning significantly reduces the model's size, structure, and the number of parameters. For our improved YOLOv7-tiny model, we employed channel pruning through a method that involves pruning the modified model [29]. Initially, we trained the original model with sparse regularization, assigning a scaling factor to each channel to signify its importance. Subsequently, we conducted channel sparsity training to distinguish between essential and non-essential channels. The Batch Normalization (BN) layer, a powerful operator that ensures rapid convergence and enhances generalization capabilities, is typically connected after the convolution layer in most traditional network architectures. Specifically, the BN layer employs batch statistics to normalize feature activations, as detailed below:  
\[Z_{\text{out}}=\gamma\frac{Z_{in}-\mu}{\sqrt{\sigma^{2}+\in}}+\beta \tag{8}\]  
Figure 10: Pruning schematic diagram of channel pruning algorithm. Figure 9: Structure of the Decoupled head. In this context, \(\mu\) and \(\sigma\) represent the mean and standard deviation of the features in a mini-batch, respectively. \(\gamma\) and \(\beta\) are trainable factors for scaling and shifting, respectively, and \(Z_{out}\) is the output of the Batch Normalization (BN) layer. The Batch Normalization (BN) layers are inserted following the convolutional layers that have trainable factors [30], where the scaling factor serves as an indicator of channel importance. The loss function used during the sparse training process is as follows:  
Figure 5:0  
The first term represents the training loss of the network, where \(x\) and \(y\) denote the input and output of the training, respectively, and \(W\) represents the training parameters of the network. The second term is the \(L1\) regularization of the scale coefficient \(\gamma\) of the Batch Normalization (BN) layer. The penalty factor \(\alpha\) is used to balance these two terms, while \(\gamma\) serves as a scaling factor introduced for each channel.