# Real-Time Detection of Insulator Defects with Channel Pruning and Channel Distillation
## 3 Materials and Methods
### The Improved YOLOv7-tiny Network Model  
The improved YOLOv7-tiny builds upon the original YOLOv7-tiny by incorporating the FasterNet architecture as the backbone network, introducing the OD-SlimNeck feature extraction framework in the neck part, and utilizing a decoupled detection head. These enhancements reduce the number of parameters while improving the accuracy of insulator defect detection, making it more suitable for target detection in drone aerial images. The framework of the improved YOLOv7-tiny algorithm is illustrated in Figure 2, with its specific structure detailed in the following sections. Figure 1: The YOLOv7-tiny algorithm framework. 1. FasterNet Backbone  
When conducting insulator defect detection tasks using drone aerial photography, it is crucial to ensure that the detection model can be deployed on mobile devices like drones while maintaining high detection speed. To achieve this, we focus on designing cost-effective, high-speed neural networks. The YOLOv7-tiny neural network architecture includes some redundant computations, leading to a higher number of floating-point operations (FLOPs). This increase in FLOPs contributes to the model's latency. The relationship among latency, FLOPs, and FLOPS is shown below:  
\[L\text{atency}= \frac{FLOPs}{FLOPS} \tag{1}\]  
FLOPs indicates the quantity of floating-point operations and FLOPS indicates floating-point operations per second. The division of FLOPs and FLOPS serves as a measure of computational latency. From Formula (1), the FasterNet network backbone improves the detection speed by effectively reducing FLOPs while increasing FLOPS [21], thereby reducing latency and increasing computation speed without compromising accuracy. Depthwise Convolution (DWConv) is a prevalent optimization technique used in backbone networks. Unlike traditional convolution methods, DWConv allocates a single convolution kernel to each channel, ensuring that each channel is processed by just one kernel. This approach significantly reduces redundant computations and floating-point operations per second (FLOPs). However, substituting regular convolution with DWConv alone may lead to reduced network accuracy. To counter this, pointwise convolution (PWConv) is often employed alongside DWConv to enhance precision [22]. In this setup, the number of DWConv channels is expanded from \(c\) to \(c^{\prime}\), exceeding the original channel count c of standard convolution to offset the accuracy loss incurred by DWConv. Nevertheless, this increase in channels necessitates more memory access, thereby elevating latency and diminishing overall computational efficiency. Memory accesses for DWConv are detailed in Equation (2), where \(h\) and \(w\) denote the height and width of the image, respectively, and c represents the channel count. \[h\times w\times 2c^{\prime}+k^{2}\times c\approx h\times w\times 2c \tag{2}\]  
The memory accesses for regular convolution are as follows:  
\[h\times w\times 2c+k^{2}\times c\approx h\times w\times 2c \tag{3}\]  
Figure 2: The framework of the improved YOLOv7-tiny algorithm. When \(c^{\prime}\) exceeds \(c\), it is clear that the memory demand for DWConv surpasses that of standard convolution. There is a need for a novel convolution module that can replace the inefficiencies associated with both regular convolution and DWConv, thereby enhancing detection speeds. Compared with regular Conv and DWConv, PConv in FasterNet only needs to apply conventional Conv to a part of the input channel to extract spatial features. The remaining channels remain unchanged. If the feature graph is stored continuously or regularly in memory, the first or last continuous channel is considered as a representative of the entire feature graph. According to the experimental results, compared to regular convolution, PConv achieves a significant reduction in FLOPs, with only 1/16 th of the computational operations required. Similarly, the memory access in PConv is also reduced to just 1/4 th of that in regular convolution.