# Real-Time Detection of Insulator Defects with Channel Pruning and Channel Distillation
## 2 New Feature Fusion Network: OD-Slimneck  
Object detection is a critical downstream task in computer vision. For various mobile devices, models with large numbers of parameters often struggle to meet the requirements for real-time detection. In order to accelerate the prediction computation, the input image in the CNN must almost undergo a similar transformation process in the backbone: the spatial information is gradually transferred to the channel. Each spatial (width and height) compression and channel expansion of the feature map may result in the partial loss of semantic information. Dense convolutional computation maximally preserves the hidden connections between individual channels, while sparse convolution completely disconnects these connections. GSConv aims to maintain connectivity throughout its application [24], but deploying it across all stages of the model can lead to increased network depth. This additional depth hinders data flow, substantially extending inference times. By the time feature maps reach the neck stage, they are at their most condensed--maximized in channel dimensions and minimized in spatial dimensions--rendering further transformations unnecessary. Consequently, a more efficient strategy is to employ GSConv solely at the neck. Initially, GSConv replaces the lightweight convolutional method, SC, offering a computational cost of about 60-70% of SC, while still providing a comparable impact on the model's learning capabilities. Subsequently, the GSDottleneck layer is integrated atop GSConv to further enhance the model's performance. Figure 5 illustrates the structure of the GSConv module. The construction of the GS bottleneck module is illustrated in Figure 6a. Following this, we have designed the cross-layer part of the network, the VoV-GSCSP module, using a one-time aggregation approach. The VoV-GSCSP module maintains satisfactory accuracy while  
Figure 4: Structure of FasterNet Block. Figure 5: The structure of the GSConv module. reducing computational and architectural complexity. Figure 6b depicts the construction of the VoV-GSCSP module. Moreover, the SlimNeck was designed with hardware-friendliness in mind, facilitating easier deployment and operation across various hardware platforms. Therefore, replacing the neck part of YOLOv7-tiny with SlimNeck not only maintains good performance, but also enhances efficiency and broadens potential applications. Due to the parameter-sharing characteristics of convolutional neural networks (CNNs) [25], traditional feature filters utilize convolutional kernel parameters that are fixed for the entire image. Consequently, when optimizing CNNs, there is often a need to upscale both the model size and parameters, as well as to utilize larger datasets. This requirement inevitably results in a significant increase in computational overhead. Omni-dimensional dynamic convolution (ODConv) expands upon traditional dynamic convolution kernels by incorporating the spatial kernel size and the number of input and output channels within the kernel space [26]. The ODConv is represented in Figure 7. This approach establishes a four-dimensional parallel strategy that leverages complementary attention mechanisms. The ODConv module substitutes the standard convolution in the backbone network. This substitution enhances network performance, rendering it more efficient and streamlined. The computational equations are as follows:  
\[\mathbf{z}=(\alpha_{1}\odot W_{1}+\dots+\alpha_{N}\odot W_{N})*y \tag{6}\]  
\[\alpha_{i}=\alpha_{w_{i}}\odot\alpha_{f_{i}}\odot\alpha_{c_{i}}\odot\alpha_{s _{i}},i=1,2,\dots,N \tag{7}\]  
where \(y\ \in\ \mathbb{R}^{\ h\times w\times c_{in}}\) and \(z\in\mathbb{R}^{\ h\times w\times c_{out}}\) represent the input and output features, respectively. \(w_{i}\), the \(i^{th}\) convolution kernel with \(c_{out}\) filters, is defined as \(W_{m}^{i}\in\mathbb{R}^{\ k\times k\times c_{in}}\), where \(m=1,\dots,\ c_{out}\).