Adaptive Pruning of Channel Spatial Dependability in Convolutional Neural Networks

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-08 05:08:28

Abstract: Deep Convolutional Neural Networks (CNNs) have demonstrated excellent performance in various multimedia application scenarios. However, complex models often require significant computational resources and energy costs. Therefore, CNN compression is crucial for addressing deployment challenges of multimedia application on resource constrained edge devices. However, existing CNN channel pruning strategies primarily focus on the "weights" or "activations" of the model, overlooking its "interpretability" information. In this paper, we explore CNN pruning strategies from the perspective of model interpretability. We model the correspondence between channel feature maps and interpretable visual perception based on class saliency maps, aiming to assess the contribution of each channel to the desired output. Additionally, we utilize Discrete Wavelet Transform (DWT) to capture the global features and structure of class saliency maps. Based on this, we propose a Channel Spatial Dependability (CSD) metric, evaluating the importance and contribution of channels in a bidirectional manner to guide model quantization pruning. And we dynamically adjust the pruning rate of each layer based on performance changes, in order to achieve more accurate and efficient adaptive pruning. Experimental results demonstrate that our method achieves significant results across a range of different networks and datasets. For instance, we achieved a 51.3% pruning on the ResNet-56 model while maintaining an accuracy of 94.16%, outperforming feature-map or weight-based pruning and other State-of-the-Art (SOTA).

Main paper: 
 related work 2.1 structed pruning: 
 Structured pruning typically prunes channels or filters as the basic units in convolutional layers, employing two main approaches: one based on the filters themselves and the other based on feature maps. Determining their saliency is a critical step in model compression techniques, and various approaches exist to address this challenge, each with its own advantages and limitations. Noteworthy methods using filter-centric approaches include [26] , who suggest sorting based on the L1 norm of convolutional kernels, considering smaller norms as less important and eligible for pruning. Additionally, approaches such as those by [18] utilize the Geometric Median for model pruning, replacing filters that are deemed too similar to others. Molchanov et al. [39] estimate filter significance by prioritizing the ranking of first-order Taylor coefficients. On the other hand, methods relying on feature maps for judgment, such as the one proposed by Lin et al. [31] , determine filter importance based on the rank of feature maps. Sui et al. [46] suggest using channel independence to measure the correlation between different feature maps, thereby enabling effective filter pruning. Among structured pruning techniques, channel pruning is particularly popular as it operates at a finer level compared to filter pruning, making it well-suited for deep learning frameworks, as shown in Figure 1 .Due to the overparameterized nature of CNNs, involving millions of parameters stacked over hundreds of layers, the prediction results of deep models are often challenging to interpret [43] . Several interpretability tools [22, 44] have been proposed to explain or reveal the decision-making process of deep models. For a given input sample, the model's output is calculated relative to the input feature maps using the backpropagation algorithm, computing the gradients of the model output with respect to each pixel in the input feature maps. These gradients indicate the influence of each pixel in the input feature maps on the output. Transforming these gradients . CSD is a method aimed at finding the optimal pruning strategy to enhance interpretability for each convolutional layer. This approach transforms the problem into a global pruning optimization problem, by calculating the contribution of each channel to predictions (by analyzing the class activation maps corresponding to each channel), and utilizing discrete wavelet transformation to preserve low-frequency components as the criterion for evaluating filter importance. Under local constraints, this method aims to maximize the compression ratio of each convolutional layer 𝑛 𝑙 , while dynamically searching for the optimal pruning rate for that layer based on performance loss. This method provides an adaptive filter pruning strategy based on model interpretability.into importance or relevance maps of the input feature maps can be achieved through post-processing or transformations such as taking absolute values, squaring or other operations to represent the impact of features on the output. Visualizing these importance or relevance maps provides a basis for interpreting the model's decisions. These images can reveal which input features play a crucial role in the model's output, offering some explanation for the model's decision. By visualizing feature map gradients or employing other methods, researchers and practitioners can better understand how the model makes predictions or decisions based on input features. Such explanations contribute to increasing the transparency of the model and aiding users in gaining a deeper understanding of the model's behavior.Based on extensive research, we have identified a significant advantage in measuring channel importance based on feature maps. However, most of these methods primarily focus on feature map information while neglecting interpretability. Therefore, we aim to leverage the interpretability of the model to provide more comprehensive guidance for a more efficient pruning process. In addition, traditional pruning methods often require extensive hyperparameter tuning and complex training, leading to substantial human resource costs. We propose an automatic method to calculate the pruning rate for each layer, aiming to achieve more efficient acceleration and achieve stronger performance. 
method 3.1 notations: 
 Let's assume a standard CNN model with L convolutional layers, indexed by 𝑙 ∈ {0, . . . , 𝐿 -1}. For the l-th convolutional layer, which contains 𝑛 𝑙 filters 𝐹 𝑙 𝑖 ∈ R 𝑛 𝑙 -1 ×𝑘 𝑙 ×𝑘 𝑙 , we can compute the number of parameters for this layer as𝑊 𝑙 = {𝐹 𝑙 1 , 𝐹 𝑙 2 , ..., 𝐹 𝑙 𝑛 𝑙 } ∈ R 𝑛 𝑙 ×𝑛 𝑙 -1 ×𝑘 𝑙 ×𝑘 𝑙 ,where 𝑛 𝑙 , 𝑛 𝑙 -1 , and 𝑘 𝑙 represent the number of output channels, number of input channels, and kernel size for the l-th layer. The total number of parameters for all convolutional layers in the entire network can be denoted as 𝑇 = 𝐿- We introduce a new symbol 𝑝 𝑙 to represent the pruning rate for the 𝑙-th layer, where 𝑝 𝑙 ranges between 0 and 1. With this, we can calculate the new number of filters for this layer as 𝑛 𝑙 new = ⌈𝑝 𝑙 × 𝑛 𝑙 ⌉. Consequently, the parameter quantity for this layer is given by𝑊 𝑙 new ∈ R 𝑛 𝑙 new ×𝑛 𝑙 -1 new ×𝑘 𝑙 ×𝑘 𝑙. Similarly, after pruning, the total parameters for the CNNs convolutional layers can be expressed as𝑇 new = 𝐿-1 0 𝑊 𝑙 new .Therefore, network pruning can be formulated as the following optimization problem:EQUATIONwhere L (𝑦, 𝑓 (𝑋,𝑊 𝑙 )) represents the loss function, 𝑦 denotes the ground truth labels, 𝑋 is the input data, 𝑓 (𝑋,𝑊 𝑙 ) is the CNN model's output function with parameters {𝑊 𝑙 𝑖 } 𝑖=1 , 𝑇 × 𝑃 𝑡 denotes the desired size of the pruned model's parameters, and 𝑃 𝑡 is the targeted total pruning rate.Over time, channel importance has been predominantly based on feature maps. However, feature maps do not comprehensively reflect the model's understanding of the data and fail to capture the dynamic changes during model training. Considering that gradients indicate the update direction and importance of each parameter in the model, they not only express the model's sensitivity to each parameter but also reflect the model's attention to different features during training. Therefore, we propose gradient-enhanced feature maps 𝐶 𝑙 𝑖 , which complement the limitations of feature map evaluation by utilizing gradient information. This approach allows for a comprehensive consideration of the model's understanding of the data and its attention to features during training. This combination enables a more accurate assessment of each channel's contribution to model performance, facilitating the selective retention of channels crucial for model decision-making and further optimizing model performance and efficiency, as shown in Figure 3 .Firstly, we conduct forward propagation through the CNNs to obtain feature maps 𝐹 𝑙 𝑖 . Subsequently, we compute the score list 𝑠 for all classes. Assuming there are 𝐾 classes, we set the score corresponding to the correct class index 𝑐 to 1 and the rest to 0, yielding the score 𝑆 𝑐 for the specific class 𝑐:EQUATIONSubsequently, we utilize 𝑆 𝑐 to backpropagate and obtain gradient information, denoted as grad:EQUATIONwhere ∇ inputs represents the gradient of the model output with respect to the inputs, and ⊙ denotes element-wise multiplication. This allows us to track the class saliency information 𝑤 𝑐 of the 𝑖-th filter in the 𝑙-th convolutional layer for a specific class 𝑐:EQUATIONwhere 𝐻 and 𝑊 represent the height and width of the feature map, respectively, and grad (𝑖,𝑗 ) denotes the gradient value at position (𝑖, 𝑗) on the feature map, obtained during model backpropagation.When attempting to interpret model decisions, it is crucial to understand which channels play a critical role in the model's final predictions. We expect convolutional layers' channels to strike the optimal balance between feature analysis and sensitivity analysis. Feature maps are commonly utilized for feature analysis and extraction, aiding in understanding how networks gradually extract abstract and useful features to accomplish specific tasks. Gradient information flowing into CNNs assigns importance values to each neuron for specific decisions. Combining these aspects allows us to visualize the image regions on which the model's predictions for specific classes rely, facilitating the interpretation of the model's decision-making process. This enhances the model's interpretability and aids in verifying whether the model makes predictions based on reasonable features.Therefore, we propose utilizing gradient-enhanced feature maps 𝐶 𝑙 𝑖 to better understand the CNNs internal structure:EQUATIONwhere 𝐹 𝑙 𝑖 denotes the feature map of the 𝑖-th channel in the 𝑙-th convolutional layer, and 𝑤 𝑐 represents the gradient information for a specific class in the feature map. Multiply these gradients with their corresponding feature maps element-wise to abtain gradient enhanced feature maps.If the feature information is directly fused with gradient information, it may lead to excessive redundancy or blurring, making it difficult to distinguish the truly beneficial parts for decision-making. To address this issue, we propose a novel filter evaluation criteria-CSD, inspired by the interpretability of the model and Discrete Wavelet Transform (DWT), as shown in Figure 3 . DWT can decompose an image into frequency components at different scales, where high-frequency components often contain noise, and extracting the low-frequency components containing the overall trend and basic structural information of the signal helps reduce redundant information. Therefore, we use DWT to decompose the gradientenhanced feature map 𝐶 𝑙 𝑖 from Equations ( 2)-( 5) into low-frequency and high-frequency parts:EQUATIONwhere Dec represents the decomposition process, 𝐿 and 𝐻 represent the low-pass and high-pass filters used for DWT, and F Low and F High represent the low-frequency and high-frequency parts of the gradient-enhanced feature map obtained through wavelet decomposition. The high-frequency part typically contains image details, while the low-frequency part contains the overall structure and global information of the image. Retaining only the low-frequency part of the gradient-enhanced feature map allows for aggregating global information and better capturing its overall characteristics:EQUATIONwhere Rec represents the reconstruction process, L represents the conjugate filter of 𝐿. We choose the Coiflet wavelet for its good compact support, helping to preserve more information of the original signal in wavelet transform. Additionally, considering that the For each layer, prune the 𝑘 𝑙 channel with the minimum CSD (which needs to be initialized), and calculate the pruning rate for each layer based on the performance loss threshold of dynamic iteration and the global pruning rate.high-frequency part may contain noise or redundant information, retaining the low-frequency part helps reduce or eliminate unnecessary details, improving the robustness and generalization ability of the features. Subsequently, we quantify the CSD of the low-frequency part of the gradient enhanced feature map by calculating its L2 norm:EQUATIONwhere ∥ • ∥ 2 denotes the L2 norm. A higher value of CSD implies that the gradient enhanced feature map may carry crucial information for model decision-making, and the corresponding channel is worth retaining.The adaptive pruning algorithm dynamically computes the pruning rate for each convolutional layer and adjusts the pruning strategy based on the model's performance, thus achieving more precise and efficient model compression, as shown in Figure 4 .Initially, we decompose the global threshold for performance degradation into progressive descent constraints, represented by the formula:EQUATIONwhere 𝑑 1 represents the initial loss threshold for the first layer (indicating the maximum acceptable performance loss for pruning the first layer), 𝛼 is the threshold for global performance degradation, and 𝜆 represents a constant scaling factor for each step based on Calculate CSD by Eqs. ( 6)-( 8).Select 𝑘 𝑙 filters with lowest CSD to prune. EQUATIONThen, for the 𝑙-th layer, the desired number of filters 𝑘 𝑙 to be pruned can be calculated as follows:maxEQUATIONwhere 𝑛 𝑙 represents the number of filters in the 𝑙-th layer, and 𝑘 𝑙 denotes the number of filters with the minimum global contribution 𝐺𝐶 (𝐶 𝑙 𝑖 ) in the 𝑙-th layer. By sorting the global contribution 𝐺𝐶 (𝐶 𝑙 𝑖 ) in the 𝑙-th layer and selecting a set of filters with the minimum global contribution 𝐺𝐶 (𝐶 𝑙 𝑖 ), we find the maximum value of 𝑘 𝑙 under the constraint of performance loss Δ𝐿 ≤ 𝑑 𝑙 . This approach allows us to retain filters that contribute more to performance while controlling the extent of pruning by limiting performance loss.In each layer, we compute the global contribution of each filter, select and prune the filters with the lowest scores, and then evaluate the model's performance after pruning. Based on the degree of performance degradation, we dynamically adjust the pruning strategy until the specified pruning rate and performance degradation threshold are reached. 
experiments 4.1 implementation details: 
 Datasets. We selected three classification datasets to evaluate the performance of our method: 1) CIFAR-10 [24] ; 2) CIFAR-100 [24] ; Evaluation Metrics. To assess the model's performance accurately, we adopted three commonly used metrics based on the SOTA: Top-1%, FLOPs and Params. Top-1% reflects the model's recognition ability for the most probable class on a specific dataset. Params and FLOPs evaluate the model's size and computational requirements, respectively. For the ImageNet dataset, due to its difficulty, we included the commonly used Top-5% accuracy as an evaluation metric to comprehensively measure the model's classification performance.Configuration. Our method is implemented with Pytorch. During the training process, a uniform preprocessing of the datasets was applied. For the CIFAR dataset, standard data augmentation techniques such as random scaling, cropping, and rotation were employed. This ensured that the size of the images used for training was uniformly set to 32×32×3. Specifically, the ResNet-32, -56, and -110 are trained for 300 epochs of fine-tuning with a batch size of 218. The momentum is 0.9, the weight decay is 0.005, and the initial learning rate is 0.01. To determine the importance of each filter, 5 batches (640 input images) were randomly sampled to calculate the CSD of each gradient-enhanced feature map in all experiments. For ImageNet and ResNet-50 is trained for 100 epochs with batch size of 256, weight decay of 1e-4, and momentum of 0.9. After pruning, the pruned model was fine-tuned using Stochastic Gradient Descent (SGD) as the optimizer on 8 NVIDIA A100-SXM GPUs.In Table 1 , we evaluated our method on the CIFAR-10 dataset in both single branch networks (VGGNet) and multi branch networks (ResNet), and compared it with existing pruning methods.For the ResNet-32 model, our method showed significant advantages compared to other pruning algorithms [5-7, 17, 26] . Our method achieved a 0.78% improvement in Top-1 accuracy, and reduced FLOPs and parameter count by 53.1% and 44.3%, respectively.For the ResNet-56 model, our CSD prunig approach achieves an accuracy improvement of 0.90% compared to the baseline model. Simultaneously, the number of parameters and FLOPs is reduced by 44.3% and 51.3%, respectively, highlighting the excellent compression performance of this method. Additionally, we compare the experimental results with other methods such as HRank [31] , LRMF [57] , FTWT [8] , and CFDP [23] , our approach not only maintains model performance but also further improves predictive accuracy, as shown in Table 1 .Table 2 : Experimental results on CIFAR-100 dataset.Algorithm Top-1(%) Δ(%) FLOPs(↓)ResNet-32SFP [17] 68.37 -0.24 50% TAS [7] 72.41 -0.18 38.5% LCCL [6] 67.39 -0.24 50% FPGM [26] 68.52 -0.66 52.6% DCPH [5] 69 For the ResNet-110 model, our algorithm demonstrates superior performance in accuracy compared to other methods, achieving a remarkable 94.43%, significantly outperforming algorithms such as HRank [31] , GAL [33] , EPruner [30] , and APIB [14] , as shown in Table 1 . This implies that our method excels in preserving predictive performance on the pruned model. Relative to the baseline model, our algorithm reduces the parameter count by 46.1% and FLOPs by 51.9%. This highlights the outstanding effectiveness of our method in model compression, resulting in a more lightweight pruned model suitable for resource-constrained environments.Finally, we validated the performance of our proposed method using VGG-16, as shown in Table 1 . Not surprisingly, our method demonstrated superior performance in deep compression. Compared to HRank [31] , GAL [33] , FPMG [18] , FalCon [52] , and SSS [21] , our method exhibited significant advantages in terms of Top-1 accuracy. Specifically, our approach achieved up to a 45.4% reduction in FLOPs (from 93.96% to 93.69%) with only a 0.27% accuracy loss. Additionally, despite achieving a similar reduction in FLOPs, our method incurred less Top-1 accuracy loss compared to FPGM (-0.27% vs. -0.35%). These results underscore the effectiveness of our proposed method.Overall, our method has better generalization ability and model compression performance on the CIFAR-10 dataset.For the ResNet-32 model, our method achieved significant performance improvement. After pruning, our model achieved a Top-1 accuracy of 71.28%, an increase of 0.13% compared to the baseline, as shown in Table 2 . This result outperforms other pruning methods (such as SFP and FPGM) and maintains a high pruning rate (50%). This indicates that our method effectively reduces the computational burden of the model while maintaining performance.For the ResNet-56 model, our method also yielded satisfactory results. After pruning, our model achieved a top-1 accuracy of 71.72%, an improvement of 2.03% compared to the baseline. Compared to other methods, our method demonstrates a more significant improvement in accuracy while still achieving a high pruning rate (51.5%). This suggests that our method has a performance advantage, even on larger models.In the case of the ResNet-110 model, our method also delivered notably significant results. Our model achieved a top-1 accuracy of 72.97%, an increase of 1.71% compared to the baseline. Compared to other pruning methods, our approach shows competitiveness in accuracy improvement while maintaining a high pruning rate (51.9%). This further demonstrates the effectiveness of our method across models of various sizes.Overall, our method consistently demonstrates superior performance in improving model accuracy while maintaining high pruning rates across ResNet models of different depths. These results underscore the effectiveness and versatility of our proposed pruning approach.We tested ResNet-50 on ImageNet. Our algorithm achieves a minimal decrease of 0.04% in Top-1 accuracy, indicating performance close to the original model. The slight reduction in Top-5 accuracy, by only 0.08%, signifies a notable success in preserving the primary predictive performance, as shown in Table 3 . In comparison to other algorithms such as ABCPruner [32] , CPS [53] , FalCon [52] , APIB [14] , and AutoPruner [37] , our method demonstrates a wellbalanced performance in maintaining accuracy and compressing the model. This highlights superior overall effectiveness.In this section, we delve into the motivations behind the CSD pruning design choices. Specifically, we examine how each component of our framework influences the overall performance of our approach. For consistency, we conduct all ablation studies on the CIFAR-10 dataset using ResNet-56.4.5.1 Gradient Enhanced Feature Map. We investigate the impact of the Gradient Enhanced Feature Map (G-E Feature Map) on the CSD pruning method. The G-E Feature Map is crucial for understanding channel importance. We compared the different performances of channel importance guided pruning using G-E feature map ensemble and feature map set calculation, as shown in Table 4. In the presence of DWT, enabling G-E feature map further enhances the model performance, resulting in a Top-1 accuracy of 94.18%. Compared to using DWT alone without enabling G-E feature map, the accuracy improves by 0.76% (93.72% -> 94.18%). Enabling G-E feature map also brings additional reductions in FLOPs and parameters, by 40% and 45%, respectively, as shown in Table 4 . This indicates that G-E feature map play a positive role in further compressing the model's computational burden and parameter count.We assess the significance of the DWT step in the CSD pruning method. We compare the pruning performance with and without the DWT step. When DWT is applied, the CSD pruning method shows a significant improvement in Top-1 accuracy, reaching 94.18%, an increase of 0.16% (94.02% -> 94.18%) points compared to the scenario without DWT. Simultaneously, under the usage of DWT, there is a 40% reduction in FLOPs and a 45% reduction in parameters, as shown in Table 4 . This indicates that DWT, while enhancing model accuracy, effectively reduces the computational burden and parameter count of the model.Rate. We explore different designs for adaptive pruning rates in the CSD pruning method. The choice of pruning rate calculation method significantly influences the overall pruning performance, comparing performance with and without adaptive pruning rate designs. When adaptive pruning rate designs were enabled, we observed variations in model performance under different pruning rates 𝑝 𝑡 . Taking a 35% pruning rate as an example, the Top-1 accuracy reached 94.21%. Compared to the scenario without adaptive pruning rate designs, FLOPs were reduced by 35%, and parameters decreased by 34%. We further validated performance at 40% and 50% pruning rates, as shown in Table 5 .In both cases, the model achieved higher accuracy while achieving more significant reductions in FLOPs and parameters. This indicates that adaptive pruning rate designs contribute to optimizing model performance under different pruning rates, leading to more effective model compression.In addition, we observed that when different batches of images are used as input, but similar or identical total pruning rates are set, the adaptive mechanism calculates approximately equal pruning rates for each layer, as shown in Figure 5 . This indicates that the model maintains a consistent pruning trend for each layer under various input conditions. This aligns with our initial hypothesis, where shallow layers exhibit high spatial dependencies, resulting in smaller pruning rates, while deep layers with lower spatial dependencies have larger pruning rates.These ablation studies provide insights into the importance of each component in the CSD pruning framework, guiding future applications and improvements.In this paper, we introduced CSD pruning, a novel pruning strategy that adaptively calculates pruning rates for each layer, making the pruning process more intelligent. Our algorithm achieves high accuracy on CIFAR-10, CIFAR-100 and ImageNet while significantly reducing the model's size and computational complexity, showcasing the exceptional performance of our approach. Furthermore, we conducted several ablation studies demonstrating the robustness of each proposed component to the initialization of the model. Overall, we have demonstrated the true advantages of our framework and evaluation criteria against benchmark standards. In future work, we aim to delve deeper into theoretical analyses on how interpretability can be combined with channel prunability.  
: 
 where the input dimension is represented as 𝑁 𝑙 , the output dimension as 𝑁 𝑙+1 , and the kernel size as 𝐾 2 , (a) filter-wise and (b) channel-wise both constitute structured pruning methods. Conversely, (c) group-wise is non-structured pruning involves grouping weights along the output dimension 𝑁 𝑙+1 .their feature representation capability, the models become excessively parameterized, making it challenging to deploy them on resource-constrained embedded devices. Therefore, effectively reducing the parameter count and floating-point operations of CNN models while ensuring that their performance does not significantly degrade is crucial for the practical application of deep learning technologies. To address this challenge, various model compression strategies have emerged, including network pruning [2, 10, 41] , parameter quantization [29, 34, 35] , low-rank approximation [1, 27, 40] , knowledge distillation [54, 58, 59 ] and others. Network pruning is an effective approach for reducing the parameter count and decreasing the computational workload of CNNs, with broad prospects for applications. Typically, based on whether the structure of the network changes before and after pruning, network pruning can be divided into two categories: structured pruning [18, 23, 31, 56] and unstructured (weight) pruning [11, 28, 36] , as shown in Figure 1 . Unstructured pruning usually refers to finegrained pruning methods with relatively high pruning accuracy. Structured pruning typically takes channels or filters in convolutional layers as the basic pruning units, preserving the model's structure. While the former can achieve higher pruning rates, it cannot leverage general-purpose hardware for acceleration, whereas the latter can be accelerated using commonly available hardware.In this paper, we focus on structured pruning, where the core challenge lies in reducing the number of intermediate features. Classic methods involve evaluating the importance of channels and pruning under certain constraints to make the model sparse. Traditional evaluation methods include norms-based [26] , geometric median-based [18] , and Hessian-based [55] approaches. Although these methods can compress the model, they may not fully capture the contribution of channels to the overall performance of the model. Norms-based methods may not provide a complete understanding of feature and gradient information, potentially overlooking important channels or retaining some less important ones. Geometric median-based methods may lack flexibility when dealing with complex datasets or large-scale models, making it challenging to adapt to various data distributions and model structures, resulting in inaccurate evaluations. Hessian-based methods typically require computing higher-order derivative information, which increases computational costs. Additionally, Hessian methods may encounter issues with local optima and computational stability in non-convex optimization problems. In summary, while these evaluations provide some guidance for channel pruning.Recent findings have demonstrated that layer-wise adaptive sparsity [9, 25, 50] is a superior pruning approach. However, these methods only consider existing evaluation criteria, as shown in Figure 2 . Inspired by these studies, we propose layer-wise adaptive compression method based on a novel filter metric standards. Our goal is to develop an effective and efficient compression method that identifies the most valuable channels in the network under performance loss constraints. We integrate forward feature information and gradient information from specific class backpropagation to extract high-level semantic information for a given task. Additionally, we employ novel metrics, such as Channel Spatial Dependability (CSD), constructed using other methods like Discrete Wavelet Transform (DWT). Unlike previous approaches, we not only focus on filter evaluation criteria but also integrate them with layer-wise adaptive sparsity methods. We conduct tests on CIFAR-10 [24], CIFAR-100 [24] and ImageNet [42] datasets across various architectures. Furthermore, we perform comprehensive tests through ablation studies to examine the robustness of our method. Our experimental results demonstrate competitive performance against current state-of-the-art (SOTA) benchmarks [23, 30, 31, 52] . We summarize the main contributions of our paper as follows:• In order to better understand the internal structure of CNNs, we propose the CSD evaluation criterion. By assessing the spatial dependability of each channel, we can determine which channels carry crucial information for model decisions, enabling effective pruning. • We integrate the CSD with layer-wise adaptive sparsity algorithms to more accurately determine the optimal number of filters for each layer of the model. This makes the pruning process more intelligent and efficient, preserving model performance stability and accuracy while reducing the number of model parameters.• We validate our algorithm on datasets such as CIFAR-10, CIFAR-100, and ImageNet, achieving state-of-the-art performance. Through extensive experimentation, we demonstrate the effectiveness and reliability of our proposed method across different datasets. 
 
 Appendix:  


Decision: This paper presents a novel CNN pruning method using Channel Spatial Dependability (CSD) metric and an adaptive pruning strategy. Some concerns were raised about generalizability beyond classification tasks and the computational overhead of the method, the overall consensus is that the paper makes a significant contribution to model compression research. Given the paper's novelty, solid rationale, and impressive experimental results, I recommend accepting this submission with the condition that the authors address the minor writing issues and clarify some technical details in the final version.

Reviews: review: paper_summary: paper_summary: This paper proposed a novel channel pruning procedure for convolutional neural networks focusing on taking the advantage of the interpretable information corresponding to each channel for accurate measurement of their importance. An end-to-end framework has been presented for prunning off less contributive channels of any 2D convolutional layers that distinctively incorporating multiple techniques namely gradient-enhanced feature map, DWT filter criterion, and layered adaptive pruning rate. Extensive experiments are covered in this study which provides all-round evidence that the proposed method is significantly better than state of arts in accuracy preservation and compression efficiency.
strength_weakness: strength_weakness: 1. *Impressive experimental results and gain from the state of arts.* The contribution of the proposed method is reasonably validated by its consistently better accuracy preservation performance across all the dimensions of the experiment setup, including dataset, network architecture, and compresssion rate.

2. *The proposed idea has solid novelty and provided sound rationale behind.* As the interpretability characteristics of machine learning models have been broadly studied recently, making it benefit model compression techniques are naturally a promissing direction.

3. *High presentation quality.* The overall writing of this draft is at a high standard. Except the minor artifacts to be resolved (as pointed out below), this draft is already in a feasible state for publishing.
limitations: limitations: 1. *Lack of perceptual results and analysis.* Sinc the proposed idea is stacked upon the saliency map based criterion, it is surprising to readers when we are not shown the actual saliency map or similar visualization of the experiemented model internals at all. It would be strong evidence to support that adopting the gradient-enhanced feature maps has a key contribution to the end pruning effect improvement if the difference can be demonstrated intuitively. Is the infusion of interpretability information perceivable from the feature map 
to explain it would be a better way to guide the selection of pruned channel?

2. *Experimetal flaws.* 
    1. In the ablation study, it is claimed enabling G-E feature map brings additional reduction in FLOPS. This is not a straightforward conclusion to draw and authors are suggested to add some derivational content to explain how this is the case.
    2. As shown in Table 4, the application of DWT has minimal effect on the accuracy gain. Besides, looking from the implementation details, this is an independent technique from CSD. It should be elaborated how DWT matters to this work and how it is related to completeness of the proposed idea.
    3. For the experiment on ImageNet, comparing the accuracy drop to [19], it is interesting to see the Top-5 accuracy drop is even higher, contradicting to Top-1 accuracy result. Any observations to why?
    4. No runtime evaluation for the efficiency of the pruning procedure, as compared to other methods, and relative to the plain training job. 

3. *Writing artifacts and nitpicking.*
    1. Abstract, "quantization pruning" -> "pruning".
    2. Introduction, "In summary, while these ..."
    3. Section 2.1, "who suggest ..." -> "which suggest ..."
    4. Figure 3, the caption is too abstract and should be focusing on explaining the quoted figure
    5. Section 3.3, the title, "Adaptive the Pruning Rate for Each Layer", remove "the"
paper_summary: paper_summary: This paper presents a method for structured CNN pruning aiming for model compression on resource-constrained devices. This method improves upon traditional pruning strategies by focusing on model interpretability and the internal structure of CNNs, using class saliency maps and Discrete Wavelet Transform (DWT) to assess the importance of channels within the model without being affected by high frequency noise. This paper also proposed an adaptive approach to dynamically adjust the pruning rates layer by layer based on performance metrics, leading to significant size and computation reduction while preserving or even improving model accuracy. This paper also provided extensive experiments and ablation study to verify the effectiveness of the proposed method.
strength_weakness: strength_weakness: 1. Unlike other methods that prune CNNs using norms or shapes, this paper introduces a new way that uses gradient-enhanced feature maps. This method provides a clearer view of how gradients affect model pruning and helps us better understand the importance of each channel.

2. This method also uses a Discrete Wavelet Transform module to remove high-frequency noise when assessing the importance of channels and filters. This improvement makes the pruning process more reliable.

3. The paper describes an adaptive pruning strategy that adjusts to changes in channel importance as pruning happens. This flexible approach ensures that the model remains accurate while becoming simpler.

4. The overall presentation is clear and effective.
limitations: limitations: 1. The method is designed only for CNNs used in classification tasks, which limits its wider application. It would be more useful if the author could extend this approach to work with other types of neural networks, like transformers, and for different tasks beyond classification.

2. The explanation of how pruning rates are adjusted for each layer needs to be clearer. It's unclear how the parameters $\lambda$, $d_i$, and $\alpha$ are chosen when the only given input is the overall pruning rate $p$.

3. A basic concern is that the main goal is to reduce the number of floating-point operations (FLOP). However, just having fewer FLOPs doesn’t always mean a model will run more efficiently on devices with limited resources (like being faster than a model with more FLOPs). This assumption should be examined more closely to confirm if minimizing FLOPs is the best way to define the problem.
paper_summary: paper_summary: This paper presents a new metric called CSD (Channel Spatial Dependability), which is used to rank the importance of convolutional filters. The main idea is to retain filters with high global contribution towards the network's performance, and remove redundant filters to obtain lightweight CNNs.
strength_weakness: strength_weakness: -- This work is a good step towards the evolving research on model compression for resource-constrained execution. Authors motivate that neural networks are overparametrized, hence its an important scientific question to obtain small models with considerable accuracy. 

-- The main innovation made by authors appears to be in the form of CSD metric to evaluate filter importance. 

-- In particular, authors apply wavelet transform on feature maps to obtain low and high frequency components. Then, l2-norm over low-frequency components is used to decide on high and less important features. The idea is interesting, with an integration of adapting layer-wise sparsities as the pruning is carried out.  

--Overall, the paper is well motivated and contributions are clearly presented.
limitations: limitations: --While overall write-up is good, there is still room for improvement. There are several typos, for example, in Table 5, '' Adptive -> Adaptive ''. 

-- Similar inconsistencies exist across the paper, for example, in caption, "nl" is referred as maximum compression ratio, but in the text, it's again associated to other terms like the number of filters and output channels, causing confusion to readers. Again, in the text, the final compression ratio for layer L is written as "nl_new" and which should be consistent across text and caption too. 

-- It is also confusing whether authors are trying to prune 'channels' or 'filters', because at some instance, 'channels' are pruned, while 'filters' at others. Please see line 136 and 351 as an example. 

--It looks like authors have not established a clear difference between channels and filters yet. There seems to be a gap in understanding both of these important structural components of CNNs as removal of either may have different impact on overall architecture, performance and complexity. 

-- Similarly, line 98 in Intro should be more clarified as to what structural changes are evident in structured and unstructured pruning. I think change happens across both methods, but what authors mentioned seems to be contrasting. 

-- It would be also important to see how does the performance varies if pruning rates are relaxed, and allowed to go beyond 0.5. I assume a severe degradation in performance which might question the effectiveness of the proposed DWT based importance estimation. 

-- The biggest concern I have is that while pruning aims to relieve the overall computational burden of CNNs, the proposed pruning pipeline seems to introduce additional computational overhead by calculating complex wavelet coefficients and then projecting L2-norm over each component. There needs to be well established reasons as to why these computationally complex steps are necessarily taken? Whether they help in retaining accuracy for extremely compressed models?

-- Finally, it would be interesting to see how effective is the proposed pruning is for already mobile-optimized models such as MobileNets or EfficientNets.
paper_summary: paper_summary: This paper focuses on CNN model pruning and establishes a description of the contribution of channels based on Discrete Wavelet Transform (DWT), referred to as the Channel Spatial Dependability (CSD) metric. It also presents an adaptive pruning strategy.
strength_weakness: strength_weakness: 1. Evaluating the contribution of channels based on Discrete Wavelet Transform (DWT) is novel for me.

2. The experiments are comprehensive.
limitations: limitations: 1. The paper focuses on pruning CNNs, yet CNNs are not only applied to classification tasks. From Equation 2 and related content, it seems that the operations described in the paper are limited to classification tasks, leading me to question the generalizability of this method. It seems that a more rigorous description of gradient calculation is necessary, rather than solely deriving conclusions from classification results. This is a part where the authors need to make improvements.

2. The abstract mentions that existing pruning methods "overlook its 'interpretability' information" and that "we explore CNN pruning strategies from the perspective of model interpretability." However, to my knowledge, this paper merely evaluates channel importance based on the Discrete Wavelet Transform (DWT), while existing methods have various ways to assess this importance, such as rank, similarity, and gradient. These methods, including the one proposed in this paper, are valid to some extent. Nonetheless, I do not agree that the proposed method is inherently interpretable, as DWT cannot be proven to be the sole metric for evaluating channel importance (if the authors consider DWT interpretable in terms of importance, then other methods could also be regarded as interpretable).

3. The overall writing quality needs improvement. There are many minor errors in both the figures and the text, such as "map map" in line 505 and "Feature Mas" in line 833. Additionally, in Figure 3, there is a "GF" label on the left side, but I couldn't find its definition anywhere in the paper. Furthermore, the structure of Figure 3 seems to require enhancement; it appears that the forward and backward processes of the network could be illustrated in a single area. The tables contain too many instances of "N/A," and it is recommended to standardize the decimal places for "FLOPs" (some are integers, while others have one decimal place).