Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper proposes a channel pruning method named CHIP to improve the existing channel pruning frameworks. The proposed method measures the importance of channels through channel independence. Extensive experiments show the effectiveness of trunk pruning.
main_review: Pros. 
(1)	The paper is well motivated and well-written. The contributions are clearly presented.

(2)	Model compression is an important topic in scaling deep models on mobile devices. It is essential to design an effective channel pruning method.

(3)	Extensive experiments are conducted to show the effectiveness of CHIP.

Cons.
(1)	The authors utilize the nuclear norm to measure the independence among channels. It is not clear for me to see the relationship between the nuclear norm and correlations between channels. Experiments should be done to show this.

(2)	The hyper-parameter $\mathcal{k}^l$ is crucial to determine the model size of the pruned model. But it is not specified in the paper. Note that the performance of the pruned model is strongly related to the layer-wise configuration of $\mathcal{k}^l$.

(3)	Since the motivation is to design the pruning method by exploring the correlation between channels. It would be great to visualize the correlation before and after pruning.

(4)	It would be better to see how CHIP performs on MobileNet. EfficientNet, etc.

limitations_and_societal_impact: The authors do not address the limitations and potential negative social impact of their work. The authors can discuss how the proposed pruning method affects the development of edge AI.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: To determine the importance of filters in DNNs, this paper proposes a new concept, channel independence, which measures the correlation of multiple feature maps. Also, this paper systematically investigates and analyzes the quantification metric, measuring scheme, and sensitiveness & reliability of channel independence in the context of filter pruning. As a result, the proposed scheme has achieved outstanding performance compared to existing methods. Although this paper is well-written and presents a solution for network pruning that has steadily gained a lot of attention, there are some minor comments as a result of my review of this paper. (Please refer to my main review)
main_review: 1. Scalability/Compatibility of the proposed method (Request for additional experiments)

1-1) Recently, activation functions with negative values such as Swish and Mish are widely used. In general, pruning is vulnerable to activation functions including negative values due to the influence of shift parameters, so please show whether the proposed method has robust properties to these activation functions.

1-2) As you know, fine-tuning requires a significant training cost, so many filter pruning techniques that achieve excellent performance without fine-tuning have been proposed. It is necessary to show whether good performance can be maintained even if fine-tuning is removed from the proposed method.

1-3) It would be good to present the results of various networks for ImageNet dataset such as ResNet-34/101 or MobileNet.

1-4) It is necessary to show whether the proposed method is effective not only for classification but also for more practical detection and segmentation networks.

1-5) Recently, there have been many attempts to perform dynamic pruning rather than static pruning within filter pruning, and therefore it would be good to include the comparison result and analysis with dynamic pruning schemes.

2. Request for additional explanation

2-1) Authors need to present an example in which the difference in the importance of filters occurs in the intra-channel and the inter-channel at the beginning of Chapter 1 or Chapter 3. And if possible, it would be good to present a numerical value indicating the proportion of such filters among all filters.

2-2) It seems that the complexity of including the inter-channel is relatively higher than considering only the intra-channel. Therefore, it is necessary to analyze the complexity.

2-3) Recently, many kernel pruning studies have been attempted, and therefore it would be good to explain the advantages of filter pruning considering inter-channel compared to kernel pruning targeting the kernel in the filter.

2-4) It seems that the baseline accuracy is slightly lower than the values originally presented in ResNet and other papers. Authors should explain how to obtain baseline accuracy.
limitations_and_societal_impact: See my main review.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper presents a metric to evaluate pre-trained dense networks to identify the feature maps that need to be dropped in an optimal manner, such that the performance of the pruned network is maximized. Authors present a simple, yet effective approach to identify the feature maps that need to be removed from the network such that it does not compromise the performance. Authors demonstrate the improvement on ImageNet dataset, which clearly demonstrates the efficacy of the proposed metric.
main_review: The paper appropriately addresses the scientific question of how to choose the correct set of feature maps for a pre-trained dense network and prune it. The idea of the paper is clear and the paper is well written. However, I have some comments/suggestions which need to be addressed during the rebuttal.

1. Authors state that the existing methods do not consider the inter-feature relations when choosing the feature maps to be eliminated during pruning. While I am not aware if any such method exists or not, it is important to note that most of the pruning method methods such that add additional objective (pruning-related) do actually consider this aspect in a implicit sense. For example, the “L1-regularization” approach increases sparsity at a layer level or at the entire network level. During this process, it takes an aggregate over a set of feature maps, thus tying them together. Similarly, there exist budget-aware pruning methods (“BAR” and “CHIPNET”) that introduce budget constraints in the pruning process, and this budget ties the different feature maps together during the selection process. My point here is that in such schemes, the mask corresponding to any feature map when goes towards a value of 0, it helps the other maps to go towards 1 in some sens – thus they are not really independent of each other. Authors seem to have completely missed this whole set of literature in their discussion. I would recommend the authors to accordingly soften their claim and add a discussion related to the above methods in the paper. 

2. Authors claim that their one-shot static pruning approach prunes the networks in an optimal sense, and state that they have experimented with dynamic learning methods where further to-be-pruned feature maps are identified as a part of the learning process. However, in the supplementary material authors demonstrate this only in 3-4 lines with a figure that contains insufficient details. My personnel experience with static and dynamic pruning has has revealed that no matter how well the static pruning  is done, dynamic learning contrbutes significantly to the pruning results. It is certainly possible that the method proposed in the paper is better and its static nature is good enough, however, this needs to be rigorously evaluated before such a claim can be made. In this regard, I would like the authors to present thorough details for the extensive set of experiments that they have conducted to reach this conclusion.

limitations_and_societal_impact: Authors have not provided limitations of their work. I would recommend that the authors discuss limitations if any. Furthers, authors are recommended to add a line or 2 on the societal impact of their work - for example are there any assumptions made in designing the method that could lead to bias in any form that could adversely impact the society.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors propose to explore and leverage the cross-channel feature information for filter pruning. Specifically, the authors propose a cross-channel correlation-based metric to measure the importance of filters. In this case, the low-independence feature maps are considered redundant. As a result, the corresponding filters that output the low-independence feature maps are viewed as uninformative and can be safely removed. Experiments on CIFAR-10 and ImageNet demonstrate the effectiveness of the proposed method. However, the experiments are not sufficient. Please see my detailed comments below.
main_review: **Contributions**: 
1.	The authors propose a new metric to measure the correlation of multiple feature maps. Relying on the proposed metric, the proposed method is able to identify important filters from a global perspective.

2.	The authors develop a low-cost fine-grained channel independence metric for efficient filter pruning.

3.	Experiments on CIFAR-10 and ImageNet show that the proposed method is able to achieve high pruning ratios while still achieving promising performance.

**Questions and points needed to be improved**: 
1.	CCP [1] also considers the inter-channel relationship during channel pruning. More discussions on the difference between the proposed method and CCP are required. Moreover, it would be better for the authors to provide more discussion on several methods that use gradient-based metrics [2-4] to measure the importance of filters.

2.	In the experiments on ImageNet, it takes too many epochs (180) to finetune the pruned models comparing with other methods (20 in SCOP [5]). It is unclear whether the good performance is resulting from longtime training.

3.	To demonstrate the effectiveness of the proposed pruning metric, it would be better for the authors to show the accuracy-pruning rate trade-off curve of different methods, which will strengthen the paper.

4.	Some important details are missing. How to set the pruning rate of each layer? Do the authors set the same pruning rate for each layer?

5.	In Algorithm 1, the authors compute the channel independence (CI) criterion among $N$ samples. However, the effect of different $N$ is unclear. It would be better for the authors to conduct more experiments to investigate the effect of different $N$.

6.	The authors conduct experiments on heavyweight networks (e.g., ResNet-56, ResNet-110, VGG-16, ResNet-50). However, the performance of the proposed method on lightweight networks is still unknown. It would be better for the authors to conduct more experiments on lightweight models (e.g., MobileNetV1 [6] and MobileNetV2 [7]).

**Reference**: 

[1] Collaborative Channel Pruning for Deep Networks. ICML 2019.

[2] Importance Estimation for Neural Network Pruning. CVPR 2019.

[3] SNIP: Single-shot Network Pruning based on Connection Sensitivity. ICLR 2019.

[4] Discrimination-aware Network Pruning for Deep Model Compression. TPAMI 2021.

[5] SCOP: Scientific Control for Reliable Neural Network Pruning. NeurIPS 2020.

[6] MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv 2017.

[7] MobileNetV2: Inverted Residuals and Linear Bottlenecks. CVPR 2018.

limitations_and_societal_impact: Please discuss the limitations of the proposed method.
review_title: null

Meta-review: This paper proposes a new pruning method for model compression that utilizes channel independence (CI). CI is given by the change of nuclear norm of the entire set of feature maps between before and after removing a channel. The method is compared with other methods extensively, and it is reported that the proposed method shows a better performance than other existing ones.  

This paper is overall well-written and it is easy to follow. The proposed method can be easily implemented. It is nice that such a simple method gives good performance improvement.  
On the other hand, there are still some concerns as follows:   
1. The authors state that the existing methods do not consider inter-channel correlation. However, it is a too strong claim and ignores several related works. For example, CCP [R1] and Spectral-Pruning [R2] consider correlation between channels, and other methods implicitly takes inter-channel correlation into account. The authors should add discussions about relation to those existing methods, and tone down the claim.    
2. The proposed method utilizes a nuclear norm type criterion to measure channel independence instead of directly measuring channel correlations. The current form of the paper does not give sufficient justification about why the proposed one is appropriate. This point should be discussed properly, for example, by showing the strong correlation between channel independence (correlation score) and nuclear norm in numerical experiments.    
3.  Some details of experimental settings such as the number of preserved filters $k_l$ are missing. These information should be included. I also recommend the authors to include accuracy-pruning rate trade-off curve of different pruning metrics.  

[R1] Collaborative Channel Pruning for Deep Networks. ICML 2019.   
[R2]  Spectral Pruning: Compressing Deep Neural Networks via Spectral Analysis and its Generalization Error. IJCAI 2020.

The concerns I listed above should be addressed. On the other hand, the proposed method shows fairly nice performance which is informative for the community. In summary, I recommend this paper for acceptance, while I think the authors should fix the issues listed above.  
Decision: Accept (Poster)