From: NIPS.cc/2018/Workshop/CDNNRIA/Reviewers
comment: Thanks!

From: ~Yuxin_Zhang1
comment: Thanks for your reply!
The accuracies after retraining are listed in Tab1 and Tab2 in the paper, briefly summarized as follows：

C3D                              Increased err. (%)
(baseline: 79.94%)          2x          4x
TP(our impl.)                11.50      21.19
FP(our impl.)                 4.92      10.96
Ours                               3.56        7.02

3D-ResNet18             Increased err. (%)
(baseline: 72.50%)       2x          4x
TP(our impl.)               5.72     14.24
FP(our impl.)               1.60       6.92
Ours                              0.91      3.50
title: accuracy after retraining

From: ~Huixiang_Chen1
comment: Thank you very much for your response!
What about the accuracy after retraining?
Cause people need to make sure that the pruned network could not degrade the accuracy too much. 
title: accuracy after retraining

From: ~Yuxin_Zhang1
comment: Thanks for your interest! 

1.The accuracies after pruning (not retrained) are shown as follows:

C3D                     Increased err. (%)
                                2x          4x
TP(our impl.)    14.12%   24.57%
FP(our impl.)     9.42%    19.61%
Ours                   4.90%     7.89%

3D-ResNet18    Increased err. (%)
                                2x          4x
TP(our impl.)   12.84%   24.31%
FP(our impl.)    7.62%    16.24%
Ours                   2.12%     6.20%

The results show that our method is also better than the other two methods before retraining.

2.The percentage of the shape-sparsity is 50% for 2x acceleration and 75% for 4x acceleration.

3.The pro of structured pruning is that it generates hardware-friendly sparsity which is easy for acceleration. With im2col implementation, we remove all-zero rows and columns and concatenate the remaining ones. Then we apply GEMM with off-the-shelf libraries. While unstructured pruning often produces unstructured and random connections which is hard to implement for speedup on general hardware platforms. Even with sparse matrix kernels, the speedup is very limited. 

Generally, structured pruning focuses on acceleration and unstructured pruning focuses on compression. Pros and cons can depend on specific application scenarios. For an application caring more about low latency, structured pruning counts more; while for an application caring more about storage, unstructured pruning will be better.
title: accuracy

From: ~Huixiang_Chen1
comment: This work presents a structured pruning technique tailored for 3D NN networks since 3D NN leads to massive computation and storage requirement. Seems like this work is very similar to "Learning Structured Sparsity in Deep Neural Networks NIPS 2016" in essence.

I have a question about the results. What is the accuracy after pruning? For UCF101. How many (what is the percentage) of filters/shape/channels reduced after pruning?

Another question is, how do you compare the pros and cons for structured pruning and unstructured pruning (like deep compression by Song Han) for 3D NN?

Thanks!
title: accuracy

From: NIPS.cc/2018/Workshop/CDNNRIA
title: Acceptance Decision
decision: accept

From: NIPS.cc/2018/Workshop/CDNNRIA/Paper30/Authors
comment: Thanks to the Reviewer for the valuable comments.

We believe the proposed method is very useful for the acceleration of 3D-CNN. TP and FP are two popular pruning methods closely related with our method. We believe that these two methods are more comparable to our method than other acceleration methods (such as Low-rank Decomposition and Knowledge Distillation). That’s the reason why we chose these two methods to compare. Admittedly, given the rapid development of this area, there are more recent pruning methods (such as [*1-*3]), but due to the limited time, we haven’t compared with them on 3D-CNN, which we leave for the future work. Thanks to the Reviewer for pointing this out.

In addition, since TP and FP didn’t experiment on C3D and 3D-ResNet18 in their papers, we have to implement their methods ourselves. TP and FP are quite straight to implement, so we believe our comparison is affirmatively valid.

[*1] Liu, Zhuang, et al. "Learning efficient convolutional networks through network slimming." Computer Vision (ICCV), 2017 IEEE International Conference on. IEEE, 2017.
[*2] He, Yihui, et al. "Channel pruning for accelerating very deep neural networks." International Conference on Computer Vision (ICCV). Vol. 2. No. 6. 2017.
[*3] Gordon, Ariel, et al. "Morphnet: Fast & simple resource-constrained structure learning of deep networks." IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2018.

From: NIPS.cc/2018/Workshop/CDNNRIA/Paper30/AnonReviewer2
title: No Title
review: This work extends the pruning method in [9] to 3D CNN. By assigning different regularizations for the weight groups, the method is able to  remove unimportant groups to the networks. Although the method is not new, it is rather important for 3D CNN as it widely applied in many applications in CV. The experiments show promising results of this method in the sub-area. I am fine to accept this paper.
rating: 3: Marginally above acceptance threshold
confidence: 2: The reviewer is fairly confident that the evaluation is correct

From: NIPS.cc/2018/Workshop/CDNNRIA/Paper30/AnonReviewer3
title:    
review: 
Summary: 
The authors propose a three-dimensional regularization-based pruning method to assign different regularization parameters to different weight groups based on their importance to the network.

The authors implement the regularization based method for 3D CNN acceleration. According to the experimental results, the proposed method achieves better performance than other two state-of-the-art methods: TP and FP. However, the experiment part is somewhat insufficient. The two comparing methods are implemented by authors themselves and old (2016). 

rating: 3: Marginally above acceptance threshold
confidence: 2: The reviewer is fairly confident that the evaluation is correct

From: NIPS.cc/2018/Workshop/CDNNRIA/Paper30/Authors
comment: We thank Reviewer for the constructive comments. 

1. We refer to Wang’s article[9] to get the Eq.(2), the main reason for its setting is as follows:
(1) This function must be decremented because we need to decrease the regularization parameter of the important groups.
(2) This function needs to be equal to 0 when the average rank is $RN_g$, which ensures that at the end of pruning, there are exactly $RN_g$ weight groups to be pruned. Namely, if the average ranks less than $RN_g$, we need to increase their regularization parameters to further decrease their L1 norms; and those with greater L1 norms and rank above $RN_g$, we need to decrease regularization parameters to further increase their L1 norms. Since we decrease the L1 norms of weight groups whose ranks are less than $RN_g$ and increase the L1 norms of weight groups whose ranks are greater than $RN_g$, there should be $RN_g$ pruned weight groups at the convergence point.

2. The reason for setting $A$ to $0.5*lambda$ is that we want to make a connection between $A$ and the weight decay. The weight decay is also an important hyper-parameter in regularization. This setting prevents us from blindly setting $A$ when experimenting with different models. $A$ should be on the same order of magnitude as the weight decay while the specific number 0.5 is not so important. However, $A$ can control the speed of convergence. Greater value of $A$ results in faster convergence.

3. For a 2-D convolutional neural network with L layers, the weights of the lth convolutional layer W has the shape of N(l)*C(l)*W(l)*H(l). Here N(l), C(l), W(l) and H(l) are the dimensions along the axes of filter, channel, spatial width and spatial height. The feature map of l+1 layer has the shape of w(l)*h(l)*N(l). The computational cost will be W(l)*H(l)*C(l)*w(l)*h(l)*N(l).

While for a 3-D convolutional neural network with L layers, the weights of the lth convolutional layer W has the shape of N(l)*C(l)*W(l)*H(l)*D(l). Here N(l), C(l), W(l), H(l) and D(l) are the dimensions along the axes of filter, channel, spatial width, spatial height and spatial depth. The feature map of l+1 layer has the shape of w(l)*h(l)*d(l)*N(l). The computational cost will be W(l)*H(l)*D(l)*C(l)*w(l)*h(l)*d(l)*N(l).
 
4. We are sorry for the typo, the $\omega^\{g\}$ should be $W^{(l)}_g$, which represents the different groups of weights. We have modified it in the latest uploaded article.

From: NIPS.cc/2018/Workshop/CDNNRIA/Paper30/AnonReviewer1
title:  
review: Pros: 
- Simple idea
- Clearly presented
- Significant results

Cons:
- Conceptual originality limited, though simplicity of the idea is a plus.  
- Lack of intuition underlying the heuristics of multiplier adjustment (Eq. 2), and lack of details on the rationales and practical considerations in setting hyperparameter $A = 0.5 \lambda$.  
- Computational acceleration compared to TP and FP only briefly mentioned, how does cost scale from 2D convolution to 3D?
- Potential typo?  Line 34, what is $\omega^\{g\}$?  
rating: 4: Top 50% of accepted papers, clear accept
confidence: 2: The reviewer is fairly confident that the evaluation is correct