Venue: ICLR 2017 conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The idea of "pruning where it matters" is great. The authors do a very good job of thinking it through, and taking to the next level by studying pruning across different layers too.

Extra points for clarity of the description and good pictures. Even more extra points for actually specifying what spaces are which layers are mapping into which (\mathbb symbol - two thumbs up!).

The experiments are well done and the results are encouraging. Of course, more experiments would be even nicer, but is it ever not the case?

My question/issue - is the proposed pruning criterion proposed? Yes, pruning on the filter level is what in my opinion is the way to go, but I would be curious how the "min sum of weights" criterion compares to other approaches.
How does it compare to other pruning criteria? Is it better than "pruning at random"?

Overall, I liked the paper.
review_title: Good idea, well thought through and decently tested

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper proposes a simple method for pruning filters in two types of architecture to decrease the time for execution.

Pros:
- Impressively retains accuracy on popular models on ImageNet and Cifar10

Cons:
- There is no justification for for low L1 or L2 norm being a good selection criteria. There are two easy critical missing baselines of 1) randomly pruning filters, 2) pruning filters with low activation pattern norms on training set.
- There is no direct comparison to the multitude of other pruning and speedup methods.
- While FLOPs are reported, it is not clear what empirical speedup this method gives, which is what people interested in these methods care about. Wall-clock speedup is trivial to report, so the lack of wall-clock speedup is suspect.


review_title: Review

final_score: rating: 7: Good paper, accept
review_content: This paper proposes a very simple idea (prune low-weight filters from ConvNets) in order to reduce FLOPs and memory consumption. The proposed method is experimented on with VGG-16 and ResNets on CIFAR10 and ImageNet.

Pros:
- Creates *structured* sparsity, which automatically improves performance without changing the underlying convolution implementation
- Very simple to implement

Cons:
- No evaluation of how pruning impacts transfer learning

I'm generally positive about this work. While the main idea is almost trivial, I am not aware of any other papers that propose exactly the same idea and show a good set of experimental results. Therefore I'm inclined to accept it. The only major downside is that the paper does not evaluate the impact of filter pruning on transfer learning. For example, there is not much interest in the tasks of CIFAR10 or even ImageNet. Instead, the main interest in both academia and industry is the value of the learned representation for transferring to other tasks. One might expect filter pruning (or any other kind of pruning) to harm transfer learning. It's possible that the while the main task has about the same performance, transfer learning is strongly hurt. This paper has missed an opportunity to explore that direction.

Nit: Fig 2 title says VGG-16 in (b) and VGG_BN in (c). Are these the same models?
review_title: Simple idea with good experiments; transfer learning results would improve it

final_score: rating: 7: Good paper, accept
review_content: This paper prunes entire groups of filters in CNN so that they reduce computational cost and at the same time do not result in sparse connectivity. This result is important to speed up and compress neural networks while being able to use standard fully-connected linear algebra routines. 
The results are a 10% improvements in ResNet-like and ImageNet, which may be also achieved with better design of networks. New networks should have been also compared, but this we know it is time-consuming.
A good paper with some useful results.
review_title: Pruning Filters for Efficient ConvNets

Meta-review: The paper presents a simple but effective approach for pruning ConvNet filters with extensive evaluation using several architectures on ImageNet and CIFAR-10.
Decision: Accept (Poster)