From: ~Harris_Teague1
comment: Smallify is referred to in the comparison tables.  I assume this is Leclerc et. al. 2018, but you may want to be specific.
title: Ref of Smallify suggested

From: NIPS.cc/2018/Workshop/CDNNRIA
title: Acceptance Decision
decision: accept

From: NIPS.cc/2018/Workshop/CDNNRIA/Paper36/Authors
comment: We would like to thank the reviewer for their comments. Thank you for bringing up the confusion regarding citation [12], we have removed [12] from the descriptions of "weight" and "unit" pruning, replacing them with earlier work presenting the techniques. Regarding the explanation of weight pruning, we thank the reviewer for pointing out the description error, we have corrected this in the revision.

Thank you for your time and comments,
Authors

From: NIPS.cc/2018/Workshop/CDNNRIA/Paper36/Authors
comment: We thank the reviewer for the comments, we do have considerably more empirical results, as well as discussion on some possible explanations for the methods effectiveness that we can add to the final version of the paper.

In particular, we benchmark on MLPs, LeNet, and VGG16. All of which show results confirming the method's success. There are also visualizations of the change in function sensitivity as a consequence of our technique that we will add the appendix.

Again, we would like to thank the reviewer for their time and constructive comments,
Authors

From: NIPS.cc/2018/Workshop/CDNNRIA/Paper36/AnonReviewer5
title: An important direction to explore.
review: 
This paper proposed to explore an important direction i.e. to incorporate stochastic dropout into the sparsity regularisation during learning, and justified its argument with preliminary yet convincing experimental results.

My main concern is the lacking of the proof of convergence, both empirical and theoretical, for such a stochastic pruning and dropout approach. Even though there is a page limit, adding some detailed analysis or empirical studies in appendix would greatly improve the paper.

Overall, this is a well motivated work with promising results that should be of interest to large audience at the conference. 
rating: 4: Top 50% of accepted papers, clear accept
confidence: 2: The reviewer is fairly confident that the evaluation is correct

From: NIPS.cc/2018/Workshop/CDNNRIA/Reviewers
comment: The authors presented a novel regularization method for fusing post hoc pruning methods with the training process, the method is based on using the pruning strategy information and bias the dropout criteria based on the information.

I believe the intuition behind the idea is solid, it introduced a bridge between known pruning techniques such as [6] and [12], and dropout. However, there are some parts that confused me; line 56 and 57 contain multiple points:

- Citing [6] and [12] as weight pruning techniques, while it holds true for the first method, the second method prunes feature maps, not weights.

- In the explanation of weight pruning, it is mentioned that it considers whole filters, which is not the case for weight pruning nor [6]

- [12] is cited in unit and weight pruning, does it belong to both?

Pros:

- Novelty
- Methods used

Cons:

- Parts which caused confusion (mentioned above)
- Not considering comparing the results to the original methods (e.g, produce tests on LeNet and compare with results of [6] on LeNet)

Rating: 3/5
Confidence: 3/3


[6] Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections
for efficient neural network. In Advances in neural information processing systems

[12] Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, and Jan Kautz. Pruning convolutional
neural networks for resource efficient inference. 2016.