Venue: ICLR 2018 conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes to use reinforcement learning instead of pre-defined heuristics to determine the structure of the compressed model in the knowledge distillation process.

The draft is well-written, and the method is clearly explained. However, I have the following concerns for this draft:

1. The technical contribution is not enough. First, the use of reinforcement learning is quite straightforward. Second, the proposed method seems not significantly different from the architecture search method in [1][2] – their major difference seems to be the use of “remove” instead of “add” when manipulating the parameters. It is unclear whether this difference is substantial, and whether the proposed method is better than the architecture search method.

2. I also have concern with the time efficiency of the proposed method. Reinforcement learning involves multiple rounds of knowledge distillation, and each knowledge distillation is an independent training process that requires many rounds of forward and backward propagations. Therefore, the whole reinforcement learning process seems very time-consuming and difficult to be generalized to big models and large datasets (such as ImageNet). It would be necessary for the authors to make direct discussions on this issue, in order to convince others that their proposed method has practical value.

[1] Zoph, Barret, and Quoc V. Le. "Neural architecture search with reinforcement learning." ICLR (2017).
[2] Baker, Bowen, et al. "Designing Neural Network Architectures using Reinforcement Learning." ICLR (2017).

review_title: This paper proposes to use reinforcement learning instead of pre-defined heuristics to determine the structure of the compressed model in the knowledge distillation process.

final_score: rating: 4: Ok but not good enough - rejection
review_content: On the positive side the paper is well written and the problem is interesting. 

On the negative side there is very limited innovation in the techniques proposed, that are indeed small variations of existing methods. 

review_title: Reinforcement learning for estimating the structure of the compressed model in the knowledge distillation process

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: Summary:
The manuscript introduces a principled way of network to network compression, which uses policy gradients for optimizing two policies which compress a strong teacher into a strong but smaller student model. The first policy, specialized on architecture selection, iteratively removes layers, starting with architecture of the teacher model. After the first policy is finished, the second policy reduces the size of each layer by iteratively outputting shrinkage ratios for hyperparameters such as kernel size or padding. This organization of the action space, together with a smart reward design achieves impressive compression results, given that this approach automates tedious architecture selection. The reward design favors low compression/high accuracy over high compression/low performance while the reward still monotonically increases with both compression and accuracy. As a bonus, the authors also demonstrate how to include hard constraints such as parameter count limitations into the reward model and show that policies trained on small teachers generalize to larger teacher models.

Review:
The manuscript describes the proposed algorithm in great detail and the description is easy to follow. The experimental analysis of the approach is very convincing and confirms the author’s claims. 
Using the teacher network as starting point for the architecture search is a good choice, as initialization strategies are a critical component in knowledge distillation. I am looking forward to seeing work on the research goals outlined in the Future Directions section.

A few questions/comments:
1) I understand that L_{1,2} in Algorithm 1 correspond to the number of layers in the network, but what do N_{1,2} correspond to? Are these multiple rollouts of the policies? If so, shouldn’t the parameter update theta_{{shrink,remove},i} be outside the loop over N and apply the average over rollouts according to Equation (2)? I think I might have missed something here.
2) Minor: some of the citations are a bit awkward, e.g. on page 7: “algorithm from Williams Williams (1992). I would use the \citet command from natbib for such citations and \citep for parenthesized citations, e.g. “... incorporate dark knowledge (Hinton et al., 2015)” or “The MNIST (LeCun et al., 1998) dataset...” 
3) In Section 4.6 (the transfer learning experiment), it would be interesting to compare the performance measures for different numbers of policy update iterations.
4) Appendix: Section 8 states “Below are the results”, but the figure landed on the next page. I would either try to force the figures to be output at that position (not in or after Section 9) or write "Figures X-Y show the results". Also in Section 11, Figure 13 should be referenced with the \ref command
5) Just to get a rough idea of training time: Could you share how long some of the experiments took with the setup you described (using 4 TitanX GPUs)?
6) Did you use data augmentation for both teacher and student models in the CIFAR10/100 and Caltech256 experiments?
7) What is the threshold you used to decide if the size of the FC layer input yields a degenerate solution?

Overall, this manuscript is a submission of exceptional quality and if minor details of the experimental setup are added to the manuscript, I would consider giving it the full score.
review_title: very good paper

Meta-review: This is a meta-learning approach to model compression which trains 2 policies using RL to reduce the capacity (computational cost) of a trained network while maintaining performance, such that it can be effectively transferred to a smaller student network. The approach has similarities to recently proposed methods for architecture search, but is significantly different. The paper is well written and the experiments are clear and convincing. One of the reviews was unacceptable; I am not considering it (R1).
Decision: Accept (Poster)