Venue: ICLR 2017 conference

Reviews:

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: This paper presents search for optimal neural-net architectures based on actor-critic framework. The method treats DNN as a variable length sequence, and uses RL to find the target architecture, which acts as an actor. The node selection is an action in the RL context, and evaluation error of the outcome architecture corresponds to reward. A auto-regressive two-layer LSTM is used as a controller and critic. The method is evaluated on two different problems, and each compared with number of other human-created architectures.

This is very exciting paper! Hand selecting architectures is difficult, and it is hard to know how far from optimal results the hand-designed networks are. The presented method is  novel. The authors do an excellent job of describing it in detail, with all the improvements that needed to be done. The tested data represents well the capability of the method. It is very interesting to see the differences between the generated architectures and human generated ones. The paper is written very clearly, and is very accessible. The coverage and contrast with the related literature is done well.

It would be interesting to see the data about the time needed for training, and correlation between time/resources needed to train and the quality of the model. It would also be interesting to see how human bootstrapped models perform and involve.

Overall, an excellent and interesting paper.
review_title: Review

final_score: rating: null
review_content: This paper is intriguing.

For a question more out of curiousity than anything else, why did you use CPUs for training the language modeling task rather than GPUs? Just an excess number of CPUs at your disposal?

Regarding the recurrent cell search, you force some of the cells, such as c_t and c_t-1 / h_t and h_t-1. Did you think of or experiment with allowing the model to add these itself?

This may be my lack of knowledge, but what's the reasoning for some of the reward choices? Last five epochs _cubed_ or c / validation_ppl**2? Obviously these are all generally ad hoc in RL but I'd just not seen these as standard choices in the past.

What was the BPTT length for the word level language modeling? Was it 35 as in Zaremba et al.?

Finally, you tested generalization on the character level Penn Treebank task, though that is quite small and odd in some ways (being a letter for letter version of the word level PTB, it also has a vocabulary of 10,000 words, no numbers / punctuation, and <unk> being a character by character "word"). Testing the generalization of the architecture to essentially the same dataset but at a character level doesn't seem an extreme generalization. Were you interested in running experiments over larger datasets such as text8 (character level), One Billion LM (word level, sentential context), or WikiText-103 (word level with context)?

Overall I'm excited for the potential that policy gradient based architecture search may bring!
review_title: Why CPUs, reward choices, generalization, BPTT length, dataset choice

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: This paper explores an important part of our field, that of automating architecture search. While the technique is currently computationally intensive, this trade-off will likely become better in the near future as technology continues to improve.

The paper covers both standard vision and text tasks and tackle many benchmark datasets, showing there are gains to be made by exploring beyond the standard RNN and CNN search space. While one would always want to see the technique applied to more datasets, this is already far more sufficient to show the technique is not only competitive with human architectural intuition but may even surpass it. This also suggests an approach to tailor the architecture to specific datasets without resulting in hand engineering at each stage.

This is a well written paper on an interesting topic with strong results. I recommend it be accepted.
review_title: Review

final_score: rating: null
review_content: Quite interesting paper!

1. Can you please comment on how generalizable the method is? Is there anything inherent to the two selected problems that make it work? What types of problems would be difficult for this method?
2. The method is clearly computationally very intensive. Would it help to bootstrap it with an initial human-architected DNN? 
review_title: Generalization

final_score: rating: null
review_content: The paper reports that "[a]fter the controller RNN is done training, we take the best RNN cell according to the lowest validation perplexity and then run a grid search over learning rate, weight initialization, dropout rates and decay epoch. The best cell found was then run with three different configurations and sizes to increase its capacity."

Is it possible for you to include (or provide here) the hyperparameters and type of dropout (i.e. recurrent dropout, embedding dropout, ...) used? Without them, replication would at best require a great deal of trial and error. As with "Recurrent Neural Network Regularization" (Zaremba et al., 2014), releasing a base set of hyper parameters greatly assists in the future work of the field.

This will likely also be desired for the other experiments, such as character LM.
review_title: Hyperparameters for SotA PTB word level LM

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: This paper proposed to use RL and RNN to design the architecture of networks for specific tasks. The idea of the paper is quite promising and the experimental results on two datasets show that method is solid.
The pros of the paper are:
1. The idea of using RNN to produce the description of the network and using RL to train the RNN is interesting and promising.
2. The generated architecture looks similar to what human designed, which shows that the human expertise and the generated network architectures are compatible.

The cons of the paper are:
1. The training time of the network is long, even with a lot of computing resources. 
2. The experiments did not provide the generality of the generated architectures. It would be nice to see the performances of the generated architecture on other similar but different datasets, especially the generated sequential models.

Overall, I believe this is a nice paper. But it need more experiments to show its potential advantage over the human designed models.
review_title: A nice paper

final_score: rating: null
review_content: 1. Why does the distributed systems use S shards? It seems that the model does not have a huge number of parameters.

2. In section 4.2, why does the model use base number of 8, not 2? It seems that it is more fair to compare with LSTM with base number 2.

review_title: several questions

Meta-review: This was one of the most highly rated papers submitted to the conference. The reviewers all enjoyed the idea and found the experiments rigorous, interesting and compelling. Especially interesting was the empirical result that the model found architectures that outperform those that are used extensively in the literature (i.e. LSTMs).
Decision: Accept (Oral)