Venue: ICLR 2017 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper introduces a reinforcement learning framework for designing a neural network architecture. For each time-step, the agent picks a new layer type with corresponding layer parameters (e.g., #filters). In order to reduce the size of state-action space, they used a small set of design choices.

Strengths:
- A novel approach for automatic design of neural network architectures.
- Shows quite promising results on several datasets (MNIST, CIFAR-10).

Weakness:
- Limited architecture design choices due to many prior assumptions (e.g., a set of possible number of convolution filters, at most 2 fully-connected layers, maximum depth, hard-coded dropout, etc.)
- The method is demonstrated in tabular Q-learning setting, but it is unclear whether the proposed method would work in a large state-action space.

Overall, this is an interesting and novel approach for neural network architecture design, and it seems to be worth publication despite some weaknesses.
review_title: review

final_score: rating: null
review_content: - Tables with top architectures in appendix are hard to read, it would be great to present them visually, in a similar style to Inception graphics.
- A qualitative analysis of the difference between the best existing human-optimized architecture would make a much stronger case: are there some common patterns the Qlearning finds over human designed architectures? are they very similar to human architectures which are then already optimal? What lessons can human designers learn from the machine designer? Do learned models tend to be deeper, wider, etc?
- Have you considered adding penalties in the rewards for adding expensive layers, for too long running time, for too much depth or width?

review_title: Details

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper looks solid and the idea is natural. Results seem promising as well.

I am mostly concerned about the computational cost of the method. 8-10 days on 10 GPUs for relatively tiny datasets is quite prohibitive for most applications I would ever encounter.
 I think the main question is how this approach scales to larger images and also when applied to more exotic and possibly tiny datasets. Can you run an experiment on Caltech-101 for instance? I would be very curious to see if your approach is suitable for the low-data regime and areas where we all do not know right away how a suitable architecture looks like. For Cifar-10/100, MNIST and SVHN, everyone knows very well what a reasonable model initialization looks like.

If you show proof that you can discover a competitive architecture for something like Caltech-101, I would recommend the paper for publication.

Minor: 
- ResNets should be mentioned in Table 
review_title: null

final_score: rating: 6: Marginally above acceptance threshold
review_content: Authors learn deep architectures on a few small vision problems using Q-learning and obtain solid results, SOTA results when limiting to certain types of layers and competitive against everything else. It would be good to know how well this performs when allowing more complex structures. Paper would be much more convincing on a real-size task such as ImageNet.
review_title: Review

final_score: rating: null
review_content: Very interesting work! 

Some questions below:

What is the size of the state space (approximately)?

Can you define the action space more formally? What is the size of the action space (is it a fixed dimension)? For example, a state of type convolution (C) "can transition to any other layer type, but each layer type has a different number of hyperparameters. How do you represent the action as a fixed dimensional vector?

How stable is the proposed Q-learning process? Have you measured the standard deviation from multiple independent runs? 

What is the network training objective if softmax is not the top layer?

How practical is the method for large-scale neural network design (e.g., for ImageNet classification) in terms of time consumption? Is the network designed on CIFAR transferable to a larger scale (ImageNet)?

review_title: questions

final_score: rating: null
review_content: How do you position yourself to: https://arxiv.org/abs/1606.02492
"Convolutional Neural Fabrics"
review_title: CNF

Meta-review: This paper comes up with a novel approach to searching the space of architectures for deep neural networks using reinforcement learning. The idea is straightforward and sensible: use a reinforcement learning strategy to iteratively grow a deep net graph (the space of actions is e.g. adding different layer types) via Q-learning. The reviewers agree that the idea is interesting, novel and promising but are underwhelmed with the execution of the experiments and the empirical results. 
 
 The idea behind the paper and the formulation of the problem are quite similar to a concurrent submission (https://openreview.net/forum?id=r1Ue8Hcxg) which has much higher scores. That paper formulates their reinforcement learning strategy using the REINFORCE algorithm while this one uses Q-learning. The major discrepancy between the papers is in the execution of the experiments. This paper had a much more restricted set of experiments on a smaller search space and thus had less impressive results. The other paper explored a much larger space of architectures and explored recurrent neural networks, achieving state-of-the-art on multiple tasks and finding exciting novel architectures that challenge current standard models (LSTMs). As a result that paper received three 9's and this one three 6's.
 
 The authors argue that they had almost the same idea but did not have access to the same amount of computing resources as the other paper. That certainly is warranted as the other paper used an *extreme* amount of compute resources for their experiments. 
 
 Given the average scores of ICLR and target acceptance rate, three 6s should normally be considered a reject. Specifically, in the absence of the other paper this would be rejected. However, the authors argue that this would be unfair, since the major difference was simply in compute resources. A major concern of the reviewers was that this approach was too computationally expensive, which is strongly contradictory to the scores for the other paper. In reality, the other paper really is just much more interesting because of the empirical results. However, I sympathize with the authors as the other paper essentially demonstrates that their idea is sensible and effective (albeit only with obscene compute resources), and the paper would effectively be "scooped" from further submissions.
 
 (Note a contrarian viewpoint might take an analogy from other fields. e.g. experimental physics papers are likely rejected if they don't have multi-billion dollar equipment to run the right set of exhaustive experiments, which is the difference between e.g. the large hadron collider and most university depts.)
 
 Pros:
 - The paper proposes a novel approach to architecture search using reinforcement learning
 - The approach is technically sound and interesting
 - The authors achieve good results automatically on benchmark tasks using their approach
 - They empirically demonstrate that their method works (via plots demonstrating the behavior of the algorithm while varying how exploratory the algorithm is)
 
 Cons:
 - The search space is very constrained (too much for us to learn anything interesting from the results)
 - The experimental results are good but underwhelming
 - Compared to e.g. Bayesian optimization methods, this approach seems very wasteful in compute resources (e.g. 1500 runs of training before the approach starts exploiting).
 
 The Program Chairs have also reviewed this paper, and taken everything into account, have reccommended this paper for poster presentation at the main conference.
Decision: Accept (Poster)