Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: This paper proposed to train a network with training curves and corresponding parameters, and use policy search to find optimal parameter to replace hundreds or thousands of training in real case scenario, and it is clearly much faster using the trained network to infer parameters, instead of tuning the network manually.

The first point would be: what's the meaning of synthetically generating training curves other than proving that transformer achieves good performance in modeling discrete distribution? Most practical problems would not have the same distribution as the previously gathered public dataset, thus the data is not representative, and synthetic training curves just does not make sence.

The cited paper 'Learning an adaptive learning rate schedule' does not appear online.


review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: The paper investigates the possibility of learning a model to predict the training behaviour of deep learning architectures from hyperparameter information and a history of training observations. The model can then be used by researchers or a reinforcement learning agent to make better hyperparameter choices. The paper first adapts the Transformer model to be suitable to this prediction task by introducing a discretization scheme that prevents the transformer decoder's predictions from collapsing to a single curve. Next, the problem is formalized as a partially-observable MDP with a discrete action set, and PPO and SimPLe are introduced. The proposed model-based method is compared against a human and a model-free baseline training a Wide ResNet on CIFAR-10. The model-based method achieves better validation error than the other baselines that use actual data. Next, the method is compared against a human and a model-free baseline training Transformer models on the Penn Treebank dataset. While the human achieves the best performance at the end of the run, the proposed method appears to learn more quickly than the others and finishes with performance comparable to the model-free baseline.

Currently I lean towards accepting this paper for publication, despite a few issues. It asks an interesting question: can we learn a model of the training dynamics to avoid actually having to do the training? This could potentially prevent a lot of unnecessary computation and also lead to better-performing models. It then shows some experimental evidence suggesting that this is possible.

Most importantly, I would like to see a measure of variance/uncertainty like confidence intervals included in the results; otherwise it's impossible to assess whether the results are likely to be significant or not. Other questions:
1. In the PTB experiment, it looks like the human only adapts the learning rate and leaves the rest of the hyperparameters alone. Why was this policy used as the baseline? It seems extremely basic and unlikely to truly lead to optimal performance.
2. Why were more baselines from the related work not included? I understand the experiments are a proof of concept, but it would be nice to get a feeling for what some of the other methods do.
3. How do PPO and SimPLe handle partial observability? Is it principled to apply them to partially-observable environments?
4. Why not use continuous actions with a parameterized policy (e.g. Gaussian)?
5. Is it reasonable to assume that the learning dynamics of all deep learning architectures are similar enough that a model trained on one set of deep learning architectures and problems will generalize to new architectures and problems?
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: This work focuses on learning a good policy for hyperparameters schedulers, for example learning rate or weight decay, using reinforcement learning. The main contributions include 1) a discretization on the learning curves such that transformer can be applied to predict the them; 2) an empirical evaluations using the predicted learning curves to train the policy. 

The main novelties are two folds. On the methodology side, using predicted learning curves instead of real ones can speed up training significantly. On the technical side, the author presented a discretization step to use transformer for learning curve predictions. The results are mixed, we see slightly advantage over human baseline on one task but worse in the other. Human baseline does not need any training! On the writing part, it would be nice to provide more context for both transformer, Proximal Policy Optimization and Simulated Policy Learning to make the paper more self-complete.

I like the directions using surrogate to speed up HPO in general but I feel the learning curve prediction part can be improved. There are already some works, not using deep learning method, for example the following:

* Baker, Bowen, et al. "Accelerating neural architecture search using performance prediction." arXiv preprint arXiv:1705.10823 (2017).
* Domhan, Tobias, Jost Tobias Springenberg, and Frank Hutter. "Speeding up automatic hyperparameter optimization of deep neural networks by extrapolation of learning curves." Twenty-Fourth International Joint Conference on Artificial Intelligence. 2015.

Why these methods are not considered in the beginning? In my opinion, transformer is good for modeling long term dependency and concurrent predictions which is not necessarily the case for learning curves. How does the transformer based method comparing to others? 
review_title: Official Blind Review #3

Meta-review: This paper trains a transformer to extrapolate learning curves, and uses this in a model-based RL framework to automatically tune hyperparameters. This might be a good approach, but it's hard to know because the experiments don't include direct comparisons against existing hyperparameter optimization/adaptation techniques (either the ones based on extrapolating training curves, or standard ones like BayesOpt or PBT). The presentation is also fairly informal, and it's not clear if a reader would be able to reproduce the results. Overall, I think there's significant cleanup and additional experiments needed before publication in ICLR.

Decision: Reject