From: ICML.cc/2021/Workshop/AutoML/Program_Chairs
title: Paper Decision
decision: Accept (Poster)

From: ICML.cc/2021/Workshop/AutoML/Paper28/Reviewer_LnoL
title: Preliminary work on an important topic
review: The paper addresses the problem of reducing the search space during NAS-type of searches, by introducing a set of rules and constraints that can prune the search. The method is evaluated on symbolic regression with promising results.

Incorporating rules and prior knowledge into search and learning methods is very important area of the study. The presented method does this through several proposed roles. The presentation is clear and the paper is well positioned within the field.

Overall, the idea is promising and is an appropriate workshop paper. To make the paper stronger, the authors should:
- Discuss the design choice of the selected priors. Why those? What are the limitations of the given set? What use cases they do not cover? And should we think about the selection of the priors in the pricipled way? 
- Evaluate the method on more than one problem. And add some other baselines -- pruning techniques have been present for a while, and adding some heuristic and learned baselines would go along way to evaluate the approach' strengths and weaknesses. 
rating: 6: Marginally above acceptance threshold
confidence: 4: The reviewer is confident but not absolutely certain that the evaluation is correct

From: ICML.cc/2021/Workshop/AutoML/Paper28/Reviewer_2SPj
title: Well written but lacking AutoML experiments
review: **Summary

Many methods (such as NAS) use autoregressive models when optimizing blackbox functions. This paper presents an approach to alter the output of such a model at an intermediate stage, in order to inject domain knowledge into the final output. Results are presented for a simple symbolic regression task.

My review score is borderline reject for two reasons: 1) The experiments are unrelated to AutoML 2) The motivation is how to inject domain knowledge, which goes against many AutoML works which seek to automate rather than add human knowledge. If there was an experiment using ML in a real setting it could be a good fit for this workshop. Without that, given my relatively limited knowledge of this area, I struggle to see the utility.

**Strengths
- The work is sufficiently broad that it could 1) aid others and 2) inspire future work.
- The paper is well written.
- If indeed there are settings where it could help with NAS, then that is an active area of research in AutoML.

**Weaknesses
- I am not convinced how useful the problem setting is. Do we really want a way to inject domain knowledge? Doesn't that go against the point of AutoML? 
- The experiments are a toy symbolic regression, where the method presented trivially wins. Adding priors into a problem *should* make it perform better, but the challenge with AutoML is we often do not know this?
rating: 5: Marginally below acceptance threshold
confidence: 2: The reviewer is willing to defend the evaluation, but it is quite likely that the reviewer did not understand central parts of the paper