Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 8: Accept
review_content: Summary:

This paper introduces the use of asymptotic constraints to find
prune the search space of mathematical expressions for symbolic
regression.  This is done by training a neural network to
generate production rules conditioned on being given the
asymptotic constraints and previously generated production
rules. This neural network is then itself also used to guide a
MCTS to generate mathematical expressions.

The algorithms is compared against a reasonable set of baselines
and related algorithms.

Feedback:

This is a very clear and well-written paper. It was
straightforward to understand and very easy to see how it fits in
with the broader literature. The insight about using asymptotic
constraints makes the result a bit limited to only generating
mathematical expressions, and it would have been a bit nicer if
there was something more generically applicable to program
synthesis in general. It's not really clear to me how the
existing work extends to programs.

The evaluation was very thorough and the appropriate algorithms
were compared against the work. I came away with a good
understanding of how well the model generalizes to larger
expressions compared to existing work.

Minor notes:

The abstract is a bit inaccurate as the NN generates production rules
and not expressions.
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: Summary:
The authors of this paper propose a novel approach for symbolic regression. The simulation results demonstrate that the proposed approach can find a better function g producing similar results as desired function f by providing the additional information – the leading power of function f.

Paper strength:
1.	The proposed NG-MCTS is elegant to solve the problem of symbolic regression.
2.	Experiment results illustrate the superiority of the proposed approach

Paper weakness:

1.	I can follow most of the mathematics in the paper. But the most confusing part for me is why you feed the random partial sequence for training. Besides, how you do inference to generate a sequence of the production rules.
2.	What is the final objective function? If I do not misunderstand, it could be the cross-entropy loss between the output of GRU and the next target production rule, RMSE and the error on leading power. Then how you optimize it? Please describe more details about this.
3.	The authors of this paper introduce more information – leading power of desired function for symbolic regression but they incorporate the additional information by introducing a simple loss function term. How about the performance of baseline approaches with those kinds of information?
4.	The whole systems seem like very complicate and it would be more interesting if the authors provide sufficient ablations to decompose the complex algorithm.

review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: This paper considers a task of symbolic regression (SR) when additional information called 'asymptotic constraints' on the target expression is given. For example, when the groundtruth expression is 3 x^2 + 5 x, it behaves like x^2 in the limit x -> infinity, thus the power '2' is given as additional information 'asymptotic constraint'. In the paper's setting, for SR with univariate groundtruth functions f(x), 'asymptotic constraints' for x-> infinity and x -> 0 are given. For this situation, the paper proposes a method called NG-MCTS with an RNN-based generator and MCTS guided by it to consider asymptotic constraints. In SR, a learner is asked to acquire an explicit symbolic expression \hat{f}(x) from a given set of datapoints {x_i, f(x_i)}, but unlike the parameter optimization aspect of a standard supervised ML setting, the problem is essentially combinatorial optimization over exponentially large space of symbolic expressions of a given context-free grammar (CFG). For a given symbolic space with a prespecified CFG, extensive experimental evaluations are performed and demonstrated significant performance gain over existing alternatives based on EA. Also, quantitative evaluations about extrapolative performance and detailed evaluation of the RNN generator are also reported.

Though it includes a lot of quite interesting methods and results, the paper has two major issues: 

(1) the proposed method NG-MCTS explicitly uses the fact that the target expressions are generated from a CFG, but this assumption sounds unnatural if the target problem is SR (unlike the case of GVAE). Apparently, all results except a toy case study in 5.2 depend on artificial datasets from a CFG, and any practical advantages or impacts are still unclear because the experimental settings are very artificial. 

(2) the most important claim of this paper would be the proposal to use 'asymptotic constraints', but the availability of this information sounds too strong and again artificial in practical situations. A one-line motivation saying that 'this property is known a priori for some physical systems before the detailed law is derived' is not convincing enough.
review_title: Official Blind Review #2

Meta-review: This paper proposes 1) using neural-guided Monte-Carlo Tree Search to search for expressions that match a dataset and 2) Augments the loss to match the asymptotics of the true function when these are given.

The use of MCTS sounds more sensible than standard evolutionary search.  The augmented loss could make sense but seems extremely niche, requiring specific side information about the problem being solved.

Overall, the task is so niche that I don't think it'll be of wide interest.  It's not clear that it's solving a real problem.
Decision: Reject