Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: This paper presents a RNN-RL based method for the symbolic regression problem. The problem is new (to Deep RL) and interesting. My main concern is about the proposed method, where the three RL related equations (not numbered) at page 5 are also direct copy-from-textbook policy gradient equations without specific adaptation to the new application considered in this paper, which is very strange. The two conditional probability definitions considered at page 3 are not mentioned in later text. These are only fractions of the underlying method and by reading the paper back and forth several times, it is not clear of the basic algorithmic flowchart, let alone more detailed description of the related parameters. Without these information, it is impossible to have a fair judge of the novelty and feasibility of the proposed method. The empirical results are also limited in small dataset, which makes it hard to verify the generality of the superior claim.

review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper presents deep symbolic regression (DSR), which uses a recurrent neural network to learn a distribution over mathematical expressions and uses policy gradient to train the RNN for generating desired expressions given a set of points. The RNN model is used to sample expressions from the learned distribution, which are then instantiated into corresponding trees and evaluated on a dataset. The fitness on the dataset is used as the reward to train the RNN using policy gradient. In comparison to GP, the presented DSR approach recovers exact symbolic expressions in majority of the benchmarks.

Overall, this paper presents a novel technique of using an RNN to learn a distribution over mathematical expressions, and using the fitness of sampled expressions as the reward signal to train the RNN using policy gradient. The idea of using the parent and sibling nodes to predict expression nodes in an autoregressive fashion is also interesting, which exploits the fact that the operators are either binary or unary. The experimental results show that it outperforms genetic programming as well as the ablation study shows the usefulness of different extensions.

Grammar VAE (Kusner et al. 2017) learns a distribution over parse trees in a grammar and then uses Bayesian optimization to search over this space to perform symbolic regression. It would be important to empirically evaluate how GVAE performs on these symbolic regression tasks compared to DSR. 

During the search for expressions using DSR, it wasn’t clear why the algorithm chose all constants to be 1 for the first 12 benchmarks. Is it because the RNN never chose the constants in the learnt distribution or the BFGS algorithm prefers constants to be 1? Also, could it be the case that GP is being unfairly penalized for such cases as it might be trying to learn real-valued constants. Would it be possible to report what expressions GP came up with for the first 12 benchmarks?

How was the Recovery metric calculated? Does it require exact syntactic match or it also allows for semantically equivalent expression (that might be different syntactically)?

What are the runtimes for the REINFORCE and GP methods? It wasn’t clear how big the total search space of expressions was without constants. How would a random search that enumerates over all expressions and uses BFGS to compute constants work?

The evaluation is only performed on 16 expressions and only 4 expressions have real-valued constants. It would be good to evaluate the technique on more benchmarks especially with real-valued constants, and larger expressions. Is Nguyen-4 already getting to maximum length of 30? Can the expressions from AI Feyman (Udrescu & Tegmark 2019) or synthetically generated expression be used?

From the experiment benchmarks, it seems only 4 expressions had real-valued constants, and for these benchmarks GP did quite well in terms of NRMSE. What expression is GP coming up with for these benchmarks?

Matt J. Kusner, Brooks Paige, and José Miguel Hernández-Lobato. 2017. Grammar variational autoencoder. ICML 2017 
review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: This paper does a good job at specifying a solution, but never states the problem.

For the problem specification, please see the introductory paragraph in https://arxiv.org/pdf/1905.11481.pdf which I quote here to inform other readers: 
"In 1601, Johannes Kepler got access to the world’s best data tables on planetary orbits, and after 4 years and about 40 failed attempts to fit the Mars data to various ovoid shapes, he launched a scientific revolution by discovering that Mars’ orbit was an ellipse [1]. This was an example of symbolic regression: discovering a symbolic expression that accurately matches a given data set. More specifically, we are given a table of numbers, whose rows are of the form {x1, ..., xn, y} where
y = f(x1, ..., xn), and our task is to discover the correct symbolic expression for the unknown mystery function f, optionally including the complication of noise."

For people familiar with policy gradients and RNNs, you need only look at the policy RNN in Figure 1. This is a standard approach for sampling a symbolic expression (just as is often done when an RNN composes another net in AutoML). However, note that the authors introduce a bias (input keeps track of parents and siblings) to effectively incorporate hierarchy. Could the authors please expand on their heuristics for automatically choosing the (parent, sibling) input pair? Adding this to Algorithm 1 would help. It would also help with clarity if you could please add the fitting of the parameters of the symbolic expressions using BFGS to the pseudocode.

The RL approach is standard and the authors have executed it carefully and conducted the necessary ablations. However, the axes in Figure 2 should be improved. 

The experiments indicate that the proposed RL approach works better than genetic programming (GP) for what appears to be a simple benchmark. However, this is hard to judge. To truly understand the experiments, I advise readers to first look at: https://researchrepository.ucd.ie/bitstream/10197/3528/1/uy_gpem.pdf   

I would have loved to see training and test curves, mathematical expressions for the reward so it is unambiguous, ablations of the dataset (eg varying the number of data). I assume a single net with the same parameters applies to all expressions. Is this correct?

While applying policy gradients to symbolic regression is a great idea, the write up of this paper needs to improve substantially.  


review_title: Official Blind Review #1

Meta-review: This paper suggests using RNN and policy gradient methods for improving symbolic regression. The reviewers could not reach a consensus, and due to concerns about the clarity of the paper and the extensiveness of the experimental results, the paper does not appear to currently meet the level of publication. 

Also, while not mentioned in the reviews, there appears to be some work on symbolic regression aided by deep learning, (see for example, https://twhughes.github.io/pdfs/cs221_final.pdf, which was found by searching "symbolic regression deep learning")---I would thus also recommend the authors do a more thorough literature search for future revisions. 
Decision: Reject