Paper:
Discovering the underlying mathematical expressions describing a dataset is a core challenge for artificial intelligence. This is the problem of symbolic regression. Despite recent advances in training neural networks to solve complex tasks, deep learning approaches to symbolic regression are lacking. We propose a framework that combines deep learning with symbolic regression via a simple idea: use a large model to search the space of small models. More specifically, we use a recurrent neural network to emit a distribution over tractable mathematical expressions, and employ reinforcement learning to train the network to generate better-fitting expressions. Our algorithm significantly outperforms standard genetic programming-based symbolic regression in its ability to exactly recover symbolic expressions on a series of benchmark problems, both with and without added noise. More broadly, our contributions include a framework that can be applied to optimize hierarchical, variable-length objects under a black-box performance metric, with the ability to incorporate a priori constraints in situ.

Review:
This paper presents deep symbolic regression (DSR), which uses a recurrent neural network to learn a distribution over mathematical expressions and uses policy gradient to train the RNN for generating desired expressions given a set of points. The RNN model is used to sample expressions from the learned distribution, which are then instantiated into corresponding trees and evaluated on a dataset. The fitness on the dataset is used as the reward to train the RNN using policy gradient. In comparison to GP, the presented DSR approach recovers exact symbolic expressions in majority of the benchmarks.
Overall, this paper presents a novel technique of using an RNN to learn a distribution over mathematical expressions, and using the fitness of sampled expressions as the reward signal to train the RNN using policy gradient. The idea of using the parent and sibling nodes to predict expression nodes in an autoregressive fashion is also interesting, which exploits the fact that the operators are either binary or unary. The experimental results show that it outperforms genetic programming as well as the ablation study shows the usefulness of different extensions.
Grammar VAE (Kusner et al. 2017) learns a distribution over parse trees in a grammar and then uses Bayesian optimization to search over this space to perform symbolic regression. It would be important to empirically evaluate how GVAE performs on these symbolic regression tasks compared to DSR. 
During the search for expressions using DSR, it wasn’t clear why the algorithm chose all constants to be 1 for the first 12 benchmarks. Is it because the RNN never chose the constants in the learnt distribution or the BFGS algorithm prefers constants to be 1? Also, could it be the case that GP is being unfairly penalized for such cases as it might be trying to learn real-valued constants. Would it be possible to report what expressions GP came up with for the first 12 benchmarks?
How was the Recovery metric calculated? Does it require exact syntactic match or it also allows for semantically equivalent expression (that might be different syntactically)?
What are the runtimes for the REINFORCE and GP methods? It wasn’t clear how big the total search space of expressions was without constants. How would a random search that enumerates over all expressions and uses BFGS to compute constants work?
The evaluation is only performed on 16 expressions and only 4 expressions have real-valued constants. It would be good to evaluate the technique on more benchmarks especially with real-valued constants, and larger expressions. Is Nguyen-4 already getting to maximum length of 30? Can the expressions from AI Feyman (Udrescu & Tegmark 2019) or synthetically generated expression be used?
From the experiment benchmarks, it seems only 4 expressions had real-valued constants, and for these benchmarks GP did quite well in terms of NRMSE. What expression is GP coming up with for these benchmarks?
Matt J. Kusner, Brooks Paige, and José Miguel Hernández-Lobato. 2017. Grammar variational autoencoder. ICML 2017