Deep symbolic regression
Authors: Brenden K. Petersen
Venue:  (2020)
Keywords: symbolic regression
reinforcement learning
automated machine learning

Abstract: Discovering the underlying mathematical expressions describing a dataset is a core challenge for artificial intelligence. This is the problem of symbolic regression. Despite recent advances in training neural networks to solve complex tasks, deep learning approaches to symbolic regression are lacking. We propose a framework that combines deep learning with symbolic regression via a simple idea: use a large model to search the space of small models. More specifically, we use a recurrent neural network to emit a distribution over tractable mathematical expressions, and employ reinforcement learning to train the network to generate better-fitting expressions. Our algorithm significantly outperforms standard genetic programming-based symbolic regression in its ability to exactly recover symbolic expressions on a series of benchmark problems, both with and without added noise. More broadly, our contributions include a framework that can be applied to optimize hierarchical, variable-length objects under a black-box performance metric, with the ability to incorporate a priori constraints in situ.

Under review conference paper at ICLR 2020
DEEP SYMBOLIC REGRESSION
A Anonymous authors
Paper under double-blind review
ABSTRACT
Discovering the underlying mathematical expressions describing dataset is core chal- lenge for artificial intelligence. This is the problem of symbolic regression. Despite recent advances in training neural networks to solve complex tasks, deep learning approaches to symbolic regression are lacking. We propose framework that combines deep learn- ing with symbolic regression via a simple idea: use a large model to search the space of small models. More specifically, we use recurrent neural network to emit distribution over tractable mathematical expressions, and employ reinforcement learning to train the network to generate better-fitting expressions. Our algorithm significantly outperforms standard genetic programming-han -haiid symbolic regression in its ability to exactly recover symbolic expressions
series of benchmark problems, both with and without added noise. More broadly, our contributions include a framework that can be applied to opti- mize hierarchical, variable-length objects under black-box performance metric, with the ability to incorporate a priori constraints in situ.
INTRODUCTION
Understanding the mathematical relationships among variables in a physical system is an integral component of the scientific process. Symbolic regression aims to identify these relationships by searching over the space of tractable mathematical expressions to best fit dataset. Specifically, given dataset of (X, y) pairs, where X E Rn and E IR, symbolic regression aims to identify a function [(X) R - R that minimizes a distance metric D(y: J [(X)) between real and predicted values. That is, symbolic regression seeks find the optimal f = arg min, D (y. f(X),, where the functional form of f is a tractable expression.
The resulting expression f may be readily interpretable and/or provide useful scientific insights simply by inspection. In contrast, conventional regression imposes a single model structure that is fixed during training. often chosen be expressive (e-g. a neural network) at the expense of being easily interpretable. However, the space of mathematical expressions is discrete (in model structure) and continuous (in model parameters), growing exponentially with the length of the expression, rendering symbolic regression an extremely challenging machine learning problem.
Given the large and combinatorial search space, traditional approaches to symbolic regression typically utilize evolutionary algorithms, especially genetic programming (GP) (Koza) 1992] Bäck et al.] 2018). In GP-based symbolic regression, population mathematical expressions is "evolved" using evolutionary operations like selection, crossover, and mutation to improve fitness function. While GP can be effective, it is also known to scale poorly to larger problems and to exhibit high sensitivity to hyperparameters.
Deep learning has permeated almost all areas of artificial intelligence, from computer vision (Krizhevsky et al. (0122) to optimal control (Mnih et al.] 2015).. However, deep learning may seem incongruous with even antithetical toward symbolic regression, given that neural networks are typically highly complex, difficult to interpret, and rely on gradient information. We propose a framework that resolves this incongruity by tying deep learning and symbolic regression together with simple idea: large model (i.e. neural

---

Under review conference paper at ICLR 2020
network) to search the space of small models (i.e. symbolic expressions). This framework leverages the representational capacity of neural networks while entirely bypassing the need to interpret network.
We present deep symbolic regression (DSR), a gradient-based approach for symbolic regression based on reinforcement learning. In DSR, recurrent neural network (RNN) emits distribution over mathematical expressions. Expressions are sampled from the distribution, instantiated, and evaluated based on their fitness to the dataset. This fitness is used as the reward signal to train the RNN parameters using policy gradient algorithm. training proceeds, the RNN adjusts the likelihood of an expression relative to its reward, assigning higher probabilities to better fitting expressions.
We demonstrate that DSR outperforms a standard GP implementation in its ability to recover exact symbolic expressions from data, both with and without added noise. We summarize our contributions as follows: 1) novel method for solving symbolic regression that outperforms standard GP, 2) am autoregressive generative modeling framework for optimizing hierarchical, variable-lene leabhe objects, 3) a framework that accommodates in situ constraints, and 4) a novel risk-seeking strategy that optimizes for best-case performance.
2 RELATED WORK
Symbolic regression. Symbolic regression has a long history of evolutionary strategies, especially GP (Koza 1992 Bäck al., 2018 [y al. 201T). Among non-evolutionary approaches, the recent Al Feyn- man algorithm (Udrescu & Tegmark 2019) is multi-staged approach to symbolic regression leveraging the observation that physical equations often exhibit simplifying properties like multiplicative separability and translational symmetry. The algorithm identifies and exploits such properties to recursively define simplified sub-probie sub-problems that can eventually be solved using simple techniques like polynomial fit or small brute force search. Brunton et al. (2016) develop sparse regression approach to recover nonlinear dynamics equations from data; however, their search space is limited to linear combinations of a library of basis functions.
Aut Our framework has many parallels to a body of works within automated machine learning (Au- toML) that use an autoregressive RNN to define a distribution over discrete objects and use reinforcement learning to optimize this distribution under black-box performance metric (Zoph & Le, 2017 Ramachan- dran et al. 2017] [Blllo et al. 2017). The key methodological difference to our framework is that these works optimize objects that are both sequential and fixed length. For example, in neural architecture search (Zoph & Le 2017). an RNN searches the space of neural network architectures, which are encoded by sequence of discrete "tokens" specifying architectural properties (e.g. number of neurons) of each layer. The length of the sequence is fixed or scheduled during training. In contrast, major contribution of our framework is defining search space that both inherently hierarchical and variable length.
The most similar AutoML work searches for neural network activation functions (Ramachandran et al. 2017). While the space of activation functions is hierarchical in nature. the authors (rightfully) constrain this space substantially by positing functional unit that is repeated sequentially, thus restricting their search space back to a lixed-length sequence. This constraint is well-justified for learning activation functions, which tend to exhibit similar hicrarchical structures. However, repeating-uni constraint is not practical for symbolic regression because the ground truth expression may have arbitrary structure.
Autoregressive models. The RNN-based distribution over expressions used in DSR is autoregressive, mean- ing each token is conditioned on the previously sampled tokens. Autoregressive models have proven to be useful for audio and image data (Oord et al. 2016a| b) in addition to the AutoML works discussed above; we further demonstrate their efficacy for hierarchical expressions.
GraphRNN defines a distribution over graphs that generates adjacency matrix column at time in autoregressive fashion (You et al.] 2018). In principle, we could have constrained GraphRNN to define the distribution over expressions, since trees are special case of graphs. However, GraphRNN constructs
2

---

Under review conference paper at ICLR 2020
graphs breadth-first, whereas expressions are more naturally represented using depth-first first traversals (Li al., 2005). Further, DSR exploits the hierarchical nature of trees by providing the parent and sibling as inputs to the RNN, and leverages the additional structure of expression trees that node's value determines number of children (e.g. cosine is unary node).
3 METHODS
Our overall approach involves representing mathematical expressions by the pre-order traversals of their corresponding symbolic expression trees, developing an autoregressive model to generate expression trees under pre-specified set of constraints, and using reinforcement learning to train the model to generate better-fitting expressions.
3.1 GENERATING EXPRESSIONS WITH A RECURRENT NEURAL NETWORK
We leverage the fact that algebraic expressions can be represented using symbolic expression trees, type of binary tree in which nodes map to mathematical operators, input variables, or constants. Operators are internal nodes and may be unary (e.g. sine) or binary (e.g. multiply). Input variables and constants terminal nodes. We encode an expression t by the pre-order traversal (i.e. depth-first, then left-to-right) of its corresponding expression tree_ We denote the 7th node in the traversal as and the length of the traversal as |r| T. Each node has a value within a given library L of possible node values or "tokens," c.g.
{+, -, sin, 2)..
Expressions are generated one node at a time along the pre-order traversal (from Ti to TT). For each node, a categorical distribution with parameters & defines the probabilities of selecting each node value from L. To capture the "context" of the expression as it is being generated. we condition this probability upon the selections of all previous nodes in that traversal. This conditional dependence can be achieved very generally using RNN with parameters 0 that outputs a probability vector v in autoregressive manner.
Specifically, the j output vector g(()) of the RNN defines the probability distribution for selecting the 8h node value Tı conditioned the previously selected node values 71:(1-1);
p p 0) = ((G)))
where L(T,) is the index in L corresponding to node value тę. The likelihood of the sampled expression is computed using the chain rule of conditional probability:
|]]
|r|
II p(T (m 1=1
((T)
The sampling process is illustrated in Figure D and described Algorithm Additional algorithmic details of the sampling process are described in Subroutines and [and Appendix Starting at the root node, a node value is sampled according to (5(1), Subsequent node values are sampled autoregressively in a depth-first, left-to-right manner until the tree is complete (i.e. all tree branches reach terminal nodes). The resulting sequence of node values is the tree's pre-order traversal, which can be used to reconstruct the tree and its
'Given expression tree (or equivalently, its pre-order traversal), the corresponding mathematical expression is unique; however, given expression, its expression tree (or its corresponding traversal) is unique. For example, x² and x x are equivalent expressions but yield different trees. For simplicity, we use somewhat abusively to refer to an expression where it technically refers to an expression tree (or cquivalently, its corresponding traversal).
In general, pre-order traversal is insufficient to uniquely reconstruct the tree. However, in this context, know how many child nodes each node has based on its value, e.g. "multiply" is binary operator and thus has children.
3

---

Under review conference paper at ICLR 2020
Pre-ordertraversal
Library
Sampled token:
Categorical distribution:
Expression free
RNN:
Parent:
Sibling:
Figure 1: A. Sampling expression from the RNN, Nodes selected one at time in autoregressive fashion along the pre-order traversal of the corresponding expression tree. For each token, the RNN outputs categorical distribution over tokens, a token is sampled, and the parent and sibling of the next token are used as the next input to the RNN. In this example, the sampled expression sin(cz)/ log(y), where the value of the constant is optimized with respect an input dataset. Numbers indicate the order in which tokens were sampled. Colors correspond to the arity of the token. White circles represent empty tokens. B. The library of tokens. C. The expression tree sampled in A.
corresponding expression. Note that different samples of the distribution have different tree structures of different size. Thus, the search space is inherently both hierarchical and variable length.
Providing hierarchical inputs to the RNN. Naively, the input to the RNN when sampling тN would be a representation (i.c. embedding one-hot encoding) of the previously sampled token, T,-1. Indeed, this is typical in related autoregressive models, e.g. when generating sentences (Vaswani et al. 2017) or for neural architecture search (Zoph & Le (017). However, the search space for symbolic regression is inherently hi- erarchical, and the previously sampled token may actually be very distant from the next token to be sampled in the expression tree. For example, the fifth and sixth tokens sampled in Figure [ are adjacent nodes in the traversal but are four edges apart in the expression tree. To better capture hierarchical information, we provide as inputs to the RNN representation of the parent and sibling node of the token being sampled. We introduce an empty token for cases in which a node does not have parent or sibling. Pseudocode for identifying the parent and sibling nodes given partial traversal is provided in Subroutine 2 in Appendix A
Constraining the search space. Under our framework, it is straightforward to apply priori constraints reduce the search space. To demonstrate, we impose several simple, domain-agnostic constraints: (1) Expressions are limited to pre-specified minimum and maximum length. We selected minimum length of 2 to prevent trivial expressions and a maximum length children of an operator should not all be constants, as the result would simply be a different constant. (3)
to ensure expressions are tractable. (2) The
The child of unary operator should not be the inverse of that operator, e.g. log(exp(x)) is not allowed. (4) Direct descendants of trigonometric operators should not be trigonometric operators, e.g. sin(x + cos(x)) is not allowed because cosine a descendant of sine. While still semantically meaningful, such composed trigonometric operators do not appear in virtually any scientific domain.
We apply these constraints in situ (concurrently with autoregressive sampling) by zeroing out the probabili- ties of selecting tokens that would violate constraint. Pseudocode for this process provided in Subroutine
For domains without this property, the number of children can be sampled from additional RNN output. A pre-order traversal plus the corresponding number of children for each node is sufficient uniquely reconstruct the tree.
4

---

Under review conference paper at ICLR 2020
Algorithm 1: Sampling expression from the RNN
function SampleExpression(e L)
input RNN with parameters 0; library of tokens &
output: Pre-order traversal т of an expression sampled from the RNN
// Empty list // Initial RNN input is empty parent and sibling // Initialize RNN cell state to zero
x = empty/ empty
ho 0 for , do
(16(0) 14) (oN(a,,
(6()) + ApplyConsttaintts L)
// Adjust probabilities // Sample next token // Append token to traversal
T. (ategorical(e((t))
T < =||
if ExxpresionCComleee(r) then return
11
12
2 ParentSibling(7)
// Compute next parent and sibling
end
14 return
Algorithm 2: Deep symbolic regression
function DSR(a, N, C, X)
input : learning rate a; batch size N: library of tokens L; input dataset (.X, y)
output: Best fitting expression 7*
Initialize RNN with parameters 0, defining distribution over expressions p(-10) null
b=0 -
repeat
= (7(0) p(-10)]==1x
1/ Sample expressions (Algorithm // Optimize constants
T t (OptimizeConstants "X)
R (0)) (cC(-(()) Ji=
// Compute rewards Compute policy gradient // Apply gradients
9 = Ei= R(r(0)V, logp(r(0(0)
10
0 aggi 92)
11
if max R > R(r*) then T* (arg max R)
// Update best expression
12
return
in Appendix A This process ensures that all samples adhere all constraints, without rejecting samples post hoc. In contrast, imposing constraints in GP-based symbolic regression can be problematic (Craenen et al. 2001). In practice evolutionary operations that violate constraints typically rejected post hoc Fortin et al. 2012).
3.2 TRAINING THE RNN USING POLICY GRADIENTS
Optimizing the parameters of the sampled expressions. Once pre-order traversal is sampled, instan- tiate the corresponding symbolic expression. The expression may have several constant tokens, which can be viewed as model parameters. We train these model parameters by minimizing the mean-squared emror
5

---

Under review conference paper at ICLR 2020
with respect to an imput dataset using nonlinear optimization algorithm, e.g. BFGS (Fletcher 2013). We perform this inner optimization loop for each sampled expression before training the RNN.
Training the RNN using policy gradients. Given a distribution over mathematical expressions p(T and a measure of performance of an expression R(T), we consider the objective to maximize J(f), defined as the expectation of under expressions sampled from the distribution:
(() (=)p(( [R((T)]
We REINFORCE (Williams 1992 to maximize this expectation via gradient ascent:
voJ(O) Vo [R(T)]
1_-p((100) V p(T
This result allows us to estimate the expectation using samples from the distribution. Specifically, we can ob- tain an unbiased estimate of [() by computing the sample mean over a batch N sampled expressions
= (7)) B=
N R(T (1) Ve pog
VoJ(O) ~
N
Reward function. A standard fitness measure in GP-based symbolic regression is normalized root-mean- square error (NRMSE), the root-mean-square error normalized by the standard deviation of the target values,
Ty. That is, given dataset of n number of (X, v) pairs, NRMSE - V A Σi= (v: ot)², where ý T. f()) are the predicted values computed using the candidate expression f. Normalization by Jy makes the metric commensurate across different datasets with potentially different ranges. However, metrics based on mean-square error exhibit extraordinarily large values for some expressions, e.g. an expression that incorrectly divides by an input variable with values near zero. For gradient-basee approach like DSR, this results in the gradient being dominated by the worst expressions, which can lead instability. We found that a bounded reward function is more stable; thus, we applied a squashing function, yielding the reward function R(T) 1/(1 NRMSE)
We introduce the "vanilla" version of DSR in Algorithm 2 Below we describe several simple extensions.
Reward baseline. The above approximation to (oJ is unbiased gradient estimate, but in practice has high variance. To reduce variance, we include a baseline function b:
N
VoJ(O) ~
1 N
Σ
R((
- -b] Vo log
As long as the baseline is not a function of the current batch of expressions, the gradient estimate is still unbiased. We define the baseline function as an apponentillly-weihheee moving average batches of re- wards. Intuitively, the gradient step increases the likelihood of expressions above the baseline and decreases the likelihood of expressions below the bascline.
Complexity penalty. We include optional complexity penalty that is added to the reward function. For simplicity, we consider the complexity metric [r]. i.e. the number of nodes in the expression tree. More complicated metrics have been proposed that capture hierarchical features of the tree and/or deduced properties of the resulting expression (Vladislavleva et al.) 2008).
Since GP-based approaches using tournament selection only rely the rankings of the fitness measure within the population, large fitness values are not problematic. Since R(T) is monotonic in NRMSE, GP unaffected by squashing.
6

---

Under review conference paper at ICLR 2020
Algorithm 3: Deep symbolic regression with baseline, risk-seeking, entropy bonus, and complexity penalty
function DSR(a, 3, Åc, Ax, 6, N, C, y)
input learning rate o; moving average coefficient 3; complexity coefficient Ac; entropy coefficient Ay; risk factor batch size N; library of tokens L; input dataset (X; y)
output: Best fitting expression 7*
Initialize RNN with parameters defining distribution over expressions p(-10)
r² = r'=null
b=0
repeat
T = {-(0) ~ p(110)]i-1.N
1/ Sample expressions (Algorithm.
1
T < (OptimizeConnttntss7)) (700,, xX,))
// Optimize constants / Compute zewards // Compute threshold
(R(7()) -cC(((()) K=IN
R. = (1 -e)
e) percentile of R
T < (r()) R()(0)) 2 Re}
// Select subset of expressions
11
+- (R(r(0): R(T 2 R₂}
// Select subset of rewards // Compute policy gradient
91 ReduceMean (R b) p((())
13
92 ReduceMcan YH V (T)
// Compute entropy gradient
0 < + a(ği G2)
// Apply gradients
// Update baseline
15
--++ ReduceMean R + (1 - 3)b
16
if max R > R(T*) then T* t (arg max R.)
// Update best expression
17
return 7'
Entropy bonus. We provide bonus to the loss function proportional to the entropy of the sampled ex- pressions. In accordance with the maximum entropy reinforcement learning framework (Haarnoja et al.) 2018), this bonus serves two purposes. First, it encourages the RNN to explore more expressions, prevent- ing premature convergence to a local optimum. In practice, this often leads to better end result. Second, it encourages the RNN to assign equal likelihood to different expressions that have equal fitness.
Risk-seeking. The policy performance, J. is defined as an expectation. However, in practice, the perfor- mance of symbolic regression is measured by the single or few best expressions. Thus, we employ novel risk-seeking technique which only the top percentile samples from each batch are used in the gradient computation. This has the effect of increasing best-case performance the expense of lower worst-case and average performances. This process is essentially the opposite of the EpOpt technique (Rajeswaran et al., 2016) used for risk-averse reinforcement learning, in which only the bottom percentile samples from each batch are used.
The complete algorithm, including reward baseline, complexity penalty, entropy bonus, and risk-seeking, shown in Algorithm 3
4 RESULTS AND DISCUSSION
Evaluating DSR. We evaluated DSR of 12 commonly used symbolic regression benchmarks (Uy jet al.] 2011). as well as additional variants in which we introduced real-valued constants to demonstrate the inner optimization loop. Each benchmark is defined by ground truth expression, training and testing dataset, and set of allowed operators, described in Table 2 in Appendix A The training data is used to compute the reward for each candidate expression, the test data is used to evaluate the best found candidate
7

---

Under review conference paper at ICLR 2020
Table 1: Performance comparison DSR and GP-based symbolic regression on 16 symbolic regression benchmarks. Bold values represent statistical significance (two-sample t-test, p 0.05). Errors represent standard deviation (n = 100 for Nguyen benchmarks; n 10 for Constant henchmarks).
GP
DSR
Benchmark
Expression
Recovery
NRMSE
Recovery
NRMSE
Nguyen-1
x3 + +x² +
51%
0.009 = 0.012
99% %
0.000 + 0.000
2
a + xd + +x² +x
22%
0.021 + 0.016
100%
0.000 0.000
3
2 x + xx² t +
6%
0.025 = 0.025
100%
0.000 + 0.000
Nguyen-4
6 + x + x² + +x3 +x² +
3%
0.024 0.012
100%
0,000 + 0.000
5
sin(x²) cos(x))
20%
0.048 = 0.4
17%
0.031 + 0.042
6guyen-6
sin(z) + sin(x +
29%
0.019 = 0.024
100%
0.000 + 0.000
Nguyen-7
log(x + 1) + log( (2² +1
0%
0.017 0.021
1%
0.020 + 0.021
Nguyen-8
v
1%
0.063 + 0.032
34%
0.081 0.112
Nguyen-9
sin(x) + sin(y²)
100%
0.000 + 0.000
100%
0.000 + 0.000
Nguyen-10
2si (r) cos(y)
61%
0.016 + 0.028
100%
0.000 + 0.000
Nguyen-11
5%
0.272 = 0.196
100%
0.000 1
Nguyen-12
x² - x²
1
0%
0.202 + 0.101
0%
0.301 0.077
Average
24.8%
0.060 = 0.023
71.0%
0.036
Constant-1
3.39x 3 + 2.12r + 1.78x
100%
0.000 = 0.000
100%
0.000 + 0.000
Constant-2
sin(x²) cos(x) 0.75
60%
0.000 = 0.001
100%
0.000 + 0.000
Constant-3
sin din(1.zz) (0.5g)
20%
0.001 = 0.002
90 %
0.001 + 0.002
Constant-4
2.7x
0%
0.184 = 0.038
80% %
0.034 +
Average
45.0%
0.046 = 0.009
92.5%
0.009
expression at the end of training, and the ground truth function is used to determine whether the best found candidate expression was correctly recovered.
As baseline, compared against standard GP-based symbolic regression. To ensure fair comparison, the same constant optimizer (BFGS) was used for both methods. We ran independent training runs for GP and DSR for each benchmark expression (n 100 for benchmarks without constants; with constants), For each experiment, we generated 1,000 candidate expressions per generation/iteration for 1,000 generations//terationn, resulting in 1,000,000 total expressions. For each training run, the expression with the best reward is selected and we record the NRMSE on the test data.
10 for benchmarks
For GP, we used the open-source software package "deap" (Fortin et al. 2012). For DSR, the RNN com- prised a single-layer LSTM of 32 hidden units. Additional hyperparameters and experiment details are provided in Appendix
In Table T we report the percentage of runs that correctly recover the expression and NRMSE on the test data for each benchmark. DSR significantly outperforms GP its ability to exactly recover benchmark expressions. DSR also outperforms GP in the average NRMSE across all expressions; however, we observe that for the few expressions with low or zero recovery rate (e.g. Nguyen-7, Nguyen-8, and Nguyen-12), GP sometimes exhibits lower NRMSE. One explanation is that GP is more prone overfitting the expression to the datasct. As an evolutionary approach, GP directly modifies the previous generation's 's expressions, allowing it make small "corrections" that decrease error each generation even if the functional form is far from correct. In contrast, DSR the RNN "rewrites" each expression from scratch each iteration after learning from gradient update, making it less prone overfitting.
8

---

Under review conference paper at ICLR 2020
Recovery
50%
0%
0.10
NRMSE
0.00
No ablations
Parentisibling
Risk seaking
Entropy bonus
Reward baseline
All improvmments
Constrain inverse
Constrain min/max
Constrain trig
All constraints
All Intronamen
Figure 2: Average recovery (top) and NRMSE (bottom) for various ablations across the Nguyen bench- marks. Dotted lines correspond to DSR (no ablations) and GP baselines. Error bars represented standard enror (n = 10). Additional descriptions for each ablation experiment are provided in Appendix
0.3
GP
Recovery
10x data
NRMSE
0.1
0.04
0.08
0.10
Noise level
Noise level
Figure 3: Average recovery (left) and NRMSE (right) for various noise levels across the 12 Nguyen bench- marks. Solid lines represent 20 data points benchmark (default); dashed lines represent 200 data points per benchmark (10-fold increase). Error bars represent standard error (n 10).
Surprisingly, DSR consistently performed best without a complexity penalty, i.c. Ac = 0. Due to the autoregressive nature of the RNN, shorter expressions tend exhibit higher likelihood than longer ones. We postulate that this property produces seff-regularizztion effect that precludes the need for an explicit complexity penalty.
Ablation studies. Algorithm 3 includes several additional components relative to the "vanilla" Algorithm
2 We performed a series of ablation studies quantify the effect of each of these components, along with the effects of the various constraints on the search space, and including the parent and sibling as input to the RNN instead of the previous node value. In Figure 2 we performed DSR on the set of 12 Nguyen benchmarks for each ablation. DSR is still competitive with GP even when removing all improvements and all constraints.
Noisy data and amount of data. We evaluated the robustness of DSR noisy data by adding indepen- dent Gaussian noise to the dependent variable, with mean zero and standard deviation proportional to the oot-mean-ssqars of the dependent variable in the training data. In Figure constant from 0 (noiseless) to 10-1 and compared the performance of GP and DSR across the set of 12 Nguyen benchmarks. DSR still outperforms GP in both recovery rate and NRMSE across noise levels.
varied the proportionality
Symbolic regression excels in the low-data setting when data is noiseless, hence, the benchmark expressions included herein include only 20 data points (see Table 2). With added noise, increasing the amount of data
9

---

Under review conference paper at ICLR 2020
smooths the reward function and may help prevent overfitting. Thus, we repeated the noise experiments using the same benchmarks but with 10-fold larger training datasets (200 points data points). As expected, recovery rates tend to increase for both methods; however, DSR maintains much larger improvement than GP at higher noise levels.
5 CONCLUSION
We introduce an unconventional approach to symbolic regression based on reinforcement learning that out- performs standard GP-based method recovering exact expressions on benchmark problems, both with and without added noise. Since both DSR and GP generate expression trees, there are many opportunities for hybrid methods, for example including several generations of evolutionary operations in the inner optimiza- tion loop. From the perspective of AutoML, the main contributions are defining flexible distribution over hierarchical, variable-length objects that allows imposing in situ constraints, and using risk-secking training to optimize best-case performance. Thus, we note that our framework is easily extensible to domains outside symbolic regression, which we save for future work; for example, searching the space of organic molecular structures for high binding affinity to reference compound. We chose symbolic regression to demonstrate our framework in part because of the large search space, broad applicability, computationally expedient inner optimization loop (sub-second), and availability of vetted benchmark problems and baseline methods.
REFERENCES
Thomas Bäck, David B Fogel, and Zbigniew Michalewicz. Evolutionarr Computation 1: Basic Algorithmss and Operators. CRC press, 2018.
Irwan Bello, Barret Zoph, Vijay Vasudevan, and Quoc V Le. Neural optimizer search with reinforcement learning. In Proceedings of the 34th International Conference on Machine Crarning-Volume 70, pp. 459- 468. JMLR. org, 2017.
Steven L Brunton, Joshua L Proctor, and J Nathan Kutz. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the National Academy of Sciences, 113(15):3932-399) -937, 2016.
BGW Craenen, AE Eiben, and E Marchiori. How to handle constraints with evolutionary algorithms. Prac- tical Handbook Of Genetic Algorithms: Applications, pp. 341-361, 2001.
Roger Fletcher. Practical Methods of Optimization. John Wiley Sons, 2013.
Félix-Antoine Fortin, François-Michel De Rainville, Mare-André Gardner, Mare Parizeau, and Christian Gagné. Deap: Evolutionary algorithms made easy. Journal of Machine Learning Research, 13(Jul): 2171-2175, 2012.
Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. arXiv preprint
2018.
John Koza. Genetic Programming: On the Programming of Computers by Means of Natural Selection, volume 1. MIT press, 1992.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pp. 1097-1105, 2012.
Xin Li, Chi Zhou, Weimin Xiao, and Peter C Nelson. Prefix gene expression programming. In Late breaking paper at Genetic and Evolutionary Computation Conference (GECCO Washington, DC, USA, pp. 25-29, 2005.
10

---

Under review conference paper at ICLR 2020
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Mare Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529, 2015.
Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499 2016a.
Aaron van den Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. Pixel recurrent neural networks. arXiv preprint arXiv:1601.06755 2016b.
Aravind Rajeswaran, Sarvjeet Ghotra, Balaraman Ravindran, and Sergey Levine. Epopt: Learning robust neural network policies using model ensembles. arXiv preprint arXiv:161000/283, 2016.
Prajit Ramachandran, Barret Zoph, and Quoe V Le. Searching for activation functions. arXiv preprint arXiv:1710.0594112011 2017.
Silviu-Marian Udrescu and Max Tegmark. Ai feynman: a physics-inspired method for symbolic regression. arXiv preprint 1905 11481, 2019.
Nguyen Quang Uy, Nguyen Xuan Hoai, Michael O'Neill, Robert I McKay, and Edgar Galván-López. Semantically crossover in genetic programming: application to real-valued symbolic regression. Genetic Programming and Evolvable Machines, 12(2):91-119, 2011.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008, 2017.
Ekaterina J Vladislavleva, Guido F Smits, and Dick Den Hertog. Order of nonlinearity as a complexity measure for models generated by symbolic regression via pareto genetic programming. IEEE Transactions on Evolutionary Computation, 13(2):333-349 2008.
Ronald J Williams. Simple statistical gralient-following algorithms for connectionist reinforcement learn- ing. Machine learning, 8(3-4)2229-555 1992.
Jiaxuan You, Rex Ying, Xiang Ren, William Hamilton, and Jure Leskovec, Graphrmm: Generating realistic graphs with deep auto-regressive models. arXiv preprint arXiv:1802.08773, 2018.
Barret Zoph and Quoc V Le. Neural architecture search with reinforcement learning. International Confer- ence on Learning Representations, 2017.
11

---

Expression Under review conference paper at ICLR 2020
APPENDIX A
Hyperparameters. DSR hyperparameters are listed Table 3 GP byperparameters are listed in Table The same hyperparameters used for all experiments and all benchmark expressions.
Additional details for performance comparison experiments. Details of the benchmark symbolic regres- sion problems shown in Table 2 All benchmarks use the function set {+, -, x, +, ,in, cos, exp, log): To ensure closures, we use protected versions of operators: log returns the logarithm of the absolute value of its argument, and +, exp, and log return I for arguments that would cause overflow or other errors. Bench- marks without constants can be recovered exactly, thus recovery is defined by exact correctness (modulo floating point precision error). Note Nguyen-8 can be recovered via expl 2+ log(x)) and Nguyen- 1l can be recovered via exp(y log(r)).
Table 2: Benchmark symbolic regression problem specifications. U(a, b, n) denotes n random points uni- formly sampled between and for each input variable. Training and testing datasets use different random seeds.
Name N N
Variables
Dataset
Constant?
x + x² +
U(-1, 20)
No
x
r + x² tx + x + x² + +x² +x
U(-1,1,20)
No
Nguyen-3
2
U(-1,1,20)
No
N
2.6
x + x² + x² xx²
U(-1,1,20)
No
5
sin(x²) cos( (z) -
U(-1,1,20)
No
Nguyen-6
sin(x) + sin x +x²)
U(-1,1,20)
No
Nguyen-7
log(æ + 1) + log (x +1) V
U(0,2,20)
No
Nguyen-8
U(0,4,20)
No
9
sin(r) + sin
U(0,1,20) No
Nguyen-10
sin (z) cos
U(0,1,20) No
Nguyen-11 11
2
x x. - x3 + y 3.39r 2.12x 1.78x
U(0,1,20) No
Nguyen-12 12
U(0,1,20)
No
Constant-1
U(-1,1,20)
Yes
Constant-2 2
sin(x²) cos(x)) - 0.75
U(-1,1,20)
Yes
Constant-3
sin (1.5z) coscco.5)) (0.5g) 2.7x
U(0,1,20)
Yes
Constant-4
2
U,2,1,20)
Yes
Table DSR hyperparameters
Table 4: GP hyperparameters
Parameter
Value
Parameter
Value
Batch size Iterations
1,000
Population size
1,000
1,000
Generations
1,000
Learning rate (a)
0.0003
Fitness function
NRMSE
Entropy coefficient (Ax)
0.08
Initialization method
Full
Complexity coefficient (Ac)
0
Sclection type
Tournament
Moving average coefficient (9)
0.5
Tournament size (A)
3
Risk factor (c)
0.1
Crossover probability
0.5
Mutation probability
0.1
Minimum subtree depth (dmin)
0
Maximum subtree depth (dmax)
2
12

---

Under review conference paper at ICLR 2020
For benchmarks with constants, constants are optimized using BFGS with an initial guess of 1.0 for each constant. We ensured that all benchmarks with constants do not get stuck in a poor local optimum when optimizing with BFGS and the candidate functional form is correct. Since floating point constants cannot be recovered exactly, for benchmarks with constants we manually determined correctness of the functional form by inspection. Since constant optimization a computational bottleneck, we limited each expression to three constants for both DSR and GP experiments.
For GP, the initial population of expressions is generated using the "full" method (Koza 1992) with depth randomly selected between dmin and dmax. The selection operator is defined by deterministic tournament se- lection, in which the expression with the best fitness among k randomly selected expressions is chosen. The crossover operator is defined by swapping random subtrees between two expressions. The point mutation operator is defined by replacing a random subtree with a new subtree initialized using the "full" method with depth randomly selected between dmin and dmax-
Additional details for ablation studies. In Figure 2 "Parent/sibling" denotes that the previous node of the traversal is provided as input to the RNN, rather than the parent and sibling node. "Risk-seeking" denotes no risk-seeking, equivalent to e = 1. "Entropy bonus" denotes no entropy bonus, equivalent to AH = 0. "Reward baseline" denotes no reward baseline, equivalent to 3 = "All improvements" denotes combining ablations for Parent/sibling, Risk, Entropy, and Baseline. "Constrain trig" denotes no constraint precluding nested trigonometric operators. "Constrain inverse" denotes no constraint precluding inverse unary operators. "Constrain min/max" denotes no constraint precluding minimum or maximum length. (If the maximum length is reached, the expression appended with until complete.) "Constraints" denotes combining ablations for Trig, Inverse, and Min/max. "All constraints & improvements" denotes combining all ablations.
Training curves. Figures and 5 show the reward (1/(1 + NRMSE)) and recovery rate, respectively, as a function of training step (DSR ieeaation or GP generation). For benchmarks with constants, the constant optimizer can allow both algorithms to quickly reach reward near 1.0. For these benchmarks, we provide zoomed inset plots demonstrate if and when all independent training runs correctly recovery the expression (Figure 4 Constant-1, I, Constant-2, and Constant-3). Note that the NRMSE and recovery values in Table 1 correspond to the final point on each curve in Figures 4 and 5
respectively.
Additional subroutines. DSR includes several subroutines used when sampling expression from the RNN and during training. In Subroutine [ we describe the function ApplyConstraintss(ts L) used in Algorithm [ which zeros out the probabilities of tokens that would violate any given constraints. Within this subroutine, the user-specific function ViolatesConstrain((. L.) returns TRUE if adding the ith token of the library to the partial traversal 7 would violate any user-spedified constraints, and FALSE otherwise.
In Subroutine 2 we describe the function ParentSibling(r) used in Algorithm I which computes the parent and sibling of the next token to be sampled. This subroutine uses the following logic. the final node in the partial traversal is unary or binary operator, then that node is the parent and there is no sibling. Otherwise, the subroutine iterates backward through the traversal until finding node with an unselected child node. That node is the parent and the subsequent node is the child. Within this subroutine, the function (rity(T.) simply returns the arity (number of arguments) of token Tú, i.e two for binary operators, one for unary operators, or zero for input variables and constants.
Subroutine 3 we describe the function OptimizeCCnstanttsrr, X,y) used in Algorithms and 3, which optimizes the placeholder constants of an expression with respect to input dataset (X, y) using black-box optimizer, c.g. BFGS. Within this subroutine, the function Instantiate(7) instantiates the symbolic expression function c( with inputs X and parameters (constants) c, and the function ReplaceConstants((,cc replaces the placeholder constants in the expression with the optimized constants.
13

---

Under review conference paper at ICLR 2020
Nguyen-1
Nguyen-2
Nguyen-3
Nguyen-4
1.00
0.95
Reward
0.90
0.85
0.80
DSR
0.75
Nguyen-5
Nguyen-6
Nguyen-7
Nguyen-8
1.00
0.95
Reward
0.90
0.85
0.80
0.75
Nguyen-9
Nguyen-10 10
Nguyen-11 11
Nguyen -22
1.00
0.95
Reward
0.90
Constant-1
Constant-2
Constant-3
Constant-4
1.00
0.95
Reward
0.90
1.000
1.000
0.85
0.80
0.599
0.989
150
200
250
250
320
350
900
950
1000
0.75
1000
Training step
Training step
Training
Training
Figure 4: Reward training curves for DSR and GP for each of the 16 benchmark expressions. Each curve shows the best reward (1/(1 + NRMSE)) found so far as function of training step, averaged across all 100 independent training runs. A value of 1.0 denotes that all training runs have recovered the correct expression by that training step. Error bands represent standard deviation (n 100).
14

---

Under review conference paper at ICLR 2020
Nguyen-1
Nguyen-2
Nguyen-3
Nguyen-4
Recovery rate
09
N
6
Nguyen-7
N 8
100%
Recovery rate
Nguyen-9
Nguyen-10
Nguyen-11
Nguyen-12
100%
Recovery rate
50%
Constant-1
Constant-2
Constant-3
Constant-4
100%
Recovery rate
50%
0%
250
500
750
1000
250
500
750
1000
250
500
750
1000
250
500
750
1000
Training step
Training step
Training step
Training step
Figure 5: Recovery rate training curves for DSR and GP for each of the 16 benchmark expressions. Each curve shows the fraction of independent training runs that correctly recovered the benchmark expression as a function of training step. A value of 100% denotes that all training runs have recovered the correct expression by that training step.
15

---

Under review conference paper at ICLR 2020
Subroutine 1: Apply generic constraints in situ when sampling from the RNN
function ApplyConstraints(e)) T)
input : :Categorical probabilities v corresponding library of tokens L; partially sampled traversal
output: Adjusted categorical probabilities /
L= |p for L..... L do
// Length of library
if CoglattsConstrain((,... C,) then
1/4 < 0
// Constrain that token
end
// Norma lize back to
M return V
Subroutine 2: Computing parent and sibling inputs to the RNN
function ParentSibling(7)
input Partially sampled traversal т
output: Parent and sibling tokens of the next token to be sampled
L= |r|
// Length of partial traversal
counter 0
// Counter for number of unselected nodes
if Arity(Tr.) > then
parent TL sibling = empty return parent||sibling
for ,,.., do
// Iterate backward
counter + counter + Arity(T:) - if counter then
10
11
parent
12
sibling T
13
return parent||sibling
end
Subroutine 3: Optimize the constants an expression (inner optimization loop)
function OptimizeConstants(fr XX y)
input Expression т with placeholder constants c: input dataset (X, y)
output: Expression r* with optimized constants
f(X,c) c) Instantiate(r)
// Instantiate the symbolic expression // Minimize error (e.g. with BFGS)
c* arg min, 1ly c)
ReplaceConsstans((...)
// Replace placeholder constants
return 7'
16


Reviews:
[{"review_id": "SyeAQpbg9S", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a RNN-RL based method for the symbolic regression problem. The problem is new (to Deep RL) and interesting. My main concern is about the proposed method, where the three RL related equations (not numbered) at page 5 are also direct copy-from-textbook policy gradient equations without specific adaptation to the new application considered in this paper, which is very strange. The two conditional probability definitions considered at page 3 are not mentioned in later text. These are only fractions of the underlying method and by reading the paper back and forth several times, it is not clear of the basic algorithmic flowchart, let alone more detailed description of the related parameters. Without these information, it is impossible to have a fair judge of the novelty and feasibility of the proposed method. The empirical results are also limited in small dataset, which makes it hard to verify the generality of the superior claim.\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "Skgkm5WIKr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper does a good job at specifying a solution, but never states the problem.\n\nFor the problem specification, please see the introductory paragraph in https://arxiv.org/pdf/1905.11481.pdf which I quote here to inform other readers: \n\"In 1601, Johannes Kepler got access to the world’s best data tables on planetary orbits, and after 4 years and about 40 failed attempts to fit the Mars data to various ovoid shapes, he launched a scientific revolution by discovering that Mars’ orbit was an ellipse [1]. This was an example of symbolic regression: discovering a symbolic expression that accurately matches a given data set. More specifically, we are given a table of numbers, whose rows are of the form {x1, ..., xn, y} where\ny = f(x1, ..., xn), and our task is to discover the correct symbolic expression for the unknown mystery function f, optionally including the complication of noise.\"\n\nFor people familiar with policy gradients and RNNs, you need only look at the policy RNN in Figure 1. This is a standard approach for sampling a symbolic expression (just as is often done when an RNN composes another net in AutoML). However, note that the authors introduce a bias (input keeps track of parents and siblings) to effectively incorporate hierarchy. Could the authors please expand on their heuristics for automatically choosing the (parent, sibling) input pair? Adding this to Algorithm 1 would help. It would also help with clarity if you could please add the fitting of the parameters of the symbolic expressions using BFGS to the pseudocode.\n\nThe RL approach is standard and the authors have executed it carefully and conducted the necessary ablations. However, the axes in Figure 2 should be improved. \n\nThe experiments indicate that the proposed RL approach works better than genetic programming (GP) for what appears to be a simple benchmark. However, this is hard to judge. To truly understand the experiments, I advise readers to first look at: https://researchrepository.ucd.ie/bitstream/10197/3528/1/uy_gpem.pdf   \n\nI would have loved to see training and test curves, mathematical expressions for the reward so it is unambiguous, ablations of the dataset (eg varying the number of data). I assume a single net with the same parameters applies to all expressions. Is this correct?\n\nWhile applying policy gradients to symbolic regression is a great idea, the write up of this paper needs to improve substantially.  \n\n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "HkgJNlyCYB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents deep symbolic regression (DSR), which uses a recurrent neural network to learn a distribution over mathematical expressions and uses policy gradient to train the RNN for generating desired expressions given a set of points. The RNN model is used to sample expressions from the learned distribution, which are then instantiated into corresponding trees and evaluated on a dataset. The fitness on the dataset is used as the reward to train the RNN using policy gradient. In comparison to GP, the presented DSR approach recovers exact symbolic expressions in majority of the benchmarks.\n\nOverall, this paper presents a novel technique of using an RNN to learn a distribution over mathematical expressions, and using the fitness of sampled expressions as the reward signal to train the RNN using policy gradient. The idea of using the parent and sibling nodes to predict expression nodes in an autoregressive fashion is also interesting, which exploits the fact that the operators are either binary or unary. The experimental results show that it outperforms genetic programming as well as the ablation study shows the usefulness of different extensions.\n\nGrammar VAE (Kusner et al. 2017) learns a distribution over parse trees in a grammar and then uses Bayesian optimization to search over this space to perform symbolic regression. It would be important to empirically evaluate how GVAE performs on these symbolic regression tasks compared to DSR. \n\nDuring the search for expressions using DSR, it wasn’t clear why the algorithm chose all constants to be 1 for the first 12 benchmarks. Is it because the RNN never chose the constants in the learnt distribution or the BFGS algorithm prefers constants to be 1? Also, could it be the case that GP is being unfairly penalized for such cases as it might be trying to learn real-valued constants. Would it be possible to report what expressions GP came up with for the first 12 benchmarks?\n\nHow was the Recovery metric calculated? Does it require exact syntactic match or it also allows for semantically equivalent expression (that might be different syntactically)?\n\nWhat are the runtimes for the REINFORCE and GP methods? It wasn’t clear how big the total search space of expressions was without constants. How would a random search that enumerates over all expressions and uses BFGS to compute constants work?\n\nThe evaluation is only performed on 16 expressions and only 4 expressions have real-valued constants. It would be good to evaluate the technique on more benchmarks especially with real-valued constants, and larger expressions. Is Nguyen-4 already getting to maximum length of 30? Can the expressions from AI Feyman (Udrescu & Tegmark 2019) or synthetically generated expression be used?\n\nFrom the experiment benchmarks, it seems only 4 expressions had real-valued constants, and for these benchmarks GP did quite well in terms of NRMSE. What expression is GP coming up with for these benchmarks?\n\nMatt J. Kusner, Brooks Paige, and José Miguel Hernández-Lobato. 2017. Grammar variational autoencoder. ICML 2017 ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper suggests using RNN and policy gradient methods for improving symbolic regression. The reviewers could not reach a consensus, and due to concerns about the clarity of the paper and the extensiveness of the experimental results, the paper does not appear to currently meet the level of publication. 

Also, while not mentioned in the reviews, there appears to be some work on symbolic regression aided by deep learning, (see for example, https://twhughes.github.io/pdfs/cs221_final.pdf, which was found by searching "symbolic regression deep learning")---I would thus also recommend the authors do a more thorough literature search for future revisions. 
Decision: Reject