Deep symbolic regression

1 INTRODUCTION . Understanding the mathematical relationships among variables in a physical system is an integral component of the scientific process . Symbolic regression aims to identify these relationships by searching over the space of tractable mathematical expressions to best fit a dataset . Specifically , given a dataset of ( X , y ) pairs , where X ∈ Rn and y ∈ R , symbolic regression aims to identify a function f ( X ) : Rn → R that minimizes a distance metric D ( y , f ( X ) ) between real and predicted values . That is , symbolic regression seeks to find the optimal f ? = argminf D ( y , f ( X ) ) , where the functional form of f is a tractable expression . The resulting expression f ? may be readily interpretable and/or provide useful scientific insights simply by inspection . In contrast , conventional regression imposes a single model structure that is fixed during training , often chosen to be expressive ( e.g . a neural network ) at the expense of being easily interpretable . However , the space of mathematical expressions is discrete ( in model structure ) and continuous ( in model parameters ) , growing exponentially with the length of the expression , rendering symbolic regression an extremely challenging machine learning problem . Given the large and combinatorial search space , traditional approaches to symbolic regression typically utilize evolutionary algorithms , especially genetic programming ( GP ) ( Koza , 1992 ; Bäck et al. , 2018 ) . In GP-based symbolic regression , a population of mathematical expressions is “ evolved ” using evolutionary operations like selection , crossover , and mutation to improve a fitness function . While GP can be effective , it is also known to scale poorly to larger problems and to exhibit high sensitivity to hyperparameters . Deep learning has permeated almost all areas of artificial intelligence , from computer vision ( Krizhevsky et al. , 2012 ) to optimal control ( Mnih et al. , 2015 ) . However , deep learning may seem incongruous with or even antithetical toward symbolic regression , given that neural networks are typically highly complex , difficult to interpret , and rely on gradient information . We propose a framework that resolves this incongruity by tying deep learning and symbolic regression together with a simple idea : use a large model ( i.e . neural network ) to search the space of small models ( i.e . symbolic expressions ) . This framework leverages the representational capacity of neural networks while entirely bypassing the need to interpret a network . We present deep symbolic regression ( DSR ) , a gradient-based approach for symbolic regression based on reinforcement learning . In DSR , a recurrent neural network ( RNN ) emits a distribution over mathematical expressions . Expressions are sampled from the distribution , instantiated , and evaluated based on their fitness to the dataset . This fitness is used as the reward signal to train the RNN parameters using a policy gradient algorithm . As training proceeds , the RNN adjusts the likelihood of an expression relative to its reward , assigning higher probabilities to better fitting expressions . We demonstrate that DSR outperforms a standard GP implementation in its ability to recover exact symbolic expressions from data , both with and without added noise . We summarize our contributions as follows : 1 ) a novel method for solving symbolic regression that outperforms standard GP , 2 ) an autoregressive generative modeling framework for optimizing hierarchical , variable-length objects , 3 ) a framework that accommodates in situ constraints , and 4 ) a novel risk-seeking strategy that optimizes for best-case performance . 2 RELATED WORK . Symbolic regression . Symbolic regression has a long history of evolutionary strategies , especially GP ( Koza , 1992 ; Bäck et al. , 2018 ; Uy et al. , 2011 ) . Among non-evolutionary approaches , the recent AI Feynman algorithm ( Udrescu & Tegmark , 2019 ) is a multi-staged approach to symbolic regression leveraging the observation that physical equations often exhibit simplifying properties like multiplicative separability and translational symmetry . The algorithm identifies and exploits such properties to recursively define simplified sub-problems that can eventually be solved using simple techniques like a polynomial fit or small brute force search . Brunton et al . ( 2016 ) develop a sparse regression approach to recover nonlinear dynamics equations from data ; however , their search space is limited to linear combinations of a library of basis functions . AutoML . Our framework has many parallels to a body of works within automated machine learning ( AutoML ) that use an autoregressive RNN to define a distribution over discrete objects and use reinforcement learning to optimize this distribution under a black-box performance metric ( Zoph & Le , 2017 ; Ramachandran et al. , 2017 ; Bello et al. , 2017 ) . The key methodological difference to our framework is that these works optimize objects that are both sequential and fixed length . For example , in neural architecture search ( Zoph & Le , 2017 ) , an RNN searches the space of neural network architectures , which are encoded by a sequence of discrete “ tokens ” specifying architectural properties ( e.g . number of neurons ) of each layer . The length of the sequence is fixed or scheduled during training . In contrast , a major contribution of our framework is defining a search space that is both inherently hierarchical and variable length . The most similar AutoML work searches for neural network activation functions ( Ramachandran et al. , 2017 ) . While the space of activation functions is hierarchical in nature , the authors ( rightfully ) constrain this space substantially by positing a functional unit that is repeated sequentially , thus restricting their search space back to a fixed-length sequence . This constraint is well-justified for learning activation functions , which tend to exhibit similar hierarchical structures . However , a repeating-unit constraint is not practical for symbolic regression because the ground truth expression may have arbitrary structure . Autoregressive models . The RNN-based distribution over expressions used in DSR is autoregressive , meaning each token is conditioned on the previously sampled tokens . Autoregressive models have proven to be useful for audio and image data ( Oord et al. , 2016a ; b ) in addition to the AutoML works discussed above ; we further demonstrate their efficacy for hierarchical expressions . GraphRNN defines a distribution over graphs that generates an adjacency matrix one column at a time in autoregressive fashion ( You et al. , 2018 ) . In principle , we could have constrained GraphRNN to define the distribution over expressions , since trees are a special case of graphs . However , GraphRNN constructs graphs breadth-first , whereas expressions are more naturally represented using depth-first traversals ( Li et al. , 2005 ) . Further , DSR exploits the hierarchical nature of trees by providing the parent and sibling as inputs to the RNN , and leverages the additional structure of expression trees that a node ’ s value determines its number of children ( e.g . cosine is a unary node ) . 3 METHODS . Our overall approach involves representing mathematical expressions by the pre-order traversals of their corresponding symbolic expression trees , developing an autoregressive model to generate expression trees under a pre-specified set of constraints , and using reinforcement learning to train the model to generate better-fitting expressions . 3.1 GENERATING EXPRESSIONS WITH A RECURRENT NEURAL NETWORK . We leverage the fact that algebraic expressions can be represented using symbolic expression trees , a type of binary tree in which nodes map to mathematical operators , input variables , or constants . Operators are internal nodes and may be unary ( e.g . sine ) or binary ( e.g . multiply ) . Input variables and constants are terminal nodes . We encode an expression τ by the pre-order traversal ( i.e . depth-first , then left-to-right ) of its corresponding expression tree.1 We denote the ith node in the traversal as τi and the length of the traversal as |τ | = T . Each node has a value within a given library L of possible node values or “ tokens , ” e.g . { + , − , × , ÷ , sin , cos , x } . Expressions are generated one node at a time along the pre-order traversal ( from τ1 to τT ) . For each node , a categorical distribution with parameters ψ defines the probabilities of selecting each node value from L. To capture the “ context ” of the expression as it is being generated , we condition this probability upon the selections of all previous nodes in that traversal . This conditional dependence can be achieved very generally using an RNN with parameters θ that outputs a probability vector ψ in autoregressive manner . Specifically , the ith output vector ψ ( i ) of the RNN defines the probability distribution for selecting the ith node value τi , conditioned on the previously selected node values τ1 : ( i−1 ) : p ( τi|τ1 : ( i−1 ) ; θ ) = ψ ( i ) L ( τi ) , where L ( τi ) is the index in L corresponding to node value τi . The likelihood of the sampled expression is computed using the chain rule of conditional probability : p ( τ |θ ) = |τ |∏ i=1 p ( τi|τ1 : ( i−1 ) ; θ ) = |τ |∏ i=1 ψ ( i ) L ( τi ) The sampling process is illustrated in Figure 1 and described in Algorithm 1 . Additional algorithmic details of the sampling process are described in Subroutines 1 and 2 in Appendix A . Starting at the root node , a node value is sampled according to ψ ( 1 ) . Subsequent node values are sampled autoregressively in a depth-first , left-to-right manner until the tree is complete ( i.e . all tree branches reach terminal nodes ) . The resulting sequence of node values is the tree ’ s pre-order traversal , which can be used to reconstruct the tree2 and its 1Given an expression tree ( or equivalently , its pre-order traversal ) , the corresponding mathematical expression is unique ; however , given an expression , its expression tree ( or its corresponding traversal ) is not unique . For example , x2 and x · x are equivalent expressions but yield different trees . For simplicity , we use τ somewhat abusively to refer to an expression where it technically refers to an expression tree ( or equivalently , its corresponding traversal ) . 2In general , a pre-order traversal is insufficient to uniquely reconstruct the tree . However , in this context , we know how many child nodes each node has based on its value , e.g . “ multiply ” is a binary operator and thus has two children . corresponding expression . Note that different samples of the distribution have different tree structures of different size . Thus , the search space is inherently both hierarchical and variable length . Providing hierarchical inputs to the RNN . Naively , the input to the RNN when sampling τi would be a representation ( i.e . embedding or one-hot encoding ) of the previously sampled token , τi−1 . Indeed , this is typical in related autoregressive models , e.g . when generating sentences ( Vaswani et al. , 2017 ) or for neural architecture search ( Zoph & Le , 2017 ) . However , the search space for symbolic regression is inherently hierarchical , and the previously sampled token may actually be very distant from the next token to be sampled in the expression tree . For example , the fifth and sixth tokens sampled in Figure 1 are adjacent nodes in the traversal but are four edges apart in the expression tree . To better capture hierarchical information , we provide as inputs to the RNN a representation of the parent and sibling node of the token being sampled . We introduce an empty token for cases in which a node does not have a parent or sibling . Pseudocode for identifying the parent and sibling nodes given a partial traversal is provided in Subroutine 2 in Appendix A. Constraining the search space . Under our framework , it is straightforward to apply a priori constraints to reduce the search space . To demonstrate , we impose several simple , domain-agnostic constraints : ( 1 ) Expressions are limited to a pre-specified minimum and maximum length . We selected minimum length of 2 to prevent trivial expressions and a maximum length of 30 to ensure expressions are tractable . ( 2 ) The children of an operator should not all be constants , as the result would simply be a different constant . ( 3 ) The child of a unary operator should not be the inverse of that operator , e.g . log ( exp ( x ) ) is not allowed . ( 4 ) Direct descendants of trigonometric operators should not be trigonometric operators , e.g . sin ( x + cos ( x ) ) is not allowed because cosine is a descendant of sine . While still semantically meaningful , such composed trigonometric operators do not appear in virtually any scientific domain . We apply these constraints in situ ( concurrently with autoregressive sampling ) by zeroing out the probabilities of selecting tokens that would violate a constraint . Pseudocode for this process is provided in Subroutine For domains without this property , the number of children can be sampled from an additional RNN output . A pre-order traversal plus the corresponding number of children for each node is sufficient to uniquely reconstruct the tree . Algorithm 1 : Sampling an expression from the RNN 1 function SampleExpression ( θ , L ) input : RNN with parameters θ ; library of tokens L output : Pre-order traversal τ of an expression sampled from the RNN 2 τ = [ ] // Empty list 3 x = empty‖empty // Initial RNN input is empty parent and sibling 4 h0 = ~0 // Initialize RNN cell state to zero 5 for i = 1 , . . . , T do 6 ( ψ ( i ) , hi ) = RNN ( x , hi−1 ; θ ) 7 ψ ( i ) ← ApplyConstraints ( ψ ( i ) , L , τ ) // Adjust probabilities 8 τi = Categorical ( ψ ( i ) ) // Sample next token 9 τ ← τ‖τi // Append token to traversal 10 if ExpressionComplete ( τ ) then 11 return τ 12 x← ParentSibling ( τ ) // Compute next parent and sibling 13 end 14 return τ Algorithm 2 : Deep symbolic regression 1 function DSR ( α , N , L , X , y ) input : learning rate α ; batch size N ; library of tokens L ; input dataset ( X , y ) output : Best fitting expression τ ? 2 Initialize RNN with parameters θ , defining distribution over expressions p ( ·|θ ) 3 τ ? = null 4 b = 0 5 repeat 6 T = { τ ( i ) ∼ p ( ·|θ ) } i=1 : N // Sample expressions ( Algorithm 1 ) 7 T ← { OptimizeConstants ( τ ( i ) , X , y ) } i=1 : N // Optimize constants 8 R = { R ( τ ( i ) ) − λCC ( τ ( i ) ) } i=1 : N // Compute rewards 9 ĝ = 1N ∑N i=1R ( τ ( i ) ) ∇θ log p ( τ ( i ) |θ ) // Compute policy gradient 10 θ ← θ + α ( ĝ1 + ĝ2 ) // Apply gradients 11 if maxR > R ( τ ? ) then τ ? ← τ ( argmaxR ) // Update best expression 12 return τ ? 1 in Appendix A . This process ensures that all samples adhere to all constraints , without rejecting samples post hoc . In contrast , imposing constraints in GP-based symbolic regression can be problematic ( Craenen et al. , 2001 ) . In practice , evolutionary operations that violate constraints are typically rejected post hoc ( Fortin et al. , 2012 ) .

Summary: This paper presents a RNN-RL based method for the symbolic regression problem. The problem is new (to Deep RL) and interesting. My main concern is about the proposed method, where the three RL related equations (not numbered) at page 5 are also direct copy-from-textbook policy gradient equations without specific adaptation to the new application considered in this paper, which is very strange. The two conditional probability definitions considered at page 3 are not mentioned in later text. These are only fractions of the underlying method and by reading the paper back and forth several times, it is not clear of the basic algorithmic flowchart, let alone more detailed description of the related parameters. Without these information, it is impossible to have a fair judge of the novelty and feasibility of the proposed method. The empirical results are also limited in small dataset, which makes it hard to verify the generality of the superior claim.