Original: C O NS O L E : Convex Neural Symbolic Learning
Revised: C O NS O L E : Convex Neural Symbolic Learning

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Learning the underlying equation from data is a fundamental problem in manydisciplines.
Revised: Learning the underlying equation from data is a fundamental problem in manydisciplines.

Original: Recent advances rely on Neural Networks (NNs) but do not providetheoretical guarantees in obtaining the exact equations owing to the non-convexityof NNs.
Revised: Recent advances rely on Neural Networks (NNs) but do not providetheoretical guarantees in obtaining the exact equations owing to the non-convexityof NNs.

Original: In this paper, we propose Convex Neural Symbolic Learning (C O NS O L E )to seek convexity under mild conditions.
Revised: In this paper, we propose Convex Neural Symbolic Learning (C O NS O L E )to seek convexity under mild conditions.

Original: The main idea is to decompose therecovering process into two steps and convexify each step.
Revised: The main idea is to decompose therecovering process into two steps and convexify each step.

Original: In the first step ofsearching for right symbols, we convexify the deep Q-learning.
Revised: In the first step ofsearching for right symbols, we convexify the deep Q-learning.

Original: The key is tomaintain double convexity for both the negative Q-function and the negative rewardfunction in each iteration, leading to provable convexity of the negative optimal Qfunction to learn the true symbol connections.
Revised: The key is tomaintain double convexity for both the negative Q-function and the negative rewardfunction in each iteration, leading to provable convexity of the negative optimal Qfunction to learn the true symbol connections.

Original: Conditioned on the exact searchingresult, we construct a Locally-Convex equation Learner (
Revised: Conditioned on the exact searchingresult, we construct a Locally-Convex equation Learner (

Original: L O C
Revised: L O C

Original: A L) neural networkto convexify the estimation of symbol coefficients.
Revised: A L) neural networkto convexify the estimation of symbol coefficients.

Original: With such a design, we quantifya large region with strict convexity in the loss surface of L O C A L for commonlyused physical functions.
Revised: With such a design, we quantifya large region with strict convexity in the loss surface of L O C A L for commonlyused physical functions.

Original: Finally, we demonstrate the superior performance of the
Revised: Finally, we demonstrate the superior performance of the

Original: C O NS O L E framework over the state-of-the-art on a diverse set of datasets.
Revised: C O NS O L E framework over the state-of-the-art on a diverse set of datasets.

Original:  Introduction
Revised:  Introduction

Original: Identifying the underlying mathematical expressions from data plays a key role in multiple domains.
Revised: Identifying the underlying mathematical expressions from data plays a key role in multiple domains.

Original: For example, scientific discovery naturally requires learning analytical solutions to fit data.
Revised: For example, scientific discovery naturally requires learning analytical solutions to fit data.

Original: Engineering systems also need to frequently re-estimate system equations due to events, maintenance,upgrading and new constructions [1] etc.
Revised: Engineering systems also need to frequently re-estimate system equations due to events, maintenance,upgrading and new constructions [1] etc.

Original: In general, the problem, known as Symbolic Regression (SR) [2], learns the underlying equations y
Revised: In general, the problem, known as Symbolic Regression (SR) [2], learns the underlying equations y

Original: = g ( x ) constructed via certain symbols, where xand y represent the input/output vector of the equations.
Revised: = g ( x ) constructed via certain symbols, where xand y represent the input/output vector of the equations.

Original: If successfully learned, the equation canenjoy many important properties like high interpretability and generalizability, which will in turnsignificantly benefit scientific understanding and engineering planning, monitoring and control.
Revised: If successfully learned, the equation canenjoy many important properties like high interpretability and generalizability, which will in turnsignificantly benefit scientific understanding and engineering planning, monitoring and control.

Original: Promising as it might be, SR is NP-hard [3, 4].
Revised: Promising as it might be, SR is NP-hard [3, 4].

Original: Mathematically, one can cast SR as an optimizationproblem over both discrete variables to select symbols and continuous variables to represent thesymbol coefficients.
Revised: Mathematically, one can cast SR as an optimizationproblem over both discrete variables to select symbols and continuous variables to represent thesymbol coefficients.

Original: Traditional solutions employ evolutionary algorithms like Genetic Programming(GP)
Revised: Traditional solutions employ evolutionary algorithms like Genetic Programming(GP)

Original: These methods start from an initial set of expressions and continue evolving via operationslike crossover or mutation.
Revised: These methods start from an initial set of expressions and continue evolving via operationslike crossover or mutation.

Original: With fitness measures, GP-based algorithms can evaluate and select thebest equations.
Revised: With fitness measures, GP-based algorithms can evaluate and select thebest equations.

Original: However, these methods have poor scalability and limited theoretical guarantees [3].
Revised: However, these methods have poor scalability and limited theoretical guarantees [3].

Original: More recent SR studies leverage Neural Networks (NNs) with high representational power.
Revised: More recent SR studies leverage Neural Networks (NNs) with high representational power.

Original: For the
Revised: For the

Original: NN-based SR, we mainly categorize them into two groups based on the roles of the NNs.
Revised: NN-based SR, we mainly categorize them into two groups based on the roles of the NNs.

Original: The firstgroup employs NNs to directly model the equations, where sparsity of the NN weights is enforced toselect symbols [6–10].
Revised: The firstgroup employs NNs to directly model the equations, where sparsity of the NN weights is enforced toselect symbols [6–10].

Original: Thus, the problem is transformed into training the designed NN with sparsityregularization.
Revised: Thus, the problem is transformed into training the designed NN with sparsityregularization.

Original: However, due to the non-convexity of NNs, the weight selection and updating can beeasily stuck in local optima, failing to find the exact equations.
Revised: However, due to the non-convexity of NNs, the weight selection and updating can beeasily stuck in local optima, failing to find the exact equations.

Original: The second group employs NNs to search the symbol connections, and non-linear optimizations like
Revised: The second group employs NNs to search the symbol connections, and non-linear optimizations like

Original: BFGS [11] can be employed to estimate symbol coefficients.
Revised: BFGS [11] can be employed to estimate symbol coefficients.

Original: For the searching procedure, [3, 12–14] leverage a Recursive Neural Network (RNN) as a policy network to iteratively generate optimalactions that can select and connect symbols.
Revised: For the searching procedure, [3, 12–14] leverage a Recursive Neural Network (RNN) as a policy network to iteratively generate optimalactions that can select and connect symbols.

Original: [15] employs large-scale pre-training to directly mapfrom data to the symbolic equations.
Revised: [15] employs large-scale pre-training to directly mapfrom data to the symbolic equations.

Original: While these methods restrict the utilization of NNs in thesearch phase, the non-convexity of NNs can still suffer the risk of sub-optimal decisions to formulateequations.
Revised: While these methods restrict the utilization of NNs in thesearch phase, the non-convexity of NNs can still suffer the risk of sub-optimal decisions to formulateequations.

Original: To mitigate this issue, some efforts have been made such as risk-seeking policy gradient tofind the best equations [3], restricting the searching space via domain knowledge [16] and entropyregularization [13], and re-initialization [13], etc.
Revised: To mitigate this issue, some efforts have been made such as risk-seeking policy gradient tofind the best equations [3], restricting the searching space via domain knowledge [16] and entropyregularization [13], and re-initialization [13], etc.

Original: However, they have limited theoretical guarantees.
Revised: However, they have limited theoretical guarantees.

Original: In this paper, we propose Convex Neural Symbolic Learning (C O NS O L E ) with convexity undermoderate conditions.
Revised: In this paper, we propose Convex Neural Symbolic Learning (C O NS O L E ) with convexity undermoderate conditions.

Original: To our best knowledge, we are the first to provide provable guarantees to learnthe exact equations.
Revised: To our best knowledge, we are the first to provide provable guarantees to learnthe exact equations.

Original: In general, we decompose the SR problem into two sub-problems and seekconvexity, respectively.
Revised: In general, we decompose the SR problem into two sub-problems and seekconvexity, respectively.

Original: In the first problem of searching symbols, we propose a double-convexifieddeep Q-learning to maintain the convexity of negative Q-function and negative reward functionswith continuous action variables.
Revised: In the first problem of searching symbols, we propose a double-convexifieddeep Q-learning to maintain the convexity of negative Q-function and negative reward functionswith continuous action variables.

Original: Specifically, we use the Input Convex Neural Network (ICNN)[17] to represent both the negative Q-function and the negative reward function in each iteration.
Revised: Specifically, we use the Input Convex Neural Network (ICNN)[17] to represent both the negative Q-function and the negative reward function in each iteration.

Original: Subsequently, we prove that such a design (1) guarantees an optimal action selection at each stepand (2) ensures the negative optimal Q-function, if successfully found, to be convex.
Revised: Subsequently, we prove that such a design (1) guarantees an optimal action selection at each stepand (2) ensures the negative optimal Q-function, if successfully found, to be convex.

Original: Therefore, theconvex negative optimal Q-function can yield global optimal decisions of equation constructions.
Revised: Therefore, theconvex negative optimal Q-function can yield global optimal decisions of equation constructions.

Original: In the second problem of coefficient estimation, we use the search result to build a Locally-Convex
Revised: In the second problem of coefficient estimation, we use the search result to build a Locally-Convex

Original: Equation Learner (L O C A L) neural network.
Revised: Equation Learner (L O C A L) neural network.

Original: The key insight is that if the search result is correct,the loss surface of L O C A L has local regions that contain the global optima and has strict convexity.
Revised: The key insight is that if the search result is correct,the loss surface of L O C A L has local regions that contain the global optima and has strict convexity.

Original: Moreover, we quantify the local regions and show the range of the region is large under mildassumptions.
Revised: Moreover, we quantify the local regions and show the range of the region is large under mildassumptions.

Original: Therefore, initializations based on prior knowledge can often lie in the convex region,bringing the accurate coefficient estimation.
Revised: Therefore, initializations based on prior knowledge can often lie in the convex region,bringing the accurate coefficient estimation.

Original: Finally, we demonstrate that our C O NS O L E outperformsstate-of-the-art methods on a diverse set of datasets.
Revised: Finally, we demonstrate that our C O NS O L E outperformsstate-of-the-art methods on a diverse set of datasets.

Original:  Related Work
Revised:  Related Work

Original: Symbolic regression using neural networks.
Revised: Symbolic regression using neural networks.

Original: In addition to the direct use of NNto help SR asdiscussed in the Introduction, there are studies with the indirect use.
Revised: In addition to the direct use of NN in the Introduction,there are studies for the indirect use.

Original: They typically treat NNs as adata augmentation tool to create high-quality data for learning or evaluation [18, 2].
Revised: They typically treat NNs as a data augmentation tool to createhigh-quality data for learning or evaluation [18, 2].

Original: Neural architecture search.
Revised: Neural architecture search.

Original: Searching the connections of L O C A L falls into the area of Neural
Revised: Searching the connections of L O C A L falls into the area of Neural

Original: Architecture Search (NAS).
Revised: Architecture Search (NAS).

Original: NAS tries to find an optimal architecture of a target NN with the bestperformance [19].
Revised: NAS tries to find an optimal architecture of a target NN with the bestperformance [19].

Original: The searching algorithms can be divided into RL-based, evolutionary algorithmbased, sequential optimization-based, and gradient descent-based methods.
Revised: The searching algorithms can be divided into RL-based, evolutionary algorithmbased, sequential optimization-based, and gradient descent-based methods.

Original: For RL-based methods,[20] employs an RNN model to sample the architecture and utilizes the accuracy of the samplednetwork as the reward.
Revised: For RL-based methods,[20] employs an RNN model to sample the architecture and utilizes the accuracy of the samplednetwork as the reward.

Original: [21] uses tabular Q-learning to find the connections of a target NN.
Revised: [21] uses tabular Q-learning to find the connections of a target NN.

Original: However,tabular Q-learning can hardly be applicable when the state and action spaces are large, e.g., SRproblem.
Revised: However,tabular Q-learning can hardly be applicable when the state and action spaces are large, e.g., SRproblem.

Original: The evolutionary algorithm employs methods such as GP [22] and tournament selection [23].
Revised: The evolutionary algorithm employs methods such as GP [22] and tournament selection [23].

Original: However, these methods may lack the scalability for SR problem [3].
Revised: However, these methods may lack the scalability for SR problem [3].

Original: The sequential optimizationbased method is more scalable as the model complexity increases in a sequential manner [24].
Revised: The sequential optimizationbased method is more scalable as the model complexity increases in a sequential manner [24].

Original: Finally,the gradient descent-based method [25] builds a large and over-parameterized network to search andtrain.
Revised: Finally,the gradient descent-based method [25] builds a large and over-parameterized network to search andtrain.

Original: Then, regularizations like dropout are added to find the best connections.
Revised: Then, regularizations like dropout are added to find the best connections.

Original: However, for thesemethods, the theoretical guarantees remain opaque for SR problem.
Revised: However, for thesemethods, the theoretical guarantees remain opaque for SR problem.

Original: Global optimum in neural networks.
Revised: Global optimum in neural networks.

Original: Many studies have been conducted to seek global optimalityin NNs, and they can be categorized into finding global optima for weights or input variables.
Revised: Many studies have been conducted to seek global optimalityin NNs, and they can be categorized into finding global optima for weights or input variables.

Original: Theweight optimization is directly linked to finding symbol coefficients in L O C A L. Specifically, [26]investigates a single hidden layer with unbounded neurons and non-Euclidean regularization.
Revised: Theweight optimization is directly linked to finding symbol coefficients in L O C A L. Specifically, [26]investigates a single hidden layer with unbounded neurons and non-Euclidean regularization.

Original: Theauthors show the training can be done via convex optimization problems.
Revised: Theauthors show the training can be done via convex optimization problems.

Original: [27] considers finiteneurons and develops a novel duality theory to train two-layer NNs as convex optimization problems.[28] establishes a strong result that every local optimum is a global optimum for deep non-linearnetworks under several assumptions.
Revised: [27] considers finite neuronsand develops a novel duality theory to train two-layer NNs with convexity. [28] establishes a strongresult that every local optimum is a global optimum for deep non-linear networks under severalassumptions.

Original: [29] eliminates these assumptions and finds that with weightdecay regularization (e.g., l 2 norm), the loss function of NN with ReLU activations is piece-wisestrongly convex in local regions.
Revised: [29] eliminates these assumptions and finds that with weight decay regularization(e.g. , l 2 norm), the loss function of NN with ReLU activations is piece-wise strongly convex in localregions.

Original: However, L O C A L doesn’t fit the above conditions.
Revised: However, L O C A L doesn’t fit the above conditions.

Original: The secondgroup of input variable optimization can help search for optimal inputs.
Revised: The second group of input variableoptimization can help search for optimal inputs.

Original: [17] proposes ICNN suchthat the output of ICNN is convex in input variables.
Revised: [17] proposes ICNN such that the output of ICNN isconvex in input variables.

Original: The key of ICNN is to restrict some weights andactivation functions to preserve convexity.
Revised: The key of ICNN is to restrict some weights and activation functions topreserve convexity.

Original: ICNN facilitates to design a convexified search algorithm.
Revised: ICNN facilitates to design a convexified search algorithm.

Original:  Convex Neural Symbolic Learning
Revised:  Convex Neural Symbolic Learning

Original: L O C A L to Hierarchically Represent the Equations and Learn Symbol Coefficients
Revised: L O C A L to Hierarchically Represent the Equations and Learn Symbol Coefficients

Original: We utilize a neural network to represent the multi-input multi-output equations due to the efficiency[6].
Revised: We utilize a neural network to represent the multi-input multi-output equations due to the efficiency[6].

Original: We prove in Theorem 3 that under certain conditions, there are local regions in the loss surfacewith strict convexity.
Revised: We prove in Theorem 3 that under certain conditions, there are local regions in the loss surfacewith strict convexity.

Original: Thus, we name our NN as Locally Convex Equation Learner (L O C
Revised: Thus, we name our NN as Locally Convex Equation Learner (L O C

Original: A L).
Revised: A L).

Original: We assume the underlying equation y
Revised: We assume the underlying equation y

Original: = g ( x ) follows compositionality and smoothness assumptionsin [2], which are often the case in physics and many other scientific applications.
Revised: = g ( x ) follows compositionality and smoothness assumptionsin [2], which are often the case in physics and many other scientific applications.

Original: Then, wecan build
Revised: Then, we build

Original: L O C A L that can be trained via input/output pairs { x i , y i } Ni
Revised: L O C A L that can be trained via input/output pairs { x i , y i } Ni

Original: =1 , where x i ∈ X and
Revised: =1 , where x i ∈ X and

Original: y i ∈ Y are the i thsamples and N is the number of data.
Revised: y i ∈ Y are the i thsamples and N is the number of data.

Original: X and Y are the input and output data spaces, respectively.
Revised: X and Y are the input and output data spaces, respectively.

Original: Bythe compositionality, we propose to hierarchically map x to y with correct symbols.
Revised: Bythe compositionality, we propose to hierarchically map x to y with correct symbols.

Original: Fig.
Revised: Fig.

Original: 1 showsan example of the L O C A L structure.
Revised: 1 showsan example of the L O C A L structure.

Original: Specifically, each input entry first goes through the activationfunctions from a symbol library Φ .
Revised: Specifically, each input entry first goes through the activationfunctions from a symbol library Φ .

Original: For example, in Fig. 1, we denote Φ =
Revised: For example, in Fig. 1, we denote Φ =

Original: { x, x 2 , cos( wx ) } tocorrespond with three neurons from top to bottom in the dotted black box, where w is a learnableweight.
Revised: { x, x 2 , cos( wx ) } tocorrespond with three neurons from top to bottom in the dotted black box, where w is a learnableweight.

Original: As for weights of x and x 2 , they only need to appear in the later summation layer.
Revised: As for weights of x and x 2 , they only need to appear in the later summation layer.

Original: Then, someactivation outputs will be selected as multipliers for the multiplication.
Revised: Then, someactivation outputs will be selected as multipliers for the multiplication.

Original: Subsequently, the multipliedoutputs are selected and summed together.
Revised: Subsequently, the multipliedoutputs are selected and summed together.

Original: The repetition of symbolic activation, multiplication, andsummation formulates final equations.
Revised: The repetition of symbolic activation, multiplication, andsummation formulates final equations.

Original: For instance, the L O C A L structure in Fig.
Revised: For instance, the L O C A L structure in Fig.

Original: 1 can correctlyrepresents y 1 and y 2 shown on the top right of Fig.
Revised: 1 can correctlyrepresents y 1 and y 2 shown on the top right of Fig.

Revised: The layer-wise connectivity and the parameters of L O C

Revised: A L are optimization variables to let L O C A Lfit the training data.

Original: Mathematically, we denote Z k ∈ { 0 , 1 } n k × n k +1 and W k ∈ R n k × n k +1 as the indicator and weightmatrix between the k th and the ( k + 1) th layer of the L O C A L, respectively.
Revised: Mathematically, we denote Z k ∈ { 0 , 1 } n k × n k +1 and W k ∈ R n k × n k +1 as theindicator and weight matrix between the k th and the ( k + 1) th layer of the L O C A L, respectively.

Original: n k is the number ofneurons for the k th layers.
Revised: n kis the number of neurons for the k th layers.

Original: Z k
Revised: Z k

Original: [ i, j ] = 1 indicates that the connection exists between the i th neuronin the k th layer and the j th neuron in the ( k + 1) th layer, where Z k [ i, j ] is the ( i, j ) th entry of 
Revised: [ i, j ] = 1 indicates that the connection exists betweenthe i th neuron in the k th layer and the j th neuron in the ( k + 1) th layer, where Z k [ i, j ] is the ( i, j ) thentry of Z k .

Original: Z k .

Original: We assume there are ( K + 1) layers in L O C A L and denote h k ∈ R n k to be the output ofthe k th layer.
Revised: We assume there are ( K + 1) layers in L O C A L and denote h k ∈ R n k to be the outputof the k th layer.

Original: Naturally, we have h 0 = x
Revised: Naturally, we have h 0 = x

Original: and h K
Revised: and h K

Original: = ˆ y , where ˆ y is the output of L O C A L. For thesymbolic activation or summation layer, we have h k +

Revised: = ˆ y , where ˆ y is the output of L O C A L. Forthe symbolic activation or summation layer, we have h k +

Original: =
Revised: =

Original: Z Tk ◦ W Tk h k , where ◦ represents the Hadamard product and helps to zero out some connections.
Revised: Z Tk ◦ W Tk h k , where ◦ representsthe Hadamard product and helps to zero out some connections.

Original: For the multiplication layer, we haveh k +

Revised: For the multiplication layer, wehave h k +

Original: [ j ] = (cid:81)
Revised: [ j ] = (cid:81)

Original: Z k
Revised: Z k

Original: [ i,j ]=1 h k
Revised: [ i,j ]=1 h k

Original: [ i ] .
Revised: [ i ] with no weight matrix involved.

Original: In general, we denote L O C
Revised: In general, we denote L O C

Original: A L as f ( x ; { Z k } K − 1 k =0 , { W k } K − 1 k =0 ) .
Revised: A L asf ( x ; { Z k } K − 1 k =0 , { W k } K − 1 k =0 ) .

Original: Thesearch algorithm identifiesthe locations of the entry 1 in { Z k } K − 1 k =0{+=
+} and the estimation process learnsthe corresponding weights in { W k } K − 1 k [-=
-] using { x i , y i } Ni =1 .
Revised: The search algorithm identifies { Z k } K − 1 k {+=
+} and the estimation processlearns the corresponding weights in { W k } K − 1 k [-=
-]=0 using { x i , y i } Ni =1 .

Original: 3.2 Search L O C A L Structures with Double Convex Deep Q-Learning
Revised: 3.2 Search L O C A L Structures with Double Convex Deep Q-Learning

Original: The high dimensionality of the search space is a general problem for L O C A L and other SR models.

Original: For an efficient search, NN-based RL methods are often utilized, e.g., deep Q-learning [30].

Original: In thissubsection, we employ the Input Convex Neural Network (ICNN) [17] to design a double convexdeep Q-learning with provable performance guarantees.
Revised: In this subsection, we employ the Input Convex Neural Network (ICNN) [17] to design a doubleconvex deep Q-learning with provable performance guarantees.

Original: Search the structure of L O C A L. To search the structure of L O C A L, we treat the status of each layerof L O C A L as a state and the connections between layers are actions.
Revised: State and action definitions in the search process. To search the structure of L O C A L, we treat thestatus of each layer of L O C A L as a state and the connections between layers, i.e., Z k in L O C A L, asactions.

Original: Specifically, we denote a k ∈ { 0 , 1 } n a as the k th action vector where a k [( i − 1) n k + j ] = 1 implies that the connection ij exists forthe i th neuron in the k th layer and the j th neuron in the ( k + 1) th layer.
Revised: Specifically, we denote a k ∈ { 0 , 1 } n a as the k th action vector where a k [( i − 1) n k + j ] = 1implies that the connection ij exists for the i th neuron in the k th layer and the j th neuron in the ( k + 1) th layer.

Original: n a = max { n k n k +1 } K − 1 k [-=
 isthe-] dimensionality of the action space.
Revised: n a = max { n k n k +1 } K − 1 k [-=
 isthe-]=0 is the dimensionality of the action space.

Original: Also, we denote s k ∈
Revised: Also,we denote s k ∈

Original: Z n s as the state vector to represent thecurrent state for the k th layers, where n s = max { n k } Kk =0{+=
 + 1+} is the dimensionality of the state space.
Revised: Z n s as the state vector to represent the current state for the k th layers, wheren s = max { n k } Kk {+=
 + 1+} is the dimensionality of the state space.

Original: Fora known state and a fixed action, thenext state  is exactly known due to the exact neuron connections,which implies a deterministic state transition.
Revised: More specifically, the state vectors k is composed of the values for each neuron in the k th layer and an entry of state index k . Theappending of the value k can avoid the duplicate state vectors for different layers. Otherwise, theduplicated states may exist and require different optimal actions, deteriorating the correct search forthe true structure of L O C A L.

Original: Thus, we utilize a linear transformation to representthe state transition process.
Revised: To further quantify state vectors, we utilize a linear transformation torepresent the state transition process.

Original: ∀ 0 ≤ k ≤ K − 1 , we define
Revised: ∀ 0 ≤ k ≤ K − 1 , we define

Original: [Equation]
Revised: [Equation]

Original: where  Mat ( · ) is the operation to convert a vector to a matrix and we fill 0 s  to some s k , a k , and Z ′ k tomaintain the fixed size.
Revised: where the number of 1 s in s 0 corresponds to the input dimension. Mat ( · ) is the operation to reshapea vector to a matrix, and we utilize 0 s for padding to maintain the dimensions.

Original: [ ·; · ] is the row concatenation process.
Revised: c = [0 , 0 , · · · , 1] Tis a constant vector to increase the entry of the state index from k to k + 1 .

Original: We show this linear state transition isessential to guarantee the convexity of negative optimal Q-function in Theorem 1.
Revised: We show this linearstate transition is essential to guarantee the convexity of negative optimal Q-function in Theorem1.

Original: For the calculationexample of state transition, one can refer to Fig.
Revised: For the calculation example of state transition, one can refer to Fig.

Original: Then, we can obtain Z k from Mat ( a k ) by deletingthe filled 0s.
Revised: Then, we can obtain Z kfrom Mat ( a k ) by deleting the filled 0s.

Original: By the designed states and actions, the search will always start at s 0 and end at s K inone episode, thus formulating the L O C A L function f ( x ; { Z k } K − 1 k =0 , { W k } K − 1 k =0 ) .
Revised: By the designed states and actions, the search will alwaysstart at s 0 and end at s K in one episode. Thus, we define a trajectory as a sequence of searchedstate-action pairs for { ( s 0 , a 0 ) , ( s 1 , a 1 ) , · · · , ( s K − 1 , a K − 1 ) } . This trajectory formulates { Z k } K − 1 k =0in the L O C A L function f ( x ; { Z k } K − 1 k =0 , { W k } K − 1 k =0 ) .

Original: The above states, actions, state transitions, initial state s 0 , and a discount factor γ can form a
Revised: The above states, actions, state transitions, initial state s 0 , and a discount factor γ can form a

Original: Controlled Markov Process (CMP) [31, 32], which is a Markov Decision Process (MDP) withouta reward function [32].
Revised: Controlled Markov Process (CMP) [30, 31], which is a Markov Decision Process (MDP) withouta reward function [31].

Original: In the following part, we define our reward function based on an end-oftrajectory reward [31].
Revised: In the following part, we define our reward function based on an end-oftrajectory reward [30].

Original: Although the reward function is non-Markov, we prove in Theorem 1 thatunder our settings, there exists an optimal Q-function.
Revised: Although the reward function is non-Markov, we prove in Theorem 1 thatunder our settings, there exists an optimal Q-function.

Original: The proof can be seen in Appendix A.2.
Revised: The proof can be seen in Appendix A.3.

Original: Double convexity for the negative reward and Q-function in the search model.
Revised: Double convex deep Q-learning to search optimal actions.

Original: Training L O C A Lcan bring an end-of-trajectory reward [3] to evaluate the episode.
Revised: Training L O C A L can bring an end-oftrajectory reward [3] to evaluate the episode.

Original: However, this reward is insufficientto enable a convex evaluation for instant state-action pairs.
Revised: However, this reward is insufficient to enable a convexevaluation for instant state-action pairs.

Original: Thus, we propose to restrict the negativereward function to be convex.
Revised: Thus, we propose to restrict the negative reward function tobe convex.

Original: Based on Bellman equations [33], this design will ensure the convexityof the negative optimal Q-function.
Revised: Based on Bellman equations [32], this design will ensure the convexity of the negativeoptimal Q-function.

Original: More proof details can be referred to Theorem 1 and Appendix A.2.
Revised: More proof details can be referred to Theorem 1 and Appendix A.3.

Original: Specifically, we utilize an ICNN to model the reward function − R ( s k , a k )
Revised: Specifically, we utilize an ICNN to model the reward function − R ( s k , a k )

Original: such that − R ( s k , a k )is convex in states and actions in continuous spaces.
Revised: such that − R ( s k , a k )is convex in states and actions in continuous spaces.

Original: − R ( s k , a k ) requires proper training to do thecorrect evaluation.
Revised: − R ( s k , a k ) requires proper training to do thecorrect evaluation.

Original: In the t th episode, we collect the training input of state-action pairs {  s tk , a tk  } K − 1 k =0 .
Revised: In the t th episode, we collect the t th trajectory sample { ( s tk , a tk ) } K − 1 k =0 .

Original: Then, the output can be defined as the end-of-trajectory reward to evaluate the obtained L O C A Lwhich is denoted as f t ( · ) .
Revised: Then,the output can be defined as the end-of-trajectory reward to evaluate the obtained L O C A L whichis denoted as f t ( x ; W t ) , where W t is the weight set of L O C A L. Basically, we utilize gradientmethod (e.g., Adam [33]) to train f t ( x ; W t ) and obtain the optimal set of weights W ∗ t for thet th episode of L O C A L by minimizing the Mean Square Error (MSE). Then, we can calculate the

Original: Basically, we calculate the Normalized Root-Mean-Square Error (NRMSE)[3] of the trained f t ( · )function such that NRMSE t = 1 σ y (cid:113)1 N (cid:80) Ni =

Revised: Normalized Root-Mean-Square Error (NRMSE) [3] of the trained f t ( x ; W ∗ ) such that NRMSE t =1 σ y (cid:113) N (cid:80) N i =

Original: ( y i − f t (cid:0) x i  ) (cid:1) 2 , where σ y is thestandard deviation of the outputs.
Revised: ( y i − f t (cid:0) x i ; W ∗ t ) (cid:1) 2 , where σ y is the standard deviation of the outputs.

Original: Then, the output of the reward  can be calculated as R t = 1 1+ NRMSE t .
Revised: The outputof the reward function can be calculated as R t = 1 1+ NRMSE t .

Original: Therefore, we can train − R ( · ) using {{ s tk , a tk } K − 1 k =0 , − R t } .
Revised: Therefore, we can train − R ( · ) using{{ s tk , a tk } K − 1 k =0 , − R t } .

Original: It’s important to note that building this convex function − R ( · ) is better than directly leveraging theend-of-trajectory reward for evaluations.
Revised: It’s important to note that building this convex function − R ( · ) is better than directly leveraging theend-of-trajectory reward for evaluations.

Original: The reason is that the former process not only providestheoretical guarantees in Theorem 1 but also evaluates the actions in a continuous space with convexity.
Revised: The reason is that the former process not only providestheoretical guarantees in Theorem 1 but also evaluates the actions in a continuous space with convexity.

Original: Namely, one can utilize R ( s k , ˜ a k ) to evaluate a continuous action ˜ a k , given that there is a muchhigher probability to obtain ˜ a k rather than the discrete action a k in the deep Q-learning.
Revised: Namely, one can utilize R ( s k , ˜ a k ) to evaluate a continuous action ˜ a k , given that there is a muchhigher probability to obtain ˜ a k rather than the discrete action a k in the deep Q-learning.

Original: Then, toupdate Q values, we have the following iterative computations based on the temporal difference [21]:
Revised: Then, toupdate Q values, we have the following iterative computations based on the temporal difference [21]:

Original: [Equation]
Revised: [Equation]

Original: where Q t ( s k , ˜ a k ) is the Q value at the t th episode for state s k and action ˜ a k .
Revised: where Q t ( s k , ˜ a k ) is the Q value at the t th episode for state s k and action ˜ a k .

Original: α and γ are pre-definedlearning rate and discount factor, respectively.
Revised: α and γ are pre-definedlearning rate and discount factor, respectively.

Original: Similarly, we utilize another ICNN to represent− Q ( s k , ˜ a k ) such that − Q ( s k , ˜ a k ) is convex in ˜ a k given fixed s k .
Revised: Similarly, we utilize another ICNN to represent− Q ( s k , ˜ a k ) such that − Q ( s k , ˜ a k ) is convex in ˜ a k given fixed s k .

Original: Therefore, one can solve a convexoptimization problem max a Q t ( s k +1 , ˜ a ) to obtain the (approximately) global optimal action for
Revised: Therefore, one can solve a convexoptimization problem max a Q t ( s k +1 , ˜ a ) to obtain the (approximately) global optimal action for

Original: Equation (2) in each iteration.
Revised: Equation (2) in each iteration.

Original: By definitions of the discrete actions, we can restrict the continuousaction space in a hypercube conv ( { 0 , 1 } n a ) , i.e., a convex hull of the discrete actions.
Revised: By definitions of the discrete actions, we can restrict the continuous action space in a hypercube conv ( { 0 , 1 } n a ) , i.e., a convex hull of the discrete actions.

Original: Thus, theoptimization problem is:
Revised: Thus, theoptimization problem is:

Original: [Equation]
Revised: [Equation]

Original: Based on [17], this convex optimization problem can be solved via a bundle entropy method.
Revised: Based on [17], this convex optimization problem can be solved via a bundle entropy method.

Original: Afterobtaining a continuous solution ˜ a ∗ , we discretize it to a discrete vector a ∗ to build L O C A L. One simple method is to enforce a ∗
Revised: Afterobtaining a continuous solution ˜ a ∗ , we discretize it to a discrete vector a ∗ to build L O C A L. Onesimple method is to enforce a ∗

Original: [ i ] = 

Revised: [ i ] =

Original: if ˜ a ∗ [ i ] ≥ 0 . 5 , and otherwise a ∗ [ i ] = 0 .
Revised: if ˜ a ∗ [ i ] ≥ 0 . 5 , and otherwise a ∗ [ i ] = 0 .

Original: Thus, both
Revised: Thus, both

Original: Q ( s k , ˜ a k ) and Q ( s k , a k ) can be trained using Equation (2).
Revised: Q ( s k , ˜ a k ) and Q ( s k , a k ) can be trained using Equation (2).

Original: Practically, we introduce ϵ -greedystrategy [34], experience replay [35] and a target Q-network [30] to update Q-function, thus boostingthe convergence to the optimal Q-function.
Revised: Practically, we introduce ϵ -greedystrategy [34], experience replay [35] and a target Q-network [36] to update Q-function, thus boostingthe convergence to the optimal Q-function.

Original: The specific algorithm can be seen in Algorithm 1.
Revised: The overview of our framework is in Algorithm 1 and thespecific algorithm is in Algorithm 2.

Original: Finally,based on Theorems 1-2,if the optimal Q-function is found, solving the convex optimization via theoptimal Q-function can generate the correct structures of L O C A L and exact equations.
Revised: Finally, based on Theorems 1-2, solving the convex optimizationvia the optimal Q-function can generate the correct structures of L O C A L and exact equations.

Original: Symbolic static and dynamic constraints.
Revised: Symbolic static and dynamic constraints.

Original: Constraints can be added to accelerate the search process[3, 2].
Revised: Constraints can be added to accelerate the search process[3, 2].

Original: For example, in Algorithm 1, we propose a constraint checking program for the state-actionpairs to avoid the invalid search, suitable for arbitrary restrictions.
Revised: For example, in Algorithm 2, we propose a constraint checking program for the state-actionpairs to avoid the invalid search, suitable for arbitrary restrictions.

Original: Then, we emphasize a generaltype of constraint, symbolic constraint, for the SR problem.
Revised: Then, we emphasize a generaltype of constraint, symbolic constraint, for the SR problem.

Original: The symbolic static constraint requiresthat each equation contains only a subset of symbols.
Revised: The symbolic static constraint requiresthat each equation contains only a subset of symbols.

Original: For example, the equation y 1 = x 1 x 2 · · · x 100shouldn’t exist since it is too complicated for real-world systems.
Revised: For example, the equation y 1 = x 1 x 2 · · · x 100shouldn’t exist since it is too complicated for real-world systems.

Original: This constraint eliminates part ofthe action space and can be checked by counting the number of 1 s in the action vector.
Revised: This constraint eliminates part ofthe action space and can be checked by counting the number of 1 s in the action vector.

Original: The symbolicdynamic constraint can gradually reduce the search space based on symbol correlations.
Revised: The symbolicdynamic constraint can gradually reduce the search space based on symbol correlations.

Original: Specifically,we investigate the ( K − 1) th layer’s neurons that are linearly summed to form the equation.
Revised: Specifically,we investigate the ( K − 1) th layer’s neurons that are linearly summed to form the equation.

Original: Ifsome of these neurons have strong linear correlations to the output neuron (e.g., Pearson correlationcoefficient larger than 0 . 99 ), they should be kept subsequently.
Revised: Ifsome of these neurons have strong linear correlations to the output neuron (e.g., Pearson correlationcoefficient larger than 0 . 99 ), they should be kept subsequently.

Original: Namely, we can maintain the pathfrom input neurons to the neurons to be kept and reduce the search space.
Revised: Namely, we can maintain the pathfrom input neurons to the neurons to be kept and reduce the search space.

Original:  Theoretical Analysis
Revised:  Theoretical Analysis

Original: We employ explorations, experience replay, and a target Q-network to boost the convergence to theoptimal Q function.
Revised: We employ explorations, experience replay, and a target Q-network to boost the convergence to theoptimal Q function.

Original: However, our extra requirement of the convex shape for the negative Q-functionmay deteriorate the convergence performance.
Revised: However, our extra requirement of the convex shape for the negative Q-functionmay deteriorate the convergence performance.

Original: Thus, we first prove that in C O NS O L E , the negativeoptimal Q-function is also convex so that the convex design doesn’t affect the convergence.
Revised: Thus, we first prove that in C O NS O L E , the negativeoptimal Q-function is also convex so that the convex design doesn’t affect the convergence.

Original: Then, weprove that the convexity of the negative optimal Q-function eventually yields the exact equations.
Revised: Then, weprove that the convexity of the negative optimal Q-function eventually yields the exact equations.

Original: Theorem 1.
Revised: Theorem 1.

Original: ∀ 0 ≤ k ≤ K − 1 , the negative optimal Q-function − Q ∗ ( s k , ˜ a k ) in the proposed
Revised: ∀ 0 ≤ k ≤ K − 1 , the negative optimal Q-function − Q ∗ ( s k , ˜ a k ) in the proposed

Original: C O NS O L E framework exists and is convex in s k and ˜ a k , where s k is the discrete state and ˜ a k is thecontinuous action at the k th stage.
Revised: C O NS O L E framework exists and is convex in s k and ˜ a k , where s k is the discrete state and ˜ a k is thecontinuous action at the k th stage.

Original: The proof can be seen in Appendix A.2.
Revised: The proof can be seen in Appendix A.3.

Original: Based on the convexity of negative optimal Q-function,the optimal sequence of states ( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) can be found viasolving the convex optimization problem.
Revised: Based on the convexity of negative optimal Q-function,the optimal sequence of states ( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) can be found viasolving the convex optimization problem.

Original: Then, we have the following theorem.
Revised: Then, we have the following theorem.

Original: Theorem 2.
Revised: Theorem 2.

Original: Let f ∗ ( · ; W ) denote the L O C A L constructed by the optimal sequences of states( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) from − Q ∗ ( · ) , where W is the set of weightsof f ∗ ( · ; W ) .
Revised: Let f ∗ ( · ; W ) denote the L O C A L constructed by the optimal sequences of states( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) from − Q ∗ ( · ) , where W is the set of weightsof f ∗ ( · ; W ) .

Original: If f ∗ ( · ; W ) can be trained with noiseless datasets and the training can achieve theglobal optimal weights W ∗ , f ∗ ( · ; W ∗ ) can be simplified to the true equation g ( · ) .
Revised: If f ∗ ( · ; W ) can be trained with noiseless datasets and the training can achieve theglobal optimal weights W ∗ , f ∗ ( · ; W ∗ ) can be simplified to the true equation g ( · ) .

Revised: For the optimal search structure of L O C A L, i.e., f ∗ ( · ; W ) , the optimal weight set W ∗ is also trainedvia gradient method like Adam [33].

Original: The proof can be seen in Appendix A.3.
Revised: The proof can be seen in Appendix A.4.

Original: Theorem 2 requires that L O C A L can learn the global optimalweights.
Revised: Theorem 2 requiresthat L O C A L can learn the global optimal weights.

Original: The requirement is generally hard to achieve due to the non-linearity and non-convexityof L O C A
Revised: The requirement is generally hard to achieve dueto the non-linearity and non-convexity of L O C A

Original: L.
Revised: L.

Original: However, we show that with mild assumptions, there are local regions in the L O C A Lloss surface with strict convexity.
Revised: However, we show that with mild assumptions,there are local regions in the L O C A L loss surface with strict convexity.

Original: Then, if we have proper initializations, the gradient-based weightupdating can find the global optimum.
Revised: Then, if we have properinitializations, the gradient-based weight updating can find the global optimum.

Original: Specifically, we have the following theorem.
Revised: Specifically, we havethe following theorem.

Original: Theorem 3.
Revised: Theorem 3.

Original: Assume the following conditions hold: (1) the equation g ( x ) is C 2 smooth and hasbounded second derivatives with respect to weights, (2) ∃ x ∈ X , g ( x ) has non-zero gradients withrespect to weights, (3) the structure of L O C
Revised: Assume the following conditions hold: (1) the equation g ( x ) is C 2 smooth and hasbounded second derivatives with respect to weights, (2) ∃ x ∈ X , g ( x ) has non-zero gradients withrespect to weights, (3) the structure of L O C

Original: A L is correctly searched to exactly represent symbolsand symbol connections in g ( x ) , and (4) the training dataset of L O C
Revised: A L is correctly searched to exactly represent symbolsand symbol connections in g ( x ) , and (4) the training dataset of L O C

Original: A L is noiseless.
Revised: A L is noiseless.

Original: Then, for the
Revised: Then, for the

Original: MSE loss surface of L O C A L , each global optimal point has a strictly convex local region.
Revised: MSE loss surface of L O C A L , each global optimal point has a strictly convex local region.

Original: The proofs can be seen in Appendix A.4.
Revised: The proofs can be seen in Appendix A.5.

Original: Note that Assumptions 1-2 easily hold for common physicalequations in nature [2].
Revised: Note that Assumptions 1-2 easily hold for common physicalequations in nature [2].

Original: Assumption 3 relies on the search algorithm, and we show, both theoreticallyand numerically, that our double convex deep Q-learning has good performances.
Revised: Assumption 3 relies on the search algorithm, and we show, both theoretically and numerically, that our double convex deep Q-learning has good performances.

Original: Assumption 4relies on the quality of data and we focus on the noiseless data in this paper.
Revised: Assumption 4relies on the quality of data and we focus on the noiseless data in this paper.

Original: What’s more, Equation(6) in the proof suggests that if the absolute noise values are small, the locally convex region stillexists.
Revised: What’s more, Equation(6) in the proof suggests that if the absolute noise values are small, the locally convex region stillexists.

Original: We also numerically prove C O NS O L E is robust under certain noise levels in Section 5.5.
Revised: We also numerically prove C O NS O L E is robust under certain noise levels in Section 5.5.

Original: Tosummarize, these assumptions are acceptable.
Revised: Tosummarize, these assumptions are acceptable.

Original: To quantify the range of the local region for a L O C A
Revised: To quantify the range of the local region for a L O C A

Original: Lwith a certain complexity, we have the following theorem.
Revised: Lwith a certain complexity, we have the following theorem.

Original: Theorem 4.
Revised: Theorem 4.

Original: Suppose Assumptions 1-4 in Theorem 3 hold.
Revised: Suppose Assumptions 1-4 in Theorem 3 hold.

Original: For a L O C A L with one symbolicactivation, multiplication, and summation layer, the set of local convex regions with global optima isU = { W
Revised: For a L O C A L with one symbolicactivation, multiplication, and summation layer, the set of local convex regions with global optima is

Original: [Equation]
Revised: [Equation]

Original: (cid:12)(cid:12)

Original: [-[Equation] ( x i ,W + tX ) (cid:12)(cid:12)η (cid:12)(cid:12) ddt (cid:12)(cid:12) t =
 ˆ y ( x j ,W + tX ) (cid:12)(cid:12) > | ˆ y ( x k , W ) − y k |}-] , where notations are defined in the proof.
Revised: [-[Equation] ( x i ,W + tX ) (cid:12)(cid:12)η (cid:12)(cid:12) ddt (cid:12)(cid:12) t =
 ˆ y ( x j ,W + tX ) (cid:12)(cid:12) > | ˆ y ( x k , W ) − y k |}-] , where notations are defined in the proof.

Original: The proofs can be seen in Appendix A.5.
Revised: The proofs can be seen in Appendix A.6.

Original: We explain the region range is large for a stable system thatsatisfies all assumptions in Theorem 4.
Revised: We explain the region range is large for a stable system thatsatisfies all assumptions in Theorem 4.

Original: Physical interpretations of the convex region size.
Revised: Physical interpretations of the convex region size.

Original: Physical systems in scientific and engineeringdomains have certain stability that can withstand parameter changes to some extent.
Revised: Physical systems in scientific and engineeringdomains have certain stability that can withstand parameter changes to some extent.

Original: Further, thisability should hold for arbitrary x ∈ X and y ∈ Y .
Revised: Further, thisability should hold for arbitrary x ∈ X and y ∈ Y .

Original: Thus, we can assume ddt (cid:12)(cid:12) t =

Revised: Thus, we can assume ddt (cid:12)(cid:12) t =

Original: ˆ y
Revised: ˆ y

Original: ( x i , W + tX ) ≈ddt (cid:12)(cid:12) t
Revised: ( x i , W + tX ) ≈ddt (cid:12)(cid:12) t

Original: =

Revised: =

Original: ˆ y ( x j , W + tX )
Revised: ˆ y ( x j , W + tX )

Original: ≈ ddt (cid:12)(cid:12) t
Revised: ≈ ddt (cid:12)(cid:12) t

Original: =

Revised: =

Original: ˆ y
Revised: ˆ y

Original: ( x k , W + tX )
Revised: ( x k , W + tX )

Original: for x i , x j , x k ∈ X .
Revised: for x i , x j , x k ∈ X .

Original: Then, the inequality in set Ucan be approximately rewritten as 1 η > || w − w ∗
Revised: Then, the inequality in set Ucan be approximately rewritten as 1 η > || w − w ∗

Original: || 2 , where w and w ∗ are the vectorized W and W ∗ ,respectively.
Revised: || 2 , where w and w ∗ are the vectorized W and W ∗ ,respectively.

Original: Namely, the distance between any point W in the region to the global optimal point
Revised: Namely, the distance between any point W in the region to the global optimal point

Original: W ∗ in the region is bounded by 1 η .
Revised: W ∗ in the region is bounded by 1 η .

Original: Based on Equation (19), η is the bound of the ratio of secondderivative to the first derivative.
Revised: Based on Equation (19), η is the bound of the ratio of secondderivative to the first derivative.

Original: For a stable system, this ratio should be small.
Revised: For a stable system, this ratio should be small.

Original: Otherwise, the systemcan easily crash with a small parameter disturbance.
Revised: Otherwise, the systemcan easily crash with a small parameter disturbance.

Original: Thus, 1 η is relatively large and so is the regionof U .
Revised: Thus, 1 η is relatively large and so is the regionof U .

Original: An example of the range is displayed in Section 5.2.
Revised: An example of the range is displayed in Section 5.2.

Original: Finally, the above analysis also holdswhen the L O C
Revised: Finally, the above analysis also holdswhen the L O C

Original: A L of the system equation has more than one symbolic activation, multiplication, andsummation layers.
Revised: A L of the system equation has more than one symbolic activation, multiplication, andsummation layers.

Original: This is because Equation (20) always holds as long as we can find a η to boundthe ratio of the second derivative to the first derivative, which is irrelevant to the structure of L O C A L.
Revised: This is because Equation (20) always holds as long as we can find a η to boundthe ratio of the second derivative to the first derivative, which is irrelevant to the structure of L O C A L.

Original:  Experiments5.1 Settings
Revised:  Experiments

Revised: 5.1 Settings

Original: Datasets.
Revised: Datasets.

Original: We use the following datasets for testing.
Revised: We use the following datasets for testing.

Original: (1) Synthetic datasets .
Revised: (1) Synthetic datasets .

Original: We create two datasets,
Revised: We create two datasets,

Original: Syn 1 and Syn 2 , for testing.
Revised: Syn 1 and Syn 2 , for testing.

Original: Syn 1 has the following equations: y 1 = 3 x 21 cos(2 .
Revised: Syn 1 has the following equations: y 1 = 3 x 21 cos(2 .

Original: 5 x 2 ) , y 

Revised: 5 x 2 ) , y

Original: = 4 x 1 x 3 ,and y 

Revised: = 4 x 1 x 3 ,and y

Original: = 3 x 23 .
Revised: = 3 x 23 .

Original: Syn 2 is more complex with the following equations y 1 = √ 2 .
Revised: Syn 2 is more complex with the following equations y 1 = √ 2 .

Original: 2 x 1 x 2 + x 1 x 22 ,y 2 = sin(1 .
Revised: 2 x 1 x 2 + x 1 x 22 ,y 2 = sin(1 .

Original: 8 x 1 ) (cid:0) log(3 x 2 ) + √ x 3 (cid:1) , y 3 = √ 3 .
Revised: 8 x 1 ) (cid:0) log(3 x 2 ) + √ x 3 (cid:1) , y 3 = √ 3 .

Original: 7 x 3 log(1 .
Revised: 7 x 3 log(1 .

Original: 6 x 1 ) + x 21 .
Revised: 6 x 1 ) + x 21 .

Original: For the training data, each inputvariable is randomly sampled from a uniform distribution of U (1 , 2) to avoid invalid values likelog(0) .
Revised: For the training data, each inputvariable is randomly sampled from a uniform distribution of U (1 , 2) to avoid invalid values likelog(0) .

Original: Totally, we create 2 , 000 samples for training.
Revised: Totally, we create 2 , 000 samples for training.

Original: Then, in the test phase, we utilize another2 , 000 samples whose input variables are sampled from U (3 , 4) .
Revised: Then, in the test phase, we utilize another2 , 000 samples whose input variables are sampled from U (3 , 4) .

Original: The symbolic activation pools are{ x, x 2 , cos( x ) } and {√ x, x, x 2 , log( x ) , sin( x ) } for Syn 1 and Syn 2 , respectively.(2) Power system dataset.
Revised: The symbolic activation pools are{ x, x 2 , cos( x ) } and {√ x, x, x 2 , log( x ) , sin( x ) } for Syn 1 and Syn 2 , respectively.(2) Power system dataset.

Original: Power flow equation determines the operations of electric systems [36].
Revised: Power flow equation determines the operations of electric systems [37].

Original: For node i in an M -node system, the equation can be written as p i
Revised: For node i in an M -node system, the equation can be written as p i

Original: = (cid:80)
Revised: = (cid:80)

Original: | M | m =1 G im ( u i u m + v i v m )
Revised: | M | m =1 G im ( u i u m + v i v m )

Original: +
Revised: +

Original: B im ( v i u m − u i v m ) and q i = (cid:80) [-Mm =
-]
Revised: B im ( v i u m − u i v m ) and q i = (cid:80) [-Mm =
-]M m =1 G

Original: G im ( v i u m − u i v m ) − B im ( v i u m − u i v m ) , where u i and v iare the real and imaginary components of the voltage phasor at node i .
Revised: im ( v i u m − u i v m ) − B im ( v i u m − u i v m ) , where u i and v iare the real and imaginary components of the voltage phasor at node i .

Original: p i and q
Revised: p i and q

Original: i are the activeand reactive power at node i .
Revised: i are the activeand reactive power at node i .

Original: G
Revised: G

Original: im and B im represent the physical parameters of line im .
Revised: im and B im represent the physical parameters of line im .

Original: If lineim does not exist, G im =
Revised: If lineim does not exist, G im =

Original: B im = 0 .
Revised: B im = 0 .

Original: Therefore, we can treat x =
Revised: Therefore, we can treat x =

Original: [ u 1 , v 1 , · · · , u M , v M ] T andy =
Revised: [ u 1 , v 1 , · · · , u M , v M ] T andy =

Original: [ p 1 , q 1 , · · · , p M , q M ] T .
Revised: [ p 1 , q 1 , · · · , p M , q M ] T .

Original: The target is to learn the underlying system topology and parameters,which has broad impacts on the power domains [37].
Revised: The target is to learn the underlying system topology and parameters,which has broad impacts on the power domains [38].

Original: In this experiment, we implement simulationfrom a 5 -node system using MATPOWER [38] and two year’s hourly data.
Revised: In this experiment, we implement simulationfrom a 5 -node system using MATPOWER [39] and two year’s hourly data.

Original: The first 8 , 760 points areused for training while the remaining samples are used for testing.
Revised: The first 8 , 760 points areused for training while the remaining samples are used for testing.

Original: The symbolic activation pool is{ x } .
Revised: The symbolic activation pool is{ x } .

Original: We denote this dataset as Pow.(3) Mass-damper system dataset.
Revised: We denote this dataset as Pow.(3) Mass-damper system dataset.

Original: Equations of the mass-damper system can be approximatelywritten as: ˙ q = − DRD ⊤ M − 1 q , where ˙ q is a vector of momenta, D is the incidence matrix ofthe system, R is the diagonal matrix of the damping coefficients for each line of the system, and M is the diagonal matrix of each node mass of the system.
Revised: Equations of the mass-damper system can be written as: ˙ q =− DRD ⊤ M − 1 q , where ˙ q is a vector of momenta, D is the incidence matrix of the system, R is thediagonal matrix of the damping coefficients for each line of the system, and M is the diagonal matrixof each node mass of the system.

Original: Thus, we can set y = ˙
Revised: Thus, we can set y = ˙

Original: q and x = q andthe goal is to learn the parameter matrix − DRD ⊤ M − 1 .
Revised: q and x = q and the goal is to learn theparameter matrix − DRD ⊤ M − 1 .

Original: We conduct the simulation via MATLABfor a 10 -node system and obtain 6 , 000 points for 1 min simulationwith a step size to be 0 . 01 s. Thefirst 3 , 000 samples are used for training while the rest samples are used for testing. The symbolicactivation pool is { x } . Then, we denote the dataset as Mas.
Revised: We conduct the simulation via MATLAB for a 10 -node systemand obtain 6 , 000 points for 1 min simulation

Original: Benchmark methods .
Revised: Benchmark methods .

Original: The following benchmark methods are utilized.
Revised: The following benchmark methods are utilized.

Original: (1) Deep Symbolic Regression( DSR ) [3].
Revised: (1) Deep Symbolic Regression( DSR ) [3].

Original: DSR develops an RNN-based framework to search the expression tree.
Revised: DSR develops an RNN-based framework to search the expression tree.

Original: Especially, therisk-seeking policy gradient is utilized to seek the best performance.
Revised: Especially, therisk-seeking policy gradient is utilized to seek the best performance.

Original: Then, BFGS [11] can solve thenon-linear optimization and estimate the symbol coefficients
Revised: Then, BFGS [11] can solve thenon-linear optimization and estimate the symbol coefficients.

Original:  VPG is a vanilla version of DSR with a normal policy gradient rather than the risk-seeking method.(3) Equation Learner ( EQL ) [6, 7].
Revised: (2) Vanilla Policy Gradient ( VPG ) [3]. VPG is a vanilla version of DSR with a normal policy gradient rather than the risk-seeking method.(3) Equation Learner ( EQL ) [6, 7].

Original: EQL creates an end-to-end NN to select symbols and estimatethe coefficients.
Revised: EQL creates an end-to-end NN to select symbols and estimatethe coefficients.

Original: The sparse regularization is enforced for the NN weights to search symbols.
Revised: The sparse regularization is enforced for the NN weights to search symbols.

Original: Multilayer Perceptron ( MLP ).
Revised: Multilayer Perceptron ( MLP ).

Original: We also employ a standard MLP to learn the regression from x to y .
Revised: We also employ a standard MLP to learn the regression from x to y .

Original: We only evaluate the extrapolation capacity of MLP in the test dataset.
Revised: We only evaluate the extrapolation capacity of MLP in the test dataset.

Original: For DSR, VPG, and EQL methods, based on the input datasets, we adjust the symbol and operatorlibrary to enable the same searching space as C O NS O L E for fair comparisons.
Revised: For DSR, VPG, and EQL methods, based on the input datasets, we adjust the symbol and operatorlibrary to enable the same searching space as C O NS O L E for fair comparisons.

Original: We run the benchmarkmethods 5 times with different random seeds and present their best results.
Revised: We run the benchmarkmethods 5 times with different random seeds and present their best results.

Original: As for our method, weonly run 1 time and obtain good results due to the convex design and the ϵ -greedy strategy.
Revised: As for our method, weonly run 1 time and obtain good results due to the convex design and the ϵ -greedy strategy.

Original: Metrics for evaluation.
Revised: Metrics for evaluation.

Original: We employ the following metrics.
Revised: We employ the following metrics.

Original: (1) Average coefficient estimation percentage error E c .
Revised: (1) Average coefficient estimation percentage error E c .

Original: For an equation with H symbols, We calculate the error as E c = 1 H (cid:80)
Revised: For an equation with H symbols, We calculate the error as E c = 1 H (cid:80)

Original: h
Revised: h

Original: PE ( w h , ˆ w h ) ,where w h and ˆ w h represents the true and the estimated coefficients for the h th symbol, respectively.
Revised: PE ( w h , ˆ w h ) ,where w h and ˆ w h represents the true and the estimated coefficients for the h th symbol, respectively.

Original: PE is the operation to calculate the percentage error.
Revised: PE is the operation to calculate the percentage error.

Original: If there is no matched symbol for the h th truesymbol, we denote PE ( w h , ˆ w h ) = 100% .
Revised: If there is no matched symbol for the h th truesymbol, we denote PE ( w h , ˆ w h ) = 100% .

Original: Note that when calculating E c , proper simplifications maybe needed.
Revised: Note that when calculating E c , proper simplifications maybe needed.

Original: For example, cos (cid:0) 2 . 5( √ x ) 2 (cid:1) = cos(2 .
Revised: For example, cos (cid:0) 2 . 5( √ x ) 2 (cid:1) = cos(2 .

Original: 5 x ) .
Revised: 5 x ) .

Original: (2) NRMSE in the test dataset.
Revised: (2) NRMSE in the test dataset.

Original: We measurethe extrapolation capacity in the test dataset and utilize NRMSE employed in Section 3.2.
Revised: We measurethe extrapolation capacity in the test dataset and utilize NRMSE employed in Section 3.2.

Original: Implementing details of C O NS O L E .
Revised: Implementing details of C O NS O L E .

Original: Hyper-parameters of C O NS
Revised: Hyper-parameters of C O NS

Original: O L E exist for both the doubleconvex deep Q-learning and the L O C A L.
Revised: O L E exist for both the doubleconvex deep Q-learning and the L O C A L.

Original: In the deep Q-learning, we set γ = 0 .
Revised: In the deep Q-learning, we set γ = 0 .

Original: 2 , ϵ = 0 .
Revised: 2 , ϵ = 0 .

Original: 4 , T = 600 ,λ = 10 − 2 , T 0 = 10 for Algorithm 1.
Revised: 4 , T = 600 ,λ = 10 − 2 , T 0 = 10 for Algorithm 2.

Original: Furthermore, to train the negative Q-function and the rewardfunction, we set the learning rate to be 5 × 10 − 3 and the number of epochs for training to be 50 .
Revised: Furthermore, to train the negative Q-function and the rewardfunction, we set the learning rate to be 5 × 10 − 3 and the number of epochs for training to be 50 .

Original: Then, we set the batch size for the negative Q-function to be 100 .
Revised: Then, we set the batch size for the negative Q-function to be 100 .

Original: If the number of data in the replaybuffer is less than 100 , no training happens for the negative Q-function.
Revised: If the number of data in the replaybuffer is less than 100 , no training happens for the negative Q-function.

Original: Additionally, all the datagathered in one episode are used to train the negative reward function.
Revised: Additionally, all the datagathered in one episode are used to train the negative reward function.

Original: As for the L O C A L, we set
Revised: As for the L O C A L, we set

Original: K = 3 , the learning rate to be 1 × 10 − 2 and the number of training epochs to be 8 .
Revised: K = 3 , the learning rate to be 1 × 10 − 2 and the number of training epochs to be 8 .

Original: We make thesetraining epochs to be small since training the L O C
Revised: We make thesetraining epochs to be small since training the L O C

Original: A L is the most time-consuming part of C O NS O L E .
Revised: A L is the most time-consuming part of C O NS O L E .

Original: Furthermore, if the structure of L O C A L is correctly searched, a small number of iterations can help
Revised: Furthermore, if the structure of L O C A L is correctly searched, a small number of iterations can help

Original: L O C A L to gain the global optimal weights.
Revised: L O C A L to gain the global optimal weights.

Original: Finally, we initialize all trainable weights in L O C
Revised: Finally, we initialize all trainable weights in L O C

Original: A L tobe 1 .
Revised: A L tobe 1 .

Original: The following results show that a relatively large area is suitable for an initial guess of L O C A L.
Revised: The following results show that a relatively large area is suitable for an initial guess of L O C A L.

Original: 5.2 Verification and 3-D Visualization of Convex Mechanisms
Revised: 5.2 Verification and 3-D Visualization of Convex Mechanisms

Original: We first utilize a toy example to verify the benefits of convex designs for two sub problems.
Revised: We first utilize a toy example to verify the benefits of convex designs for two sub problems.

Original: Specifically, we consider to learn y 1 = 3 x 21 cos(2 .
Revised: Specifically, we consider to learn y 1 = 3 x 21 cos(2 .

Original: 5 x 2 ) with the loss function L = (cid:80) Ni
Revised: 5 x 2 ) with the loss function L = (cid:80) Ni

Original: ( y i [1] −w 1 x i [1] 2 cos( w 2 x i [2]) 2 , where the data sample notations are defined in Section 3.1.
Revised: ( y i [1] −w 1 x i [1] 2 cos( w 2 x i [2]) 2 , where the data sample notations are defined in Section 3.1.

Original: As shown in
Revised: As shown in

Original: Fig.
Revised: Fig.

Original: 2a, we design a 4 -layer L O C A L with two learnable parameters w 1 and w 2 to represent thecoefficients.
Revised: 2a, we design a 4 -layer L O C A L with two learnable parameters w 1 and w 2 to represent the coefficients.

Original: Thus, in the search phase, the goal is to identify the action a 1 .
Revised: Thus, in the search phase, the goal is to identify the action a 1 .

Original: We plot − Q t ( · ) whent = 1 , 5 , 10 in Fig. 2b.
Revised: We plot − Q t ( · ) whent = 1 , 5 , 10 in Fig. 2b.

Original: The convexity always exists so that the algorithm can quickly find the global optimal solutions in red dots.
Revised: The convexity always exists so that the algorithm can quickly find the globaloptimal solutions in red dots.

Original: Next, as the agent update Q values with true rewards, the Q-functionconverges to the optimal function within 10 episodes.
Revised: Next, as the agent update Q values with true rewards, the Q-functionconverges to the optimal function within 10 episodes.

Original: Finally, the convex − Q ∗ ( · ) remains unchangedand can bring the optimal action and the true equation, which supports Theorem 2.
Revised: Finally, the convex − Q ∗ ( · ) remains unchangedand can bring the optimal action and the true equation, which supports Theorem 2.

Original: Subsequently, we plot the loss surface of L in Fig.
Revised: Subsequently, we plot the loss surface of L in Fig.

Original: 2c.
Revised: 2c.

Original: We find that around the two global optima(3 , 2 . 5) and (3 , − 2 . 5) , there are convex regions that have an approximate quadratic shape.
Revised: We find that around the two global optima(3 , 2 . 5) and (3 , − 2 . 5) , there are convex regions that have an approximate quadratic shape.

Original: To furtherquantify the range for proper initialization, we vary the initial weight w 0 ∈ {− 10 , − 9 , · · · , 10 } forw 1 and w 2 in L O C A L. Fig.
Revised: To furtherquantify the range for proper initialization, we vary the initial weight w 0 ∈ {− 10 , − 9 , · · · , 10 } forw 1 and w 2 in L O C A L. Fig.

Original: 2 reports the final training loss with respect to different w 0 , and we findthe safe range for initialization is [ − 3 , 7] \ { 0 } .
Revised: 2 reports the final training loss with respect to different w 0 , and we findthe safe range for initialization is [ − 3 , 7] \ { 0 } .

Original: This range is relatively large when compared to theoptimal values.
Revised: This range is relatively large when compared to theoptimal values.

Original: These observations support our Theorems 3 and 4.
Revised: These observations support our Theorems 3 and 4.

Original: Finally, w 0 = 0 does not worksince ∂L∂w 2 (cid:12)(cid:12) w 2 =

Revised: Finally, w 0 = 0 does not worksince ∂L∂w 2 (cid:12)(cid:12) w 2 =

Original: = 0 always holds, which prevents the weight updating using gradient methods.
Revised: = 0 always holds, which prevents the weight updating using gradient methods.

Original: [Table]
Revised: [Table]

Original: y

Original: [Table]

Original: =

Original: [Table]

Original: y

Original: [Table]

Original: = [Table] [Table] 5.3 Convexity Guarantees of C O NS O L E to Learn Correct Equations
Revised: .3 Convexity Guarantees of C O NS O L E to Learn Correct Equations

Original: In this subsection [Table] we report the results of the equation learning.
Revised: In this subsection, we report the results of the equation learning.

Original: First, we list the learned equationsfor Syn 1 and Syn 2 in Table 1.
Revised: First, we list the learned equationsfor Syn 1 and Syn 2 in Table 1.

Original: Table 1 presents that
Revised: Table 1 presents that

Original: C
Revised: C

Original: O
Revised: O

Original: NS
Revised: NS

Original: O
Revised: O

Original: L E has the best performance in most ofthe equations while DSR ranks second.
Revised: L E has the best performance in most ofthe equations while DSR ranks second.

Original: In particular, C O NS O L E can accurately learn all equations in
Revised: In particular, C O NS O L E can accurately learn all equations in

Original: Syn 1 and Syn 2 .
Revised: Syn 1 and Syn 2 .

Original: The superior performance is mostly due to the convex design of the search and thecoefficient estimation process with provable guarantees.
Revised: The superior performance is mostly due to the convex design of the search and thecoefficient estimation process with provable guarantees.

Original: In addition, we observe that for the resultof C O NS O L E in learning y 3 of Syn 1 , we have 3 x 23 cos(0 .
Revised: In addition, we observe that for the resultof C O NS O L E in learning y 3 of Syn 1 , we have 3 x 23 cos(0 .

Original: 005 x 1 ) ≈ 3 x 23 .
Revised: 005 x 1 ) ≈ 3 x 2 3 .

Original: This shows that there is apossibility that the search result of C O NS O L E might not be optimal (i.e., an extra consine term existsbut is close to 1 ), but the learned equation is still highly accurate.
Revised: This shows that there is apossibility that the search result of C O NS O L E might not be optimal (i.e., an extra consine term existsbut is close to 1 ), but the learned equation is still highly accurate.

Original: Such an observation nonethelessguides further study of the coupling relationship between the search and the estimation procedures.
Revised: Such an observation nonethelessguides further study of the coupling relationship between the search and the estimation procedures.

Original: [Figure]
Revised: [Figure]

Original: DSR doesn’t perform well when the underlying equation is relatively complex, e.g., y 1 in Syn 1 and y 1and y 2 in Syn 2 .
Revised: DSR doesn’t perform well when the underlying equation is relatively complex, e.g., y 1 in Syn 1 and y 1and y 2 in Syn 2 .

Original: This is because DSR may still fall into a local optimal solution despite risk-seeking policy design.
Revised: This is because DSR may still fall into a local optimal solution despite risk-seekingpolicy design.

Original: VPG method performs worse than DSR since VPG considers an expected reward [3].
Revised: VPG method performs worse than DSR since VPG considers an expected reward [3].

Original: Finally, EQL performs the worst as it merges the symbol search and the coefficient estimation in one
Revised: Finally, EQL performs the worst as it merges the symbol search and the coefficient estimation in one

Original: NN model, which provides few guarantees of accurate learning.
Revised: NN model, which provides few guarantees of accurate learning.

Original: The above observations and analysesare consistent with the result of coefficient estimation errors and prediction errors in Fig. 3a and 3b.
Revised: The above observations and analysesare consistent with the result of coefficient estimation errors and prediction errors in Fig. 3a and 3b.

Original: For the Pow and Mas, C O NS O L E doesn’t learn completely exact equations within T = 600 episodes.
Revised: For the Pow and Mas, C O NS O L E doesn’t learn completely exact equations within T = 600 episodes.

Original: This is because they have a large number of variables to be considered.
Revised: This is because they have a large number of variables to be considered.

Original: However, C O NS O L E is stillbetter than other methods.
Revised: However, C O NS O L E is stillbetter than other methods.

Original: 5.4 Ablation Study: Exploration and Convex Search are Essential
Revised: 5.4 Ablation Study: Exploration and Convex Search are Essential

Original: We conduct an ablation study to further understand what factors are important in the C O NS O L E .
Revised: We conduct an ablation study to further understand what factors are important in the C O NS O L E .

Original: We test the result with Syn 1 and Syn 1 and report the E c (%) values.
Revised: We test the result with Syn 1 and Syn 1 and report the E c (%) values.

Original: Specifically, we investigate thefollowing cases.
Revised: Specifically, we investigate thefollowing cases.

Original: (1) No ablation.
Revised: (1) No ablation.

Original: (2) Drop exploration in deep Q-learning.
Revised: (2) Drop exploration in deep Q-learning.

Original: We delete the ϵ -greedystrategy.
Revised: We delete the ϵ -greedystrategy.

Original: (3) Drop double-convex deep-Q learning.
Revised: (3) Drop double-convex deep-Q learning.

Original: We replace this design with a traditional deep-Qlearning.
Revised: We replace this design with a traditional deep-Qlearning.

Original: (4) Drop coefficient estimation using L O C A L. After learning the structure of L O C A L, wereformulate a non-linear optimization and utilize BFGS [11] in DSR, instead of gradient descent in
Revised: (4) Drop coefficient estimation using L O C A L. After learning the structure of L O C A L, wereformulate a non-linear optimization and utilize BFGS [11] in DSR, instead of gradient descent in

Original: L O C A L, to estimate the coefficient.
Revised: L O C A L, to estimate the coefficient.

Original: (5) Drop static symbolic constraint.
Revised: (5) Drop static symbolic constraint.

Original: (6) Drop dynamic symbolicconstraint.
Revised: (6) Drop dynamic symbolicconstraint.

Original: These two constraints are mentioned in Section 3.2.
Revised: These two constraints are mentioned in Section 3.2.

Original: Then, Fig. 4 shows that cases (2) and(3) cause large errors.
Revised: Then, Fig. 4 shows that cases (2) and(3) cause large errors.

Original: For case (2) , if no exploration strategy is added, the updating of the Q-functionand the reward function is slow.
Revised: For case (2) , if no exploration strategy is added, the updating of the Q-functionand the reward function is slow.

Original: For case (3) , the non-convex search induces many sub-optimalactions in the search process.
Revised: For case (3) , the non-convex search induces many sub-optimalactions in the search process.

Original: Thus, these two cases cause a slow search process and significant errorsafter T = 600 episodes.
Revised: Thus, these two cases cause a slow search process and significant errorsafter T = 600 episodes.

Original: For symbolic constraints in (5) and (6) , removing them slightly increasesthe error for Syn 1 .
Revised: For symbolic constraints in (5) and (6) , removing them increases the errorfor Syn 2 and Syn 1 , respectively.

Original: This shows these constraints are beneficial to the search process.
Revised: This shows these constraints are beneficial to the search process.

Original: Finally, wefind that utilizing BFGS in (4) can bring good results with initialization in the locally convex region.
Revised: Finally, we find that utilizing BFGS in (4) can bring good results with initialization in the locallyconvex region.

Original: Since the non-linear optimization has the same loss surface as L O C A L, the locally convex region in Theorems 3 and 4 can prove the good performance of BFGS.
Revised: Since the non-linear optimization has the same loss surface as L O C A L, the locallyconvex region in Theorems 3 and 4 can prove the good performance of BFGS.

Original: [Figure]
Revised: [Figure]

Original: 5. C O NS O L E is Robust with Changing Noise Levels and Data Volume
Revised: 5. C O NS O L E is Robust with Changing Noise Levels and Data Volume

Original: We utilize Syn 1 and Syn 2 to examine the robustness of the framework with changing noiselevels and data volumes.
Revised: We utilize Syn 1 and Syn 2 to examine the robustness of the framework with changing noiselevels and data volumes.

Original: For the noise level, we consider the Signal-to-Noise Ratio (SNR)such that SNR ∈ { 80 , 90 , 100 , 110 , 120 } .
Revised: For the noise level, we consider the Signal-to-Noise Ratio (SNR)such that SNR ∈ { 80 , 90 , 100 , 110 , 120 } .

Original: For the data volume, we fix SNR = 100 and vary
Revised: For the data volume, we fix SNR = 100 and vary

Original: N ∈ { 500 , 1000 , 1500 , 2000 , 2500 } .
Revised: N ∈ { 500 , 1000 , 1500 , 2000 , 2500 } .

Original: Fig.
Revised: Fig.

Original: 5a and 5b demonstrate the results.
Revised: 5a and 5b demonstrate the results.

Original: We find that when
Revised: We find that when

Original: SNR ≥ 100 , the error can be less than 1% .
Revised: SNR ≥ 100 , the error can be less than 1% .

Original: This noise level is suitable to real-word systems.
Revised: This noise level is suitable to real-word systems.

Original: Forexample, SNR = 125 for electric measurements [37].
Revised: Forexample, SNR = 125 for electric measurements [38].

Original: For the data volume, the overall error is lessthan 2% when N ≥ 500 , which shows a robust performance of C O NS O L E .
Revised: For the data volume, the overall error is lessthan 1 . 5% when N ≥ 500 , which shows a robust performance of C O NS O L E .

Original:  Conclusions and Future Work
Revised:  Conclusions and Future Work

Original: In this paper, we propose C O NS O L E , a novel Convex Neural Symbolic Learning method, whichenjoys convexity with certain conditions to tackle Symbolic Regression with guaranteed performances.
Revised: In this paper, we propose C O NS O L E , a novel Convex Neural Symbolic Learning method, whichenjoys convexity with certain conditions to tackle Symbolic Regression with guaranteed performances.

Original: Specifically, we convexify the search problem by proposing a double convex deep-Q learning.
Revised: Specifically, we convexify the search problem by proposing a double convex deep-Q learning.

Original: In themeantime, we prove the local and strict convexity of the coefficient estimation in our Locally-Convexequation Learner (L O C
Revised: In themeantime, we prove the local and strict convexity of the coefficient estimation in our Locally-Convexequation Learner (L O C

Original: A L).
Revised: A L).

Original: To our best knowledge, C O NS O
Revised: To our best knowledge, C O NS O

Original: L E is the first method that providesguarantees with reasonable assumptions to learn exact equations.
Revised: L E is the first method that providesguarantees with reasonable assumptions to learn exact equations.

Original: Besides, C O NS O L E has, at aminimum, a broader impact on the following domains.
Revised: Besides, C O NS O L E has, at aminimum, a broader impact on the following domains.

Original: (1) Convex control for physical systems using
Revised: (1) Convex control for physical systems using

Revised: Reinforcement Learning.

Revised: (2) Neural networks in engineering applications with local convexity.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] We specify assumptions needed
Revised: [Yes] We specify assumptions needed

Original: for good performances in Section 4.
Revised: for good performances in Section 4.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A]
Revised: [N/A]

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] See Section 4.
Revised: [Yes] See Section 4.

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] See Appendix A.2 toA.
Revised: [Yes] See Appendix A.3 to A.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [No] We will release
Revised: [No] We will release

Original: the code if the paper is accepted.
Revised: the code if the paper is accepted.

Original: (b) Did you specify all the training details (e g., data splits, hyperparameters,
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters,

Original: how they
Revised: how they

Original: were chosen)?
Revised: were chosen)?

Original: [Yes] See Section 5.1.
Revised: [Yes] See Section 5.1.

Original: (c) Did you report error bars (e
Revised: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experi-
Revised: g., with respect to the random seed after running experi-

Original: ments multiple times)?
Revised: ments multiple times)?

Original: [Yes] See Section 5.3.
Revised: [Yes] See Section 5.3.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [N/A
Revised: [N/A]

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...(
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: a)
Revised: (a)

Original: If your work uses existing assets , did you cite 
Revised: If your work uses existing assets, did you cite the creators?

Original: the

Original: creators

Original: [N/A
Revised: [N/A]

Original: b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A
Revised: [N/A]

Original: c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]
Revised: [N/A]

Original: d)

Original:  Did you discuss whether and how consent was obtained 
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: from people whose data you’reusing curating
Revised: using/curating?

Original: [N/A
Revised: [N/A]

Original: e) Did you discuss whether the data you are using/ curating
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: contains personally identifiableinformation or offensive content
Revised: information or offensive content?

Original: N/A
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...(
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
Revised: (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?

Original: [N/A](b)
Revised: [N/A](b)

Original: Did you describe any potential participant risks, with links to Institutional Review
Revised: Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N A](
Revised: [N/A]

Original: c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A
Revised: [N/A]

Original: A Appendix
Revised: A AppendixA.1 Model Overview with Pseudo Codes

Revised: In this subsection, we provide a high-level summary of our framework for better understanding. Wepresent the summary in the form of pseudo codes, shown in Algorithm 1. Algorithm 1 The Overview of the Proposed Framework.

Revised: Input: Training dataset { x i , y i } Ni =1 .

Revised: Step 1: Design L O C A L. L O C

Revised: A L has repeated blocks of symbolic activation, multiplication, and summation layers.

Revised: For example, Fig. 1 presents a L O C A L with 2 blocks.

Revised: Step 2:

Revised: Denote L O C A L Function.

Revised: L O C A L represents the map f

Revised: ( x ; { Z k } K − 1 k =0 , { W k } K − 1 k =0 ) from x to y .

Revised: With global optimal solutions of Z k and W k , f ( x ) can be simplified to the true equation g ( x ) .

Revised: while L O C A L does not have the optimal performance do

Revised: Step 3: Search L O C A L Structure.

Revised: Step 3.1: Model the Search Process.

Revised: Build the CMP and the reward function R ( · ) based on states and actions defined over L O C A L. Formulate a sequential optimization.

Revised: Step 3.2: Solve the Optimization.

Revised: Utilize the proposed double convex Q-learning to find optimal actions.

Revised: Generate a search result of { Z k } K − 1 k =0 .

Revised: Step 4:

Revised: Estimate L O C

Revised: A L Parameters.

Revised: Train the searched L O C A L by minimizing the MSE via Adam.

Revised: Estimate values in { W k } K − 1 k =0 .

Revised: Step 5: Evaluate the Search and Estimation Results.

Revised: The results can formulate f t ( x ) for the t th episode.

Revised: Calculate the end-of-trajectory reward R t to evaluate f t ( x ) .

Revised: Output: L O C A L with the best performance and the corresponding equations.

Original: A.1 Training Algorithm for C O NS O L E .
Revised: A.2 Training Algorithm for C O NS O L E .

Original: The training algorithm can be seen in Algorithm 1.
Revised: The training algorithm can be seen in Algorithm 2.

Revised: A.3 Proofs of Theorem

Revised: Theorem.

Revised: ∀ 0 ≤ k ≤ K − 1 , the negative optimal Q-function − Q ∗ ( s k , ˜ a k ) in the proposed

Revised: C O NS O L E framework exists and is convex in s k and ˜ a k , where s k is the discrete state and ˜ a k is thecontinuous action at the k th stage.

Revised: Proof.

Original: Input: training dataset { x i , y i } Ni =1 .
Revised: First, we show our state transition satisfies the Markov property. Specifically, Equation (1)in our paper shows that the next state s k +1 equals the matrix multiplication between the currentstate s k and the matrix Z k that is a matricization of the current action a k , where k is the index ofthe state. Therefore, the state transition satisfies Markov property with the transition probability P ( s k +1 | s k , a k ) = 1 .

Revised: Due to the Markov property of the state transition, we define our search process as Controlled Markov

Revised: Process (CMP) [31, 30].

Revised: By the CMP definition [30], our CMP is composed of our state, action, statetransition probability, a discounter factor γ , and a start state (i.e., s 0 in Equation (1)).

Revised: In general,

Revised: CMP is a Markov Decision Process (MDP) without a reward function [31].

Revised: For one CMP, Trajectory Ordering (TO) ranks trajectories of state action pairs [31].

Revised: In our paper,we define the trajectory from ( s 0 , a 0 ) to ( s K − 1 , a K − 1 ) for K -layer L O C A L.

Revised: Then, our rewardfunction R ( · ) realizes a TO for our defined trajectories [31] since the ordering of trajectories can bedetermined by R ( · ) .

Revised: More specifically, R ( · ) is trained with the end-of-trajectory reward R t for thet th trajectory in our paper and can rank trajectories.

Revised: A reward bundle is an automation-like structureto produce rewards for a CMP [30].

Revised: By Corollary 2 of [30], there exists a reward bundle for ourdefined CMP and TO realized by R ( · ) .

Revised: We pair our CMP with the reward bundle to form a Split Partially Observable MDP (Split-POMDP)[30].

Revised: Then, by Proposition 1 and Corollary 1 in [30], our Split-POMDP will always have an optimaldeterministic policy that only depends on states in our CMP.

Revised: By the proof of Proposition 1 in [30],

Revised: Algorithm

Revised: Input: Training dataset { x i , y i } Ni =1 .

Original: Initialize:
Revised: Initialize:

Original: L O C
Revised: L O C

Original: A L layer number K , initial state s 

Revised: A L layer number K , initial state s

Original: =
Revised: =

Original: [ 1 , 0 ] T , discount factor γ ∈ (0 , 1) , ϵ for ϵ -greedy strategy, λ as a threshold to stop searching, ICNN for reward function − R ( s , a ) , ICNN for Q-function − Q ( s , a ) , replay buffer B = ∅ , maximum episode T , target network Q ′ ( · ) = Q ( · ) , and target network update interval T 0 .
Revised: [ 1 , 0 ] T , discount factor γ ∈ (0 , 1) , ϵ for ϵ -greedy strategy, λ as a threshold to stop searching, ICNN for reward function − R ( s , a ) , ICNN for Q-function − Q ( s , a ) , replay buffer B = ∅ , maximum episode T , target network Q ′ ( · ) = Q ( · ) , and target network update interval T 0 .

Original: while t ≤ T do while k ≤ K do
Revised: while t ≤ T do while k ≤ K do

Original: Solve Optimization in Equation (3) with − Q ( s tk , a ) to obtain ˜ a ∗ k .
Revised: Solve Optimization in Equation (3) with − Q ( s tk , a ) to obtain ˜ a ∗ k .

Original: Use ϵ -greedy to select ˜ a tk from ˜ a ∗ k and a random action.
Revised: Use ϵ -greedy to select ˜ a tk from ˜ a ∗ k and a random action.

Original: ▷ ϵ -greedy strategy.
Revised: ▷ ϵ -greedy strategy.

Original: Discretize ˜ a tk to obtain a tk .
Revised: Discretize ˜ a tk to obtain a tk .

Original: Execute a t k and use Equation (1) to obtain s k k +1 .
Revised: Execute a tk and use Equation (1) to obtain s kk +1 .

Original: Check if a tk and s kk +1 satisfy certain constraints.
Revised: Check if a tk and s kk +1 satisfy certain constraints.

Original: Otherwise, delete this state transition and restart the iteration from s tk .
Revised: Otherwise, delete this state transition and restart the iteration from s tk .

Original: ▷ Constraint checking.
Revised: ▷ Constraint checking.

Original: Formulate L O C
Revised: Formulate L O C

Original: A L, train L O C
Revised: A L, train L O C

Original: A L with { x i , y i } Ni =1 , and calculate R t .
Revised: A L with { x i , y i } Ni =1 , and calculate R t .

Original: Train the reward function − R ( · ) using training data {{ s tk , a tk } K − 1 k =0 , − R t } .
Revised: Train the reward function − R ( · ) using training data {{ s tk , a tk } K − 1 k =0 , − R t } .

Original: ∀ 0 ≤ k ≤ K , insert ( s tk , a tk , s tk +1 , R t ) and ( s tk , ˜ a tk , s tk +1 , R ( s tk , ˜ a tk )) to B 0 .
Revised: ∀ 0 ≤ k ≤ K , insert ( s tk , a tk , s tk +1 , R t ) and ( s tk , ˜ a t k , s tk +1 , R ( s tk , ˜ a t k )) to B 0 .

Original: Sample a minibatch B 0 ⊂ B for ( s m , a m , s m +1 , R m ) ∈ B do ▷ Experience replay.
Revised: Sample a minibatch B 0 ⊂ B for ( s m , a m , s m +1 , R m ) ∈ B do ▷ Experience replay.

Original: Solve Optimization in Equation (3) with − Q ′ ( s m +1 , a ) to obtain ˜ a m +1 .
Revised: Solve Optimization in Equation (3) with − Q ′ ( s m +1 , a ) to obtain ˜ a m +1 .

Original: y m
Revised: y m

Original: = R m + γQ ′
Revised: = R m + γQ ′

Original: ( s m , a m )
Revised: ( s m , a m )

Original: Train Q ( · ) using training data { s m +1 , a m +1 , y m } m if t mod T 0 = 

Revised: Train Q ( · ) using training data { s m +1 , a m +1 , y m } m if t mod T 0 =

Original: thenif | R t − 1 | ≤ λ then End the search process. Output: L O C A L with the best performance and the corresponding equations.
Revised: then

Original: Theorem. ∀ 0 ≤ k ≤ K − 1 , the negative optimal Q-function − Q∗ ( s k , ˜ a k ) in the proposed C O NS O L E framework exists and is convex in s k and ˜ a k , where s k is the discrete state and ˜ a k is thecontinuous action at the k th stage.
Revised: Q

Original: Proof.
Revised: [Equation]

Original: First, we show our state transition satisfies the Markov property. Specifically, Equation (1)in our paper shows that the next state s k +1 equals the matrix multiplication between the currentstate s k and the matrix Z k that is a matricization of the current action a k , where k is the index ofthe state. Therefore, the state transition satisfies Markov property with the transition probability
Revised: ▷ Update target Q-network.

Original: P ( s k +1 | s k , a k ) = 1 .
Revised: if | R t − 1 | ≤ λ then

Original: Due to the Markov property of the state transition, we define our search process as Controlled Markov

Original: Process (CMP) [32, 31]. By the CMP definition [31], our CMP is composed of our state, action, statetransition probability, a discounter factor γ , and a start state (i.e., s 0 in Equation (1)). In general, CMP is a Markov Decision Process (MDP) without a reward function [32].
Revised: End the search process.

Original: For one CMP, Trajectory Ordering (TO) ranks trajectories of state action pairs [32].

Original: In our paper,we define the trajectory from ( s 0 , a 0 ) to ( s K − 1 , a K − 1 ) for K -layer L O C A L. Then, our rewardfunction R ( · ) realizes a TO for our defined trajectories [32] since the ordering of trajectories can bedetermined by R ( · ) . More specifically, R ( · ) is trained with the end-of-trajectory reward R t for the t th trajectory in our paper and can rank trajectories. A reward bundle is an automation-like structureto produce rewards for a CMP [31].
Revised: Output: L O C A L with the best performance and the corresponding equations.

Original: By Corollary 2 of [31], there exists a reward bundle for ourdefined CMP and TO realized by R ( · ) . We pair our CMP with the reward bundle to form a Split Partially Observable MDP (Split-POMDP)[31]. Then, by Proposition 1 and Corollary 1 in [31], our Split-POMDP will always have an optimaldeterministic policy that only depends on states in our CMP. By theproof of Proposition 1 in [31],the optimal policy optimizes the value function over states in CMP.
Revised: the optimal policy optimizes the value function over states in CMP.

Original: Further, the value function isan evaluation of trajectories for our TO by the proof in Corollary 2 in [31].
Revised: Further, the value function isan evaluation of trajectories for our TO by the proof in Corollary 2 in [30].

Original: Additionally, our TO isrealized by our proposed reward function R ( · ) .
Revised: Additionally, our TO isrealized by our proposed reward function R ( · ) .

Original: Therefore, the optimal Q-function exists for our CMPand our proposed R ( · ) .
Revised: Therefore, the optimal Q-function exists for our CMPand our proposed R ( · ) .

Original: Then, we consider the Bellman Equation of Q ∗ ( · ) :
Revised: Then, we consider the Bellman Equation of Q ∗ ( · ) :

Original: [Equation]
Revised: [Equation]

Original: where the second equality holds since our state transitions are deterministic by Equation (1).
Revised: where the second equality holds since our state transitions are deterministic by Equation (1).

Original: Weprove the convexity from the induction method.
Revised: Weprove the convexity from the induction method.

Original: When k = K − 1 , the ( k + 1) th state is the terminalstate without action selections.
Revised: When k = K − 1 , the ( k + 1) th state is the terminalstate without action selections.

Original: Thus, we have
Revised: Thus, we have

Original: [Equation]
Revised: [Equation]

Original: Since − R ( · ) is an ICNN and is convex in input, − Q ∗ ( s K − 1 , ˜ a K − 1 ) is convex in s K − 1 and ˜ a K − 1 .
Revised: Since − R ( · ) is an ICNN and is convex in input, − Q ∗ ( s K − 1 , ˜ a K − 1 ) is convex in s K − 1 and ˜ a K − 1 .

Original: When 0 ≤ k < K − 1 and assume − Q ∗ ( s k +1 , ˜ a k +1 ) is convex in s k +1 and ˜ a k +1 , we have− max ˜ a Q ∗ ( s k +1 , ˜ a )
Revised: When 0 ≤ k < K − 1 and assume − Q ∗ ( s k +1 , ˜ a k +1 ) is convex in s k +1 and ˜ a k +1 , we have− max ˜ a Q ∗ ( s k +1 , ˜ a )

Original: =
Revised: =

Original: min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in s k +1 given the fixed optimal action.
Revised: min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in s k +1 given the fixed optimal action.

Original: Let
Revised: Let

Original: H denote the Hessian matrix of min ˜ a − Q ∗ ( s k +1 , ˜ a ) with respect to s k +1 .
Revised: H denote the Hessian matrix of min ˜ a − Q ∗ ( s k +1 , ˜ a ) with respect to s k +1 .

Original: Due to the convexity,
Revised: Due to the convexity,

Original: H is positive semi-definite.
Revised: H is positive semi-definite.

Original: Thus, by Equation (1) and the chain rule, the Hessian matrix ofmin ˜ a − Q ∗ ( s k +1 , ˜ a ) with respect to s k can be written as:
Revised: Thus, by Equation (1) and the chain rule, the Hessian matrix ofmin ˜ a − Q ∗ ( s k +1 , ˜ a ) with respect to s k can be written as:

Original: [Equation]
Revised: [Equation]

Original: H ′ is also positive semi-definite.
Revised: H ′ is also positive semi-definite.

Original: Therefore, min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in s k .
Revised: Therefore, min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in s k .

Original: Since − R ( s k , ˜ a k )is convex in s k , − Q ∗ ( s k , ˜ a k ) is convex in s k .
Revised: Since − R ( s k , ˜ a k )is convex in s k , − Q ∗ ( s k , ˜ a k ) is convex in s k .

Original: Similarly, vectorizing the state transition equation can give:
Revised: Similarly, vectorizing the state transition equation can give:

Original: [Equation]
Revised: [Equation]

Original: where I n s is the n s × n s identity matrix and (cid:78) is the Kronecker product.
Revised: where I n s is the n s × n s identity matrix and (cid:78) is the Kronecker product.

Original: a ′ k
Revised: a ′ k

Original: =
Revised: =

Original: [( a k ) T , 0 ] T is theconcatenation of the discrete action a k and a zero vector to maintain the fixed dimensionality ofaction vectors.
Revised: [( a k ) T , 0 ] T is theconcatenation of the discrete action a k and a zero vector to maintain the fixed dimensionality ofaction vectors.

Original: With similar proofs based on the Hessian matrix and the fact that − Q ∗ ( s k +1 , ˜ a k ) isconvex in s k +1 , we have min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in a ′ k and also a k .
Revised: With similar proofs based on the Hessian matrix and the fact that − Q ∗ ( s k +1 , ˜ a k ) isconvex in s k +1 , we have min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in a ′ k and also a k .

Original: Subsequently, arbitrary˜ a k ∈ conv ( { 0 , 1 } n a ) can be written as a convex combination of the discrete actions a k .
Revised: Subsequently, arbitrary˜ a k ∈ conv ( { 0 , 1 } n a ) can be written as a convex combination of the discrete actions a k .

Original: Thus,min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in ˜ a k .
Revised: Thus,min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in ˜ a k .

Original: Since − R ( s k , ˜ a k ) is convex in ˜ a k , − Q ∗ ( s k , ˜ a k ) is convex in˜ a k .
Revised: Since − R ( s k , ˜ a k ) is convex in ˜ a k , − Q ∗ ( s k , ˜ a k ) is convex in˜ a k .

Original: Eventually, − Q ∗ ( s k , ˜ a k ) is convex in s k and ˜ a k , which concludes the proof.
Revised: Eventually, − Q ∗ ( s k , ˜ a k ) is convex in s k and ˜ a k , which concludes the proof.

Original: ■A.3 Proofs of Theorem2
Revised: ■ A.4 Proofs of Theorem

Original: Theorem.
Revised: Theorem.

Original: Let f ∗ ( · ; W ) denote the L O C A L constructed by the optimal sequences of states( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) from − Q ∗ ( · ) , where W is the set of weightsof f ∗ ( · ; W ) .
Revised: Let f ∗ ( · ; W ) denote the L O C A L constructed by the optimal sequences of states( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) from − Q ∗ ( · ) , where W is the set of weightsof f ∗ ( · ; W ) .

Original: If f ∗ ( · ; W ) can be trained with noiseless datasets and the training can achieve theglobal optimal weights W ∗ , f ∗ ( · ; W ∗ ) can be simplified to the true equation g ( · ) .
Revised: If f ∗ ( · ; W ) can be trained with noiseless datasets and the training can achieve theglobal optimal weights W ∗ , f ∗ ( · ; W ∗ ) can be simplified to the true equation g ( · ) .

Original: Proof.
Revised: Proof.

Original: If f ∗ ( · ; W ) can’t represent the exact equations, there are two cases: (1) the structure off ∗ ( · ; W ) is correct to represent the equations, but the learned weights W ∗ don’t represent thesymbol coefficients, and (2) the structure of f ∗ ( · ; W ) can’t represent the equations.
Revised: If f ∗ ( · ; W ) can’t represent the exact equations, there are two cases: (1) the structure off ∗ ( · ; W ) is correct to represent the equations, but the learned weights W ∗ don’t represent thesymbol coefficients, and (2) the structure of f ∗ ( · ; W ) can’t represent the equations.

Original: Case (1) doesn’thold since we assume W ∗ is the global optimal weights for noiseless data.
Revised: Case (1) doesn’thold since we assume W ∗ is the global optimal weights for noiseless data.

Original: If case (2) holds,∃ 

Revised: If case (2) holds,∃

Original: ≤ j ≤ K − 1 , b ∗ j = min ˜ a j − Q ∗ ( s j , ˜ a j ) and b ∗ j doesn’t represent the symbol connections in theunderlying equations.
Revised: ≤ j ≤ K − 1 , b ∗ j = min ˜ a j − Q ∗ ( s j , ˜ a j ) and b ∗ j doesn’t represent the symbol connections in theunderlying equations.

Original: Further, we assume ∀ 0 ≤ i < j , a ∗ i =
Revised: Further, we assume ∀ 0 ≤ i < j , a ∗ i =

Original: min
Revised: min

Original: ˜ a i − Q ( s i , ˜ a i ) and a ∗ i representsthe true connections.
Revised: ˜ a i − Q ( s i , ˜ a i ) and a ∗ i representsthe true connections.

Original: If j = K − 1 , Equation (4) implies that ˜ a ∗ j = min ˜ a j − Q ∗ ( s j , ˜ a j ) = arg min ˜ a − R ( s j , ˜ a ) .
Revised: If j = K − 1 , Equation (4) implies that ˜ a ∗ j = min ˜ a j − Q ∗ ( s j , ˜ a j ) = arg min ˜ a − R ( s j , ˜ a ) .

Original: Since− R ( s j , ˜ a ) is convex in ˜ a
Revised: Since− R ( s j , ˜ a ) is convex in ˜ a

Original: we know the discrete version of ˜ a ∗ j , namely a ∗ j , represents the trueconnection of the last layer for the underlying equations.
Revised: we know the discrete version of ˜ a ∗ j , namely a ∗ j , represents the trueconnection of the last layer for the underlying equations.

Original: Otherwise, the reward is not maximized.
Revised: Otherwise, the reward is not maximized.

Original: However, by definition of b ∗ j , b ∗ j ̸ = a ∗ j .
Revised: However, by definition of b ∗ j , b ∗ j ̸ = a ∗ j .

Original: If j < K − 1 , Equation (4) implies:
Revised: If j < K − 1 , Equation (4) implies:

Original: [Equation]
Revised: [Equation]

Original: By definition of b ∗ j , b ∗ j is not the solution of Equation (5).
Revised: By definition of b ∗ j , b ∗ j is not the solution of Equation (5).

Original: This is because b ∗ j can’t achieve theminimum value for each summation term on the right hand side of Equation (5), according to theconvexity of the reward function.
Revised: This is because b ∗ j can’t achieve theminimum value for each summation term on the right hand side of Equation (5), according to theconvexity of the reward function.

Original: In general, b ∗ j ̸ = min ˜ a j − Q ∗ ( s j , ˜ a j ) , which contradicts thedefinition of b ∗ j .
Revised: In general, b ∗ j ̸ = min ˜ a j − Q ∗ ( s j , ˜ a j ) , which contradicts thedefinition of b ∗ j .

Original: Thus, b ∗ j doesn’t exist.
Revised: Thus, b ∗ j doesn’t exist.

Original: Therefore, case (2) doesn’t hold and f ∗ ( · ; W ∗ ) representsthe exact equations.
Revised: Therefore, case (2) doesn’t hold and f ∗ ( · ; W ∗ ) representsthe exact equations.

Original: ■
Revised: ■

Original: A.4 Proofs of Theorem 
Revised: A.5 Proofs of Theorem 3

Original: Theorem.
Revised: Theorem.

Original: Assume the following conditions hold: (1) the equation g ( x ) is C 2 smooth and hasbounded second derivatives with respect to weights, (2) ∃ x ∈ X , g ( x ) has non-zero gradients withrespect to weights, (3) the structure of L O C A L is correctly searched to exactly represent symbolsand symbol connections in g ( x ) , and (4) the training dataset of L O C
Revised: Assume the following conditions hold: (1) the equation g ( x ) is C 2 smooth and hasbounded second derivatives with respect to weights, (2) ∃ x ∈ X , g ( x ) has non-zero gradients withrespect to weights, (3) the structure of L O C A L is correctly searched to exactly represent symbolsand symbol connections in g ( x ) , and (4) the training dataset of L O C

Original: A L is noiseless.
Revised: A L is noiseless.

Original: Then, for the
Revised: Then, for the

Original: MSE loss surface of L O C A L , each global optimal point has a strictly convex local region.
Revised: MSE loss surface of L O C A L , each global optimal point has a strictly convex local region.

Original: Proof.
Revised: Proof.

Original: To simplify the proof, we consider scalar output of the L O C
Revised: To simplify the proof, we consider scalar output of the L O C

Original: A L, i.e., one equation, and theproof can be easily extended to the multi-output case.
Revised: A L, i.e., one equation, and theproof can be easily extended to the multi-output case.

Original: We follow the idea of [29] to study the secondderivative of L O C A L with perturbations.
Revised: We follow the idea of [29] to study the secondderivative of L O C A L with perturbations.

Original: Let ˆ y ( x , W ) denote the L O C A L with input to be x and theweight set to be W .
Revised: Let ˆ y ( x , W ) denote the L O C A L with input to be x and theweight set to be W .

Original: Let X be a perturbation direction of W and t be a small step size.
Revised: Let X be a perturbation direction of W and t be a small step size.

Original: For the i thnoiseless instance ( x i , y i ) , we denote e ( x i , W + tX )
Revised: For the i thnoiseless instance ( x i , y i ) , we denote e ( x i , W + tX )

Original: =
Revised: =

Original: ˆ y ( x i , W + tX ) − y i .
Revised: ˆ y ( x i , W + tX ) − y i .

Original: Obviously, the lossfunction can be written as L ( W + tX )
Revised: Obviously, the lossfunction can be written as L ( W + tX )

Original: = 12 N (cid:80) Ni
Revised: = 12 N (cid:80) Ni

Original: =1 ( e ( x i , W + tX )) 2 .
Revised: =1 ( e ( x i , W + tX )) 2 .

Original: Then, we can calculate thesecond-order derivative based on the chain rule:
Revised: Then, we can calculate the second-order derivative based on the chain rule:

Original: [Equation]
Revised: [Equation]

Original: Next, we denote the global optimal solution to be W ∗ .
Revised: Next, we denote the global optimal solution to be W ∗ .

Original: Based on the Assumptions (3)and (4) , ∀ i, ˆ y ( x i , W ∗ )
Revised: Based on the Assumptions (3)and (4) , ∀ i, ˆ y ( x i , W ∗ )

Original: =
Revised: =

Original: g ( x i ) =
Revised: g ( x i ) =

Original: y i .
Revised: y i .

Original: Therefore, we have d 2 dt 2 (cid:12)(cid:12) t =

Revised: Therefore, we have d 2 dt 2 (cid:12)(cid:12) t =

Original: L ( W ∗ + tX ) =
Revised: L ( W ∗ + tX ) =

Original:  N  (cid:0)  t {+=
+}
Revised: 1 N (cid:80) Ni =1 (cid:0) ddt (cid:12)(cid:12) t {+=
+}

Original: ˆ y ( x i , W ∗ ) (cid:1) 2 > 0 , where the inequality strictly holds.
Revised: ˆ y ( x i , W ∗ ) (cid:1) 2 > 0 , where the inequality strictly holds.

Original: This is because by Assumptions (3) , ˆ y ( x , W ∗ ) can be mathematically simplified to obtain g ( x ) .
Revised: This is because by Assumptions (3) , ˆ y ( x , W ∗ ) can be mathematically simplified to obtain g ( x ) .

Original: Then, by Assumption (2) , (cid:0)  (cid:12)(cid:12) t
Revised: Then, by Assumption (2) ,1 N (cid:80) Ni =1 (cid:0) ddt (cid:12)(cid:12) t

Original: =

Revised: =

Original: ˆ y
Revised: ˆ y

Original: ( x i , W ∗ ) (cid:1) 2 > 0 .
Revised: ( x i , W ∗ ) (cid:1) 2 > 0 .

Original: Finally, by Assumption (1) and (3) , d 2 dt 2 (cid:12)(cid:12) t =

Revised: Finally, by Assumption (1) and (3) , d 2 dt 2 (cid:12)(cid:12) t =

Original: ˆ y
Revised: ˆ y

Original: ( x i , W + tX ) isbounded and there is a local region around W ∗ such that d 2 dt 2 (cid:12)(cid:12) t
Revised: ( x i , W + tX ) isbounded and there is a local region around W ∗ such that d 2 dt 2 (cid:12)(cid:12) t

Original: =

Revised: =

Original: L ( W + tX ) > 0 , which concludesthe proof.
Revised: L ( W + tX ) > 0 , which concludesthe proof.

Original: ■
Revised: ■

Original: A.5 Proofs of Theorem 
Revised: A.6 Proofs of Theorem 4

Original: Theorem.
Revised: Theorem.

Original: Suppose Assumptions 1-4 in Theorem 3 hold.
Revised: Suppose Assumptions 1-4 in Theorem 3 hold.

Original: For a L O C A L with one symbolic activation,multiplication, and summation layer, the set of local convex regions with global optima is U =
Revised: For a L O C A L with one symbolic activation,multiplication, and summation layer, the set of local convex regions with global optima is U =

Original: { W (cid:12)(cid:12)(cid:12) (cid:12)(cid:12) ddt (cid:12)(cid:12) t

Original: [-=
 ˆ y-]
Revised: [-=
 ˆ y-][Equation]

Original: [-( x i ,W + tX ) (cid:12)(cid:12)η (cid:12)(cid:12) ddt (cid:12)(cid:12) t =
 ˆ y ( x j ,W + tX ) (cid:12)(cid:12) > | ˆ y ( x k , W ) − y k-] |} , where notations are defined in the proof.
Revised: [-( x i ,W + tX ) (cid:12)(cid:12)η (cid:12)(cid:12) ddt (cid:12)(cid:12) t =
 ˆ y ( x j ,W + tX ) (cid:12)(cid:12) > | ˆ y ( x k , W ) − y k-] |} , where notations are defined in the proof.

Original: Proof.
Revised: Proof.

Original: For the target L O C A L, we similarly consider the scalar output and write the function analytically:
Revised: For the target L O C A L, we similarly consider the scalar output and write the function analytically:

Original: [Equation]
Revised: [Equation]

Original: where W 0 ∈ R n 0 × n 1 is the weight matrix for activation, Φ : R n 1 → R n 1 represents the activationwith symbol functions like x 2 , cos ( x ) , and log( x ) , etc.
Revised: where W 0 ∈ R n 0 × n 1 is the weight matrix for activation, Φ : R n 1 → R n 1 represents the activationwith symbol functions like x 2 , cos ( x ) , and log( x ) , etc.

Original: Ψ : R n 1 → R n 2 is the function to selectsome activated neurons for multiplications, and W 1 ∈ R n 2 × n 3 ( n 3 = 1 ) represents the weight forsummation.
Revised: Ψ : R n 1 → R n 2 is the function to selectsome activated neurons for multiplications, and W 1 ∈ R n 2 × n 3 ( n 3 = 1 ) represents the weight forsummation.

Original: We rewrite Equation (7) with the help of exponential and logarithm mappings.
Revised: We rewrite Equation (7) with the help of exponential and logarithm mappings.

Original: [Equation]
Revised: [Equation]

Original: where S ∈ R n 1 × n 2 represents a selection matrix such that S [ i, j ] = 1 if and only if the i th neuronis selected as the multiplicative factor for the j th neuron in the multiplication layer.
Revised: where S ∈ R n 1 × n 2 represents a selection matrix such that S [ i, j ] = 1 if and only if the i th neuronis selected as the multiplicative factor for the j th neuron in the multiplication layer.

Original: Given thefixed structure of ˆ y ( · ) from the deep Q-learning, S is a known matrix.
Revised: Given thefixed structure of ˆ y ( · ) from the deep Q-learning, S is a known matrix.

Original: log( · ) and exp( · ) representthe element-wise logarithm and exponential functions.
Revised: log( · ) and exp( · ) representthe element-wise logarithm and exponential functions.

Original: Notably, the corresponding element inΦ( W T 0 x ) should be positive in Equation (8).
Revised: Notably, the corresponding element inΦ( W T 0 x ) should be positive in Equation (8).

Original: If there are negative entries, one can utilize W T 1 s ◦exp (cid:18)
Revised: If there are negative entries, one can utilize W T 1 s ◦exp (cid:18)

Original: S T log (cid:16) | Φ( W T 0 x ) | (cid:17) (cid:19) to take place of the right hand side term in Equation (8), where s [ i ] =( − 1) n i − and 0 ≤ n i − ≤ n 1 represents the number of negative entries selected for the i th neuron ofthe multiplication layer.
Revised: S T log (cid:16) | Φ( W T 0 x ) | (cid:17)(cid:19) to take place of the right hand side term in Equation (8), where s [ i ] =( − 1) n i − and 0 ≤ n i − ≤ n 1 represents the number of negative entries selected for the i th neuron ofthe multiplication layer.

Original: ◦ represents the Hadamard product.
Revised: ◦ represents the Hadamard product.

Original: However, both expressions have thesame values and gradients.
Revised: However, both expressions have thesame values and gradients.

Original: Thus, we utilize Equation (8) in later derivations.
Revised: Thus, we utilize Equation (8) in later derivations.

Original: Then, let X be a perturbation direction such that X
Revised: Then, let X be a perturbation direction such that X

Original: = { X 0 , X 1 } .
Revised: = { X 0 , X 1 } .

Original: Thus, for a small step t , we have:
Revised: Thus, for a small step t , we have:

Original: [Equation]
Revised: [Equation]

Original: Based on Equation (9), we can compute:
Revised: Based on Equation (9), we can compute:

Original: [Equation]
Revised: [Equation]

Original: where 1 Φ(( W 0 + t X 0 )
Revised: where 1 Φ(( W 0 + t X 0 )

Original: T x i ) ∈ R n 1 is the element-wise division and Φ ′ is the element-wise first derivativeof Φ ′ .
Revised: T x i ) ∈ R n 1 is the element-wise division and Φ ′ is the element-wise first derivativeof Φ ′ .

Original: Without special notifications, we assume all the division for vectors is element-wise in thefollowing derivations.
Revised: Without special notifications, we assume all the division for vectors is element-wise in thefollowing derivations.

Original: Then, we denote
Revised: Then, we denote

Original: [Equation]
Revised: [Equation]

Original: With above definitions, we can calculate:
Revised: With above definitions [Equation] we can calculate:

Original: [Equation]
Revised: [Equation]

Original: Further, we calculate the second derivative based on Equation (10) and the fact that element-wiseoperations for vectors are commutative:
Revised: Further, we calculate the second derivative based on Equation (10) and the fact that element-wiseoperations for vectors are commutative:

Original: [Equation]
Revised: [Equation]

Original: When t → [Equation] we have:
Revised: When t → 0 , we have:

Original: [Equation]
Revised: [Equation]

Original: The above equation can reflect the relationship between the second and the first derivative.
Revised: The above equation can reflect the relationship between the second and the first derivative.

Original: However,we first identify the inequality between these two derivatives to enable a strictly convex region.
Revised: However,we first identify the inequality between these two derivatives to enable a strictly convex region.

Original: Let ˆ
Revised: Let ˆ

Original: [Equation]
Revised: [Equation]

Revised: y ( x 1 , W +tX ) , · · · , ddt 2 (cid:12)(cid:12) t

Original: {+=
+}
Revised: {+=
+}

Original:  N , W + tX )]
Revised: ˆ y ( x N , W + tX )]

Original: T , and e =
Revised: T , and e =

Original: [ e ( x 1 , W ) , · · · , e ( x N , W )] T .
Revised: [ e ( x 1 , W ) , · · · , e ( x N , W )] T .

Original: Equation (6) impliesthat:
Revised: Equation (6) impliesthat:

Original: [Equation]
Revised: [Equation]

Original: To find a region to restrict the convexity, we restrict the lower bound of the second derivative to bepositive and compute:
Revised: To find a region to restrict the convexity, we restrict the lower bound of the second derivative to bepositive and compute:

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: The right hand side of Equation (16) can be easily bounded by:
Revised: The right hand side of Equation (16) can be easily bounded by:

Original: [Equation]
Revised: [Equation]

Original: where |·| for a vector is to calculate the absolute value for each element of the vector, i = arg min( | ˆ y ′ | )and j = arg max( | ˆ y ′′ | ) .
Revised: where |·| for a vector is to calculate the absolute value for each element of the vector [Equation] i = arg min( | ˆ y ′ | )and j = arg max( | ˆ y ′′ | ) .

Original: Namely, we consider a sufficient condition for convexity.
Revised: Namely, we consider a sufficient condition for convexity.

Original: [Equation]
Revised: [Equation]

Original: Next, Equation (14) indicates that:
Revised: Next, Equation (14) indicates that:

Original: [Equation]
Revised: [Equation]

Original: where η is a positive constant.
Revised: where η is a positive constant.

Original: Note that η < ∞ by Assumptions (1) and (2) in Theorem 3.
Revised: Note that η < ∞ by Assumptions (1) and (2) in Theorem 3.

Original: Therefore,we have the following sufficient condition to make d 2 dt 2 (cid:12)(cid:12) t =

Revised: Therefore,we have the following sufficient condition to make d 2 dt 2 (cid:12)(cid:12) t =

Original: L ( W + tX ) > 0 always hold.
Revised: L ( W + tX ) > 0 always hold.

Original: [Equation]
Revised: [Equation]

Original: where k = arg max( | e | )
Revised: where k = arg max( | e | )

Original: The above equation leads to a set U of local regions that have strongconvexity.
Revised: The above equation leads to a set U of local regions that have strongconvexity.

Original: Namely,
Revised: Namely,

Original: [Equation]
Revised: [Equation]

Original: Clearly, the global optimal solution W ∗ ∈ U since ˆ y ( x k , W ∗ ) − y k = 0 .
Revised: Clearly, the global optimal solution W ∗ ∈ U since ˆ y ( x k , W ∗ ) − y k = 0 .

Original: Note that there may bemultiple global optimal solutions of the loss minimization in L O C A
Revised: Note that there may bemultiple global optimal solutions of the loss minimization in L O C A

Original: L. Thus, U is the set of localconvex regions that contain global optima.
Revised: L. Thus, U is the set of localconvex regions that contain global optima.

Original: This implies that for each W ∗ ∈
Revised: This implies that for each W ∗ ∈

Original: U , we can find a locallyand strictly convex region U ∗ =
Revised: U , we can find a locallyand strictly convex region U ∗ =

Original: U ∩ B ( r ) , where B ( r ) =
Revised: U ∩ B ( r ) , where B ( r ) =

Original: ||
Revised: ||

Original: w − w ∗
Revised: w − w ∗

Original: ||
Revised: ||

Original: 2 ≤ r is a norm ball and wevectorize W and W ∗ to obtain w and w ∗ , respectively.
Revised: 2 ≤ r is a norm ball and wevectorize W and W ∗ to obtain w and w ∗ , respectively.

Original: Subsequently, range r can be set relativelylarge such that U ∗ ⊂ B ( r ) and U ∗∗ ∩ B ( r ) = ∅ , where U ∗∗ is the local region for another globaloptimal point W ∗∗ if it exists.
Revised: Subsequently, range r can be set relativelylarge such that U ∗ ⊂ B ( r ) and U ∗∗ ∩ B ( r ) = ∅ , where U ∗∗ is the local region for another globaloptimal point W ∗∗ if it exists.

Original: Then, the range for U ∗ still depends on the inequality in Equation(21).
Revised: Then, the range for U ∗ still depends on the inequality in Equation(21).

Original: ■
Revised: ■