Original: C O NS O L E : Convex Neural Symbolic Learning
Revised: C O NS O L E : Convex Neural Symbolic Learning

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Learning the underlying equation from data is a fundamental problem in manydisciplines.
Revised: Learning the underlying equation from data is a fundamental problem in manydisciplines.

Original: Recent advances rely on Neural Networks (NNs) but do not providetheoretical guarantees in obtaining the exact equations owing to the non-convexityof NNs.
Revised: Recent advances rely on Neural Networks (NNs) but do not providetheoretical guarantees in obtaining the exact equations owing to the non-convexityof NNs.

Original: In this paper, we propose Convex Neural Symbolic Learning (C O NS O L E )to seek convexity under mild conditions.
Revised: In this paper, we propose Convex Neural Symbolic Learning (C O NS O L E )to seek convexity under mild conditions.

Original: The main idea is to decompose therecovering process into two steps and convexify each step.
Revised: The main idea is to decompose therecovering process into two steps and convexify each step.

Original: In the first step ofsearching for right symbols, we convexify the deep Q-learning.
Revised: In the first step ofsearching for right symbols, we convexify the deep Q-learning.

Original: The key is tomaintain double convexity for both the negative Q-function and the negative rewardfunction in each iteration, leading to provable convexity of the negative optimal Qfunction to learn the true symbol connections.
Revised: The key is tomaintain double convexity for both the negative Q-function and the negative rewardfunction in each iteration, leading to provable convexity of the negative optimal Qfunction to learn the true symbol connections.

Original: Conditioned on the exact searchingresult, we construct a Locally-Convex equation Learner (
Revised: Conditioned on the exact searchingresult, we construct a Locally-Convex equation Learner (

Original: L O C
Revised: L O C

Original: A L) neural networkto convexify the estimation of symbol coefficients.
Revised: A L) neural networkto convexify the estimation of symbol coefficients.

Original: With such a design, we quantifya large region with strict convexity in the loss surface of L O C A L for commonlyused physical functions.
Revised: With such a design, we quantifya large region with strict convexity in the loss surface of L O C A L for commonlyused physical functions.

Original: Finally, we demonstrate the superior performance of the
Revised: Finally, we demonstrate the superior performance of the

Original: C O NS O L E framework over the state-of-the-art on a diverse set of datasets.
Revised: C O NS O L E framework over the state-of-the-art on a diverse set of datasets.

Original:  Introduction
Revised:  Introduction

Original: Identifying the underlying mathematical expressions from data plays a key role in multiple domains.
Revised: Identifying the underlying mathematical expressions from data plays a key role in multiple domains.

Original: For example, scientific discovery naturally requires learning analytical solutions to fit data.
Revised: For example, scientific discovery naturally requires learning analytical solutions to fit data.

Original: Engineering systems also need to frequently re-estimate system equations due to events, maintenance,upgrading and new constructions [1] etc.
Revised: Engineering systems also need to frequently re-estimate system equations due to events, maintenance,upgrading and new constructions [1] etc.

Original: In general, the problem, known as Symbolic Regression (SR) [2], learns the underlying equations y
Revised: In general, the problem, known as Symbolic Regression (SR) [2], learns the underlying equations y

Original: = g ( x ) constructed via certain symbols, where xand y represent the input/output vector of the equations.
Revised: = g ( x ) constructed via certain symbols, where xand y represent the input/output vector of the equations.

Original: If successfully learned, the equation canenjoy many important properties like high interpretability and generalizability which will in turnsignificantly benefit scientific understanding and engineering planning, monitoring and control.
Revised: If successfully learned, the equation canenjoy many important properties like high interpretability and generalizability, which will in turnsignificantly benefit scientific understanding and engineering planning, monitoring and control.

Original: Promising as it might be, SR is NP-hard [3, 4].
Revised: Promising as it might be, SR is NP-hard [3, 4].

Original: Mathematically one can cast SR as an optimizationproblem over both discrete variables to select symbols and continuous variables to represent thesymbol coefficients.
Revised: Mathematically, one can cast SR as an optimizationproblem over both discrete variables to select symbols and continuous variables to represent thesymbol coefficients.

Original: Traditional solutions employ evolutionary algorithms like Genetic Programming(GP)
Revised: Traditional solutions employ evolutionary algorithms like Genetic Programming(GP)

Original: These methods start from an initial set of expressions and continue evolving via operationslike crossover or mutation.
Revised: These methods start from an initial set of expressions and continue evolving via operationslike crossover or mutation.

Original: With fitness measures, GP-based algorithms can evaluate and select thebest equations.
Revised: With fitness measures, GP-based algorithms can evaluate and select thebest equations.

Original: However, these methods have poor scalability and limited theoretical guarantees [3].
Revised: However, these methods have poor scalability and limited theoretical guarantees [3].

Original: More recent SR studies leverage Neural Networks (NNs) with high representational power.
Revised: More recent SR studies leverage Neural Networks (NNs) with high representational power.

Original: For the
Revised: For the

Original: NN-based SR, we mainly categorize them into two groups based on the roles of the NNs.
Revised: NN-based SR, we mainly categorize them into two groups based on the roles of the NNs.

Original: The firstgroup employs NNs to directly model the equations, where sparsity of the NN weights is enforced toselect symbols [6–10].
Revised: The firstgroup employs NNs to directly model the equations, where sparsity of the NN weights is enforced toselect symbols [6–10].

Original: Thus, the problem is transformed into training the designed NN with sparsityregularization.
Revised: Thus, the problem is transformed into training the designed NN with sparsityregularization.

Original: However, due to the non-convexity of NNs, the weight selection and updating can beeasily stuck in local optima, failing to find the exact equations.
Revised: However, due to the non-convexity of NNs, the weight selection and updating can beeasily stuck in local optima, failing to find the exact equations.

Original: The second group employs NNs to search the symbol connections, and non-linear optimizations like
Revised: The second group employs NNs to search the symbol connections, and non-linear optimizations like

Original: BFGS [11] can be employed to estimate symbol coefficients.
Revised: BFGS [11] can be employed to estimate symbol coefficients.

Original: For the searching procedure, [3, 12–14]leverage a Recursive Neural Network (RNN) as a policy network to iteratively generate optimalactions that can select and connect symbols.
Revised: For the searching procedure, [3, 12–14] leverage a Recursive Neural Network (RNN) as a policy network to iteratively generate optimalactions that can select and connect symbols.

Original: [15] employs large-scale pre-training to directly mapfrom data to the symbolic equations.
Revised: [15] employs large-scale pre-training to directly mapfrom data to the symbolic equations.

Original: While these methods restrict the utilization of NNs in thesearch phase, the non-convexity of NNs can still suffer the risk of sub-optimal decisions to formulateequations.
Revised: While these methods restrict the utilization of NNs in thesearch phase, the non-convexity of NNs can still suffer the risk of sub-optimal decisions to formulateequations.

Original: To mitigate this issue, some efforts have been made such as risk-seeking policy gradient tofind the best equations [3], restricting the searching space via domain knowledge [16] and entropyregularization [13], and re-initialization [13], etc.
Revised: To mitigate this issue, some efforts have been made such as risk-seeking policy gradient tofind the best equations [3], restricting the searching space via domain knowledge [16] and entropyregularization [13], and re-initialization [13], etc.

Original: However, they have limited theoretical guarantees.
Revised: However, they have limited theoretical guarantees.

Original: In this paper, we propose Convex Neural Symbolic Learning (C O NS O L E ) with convexity undermoderate conditions.
Revised: In this paper, we propose Convex Neural Symbolic Learning (C O NS O L E ) with convexity undermoderate conditions.

Original: To our best knowledge, we are the first to provide provable guarantees to learnthe exact equations.
Revised: To our best knowledge, we are the first to provide provable guarantees to learnthe exact equations.

Original: In general, we decompose the SR problem into two sub-problems and seekconvexity, respectively.
Revised: In general, we decompose the SR problem into two sub-problems and seekconvexity, respectively.

Original: In the first problem of searching symbols, we propose a double-convexifieddeep Q-learning to maintain the convexity of negative Q-function and negative reward functionswith continuous action variables.
Revised: In the first problem of searching symbols, we propose a double-convexifieddeep Q-learning to maintain the convexity of negative Q-function and negative reward functionswith continuous action variables.

Original: Specifically, we use the Input Convex Neural Network (ICNN)[17] to represent both the negative Q-function and the negative reward function in each iteration.
Revised: Specifically, we use the Input Convex Neural Network (ICNN)[17] to represent both the negative Q-function and the negative reward function in each iteration.

Original: Subsequently, we prove that such a design (1) guarantees an optimal action selection at each stepand (2) ensures the negative optimal Q-function, if successfully found, to be convex.
Revised: Subsequently, we prove that such a design (1) guarantees an optimal action selection at each stepand (2) ensures the negative optimal Q-function, if successfully found, to be convex.

Original: Therefore, theconvex negative optimal Q-function can yield global optimal decisions of equation constructions.
Revised: Therefore, theconvex negative optimal Q-function can yield global optimal decisions of equation constructions.

Original: In the second problem of coefficient estimation, we use the search result to build a Locally-Convex
Revised: In the second problem of coefficient estimation, we use the search result to build a Locally-Convex

Original: Equation Learner (L O C A L) neural network.
Revised: Equation Learner (L O C A L) neural network.

Original: The key insight is that if the search result is correct,the loss surface of L O C A L has local regions that contain the global optima and has strict convexity.
Revised: The key insight is that if the search result is correct,the loss surface of L O C A L has local regions that contain the global optima and has strict convexity.

Original: Moreover, we quantify the local regions and show the range of the region is large under mildassumptions.
Revised: Moreover, we quantify the local regions and show the range of the region is large under mildassumptions.

Original: Therefore, initializations based on prior knowledge can often lie in the convex region,bringing the accurate coefficient estimation.
Revised: Therefore, initializations based on prior knowledge can often lie in the convex region,bringing the accurate coefficient estimation.

Original: Finally, we demonstrate that our C O NS O L E outperformsstate-of-the-art methods on a diverse set of datasets.
Revised: Finally, we demonstrate that our C O NS O L E outperformsstate-of-the-art methods on a diverse set of datasets.

Original:  Related Work
Revised:  Related Work

Original: Symbolic regression using neural networks.
Revised: Symbolic regression using neural networks.

Original: As discussed in Section 1, the high approximationcapacity of NNs can facilitate SR in both the symbol search [3, 12–15] and the expression representation [6–10].

Original: In addition, there are studies toindirectly help SR  with NNs.
Revised: In addition to the direct use of NN to help SR asdiscussed in the Introduction, there are studies with the indirect use.

Original: They typically treat NNs asa data augmentation tool to create high-quality data for learning or evaluation [18, 2].
Revised: They typically treat NNs as adata augmentation tool to create high-quality data for learning or evaluation [18, 2].

Original: Neural architecture search.
Revised: Neural architecture search.

Original: Searching the connections of L O C A L falls into the area of Neural
Revised: Searching the connections of L O C A L falls into the area of Neural

Original: Architecture Search (NAS).
Revised: Architecture Search (NAS).

Original: NAS tries to find an optimal architecture of a target NN with the bestperformance [19].
Revised: NAS tries to find an optimal architecture of a target NN with the bestperformance [19].

Original: The searching algorithms can be divided into RL-based, evolutionary algorithmbased, sequential optimization-based, and gradient descent-based methods.
Revised: The searching algorithms can be divided into RL-based, evolutionary algorithmbased, sequential optimization-based, and gradient descent-based methods.

Original: For RL-based methods,[20] employs an RNN model to sample the architecture and utilizes the accuracy of the samplednetwork as the reward.
Revised: For RL-based methods,[20] employs an RNN model to sample the architecture and utilizes the accuracy of the samplednetwork as the reward.

Original: [21] uses  Q-learning to find the connections of a target NN.
Revised: [21] uses tabular Q-learning to find the connections of a target NN.

Original: However, Qlearning can hardly be applicable when the state and action spaces are large, e.g., SR problem.
Revised: However,tabular Q-learning can hardly be applicable when the state and action spaces are large, e.g., SRproblem.

Original: Theevolutionary algorithm employs methods such as GP [22] and tournament selection [23].
Revised: The evolutionary algorithm employs methods such as GP [22] and tournament selection [23].

Original: However,these methods may lack the scalability for SR problem [3].
Revised: However, these methods may lack the scalability for SR problem [3].

Original: The sequential optimization-based methodis more scalable as the model complexity increases in a sequential manner [24].
Revised: The sequential optimizationbased method is more scalable as the model complexity increases in a sequential manner [24].

Original: Finally, the gradientdescent-based method [25] builds a large and over-parameterized network to search and train.
Revised: Finally,the gradient descent-based method [25] builds a large and over-parameterized network to search andtrain.

Original: Then,regularizations like dropout are added to find the best connections.
Revised: Then, regularizations like dropout are added to find the best connections.

Original: However, for these methods, thetheoretical guarantees remain opaque for SR problem.
Revised: However, for thesemethods, the theoretical guarantees remain opaque for SR problem.

Original: Global optimum in neural networks.
Revised: Global optimum in neural networks.

Original: Many studies have been conducted to seek global optimalityin NNs, and they can be categorized into finding global optima for weights or input variables.
Revised: Many studies have been conducted to seek global optimalityin NNs, and they can be categorized into finding global optima for weights or input variables.

Original: Theweight optimization is directly linked to finding symbol coefficients in L O C A L. Specifically, [26]establishes a strong result that every local optimum is a global optimum for deep non-linear networksunder several assumptions.
Revised: Theweight optimization is directly linked to finding symbol coefficients in L O C A L. Specifically, [26]investigates a single hidden layer with unbounded neurons and non-Euclidean regularization. Theauthors show the training can be done via convex optimization problems. [27] considers finiteneurons and develops a novel duality theory to train two-layer NNs as convex optimization problems.[28] establishes a strong result that every local optimum is a global optimum for deep non-linearnetworks under several assumptions.

Original: [27] eliminates these assumptions and finds that with weight decayregularization (e.g., l 2 norm), the loss function of NN with ReLU activations is piece-wise stronglyconvex in local regions.
Revised: [29] eliminates these assumptions and finds that with weightdecay regularization (e.g., l 2 norm), the loss function of NN with ReLU activations is piece-wisestrongly convex in local regions.

Original: However, L O C A L doesn’t fit the above conditions.
Revised: However, L O C A L doesn’t fit the above conditions.

Original: The second groupof input variable optimization can help search for optimal inputs.
Revised: The secondgroup of input variable optimization can help search for optimal inputs.

Original: [17] proposes ICNN such thatthe output of ICNN is convex in input variables.
Revised: [17] proposes ICNN suchthat the output of ICNN is convex in input variables.

Original: The key of ICNN is to restrict some weights andactivation functions to preserve convexity.
Revised: The key of ICNN is to restrict some weights andactivation functions to preserve convexity.

Original: ICNN facilitates to design a convexified search algorithm.
Revised: ICNN facilitates to design a convexified search algorithm.

Original:  Convex Neural Symbolic Learning
Revised:  Convex Neural Symbolic Learning

Original: L O C A L to Hierarchically Represent the Equations and Learn Symbol Coefficients
Revised: L O C A L to Hierarchically Represent the Equations and Learn Symbol Coefficients

Original: We utilize a neural network to represent the multi-input multi-output equations due to the efficiency[6].
Revised: We utilize a neural network to represent the multi-input multi-output equations due to the efficiency[6].

Original: We prove in Theorem 3 that under certain conditions, there are local regions in the loss surfacewith strict convexity.
Revised: We prove in Theorem 3 that under certain conditions, there are local regions in the loss surfacewith strict convexity.

Original: Thus, we name our NN as Locally Convex Equation Learner (L O C
Revised: Thus, we name our NN as Locally Convex Equation Learner (L O C

Original: A L).
Revised: A L).

Original: We assume the underlying equation y
Revised: We assume the underlying equation y

Original: = g ( x ) follows compositionality and smoothness assumptionsin [2], which are often the case in physics and many other scientific applications.
Revised: = g ( x ) follows compositionality and smoothness assumptionsin [2], which are often the case in physics and many other scientific applications.

Original: Then, we can build
Revised: Then, we can build

Original: L O C A L that can be trained via input/output pairs { x i , y i } Ni
Revised: L O C A L that can be trained via input/output pairs { x i , y i } Ni

Original: =1 , where x i ∈ X and
Revised: =1 , where x i ∈ X and

Original: y i ∈ Y are the i thsamples and N is the number of data.
Revised: y i ∈ Y are the i thsamples and N is the number of data.

Original: X and Y are the input and output data spaces, respectively.
Revised: X and Y are the input and output data spaces, respectively.

Original: Bythe compositionality, we propose to hierarchically map x to y with correct symbols.
Revised: Bythe compositionality, we propose to hierarchically map x to y with correct symbols.

Original: Fig.
Revised: Fig.

Original: 1 showsan example of the L O C A L structure.
Revised: 1 showsan example of the L O C A L structure.

Original: Specifically, each input entry first goes through the activationfunctions from a symbol library Φ .
Revised: Specifically, each input entry first goes through the activationfunctions from a symbol library Φ .

Original: For example, in Fig. 1, we denote Φ =
Revised: For example, in Fig. 1, we denote Φ =

Original: { x, x 2 , cos( wx ) } tocorrespond with three neurons from top to bottom in the dotted black box, where w is a learnableweight.
Revised: { x, x 2 , cos( wx ) } tocorrespond with three neurons from top to bottom in the dotted black box, where w is a learnableweight.

Original: As for weights of x and x 2 , they only need to appear in the later summation layer.
Revised: As for weights of x and x 2 , they only need to appear in the later summation layer.

Original: Then, someactivation outputs will be selected as multipliers for the multiplication.
Revised: Then, someactivation outputs will be selected as multipliers for the multiplication.

Original: Subsequently, the multipliedoutputs are selected and summed together.
Revised: Subsequently, the multipliedoutputs are selected and summed together.

Original: The repetition of symbolic activation, multiplication, andsummation formulates final equations.
Revised: The repetition of symbolic activation, multiplication, andsummation formulates final equations.

Original: For instance, the L O C A L structure in Fig.
Revised: For instance, the L O C A L structure in Fig.

Original: 1 can correctlyrepresents y 1 and y 2 shown on the top right of Fig.
Revised: 1 can correctlyrepresents y 1 and y 2 shown on the top right of Fig.

Original: Mathematically, we denote Z k ∈ { 0 , 1 } n k × n k +1 and W k ∈ R n k × n k +1 as the indicator and weightmatrix between the k th and the ( k + 1) th layer of the L O C A L, respectively.
Revised: Mathematically, we denote Z k ∈ { 0 , 1 } n k × n k +1 and W k ∈ R n k × n k +1 as the indicator and weightmatrix between the k th and the ( k + 1) th layer of the L O C A L, respectively.

Original: n k is the number ofneurons for the k th layers.
Revised: n k is the number ofneurons for the k th layers.

Original: Z k
Revised: Z k

Original: [ i, j ] = 1 indicates that the connection exists between the i th neuronin the k th layer and the j th neuron in the ( k + 1) th layer, where Z k [ i, j ] is the ( i, j ) th entry of Z k .
Revised: [ i, j ] = 1 indicates that the connection exists between the i th neuronin the k th layer and the j th neuron in the ( k + 1) th layer, where Z k [ i, j ] is the ( i, j ) th entry of Z k .

Original: We assume there are ( K + 1) layers in L O C A L and denote h k ∈ R n k to be the output of the k thlayer.
Revised: We assume there are ( K + 1) layers in L O C A L and denote h k ∈ R n k to be the output ofthe k th layer.

Original: Naturally, we have h 0 = x
Revised: Naturally, we have h 0 = x

Original: and h K=
Revised: and h K

Original:  y .
Revised: = ˆ y , where ˆ y is the output of L O C A L.

Original: For the symbolic activation or summation layer,we have h k +

Revised: For thesymbolic activation or summation layer, we have h k +

Original: =
Revised: =

Original: Z Tk ◦ W Tk h k , where ◦ represents the Hadamard product and helps to zero outsome connections.
Revised: Z Tk ◦ W Tk h k , where ◦ represents the Hadamard product and helps to zero out some connections.

Original: For the multiplication layer, we have h k +

Revised: For the multiplication layer, we haveh k +

Original: [ j ] = (cid:81)
Revised: [ j ] = (cid:81)

Original: Z k
Revised: Z k

Original: [ i,j ]=1 h k
Revised: [ i,j ]=1 h k

Original: [ i ] .
Revised: [ i ] .

Original: In general,we denote L O C
Revised: In general, we denote L O C

Original: A L as f ( x ; { Z k } K − 1 k =0 , { W k } K − 1 k =0 ) .
Revised: A L as f ( x ; { Z k } K − 1 k =0 , { W k } K − 1 k =0 ) .

Original: The search algorithm identifies the locations ofthe entry 1 in { Z k } K − 1 k [-=
-] and the estimation process learns the corresponding weights in { W k } K − 1 k = 0using{+=
+}
Revised: Thesearch algorithm identifies the locations of the entry 1 in { Z k } K − 1 k [-=
-]=0 and the estimation process learnsthe corresponding weights in { W k } K − 1 k {+=
+}

Original:  x i , y i } Ni =1 .
Revised: using { x i , y i } Ni =1 .

Original: 3.2 Search L O C A L Structures with Double Convex Deep Q-Learning
Revised: 3.2 Search L O C A L Structures with Double Convex Deep Q-Learning

Original: The high dimensionality of the search space is a general problem for L O C A L and other SR models.
Revised: The high dimensionality of the search space is a general problem for L O C A L and other SR models.

Original: For an efficient search, NN-based RL methods are often utilized, e.g., deep Q-learning [28].
Revised: For an efficient search, NN-based RL methods are often utilized, e.g., deep Q-learning [30].

Original: In thissubsection, we employ the Input Convex Neural Network (ICNN) [17] to design a double convexdeep Q-learning with provable performance guarantees.
Revised: In thissubsection, we employ the Input Convex Neural Network (ICNN) [17] to design a double convexdeep Q-learning with provable performance guarantees.

Original: Markov Decision Process (MDP) to search the structure of L O C A L.The search procedure is an
Revised: Search the structure of L O C A L.

Original: MDP where each layer of  L O C A Lcan be treated as a state and the connections between layers areactions.
Revised: To search the structure of L O C A L, we treat the status of each layerof L O C A L as a state and the connections between layers are actions.

Original: At each state, the deep Q-learning agent aims to find the optimal policy to decide the nextaction [21].
Revised: Specifically, we denote a k ∈

Original: Therefore, we denote a k ∈ { 0 , 1 } n a as the k th action vector where a k [( i − 1) n k + j ] = 1implies that the connection ij exists for the i th neuron in the k th layer and the j th neuron in the( k + 1) th layer.
Revised: { 0 , 1 } n a as the k th action vector where a k [( i − 1) n k + j ] = 1 implies that the connection ij exists forthe i th neuron in the k th layer and the j th neuron in the ( k + 1) th layer.

Original: n a = max { n k n k +1 } K − 1 k =0 is the{+=
 isthe+} dimensionality of the action space.
Revised: n a = max { n k n k +1 } K − 1 k {+=
 isthe+} dimensionality of the action space.

Original: Also, we denotes k ∈
Revised: Also, we denote s k ∈

Original: Z n s as the state vector to represent the current state for the k th layers, where n s = max { n k } Kk =0is the dimensionality of the state space.
Revised: Z n s as the state vector to represent thecurrent state for the k th layers, where n s = max { n k } Kk =0 is the dimensionality of the state space.

Original: For a known state and a fixed action, the next state is exactlyknown due to the exact neuron connections, which implies a deterministic state transition.
Revised: Fora known state and a fixed action, the next state is exactly known due to the exact neuron connections,which implies a deterministic state transition.

Original: Thus, weutilize a linear transformation to represent the state transition process.
Revised: Thus, we utilize a linear transformation to representthe state transition process.

Original: ∀ 0 ≤ k ≤ K − 1 , we define
Revised: ∀ 0 ≤ k ≤ K − 1 , we define

Original: [Equation]
Revised: [Equation]

Original: where Mat ( · ) is the operation to convert a vector to a matrix and we fill 0 s to some s k , a k , and
Revised: where Mat ( · ) is the operation to convert a vector to a matrix and we fill 0 s to some s k , a k , and

Original: Z ′ k tomaintain the fixed size.
Revised: Z ′ k tomaintain the fixed size.

Original: [ · ; · ] is the row concatenation process.
Revised: [ · ; · ] is the row concatenation process.

Original: We show this linear state transition isessential to guarantee the convexity of negative optimal Q-function in Theorem 1.
Revised: We show this linear state transition isessential to guarantee the convexity of negative optimal Q-function in Theorem 1.

Original: For the calculationexample of state transition, one can refer to Fig.
Revised: For the calculationexample of state transition, one can refer to Fig.

Original: Finally, we can obtain Z k from Mat ( a k ) bydeleting the filled 0s.
Revised: Then, we can obtain Z k from Mat ( a k ) by deletingthe filled 0s.

Original: By the designed states and actions, the search will always start at s 0 and end ats K in one episode, thus formulating the L O C A L function f ( x ; { Z k } K − 1 k =0 , { W k } K − 1 k =0 ) . 
Revised: By the designed states and actions, the search will always start at s 0 and end at s K inone episode, thus formulating the L O C A L function f ( x ; { Z k } K − 1 k =0 , { W k } K − 1 k =0 ) . The above states, actions, state transitions, initial state s 0 , and a discount factor γ can form a

Revised: Controlled Markov Process (CMP) [31, 32], which is a Markov Decision Process (MDP) withouta reward function [32].

Revised: In the following part, we define our reward function based on an end-oftrajectory reward [31].

Revised: Although the reward function is non-Markov, we prove in Theorem 1 thatunder our settings, there exists an optimal Q-function.

Original:  Double convexity for the negative reward and Q-function in the search model.
Revised: The proof can be seen in Appendix A.2. Double convexity for the negative reward and Q-function in the search model.

Original: Training L O C A Lcan bring a terminal reward [3] to evaluate the episode.
Revised: Training L O C A Lcan bring an end-of-trajectory reward [3] to evaluate the episode.

Original: However, this reward is insufficient to enablea convex evaluation for instant state-action pairs.
Revised: However, this reward is insufficientto enable a convex evaluation for instant state-action pairs.

Original: Thus, we propose to restrict the negative rewardfunction to be convex.
Revised: Thus, we propose to restrict the negativereward function to be convex.

Original: Based on Bellman equations [29], this design will ensure the convexity of thenegative optimal Q-function.
Revised: Based on Bellman equations [33], this design will ensure the convexityof the negative optimal Q-function.

Original: More proof details can be referred to Theorem 1 and Appendix A.2.
Revised: More proof details can be referred to Theorem 1 and Appendix A.2.

Original: Specifically, we utilize an ICNN to model the reward function − R ( s k , a k )
Revised: Specifically, we utilize an ICNN to model the reward function − R ( s k , a k )

Original: such that − R ( s k , a k )is convex in states and actions in continuous spaces.
Revised: such that − R ( s k , a k )is convex in states and actions in continuous spaces.

Original: − R ( s k , a k ) requires proper training to do thecorrect evaluation.
Revised: − R ( s k , a k ) requires proper training to do thecorrect evaluation.

Original: In the t th episode, we collect the training input of state-action pairs { s tk , a tk } K − 1 k =0 .
Revised: In the t th episode, we collect the training input of state-action pairs { s tk , a tk } K − 1 k =0 .

Original: Then, the output can be defined as the terminal reward to evaluate the obtained L O C A L which isdenoted as f t ( · ) .
Revised: Then, the output can be defined as the end-of-trajectory reward to evaluate the obtained L O C A Lwhich is denoted as f t ( · ) .

Original: Basically, we calculate the Normalized Root-Mean-Square Error (NRMSE) [3]of the trained f t ( · ) function such that NRMSE t = 1 σ y (cid:113) 1 N (cid:80) 
Revised: Basically, we calculate the Normalized Root-Mean-Square Error (NRMSE)[3] of the trained f t ( · ) function such that NRMSE t = 1 σ y (cid:113) 1 N (cid:80) Ni

Original: Ni =

Revised: =

Original: ( y i − f t (cid:0) x i ) (cid:1) 2 , where σ y is thestandard deviation of the outputs.
Revised: ( y i − f t (cid:0) x i ) (cid:1) 2 , where σ y is thestandard deviation of the outputs.

Original: Then, the output of the reward can be calculated as R t = 1 1+ NRMSE t .
Revised: Then, the output of the reward can be calculated as R t = 1 1+ NRMSE t .

Original: Therefore, we can train − R ( · ) using {{ s tk , a tk } K − 1 k =0 , − R t } .
Revised: Therefore, we can train − R ( · ) using {{ s tk , a tk } K − 1 k =0 , − R t } .

Original: It’s important to note that building this convex function − R ( · ) is better than directly leveraging theterminal reward for evaluations.
Revised: It’s important to note that building this convex function − R ( · ) is better than directly leveraging theend-of-trajectory reward for evaluations.

Original: The reason is that the former process not only provides theoreticalguarantees in Theorem 1 but also evaluates the actions in a continuous space with convexity.
Revised: The reason is that the former process not only providestheoretical guarantees in Theorem 1 but also evaluates the actions in a continuous space with convexity.

Original: Namely,one can utilize R ( s k , ˜ a k ) to evaluate a continuous action ˜ a k , given that there is a much higherprobability to obtain ˜ a k rather than the discrete action a k in the deep Q-learning.
Revised: Namely, one can utilize R ( s k , ˜ a k ) to evaluate a continuous action ˜ a k , given that there is a muchhigher probability to obtain ˜ a k rather than the discrete action a k in the deep Q-learning.

Original: Then, to update Qvalues, we have the following iterative computations based on the temporal difference [21]:
Revised: Then, toupdate Q values, we have the following iterative computations based on the temporal difference [21]:

Original: [Equation]
Revised: [Equation]

Original: where Q t ( s k , ˜ a k ) is the Q value at the t th episode for state s k and action ˜ a k .
Revised: where Q t ( s k , ˜ a k ) is the Q value at the t th episode for state s k and action ˜ a k .

Original: α and γ are pre-definedlearning rate and discount factor, respectively.
Revised: α and γ are pre-definedlearning rate and discount factor, respectively.

Original: Similarly, we utilize another ICNN to represent− Q ( s k , ˜ a k ) such that − Q ( s k , ˜ a k ) is convex in ˜ a k given fixed s k .
Revised: Similarly, we utilize another ICNN to represent− Q ( s k , ˜ a k ) such that − Q ( s k , ˜ a k ) is convex in ˜ a k given fixed s k .

Original: Therefore, one can solve a convexoptimization problem max a Q t ( s k +1 , ˜ a ) to obtain the (approximately) global optimal action for
Revised: Therefore, one can solve a convexoptimization problem max a Q t ( s k +1 , ˜ a ) to obtain the (approximately) global optimal action for

Original: Equation (2) in each iteration.
Revised: Equation (2) in each iteration.

Original: By definitions of the discrete actions, we can restrict the continuousaction space in a hypercube conv ( { 0 , 1 } n a ) , i.e., a convex hull of the discrete actions.
Revised: By definitions of the discrete actions, we can restrict the continuousaction space in a hypercube conv ( { 0 , 1 } n a ) , i.e., a convex hull of the discrete actions.

Original: Thus, theoptimization problem is:
Revised: Thus, theoptimization problem is:

Original: [Equation]
Revised: [Equation]

Original: Based on [17], this convex optimization problem can be solved via a bundle entropy method.
Revised: Based on [17], this convex optimization problem can be solved via a bundle entropy method.

Original: Afterobtaining a continuous solution ˜ a ∗ , we discretize it to a discrete vector a ∗ to build L O C A L. Onesimple method is to enforce a ∗
Revised: Afterobtaining a continuous solution ˜ a ∗ , we discretize it to a discrete vector a ∗ to build L O C A L. One simple method is to enforce a ∗

Original: [ i ] = 

Revised: [ i ] =

Original: if ˜ a ∗ [ i ] ≥ 0 . 5 , and otherwise a ∗ [ i ] = 0 .
Revised: if ˜ a ∗ [ i ] ≥ 0 . 5 , and otherwise a ∗ [ i ] = 0 .

Original: Thus, both
Revised: Thus, both

Original: Q ( s k , ˜ a k ) and Q ( s k , a k ) can be trained using Equation (2).
Revised: Q ( s k , ˜ a k ) and Q ( s k , a k ) can be trained using Equation (2).

Original: Practically, we introduce ϵ -greedystrategy [30], experience replay [31] and a target Q-network [28] to update Q-function, thus boostingthe convergence to the optimal Q-function.
Revised: Practically, we introduce ϵ -greedystrategy [34], experience replay [35] and a target Q-network [30] to update Q-function, thus boostingthe convergence to the optimal Q-function.

Original: The specific algorithm can be seen in Algorithm 1.
Revised: The specific algorithm can be seen in Algorithm 1.

Original: Finally, based on Theorems 1-2, if the optimal Q-function is found, solving the convex optimization via theoptimal Q-function can generate the correct structures of L O C A L and exact equations.
Revised: Finally,based on Theorems 1-2, if the optimal Q-function is found, solving the convex optimization via theoptimal Q-function can generate the correct structures of L O C A L and exact equations.

Original: Symbolic static and dynamic constraints.
Revised: Symbolic static and dynamic constraints.

Original: Constraints can be added to accelerate the search process[3, 2].
Revised: Constraints can be added to accelerate the search process[3, 2].

Original: For example, in Algorithm 1, we propose a constraint checking program for the state-actionpairs to avoid the invalid search, suitable for arbitrary restrictions.
Revised: For example, in Algorithm 1, we propose a constraint checking program for the state-actionpairs to avoid the invalid search, suitable for arbitrary restrictions.

Original: Then, we emphasize a generaltype of constraint, symbolic constraint, for the SR problem.
Revised: Then, we emphasize a generaltype of constraint, symbolic constraint, for the SR problem.

Original: The symbolic static constraint requiresthat each equation contains only a subset of symbols.
Revised: The symbolic static constraint requiresthat each equation contains only a subset of symbols.

Original: For example, the equation y 1 = x 1 x 2 · · · x 100shouldn’t exist since it is too complicated for real-world systems.
Revised: For example, the equation y 1 = x 1 x 2 · · · x 100shouldn’t exist since it is too complicated for real-world systems.

Original: This constraint eliminates part ofthe action space and can be checked by counting the number of 1 s in the action vector.
Revised: This constraint eliminates part ofthe action space and can be checked by counting the number of 1 s in the action vector.

Original: The symbolicdynamic constraint can gradually reduce the search space based on symbol correlations.
Revised: The symbolicdynamic constraint can gradually reduce the search space based on symbol correlations.

Original: Specifically,we investigate the ( K − 1) th layer’s neurons that are linearly summed to form the equation.
Revised: Specifically,we investigate the ( K − 1) th layer’s neurons that are linearly summed to form the equation.

Original: Ifsome of these neurons have strong linear correlations to the output neuron (e.g., Pearson correlationcoefficient larger than 0 . 99 ), they should be kept subsequently.
Revised: Ifsome of these neurons have strong linear correlations to the output neuron (e.g., Pearson correlationcoefficient larger than 0 . 99 ), they should be kept subsequently.

Original: Namely, we can maintain the pathfrom input neurons to the neurons to be kept and reduce the search space.
Revised: Namely, we can maintain the pathfrom input neurons to the neurons to be kept and reduce the search space.

Original:  Theoretical Analysis
Revised:  Theoretical Analysis

Original: We employ explorations, experience replay, and a target Q-network to boost the convergence to theoptimal Q function.
Revised: We employ explorations, experience replay, and a target Q-network to boost the convergence to theoptimal Q function.

Original: However, our extra requirement of the convex shape for the negative Q-functionmay deteriorate the convergence performance.
Revised: However, our extra requirement of the convex shape for the negative Q-functionmay deteriorate the convergence performance.

Original: Thus, we first prove that in C O NS O L E , the negativeoptimal Q-function is also convex so that the convex design doesn’t affect the convergence.
Revised: Thus, we first prove that in C O NS O L E , the negativeoptimal Q-function is also convex so that the convex design doesn’t affect the convergence.

Original: Then, weprove that the convexity of the negative optimal Q-function eventually yields the exact equations.
Revised: Then, weprove that the convexity of the negative optimal Q-function eventually yields the exact equations.

Original: Theorem 1.
Revised: Theorem 1.

Original: ∀ 0 ≤ k ≤ K − 1 , the negative optimal Q-function − Q ∗ ( s k , ˜ a k ) in the proposed
Revised: ∀ 0 ≤ k ≤ K − 1 , the negative optimal Q-function − Q ∗ ( s k , ˜ a k ) in the proposed

Original: C O NS O L E framework  is convex in s k and ˜ a k , where s k is the discrete state and ˜ a k is the continuousaction at the k th stage.
Revised: C O NS O L E framework exists and is convex in s k and ˜ a k , where s k is the discrete state and ˜ a k is thecontinuous action at the k th stage.

Original: The proof can be seen in Appendix A.2.
Revised: The proof can be seen in Appendix A.2.

Original: Based on the convexity of negative optimal Q-function,the optimal sequence of states ( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) can be found viasolving the convex optimization problem.
Revised: Based on the convexity of negative optimal Q-function,the optimal sequence of states ( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) can be found viasolving the convex optimization problem.

Original: Then, we have the following theorem.
Revised: Then, we have the following theorem.

Original: Theorem 2.
Revised: Theorem 2.

Original: Let f ∗ ( · ; W ) denote the L O C A L constructed by the optimal sequences of states( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) from − Q ∗ ( · ) , where W is the set of weightsof f ∗ ( · ; W ) .
Revised: Let f ∗ ( · ; W ) denote the L O C A L constructed by the optimal sequences of states( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) from − Q ∗ ( · ) , where W is the set of weightsof f ∗ ( · ; W ) .

Original: If f ∗ ( · ; W ) can be trained with noiseless datasets and the training can achieve theglobal optimal weights W ∗ , f ∗ ( · ; W ∗ ) represents the exact equations.
Revised: If f ∗ ( · ; W ) can be trained with noiseless datasets and the training can achieve theglobal optimal weights W ∗ , f ∗ ( · ; W ∗ ) can be simplified to the true equation g ( · ) .

Original: The proof can be seen in Appendix A.3.
Revised: The proof can be seen in Appendix A.3.

Original: Theorem 2 requires that L O C A L can learn the global optimalweights.
Revised: Theorem 2 requires that L O C A L can learn the global optimalweights.

Original: The requirement is generally hard to achieve due to the non-linearity and non-convexityof L O C A
Revised: The requirement is generally hard to achieve due to the non-linearity and non-convexityof L O C A

Original: L.
Revised: L.

Original: However, we show that with mild assumptions, there are local regions in the L O C A Lloss surface with strict convexity.
Revised: However, we show that with mild assumptions, there are local regions in the L O C A Lloss surface with strict convexity.

Original: Then, if we have proper initializations, the gradient-based weightupdating can find the global optimum.
Revised: Then, if we have proper initializations, the gradient-based weightupdating can find the global optimum.

Original: Specifically, we have the following theorem.
Revised: Specifically, we have the following theorem.

Original: Theorem 3.
Revised: Theorem 3.

Original: Assume the following conditions hold: (1) the equation g ( x ) is C 2 smooth and hasbounded second derivatives with respect to weights, (2) ∃ x ∈ X , g ( x ) has non-zero gradients withrespect to weights, (3) the structure of L O C
Revised: Assume the following conditions hold: (1) the equation g ( x ) is C 2 smooth and hasbounded second derivatives with respect to weights, (2) ∃ x ∈ X , g ( x ) has non-zero gradients withrespect to weights, (3) the structure of L O C

Original: A L is correctly searched to exactly represent symbolsand symbol connections in g ( x ) , and (4) the training dataset of L O C
Revised: A L is correctly searched to exactly represent symbolsand symbol connections in g ( x ) , and (4) the training dataset of L O C

Original: A L is noiseless.
Revised: A L is noiseless.

Original: Then, for the
Revised: Then, for the

Original: MSE loss surface of L O C A L , each global optimal point has a strictly convex local region.
Revised: MSE loss surface of L O C A L , each global optimal point has a strictly convex local region.

Original: The proofs can be seen in Appendix A.4.
Revised: The proofs can be seen in Appendix A.4.

Original: Note that Assumptions 1-2 easily hold for common physicalequations in nature [2].
Revised: Note that Assumptions 1-2 easily hold for common physicalequations in nature [2].

Original: Assumption 3 relies on the search algorithm, and we show, both theoreticallyand numerically, that our double convex deep Q-learning has good performances.
Revised: Assumption 3 relies on the search algorithm, and we show, both theoreticallyand numerically, that our double convex deep Q-learning has good performances.

Original: Assumption 4relies on the quality of data and we focus on the noiseless data in this paper.
Revised: Assumption 4relies on the quality of data and we focus on the noiseless data in this paper.

Original: What’s more, Equation(6) in the proof suggests that if the absolute noise values are small, the locally convex region stillexists.
Revised: What’s more, Equation(6) in the proof suggests that if the absolute noise values are small, the locally convex region stillexists.

Original: We also numerically prove C O NS O L E is robust under certain noise levels in Section 5.5.
Revised: We also numerically prove C O NS O L E is robust under certain noise levels in Section 5.5.

Original: Tosummarize, these assumptions are acceptable.
Revised: Tosummarize, these assumptions are acceptable.

Original: To quantify the range of the local region for a L O C A
Revised: To quantify the range of the local region for a L O C A

Original: Lwith a certain complexity, we have the following theorem.
Revised: Lwith a certain complexity, we have the following theorem.

Original: Theorem 4.
Revised: Theorem 4.

Original: Suppose Assumptions 1-4 in Theorem 3 hold.
Revised: Suppose Assumptions 1-4 in Theorem 3 hold.

Original: For a L O C A L with one symbolicactivation, multiplication, and summation layer, the set of local convex regions with global optima is 
Revised: For a L O C A L with one symbolicactivation, multiplication, and summation layer, the set of local convex regions with global optima is U = { W [Equation]

Revised: (cid:12)(cid:12)

Original: [Equation]
Revised: [Equation]

Original: {+( x i ,W + tX ) (cid:12)(cid:12)η (cid:12)(cid:12) ddt (cid:12)(cid:12) t =
+}
Revised: {+( x i ,W + tX ) (cid:12)(cid:12)η (cid:12)(cid:12) ddt (cid:12)(cid:12) t =
+}

Original:  > | ˆ y ( x k , W ) − y k |} , where notations are defined in the proof.
Revised: ˆ y ( x j ,W + tX ) (cid:12)(cid:12) > | ˆ y ( x k , W ) − y k |} , where notations are defined in the proof.

Original: The proofs can be seen in Appendix A.5.
Revised: The proofs can be seen in Appendix A.5.

Original: We explain the region range is large for a stable system thatsatisfies all assumptions in Theorem 4.
Revised: We explain the region range is large for a stable system thatsatisfies all assumptions in Theorem 4.

Original: Physical interpretations of the convex region size.
Revised: Physical interpretations of the convex region size.

Original: Physical systems in scientific and engineeringdomains have certain stability that can withstand parameter changes to some extent.
Revised: Physical systems in scientific and engineeringdomains have certain stability that can withstand parameter changes to some extent.

Original: Further, thisability should hold for arbitrary x ∈ X and y ∈ Y .
Revised: Further, thisability should hold for arbitrary x ∈ X and y ∈ Y .

Original: Thus, we can assume ddt (cid:12)(cid:12) t =

Revised: Thus, we can assume ddt (cid:12)(cid:12) t =

Original: ˆ y
Revised: ˆ y

Original: ( x i , W + tX ) ≈ddt (cid:12)(cid:12) t
Revised: ( x i , W + tX ) ≈ddt (cid:12)(cid:12) t

Original: =

Revised: =

Original: ˆ y ( x j , W + tX )
Revised: ˆ y ( x j , W + tX )

Original: ≈ ddt (cid:12)(cid:12) t
Revised: ≈ ddt (cid:12)(cid:12) t

Original: =

Revised: =

Original: ˆ y
Revised: ˆ y

Original: ( x k , W + tX )
Revised: ( x k , W + tX )

Original: for x i , x j , x k ∈ X .
Revised: for x i , x j , x k ∈ X .

Original: Then, the inequality in set Ucan be approximately rewritten as 1 η > || w − w ∗
Revised: Then, the inequality in set Ucan be approximately rewritten as 1 η > || w − w ∗

Original: || 2 , where w and w ∗ are the vectorized W and W ∗ ,respectively.
Revised: || 2 , where w and w ∗ are the vectorized W and W ∗ ,respectively.

Original: Namely, the distance between any point W in the region to the global optimal point
Revised: Namely, the distance between any point W in the region to the global optimal point

Original: W ∗ in the region is bounded by 1 η .
Revised: W ∗ in the region is bounded by 1 η .

Original: Based on Equation (19), η is the bound of the ratio of secondderivative to the first derivative.
Revised: Based on Equation (19), η is the bound of the ratio of secondderivative to the first derivative.

Original: For a stable system, this ratio should be small.
Revised: For a stable system, this ratio should be small.

Original: Otherwise, the systemcan easily crash with a small parameter disturbance.
Revised: Otherwise, the systemcan easily crash with a small parameter disturbance.

Original: Thus, 1 η is relatively large and so is the regionof U .
Revised: Thus, 1 η is relatively large and so is the regionof U .

Original: An example of the range is displayed in Section 5.2.
Revised: An example of the range is displayed in Section 5.2.

Original: Finally, the above analysis also holdswhen the L O C
Revised: Finally, the above analysis also holdswhen the L O C

Original: A L of the system equation has more than one symbolic activation, multiplication, andsummation layers.
Revised: A L of the system equation has more than one symbolic activation, multiplication, andsummation layers.

Original: This is because Equations (20) always hold as long as we can find a η to boundthe ratio of  second derivative to the first derivative, which is irrelevant to the structure of L O C A L.
Revised: This is because Equation (20) always holds as long as we can find a η to boundthe ratio of the second derivative to the first derivative, which is irrelevant to the structure of L O C A L.

Original:  Experiments5.1 Settings
Revised:  Experiments5.1 Settings

Original: Datasets.
Revised: Datasets.

Original: We use the following datasets for testing.
Revised: We use the following datasets for testing.

Original: (1) Synthetic datasets .
Revised: (1) Synthetic datasets .

Original: We create two datasets,
Revised: We create two datasets,

Original: Syn 1 and Syn 2 , for testing.
Revised: Syn 1 and Syn 2 , for testing.

Original: Syn 1 has the following equations: y 1 = 3 x 21 cos(2 .
Revised: Syn 1 has the following equations: y 1 = 3 x 21 cos(2 .

Original: 5 x 2 ) , y 

Revised: 5 x 2 ) , y

Original: = 4 x 1 x 3 ,and y 

Revised: = 4 x 1 x 3 ,and y

Original: = 3 x 23 .
Revised: = 3 x 23 .

Original: Syn 2 is more complex with the following equations y 1 = √ 2 .
Revised: Syn 2 is more complex with the following equations y 1 = √ 2 .

Original: 2 x 1 x 2 + x 1 x 22 ,y 2 = sin(1 .
Revised: 2 x 1 x 2 + x 1 x 22 ,y 2 = sin(1 .

Original: 8 x 1 ) (cid:0) log(3 x 2 ) + √ x 3 (cid:1) , y 3 = √ 3 .
Revised: 8 x 1 ) (cid:0) log(3 x 2 ) + √ x 3 (cid:1) , y 3 = √ 3 .

Original: 7 x 3 log(1 .
Revised: 7 x 3 log(1 .

Original: 6 x 1 ) + x 21 .
Revised: 6 x 1 ) + x 21 .

Original: For the training data, each inputvariable is randomly sampled from a uniform distribution of U (1 , 2) to avoid invalid values likelog(0) .
Revised: For the training data, each inputvariable is randomly sampled from a uniform distribution of U (1 , 2) to avoid invalid values likelog(0) .

Original: Totally, we create 2 , 000 samples for training.
Revised: Totally, we create 2 , 000 samples for training.

Original: Then, in the test phase, we utilize another2 , 000 samples whose input variables are sampled from U (3 , 4) .
Revised: Then, in the test phase, we utilize another2 , 000 samples whose input variables are sampled from U (3 , 4) .

Original: The symbolic activation pools are{ x, x 2 , cos( x ) } and {√ x, x, x 2 , log( x ) , sin( x ) } for Syn 1 and Syn 2 , respectively.(2) Power system dataset.
Revised: The symbolic activation pools are{ x, x 2 , cos( x ) } and {√ x, x, x 2 , log( x ) , sin( x ) } for Syn 1 and Syn 2 , respectively.(2) Power system dataset.

Original: Power flow equation determines the operations of electric systems [32].
Revised: Power flow equation determines the operations of electric systems [36].

Original: For node i in an M -node system, the equation can be written as p i
Revised: For node i in an M -node system, the equation can be written as p i

Original: = (cid:80)
Revised: = (cid:80)

Original: | M | m =1 G im ( u i u m + v i v m )
Revised: | M | m =1 G im ( u i u m + v i v m )

Original: +
Revised: +

Original: B im ( v i u m − u i v m ) and q i = (cid:80)
Revised: B im ( v i u m − u i v m ) and q i = (cid:80)

Original: Mm =

Revised: Mm =

Original: G im ( v i u m −
Revised: G im ( v i u m −

Original: u
Revised: u

Original: i v m ) − B im ( v i u m − u i v m ) , where u i and v iare the real and imaginary components of the voltage phasor at node i .
Revised: i v m ) − B im ( v i u m − u i v m ) , where u i and v iare the real and imaginary components of the voltage phasor at node i .

Original: p i and q
Revised: p i and q

Original: i are the activeand reactive power at node i .
Revised: i are the activeand reactive power at node i .

Original: G
Revised: G

Original: im and B im represent the physical parameters of line im .
Revised: im and B im represent the physical parameters of line im .

Original: If lineim does not exist, G im =
Revised: If lineim does not exist, G im =

Original: B im = 0 .
Revised: B im = 0 .

Original: Therefore, we can treat x =
Revised: Therefore, we can treat x =

Original: [ u 1 , v 1 , · · · , u M , v M ] T andy =
Revised: [ u 1 , v 1 , · · · , u M , v M ] T andy =

Original: [ p 1 , q 1 , · · · , p M , q M ] T .
Revised: [ p 1 , q 1 , · · · , p M , q M ] T .

Original: The target is to learn the underlying system topology and parameters,which has broad impacts on the power domains [33].
Revised: The target is to learn the underlying system topology and parameters,which has broad impacts on the power domains [37].

Original: In this experiment, we implement simulationfrom a 5 -node system using MATPOWER [34] and two year’s hourly data.
Revised: In this experiment, we implement simulationfrom a 5 -node system using MATPOWER [38] and two year’s hourly data.

Original: The first 8 , 760 points areused for training while the remaining samples are used for testing.
Revised: The first 8 , 760 points areused for training while the remaining samples are used for testing.

Original: The symbolic activation pool is{ x } .
Revised: The symbolic activation pool is{ x } .

Original: We denote this dataset as Pow.(3) Mass-damper system dataset.
Revised: We denote this dataset as Pow.(3) Mass-damper system dataset.

Original: Equations of the mass-damper system can be approximatelywritten as: ˙ q = − DRD ⊤ M − 1 q , where ˙ q is a vector of momenta, D is the incidence matrix ofthe system, R is the diagonal matrix of the damping coefficients for each line of the system, and
Revised: Equations of the mass-damper system can be approximatelywritten as: ˙ q = − DRD ⊤ M − 1 q , where ˙ q is a vector of momenta, D is the incidence matrix ofthe system, R is the diagonal matrix of the damping coefficients for each line of the system, and

Original: M is the diagonal matrix of each node mass of the system.
Revised: M is the diagonal matrix of each node mass of the system.

Original: Thus, we can set y = ˙
Revised: Thus, we can set y = ˙

Original: q and x = q andthe goal is to learn the parameter matrix − DRD ⊤ M − 1 .
Revised: q and x = q andthe goal is to learn the parameter matrix − DRD ⊤ M − 1 .

Original: We conduct the simulation via MATLABfor a 10 -node system and obtain 6 , 000 points for 1 min simulation with a step size to be 0 .
Revised: We conduct the simulation via MATLABfor a 10 -node system and obtain 6 , 000 points for 1 min simulation with a step size to be 0 .

Original: 01 s. Thefirst 3 , 000 samples are used for training while the rest samples are used for testing.
Revised: 01 s. Thefirst 3 , 000 samples are used for training while the rest samples are used for testing.

Original: The symbolicactivation pool is { x } .
Revised: The symbolicactivation pool is { x } .

Original: Then, we denote the dataset as Mas.
Revised: Then, we denote the dataset as Mas.

Original: Benchmark methods .
Revised: Benchmark methods .

Original: The following benchmark methods are utilized.
Revised: The following benchmark methods are utilized.

Original: (1) Deep Symbolic Regression( DSR ) [3].
Revised: (1) Deep Symbolic Regression( DSR ) [3].

Original: DSR develops an RNN-based framework to search the expression tree.
Revised: DSR develops an RNN-based framework to search the expression tree.

Original: Especially, therisk-seeking policy gradient is utilized to seek the best performance.
Revised: Especially, therisk-seeking policy gradient is utilized to seek the best performance.

Original: Then, BFGS [11] can solve thenon-linear optimization and estimate the symbol coefficients.
Revised: Then, BFGS [11] can solve thenon-linear optimization and estimate the symbol coefficients

Original: (2) Vanilla Policy Gradient ( VPG ) [3]. VPG is a vanilla version of DSR with a normal policy gradient rather than the risk-seeking method.(3) Equation Learner ( EQL ) [6, 7].
Revised: VPG is a vanilla version of DSR with a normal policy gradient rather than the risk-seeking method.(3) Equation Learner ( EQL ) [6, 7].

Original: EQL creates an end-to-end NN to select symbols and estimatethe coefficients.
Revised: EQL creates an end-to-end NN to select symbols and estimatethe coefficients.

Original: The sparse regularization is enforced for the NN weights to search symbols.
Revised: The sparse regularization is enforced for the NN weights to search symbols.

Original: Multilayer Perceptron ( MLP ).
Revised: Multilayer Perceptron ( MLP ).

Original: We also employ a standard MLP to learn the regression from x to y .
Revised: We also employ a standard MLP to learn the regression from x to y .

Original: We only evaluate the extrapolation capacity of MLP in the test dataset.
Revised: We only evaluate the extrapolation capacity of MLP in the test dataset.

Original: For DSR, VPG, and EQL methods, based on the input datasets, we adjust the symbol and operatorlibrary to enable the same searching space as C O NS O L E for fair comparisons.
Revised: For DSR, VPG, and EQL methods, based on the input datasets, we adjust the symbol and operatorlibrary to enable the same searching space as C O NS O L E for fair comparisons.

Original: We run the benchmarkmethods 5 times with different random seeds and present their best results.
Revised: We run the benchmarkmethods 5 times with different random seeds and present their best results.

Original: As for our method, weonly run 1 time and obtain good results due to the convex design and the ϵ -greedy strategy.
Revised: As for our method, weonly run 1 time and obtain good results due to the convex design and the ϵ -greedy strategy.

Original: Metrics for evaluation.
Revised: Metrics for evaluation.

Original: We employ the following metrics.
Revised: We employ the following metrics.

Original: (1) Average coefficient estimation percentage error E c .
Revised: (1) Average coefficient estimation percentage error E c .

Original: For an equation with H symbols, We calculate the error as E c = 1 H (cid:80)
Revised: For an equation with H symbols, We calculate the error as E c = 1 H (cid:80)

Original: h
Revised: h

Original: PE ( w h , ˆ w h ) ,where w h and ˆ w h represents the true and the estimated coefficients for the h th symbol, respectively.
Revised: PE ( w h , ˆ w h ) ,where w h and ˆ w h represents the true and the estimated coefficients for the h th symbol, respectively.

Original: PE is the operation to calculate the percentage error.
Revised: PE is the operation to calculate the percentage error.

Original: If there is no matched symbol for the h th truesymbol, we denote PE ( w h , ˆ w h ) = 100% .
Revised: If there is no matched symbol for the h th truesymbol, we denote PE ( w h , ˆ w h ) = 100% .

Original: Note that when calculating E c , proper simplifications maybe needed.
Revised: Note that when calculating E c , proper simplifications maybe needed.

Original: For example, cos (cid:0) 2 . 5( √ x ) 2 (cid:1) = cos(2 .
Revised: For example, cos (cid:0) 2 . 5( √ x ) 2 (cid:1) = cos(2 .

Original: 5 x ) .
Revised: 5 x ) .

Original: (2) NRMSE in the test dataset.
Revised: (2) NRMSE in the test dataset.

Original: We measurethe extrapolation capacity in the test dataset and utilize NRMSE employed in Section 3.2.
Revised: We measurethe extrapolation capacity in the test dataset and utilize NRMSE employed in Section 3.2.

Original: Implementing details of C O NS O L E .
Revised: Implementing details of C O NS O L E .

Original: Hyper-parameters of C O NS
Revised: Hyper-parameters of C O NS

Original: O L E exist for both the doubleconvex deep Q-learning and the L O C A L.
Revised: O L E exist for both the doubleconvex deep Q-learning and the L O C A L.

Original: In the deep Q-learning, we set γ = 0 .
Revised: In the deep Q-learning, we set γ = 0 .

Original: 2 , ϵ = 0 .
Revised: 2 , ϵ = 0 .

Original: 4 , T = 600 ,λ = 10 − 2 , T 0 = 10 for Algorithm 1.
Revised: 4 , T = 600 ,λ = 10 − 2 , T 0 = 10 for Algorithm 1.

Original: Furthermore, to train the negative Q-function and the rewardfunction, we set the learning rate to be 5 × 10 − 3 and the number of epochs for training to be 50 .
Revised: Furthermore, to train the negative Q-function and the rewardfunction, we set the learning rate to be 5 × 10 − 3 and the number of epochs for training to be 50 .

Original: Then, we set the batch size for the negative Q-function to be 100 .
Revised: Then, we set the batch size for the negative Q-function to be 100 .

Original: If the number of data in the replaybuffer is less than 100 , no training happens for the negative Q-function.
Revised: If the number of data in the replaybuffer is less than 100 , no training happens for the negative Q-function.

Original: Additionally, all the datagathered in one episode are used to train the negative reward function.
Revised: Additionally, all the datagathered in one episode are used to train the negative reward function.

Original: As for the L O C A L, we set
Revised: As for the L O C A L, we set

Original: K = 3 , the learning rate to be 1 × 10 − 2 and the number of training epochs to be 8 .
Revised: K = 3 , the learning rate to be 1 × 10 − 2 and the number of training epochs to be 8 .

Original: We make thesetraining epochs to be small since training the L O C
Revised: We make thesetraining epochs to be small since training the L O C

Original: A L is the most time-consuming part of C O NS O L E .
Revised: A L is the most time-consuming part of C O NS O L E .

Original: Furthermore, if the structure of L O C A L is correctly searched, a small number of iterations can help
Revised: Furthermore, if the structure of L O C A L is correctly searched, a small number of iterations can help

Original: L O C A L to gain the global optimal weights.
Revised: L O C A L to gain the global optimal weights.

Original: Finally, we initialize all trainable weights in L O C
Revised: Finally, we initialize all trainable weights in L O C

Original: A L tobe 1 .
Revised: A L tobe 1 .

Original: The following results show that a relatively large area is suitable for an initial guess of L O C A L.
Revised: The following results show that a relatively large area is suitable for an initial guess of L O C A L.

Original: 5.2 Verification and 3-D Visualization of Convex Mechanisms
Revised: 5.2 Verification and 3-D Visualization of Convex Mechanisms

Original: We first utilize a toy example to verify the benefits of convex designs for two sub problems.
Revised: We first utilize a toy example to verify the benefits of convex designs for two sub problems.

Original: Specifically, we consider to learn y 1 = 3 x 21 cos(2 .
Revised: Specifically, we consider to learn y 1 = 3 x 21 cos(2 .

Original: 5 x 2 ) with the loss function L = (cid:80) Ni
Revised: 5 x 2 ) with the loss function L = (cid:80) Ni

Original: ( y i [1] −w 1 x i [1] 2 cos( w 2 x i [2]) 2 , where the data sample notations are defined in Section 3.1.
Revised: ( y i [1] −w 1 x i [1] 2 cos( w 2 x i [2]) 2 , where the data sample notations are defined in Section 3.1.

Original: As shown in
Revised: As shown in

Original: Fig.
Revised: Fig.

Original: 2a, we design a 4 -layer L O C A L with two learnable parameters w 1 and w 2 to represent thecoefficients.
Revised: 2a, we design a 4 -layer L O C A L with two learnable parameters w 1 and w 2 to represent thecoefficients.

Original: Thus, in the search phase, the goal is to identify the action a 1 .
Revised: Thus, in the search phase, the goal is to identify the action a 1 .

Original: We plot − Q t ( · ) whent = 1 , 5 , 10 in Fig. 2b.
Revised: We plot − Q t ( · ) whent = 1 , 5 , 10 in Fig. 2b.

Original: The convexity always exists so that the algorithm can quickly find the globaloptimal solutions in red dots.
Revised: The convexity always exists so that the algorithm can quickly find the global optimal solutions in red dots.

Original: Next, as the agent update Q values with true rewards, the Q-functionconverges to the optimal function within 10 episodes.
Revised: Next, as the agent update Q values with true rewards, the Q-functionconverges to the optimal function within 10 episodes.

Original: Finally, the convex − Q ∗ ( · ) remains unchangedand can bring the optimal action and the true equation, which supports Theorem 2.
Revised: Finally, the convex − Q ∗ ( · ) remains unchangedand can bring the optimal action and the true equation, which supports Theorem 2.

Original: Subsequently, we plot the loss surface of L in Fig.
Revised: Subsequently, we plot the loss surface of L in Fig.

Original: 2c.
Revised: 2c.

Original: We find that around the two global optima(3 , 2 . 5) and (3 , − 2 . 5) , there are convex regions that have an approximate quadratic shape.
Revised: We find that around the two global optima(3 , 2 . 5) and (3 , − 2 . 5) , there are convex regions that have an approximate quadratic shape.

Original: To furtherquantify the range for proper initialization, we vary the initial weight w 0 ∈ {− 10 , − 9 , · · · , 10 } forw 1 and w 2 in L O C A L. Fig.
Revised: To furtherquantify the range for proper initialization, we vary the initial weight w 0 ∈ {− 10 , − 9 , · · · , 10 } forw 1 and w 2 in L O C A L. Fig.

Original: 2 reports the final training loss with respect to different w 0 , and we findthe safe range for initialization is [ − 3 , 7] \ { 0 } .
Revised: 2 reports the final training loss with respect to different w 0 , and we findthe safe range for initialization is [ − 3 , 7] \ { 0 } .

Original: This range is relatively large when compared to theoptimal values.
Revised: This range is relatively large when compared to theoptimal values.

Original: These observations support our Theorems 3 and 4.
Revised: These observations support our Theorems 3 and 4.

Original: Finally, w 0 = 0 does not worksince ∂L∂w 2 (cid:12)(cid:12) w 2 =

Revised: Finally, w 0 = 0 does not worksince ∂L∂w 2 (cid:12)(cid:12) w 2 =

Original: = 0 always holds, which prevents the weight updating using gradient methods. 
Revised: = 0 always holds, which prevents the weight updating using gradient methods. [Table] y [Table]

Revised: =

Original: [Table]
Revised: [Table]

Original: 5.3 Convexity Guarantees of C O NS O L E to Learn Correct Equations
Revised: y [Table] = [Table]

Revised: [Table]

Revised: 5.3 Convexity Guarantees of C O NS O L E to Learn Correct Equations

Original: In this subsection
Revised: In this subsection

Original: [Table]
Revised: [Table]

Original: we report the results of the equation learning.
Revised: we report the results of the equation learning.

Original: First, we list the learned equationsfor Syn 1 and Syn 2 in Table 1.
Revised: First, we list the learned equationsfor Syn 1 and Syn 2 in Table 1.

Original: Table 1 presents that
Revised: Table 1 presents that

Original: C
Revised: C

Original: O
Revised: O

Original: NS
Revised: NS

Original: O
Revised: O

Original: L E has the best performance in most ofthe equations while DSR ranks second.
Revised: L E has the best performance in most ofthe equations while DSR ranks second.

Original: In particular, C O NS O L E can accurately learn all equations in
Revised: In particular, C O NS O L E can accurately learn all equations in

Original: Syn 1 and Syn 2 .
Revised: Syn 1 and Syn 2 .

Original: The superior performance is mostly due to the convex design of the search and thecoefficient estimation process with provable guarantees.
Revised: The superior performance is mostly due to the convex design of the search and thecoefficient estimation process with provable guarantees.

Original: In addition, we observe that for the resultof C O NS O L E in learning y 3 of Syn 1 , we have 3 x 23 cos(0 .
Revised: In addition, we observe that for the resultof C O NS O L E in learning y 3 of Syn 1 , we have 3 x 23 cos(0 .

Original: 005 x 1 ) ≈ 3 x 23 .
Revised: 005 x 1 ) ≈ 3 x 23 .

Original: This shows that there is apossibility that the search result of C O NS O L E might not be optimal (i.e., an extra consine term existsbut is close to 1 ), but the learned equation is still highly accurate.
Revised: This shows that there is apossibility that the search result of C O NS O L E might not be optimal (i.e., an extra consine term existsbut is close to 1 ), but the learned equation is still highly accurate.

Original: Such an observation nonethelessguides further study of the coupling relationship between the search and the estimation procedures.
Revised: Such an observation nonethelessguides further study of the coupling relationship between the search and the estimation procedures.

Original: [Figure]
Revised: [Figure]

Original: DSR doesn’t perform well when the underlying equation is relatively complex, e.g., y 1 in Syn 1 and y 1and y 2 in Syn 2 .
Revised: DSR doesn’t perform well when the underlying equation is relatively complex, e.g., y 1 in Syn 1 and y 1and y 2 in Syn 2 .

Original: This is because DSR may still fall into a local optimal solution despite risk-seekingpolicy design.
Revised: This is because DSR may still fall into a local optimal solution despite risk-seeking policy design.

Original: VPG method performs worse than DSR since VPG considers an expected reward [3].
Revised: VPG method performs worse than DSR since VPG considers an expected reward [3].

Original: Finally, EQL performs the worst as it merges the symbol search and the coefficient estimation in one
Revised: Finally, EQL performs the worst as it merges the symbol search and the coefficient estimation in one

Original: NN model, which provides few guarantees of accurate learning.
Revised: NN model, which provides few guarantees of accurate learning.

Original: The above observations and analyses are consistent with the result of coefficient estimation errors and prediction errors in Fig. 3a and 3b.
Revised: The above observations and analysesare consistent with the result of coefficient estimation errors and prediction errors in Fig. 3a and 3b.

Original: For the Pow and Mas, C O NS O L E doesn’t learn completely exact equations within T = 600 episodes.
Revised: For the Pow and Mas, C O NS O L E doesn’t learn completely exact equations within T = 600 episodes.

Original: This is because they have a large number of variables to be considered.
Revised: This is because they have a large number of variables to be considered.

Original: However, C O NS O L E is stillbetter than other methods.
Revised: However, C O NS O L E is stillbetter than other methods.

Original: 5.4 Ablation Study: Exploration and Convex Search are Essential
Revised: 5.4 Ablation Study: Exploration and Convex Search are Essential

Original: We conduct an ablation study to further understand what factors are important in the C O NS O L E .
Revised: We conduct an ablation study to further understand what factors are important in the C O NS O L E .

Original: We test the result with Syn 1 and Syn 1 and report the E c (%) values.
Revised: We test the result with Syn 1 and Syn 1 and report the E c (%) values.

Original: Specifically, we investigate thefollowing cases.
Revised: Specifically, we investigate thefollowing cases.

Original: (1) No ablation.
Revised: (1) No ablation.

Original: (2) Drop exploration in deep Q-learning.
Revised: (2) Drop exploration in deep Q-learning.

Original: We delete the ϵ -greedystrategy.
Revised: We delete the ϵ -greedystrategy.

Original: (3) Drop double-convex deep-Q learning.
Revised: (3) Drop double-convex deep-Q learning.

Original: We replace this design with a traditional deep-Qlearning.
Revised: We replace this design with a traditional deep-Qlearning.

Original: (4) Drop coefficient estimation using L O C A L. After learning the structure of L O C A L, wereformulate a non-linear optimization and utilize BFGS [11] in DSR, instead of gradient descent in
Revised: (4) Drop coefficient estimation using L O C A L. After learning the structure of L O C A L, wereformulate a non-linear optimization and utilize BFGS [11] in DSR, instead of gradient descent in

Original: L O C A L, to estimate the coefficient.
Revised: L O C A L, to estimate the coefficient.

Original: (5) Drop static symbolic constraint.
Revised: (5) Drop static symbolic constraint.

Original: (6) Drop dynamic symbolicconstraint.
Revised: (6) Drop dynamic symbolicconstraint.

Original: These two constraints are mentioned in Section 3.2.
Revised: These two constraints are mentioned in Section 3.2.

Original: Then, Fig. 4 shows that cases (2) and(3) cause large errors.
Revised: Then, Fig. 4 shows that cases (2) and(3) cause large errors.

Original: For case (2) , if no exploration strategy is added, the updating of the Q-functionand the reward function is slow.
Revised: For case (2) , if no exploration strategy is added, the updating of the Q-functionand the reward function is slow.

Original: For case (3) , the non-convex search induces many sub-optimalactions in the search process.
Revised: For case (3) , the non-convex search induces many sub-optimalactions in the search process.

Original: Thus, these two cases cause a slow search process and significant errorsafter T = 600 episodes.
Revised: Thus, these two cases cause a slow search process and significant errorsafter T = 600 episodes.

Original: For symbolic constraints in (5) and (6) , removing them slightly increasesthe error for Syn 1 .
Revised: For symbolic constraints in (5) and (6) , removing them slightly increasesthe error for Syn 1 .

Original: This shows these constraints are beneficial to the search process.
Revised: This shows these constraints are beneficial to the search process.

Original: Finally, wefind that utilizing BFGS in (4) can bring good results with initialization in the locally convex region.
Revised: Finally, wefind that utilizing BFGS in (4) can bring good results with initialization in the locally convex region.

Original: Since the non-linear optimization has the same loss surface as L O C A L, the locally convex region in
Revised: Since the non-linear optimization has the same loss surface as L O C A L, the locally convex region in

Original: Theorems 3 and 4 can prove the good performance of BFGS.
Revised: Theorems 3 and 4 can prove the good performance of BFGS.

Original: (%) of ablation study.

Original: [Figure]
Revised: [Figure]

Original:  5. C O NS O L E is Robust with Changing Noise Levels and Data Volume
Revised:  5. C O NS O L E is Robust with Changing Noise Levels and Data Volume

Original: We utilize Syn 1 and Syn 2 to examine the robustness of the framework with changing noiselevels and data volumes.
Revised: We utilize Syn 1 and Syn 2 to examine the robustness of the framework with changing noiselevels and data volumes.

Original: For the noise level, we consider the Signal-to-Noise Ratio (SNR)such that SNR ∈ { 80 , 90 , 100 , 110 , 120 } .
Revised: For the noise level, we consider the Signal-to-Noise Ratio (SNR)such that SNR ∈ { 80 , 90 , 100 , 110 , 120 } .

Original: For the data volume, we fix SNR = 100 and vary
Revised: For the data volume, we fix SNR = 100 and vary

Original: N ∈ { 500 , 1000 , 1500 , 2000 , 2500 } .
Revised: N ∈ { 500 , 1000 , 1500 , 2000 , 2500 } .

Original: Fig.
Revised: Fig.

Original: 5a and 5b demonstrate the results.
Revised: 5a and 5b demonstrate the results.

Original: We find that when
Revised: We find that when

Original: SNR ≥ 100 , the error can be less than 1% .
Revised: SNR ≥ 100 , the error can be less than 1% .

Original: This noise level is suitable to real-word systems.
Revised: This noise level is suitable to real-word systems.

Original: Forexample, SNR = 125 for electric measurements [33].
Revised: Forexample, SNR = 125 for electric measurements [37].

Original: For the data volume, the overall error is lessthan 2% when N ≥ 500 , which shows a robust performance of C O NS O L E .
Revised: For the data volume, the overall error is lessthan 2% when N ≥ 500 , which shows a robust performance of C O NS O L E .

Original:  Conclusions and Future Work
Revised:  Conclusions and Future Work

Original: In this paper, we propose C O NS O L E , a novel Convex Neural Symbolic Learning method, whichenjoys convexity with certain conditions to tackle Symbolic Regression with guaranteed performances.
Revised: In this paper, we propose C O NS O L E , a novel Convex Neural Symbolic Learning method, whichenjoys convexity with certain conditions to tackle Symbolic Regression with guaranteed performances.

Original: Specifically, we convexify the search problem by proposing a double convex deep-Q learning.
Revised: Specifically, we convexify the search problem by proposing a double convex deep-Q learning.

Original: In themeantime, we prove the local and strict convexity of the coefficient estimation in our Locally-Convexequation Learner (L O C
Revised: In themeantime, we prove the local and strict convexity of the coefficient estimation in our Locally-Convexequation Learner (L O C

Original: A L).
Revised: A L).

Original: To our best knowledge, C O NS O
Revised: To our best knowledge, C O NS O

Original: L E is the first method that providesguarantees with reasonable assumptions to learn exact equations.
Revised: L E is the first method that providesguarantees with reasonable assumptions to learn exact equations.

Original: Besides, C O NS O L E has, at aminimum, a broader impact on the following domains.
Revised: Besides, C O NS O L E has, at aminimum, a broader impact on the following domains.

Original: (1) Convex control for physical systems using
Revised: (1) Convex control for physical systems using

Original: Reinforcement Learning.

Original: (2) Neural networks in engineering applications with local convexity.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] We specify assumptions needed
Revised: [Yes] We specify assumptions needed

Original: for good performances in Section 4.
Revised: for good performances in Section 4.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A]
Revised: [N/A]

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] See Section 4.
Revised: [Yes] See Section 4.

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] See Appendix A.2 to A.
Revised: [Yes] See Appendix A.2 toA.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [No] We will release
Revised: [No] We will release

Original: the code if the paper is accepted.
Revised: the code if the paper is accepted.

Original: (b) Did you specify all the training details (e
Revised: (b) Did you specify all the training details (e

Original: g., data splits, hyperparameters,
Revised: g., data splits, hyperparameters,

Original: how they
Revised: how they

Original: were chosen)?
Revised: were chosen)?

Original: [Yes] See Section 5.1.
Revised: [Yes] See Section 5.1.

Original: (c) Did you report error bars (e
Revised: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experi-
Revised: g., with respect to the random seed after running experi-

Original: ments multiple times)?
Revised: ments multiple times)?

Original: [Yes] See Section 5.3.
Revised: [Yes] See Section 5.3.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [N/A]
Revised: [N/A

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...( a)

Original: (a) If your work uses existing assets, did you citethe creators?
Revised: If your work uses existing assets , did you cite

Revised: the

Revised: creators

Original: [N/A]
Revised: [N/A

Original: (b) Did you mention the license of the assets?
Revised: b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]
Revised: [N/A]

Revised: d)

Original: (d) Did you discuss whether and how consent was obtainedfrom people whose data you’re
Revised: Did you discuss whether and how consent was obtained

Original: using/curating?
Revised: from people whose data you’reusing curating

Original: [N/A]
Revised: [N/A

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: e) Did you discuss whether the data you are using/ curating

Original: information or offensive content?
Revised: contains personally identifiableinformation or offensive content

Original: [N/A]
Revised: N/A

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...(

Original: (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?

Original: [N/A]
Revised: [N/A](b)

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N A](

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation [N/A A Appendix

Revised: A.1 Training Algorithm for C O NS O L E .

Original: spent on participant compensation?
Revised: The training algorithm can be seen in Algorithm 1. Input: training dataset { x i , y i } Ni =1 . Initialize: L O C

Original: {+A L layer number K , initial state s 
+}
Revised: {+A L layer number K , initial state s 
+}

Revised: =

Revised: [ 1 , 0 ] T , discount factor γ ∈ (0 , 1) , ϵ for ϵ -greedy strategy, λ as a threshold to stop searching, ICNN for reward function − R ( s , a ) , ICNN for Q-function − Q ( s , a ) , replay buffer B = ∅ , maximum episode T , target network Q ′ ( · ) = Q ( · ) , and target network update interval T 0 .

Revised: while t ≤ T do while k ≤ K do

Revised: Solve Optimization in Equation (3) with − Q ( s tk , a ) to obtain ˜ a ∗ k .

Revised: Use ϵ -greedy to select ˜ a tk from ˜ a ∗ k and a random action.

Revised: ▷ ϵ -greedy strategy.

Revised: Discretize ˜ a tk to obtain a tk .

Revised: Execute a t k and use Equation (1) to obtain s k k +1 .

Revised: Check if a tk and s kk +1 satisfy certain constraints.

Revised: Otherwise, delete this state transition and restart the iteration from s tk .

Revised: ▷ Constraint checking.

Revised: Formulate L O C

Revised: A L, train L O C

Revised: A L with { x i , y i } Ni =1 , and calculate R t .

Revised: Train the reward function − R ( · ) using training data {{ s tk , a tk } K − 1 k =0 , − R t } .

Revised: ∀ 0 ≤ k ≤ K , insert ( s tk , a tk , s tk +1 , R t ) and ( s tk , ˜ a tk , s tk +1 , R ( s tk , ˜ a tk )) to B 0 .

Revised: Sample a minibatch B 0 ⊂ B for ( s m , a m , s m +1 , R m ) ∈ B do ▷ Experience replay.

Revised: Solve Optimization in Equation (3) with − Q ′ ( s m +1 , a ) to obtain ˜ a m +1 .

Revised: y m

Revised: = R m + γQ ′

Revised: ( s m , a m )

Original: {+Train Q ( · ) using training data { s m +1 , a m +1 , y m } m if t mod T 0 = 
+}
Revised: {+Train Q ( · ) using training data { s m +1 , a m +1 , y m } m if t mod T 0 = 
+}

Revised: then if | R t − 1 | ≤ λ then

Revised: End the search process.

Revised: Output: L O C A L with the best performance and the corresponding equations.

Revised: A.2 Proofs of Theorem 1

Revised: Theorem.

Revised: ∀ 0 ≤ k ≤ K − 1 , the negative optimal Q-function − Q ∗ ( s k , ˜ a k ) in the proposed

Revised: C O NS O L E framework exists and is convex in s k and ˜ a k , where s k is the discrete state and ˜ a k is thecontinuous action at the k th stage.

Revised: Proof.

Revised: First, we show our state transition satisfies the Markov property.

Revised: Specifically, Equation (1)in our paper shows that the next state s k +1 equals the matrix multiplication between the currentstate s k and the matrix Z k that is a matricization of the current action a k , where k is the index ofthe state.

Revised: Therefore, the state transition satisfies Markov property with the transition probability

Revised: P ( s k +1 | s k , a k ) = 1 .

Revised: Due to the Markov property of the state transition, we define our search process as Controlled Markov

Revised: Process (CMP) [32, 31].

Revised: By the CMP definition [31], our CMP is composed of our state, action, statetransition probability, a discounter factor γ , and a start state (i.e., s 0 in Equation (1)).

Revised: In general,

Revised: CMP is a Markov Decision Process (MDP) without a reward function [32].

Revised: For one CMP, Trajectory Ordering (TO) ranks trajectories of state action pairs [32].

Revised: In our paper,we define the trajectory from ( s 0 , a 0 ) to ( s K − 1 , a K − 1 ) for K -layer L O C A L.

Revised: Then, our rewardfunction R ( · ) realizes a TO for our defined trajectories [32] since the ordering of trajectories can bedetermined by R ( · ) .

Revised: More specifically, R ( · ) is trained with the end-of-trajectory reward R t for the t th trajectory in our paper and can rank trajectories.

Revised: A reward bundle is an automation-like structureto produce rewards for a CMP [31].

Revised: By Corollary 2 of [31], there exists a reward bundle for ourdefined CMP and TO realized by R ( · ) .

Revised: We pair our CMP with the reward bundle to form a Split Partially Observable MDP (Split-POMDP)[31].

Revised: Then, by Proposition 1 and Corollary 1 in [31], our Split-POMDP will always have an optimaldeterministic policy that only depends on states in our CMP.

Revised: By the proof of Proposition 1 in [31],the optimal policy optimizes the value function over states in CMP.

Revised: Further, the value function isan evaluation of trajectories for our TO by the proof in Corollary 2 in [31].

Revised: Additionally, our TO isrealized by our proposed reward function R ( · ) .

Revised: Therefore, the optimal Q-function exists for our CMPand our proposed R ( · ) .

Revised: Then, we consider the Bellman Equation of Q ∗ ( · ) :

Revised: [Equation]

Revised: where the second equality holds since our state transitions are deterministic by Equation (1).

Revised: Weprove the convexity from the induction method.

Revised: When k = K − 1 , the ( k + 1) th state is the terminalstate without action selections.

Revised: Thus, we have

Revised: [Equation]

Revised: Since − R ( · ) is an ICNN and is convex in input, − Q ∗ ( s K − 1 , ˜ a K − 1 ) is convex in s K − 1 and ˜ a K − 1 .

Revised: When 0 ≤ k < K − 1 and assume − Q ∗ ( s k +1 , ˜ a k +1 ) is convex in s k +1 and ˜ a k +1 , we have− max ˜ a Q ∗ ( s k +1 , ˜ a )

Revised: =

Revised: min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in s k +1 given the fixed optimal action.

Revised: Let

Revised: H denote the Hessian matrix of min ˜ a − Q ∗ ( s k +1 , ˜ a ) with respect to s k +1 .

Revised: Due to the convexity,

Revised: H is positive semi-definite.

Revised: Thus, by Equation (1) and the chain rule, the Hessian matrix ofmin ˜ a − Q ∗ ( s k +1 , ˜ a ) with respect to s k can be written as:

Revised: [Equation]

Revised: H ′ is also positive semi-definite.

Revised: Therefore, min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in s k .

Revised: Since − R ( s k , ˜ a k )is convex in s k , − Q ∗ ( s k , ˜ a k ) is convex in s k .

Revised: Similarly, vectorizing the state transition equation can give:

Revised: [Equation]

Revised: where I n s is the n s × n s identity matrix and (cid:78) is the Kronecker product.

Revised: a ′ k

Revised: =

Revised: [( a k ) T , 0 ] T is theconcatenation of the discrete action a k and a zero vector to maintain the fixed dimensionality ofaction vectors.

Revised: With similar proofs based on the Hessian matrix and the fact that − Q ∗ ( s k +1 , ˜ a k ) isconvex in s k +1 , we have min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in a ′ k and also a k .

Revised: Subsequently, arbitrary˜ a k ∈ conv ( { 0 , 1 } n a ) can be written as a convex combination of the discrete actions a k .

Revised: Thus,min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in ˜ a k .

Revised: Since − R ( s k , ˜ a k ) is convex in ˜ a k , − Q ∗ ( s k , ˜ a k ) is convex in˜ a k .

Revised: Eventually, − Q ∗ ( s k , ˜ a k ) is convex in s k and ˜ a k , which concludes the proof.

Revised: ■A.3 Proofs of Theorem 2

Revised: Theorem.

Revised: Let f ∗ ( · ; W ) denote the L O C A L constructed by the optimal sequences of states( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) from − Q ∗ ( · ) , where W is the set of weightsof f ∗ ( · ; W ) .

Revised: If f ∗ ( · ; W ) can be trained with noiseless datasets and the training can achieve theglobal optimal weights W ∗ , f ∗ ( · ; W ∗ ) can be simplified to the true equation g ( · ) .

Revised: Proof.

Revised: If f ∗ ( · ; W ) can’t represent the exact equations, there are two cases: (1) the structure off ∗ ( · ; W ) is correct to represent the equations, but the learned weights W ∗ don’t represent thesymbol coefficients, and (2) the structure of f ∗ ( · ; W ) can’t represent the equations.

Revised: Case (1) doesn’thold since we assume W ∗ is the global optimal weights for noiseless data.

Original: {+If case (2) holds,∃ 
+}
Revised: {+If case (2) holds,∃ 
+}

Revised: ≤ j ≤ K − 1 , b ∗ j = min ˜ a j − Q ∗ ( s j , ˜ a j ) and b ∗ j doesn’t represent the symbol connections in theunderlying equations.

Revised: Further, we assume ∀ 0 ≤ i < j , a ∗ i =

Revised: min

Revised: ˜ a i − Q ( s i , ˜ a i ) and a ∗ i representsthe true connections.

Revised: If j = K − 1 , Equation (4) implies that ˜ a ∗ j = min ˜ a j − Q ∗ ( s j , ˜ a j ) = arg min ˜ a − R ( s j , ˜ a ) .

Revised: Since− R ( s j , ˜ a ) is convex in ˜ a

Revised: we know the discrete version of ˜ a ∗ j , namely a ∗ j , represents the trueconnection of the last layer for the underlying equations.

Revised: Otherwise, the reward is not maximized.

Revised: However, by definition of b ∗ j , b ∗ j ̸ = a ∗ j .

Revised: If j < K − 1 , Equation (4) implies:

Revised: [Equation]

Revised: By definition of b ∗ j , b ∗ j is not the solution of Equation (5).

Revised: This is because b ∗ j can’t achieve theminimum value for each summation term on the right hand side of Equation (5), according to theconvexity of the reward function.

Revised: In general, b ∗ j ̸ = min ˜ a j − Q ∗ ( s j , ˜ a j ) , which contradicts thedefinition of b ∗ j .

Revised: Thus, b ∗ j doesn’t exist.

Revised: Therefore, case (2) doesn’t hold and f ∗ ( · ; W ∗ ) representsthe exact equations.

Revised: ■

Revised: A.4 Proofs of Theorem

Revised: Theorem.

Revised: Assume the following conditions hold: (1) the equation g ( x ) is C 2 smooth and hasbounded second derivatives with respect to weights, (2) ∃ x ∈ X , g ( x ) has non-zero gradients withrespect to weights, (3) the structure of L O C A L is correctly searched to exactly represent symbolsand symbol connections in g ( x ) , and (4) the training dataset of L O C

Revised: A L is noiseless.

Revised: Then, for the

Revised: MSE loss surface of L O C A L , each global optimal point has a strictly convex local region.

Revised: Proof.

Revised: To simplify the proof, we consider scalar output of the L O C

Revised: A L, i.e., one equation, and theproof can be easily extended to the multi-output case.

Revised: We follow the idea of [29] to study the secondderivative of L O C A L with perturbations.

Revised: Let ˆ y ( x , W ) denote the L O C A L with input to be x and theweight set to be W .

Revised: Let X be a perturbation direction of W and t be a small step size.

Revised: For the i thnoiseless instance ( x i , y i ) , we denote e ( x i , W + tX )

Revised: =

Revised: ˆ y ( x i , W + tX ) − y i .

Revised: Obviously, the lossfunction can be written as L ( W + tX )

Revised: = 12 N (cid:80) Ni

Revised: =1 ( e ( x i , W + tX )) 2 .

Revised: Then, we can calculate thesecond-order derivative based on the chain rule:

Revised: [Equation]

Revised: Next, we denote the global optimal solution to be W ∗ .

Revised: Based on the Assumptions (3)and (4) , ∀ i, ˆ y ( x i , W ∗ )

Revised: =

Revised: g ( x i ) =

Revised: y i .

Original: {+Therefore, we have d 2 dt 2 (cid:12)(cid:12) t =
+}
Revised: {+Therefore, we have d 2 dt 2 (cid:12)(cid:12) t =
+}

Revised: L ( W ∗ + tX ) =

Revised: N

Revised: (cid:0) t

Revised: ˆ y ( x i , W ∗ ) (cid:1) 2 > 0 , where the inequality strictly holds.

Revised: This is because by Assumptions (3) , ˆ y ( x , W ∗ ) can be mathematically simplified to obtain g ( x ) .

Revised: Then, by Assumption (2) ,

Revised: (cid:0)

Revised: (cid:12)(cid:12) t

Original: {+=
+}
Revised: {+=
+}

Revised: ˆ y

Revised: ( x i , W ∗ ) (cid:1) 2 > 0 .

Original: {+Finally, by Assumption (1) and (3) , d 2 dt 2 (cid:12)(cid:12) t =
+}
Revised: {+Finally, by Assumption (1) and (3) , d 2 dt 2 (cid:12)(cid:12) t =
+}

Revised: ˆ y

Revised: ( x i , W + tX ) isbounded and there is a local region around W ∗ such that d 2 dt 2 (cid:12)(cid:12) t

Original: {+=
+}
Revised: {+=
+}

Revised: L ( W + tX ) > 0 , which concludesthe proof.

Revised: ■

Revised: A.5 Proofs of Theorem

Revised: Theorem.

Revised: Suppose Assumptions 1-4 in Theorem 3 hold.

Revised: For a L O C A L with one symbolic activation,multiplication, and summation layer, the set of local convex regions with global optima is U

Revised: =

Revised: { W (cid:12)(cid:12)(cid:12) (cid:12)(cid:12) ddt (cid:12)(cid:12) t

Original: {+=
+}
Revised: {+=
+}

Revised: ˆ y

Revised: ( x i ,W + tX ) (cid:12)(cid:12)η (cid:12)(cid:12) ddt (cid:12)(cid:12) t

Original: {+=
+}
Revised: {+=
+}

Revised: ˆ y ( x j ,W + tX ) (cid:12)(cid:12) >

Revised: | ˆ y ( x k , W ) − y k |} , where notations are defined in the proof.

Revised: Proof.

Revised: For the target L O C A L, we similarly consider the scalar output and write the function analytically:

Revised: [Equation]

Revised: where W 0 ∈ R n 0 × n 1 is the weight matrix for activation, Φ : R n 1 → R n 1 represents the activationwith symbol functions like x 2 , cos ( x ) , and log( x ) , etc.

Revised: Ψ : R n 1 → R n 2 is the function to selectsome activated neurons for multiplications, and W 1 ∈ R n 2 × n 3 ( n 3 = 1 ) represents the weight forsummation.

Revised: We rewrite Equation (7) with the help of exponential and logarithm mappings.

Revised: [Equation]

Revised: where S ∈ R n 1 × n 2 represents a selection matrix such that S [ i, j ] = 1 if and only if the i th neuronis selected as the multiplicative factor for the j th neuron in the multiplication layer.

Revised: Given thefixed structure of ˆ y ( · ) from the deep Q-learning, S is a known matrix.

Revised: log( · ) and exp( · ) representthe element-wise logarithm and exponential functions.

Revised: Notably, the corresponding element inΦ( W T 0 x ) should be positive in Equation (8).

Revised: If there are negative entries, one can utilize W T 1 s ◦exp (cid:18)

Revised: S T log (cid:16) | Φ( W T 0 x ) | (cid:17) (cid:19) to take place of the right hand side term in Equation (8), where s [ i ] =( − 1) n i − and 0 ≤ n i − ≤ n 1 represents the number of negative entries selected for the i th neuron ofthe multiplication layer.

Revised: ◦ represents the Hadamard product.

Revised: However, both expressions have thesame values and gradients.

Revised: Thus, we utilize Equation (8) in later derivations.

Revised: Then, let X be a perturbation direction such that X

Revised: = { X 0 , X 1 } .

Revised: Thus, for a small step t , we have:

Revised: [Equation]

Revised: Based on Equation (9), we can compute:

Revised: [Equation]

Revised: where 1 Φ(( W 0 + t X 0 )

Revised: T x i ) ∈ R n 1 is the element-wise division and Φ ′ is the element-wise first derivativeof Φ ′ .

Revised: Without special notifications, we assume all the division for vectors is element-wise in thefollowing derivations.

Revised: Then, we denote

Revised: [Equation]

Revised: With above definitions, we can calculate:

Original: [N/A]
Revised: [Equation]

Revised: Further, we calculate the second derivative based on Equation (10) and the fact that element-wiseoperations for vectors are commutative:

Revised: [Equation]

Revised: When t →

Revised: [Equation]

Revised: we have:

Revised: [Equation]

Revised: The above equation can reflect the relationship between the second and the first derivative.

Revised: However,we first identify the inequality between these two derivatives to enable a strictly convex region.

Revised: Let ˆ

Revised: [Equation]

Revised: N , W + tX )]

Revised: T , and e =

Revised: [ e ( x 1 , W ) , · · · , e ( x N , W )] T .

Revised: Equation (6) impliesthat:

Revised: [Equation]

Revised: To find a region to restrict the convexity, we restrict the lower bound of the second derivative to bepositive and compute:

Revised: [Equation]

Revised: [Equation]

Revised: The right hand side of Equation (16) can be easily bounded by:

Revised: [Equation]

Revised: where |·| for a vector is to calculate the absolute value for each element of the vector, i = arg min( | ˆ y ′ | )and j = arg max( | ˆ y ′′ | ) .

Revised: Namely, we consider a sufficient condition for convexity.

Revised: [Equation]

Revised: Next, Equation (14) indicates that:

Revised: [Equation]

Revised: where η is a positive constant.

Revised: Note that η < ∞ by Assumptions (1) and (2) in Theorem 3.

Original: {+Therefore,we have the following sufficient condition to make d 2 dt 2 (cid:12)(cid:12) t =
+}
Revised: {+Therefore,we have the following sufficient condition to make d 2 dt 2 (cid:12)(cid:12) t =
+}

Revised: L ( W + tX ) > 0 always hold.

Revised: [Equation]

Revised: where k = arg max( | e | )

Revised: The above equation leads to a set U of local regions that have strongconvexity.

Revised: Namely,

Revised: [Equation]

Revised: Clearly, the global optimal solution W ∗ ∈ U since ˆ y ( x k , W ∗ ) − y k = 0 .

Revised: Note that there may bemultiple global optimal solutions of the loss minimization in L O C A

Revised: L. Thus, U is the set of localconvex regions that contain global optima.

Revised: This implies that for each W ∗ ∈

Revised: U , we can find a locallyand strictly convex region U ∗ =

Revised: U ∩ B ( r ) , where B ( r ) =

Revised: ||

Revised: w − w ∗

Revised: ||

Revised: 2 ≤ r is a norm ball and wevectorize W and W ∗ to obtain w and w ∗ , respectively.

Revised: Subsequently, range r can be set relativelylarge such that U ∗ ⊂ B ( r ) and U ∗∗ ∩ B ( r ) = ∅ , where U ∗∗ is the local region for another globaloptimal point W ∗∗ if it exists.

Revised: Then, the range for U ∗ still depends on the inequality in Equation(21).

Revised: ■