Original: C O NS O L E : Convex Neural Symbolic Learning
Revised: C O NS O L E : Convex Neural Symbolic Learning

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Learning the underlying equation from data is a fundamental problem in manydisciplines.
Revised: Learning the underlying equation from data is a fundamental problem in manydisciplines.

Original: Recent advances rely on Neural Networks (NNs) but do not providetheoretical guarantees in obtaining the exact equations owing to the non-convexityof NNs.
Revised: Recent advances rely on Neural Networks (NNs) but do not providetheoretical guarantees in obtaining the exact equations owing to the non-convexityof NNs.

Original: In this paper, we propose Convex Neural Symbolic Learning (C O NS O L E )to seek convexity under mild conditions.
Revised: In this paper, we propose Convex Neural Symbolic Learning (C O NS O L E )to seek convexity under mild conditions.

Original: The main idea is to decompose therecovering process into two steps and convexify each step.
Revised: The main idea is to decompose therecovering process into two steps and convexify each step.

Original: In the first step ofsearching for right symbols, we convexify the deep Q-learning.
Revised: In the first step ofsearching for right symbols, we convexify the deep Q-learning.

Original: The key is tomaintain double convexity for both the negative Q-function and the negative rewardfunction in each iteration, leading to provable convexity of the negative optimal Qfunction to learn the true symbol connections.
Revised: The key is tomaintain double convexity for both the negative Q-function and the negative rewardfunction in each iteration, leading to provable convexity of the negative optimal Qfunction to learn the true symbol connections.

Original: Conditioned on the exact searchingresult, we construct a Locally-Convex equation Learner (
Revised: Conditioned on the exact searchingresult, we construct a Locally-Convex equation Learner (

Original: L O C
Revised: L O C

Original: A L) neural networkto convexify the estimation of symbol coefficients.
Revised: A L) neural networkto convexify the estimation of symbol coefficients.

Original: With such a design, we quantifya large region with strict convexity in the loss surface of L O C A L for commonlyused physical functions.
Revised: With such a design, we quantifya large region with strict convexity in the loss surface of L O C A L for commonlyused physical functions.

Original: Finally, we demonstrate the superior performance of the
Revised: Finally, we demonstrate the superior performance of the

Original: C O NS O L E framework over the state-of-the-art on a diverse set of datasets.
Revised: C O NS O L E framework over the state-of-the-art on a diverse set of datasets.

Original:  Introduction
Revised:  Introduction

Original: Identifying the underlying mathematical expressions from data plays a key role in multiple domains.
Revised: Identifying the underlying mathematical expressions from data plays a key role in multiple domains.

Original: For example, scientific discovery naturally requires learning analytical solutions to fit data.
Revised: For example, scientific discovery naturally requires learning analytical solutions to fit data.

Original: Engineering systems also need to frequently re-estimate system equations due to events, maintenance,upgrading and new constructions [1] etc.
Revised: Engineering systems also need to frequently re-estimate system equations due to events, maintenance,upgrading and new constructions [1] etc.

Original: In general, the problem, known as Symbolic Regression (SR) [2], learns the underlying equations y
Revised: In general, the problem, known as Symbolic Regression (SR) [2], learns the underlying equations y

Original: = g ( x ) constructed via certain symbols, where xand y represent the input/output vector of the equations.
Revised: = g ( x ) constructed via certain symbols, where xand y represent the input/output vector of the equations.

Original: If successfully learned, the equation canenjoy many important properties like high interpretability and generalizability, which will in turnsignificantly benefit scientific understanding and engineering planning, monitoring and control.
Revised: If successfully learned, the equation canenjoy many important properties like high interpretability and generalizability, which will in turnsignificantly benefit scientific understanding and engineering planning, monitoring and control.

Original: Promising as it might be, SR is NP-hard [3, 4].
Revised: Promising as it might be, SR is NP-hard [3, 4].

Original: Mathematically, one can cast SR as an optimizationproblem over both discrete variables to select symbols and continuous variables to represent thesymbol coefficients.
Revised: Mathematically, one can cast SR as an optimizationproblem over both discrete variables to select symbols and continuous variables to represent thesymbol coefficients.

Original: Traditional solutions employ evolutionary algorithms like Genetic Programming(GP)
Revised: Traditional solutions employ evolutionary algorithms like Genetic Programming(GP)

Original: These methods start from an initial set of expressions and continue evolving via operationslike crossover or mutation.
Revised: These methods start from an initial set of expressions and continue evolving via operationslike crossover or mutation.

Original: With fitness measures, GP-based algorithms can evaluate and select thebest equations.
Revised: With fitness measures, GP-based algorithms can evaluate and select thebest equations.

Original: However, these methods have poor scalability and limited theoretical guarantees [3].
Revised: However, these methods have poor scalability and limited theoretical guarantees [3].

Original: More recent SR studies leverage Neural Networks (NNs) with high representational power.
Revised: More recent SR studies leverage Neural Networks (NNs) with high representational power.

Original: For the
Revised: For the

Original: NN-based SR, we mainly categorize them into two groups based on the roles of the NNs.
Revised: NN-based SR, we mainly categorize them into two groups based on the roles of the NNs.

Original: The firstgroup employs NNs to directly model the equations, where sparsity of the NN weights is enforced toselect symbols [6–10].
Revised: The firstgroup employs NNs to directly model the equations, where sparsity of the NN weights is enforced toselect symbols [6–10].

Original: Thus, the problem is transformed into training the designed NN with sparsityregularization.
Revised: Thus, the problem is transformed into training the designed NN with sparsityregularization.

Original: However, due to the non-convexity of NNs, the weight selection and updating can beeasily stuck in local optima, failing to find the exact equations.
Revised: However, due to the non-convexity of NNs, the weight selection and updating can beeasily stuck in local optima, failing to find the exact equations.

Original: The second group employs NNs to search the symbol connections, and non-linear optimizations like
Revised: The second group employs NNs to search the symbol connections, and non-linear optimizations like

Original: BFGS [11] can be employed to estimate symbol coefficients.
Revised: BFGS [11] can be employed to estimate symbol coefficients.

Original: For the searching procedure, [3, 12–14] leverage a Recursive Neural Network (RNN) as a policy network to iteratively generate optimalactions that can select and connect symbols.
Revised: For the searching procedure, [3, 12–14]leverage a Recursive Neural Network (RNN) as a policy network to iteratively generate optimal actions that can select and connect symbols.

Original: [15] employs large-scale pre-training to directly mapfrom data to the symbolic equations.
Revised: [15] employs large-scale pre-training to directly mapfrom data to the symbolic equations.

Original: While these methods restrict the utilization of NNs in thesearch phase, the non-convexity of NNs can still suffer the risk of sub-optimal decisions to formulateequations.
Revised: While these methods restrict the utilization of NNs in thesearch phase, the non-convexity of NNs can still suffer the risk of sub-optimal decisions to formulateequations.

Original: To mitigate this issue, some efforts have been made such as risk-seeking policy gradient tofind the best equations [3], restricting the searching space via domain knowledge [16] and entropyregularization [13], and re-initialization [13], etc.
Revised: To mitigate this issue, some efforts have been made such as risk-seeking policy gradient tofind the best equations [3], restricting the searching space via domain knowledge [16] and entropyregularization [13], and re-initialization [13], etc.

Original: However, they have limited theoretical guarantees.
Revised: However, they have limited theoretical guarantees.

Original: In this paper, we propose Convex Neural Symbolic Learning (C O NS O L E ) with convexity undermoderate conditions.
Revised: In this paper, we propose Convex Neural Symbolic Learning (C O NS O L E ) with convexity undermoderate conditions.

Original: To our best knowledge, we are the first to provide provable guarantees to learnthe exact equations.
Revised: To our best knowledge, we are the first to provide provable guarantees to learnthe exact equations.

Original: In general, we decompose the SR problem into two sub-problems and seekconvexity, respectively.
Revised: In general, we decompose the SR problem into two sub-problems and seekconvexity, respectively.

Original: In the first problem of searching symbols, we propose a double-convexifieddeep Q-learning to maintain the convexity of negative Q-function and negative reward functionswith continuous action variables.
Revised: In the first problem of searching symbols, we propose a double-convexifieddeep Q-learning to maintain the convexity of negative Q-function and negative reward functionswith continuous action variables.

Original: Specifically, we use the Input Convex Neural Network (ICNN)[17] to represent both the negative Q-function and the negative reward function in each iteration.
Revised: Specifically, we use the Input Convex Neural Network (ICNN)[17] to represent both the negative Q-function and the negative reward function in each iteration.

Original: Subsequently, we prove that such a design (1) guarantees an optimal action selection at each stepand (2) ensures the negative optimal Q-function, if successfully found, to be convex.
Revised: Subsequently, we prove that such a design (1) guarantees an optimal action selection at each stepand (2) ensures the negative optimal Q-function, if successfully found, to be convex.

Original: Therefore, theconvex negative optimal Q-function can yield global optimal decisions of equation constructions.
Revised: Therefore, theconvex negative optimal Q-function can yield global optimal decisions of equation constructions.

Original: In the second problem of coefficient estimation, we use the search result to build a Locally-Convex
Revised: In the second problem of coefficient estimation, we use the search result to build a Locally-Convex

Original: Equation Learner (L O C A L) neural network.
Revised: Equation Learner (L O C A L) neural network.

Original: The key insight is that if the search result is correct,the loss surface of L O C A L has local regions that contain the global optima and has strict convexity.
Revised: The key insight is that if the search result is correct,the loss surface of L O C A L has local regions that contain the global optima and has strict convexity.

Original: Moreover, we quantify the local regions and show the range of the region is large under mildassumptions.
Revised: Moreover, we quantify the local regions and show the range of the region is large under mildassumptions.

Original: Therefore, initializations based on prior knowledge can often lie in the convex region,bringing the accurate coefficient estimation.
Revised: Therefore, initializations based on prior knowledge can often lie in the convex region,bringing the accurate coefficient estimation.

Original: Finally, we demonstrate that our C O NS O L E outperformsstate-of-the-art methods on a diverse set of datasets.
Revised: Finally, we demonstrate that our C O NS O L E outperformsstate-of-the-art methods on a diverse set of datasets.

Original:  Related Work
Revised:  Related Work

Original: Symbolic regression using neural networks.
Revised: Symbolic regression using neural networks.

Original: In addition to the direct use of NN in the Introduction,there are studies forthe indirect use.
Revised: In addition to the direct use of NN in the Introduction,there are studies for indirect use.

Original: They typically treat NNs as a data augmentation tool to createhigh-quality data for learning or evaluation [18, 2].
Revised: They typically treat NNs as a data augmentation tool to createhigh-quality data for learning or evaluation [18, 2].

Original: Neural architecture search.
Revised: Neural architecture search.

Original: Searching the connections of L O C A L falls into the area of Neural
Revised: Searching the connections of L O C A L falls into the area of Neural

Original: Architecture Search (NAS).
Revised: Architecture Search (NAS).

Original: NAS tries to find an optimal architecture of a target NN with the bestperformance [19].
Revised: NAS tries to find an optimal architecture of a target NN with the bestperformance [19].

Original: The searching algorithms can be divided into RL-based, evolutionary algorithmbased, sequential optimization-based, and gradient descent-based methods.
Revised: The searching algorithms can be divided into RL-based, evolutionary algorithmbased, sequential optimization-based, and gradient descent-based methods.

Original: For RL-based methods,[20] employs an RNN model to sample the architecture and utilizes the accuracy of the samplednetwork as the reward.
Revised: For RL-based methods,[20] employs an RNN model to sample the architecture and utilizes the accuracy of the samplednetwork as the reward.

Original: [21] uses tabular Q-learning to find the connections of a target NN.
Revised: [21] uses tabular Q-learning to find the connections of a target NN.

Original: However,tabular Q-learning can hardly be applicable when the state and action spaces are large, e.g., SRproblem.
Revised: However,tabular Q-learning can hardly be applicable when the state and action spaces are large, e.g., SRproblem.

Original: The evolutionary algorithm employs methods such as GP [22] and tournament selection [23].
Revised: The evolutionary algorithm employs methods such as GP [22] and tournament selection [23].

Original: However, these methods may lack the scalability for SR problem [3].
Revised: However, these methods may lack the scalability for SR problem [3].

Original: The sequential optimizationbased method is more scalable as the model complexity increases in a sequential manner [24].
Revised: The sequential optimizationbased method is more scalable as the model complexity increases in a sequential manner [24].

Original: Finally,the gradient descent-based method [25] builds a large and over-parameterized network to search andtrain.
Revised: Finally,the gradient descent-based method [25] builds a large and over-parameterized network to search andtrain.

Original: Then, regularizations like dropout are added to find the best connections.
Revised: Then, regularizations like dropout are added to find the best connections.

Original: However, for thesemethods, the theoretical guarantees remain opaque for SR problem.
Revised: However, for thesemethods, the theoretical guarantees remain opaque for SR problem.

Original: Global optimum in neural networks.
Revised: Global optimum in neural networks.

Original: Many studies have been conducted to seek global optimalityin NNs, and they can be categorized into finding global optima for weights or input variables.
Revised: Many studies have been conducted to seek global optimalityin NNs, and they can be categorized into finding global optima for weights or input variables.

Original: Theweight optimization is directly linked to finding symbol coefficients in L O C A L. Specifically, [26]investigates a single hidden layer with unbounded neurons and non-Euclidean regularization.
Revised: Theweight optimization is directly linked to finding symbol coefficients in L O C A L. Specifically, [26]investigates a single hidden layer with unbounded neurons and non-Euclidean regularization.

Original: Theauthors show the training can be done via convex optimization problems.
Revised: Theauthors show the training can be done via convex optimization problems.

Original: [27] considers finite neuronsand develops a novel duality theory to train two-layer NNs with convexity.
Revised: [27] considers finite neuronsand develops a novel duality theory to train two-layer NNs with convexity.

Original: [28] establishes a strongresult that every local optimum is a global optimum for deep non-linear networks under severalassumptions.
Revised: [28] establishes a strongresult that every local optimum is a global optimum for deep non-linear networks under severalassumptions.

Original: [29] eliminates these assumptions and finds that with weight decay regularization(e.g.
Revised: [29] eliminates these assumptions and finds that with weight decay regularization(e.g.

Original: , l 2 norm), the loss function of NN with ReLU activations is piece-wise strongly convex in localregions.
Revised: , l 2 norm), the loss function of NN with ReLU activations is piece-wise strongly convex in localregions.

Original: However, L O C A L doesn’t fit the above conditions.
Revised: However, L O C A L doesn’t fit the above conditions.

Original: The second group of input variableoptimization can help search for optimal inputs.
Revised: The second group of input variableoptimization can help search for optimal inputs.

Original: [17] proposes ICNN such that the output of ICNN isconvex in input variables.
Revised: [17] proposes ICNN such that the output of ICNN isconvex in input variables.

Original: The key of ICNN is to restrict some weights and activation functions topreserve convexity.
Revised: The key of ICNN is to restrict some weights and activation functions topreserve convexity.

Original: ICNN facilitates to design a convexified search algorithm.
Revised: ICNN facilitates to design a convexified search algorithm.

Original:  Convex Neural Symbolic Learning
Revised:  Convex Neural Symbolic Learning

Original: L O C A L to Hierarchically Represent the Equations and Learn Symbol Coefficients
Revised: L O C A L to Hierarchically Represent the Equations and Learn Symbol Coefficients

Revised: SR problem can be decomposed into searching the symbol connections and estimating the symbolcoefficients.

Revised: The link of these two sub-problems is a proper representation of the underlying equationwith unknown structures and weights.

Revised: Then, we need to search the structures and estimate the weights.

Original: We utilize a neural network to represent the multi-input multi-output equations due to the efficiency[6].
Revised: We utilize a neural network to represent the multi-input multi-output equations due to the efficiency[6].

Original: We prove in Theorem 3 that under certain conditions, there are local regions in the loss surfacewith strict convexity.
Revised: We prove in Theorem 3 that under mild conditions and given correct structures of the NN, thereare local regions in the loss surface with strict convexity.

Original: Thus, we name our NN as Locally Convex Equation Learner (L O C 
Revised: Thus, we name our NN as Locally Convex Equation Learner (L O C A L).

Original:  A L).
Revised: L O C A L should have the capacity to represent the true equation.

Original: We assume the underlying equation y
Revised: We assume the underlying equationy =

Original: = g ( x ) follows compositionality and smoothness assumptionsin [2], which are often the case in physics and many other scientific applications.
Revised: g ( x ) follows compositionality and smoothness assumptions in [2], which are often the casein physics and many other scientific applications.

Original: Then, we build L O C A L that can be trained via input/output pairs { x i , y i } Ni =1 , where x i ∈ Xand
Revised: Then, we build L O C A L that can be trained viainput/output pairs { x i , y i } Ni =1 , where x i ∈ X

Original:  y i ∈ Y are the i thsamples and N is the number of data.
Revised: and y i ∈ Y are the i th samples and N is the numberof data.

Original: X and Y are the input and output data spaces, respectively.
Revised: X and Y are the input and output data spaces, respectively.

Original: Bythe compositionality, we propose to hierarchically map x to y with correct symbols.
Revised: By the compositionality, wepropose to hierarchically map x to y with correct symbols.

Original: Fig.
Revised: Fig.

Original: 1 showsan example of the L O C A L structure.
Revised: 1 shows an example of the L O C A Lstructure.

Original: Specifically, each input entry first goes through the activationfunctions from a symbol library Φ .
Revised: Specifically, each input entry first goes through the activation functions from a symbollibrary Φ .

Original: For example, in Fig. 1, we denote Φ =
Revised: For example, in Fig. 1, we denote Φ =

Original: { x, x 2 , cos( wx ) } tocorrespond with three neurons from top to bottom in the dotted black box, where w is a learnableweight.
Revised: { x, x 2 , cos( wx ) } to correspond with three neuronsfrom top to bottom in the dotted black box, where w is a learnable weight.

Original: As for weights of x and x 2 , they only need to appear in the later summation layer.
Revised: As for weights of x andx 2 , they only need to appear in the later summation layer.

Original: Then, someactivation outputs will be selected as multipliers for the multiplication.
Revised: Then, some activation outputs will beselected as multipliers for the multiplication.

Original: Subsequently, the multipliedoutputs are selected and summed together.
Revised: Subsequently, the multiplied outputs are selected andsummed together.

Original: The repetition of symbolic activation, multiplication, andsummation formulates final equations.
Revised: The repetition of symbolic activation, multiplication, and summation formulatesfinal equations.

Original: For instance, the L O C A L structure in Fig.
Revised: For instance, the L O C A L structure in Fig.

Original: 1 can correctlyrepresents y 1 and y 2 shown on the top right of Fig.
Revised: 1 can correctly represents y 1 and y 2 shownon the top right of Fig.

Original: The layer-wise connectivity and the parameters of L O C
Revised: The layer-wise connectivity and the weights of L O C

Original: A L are optimization variables to let L O C A Lfit the training data.
Revised: A L are optimization variables for the SR problem.

Original: Mathematically, we denote Z k ∈ { 0 , 1 } n k × n k +1 and W k ∈ R n k × n k +1 as theindicator and weight matrix between the k th and the ( k + 1) th layer of the L O C A L, respectively.
Revised: Mathematically, we denote Z k ∈ { 0 , 1 } n k × n k +1 and W k ∈ R n k × n k +1 as the indicator and weightmatrix between the k th and the ( k + 1) th layer of the L O C A L, respectively.

Original: n kis the number of neurons for the k th layers.
Revised: n k is the number ofneurons for the k th layers.

Original: Z k
Revised: Z k

Original: [ i, j ] = 1 indicates that the connection exists betweenthe i th neuron in the k th layer and the j th neuron in the ( k + 1) th layer, where Z k [ i, j ] is the ( i, j ) thentry of Z k .
Revised: [ i, j ] = 1 indicates that the connection exists between the i th neuronin the k th layer and the j th neuron in the ( k + 1) th layer, where Z k [ i, j ] is the ( i, j ) th entry of Z k .

Original: We assume there are ( K + 1) layers in L O C A L and denote h k ∈ R n k to be the outputof the k th layer.
Revised: We assume there are ( K + 1) layers in L O C A L and denote h k ∈ R n k to be the output ofthe k th layer.

Original: Naturally, we have h 0 = x
Revised: Naturally, we have h 0 = x

Original: and h K
Revised: and h K

Original: = ˆ y , where ˆ y is the output of L O C A L. Forthe symbolic activation or summation layer, we have h k +

Revised: = ˆ y , where ˆ y is the output of L O C A L. Forthe symbolic activation or summation layer, we have h k +

Original: =
Revised: =

Original: Z Tk ◦ W Tk h k , where ◦ representsthe Hadamard product and helps to zero out some connections.
Revised: Z Tk ◦ W Tk h k , where ◦ representsthe Hadamard product and helps to zero out some connections.

Original: For the multiplication layer, wehave h k +

Revised: For the multiplication layer, wehave h k +

Original: [ j ] = (cid:81)
Revised: [ j ] = (cid:81)

Original: Z k
Revised: Z k

Original: [ i,j ]=1 h k
Revised: [ i,j ]=1 h k

Original: [ i ] with no weight matrix involved.
Revised: [ i ] with no weight matrix involved.

Original: In general, we denote L O C
Revised: In general, we denote L O C

Original: A L asf ( x ; { Z k } K − 1 k =0 , { W k } K − 1 k =0 ) .
Revised: A L asf ( x ; { Z k } K − 1 k =0 , { W k } K − 1 k =0 ) .

Original: The search algorithm identifies { Z k } K − 1 k =

Revised: The search algorithm identifies { Z k } K − 1 k =

Original: and the estimation processlearns the corresponding weights in { W k } K − 1 k =0 using { x i , y i } Ni =1 .
Revised: and the estimation processlearns the corresponding weights in { W k } K − 1 k =0 using { x i , y i } Ni =1 .

Original: 3.2 Search L O C A L Structures with Double Convex Deep Q-Learning
Revised: 3.2 Search L O C A L Structures with Double Convex Deep Q-Learning

Original: In this subsection, we employ the Input Convex Neural Network (ICNN) [17] to design a doubleconvex deep Q-learning with provable performance guarantees.

Original: State and action definitions in the search process.
Revised: State and action definitions in the search process.

Original: To search the structure of L O C A L, we treat thestatus of each layer of L O C A L as a state and the connections between layers, i.e., Z k in L O C A L, asactions.
Revised: To search the structure of L O C A L, we treat thestatus of each layer of L O C A L as a state and the connections between layers, i.e., Z k in L O C A L, as actions.

Original: Specifically, we denote a k ∈ { 0 , 1 } n a as the k th action vector where a k [( i − 1) n k + j ] = 1implies that the connection ij exists for the i th neuron in the k th layer and the j th neuron in the ( k + 1) th layer.
Revised: Specifically, we denote a k ∈ { 0 , 1 } n a as the k th action vector where a k [( i − 1) n k + j ] = 1implies that the connection ij exists for the i th neuron in the k th layer and the j th neuron inthe ( k + 1) th layer.

Original: n a = max { n k n k +1 } K − 1 k =0 is the dimensionality of the action space.
Revised: n a = max { n k n k +1 } K − 1 k =0 is the dimensionality of the action space.

Original: Also,we denote s k ∈
Revised: Also,we denote s k ∈

Original: Z n s as the state vector to represent the current state for the k th layers, wheren s = max { n k } Kk =

Revised: Z n s as the state vector to represent the current state for the k th layers, wheren s = max { n k } Kk =

Original: + 1 is the dimensionality of the state space.
Revised: + 1 is the dimensionality of the state space.

Original: More specifically, the state vectors k is composed of the values for each neuron in the k th layer and an entry of state index k .
Revised: More specifically, the state vectors k is composed of the values for each neuron in the k th layer and an entry of state index k .

Original: Theappending of the value k can avoid the duplicate state vectors for different layers.
Revised: Theappending of the value k can avoid the duplicate state vectors for different layers.

Original: Otherwise, theduplicated states may exist and require different optimal actions, deteriorating the correct search forthe true structure of L O C A L.
Revised: Otherwise, theduplicated states may exist and require different optimal actions, deteriorating the correct search forthe true structure of L O C A L.

Original: To further quantify state vectors, we utilize a linear transformation torepresent the state transition process.
Revised: To further quantify state vectors, we utilize a linear transformation torepresent the state transition process.

Original: ∀ 0 ≤ k ≤ K − 1 , we define
Revised: ∀ 0 ≤ k ≤ K − 1 , we define

Original: [Equation]
Revised: [Equation]

Original: where the number of 1 s in s 0 corresponds to the input dimension.
Revised: where the number of 1 s in s 0 corresponds to the input dimension.

Original: Mat ( · ) is the operation to reshapea vector to a matrix, and we utilize 0 s for padding to maintain the dimensions.
Revised: Mat ( · ) is the operation to reshapea vector to a matrix, and we utilize 0 s for padding to maintain the dimensions.

Original: c =
Revised: c =

Original: [0 , 0 , · · · , 1] Tis a constant vector to increase the entry of the state index from k to k + 1 .
Revised: [0 , 0 , · · · , 1] Tis a constant vector to increase the entry of the state index from k to k + 1 .

Original: We show this linearstate transition is essential to guarantee the convexity of negative optimal Q-function in Theorem1.
Revised: We show this linearstate transition is essential to guarantee the convexity of negative optimal Q-function in Theorem1.

Original: For the calculation example of state transition, one can refer to Fig.
Revised: For the calculation example of state transition, one can refer to Fig.

Original: Then, we can obtain Z kfrom Mat ( a k ) by deleting the filled 0s.
Revised: Then, we can obtain Z kfrom Mat ( a k ) by deleting the filled 0s.

Original: By the designed states and actions, the search will alwaysstart at s 0 and end at s K in one episode.
Revised: Then, the search will always start at s 0 and end at s K in oneepisode.

Original: Thus, we define a trajectory as a sequence of searchedstate-action pairs for { ( s 0 , a 0 ) , ( s 1 , a 1 ) , · · · , ( s K − 1 , a K − 1 ) } .
Revised: Thus, we define a trajectory as a sequence of searched state-action pairs for { ( s k , a k ) } K − 1 k =0 .

Original: This trajectory formulates { Z k } K − 1 k =0in the L O C
Revised: This trajectory formulates { Z k } K − 1 k =0 in the L O C

Original: A L function f ( x ; { Z k } K − 1 k =0 , { W k } K − 1 k =0 ) .
Revised: A L function f ( x ; { Z k } K − 1 k =0 , { W k } K − 1 k =0 ) .

Original: The above states, actions, state transitions, initial state s 0 , and a discount factor γ can form a
Revised: The above states, actions, state transitions, initial state s 0 , and a discount factor γ can form a

Original: Controlled Markov Process (CMP) [30, 31], which is a Markov Decision Process (MDP) withouta reward function [31].
Revised: Controlled Markov Process (CMP) [30, 31], which is a Markov Decision Process (MDP) withouta reward function [31].

Original: In the following part, we define our reward function based on an end-oftrajectory reward [30].
Revised: In the following part, we define our reward function based on an end-oftrajectory reward [30].

Original: Although the reward function is non-Markov, we prove in Theorem 1 thatunder our settings, there exists an optimal Q-function.
Revised: Although the reward function is non-Markov, we prove in Theorem 1 thatunder our settings, there exists an optimal Q-function.

Original: The proof can be seen in Appendix A.3.
Revised: The proof can be seen in Appendix A.3.

Original: Double convex deep Q-learning to search optimal actions.
Revised: Double convex deep Q-learning to search optimal actions.

Original: Training L O C A L can bring an end-oftrajectory reward [3] to evaluate the episode. However, this reward is insufficient to enable a convexevaluation for instant state-action pairs.
Revised: To optimize over the defined CMP foroptimal actions, we seek certain convexity with provable optimal results.

Original: Thus, we propose to restrict the  negative reward function tobe convex.
Revised: Thus, we propose a doubleconvex deep Q-learning with the convex negative reward function and negative value function (i.e., Q-function).

Original: Based on Bellman equations [32], this design will ensure the convexity of the negativeoptimal Q-function.
Revised: Based on Bellman equations [32], this design will ensure the convexity of the negativeoptimal Q-function and global optimal solutions.

Original: More proof details can be referred to Theorem 1 and Appendix A.3.
Revised: More proof details can be referred to Theorem 1and Appendix A.3.

Original: Specifically, we utilize an ICNN to model the reward function − R( s k , a k )
Revised: Specifically, we utilize an Input Convex Neural Network (s) [17] to model the reward function− R

Original:  such that − R ( s k , a k )is convex in states and actions in continuous spaces.
Revised: ( s k , a k ) such that − R ( s k , a k ) is convex in states and actions.

Original: − R ( s k , a k ) requires proper training to do thecorrect evaluation.
Revised: − R ( s k , a k ) requires propertraining to do the correct evaluation.

Original: In the t th episode, we collect the t th trajectory sample { ( s tk , a tk ) } K − 1 k =0 .
Revised: In the t th episode, we collect the t th trajectory sample{ ( s tk , a tk ) } K − 1 k =0 .

Original: Then,the output can be defined as the end-of-trajectory reward to evaluate the obtained L O C A L whichis denoted as f t ( x ; W t ) , where W t is the weight set of L O C A L. Basically, we utilize gradientmethod (e.g., Adam [33]) to train f t ( x ; W t ) and obtain the optimal set of weights W ∗ t for thet th episode of L O C A L by minimizing the Mean Square Error (MSE).
Revised: Then, the output can be defined as the end-of-trajectory reward to evaluate theobtained L O C A L which is denoted as f t ( x ; W t ) , where W t is the weight set of L O C A L. Basically,we utilize gradient method (e.g., Adam [33]) to train f t ( x ; W t ) and obtain the optimal set of weights W ∗ t for the t th episode of L O C A L by minimizing the Mean Square Error (MSE).

Original: Then, we can calculate the Normalized Root-Mean-Square Error (NRMSE) [3] of the trained f t ( x ; W ∗ ) such that NRMSE t =1 σ y (cid:113)
Revised: Then, we cancalculate the Normalized Root-Mean-Square Error (NRMSE) [3] of the trained f t ( x ; W ∗ ) such that NRMSE t = 1 σ y (cid:113)

Original: N (cid:80) N i =

Revised: N (cid:80) Ni =

Original: ( y i − f t (cid:0) x i ; W ∗ t ) (cid:1) 2 , where σ y is the standard deviation of the outputs.
Revised: ( y i − f t (cid:0) x i ; W ∗ t ) (cid:1) 2 , where σ y is the standard deviation of the outputs.

Original: The outputof the reward function can be calculated as R t = 1 1+ NRMSE t .
Revised: The output of the reward function can be calculated as R t = 1 1+ NRMSE t .

Original: Therefore, we can train − R ( · ) using{{ s tk , a tk } K − 1 k =0 , − R t } .
Revised: Therefore, we can train− R ( · ) using {{ s tk , a tk } K − 1 k =0 , − R t } .

Original: It’s important to note that building this convex function − R ( · ) is better than directly leveraging theend-of-trajectory reward for evaluations.
Revised: Although training R ( · ) utilizes the samples of discrete states and actions, we aim to solve thecontinuous convex optimization for optimal sequential decisions. Thus, we first show that R ( · ) canbe defined over the continuous action space.

Original: The reason is that the former process not only providestheoretical guarantees in Theorem 1 but also evaluates the  actions  in a continuous space with convexity.
Revised: By definitions of the discrete actions, we restrict thecontinuous action space in a hypercube conv ( { 0 , 1 } n a ) , i.e., a convex hull of the discrete actions. Then, the discrete actions are the endpoints of the hypercube. Notably, it is doable to utilize acontinuous convex function to fit these endpoints with end-of-trajectory rewards. Especially, thestrictly minimal reward value − R t only lies in the endpoint that guarantees the correct structureof L O C A L. Therefore, one can utilize piece-wise linear functions with convexity to represent R ( · )over conv ( { 0 , 1 } n a ) and ensure one endpoint has the minimal value.

Original: Namely, one can utilize R (  s k , ˜ a k ) to evaluate a continuous action ˜ a k ,given that there is a muchhigher probability to obtain ˜ a k rather than the discrete action  a k in the deep Q-learning.
Revised: The piece-wise linearity can beachieved via using ReLu activations to R ( · ) . With the defined continuous action space, we utilize another ICNN to represent − Q ( s k , ˜ a k ) suchthat − Q ( s k , ˜ a k ) is convex in the continuous action ˜ a k ∈ conv ( { 0 , 1 } n a ) given fixed s k .

Original: Then, toupdate Q values, we have the following iterative computations based on the temporal difference [21]:
Revised: To update Q values, we have the following iterative computations based on the temporal difference [21]:

Original: [Equation]
Revised: [Equation]

Original: where Q t ( s k , ˜ a k ) is the Q value at the t th episode for state s k and action ˜ a k .
Revised: where Q t ( s k , ˜ a k ) is the Q value at the t th episode for state s k and action ˜ a k .

Original: α and γ are pre-definedlearning rate and discount factor, respectively.
Revised: α and γ are pre-definedlearning rate and discount factor, respectively.

Original: Similarly, we utilize another ICNN to represent− Q ( s k , ˜ a k ) such that − Q ( s k , ˜ a k ) is convex in ˜ a k given fixed s k .

Original: Therefore, one can solve a convexoptimization problem max a Q t ( s k +1 , ˜ a ) to obtain the (approximately) global optimal action for Equation (2) in each iteration.
Revised: Therefore, one can solve a convex optimizationproblem max a Q t ( s k +1 , ˜ a ) to obtain the (approximately) global optimal action for Equation (2) ineach iteration.

Original: By definitions of the discrete actions, we can restrict the continuous action space in a hypercube conv ( { 0 , 1 } n a ) , i.e., a convex hull of the discrete actions.

Original: Thus, theoptimization problem is:
Revised: Thus, the optimization problem is:

Original: [Equation]
Revised: [Equation]

Original: Based on [17], this convex optimization problem can be solved via a bundle entropy method.
Revised: Based on [17], this convex optimization problem can be solved via a bundle entropy method.

Original: Afterobtaining a continuous solution ˜ a ∗ , we discretize it to a discrete vector a ∗ to build L O C A L. Onesimple method is to enforce a ∗
Revised: Afterobtaining a continuous solution ˜ a ∗ , we discretize it to a discrete vector a ∗ to build L O C A L. Onesimple method is to enforce a ∗

Original: [ i ] = 

Revised: [ i ] =

Original: if ˜ a ∗ [ i ] ≥ 0 . 5 , and otherwise a ∗ [ i ] = 0 .
Revised: if ˜ a ∗ [ i ] ≥ 0 . 5 , and otherwise a ∗ [ i ] = 0 .

Original: Thus, both
Revised: Thus, both

Original: Q ( s k , ˜ a k ) and Q ( s k , a k ) can be trained using Equation (2).
Revised: Q ( s k , ˜ a k ) and Q ( s k , a k ) can be trained using Equation (2).

Original: Practically, we introduce ϵ -greedystrategy [34], experience replay [35] and a target Q-network [36] to update Q-function, thus boostingthe convergence to the optimal Q-function.
Revised: Practically, we introduce ϵ -greedystrategy [34], experience replay [35] and a target Q-network [36] to update Q-function, thus boostingthe convergence to the optimal Q-function.

Original: The overview of our framework is in Algorithm 1 and thespecific algorithm is in Algorithm 2.
Revised: The overview of our framework is in Algorithm 1 andthe specific algorithm is in Algorithm 2.

Original: Finally, based on Theorems 1-2,solving the convex optimizationvia the optimal Q-function can generate the correct structures of L O C A L and exact equations.
Revised: Finally, based on Theorems 1-2, the discrete version of theoptimal actions can generate the correct structures of L O C A L and exact equations.

Original: Symbolic static and dynamic constraints.
Revised: Symbolic static and dynamic constraints.

Original: Constraints can be added to accelerate the search process[3, 2].
Revised: Constraints can be added to accelerate the search process[3, 2].

Original: For example, in Algorithm 2, we propose a constraint checking program for the state-actionpairs to avoid the invalid search, suitable for arbitrary restrictions.
Revised: For example, in Algorithm 2, we propose a constraint checking program for the state-actionpairs to avoid the invalid search, suitable for arbitrary restrictions.

Original: Then, we emphasize a generaltype of constraint, symbolic constraint, for the SR problem.
Revised: Then, we emphasize a generaltype of constraint, symbolic constraint, for the SR problem.

Original: The symbolic static constraint requiresthat each equation contains only a subset of symbols.
Revised: The symbolic static constraint requiresthat each equation contains only a subset of symbols.

Original: For example, the equation y 1 = x 1 x 2 · · · x 100shouldn’t exist since it is too complicated for real-world systems.
Revised: For example, the equation y 1 = x 1 x 2 · · · x 100shouldn’t exist since it is too complicated for real-world systems.

Original: This constraint eliminates part ofthe action space and can be checked by counting the number of 1 s in the action vector.
Revised: This constraint eliminates part ofthe action space and can be checked by counting the number of 1 s in the action vector.

Original: The symbolicdynamic constraint can gradually reduce the search space based on symbol correlations.
Revised: The symbolicdynamic constraint can gradually reduce the search space based on symbol correlations.

Original: Specifically,we investigate the ( K − 1) th layer’s neurons that are linearly summed to form the equation.
Revised: Specifically,we investigate the ( K − 1) th layer’s neurons that are linearly summed to form the equation.

Original: Ifsome of these neurons have strong linear correlations to the output neuron (e.g., Pearson correlationcoefficient larger than 0 . 99 ), they should be kept subsequently.
Revised: Ifsome of these neurons have strong linear correlations to the output neuron (e.g., Pearson correlationcoefficient larger than 0 . 99 ), they should be kept subsequently.

Original: Namely, we can maintain the pathfrom input neurons to the neurons to be kept and reduce the search space.
Revised: Namely, we can maintain the pathfrom input neurons to the neurons to be kept and reduce the search space.

Original:  Theoretical Analysis
Revised:  Theoretical Analysis

Original: We employ explorations, experience replay, and a target Q-network to boost the convergence to theoptimal Q function.
Revised: We employ explorations, experience replay, and a target Q-network to boost the convergence to theoptimal Q function.

Original: However, our extra requirement of the convex shape for the negative Q-functionmay deteriorate the convergence performance.
Revised: However, our extra requirement of the convex shape for the negative Q-functionmay deteriorate the convergence performance.

Original: Thus, we first prove that in C O NS O L E , the negativeoptimal Q-function is also convex so that the convex design doesn’t affect the convergence.
Revised: Thus, we first prove that in C O NS O L E , the negativeoptimal Q-function is also convex so that the convex design doesn’t affect the convergence.

Original: Then, weprove that the convexity of the negative optimal Q-function eventually yields the exact equations.
Revised: Then, weprove that the convexity of the negative optimal Q-function eventually yields the exact equations.

Original: Theorem 1.
Revised: Theorem 1.

Original: ∀ 0 ≤ k ≤ K − 1 , the negative optimal Q-function − Q ∗ ( s k , ˜ a k ) in the proposed
Revised: ∀ 0 ≤ k ≤ K − 1 , the negative optimal Q-function − Q ∗ ( s k , ˜ a k ) in the proposed

Original: C O NS O L E framework exists and is convex in s k and ˜ a k , where s k is the discrete state and ˜ a k is thecontinuous action at the k th stage.
Revised: C O NS O L E framework exists and is convex in s k and ˜ a k , where s k is the discrete state and ˜ a k is thecontinuous action at the k th stage.

Original: The proof can be seen in Appendix A.3.
Revised: The proof can be seen in Appendix A.3.

Original: Based on the convexity of negative optimal Q-function,the optimal sequence of states ( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) can be found viasolving the convex optimization problem.
Revised: Based on the convexity of negative optimal Q-function,the optimal sequence of states ( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) can be found viasolving the convex optimization problem.

Original: Then, we have the following theorem.
Revised: Then, we have the following theorem.

Original: Theorem 2.
Revised: Theorem 2.

Original: Let f ∗ ( · ; W ) denote the L O C A L constructed by the optimal sequences of states( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) from − Q ∗ ( · ) , where W is the set of weightsof f ∗ ( · ; W ) .
Revised: Let f ∗ ( · ; W ) denote the L O C A L constructed by the optimal sequences of states( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) from − Q ∗ ( · ) , where W is the set of weightsof f ∗ ( · ; W ) .

Original: If f ∗ ( · ; W ) can be trained with noiseless datasets and the training can achieve theglobal optimal weights W ∗ , f ∗ ( · ; W ∗ ) can be simplified to the true equation g ( · ) .
Revised: If f ∗ ( · ; W ) can be trained with noiseless datasets and the training can achieve theglobal optimal weights W ∗ , f ∗ ( · ; W ∗ ) can be simplified to the true equation g ( · ) .

Original: For the optimal search structure of L O C A L, i.e., f ∗ ( · ; W ) , the optimal weight set W ∗ is also trainedvia gradient method like Adam [33].
Revised: For the optimal search structure of L O C A L, i.e., f ∗ ( · ; W ) , the optimal weight set W ∗ is also trainedvia gradient method like Adam [33].

Original: The proof can be seen in Appendix A.4.
Revised: The proof can be seen in Appendix A.4.

Original: Theorem 2 requiresthat L O C
Revised: Theorem 2 requiresthat L O C

Original: A L can learn the global optimal weights.
Revised: A L can learn the global optimal weights.

Original: The requirement is generally hard to achieve dueto the non-linearity and non-convexity of L O C A
Revised: The requirement is generally hard to achieve dueto the non-linearity and non-convexity of L O C A

Original: L.
Revised: L.

Original: However, we show that with mild assumptions,there are local regions in
Revised: However, we show that with mild assumptions,there are local regions in

Original: the L O C
Revised: the L O C

Original: A L loss surface with strict convexity.
Revised: A L loss surface with strict convexity.

Original: Then, if we have properinitializations, the gradient-based weight updating can find the global optimum.
Revised: Then, if we have properinitializations, the gradient-based weight updating can find the global optimum.

Original: Specifically, we havethe following theorem.
Revised: Specifically, we havethe following theorem.

Original: Theorem 3.
Revised: Theorem 3.

Original: Assume the following conditions hold: (1) the equation g ( x ) is C 2 smooth and hasbounded second derivatives with respect to weights, (2) ∃ x ∈ X , g ( x ) has non-zero gradients withrespect to weights, (3) the structure of L O C
Revised: Assume the following conditions hold: (1) the equation g ( x ) is C 2 smooth and hasbounded second derivatives with respect to weights, (2) ∃ x ∈ X , g ( x ) has non-zero gradients withrespect to weights, (3) the structure of L O C

Original: A L is correctly searched to exactly represent symbolsand symbol connections in g ( x ) , and (4) the training dataset of L O C
Revised: A L is correctly searched to exactly represent symbolsand symbol connections in g ( x ) , and (4) the training dataset of L O C

Original: A L is noiseless.
Revised: A L is noiseless.

Original: Then, for the
Revised: Then, for the

Original: MSE loss surface of L O C A L , each global optimal point has a strictly convex local region.
Revised: MSE loss surface of L O C A L , each global optimal point has a strictly convex local region.

Original: The proofs can be seen in Appendix A.5.
Revised: The proofs can be seen in Appendix A.5.

Original: Note that Assumptions 1-2 easily hold for common physicalequations in nature [2].
Revised: Note that Assumptions 1-2 easily hold for common physicalequations in nature [2].

Original: Assumption 3 relies on the search algorithm, and we show, both theoretically and numerically, that our double convex deep Q-learning has good performances.
Revised: Assumption 3 relies on the search algorithm, and we show, both theoreticallyand numerically, that our double convex deep Q-learning has good performances.

Original: Assumption 4relies on the quality of data and we focus on the noiseless data in this paper.
Revised: Assumption 4relies on the quality of data and we focus on the noiseless data in this paper.

Original: What’s more, Equation(6) in the proof suggests that if the absolute noise values are small, the locally convex region stillexists.
Revised: What’s more, Equation(6) in the proof suggests that if the absolute noise values are small, the locally convex region stillexists.

Original: We also numerically prove C O NS O L E is robust under certain noise levels in Section 5.5.
Revised: We also numerically prove C O NS O L E is robust under certain noise levels in Section 5.5.

Original: Tosummarize, these assumptions are acceptable.
Revised: Tosummarize, these assumptions are acceptable.

Original: To quantify the range of the local region for a L O C A
Revised: To quantify the range of the local region for a L O C A

Original: Lwith a certain complexity, we have the following theorem.
Revised: Lwith a certain complexity, we have the following theorem.

Original: Theorem 4.
Revised: Theorem 4.

Original: Suppose Assumptions 1-4 in Theorem 3 hold.
Revised: Suppose Assumptions 1-4 in Theorem 3 hold.

Original: For a L O C A L with one symbolicactivation, multiplication, and summation layer, the set of local convex regions with global optima is 
Revised: For a L O C A L with one symbolicactivation, multiplication, and summation layer, the set of local convex regions with global optima is [Equation] [Equation] W )

Original: [Equation]
Revised: [Equation]

Original:  , where notations are defined in the proof.
Revised: y k |} , where notations are defined in the proof.

Original: The proofs can be seen in Appendix A.6.
Revised: The proofs can be seen in Appendix A.6.

Original: We explain the region range is large for a stable system thatsatisfies all assumptions in Theorem 4.
Revised: We explain the region range is large for a stable system thatsatisfies all assumptions in Theorem 4.

Original: Physical interpretations of the convex region size.
Revised: Physical interpretations of the convex region size.

Original: Physical systems in scientific and engineeringdomains have certain stability that can withstand parameter changes to some extent.
Revised: Physical systems in scientific and engineeringdomains have certain stability that can withstand parameter changes to some extent.

Original: Further, thisability should hold for arbitrary x ∈ X and y ∈ Y .
Revised: Further, thisability should hold for arbitrary x ∈ X and y ∈ Y .

Original: Thus, we can assume ddt (cid:12)(cid:12) t =

Revised: Thus, we can assume ddt (cid:12)(cid:12) t =

Original: ˆ y
Revised: ˆ y

Original: ( x i , W + tX ) ≈ddt (cid:12)(cid:12) t
Revised: ( x i , W + tX ) ≈ddt (cid:12)(cid:12) t

Original: =

Revised: =

Original: ˆ y ( x j , W + tX )
Revised: ˆ y ( x j , W + tX )

Original: ≈ ddt (cid:12)(cid:12) t
Revised: ≈ ddt (cid:12)(cid:12) t

Original: =

Revised: =

Original: ˆ y
Revised: ˆ y

Original: ( x k , W + tX )
Revised: ( x k , W + tX )

Original: for x i , x j , x k ∈ X .
Revised: for x i , x j , x k ∈ X .

Original: Then, the inequality in set Ucan be approximately rewritten as 1 η > || w − w ∗
Revised: Then, the inequality in set Ucan be approximately rewritten as 1 η > || w − w ∗

Original: || 2 , where w and w ∗ are the vectorized W and W ∗ ,respectively.
Revised: || 2 , where w and w ∗ are the vectorized W and W ∗ ,respectively.

Original: Namely, the distance between any point W in the region to the global optimal point
Revised: Namely, the distance between any point W in the region to the global optimal point

Original: W ∗ in the region is bounded by 1 η .
Revised: W ∗ in the region is bounded by 1 η .

Original: Based on Equation (19), η is the bound of the ratio of secondderivative to the first derivative.
Revised: Based on Equation (19), η is the bound of the ratio of secondderivative to the first derivative.

Original: For a stable system, this ratio should be small.
Revised: For a stable system, this ratio should be small.

Original: Otherwise, the systemcan easily crash with a small parameter disturbance.
Revised: Otherwise, the systemcan easily crash with a small parameter disturbance.

Original: Thus, 1 η is relatively large and so is the regionof U .
Revised: Thus, 1 η is relatively large and so is the regionof U .

Original: An example of the range is displayed in Section 5.2.
Revised: An example of the range is displayed in Section 5.2.

Original: Finally, the above analysis also holdswhen the L O C
Revised: Finally, the above analysis also holdswhen the L O C

Original: A L of the system equation has more than one symbolic activation, multiplication, andsummation layers.
Revised: A L of the system equation has more than one symbolic activation, multiplication, andsummation layers.

Original: This is because Equation (20) always holds as long as we can find a η to boundthe ratio of the second derivative to the first derivative, which is irrelevant to the structure of L O C A L.
Revised: This is because Equation (20) always holds as long as we can find a η to boundthe ratio of the second derivative to the first derivative, which is irrelevant to the structure of L O C A L.

Original:  Experiments
Revised:  Experiments

Original: 5.1 Settings
Revised: 5.1 Settings

Original: Datasets.
Revised: Datasets.

Original: We use the following datasets for testing.
Revised: We use the following datasets for testing.

Original: (1) Synthetic datasets .
Revised: (1) Synthetic datasets .

Original: We create two datasets,
Revised: We create two datasets,

Original: Syn 1 and Syn 2 , for testing.
Revised: Syn 1 and Syn 2 , for testing.

Original: Syn 1 has the following equations: y 1 = 3 x 21 cos(2 .
Revised: Syn 1 has the following equations: y 1 = 3 x 21 cos(2 .

Original: 5 x 2 ) , y 

Revised: 5 x 2 ) , y

Original: = 4 x 1 x 3 ,and y 

Revised: = 4 x 1 x 3 ,and y

Original: = 3 x 23 .
Revised: = 3 x 2 3 .

Original: Syn 2 is more complex with the following equations y 1 = √ 2 .
Revised: Syn 2 is more complex with the following equations y 1 = √ 2 .

Original: 2 x 1 x 2 + x 1 x 22 ,y 2 = sin(1 .
Revised: 2 x 1 x 2 + x 1 x 2 2 ,y 2 = sin(1 .

Original: 8 x 1 ) (cid:0) log(3 x 2 ) + √ x 3 (cid:1) , y 3 = √ 3 .
Revised: 8 x 1 ) (cid:0) log(3 x 2 ) + √ x 3 (cid:1) , y 3 = √ 3 .

Original: 7 x 3 log(1 .
Revised: 7 x 3 log(1 .

Original: 6 x 1 ) + x 21 .
Revised: 6 x 1 ) + x 2 1 .

Original: For the training data, each inputvariable is randomly sampled from a uniform distribution of U (1 , 2) to avoid invalid values likelog(0) .
Revised: For the training data, each inputvariable is randomly sampled from a uniform distribution of U (1 , 2) to avoid invalid values likelog(0) .

Original: Totally, we create 2 , 000 samples for training.
Revised: Totally, we create 2 , 000 samples for training.

Original: Then, in the test phase, we utilize another2 , 000 samples whose input variables are sampled from U (3 , 4) .
Revised: Then, in the test phase, we utilize another2 , 000 samples whose input variables are sampled from U (3 , 4) .

Original: The symbolic activation pools are{ x, x 2 , cos( x ) } and {√ x, x, x 2 , log( x ) , sin( x ) } for Syn 1 and Syn 2 , respectively.(2) Power system dataset.
Revised: The symbolic activation pools are{ x, x 2 , cos( x ) } and {√ x, x, x 2 , log( x ) , sin( x ) } for Syn 1 and Syn 2 , respectively.(2) Power system dataset.

Original: Power flow equation determines the operations of electric systems [37].
Revised: Power flow equation determines the operations of electric systems [37].

Original: For node i in an M -node system, the equation can be written as p i
Revised: For node i in an M -node system, the equation can be written as p i

Original: = (cid:80)
Revised: = (cid:80)

Original: | M | m =1 G im ( u i u m + v i v m )
Revised: | M | m =1 G im ( u i u m + v i v m )

Original: +
Revised: +

Original: B im ( v i u m − u i v m ) and q i = (cid:80) M m =1 G{+Mm =
+}
Revised: B im ( v i u m − u i v m ) and q i = (cid:80) {+Mm =
+}

Original:  im ( v i u m − u i v m ) − B im ( v i u m − u i v m ) , where u i and v iare the real and imaginary components of the voltage phasor at node i .
Revised: G im ( v i u m − u i v m ) − B im ( v i u m − u i v m ) , where u i and v iare the real and imaginary components of the voltage phasor at node i .

Original: p i and q
Revised: p i and q

Original: i are the activeand reactive power at node i .
Revised: i are the activeand reactive power at node i .

Original: G
Revised: G

Original: im and B im represent the physical parameters of line im .
Revised: im and B im represent the physical parameters of line im .

Original: If lineim does not exist, G im =
Revised: If lineim does not exist, G im =

Original: B im = 0 .
Revised: B im = 0 .

Original: Therefore, we can treat x =
Revised: Therefore, we can treat x =

Original: [ u 1 , v 1 , · · · , u M , v M ] T andy =
Revised: [ u 1 , v 1 , · · · , u M , v M ] T andy =

Original: [ p 1 , q 1 , · · · , p M , q M ] T .
Revised: [ p 1 , q 1 , · · · , p M , q M ] T .

Original: The target is to learn the underlying system topology and parameters,which has broad impacts on the power domains [38].
Revised: The target is to learn the underlying system topology and parameters,which has broad impacts on the power domains [38].

Original: In this experiment, we implement simulationfrom a 5 -node system using MATPOWER [39] and two year’s hourly data.
Revised: In this experiment, we implement simulationfrom a 5 -node system using MATPOWER [39] and two year’s hourly data.

Original: The first 8 ,760 points areused for training while the remaining samples are used for testing. The symbolic activation pool is{ x } .
Revised: The first 8 ,

Original: We denote this dataset as Pow.(3) Mass-damper system dataset.
Revised: (3) Mass-damper system dataset.

Original: Equations of the mass-damper system can be written as: ˙ q =− DRD ⊤ M − 1 q , where ˙ q is a vector of momenta, D is the incidence matrix of the system, R is thediagonal matrix of the damping coefficients for each line of the system, and M is the diagonal matrixof each node mass of the system.
Revised: Equations of the mass-damper system can be written as: ˙ q =− DRD ⊤ M − 1 q , where ˙ q is a vector of momenta, D is the incidence matrix of the system, R is thediagonal matrix of the damping coefficients for each line of the system, and M is the diagonal matrixof each node mass of the system.

Original: Thus, we can set y = ˙
Revised: Thus, we can set y = ˙

Original: q and x = q
Revised: q and x = q

Original: and the goal is to learn theparameter matrix − DRD ⊤ M − 1 .
Revised: and the goal is to learn theparameter matrix − DRD ⊤ M − 1 .

Original: We conduct the simulation via MATLAB for a 10 -node systemand obtain 6 , 000 points for 1 min simulation 
Revised: We conduct the simulation via MATLAB for a 10 -node systemand obtain 6 , 000 points for 1 min simulation with a step size to be 0 . 01 s.

Revised: The first 3 , 000 samples areused for training while the rest samples are used for testing.

Revised: The symbolic activation pool is { x } .

Revised: Then, we denote the dataset as Mas.

Original: Benchmark methods .
Revised: Benchmark methods .

Original: The following benchmark methods are utilized.
Revised: The following benchmark methods are utilized.

Original: (1) Deep Symbolic Regression( DSR ) [3].
Revised: (1) Deep Symbolic Regression( DSR ) [3].

Original: DSR develops an RNN-based framework to search the expression tree.
Revised: DSR develops an RNN-based framework to search the expression tree.

Original: Especially, therisk-seeking policy gradient is utilized to seek the best performance.
Revised: Especially, therisk-seeking policy gradient is utilized to seek the best performance.

Original: Then, BFGS [11] can solve thenon-linear optimization and estimate the symbol coefficients.
Revised: Then, BFGS [11] can solve thenon-linear optimization and estimate the symbol coefficients.

Original: (2) Vanilla Policy Gradient ( VPG ) [3].
Revised: (2) Vanilla Policy Gradient ( VPG ) [3].

Original: VPG is a vanilla version of DSR with a normal policy gradient rather than the risk-seeking method.(3) Equation Learner ( EQL ) [6, 7].
Revised: VPG is a vanilla version of DSR with a normal policy gradient rather than the risk-seeking method.(3) Equation Learner ( EQL ) [6, 7].

Original: EQL creates an end-to-end NN to select symbols and estimatethe coefficients.
Revised: EQL creates an end-to-end NN to select symbols and estimatethe coefficients.

Original: The sparse regularization is enforced for the NN weights to search symbols.
Revised: The sparse regularization is enforced for the NN weights to search symbols.

Original: Multilayer Perceptron ( MLP ).
Revised: Multilayer Perceptron ( MLP ).

Original: We also employ a standard MLP to learn the regression from x to y .
Revised: We also employ a standard MLP to learn the regression from x to y .

Original: We only evaluate the extrapolation capacity of MLP in the test dataset.
Revised: We only evaluate the extrapolation capacity of MLP in the test dataset.

Original: For DSR, VPG, and EQL methods, based on the input datasets, we adjust the symbol and operatorlibrary to enable the same searching space as C O NS O L E for fair comparisons.
Revised: For DSR, VPG, and EQL methods, based on the input datasets, we adjust the symbol and operatorlibrary to enable the same searching space as C O NS O L E for fair comparisons.

Original: We run the benchmarkmethods 5 times with different random seeds and present their best results.
Revised: We run the benchmarkmethods 5 times with different random seeds and present their best results.

Original: As for our method, weonly run 1 time and obtain good results due to the convex design and the ϵ -greedy strategy.
Revised: As for our method, weonly run 1 time and obtain good results due to the convex design and the ϵ -greedy strategy.

Original: Metrics for evaluation.
Revised: Metrics for evaluation.

Original: We employ the following metrics.
Revised: We employ the following metrics.

Original: (1) Average coefficient estimation percentage error E c .
Revised: (1) Average coefficient estimation percentage error E c .

Original: For an equation with H symbols, We calculate the error as E c = 1 H (cid:80)
Revised: For an equation with H symbols, We calculate the error as E c = 1 H (cid:80)

Original: h
Revised: h

Original: PE ( w h , ˆ w h ) ,where w h and ˆ w h represents the true and the estimated coefficients for the h th symbol, respectively.
Revised: PE ( w h , ˆ w h ) ,where w h and ˆ w h represents the true and the estimated coefficients for the h th symbol, respectively.

Original: PE is the operation to calculate the percentage error.
Revised: PE is the operation to calculate the percentage error.

Original: If there is no matched symbol for the h th truesymbol, we denote PE ( w h , ˆ w h ) = 100% .
Revised: If there is no matched symbol for the h th truesymbol, we denote PE ( w h , ˆ w h ) = 100% .

Original: Note that when calculating E c , proper simplifications maybe needed.
Revised: Note that when calculating E c , proper simplifications maybe needed.

Original: For example, cos (cid:0) 2 . 5( √ x ) 2 (cid:1) = cos(2 .
Revised: For example, cos (cid:0) 2 . 5( √ x ) 2 (cid:1) = cos(2 .

Original: 5 x ) .
Revised: 5 x ) .

Original: (2) NRMSE in the test dataset.
Revised: (2) NRMSE in the test dataset.

Original: We measurethe extrapolation capacity in the test dataset and utilize NRMSE employed in Section 3.2.
Revised: We measurethe extrapolation capacity in the test dataset and utilize NRMSE employed in Section 3.2.

Original: Implementing details of C O NS O L E .
Revised: Implementing details of C O NS O L E .

Original: Hyper-parameters of C O NS
Revised: Hyper-parameters of C O NS

Original: O L E exist for both the doubleconvex deep Q-learning and the L O C A L.
Revised: O L E exist for both the doubleconvex deep Q-learning and the L O C A L.

Original: In the deep Q-learning, we set γ = 0 .
Revised: In the deep Q-learning, we set γ = 0 .

Original: 2 , ϵ = 0 .
Revised: 2 , ϵ = 0 .

Original: 4 , T = 600 ,λ = 10 − 2 , T 0 = 10 for Algorithm 2.
Revised: 4 , T = 600 ,λ = 10 − 2 , T 0 = 10 for Algorithm 2.

Original: Furthermore, to train the negative Q-function and the rewardfunction, we set the learning rate to be 5 × 10 − 3 and the number of epochs for training to be 50 .
Revised: Furthermore, to train the negative Q-function and the rewardfunction, we set the learning rate to be 5 × 10 − 3 and the number of epochs for training to be 50 .

Original: Then, we set the batch size for the negative Q-function to be 100 .
Revised: Then, we set the batch size for the negative Q-function to be 100 .

Original: If the number of data in the replaybuffer is less than 100 , no training happens for the negative Q-function.
Revised: If the number of data in the replaybuffer is less than 100 , no training happens for the negative Q-function.

Original: Additionally, all the datagathered in one episode are used to train the negative reward function.
Revised: Additionally, all the datagathered in one episode are used to train the negative reward function.

Original: As for the L O C A L, we set
Revised: As for the L O C A L, we set

Original: K = 3 , the learning rate to be 1 × 10 − 2 and the number of training epochs to be 8 .
Revised: K = 3 , the learning rate to be 1 × 10 − 2 and the number of training epochs to be 8 .

Original: We make thesetraining epochs to be small since training the L O C
Revised: We make thesetraining epochs to be small since training the L O C

Original: A L is the most time-consuming part of C O NS O L E .
Revised: A L is the most time-consuming part of C O NS O L E .

Original: Furthermore, if the structure of L O C A L is correctly searched, a small number of iterations can help
Revised: Furthermore, if the structure of L O C A L is correctly searched, a small number of iterations can help

Original: L O C A L to gain the global optimal weights.
Revised: L O C A L to gain the global optimal weights.

Original: Finally, we initialize all trainable weights in L O C
Revised: Finally, we initialize all trainable weights in L O C

Original: A L tobe 1 .
Revised: A L tobe 1 .

Original: The following results show that a relatively large area is suitable for an initial guess of L O C A L.
Revised: The following results show that a relatively large area is suitable for an initial guess of L O C A L.

Original: 5.2 Verification and 3-D Visualization of Convex Mechanisms
Revised: 5.2 Verification and 3-D Visualization of Convex Mechanisms

Original: We first utilize a toy example to verify the benefits of convex designs for two sub problems.
Revised: We first utilize a toy example to verify the benefits of convex designs for two sub problems.

Original: Specifically, we consider to learn y 1 = 3 x 21 cos(2 .
Revised: Specifically, we consider to learn y 1 = 3 x 21 cos(2 .

Original: 5 x 2 ) with the loss function L = (cid:80) Ni
Revised: 5 x 2 ) with the loss function L = (cid:80) Ni

Original: ( y i [1] −w 1 x i [1] 2 cos( w 2 x i [2]) 2 , where the data sample notations are defined in Section 3.1.
Revised: ( y i [1] −w 1 x i [1] 2 cos( w 2 x i [2]) 2 , where the data sample notations are defined in Section 3.1.

Original: As shown in
Revised: As shown in

Original: Fig.
Revised: Fig.

Original: 2a, we design a 4 -layer L O C A L with two learnable parameters w 1 and w 2 to represent the coefficients.
Revised: 2a, we design a 4 -layer L O C A L with two learnable parameters w 1 and w 2 to represent thecoefficients.

Original: Thus, in the search phase, the goal is to identify the action a 1 .
Revised: Thus, in the search phase, the goal is to identify the action a 1 .

Original: We plot − Q t ( · ) whent = 1 , 5 , 10 in Fig. 2b.
Revised: We plot − Q t ( · ) whent = 1 , 5 , 10 in Fig. 2b.

Original: The convexity always exists so that the algorithm can quickly find the globaloptimal solutions in red dots.
Revised: The convexity always exists so that the algorithm can quickly find the globaloptimal solutions in red dots.

Original: Next, as the agent update Q values with true rewards, the Q-functionconverges to the optimal function within 10 episodes.
Revised: Next, as the agent update Q values with true rewards, the Q-functionconverges to the optimal function within 10 episodes.

Original: Finally, the convex − Q ∗ ( · ) remains unchangedand can bring the optimal action and the true equation, which supports Theorem 2.
Revised: Finally, the convex − Q ∗ ( · ) remains unchangedand can bring the optimal action and the true equation, which supports Theorem 2.

Original: Subsequently, we plot the loss surface of L in Fig.
Revised: Subsequently, we plot the loss surface of L in Fig.

Original: 2c.
Revised: 2c.

Original: We find that around the two global optima(3 , 2 . 5) and (3 , − 2 . 5) , there are convex regions that have an approximate quadratic shape.
Revised: We find that around the two global optima(3 , 2 . 5) and (3 , − 2 . 5) , there are convex regions that have an approximate quadratic shape.

Original: To furtherquantify the range for proper initialization, we vary the initial weight w 0 ∈ {− 10 , − 9 , · · · , 10 } forw 1 and w 2 in L O C A L. Fig.
Revised: To furtherquantify the range for proper initialization, we vary the initial weight w 0 ∈ {− 10 , − 9 , · · · , 10 } forw 1 and w 2 in L O C A L. Fig.

Original: 2 reports the final training loss with respect to different w 0 , and we findthe safe range for initialization is [ − 3 , 7] \ { 0 } .
Revised: 2 reports the final training loss with respect to different w 0 , and we find the safe range for initialization is [ − 3 , 7] \ { 0 } .

Original: This range is relatively large when compared to theoptimal values.
Revised: This range is relatively large when compared to theoptimal values.

Original: These observations support our Theorems 3 and 4.
Revised: These observations support our Theorems 3 and 4.

Original: Finally, w 0 = 0 does not worksince ∂L∂w 2 (cid:12)(cid:12) w 2 =

Revised: Finally, w 0 = 0 does not worksince ∂L∂w 2 (cid:12)(cid:12) w 2 =

Original: = 0 always holds, which prevents the weight updating using gradient methods.
Revised: = 0 always holds, which prevents the weight updating using gradient methods.

Original: [Table]
Revised: [Table]

Original: .3 Convexity Guarantees of C O NS O L E to Learn Correct Equations
Revised: 5.3 Convexity Guarantees of C O NS O L E to Learn Correct Equations

Original: In this subsection, we report the results of the equation learning.
Revised: In this subsection, we report the results of the equation learning.

Original: First, we list the learned equationsfor Syn 1 and Syn 2 in Table 1.
Revised: First, we list the learned equationsfor Syn 1 and Syn 2 in Table 1.

Original: Table 1 presents that
Revised: Table 1 presents that

Original: C
Revised: C

Original: O
Revised: O

Original: NS
Revised: NS

Original: O
Revised: O

Original: L E has the best performance in most ofthe equations while DSR ranks second.
Revised: L E has the best performance in most ofthe equations while DSR ranks second.

Original: In particular, C O NS O L E can accurately learn all equations in
Revised: In particular, C O NS O L E can accurately learn all equations in

Original: Syn 1 and Syn 2 .
Revised: Syn 1 and Syn 2 .

Original: The superior performance is mostly due to the convex design of the search and thecoefficient estimation process with provable guarantees.
Revised: The superior performance is mostly due to the convex design of the search and thecoefficient estimation process with provable guarantees.

Original: In addition, we observe that for the resultof C O NS O L E in learning y 3 of Syn 1 , we have 3 x 23 cos(0 .
Revised: In addition, we observe that for the resultof C O NS O L E in learning y 3 of Syn 1 , we have 3 x 23 cos(0 .

Original: 005 x 1 ) ≈ 3 x 2 3 .
Revised: 005 x 1 ) ≈ 3 x 23 .

Original: This shows that there is apossibility that the search result of C O NS O L E might not be optimal (i.e., an extra consine term existsbut is close to 1 ), but the learned equation is still highly accurate.
Revised: This shows that there is apossibility that the search result of C O NS O L E might not be optimal (i.e., an extra consine term existsbut is close to 1 ), but the learned equation is still highly accurate.

Original: Such an observation nonethelessguides further study of the coupling relationship between the search and the estimation procedures.
Revised: Such an observation nonethelessguides further study of the coupling relationship between the search and the estimation procedures.

Original: [Figure] DSR doesn’t perform well when the underlying equation is relatively complex, e.g., y 1 in Syn 1 and y 1and y 2 in Syn 2 .
Revised: DSR doesn’t perform well when the underlying equation is relatively complex, e.g., y 1 in Syn 1 and y 1and y 2 in Syn 2 .

Original: This is because DSR may still fall into a local optimal solution despite risk-seekingpolicy design.
Revised: This is because DSR may still fall into a local optimal solution despite risk-seekingpolicy design.

Original: VPG method performs worse than DSR since VPG considers an expected reward [3].
Revised: VPG method performs worse than DSR since VPG considers an expected reward [3].

Original: Finally, EQL performs the worst as it merges the symbol search and the coefficient estimation in one
Revised: Finally, EQL performs the worst as it merges the symbol search and the coefficient estimation in one

Revised: [Figure]

Original: NN model, which provides few guarantees of accurate learning.
Revised: NN model, which provides few guarantees of accurate learning.

Original: The above observations and analysesare consistent with the result of coefficient estimation errors and prediction errors in Fig. 3a and 3b.
Revised: The above observations and analysesare consistent with the result of coefficient estimation errors and prediction errors in Fig. 3a and 3b.

Original: For the Pow and Mas, C O NS O L E doesn’t learn completely exact equations within T = 600 episodes.
Revised: For the Pow and Mas, C O NS O L E doesn’t learn completely exact equations within T = 600 episodes.

Original: This is because they have a large number of variables to be considered.
Revised: This is because they have a large number of variables to be considered.

Original: However, C O NS O L E is stillbetter than other methods.
Revised: However, C O NS O L E is stillbetter than other methods.

Original: 5.4 Ablation Study: Exploration and Convex Search are Essential
Revised: 5.4 Ablation Study: Exploration and Convex Search are Essential

Original: We conduct an ablation study to further understand what factors are important in the C O NS O L E .
Revised: We conduct an ablation study to further understand what factors are important in the C O NS O L E .

Original: We test the result with Syn 1 and Syn 1 and report the E c (%) values.
Revised: We test the result with Syn 1 and Syn 1 and report the E c (%) values.

Original: Specifically, we investigate thefollowing cases.
Revised: Specifically, we investigate thefollowing cases.

Original: (1) No ablation.
Revised: (1) No ablation.

Original: (2) Drop exploration in deep Q-learning.
Revised: (2) Drop exploration in deep Q-learning.

Original: We delete the ϵ -greedystrategy.
Revised: We delete the ϵ -greedystrategy.

Original: (3) Drop double-convex deep-Q learning.
Revised: (3) Drop double-convex deep-Q learning.

Original: We replace this design with a traditional deep-Qlearning.
Revised: We replace this design with a traditional deep-Qlearning.

Original: (4) Drop coefficient estimation using L O C A L. After learning the structure of L O C A L, wereformulate a non-linear optimization and utilize BFGS [11] in DSR, instead of gradient descent in
Revised: (4) Drop coefficient estimation using L O C A L. After learning the structure of L O C A L, wereformulate a non-linear optimization and utilize BFGS [11] in DSR, instead of gradient descent in

Original: L O C A L, to estimate the coefficient.
Revised: L O C A L, to estimate the coefficient.

Original: (5) Drop static symbolic constraint.
Revised: (5) Drop static symbolic constraint.

Original: (6) Drop dynamic symbolicconstraint.
Revised: (6) Drop dynamic symbolicconstraint.

Original: These two constraints are mentioned in Section 3.2.
Revised: These two constraints are mentioned in Section 3.2.

Original: Then, Fig. 4 shows that cases (2) and(3) cause large errors.
Revised: Then, Fig. 4 shows that cases (2) and(3) cause large errors.

Original: For case (2) , if no exploration strategy is added, the updating of the Q-functionand the reward function is slow.
Revised: For case (2) , if no exploration strategy is added, the updating of the Q-functionand the reward function is slow.

Original: For case (3) , the non-convex search induces many sub-optimalactions in the search process.
Revised: For case (3) , the non-convex search induces many sub-optimalactions in the search process.

Original: Thus, these two cases cause a slow search process and significant errorsafter T = 600 episodes.
Revised: Thus, these two cases cause a slow search process and significant errorsafter T = 600 episodes.

Original: For symbolic constraints in (5) and (6) , removing them increases the errorfor Syn 2 and Syn 1 , respectively.
Revised: For symbolic constraints in (5) and (6) , removing them increases the errorfor Syn 2 and Syn 1 , respectively.

Original: This shows these constraints are beneficial to the search process.
Revised: This shows these constraints are beneficial to the search process.

Original: Finally, we find that utilizing BFGS in (4) can bring good results with initialization in the locallyconvex region.
Revised: Finally, we find that utilizing BFGS in (4) can bring good results with initialization in the locallyconvex region.

Original: Since the non-linear optimization has the same loss surface as L O C A L, the locallyconvex region in Theorems 3 and 4 can prove the good performance of BFGS.
Revised: Since the non-linear optimization has the same loss surface as L O C A L, the locallyconvex region in Theorems 3 and 4 can prove the good performance of BFGS.

Original: [Figure]
Revised: [Figure]

Original: 5. C O NS O L E is Robust with Changing Noise Levels and Data Volume
Revised: C O NS O L E is Robust with Changing Noise Levels and Data Volume

Original: We utilize Syn 1 and Syn 2 to examine the robustness of the framework with changing noiselevels and data volumes.
Revised: We utilize Syn 1 and Syn 2 to examine the robustness of the framework with changing noiselevels and data volumes.

Original: For the noise level, we consider the Signal-to-Noise Ratio (SNR)such that SNR ∈ { 80 , 90 , 100 , 110 , 120 } .
Revised: For the noise level, we consider the Signal-to-Noise Ratio (SNR)such that SNR ∈ { 80 , 90 , 100 , 110 , 120 } .

Original: For the data volume, we fix SNR = 100 and vary
Revised: For the data volume, we fix SNR = 100 and vary

Original: N ∈ { 500 , 1000 , 1500 , 2000 , 2500 } .
Revised: N ∈ { 500 , 1000 , 1500 , 2000 , 2500 } .

Original: Fig.
Revised: Fig.

Original: 5a and 5b demonstrate the results.
Revised: 5a and 5b demonstrate the results.

Original: We find that when
Revised: We find that when

Original: SNR ≥ 100 , the error can be less than 1% .
Revised: SNR ≥ 100 , the error can be less than 1% .

Original: This noise level is suitable to real-word systems.
Revised: This noise level is suitable to real-word systems.

Original: Forexample, SNR = 125 for electric measurements [38].
Revised: Forexample, SNR = 125 for electric measurements [38].

Original: For the data volume, the overall error is lessthan 1 .
Revised: For the data volume, the overall error is lessthan 1 .

Original: 5% when N ≥ 500 , which shows a robust performance of C O NS O L E .
Revised: 5% when N ≥ 500 , which shows a robust performance of C O NS O L E .

Original:  Conclusions and Future Work
Revised:  Conclusions and Future Work

Original: In this paper, we propose C O NS O L E , a novel Convex Neural Symbolic Learning method, whichenjoys convexity with certain conditions to tackle Symbolic Regression with guaranteed performances.
Revised: In this paper we propose C O NS O L E , a novel Convex Neural Symbolic Learning method, whichenjoys convexity with certain conditions to tackle Symbolic Regression with guaranteed performances.

Original: Specifically, we convexify the search problem by proposing a double convex deep-Q learning.
Revised: Specifically, we convexify the search problem by proposing a double convex deep-Q learning.

Original: In themeantime, we prove the local and strict convexity of the coefficient estimation in our Locally-Convexequation Learner (L O C
Revised: In themeantime, we prove the local and strict convexity of the coefficient estimation in our Locally-Convexequation Learner (L O C

Original: A L).
Revised: A L).

Original: To our best knowledge, C O NS O
Revised: To our best knowledge, C O NS O

Original: L E is the first method that providesguarantees with reasonable assumptions to learn exact equations.
Revised: L E is the first method that providesguarantees with reasonable assumptions to learn exact equations.

Original: Besides, C O NS O L E has, at aminimum, a broader impact on the following domains.
Revised: Besides, C O NS O L E has, at aminimum, a broader impact on the following domains.

Original: (1) Convex control for physical systems using
Revised: (1) Convex control for physical systems using

Original: Reinforcement Learning.
Revised: Reinforcement Learning.

Original: (2) Neural networks in engineering applications with local convexity.
Revised: (2) Neural networks in engineering applications with local convexity.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...(

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] We specify assumptionsneeded
Revised: Yes We specify assumptions

Original: for good performances in Section 4.
Revised: neededfor good performances in Section 4.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A]
Revised: [N/A]

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] See Section 4.
Revised: [Yes] See Section 4.(b)

Original: (b) Did you include complete proofs of all theoretical results?
Revised: Did you include complete proofs of all theoretical results?

Original: [Yes] See Appendix A.3 to A.
Revised: [Yes] See Appendix A.3 toA.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [No] We will release
Revised: [No] We will release

Original: the code if the paper is accepted.
Revised: the code if the paper is accepted.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters,
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters,

Original: how they
Revised: how they

Original: were chosen)?
Revised: were chosen)?

Original: [Yes] See Section 5.1.
Revised: [Yes] See Section 5.1.

Original: (c) Did you report error bars (e g., with respect to the random seed after running experi- ments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] See Section 5.3.
Revised: [Yes] See Section 5.3.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [N/A]
Revised: [N/A]

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]
Revised: [N/A]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if [Table] applicable?

Original: [N/A](b)
Revised: [N/A]

Original:  Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Revised: [Table]

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Revised: [Table]

Original: A AppendixA.1 Model Overview with Pseudo Codes
Revised: A AppendixA.1 Model Overview with Pseudo Codes

Original: In this subsection, we provide a high-level summary of our framework for better understanding.
Revised: In this subsection, we provide a high-level summary of our framework for better understanding.

Original: Wepresent the summary in the form of pseudo codes, shown in Algorithm 1.
Revised: Wepresent the summary in the form of pseudo codes, shown in Algorithm 1.

Original: Algorithm 1 The Overview of the Proposed Framework.
Revised: Algorithm 1 The Overview of the Proposed Framework.

Original: Input: Training dataset { x i , y i } Ni =1 .
Revised: Input: Training dataset { x i , y i } Ni =1 .

Original: Step 1: Design L O C A L. L O C
Revised: Step 1: Design L O C A L. L O C

Original: A L has repeated blocks of symbolic activation, multiplication, and summation layers.
Revised: A L has repeated blocks of symbolic activation, multiplication, and summation layers.

Original: For example, Fig. 1 presents a L O C A L with 2 blocks.
Revised: For example, Fig. 1 presents a L O C A L with 2 blocks.

Original: Step 2:
Revised: Step 2:

Original: Denote L O C A L Function.
Revised: Denote L O C A L Function.

Original: L O C A L represents the map f
Revised: L O C A L represents the map f

Original: ( x ; { Z k } K − 1 k =0 , { W k } K − 1 k =0 ) from x to y .
Revised: ( x ; { Z k } K − 1 k =0 , { W k } K − 1 k =0 ) from x to y .

Original: With global optimal solutions of Z k and W k , f ( x ) can be simplified to the true equation g ( x ) .
Revised: With global optimal solutions of Z k and W k , f ( x ) can be simplified to the true equation g ( x ) .

Original: while L O C A L does not have the optimal performance do
Revised: while L O C A L does not have the optimal performance do

Original: Step 3: Search L O C A L Structure.
Revised: Step 3: Search L O C A L Structure.

Original: Step 3.1: Model the Search Process.
Revised: Step 3.1: Model the Search Process.

Original: Build the CMP and the reward function R ( · ) based on states and actions defined over L O C A L. Formulate a sequential optimization.
Revised: Build the CMP and the reward function R ( · ) based on states and actions defined over L O C A L. Formulate a sequential optimization.

Original: Step 3.2: Solve the Optimization.
Revised: Step 3.2: Solve the Optimization.

Original: Utilize the proposed double convex Q-learning to find optimal actions.
Revised: Utilize the proposed double convex Q-learning to find optimal actions.

Original: Generate a search result of { Z k } K − 1 k =0 .
Revised: Generate a search result of { Z k } K − 1 k =0 .

Original: Step 4:
Revised: Step 4:

Original: Estimate L O C
Revised: Estimate L O C

Original: A L Parameters.
Revised: A L Parameters.

Original: Train the searched L O C A L by minimizing the MSE via Adam.
Revised: Train the searched L O C A L by minimizing the MSE via Adam.

Original: Estimate values in { W k } K − 1 k =0 .
Revised: Estimate values in { W k } K − 1 k =0 .

Original: Step 5: Evaluate the Search and Estimation Results.
Revised: Step 5: Evaluate the Search and Estimation Results.

Original: The results can formulate f t ( x ) for the t th episode.
Revised: The results can formulate f t ( x ) for the t th episode.

Original: Calculate the end-of-trajectory reward R t to evaluate f t ( x ) .
Revised: Calculate the end-of-trajectory reward R t to evaluate f t ( x ) .

Original: Output: L O C A L with the best performance and the corresponding equations.
Revised: Output: L O C A L with the best performance and the corresponding equations.

Original: A.2 Training Algorithm for C O NS O L E .
Revised: A.2 Training Algorithm for C O NS O L E .

Original: The training algorithm can be seen in Algorithm 2.
Revised: The training algorithm can be seen in Algorithm 2.

Original: A.3 Proofs of Theorem
Revised: A.3 Proofs of Theorem

Original: Theorem.
Revised: Theorem.

Original: ∀ 0 ≤ k ≤ K − 1 , the negative optimal Q-function − Q ∗ ( s k , ˜ a k ) in the proposed
Revised: ∀ 0 ≤ k ≤ K − 1 , the negative optimal Q-function − Q ∗ ( s k , ˜ a k ) in the proposed

Original: C O NS O L E framework exists and is convex in s k and ˜ a k , where s k is the discrete state and ˜ a k is thecontinuous action at the k th stage.
Revised: C O NS O L E framework exists and is convex in s k and ˜ a k , where s k is the discrete state and ˜ a k is thecontinuous action at the k th stage.

Original: Proof.
Revised: Proof.

Original: First, we show our state transition satisfies the Markov property.
Revised: First, we show our state transition satisfies the Markov property.

Original: Specifically, Equation (1)in our paper shows that the next state s k +1 equals the matrix multiplication between the currentstate s k and the matrix Z k that is a matricization of the current action a k , where k is the index ofthe state.
Revised: Specifically, Equation (1)in our paper shows that the next state s k +1 equals the matrix multiplication between the currentstate s k and the matrix Z k that is a matricization of the current action a k , where k is the index ofthe state.

Original: Therefore, the state transition satisfies Markov property with the transition probability
Revised: Therefore, the state transition satisfies Markov property with the transition probability

Original: P ( s k +1 | s k , a k ) = 1 .
Revised: P ( s k +1 | s k , a k ) = 1 .

Original: Due to the Markov property of the state transition, we define our search process as Controlled Markov
Revised: Due to the Markov property of the state transition, we define our search process as Controlled Markov

Original: Process (CMP) [31, 30].
Revised: Process (CMP) [31, 30].

Original: By the CMP definition [30], our CMP is composed of our state, action, statetransition probability, a discounter factor γ , and a start state (i.e., s 0 in Equation (1)).
Revised: By the CMP definition [30], our CMP is composed of our state, action, statetransition probability, a discounter factor γ , and a start state (i.e., s 0 in Equation (1)).

Original: In general,
Revised: In general,

Original: CMP is a Markov Decision Process (MDP) without a reward function [31].
Revised: CMP is a Markov Decision Process (MDP) without a reward function [31].

Original: For one CMP, Trajectory Ordering (TO) ranks trajectories of state action pairs [31].
Revised: For one CMP, Trajectory Ordering (TO) ranks trajectories of state action pairs [31].

Original: In our paper,we define the trajectory from ( s 0 , a 0 ) to ( s K − 1 , a K − 1 ) for K -layer L O C A L.
Revised: In our paper,we define the trajectory from ( s 0 , a 0 ) to ( s K − 1 , a K − 1 ) for K -layer L O C A L.

Original: Then, our rewardfunction R ( · ) realizes a TO for our defined trajectories [31] since the ordering of trajectories can bedetermined by R ( · ) .
Revised: Then, our rewardfunction R ( · ) realizes a TO for our defined trajectories [31] since the ordering of trajectories can bedetermined by R ( · ) .

Original: More specifically, R ( · ) is trained with the end-of-trajectory reward R t for thet th trajectory in our paper and can rank trajectories.
Revised: More specifically, R ( · ) is trained with the end-of-trajectory reward R t for thet th trajectory in our paper and can rank trajectories.

Original: A reward bundle is an automation-like structureto produce rewards for a CMP [30].
Revised: A reward bundle is an automation-like structureto produce rewards for a CMP [30].

Original: By Corollary 2 of [30], there exists a reward bundle for ourdefined CMP and TO realized by R ( · ) .
Revised: By Corollary 2 of [30], there exists a reward bundle for ourdefined CMP and TO realized by R ( · ) .

Original: We pair our CMP with the reward bundle to form a Split Partially Observable MDP (Split-POMDP)[30].
Revised: We pair our CMP with the reward bundle to form a Split Partially Observable MDP (Split-POMDP)[30].

Original: Then, by Proposition 1 and Corollary 1 in [30], our Split-POMDP will always have an optimaldeterministic policy that only depends on states in our CMP.
Revised: Then, by Proposition 1 and Corollary 1 in [30], our Split-POMDP will always have an optimaldeterministic policy that only depends on states in our CMP.

Original: By the proof of Proposition 1 in [30],
Revised: By the proof of Proposition 1 in [30],

Original: Algorithm
Revised: Algorithm

Original: Input: Training dataset { x i , y i } Ni =1 .
Revised: Input: Training dataset { x i , y i } Ni =1 .

Original: Initialize:
Revised: Initialize:

Original: L O C
Revised: L O C

Original: A L layer number K , initial state s 

Revised: A L layer number K , initial state s

Original: =
Revised: =

Original: [ 1 , 0 ] T , discount factor γ ∈ (0 , 1) , ϵ for ϵ -greedy strategy, λ as a threshold to stop searching, ICNN for reward function − R ( s , a ) , ICNN for Q-function − Q ( s , a ) , replay buffer B = ∅ , maximum episode T , target network Q ′ ( · ) = Q ( · ) , and target network update interval T 0 .
Revised: [ 1 , 0 ] T , discount factor γ ∈ (0 , 1) , ϵ for ϵ -greedy strategy, λ as a threshold to stop searching, ICNN for reward function − R ( s , a ) , ICNN for Q-function − Q ( s , a ) , replay buffer B = ∅ , maximum episode T , target network Q ′ ( · ) = Q ( · ) , and target network update interval T 0 .

Original: while t ≤ T do while k ≤ K do
Revised: while t ≤ T do while k ≤ K do

Original: Solve Optimization in Equation (3) with − Q ( s tk , a ) to obtain ˜ a ∗ k .
Revised: Solve Optimization in Equation (3) with − Q ( s tk , a ) to obtain ˜ a ∗ k .

Original: Use ϵ -greedy to select ˜ a tk from ˜ a ∗ k and a random action.
Revised: Use ϵ -greedy to select ˜ a tk from ˜ a ∗ k and a random action.

Original: ▷ ϵ -greedy strategy.
Revised: ▷ ϵ -greedy strategy.

Original: Discretize ˜ a tk to obtain a tk .
Revised: Discretize ˜ a tk to obtain a tk .

Original: Execute a tk and use Equation (1) to obtain s kk +1 .
Revised: Execute a tk and use Equation (1) to obtain s kk +1 .

Original: Check if a tk and s kk +1 satisfy certain constraints.
Revised: Check if a tk and s kk +1 satisfy certain constraints.

Original: Otherwise, delete this state transition and restart the iteration from s tk .
Revised: Otherwise, delete this state transition and restart the iteration from s tk .

Original: ▷ Constraint checking.
Revised: ▷ Constraint checking.

Original: Formulate L O C
Revised: Formulate L O C

Original: A L, train L O C
Revised: A L, train L O C

Original: A L with { x i , y i } Ni =1 , and calculate R t .
Revised: A L with { x i , y i } Ni =1 , and calculate R t .

Original: Train the reward function − R ( · ) using training data {{ s tk , a tk } K − 1 k =0 , − R t } .
Revised: Train the reward function − R ( · ) using training data {{ s tk , a tk } K − 1 k =0 , − R t } .

Original: ∀ 0 ≤ k ≤ K , insert ( s tk , a tk , s tk +1 , R t ) and ( s tk , ˜ a t k , s tk
Revised: ∀ 0 ≤ k ≤ K , insert ( s tk , a tk , s tk +1 , R t ) and ( s tk , ˜ a t k , s tk

Original: +1 , R ( s tk , ˜ a t k )) to B 0 .
Revised: +1 , R ( s tk , ˜ a t k )) to B 0 .

Original: Sample a minibatch B 0 ⊂ B for ( s m , a m , s m +1 , R m ) ∈ B do ▷ Experience replay.
Revised: Sample a minibatch B 0 ⊂ B for ( s m , a m , s m +1 , R m ) ∈ B do ▷ Experience replay.

Original: Solve Optimization in Equation (3) with − Q ′ ( s m +1 , a ) to obtain ˜ a m +1 .
Revised: Solve Optimization in Equation (3) with − Q ′ ( s m +1 , a ) to obtain ˜ a m +1 .

Original: y m
Revised: y m

Original: = R m + γQ ′
Revised: = R m + γQ ′

Original: ( s m , a m )
Revised: ( s m , a m )

Original: Train Q ( · ) using training data { s m +1 , a m +1 , y m } m if t mod T 0 = 

Revised: Train Q ( · ) using training data { s m +1 , a m +1 , y m } m if t mod T 0 =

Original: then
Revised: then

Original: Q
Revised: Q

Original: [Equation]
Revised: [Equation]

Original: ▷ Update target Q-network.
Revised: ▷ Update target Q-network.

Original: if | R t − 1 | ≤ λ then
Revised: if | R t − 1 | ≤ λ then

Original: End the search process.
Revised: End the search process.

Original: Output: L O C A L with the best performance and the corresponding equations.
Revised: Output: L O C A L with the best performance and the corresponding equations.

Original: the optimal policy optimizes the value function over states in CMP.
Revised: the optimal policy optimizes the value function over states in CMP.

Original: Further, the value function isan evaluation of trajectories for our TO by the proof in Corollary 2 in [30].
Revised: Further, the value function isan evaluation of trajectories for our TO by the proof in Corollary 2 in [30].

Original: Additionally, our TO isrealized by our proposed reward function R ( · ) .
Revised: Additionally, our TO isrealized by our proposed reward function R ( · ) .

Original: Therefore, the optimal Q-function exists for our CMPand our proposed R ( · ) .
Revised: Therefore, the optimal Q-function exists for our CMPand our proposed R ( · ) .

Original: Then, we consider the Bellman Equation of Q ∗ ( · ) :
Revised: Then, we consider the Bellman Equation of Q ∗ ( · ) :

Original: [Equation]
Revised: [Equation]

Original: where the second equality holds since our state transitions are deterministic by Equation (1).
Revised: where the second equality holds since our state transitions are deterministic by Equation (1).

Original: Weprove the convexity from the induction method.
Revised: Weprove the convexity from the induction method.

Original: When k = K − 1 , the ( k + 1) th state is the terminalstate without action selections.
Revised: When k = K − 1 , the ( k + 1) th state is the terminalstate without action selections.

Original: Thus, we have
Revised: Thus, we have

Original: [Equation]
Revised: [Equation]

Original: Since − R ( · ) is an ICNN and is convex in input, − Q ∗ ( s K − 1 , ˜ a K − 1 ) is convex in s K − 1 and ˜ a K − 1 .
Revised: Since − R ( · ) is an ICNN and is convex in input, − Q ∗ ( s K − 1 , ˜ a K − 1 ) is convex in s K − 1 and ˜ a K − 1 .

Original: When 0 ≤ k < K − 1 and assume − Q ∗ ( s k +1 , ˜ a k +1 ) is convex in s k +1 and ˜ a k +1 , we have− max ˜ a Q ∗ ( s k +1 , ˜ a )
Revised: When 0 ≤ k < K − 1 and assume − Q ∗ ( s k +1 , ˜ a k +1 ) is convex in s k +1 and ˜ a k +1 , we have− max ˜ a Q ∗ ( s k +1 , ˜ a )

Original: =
Revised: =

Original: min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in s k +1 given the fixed optimal action.
Revised: min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in s k +1 given the fixed optimal action.

Original: Let
Revised: Let

Original: H denote the Hessian matrix of min ˜ a − Q ∗ ( s k +1 , ˜ a ) with respect to s k +1 .
Revised: H denote the Hessian matrix of min ˜ a − Q ∗ ( s k +1 , ˜ a ) with respect to s k +1 .

Original: Due to the convexity,
Revised: Due to the convexity,

Original: H is positive semi-definite.
Revised: H is positive semi-definite.

Original: Thus, by Equation (1) and the chain rule, the Hessian matrix ofmin ˜ a − Q ∗ ( s k +1 , ˜ a ) with respect to s k can be written as:
Revised: Thus, by Equation (1) and the chain rule, the Hessian matrix ofmin ˜ a − Q ∗ ( s k +1 , ˜ a ) with respect to s k can be written as:

Original: [Equation]
Revised: [Equation]

Original: H ′ is also positive semi-definite.
Revised: H ′ is also positive semi-definite.

Original: Therefore, min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in s k .
Revised: Therefore, min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in s k .

Original: Since − R ( s k , ˜ a k )is convex in s k , − Q ∗ ( s k , ˜ a k ) is convex in s k .
Revised: Since − R ( s k , ˜ a k )is convex in s k , − Q ∗ ( s k , ˜ a k ) is convex in s k .

Original: Similarly, vectorizing the state transition equation can give:
Revised: Similarly, vectorizing the state transition equation can give:

Original: [Equation]
Revised: [Equation]

Original: where I n s is the n s × n s identity matrix and (cid:78) is the Kronecker product.
Revised: where I n s is the n s × n s identity matrix and (cid:78) is the Kronecker product.

Original: a ′ k
Revised: a ′ k

Original: =
Revised: =

Original: [( a k ) T , 0 ] T is theconcatenation of the discrete action a k and a zero vector to maintain the fixed dimensionality ofaction vectors.
Revised: [( a k ) T , 0 ] T is theconcatenation of the discrete action a k and a zero vector to maintain the fixed dimensionality ofaction vectors.

Original: With similar proofs based on the Hessian matrix and the fact that − Q ∗ ( s k +1 , ˜ a k ) isconvex in s k +1 , we have min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in a ′ k and also a k .
Revised: With similar proofs based on the Hessian matrix and the fact that − Q ∗ ( s k +1 , ˜ a k ) isconvex in s k +1 , we have min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in a ′ k and also a k .

Original: Subsequently, arbitrary˜ a k ∈ conv ( { 0 , 1 } n a ) can be written as a convex combination of the discrete actions a k .
Revised: Subsequently, arbitrary˜ a k ∈ conv ( { 0 , 1 } n a ) can be written as a convex combination of the discrete actions a k .

Original: Thus,min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in ˜ a k .
Revised: Thus,min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in ˜ a k .

Original: Since − R ( s k , ˜ a k ) is convex in ˜ a k , − Q ∗ ( s k , ˜ a k ) is convex in˜ a k .
Revised: Since − R ( s k , ˜ a k ) is convex in ˜ a k , − Q ∗ ( s k , ˜ a k ) is convex in˜ a k .

Original: Eventually, − Q ∗ ( s k , ˜ a k ) is convex in s k and ˜ a k , which concludes the proof.
Revised: Eventually, − Q ∗ ( s k , ˜ a k ) is convex in s k and ˜ a k , which concludes the proof.

Original: ■
Revised: ■

Original: A.4 Proofs of Theorem
Revised: A.4 Proofs of Theorem

Original: Theorem.
Revised: Theorem.

Original: Let f ∗ ( · ; W ) denote the L O C A L constructed by the optimal sequences of states( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) from − Q ∗ ( · ) , where W is the set of weightsof f ∗ ( · ; W ) .
Revised: Let f ∗ ( · ; W ) denote the L O C A L constructed by the optimal sequences of states( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) from − Q ∗ ( · ) , where W is the set of weightsof f ∗ ( · ; W ) .

Original: If f ∗ ( · ; W ) can be trained with noiseless datasets and the training can achieve theglobal optimal weights W ∗ , f ∗ ( · ; W ∗ ) can be simplified to the true equation g ( · ) .
Revised: If f ∗ ( · ; W ) can be trained with noiseless datasets and the training can achieve theglobal optimal weights W ∗ , f ∗ ( · ; W ∗ ) can be simplified to the true equation g ( · ) .

Original: Proof.
Revised: Proof.

Original: If f ∗ ( · ; W ) can’t represent the exact equations, there are two cases: (1) the structure off ∗ ( · ; W ) is correct to represent the equations, but the learned weights W ∗ don’t represent thesymbol coefficients, and (2) the structure of f ∗ ( · ; W ) can’t represent the equations.
Revised: If f ∗ ( · ; W ) can’t represent the exact equations, there are two cases: (1) the structure off ∗ ( · ; W ) is correct to represent the equations, but the learned weights W ∗ don’t represent thesymbol coefficients, and (2) the structure of f ∗ ( · ; W ) can’t represent the equations.

Original: Case (1) doesn’thold since we assume W ∗ is the global optimal weights for noiseless data.
Revised: Case (1) doesn’thold since we assume W ∗ is the global optimal weights for noiseless data.

Original: If case (2) holds,∃ 

Revised: If case (2) holds,∃

Original: ≤ j ≤ K − 1 , b ∗ j = min ˜ a j − Q ∗ ( s j , ˜ a j ) and b ∗ j doesn’t represent the symbol connections in theunderlying equations.
Revised: ≤ j ≤ K − 1 , b ∗ j = min ˜ a j − Q ∗ ( s j , ˜ a j ) and b ∗ j doesn’t represent the symbol connections in theunderlying equations.

Original: Further, we assume ∀ 0 ≤ i < j , a ∗ i =
Revised: Further, we assume ∀ 0 ≤ i < j , a ∗ i =

Original: min
Revised: min

Original: ˜ a i − Q ( s i , ˜ a i ) and a ∗ i representsthe true connections.
Revised: ˜ a i − Q ( s i , ˜ a i ) and a ∗ i representsthe true connections.

Original: If j = K − 1 , Equation (4) implies that ˜ a ∗ j = min ˜ a j − Q ∗ ( s j , ˜ a j ) = arg min ˜ a − R ( s j , ˜ a ) .
Revised: If j = K − 1 , Equation (4) implies that ˜ a ∗ j = min ˜ a j − Q ∗ ( s j , ˜ a j ) = arg min ˜ a − R ( s j , ˜ a ) .

Original: Since− R ( s j , ˜ a ) is convex in ˜ a
Revised: Since− R ( s j , ˜ a ) is convex in ˜ a

Original: we know the discrete version of ˜ a ∗ j , namely a ∗ j , represents the trueconnection of the last layer for the underlying equations.
Revised: we know the discrete version of ˜ a ∗ j , namely a ∗ j , represents the trueconnection of the last layer for the underlying equations.

Original: Otherwise, the reward is not maximized.
Revised: Otherwise, the reward is not maximized.

Original: However, by definition of b ∗ j , b ∗ j ̸ = a ∗ j .
Revised: However, by definition of b ∗ j , b ∗ j ̸ = a ∗ j .

Original: If j < K − 1 , Equation (4) implies:
Revised: If j < K − 1 , Equation (4) implies:

Original: [Equation]
Revised: [Equation]

Original: By definition of b ∗ j , b ∗ j is not the solution of Equation (5).
Revised: By definition of b ∗ j , b ∗ j is not the solution of Equation (5).

Original: This is because b ∗ j can’t achieve theminimum value for each summation term on the right hand side of Equation (5), according to theconvexity of the reward function.
Revised: This is because b ∗ j can’t achieve theminimum value for each summation term on the right hand side of Equation (5), according to theconvexity of the reward function.

Original: In general, b ∗ j ̸ = min ˜ a j − Q ∗ ( s j , ˜ a j ) , which contradicts thedefinition of b ∗ j .
Revised: In general, b ∗ j ̸ = min ˜ a j − Q ∗ ( s j , ˜ a j ) , which contradicts thedefinition of b ∗ j .

Original: Thus, b ∗ j doesn’t exist.
Revised: Thus, b ∗ j doesn’t exist.

Original: Therefore, case (2) doesn’t hold and f ∗ ( · ; W ∗ ) representsthe exact equations.
Revised: Therefore, case (2) doesn’t hold and f ∗ ( · ; W ∗ ) representsthe exact equations.

Original: ■ A.5 Proofs of Theorem 3
Revised: ■A.5 Proofs of Theorem 3

Original: Theorem.
Revised: Theorem.

Original: Assume the following conditions hold: (1) the equation g ( x ) is C 2 smooth and hasbounded second derivatives with respect to weights, (2) ∃ x ∈ X , g ( x ) has non-zero gradients withrespect to weights, (3) the structure of L O C A L is correctly searched to exactly represent symbolsand symbol connections in g ( x ) , and (4) the training dataset of L O C
Revised: Assume the following conditions hold: (1) the equation g ( x ) is C 2 smooth and hasbounded second derivatives with respect to weights, (2) ∃ x ∈ X , g ( x ) has non-zero gradients withrespect to weights, (3) the structure of L O C A L is correctly searched to exactly represent symbolsand symbol connections in g ( x ) , and (4) the training dataset of L O C

Original: A L is noiseless.
Revised: A L is noiseless.

Original: Then, for the
Revised: Then, for the

Original: MSE loss surface of L O C A L , each global optimal point has a strictly convex local region.
Revised: MSE loss surface of L O C A L , each global optimal point has a strictly convex local region.

Original: Proof.
Revised: Proof.

Original: To simplify the proof, we consider scalar output of the L O C
Revised: To simplify the proof, we consider scalar output of the L O C

Original: A L, i.e., one equation, and theproof can be easily extended to the multi-output case.
Revised: A L, i.e., one equation, and theproof can be easily extended to the multi-output case.

Original: We follow the idea of [29] to study the secondderivative of L O C A L with perturbations.
Revised: We follow the idea of [29] to study the secondderivative of L O C A L with perturbations.

Original: Let ˆ y ( x , W ) denote the L O C A L with input to be x and theweight set to be W .
Revised: Let ˆ y ( x , W ) denote the L O C A L with input to be x and theweight set to be W .

Original: Let X be a perturbation direction of W and t be a small step size.
Revised: Let X be a perturbation direction of W and t be a small step size.

Original: For the i thnoiseless instance ( x i , y i ) , we denote e ( x i , W + tX )
Revised: For the i thnoiseless instance ( x i , y i ) , we denote e ( x i , W + tX )

Original: =
Revised: =

Original: ˆ y ( x i , W + tX ) − y i .
Revised: ˆ y ( x i , W + tX ) − y i .

Original: Obviously, the lossfunction can be written as L ( W + tX )
Revised: Obviously, the lossfunction can be written as L ( W + tX )

Original: = 12 N (cid:80) Ni
Revised: = 12 N (cid:80) Ni

Original: =1 ( e ( x i , W + tX )) 2 .
Revised: =1 ( e ( x i , W + tX )) 2 .

Original: Then, we can calculate the second-order derivative based on the chain rule:
Revised: Then, we can calculate the second-order derivative based on the chain rule:

Original: [Equation]
Revised: [Equation]

Original: Next, we denote the global optimal solution to be W ∗ .
Revised: Next, we denote the global optimal solution to be W ∗ .

Original: Based on the Assumptions (3)and (4) , ∀ i, ˆ y ( x i , W ∗ )
Revised: Based on the Assumptions (3)and (4) , ∀ i, ˆ y ( x i , W ∗ )

Original: =
Revised: =

Original: g ( x i ) =
Revised: g ( x i ) =

Original: y i .
Revised: y i .

Original: Therefore, we have d 2 dt 2 (cid:12)(cid:12) t =

Revised: Therefore, we have d 2 dt 2 (cid:12)(cid:12) t =

Original: L ( W ∗ + tX ) =
Revised: L ( W ∗ + tX ) =

Original: 1 N (cid:80) Ni =1 (cid:0) ddt (cid:12)(cid:12) t =

Revised: 1 N [Equation] Ni =1 (cid:0) ddt (cid:12)(cid:12) t =

Original: ˆ y
Revised: ˆ y

Original: ( x i , W ∗ ) (cid:1) 2 > 0 , where the inequality strictly holds.
Revised: ( x i , W ∗ ) (cid:1) 2 > 0 , where the inequality strictly holds.

Original: This is because by Assumptions (3) , ˆ y ( x , W ∗ ) can be mathematically simplified to obtain g ( x ) .
Revised: This is because by Assumptions (3) , ˆ y ( x , W ∗ ) can be mathematically simplified to obtain g ( x ) .

Original: Then, by Assumption (2) ,1 N (cid:80) Ni =1 (cid:0) ddt (cid:12)(cid:12) t
Revised: Then, by Assumption (2) ,1 N (cid:80) Ni =1 (cid:0) ddt (cid:12)(cid:12) t

Original: =

Revised: =

Original: ˆ y
Revised: ˆ y

Original: ( x i , W ∗ ) (cid:1) 2 > 0 .
Revised: ( x i , W ∗ ) (cid:1) 2 > 0 .

Original: Finally, by Assumption (1) and (3) , d 2 dt 2 (cid:12)(cid:12) t =

Revised: Finally, by Assumption (1) and (3) , d 2 dt 2 (cid:12)(cid:12) t =

Original: ˆ y
Revised: ˆ y

Original: ( x i , W + tX ) isbounded and there is a local region around W ∗ such that d 2 dt 2 (cid:12)(cid:12) t
Revised: ( x i , W + tX ) isbounded and there is a local region around W ∗ such that d 2 dt 2 (cid:12)(cid:12) t

Original: =

Revised: =

Original: L ( W + tX ) > 0 , which concludesthe proof.
Revised: L ( W + tX ) > 0 , which concludesthe proof.

Original: ■
Revised: ■

Revised: [Table]

Original: A.6 Proofs of Theorem 4
Revised: A.6 Proofs of Theorem 4

Original: Theorem.
Revised: Theorem.

Original: Suppose Assumptions 1-4 in Theorem 3 hold.
Revised: Suppose Assumptions 1-4 in Theorem 3 hold.

Original: For a L O C A L with one symbolic activation,multiplication, and summation layer, the set of local convex regions with global optima is U =
Revised: For a L O C A L with one symbolic activation,multiplication, and summation layer, the set of local convex regions with global optima is U =

Original: [Equation]
Revised: [Equation]

Original: |} , where notations are defined in the proof.
Revised: } , where notations are defined in the proof.

Original: Proof.
Revised: Proof.

Original: For the target L O C A L, we similarly consider the scalar output and write the function analytically:
Revised: For the target L O C A L, we similarly consider the scalar output and write the function analytically:

Original: [Equation]
Revised: [Equation]

Original: where W 0 ∈ R n 0 × n 1 is the weight matrix for activation, Φ : R n 1 → R n 1 represents the activationwith symbol functions like x 2 , cos ( x ) , and log( x ) , etc.
Revised: where W 0 ∈ R n 0 × n 1 is the weight matrix for activation, Φ : R n 1 → R n 1 represents the activationwith symbol functions like x 2 , cos ( x ) , and log( x ) , etc.

Original: Ψ : R n 1 → R n 2 is the function to selectsome activated neurons for multiplications, and W 1 ∈ R n 2 × n 3 ( n 3 = 1 ) represents the weight forsummation.
Revised: Ψ : R n 1 → R n 2 is the function to selectsome activated neurons for multiplications, and W 1 ∈ R n 2 × n 3 ( n 3 = 1 ) represents the weight forsummation.

Original: We rewrite Equation (7) with the help of exponential and logarithm mappings.
Revised: We rewrite Equation (7) with the help of exponential and logarithm mappings.

Original: [Equation]
Revised: [Equation]

Original: where S ∈ R n 1 × n 2 represents a selection matrix such that S [ i, j ] = 1 if and only if the i th neuronis selected as the multiplicative factor for the j th neuron in the multiplication layer.
Revised: where S ∈ R n 1 × n 2 represents a selection matrix such that S [ i, j ] = 1 if and only if the i th neuronis selected as the multiplicative factor for the j th neuron in the multiplication layer.

Original: Given thefixed structure of ˆ y ( · ) from the deep Q-learning, S is a known matrix.
Revised: Given thefixed structure of ˆ y ( · ) from the deep Q-learning, S is a known matrix.

Original: log( · ) and exp( · ) representthe element-wise logarithm and exponential functions.
Revised: log( · ) and exp( · ) representthe element-wise logarithm and exponential functions.

Original: Notably, the corresponding element inΦ( W T 0 x ) should be positive in Equation (8).
Revised: Notably, the corresponding element inΦ( W T 0 x ) should be positive in Equation (8).

Original: If there are negative entries, one can utilize W T 1 s ◦exp (cid:18)
Revised: If there are negative entries, one can utilize W T 1 s ◦exp (cid:18)

Original: S T log (cid:16) | Φ( W T 0 x ) | (cid:17)(cid:19) to take place of the right hand side term in Equation (8), where s [ i ] =( − 1) n i − and 0 ≤ n i − ≤ n 1 represents the number of negative entries selected for the i th neuron ofthe multiplication layer.
Revised: S T log (cid:16) | Φ( W T 0 x ) | (cid:17)(cid:19) to take place of the right hand side term in Equation (8), where s [ i ] =( − 1) n i − and 0 ≤ n i − ≤ n 1 represents the number of negative entries selected for the i th neuron ofthe multiplication layer.

Original: ◦ represents the Hadamard product.
Revised: ◦ represents the Hadamard product.

Original: However, both expressions have thesame values and gradients.
Revised: However, both expressions have thesame values and gradients.

Original: Thus, we utilize Equation (8) in later derivations.
Revised: Thus, we utilize Equation (8) in later derivations.

Original: Then, let X be a perturbation direction such that X
Revised: Then, let X be a perturbation direction such that X

Original: = { X 0 , X 1 } .
Revised: = { X 0 , X 1 } .

Original: Thus, for a small step t , we have:
Revised: Thus, for a small step t , we have:

Original: [Equation]
Revised: [Equation]

Original: Based on Equation (9), we can compute:
Revised: Based on Equation (9), we can compute:

Original: [Equation]
Revised: [Equation]

Original: where 1 Φ(( W 0 + t X 0 )
Revised: where 1 Φ(( W 0 + t X 0 )

Original: T x i ) ∈ R n 1 is the element-wise division and Φ ′ is the element-wise first derivativeof Φ ′ .
Revised: T x i ) ∈ R n 1 is the element-wise division and Φ ′ is the element-wise first derivativeof Φ ′ .

Original: Without special notifications, we assume all the division for vectors is element-wise in thefollowing derivations.
Revised: Without special notifications, we assume all the division for vectors is element-wise in thefollowing derivations.

Original: Then, we denote
Revised: Then, we denote

Original: [Equation]
Revised: [Equation]

Original: With above definitions
Revised: With above definitions

Original: [Equation]
Revised: [Equation]

Original: we can calculate:
Revised: we can calculate:

Original: [Equation]
Revised: [Equation]

Original: Further, we calculate the second derivative based on Equation (10) and the fact that element-wiseoperations for vectors are commutative:
Revised: Further, we calculate the second derivative based on Equation (10) and the fact that element-wiseoperations for vectors are commutative:

Original: [Equation]
Revised: [Equation]

Original: When t → 0 , we have:
Revised: When t → 0 , we have:

Original: [Equation]
Revised: [Equation]

Original: The above equation can reflect the relationship between the second and the first derivative.
Revised: The above equation can reflect the relationship between the second and the first derivative.

Original: However,we first identify the inequality between these two derivatives to enable a strictly convex region.
Revised: However,we first identify the inequality between these two derivatives to enable a strictly convex region.

Original: Let ˆ
Revised: Let ˆ

Original: [Equation]
Revised: [Equation]

Original: y ( x 1 , W +tX ) , · · · , ddt 2 (cid:12)(cid:12) t
Revised: y ( x 1 , W +tX ) , · · · , ddt 2 (cid:12)(cid:12) t

Original: =

Revised: =

Original: ˆ y ( x N , W + tX )]
Revised: ˆ y ( x N , W + tX )]

Original: T , and e =
Revised: T , and e =

Original: [ e ( x 1 , W ) , · · · , e ( x N , W )] T .
Revised: [ e ( x 1 , W ) , · · · , e ( x N , W )] T .

Original: Equation (6) impliesthat:
Revised: Equation (6) impliesthat:

Original: [Equation]
Revised: [Equation]

Original: To find a region to restrict the convexity, we restrict the lower bound of the second derivative to bepositive and compute:
Revised: To find a region to restrict the convexity, we restrict the lower bound of the second derivative to bepositive and compute:

Original: [Equation]
Revised: [Equation]

Original: The right hand side of Equation (16) can be easily bounded by:
Revised: The right hand side of Equation (16) can be easily bounded by:

Original: [Equation]
Revised: [Equation]

Original: where |·| for a vector is to calculate the absolute value for each element of the vector
Revised: where |·| for a vector is to calculate the absolute value for each element of the vector

Original: [Equation]
Revised: [Equation]

Original: i = arg min( | ˆ y ′ | )and j = arg max( | ˆ y ′′ | ) .
Revised: i = arg min( | ˆ y ′ | )and j = arg max( | ˆ y ′′ | ) .

Original: Namely, we consider a sufficient condition for convexity.
Revised: Namely, we consider a sufficient condition for convexity.

Original: [Equation]
Revised: [Equation]

Original: Next, Equation (14) indicates that:
Revised: Next, Equation (14) indicates that:

Original: [Equation]
Revised: [Equation]

Original: where η is a positive constant.
Revised: where η is a positive constant.

Original: Note that η < ∞ by Assumptions (1) and (2) in Theorem 3.
Revised: Note that η < ∞ by Assumptions (1) and (2) in Theorem 3.

Original: Therefore,we have the following sufficient condition to make d 2 dt 2 (cid:12)(cid:12) t =

Revised: Therefore,we have the following sufficient condition to make d 2 dt 2 (cid:12)(cid:12) t =

Original: L ( W + tX ) > 0 always hold.
Revised: L ( W + tX ) > 0 always hold.

Original: [Equation]
Revised: [Equation]

Original: where k = arg max( | e | )
Revised: where k = arg max( | e | )

Original: The above equation leads to a set U of local regions that have strongconvexity.
Revised: The above equation leads to a set U of local regions that have strongconvexity.

Original: Namely,
Revised: Namely,

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Clearly, the global optimal solution W ∗ ∈ U since ˆ y ( x k , W ∗ ) − y k = 0 .
Revised: Clearly, the global optimal solution W ∗ ∈ U since ˆ y ( x k , W ∗ ) − y k = 0 .

Original: Note that there may bemultiple global optimal solutions of the loss minimization in L O C A
Revised: Note that there may bemultiple global optimal solutions of the loss minimization in L O C A

Original: L. Thus, U is the set of localconvex regions that contain global optima.
Revised: L. Thus, U is the set of localconvex regions that contain global optima.

Original: This implies that for each W ∗ ∈
Revised: This implies that for each W ∗ ∈

Original: U , we can find a locallyand strictly convex region U ∗ =
Revised: U , we can find a locallyand strictly convex region U ∗ =

Original: U ∩ B ( r ) , where B ( r ) =
Revised: U ∩ B ( r ) , where B ( r ) =

Original: ||
Revised: ||

Original: w − w ∗
Revised: w − w ∗

Original: ||
Revised: ||

Original: 2 ≤ r is a norm ball and wevectorize W and W ∗ to obtain w and w ∗ , respectively.
Revised: 2 ≤ r is a norm ball and wevectorize W and W ∗ to obtain w and w ∗ , respectively.

Original: Subsequently, range r can be set relativelylarge such that U ∗ ⊂ B ( r ) and U ∗∗ ∩ B ( r ) = ∅ , where U ∗∗ is the local region for another globaloptimal point W ∗∗ if it exists.
Revised: Subsequently, range r can be set relativelylarge such that U ∗ ⊂ B ( r ) and U ∗∗ ∩ B ( r ) = ∅ , where U ∗∗ is the local region for another globaloptimal point W ∗∗ if it exists.

Original: Then, the range for U ∗ still depends on the inequality in Equation(21).
Revised: Then, the range for U ∗ still depends on the inequality in Equation(21).

Original: ■
Revised: ■