Original: C O NS O L E : Convex Neural Symbolic LearningAnonymous Author(s)
Revised: C O NS O L E : Convex Neural Symbolic Learning

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Learning the underlying equation from data is a fundamental problem in manydisciplines.
Revised: Learning the underlying equation from data is a fundamental problem in many disciplines.

Original: Recent advances rely on Neural Networks (NNs) but do not providetheoretical guarantees in obtaining the exact equations owing to the non-convexityof NNs.
Revised: Recent advances rely on Neural Networks (NNs) but do not provide theoretical guarantees in obtaining the exact equations owing to the non-convexity of NNs.

Original: In this paper, we propose Convex Neural Symbolic Learning (C O NS O L E )to seek convexity under mild conditions.
Revised: In this paper, we propose Convex Neural Symbolic Learning (C O NS O L E ) to seek convexity under mild conditions.

Original: The main idea is to decompose therecovering process into two steps and convexify each step.
Revised: The main idea is to decompose the recovering process into two steps and convexify each step.

Original: In the first step ofsearching for right symbols, we convexify the deep Q-learning.
Revised: In the first step of searching for right symbols, we convexify the deep Q-learning.

Original: The key is tomaintain double convexity for both the negative Q-function and the negative rewardfunction in each iteration, leading to provable convexity of the negative optimal Qfunction to learn the true symbol connections.
Revised: The key is to maintain double convexity for both the negative Q-function and the negative reward function in each iteration, leading to provable convexity of the negative optimal Q function to learn the true symbol connections.

Original: Conditioned on the exact searchingresult, we construct a Locally-Convex equation Learner ( L O C A L) neural networkto convexify the estimation of symbol coefficients.
Revised: Conditioned on the exact searching result, we construct a Locally-Convex equation Learner (L O C A L) neural network to convexify the estimation of symbol coefficients.

Original: With such a design, we quantifya large region with strict convexity in the loss surface of L O C A L for commonlyused physical functions.
Revised: With such a design, we quantify a large region with strict convexity in the loss surface of L O C A L for commonly used physical functions.

Original: Finally, we demonstrate the superior performance of the
Revised: Finally, we demonstrate the superior performance of the

Original: C O NS O L E framework over the state-of-the-art on a diverse set of datasets.
Revised: C O NS O L E framework over the state-of-the-art on a diverse set of datasets.

Original:  Introduction
Revised:  Introduction

Original: Identifying the underlying mathematical expressions from data plays a key role in multiple domains.
Revised: Identifying the underlying mathematical expressions from data plays a key role in multiple domains.

Original: For example, scientific discovery naturally requires learning analytical solutions to fit data.
Revised: For example, scientific discovery naturally requires learning analytical solutions to fit data.

Original: Engineering systems also need to frequently re-estimate system equations due to events, maintenance,upgrading and new constructions [1] etc.
Revised: Engineering systems also need to frequently re-estimate system equations due to events, maintenance, upgrading, and new constructions [1], etc.

Original: In general, the problem, known as Symbolic Regression (SR) [2], learns the underlying equations y
Revised: In general, the problem, known as Symbolic Regression (SR) [2], learns the underlying equations y

Original: = g ( x ) constructed via certain symbols, where xand y represent the input/output vector of the equations.
Revised: = g ( x ) constructed via certain symbols, where x and y represent the input/output vector of the equations.

Original: If successfully learned, the equation canenjoy many important properties like high interpretability and generalizability, which will in turnsignificantly benefit scientific understanding and engineering planning, monitoring and control.
Revised: If successfully learned, the equation can enjoy many important properties like high interpretability and generalizability, which will in turn significantly benefit scientific understanding and engineering planning, monitoring and control.

Original: Promising as it might be, SR is NP-hard [3, 4].
Revised: Promising as it might be, SR is NP-hard [3, 4].

Original: Mathematically, one can cast SR as an optimizationproblem over both discrete variables to select symbols and continuous variables to represent thesymbol coefficients.
Revised: Mathematically, one can cast SR as an optimization problem over both discrete variables to select symbols and continuous variables to represent the symbol coefficients.

Original: Traditional solutions employ evolutionary algorithms like Genetic Programming(GP)
Revised: Traditional solutions employ evolutionary algorithms like Genetic Programming (GP) [5].

Original: These methods start from an initial set of expressions and continue evolving via operationslike crossover or mutation.
Revised: These methods start from an initial set of expressions and continue evolving via operations like crossover or mutation.

Original: With fitness measures, GP-based algorithms can evaluate and select thebest equations.
Revised: With fitness measures, GP-based algorithms can evaluate and select the best equations.

Original: However, these methods have poor scalability and limited theoretical guarantees [3].
Revised: However, these methods have poor scalability and limited theoretical guarantees [3].

Original: More recent SR studies leverage Neural Networks (NNs) with high representational power.
Revised: More recent SR studies leverage Neural Networks (NNs) with high representational power.

Original: For the
Revised: For the

Original: NN-based SR, we mainly categorize them into two groups based on the roles of the NNs.
Revised: NN-based SR, we mainly categorize them into two groups based on the roles of the NNs.

Original: The firstgroup employs NNs to directly model the equations, where sparsity of the NN weights is enforced toselect symbols [6–10].
Revised: The first group employs NNs to directly model the equations, where sparsity of the NN weights is enforced to select symbols [6–10].

Original: Thus, the problem is transformed into training the designed NN with sparsityregularization.
Revised: Thus, the problem is transformed into training the designed NN with sparsity regularization.

Original: However, due to the non-convexity of NNs, the weight selection and updating can beeasily stuck in local optima, failing to find the exact equations.
Revised: However, due to the non-convexity of NNs, the weight selection and updating can be easily stuck in local optima, failing to find the exact equations.

Original: The second group employs NNs to search the symbol connections, and non-linear optimizations like
Revised: The second group employs NNs to search the symbol connections, and non-linear optimizations like

Original: BFGS [11] can be employed to estimate symbol coefficients.
Revised: BFGS [11] can be employed to estimate symbol coefficients.

Original: For the searching procedure, [3, 12–14]leverage a Recursive Neural Network (RNN) as a policy network to iteratively generate optimal actions that can select and connect symbols.
Revised: For the searching procedure, [3, 12–14] leverage a Recursive Neural Network (RNN) as a policy network to iteratively generate optimal actions that can select and connect symbols.

Original: [15] employs large-scale pre-training to directly mapfrom data to the symbolic equations.
Revised: [15] employs large-scale pre-training to directly map from data to the symbolic equations.

Original: While these methods restrict the utilization of NNs in thesearch phase, the non-convexity of NNs can still suffer the risk of sub-optimal decisions to formulateequations.
Revised: While these methods restrict the utilization of NNs in the search phase, the non-convexity of NNs can still suffer the risk of sub-optimal decisions to formulate equations.

Original: To mitigate this issue, some efforts have been made such as risk-seeking policy gradient tofind the best equations [3], restricting the searching space via domain knowledge [16] and entropyregularization [13], and re-initialization [13], etc.
Revised: To mitigate this issue, some efforts have been made such as risk-seeking policy gradient to find the best equations [3], restricting the searching space via domain knowledge [16] and entropy regularization [13], and re-initialization [13], etc.

Original: However, they have limited theoretical guarantees.
Revised: However, they have limited theoretical guarantees.

Original: In this paper, we propose Convex Neural Symbolic Learning (C O NS O L E ) with convexity undermoderate conditions.
Revised: In this paper, we propose Convex Neural Symbolic Learning (C O NS O L E ) with convexity under moderate conditions.

Original: To our best knowledge, we are the first to provide provable guarantees to learnthe exact equations.
Revised: To our best knowledge, we are the first to provide provable guarantees to learn the exact equations.

Original: In general, we decompose the SR problem into two sub-problems and seekconvexity, respectively.
Revised: In general, we decompose the SR problem into two sub-problems and seek convexity, respectively.

Original: In the first problem of searching symbols, we propose a double-convexifieddeep Q-learning to maintain the convexity of negative Q-function and negative reward functionswith continuous action variables.
Revised: In the first problem of searching symbols, we propose a double-convexified deep Q-learning to maintain the convexity of negative Q-function and negative reward functions with continuous action variables.

Original: Specifically, we use the Input Convex Neural Network (ICNN)[17] to represent both the negative Q-function and the negative reward function in each iteration.
Revised: Specifically, we use the Input Convex Neural Network (ICNN) [17] to represent both the negative Q-function and the negative reward function in each iteration.

Original: Subsequently, we prove that such a design (1) guarantees an optimal action selection at each stepand (2) ensures the negative optimal Q-function, if successfully found, to be convex.
Revised: Subsequently, we prove that such a design (1) guarantees an optimal action selection at each step and (2) ensures the negative optimal Q-function, if successfully found, to be convex.

Original: Therefore, theconvex negative optimal Q-function can yield global optimal decisions of equation constructions.
Revised: Therefore, the convex negative optimal Q-function can yield global optimal decisions of equation constructions.

Original: In the second problem of coefficient estimation, we use the search result to build a Locally-Convex
Revised: In the second problem of coefficient estimation, we use the search result to build a Locally-Convex

Original: Equation Learner (L O C A L) neural network.
Revised: Equation Learner (L O C A L) neural network.

Original: The key insight is that if the search result is correct,the loss surface of L O C A L has local regions that contain the global optima and has strict convexity.
Revised: The key insight is that if the search result is correct, the loss surface of L O C A L has local regions that contain the global optima and has strict convexity.

Original: Moreover, we quantify the local regions and show the range of the region is large under mildassumptions.
Revised: Moreover, we quantify the local regions and show the range of the region is large under mild assumptions.

Original: Therefore, initializations based on prior knowledge can often lie in the convex region,bringing the accurate coefficient estimation.
Revised: Therefore, initializations based on prior knowledge can often lie in the convex region, bringing the accurate coefficient estimation.

Original: Finally, we demonstrate that our C O NS O L E outperformsstate-of-the-art methods on a diverse set of datasets.
Revised: Finally, we demonstrate that our C O NS O L E outperforms state-of-the-art methods on a diverse set of datasets.

Original:  Related Work
Revised:  Related Work

Original: Symbolic regression using neural networks.
Revised: Symbolic regression using neural networks.

Original: In addition to the direct use of NN in the Introduction,there are studies for indirect use. They typically treat NNs as a data augmentation tool to createhigh-quality data for learning or evaluation [18, 2].
Revised: In addition to the review in the Introduction, there are studies treating NNs as a data augmentation tool to create high-quality data for SR [18, 2].

Original: Neural architecture search.
Revised: Neural architecture search.

Original: Searching the connections of L O C A L falls into the area of Neural
Revised: Searching the connections of L O C A L falls into the area of Neural

Original: Architecture Search (NAS).
Revised: Architecture Search (NAS).

Original: NAS tries to find an optimal architecture of a target NN with the bestperformance [19].
Revised: NAS tries to find an optimal architecture of a target NN with the best performance [19].

Original: The searching algorithms can be divided into RL-based, evolutionary algorithmbased, sequential optimization-based, and gradient descent-based methods.
Revised: The searching algorithms can be divided into RL-based, evolutionary algorithmbased, sequential optimization-based, and gradient descent-based methods.

Original: For RL-based methods,[20] employs an RNN model to sample the architecture and utilizes the accuracy of the samplednetwork as the reward.
Revised: For RL-based methods, [20] employs an RNN model to sample the architecture and utilizes the accuracy of the sampled network as the reward.

Original: [21] uses tabular Q-learning to find the connections of a target NN.
Revised: [21] uses tabular Q-learning to find the connections of a target NN.

Original: However,tabular Q-learning can hardly be applicable when the state and action spaces are large, e.g., SRproblem.
Revised: However, tabular Q-learning can hardly be applicable when the state and action spaces are large, e.g., SR problem.

Original: The evolutionary algorithm employs methods such as GP [22] and tournament selection [23].
Revised: The evolutionary algorithm employs methods such as GP [22] and tournament selection [23].

Original: However, these methods may lack the scalability for SR problem [3].
Revised: However, these methods may lack the scalability for SR problem [3].

Original: The sequential optimizationbased method is more scalable as the model complexity increases in a sequential manner [24].
Revised: The sequential optimizationbased method is more scalable as the model complexity increases in a sequential manner [24].

Original: Finally,the gradient descent-based method [25] builds a large and over-parameterized network to search andtrain.
Revised: Finally, the gradient descent-based method [25] builds a large and over-parameterized network to search and train.

Original: Then, regularizations like dropout are added to find the best connections.
Revised: Then, regularizations like dropout are added to find the best connections.

Original: However, for thesemethods, the theoretical guarantees remain opaque for SR problem.
Revised: However, for these methods, the theoretical guarantees remain opaque for SR problem.

Original: Global optimum in neural networks.
Revised: Global optimum in neural networks.

Original: Many studies have been conducted to seek global optimalityin NNs, and they can be categorized into finding global optima for weights or input variables.
Revised: Many studies have been conducted to seek global optimality in NNs, and they can be categorized into finding global optima for weights or input variables.

Original: Theweight optimization is directly linked to finding symbol coefficients in L O C A L. Specifically, [26]investigates a single hidden layer with unbounded neurons and non-Euclidean regularization.
Revised: The weight optimization is directly linked to finding symbol coefficients in L O C A L. Specifically, [26] investigates a single hidden layer with unbounded neurons and non-Euclidean regularization.

Original: Theauthors show the training can be done via convex optimization problems.
Revised: The authors show the training can be done via convex optimization problems.

Original: [27] considers finite neuronsand develops a novel duality theory to train two-layer NNs with convexity.
Revised: [27] considers finite neurons and develops a novel duality theory to train two-layer NNs with convexity.

Original: [28] establishes a strongresult that every local optimum is a global optimum for deep non-linear networks under severalassumptions.
Revised: [28] establishes a strong result that every local optimum is a global optimum for deep non-linear networks under several assumptions.

Original: [29] eliminates these assumptions and finds that with weight decay regularization(e.g. , l 2 norm), the loss function of NN with ReLU activations is piece-wise strongly convex in localregions.
Revised: [29] eliminates these assumptions and finds that with weight decay regularization (e.g., l 2 norm), the loss function of NN with ReLU activations is piece-wise strongly convex in local regions.

Original: However, L O C A L doesn’t fit the above conditions.
Revised: However, L O C A L doesn’t fit the above conditions.

Original: The second group of input variableoptimization can help search for optimal inputs.
Revised: The second group of input variable optimization can help search for optimal inputs.

Original: [17] proposes ICNN such that the output of ICNN isconvex in input variables.
Revised: [17] proposes ICNN such that the output of ICNN is convex in input variables.

Original: The key of ICNN is to restrict some weights and activation functions topreserve convexity.
Revised: The key of ICNN is to restrict some weights and activation functions to preserve convexity.

Original: ICNN facilitates to design a convexified search algorithm.
Revised: ICNN facilitates to design a convexified search algorithm.

Original:  Convex Neural Symbolic Learning 
Revised:  Convex Neural Symbolic Learning 3.

Original: L O C A L to Hierarchically Represent the Equations and Learn Symbol Coefficients
Revised: L O C A L to Hierarchically Represent the Equations and Learn Symbol Coefficients

Original: SR problem can be decomposed into searching the symbol connections and estimating the symbolcoefficients.
Revised: SR problem can be decomposed into searching the symbol connections and estimating the symbol coefficients.

Original: The link of these two sub-problems is a proper representation of the underlying equationwith unknown structures and weights.
Revised: The link of these two sub-problems is a proper representation of the underlying equation with unknown structures and weights.

Original: Then, we need to search the structures and estimate the weights.
Revised: Then, we need to search the structures and estimate the weights.

Original: We utilize a neural network to represent the multi-input multi-output equations due to the efficiency[6].
Revised: We utilize a neural network to represent the multi-input multi-output equations due to the efficiency [6].

Original: We prove in Theorem 3 that under mild conditions and given correct structures of the NN, thereare local regions in the loss surface with strict convexity.
Revised: We prove in Theorem 3 that under mild conditions and given correct structures of the NN, there are local regions in the loss surface with strict convexity.

Original: Thus, we name our NN as Locally Convex
Revised: Thus, we name our NN as Locally Convex

Original: Equation Learner (L O C A L).
Revised: Equation Learner (L O C A L).

Original: L O C A L should have the capacity to represent the true equation.
Revised: L O C A L should have the capacity to represent the true equation.

Original: We assume the underlying equationy = g ( x ) follows compositionality and smoothness assumptions in [2], which are often the casein physics and many other scientific applications.
Revised: We assume the underlying equation y = g ( x ) follows compositionality and smoothness assumptions in [2], which are often the case in physics and many other scientific applications.

Original: Then, we build L O C A L that can be trained viainput/output pairs { x i , y i } Ni =1 , where x i ∈ X
Revised: Then, we build L O C A L that can be trained via input/output pairs { x i , y i } Ni =1 , where x i ∈ X

Original: and y i ∈ Y are the i th samples and N is the numberof data.
Revised: and y i ∈ Y are the i th samples and N is the number of data.

Original: X and Y are the input and output data spaces, respectively.
Revised: X and Y are the input and output data spaces, respectively.

Original: By the compositionality, wepropose to hierarchically map x to y with correct symbols.
Revised: By the compositionality, we propose to hierarchically map x to y with correct symbols.

Original: Fig.
Revised: Fig.

Original: 1 shows an example of the L O C A Lstructure.
Revised: 1 shows an example of the L O C A L structure.

Original: Specifically, each input entry first goes through the activation functions from a symbollibrary Φ .
Revised: Specifically, each input entry first goes through the activation functions from a symbol library Φ .

Original: For example, in Fig. 1, we denote Φ =
Revised: For example, in Fig. 1, we denote Φ =

Original: { x, x 2 , cos( wx ) } to correspond with three neuronsfrom top to bottom in the dotted black box, where w is a learnable weight.
Revised: { x, x 2 , cos( wx ) } to correspond with three neurons from top to bottom in the dotted black box, where w is a learnable weight.

Original: As for weights of x andx 2 , they only need to appear in the later summation layer.
Revised: As for weights of x and x 2 , they only need to appear in the later summation layer.

Original: Then, some activation outputs will beselected as multipliers for the multiplication.
Revised: Then, some activation outputs will be selected as multipliers for the multiplication.

Original: Subsequently, the multiplied outputs are selected andsummed together.
Revised: Subsequently, the multiplied outputs are selected and summed together.

Original: The repetition of symbolic activation, multiplication, and summation formulatesfinal equations.
Revised: The repetition of symbolic activation, multiplication, and summation formulates final equations.

Original: For instance, the L O C A L structure in Fig.
Revised: For instance, the L O C A L structure in Fig.

Original: 1 can correctly represents y 1 and y 2 shownon the top right of Fig.
Revised: 1 can correctly represents y 1 and y 2 shown on the top right of Fig.

Original: The layer-wise connectivity and the weights of L O C
Revised: The layer-wise connectivity and the weights of L O C

Original: A L are optimization variables for the SR problem.
Revised: A L are optimization variables for the SR problem.

Original: Mathematically, we denote Z k ∈ { 0 , 1 } n k × n k +1 and W k ∈ R n k × n k +1 as the indicator and weightmatrix between the k th and the ( k + 1) th layer of the L O C A L, respectively.
Revised: Mathematically, we denote Z k ∈ { 0 , 1 } n k × n k +1 and W k ∈ R n k × n k +1 as the indicator and weight matrix between the k th and the ( k + 1) th layer of the L O C A L, respectively.

Original: n k is the number ofneurons for the k th layers.
Revised: n k is the number of neurons for the k th layers.

Original: Z k
Revised: Z k

Original: [ i, j ] = 1 indicates that the connection exists between the i th neuronin the k th layer and the j th neuron in the ( k + 1) th layer, where Z k [ i, j ] is the ( i, j ) th entry of
Revised: [ i, j ] = 1 indicates that the connection exists between the i th neuron in the k th layer and the j th neuron in the ( k + 1) th layer, where Z k [ i, j ] is the ( i, j ) th entry of

Original: Z k .
Revised: Z k .

Original: We assume there are ( K + 1) layers in L O C A L and denote h k ∈ R n k to be the output ofthe k th layer.
Revised: We assume there are ( K + 1) layers in L O C A L and denote h k ∈ R n k to be the output of the k th layer.

Original: Naturally, we have h 0 = x
Revised: Naturally, we have h 0 = x

Original: and h K
Revised: and h K

Original: = ˆ y , where ˆ y is the output of L O C A L. Forthe symbolic activation or summation layer, we have h k +

Revised: = ˆ y , where ˆ y is the output of L O C A L. For the symbolic activation or summation layer, we have h k +

Original: =
Revised: =

Original: Z Tk ◦ W Tk h k , where ◦ representsthe Hadamard product and helps to zero out some connections.
Revised: Z Tk ◦ W Tk h k , where ◦ represents the Hadamard product and helps to zero out some connections.

Original: For the multiplication layer, wehave h k +

Revised: For the multiplication layer, we have h k +

Original: [ j ] = (cid:81)
Revised: [ j ] = (cid:81)

Original: Z k
Revised: Z k

Original: [ i,j ]=1 h k
Revised: [ i,j ]=1 h k

Original: [ i ] with no weight matrix involved.
Revised: [ i ] with no weight matrix involved.

Original: In general, we denote L O C
Revised: In general, we denote L O C

Original: A L asf ( x ; { Z k } K − 1 k =0 , { W k } K − 1 k =0 ) .
Revised: A L as f ( x ; { Z k } K − 1 k =0 , { W k } K − 1 k =0 ) .

Original: The search algorithm identifies { Z k } K − 1 k =

Revised: The search algorithm identifies { Z k } K − 1 k =

Original: and the estimation processlearns the corresponding weights in { W k } K − 1 k =0 using { x i , y i } Ni =1 .
Revised: and the estimation process learns the corresponding weights in { W k } K − 1 k =0 using { x i , y i } Ni =1 .

Original: 3.2 Search L O C A L Structures with Double Convex Deep Q-Learning
Revised: 3.2 Search L O C A L Structures with Double Convex Deep Q-Learning

Original: State and action definitions in the search process.
Revised: State and action definitions in the search process.

Original: To search the structure of L O C A L, we treat thestatus of each layer of L O C A L as a state and the connections between layers, i.e., Z k in L O C A L, as actions.
Revised: To search the structure of L O C A L, we treat the status of each layer of L O C A L as a state and the connections between layers, i.e., Z k in L O C A L, as actions.

Original: Specifically, we denote a k ∈ { 0 , 1 } n a as the k th action vector where a k [( i − 1) n k + j ] = 1implies that the connection ij exists for the i th neuron in the k th layer and the j th neuron inthe ( k + 1) th layer.
Revised: Specifically, we denote a k ∈ { 0 , 1 } n a as the k th action vector where a k [( i − 1) n k + j ] = implies that the connection ij exists for the i th neuron in the k th layer and the j th neuron in the ( k + 1) th layer.

Original: n a = max { n k n k +1 } K − 1 k =0 is the dimensionality of the action space.
Revised: n a = max { n k n k +1 } K − 1 k =0 is the dimensionality of the action space.

Original: Also,we denote s k ∈
Revised: Also, we denote s k ∈

Original: Z n s as the state vector to represent the current state for the k th layers, wheren s = max { n k } Kk =

Revised: Z n s as the state vector to represent the current state for the k th layers, where n s = max { n k } Kk =

Original: + 1 is the dimensionality of the state space.
Revised: + 1 is the dimensionality of the state space.

Original: More specifically, the state vectors k is composed of the values for each neuron in the k th layer and an entry of state index k .
Revised: More specifically, the state vector s k is composed of the values for each neuron in the k th layer and an entry of state index k .

Original: Theappending of the value k can avoid the duplicate state vectors for different layers.
Revised: The appending of the value k can avoid the duplicate state vectors for different layers.

Original: Otherwise, theduplicated states may exist and require different optimal actions, deteriorating the correct search forthe true structure of L O C A L. To further quantify state vectors, we utilize a linear transformation torepresent the state transition process.
Revised: Otherwise, the duplicated states may exist and require different optimal actions, deteriorating the correct search for the true structure of L O C A L. To further quantify state vectors, we utilize a linear transformation to represent the state transition process.

Original: ∀ 0 ≤ k ≤ K − 1 , we define
Revised: ∀ 0 ≤ k ≤ K − 1 , we define

Original: [Equation]
Revised: [Equation]

Original: where the number of 1 s in s 0 corresponds to the input dimension.
Revised: where the number of 1 s in s 0 corresponds to the input dimension.

Original: Mat ( · ) is the operation to reshapea vector to a matrix, and we utilize 0 s for padding to maintain the dimensions.
Revised: Mat ( · ) is the operation to reshape a vector to a matrix, and we utilize 0 s for padding to maintain the dimensions.

Original: c =
Revised: c =

Original: [0 , 0 , · · · , 1] Tis a constant vector to increase the entry of the state index from k to k + 1 .
Revised: [0 , 0 , · · · , 1] T is a constant vector to increase the entry of the state index from k to k + 1 .

Original: We show this linearstate transition is essential to guarantee the convexity of negative optimal Q-function in Theorem1.
Revised: We show this linear state transition is essential to guarantee the convexity of negative optimal Q-function in Theorem 1.

Original: For the calculation example of state transition, one can refer to Fig.
Revised: For the calculation example of state transition, one can refer to Fig.

Original: Then, we can obtain Z kfrom Mat ( a k ) by deleting the filled 0s.
Revised: Then, we can obtain Z k from Mat ( a k ) by deleting the filled 0s.

Original: Then, the search will always start at s 0 and end at s K in oneepisode.
Revised: Then, the search will always start at s 0 and end at s K in one episode.

Original: Thus, we define a trajectory as a sequence of searched state-action pairs for { ( s k , a k ) } K − 1 k =0 .
Revised: Thus, we define a trajectory as a sequence of searched state-action pairs for { ( s k , a k ) } K − 1 k =0 .

Original: This trajectory formulates { Z k } K − 1 k =0 in the L O C
Revised: This trajectory formulates { Z k } K − 1 k =0 in the L O C

Original: A L function f ( x ; { Z k } K − 1 k =0 , { W k } K − 1 k =0 ) .
Revised: A L function f ( x ; { Z k } K − 1 k =0 , { W k } K − 1 k =0 ) .

Original: The above states, actions, state transitions, initial state s 0 , and a discount factor γ can form a
Revised: The above states, actions, state transitions, initial state s 0 , and a discount factor γ can form a

Original: Controlled Markov Process (CMP) [30, 31], which is a Markov Decision Process (MDP) withouta reward function [31].
Revised: Controlled Markov Process (CMP) [30, 31], which is a Markov Decision Process (MDP) without a reward function [31].

Original: In the following part, we define our reward function based on an end-oftrajectory reward [30].
Revised: In the following part, we define our reward function based on an end-oftrajectory reward [30].

Original: Although the reward function is non-Markov, we prove in Theorem 1 thatunder our settings, there exists an optimal Q-function.
Revised: Although the reward function is non-Markov, we prove in Theorem 1 that under our settings, there exists an optimal Q-function.

Original: The proof can be seen in Appendix A.3.
Revised: The proof can be seen in Appendix A.3.

Original: Double convex deep Q-learning to search optimal actions.
Revised: Double convex deep Q-learning to search optimal actions.

Original: To optimize over the defined CMP foroptimal actions, we seek certain convexity with provable optimal results.
Revised: To optimize over the defined CMP for optimal actions, we seek certain convexity with provable optimal results.

Original: Thus, we propose a doubleconvex deep Q-learning with the convex negative reward function and negative value function (i.e.,
Revised: Thus, we propose a double convex deep Q-learning with the convex negative reward function and negative value function (i.e.,

Original: Q-function).
Revised: Q-function).

Original: Based on Bellman equations [32], this design will ensure the convexity of the negativeoptimal Q-function and global optimal solutions.
Revised: Based on Bellman equations [32], this design will ensure the convexity of the negative optimal Q-function and global optimal solutions.

Original: More proof details can be referred to Theorem 1and Appendix A.3.
Revised: More proof details can be referred to Theoremand Appendix A.3.

Original: Specifically, we utilize an Input Convex Neural Network (s) [17] to model the reward function− R 
Revised: Specifically, we utilize an Input Convex Neural Network (s) [17] to model the reward function − R ( s k , a k )

Original: ( s k , a k ) such that − R ( s k , a k ) is convex in states and actions.
Revised: such that − R ( s k , a k ) is convex in states and actions.

Original: − R ( s k , a k ) requires propertraining to do the correct evaluation.
Revised: − R ( s k , a k ) requires proper training to do the correct evaluation.

Original: In the t th episode, we collect the t th trajectory sample{ ( s tk , a tk ) } K − 1 k =0 .
Revised: In the t th episode, we collect the t th trajectory sample { ( s tk , a tk ) } K − 1 k =0 .

Original: Then, the output can be defined as the end-of-trajectory reward to evaluate theobtained L O C A L which is denoted as f t ( x ; W t ) , where W t is the weight set of L O C A L. Basically,we utilize gradient method (e.g., Adam [33]) to train f t ( x ; W t ) and obtain the optimal set of weights
Revised: Then, the output can be defined as the end-of-trajectory reward to evaluate the obtained L O C A L which is denoted as f t ( x ; W t ) , where W t is the weight set of L O C A L. Basically, we utilize gradient method (e.g., Adam [33]) to train f t ( x ; W t ) and obtain the optimal set of weights

Original: W ∗ t for the t th episode of L O C A L by minimizing the Mean Square Error (MSE).
Revised: W ∗ t for the t th episode of L O C A L by minimizing the Mean Square Error (MSE).

Original: Then, we cancalculate the Normalized Root-Mean-Square Error (NRMSE) [3] of the trained f t ( x ; W ∗ ) such that
Revised: Then, we can calculate the Normalized Root-Mean-Square Error (NRMSE) [3] of the trained f t ( x ; W ∗ ) such that

Original: NRMSE t = 1 σ y (cid:113)
Revised: NRMSE t = 1 σ y (cid:113)

Original: N (cid:80) Ni =

Revised: N (cid:80) Ni =

Original: ( y i − f t (cid:0) x i ; W ∗ t ) (cid:1) 2 , where σ y is the standard deviation of the outputs.
Revised: ( y i − f t (cid:0) x i ; W ∗ t ) (cid:1) 2 , where σ y is the standard deviation of the outputs.

Original: The output of the reward function can be calculated as R t = 1 1+ NRMSE t .
Revised: The output of the reward function can be calculated as R t = 1 1+ NRMSE t .

Original: Therefore, we can train− R ( · ) using {{ s tk , a tk } K − 1 k =0 , − R t } .
Revised: Therefore, we can train − R ( · ) using {{ s tk , a tk } K − 1 k =0 , − R t } .

Original: Although training R ( · ) utilizes the samples of discrete states and actions, we aim to solve thecontinuous convex optimization for optimal sequential decisions.
Revised: Although training R ( · ) utilizes the samples of discrete states and actions, we aim to solve the continuous convex optimization for optimal sequential decisions.

Original: Thus, we first show that R ( · ) canbe defined over the continuous action space.
Revised: Thus, we first show that R ( · ) can be defined over the continuous action space.

Original: By definitions of the discrete actions, we restrict thecontinuous action space in a hypercube conv ( { 0 , 1 } n a ) , i.e., a convex hull of the discrete actions.
Revised: By definitions of the discrete actions, we restrict the continuous action space in a hypercube conv ( { 0 , 1 } n a ) , i.e., a convex hull of the discrete actions.

Original: Then, the discrete actions are the endpoints of the hypercube.
Revised: Then, the discrete actions are the endpoints of the hypercube.

Original: Notably, it is doable to utilize acontinuous convex function to fit these endpoints with end-of-trajectory rewards.
Revised: Notably, it is doable to utilize a continuous convex function to fit these endpoints with end-of-trajectory rewards.

Original: Especially, thestrictly minimal reward value − R t only lies in the endpoint that guarantees the correct structureof L O C A L. Therefore, one can utilize piece-wise linear functions with convexity to represent R ( · )over conv ( { 0 , 1 } n a ) and ensure one endpoint has the minimal value.
Revised: Especially, the strictly minimal reward value − R t only lies in the endpoint that guarantees the correct structure of L O C A L. Therefore, one can utilize piece-wise linear functions with convexity to represent R ( · ) over conv ( { 0 , 1 } n a ) and ensure one endpoint has the minimal value.

Original: The piece-wise linearity can beachieved via using ReLu activations to R ( · ) .
Revised: The piece-wise linearity can be achieved via using ReLu activations to R ( · ) .

Original: With the defined continuous action space, we utilize another ICNN to represent − Q ( s k , ˜ a k ) suchthat − Q ( s k , ˜ a k ) is convex in the continuous action ˜ a k ∈ conv ( { 0 , 1 } n a ) given fixed s k .
Revised: With the defined continuous action space, we utilize another ICNN to represent − Q ( s k , ˜ a k ) such that − Q ( s k , ˜ a k ) is convex in the continuous action ˜ a k ∈ conv ( { 0 , 1 } n a ) given fixed s k .

Original: To update
Revised: To update

Original: Q values, we have the following iterative computations based on the temporal difference [21]:
Revised: Q values, we have the following iterative computations based on the temporal difference [21]:

Original: [Equation]
Revised: [Equation]

Original: where Q t ( s k , ˜ a k ) is the Q value at the t th episode for state s k and action ˜ a k .
Revised: where Q t ( s k , ˜ a k ) is the Q value at the t th episode for state s k and action ˜ a k .

Original: α and γ are pre-definedlearning rate and discount factor, respectively.
Revised: α and γ are pre-defined learning rate and discount factor, respectively.

Original: Therefore, one can solve a convex optimizationproblem max a Q t ( s k +1 , ˜ a ) to obtain the (approximately) global optimal action for Equation (2) ineach iteration.
Revised: Therefore, one can solve a convex optimization problem max a Q t ( s k +1 , ˜ a ) to obtain the (approximately) global optimal action for Equation (2) in each iteration.

Original: Thus, the optimization problem is:
Revised: Thus, the optimization problem is:

Original: [Equation]
Revised: [Equation]

Original: Based on [17], this convex optimization problem can be solved via a bundle entropy method.
Revised: Based on [17], this convex optimization problem can be solved via a bundle entropy method.

Original: Afterobtaining a continuous solution ˜ a ∗ , we discretize it to a discrete vector a ∗ to build L O C A L. Onesimple method is to enforce a ∗ {+[ i ] = 
+}
Revised: After obtaining a continuous solution ˜ a ∗ , we discretize it to a discrete vector a ∗ to build L O C A L. One simple method is to enforce a ∗ {+[ i ] = 
+}

Original: [-[ i ] = 
-] if ˜ a ∗ [ i ] ≥ 0 . 5 , and otherwise a ∗ [ i ] = 0 .
Revised: [-[ i ] = 
-] if ˜ a ∗ [ i ] ≥ 0 . 5 , and otherwise a ∗ [ i ] = 0 .

Original: Thus, both
Revised: Thus, both

Original: Q ( s k , ˜ a k ) and Q ( s k , a k ) can be trained using Equation (2).
Revised: Q ( s k , ˜ a k ) and Q ( s k , a k ) can be trained using Equation (2).

Original: Practically, we introduce ϵ -greedystrategy [34], experience replay [35] and a target Q-network [36] to update Q-function, thus boostingthe convergence to the optimal Q-function.
Revised: Practically, we introduce ϵ -greedy strategy [34], experience replay [35] and a target Q-network [36] to update Q-function, thus boosting the convergence to the optimal Q-function.

Original: The overview of our framework is in  Algorithm 1 andthe specific algorithm is in  Algorithm 2.
Revised: The overview of our framework is in Appendix A.1, Algorithm 1. The specific algorithm is in Appendix A.2, Algorithm 2.

Original: Finally, based on Theorems 1-2, the discrete version of theoptimal actions can generate the correct structures of L O C A L and exact equations.
Revised: Finally, by Theorems 1-2, the discrete optimal actions can generate the correct structures of L O C A L and exact equations.

Original: Symbolic static and dynamic constraints.
Revised: Symbolic static and dynamic constraints.

Original: Constraints can be added to accelerate the search process[3, 2].
Revised: Constraints can be added to accelerate the search process [3, 2].

Original: For example, in Algorithm 2, we propose a constraint checking program for the state-actionpairs to avoid the invalid search, suitable for arbitrary restrictions.
Revised: For example, in Algorithm 2, we propose a constraint checking program for the state-action pairs to avoid the invalid search, suitable for arbitrary restrictions.

Original: Then, we emphasize a generaltype of constraint, symbolic constraint, for the SR problem.
Revised: Then, we emphasize a general type of constraint, symbolic constraint, for the SR problem.

Original: The symbolic static constraint requiresthat each equation contains only a subset of symbols.
Revised: The symbolic static constraint requires that each equation contains only a subset of symbols.

Original: For example, the equation y 1 = x 1 x 2 · · · x 100shouldn’t exist since it is too complicated for real-world systems.
Revised: For example, the equation y 1 = x 1 x 2 · · · xshouldn’t exist since it is too complicated for real-world systems.

Original: This constraint eliminates part ofthe action space and can be checked by counting the number of 1 s in the action vector.
Revised: This constraint eliminates part of the action space and can be checked by counting the number of 1 s in the action vector.

Original: The symbolicdynamic constraint can gradually reduce the search space based on symbol correlations.
Revised: The symbolic dynamic constraint can gradually reduce the search space based on symbol correlations.

Original: Specifically,we investigate the ( K − 1) th layer’s neurons that are linearly summed to form the equation.
Revised: Specifically, we investigate the ( K − 1) th layer’s neurons that are linearly summed to form the equation.

Original: Ifsome of these neurons have strong linear correlations to the output neuron (e.g., Pearson correlationcoefficient larger than 0 . 99 ), they should be kept subsequently.
Revised: If some of these neurons have strong linear correlations to the output neuron (e.g., Pearson correlation coefficient larger than 0 . 99 ), they should be kept subsequently.

Original: Namely, we can maintain the pathfrom input neurons to the neurons to be kept and reduce the search space.
Revised: Namely, we can maintain the path from input neurons to the neurons to be kept and reduce the search space.

Original:  Theoretical Analysis
Revised:  Theoretical Analysis

Original: We employ explorations, experience replay, and a target Q-network to boost the convergence to theoptimal Q function.
Revised: We employ explorations, experience replay, and a target Q-network to boost the convergence to the optimal Q function.

Original: However, our extra requirement of the convex shape for the negative Q-functionmay deteriorate the convergence performance.
Revised: However, our extra requirement of the convex shape for the negative Q-function may deteriorate the convergence performance.

Original: Thus, we first prove that in C O NS O L E , the negativeoptimal Q-function is also convex so that the convex design doesn’t affect the convergence.
Revised: Thus, we first prove that in C O NS O L E , the negative optimal Q-function is also convex so that the convex design doesn’t affect the convergence.

Original: Then, weprove that the convexity of the negative optimal Q-function eventually yields the exact equations.
Revised: Then, we prove that the convexity of the negative optimal Q-function eventually yields the exact equations.

Original: Theorem 1.
Revised: Theorem 1.

Original: ∀ 0 ≤ k ≤ K − 1 , the negative optimal Q-function − Q ∗ ( s k , ˜ a k ) in the proposed
Revised: ∀ 0 ≤ k ≤ K − 1 , the negative optimal Q-function − Q ∗ ( s k , ˜ a k ) in the proposed

Original: C O NS O L E framework exists and is convex in s k and ˜ a k , where s k is the discrete state and ˜ a k is thecontinuous action at the k th stage.
Revised: C O NS O L E framework exists and is convex in s k and ˜ a k , where s k is the discrete state and ˜ a k is the continuous action at the k th stage.

Original: The proof can be seen in Appendix A.3.
Revised: The proof can be seen in Appendix A.3.

Original: Based on the convexity of negative optimal Q-function,the optimal sequence of states ( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) can be found viasolving the convex optimization problem.
Revised: Based on the convexity of negative optimal Q-function, the optimal sequence of states ( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) can be found via solving the convex optimization problem.

Original: Then, we have the following theorem.
Revised: Then, we have the following theorem.

Original: Theorem 2.
Revised: Theorem 2.

Original: Let f ∗ ( · ; W ) denote the L O C A L constructed by the optimal sequences of states( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) from − Q ∗ ( · ) , where W is the set of weightsof f ∗ ( · ; W ) .
Revised: Let f ∗ ( · ; W ) denote the L O C A L constructed by the optimal sequences of states ( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) from − Q ∗ ( · ) , where W is the set of weights of f ∗ ( · ; W ) .

Original: If f ∗ ( · ; W ) can be trained with noiseless datasets and the training can achieve theglobal optimal weights W ∗ , f ∗ ( · ; W ∗ ) can be simplified to the true equation g ( · ) .
Revised: If f ∗ ( · ; W ) can be trained with noiseless datasets and the training can achieve the global optimal weights W ∗ , f ∗ ( · ; W ∗ ) can be simplified to the true equation g ( · ) .

Original: For the optimal search structure of L O C A L, i.e., f ∗ ( · ; W ) , the optimal weight set W ∗ is also trainedvia gradient method like Adam [33].
Revised: For the optimal search structure of L O C A L, i.e., f ∗ ( · ; W ) , the optimal weight set W ∗ is also trained via gradient method like Adam [33].

Original: The proof can be seen in Appendix A.4.
Revised: The proof can be seen in Appendix A.4.

Original: Theorem 2 requiresthat L O C A L can learn the global optimal weights.
Revised: Theorem 2 requires that L O C A L can learn the global optimal weights.

Original: The requirement is generally hard to achieve dueto the non-linearity and non-convexity of L O C A
Revised: The requirement is generally hard to achieve due to the non-linearity and non-convexity of L O C A

Original: L.
Revised: L.

Original: However, we show that with mild assumptions,there are local regions in
Revised: However, we show that with mild assumptions, there are local regions in

Original: the L O C
Revised: the L O C

Original: A L loss surface with strict convexity.
Revised: A L loss surface with strict convexity.

Original: Then, if we have properinitializations, the gradient-based weight updating can find the global optimum.
Revised: Then, if we have proper initializations, the gradient-based weight updating can find the global optimum.

Original: Specifically, we havethe following theorem.
Revised: Specifically, we have the following theorem.

Original: Theorem 3.
Revised: Theorem 3.

Original: Assume the following conditions hold: (1) the equation g ( x ) is C 2 smooth and hasbounded second derivatives with respect to weights, (2) ∃ x ∈ X , g ( x ) has non-zero gradients withrespect to weights, (3) the structure of L O C
Revised: Assume the following conditions hold: (1) the equation g ( x ) is C 2 smooth and has bounded second derivatives with respect to weights, (2) ∃ x ∈ X , g ( x ) has non-zero gradients with respect to weights, (3) the structure of L O C

Original: A L is correctly searched to exactly represent symbolsand symbol connections in g ( x ) , and (4) the training dataset of L O C
Revised: A L is correctly searched to exactly represent symbols and symbol connections in g ( x ) , and (4) the training dataset of L O C

Original: A L is noiseless.
Revised: A L is noiseless.

Original: Then, for the
Revised: Then, for the

Original: MSE loss surface of L O C A L , each global optimal point has a strictly convex local region.
Revised: MSE loss surface of L O C A L , each global optimal point has a strictly convex local region.

Original: The proofs can be seen in Appendix A.5.
Revised: The proofs can be seen in Appendix A.5.

Original: Note that Assumptions 1-2 easily hold for common physicalequations in nature [2].
Revised: Note that Assumptions 1-2 easily hold for common physical equations in nature [2].

Original: Assumption 3 relies on the search algorithm, and we show, both theoreticallyand numerically, that our double convex deep Q-learning has good performances.
Revised: Assumption 3 relies on the search algorithm, and we show, both theoretically and numerically, that our double convex deep Q-learning has good performances.

Original: Assumption 4relies on the quality of data and we focus on the noiseless data in this paper.
Revised: Assumptionrelies on the quality of data and we focus on the noiseless data in this paper.

Original: What’s more, Equation(6) in the proof suggests that if the absolute noise values are small, the locally convex region stillexists.
Revised: What’s more, Equation (6) in the proof suggests that if the absolute noise values are small, the locally convex region still exists.

Original: We also numerically prove C O NS O L E is robust under certain noise levels in Section 5.5.
Revised: We also numerically prove C O NS O L E is robust under certain noise levels in Section 5.5.

Original: Tosummarize, these assumptions are acceptable.
Revised: To summarize, these assumptions are acceptable.

Original: To quantify the range of the local region for a L O C A Lwith a certain complexity, we have the following theorem.
Revised: To quantify the range of the local region for a L O C A L with a certain complexity, we have the following theorem.

Original: Theorem 4.
Revised: Theorem 4.

Original: Suppose Assumptions 1-4 in Theorem 3 hold.
Revised: Suppose Assumptions 1-4 in Theorem 3 hold.

Original: For a L O C A L with one symbolicactivation, multiplication, and summation layer, the set of local convex regions with global optima is 
Revised: For a L O C A L with one symbolic activation, multiplication, and summation layer, the set of local convex regions with global optima is [Equation] |

Original: [Equation]
Revised: [Equation]

Revised: y

Original: [Equation]
Revised: [Equation]

Original: W)
Revised: W

Original: [Equation]
Revised: [Equation]

Original: y k |} , where notations are defined in the proof.
Revised: y [Equation] |} , where notations are defined in the proof.

Original: The proofs can be seen in Appendix A.6.
Revised: The proofs can be seen in Appendix A.6.

Original: We explain the region range is large for a stable system thatsatisfies all assumptions in Theorem 4.
Revised: We explain the region range is large for a stable system that satisfies all assumptions in Theorem 4.

Original: Physical interpretations of the convex region size.
Revised: Physical interpretations of the convex region size.

Original: Physical systems in scientific and engineeringdomains have certain stability that can withstand parameter changes to some extent.
Revised: Physical systems in scientific and engineering domains have certain stability that can withstand parameter changes to some extent.

Original: Further, thisability should hold for arbitrary x ∈ X and y ∈ Y .
Revised: Further, this ability should hold for arbitrary x ∈ X and y ∈ Y .

Original: Thus, we can assume ddt (cid:12)(cid:12) t =

Revised: Thus, we can assume ddt (cid:12)(cid:12) t =

Original: ˆ y
Revised: ˆ y

Original: ( x i , W + tX ) ≈ddt (cid:12)(cid:12) t
Revised: ( x i , W + tX ) ≈ ddt (cid:12)(cid:12) t

Original: =

Revised: =

Original: ˆ y ( x j , W + tX )
Revised: ˆ y ( x j , W + tX )

Original: ≈ ddt (cid:12)(cid:12) t
Revised: ≈ ddt (cid:12)(cid:12) t

Original: =

Revised: =

Original: ˆ y
Revised: ˆ y

Original: ( x k , W + tX )
Revised: ( x k , W + tX )

Original: for x i , x j , x k ∈ X .
Revised: for x i , x j , x k ∈ X .

Original: Then, the inequality in set Ucan be approximately rewritten as 1 η > || w − w ∗
Revised: Then, the inequality in set U can be approximately rewritten as 1 η > || w − w ∗

Original: || 2 , where w and w ∗ are the vectorized W and W ∗ ,respectively.
Revised: || 2 , where w and w ∗ are the vectorized W and W ∗ , respectively.

Original: Namely, the distance between any point W in the region to the global optimal point
Revised: Namely, the distance between any point W in the region to the global optimal point

Original: W ∗ in the region is bounded by 1 η .
Revised: W ∗ in the region is bounded by 1 η .

Original: Based on Equation (19), η is the bound of the ratio of secondderivative to the first derivative.
Revised: Based on Equation (19), η is the bound of the ratio of second derivative to the first derivative.

Original: For a stable system, this ratio should be small.
Revised: For a stable system, this ratio should be small.

Original: Otherwise, the systemcan easily crash with a small parameter disturbance.
Revised: Otherwise, the system can easily crash with a small parameter disturbance.

Original: Thus, 1 η is relatively large and so is the regionof U .
Revised: Thus, 1 η is relatively large and so is the region of U .

Original: An example of the range is displayed in Section 5.2.
Revised: An example of the range is displayed in Section 5.2.

Original: Finally, the above analysis also holdswhen the L O C A L of the system equation has more than one symbolic activation, multiplication, andsummation layers.
Revised: Finally, the above analysis also holds when the L O C A L of the system equation has more than one symbolic activation, multiplication, and summation layers.

Original: This is because Equation (20) always holds as long as we can find a η to boundthe ratio of the second derivative to the first derivative, which is irrelevant to the structure of L O C A L.
Revised: This is because Equation (20) always holds as long as we can find a η to bound the ratio of the second derivative to the first derivative, which is irrelevant to the structure of L O C A L.

Original:  Experiments 
Revised:  Experiments 5.1 Settings

Original: 5.1 Settings

Original: Datasets.
Revised: Datasets.

Original: We use the following datasets for testing.
Revised: We use the following datasets for testing.

Original: (1) Synthetic datasets .
Revised: (1) Synthetic datasets .

Original: We create two datasets,
Revised: We create two datasets,

Original: Syn 1 and Syn 2 , for testing.
Revised: Syn 1 and Syn 2 , for testing.

Original: Syn 1 has the following equations: y 1 = 3 x 21 cos(2 .
Revised: Syn 1 has the following equations: y 1 = 3 x 2 1 cos(2 .

Original: 5 x 2 ) , y 

Revised: 5 x 2 ) , y

Original: = 4 x 1 x 3 ,and y 

Revised: = 4 x 1 x 3 , and y

Original: = 3 x 2 3 .
Revised: = 3 x 23 .

Original: Syn 2 is more complex with the following equations y 1 = √ 2 .
Revised: Syn 2 is more complex with the following equations y 1 = √ 2 .

Original: 2 x 1 x 2 + x 1 x 2 2 ,y 2 = sin(1 .
Revised: 2 x 1 x 2 + x 1 x 22 , y 2 = sin(1 .

Original: 8 x 1 ) (cid:0) log(3 x 2 ) + √ x 3 (cid:1) , y 3 = √ 3 .
Revised: 8 x 1 ) (cid:0) log(3 x 2 ) + √ x 3 (cid:1) , y 3 = √ 3 .

Original: 7 x 3 log(1 .
Revised: 7 x 3 log(1 .

Original: 6 x 1 ) + x 2 1 .
Revised: 6 x 1 ) + x 21 .

Original: For the training data, each inputvariable is randomly sampled from a uniform distribution of U (1 , 2) to avoid invalid values likelog(0) .
Revised: For the training data, each input variable is randomly sampled from a uniform distribution of U (1 , 2) to avoid invalid values like log(0) .

Original: Totally, we create 2 , 000 samples for training.
Revised: Totally, we create 2 , 000 samples for training.

Original: Then, in the test phase, we utilize another2 , 000 samples whose input variables are sampled from U (3 , 4) .
Revised: Then, in the test phase, we utilize another 2 , 000 samples whose input variables are sampled from U (3 , 4) .

Original: The symbolic activation pools are{ x, x 2 , cos( x ) } and {√ x, x, x 2 , log( x ) , sin( x ) } for Syn 1 and Syn 2 , respectively.(2) Power system dataset.
Revised: The symbolic activation pools are { x, x 2 , cos( x ) } and {√ x, x, x 2 , log( x ) , sin( x ) } for Syn 1 and Syn 2 , respectively. (2) Power system dataset.

Original: Power flow equation determines the operations of electric systems [37].
Revised: Power flow equation determines the operations of electric systems [37].

Original: For node i in an M -node system, the equation can be written as p i
Revised: For node i in an Mnode system, the equation can be written as p i

Original: = (cid:80)
Revised: = (cid:80)

Original: | M | m =1 G im ( u i u m + v i v m )
Revised: | M | m =1 G im ( u i u m + v i v m )

Original: + B im ( v i u m − ui v m ) and q i = (cid:80)
Revised: + B im ( v i u m − u

Original:  Mm [-=
-] G im ( v i u m − u i v m ) − B im ( v i u m − u i v m ) , where u i and v iare the real and imaginary components of the voltage phasor at node i .
Revised: i v m ) and q i = (cid:80) Mm [-=
-]=1 G im ( v i u m − u i v m ) − B im ( v i u m − u i v m ) , where u i and v i are the real and imaginary components of the voltage phasor at node i .

Original: p i and q
Revised: p i and q

Original: i are the activeand reactive power at node i .
Revised: i are the active and reactive power at node i .

Original: G
Revised: G

Original: im and B im represent the physical parameters of line im .
Revised: im and B im represent the physical parameters of line im .

Original: If lineim does not exist, G im =
Revised: If line im does not exist, G im =

Original: B im = 0 .
Revised: B im = 0 .

Original: Therefore, we can treat x =
Revised: Therefore, we can treat x =

Original: [ u 1 , v 1 , · · · , u M , v M ] T andy =
Revised: [ u 1 , v 1 , · · · , u M , v M ] T and y =

Original: [ p 1 , q 1 , · · · , p M , q M ] T .
Revised: [ p 1 , q 1 , · · · , p M , q M ] T .

Original: The target is to learn the underlying system topology and parameters,which has broad impacts on the power domains [38].
Revised: The target is to learn the underlying system topology and parameters, which has broad impacts on the power domains [38].

Original: In this experiment, we implement simulationfrom a 5 -node system using MATPOWER [39] and two year’s hourly data.
Revised: In this experiment, we implement simulation from a 5 -node system using MATPOWER [39] and two year’s hourly data.

Original: The first 8 , 
Revised: The first 8 , 760 points are used for training while the remaining samples are used for testing. The symbolic activation pool is { x } . We denote this dataset as Pow.

Original: (3) Mass-damper system dataset.
Revised: (3) Mass-damper system dataset.

Original: Equations of the mass-damper system can be written as: ˙ q =− DRD ⊤ M − 1 q , where ˙ q is a vector of momenta, D is the incidence matrix of the system, R is thediagonal matrix of the damping coefficients for each line of the system, and M is the diagonal matrixof each node mass of the system.
Revised: Equations of the mass-damper system can be written as: ˙ q = − DRD ⊤ M − 1 q , where ˙ q is a vector of momenta, D is the incidence matrix of the system, R is the diagonal matrix of the damping coefficients for each line of the system, and M is the diagonal matrix of each node mass of the system.

Original: Thus, we can set y = ˙
Revised: Thus, we can set y = ˙

Original: q and x = q
Revised: q and x = q

Original: and the goal is to learn theparameter matrix − DRD ⊤ M − 1 .
Revised: and the goal is to learn the parameter matrix − DRD ⊤ M − 1 .

Original: We conduct the simulation via MATLAB for a 10 -node systemand obtain 6 , 000 points for 1 min simulation with a step size to be 0 .
Revised: We conduct the simulation via MATLAB for a 10 -node system and obtain 6 , 000 points for 1 min simulation with a step size to be 0 .

Original: 01 s.
Revised: 01 s.

Original: The first 3 , 000 samples areused for training while the rest samples are used for testing.
Revised: The first 3 ,samples are used for training while the rest samples are used for testing.

Original: The symbolic activation pool is { x } .
Revised: The symbolic activation pool is { x } .

Original: Then, we denote the dataset as Mas.
Revised: Then, we denote the dataset as Mas.

Original: Benchmark methods .
Revised: Benchmark methods .

Original: The following benchmark methods are utilized.
Revised: The following benchmark methods are utilized.

Original: (1) Deep Symbolic Regression( DSR ) [3].
Revised: (1) Deep Symbolic Regression ( DSR ) [3].

Original: DSR develops an RNN-based framework to search the expression tree.
Revised: DSR develops an RNN-based framework to search the expression tree.

Original: Especially, therisk-seeking policy gradient is utilized to seek the best performance.
Revised: Especially, the risk-seeking policy gradient is utilized to seek the best performance.

Original: Then, BFGS [11] can solve thenon-linear optimization and estimate the symbol coefficients.
Revised: Then, BFGS [11] can solve the non-linear optimization and estimate the symbol coefficients.

Original: (2) Vanilla Policy Gradient ( VPG ) [3].
Revised: (2) Vanilla Policy Gradient ( VPG ) [3].

Original: VPG is a vanilla version of DSR with a normal policy gradient rather than the risk-seeking method.(3) Equation Learner ( EQL ) [6, 7].
Revised: VPG is a vanilla version of DSR with a normal policy gradient rather than the risk-seeking method. (3) Equation Learner ( EQL ) [6, 7].

Original: EQL creates an end-to-end NN to select symbols and estimatethe coefficients.
Revised: EQL creates an end-to-end NN to select symbols and estimate the coefficients.

Original: The sparse regularization is enforced for the NN weights to search symbols.
Revised: The sparse regularization is enforced for the NN weights to search symbols.

Original: Multilayer Perceptron ( MLP ).
Revised: Multilayer Perceptron ( MLP ).

Original: We also employ a standard MLP to learn the regression from x to y .
Revised: We also employ a standard MLP to learn the regression from x to y .

Original: We only evaluate the extrapolation capacity of MLP in the test dataset.
Revised: We only evaluate the extrapolation capacity of MLP in the test dataset.

Original: For DSR, VPG, and EQL methods, based on the input datasets, we adjust the symbol and operatorlibrary to enable the same searching space as C O NS O L E for fair comparisons.
Revised: For DSR, VPG, and EQL methods, based on the input datasets, we adjust the symbol and operator library to enable the same searching space as C O NS O L E for fair comparisons.

Original: We run the benchmarkmethods 5 times with different random seeds and present their best results.
Revised: We run the benchmark methods 5 times with different random seeds and present their best results.

Original: As for our method, weonly run 1 time and obtain good results due to the convex design and the ϵ -greedy strategy.
Revised: As for our method, we only run 1 time and obtain good results due to the convex design and the ϵ -greedy strategy.

Original: Metrics for evaluation.
Revised: Metrics for evaluation.

Original: We employ the following metrics.
Revised: We employ the following metrics.

Original: (1) Average coefficient estimation percentage error E c .
Revised: (1) Average coefficient estimation percentage error E c .

Original: For an equation with H symbols, We calculate the error as E c = 1 H (cid:80)
Revised: For an equation with H symbols, We calculate the error as E c = 1 H (cid:80)

Original: h
Revised: h

Original: PE ( w h , ˆ w h ) ,where w h and ˆ w h represents the true and the estimated coefficients for the h th symbol, respectively.
Revised: PE ( w h , ˆ w h ) , where w h and ˆ w h represents the true and the estimated coefficients for the h th symbol, respectively.

Original: PE is the operation to calculate the percentage error.
Revised: PE is the operation to calculate the percentage error.

Original: If there is no matched symbol for the h th truesymbol, we denote PE ( w h , ˆ w h ) = 100% .
Revised: If there is no matched symbol for the h th true symbol, we denote PE ( w h , ˆ w h ) = 100% .

Original: Note that when calculating E c , proper simplifications maybe needed.
Revised: Note that when calculating E c , proper simplifications may be needed.

Original: For example, cos (cid:0) 2 . 5( √ x ) 2 (cid:1) = cos(2 .
Revised: For example, cos (cid:0) 2 . 5( √ x ) 2 (cid:1) = cos(2 .

Original: 5 x ) .
Revised: 5 x ) .

Original: (2) NRMSE in the test dataset.
Revised: (2) NRMSE in the test dataset.

Original: We measurethe extrapolation capacity in the test dataset and utilize NRMSE employed in Section 3.2.Implementing details of C O NS O L E . Hyper-parameters of C O NS
Revised: We measure the extrapolation capacity in the test dataset and utilize NRMSE employed in Section 3.2.

Original: O L E exist for both the doubleconvex deep Q-learning and the L O C A L.

Original: In the deep Q-learning, we set γ = 0 .

Original: 2 , ϵ = 0 .

Original: 4 , T = 600 ,λ = 10 − 2 , T 0 = 10 for Algorithm 2.

Original: Furthermore, to train the negative Q-function and the rewardfunction, we set the learning rate to be 5 × 10 − 3 and the number of epochs for training to be 50 .

Original: Then, we set the batch size for the negative Q-function to be 100 .

Original: If the number of data in the replaybuffer is less than 100 , no training happens for the negative Q-function.

Original: Additionally, all the datagathered in one episode are used to train the negative reward function.

Original: As for the L O C A L, we set

Original: K = 3 , the learning rate to be 1 × 10 − 2 and the number of training epochs to be 8 .

Original: We make thesetraining epochs to be small since training the L O C

Original: A L is the most time-consuming part of C O NS O L E .

Original: Furthermore, if the structure of L O C A L is correctly searched, a small number of iterations can help L O C A L to gain the global optimal weights. Finally, we initialize all trainable weights in L O C A L tobe 1 .
Revised: Finally, the hyper-parameter settings can be seen in Appendix A.7.

Original: The following results show that a relatively large area is suitable for an initial guess of L O C A L.

Original: 5.2 Verification and 3-D Visualization of Convex Mechanisms
Revised: 5.2 Verification and 3-D Visualization of Convex Mechanisms

Original: We first utilize a toy example to verify the benefits of convex designs for two sub problems.
Revised: We first utilize a toy example to verify the benefits of convex designs for two sub problems.

Original: Specifically, we consider to learn y 1 = 3 x 21 cos(2 .
Revised: Specifically, we consider to learn y 1 = 3 x 2 1 cos(2 .

Original: 5 x 2 ) with the loss function L = (cid:80) Ni ( y i [1] −w 1 x i [1] 2 cos( w 2 x i [2]) 2 , where the data sample notations are defined in Section 3.1.
Revised: 5 x 2 ) with the loss function L = (cid:80) N i ( y i [1] − w 1 x i [1] 2 cos( w 2 x i [2]) 2 , where the data sample notations are defined in Section 3.1.

Original: As shown in
Revised: As shown in

Original: Fig.
Revised: Fig.

Original: 2a, we design a 4 -layer L O C A L with two learnable parameters w 1 and w 2 to represent thecoefficients.
Revised: 2a, we design a 4 -layer L O C A L with two learnable parameters w 1 and w 2 to represent the coefficients.

Original: Thus, in the search phase, the goal is to identify the action a 1 .
Revised: Thus, in the search phase, the goal is to identify the action a 1 .

Original: We plot − Q t ( · ) whent = 1 , 5 , 10 in Fig. 2b.
Revised: We plot − Q t ( · ) when t = 1 , 5 , 10 in Fig. 2b.

Original: The convexity always exists so that the algorithm can quickly find the globaloptimal solutions in red dots.
Revised: The convexity always exists so that the algorithm can quickly find the global optimal solutions in red dots.

Original: Next, as the agent update Q values with true rewards, the Q-functionconverges to the optimal function within 10 episodes.
Revised: Next, as the agent update Q values with true rewards, the Q-function converges to the optimal function within 10 episodes.

Original: Finally, the convex − Q ∗ ( · ) remains unchangedand can bring the optimal action and the true equation, which supports Theorem 2.
Revised: Finally, the convex − Q ∗ ( · ) remains unchanged and can bring the optimal action and the true equation, which supports Theorem 2.

Original: Subsequently, we plot the loss surface of L in Fig.
Revised: Subsequently, we plot the loss surface of L in Fig.

Original: 2c.
Revised: 2c.

Original: We find that around the two global optima(3 , 2 . 5) and (3 , − 2 . 5) , there are convex regions that have an approximate quadratic shape.
Revised: We find that around the two global optima (3 , 2 . 5) and (3 , − 2 . 5) , there are convex regions that have an approximate quadratic shape.

Original: To furtherquantify the range for proper initialization, we vary the initial weight w 0 ∈ {− 10 , − 9 , · · · , 10 } forw 1 and w 2 in L O C A L. Fig.
Revised: To further quantify the range for proper initialization, we vary the initial weight w 0 ∈ {− 10 , − 9 , · · · , 10 } for w 1 and w 2 in L O C A L. Fig.

Original: 2 reports the final training loss with respect to different w 0 , and we find the safe range for initialization is [ − 3 , 7] \ { 0 } .
Revised: 2 reports the final training loss with respect to different w [Table] and we find the safe range for initialization is [ − 3 , 7] \ { 0 } .

Original: This range is relatively large when compared to theoptimal values.
Revised: This range is relatively large when compared to the optimal values.

Original: These observations support our Theorems 3 and 4.
Revised: These observations support our Theorems 3 and 4.

Original: Finally, w 0 = 0 does not worksince ∂L∂w 2 (cid:12)(cid:12) w 2 =

Revised: Finally, w 0 = 0 does not work since ∂L∂w 2 (cid:12)(cid:12) w 2 =

Original: = 0 always holds, which prevents the weight updating using gradient methods.
Revised: = 0 always holds, which prevents the weight updating using gradient methods

Revised: [Table]

Original: [Table]
Revised: [Table]

Original: 5.3 Convexity Guarantees of C O NS O L E to Learn Correct Equations
Revised: 5.3 Convexity Guarantees of C O NS O L E to Learn Correct Equations

Original: In this subsection, we report the results of the equation learning.
Revised: In this subsection [Table] we report the results of the equation learning.

Original: First, we list the learned equationsfor Syn 1 and Syn 2 in Table 1.
Revised: First [Table] we list the learned equations for Syn 1 and Syn 2 in Table 1.

Original: Table 1 presents that
Revised: Table 1 presents that

Original: C
Revised: C

Original: O
Revised: O

Original: NS
Revised: NS

Original: O
Revised: O

Original: L E has the best performance in most ofthe equations while DSR ranks second.
Revised: L E has the best performance in most of the equations while DSR ranks second.

Original: In particular, C O NS O L E can accurately learn all equations in
Revised: In particular, C O NS O L E can accurately learn all equations in

Original: Syn 1 and Syn 2 .
Revised: Syn 1 and Syn 2 .

Original: The superior performance is mostly due to the convex design of the search and thecoefficient estimation process with provable guarantees.
Revised: The superior performance is mostly due to the convex design of the search and the coefficient estimation process with provable guarantees.

Original: In addition, we observe that for the resultof C O NS O L E in learning y 3 of Syn 1 , we have 3 x 23 cos(0 .
Revised: In addition, we observe that for the result of C O NS O L E in learning y 3 of Syn 1 , we have 3 x 23 cos(0 .

Original: 005 x 1 ) ≈ 3 x 23 .
Revised: 005 x 1 ) ≈ 3 x 23 .

Original: This shows that there is apossibility that the search result of C O NS O L E might not be optimal (i.e., an extra consine term existsbut is close to 1 ), but the learned equation is still highly accurate.
Revised: This shows that there is a possibility that the search result of C O NS O L E might not be optimal (i.e., an extra consine term exists but is close to 1 ), but the learned equation is still highly accurate.

Original: Such an observation nonethelessguides further study of the coupling relationship between the search and the estimation procedures.
Revised: Such an observation nonetheless guides further study of the coupling relationship between the search and the estimation procedures.

Original:  DSR doesn’t perform well when the underlying equation is relatively complex, e.g., y 1 in Syn 1 and y 1and y 2 in Syn 2 .
Revised: [Figure] DSR doesn’t perform well when the underlying equation is relatively complex, e.g., y 1 in Syn 1 and yand y 2 in Syn 2 .

Original: This is because DSR may still fall into a local optimal solution despite risk-seekingpolicy design.
Revised: This is because DSR may still fall into a local optimal solution despite risk-seeking policy design.

Original: VPG method performs worse than DSR since VPG considers an expected reward [3].
Revised: VPG method performs worse than DSR since VPG considers an expected reward [3].

Original: Finally, EQL performs the worst as it merges the symbol search and the coefficient estimation in one
Revised: Finally, EQL performs the worst as it merges the symbol search and the coefficient estimation in one

Original: [Figure] NN model, which provides few guarantees of accurate learning.
Revised: NN model, which provides few guarantees of accurate learning.

Original: The above observations and analysesare consistent with the result of coefficient estimation errors and prediction errors in Fig. 3a and 3b.
Revised: The above observations and analyses are consistent with the result of coefficient estimation errors and prediction errors in Fig. 3a and 3b.

Original: For the Pow and Mas, C O NS O L E doesn’t learn completely exact equations within T = 600 episodes.
Revised: For the Pow and Mas, C O NS O L E doesn’t learn completely exact equations within T = 600 episodes.

Original: This is because they have a large number of variables to be considered.
Revised: This is because they have a large number of variables to be considered.

Original: However, C O NS O L E is stillbetter than other methods.
Revised: However, C O NS O L E is still better than other methods.

Original: 5.4 Ablation Study: Exploration and Convex Search are Essential
Revised: 5.4 Ablation Study: Exploration and Convex Search are Essential

Original: We conduct an ablation study to further understand what factors are important in the C O NS O L E .
Revised: We conduct an ablation study to further understand what factors are important in the C O NS O L E .

Original: We test the result with Syn 1 and Syn 1 and report the E c (%) values.
Revised: We test the result with Syn 1 and Syn 1 and report the E c (%) values.

Original: Specifically, we investigate thefollowing cases.
Revised: Specifically, we investigate the following cases.

Original: (1) No ablation.
Revised: (1) No ablation.

Original: (2) Drop exploration in deep Q-learning.
Revised: (2) Drop exploration in deep Q-learning.

Original: We delete the ϵ -greedystrategy.
Revised: We delete the ϵ -greedy strategy.

Original: (3) Drop double-convex deep-Q learning.
Revised: (3) Drop double-convex deep-Q learning.

Original: We replace this design with a traditional deep-Qlearning.
Revised: We replace this design with a traditional deep-Q learning.

Original: (4) Drop coefficient estimation using L O C A L. After learning the structure of L O C A L, wereformulate a non-linear optimization and utilize BFGS [11] in DSR, instead of gradient descent in
Revised: (4) Drop coefficient estimation using L O C A L. After learning the structure of L O C A L, we reformulate a non-linear optimization and utilize BFGS [11] in DSR, instead of gradient descent in

Original: L O C A L, to estimate the coefficient.
Revised: L O C A L, to estimate the coefficient.

Original: (5) Drop static symbolic constraint.
Revised: (5) Drop static symbolic constraint.

Original: (6) Drop dynamic symbolicconstraint.
Revised: (6) Drop dynamic symbolic constraint.

Original: These two constraints are mentioned in Section 3.2.
Revised: These two constraints are mentioned in Section 3.2.

Original: Then, Fig. 4 shows that cases (2) and(3) cause large errors.
Revised: Then, Fig. 4 shows that cases (2) and (3) cause large errors.

Original: For case (2) , if no exploration strategy is added, the updating of the Q-functionand the reward function is slow.
Revised: For case (2) , if no exploration strategy is added, the updating of the Q-function and the reward function is slow.

Original: For case (3) , the non-convex search induces many sub-optimalactions in the search process.
Revised: For case (3) , the non-convex search induces many sub-optimal actions in the search process.

Original: Thus, these two cases cause a slow search process and significant errorsafter T = 600 episodes.
Revised: Thus, these two cases cause a slow search process and significant errors after T = 600 episodes.

Original: For symbolic constraints in (5) and (6) , removing them increases the errorfor Syn 2 and Syn 1 , respectively.
Revised: For symbolic constraints in (5) and (6) , removing them increases the error for Syn 2 and Syn 1 , respectively.

Original: This shows these constraints are beneficial to the search process.
Revised: This shows these constraints are beneficial to the search process.

Original: Finally, we find that utilizing BFGS in (4) can bring good results with initialization in the locallyconvex region.
Revised: Finally, we find that utilizing BFGS in (4) can bring good results with initialization in the locally convex region.

Original: Since the non-linear optimization has the same loss surface as L O C A L, the locallyconvex region in Theorems 3 and 4 can prove the good performance of BFGS. 
Revised: Since the non-linear optimization has the same loss surface as L O C A L, the locally convex region in Theorems 3 and 4 can prove the good performance of BFGS. [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original:  C O NS O L E is Robust with Changing Noise Levels and Data Volume
Revised: 5. C O NS O L E is Robust with Changing Noise Levels and Data Volume

Original: We utilize Syn 1 and Syn 2 to examine the robustness of the framework with changing noiselevels and data volumes.
Revised: We utilize Syn 1 and Syn 2 to examine the robustness of the framework with changing noise levels and data volumes.

Original: For the noise level, we consider the Signal-to-Noise Ratio (SNR)such that SNR ∈ { 80 , 90 , 100 , 110 , 120 } .
Revised: For the noise level, we consider the Signal-to-Noise Ratio (SNR) such that SNR ∈ { 80 , 90 , 100 , 110 , 120 } .

Original: For the data volume, we fix SNR = 100 and vary
Revised: For the data volume, we fix SNR = 100 and vary

Original: N ∈ { 500 , 1000 , 1500 , 2000 , 2500 } .
Revised: N ∈ { 500 , 1000 , 1500 , 2000 , 2500 } .

Original: Fig.
Revised: Fig.

Original: 5a and 5b demonstrate the results.
Revised: 5a and 5b demonstrate the results.

Original: We find that when
Revised: We find that when

Original: SNR ≥ 100 , the error can be less than 1% .
Revised: SNR ≥ 100 , the error can be less than 1% .

Original: This noise level is suitable to real-word systems.
Revised: This noise level is suitable to real-word systems.

Original: Forexample, SNR = 125 for electric measurements [38].
Revised: For example, SNR = 125 for electric measurements [38].

Original: For the data volume, the overall error is lessthan 1 .
Revised: For the data volume, the overall error is less than 1 .

Original: 5% when N ≥ 500 , which shows a robust performance of C O NS O L E .
Revised: 5% when N ≥ 500 , which shows a robust performance of C O NS O L E .

Original:  Conclusions and Future Work
Revised:  Conclusions and Future Work

Original: In this paper we propose C O NS O L E , a novel Convex Neural Symbolic Learning method, whichenjoys convexity with certain conditions to tackle Symbolic Regression with guaranteed performances.
Revised: In this paper, we propose C O NS O L E , a novel Convex Neural Symbolic Learning method, which enjoys convexity with certain conditions to tackle Symbolic Regression with guaranteed performances.

Original: Specifically, we convexify the search problem by proposing a double convex deep-Q learning.
Revised: Specifically, we convexify the search problem by proposing a double convex deep-Q learning.

Original: In themeantime, we prove the local and strict convexity of the coefficient estimation in our Locally-Convexequation Learner (L O C A L).
Revised: In the meantime, we prove the local and strict convexity of the coefficient estimation in our Locally-Convex equation Learner (L O C A L).

Original: To our best knowledge, C O NS O
Revised: To our best knowledge, C O NS O

Original: L E is the first method that providesguarantees with reasonable assumptions to learn exact equations.
Revised: L E is the first method that provides guarantees with reasonable assumptions to learn exact equations.

Original: Besides, C O NS O L E has, at aminimum, a broader impact on the following domains.
Revised: Besides, C O NS O L E has, at a minimum, a broader impact on the following domains.

Original: (1) Convex control for physical systems using
Revised: (1) Convex control for physical systems using

Original: Reinforcement Learning.
Revised: Reinforcement Learning.

Original: (2) Neural networks in engineering applications with local convexity.Checklist For all authors...( a)
Revised: (2) Neural networks in engineering applications with local convexity.

Original: Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?

Original: [Yes]

Original: (b)

Original: Did you describe the limitations of your work?

Original: Yes We specify assumptions
Revised: Acknowledgments and Disclosure of Funding

Original: neededfor good

Original: performances

Original: in

Original: Section 4.

Original: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A]

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?

Original: [Yes]

Original: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] See Section 4.(b)

Original: Did you include complete proofs of all theoretical results?

Original: [Yes] See Appendix A.3 toA.

Original: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?

Original: [No] We will release

Original: the code if the paper is accepted.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters,

Original: how they

Original: were chosen)?

Original: [Yes] See Section 5.1. (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)? [Yes] See Section 5.3. (d) Did you include the total amount of compute andthe type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: This work is partially supported by NSF (1947135, 2134079, 1939725, ECCS-1810537, and ECCS2048288), DARPA (HR001121C0165), ARO (W911NF2110088), DOE (DE-AR00001858-1631 and

Original: [N/A]

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?

Original: [N/A]

Original: (b) Did you mention the license of the assets?

Original: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?

Original: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?

Original: [N/A]

Original: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: [Table]

Original: applicable?

Original: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?

Original: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: [Table]

Original: spent on participant compensation?

Original: [N/A]

Original: [Table]

Original: A AppendixA.1 Model Overview with Pseudo Codes

Original: In this subsection, we provide a high-level summary of our framework for better understanding.

Original: Wepresent the summary in the form of pseudo codes, shown in Algorithm 1.

Original: Algorithm 1 The Overview of the Proposed Framework.

Original: Input: Training dataset { x i , y i } Ni =1 .

Original: Step 1: Design L O C A L. L O C

Original: A L has repeated blocks of symbolic activation, multiplication, and summation layers.

Original: For example, Fig. 1 presents a L O C A L with 2 blocks.

Original: Step 2:

Original: Denote L O C A L Function.

Original: L O C A L represents the map f

Original: ( x ; { Z k } K − 1 k =0 , { W k } K − 1 k =0 ) from x to y .

Original: With global optimal solutions of Z k and W k , f ( x ) can be simplified to the true equation g ( x ) .

Original: while L O C A L does not have the optimal performance do

Original: Step 3: Search L O C A L Structure.

Original: Step 3.1: Model the Search Process.

Original: Build the CMP and the reward function R ( · ) based on states and actions defined over L O C A L. Formulate a sequential optimization.

Original: Step 3.2: Solve the Optimization.

Original: Utilize the proposed double convex Q-learning to find optimal actions.

Original: Generate a search result of { Z k } K − 1 k =0 .

Original: Step 4:

Original: Estimate L O C

Original: A L Parameters.

Original: Train the searched L O C A L by minimizing the MSE via Adam.

Original: Estimate values in { W k } K − 1 k =0 .

Original: Step 5: Evaluate the Search and Estimation Results.

Original: The results can formulate f t ( x ) for the t th episode.

Original: Calculate the end-of-trajectory reward R t to evaluate f t ( x ) .

Original: Output: L O C A L with the best performance and the corresponding equations.

Original: A.2 Training Algorithm for C O NS O L E .

Original: The training algorithm can be seen in Algorithm 2.

Original: A.3 Proofs of Theorem

Original: Theorem.

Original: ∀ 0 ≤ k ≤ K − 1 , the negative optimal Q-function − Q ∗ ( s k , ˜ a k ) in the proposed

Original: C O NS O L E framework exists and is convex in s k and ˜ a k , where s k is the discrete state and ˜ a k is thecontinuous action at the k th stage.

Original: Proof.

Original: First, we show our state transition satisfies the Markov property.

Original: Specifically, Equation (1)in our paper shows that the next state s k +1 equals the matrix multiplication between the currentstate s k and the matrix Z k that is a matricization of the current action a k , where k is the index ofthe state.

Original: Therefore, the state transition satisfies Markov property with the transition probability

Original: P ( s k +1 | s k , a k ) = 1 .

Original: Due to the Markov property of the state transition, we define our search process as Controlled Markov

Original: Process (CMP) [31, 30].

Original: By the CMP definition [30], our CMP is composed of our state, action, statetransition probability, a discounter factor γ , and a start state (i.e., s 0 in Equation (1)).

Original: In general,

Original: CMP is a Markov Decision Process (MDP) without a reward function [31].

Original: For one CMP, Trajectory Ordering (TO) ranks trajectories of state action pairs [31].

Original: In our paper,we define the trajectory from ( s 0 , a 0 ) to ( s K − 1 , a K − 1 ) for K -layer L O C A L.

Original: Then, our rewardfunction R ( · ) realizes a TO for our defined trajectories [31] since the ordering of trajectories can bedetermined by R ( · ) .

Original: More specifically, R ( · ) is trained with the end-of-trajectory reward R t for thet th trajectory in our paper and can rank trajectories.

Original: A reward bundle is an automation-like structureto produce rewards for a CMP [30].

Original: By Corollary 2 of [30], there exists a reward bundle for ourdefined CMP and TO realized by R ( · ) .

Original: We pair our CMP with the reward bundle to form a Split Partially Observable MDP (Split-POMDP)[30].

Original: Then, by Proposition 1 and Corollary 1 in [30], our Split-POMDP will always have an optimaldeterministic policy that only depends on states in our CMP.

Original: By the proof of Proposition 1 in [30],

Original: Algorithm

Original: Input: Training dataset { x i , y i } Ni =1 .

Original: Initialize:

Original: L O C

Original: [-A L layer number K , initial state s 
-]
Revised: [-A L layer number K , initial state s 
-]

Original: =

Original: [ 1 , 0 ] T , discount factor γ ∈ (0 , 1) , ϵ for ϵ -greedy strategy, λ as a threshold to stop searching, ICNN for reward function − R ( s , a ) , ICNN for Q-function − Q ( s , a ) , replay buffer B = ∅ , maximum episode T , target network Q ′ ( · ) = Q ( · ) , and target network update interval T 0 .

Original: while t ≤ T do while k ≤ K do

Original: Solve Optimization in Equation (3) with − Q ( s tk , a ) to obtain ˜ a ∗ k .

Original: Use ϵ -greedy to select ˜ a tk from ˜ a ∗ k and a random action.

Original: ▷ ϵ -greedy strategy.

Original: Discretize ˜ a tk to obtain a tk .

Original: Execute a tk and use Equation (1) to obtain s kk +1 .

Original: Check if a tk and s kk +1 satisfy certain constraints.

Original: Otherwise, delete this state transition and restart the iteration from s tk .

Original: ▷ Constraint checking.

Original: Formulate L O C

Original: A L, train L O C

Original: A L with { x i , y i } Ni =1 , and calculate R t .

Original: Train the reward function − R ( · ) using training data {{ s tk , a tk } K − 1 k =0 , − R t } .

Original: ∀ 0 ≤ k ≤ K , insert ( s tk , a tk , s tk +1 , R t ) and ( s tk , ˜ a t k , s tk

Original: +1 , R ( s tk , ˜ a t k )) to B 0 .

Original: Sample a minibatch B 0 ⊂ B for ( s m , a m , s m +1 , R m ) ∈ B do ▷ Experience replay.

Original: Solve Optimization in Equation (3) with − Q ′ ( s m +1 , a ) to obtain ˜ a m +1 .

Original: y m

Original: = R m + γQ ′

Original: ( s m , a m )

Original: [-Train Q ( · ) using training data { s m +1 , a m +1 , y m } m if t mod T 0 = 
-]
Revised: [-Train Q ( · ) using training data { s m +1 , a m +1 , y m } m if t mod T 0 = 
-]

Original: then

Original: Q

Original: [Equation]

Original: ▷ Update target Q-network.

Original: if | R t − 1 | ≤ λ then

Original: End the search process.

Original: Output: L O C A L with the best performance and the corresponding equations.

Original: the optimal policy optimizes the value function over states in CMP.

Original: Further, the value function isan evaluation of trajectories for our TO by the proof in Corollary 2 in [30].

Original: Additionally, our TO isrealized by our proposed reward function R ( · ) .

Original: Therefore, the optimal Q-function exists for our CMPand our proposed R ( · ) .

Original: Then, we consider the Bellman Equation of Q ∗ ( · ) :

Original: [Equation]

Original: where the second equality holds since our state transitions are deterministic by Equation (1).

Original: Weprove the convexity from the induction method.

Original: When k = K − 1 , the ( k + 1) th state is the terminalstate without action selections.

Original: Thus, we have

Original: [Equation]

Original: Since − R ( · ) is an ICNN and is convex in input, − Q ∗ ( s K − 1 , ˜ a K − 1 ) is convex in s K − 1 and ˜ a K − 1 .

Original: When 0 ≤ k < K − 1 and assume − Q ∗ ( s k +1 , ˜ a k +1 ) is convex in s k +1 and ˜ a k +1 , we have− max ˜ a Q ∗ ( s k +1 , ˜ a )

Original: =

Original: min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in s k +1 given the fixed optimal action.

Original: Let

Original: H denote the Hessian matrix of min ˜ a − Q ∗ ( s k +1 , ˜ a ) with respect to s k +1 .

Original: Due to the convexity,

Original: H is positive semi-definite.

Original: Thus, by Equation (1) and the chain rule, the Hessian matrix ofmin ˜ a − Q ∗ ( s k +1 , ˜ a ) with respect to s k can be written as:

Original: [Equation]

Original: H ′ is also positive semi-definite.

Original: Therefore, min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in s k .

Original: Since − R ( s k , ˜ a k )is convex in s k , − Q ∗ ( s k , ˜ a k ) is convex in s k .

Original: Similarly, vectorizing the state transition equation can give:

Original: [Equation]

Original: where I n s is the n s × n s identity matrix and (cid:78) is the Kronecker product.

Original: a ′ k

Original: =

Original: [( a k ) T , 0 ] T is theconcatenation of the discrete action a k and a zero vector to maintain the fixed dimensionality ofaction vectors.

Original: With similar proofs based on the Hessian matrix and the fact that − Q ∗ ( s k +1 , ˜ a k ) isconvex in s k +1 , we have min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in a ′ k and also a k .

Original: Subsequently, arbitrary˜ a k ∈ conv ( { 0 , 1 } n a ) can be written as a convex combination of the discrete actions a k .

Original: Thus,min ˜ a − Q ∗ ( s k +1 , ˜ a ) is convex in ˜ a k .

Original: Since − R ( s k , ˜ a k ) is convex in ˜ a k , − Q ∗ ( s k , ˜ a k ) is convex in˜ a k .

Original: Eventually, − Q ∗ ( s k , ˜ a k ) is convex in s k and ˜ a k , which concludes the proof.

Original: ■

Original: A.4 Proofs of Theorem

Original: Theorem.

Original: Let f ∗ ( · ; W ) denote the L O C A L constructed by the optimal sequences of states( s 0 , s ∗ 1 , · · · , s ∗ K ) and actions ( a ∗ 0 , a ∗ 1 , · · · , a ∗ K − 1 ) from − Q ∗ ( · ) , where W is the set of weightsof f ∗ ( · ; W ) .

Original: If f ∗ ( · ; W ) can be trained with noiseless datasets and the training can achieve theglobal optimal weights W ∗ , f ∗ ( · ; W ∗ ) can be simplified to the true equation g ( · ) .

Original: Proof.

Original: If f ∗ ( · ; W ) can’t represent the exact equations, there are two cases: (1) the structure off ∗ ( · ; W ) is correct to represent the equations, but the learned weights W ∗ don’t represent thesymbol coefficients, and (2) the structure of f ∗ ( · ; W ) can’t represent the equations.

Original: Case (1) doesn’thold since we assume W ∗ is the global optimal weights for noiseless data.

Original: [-If case (2) holds,∃ 
-]
Revised: [-If case (2) holds,∃ 
-]

Original: ≤ j ≤ K − 1 , b ∗ j = min ˜ a j − Q ∗ ( s j , ˜ a j ) and b ∗ j doesn’t represent the symbol connections in theunderlying equations.

Original: Further, we assume ∀ 0 ≤ i < j , a ∗ i =

Original: min

Original: ˜ a i − Q ( s i , ˜ a i ) and a ∗ i representsthe true connections.

Original: If j = K − 1 , Equation (4) implies that ˜ a ∗ j = min ˜ a j − Q ∗ ( s j , ˜ a j ) = arg min ˜ a − R ( s j , ˜ a ) .

Original: Since− R ( s j , ˜ a ) is convex in ˜ a

Original: we know the discrete version of ˜ a ∗ j , namely a ∗ j , represents the trueconnection of the last layer for the underlying equations.

Original: Otherwise, the reward is not maximized.

Original: However, by definition of b ∗ j , b ∗ j ̸ = a ∗ j .

Original: If j < K − 1 , Equation (4) implies:

Original: [Equation]

Original: By definition of b ∗ j , b ∗ j is not the solution of Equation (5).

Original: This is because b ∗ j can’t achieve theminimum value for each summation term on the right hand side of Equation (5), according to theconvexity of the reward function.

Original: In general, b ∗ j ̸ = min ˜ a j − Q ∗ ( s j , ˜ a j ) , which contradicts thedefinition of b ∗ j .

Original: Thus, b ∗ j doesn’t exist.

Original: Therefore, case (2) doesn’t hold and f ∗ ( · ; W ∗ ) representsthe exact equations.

Original: ■A.5 Proofs of Theorem 3

Original: Theorem.

Original: Assume the following conditions hold: (1) the equation g ( x ) is C 2 smooth and hasbounded second derivatives with respect to weights, (2) ∃ x ∈ X , g ( x ) has non-zero gradients withrespect to weights, (3) the structure of L O C A L is correctly searched to exactly represent symbolsand symbol connections in g ( x ) , and (4) the training dataset of L O C

Original: A L is noiseless.

Original: Then, for the

Original: MSE loss surface of L O C A L , each global optimal point has a strictly convex local region.

Original: Proof.

Original: To simplify the proof, we consider scalar output of the L O C

Original: A L, i.e., one equation, and theproof can be easily extended to the multi-output case.

Original: We follow the idea of [29] to study the secondderivative of L O C A L with perturbations.

Original: Let ˆ y ( x , W ) denote the L O C A L with input to be x and theweight set to be W .

Original: Let X be a perturbation direction of W and t be a small step size.

Original: For the i thnoiseless instance ( x i , y i ) , we denote e ( x i , W + tX )

Original: =

Original: ˆ y ( x i , W + tX ) − y i .

Original: Obviously, the lossfunction can be written as L ( W + tX )

Original: = 12 N (cid:80) Ni

Original: =1 ( e ( x i , W + tX )) 2 .

Original: Then, we can calculate the second-order derivative based on the chain rule:

Original: [Equation]

Original: Next, we denote the global optimal solution to be W ∗ .

Original: Based on the Assumptions (3)and (4) , ∀ i, ˆ y ( x i , W ∗ )

Original: =

Original: g ( x i ) =

Original: y i .

Original: [-Therefore, we have d 2 dt 2 (cid:12)(cid:12) t =
-]
Revised: [-Therefore, we have d 2 dt 2 (cid:12)(cid:12) t =
-]

Original: L ( W ∗ + tX ) =

Original: 1 N

Original: [Equation]

Original: Ni =1 (cid:0) ddt (cid:12)(cid:12) t

Original: [-=
-]
Revised: [-=
-]

Original: ˆ y

Original: ( x i , W ∗ ) (cid:1) 2 > 0 , where the inequality strictly holds.

Original: This is because by Assumptions (3) , ˆ y ( x , W ∗ ) can be mathematically simplified to obtain g ( x ) .

Original: Then, by Assumption (2) ,1 N (cid:80) Ni =1 (cid:0) ddt (cid:12)(cid:12) t

Original: [-=
-]
Revised: [-=
-]

Original: ˆ y

Original: ( x i , W ∗ ) (cid:1) 2 > 0 .

Original: [-Finally, by Assumption (1) and (3) , d 2 dt 2 (cid:12)(cid:12) t =
-]
Revised: [-Finally, by Assumption (1) and (3) , d 2 dt 2 (cid:12)(cid:12) t =
-]

Original: ˆ y

Original: ( x i , W + tX ) isbounded and there is a local region around W ∗ such that d 2 dt 2 (cid:12)(cid:12) t

Original: [-=
-]
Revised: [-=
-]

Original: L ( W + tX ) > 0 , which concludesthe proof.

Original: ■

Original: [Table]

Original: A.6 Proofs of Theorem 4

Original: Theorem.

Original: Suppose Assumptions 1-4 in Theorem 3 hold.

Original: For a L O C A L with one symbolic activation,multiplication, and summation layer, the set of local convex regions with global optima is U =

Original: [Equation]

Original: } , where notations are defined in the proof.

Original: Proof.

Original: For the target L O C A L, we similarly consider the scalar output and write the function analytically:

Original: [Equation]

Original: where W 0 ∈ R n 0 × n 1 is the weight matrix for activation, Φ : R n 1 → R n 1 represents the activationwith symbol functions like x 2 , cos ( x ) , and log( x ) , etc.

Original: Ψ : R n 1 → R n 2 is the function to selectsome activated neurons for multiplications, and W 1 ∈ R n 2 × n 3 ( n 3 = 1 ) represents the weight forsummation.

Original: We rewrite Equation (7) with the help of exponential and logarithm mappings.

Original: [Equation]

Original: where S ∈ R n 1 × n 2 represents a selection matrix such that S [ i, j ] = 1 if and only if the i th neuronis selected as the multiplicative factor for the j th neuron in the multiplication layer.

Original: Given thefixed structure of ˆ y ( · ) from the deep Q-learning, S is a known matrix.

Original: log( · ) and exp( · ) representthe element-wise logarithm and exponential functions.

Original: Notably, the corresponding element inΦ( W T 0 x ) should be positive in Equation (8).

Original: If there are negative entries, one can utilize W T 1 s ◦exp (cid:18)

Original: S T log (cid:16) | Φ( W T 0 x ) | (cid:17)(cid:19) to take place of the right hand side term in Equation (8), where s [ i ] =( − 1) n i − and 0 ≤ n i − ≤ n 1 represents the number of negative entries selected for the i th neuron ofthe multiplication layer.

Original: ◦ represents the Hadamard product.

Original: However, both expressions have thesame values and gradients.

Original: Thus, we utilize Equation (8) in later derivations.

Original: Then, let X be a perturbation direction such that X

Original: = { X 0 , X 1 } .

Original: Thus, for a small step t , we have:

Original: [Equation]

Original: Based on Equation (9), we can compute:

Original: [Equation]

Original: where 1 Φ(( W 0 + t X 0 )

Original: T x i ) ∈ R n 1 is the element-wise division and Φ ′ is the element-wise first derivativeof Φ ′ .

Original: Without special notifications, we assume all the division for vectors is element-wise in thefollowing derivations.

Original: Then, we denote

Original: [Equation]

Original: With above definitions

Original: [Equation]

Original: we can calculate:

Original: [Equation]

Original: Further, we calculate the second derivative based on Equation (10) and the fact that element-wiseoperations for vectors are commutative:

Original: [Equation]

Original: When t → 0 , we have:

Original: [Equation]

Original: The above equation can reflect the relationship between the second and the first derivative.

Original: However,we first identify the inequality between these two derivatives to enable a strictly convex region.

Original: Let ˆ

Original: [Equation]

Original: y ( x 1 , W +tX ) , · · · , ddt 2 (cid:12)(cid:12) t

Original: [-=
-]
Revised: [-=
-]

Original: ˆ y ( x N , W + tX )]

Original: T , and e =

Original: [ e ( x 1 , W ) , · · · , e ( x N , W )] T .

Original: Equation (6) impliesthat:

Original: [Equation]

Original: To find a region to restrict the convexity, we restrict the lower bound of the second derivative to bepositive and compute:

Original: [Equation]

Original: The right hand side of Equation (16) can be easily bounded by:

Original: [Equation]

Original: where |·| for a vector is to calculate the absolute value for each element of the vector

Original: [Equation]

Original: i = arg min( | ˆ y ′ | )and j = arg max( | ˆ y ′′ | ) .

Original: Namely, we consider a sufficient condition for convexity.

Original: [Equation]

Original: Next, Equation (14) indicates that:

Original: [Equation]

Original: where η is a positive constant.

Original: Note that η < ∞ by Assumptions (1) and (2) in Theorem 3.

Original: [-Therefore,we have the following sufficient condition to make d 2 dt 2 (cid:12)(cid:12) t =
-]
Revised: [-Therefore,we have the following sufficient condition to make d 2 dt 2 (cid:12)(cid:12) t =
-]

Original: L ( W + tX ) > 0 always hold.

Original: [Equation]

Original: where k = arg max( | e | )

Original: The above equation leads to a set U of local regions that have strongconvexity.

Original: Namely,

Original: [Equation]

Original: [Equation]

Original: Clearly, the global optimal solution W ∗ ∈ U since ˆ y ( x k , W ∗ ) − y k = 0 .

Original: Note that there may bemultiple global optimal solutions of the loss minimization in L O C A

Original: L. Thus, U is the set of localconvex regions that contain global optima.

Original: This implies that for each W ∗ ∈

Original: U , we can find a locallyand strictly convex region U ∗ =

Original: U ∩ B ( r ) , where B ( r ) = || w − w ∗ ||
Revised: DE-EE0009355)

Original: 2 ≤ r is a norm ball and wevectorize W and W ∗ to obtain w and w ∗ , respectively.

Original: Subsequently, range r can be set relativelylarge such that U ∗ ⊂ B ( r ) and U ∗∗ ∩ B ( r ) = ∅ , where U ∗∗ is the local region for another globaloptimal point W ∗∗ if it exists.

Original: Then, the range for U ∗ still depends on the inequality in Equation(21).

Original: ■