CoNSoLe: Convex Neural Symbolic Learning
Authors: Haoran Li
Yang Weng
Hanghang Tong
Venue: NeurIPS 2022 Accept (2022)
Keywords: Symbolic Regression
Physical Equations
Neural Networks
Convex Guarantees
Deep Q Learning

Abstract: Learning the underlying equation from data is a fundamental problem in many disciplines. Recent advances rely on Neural Networks (NNs) but do not provide theoretical guarantees in obtaining the exact equations owing to the non-convexity of NNs. In this paper, we propose Convex Neural Symbolic Learning (CoNSoLe) to seek convexity under mild conditions. The main idea is to decompose the recovering process into two steps and convexify each step. In the first step of searching for right symbols, we convexify the deep Q-learning. The key is to maintain double convexity for both the negative Q-function and the negative reward function in each iteration, leading to provable convexity of the negative optimal Q  function to learn the true symbol connections. Conditioned on the exact searching result, we construct a Locally Convex equation Learning (LoCaL) neural network to convexify the estimation of symbol coefficients. With such a design, we quantify a large region with strict convexity in the loss surface of LoCaL for commonly used physical functions. Finally, we demonstrate the superior performance of the CoNSoLe framework over the state-of-the-art on a diverse set of datasets.

CONSOLE: Convex Neural Symbolic Learning
Haoran Li
Yang Weng Arizona State University Tempe, AZ, 85287
Hanghang Tong
University Illinois Trtban-Champaigh Champaign, II, 61820 htong @ @illinois.edu
(1haoran, yang. weng) Basu. edu
Abstract
Learning the underlying equation from data is a fundamental problem in many disciplines. Recent advances rely on Neural Networks (NNs) but do not provide theoretical guarantees in obtaining the exact equations owing to the non-convexity NNs. In this paper, we propose Convex Neural Symbolic Learning (CoNSoLE) seek convexity under mild conditions. The main idea is decompose the recovering process into two steps and convexify each step. In the first step of searching for right symbols, we convexify the deep Q-learning. The key is maintain double convexity for both the negative Q-function and the negative reward function in each iteration, leading to provable convexity of the negative optimal Q function to Iearn the true symbol connections. Conditioned on the exact searching result, we construct to convexify the estimation of symbol coefficients. With such design, we quantify large region with strict convexity in the loss surface of LoCAL for commonly used physical functions. Finally, we demonstrate the superior performance of the CoNSOLE framework over the state-of-the of-thheart on a diverse of datasets.
Docally-Convee equation Learner (LoCaL) neural network
Introduction
Identifying the underlying mathematical expressions from data plays a key role in multiple domains. For example, scientific discovery naturally requires learning analytical solutions to fit data. Engi- neering systems also need to frequently re-estimate system equations due events, maintenance, upgrading. and new constructions [1], etc. In general, the problem, known as Symbolic Regres- sion (SR) [2], learns the underlying equations y g(x) constructed via certain symbols, where and y represent the input/output vector of the equations. If successfully learned, the equation can enjoy many important properties like high interpretability and generalizability, which will in turn significantly benefit scientific understanding and engineering planning, monitoring, and control.
Promising as it might be, SR is NP-hard [3, 4]. Mathematically, problem over both discrete variables to select symbols and continuous variables to represent the symbol coefficients. Traditional solutions employ evolutionary algorithms like Genetie Programming (GP) [5]. These methods start from an initial set of expressions and continue evolving via operations like crossover mutation. With fitness measures, GP- algorithms best equations. However, these methods have poor scalability and limited theoretical guarantees [3].
can cast SR as optimization
evaluate and select the
More recent SR studies leverage Neural Networks (NNs) with high representational power: For the NN-based SR, we mainly categorize them into two groups based on the roles of the NNs. The first group employs NNs to directly model the equations, where sparsity of the NN weights is enforced to select symbols [6-10]. Thus, the problem is transformed into training the designed NN with sparsity regularization. However, due to the non-convexity of NNs, the weight selection and updating can be easily stuck local optima, failing to find the exact equations.
The second group employs NNs to search the symbol connections, and non-lineat optimizations like BFGS [11] can be employed estimate symbol coefficients. For the searching procedure, [3, 12-14] leverage Recursive Neural Network (RNN) as policy network to iteratively generate optimal
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

actions that can select and connect symbols. [15] employs large-scale pre- tre-irrining to directly map from data to the symbolic equations. While these methods restrict the utilization of NNs in the search phase, the non-convexity of NNs can still suffer the risk of sub-optimal decisions to formulate equations. To mitigate this issue, some efforts have been made such as risk-seeking policy gradient to find the best equations [3], restricting the searching space via domain knowledge [16] and entropy regularization [13], and re-initialization [13], etc. However, they have limited theoretical guarantees.
In this paper, we propose Convex Neural Symbolic Learning (CONSOLE) with convexity under moderate conditions. To our best knowledge, we are the first to provide provable guarantees to learn the exact equations. In general, we decompose the SR problem into two sub-problems and seek convexity, respectively. In the first problem of searching symbols, we propose a double-convexifted deep Q-learning to maintain the convexity of negative Q-function and negative reward functions with continuous action variables. Specifically, we use the Input Convex Neural Network (ICNN) [17] to represent both the negative Q-function and the negative reward function in each iteration. Subsequently, we prove that such a design (1) guarantees an optimal action selection at each step and (2) ensures the negative optimal Q-function, if successfully found, to be convex. Therefore, the convex negative optimal Q-function can yield global optimal decisions equation constructions.
In the second problem of coefficient estimation, we use the search result to build a Locally-Conve Equation Learner (LoCaL) neural network. The key insight is that if the search result is correct, the loss surface of LoCAL has local regions that contain the global optima and has strict convexity. Moreover, we quantify the local regions and show the range of the region is large under mild assumptions. Therefore, initializations based on prior knowledge can often lie in the convex region, bringing the accurate coefficient estimation. Finally, we demonstrate that our CoNSoL. outperforms state-of-the-art methods on a diverse set of datasets.
2 Related Work
Symbolic regression using neural networks. In addition to the review in the Introduction, there are studies treating NNs as a data augmentation tool to create high-quality data for SR [18, 2].
Neural architecture search. Searching the connections of LoCAL falls into the area of Neural Architecture Search (NAS). NAS tries to find optimal architecture of a target NN with the best performance [19]. The searching algorithms can be divided into RL-based, evolutionary algorithm- based, sequential optimization-basee and gradient descent-
methods. For RL-based methods,
[20] employs an RNN model to sample the architecture and utilizes the accuracy of the sampled network as the reward. [21] uses tabular Q-learning to find the connections of a target NN. However, tabular Q-learning can hardly be applicable when the state and action spaces are large, e.g., SR problem. The evolutionary algorithm employs methods such as GP [22] and tournament selection [23]. However, these methods may lack the scalability for SR problem [3]. The sequential optimization- based method more scalable as the model complexity increases in a sequential manner [24]. Finally, the gradient descent-based method [25] builds a large and over- parameterized network to search and train. Then, regularizations like dropout are added to find the best connections. However, for these methods, the theoretical guarantees remain opaque for SR problem.
Global optimum in neural networks. Many studies have been conducted to seek global optimality in NNs, and they can be categorized into finding global optima for weights or input variables. The weight optimization is directly linked finding symbol coefficients LOCAL. Specificallyy [26] investigates a single hidden layer with unbounded neurons and non-Euclidean regularization. The authors show the training can be done via convex optimization problems. [27] considers finite neurons and develops a novel duality theory to train two-layer NNs with convexity. [28] establishes a strong result that every local optimum is a global optimum for deep non-linear networks under several assumptions. [29] eliminates these assumptions and finds that with weight decay regularization (e.g., l2 norm), the loss function of NN with ReLU activations is piece- wise strongly convex in local regions. However, LOCAL doesn't 't fit the above conditions. The second group of input variable optimization can help search for optimal inputs. [17] proposes ICNN such that the output of ICNN is convex in input variables. The key of ICNN is to restrict some weights and activation functions to preserve convexity. ICNN facilitates to design a convexified search algorithm.
3 Convex Neural Symbolic Learning
3.1 LOCAL to Hierarchically Represent the Equations and Learn Symbol Coefficients
2

---

SR problem can be decomposed into searching the symbol connections and estimating the symbol coefficients. The link of these two sub-pree problems a proper representation of the underlying equation with unknown structures and weights. Then, we need to search the structures and estimate the weights. We utilize a neural network to represent the multi-input multi-output equations due to the efficiency [6]. We prove in Theorem 3 that under mild conditions and given correct structures of the NN, there are local regions in the loss surface with strict convexity. Thus, we name our NN as Locally Convex Equation Learner (LoCaL).
Steps: 1. Symbolic Activation c- = {x,x², cos(wx))
2. Multiplication
3. Summation
Y1 3xf cos cos(2.5x2 (2.5x))
X2
y/ (3x)s(3x,) 1.5x3)2
$O
T
0
0
0
0
0
1
0
0
Example of State
0
0
0
Transition $1=
0
0
0
0
+
(Z,)²s, So +
1
1
0
0
1
0
0
1
0 0
Figure 1:
An example of LoCAL structure and the state transition calculation.
LoCAL should have the capacity to represent the true equation. We assume the underlying equation = g(x)) follows compositionality and smoothness assumptions in [2], which are often the case in physics and many other scientific applications. Then, we build LoCAL that can be trained via input/output pairs {æ,, V4 ).)a where I; E X' and y; E y are the 2th samples and N is the number of data. X' and y are the input and output data spaces, respectively. By the compositionality, we propose to hierarchically map to y with correct symbols. Fig. 1 shows an example of the LoCAL. structure. Specifically, each input entry first goes through the activation functions from a symbol library Φ. For example, in Fig. 1, we denote + = {ar, x², cos (wxz) to correspond with three neurons from top to bottom in the dotted black box, where w is a learnable weight. As for weights of x and x², they only need to appear in the later summation layer. Then, some activation outputs will be selected as multipliers for the multiplication. Subsequently, the multiplied outputs are selected and summed together. The repetition of symbolic activation, multiplication, and summation formulates final equations. For instance, the LOCAL structure in Fig. 1 can correctly represents y/1 and y/2 shown on the top right of Fig. 1.
The layer-wise connectivity and the weights of LOCAL are optimization variables for the SR problem. x x and W Tg as the indicator and weight matrix between the htth and the (k: + 1)th layer of the LOCAL, respectively, n& is the number of neurons for the 4tth layers. Zx(i,j] = 1 indicates that the connection exists between the tt neuron in the h layer and the jth neuron the (k: + 1)th layer, where [ j] the (i,j) entry of Zr. We assume there are (K + 1) layers in LOCAL and denote h2 R to be the output of the tt layer. Naturally, we have hg x and hx = ÿ. where ÿ is the output of LoCAL. For the symbolic activation or summation layer, we have ha+1 2 o W.,hb, where represents the Hadamard product and helps to zero out some connections. For the multiplication layer, we have h [i] = I [ h& with no weight matrix involved. In general, we denote LoCAL as K-1
Mathematically, we denote Z E (0, 1 1
(m;; ( &
k=o | (W*)k=) ). The search algorithm identifies { {Zx ) K-1 and the estimation process
I==0 learns the corresponding weights in ( Wikk
K-1 using (x,, V: 1
3.2 Search LOCAL Structures with Double Convex Deep Q-Learning
State and action definitions in the search process. To search the structure of LOCAL, we treat the status of each layer of LOCAL as a state and the connections between layers, i.e., Z5 LoCAL, as actions. Specifically, we denote ag {0, 1 na as the th action vector where aul(i 1)ny + =
3

---

implies that the connection ij exists for the tth neuron in the kth layer and the jth neuron in the (k + 1)th layer. na = max K KK11
is the dimensionality of the action space. Also,
we denote S/: E td as the state vector to represent the current state for the htth layers, where n, = max T===== k== + 1 is the dimensionality of the state space. More specifically, the state vector Sy is composed of the values for each neuron in the kth layer and an entry of state index k. The appending of the value k can avoid the duplicate state vectors for different layers. Otherwise, the duplicated states may exist and require different optimal actions, deteriorating the correct search for the true structure of LOCAL. To further quantify state vectors, we utilize a linear transformation to represent the state transition process. 4A S I K 1, we define
80 [1,0]" 8k+1 (2 cc= 8 Mat((a)) O
0
8k c c, 1
(I)
where the number of Is in so corresponds to the input dimension. Mat(-) is the operation to reshape a vector to a matrix, and we utilize Os for padding to maintain the dimensions. c = [0, 0...... ..]' 1|T is constant vector to increase the entry of the state index from &o to le + 1. We show this linear state transition is essential to guarantee the convexity of negative optimal Q-function in Theorem 1. For the calculation example of state transition, one refer to Fig. 1. Then, we can obtain Z from Mat(as) (ax) by deleting the filled Os. Then, the search will always start at 8n and end at 8 in one episode. Thus, we define a trajectory as a sequence of searched state-action pairs for {(8), ]x) K-1 in the LoCAL function f(x:; [ K-1 [W&) This trajectory formulates {Zx k=
The above states, actions, state transitions, initial state 8o: and discount factor y can form Controlled Markov Process (CMP) [30, 31], which is a Markov Decision Process (MDP) without a reward function [31]. In the following part, we define our reward function based on an end-of- trajectory reward [30]. Although the reward function is non-Markov, we prove in Theorem I that under our settings, there exists an optimal Q-function. The proof can be seen in Appendix A.3.
Double convex deep Q-learning to search optimal actions. To optimize over the defined CMP for optimal actions, seek certain convexity with provable optimal results. Thus, we propose double convex deep Q-learning with the convex negative reward function and negative value function (i.e., Q-function). Based on Bellman equations [32], this design will ensure the convexity of the negative optimal Q-function and global optimal solutions. More proof details can be referred to Theorem and Appendix A.3.
Specifically, we utilize an Input Convex Neural Network (s) [17] to model the reward function R(sk, as) such that -f((s,, ax) is convex in states and actions. - -R(sk, ax) requires proper training to do the correct evaluation. In the 1th episode, we collect the 1th trajectory sample {( ]{]]-- K-1 Then, the output can be defined as the end-of-trajectory reward to evaluate the obtained LOCAL which is denoted as far; Wt), where W, the weight set of LoCAL. Basically, we utilize gradient method (e.g., Adam [33]) to train fæ: W.) and obtain the optimal set of weights W for the t episode of LOCAL by minimizing the Mean Square Error (MSE). Then, we can calculate the Normalized Root-Mean-Squanl Error (NRMSE) [3] of the trained fe(:;; W*) such that
where o, is the standard deviation of the outputs. 1 The output of the reward function can be calculated as RS = I+NRMSE, K-1 ).
NRMSE, = J. N (v, - Sz (a,;; W;))² 2
Therefore, we can train
- R()) using (1 at
Although training R(-) utilizes the samples of discrete states and actions, we aim to solve the continuous convex optimization for optimal sequential decisions. Thus, we first show that R(-) can be defined over the continuous action space. By definitions of the discrete actions, we restrict the continuous action space in a hypercube conv (10, 1)*²), ), i.e., convex hull of the discrete actions. Then, the discrete actions are the endpoints of the hypercube. Notably, it is doable to utilize a continuous convex function to fit these endpoints with end-of-trajectory rewards. Especially, the strictly minimal reward value - -R, only lies in the endpoint that guarantees the correct structure of LOCAL. Therefore, one can utilize piece-wise linear functions with convexity to represent R(-) over conv (0, 1 )m and ensure one endpoint has the minimal value. The piece-wise linearity can be achieved via using ReLu activations to R(-).
With the defined continuous action space, we utilize another ICNN to represent -Q(sk, āg) such that -Q (sx, ā&) is convex in the continuous action à E conv((O, 1)**) given fixed sg. To update

---

Q values, we have the following iterative computations based on the temporal difference [21]:
Q4+1(Sk, ax) (sks,, āx) + a ((((ss: ax) + 7 max Qc(sk+1, Qr(sk, āx)):
(2)
where Qs, a&) is the Q value at the tth episode for state 8u and action ag. a and 7 are pre-defined learning rate and discount factor, respectively. Therefore, one can solve a convex optimization problem maxa (sk+1 â) obtain the (approximately) global optimal action for Equation (2) in each iteration. Thus, the optimization problem is:
a' arg min - ((s,)) conv((0,1)**))
Based on [17], this convex optimization problem can be solved via bundle entropy method. After obtaining continuous solution ä", we discretize it to a discrete vector a" to build LoCAL. One simple method to enforce a [ = 1 if à a'li] 2 0.5, and otherwise a [ = 0. Thus, both (8s, āx) and (skk, a&)) can be trained using Equation (2). Practically, we introduce e-greedy strategy [34], experience replay [35] and target Q-network [36] to update Q-function, thus boosting the convergence to the optimal Q-function. The overview of our framework is in Appendix A. Algorithm 1. The specific algorithm is in Appendix A.2, Algorithm Finally, by Theorems 1-2, the discrete optimal actions can generate the correct structures of LoCAL and exact equations.
Symbolic static and dynamic constraints. Constraints can be added to accelerate the search process [3, 2]. For example, in Algorithm/ 2, we propose constraint checking program for the state-action pairs to avoid the invalid search, suitable for arbitrary restrictions. Then, we emphasize a general type of constraint, symbolic constraint, for the SR problem. The symbolic static constraint requires that each equation contains only a subset of symbols. For example, the equation y/1 = -2 . 100 shouldn't It exist since it is too complicated for real-world systems. This constraint eliminates part of the action space and can be checked by counting the number Is in the action vector. The symbolic dynamic constraint can gradually reduce the search space based on symbol correlations. Specifically, we investigate the (K - 1)th layer's neurons that are linearly summed to form the equation. some of these neurons have strong linear correlations to the output neuron (e.g., Pearson correlation coefficient larger than 0.99), they should be kept subsequently. Namely, we can maintain the path from input neurons to the neurons to be kept and reduce the search space.
4 Theoretical Analysis
We employ explorations, experience replay, and target Q-network boost the convergence to the optimal Q function. However, our extra requirement of the convex shape for the negative Q-function may deteriorate the convergence performance. Thus, we first prove that in CONSOLE, the negative optimal Q-function also convex so that the convex design docsn't 't affect the convergence. Then, we prove that the convexity of the negative optimal Q-function eventually yields the exact equations.
Theorem 1. 4A k S K 1, the negative optimal Q-function -Q" (8k: a&) in the proposed CONSOLE framework fraSoLerram exists and is convex in sy and ag where Sy is the discrete state and āg is the continuous action the kth stage.
The proof can be seen in Appendix A.3. Based on the convexity of negative optimal Q-function, the optimal sequence of states (su, ...... .xx) and actions (a,j aj.... .... .a.....aa... a can be found via solving the convex optimization problem. Then, we have the following theorem.
Theorem 2. Let (:::W) denote the LOCAL constructed by the optimal sequences of states (So. ....... ,sk) and actions (an, aj ai...................... ax=1) from (-), where W is the set of weights of (;) If f=; ()) can be trained with noiseless datasets and the training can achieve the global optimal weights W [=; (="W")) can be simplified to the true equation g(-).
For the optimal search structure of LOCAL, ie., f" W), the optimal weight set W is also trained via gradient method like Adam [33]. The proof can be seen in Appendix A.4. Theorem 2 requires that LOCAL can learn the global optimal weights. The requirement is generally hard to achieve due to the non-linearity and non-convexity of LoCAL. However, we show that with mild assumptions, there are local regions in the LOCAL loss surface with strict convexity. Then, if we have proper initializations, the gradient- based weight updating can find the global optimum. Specifically, we have the following theorem.
5

---

Theorem 3. Assume the following conditions hold: (1) the equation g(x) is C² smooth and has bounded second derivatives with respect to weights, (2) E E X. g(x) has non-zero gradients with respect to weights, (3) the structure of LOCAL is correctly searched to exactly represent symbols and symbol connections in g(æ),, and (4) the training dataset of LoCAL noiseless. Then, for the MSE loss surface of LoCAl,, each global optimal point has strictly convex local region.
The proofs can be seen in Appendix A.5. Note that Assumptions 1-2 easily hold for common physical equations in nature [2]. Assumption 3 relies the search algorithm, and we show, both theoretically and numerically, that our double convex deep Q-learning has good performances. Assumption relies on the quality of data and we focus the noiseless data in this paper. What's more, Equation (6) in the proof suggests that if the absolute noise values are small, the locally convex region still exists. We also numerically prove CONSOLE is robust under certain noise levels in Section 5.5. To summarize, these assumptions are acceptable. To quantify the range of the local region for a LOCAL with certain complexity, we have the following theorem.
Theorem 4. Suppose Assumptions 1-4 in Theorem 3 hold. For LoCAL with one symbolic activation, multiplication, and summation layer, the set of local convex regions with global optima is ((m. ((n. ++X)
U = {W] c-0
10(Z W) -yal3. where notations are defined in the proof.
5(
The proofs can be seen in Appendix A.6. We explain the region range is large for stable system that satisfies all assumptions in Theorem 4.
Physical interpretations of the convex region size. Physical systems in scientific and engineering domains have certain stability that can withstand parameter changes some extent. Further, this ability should hold for arbitrary
X' and y V. Thus, we assume d ( W + tX) s de 1== W + +X)= x de ( W + fX) for B;, Tj, 2% E X. Then, the inequality in set U can be approximately rewritten as 1 > || - w*lle. where w and w² are the vectorized W and W respectively. Namely, the distance between any point W in the region to the global optimal point W* in the region is bounded by +. Based on Equation (19), 7) is the bound of the ratio of second derivative to the first derivative. For a stable system, this ratio should small. Otherwise, the system can easily crash with small parameter disturbance. Thus. 1 is relatively large and so is the region of U. An example of the range is displayed in Section 5.2. Finally, the above analysis also holds when the LOCAL of the system equation has more than one symbolic activation, multiplication, and summation layers. This is because Equation (20) always holds as long as we can find a 17 to bound the ratio of the second derivative to the first derivative, which is irrelevant to the structure of LoCAL.
5 Experiments
5.1 Settings
Datasets. We use the following datasets for testing. (1) Synthetic datasets. We create two datasets, Syn, and Syn,, for testing. Syn, has the following equations:
3z cos(2.52)), 1/2 4z12z, and ys = 3x3- Syn, is more complex with the following equations y1 = v2.2r102 + 21 log(322)
U2 = =in((1811)) (10g33)))) + VTa). ys- = /3.7x3 1og(1.621) + 2². For the training data, each input variable is randomly sampled from a uniform distribution of (11, 2) to avoid invalid values like log(0). Totally, we create 2, 000 samples for training. Then, in the test phase, we utilize another 2, 000 samples whose input variables are sampled from U33 4). The symbolic activation pools are (2) Power system dataset. Power flow equation determines the operations of electric systems [37]. For node i in an M- node system, the equation can be written as P: = EMM Gim (umum + v.um) + Bim (1:um - u,um) B2. (v,2 U,Un)), where u; and v; are the real and imaginary components of the voltage phasor at node i. P; and gi are the active and reactive power at node i. Gin. and Bim represent the physical parameters of line im. If line im does not exist, Gim = Bin O. Therefore, we can treat x = [M1, ".. UM, ] and = [Pi, [P1.91, to learn the underlying system topology and parameters, which has broad impacts on the power domains [38]. In this experiment, we implement simulation from a 5-node system using MATPOWER [39] and two year's 's hourly data. The first 8, 760 points are used for training while the remaining samples used for testing. The symbolic activation pool is {z). We denote this dataset as
x2 cos(z)) and {vz, 2, x², log(z)) sin(x)} } for Syn, and Synz. respectively.
2²,
and q: m=1
M
, Pas, qaх] ] The target is

---

Pow. (3) Mass-damper system dataset. Equations of the mass damper system can be written as: -DDDD' M 9. where 9 is vector of momenta, D is the incidence matrix of the system, R the diagonal matrix of the damping coefficients for each line of the system, and M is the diagonal matrix of each node mass of the system. Thus, we can set y = and æ q and the goal is to learn the parameter matrix -DRD' M We conduct the simulation via MATLAB for 10-node system and obtain 6, 000 points for 1min simulation with a step size to be 0.01s. The first 3, 000 samples are used for training while the rest samples are used for testing. The symbolic activation pool is (x). Then, we denote the dataset as Mas.
Benchmark methods. The following benchmark methods are utilized. (1) Deep Symbolic Regression (DSR) [3]. DSR develops an RNN-based framework search the expression tree. Especially, the risk-seeking policy gradient is utilized to seek the best performance. Then, BFGS [11] can solve the non-linear optimization and estimate the symbol coefficients.
(2) Vanilla Policy Gradient (VPG) [3]. VPG is vanilla version of DSR with normal policy gradient rather than the risk-seeking method. (3) Equation Learner (EQL) 16. 7]. EQL creates an end-to- tnd NN to select symbols and estimate the coefficients. The sparse regularization is enforced for the NN weights to search symbols. (4) Multilayer Perceptron (MIP). We also employ a standard ML.P to learn the regression from æ to y. We only evaluate the extrapolation capacity of MLP in the test dataset. For DSR, VPG, and EQL. methods, based on the input datasets, we adjust the symbol and operator library to enable the same searching space CONSOLE for fair comparisons. We run the benchmark methods 5 times with different random seeds and present their best results. As for our method, we only run I time and obtain good results due to the convex design and the e-greedy strategy.
Metries for evaluation. We employ the following metrics. (1) Average coefficient estimation percent- age enror E., For an equation with H symbols, We calculate the error as E. = /// Σn PE(WA, wy)), where wh and WA represents the true and the estimated coefficients for the ht symbol, respectively. PE is the operation to calculate the percentage error. there no matched symbol for the th true symbol, we denote PE(W,, wn) 100%. Note that when calculating Ec, proper simplifications may be needed. For example, cos (2.(vvZ) = cos(2.5zz). (2) NRMSE in the test dataset. We measure the extrapolation capacity in the test dataset and utilize NRMSE employed in Section 3.2. Finally, the
settings can be seen in Appendix A.7.
(0.0), (O. 1), (1.0), (1.177
-Qs
t= 5, 5.-
10, ophmal -Q"
Exucl equation
X
(o5(w)))))
(0,0)
(1,0)
(1.1)
LoCal for the Toy example.
(b) Updating of convex Q
Locally and strictly conves regions exist
0.25 L
ttink
0.2
4090
2090
0.15
0.1
M/2
W
0.05
Zero loss at a wide region
[-3, (]7]
Aokk
A
We
-10
-5
5
10
(c) Loss surface with local convexity.
(d) Training loss for different initializations.
Figure 2: Illustrations of convex mechanisms using a toy example.
5.2 Verification and 3-D Visualization of Convex Mechanisms
We first utilize a toy example to verify the benefits of convex designs for two sub problems. Specifically, we consider to learn y1 = 3z cos(2.5x22) with the loss function L = Σ. [y wr [1]² cos(u (2))² where the data sample notations are defined in Section 3.1. As shown in Fig. 2a, we design 4-layer LOCAL with two learnable parameters wi and w2 to represent the coefficients. Thus, in the search phase, the goal is to identify the action a, We plot Q.)(.) when I == 1, 10 in Fig. 2b. The convexity always exists so that the algorithm can quickly find the global optimal solutions in red dots. Next, the agent update Q values with true rewards, the Q-function
7

---

converges to the optimal function within 10 episodes. Finally, the convex - (-) remains unchanged and can bring the optimal action and the true equation, which supports Theorem
Subsequently, we plot the loss surface of I. in Fig. 2c. We find that around the two global optima (3, 2.5) and (3, 2.5),, there are convex regions that have an approximate quadratic shape. To further quantify the range for proper initialization, we vary the initial weight wo E {-10, -9, ... 10 for wi and w2 in LOCAL. Fig. 2 reports the final training loss with respect to different wo. and we find the safe range for initialization is [-3,7 7] \ {0). This range is relatively large when compared to the optimal values. These observations support our Theorems 3 and 4. Finally, wg does not work 81.
since Darg l=2=0
0 always holds, which prevents the weight updating using gradient methods.
Table I: The learned equations for Syn, and Syn..
CoNSoLE
DSR
91 fzf cos (2.5mg)
91 = 0.905 + 3.88 cos (2.48m2) + 1.742f (1s(11987))
Syn;
M2 3.999xrass
3/2 4.022,
y8 = 3u3 (0.005e))
2/3
3x3
M1 1.223/2122 + 0.1812, log(za) + 0.925m;
1 /.48/rriz 1.00.14
92 = sim (1 1.633.z (2992.6)))))
Syn2
V/ sin (1. ( log (2.999x²) + 0.999.
/Ta)
0..874 sim si7. /23
Vo
1. 1.933. log((15082)) + 1.002ef
1/3 2.081 /rz32 + 1.045.
VPG
EQI.
11 0.3642f cos 56rzg)
11 0.23rr; 0.021a3 0.283
Syn;
+ 4.707>>
0.85-Lr (0s(1.982))
1/2
0.03x1.43 + 0.488r;
1/
3 3.293.z + 0.554 cos(2 (2. 82mg)
+0.045x5 0.62g
1/3 3xg
V/ 0.3666r 0.03zr + 0.4523
1/ 1.462 log Jz² 0. 2
91 0.44. 0.2z1 0.14 0.452122
3/2 = sin(1. (1.220z)) log()))2233333. (3.02zz))
+0.51.2 + 0.24a + 0.5522 + 0.705
Syn,
sin )z2 + 0.567 sin (1.56mg)
1/2 = 0.018 + 0.012 0.0636
1/ 2.081 Fuat + 1.045z7
33 0.38332 + 0.357 + 0.31 + 0.487
5.3 Convexity Guarantees of CONSOLE to Learn Correct Equations
In this subsection, we report the results of the equation learning. First, we list the learned equations for Syn, and Syn, in Table 1. Table presents that CoNSoLE has the best performance in most of the equations while DSR ranks second. In particular, CoNSOLE can accurately learn all equations in Syn, and Syn,. The superior performance is mostly due the convex design of the search and the coefficient estimation process with provable guarantees. In addition, we observe that for the result of CONSOLE in learning 1/3 of Syn,, we have 3z3 cos c0s(0.05zz))
3xz.. This shows that there is possibility that the search result of CONSOLE might not be optimal (i.c., an extra consine term exists but close to 1), but the learned equation still highly accurate. Such an observation nonetheless guides further study of the coupling relationship between the search and the estimation procedures.
7,090
NINSE
12
04
04
10
9ym:
Powe
Syn:
Powe
(a) The average percentage error (%)% of coefficients.
(b) NRMSE of test datasets.
Figure 3: Results of equation learning for different methods and datasets.
DSR doesn't 't perform well when the underlying equation is relatively complex, e.g., 8/2 in Syn, and yr and y/2 in Syn,. This is because DSR may still fall into local optimal solution despite risk-seeking policy design. VPG method performs worse than DSR since VPG considers an expected reward [3]. Finally, EQL, performs the worst as it merges the symbol search and the coefficient estimation in one NN model, which provides few guarantees of accurate learning. The above observations and analyses
8

---

are consistent with the result of coefficient estimation errors and prediction errors in Fig. 3a and 3b. For the Pow and Mas. CoNSoLE doesn't learn completely exact equations within T 600 episodes. This is because they have large number of variables to be considered. However, CoNSoL.R is still better than other methods.
5.4 Ablation Study: Exploration and Convex Search are Essential
We conduct an ablation study further understand what factors are important in the CoNSOLE. We test the result with Syn, and Syn, and report the (%) values. Specifically, we investigate the following cases. (1) No ablation. (2) Drop exploration in deep Q-learning. We delete the e-greedy strategy. (3) Drop double-convex deep-Q learning. We replace this design with traditional deep-Q learning. (4) Drop coefficient estimation using LOCAL. After learning the structure of LOCAL, we reformulate a non-linear optimization and utilize BFGS [11] in DSR, instead of gradient descent in LOCAL, to estimate the coefficient.
(5) Drop static symbolic constraint. (6) Drop dynamic symbolic constraint. These two constraints are mentioned in Section 3.2. Then, Fig. 4 shows that cases (2) and (3) cause large errors. For case (2), if no exploration strategy is added, the updating of the Q-function and the reward function is slow. For case (3). the non-convex search induces many sub-optimal actions in the search process. Thus, these two cases cause a slow search process and significant errors after = 600 episodes. For symbolic constraints in (5) and (6), removing them increases the error for Syn, and Syn, respectively. This shows these constraints are beneficial to the search process. Finally, we find that utilizing BFGS in (4) can bring good results with initialization in the locally convex region. Since the non- non-linear optimization has the same loss surface as LOCAL, the locally convex region in Theorems 3 and 4 can prove the good performance of BFGS.
E.(%)
E, (%)
-ym
Wio dyuamic const.
Wio static cons.
Syn;
Wio exploratio
No whlonun
0.5
demaiie
Wio
Wio LoCal
Syn,
Wio comes scurch
0
oolution
(2(%)
80
90
100
110
120
10000
1500
2000
2500
SNR
Data volume
0
10
20
30
40
(a) (%) w.r.t. SNRs.
(b) (%)% w.r.t data volumes.
Figure 4: (%) of ablation study.
Figure 5: (%)% of sensitivity analysis.
5.5 CONSOLE is Robust with Changing Noise Levels and Data Volume
We utilize Syn, and Syn, to examine the robustness of the framework with changing noise levels and data volumes. For the noise level, we consider the Signal-to-Noise Ratio (SNR) such that SNR E {80, 90, 100, 110, 120]. For the data volume, we fix SNR = 100 and vary N E (500, 1000, 1500, 2000, 2500]. Fig. 5a and 5b demonstrate the results. We find that when SNR 2 100, the error can be less than 1%. This noise level is suitable to real-word systems. For example, SNR 125 for electric measurements [38]. For the data volume, the overall error is less than 1.5% when N 2 500, which shows a robust performance of CONSOLE.
6 Conclusions and Future Work
In this paper, we propose CONSOLE, a novel Convex Neural Symbolic Learning method, which enjoys convexity with certain conditions to tackle Symbolic Regression with guaranteed performances. Specifically, we convexify the search problem by proposing a double convex deep-Q learning. In the meantime, we prove the local and strict convexity of the coefficient estimation in our Locally-Conver equation Learner (LoCaL). To our best knowledge. CONSOLE is the first method that provides guarantees with reasonable assumptions to learn exact equations. Besides, CONSOLE EE has, at a minimum, a broader impact on the following domains. (1) Convex control for physical systems using Reinforcement Learning. (2) Neural networks in engineering applications with local convexity.
Acknowledgments and Disclosure of Funding
This work is partially supported by NSF (1947135, 2134079, 1939725, ECCS-1810533 and ECCS- 2048288), DARPA (HR001 121C0165), ARO (W911NF21 1008),, DOE (DF-AR00001855 1631 and DE- DE-EE0009355) and AFOSR 2A9550-22-1-022


Reviews:
[{"review_id": "wVJhoEv3zw2", "reviewer": "Reviewer_faoT", "summary": "The authors propose a two-stage symbolic learning approach, where each stage is solvable with convex optimization. They then provide experiments validating their proposed approach.", "questions": "N/A", "limitations": "Yes", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- The use of an ICNN for modeling the negative reward function is quite clever.\n- The experimental evaluation seems quite robust. The results compared to competing methods seem very strong. Note that this is not exactly my field, so I am not completely aware if these are good benchmarks to evaluate against. \n- The ablations on SNR and different components of CoNSoLe are great to see, especially because the theoretical results only analyze the noiseless case. \n\nWeaknesses\n- In the notations section (line 114) it is stated that $h_K = y$, which is not exactly correct. $h_K$ *regresses* or is in some way a *prediction* of $y$, but to say that the output of the network exactly is equal to the labels is not quite precise. \n- For global optima in neural networks, some background is lacking. For example, [1] was the first to demonstrate that two-layer ReLU neural networks can actually be expressed as convex optimization problems (over the weights), and thus can be solved to their global optimum.\n\n[1] \"Neural networks are convex regularizers\", Pilanci & Ergen, 2020", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "lHhbXEk1IuY", "reviewer": "Reviewer_AUni", "summary": "Authors propose to learn coefficients and symbols connections in the problem of Symbolic Regression (SR) by combining Q learning and neural networks. They rely on the convexity properties of of ICNN to ensure that the Q function and the rewards R() are convex wrt the state S and the action A of the MDP defined.\n\nThey use the convexity properties and the determinism of the MDP to derive several guarantees on their method - namely, that the exact symbols can be recovered for noiseless datasets because the problem is locally convex in a neighborhood of the true minimum.\n\n", "questions": "Could you clarify how W* is found ? By minimizing MSE with SGD over some randomly initialized W ? \n\nl 202 : what \"exact equations\" mean ? In general there might be no unicity of the equations, so that true equation and the one returned by CONSOLE might be different. \n\nCould you clarify what is the MDP exactly ? My whole understanding might be wrong ; but currently I feel like there is a recursive argument : a not well defined MDP is used to define a Q function (that should not exist) and a return (sum of the rewards received an the end of an episode - not markovian becasue states are too small and too poor to encode the equation being built), that is transformed into a convex reward R() given a ICNN surrogate, which is itself (using Theorem 1) used to explain that the aforementioned MDP is well defined (?) and that its Q function is convex.\n\nProof of theorem 4 (in appendix): authors uses the trick \"x = s exp(log (|x| ))\" with s the sign of x. This is always dangerous : s is not a constant, it it depends of x so one should write s(x) instead. Authors proceed with the proof for a fixed (chosen in advance) x_i until line 578 : then they introduce the objetcs y' and y'' which are defined for x_1..x_N. Since the notation s(x) is not used, it is not obvious (by reading the proof) that the differents values of s have been taken into account. I am not saying it is false, but it is worth explaining in the proof whether or not this is correct. This trick is sometimes fishy because of that.  \n\n", "limitations": "With my current understanding of the paper, the whole theoretical framework used in the paper is invalid, which seriously impedes its impact. Moreover the experiments seem to be very small, specially when there is only linear {x} symbol pools.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- i) solving complex combinatorial problem with RL is always an interesting approach\n- ii) using ICNN to enforce good properties (and particular that the minimum is uniquely defined and easy to find) is a good take-home message from the paper\n- iii) the experimental part contains numerous tasks and comparisons against other SOTA approaches, with figures and examples\n- iv) experimental results are convinving about the eficiency of the method - even though I am not familiar with the field\n\nWeaknesses:\n\nmajor issues:\n\nThe whole approach consists in using Reinforcement Learning (RL), and more specifically Q-learning (and its neural network variants). Howerver RL requires to work on a Markov Decision Process (MDP), the Markov property must hold : P(s_2 | s_1, a_1) = P( s_2 | s_1, a_1, s_0, a_0). However it seems that the object defined here is not a proper MDP. Indeed, since the state s belongs to {0, 1} ^n_k we only have access to the activations of the layer k : no information about the previous activations, not mentioning the input x itself. Here, the state s has no knowledge of how \"deep\" we are in the equation being built. It feels weird : how a neural network can take the right actions if it does not \"see\" the whole input ? Moreover, in MDP, the reward depends of the current state s and action a, or of the current state s and future state s' (depending of the literature). This is not fulfilled by the construction given by the authors in either case.\n\nIf the problem is not a MDP, the Q function has no reason to exist, and fixed point based algorithms like value iteration (and variants such as Deep Q learning) are not guaranteed to converge. Even if the optimal Q function exists, it has no reason to be convex in general. This is not a property to be enforced with ICNN, it is a property of the MDP itself, that must be defined as such (Markov property must be fufilled !).\n\nAs said by authors in literature review, the problem of SR is NP hard. Hence, an attempt to convexify the problem that would yield polynomial time algorithm would fail. So either the problem cannot be cheaply convexified, either it induces a bias that prevents to find the global optimum of the original problem.\n\nEven if it was a MDP, there are other issues. For example, the complexity of value iteration is O(S² * A) with A and S the action and space states respectively. For S defined by the authors, there is only 2^n_s states and 2^(n_s * n_s) actions. The total complexity of value iteration would be O( 2^(n_s  * n_s)). How does it compare to Deep Q learning in sample efficiency ? For dataset (1) the pool of activations is small, for (2) and (3) the pool of activations is only a singleton (!) the number of variables in input is also very small for (1) : only 3, while for (2) it is 5 nodes and for (3) it is 10 nodes. On those toy examples, the search space is so small that value iteration seems sufficient, no need for Deep Q learning\n\nTheorem 1 is false. Since it is not a MDP, Q function is not defined. There is no reward in this MDP, only a convex surrogate function R() paramatrized by ICNN, that looks like the reward R received at each time step. But in reality, this R is not a Markvovian reward (that depends of state/action pair), it is a surrogate that depends of the final return of the trajectory. \n\nForcing the surrogate R and the network Q to be convex using ICNN induces a bias, but can not be used to prove anything about the underlying MDP. \n\nl192 : \"Then, we 192 prove that the convexity of the negative optimal Q-function eventually yields the exact equations.\"\nWell, then, why is there a non zero error in experiments Fig. 3 ? Such error is not negligible.\n\nminor issues:\n\ni) It it not clear what neural networks bring here. In general their power in combinatorics is their generalization capabilities - i.e their capacity to interpolate the value of unseen inputs. Paper lack details about this precise contribution of NN.  \n\nii) Theorem 4 is very ad-hoc due to its complicated form. i,j and k variables are not defined. Even the appendix does not contain a clear statement o the theroem, only the proof. Hence, it is hard to understand what the objets clearly are. \n\nl 202 : what \"exact equations\" mean ? In general there might be no unicity of the equations, so that true equation and the one returned by the network might be different. \n\niii) theorem 2 is assumes that the optimal W* is recovered (why ? How ?) and that the sequence of actions is selected according to the optimal policy Q*, assuming it exists, such that the optimum of Q* corresponds to exact equations (using the fact that the dataset is noiseless). Authors conclude the exacts equations can be recovered. This is essentially trivial : authors are concluding that under the hypothesis that the optimum exists and has been found, it has beend found... \n\niv) theorem 3: it seems that authors are saying that in the neighborhood of the minimum, if for one of the examples there is no plateau (assumption 2), the second orer derivative (assuming it exists, assumption 1) is positive. For unconstrained minimization of twice differentiable function, this is a well known property: https://math.stackexchange.com/a/19473 (second paragraph)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "eB7FyhlcH7m", "reviewer": "Reviewer_EjER", "summary": "The authors propose a method for symbolic regression based on deep Q-learning, that learns to chose which symbols to combine in the LoCal model. To provide formal guarantees, the authors use input convex neural networks to model both the Q-function and the Reward function. Experiments suggest that the method is able to accurately recover true underlying equations from data.", "questions": "Initially I was puzzled as to why the authors were so keen on maintaining convexity in R and Q for the deep Q-learning procedure. Is it just such the optimal action is uniquely defined and there's an algorithm to find it? Or does it help with the large state and action space?\n\nWhen R is convex, the optimal Q is convex. But how reasonable is it to model R as a convex function?\n", "limitations": "The authors have adequately addressed the limitations of their work.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength\n\n- Elegant use of input convex neural networks for Deep Q-learning\n- Theorem 1 shows that modeling Q as a convex function is a consequence of modeling R as a convex function\n- Theorem 3 shows that the loss landscape of chosing optimal multiplicative weights in the model has local convex region under mild assumptions. Hence we can find it with gradient descent methods if the initialization is lucky.\n- Strong experimental evaluation on synthetic problems as well as systems from physics and electrical engineering. Several benchmark methods are considered, the ableation study highlights the importance of each component of the contribution\n- Figure 1 is very informative.\n\nWeaknesses:\n\n-The authors claim  that \"Q-learning can hardly be applicable when the state and action spaces are large, e.g., SR problem.\" but then proceed to do so anyways?\n- Code is not available to reviewers, so I cannot confidently attest to reproducibility\n\nMinor Comments\n\n- There are some redundancies between Introduction and Related Work.\n- Table 1 violates the margins\n- Authors use references [x] as subject in sentences. This should generally be avoided, use author names as subject instead.\n- Personally I'd like to have Algorithm~1 in the main text. I know space is tight, but the code helps a lot for understanding the interplay of all the components you propose\n\n\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Gz6dQcjXcws", "reviewer": "Reviewer_ipq6", "summary": "This paper considers the problem of symbolic learning using neural networks. The paper is focused on designing symbolic search algorithms with correctness guarantees. This is achieved by using a $Q$-learning framework, where the $Q$ function is modeled using an input convex neural network, ensuring exact temporal difference updates can be computed efficiently. The authors provide theoretical guarantees for the correctness of their algorithm, modulo a requirement to be able to fix neural network parameters exactly (a non-convex problem). This is balanced, however, by a follow up result showing that globally optimal points always lie in (strictly) convex neighborhoods. The method is tested on equation learning tasks, performing favorably.", "questions": "- Why are you allowed to restrict the negative reward function to be convex? Of course it is permissible to model the $Q$ function, but rewards usually come from a ground truth. \n- Is  $Q(s, a)$ differentiable in $a$? It appears that for non-integral $a$, $Q$ is computed using a thresholding at $0.5$. This seems to me to introduce discontinuities. If so, why does this not make optimizing $\\min_a Q(s,a)$ difficult?", "limitations": "Yes.", "rating": 6, "confidence": 2, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Thanks to the authors for an interesting work on the salient topic of neural-symbolic learning. I enjoyed reading the paper, and certainly felt like I learned something. Some \n\n- The convexity results are really nice, and well supported by the ablations. It is worth noting of course that the overall procedure is not fully convex (which it cannot be due to NP-hardness results) but the non-convexity comes from needing to fit  neural network parameter, arguably a fairly benign form of non-convexity. \n- Experiments cover a number of different tasks, and appear thorough. Results seem promising, for instance the proposed method is considerably closer to the ground truth equations in Table 1 than the other methods. \n\nStepping back a moment, the paper is well organized, although I found it quite information dense. The paper went through the important steps to make the work properly complete, including good, simple theoretical results, and an experiment section with comparisons to alternate methods, and ablations. The ablation section provides useful insights. For instance, the double-convexity of the $Q$ function is doing a lot of heavy lifting—removing it significantly impacts performance. This is encouraging, suggesting that the input convex network design is working well.\n\nIn all, this paper presents a method for neural-symbolic learning that appears non-trivial and effective. Importantly, it is accompanied by correctness guarantees for (large parts of) the learning algorithm. This is an especially promising sign in the context of symbolic learning, where finding exact solutions is of comparatively greater importance than in other areas. Overall I am broadly positive about this work. A disclaimer, however, is that I am not familiar with recent related works in this area. This is reason for the given confidence score, and will pay careful attention to any related works that may emerge during discussions. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept