Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description length
Authors: Zihan Yu
Jingtao Ding
Yong Li
Depeng Jin
Venue: ICLR 2025 Poster (2025)
Keywords: symbolic regression
minimum description length
trans former
neural network guided search
Primary area: neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
TL;DR: A new approach to conduct symbolic regression with minimum description legnth as the search objective

Abstract: Symbolic regression, a task discovering the formula best fitting the given data, is typically based on the heuristical search. These methods usually update candidate formulas to obtain new ones with lower prediction errors iteratively. However, since formulas with similar function shapes may have completely different symbolic forms, the prediction error does not decrease monotonously as the search approaches the target formula, causing the low recovery rate of existing methods. To solve this problem, we propose a novel search objective based on the minimum description length, which reflects the distance from the target and decreases monotonically as the search approaches the correct form of the target formula. To estimate the minimum description length of any input data, we design a neural network, MDLformer, which enables robust and scalable estimation through large-scale training. With the MDLformer's output as the search objective, we implement a symbolic regression method, SR4MDL, that can effectively recover the correct mathematical form of the formula. Extensive experiments illustrate its excellent performance in recovering formulas from data. Our method successfully recovers around 50 formulas across two benchmark datasets comprising 133 problems, outperforming state-of-the-art methods by 43.92%. Experiments on 122 unseen black-box problems further demonstrate its generalization performance. We release our code at https://github.com/tsinghua-fib-lab/SR4MDL .

Published as a conference paper at ICLR 2025
SYMBOLIC REGRESSION VIA MDLFORMER-GUIDED SEARCH: FROM MINIMIZING PREDICTION ERROR TO MINIMIZING DESCRIPTION LENGTH
Zihan Yu
Jingtao Ding"
Department of Electronic Engineering, BNRist Tsinghua University Beijing, China
Department of Electronic Engineering, BNRist Tsinghua University Beijing, China
Yong Li'
Depeng Jin
Department of Electronic Engineering, BNRist Tsinghua University Beijing, China
Department of Electronic Engineering, BNRist Tsinghua University Beijing, China
ABSTRACT
Symbolic regression, task discovering the formula best fitting the given data, is typically based on the heuristical search. These methods usually update candi- date formulas to obtain new ones with lower prediction errors iteratively. How- ever, since formulas with similar function shapes may have completely differ- symbolic forms, the prediction error does not decrease monotonously the search approaches the target formula, causing the low recovery rate of existing methods. To solve this problem, we propose a novel search objective based on the minimum description length, which reflects the distance from the target and decreases monotonically as the search approaches the correct form of the target formula. To estimate the minimum description length of any input data, we de- sign a neural network, MDLformer. which enables robust and scalable estima- tion through large-scale training. With the MDLformer's output as the search objective, we implement a symbolic regression method, SR4MDL, that can ef- fectively recover the correct mathematical form of the formula. Extensive periments illustrate its excellent performance recovering formulas from data. Our method successfully recovers around 50 formulas across two benchmark datasets comprising 133 problems, outperforming state-of-the-art methods by 43.92%. Experiments on 122 unseen black-box problems further demonstrate its generalization performance. We release our code at wotthtts//// httpps//github... com/ tsinghua fib- lab/ /R4MDDL
INTRODUCTION
Symbolic regression (SR) is task that uncovers interpretable mathematical formulas to describe the underlying relationships within observational data, which is widely used for promoting scientific discovery facilitating the modeling of diverse phenomena in many fields, such as dynamical
systems (Quade et al.) 2016 Chen et al. 2019: Cornelio et al.) 2023 Angelis et al.) 2023; Ding et al. 2024), materials science (Wang et al. 2019 Schmelzer et al. 2020, Weng et al. 2020 Sun
et al. 2019). etc. (Liu et al., 2024; Neumann et al. 2020 Shi et al. 2022). Formally. SR aims at finding symbolic function f from the given data (x. V), where
21, 22 .... Øpl E R NxD and
y E N are observed N samples points of independent and dependent variables. The discovered formulas consist of mathematical symbols like +, -, x, and +, whose specific form can provide corresponding insights into the patterns behind the data. To find the formula that best fits the data in the symbolic space, the most typical methods SR based on heuristic search, in particular, the genetic programming (GP) algorithm (Augusto & Barbosa 2000), which executes evolution
"Corresponiigg author. Email: dingit taangit15eee www.eeeettiinnnnnooeeeeooo www.nnghhaa.rrg.oooo cn, liyong0 78 retsinghua edu. cn

---

Published as a conference paper at ICLR 2025
iteratively to enhance the fit of candidate formulas to the given data. There is now large body
of commercial and open-source software (Dubčáková 2011 Stephens
2016
Cranmer
(023). as
well many influential works (Schmelzer et al. 2020 Weng al.) 2020 Liu et al. 2024). that
developed based on the heuristic search-based SR methods.
However, while existing SR methods can identify formulas with high accuracy (R² 0.99 for over 90% of cases in SRbench (Cavalab 2022), ). their effectiveness in discovering the optimal formula with the lowest prediction error limited (only around 20% success rate), even in noise-free data. This is because these methods optimize the candidate formulas' prediction errors (Makke & Chawla 2024]. which does not lead to the target formula with the minimum prediction error. As illustrated in Figure I] the mean squared eπтor of a candidate formula (f+) does not decrease monotonically when its form approaches the target one. This is because formulas with similar symbolic structures can exhibit rather different functional shapes in the numerical space. On the other hand, two formulas with similar functional shapes can have entirely different symbolic forms. This indicates that the SR task lacks an optimal substructure (Cormen et al. 2022), that is, the target formula with the minimum prediction error cannot be achieved by simply making small adjustments to formulas with sub-optimal prediction errors. This high nonlinearity of the relationship between formula's ' mathematical form and its functional shape results in a divergence between the direction of reducing prediction error and the direction toward the target formula. This complexity makes it difficult for existing methods to identify the formula with correct mathematical forms.
To solve this problem, in this work we proposed new search objective inspired by the minimum description length (MDL) (Kolmogorov 1963), which represents the size of the simplest model used describe the data, or, in SR, the minimum number of symbols required for the target formula fxx) If each mathematical symbol is regarded as a transformation step, then MDL describes the number of transformations needed to go from the independent variable, 2, to the dependent variable, y. Therefore, the search with minimal MDL as the optimization objective has an optimal substructure (Cormen et al. 2022): only when the correct transformation is executed can the MDI. reduce, thus the search direction of MDL. reduction is always consistent with the direction leading to the target formula (see Figure D. However, MDL has been shown to be incomputable (Vitányi, 2020), indicating that no algorithm can provide an accurate MDL for arbitrary input. Neverthe- less, the capacity of neural networks as universal approximators (Nielsen 2015) indicates that, with sufficient data, a suitably designed neural network effectively learn the complex mapping from the data to its MDL. To this end, we developed a Transformer-bass neural network, MDLffomer. Through large-scale training on over 130 million symbolic-numeric pairs using carefully designed training strategy that aligns the numerical space with the symbolic space, MDLformer has gained the ability estimate the MDL of any given data. With the search objective provided by the large- scale trained MDLformer, we implement a new SR method based on the Monte Carlo tree search algorithm (Browne al. 2012), a heuristic search algorithm that has been proven to be suitable for use in conjunction with neural networks successfully by many works (Silver et al.) 2016 2017 Kamienny et al. 2023).
We conduct extensive experiments to illustrate the excellent performance of our approach in recov- ering formulas from data. Across two problem sets with 133 formulas, our method successfully recovers around 50 of them, outperforming state-of-the-art methods by 43.92%. We also find this result robust to noise: even if we add noise with an intensity of 10% to the data, the recovery rate of our method is still higher than the recovery of other methods in the absence of noise. We also test our method on the black-box problem sets, finding it can discover formulas that describe the data with lower description length and higher accuracy than other methods. Further analysis of the MDLformer demonstrates its scalable and robust capability for accurate predictions of the MDL, which explains the outstanding performance of our SR method.
2 RELATED WORK
Heuristic search methods. Traditionally, symbolic regression approaches are mainly based on the genetic programming (GP) algorithmssLangdon & Poli 2013). which maintains a set of symbolic formulas and iteratively updates these candidate formulas via mutation and crossover operations. To
this day, there have been plenty of GD-based symbolic regression toolkits developed, such as Eureqa [Dubčáková] 20011) GPlearn (Stephens) 2016). PySR (Cranmer 2023). and
(Schmidt & Lip-
son 2210 de Franca & Aldeia| 2021 Cava et al. 2016 Arnldo et al. 2014 Virgolin et al. 2019
2021 Burlacu al. 2020
Zhong al.] 2018 Zhang et al. 2022 Augusto & Barbosa 2000 Kartel
2

---

Published as a conference paper at ICLR 2025
Observed Data
/2 (2x)
/3 sin(2x) ed 4 sin(2x) (2(22x))(((x))
1)
Discovered F Formula
Cly - 0,06.00
12
MSE Ily- A02P
MSB
0.5
y - f'(x)
0.0
2.5
5.0
f1
Figure 1: Comparison of the two search objectives. In the searching route leading to the target, f the prediction error (measured by the mean square error, MSE) does not decrease monotonically as the candidate formula's form gets closer to the target one, whereas the minimum description length (MDL) does. Here, Φ denotes the function f oi (2. fi) and C(o,, is its complexity.
& Djukanovic, 2023 Smits Kotanchek 2005 Searson al., 2010). In these years, people have begun to use reinforcement learning algorithms for symbolic regression, including Monte-Cate Carlo tree search (MCTS) (Sun et al. 2022), double Q-learning (Xu et al.) 2024), and deep reinforcement learning (DRL) (Petersen et al. 1021 Tenachi et al. 2023). These methods start from the empty formula and iteratively select appropriate mathematical symbols to fill in until a complete formula is obtained. Some works combine multiple methods to achieve better symbol regression (Mundhenk) jet al. 2021 Jin et al. [020)] McConaghy 2011] Landajuela et al. 2022). These search algorithms, however, all face the problem that the prediction errors usually do not decrease monotonically from the initial state to the ground-truth formula. Although recent methods introduced formula complex- ity as regularization term alongside the prediction error (Sun al., 2022 Cranmer 2023), this problem persists and their recovery rates remain low. In contrast, our approach addresses this prob- lem by optimizing the minimum description length (MDL), which decreases monotonically along the path to the target formula.
Neural network-assister search methods. Recently, some works have attempted to enhance the search methods by utilizing the powerful fitting capabilities of neural networks. For example, Mund- henk et al. (2021) proposes to train an RNN by deep reinforcement learning to generate the initial candidate formulas in genetic algorithms for speeding up evolution. While Kamienny et al. (2023) utilizes a Transformer pre-trained as next-symbol predictor to provide promising search directions for the Monte Carlo tree search algorithm. Although these works still take prediction error as the search goal and thus do not have optimal substructure, they inspire us to combine pre-trained neu- ral networks with search algorithms. AlFeynman shows another way to combine neural networks and search algorithms (Udrescu & Tegmark] 2020; [drescu et al., 2020): it discovers symmetry properties data by fitting it with neural networks and, based on this, converts the search for the target formula into searches for several smaller subformulas. However, leveraging hand-designed symmetry properties summarized on the Feynman dataset, AlFeynman cannot be adapted to other datasets without these properties, and these rules are sensitive to noise, making AlFeynman's 's recov- ery rate decrease significantly on noisy data (Cavalab) 2022). As comparison, our method can also be regarded as a method of recursively simplifying the target function, since the decrease of MDL indicates a candidate formula ʃ as a subformula of the target one. Identifying subformulas based on MDL.former 's output rather than hand-designed rules, our method does not rely on specific prior knowledge and is more robust to noise.
Regression and generative methods using neural networks. In addition to using neural networks to assist search, some works proposed neural network-based regression methods. They design fully connected neural networks with mathematical functions, such as sin, exp, X, as activation layers. After fitting weight parameters to the input data, they can extract mathematical formulas from the networks Kim et al.] (2020): La Cava et al.] (2019): Kubalik et al. (2023). However, due to us- ing mathematical functions like exponential and logarithmic functions as activation layers, these methods usually face numerical instability problems such as gradient explosion. Other works are based on generative methods. They use large-scale pre-trained Transformers to generate symbolic sequences as target formulas from the data directly.
(oaellyly.Bii (Biggio et al. 2021 Kamienny et al.) 2022 Mci- dani et al. 2023 Bendinelli al.) 2023 d Ascolli al.) 2022). ). Once pre-trained, these methods can directly generate target formulas without searching or extra training. Therefore, they are usually faster than other methods. However, it is still difficult for them to obtain the target formula with the correct form, since small changes in the input data can lead to completely different objective functions (Kamienny et al. 2023). As a comparison, based on the search method, our approach can improve the results through extended rumning durations, and the utilization of the MDL search objective enhances its efficiency, allowing it to achieve a high recovery rate for the target formula
3

---

Published as a conference paper at ICLR 2025
within reasonable timeframe. Many recent studies focus neural symbolic reasoning. i..., using neural networks for symbolic reasoning van Kricken et al. 2023 Despite their similar names, this task is different from symbolic regression, which we detailed discussed in Appendix
LEARNING TO ESTIMATE FORMULA COMPLEXITY WITH MDLFORMER
3.1
MDLFORMER
MDLformer
000R
Tokenize
Embedder
nccdee
Allention P Pooling
MLP
Ma
((t)
Apredt
Tokenize
Embedder
Encoder
Attention P Pooling
random
cifl
Figure 2: Schematic diagram of the architecture and learning process of MDLformer.
The design of MDLformer is rooted in the recent development of Transformer models, which have been proven be able to encode numeric observation data into latent space, which can be further used for inferring corresponding symbolic formulas (Kamienny et al. 2022). Meidani et al. (2023) further find that, by aligning the latent spaces of two Transformer models that encode numeric data and symbolic formulas, respectively, it's possible to predict cross-model properties. Here we exploit this approach cross-modally predict the length of the corresponding symbolic formula, i.e., the minimum description length (MDL), based on numeric observational data,
As depicted in Figure 2] the MDLformer, Ma. integrates an embedder, a Transformer encoder, an attention- based pooling, and a multilayer perceptron (MLP) as readout head, to map the input data E N and y E R R.Nx into its MDI., C[J), where f denotes the simplest symbolic function that describes y = f(x) and C[J) is its complexity, i.e., the number of symbols required to express f- Embedding. We first pad x into ND with zeros since the number of features D can vary. We then tokenize the input data using base-10 poating-poinn notation. Specifically, value to significant digits and then split it into three parts: sign (),-), mantissa (0.000- ~ 9.999), and exponent (E-100
round each
E+100). For example, value 54.321 is represented as sequence [+, 5. 43,,,,2 , E+1]. The input data is tokemized into N (D + 1) x tokens, which, increases with and D max. challenges the quadratic complexity of Transformers. Therefore, we sample no more than Nmax pairs from each row of x and y and then embed each x-y pair into the latent space with fixed dimensionality d to feed into the Transformer encoder.
Encoding. We leverage Transformer encoder (Vaswani al., 2017) to process the embedded numeric input. Notably, we remove the positional encoding in the numeric encoder since each item in the input sequence represents pair of x-y data and their order is thus not important, which aligns with the previous practices (Biggio et al. 2021 Kamienny et al.
2002 Meidani et al., 2023).
Pooling. To map the Transformer encoder's outputs, E R xxxd into a fixed-size represen- tation, we adopt an attention- based pooling mechanism (Santos et al. 2016). Specifically, we use a learnable weight, w E Rdd,, to calculate an attention weight for each row v; E V by a, softmax(w v;), where the softmax is conducted along the sequence dimension N. Then, add them up with a, as weights to get the final representation: n =E a,3;
Reading out. After pooling we obtain a compact representation for the whole numeric input, which can be used to predict mroel properties (Meidani et al. (202)). Here we predict the MDL with multilayer perceptron that uses ReLU as the activation layer.
3.2 TRAINING DATA GENERATION
The training of the MDLformer relies plenty of paired numeric and symbolic data, which is generated during the learning process on the fly. During the whole process, we generate a total of about 131 million pairs of input data.
Generate symbolic formulas. We generate symbolic formulas with the algorithm proposed by
Kamienny al. 2022 to ensure diversity: First, we sample the number of features, D
4

---

Published as a conference paper at ICLR 2025
440, ......DDuuna -mmax) }- Then, we randomly combine them with b binary operators and further sample u unary operators to insert into the random position of the resulting formula. Finally, we add non- similar transformations to each position of the formula: f. ト a;fs b, where fi are subformulas of f. a, and i; are sampled from 44[-100, 100]. To ensure that f is in its simplest form, we simplify it with sympy, open-source Python package for algehra processing. The number of operators, vari- ables, and parameters in the simplified f is its length, and therefore. is the MDL of (I, y = f(x)).
Generate numeric data. After generating the formula, we sample the number of features, D, from 4(1 -.. ) and generate numeric data for D independent variables: x E R N ~ P. Then, we dependen calculate corresponding numeric data for the dependent variable: y = f(x) E I. Nx1 Considering
that f's domain cannot be the whole RD when it contains specific operators, such as logarithm or square root, some of the sample points variable values. Therefore, we discard invalid values in y and the corresponding rows in x, ensuring each sample point of lies the domain of f. We also find that the choice of P can significantly in- fluence the performance of MDL when using for symbolic regression. We use the Gaussian mixture model (GMM) as suggested by Kamienny et al. (2022), which takes the sum of C Gaussian distributions with random mean and variance and normalizes them the distribution P. To further improve the diversity of numerical data and enhance its prediction performance in actual symbolic regression, also considered another sampling method, that is, generating it by function trans- formation on hidden variables z. Specifically, we first sample E TR-V x from a GMM distribution, and then generate random symbolic function g FRXD following the methods introduced above, and finally operate the hidden variable with the function to obtain æ g(2) E I XDD, This design comes from the fact that, when using MDLformer for symbolic regression, we need to estimate the minimal description length of the transformation results obtained by operating symbolic functions on the input data x.
may not be in its domain and lead to invalid dependent
3.3 CROSS-MODAL LEARNING FOR FORMULA COMPLEXITY
With the generated formulas f; and data (z;, y:), we train the MDLffomer using two learning objec- tives, including a primary objective that predicts the accurate MDL and an auxiliary objective that aligns the numeric latent space and the symbolic latent space, as depicted in Figure 2
Primary learning objective for prediction. To train the MDLformer for predicting the correspond- ing minimum description length (MDL) based on numerical input, we optimize the mean square error, Lpred, between MDLs estimated by the MDLformer and ground-truths:
B
Epred B
Σ
(CUA) - -Ma(;,
2=11
where denotes the batch size, C(fi] is the complexity of the symbolic function ft.
Alignment as an auxiliary learning objective. The auxiliary objective, proposed by Meidani et al. 2003)), aims to facilitate a mutual understanding of both numeric and symbolic domains and thus empower better cross- moosl prediction. Specifically, suggested by MMeidani et al. (023), introduce a symbolic encoder to map the prefix notation of f into a compact representation 8, which has structure similar the numeric encoder, depicted in Figure The latent spaces of these two encoders are aligned by optimizing a symmetric cross-entropy loss over similarity scores:
Σ log exp(n: 8;/T)
B
B
log exp(s; . n2/TT)
Lalign =
(2)
-= exp(n; . 88/T)
Expeexp(a -n,/T))
where the batch size, T the temperature parameter, s, and n. are encoded representations of i-th numeric data and symbolic function, respectively. Note that this loss is calculated per batch.
Training Strategy We adopt a two-step training strategy: First, we train the MDLffomer with the auxiliary objective and the generated formula-data pairs, where the numeric data is sampled from the GMM. ii) Then, we use the primary objective to train the MDLformer on the numeric data generated by operating symbolic functions on latent variables. The first step aligns the latent space of the numerical encoder and the symbolic encoder and thus provides a good start for the numerical encoder to predict the formula length in the symbolic space cross-most modslly. Based this, the second step uses a data distribution that is closer to the actual symbolic regression used, allowing the data encoder to estimate the description complexity of the input data accurately.
5

---

Published as a conference paper at ICLR 2025
4 MDL-GUIDED SYMBOLIC REGRESSION WITH MDLFORMER
(c,y)
Existing search methods
Our method
Inidalizz
symbolic functions (fo e Foxa
symbolic functions (fu. E Foooos
pdate
symbolic functionals Fox Toxi
symbolic functionals FAui Foxo FoxDeaL
Uiral = ffull, (FF.(
Select
select with low prediction error
select with low minimum description length
Terminate
y
(()
f" argain ly s(OII' (4f(s)
(f.B) argmin by (e(ro)))I' f" 0g /El/
Figure 3: Comparison of existing search methods and our method. E RNXD and R are observational data, Fxxq denotes all possible symbolic functions mapping from I NXP to R Nxq. i N,, denotes the indexes of candidate formulas (in total of n). while k denotes the number of loops (terminated at round K). Te. are algorithm-specinf symbolic functionals, like crossover and mutation operations in GP, whose concrete operations in different search algorithms are summarized in Table 3 dg are simple functions (like linear functions) with parameters 0 that map f into FDx1-
In this section, we explain how the trained MDLformer can help symbolic regression discover tar- get formulas with correct mathematical forms more easily. The main idea to change the search direction from the direction of prediction error reduction to the direction of minimum description length (MDL) reduction. This leads to the property of optimal substructure and makes easier for the existing search algorithms to search for the target formula with the correct form.
4.1 SEARCHING FOR MINIMUM DESCRIPTION LENGTII
As depicted in Figure existing search-basee methods focus on the update-and-seled loops over set of candidate formulas: after generating initial formula set, {fo,: FDx1). }. these methods generate algorithm-specific symbolic functionals Fi.: FDx1 + FDx1 (such as the crossover and mutation in genetic programming, see for details.) creating new candidate formulas: fK+1, (f.((f and select resulting functions with low prediction errors for the next round of iteration. This update-and and-seecct loop is repeated iteratively until formula with a low enough prediction error is encountered, at that point the loop will terminate and this formula will be read out as result: f arg minf ||y f(x)]|
In contrast, instead of maintaining a candidate set of target formulas, we maintain a candidate set of subformulas of the target formula, {Sh,i FDx D..: J. where & represents the number of iterations and represents the sample index. For candidate f. each of its item f(d) denotes part of the target, i.e., /" .( ((((1, ( ,..., ((De.)). /(e),....,,,(E..)) & reffects the transformation required from I to the target formula f", while its complexity, C(o). evaluates the "distance" between f and f Therefore, by selecting with low MDL estimated by the MDLformer, Ma(ff(), v) C(o), during the update-and-sehee thus the candidates f will get "closer" to the target one. For an fK. that sufficiently close to f after K iterations, the remaining transformation from fK,; to f" can be described by a simple parameterized function de accurately are parameters). Formally speaking, we find
loops, the remaining transformation & can be simplified iteratively, and
(f,0)
arg min ||y - 20 (f()))) se(fx.;).0
(3)
from the candidate set to determine the target formula: f = (f)
4.2 IMPLEMENTATION
In this work, we implement our method based on the Monte Carlo tree search (MCTS) Browne et al. 2012) because of its application in symbolic regression task (Sun et al.] 2022 well as its successful combination with neural networks in numerous studies (Kamienny et al. 2023 Silver
et al. 2016 2017). MCTS is classical heuristic search algorithm that maintains search tree

---

Published as a conference paper at ICLR 2025
with each node representing candidate formula and updates the tree through four steps: selection, expansion, simulation, and backpropagation. The vast majority of our implementations are consis- tent with the existing practice of using MCTS for symbolic regression (Sun et al. 2022), except that the upper confidence bound (UCB) to guide the search is based on MDLformer's 's output (see Appendix D for details). For the remaining transformation do. use the simplest linear function:
DE.
(o(Jk.)) = - 0sl d=1
(4)
where f(()) E FDx1 is the d-th item of fk... Though it is possible to use functions in other forms, such multiplications polynomials, or even brute force searches used by AlFeynman (Udrescu Tegmark] 2020), we find that even the simplest linear functions work well enough as long as MDL.frmer's estimation is accurate enough.
5 EXPERIMENTS
This section is divided into three parts. The first two parts evaluate the effectiveness of our symbolic regression method on 133 ground-truth problems and 122 black-box problems with comparison to state-of-the-art symbolic regression methods. The final part gives an in-depth discussion of the MDLformer's performance in estimating MDL.
5.1
SYMBOLIC REGRESSION ON GROUND- TRUTH PROBLEMS
Table 1:
Recovery rate and search time of different methods in both Strogatz and Feynman
datasets. Each experiment is conducted at ten random seeds and four noise levels.
Strogatz (14 problems)
Feynman (119 problems)
Type
Method
Rate f
Time (s)
R. Rate
Time (s)
Regression
FEAT
2019
0.19%
636.6
0.00%
1532
NeurSR Biggio
1.79%
15.71
2.44%
24.78
Generative
E2ESR
Kamienny
3.78%
4.044
10.40%
4.576
SNIP Meidani et
6.79%
1.457
1.60%
2.196
GPlcarn (Stophens 2016
9.21 %
966.2
16.89%
3349
AFP /Schidtt Lipson
2010
10,90%
160.7
17.51%
3845
AFP-FE Schmidr & Lipson
2010
12.86%
9532
20.80%
25138
EPLEX Cava 2016
6.02%
446.8
10.10%
11548
SBP-GP Virgolin
2.44%
20089
2.88%
28933
GP-GOMEA (Virgolin et
2021
8.46%
1100
10.32%
3456
Search-
Operon (Burlacu et
2020
4.29%
83,58
7.97%
2656
based
SPL Sun 2022
8.12%
363.7
10.48%
263.3
DSR
18.05%
784.3
18.60%
1042
RSRM (Xu et 2024)
4.43%
133.2
15.40%
127.1
AlFeynmanz Cdrescu et
2020
15.27%
241.3
27.24%
708.6
BSR [in et 2020
0.38%
25346
0.70%
30635
Ours
66.78%
338.3
33.93%
660.5
(+6.82 formulas)
(+7.96 formulas)
Experimental Settings To evaluate the capability of our method to recover formulas from data, conduct experiments on SRbench (La Cava et al.) (202]) most of the previous work does, which contains 14 Strogatz problems (Strogatz] (0188) and 119 Feynman problems (Udrescu & Tegmark 2020) with known ground-truie underlying formulas. We compare 16 baseline algorithms split into three types: the generative methods and the regression methods, both of which are based on neural networks, as well as the search methods using genetic programming or reinforcement learning. (see Appendix I. for details). For each algorithm on each problem, we test performance from 10 fixed random seeds at 4 different noise levels: 0.0, 0.001, 0.01, 0.1.
Experimental Results The experiment results are provided in Table [ where our method reaches the top recovery rate in both problem sets with reasonable search time. In the Strogatz problem

---

Published as a conference paper at ICLR 2025
set, our method improves the Recovery Rate from 15.27% to 66.17%, increasing nearly three times.
In the Feynman problem set, our method also improves the Recovery Rate by 11.94%. Overall, method recovers around 50 formulas out of 133, outperforming the best baseline (around 35 formulas by AlFeynman2) by 43.92%. We also observe that search-based algorithms outperform generative and regression methods. This could be because the latter is designed to produce formulas fitting data accurately, rather than formulas with correct forms, On the other hand, the former can improve search results by simply running for longer period, while the latter cannot Kamieeny et al. 2023), causing the former's higher recovery rates.
In Figure we also plot the recovery rate of all methods at different noise levels (see Appendix 1.2 for details). On the Strogatz problem set, our method has the highest recovery rate, significantly surpassing other methods. On the Feynman problem set, however, AlFeynman2 shows higher recovery rate than our methods when the noise level is 0.0. This is because AlFeynman2 designs series of rules based on the characteristics of formulas in the Feynman problem set help search for formulas. In contrast, as a general method with no reliance on any prior knowledge, our method can be directly applied to different problem sets. Furthermore, unlike AlFeynman, which experiences a sharp decline in performance due the failure of hand-designed rules in the presence of noise, our method is robust to noise: Even at the maximum noise level (c = 0.1), our method still outperforms other methods without noise, including the AlFeynman2 on the Strogatz dataset.
Strogatz Dataset
Feynman Dataset
FEAT
BSR
Terget Noise
Target Noise
NeurSRT
0.000
0.000
SBP.GP
0.001
0.001
EZESR
0.01
0.01
GP-GOMEA
0.1
0.1
SPI
GPleam
AFP
AFP-FE
AIFeymman2
0%
20%
40%
60%
80%
0%
10%
2015
30%
40%
50%
80%
Recovery Rate
Recovery Rate
Figure 4:
Recovery rate at different noise levels. i and t denote generative and regression methods, the others are search methods. The error bars depict the 95% confidence interval.
5.2 SYMBOLIC REGRESSION ON BLACK-BOX PROBLEMS
Black-Box Dalaset
Experimental Settings To verify the general- ization ability of our method, that is, whether can he adopted on data collected from the real world, rather than generated from formu- las, we also test our method on 122 black-box problems from Penn Machine Learning Bench-
RondomForest
24
MRGP
MEP
Karn Ragel
A AlFeytmer2
Complexity Rank
mark (PMLB) (Olson et al. 2017). We split the data of each problem into 75% training set and 25% test set, and, instead of recovery rate, we evaluate the results using the Pareto front that balances the test set accuracy and the for-
ITER
15
FEAT
12
Operaz
EPLEX
F2ESR
SNIPT
AFP.FE
mula complexity, which is a common practice in SR tasks Cavalab (2022). For the baseline methods, in addition to the three types meth- ods considered in the ground-truth problem set, we also consider decision tree-based methods (see Appendix B.I. for details), where the con- structed decision trees are treated as a gener- alized formula tree that uses conditional selec- tion as operators and the number of nodes in the trees are used as the formula complexity.
GP GOMEA
Durg
BBR
Linwe
NR
B.
04
3
21
Mean Test R² Rank
Figure 5: Pareto fronts black-box dataset. t. 3. and denote generative, regression, and decision-tree methods, respectively, the others are search methods. The colored lines mark the Pareto front in different ranks, from bottom left (best) to upper right (worst).
Experimental Results The overall perfor- mance of our method and other baselines is
8

---

Published as a conference paper at ICLR 2025
shown in Figure 5 where we can that, method achieves higher test set R with lower model complexity than other methods and is thus sharing the first rank of the Pareto front with three other search algorithms, that is, Operon (Burlacu et al.] 2020), GP-GPMEA (Virgolin et al. 202I), and DSR (Petersen et al. 2021). In contrast, decision tree methods or regression methods usually construct formulas that generative methods are unable to iteratively optimize the results and thus have low test set accuracy.
complex and lack interpretability to describe the given data; while
5.3 MDLFORMER PERFORMANCE
Table 2:
Prediction perfor- mance of MDLformer.
Experimental Settings To give an in-depth discussion about the
RMSE
R²
AUC
MDLformer's estimation performance, we generate K" 1024
3.9105
0.9035
0.8859
pairs of symbolic formulas and numeric data as described in Sec-
tion 3.2 where the independent variables are sampled from GMM. Given that the relative relation- ship of predicted values is more important than their absolute magnitudes when using MDLformer to guide symbolic regression, we consider ranking metrics in addition to the commonly used root mean squared error (RMSE) and coefficient of determination (R²). Particularly, we consider the area under the ROC curve (AUC) (Fawcett) 2006), which assesses the likelihood that the predicted values and true values maintain consistent ordering when randomly selecting pairs (formalized in Appendix ..2.T). As shown in Table the RMSE metric indicates average error of 3.9, which is only 15% of the average formula length of 25. The R2 and AUC are also close to 1.0, demonstrating the excellent performance of MDLformer in predicting the complexity of formulas corresponding to the data. We also provide the results of the other three metrics in Appendix E.2.
MDLformer indicates correct search direc- tions. Here we use a case study to illustrate that the minimum description length (MDL) es- timated by MDL.fomer has the optimal sub- structure, that is: 1) the subformula of the target formula has lower MDL than other formulas and 2) the MDL monotonically decreases along the construction route of the target formula f. We consider the Feynman III. formula, a typical example of successful recovery by our method. As shown in Figure 6 we consider the Feynman III.15.12 equation, a typical ex- ample that our method successfully recovered
y= f(U,k,d) U cos k))
dg
Estimated Minimum Description Length
(1.1%)
Ti
dg
T5
(4.7%)
da
(15.2%)
d
01 [B.A. agtoo (m
11.4.
[t 4) letia
(0.0%)
[V, cosoo
3) d,
[V.1 - coold 20]
T2
4,44
[ cowid x 40)) c tette (=
(See Appendix E..244 for more examples, in- cluding both successful and failed). This equa- tion can be obtained from four steps of transfor- mations Ti.1. Specifically, starting from do = x. we iteratively operate the transformation T that eventually maps do to d, y. For each d, we estimate its MDL concerning y with the MDL.ormer and plot the result in the red line, finding the estimated MDL. does mono- tonically decrease as the formula form gets closer to the target formula. For each d,, we also draw the estimated MDL of d' (d:) for all possible symbolic functions T" as the blue and white sectors. The blue parts show those lower than the correct transformation re- sult di+ = (d+), which can lead to incor- rect search paths. However, the proportion of blue parts is quite low, indicating that the cor- rect search direction aligns well with the direction of the steepest MDL decrease.
Estimated
ground
Estimated MDL all possiole .il lilll knwer Mit
search path) bxxei Ratio of with lower estimated MDL than T,
de
Step
Figure 6: A case study of Feynman III15.12 equation. A series of symbolic functions TV map do = x to d4 = y iteratively. The red line shows Ma (d;, v). the estimated MDL of each d, The blue and white sectors show the estimated MDL for all possible symbolic functions d' (d:),, where blue sections indicate T' with an estimated MDL. lower than T*, which can lead to incorrect search paths. However, the occurrence of these cases, as annotated by the percentage value, is nearly negligible.
Ablation study on training strategy. Figure 4 shows the ablation experiment on the training strat- egy. We considered three strategies to train the MDLformer with the alignment objective Calign and the prediction objective Eprot: 1) sequentially training for alignment and then prediction (i.c., the training scheme described in Sec 3.3). 2) training for alignment and prediction concurrently, and 3)

---

Published as a conference paper at ICLR 2025
direct training for prediction without alignment. A total of 1000 training rounds were performed for each of the three strategies, where for the first method, the first 500 rounds are for alignment and the last 500 rounds are for prediction. Among them, the first two strategies have lower RMSE than the last one, demonstrating that aligning numerical and symbolic spaces improves cross-model prediction, which is consistent with the observation of previous work (Meidani al. 2023) The Sequential strategy has a lower RMSE than the Concurrent strategy, which may be because it is more difficult to use both objectives at the same time. Figure 7 shows the relationship between the prediction error and the number of input points N, demonstrating that more data can help MDL- former predict the minimum description length more accurately. However, larger N can lead to an increase in inference time, thus we choose N 200 to balance the accuracy and efficiency.
Vigure we measure the predictive performance of MDLffrmer on formulas of different difficulty, indicated by 1) the number of vari- ables, 2) the number of unary operators, and 3) the number of binary operators. Although RMSE, consistent with intuition, increases with the formula difficulty. But considering that more difficult formulas can have longer lengths, we additionally plot the ratio of RM to the average formula length, L, in the graph. It can be that as the formula difficulty increases, the normalized RMSE remains basically unchanged, indicating that our MDLformer scales well with the formula difficulty.
Consistent predictive capability across task difficulty,
Robustness against noise. In Figure 7 add feature noise the data of independent variables: I <- x++ n, where n ~ ~OO naxx) is additive noise with intensity proportional to the standard devia- tion of xt az- We find that although the RMSE increases with the noise intensity, the ranking metric, AUC, basically remains unchanged as noise increases, suggesting that our method can identify the relative magnitude of MDL even under noisy conditions. Considering that the search process relies solely on the relative magnitude of estimated MDL, this explains the reason for the robustness of our method to noise in Section 5.1
7.0
125
1.0
Time 10gg
RMSE
RMSE
5.0
RMBE
RMSE
AUC
8.0
Time
5.5
4.5
RMSE
NoAigament
0.04
Training Strategy
Number Input Samples
Feature Noise n
1.0
1.0
RMSE
RMSE II
RMSE I
0,8
RMSE
RMSE normakzzed
RMSE
RMSE
RMSE/
0.
0.0
Number of Variables D
Number of Unary Operators
Number of Binary Operators
Figure 7: Ablation Study. The prediction performance of MDLformer with respect a: trained
with three alignment strategies, b: number of input pairs N, c: feature noise 1), d: number of variables D, e: number of unary operators, and f: number of binary operators. In d,e,f we also plot the RMSE normalized by the average formula length L.
6 DISCUSSION AND CONCLUSION
In this work, we introduce SR4MDL, a symbolic regression approach that optimizes for minimum description length (MDL) rather than prediction error. Leveraging the impressive prediction capa- bilities of MDLformer, it successfully recovers around 50 formulas across two benchmark datasets comprising 133 problems, outperforming state-of-the-arr methods by 43.92%. Additionally, it ranks in the top tier for finding formulas that balance accuracy and complexity on black-box problem set with 122 problems. While SR4MDL showcases robustness and versatility in searching for correct formulas in near-optimal direction, it has notable limitations. First, the performance of MDL- former on data with complex relationships needs to be improved. Second, although the increased efficiency caused by the MDLformer makes method faster than the regression methods, it is still lower than generative methods. Despite these limitations, SR4MDL offers a wide range of capabili- ties, making it powerful tool for uncovering symbolic laws that underlie diverse complex systems like city (Zhang et al. 2025). climate (Zhao et al. 2024), ecology (Holland et al. 2002). etc.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was supported in part by the National Natural Science Foundation of China under Grant No. No.U244B0188 80 and No.62476 No.62276152. This work is also supported in part by the National Key Research and Development Program of China under Grant No. 2022000OAO7 11403, This work is also supported in part by the Beijing National Research Center for Information Science and Technology (BNRist).
REFERENCES
Dimitrios Angelis, Filippos Sofos, and Theodoros E Karakasidis. Artificial intelligence in physical sciences: Symbolic regression trends and perspectives. Archives of Computational Methods in Engineering, 30(6):345533655 2023.
Ignacio Arnaddo, Krzysztof Krawiec, and Una-May O' O'Reilly. Multiple regression genetic program- ming. Proceedings of the 2014 Anmual Conference on Genetic and Evolutionary Computation, pp. 879-886, 2014.
Douglas Adriano Augusto and Helio JC Barbosa. Symbolic regression via genetic programming. In Proceedings. Vol. 1. Sixth Brazilian symposium on neural networks, pp. 173-178. IEEE, 2000.
Tommaso Bendinelli, Luca Biggio, and Alexae-Aeexanree Kamienny. Controllable neural symbolic regression. In /CML, pp. 2063-2077. PMLR, 2023.
Luca Biggio, Tommaso Bendinelli, Alexander Neitz, Aurelien Lucchi, and Giambatrista Parascan- dolo. Neural symbolic regression that scales. In ICML., pp. 936-945 PMLR, 2021.
Cameron B Browne, Edward Powley, Daniel Whitchouse, Simon M Lucas, Peter Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. A survey of monte carlo tree search methods. IEEE Transactions on Computational Intelligence and Al in games, 4(1):1 43, 2012.
Bogdan Burlacu, Gabriel Kronberger, and Michael Kommenda. Operon c++ an efficient genetic programming framework for symbolic regression. In Proceedings of the 2020 Genetic and Evo- lutionary Computation Conference Companion, pp. 1562-1570, 2020.
Cavalab. Srbench: A living benchmark framework for symbolic regression, 2022. URL https: 7/cava S https//enccc/reelltttssssotteeee htaps//eeeeaaaaoaeoooooooooooooa so solssouul lut ons Ac- cessed: 2024-09-26.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp. 785-794, 2016.
Yize Chen, Marco Tulio Angulo, and Yang- Yu Liu. Revealing complex ecological dynamics via symbolic regression. BioEssays, 41(1)) 2019.
Thomas H Cormen, Charles E Leiserson, Ronald L Rivest, and Clifford Stein. Introduction to algorithms. MIT press, 2022.
Cristina Cornelio, Sanjeeb Dash, Vernon Austel, Tyler R Josephson, Joao Goncalves, Kenneth I. Clarkson, Nimrod Megiddo, Bachir El Khadir, and Lior Horesh. Combining data and theory for derivable scientific discovery with ai-descartes. Nature Communications, 14(1):1777, 2023.
Miles Cranmer. Interpretable machine learning for science with pysr and symbolicregression.... arXiv preprint arXivv2305.0/582. 2023.
Stéphane d' Ascoli, symbolic regression for recurrent sequences. ICML, 2022.
Kamienny, Guillaume Lample, and François Charton. Deep
Fabricio Olivetti de Franca and Guilherme Seidyo Imai Aldeia. Interaction transformation evolu- tionary algorithm for symbolic regression. Evolutionary computation, 2973)3667-390, 2021.

---

Published as a conference paper at ICLR 2025
Jingtao Ding, Chang Liu, Yu Zheng, Yunke Zhang. Zihan Yu, Ruikun Li, Hongyi Chen, Jinghua Piao, Huandong Wang, Jiazhen Liu, et al. Artificial intelligence for complex network: Potential, methodology and application. arXiv preprint arX1v2240..16887, 2024.
Renáta Dubčáková. Eureqa: software review, 2011.
Tom Fawcett. An introduction to roc analysis. Pallern recognition letters, 27(8):861-87) 2006.
Yoav Freund and Robert E Schapire. A decision-ttheorete generalization of on-line learning and an application to boosting. Journal of computer and system sciences, 55(1):119-134) 1997.
J Nathaniel Holland, Donald L. DeAngelis, and Judith Bronstein. Population dynamics and mu- tualism: functional responses of benefits and costs. The American Naturalist, 159(3):231-244 2002.
Ying Jin, Weilin Fu, Jian Kang, Jiadong Guo, and Jian Guo. Bayesian symbolic regression. AAAI, 2020.
Cherre-Aexandrr Kamienny, Stéphane d' Ascoli, Guillaume Lample. and François Charton, End-to- end symbolic regression with transformers. NeurIPS, 15:10269-102811 2022.
Pierre-Alexandre
Kamienny, Guillaume Lample, Sylvain Lamprier, and Marco Virgolin. Deep gen- erative symbolic regression with www.-aarre-e-eaaaeeeaaa In ICML, pp. 1655-15668.. PMLR, 2023.
Aleksandar Kartelj and Marko Djukanović. Rils-rols:
Robust symbolic regression via iterated local search and ordinary least squares. Journal of Big Data, 10(71), 2023. doi: 10.1186/ -40537-023-00743-2
Guolin Ke, Meng. Thomas Finley, Taifeng Wang. Wei Chen, Weidong Ma, Qiwei Ye, and Tie- Yan Liu. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems, 30, 2017.
Samuel Kim, Peter Y Lu, Srijon Mukherjee, Michael Gilbert, Li Jing, Vladimir Čeperić, and Marin Soljačić. Integration of neural network-based symbolic regression in deep learning for scientific discovery. IEEE transactions on neural networks and learning systems,
2020.
Andrei N Kolmogorov. On tables of random numbers. Sankhyā: The Indian Journal of Statistics, Series A, pp. 369-376, 1963.
Jifí Kubalík, Erik Derner, and Robert Babuška. Toward physically plausible data-driven models: novel neural network approach to symbolic regression. JEEE Access, 1111148161501,, 2023.
William La Cava, Lee Spector, and Kourosh Danai. Epsilon-lexicase selection for regression. In Proceedings of the Genetic and Evolutionary Computation Conference 2016, pp. 741-748, 2016.
William La Cava, Tilak Raj Singh, James Taggart, Srinivas Suri, and Jason H Moore. Learning concise representations for regression by evolving networks of trees. ICLR, 2019.
William La Cava, Bogdan Burlacu, Marco Virgolin, Michael Kommenda, Patryk Orzechowski, Fabrício Olivetti de França, Ying Jin, and Jason H Moore. Contemporary symbolic regression methods and their relative performance. Advances in neural information processing systems, 2021(DB1) 2021.
Mikel Landajuela, Chak Shing Lee, Jiachen Yang, Ruben Glatt, Claudio P Santiago, Ignacio Ar- avena, Terrell Mundhenk, Garrett Mulcahy, and Brenden Petersen. A unified framework for deep symbolic regression. Advances in Neural Information Processing Systems, 35:33985-33998,, 2022.
William B Langdon and Riccardo Poli. Foundations of genetic programming. Springer Science & Business Media, 2013.
Samnyuya Liu, Qing Li, Xiaoxuan Shen, Jianwen Sun, and Zongkai Yang. Automated discovery of symbolic laws governing skill acquisition from naturally occurring data. Nature Computational Science, pp. 1-12, 2024.
12

---

Published as a conference paper at ICLR 2025
Nour Makke and Sanjay Chawla. Interpretable scientific discovery with symbolic regression: review. Artificial Intelligence Review, 57(1):2, 2024.
Trent McConaghy. Ffx: Fast, scalable, deterministic symbolic regression technology. Genetic
Programming Theory and Practice IX, pp. 235-260, 2011.
Kazem Meidani, Parshin Shojaee, Chandan Reddy, and Amir Barati Farimani. SNIP: Bridging Mathematical Symbolic and Numeric Realms with Unified Pre-training. In The Twelfth Interna- tional Conference on Learning Representations, October 2023.
Nathan Mundhenk, Mikel Landajuela, Ruben Glatt, Claudio P Santiago, Daniel M Faissol, and Brenden K Petersen. Symbolic regression via neural-guided genetic programming population seeding. NeurlPS, 2021.
Pascal Neumann, Liwei Cao, Danilo Russo, Vassilios S Vassiliadis, and Alexei A Lapkin. A new formulation for symbolic regression to identify physico-chemical laws from experimental data. Chemical Engineering Journal, 387:123412, 2020.
Michael A Nielsen. Neural networks and deep learning, volume 25. Determination press San Francisco, CA, USA, 2015.
Randal Olson, William La Cava, Patryk Orzechowski, Ryan J Urbanowicz, and Jason H Moore. Pmlb: a large benchmark suite for machine learning evaluation and comparison. BioData mining, 10:1-13, 2017.
Brenden K Petersen, Mikel Landajuela, T Nathan Mundhenk, Claudio P Santiago, Soo K Kim, and Joanne Kim. Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients. ICLR, 2021.
Markus Quade, Markus Abel, Kamran Shafi, Robert K Niven, and Bernd R Noack. Prediction of dynamical systems by symbolic regression. Physical Review E, 94(1):012214, 2016.
Steven Rigatti. Random forest. Journal of Insurance Medicine, 47(1))31-39, 2017.
Cicero dos Santos, Ming Tan, Bing Xiang, and Bowen Zhou. Attentive pooling networks. arXiv preprint (:XXi:1602.03609 2016.
Martin Schmelzer, Richard P Dwight, and Paola Cinnella. Discovery of algebraic reynolds-stress models using sparse symbolic regression. Flow, Turbulence and Combustion, 104:579-603, 2020.
Michael D Schmidt and Hod Lipson. Age-fitness pareto optimization. In Proceedings of the 12th anmual conference on Genetic and evolutionary computation, pp. 543-544, 2010.
Dominic P Searson, David E Leahy, and Mark Willis. Gptips: an open source genetic programming toolbox for multigene symbolic regression. In Proceedings of the International multiconference of engineers and computer scientists, volume 1, pp. 77-80. Citeseer, 2010.
Hongzhi Shi, Jingtao Ding, Yufan Cao, Li Liu, Yong Li, et al. Learning symbolic models for graph-structured physical mechanism. In The Eleventh International Conference on Learning Representations, 2022.
David Silver, Aja Huang, Chris Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schritwieseer, loannis Antonoglou, Veda Panneershelvam, Mare Lanctot, et al. Mastering the game of go with deep neural networks and tree search. nature, 529(7887):484 489, 2016.
David Silver, Julian Schritwieser,, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. nature, 550(7676):354 339, 2017.
Guido Smits and Mark Kotanchek. Pareto-front exploitation symbolic regression. programming theory and practice 11, pp. 283-299, 2005.
Genetic
Trevor Stephens Gplear: Genetic progaammnn in python, with scikit-learn inspired api, 2016. Accessed: 2024-09-26.
URL htt.::/g/.ggghbbooo.oo..ssssetts
13

---

Published as a conference paper at ICLR 2025
Steven H Strogatz. Nonlinear dynamics and chaos: with applications to physics, biology, chemistry, and engineering. CRC press, 2018.
Fangzheng Sun, Yang Liu, Jian-Xun Wang, and Hao Sun. Symbolic physics learner: Discovering governing equations via monte carlo tree search. arXiv preprint 1xX1::2205.13144 2022.
Sheng Sun, Runhai Ouyang, Bochao Zhang. and Tong- Yi Zhang. Data-driven discovery of formulas by symbolic regression. MRS Bulletin, 44(7):559-566) 2019.
Wassim Tenachi, Rodrigo Ibata, and Foivos Diakogiannis. Deep symbolic regression for physics guided by units constraints: toward the automated discovery of physical laws. The Astrophysical Journal, 959(2):99, 2023.
Silviu-Marian Udrescu and Max Tegmark. Ai feynman: A physics-inspired method for symbolic regression. Science Advances, 6(16):eaay6611, 2020.
Sarviu-Marian Udrescu, Andrew Tan, Jiahai Feng, Orisvaldo Neto, Tailin Wu, and Max Tegmark. Ai feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity. NeurIPS, 33: 4860-4871, 2020.
Emile Krieken, Thiviyan Thanapalasingam, Jakub M. Tomezak, Frank Harmelen, and An- nette ten Teije. A-nesi: A scalable approximate method for probabilistic neurosymbolic inference, 2023. URL https: org abs 221./222221111333 12393
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceedings of the 31 Inter- national Conference on Neural Information Processing Systems, NIPS' 17, pp. 6000-6010, Red Hook, NY, USA, December 2017. Curran Associates Inc. ISBN 978-1 1-888-5108--00 108-6096-4.
Marco Virgolin, Tanja Alderliesten, and Peter AN Bosman. Linear scaling with and within seman- tic backpropagation-basee genetic programming for symbolic regression. In Proceedings of the genetic and evolutionary computation conference, pp. 1084-1092, 2019.
Marco Virgolin, Tanja Alderliesten, Cees Witteveen, and Peter AN Bosman. Improving model-based genetic programming for symbolic regression of small expressions. Evolutionary computation, 2922):11-237,, 2021.
Paul MB Vitányi. How incomputable kolmogorov complexity? Entropy, 22(4):408, 2020.
Yiqun Wang, Nicholas Wagner, and James Rondinelli. Symbolic regression materials science.
MRS Communications, 93):79338855, 2019.
Baicheng Weng, Zhilong Song. Rilong Zhu, Qingyu Yan, Qingde Sun, Corey G Grice, Yanfa Yan, and Wan-Jian Yin. Simple descriptor derived from symbolic regression accelerating the discovery of new perovskite catalysts. Nature communications, 1(1):33513, 2020. 2020.
Yilong Xu, Yang Liu, and Hao Sun. Rsrm: Reinforcement symbolic regression machine. ICLR, 2024.
Hengzhe Zhang, Aimin Zhou, Hong Qian, and Hu Zhang, Ps-tree: A piecewise symbolic regression tree. Swarm and Evolutionary Computation, 71:101061, 2022.
Yunke Zhang, Yuming Lin, Guanjie Zheng, Yu Liu, Nicholas Sukiennik, Fengli Xu, Yongjun Xu, Feng Lu, Qi Wang, Yuan Lai, al. Metacity: Data-driven sustainable development of complex cities. The Innovation, 2025.
Sen Zhao, Fei-Fei Jin, Malte F Stuecker. Philip R Thompson, Jong-Seong Kug, Michael J McPhaden, Mark A Cane, Andrew T Wittenberg, and Wenju Cai. Explainable el niño predictabil- ity from climate mode interactions. Nature, 630(8018):88 89:188888988 -888, 2024.
Jinghui Zhong, Liang Feng, Wentong Cai, and Yew-Soon Ong. Multifactorial genetic programming for symbolic regression problems, IEEE transactions on systems, man, and cybernetics: systems, (411)):442245008 2018.
14

---

Published as a conference paper at ICLR 2025
A RELATIONSHIP WITH RELATED WORK
To better illustrate our innovation, we compare the proposed method with two types of existing methods: 1) heuristic search methods and 2) neural network-based generative methods, shown in Figure [ As illustrated the figure, our method differs from heuristic search methods in terms of its search objective, which shifts from accuracy the MDL estimated by MDLformer. On the other hand, unlike neural network-based generative methods, where the trained transformer directly generates the symbolic sequence in end-to-end manner, our proposed method incorporates the MDLformer component of the framework, which provides estimated MDL values that serve as search objectives for the search algorithm.
network-based generative
peerchprecess process
Training process
Search process
Training process
Inference process
X-V parr
Genersted formula
Cenerated KAp
PAI
Gameraned formala
Genereted
x'y par
x-V pull
chjpctas
NNT fremert
Aitiformmr
Search algoritm
Transfore
Tranderme
Tanges
Formula longth
Targes
Symbatem
Targes
Figure 8: Comparison of heuristic search methods, neural network-basc generative methods, and methods.
Furthermore, many recent studies focus on neural-symbolic reasoning, that is, adopting neural networks for symbolic reasoning (van Krieken al. 2023). These works focus the problems that, between the input feature x and target value y, there exists symbolic "concept" w that determines y in known way y c(w). The knowledge of
thus be used to help predict y and reason behind the input x. For example, if we wanna predict the sum of digits in series of MNIST images x, can benefit from the knowledge that each image x; contains digit w; and given all digits wi the sum can be determined by y E, W/, However, despite the similar names, neural- different task than symbolic regression since no such symbolic concept exists between the input data (x, y) and the target formula f in symbolic regression. This makes neural symbolic reasoning methods unsuitable for symbolic regression tasks.
symbolic reasoning
B TECHNICAL DETAILS OF MINIMUM DESCRIPTION LENGTH
The minimum description length (MDL) represents the size of the simplest model used describe the data(Kolmogorov 1963). Although it an abstract and incomputable concept in many fields, rather conerete in the field of symbolic formulas. Specifically, for pair of input-output data (æ,, v). the minimum number of symbols (i.e., variables, operators, and parameters) required to describe the target formula known formula f. For example. f(x) x² sin(x) contains five symbols: [z, 2 +. sin, æ², so the
f(x) is its MDL, which is quite easy calculate when (xi, y) generated by
MDL of data (a., y fxx) is
One question is how to ensure that the least number of symbols are used to describe f. that is, f in its simplest form. In Section we simplify the generated formulas with sympy, an open- source Python package for algebra processing, before calculating its length the training label. The sympy package provides several ways to simplify a formula, such as expand that expands formula, factor that factors the formula, col lollect that collects common powers of term in an expression, etc. We simplify each formula with different methods and choose the shortest simplification result the simplest form of the formula. Manual checking 100 simplification results shows that sympy can reach human-level simplification, simplifying most of the formulas to the shortest length that humans do.
15

---

Published as a conference paper at ICLR 2025
C DETAILS OF MDLFORMER
C.I MODEL ARCHITECTURE OF MDLFORMER
The MDLformer integrates an embedder, Transformer encoder, an attention-based pooling layer, and a multilayer perceptron (MLP) as readout head. The embedder first pad z E R RNxx x to N where Dmax = 10, then concatenate it with y to obtain (x, y) E Nx x The results are tokenized to triplets of sign, mantissa, and exponent, and are then embedded to D;- dimensional embedding space, forming the E RRNxxmax-1 X3xD. where D; = 64. To reduce the length of the input sequence of the Transformer encoder, adopt MLP with hidden layer that maps to E' RND,, where D, = 512. Without positional encoding, E' is directly fed into an 8- 8-layer Transformer encoder with 8 heads and 512 hidden units, whose feed-forward layer has a hidden size of 2048. The output of the Transformer encoder, V E NR is then pooled to RD, through the attention-based pooling layer. The n is then fed into the read-out MLP with hidden layer to obtain the estimated MDL .Ma (x, y) E R. We clip N to no more than Nmxx 200 the input of MDLformer. There are 31.9 million trainable parameters in the MDLformer and 30.7 million trainable parameters in the Symbolic encoder used for the alignment training objective.
C.2 PRE-TRAINING OF MDLFORMER
We train the MDLformer in a two-step way: First, we use the alignment loss, Calign. to train the MDLformer for 100k steps. Then, we use the prediction loss, Lpred, train the MDLformer for another 30k steps. During the training, we use a batch size of 1024, which consumes CUDA memory of around 150G. We use learning rate of 10 and the noam schedule (Vaswani et al. 2017)). where the warmup step is 4000. During the training, we use a dropout rate of 0.1. Trained on a machine with 8 A 100-SXM4-80GB GPUs and an AMD EPYC 7742 64-Core processor, the training lasted 41 hours, during which 131 million pairs of input data were generated.
D DETAILS OF OUR SR METHOD
We compare three different search methods in Table 3
Genetic Programming (GP). GP (Stephens, 2016) maintains set of candidate formulas, called population. At the initialization step, a random generation algorithm generates initial candidates in a given number as the starting population. In the update step, there are two kinds of functionals to operate on these candidates: crossover and mutation. For the crossover, two candidates are sampled from the population, whose random subformulas are then swapped with each other to obtain the new candidates. For the mutation, a candidate is sampled from the population, whose random subformula replaced with another formula generated by a formula generator.
Monte Carlo free search (MCTS). MCTS (Sun et al. 2022) maintains a search tree, where each node represents a formula Sa and some numbers, including total rewards Q, and total counts N;- At the beginning, the search tree contains only an "empty" formula / D. where O represents a placeholder to be filled. In the update step, the MCTS algorithm starts from the root node and uses the greedy algorithm to select a formula with the largest upper confidence interval (UCB) that is not in the search tree. Specifically, it considers all possible formulas fa obtained by filling in a mathematical symbol into a placeholder of current formulas and selects the one with the maximal UCB
Qi N
In In(E, No) N
UCB
(5)
where Σ, is the sum of all possible formulas. By iteratively repeating this selection until the selected formula frr is not in the search tree, MCTS finds the formula be added to the candidate set in this round. The placeholders in this formula (if any) are filled with random formulas to create several complete formulas, whose averaged prediction errors in mean square error, Rt are used to update Q: for f and its ancestors:
Qi+
+
(6)
1+ R/0
16

---

PUCT; Qi N, Cpuct M& v) N +1 Published as a conference paper at ICLR 2025
where 7 = 0.999, C[ that evaluates the length of f* is used regular term, By is the standard deviation y.
Our Method. Based on the MCTS algorithm, our method also maintains search tree. The dif- ference is that each node in the tree fü FDDD, represents D; subformulas in the target formula, where D; is no more than 10. For node f {f ... f(DD)), its child node set can be obtained by applying all possible mathematical operators (e.g., +. x, sin) all possible combinations of (f(.)....
.ID} and adding the results to f, with without keeping the operands. We set the root node the search tree as f(x) = I E FDXD. D. As an example, for a problem with D 2, initialize the root node of the search tree fo, {01, 22), and its child node can be
fi,1 = { + 22, x2 f1. { x 22, 21, 12h. f1. {sin F1, I1, N2h, f1, {sin 21,
f1S {21 + :1)
f19 = { + 22: 22
f1.18 {-1 + 22). f1.16 = {2.
Starting from the root node, we obtain the node to be added to the search tree in each round by using the greedy algorithm to select the f; with maximal PUCT Silver et al. (2016):
ΣE N
where Cpuct 1.41, the index denotes all possible formulas that can be obtained by operating an unary or binary mathematical operator on or 2 operands, which can be subformulas contained in the current node, variables, or numeric constants. Apart from these, the other parts are the same as the traditional MCTS approach introduced above.
Table 3: Comparision of existing search regression methods and ours
Method Step (Initialize, Update, Select)
fs Popullation =
[Randomly Generated f}
GP
f.) (fi): U (Cross(fi, Ji) U (Mutate(ft)):
(fi) + topk se()), 1ly f(x)]?
(fi)ie = {0} (an empty formula without any operators)
MCTS
(fi) + (ft where f has a maximum UCB selected by a greedy algorithm
[fAhieMCuce == {x}
Ours
(fit + (fili U Uff). ft has maximum PUCT selected by a greedy algorithm
E DETAILED EXPERIMENT RESULTS
E. SYMBOLIC REGRESSION
E.1.1 BASELINES
In our experiments, we considered large number of baseline methods, both the ones that come with SRbench and the most recently proposed ones, ensuring the breadth of comparison.
Search methods. The vast majority of the methods we compare are search-based algorithms. Among them, genetic programming algorithms are the primary. including GPlearn (Stephens) 2016).
AFP & AFP-FE (Schmidt Lipson| 2010). ITEA (de Franca Aldeia 2021), EPLEX (La Cava 2016), MRGP (Amaldo et al. 2014). SBP-GP (Virgolin et al. 2019), GP-GOMEA (Virgolin al.
et al. 2021). ). and Operon (Burlacu et al.) 2020). In addition to this, we also compare some recent ap- proaches based on reinforcement learning, including SPL that leverages the Monte Carlo tree search
(Sun et al. 2022), DSR that relies on the deep reinforcement learning (Petersen et al.) 202I), and
RSRM that uses double Q-learning algorithm (Xu et al. 2024). Finally, recent approaches that
17

---

Published as a conference paper at ICLR 2025
combine multiple search methods are also compared, including Al Feynman 2.0 (Udrescu et al. 2020) and BSR (Jin et al. 2020),
Other methods. We compare series of novel methods that leverage large-scale pre-trained Transformer- based neural networks to predict the target formulas directly, including NeurSR (Big-)
glo al.
202T). E2ESR (Kamienny et al. 2022). and SNIP (Meidani et al. 2023
regression method, FEAT La Cava et al. 2019), For the experiment the black box problem set, additional regression methods based on linear regression, MLP fitting, and Kernel Ridge model are included as well.
Regression methods. We also include a neural network-
Decision tree methods. For the experiment the black box problem set, also consider the XGBoost (Chen & Guestrin (0166) AdaBoost (Freund & Schapire 1997). Random Forest (Rigatti) 2017), and LightGBM (Ke al. 2011)) model, which can be considered special type of formula with discontinuous selection operators,
E. GROUND-TRUTH PROBLEM SET
We provided the detailed experiment results on the ground problem set in Table finding the recovery rate of our method always outperforms other methods, while its running time always within a reasonable range. Note that although we provide complexity in these tables, is not that smaller value is better, since the gound-trute truth formula itself has its length. We also provide the Pareto fronts balancing "recovery rate- search time" and "test R² complexity" as in Figure finding method always on the rank of the Pareto front.
and 7
E. BLACK-BOX PROBLEM SET
We provide the detailed result of the experiments on the black-box problem set as Table X where our method achieves the rank I of the Pareto front within reasonable timeframe, meaning it can balance the accuracy (measured by the test set It2) and simplicity (i.e., formula complexity) better than other baseline methods.
E.2 PREDICTION PERFORMANCE MDLFORMER
E.2.1 METRICS
In addition to the RMSE, R2, and AUC metrics we provided in the main text, we also considered another three metrics as in Table 9 All metrics illustrate the excellent performance of MDLformer in estimating the value of MDL. The definitions of these metrics are provided follows:
RMSE evaluates the average difference between the prediction value and the ground truth:
RMSE
(v. (1n-b) 0:)2
R² evaluates the strength of the linear relationship between the predicted value and the target value:
Σ K
- 0.)
Ra 2
Σ (vx 7)
AUC is a ranking metric, which evaluates the likelihood that the predicted values and the true values maintain consistent ordering when randomly selecting pairs:
Σ K Σ/e 1((, v,)( 99)) 0)
ROC
(10)
K(K - 1)
where is the indicator function.
PSpeamma is another ranking metric, which is defined as
D. (r; 1;)2
PSpearman
N(N2 -1) 1)
(11)
where Y; is the rank of yi, ranging from small to large, and the similar as f..
18

---

Published as a conference paper at ICLR 2025
Feynman Dataset
Strogatz Dataset
18
SBP-GP
BSR
18
SBP-GP
BSR
AFP.FE
AFP.FE
16
EPLEX
16
GP-GOMEA
AFP
Search Time Rank
GP-GOMEA
Search Time Rank
DSR
GPlearn
12
GPleam
Operon
12
EPLEX
FEAT
INSSR [uurs)
10
NSSR (ours DSR
FEAT
10
AlFeynman
AFP
Operon
AlFeynman
FFX
E2ESR
NeuralSR
E2ESR
NeuraiSR
4
SNIP
SNIP
ITEA
MRGP
ITEA
MRGP
12
14
18
14
Recovery Rate Rank
Recovery Rate Rank
Strogatz Dataset
Feynman Dataset
MRGP
SBP-GP
MRGP
FFX
18
SBP-GP
FFX
Operon
FEAT
16-
FEAT
Complexity Rank
14
AFP-FE
EPLEX
Complexity Rank
14
Operon
AlFeynman2
EPLEX
GPleam
12
GP.GOMEA
BSR
12
AFP.FE
AFP
10
E2ESR
10
GP-GOMEA
AFF
E2ESR
8
GPleam
NSSR (Ours)
AlFeynaan2
NSSR (Ours)
BSR
DSR
SNIP
ITEA
SNIP
SPL
DSR
NeurSR
NeurSR
SPL
10
16
10
14
16
Test R2 Rank
Test R Rank
Figure 9: Pareto fronts balancing "recovery rate search time" and "test R² - complexity" on Feynman and Strogatz datasets.
19

---

Published as a conference paper at ICLR 2025
Complete experimental results at a noise level E = 0.0. The results include test set R? recovery rate, search time, and model size of different methods
Complx
205. (15.31213)
13.28
36.02ee.80
14.87u00.00 (0)
72.43
36. (6.8702.2)
39.9 99.97919
52 22.95(41.)
489.4 49.40
34.5 34.5771155
69 19.88411.0
12. 8.888.0055
14.86u)
13.17
124.5
25.5
23 23.401.2
5
489.
Feynman Dataset (E = 0.0)
Time (s)
2417
24. 1..88
4,062
(.9010114
3900
3655
17817
12771 (4:998)
(8901(a3))
50.30s90))
2174
209
1733
116.3
854.3
29065 (a765)
467. 467.3a4ss (3435)
R. Rate
0.00%
3. 3.20%
12. 22.90%e119
1.68%)a1
16. (eces4
21.12 12.1%)
26.98 26.98%)33%
12. 22.9%
12,7 (2.72% (22)
26. (6.83% =2%)
16.55% (449)
10.37%
(9.72%
66.07%6225 (622))
57.32% (44119
2. 2.48%
39.9 39.92%eu22
Tables|56677)
in the Feynman and Strogatz datasets, each experiment conducted under 10 random seeds. (The same as Tables (/6
0.91 0.9190u004)
0. 0.3958
0. 0.8570
0.6480(20.05))
0. 0.8809
(50
0. 0.9590 c00000
0. 0.980660.00m (0.000))
0. 0,9869
a0.00))
0.9956san0O
0. 0.9889 (AL00S
0.7073
0. 0.8441 (20.00))))))
0. 0.8003(a0.01)
0. 0.9314
(..609
R²1
0. 0.9945
9
Complx
119u16
11.3c000
32. 22.49
16.77 59
28.9 (622.82
38. 17.2.
46. 16.4.)
50.09
712.2
36.43G224
59.23
14.55e2.5
15.6u1.5)
13 (3.0942.33
22. 22.26.177)
38.98 98a..4
14.07u1.9
3
Strogatz Dataset (c = 0.0)
Time (s)
1135
(4.82
3.729
1.377gmm
1195ea59
192.2cs106)
1104 1104 (2M477)
548. 26.22)25))
17591 (1770)
2760cs1290)
(6
322. 16)
1858 (2617)
121.9 9.33
762, 1e424)
31380 420922)
186 60s35)
R. Ratet
0.92%)
0.00% (00)
6.06%)55)
(.14%))0))
8.93 ((729))
15. (6
20.00%
8. (.93%
11.61 % (28%)
11 41.43%%.ol (200))
7 7.94%)c228
19.6 19.64% (L87))
28. 18.24%%camo (20%)
67.86 9 (aM)))
29, 19.46%
0,92 000000
0 0.5206-0022)
0 0.5341 0.5341e0045 (e004))
(..00))
0. 0.9248c004n
0.944280045)
(812550005)
0. 1.981 0.9812enard 1200ato
0.9925e000)
0. (.9878
0.7390e00077
0. 0.7602.0085)
1.550l
0. 0.6459
0.8455
0. a.9900ammmm
R²1
2
GP-GOMEA
AlFeynman2
Method
FEAT
NeurSR E2ESR
SNIP
GPlearn
AFP
AFP-FE
EPLEN EX
SBP-GP
Operon
SPL
DSR
RSRM
BSR
Ours
Rank
Regression
Generative
Search
Type
Table 4:
20

---

Published as a conference paper at ICLR 2025
Complx
196.5
13.24aa11)
35.85
24, 24.888018
60.49(e1))
39.33(aL6)
46.47
56.03
574.4(a:3)
23.230
69.67ct.
(3.15
16.044030)
13.044m.49
130.6rath
25. (5.55e005))
32.35
7
45.
Feynman Dataset (c = 0.001)
Time (s)
1726
24.5 ((t
3.894 90
1.926
371 lc457)
26160(a15))
12341
28940
2904
(195
275.5
830 1uzey
228.243.8)
50255ee7700
428. 428.9 (1z66)
638
R. Rate?
0.00% (00))
3. (41%))
11. 11.46% (6.6%)22))
2.02% (119))
17.27%
19. 99.66%uu28
21 11.00%
9.57%
0.78%
11.03%
13. 13.19%
10.28%
19. (4%1)11
17.18 17.18% 866228))
33. 33.67%
0.60% (1
34.22 % (82%)
Table 5: Complete experimental results at a noise level 0.001.
0.9207
0. 0.3979
0. 0.000)
66576608825
0. (8902
0. 0,9606un000,
0. 0.9805unom um000)
0. 0.9866cnmmm
0. (
0.9962cuumn)
0. 0.99 0.9916usn.n 16snnnss
0.7073
0. 0.87640n00)
0.8104
0. (
0.6538een02h
0.9079(nn0s)
R²1
9
Complx.
106.7
(1.41
33 33.83
16.66:a.85)
29. (2))))))
38. 75.4.)
48. (8.74a3)
49 49.264447)
820.5
41.14.a2.)
67.03
14.4 (4.4
18. (8.66
12.23
25 (5.64(25
37. 37.68
21 11.38.1.
6
Strogatz Dataset (c = 0.001)
Time (8)
594.4
1 15.077t
3.436
1.344.0.0
213.3
143,4 422))
8108 (58)))
416. 416.1e8)) lea0)
19596e122)
804
75. (5.681114)
358. 8
500. 8:33))
129, 129.7 7...1)
84. (4.19)74))
(4299(a388)
17.550
R. Ratet
(005))
0.00%)6
0.00%)))9)
6. 6.43% (6
9. (hc4))
13.57% (3.57%
(5.71%)
9. (.29%)))
0.00%
7. 7.14%)
5.00%)))7)
7. 7.63 (63%)
19. 20.2%%
6. 6.06% (61168)
22. 14%%
0.71 (6c228)
73.24 (199)) (19%%)
0.9244
0.5219 (0003))
(,510
0.4220G0059
0.7955 (40067)
0.9172)0055
0.9447
0. 0.8488 c00s))
0.9879
0.991 0.9914,4000) 4:000)))
0. 0.9843 (0006)
0.7526
0. 0.8067 (0048)
a..447
0. 6069)
0.8224
0.9965mm00m
R²1
GP-GOMEA
AlFeynman2
Method
FEAT
NeurSR E2ESR
SNIP
GPlearn
AFP
AFP-FE
EPLEX
SBP-GP
Operon
SPL.
DSR
RSRM
BSR
Ours
Rank
Regression
Generative
Type
Search
21

---

Published as a conference paper at ICLR 2025
Complx.
66.1le6.)
13. 33.26.00122
26.9 26.96.0.40
60.07
40 (0.86.11)
46.92a23)
54
995.5erb
44. (4.67
87 97.96113)
13.43
16.2
12.97
155.2
28.03
31.15
40.0 40.07
Feynman Dataset (c = 0.01)
Time (s)
1464x3305
24.62
4.274
2.19
3335 (0-3)
4090ku79))
27763
11043
28954(a5)
3020ka60)
3165u549)
295. (4155)
929 929. (9.88000)))
13 1.166:11
629. 629.4 4:.00)
304 3041 1cs471))
844.3 (uss
7
R. Ratef
0.00%)
3.11 111%
9.72%
2.6 %6az%)
16. 99.9%%
16. (.90%)
20. 78%%
8. 8.7 (6428))
0.00%
5. 5.09%6a11.
2.07%es10 19)
(1.24%)))5)
18.97%
15.00%42%)
13. (3.86%)c44%)
31 $1.97 51.97% (o12%)
0.09
Table 6: Complete experimental results a noise level = 0.01.
0.9212a0010
0.3942
0.8345.00.0)
0.6550ae.29)
0.8890
0.96 196610u000s
0.98 19
0. 0.99 10
0. 0.9.50
0.9967c=000
(300m)))
0.8782e00.0) (8.82.0000)
0.8092s00015
0.8732
0.6734,0015)
0.9 140g009))
R²t
C
11.63
35 15.94e18)
30 (4.3))
38. D)
48. 18.8.5.4)
53 (3.07a38)
850.9(34
42. (4.44445
81 (1.68
14.71(
18.51a12
14. 14.25
32. 12441
38. 74.2222
20.31 (
95.61am
Strogatz Dataset (c = 0.01)
Time (8)
472. 472. 9,90
15. 15.48ka114
3.392e0m
1.418
907.4.. 10)
152.2
(898
437 6:260
20783
765. 644100))
94.9 14
413.3 (3.3v-205)
492.9u287
142.2a22)
85.17
23622
505. 1u39
12
R. Ratet
0.00%)200)
1.28 (2139)
7.145))28)
9. %her8))
13.57%))
4. 4.29%))44
0.00%))09)
1. (.43%)88)
0.71 ()zzs)
7.91 %)r48)
4.3 4.31% (645%)
9. 9.29 (.29%)))))
0.00 (6c009)
62.86 %)10)))
6. 6.43%%uzz
0.9244s000)
0.51 0.5179
0.503 10031e0000 (00.04))
0.4562u00050
0.7956:4
0.9153u0055)
0.9582s00.0
0. 0.8562a0000)
0. (.
(8982))
0.7388c-0000
0.8 (8199
0.8127 (50000))
0.971 18n0057)
R²1
GP-GOMEA
AlFeynman2
Method
FEAT
NeurSR E2ESR
SNIP
GPlearn
AFP
AFP-FE
EPLEX
SBP-GP
Opernn
SPL
DSR
RSRM
BSR
Ours
Rank
Regression
Generative
Type
Search
22

---

Published as a conference paper at ICLR 2025
Complx
99. 99.48kes.0)
13.54aa11
44.09
30.8
48. 43.5))
40.79 40.79(u15)
48.88 (8.87
45.62
46. (3.43)))
59.23amm5)
(3.54
16.03aa.60
12.81a040
176.6
28.77ca.
30.86
621.9
7
Feynman Dataset (c = 0.1)
Time (s)
777. 177.9kx00
25. (11
6. 6.183
2.768
29338
3886034)
288 122051)
10283e550)
28932
3186
3090
270
8 814.911177
133 31.8
(2497
962.9.988
710.7
R. Rate?
0.00% (000)
0. 0.00% (00%)
7.01% ee
0.08% (08)
16. 16.80% (4(2)))
13. 1010%)uu)))
13.53%
(0.17%)33))
0.00% (e0es)
1.64%
0.09% (O00)))
10.00% (6.0%)
16.8 1%
13. 13.98%
0. 0.83 (33
28.79 % (ar%)
0.00%
1.
Table 7: Complete experimentas results at a noise level. 0.1.
0.9195
0. 0.3823
0.77 0.7714 14000144
0.6669
0.891 1
0. 0.9577 70077
0. 0.9826un0005)
0. 0.9901 can000))
0. 0.9905 ano0m
0.9957cumms
0. 0.9847 cenonR
0.7109
0. 0.8779 un
(.8104
0.2248
(emmn)
0.6567 0.6567cen08 (en.224)
0.9097 0.9097unam: (1000)))
R²1
8
Complx.
84.2aa15
12.74
38 (2.4
22.55(a1.2)
84.22)))))
44. (4.44us.3)
50.9 50.96
53.46 23.4642.5)
901 (23.2
43. 23.77
83. (3.44.11.2
(3.91
18.51
(3.54LLLD
23. 33.53
...4..99
20.5 (.n.. 62)
5
Strogatz Dataset (c = 0.1)
Time (s)
446.9(73
17 (7.47e11))
5.62 (149)
1.695 7. 1.14%
894. 11e
161. 4..2
10082:565) (565)
405. 405.8 81443)
2 21886:2881
402. (
97.13es1
355.3
500k
138. 3(2.4)
65.97
23292
522.6
13
R. Ratet
(.00%)
0.7 11%))22)
6.43%)
4.29%%)
2. (4%%)18%)
2. (4%)
0. 1.00%ccare)
0. 0.00%ccarrs
0. 0.00% (C)))))))
9. 9.09%..-8)
15. 15.00%
3.8 (6445)))
2. 116%)ca4))
0.00%
62.86 12.86%outt4 (a144)
0.9228
(.0054
0.5152e003)
0.5536
(.8228(4a05))
0.9110s0055
0.9496u0033
0. (8822)00))
0.9323,00000
0,9668 (0031)
0. (094))))044)
0.7715
0. 0.80862004)
0.5553
0.3170 z000))
0.7190 (4066)))))
0.968614u57)
R²4
GP-GOMEA
AlFeynman2
Method
FEAT
NeurSR E2ESR
SNIP
GPlearn
AFP
AFP-FE
EPLEX
SBP-GP
Operon
SPL.
DSR
RSRM
BSR
Ours
Rank
Regression
Generative
Type
Search
23

---

Published as a conference paper at ICLR 2025
Table 8: Complete experimental results of black box dataset.
Pareto Rank
Method
Test R² 1
Complexity
Time (s) 1
Operon
0.7945
65.69
2974
GP-GOMEA
0.7381
30.27
9636
1
NSSR (Ours)
0.6258
29.88
541.7
DSR
0.5625
9.465
36852
SBP-GP
0.7869
634
149344
FEAT
0.7621
82.49
6432
EPLEX
0,7372
53.14
15796
AFP-FE
0.6400
36.04
6184
2
AFP
0,6333
34.89
6033
GPleam
0.5390
19.06
24254
Lineart
0.4437
17.4
0.2447
NeurSRt
0.1228
13.33
11.7
XGB*
0.7496
20186
236
AdaBoost
0.6939
9481
65.12
LGBM
0.6410
5734
29.9
3
ITEA
0.6295
116.7
12183
SNIP
0,3335
38.91
3,286
BSR
0.2725
22.52
59822
RandomForest*
0.6615
1517178
120.4
KernelRidge
0.5952
1824
39.19
FFX
0,5575
1562
244.3
E2ESR
0.3612
61.09
7.101
MRGP
0.5300
10802
165007
5
MLP
0.5238
3882
30.49
AlFeynman2
0.2110
2240
86854
24

---

Published as a conference paper at ICLR 2025
Preamon or the correlation coefficient, is widely used metric for estimating the degree of correlation between two variables, which is defined as
cov (V.
Σ (yi - 0)( Uz) VE. (u 5) Ex (Ú: 5.)
PPearson
(12)
NDCG is a ranking metric that is defined as
NDCGR E, - 1)/ log2 (r +
(13)
E/ 1)/ log2(1 1)
where y, is the r-th smallest value in y, i is the index of the r-th smallest value in where we chose R = 5.
RMSE
Ra
AUC
PSpearman
Pearson
NDCGs
3.9105
0.9035
0.8859
0.9542
0.9506
0.7264
Table Detailed Prediction performance of MDLformer.
E.2.2 ABLATION STUDY
We provide the AUC metric the ablation study as in Figure 10 where we can find similar con- clusions in the main text. That is, 1) the sequential training strategy outperforms the other two strategies, 2) the performance of MDL estimation increases as the number of input samples in- creases, and 3) the RMSE performance decreases the feature noise gets stronger while the AUC performance keeps robust.
0.84
0.86
AUC
0.87
100
RMSE
10
08
09
0.82
0.7
RMSE
AUC
Nosigremntt
200
0.0
6.24
00s
abs
0.1
Training Strategy
Number of Input Samples
Feature Noise f
Figure 10: AUC metric in ablation study.
In Section 3.2 we used a variety of methods to increase the diversity of generated data, including sampling data dimensions D, randomizing formula length L. using Gaussian mixture models to sample data distribution x ~ GMM, etc. These designs enhance the diversity of the generated train- ing data, and thus ensure the generalizability of our method on specific symbolic regression tasks. To test the effectiveness of these designs, in Figure we test the recovery rate of our SR4MDL method guided by the MDLformer trained with these designed ablated. We conduct experiments on the Strogatz dataset, with noise level 0 and 10 random seeds. We considered three kinds of abla- tion: 1) Data dimensions that replace the random dimension D (4(1..10) by a constant dimension D 2; 2) formula lengths that reduce the maximal formula length T-max from 50 10; and 3) data distribution that sample data & from 24(0, 1) rather than a more diverse Gaussian mixture model (GMM). As shown in the figure, all ablations result in performance degradation, demonstrating the effectiveness of our design.
E.2.3 PERFORMANCE WITH RESPECT TO TRAINING SAMPLE SIZE
To study the impact of training sample size on MDLformer's predictive performance and com- pare it with existing neural network-based generative symbolic regression methods, in Fig- ure figrevaluateMDLLoroo training samples, well the corresponding recovery rates on Strogatz dataset (noise level 0.
we plotted its prediction performance with respect to the number of
25

---

Published as a conference paper at ICLR 2025
Recovery rate MDLformer trained with ablated designs
70%
62%
Recovery Rate
Recovery Rate
50%
Recovery Rate
007
40%
40%
42%
307
30%
30%
20%
20%
27%
10%
10%
10%
0%
1-121-10)
0%
1ra
Zrue 0%
x10,0
GALM
Data dimensions
Formula lengths
Data distribution
Figure 11:
The recovery rate of MDLformer trained with ablated designs. We test the recovery rate our SR4MDL method guided by the MDLformer trained with ablated designs, including data dimensions, formula lengths, and data distribution. The experiments are conducted on the Strogatz dataset, with noise level c and 10 random seeds.
on 10 random seeds). The prediction performance is measured by both the RMSE (blue line) and the AUC (green line). Since our training is divided into two stages, where in the first stage MDL- former is trained to align the numeric embedding space with the symbolic embedding space but not learn to predict MDL, we only plot the performance curve of the second phase. At the beginning of the second stage, the AUC is about 0.5 and the RMSE is about (i.e., the average length of the formulas in the training set), indicating that the MDLformer does not have the ability predict the MDL at this point.
As shown in the figure, the trend of RMSE-measured and AUC-measured prediction performance, as well as the recovery rate, are basically the same during the training process: The model perfor- mance and recovery rate improve sharply when the training sample size in the second stage reaches 10 (in total of 32 million) and improve again when the training sample size reaches 107 (in to- tal of 41 million). This suggests that MDLformer requires sufficient data to accurately estimate MDL,, highlighting the importance of large-scale training corpus. Furthermore, when the data size matches that used in E2ESR (Kamienny et al.) 2022) (38.4 million), SNIP (Meidani (56.3 million), and NeurSR (Biggio et al. 2021) (100 million), the recovery rates reach 42%, 66%, and 67%, respectively, which substantially higher than those of the baseline methods (j (110%). This demonstrates that the advantage of our approach lies in shifting the search objective from accuracy MDL instead of the training data volume. Finally, the model begins converge at a data size of 60 million (in total, the same below), and the performance does not differ much until 131 million. This indicates that further improvements in future work may require optimizing the data generation process rather than simply increasing the data volume.
al. 2023)
E.2.4 CASE STUDY
In this part, we provide more case studies to demonstrate the optimal substructure in MDLformer's estimation results and discuss how these results determine the success or failure of the SR4MDL. algorithm it guides.
In Figure 1 we provide typical example in the Strogatz dataset the Gilder-2 problem, where our method efficiently recovers the target formula in a few seconds. As shown in the figure, as the transformation TV iteratively maps dg = J to d3 = y, the MDL. of (d,, y) estimated by MDL.former monotonically decreases. At each point of d, all possible symbolic transformations T lead to an estimated MDL (T((((d). y) higher than the correct transformation Mp. (T (d:). : y). suggesting that there is incorrect search path. This explains why method only takes few seconds to recover this formula and achieves a significant recovery rate improvement on the Strogatz dataset.
In Figure 14 we further provide two examples on the Feynman dataset, illustrating the performance of our method on the problem of successfully discovering the target formula (left) and on the prob- lem of failing to discover the target formula (right), respectively. As shown in the left plot, for
26

---

Published as a conference paper at ICLR 2025
25
70%
0.8
Rocovery Rate
00%
00%
RIME
15
RMSE
New
Clrs
AUC
30%
20%
SNIP
0.6
3114
52M
41M
13TM
31M
30M
41M
T3TM
Number of training samples
Total training samples
Prediction performance of MDL in different numbers of training samples The figure only shows the second stage of the training process. 31M is the number of samples used the first stage of alignment training. For the convenience of comparison, we also mark the number of training samples used by recent works, including NeurSR which uses 100 million samples (Biggio al. (021). E2ESR which uses 38.4 million samples (Kamienny et al. 2022), and SNIP which uses 56.3 million Meidani et al. 2023).
Figure 12:
y (6(1)))) =x, cosGeg)
Estimated Minimum Description Length
(0.0%)
3id Ies girtt oo
dv
[ramen give
2)4 (n.costee) (o aatee
(0.0%)
3)4, m2ett2) [r] aehi
Ti
Estimated MDL of ground-brth
da
Estimated M all possible T
search
(x3de) Ratio of with lower estimated MDL than T.
Step
Figure 13: A case study of the Strogatz dataset. Here we consider the Gilde-2 problem, a typical example where our method successfully recovers the target formula few seconds.
27

---

Published as a conference paper at ICLR 2025
Feynman 1.18.4 where method successfully recovered the target formula, the estimated min- imum description length (MDL) monotonically decreases as the transformation gets closer to the target formula (shown by the red line), and the transformations other than the correct Tç result in a larger MDL. (shown by the white sectors). Transformations resulting in MDL. lower than cor- lead to incorrect search paths (shown by the blue sectors). However, there are only a few incorrect transformations (annotated by the percentages), explaining the successful recovery for this problem.
rect transformations
On the other hand, for the Feynman 111.10.19 problem shown in the right plot of Figure 14 where our method failed recover the target formula, the estimated MDL does not decrease monotonically.
Specifically, it fails to notice that T Bg + B. and Tg B B B3 + / B3 B3 Bद parts of the target formula. This may be because the square and root operations significantly change the numeric distribution in d, leading the same reason, at steps 1, 2, (square operations), and 6 (root operation), a large number of transformations (13.0% 66.9%) result lower MDL than the correct transformation, leading to many incorrect search paths. This explains the failure of our method on this problem and suggests that to further enhance our method, we have improve its robustness to these non-linear operators that significantly influence numeric distribution.
decrease in MDLformer's predictive performance. For
myT + m2?
y = (my m2
my + mig
y (m, Bz) )==
de
15
d1
d.
(4.0%)
73
(0.2%)
da
20
d2
(29.8%))
Estimated Minirum Description
(0.0%)
12
Estimated Minimum Description
(66.9%)
113 (13.0%)
da
(2.9%)
12
d4
0044- (at lehtn (ara)
mil gioo aghie 4) Dours 8 mattttt i mil gnd
(Mu,
10.5
(0.5%)
30 (muunumu
09 (o..8.8j..8) R. I 12h0e (at
d
00.0%
114,
5) (0ttrs molo2/0m, in mall e a on
714.
m [Hr 1g) gawe
0%)
30
[s. 8g.Pt]
Estimated MDL grounc-truth?
4)4.
[0, + 4-4g_95] p
All lower MDl than innerted search palt)
MDL
50
(ox
B2] g
[ou.][92 43 ag
lower estimated MDL 7?
714. (rx <ep ug xF3 o 18h (4
Step
Step
A case study of success and failure examples in the Feynman dataset. Here consider the Feynman 1.18.4 (left) and Feynman III. 10.19 (right) problems, two typical examples of successful and unsuccessful recovery by our methods, respectively.
Figure 14:
28


Reviews:
[{"review_id": "5cuIwU35z1", "reviewer": "Reviewer_LjPc", "summary": "This paper proposes a search objective based on the principle of minimum description length, which aims to streamline and enhance symbolic regression tasks. The authors generate a targeted training dataset and employ neural network techniques to develop a symbolic regression function. By using neural networks in this way, the approach is intended to improve both the efficiency and effectiveness of modeling complex data relationships through symbolic regression.", "questions": "1. In Table 1, the second-best methods are DSR (Petersen et al., 2021) and AIFeynman2 (Udrescu et al., 2020). These methods from 2020 and 2021 not only outperform those from 2022, 2023, and 2024 significantly but also achieve results up to four times greater. Could this discrepancy be due to variations in experimental settings?\n2. The proposed method is trained on an extensive dataset (1.3B), while the other methods were trained on considerably smaller datasets. Could an ablation study be conducted, where all methods are trained on the same dataset to enable a more direct performance comparison?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. This paper is well-written, with the authors presenting the algorithm process and their perspectives clearly and comprehensively.\n2. Figures 1 and 2 effectively illustrate the authors' viewpoints in a clear and intuitive way.\n3. The proposed method demonstrates efficiency and achieves state-of-the-art performance compared to baseline methods.", "weaknesses": "1. This paper demonstrates limited innovation. The proposed method comprises three main parts: the first part provides a conventional description of neural network structure; the second generates a dataset, but the symbolic formula generation method follows that of Kamienny et al. (2022). The third part introduces two loss functions in the training process, with the first being the standard mean square error. Overall, the paper offers little originality.\n2. Could larger datasets be used for experimentation? The datasets in this study are relatively small, such as Strogatz (14 problems) and Feynman (119 problems).\n3. The proposed method relies on training with a substantial dataset (1.3 billion), while other methods have not been trained on datasets of this scale. This raises the question of whether the method’s effectiveness is largely attributable to the extensive volume of training data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1DvxDWIVIl", "reviewer": "Reviewer_1vTX", "summary": "Existing symbolic regression frameworks operate by searching through the space of functions as vectors, minimizing a norm on the input-output behavior of the function. This paper instead approximates a search on the syntactic space of functions by computing a proxy metric for the minimum description length of the function necessary to take a given set of subcomponents and compute the full function. To do this, a synthetic dataset of functions is created and used to train a model that predicts a matching between functions and I/O behavior. The same dataset is then used to finetune this model to predict minimum description length. Overall, the model generally outperforms existing baselines, with particularly good relative performance in the case of noise.", "questions": "You say you sample D in line 215, which makes sense, since you need D to create the formula. However, you again sample D in line 221. However, here, shouldn’t D be the dimensionality of f?\n\nI assume that the auxiliary loss on line 252 is computed per-batch? If so, this should be noted. If not, I would like more details on how this is done.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "This paper has a very elegant approach to the problem of symbolic regression, taking a principled approach towards the search process. Despite the theoretically intractable nature of the objective function selected, in practice it seems that a transformer is able to learn it well enough to make this search strategy better than existing strategies relying on end-to-end behavior. The resulting framework is more robust to noise than existing frameworks and more generalizable to a variety of domains. The results seem quite strong, with the only unfavorable result being underperformance relative to AIFeynman2 on the Feynman dataset in the absence of noise, which seems like a fairly narrow and understandable failure case.", "weaknesses": "Firstly, the datasets being evaluated on seem relatively small and thus potentially prone to being covered by the dataset. However, this seems to be a relatively standard convention that the related work follows, so I am not particularly concerned about this.\n\nSecondly, the example in Figure 6 seems to suggest that there are very few alternate formulas that could be selected, and yet the recovery rate of this algorithm on the Feynman dataset is below 50%. Is this example atypical or is the search tree large even when heavily pruned? An explanation either way would be helpful (e.g., raw fractions for what the 4%, 0.2%, etc., are might clarify this if e.g., 4% is still 100 possibilities).\n\nMInor:\n\nFigure 6 and description: the blue sector description should probably say “all T that are lower according to the model than T^*” and emphasize that these are incorrect search paths -- it is a bit confusing to read otherwise. The caption does this better than the label on the  graph itself or the paragraph describing the figure.\n471-472: “finding that nearly few” I think you mean “very few” or “nearly none”.\n482: “for each of the three wheres” I think you mean “for each of the three strategies”\nFigure 7: there are no axis labels for the right axis for d e and f.\n\nExtremely minor nit: for figure 7 when you have two axes you color the left axis labels black and the right axis green, while the lines are blue and green. The axis labels should either both be black or blue and green.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Z7WYHD8EKT", "reviewer": "Reviewer_s6CS", "summary": "This paper addresses a fundamental challenge in symbolic regression: the traditional search objective, namely prediction error, is non-monotonic, which limits the effectiveness of the search process. The authors propose a novel approach by training a neural network called MDLformer to tackle this. This model introduces a new objective function to efficiently guide the formula search. The experimental results demonstrate that the proposed method significantly outperforms existing approaches.", "questions": "- The data contamination problem should be further discussed. The authors generate 131 million data points, which poses a potential risk of overlap between training data and testing data. Does the generated data introduce contamination problems, and how can this problem be avoided in the data generation process?\n- Some related studies should be discussed. Many recent studies [1] focus on adopting neural networks for symbolic reasoning. The authors should provide a brief discussion about the differences and advantages of the proposed method.\n\n[1] A-NeSI: A Scalable Approximate Method for Probabilistic Neurosymbolic Inference. NeurIPS 2023.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The overall idea of this paper is intuitive and reasonable. The traditional search objective, i.e., prediction error, cannot effectively capture the distance between the target and current formulas.\n- The experimental results are promising. The proposed method significantly outperforms existing methods, demonstrating that a more sophisticated objective can help improve the recovery rate.", "weaknesses": "- The evaluation dataset is limited. Unlike existing studies, this paper aims to introduce a neural network to guide the search process. As a result, it is crucial to comprehensively assess whether this model can generalize well on other tasks. However, the two datasets used in this study are relatively small. For instance, the Strogatz dataset only includes 14 problems, which is insufficient to effectively capture the performance of each method. Therefore, incorporating additional datasets [1] is necessary to thoroughly evaluate the proposed approach.\n- The proposed MDLformer model requires an in-depth discussion. Firstly, the method generates synthetic data to train the model. How do the authors ensure that this synthetic data generalizes effectively across various downstream tasks? Secondly, with 131 million data points generated, how does the model’s performance change as the data scale increases, and does it eventually reach performance convergence?\n- The technical section should be clear and self-contained. Although the concept of Minimum Description Length (MDL) originates from existing studies, further clarity is needed on its technical details, such as how it is computed in the proposed method. At present, the technical content is somewhat difficult to follow.\n\n[1] A Comparison of Recent Algorithms for Symbolic Regression to Genetic Programming. Preprint.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)