Automated Model Discovery via Multi-modal & Multi-step Pipeline
Authors: Lee Jung-Mok
Nam Hyeon-Woo
Moon Ye-Bin
Junhyun Nam
Tae-Hyun Oh
Venue: NeurIPS 2025 poster ()
Keywords: Model Discovery
Time-Series Data
Vision Language Models
Model Selection
LLM Reasoning
Primary area: deep_learning

Abstract: Automated model discovery is the process of automatically searching and identifying the most appropriate model for a given dataset over a large combinatorial search space. Existing approaches, however, often face challenges in balancing the capture of fine-grained details with ensuring generalizability beyond training data regimes with a reasonable model complexity. In this paper, we present a multi-modal \& multi-step pipeline for effective automated model discovery. Our approach leverages two vision-language-based modules (VLM), AnalyzerVLM and EvaluatorVLM, for effective model proposal and evaluation in an agentic way. AnalyzerVLM autonomously plans and executes multi-step analyses to propose effective candidate models. EvaluatorVLM assesses the candidate models both quantitatively and perceptually, regarding the fitness for local details and the generalibility for overall trends. Our results demonstrate that our pipeline effectively discovers models that capture fine details and ensure strong generalizability. Additionally, extensive ablation studies show that both multi-modality and multi-step reasoning play crucial roles in discovering favorable models.

Automated Model Discovery via Multi-modal & Multi-step Pipeline
Lee Jung-Mok'
Nam Hyeon- Woo!
Moon Ye-Bin
Junhyun Nam?
Tae-Hyun Oh
Dept. of Electrical Engineering, POSTECH
Samsung Electronics
School of Computing. KAIST
[jungmok, hyeonw nam, n yonooo Setpooonn@pottechhheo ac.kr, junh. nan@samsung com, taehyun owtkaist. ac.
Abstract
Automated model discovery is the process of automatically searching and identi- fying the most appropriate model for given dataset over large combinatorial search space. Existing approaches, however, often face challenges in balancing the capture of fine-grained details with ensuring generalizability beyond training data regimes with reasonable model complexity. In this paper, present multi-modal & multi-step pipeline for effective automated model discovery. Our approach leverages two assion-language-bin baadeeadeeeeeddd modules (VLM), AnalyzerVLM and EvaluatorV LM,, for effective model proposal and evaluation in an agentic way. AnalyzerVLM autonomously plans and executes multi-step analyses to propose effective candidate models. EvaluatorVL.M assesses the candidate models both quantitatively and perceptually, regarding the fitness for local details and the gen- eralibility for overall trends. Our results demonstrate that our pipeline effectively discovers models that capture fine details and ensure strong generalizability. Addi- tionally, extensive ablation studies show that both multi multi-modality and multi-step reasoning play crucial roles discovering favorable models.
Introduction
Model discovery aims to identify the optimal model structure and parameters that best represent given data. Historically, model discovery has been conducted manually by scientists 44 41 and has established groundbreaking advances in science and technology that have shaped our understanding of the world. However, the complexity and scale of modern datasets continue to grow, the feasibility of manual model discovery has become increasingly limited [I] 46 58). Automating this
38 46) offers the potential to accelerate the scientific progress by reducing reliance on human experts and efficient exploring of complex model spaces.
process 7 [1 23
33
Nevertheless, automatically finding the appropriate model structure is inherently challenging as it requires: 1) exploring over a vast combinatorial search space of candidate models, and 2) balancing be- tween interpretability and model fit, ensuring the model captures the data accurately while remaining sufficiently simple and understandable domain experts. Most existing systems were carefully designed to address these challenges. For instance, [14 and KK proposed predefined grammar for kernel composition within Gaussian processes, which structured the search process and reduced the manual effort required determine kernel composition. Also, the objective function relied on predefined quantifiable metrics for the model selection, rather than dynamically adapting
29
Corresponding Author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Model Proposal
2. Model Fitting
Model Evaluation Conpute score based on visuel criterion and BIC
Let's try visualization!
Find best paraneters for X (period: 0.35)
Analyzer VLM
model, data acceset) visuaizatoonmmodel data)
LIN variance: 0.3
JW W
LIN affsct :0.22
Code
period :0.15
Output
LIN varianoe:
Model
Data
LIN affect :0.4
Model
Data
penaod :0.35
It seers linnar & periodic. Let's check period!
Good fit. 8/18
Evaluator
Bad structure sin. 2/10
4. Model Selection
VLM
peñodd prinifperio))
Code Output
Select higgee-sccoee model
EvaluatorVI ((M,)
(6.2 0.8)
period: 0.35
Model 1 (score:90)
recommend
Visual Selection Critorion EvaluatorVIN BIC
LIN x PER (period: 0.35)
Model 2 (score: 70)
& multi-step pipeline contains 4 stages: model proposal, model fitting, model evaluation, and model selection. During the model proposal, AnalyzerVLM repeatedly analyzes the data until it determines the results are sufficient. Then, it proposes model structure with linearity(LIN) and periodicity(PER) then the model fitting is conducted. In the model evaluation stage, EvaluatorVLyy assesses models visually and utilizes the score in the subsequent model selection process.
Figure I:
Overall pipeline. Our
criteria as a human expert would. However, designing the balanced and sophiscate procedure for composing models requires significant modeling expertise, which compromises the flexibility needed for automation.
To achieve more flexible and intelligent model discovery system, substitute human experts' roles in the model discovery with multi mudti agents. First, we introduce AnalyzerVLM, an agent that is capable of conducting in-depth analysis of given data and models through multi-step reasoning. We task AnalyzerVI. VLM LM with generating code for analyzing the current model and proposing improved candidate models based on its findings. We observe that AnalyzerVLMM actively uses existing libraries such as NumPy 442) and Matplotlib [21]. which are commonly used by humans for analysis and visual inspection of plots. AnalyzerV iteratively analyzes the current model and data until identifies candidate models that can outperform the given model.
Second, propose Visual Information Criterion (VIC) performed by EvaluatorVLM. Our new criterion is designed to incorporate the way humans perceive a given model. Traditional criteria, e.g., Bayesian Information Criterion (BIC), quantify both goodness-of-fi fits and model complexity to identify undesirable models, but we found that they are often counter-intuitive and fail to consider inherent data characteristics and relationships. On the other hand, humans easily identify inherent data and model trends, structures, and patterns through visualization. By introducing visual representation into the model selection criteria, our VIC complements traditional criteria, enabling the discovered model to escape suboptimal. By alternating the AnalyzerVL.M and Evaluator VLM, we demonstrate that our multi-step and multi-modal pipeline effectively searches over the model space like human, enhancing the proposed model's 's quality and showing high generalizability compared to other model discovery methods.
Related Work
Automated Model Discovery. Model discovery typically includes tasks such as selecting the model structure, estimating parameters, validating the model's effectiveness, and refining the model. The discovery process requires extensive expertise across multiple data domains and involves multiple steps. Thus, model discoveries require human experts to repeatedly refine the model and reflect the data domain's 's prior knowledge. To alleviate these difficulties, automated model discovery processes have been proposed. The goal is to reduce the burden of needing human experts and prior work has been developed by defining model space on Gaussian process [ 6
27 33 35 Such methods are based on kernel structure search with pre-defined grammar, iteratively evolving its kernel structure. Automatic model discovery pipeline often uses the greedy-searde seerchh based approaches [14 33 sampling-based approaches 44 45), and employing large language models (LLMs)) [30] Also there
[4] 15]
22
26
similar approaches in symbolic regression
19 25
28
31
59
60
69
which searches

---

for the function composition that most fits to the data. Given basis function and the operators, it automatically finds out the appropriate function composition over evolving. Symbolic regression requires diverse function candidates to be generated and takes the evolving over the generation, it mostly utilizes genetic programming (2 19 60) 69]. or neural networks such as transformers [5] 24 31 551. These days symbolic regression utilizing LLMs [16 39 48 [T]. Thus, utilizing LLMS is promising in the automatic model discovery pipeline, as LLMs can be adapted to those with broad expert knowledge by providing suitable prompt. In this work, we extend the usage of LLMs to VLMs by effectively harnessing the reasoning, planning, and evaluating capabilities with visual modality.
LLMs & VLMs-based Data Understanding. Interpreting and understanding current data is crucial for model discovery and prediction, as extrapolation regimes including future events often depend on past patterns. [17] show that L.M has the ability to effectively deal with time-series data if the numeric tokens are properly designed. 13 propose to use plot visualizations instead of lengthy texts. 30 simply investigate the potential of visual plots as a replacement of text representations to reduce the number of tokens, and shows the comparable alternative to the text inputs. Like such methods for understanding data, there are also automatic data-driven discovery frameworks [37 70 which automatically analyzes and finds the relation between variables of the data. Our method leverages VLMs to gain a deeper understanding of the given data within the context of model discovery. We let VLM identify and understand visual data itself by effectively overviewing the trends and relationships.
Model Agents. LLMs and VLMs, such as GPT-4o (. have high reasoning and generalization capabilities. These strengths make them particularly well-suited for use as agents in complex tasks [0
62, 65-68). Many recent works have utilized L.LMs and VLMs for data In addition, multi-step 65 are commonly employed to enhance the decision-making and anal- ysis capabilities of LLM-based agents. We leverage this advanced capability of VLMs to accelerate and enhance the model discovery process, facilitating the efficient identification more robust and accurate models. Specifically, VLMs analyze the data and candidate models by adaptively generating codes to identify potential improvements and evaluate model accuracy through the interpretation of visual plots. By integrating these components, our approach fully capitalizes the strengths of VLMs to optimize the model discovery workflow.
20
61
analysis 20
53
63
as well as LLM/VLM for evaluator 9 18 32]
reasoning techniques 66
62
3 Method
3.1 Overview
Automated model discovery aims to identify model structure M and the corresponding parameters 0 that best describe a dataset D. This process systematically identifies optimal model structures by exploring structured search space while balancing complexity and fitness. Our pipeline discovers proper model iteratively, and it has 4 steps in each round: model (I) proposal, (2) fitting, (3) evaluation, and (4) selection, as shown in Fig. and Algorithm
The model proposal step of the r-th round needs to suggest better model candidates M = MM.......... given data DD and previous mod-
Algorithm 1 Model Discovery Pipeline.
1: Input: dataset D, rounds R. model pool PP
els 1 To effectively suggest candidates, we propose an AnalyzerVIMM which designed to propose model candidates through multi-step analysis. Further details about AnalyzerVIn VLM
2: Initialize: best model M
3: for 1 to R do
4: M" AnalyzerVLM(, D)) > Proposal
can be found in Sec. 3.2
5:
for M E M" do
Once the candidate models are suggested, they are conveyed to the model fitting step, where we determine the optimal model parameters through marginal likelihood- based optimization. To ensure robustness, we conduct the parameter optimization with multiple initialization points with AnalyzerVL.M proposal, incorporating ran- dom restarts and inheritance from model candi-
6:
0* = Optimize D)
> Fittinn
7:
SM a EvaluatorVLL
() > Evaluation
BIC
8:
end for
9:
t- PUPU
M t- arg MaX S.M
b Selection
11: end for
3

---

dates 14, 23] The parameter with the highest likelihood is then selected as the best for subsequent model evaluation.
The fitted model is evaluated using Visual Information Criterion (VIC). To assess the model in multiple aspects and perceptually plausible ways, we propose Evaluator VLM that measures visual scores: visual fitness and generalizability. The visual score is combined with the traditional measure, Bayesian Information Criterion (BIC). These criteria allow us to assess the model both in detail and holistically. Additional information about our visual scoring is provided in Sec. 3.3
Finally, scored models are put into the model pool P. The model pool is then sorted based on the VIC. In the next round, reference models are sampled from the updated pool. This round is iterated until the number of iterations reaches the maximum criterion.
3.2 AnalyzerVLM Multi-step Analysis
The reasoning process of AnalyzerVLM can be formalized as sequential decision mak- ing task defined by a policy [{ar(e)). where 1,0)) 0+) represents the context, a, E denotes the action, and o E O is the observation. Our multi-step analysis inte- grates the interaction process through the code- execution, getting observation for the ex-
Algorithm 2 Model Proposal with AnalyzerV
1: Input: prompt P. model M, dataset DD Ana- lyzerVLM, max context length Nmux 2: Initialize: c (P, (,, D,M,, step count i
(01, ai, 0 1,
3: while |c:] < Nmax do
4:
AnalyzerVLM(cs)
5:
if a; E L then
ecuted output. When there is no observation
6:
C++1 (q,aa)
Analyze
(when only reasoning happens), the context up- dates simply as C++1 = (c,, o;). When code ex-
7:
else if a, E C then
8:
= Execute(a.)
Execute
ecutions are available, the action space expands
G+1 f (c,, a, o;) to A = L C.C, C, enabling the agent to execute 10: else if a 2d then tool-related actions through the code execution.
9:
11: return a;
> Propose
If the agent selects an action E C, the interac-
12: end if
tion with the environment happens, producing an observation O2 = Execute (a)). This observa- tion is appended to the context, resulting in the updated context C++1 (Ca, 0g, 0z).
13: i=i +i+1
14: end while
At the initial step, AnalyzerVLMM LLM is provided with a prompt P that specifies task and objective, dataset DD, and candidate models P from the previous rounds. These inputs define the initial context ci. Starting from C1 AnalyzerVLM iteratively selects actions a; based a fixed policy (aalc) and finally produces or C Σ, which represents the candidate models for the next stage of model discovery. The overall algorithm is shown in Algorithm 2 The action space of AnalyzerVI.M consists of three subspaces: a language space L for the natural language reasoning process, a code space C for generating executable code to perform analysis, and a model space 2dd for generating candidate models. Since Analyzer VLM proposes multiple candidate models, the model space is represented as a power set of the search space Σ. Each specific action described as follows.
Analyze. When AnalyzerVLM performs analysis at i-th step (i.e., ag E L), it means that will generate the analysis and formulate new plan for the next step in natural language, based on the current context e. As the analysis and planning are conducted entirely in natural language, the az).
resulting analysis can be directly incorporated into the next context C2+1
Execute. When AnalyzerVLM chooses to execute code (i.e., a, E C), it generates executable code block in the python language. The generated code block is executed, and the resulting observation OF Exccute(a,) is used to update the context for the next step c = (c,, dz o,). We highlight that the observation O; is not limited to textual or numeric outputs, but can also include visual representations (e.g., plots), enabling AnalyzerVLM to perform the diverse analysis of the given data into the most-fittable format for it.
Propose. When AnalyzerVLM has sufficiently analyzed the data and model, AnalyzerVL.M chooses to perform model proposal based on the previous contexts e; (e.g., analysis) When the propose action is conducted, the multi-step analysis is terminated. When AnalyzerVI.M proposes the model structure, it also propose the initial parameters based on its analysis. Based on its iterative analysis,

---

can effectively propose the initial parameters for the model structure (e.g., period), and utilizing those parameters properly can enhance the good structure's good fitting.
Compared to the BoxLM (30] proposes the candidate model at once, our AnalyzerVI.M utilizes multi-step reasoning that acquires the sufficient information by itself propose the model. Also, our Analyzer VLM dynamically chooses the way to look at the data and model, giving degree of freedom to AnalyzerVLMM for deep analsyis. With the multi-step pipeline, our model suggestion can effectively identifies missing characteristics in the current candidate model in various ways.
3.3
WraluatorVL.M: Visual Information Criterion
From a Bayesian perspective, model selection is grounded in the marginal likelihood, which measures how well a model explains the observed data while integrating over all possible parameter values:
P(DIM) - p(D)MM ()p(0)Mdd
The parameter optimization performed in the model fitting step of automated model discovery can be interpreted as practical surrogate for intractable marginal likelihood computation. A widely adopted approach to approximate marginal likelihood is using Laplace's 's method around the maximum likelihood estimator 6", leading to the Bayesian Information Criterion (BIC):
BIC(M, -2 -2 p (DIM, 6*) + [M| (D),
where MM is the number of model parameters, and [D] is the size of the dataset.
While marginal likelihood naturally balances model fit to the train data and complexity, it may struggle to generalize to unseen regions [34]. The limitations of marginal likelihood in evaluating generalization stem from its inability to capture the structural properties of data that persist across both the training and test regions. Inspired by this, we propose a novel model selection criterion by utilizing VI.Ms identifying suitable models. Specifically, we feed a visualization of the posterior predictive results to Evaluator VLM and evaluates how well the model suits the data. This evaluation score is then incorporated into our model selection process with the model evidence.
To integrate VLM-based judgment into the model selection, we propose the Visual Information Criterion (VIC). We define VIC weighted combination of the EvaluatorVLMM score and BIC:
VIC(M,
EvaluatorVLLM 6* D) BIC.
Since BIC is traditionally lower for better models, we negate it ensure that higher VIC values indicate better models. Despite its simplicity, VIC effectively approximates the posterior model probability p(MlD) by incorporating an additional prior term p(M)), extending beyond the standard BIC-based approach. The detailed derivation of this approximation can be found in Appendix A.2
Our VIC evaluates two aspects of the model: 1) Visual Fitness and 2) Visual Generalizability The reasons of considering both aspects are that 1) visual fitness represents how well the discovered model fits to the data's trend, and 2) visual generalibzaility represents the consistency of trends across training and non- non-training regions, which can be one way of measuring model's generalization. Each score measured multiple times and is averaged due to the stochasticity of VLMs. Then VIC is computed by summing up all the following visual scores.
Visual Fitness. Visual fitness quantifies how much the data resembles the prediction visually. Specifically, fitness evaluated through comparing two plots: Given the data observation plot and the predicted posterior mean plot, EvaluatorVLM is prompted to compare two plots to measure the similarity of the prediction and data. We also evaluate visual uncertainty, how large the uncertainty region is given a predicted posterior mean and confidence region. If the uncertainty region is big, we let EvaluatorVLM give a low score, and if the uncertainty region gets suddenly large in non-train non-training data regions, a low score is given. We prompt EvaluatorVLMM to score each score in the range of [0, 50). Through the summation of mean prediction resemblance score and the plot uncertainty score, calculate the visual fitness. The prompts shown Appendix(A.1l 10
Visual Generalizability. Visual generalizability quantifies whether the predicted posterior mean preserves it structural consistency in extrapolated regions beyond the training data. Given plot
5

---

Table I: Quantitative results. We compare our pipeline with five competing methods on the train and test region, reporting RMSE. On average, our pipeline outperforms the others. Bold stands for the best, and underline for the second best.
Datuset
Method
Aifline
Solay
Maxina
Whoot
Call
Rodio
Gas
Test
Canosiae Procese (8P)
17
0108666
01:412
0.0594
0.6589
0.0130
0,0732
04/931
0.5676
0.0195
0.3120
0.9426
0.1835
0.0468
0.7605
0.0421
0.5121
ARIMA
+
0.2513
0.1583
0.2400
0.3139
0.2955
0.1758
0.1638
0.1537
0.2788
0.6291
0.3756
0.5828
0.3158
0.150
0.2151
0.2827
Pacchack Froplen
53
0.2576
0.1648
0:2218
0.3924
0.2996
0.0646
0.166
0.1268
0.3676
0.888
0.3342
0.666
0.3154
0.182
0.2630
0.3536
Aupomatt: Sulistoan
0,0156
0.2008
0/8355
0.630
0,003
1U025
00640
0.149
0.029
03197
RU053
0.556
0.0144
0.822
00254
0.3017
BodLM
0.3545
0.311
0.452
0.0255
0.312
30114
0.0073
0.2366
0.2075
0.096
0.3474
Oua (Owen2.5
0.469
0.2911
0.069
0.1123
Oun (CPPE44)
0.0057
8,839
0.099
0.2861
IUW25
ILA49
0.0086
0.1316
0.6885
0.442
01366
0,074
0. 1034
0.641
0.041
1115
Ours (GPT-4o-sio))
0U0066
0.534
08291
0.2861
0.003
100564
9.133
0.1470
0.001
00054
00602
9,007
0.883
0.134
0.1070
ARIMA
30 Facebook Prophet
Gaussian Process (SE)
Mwww
100M00 MM
MMM Mob
0.8
Radio
1946
2023
2240
2003
1246
2303
Autostats
BoxLM
Ours
www
mm
0.2
1226 Year
Figure Qualitative results. The graph on the left illustrates the data, where black line stands for the observed data and red line for the test data. The graphs on the right show the predictions of each method at the shaded region of the left graph, with the blue lines representing their outputs. The results highlight the generalization capabilities and efficient search performance of our method.
of the posterior mean and its confidence region, SyaluatorVLM is prompted to measure whether the model's predictions maintained in the extrapolated region in the range of [o, 50), without requiring ground-trr labels in extrapolated regions. This allows us to quantify how well the model generalizes beyond the observed data distribution. Specifically, we have visualized the posterior mean and confidence at the 20% extrapolated region for each side, and instructed Evaluator VLM to check whether the posterior mean flattens 2) confidence region suddenly increases at extrapolated region.
Experiments
4.1
Gaussian Process Kernel Discovery
Datasets and Competing Methods. We evaluate our multi-modal & multi-step pipeline on real- world univariate datasets (33, including Airline Passenger, Solar Irradiance, Mauna Loa, Wheat, Call-Center, Radio, and Gas Production. We will refer to the data by their representative terms for convenience, e.g., Solar for Solar Irradiance. And we compare our pipeline against five competing methods ranging from traditional forecasting methods to the latest LLM-based model discovery approaches: Gaussian Process Regression with Squared Exponential kernel, ARIMA [52] Facebook Prophet [54]. Automatic Statistician [14 [33) and BoxLM B0 We employ GPT-4o-min for methods using LLMs, including ours. Additional experimental details (e.g., basis kernels and kernel grammars) are given at the Appendix A.3
Result. As shown in Table Ref tab:quantitative, our method can discover better models by achieving consistently lower RMSE compared to other methods on average. While BoxLM and Gaussian Process (SE) exhibit low RMSE values the training set, their RMSEs significantly increases in the test region, indicating poor generalization. In contrast, our method maintains consistently low RMSES across both training and test sets, highlighting its ability generalize effectively beyond the training data. This shows the robustness of our method in the test region through Analyzer VLM and Evaluator VLM.
"The prompt is shown at the Appendix A.10
'The authors used GPT-4v, but for fair comparison with ours, we update the LLM version with GPT-4o-mint 'Note that our main result refers to the one using GPT-4o-mini. We have varied the VLM into Qwen2.5-VL. GPT-4o, and GPT-4o-min mini for the ablation study, shown at Section 4.2

---

64
mrrded (Tht only] Mall modal Va.l
AnalyzerLLM M
AnstyaerVLM
Single stop
Nubi PICP
0.4
0:00
0.2
.60662
0.64
-00071
0.0 Datacet
0.00
Datoseet
Dotaset
Figure MSE of text only and multimodal representation. We re- strict the usage of visual represen- tation during model generation and evaluation processes. The blue value indicates the reduction in MSE when using visual representation.
Figure 4: The required step num- ber LLM and VLM. When us- ing LLM instead VLM for An- alyzer, i.e., without visualizing the data and model, Analyzer requires more steps validate its analysis.
Figure 5: MSE of the multi-step and single-step restriction. We limit AnalyzerVLM to a single step. Blue values indicate the reduction in MSE when using multi-step analy- sis compared to single-step analysis.
Step
Stop2 Hostly follows the data, but the end. Check residusl.
Step 8
Stops 4
Step 5 analysis. Propose model with periodisity.
Anukyaa
Given the codk. vieualize data and model.
Residual at Letter years. Check autocormaation.
first lags, temporal feature. Visuslize periodicity.
Excoute
Model Proposal
Figure 6: Multi-step analysis of Analyzer V.M. We visualize example of the AnalyzerVI.MI multi-step analysis. Analyzer VLM begins with the data and model visualization, followed by visualizing residuals and identifying the remaining periodic pattern the data before model proposal. Finally, AnalyzerVLM proposes a refined model based on the analyses.
We visualize the model discovery result in Fig. 2 ARIMA struggles to capture trends and periodicity in the test region. The Gaussian Process successfully captures the overall trend in the train region but does not fully account for periodicity and finer details. Similarly, Autostats and BoxLM exhibit slight deviations from the trend and miss some finer patterns. Facebook Prophet effectively captures both trend and periodicity but falls slightly behind our method in capturing finer details. However, our method identifies a plausible model that generalizes well to both observed and unseen test data To further investigate the reasons behind the strong generalization performance, we conduct ablation study in the following section.
4.2 Ablation Study and Analysis
AnalyzerVLM and EvaluatorVLM.. We conduct an ablation study on the VLM-based modules in our pipeline to examine the impact of AnalyzerVLM and Evaluator as shown in Table
In experiments without AnalyzerVLM, place it with simpler model proposal approach for VLM without analysis, providing just visual- izations of the model and data while prompting the model generation following BoxLM (30)
re-
Table 2:
Ablation study on VLM-based modules in our pipeline. We conduct an ablation study Analyz- erVLM and Evaluator VLM in terms MSF. The last row stands for our complete pipeline. Bold represents the best, and underline is the second hest.
For experiments without EvaluatorVLM, we exclude the visual criterion and only rely on the BIC for model selection. The results in Ta- ble Ssoow that each component plays a crucial role in discovering better models. Our proposed pipeline with both AnalyzerVLM and Evalua- LL. finds the best model in most cases.
Analyzer VLM
Evaluator VLM
Dataset
Airline
Solar
Mauna
Wheat
0.0810
0.1791
0.2820
0.0216
0.0698
0.0820
0.2863
0.0498
0.0029
0.0822
0.0198
0.0135
0.0028
0.0819
0.0032
0.0216
Also method can be applied across different models including commercial models(GPT), and open-source models. We have varied the backbone models of AnalyzerVLM and EvaluatorVLM,
Table [ shows that our methods can be applied to diverse VLMs, including Qwen2.5-VL [3), GPT-4o-
Additional visualization can be found in Appendix.
7

---

Train HSE
Toua NM
100.0
Visud Filvess
Visuat Generalaasiit
Meuine Low
pi T9
19339,
Maa
Swed
Homan vo VUM -itess
Homanva MLM- Genens.
Gan Prod.
Roverearts
F:0.80
Figure 7: BIC and VIC compari-
Figure MSE of model over
Figure 9: Correlation of VIC
son, Comparing BIC and VIC, VIC
rounds. Train MSE gets lower as
with likelihood, generalization, and human. At first row. we visu- alize the correlation between VIC and metrics (likelihood, general- ization gap). Second row is human corrolation result with VIC.
can sort out the models with low
round goes, but without VIC, test
generalizability, by giving them the worse score. while BIC cannot penal-
MSE may increase over rounds. With
VIC, can effectively lower both
ize such low generalizability cases.
train and test MSE.
mini and GPT-4o M.. Our pipeline can work both with open-source model and commercial models, and it shows that the VLM with high capability has high performance on average.
Multi-modal Single-modal. Our pipeline leverages both visual and text representations during model proposal and evaluation. To investigate the impact of the visual representation, we restrict AnalyzerVI.M to only use text during the model proposal and rely solely on BIC for model evaluation, not giving the model prediction visualization. The result in Fig. 3 shows that incorporating visual representation improves model discovery performance. Using multimodal representation allows to consider the overall trend rather than focusing solely on the local values in the text, leading to a better understanding of the data.
Hyperparameter a selection. We have set a so that EvaluatorVLM accounts for approximately 10- 30% of the original metric (e.g., BIC), and per- formed grid search around this range to find the best. Since the scales of the selection criteria vary due to data normalization, sual score contributes a consistent proportion to the total objective. We provide the result of test RMSE of grid search varying a around 0 to 100, at Airline and Radio dataset at Table 3 As shown, there is a certain turning point (or break-down point) of alpha around 50. While a is O, linal searched models are dropping at extrapolated region, and when increases, the pipeline tends to select the model with more generality even in high value ranges near 70. This hints that our hyperparameter setting is not sensitive and starting from introducing small a would be a good rule of thumb.
Dataset
30
50
70
100
Airline Radio
0.0937
0.0574
0.0534
0,0612
0.0824
0.0766
0,0562
0,0764
0.0954
Table 3: Performance across different values. There is a break down point of performance of a around 50, indicating the trade-off between model accuracy and generalizability.
set such that the vi-
Reasoning Process of Analyzer VLM. Analyzer proposes model candidate through the multi-modal and multi-step analysis. To better understand its reasoning process, we compare the number of analysis steps required when using LLM versus VL.M the analyzer, as shown in Fig. 4 Both analyzers independently determine when to stop the analysis. AnalyzerLLM requires more steps generate proper candidate than AnalyzeVVLM, LLM is less effective than VLM at capturing the overall trend, which leads to more iteration for checking its analysis.
Next, we restrict the number of reasoning steps of AnalyzerVLM to compare single-step and multi- step analysis, observing performance improvements across steps, as shown in Fig. [5] The results indicate that additional analysis steps lead to the discovery of better model. In summary, the overall results demonstrate that the multi-modal and multi-step process enables AnalyzerVI.M to make better decisions. The qualitative result of the reasoning process is shown in Fig. 6
Senerallizabiltty-Awar Model Evaluation. We show two qualitative examples with VIC(1) and -BIC(T scores in Fig. 7 When BIC is superior, it shows deviation at the extrapolated region, but it does not detect such cases and select the right models with the deviation. However, VIC distinguishes such cases, penalizes such models and selects the left models with high generalizability. Specifically at Airline data, as shown in the right-top example of Fig. 7 the model shows a certain drop at the
'Sine BIC is better for the lower value, negate it for the explanation.
8

---

AnalyzorALM Init. IPER LIN)
Random Int. (PER + Llk)
AnatyporMLM M Init: ISE (PER + CP)
Random Init. ISE (PER + CD
Condac ves
Airline
Mauna
Figure 10: Visualization of the optimized model with different initializations. The model optimized from the MulyzerVV.MMpropaae initialization reliably identifies appropriate parameter settings through data-driven analysis, whereas random initialization frequently leads to suboptimal configurations.
extrapolated region, which is not a natural behavior. The left-top model shows the natural upward trend at the extrapolated region. BIC does not have any constraints about evaluating such naturalness, it selects out the right model. A similar phenomenon also happens in the Radio data, as shown in bottom example of Fig. 7 the right model's prediction suddenly flattens at the right extrapolate region and the structure not maintained, which may mean low the generalizability. Penalizing such generalizability, VIC can effectively distinguish such cases. Fig. 8 shows the visualization of the mean squared error of train and test region, for each round. With BIC evaluation, shows low train MSE with increasing test MSE. In contrast, our pipeline keeps the test MSE low and even reduces it across rounds, showing better generalization without overfitting.
Correlation of Visual Information Criterion. The first row's left image in Fig. shows that our visual fitness is highly correlated the likelihood, which implies that we can visually measure the model's 's fitness. The first row's 's right image in Fig. 19shows the correlation of our visual generalizability and generalization gap. Specifically, we adapt the difference of test MSE and train MSE as the generalization gap. As shown, it shows a high correlation with the generalization gap, which means that our visual generalizability is an effective criteria for evaluating a model's generalization capacity.
The second row of Fig. shows the correlation of VIC's each criterion with the human evaluation. Human evaluation is done by instructing human with the same criteria that EvaluatorVLMM visual fitness and visual generalizability. For human evaluation, we instructed human to rate the given model from to 5. The instruction shown Appendix. As shown, the PrrluuatrVL.M model evaluation follows a similar trend to the human model evaluation, showing a high correlation at both visual fitness and visual generalizability evaluation. This shows that EvaluatorVLMM closely matches human judgment at model evaluation, enabling more efficient and reliable automated assessments of the model discovery.
Hyperarameter Initialization of AnalyzerVLM. As reported in 14 and 27 hyperparameter optimization for Gaussian process regression is non-convex problem, making good initialization crucial for effective model discovery. To address this problem, [14] utilized random initialization with hyperparameter inheritance over rounds, and (27) has utilized random restarts with strong prior, sampling the hyperparameters from certain prior distribution. In our case, we utilize AnalyzerVLM propose model structures and suggest initialization point based on its analysis. In particular, we initialize the period and lengthscale of the periodic kernel and the lengthscale of the squared exponential kernel, using values suggested by AnalyzerVLLM, then start optimizing the first round. Then, such well-estimated hyperparameter values proposed by AnalyzerVLM can be carried over rounds, enabling the construction of progressively more complex and refined model structures initialized with strong hyperparameter estimates. Figure 10 shows two examples of optimized models with AnalyzerVLM proposal initialization and random initialization. As shown, with the AnalyzerVLM initialization, the model can be optimized to appropriate hyperparameters. while random initialization fails at finding the appropriate hyperparameters and leads to the whole kernel structure's failure.
4.3 Application to Symbolic Regression
In this section, we extend our discovery pipeline to symbolic regression, demonstrating its applicability beyond probabilistic model classes. Accordingly, the BIC component of our original VIC is replaced with a commonly used objective in symbolic regression: normalized mean squared error combined with complexity penalty (50).
Dataset and Competing Methods. Extending our model discovery framework into the function discovery, prompted AnalyzerVLM to generate a normal function rather than the kernels, and

---

Table 4: Quantitative results symbolic regression. We conduct experiment on symbolic regression at four datasets: R. Constant, Keijzer, Nguyen. We report R R-suurre and RMSE scores for each dataset. Bold represents the best, and underline is the second best. As shown, our method shows competitive results compared to the other methods.
3.0
-12
Constant
Keijzer
Nguyen
RP (7)
RMSE ())
R (1)
RMSE (
(*)
RMSE (4)
(F)
RMSE (})
Figure 11: Qualitative result at
SGA 36
0.8951
0.1639
0.5677
0.1086
0.3661
0.3263
0.8918
0.1261
symbolic regression. Our method
ICSR-V 150
8.9808
0.1320
0.9967
0.0289
0.9463
0.0998
0.9952
0.0801
13 M-SR
0.9717
0.08NS
0.9807
0.0225
0.9972
0.0130
0.9440
0.0340
successfully discovers function
Ours
0,9872
0.1151
0.9503
0,0411
0.9521
0.0322
0.943
0,0671
composition which fits the data.
utilized our VIC for symbolic regression score. Following [39]. we have conducted our experiments Nguyen 660, Constant
[3]] (28), Keijzer 25]. And compared our methods with ICSR (39), LLM-SR [51] and SGA [36]. Those methods utilize LLM, while ICSR-V V also incorporates the visualization of the plot at the function generation phase. The implementation details are shown in
Appendix A.3
Result. Our proposed method achieves performance that is competitive with existing approaches on symbolic regression, as shown at Table 4 Our method consistently achieves consistently high predictive accuracy and reliable generalization. Leveraging the AnalyzerVLM and EvaluatorVL.M modules, our model discovery pipeline can be employed as a symbolic regressor, enabling inter- pretable function modeling with method's iterative analysis and visual evaluation. Fig. TT shows that our approach successfully captures the underlying functional form on the Keijzer3, closely matching the ground truth in the training region and also showing similar results in extrapolated regions beyond the training data. This shows our generalization performance is robust in capturing the data's 'ss structure rather than merely fitting the data.
Conclusion
We propose multi-modal multi-step pipeline for automatic model discovery by introducing two VLM-based modules: AnalyzerVLM and EvaluatorVLM. AnalyzerVLM iteratively plans and ex- ecutes the analysis to propose the most suitable model by analyzing the given data and models. Leveraging large-scale VLM as multi-modal agent, it can generate analysis and interpret plots and data characteristics in context. Through its multi-step analysis, our method can discover better model which fits to the data's 's underlying structure. Evaluator VLM assesses the suggested model based on the visual representation, i.e., plot, evaluating fitness for local details and structure similarity for overall trends. It provides a robust mechanism for model validation beyond numeric error metrics. The experimental results demonstrate that our pipeline effectively discovers proper model, capturing fine details and interpretable model structure while ensuring strong generalizability.
Limitations. Our multi-modal multi-step pipeline focuses discovering the data's 's structures, therefore current pipeline focuses on ID datasets, and the pipeline's ' performance may depend on the quality of input visualizations. So the future work should include extending this to multivariate data to capture relationships between variables, and searching for the good quality input visualizations that fits to VLM.
10

---

6 Acknowledgement
This work was supported by Samsung Electronics Co., Ltd (Project Code: I0240508-09825-0000 T.-H. Oh was partially supported (30%) by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea BMVI)Imen((MSITT (No. 2022-0-00124, RS-222-
11220124, Development of Artificial Intelligence Technology for Self- leff-Impooving Competency-Awan Learning Capabilittiss. T.-H. Oh was partially supported (30%) by Institute of Information & commu- nications Technology Planning & Evaluation (ITTP) grant funded by the Korea (overnment(MSST) (No. RS-2024-00457882, National AT Research Lab Project). T-H. Oh work was supported (40%) by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSTT) (No.RS, ASS-2025-2544331 Physically-groundee
Intelligence: A Dual Competency Approach to Embodied AGI through Constructing and Reasoning in the Real World), T.-H. Oh was partially supported by the KAIST Cross-Generation Collaborative Lab Project.
References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXi::2303.0877 2023.
[2] Douglas Adriano Augusto and Helio JC Barbosa. Symbolic regression via genetic programming. In Proceedings. Vol. 1. Sixth Brazilian symposium on neural networks, pages 173-178. IEEE, 2000.
[3] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, Sibo Song, Kai Dang, Peng Wang. Shijie Wang, Jun Tang, et al. Qwen2. 5-v1 technical report. arXiv preprint arXiv:2502. 2025.
Fabian Berns, Jan Hüwel, and Christian Beecks. Automated model inference for gaussian processes: an overview state-of-the-art methods and algorithms. SN Computer Science, 3(4):300, 2022.
[5] Luca Biggio, Tommaso Bendinelli, Alexander Neitz, Aurelien Lucchi, and Giambattista Parascandolo. Neural symbolie regression that scales. In Marina Mcila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 936-945. PMLR, 18-24 Jul 2021.
[6] Matthias Bitzer, Mona Meister, and Christoph Zimmer. Structural kernel search via bayesian optimization and symbolical optimal transport. Advances in Neural Information Processing Systems, 35:39047- 2022.
[7] Josh Bongard and Hod Lipson. Automated reverse engineering nonlinear dynamical systems. Proceed- ings of the National Academy of Sciences, (04(24):9943-999) 9944, 2007.
[8] Jure Brence, Ljupöo Todorovski, and Saão Džeroski. Probabilistic grammars for equation discovery. Knowledge- Based Systems, 224: 107077, 2021.
[9] Dongping Chen, Ruoxi Chen, Shilin Zhang. Yaochen Wang. Yinuo Liu, Huichi Zhou, Qihui Zhang, Yao Wan, Pan Zhou, and Lichao Sun. Milm-as-a-jue Assessing multimodal tim-as-a-judge with vision- language benehmark. In Forty-first International Conference on Machine Learning, 2024.
[10] Mia Chiquier, Utkarsh Mall, and Carl Vondrick. Evolving interpretable visual classifiers with large language models, 2024.
[11] Kyle Cranmer, Johann Brehmer, and Gilles Louppe. The frontier of simulation-based inference. Proceed- ings of the National Academy of Sciences, 117(49):00055-3006,, 2020.
[12] Miles Cranmer. Interpretable machine learning for science with pysr and symbolicregression. jl. arXiv preprint arXiv:2305.01 582, 2023.
[13] Mayank Daswani, Mathias M. J. Bellaiche, Marc Wilson, Desislav Ivanov, Mikhail Papkov, Eva Schnider, Jing Tang, Kay Lamerigts, Gabriela Botea, Michael A. Sanchez, Yojan Patel, Shruthi Prabhakara, Shravya Shelty, and Umesh Telang. Plots unlock time-series understanding in multimodal models, 2024.
[14] David Duvenaud, James Robert Loydd Roger Grosse, Joshua B. Tenenbaum, and Zoubin Ghahramani. Structure discovery in nonparametric regression through compositional kernel search. In International Conference on Machine Learning (ICML), 2013.
[15] David K Duvenaud, Hannes Nickisch, and Carl Rasmussen. Additive gaussian processes. Advances in neural information processing systems, 24, 2011.

---

[16] Arya Grayeli, Atharva Sehgal, Omar Costilla Reyes, Miles Cranmer, and Swarat Chaudhuri. Symbolic regression with learned concept library. Advances in Neural Information Processing Systems, 37:44678- 44709, 2024.
[17] Nate Gruver, Mare Anton Finzi, Shikai Qiu, and Andrew Gordon Wilson. Large language models zero-shot time series forecasters. Advances in Neural Information Processing Systems (NeurIPS), 2023.
[18] Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuchao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, et al. survey on llm-as-a-judge. arXiv preprint arXiv:2411.15594 2024.
[19] Steven Gustafson, Edmund K Burke, and Natalio Krasnogor. On improving genetic programming for symbolic regression. In 2005 IEEE Congress on Evolutionary Computation, volume pages 912-919. IEEE, 2005.
[20] Shibo Hao, Yi Gu, Haodi Ma, Joshua Hong. Zhen Wang, Daisy Wang, and Zhiting Hu. Reasoning with language model is planning with world model. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 2023.
[21] John Hunter and Darren Dale. The matplotlib user's guide. Matplotlib 0.90. user's guide, 487, 2007.
[22] Yunseong Hwang, Anh Tong, and Jaesik Choi. Automatic construction of nonparametric relational regression models for multiple time series. In International Conference on Machine Learning (ICML). 2016.
[23] John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figumov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Zidek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. nature, 596(7873):583 589, 2021.
[24] Verre-Alexandee Kamienny, Stéphane AsAcol,,, Guillaume Lample, and François Charton. End-to-end symbolic regression with transformers. Advances in Neural Information Processing Systems, 35; 10:69 10281, 2022.
[25] Maarten Keijzer. Improving symbolic regression with interval arithmetic and linear scaling. In European Conference on Genetic Programming, pages 70-82. Springer, 2003.
[26] Hyunjik Kim and Yee Whye Teh. Scalable structure discovery in regression using gaussian processes. In Workshop on Automatic Machine Learning, pages 31-40. PMI.R, 2016.
[27] Hyunjik Kim and Yee Whye Tch. Scaling up the automatic statistician: Scalable structure discovery using gaussian processes. In International Conference Artificial Intelligence and Statisticx, pages 575-584. PMLR, 2018
[28] Krzysztof Krawiec and Tomasz Pawlak. Approximating geometric crossover by semantic backpropagation. In Proceedings of the 15th annual conference on Genetic and evolutionary computation. pages 941-948. 2013.
[29] Pat Langley. Bacon: A production system that discovers empirical laws. IJCAI, page 344. Citeseer. 1977.
[30] Michacl Y. Li, Emily Fox, and Noah Goodman. Automated statistical model discovery with language models. In International Conference on Machine Learning (ICML), 2024.
[31] Wenqiang Li, Weijun Li, Linjun Sun, Min Wu, Lina Yu, Jingyi Liu, Yanjie Li, and Songsong Tian. Transformer-based model for symbolic regression via joint supervised learning. In The Eleventh Interna- tional Conference on Learning Representations, 2022.
[32] Ming Liu and Wensheng Zhang. Is your video language model a reliable judge? In International Conference on Learning Representations (ICLR), 2025.
[33] James Robert Lloyd, David Duvenaud, Roger Grosse, Joshua B. Tenenbaum, and Zoubin Ghahramani. Automatic construction and natural-language description of nonparametric regression models. In AAAI Conference on Artificial Intelligence (AAAI), 2014.
[34] Sanae Lotfi, Pavel Izmailov, Gregory Benton, Micah Goldblum, and Andrew Gordon Wilson. Bayesian model selection, the marginal likelihood, and generalization. In International Conference on Machine Learning (ICML), 2022.
[35] Xiaoyu Lu, Alexis Boukouvalas, and James Hensman. Additive gaussian processes revisited. In Interna- tional conference on machine learning, pages 14358- 14383, PMLR, 2022.
12

---

[36] Pingchuan Ma, Tsun-Hsuan Wang, Minghao Guo, Zhiqing Sun, Joshua B Tenenbaum, Daniela Rus. Chuang Gan, and Wojeiech Matusik. Llm and simulation bilevel optimizers: A new paradigm advance physical scientific discovery. International Conference on Machine Learning (ICML), 2024.
[37] Bodhisattwa Prasad Majumder, Harshit Surana, Dhruv Agarwal, Bhavana Dalvi Mishra. Abhijeetsingh Meena, Aryan Prakhar, Tirth Vora, Tushar Khot, Ashish Sabharwal, and Peter Clark. Discoverybench: Towards data-driven discovery with large language models. In International Conference on Learning Representations (ICLR), 2025.
[38] Elaine C Meng. Thomas D Goddard, Eric F Pettersen, Greg S Couch, Zach J Pearson, John H Morris, and Thomas E Ferrin. Uesf chimerax: Tools for structure building and analysis. Protein Science, 32(11))e4792 2023.
[39] Matteo Merler, Katsiaryna Haitsiukevich, Nicola Dainese, and Pekka Marttinen. In-context symbolic regression: Leveraging large language models for function discovery, arXiv preprint wxXv:22444.1994 2024.
[40] Charles Morris, [newton's 's law of gravitation]. Science, os (s-2(55))361-363) 1881
[41] G.S. Ohm. Die galvanische Kette, mathematisch bearbeitet. Riemann, 1827.
[42] Travis E Oliphant et al. Guide to numpy, volume 1. Trelgol Publishing USA, 2006.
[43] Karl Popper. The logic of scientific discovery. Routledge, 2005.
[44] Feras Saad, Marco F Conumanno-Toweer Ulrich Schaechtle, Martin Rinard, and Vikash K Mansinghka. Bayesian synthesis of probabilistic programs for automatic data modeling. Proceedings of the ACM on Programming Languages (POPL), 2019.
[45] Ulrich Schacchtle, Feras Saad, Alexey Radul, and Vikash Mansinghka. Time series structure discovery via prohahilistic program synthesis, 2017.
[46] Michael Schmidt and Hod Lipson. Distilling free-form natural laws from experimental data. Science, 324(5923):81 (1-85 2009.
[47] Eric Schulz, Maarten Speekenbrink, and Andreas Krause. A tutorial on gaussian process regression: Modelling, exploring, and exploiting functions. Journal of mathematical psychology, 85:1-16, 2018.
[48] Samiha Sharlin and Tyler R Josephson. In context learning and reasoning for symbolic regression with large language models. arXiv preprint arXiv:2410. 17448, 2024.
[49] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement lcarning. Advances in Neural Information Processing Systems, 36, 2024.
[50] Parshin Shojaee. Kazem Meidani, Amir Barati Farimani, and Chandan Reddy. Transformer- planning for symbolic regression. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances Neural Information Processing Systems, volume 36, pages 45907-45919. Curran Associates, Inc., 2023.
[51] Parshin Shojace, Kazem Mcidani, Shashank Gupta, Amir Barati Farimani, and Chandan Reddy, Lim-sr: Scientific equation discovery via programming with large language models. arXiv preprint arXiv:2404. 18400, 2024.
[52] Robert H. Shumway and David S. Stoffer. ARIMA Models, pages 75-163. Springer International Publishing, Cham, 2017.
[53] Maojun Sun, Ruijian Han, Binyan Jiang, Houduo Qi. Defeng Sun, Yancheng Yuan, and Jian Huang. A sur- vey on large language model-based agents for statistics and data science. arXiv preprint 241i 14222, 2024.
[54] Sean J Taylor and Benjamin Letham. Forecasting at scale. The American Statistician, 72(1):37 45, 2018
[55] Wassim Tenachi, Rodrigo Ibata, and Foivos I Diakogiannis. Deep symbolic regression for physics guided by units constraints: toward the automated discovery of physical laws. The Astrophysical Journal, 959(2):99 2023.
[56] Kwabena Nuamah Thomas Fletcher, Alan Bundy, Gby-abed: A configurable automatic bayesian covariance discovery implementation. In International Conference Machine Learning (ICML), 2021.
13

---

[57] Ye Tian, Baolin Peng, Linfeng Song, Lifeng Jin. Dian Yu, Lei Han, Haitao Mi, and Dong Yu. Toward self-improvement of LLMs via imagination, searching, and criticizing, In Advances in Neural Information Processing Systems (NeurlPS), 2024.
[58] Silviu-Marian Udrescu and Max Tegmark. Ai feynman: physics-inspired method for symbolic regression. Science advances, 6(16)reaay2331, 2020.
[59]
Silviu-Marian Udrescu and Max Tegmark. Ai feynman: physics-inspired method for symbolic regression. Science advances, 6116)caay2631, 2020.
[60] Nguyen Quang Uy Nguyen Xuan Hoai, Michael O'Neill, Robert McKay, and Edgar Galván-Lóped Semantically-bann crossover in genetic programming: application to real-valued symbolic regression. Genetic Programming and Evolvable Machines, 12:91-119, 2011.
[61] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Le. H. Chi, Sharan Narang, Aakanksha Chowd- hery, and Denny Zhou. Self consssenns improves chain of thought reasoning in language models. In International Conference on Learning Representations (ICLR), 2023.
[62] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoe V. Le, and Denny Zhou. Chain-of ohain-of-thouuht prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
[63] Liwenhan Xie, Chengbo Zheng, Haijun Xia. Huamin Qu. and Chen Zhu-Tian. Waitgpt: Monitoring and steering conversational Ilm agent in data analysis with on the-thhhffyyyy code visualization. In Proceedings of the 37th Annual ACM Symposiun on User Interface Software and Technology, pages 1-14, 2024.
[64] Liwenhan Xie, Chengbo Zheng, Haijun Xia, Huamin Qu, and Chen Zhu-Tian. Waitgpt: Monitoring and steering conversational I1m agent data analysis with on the-fly code visualization. In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology, 2024.
[65] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Information Processing Systems (NeurlPS), 2023.
[66] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
[67] Haoran Yc, Jiarui Wang, Zhiguang Can, Federico Berto, Chuanbo Hua, Hacycon Kim, Jinkyoo Park, and Guojie Song. Reevo: Large language models as hyper-heuristies with reflective evolution, 2024.
[68] Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Jin Liu, and Gao Huang. Expel: Llm agents experiential learners, 2024.
[69] Jinghui Zhong, Liang Feng, Wentong Cai, and Yew-Soon Ong. Multifactorial genetic programming for symbolic regression problems. IEEE transactions systems, man, and cybernetics: systems, 50(1 44442 4505, 2018.
[70] Yizhang Zhu, Shiyin Du Boyan Li, Yuyu Luo, and Nan Tang. Are large language models good statisticians? arXiv preprint arXiv:2406.0ii 078/5, 2024.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes. The main claims in the abstract and introduction accurately reflect the paper's s core contributions and scope. We outlined the contribution and proposed methods in abstract and Introduction (Sec. T.
Guidelincs:
The answer NA means that the abstract and introduction do include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Yes. We discuss the limitations of our work and potential directions for future improve- ments in See.
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are discussed in the paper.
The authors are encouraged create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification asymptotic approximations only holding locally). The authors should reflect on how these assump- tions might be violated in practice and what the implications would be.
The authors should reflect on scope the claims made, e.g., if approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
applicable, the authors should discuss possible limitations of their approach to address prohlems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play important role in developing norms that preserve the integrity of the community, Reviewers will specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For cach thcoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: Yes. See Appendix for the detailed proof of VIC.
Guidelines:
The answer NA means that paper does not include thcoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
15

---

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors encouraged provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal
proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: This paper contains the information to reproduce the main experimental results at Sec. F and Appendix. We are using close-source models, and have provided the prompt at Appendix for reproducability with similar results.
Guidelines:
The answer NA means that paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example. if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation. it may be necessary either make it possible for others to replicate the model with the same datasct, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not roquire releasing code, the conference does require all submissions to pro- vide some reasonable avenue for reproducibility, which may depend on the nature the contribution.
For example
(a) the contribution primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architec- ture clearly and fully.
(c) If the contribution is new model (e.g.. a large language model). then there should either be way to
this model for reproducing the results way reproduce the model (e.g.. with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases. in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed- source models, it may that access to the model is limited in some way (e.g.. to registered users), but it should be possible for other rescarchers to have some path to reproducing or verifying the results.
Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, described supplemental material?
Answer: [Yes]
Justification: Yes. The data sources used in our experiments are fully accessible and listed in the Appendix, we provide the prompt for the main experimental results the Appendix, with instructions for setup and usage.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines httpp:::/nnip.cc//ublbl/
guidee//CoeeSubmn asi esi onPolicy) for more dctails.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g.. for open-source benchmark).
16

---

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines htttps://nips.cc/uubte Soadde//oodeuuumsssiiooo
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts reproduce all experimental results the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental sefting/details
Question: Does the paper specify all the training and test details (e.g.. data splits, hyperparameters. how they were chosen, type of optimizer, etc.) necessary understand results?
Answer: [Yes]
Justification: Yes, the data the backbone models are explained at Sec. We also have provided the detailed experimental settings at the Appendix, how the data splits are done, hyperparameters, and the optimizer.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report emror bars suitably and correctly defined other appropriate informa- tion about the statistical significance of the experiments?
Answer: [Yes]
Justification: Yes, we have provided the mean and confidence interval of the model prediction at Fig. 7 In the visualization, the shaded blue region is the confidence bounds, providing a prediction uncertainty.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at lcast for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for exam- ple, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, hootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar the standard deviation or the standard error of mean.
is OK to report 1 1-sigma error bars, one should state authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric eтror bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were caleulated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: Yes, we have provided the information about the computer resources at Appendix.
Guidelines:
17

---

The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the exper- iments reported in the paper (e.g., preliminary failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, every respect, with the NeurIPS Code of Ethics
Answer:
[Yes]
Justification:
All authors have reviewed the NeurIPS Code Ethics.
Guidelines:
The answer NA means that authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consideration due laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We will included the potential positive socictal impacts and negative impacts at Appendix.
Guidelines:
The answer NA means that there is no socictal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g.. dis- information, generating fake profiles, surveillance), fairness considerations (c.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategics (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work is based on publicly available datasets 33 and widely used, pre-existing models [30 52 54, 56),, none of which are novel or proprietary in this context. Therefore, we believe our work does not warrant specific safeguards beyond standard responsible rescarch practices.
Guidelines:
The answer NA means that the paper poses such risks.
18

---

Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled of the model, for example by requiring that users adhere usage guidelines or restrictions access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging. and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All external assets used this work, including datasets, models, and code, are properly cited in the main text and references. No proprietary or restricted materials were employed in manner that violates their intended use.
Guidelines:
The answer NA means that paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset.
For scraped data from particular source (e.g.. website), the copyright and terms of service of that source should be provided.
in the package should be provided. For popular datasets, ppaperswi whwwwe.hhoe...... . com/ datasets has curated licenses for some
If assets are released, the license, copyright information, and terms /
datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We introduce new components in the form of a multi-step multi-modal evaluation pipeline. These assets are done with clear descriptions of their functionality. input/output formats, and usage instructions.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the datesee//coeemmodllll as part of their submissions via structured templates. This includes details about training. license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, applicable, as well as details about compensation (if any)?
Answer: [Yes]
Justification: Yes. We have included the full text of the instructions provided to human evaluators, along with the screenshots the interface used during the evaluation process the Appendix.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then much detail possible should be included in the main paper.
19

---

According to the NeurIPS Code of Ethics, workers involved in data collection. curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [Yes]
Justification: Participants were clearly informed about the purpose and nature of the task before participating, and their participation was entirely voluntary. We ensured all procedures complied with ethical standards for research involving human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. you obtained IRB approval. you should clearly state this in the paper.
recognize that the procedures for this may vary significantly between institutions and locations. and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs it an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does impact the core methodology, scientific rigorousness, originality the research, declaration is not required.
Answer: [NA]
The core method development our paper does not rely on large language models (LLMs) in any essential or novel way. All key components were developed by the authors, and LLMs are only used as standard tools within the evaluation framework.
Justification:
The answer means the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer our LLM policy should or should not be described.
what
20

---

A Technical Appendices and Supplementary Material
A.1 Gaussian Process Kernel Composition
In paper, model means the kernel composition of gaussian process. Our kernel composition is done through below grammars and basis kernels. The basis kernels contain the linear(LIN), periodic(PER), squared exponen- (iE)(SE), constant(C), white (WN) following [33] And our composition grammar contains addition(++) multiplication(x), replacement. So our model search space E can be defined as:
=
U (k A E = 1 (bi). by E E,D:EOO)
(4)
:: Linear / C
(5)
x replacement
(6)
where represents set basis kernels and denotes the set kernel operations. key property of this construction is that the space E is closed under the specified operations, ensuring that any combination of basis kernels is valid and also belongs to the search space.
A.2 Derivation of Visual Information Criterion
The derivation of visual criterion starts from the posterior probability of a model M given data D:
p(M p(DDM((((pC MM)
(7)
p(D)
Typically, p(M) is assume be uniform, leading the common Bayesian Information Criterion (BIC), which approximates MD) using Laplace's ' approximation:
BIC = - =2 log ((D)M, 0*) + k log n,
(8)
where 8" the maximum likelihood estimator (MLE), & the number of model parameters, and n the sample size.
In this work, we modify the BIC by introducing:
A non-uniform prior p(M) x 8, (MM),, where 8, represents structure similarity score of the model
M, and a₂ is a scale hyperparameter.
2. A modified likelihood function:
(DD|M 0) By (DDM 8).
(9)
where sy represents fitness score of the model M for data D, and is scale hyperparameter.
The marginal likelihood is given by:
PDIM) - @(D),M,0)((00M))jl
(10)
As the fitness score also maximized at the MLE = arg mag ((DIx,(DDMMe))) 0), one can apply Laplace's approximation around the MLE:
(DIIM) = ((DIM, 0 (2x))
(11)
where the Hessian matrix of log (DD(0, M) evaluated 0":
8²
H
log (DI
(12)
202
1=--
Substituting into our approximation and taking the log on the both sides:
log lD),(D log ((D,M,,) log
(13)
Then, our log posterior log p(MlD becomes:
log p(M() == ay log sy (DIM, + log((() for some constant C.
log n a. s. + C,
(14)
Multiplying by and rearranging gives:
2 2 2(og)(M|D) = -BIC + 2 log (D)
MM 0*) + 2o, log (M) +
(15)
By modeling 2 log (D) 0*) 2 log s. (M) with our EvaluatorVLMM and simply setting a. =ay!
VIC Evv.lvaatrr
BIC.
(16)
21

---

A.3 Experimental Details
Our experiments are upon GPy and GPy-ABCD [56]. We have conducted each experiments for 5 rounds, with 10 random restarts, and used L-BFGS-B optimization. Also, we conduct top-3 sampling from model pool for each round. We have used gpt-4o-mini (for the main result) for both AnalyzerVIM and Evaluator VL.,, and we have set hyperparameter a to 50 of our Evaluator VL to halance with the BIC of the visual criterion. Also we have utiliized the current round term for scoring to select mostly on recent models from the model pool. For Symbolic Regression,
have followed 488 and utilized its dataset. We have conducted each experiments for 20 rounds, with S random restarts each, and used scipy's 's optimize curve fit for parameter optimization. The function evaluation is done similarly to the gaussian process kernel discovery, setting the hyperparameter a to 0.05. To effectively search for the parameter for kernel scarch, we initially performed 10 random restarts to explore the parameter space broadly. Then substituted the resulting parameters with those proposed by AnalyzerVL. We then conducted second-stage local optimization, using the LallyyeeeVLM-iiiiiaii points. Our experiments are conducted on CPU with cores for the precise calculation; and it may take around multiple hours for the experiments. The experiment for Gaussian Process Kernel Discovery, we have used the dataset of ressarch spliting training data into 9:1 for the validation data.
parameters as starting
For BoxLM implementation, we have followed the explanation of (30] For the fair comparison with our methods, we have set the basis kernel as linear(LIN), squared (Xponential(SE), constant(C), and white noise(WN), except the rational quadratic kernel(RQ), and also sampled top-3 models. Following our pipeline's evaluation, we have used Bayesian Information Criterion for the BoxLM's top-k model selection. For automatic statistician experiment, we have changed GPy-ABCD to work as greedy search through top-1 selection for each round. For ARIMA implementation, we have set ARIMA's p=2, d=1, q=2, and for facebook prophet implementation, have set seasonality mode multiplicative, and set changepoint prior scale to 0.1.
A.4 More Qualitative Results
We present additional quantitative results in Fig. [11 Fig Our method demonstrates superior performance overally capturing their underlying patterns. As shown, our methad not only fits the training data well but also generalizes better mostly compared to other methods.
A.5 Human Evaluation of the Model Selection
For human evaluation, the instructions are shown at Table and example on Fig.A13 We have instructed the human evaluators to evaluate given model following EvaluatorVLM's criteria: visual fitness, and visual generalizability. For each criteria, we instructed evaluators to rate the given model from to 5, giving the for the best. For fitness, we have evaluated the mean prediction's 's fitness, and also the uncertainty region. So with small uncertainty region with good fitness, evaluators are instructed give high score for visual fitness.
A.6 Function Composition and Implementation Details for Symbolic Regression
In our experiment for symbolic regression, have defined our basis function and base grammars and ex- periments for function composition following 39 We have conducted the iteration for 20 rounds, and the parameter optimization is done through Scipy's 's optimize curve fit. For EvaluatorVI.M we have utilized a to 0.05, which can halance the original symbolic regression score function and the visual evaluation. We prompted AnalyzerVLMM to propose the function composition based on the below basis functions and grammar, so our model search space of symbolic regression V is:
Σ UUISEB.I= 0i (b.), B, O},
(17)
B sin(x)) cos(z)) [an(x)) sinh(tr) cosh((x)
(18)
O ::= +| x |sqrt] exp log labs
(19)
A.7 Symbolic Regression Results at Real-World Dataset
We report the result of symbolic regression to the real-world univariate datasets 333, including Airline Passenger, Solar Irradiance, Mauna Loa, Wheat, Call-Center, Radio, and Gas Production. For the experiments, we conducted 20 rounds for each method. As shown in Table A6] SR-based methods require large number of trials to identify appropriate models. Unlike AnalyzerVLM which proposes functions based on detailed analysis of the data, symbolic regression-basee
methods like SGA (36] and LLM-SR [5T] generate naive proposals without such insight, often leading to ineffective results. Although ICSR 39 enhances function proposal by utilizing visualization of data, enabling it to show better results SGA and LR, it still falls short of achieving the same level of performance ours, due to the absence of precise, data data-driven analysis. Also, symbolic regression-baat based
22

---

Prophet
10
Observed Dato
68
Meon Forecast
66
0.5
Airline
644
6.3
1957
1860
1800
1957
1990
Autostats
BoxLM
Ours
52
80
Ybar
ARIMA
Facebook Prophet
Gaussian Process (SE)
WWW www WWW
Observed Data
Test
88
Mean Forecast
Mauna
2.0
2007
1992
1997
2022
2387
1992
1197
2002
2007
1992
1997
2002
1.0
1.
WWW
1.0
www.mm
www.
1970
1000 1980
2005
1902
1967
1902
2007
2000
3007
1902
1597
2003
Prophet
1.0
Observed Data
wwww MMM
Mean Forecast
Gas
1963
1998
1993
1943
2908
1353
1963
15403
1993
Autostats
BoxLM
Ours
MMW
WW
ww
Year
ARIMA
Facebook Prophet
Gaussian Process (SE)
Observed
Aond m M
1.0
1.0
lest Data
0.5
Forecast
1,h mam m/
C.
Call
0.6
0.0
0.
Autostats
BoxLMM
Ours
0.4
02
0.
1005
1970 Year
2013
1975
1973
1976
ARIMA
Facebook Prophet
Gaussian Process (SE)
Observed Data
1.00
1.00
1.00
lest Uata
0
2.8
Mean Forecast
0:56
0.56
0.6
Wheat
Mal Wyy yo WNll
0:34
0.34
0.34
0.6
1817
1837
1157
1877
1817
1817
1857
1817
1017
1877
1857
1877
0.4
1.03
100
0.5
www.lal
1720
1760
1800
3840
1817
3807
3857
1877
3837
1837
1857
1877
1817
1832
1957
1877
Figure A12: More qualitative results. We show more qualitative results at Airline, Mauna, Gas, Call, and Wheat data.
23

---

Please evaluate each model's fitness score. (You can give the same score to different models)
model1 1
model
model2
model4
0 M00
(worst)
(best)
O
O
Model
O
O
0
Model 3
0
0
0
0
Model
evaluate each fitness score. (You can give different models)
model1
model
model2
model4
(worst)
2
5 (best)
Model
O
O
0
Model
0
0
0
O
Model
Figure A13: Example sereenshot of visual fitness for human evaluation. The human evaluators instructed to rate the given model from 5, having same condition with the Evaluator VL...
24

---

Table AS: Human Evaluation Instruction. We have done the human evaluation similar to the EvaluatorVLMM evaluation instructions. Human evaluators are instructed to rate visual fitness, and visual generalizabilltlee.. consistency).
Evaluate the given model prediction following below criteria. Graph has three values: data(black line), mean prediction of the model(red line), and confidence region(blue shaded area).
Fitness: Compare the data plot(black line) and the model's 's mean(red line). If they are similar, give 5. If they are not similar at all(flatten line), give 1.
5: Red line perfectly matches black line
4: Red line closely follows black line with most details
3: Red line roughly follows black line (less detail, but not flat)
2: Red line is mostly flat or linear but somewhat follows data
1: Red line flat and does not match data all
Uncertainty: Check the blue shaded area is big. If blue shaded region is very small(means
small uncertainty), give 5. If it is moderate, please give them 3. If it is very big(means high uncertainty) give them 1.
5: Confidence region almost invisible (very certain)
4: Confidence region very small
3: Confidence region small in the middle, larger at edges
2: Confidence region visible and consistent throughout
1: Confidence region very large everywhere (high uncertainty)
Generalizability: At each side, the model prediction may fail(showing flatten or sudden dropping prediction at both side, showing the large uncertainty region at both side), which means the generalization has failed. In this case, give generalibility 1. If you think the model prediction maintains at both side, give high score. 5: Predictions at edges are natural with small uncertainty
4: Predictions at edges are natural (not flat) with visible uncertainty
3: Predictions at edges hold but become mostly flat lines
2: Predictions at edges degrade noticeably
1: Predictions at edges fail badly, flattening or dropping sharply with large uncertainty
methods underperform in real-world datasets since they do explicitly model the observation noise, while Gaussian process regression does.
Interestingly, we observe that symbolic regression's 's function composition can good starting point for the Gaussian process kernel discovery. To leverage this, we introduce a hybrid model discovery framework. denoted as Ours (SR + GP) in Table [66] We utilized simple SR framework [12 to generate the initial function composition and apply the top-3 function compositions' corresponding kernel structure with its initial parameters. Then, we conducted our GP kernel discovery pipeline from the kernels for rounds. With this, our hybrid model discovery results in superior performance across all evaluated datasets. Morcover, highlights the potential synergy between symbolic model discovery and probabilistic modeling for interpretable and accurate forccasting. Fig. (h14shows the qualitative results. As shown, utilizing the hybrid model discovery framework can enhance the performance and find the appropriate model across the dataset.
Table A6: Quantitative results in symbolic regression. compare our pipeline with competing methods on the train and test region, reporting RMSE as the evaluation metric. On average, our pipeline achieves superior performance across datascts. Bold stands for the best, and underline for the second best.
Dataset
Method
Airlino
Solar
Maura
Whast
Call
Raalio
Coo
Avg.
Train
Test
Train
Text
Than
Test
Trin
Tex:
Train
Tiet
Train
Text
Trais
Test
Triin
Test
SGA 35
0.700
0.1665
0.1451
0.3653
0.0332
8.0354
0,0599
0.1338
0.0630
0.8583
0.2859
0.5025
0.8574
0.3424
0.1020
0.3720
LLM--R S3
B,0692
0,3384
0.1099
1.1133
0.1317
0.2196
0.0492
0.1438
0.0438
21.025
0.1875
0.1788
0.1490
0.6521
0.0763
3.4639
JCSR
B0420
0.1029
0.1807
0.3845
0.1347
0.0543
0,0495
T,4430
0,0558
0.4725
0.1759
0.2427
008497
0.1672
0,0864
0.4067
Ours
ISR GP)
0,0154
0,0354
0.0297
0.3345
0,0937
0/9166
0,0150
0L1801
0,0095
0.1085
0U849T
0.0515
0.0159
0,8581
0,0203
0.1111
25

---

Airline
ICARI
Radio
Figure A14: Qualitative result in symbolic regression. Although conventional symbolic regression methods often exhibit limited performance real-world datasets, our hybrid model discovery framework demonstrates notable improvements.
A.8 Analyzer VLM Prompts
For the AnalyzrVLM System Prompt, have utilized WaitGPT (64)'s system prompt. Amd VLM's action choosing(e.g. analysis, code generation, model proposal) prompt is shown [A7 As shown here, do not simply give AnalyzerVLM the data itself, instead, we provide AnalyzerVLMM a small code block that can acess the data and the model's predictions. With this, AnalyzerVLM can inject this code block at the code generating action, and get the executed output of the code. In this way, Analyzer can choose how to represent the data and model, not just fully stuffing the long sequence of the numbers into itself. We also report the prompts used for AnalyzerVLMN and EvaluatorVLMM for symbolic regression experiments Table Our implementation of prompts follows the scheme of [48) utilizing our multi-step reasoning for Analyzer VLM and visual evaluation for Evaluator V...
A.9 Prompt for AnalyzerVLM and Evaluator at Symbolic Regression
We report prompts used for AnalyzerVLM EvaluatorVLMM symbolic regression experiments at Ta- ble A9 Our implementation of prompts follows the scheme of [48] utilizing our multi-step reasoning for AnalyzerVLM and visual evaluation for EvaluatorVLM..
A.10 EvaluatorVL.M Prompts
Evaluator evaluation prompts are shown at Table A8 We evaluate fitness and structure similarity seperately, and add two scores and use for the final score. For consistency, we evaluate each parts(e.g. fitness, structure similarity) twice, and averaged.
First, we evaluate the fitness based on two points: mean prediction's 's data sample matching, size of confidence area. For the mean prediction and data matching, prompted VLM to score 0-10 points if the mean prediction just constant value, 10-20
linear but follows overall trend. upper the mean prediction follows more trend than that. For the confidence area, we gave 30-50 score if confidence area is small, and gave under 30 if' confidence area gets sudden big at each side, which means it looses the confidence at extrapolation region.
Second, we evaluate the structure similarity based on the red linc's 's structure similarity at the middle of the graph (train region) is maintained at the ends of graph. We prompted VLM to give high score(40-50) when the structure similarity is maintained, else give the lower score.
A.11 Detailed Multi-step Analysis Example
And we also show the detailed outputs of multi-step analysis of AnalyzerVLM, Fig. AIS and we
Fig. A16
A17
that our AnalyzerVLM chooses how to start the analysis, first, Analyzer LLM plans to first visualize data and the predictions, and generates the code for it. When the code executed and the output O (c.g., visualization plot, the covariance visualization) is given, AnalyerVLM repeats the its process until the analysis is sufficient.
26

---

Table A7: AnalyzerVL. Prompts. We provide the prompt for analyzerVLM. First it describes about the base kernel, and provide the actions that AnalyzerVLM can select. Then AnalyzerVLMM prompted select actions.
AnalyzerVLM Analysis and action choosing.
Task Overview:
You are provided with the mean and covariance 1D array of the fitted kernel ['kernel ']. Your job is to either:
Generate Python code for further analysis, or Recommend new kernel combinations. You can only choose one action at time.
Kernel Adjustment Options:
You can adjust the current kernel by forming new combinations with base kernels using the following operations:
Addition (S B): Add new base kernel B the current kernel S.
Multiplication (S * B): Multiply the current kernel with new base kernel B.
Base Kernel Replacement: Replace the base kernel with new base kernel B'.
Base Kernels Available:
Linear (LIN)
Periodic (PER)
Sqqared Exponential (SE)
Constant (C)
White Noise (WN)
Action I:
Analyze the Fitted Kernel (Python Code)
you need further analysis before making recommendation, generate Python code for the task. You can draw insights from the mean, covariance, and confidence intervals of the fitted kernel, or analyze the parameter itself. However, please try one analysis at time. Access the Data and the Model Parameters:
"python
X, y. enX, en en_mean, en_cov, en mn_low__uannte (fitted, [ritted_modees((0) models[0]) model_printout (model) <<.
access_data
This will give you train data (X, y). enlarged data with test data (enX), the mean, covariance, and confidence intervals for the enlarged X, and the model parameters. Generate Code: If analysis is needed, provide the Python code necessary to calculate or visualize key insights.
"python
Python code goes here
Action 2: Recommend Kernel Combinations
you have already analyzed the kernel, suggest new kernel combinations using the current kernel S and the base kernels. Use the operations outlined above.
Example Recommendations:
next kernels: ["new combination1", "new combination2", "new combination3", "new combination4", "new combination5", "new combination6"
Important: Choose only one action: Either provide Python code or recommend new kernel combinations.
Do not provide both at the same time.
27

---

Table A8: EvaluatorVLM prompt. For the fitness evaluation, have evaluated how well the real data prediction fits, and how small the confidence area is. Each points are measured at 50 points, total to 100. For generalizability evaluation, we have evaluated how the structure maintained throughout the data. Structure similarity is measured out of 50 points
Evaluator VLM: Fitness evaluation of visual information criterion.
You will evaluate the similarity of the two graph, data graph and predicted mean graph. Assign a score from 0 to 50. Evaluate the Structure Similarity Between Real Data and Mean Prediction.
Please check the real data graph is similar predicted mean graph. Please check below:
- Mean graph is similar with sample graph (20-50 points).
Predicted mean graph is linear line while it shares trend with data graph (10-20 points)
Mean graph is linear and it does not share the trend at all(0-10 points).
Please generate the response in the form of Python dictionary string with keys of kernel name. score is in INTEGER, not STRING.
Please evaluate how similar the two graphs are. First data graph and second graph is predicted mean graph. Output should be the score for the kernel1. kemell:
Please evaluate how small the confidence interval area is. Evaluate the Size of the Confidence Area (LightBlue Shaded Area) Confidence scores should be assigned based on size of the lightblue shaded area. So do not consider the red line and black line, only the lightblue shaded area's size and the region of uncertainty.
Please check what the confidence area looks like. Assign score from 0 to 50 following below: Confidence interval area is hard to see, uncertainty is small(this case assign 40-50 points). in the middle of graph, but large the boundaries (30-40 points). This means the model is overfitted to the middle, so give a low score.
2. Confidence interval area is hard
Confidence interval area is normal in the middle, uncertainty remains but acceptable or becomes larger over y at the boundaries (0-30 points).
Evaluator VLM: Generalizabilite evaluation of visual information criterion.
You will evaluate how well the predicted kernel (red line) maintains based on the below criteria: Evaluate the structure similarity of middle of the graph and the ends of the graph. Check the blue line's structure similarity of the middle maintains at the left and right end of the graph. If was following the data well but suddenly changes to the constant line at the ends of the graph, assign low score for structure similarity score. But if structure similarity is maintained, assign 40-50 score.
Please generate the response in the form of Python dictionary string with keys of kernel name. score is in INTEGER, not STRING.
Please evaluate how similar the two graphs are. First is data graph and second graph is predicted mean graph. Output should be the score for the kernell. kernel::
28

---

Table A9: AnalyzerVLM and EvaluatorVLM prompts for symbolic regression. We report action choosing prompt used by AnalyzerVLM and fitness & generalizability evaluation prompt employed by EvaluatorVI.M in symbolic regression.
AnalyzerVLM, Action choosing prompt.
Your task is to give a list of five new potential functions that are different from all the ones reported below, and have a lower error value than all of the ones below. Before the function generation, please first analyze the given data points and reported functions first(e.g.. visualization, or get the statistics). Guess and list up which this function would be. If you generate the Python code that includes your analysis, will execute and give you the result. You use sympy for checking the function prediction. For saving the visualization, please avoid using plt.show(, and plt.savefig(' /zzmpings/imagennmnn using the data points, please sort them first.
when imagename is any visualization you made. Before
Please give only python code for now. Code:
"python
Python code goes here
Please try to bulid upon the function with the smallest error, then generating different ones too. Generate as diverse diverse functions!
Evaluator VLM: Fitness evaluation prompt.
You are an intelligent chatbot designed for evaluating two graph's 's similarity.
You will evaluate the structure similarity of the two graph, data graph and predicted mean graph.
Assign score from 0 to 50.
Evaluate the Structure Similarity Between Real Data and Mean Prediction.
Please check the real data graph is similar to predicted mean graph. Please check below:
Mean graph similar with sample graph (20-50 points).
Predicted mean graph is linear line while it shares trend with data graph (10-20 points)
Mean graph linear and it does not share the trend at all(0-10 points).
Please evaluate how similar the two graphs are. First is data's line plot, and second is predicted value's line plot. Please evaluate how well the predicted value fits to the data. Output should be the score for the functionl. Please generate the response in the form of Python dictionary string with keys of function name. score is in INTEGER, not STRING. functionl:
Evaluator Generalizability evaluation prompt.
You are an intelligent chatbot designed for evaluating the correctness each functions. You will evaluate how well the predicted value (red line) fits based the below criteria:
Evaluate the structure similarity middle of the graph and the ends of the graph.
Check the blue line's 's structure similarity of the middle maintains at the left and right end of the graph.
If it was following the data well but suddenly changes the constant line the ends of the graph, assign low score for structure similarity score.
But if structure similarity is maintained, assign 40-50 score.
Please generate the response in the form of a Python dictionary string with keys of function name. 'score for structure similarity' are in INTEGER, not STRING.
29

---

Let's start with **Action 1: Analyze the Fitted Kernel** by generating Python code for further analysis.
gool
moon, covarance, canfidence
Plan:
model parameters using provided function. 2. Visualize the data, mean, and confidence intervals using Matplotiib.
Access
Here's the Python code analysis:
PYTHON CODE
Recap of the Plan:
Access
model parameters.
Visualize the dets, mean, and confidence intervels using Matplotiib.
fitted kernel_ene ptarne_analysis.pon
Now; run this code perform the enslysis. Let know you need further assistance!
Analysis of Current Kernel (LIN SE (PER C))
Basad an the provided graph and the output from the previous analysis, here are the key observations:
1. *Moan Value vs. Real Data Points**;
closely
current
captures the overall trend well.
particularty suggests that the model may not be fully capturing the undertying dynemica in those regione.
2. "Covariance"*:
relatively especially at boundaries, uncertainty in the predictions, which could be due to the non stationary nature of the data or the choice of kernels. smallar covariance would generally indicate more confidence the predictions,
confidence intervals (light
higher
an area for potential improvement.
*Peremater Apgropriateness":
The current kernel effectively captures both lineer end periodic trends, but the presence the constant (C) and the squared exponential (SE) may not necessary the data is primarily Uinear with periodic fluctustions. periodic component (PER) evident data, but kernel may intraducing
complexity.
Next Steps
Given enelysis, seems that further enalysis warranted to explore the residuals of the model, which can provide insights well current kernel performing and where lacking. This will help making more informed recommendation
kerne combinations.
Python Code Residual Analysis
Here's the code to calculate and visuelize the residuals:
residuals, analysis.png
Figure A15: Example of VLM Analysis step 1,2.
30

---

Analysis of the Residuals
Besec on the residuals piot genersted from the previous analysis, here are the key obsarvations:
"Residusls Behavior**:
The residuels (black line) lluctuate eround the zero line (red dashed line), which is good sign as inclicetes that the model does not consistently overestimate or underestinaae the actual values,
However, there are periods where the residuale show significant deviations from zero, perticularly towards the end the time series. underlying petterns trends effectivaly those regions.
This suggests that the model may not be capturing
*VVariance Residuals*
The variance of the residuals oppears increase over time, espocially in the later yours. This indicates that the model's uncortainty kernel adequataly capturing the dynemics of data.
growing,
*Model Appropriateness"
The current kernel (LIN SE (PER CJj seeme to capture the overall trend well, but the incressing veriance in the residuals may simplistic flexible enough account complexities The presence of the constant (C) may not be necessory, as the data appears to have more complex structure that could be better captured
suggests
Next Steps
analysis residuals, explore the trends in the residuale, such as calculating the autocorrelation of the
analysis warranted
This can help identify if there any petterns in the residuals that the current model is feiling to capture.
-009
Python Code for Autocorrelation Analysis
autocorrelation
residual residual autocorrelation satiiiiinnnnaasssssspppp
Analysis of the Autocorrelation of Residuals
Based the outocomelaation residuals, the observations:
"AAttccoreiation Behavior**:
The autocorrelation values are significent for the first few lags, indicating that there is correlation between the residuals these fully capturing temporel dependencies Thee autocorrelation drops off quickly after the first few lags, but the presence of significant values indicates that the residuals are independent,
lags. suggests
violation assumptions many
"Imppicattons
Karnel**;
The presence of autocomeiation the residuats suggests that the current kernel (LIN SE (PER + not be flevible enough to copture underlying structure The model may benefit from incorporating edditional kernets that can better account for the temporal depandencies observed in the residusls.
could systematic predictions.
**Veriance of Residusis**:
-The residuals show varie bitit, but the sutocorrelation indicates that there patterns that current kernel not cepturing.
This could lead to increased uncertainty predictions, espocially the later periods.
Next Steps
Given the enelysis the autocorrelation of the residuell,, seems that further enelysis warranted to explore the trends in the deta itself, such calculating the period of the periedic component. This can help making informed recommendation kernel combinations.
Python Code for Period Calculation
CODE EXECUTION ERROR OCCURED
Here's 's code to calculate and visualize the period the date:
Figure A16: Example of VLM Analysis step 3,4.
31

---

Analysis of the Current Kernel (LIN SE (PER C))
Based on the previous anelyses, here are the key observations regarding the current kernet:
**MMan Value vs. Real Data Points"::
kernel follows the data points reasonably well, but noticeable deviations, especially towards the end of the time series. This indicates that the kernel may not be capturing all the underlying trends effectivaly.
The value generated
**Covariance*
predictions independent. sugrests the covariance structure of the current kernel may not be appropriate for the data, leading to increased uncertaintty predictions.
significant
indicating
3. "Parrmmtter Apprepristeness*
current captures may not be necessary. The increasing variance in the residuals suggnats that the model may simplistic floxible enough account complexities in the data.
trends, presence
(C) exponential (SE)
Next Steps
bnarve are
Given the enelysis, it seems that further anelysis warrented explore periodicity
more rabustly. previous attempt calculate the period resulted error, will provide different approach analyze the periodicity.
Python Code for Periodicity Analysis
Here's alternative analyze the perlodicity using Trensform, which can help identify dominant frequencies the date:
periodicit_aanalssis.po
Analysis of the Periodicity Analysis Graph
key
"Frequency Components"*:
The plot shows several peaks at different frequencies, indicating that the data has multiple periodic components. The presence of may not fully copture.
peaks suggests
periodic
"Magnitude of Frequencies*
The magnitudes of the frequencies indicate the strength of the periodic components. The highest peaks dominant trequencies that could be important for modeling the date. This information can useful for selecting appropriate kernels that can these periodic trends.
""Implications for the Current Kernel**:
Thee current kernel (LIN SE (PER + Cj) may not be adequately capturing the periodic nature of the data, especially given the muitiple frequency components observed. The linear and squared exponential components may be sufticient modill complexities in the data.
Next Steps
Given the analysis the periodicity, seerss that further analysis warranted to explore the specific periodic companents and how almaddy conducted analyses, may beneficial
they be incorporated recommend now kernel combinations based on the insights gained.
kemel.
Recommenned Kernel Combinations
Considering the periodic nature the data and the limitations the current kernel, here are some new kernel combinations that could explored:
plaintaxt
next kerela: ('PER (LIN SE),, 'PER (LIN SEY, 'PER SE + LIN', "UN PER + C², 'SE PER C: 'PER (LIN CJ1]
Figure A17: Example of VLM Analysis step 5,6.
32


Reviews:
[{"review_id": "Kk4Bzrrcxb", "reviewer": "Reviewer_Nkkg", "summary": "This work levrages two Vision-Language Models (AnalyzerVLM + EvaluatorVLM) to automatically discover mathematical models for time series data. AnalyzerVLM iteratively analyzes data through code generation, EvaluatorVLM evaluates models using a novel Visual Information Criterion (VIC) that combines traditional metrics with visual assessment.", "questions": "How does this work with high-dimensional data where simple plots aren't possible?\nHow do you choose α (varies 1000x between applications) without extensive tuning?\nWhen does visual assessment give wrong answers? What about noisy/counter-intuitive data?\nHow expensive is multi-step VLM inference really? Is this practical at scale?", "limitations": "See Weaknesses above.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n+ Intresting to use VLMs for automated model discovery with human-like visual reasoning\n+ Outperforms baselines (ARIMA, Prophet, etc.) across different datasets with better generalization\n+ Captures human intuition about model reasonableness through visual evaluation\n\nWeaknesses\n\n- Only 1D time series was tested\n- Multi-step VLM inference could be expensive, unclear how it scales\n- VIC weighting (α) and scoring thresholds lack principled selection", "was_revised": true, "final_justification": "Thank you for rebuttal. Increased my score.", "rebuttal": "We thank reviewer Nkkg for valuable comments, and recognizing our work as:\n\n- Interesting to use **VLMs for automated model discovery** with human-like visual reasoning\n\n- **Novel Visual Information Criterion (VIC)** that combines traditional metrics with visual assessment\n\n- **Capturing human intuition** about model reasonableness through visual evaluation\n\nWe have addressed your concerns and comments below.\n\n### **Extension to high-dimensional data & noisy data**\n\n> *Only 1D time series was tested.*\n\n> *How does this work with high-dimensional data where simple plots aren't possible?*\n\nWhile a similar vein of research [13, 29, 32], including ours, has been only focused on the univariate data setting due to its importance and its broad applications, we agree that extending to the multi-variate case is an interesting direction.\n\nGiven the limited rebuttal period, while we could not show a complete extension at this moment, we emphasize that our pipeline is not limited to the univariate case. Our proposed multi-modal multi-step pipeline is modular; thus, extending each module to multi-variate ones enables multi-variate data analysis. In this sense, we discuss the following potential directions.\n\nFirst, AnalyzerVLM can be directly applied to multivariate data without any architectural modifications. As it conducts its analysis through data access code snippets, no additional resources (e.g., tokens) are required to handle multivariate inputs. Based on this, we have conducted additional experiments of AnalyzerVLM at multivariate data, and we observed that AnalyzerVLM conducts coordinate visualization, and pairwise interactions analyses, drawing correlations across dimensions. Such analyses from AnalyzerVLM can inform per-dimension modeling of the data or the adoption of similar models with highly correlated dimensions, providing a strong foundation for the model discovery process. \n\nSecond, EvaluatorVLM visually evaluates the resemblance between the prediction and the data. A simple and straightforward extension would be to evaluate algorithmically (e.g., evaluate each output dimension independently and aggregate scores, or evaluate randomly sampled dimensions alternatively according to each iteration, like randomized algorithms), similar to how mean squared errors are averaged across dimensions in conventional scenarios. \n\nIn addition, for high-dimensional data, one can adopt alternative plot formats incorporating joint decompositions (e.g., PCA, t-SNE). While the visualization format may change for high-dimensional or multivariate datasets, the core idea of vision-based analysis and evaluation can be scaled up upon our pipeline.\n\nWe appreciate the reviewer’s suggestion regarding high-dimensional extensions; it points to a valuable direction, and we believe further exploration along these lines would be a meaningful next step to broaden our approach.\n\n\n> *What about noisy/counter-intuitive data?*\n\nTo evaluate our pipeline’s robustness, we conducted experiments on the Airline, Mauna datasets (which has the increasing trend with periodicity (shapes in Fig. 11)) by introducing controlled Gaussian noise (σ = 0.03~0.05).\nAs long as the noise did not completely override the underlying trends in the data (σ > 0.5), the final searched model was able to capture the trend of data, showing our method’s robustness. Below we report test RMSE results with the discovered structure (LIN: linearity, PER: periodicity, SE: locality) The results indicate that the pipeline maintains reasonable performance under moderate noise levels, consistently capturing core patterns in the data. We also observed that as the data becomes noisier, the identified models more frequently incorporate local components (e.g., SE), suggesting a tendency to reflect finer-scale patterns in noisier settings. This may hint that local components like SE absorb the noise or outlier factors, and it could be exploited to deal with noisy or outlier cases by post-processing those.\n\n| noise level | σ=0 | σ=0.03 | σ=0.05 |\n| :---- | :---- | :---- | :---- |\n| Airline (RMSE) | 0.0534 | 0.0864 | 0.0725 |\n| Airline (Model) | LIN \\* (PER \\+ SE) | LIN \\* PER \\+ SE | SE \\* (PER \\+ C) |\n| Mauna (RMSE) | 0.0564 | 0.0656 | 0.1073 |\n| Mauna (Model) | (SE \\+ LIN) \\* (PER \\+ C) | SE \\* PER \\+ C | (SE \\+ C) \\* (PER \\+ C) |\n\n### **Cost of Multi-step VLM inference**\n\n> *Multi-step VLM inference could be expensive, unclear how it scales*\n\n> *How expensive is multi-step VLM inference really? Is this practical at scale?*\n\nSurprisingly, it is not expensive compared to [C1, C2].\nTo ensure how multi-step VLM inference consumes, we have calculated the cost of multi-step inference in our pipeline. The result shows that 1) visual representation of data reduces each step’s token usage 2) multi-step inference has the effect of reducing the whole round, leading to a whole cost reduction. So such characteristics of multi-step VLM inference reduces the total cost of our pipeline, making it more efficient.\n\nFirst, our data encoding with plots of AnalyzerVLM is token-independent, making the token usage the same regardless of the number of data points. In contrast, the LLM-based discovery frameworks [C1, C2] often utilize data directly into textual prompts, which leads to the excessive use of tokens. \nFor example, to encode around 500-1000 points, AnalyzerVLM incorporates images to 1045 tokens, while LLM based encoding takes around 8000 tokens per round. This not only rises token usage but also limits scalability when dealing with longer sequences.\n\nSecond, for the scalability of the multi-step pipeline, our AnalyzerVLM takes the step of 6 per round on average, with a total 5 rounds through the input of 1,500 tokens including both text and image tokens per step. Therefore, the total token consumption can be approximated at around 45,000 tokens (\\\\$0.03 with GPT-4o-mini, \\\\$0.25 with GPT-4o in total). Compared to the LLM-based model discovery [C1, C2], which relies on heavy iterations(10~50 rounds) with one-step generation may require around 8000*10=80000 tokens, our methods effectively generate a reliable model, making an effect of reducing overall cost. \n\nOur method incorporates VLM with multi-step reasoning, which can lead to the reduction of total rounds through multi-step reasoning, and the vision-based encoding of the data to make the number of tokens.\n\n|  | token usage per step | Number of steps | Number of Rounds | Total token usage |\n| :---- | :---- | :---- | :---- | :---- |\n| One-step LLM (LLM as optimizer [C4]) | 8000 | 1 | \\~10 | 80,000 (\\$0.053) |\n| Ours (Multi-step VLM) | 1500 | 6 | 5 | 45,000 (\\$0.030) |\n\n\\[C1\\] Yang et al., Large Language Models as Optimizers, ICLR 2024\\.  \n\\[C2\\] Merler et al., In-Context Symbolic Regression: Leveraging Large Language Models for Function Discovery, ACL 2024\\.\n\n### **Hyperparameter selection principle for VIC weighting**\n\n> *VIC weighting (α) and scoring thresholds lack principled selection*\n\n> *How do you choose α (varies 1000x between applications) without extensive tuning?*\n\nWe set $\\alpha$ so that $\\alpha$ * EvaluatorVLM accounts for approximately 10-30% of the original metric (e.g., BIC), and performed grid search around this range to find the best $\\alpha$.\nSince the scales of the selection criteria vary due to data normalization, we set α such that the visual score contributes a consistent proportion to the total objective. As a result, α may seem vary significantly between tasks, but to maintain this relative weighting between visual and numerical components.\n\nWe also provide the result of test RMSE of grid search varying α around 0 to 100, at Airline and Radio dataset. As shown, there is a certain turning point (or a break-down point) of alpha around 50. Also, we describe qualitative results: while α is 0, final searched models are dropping at extrapolated region, and when $\\alpha$ increases, the pipeline tends to select the model with more generality even in high value ranges near 70. This hints that our hyperparameter setting is not sensitive and starting from introducing small α would be a good rule of thumb. We will add qualitative samples in Appendix. \n\n| alpha | 0 | 30 | 50 | 70 | 100 |\n| :---- | :---- | :---- | :---- | :---- | :---- |\n| Airline | 0.0937 | 0.0574 | **0.0534** | 0.0612 | 0.0824 |\n| Radio | 0.0766 | 0.0715 | **0.0562** | 0.0764 | 0.0954 |\n\n### **When does visual assessment give wrong answers?**\n\nWe have observed the cases that the visual assessment of our VIC gives wrong answers:  \n\n-  When data/prediction line is too stiff\n\n- When the mean prediction is out of the plot range and the confidence interval fills the whole plot  \n\n- When the confidence interval grows at each side (high uncertainty at OOD), but the mean prediction shows a maintained structure  \n\nTo ensure stable and reliable assessment in such cases, we incorporated numerical evaluation into VIC, providing a consistent anchor point for visual scoring. In addition, we have averaged the scores across multiple runs (2 for our main experiments, but this averaging process can be scaled further if required), which stabilizes the assessments without incurring high computational costs. \n\nSearching for a better plot format for stiff data, may enhance the performance. We believe future work for adaptive plot formatting strategies,: e.g., scaling y-axis, emphasizing residuals, or visualizing derivatives, can be adopted to our pipeline to enhance structure visibility."}, {"review_id": "0jZN3ffJ7R", "reviewer": "Reviewer_fF4a", "summary": "This paper proposes a multi-modal and multi-step pipeline for automated model discovery. The core innovation lies in integrating vision-language models into the model proposal and evaluation process. Specifically, the authors introduce AnalyzerVLM, which conducts iterative analyses by dynamically generating and executing code to understand data trends and propose candidate models, and EvaluatorVLM, which assesses model quality using a novel Visual Information Criterion. VIC combines traditional Bayesian Information Criterion with VLM-based visual evaluations to balance model fit and generalizability. The pipeline operates in four stages: model proposal, parameter fitting, evaluation, and selection, iterating these steps to refine model quality.\nThe authors evaluate their approach on several real-world time series datasets and symbolic regression tasks, showing that it outperforms traditional methods and recent LLM-based systems in both training fit and test generalization. Ablation studies highlight the importance of multi-modal representations and multi-step reasoning for achieving strong performance. Overall, the work aims to mimic human expert behaviors in automated scientific discovery, reducing reliance on predefined grammars and manual interventions.", "questions": "1.The method is currently demonstrated only on univariate datasets. Do the authors have thoughts or preliminary experiments on how their approach could handle multivariate or high-dimensional data, especially given the challenges of visualizing such data?\n2.Appendix A.5 mentions averaging VLM assessments across two runs, but how consistent are these evaluations? Could the authors provide quantitative evidence of agreement between VLM outputs, or discuss how the system handles conflicting evaluations?\n3.In scientific applications of symbolic regression, model interpretability and formula readability are often as critical as predictive accuracy. Simple, human-readable expressions can provide insights and foster understanding of the underlying phenomena. Could the authors comment on the interpretability of the symbolic regression formulas discovered by their method? Are there mechanisms within the proposed pipeline—or could there be extensions—to encourage the discovery of more concise and interpretable expressions?", "limitations": "Yes. The authors explicitly discuss the current restriction to 1D datasets and the reliance on visualization quality in Section 5. However, they could further elaborate on potential challenges in scaling their approach to high-dimensional or noisy data, and on the implications of using closed-source VLMs for reproducibility.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths：\t\n1.The paper presents a novel multi-modal, multi-step pipeline for automated model discovery, leveraging vision-language models to imitate human expert reasoning and visual judgment.\n2.The introduction of the Visual Information Criterion to combine perceptual assessment with statistical metrics is an original and impactful contribution.\n3.Experiments on time-series forecasting and symbolic regression demonstrate superior generalization performance over state-of-the-art baselines such as BoxLM and Automatic Statistician. \n\nWeaknesses:\n1.The current pipeline is demonstrated only on univariate (1D) datasets. While the approach shows promise, the visualization-based reasoning employed by EvaluatorVLM may not directly generalize to high-dimensional or multivariate data, where visual representation becomes ambiguous or infeasible. In such cases, the system would inevitably need to rely on numerical metrics or alternative modalities, raising questions about scalability.\n2.Using VLMs to judge fitness and structure similarity is a novel and intriguing idea, but it also raises concerns about reliability. This is especially critical for extrapolated regions where human-like intuition from VLMs may not align with numerical accuracy. Appendix A.5 mentions averaging VLM assessments across two runs, but it is unclear how consistent these judgments are. How often do VLMs provide conflicting evaluations, and how does the system handle such cases?\n3.The design of the Visual Information Criterion assumes that combining perceptual assessments from VLMs with BIC yields a balanced evaluation of model fitness and complexity. However, it is unclear how sensitive the final model selection is to the choice of the weighting parameter \\alpha in VIC, or whether the method is robust to different trade-offs between these components. A deeper analysis or justification of this design choice would strengthen confidence in the approach.", "was_revised": true, "final_justification": "I appreciate the authors addressed most of my questions and argued about my concerns. But I still don't think it is reliable to use VLMs to judge fitness and structure similarity. Furthermore, the rebuttal approved the correctness of the paper, but didn't change the significance and novelty of the paper, so I keep my score.", "rebuttal": "We thank reviewer fF4a for valuable comments and recognizing our work as:\n\n- **A novel multi-modal, multi-step pipeline for automated model discovery**, leveraging vision-language models to imitate human expert reasoning and visual judgment. \n\n- The introduction of the VIC to combine perceptual assessment with statistical metrics is **an original and impactful contribution.**   \n- VLMs to judge fitness and structure similarity as **a novel and intriguing idea.**\n\nWe have addressed concerns and comments in below.\n\n### **Generalibility to high-dimensional & multivariate data**\n\n> *The current pipeline is demonstrated only on univariate (1D) datasets.*\n> *Do the authors have thoughts or preliminary experiments on how their approach could handle multivariate?*\n\nWhile a similar vein of research [13, 29, 32], including ours, has been only focused on the univariate data setting due to its importance and its broad applications, we agree that extending to the multi-variate case is an interesting direction.\n\nGiven the limited rebuttal period, while we could not show a complete extension at this moment, we emphasize that our pipeline is not limited to the univariate case. Our proposed multi-modal multi-step pipeline is modular; thus, extending each module to multi-variate ones enables multi-variate data analysis. In this sense, we discuss the following potential directions.\n\n1\\) Since AnalyzerVLM conducts its analysis through code execution, AnalyzerVLM operation is **agnostic to the number of dimensions,** so applying it directly to multivariate data is possible. We conducted a preliminary experiment of AnalyzerVLM on multivariate data, and we found that AnalyzerVLM **checks the data shape** and **conducts the visualization per dimension**, and analyzes **pairwise interactions across dimensions**. Such analysis can be adopted per-dimension modeling or the adoption of similar models with highly correlated dimension, which is a good starting point of model discovery.\n\n2\\) For EvaluatorVLM, a simple and straightforward extension would be evaluating **each output dimension independently** and **aggregating scores**, similar to how mean squared errors are averaged across dimensions. \nAs high-dimensional data modeling naturally incorporates the feature extraction to use good feature space \\[C1, C2\\] and enables finding interpretability at the feature space, we can adopt the new format of the plots (e.g., PCA/t-SNE, or feature space visualization) for EvaluatorVLM. So the format of plot may change for high-dimensional or multivariate datasets, but **the core idea of vision-based analysis and evaluation can be scaled up** upon our pipeline.\n\n\\[C1\\] Wilson et al., Deep Kernel Learning, Artificial intelligence and statistics. PMLR, 2016\\.  \n\\[C2\\] Alberto et al., Feature extraction and image processing for computer vision. Academic press, 2019\\.\n\n### **VLM assessments consistency**\n\nWe appreciate the concern, especially on how VIC can be critical at extrapolate region, and visual assessment’s consistency. \n\nFirst, VIC is designed to regularize the traditional numerical metric with human intuition about data structure, allowing us to benefit from both traditional metrics and human intuition. As shown in Fig.9 in our paper, EvaluatorVLM’s evaluation shows a correlation with human evaluation, as well as the accuracy gap between interpolation and extrapolation regions, suggesting that the visual assessments capture not only numerical performance but also structural cues aligned with human intuition.\n\nSecond, to assess the consistency of EvaluatorVLM, we conduct the experiments and report the standard deviation of the evaluations with 10 runs for models that are suggested during our pipeline. Since evaluatorVLM produces discrete scores on a 5-point scale in the range \\[0, 50\\], we find the variation to be within acceptable ranges (average of 3.11 and 5.44).\n\n|  | Airline | Solar | Mauna | Wheat | Temperature | Avg. |\n| :---- | :---- | :---- | :---- | :---- | :---- | :---- |\n| visual fitness | 2.4442 | 4.328 | 3.973 | 2.923 | 1.937 | 3.11 |\n| visual generalizability | 3.4509 | 1.951 | 7.042 | 6.200 | 3.815 | 5.44 |\n\nAnd we have checked top cases that cause the EvaluatorVLM to produce inconsistent judgments:\n\n1) When the mean prediction doesn’t appear, the confidence interval covers all areas, EvaluatorVLM tends to be confused\n2) When the data or prediction is too stiff, the evaluation varies\n3) When the confidence interval expands at extrapolation (high uncertainty), but the mean prediction remains visually reasonable, EvaluatorVLM tends to assign low scores due to its uncertainty, and high scores for the mean predictions.\n\nTo ensure stable and reliable assessment at such cases, we incorporated numerical evaluation into VIC, providing a consistent anchor point for visual scoring. In addition, we have averaged the scores across multiple runs (2 for our main experiments, but this averaging process can be scaled further if required), which stabilizes the assessments without incurring high computational costs. \n\n### **Hyperparameter α selection & sensitivity**\n\n> *how sensitive the final model selection is to the choice of the weighting parameter $\\alpha$ in VIC?*\n\nWe set $\\alpha$ so that $\\alpha$ * EvaluatorVLM accounts for approximately 10-30% of the original metric (e.g., BIC), and performed grid search around this range to find the best $\\alpha$.\nSince the scales of the selection criteria vary due to data normalization or criteria (e.g., BIC vs. NMSE) we set α such that the visual score contributes a consistent proportion to the total objective. As a result, α may seem vary significantly between tasks, but to maintain this relative weighting between visual and numerical components.\n\nWe also provide the result of test RMSE of grid search varying α around 0 to 100, at Airline and Radio dataset. As shown, there is a certain turning point (or a break-down point) of alpha around 50. Also, we describe qualitative results: while α is 0, final searched models are dropping at extrapolated region, and when $\\alpha$ increases, the pipeline tends to select the model with more generality even in high value ranges near 70. This hints that our hyperparameter setting is not sensitive and starting from introducing small α would be a good rule of thumb. We will add qualitative samples in Appendix. \n\n\n| alpha | 0 | 30 | 50 | 70 | 100 |\n| :---- | :---- | :---- | :---- | :---- | :---- |\n| Airline | 0.0937 | 0.0574 | **0.0534** | 0.0612 | 0.0824 |\n| Radio | 0.0766 | 0.0715 | **0.0562** | 0.0764 | 0.0954 |\n\n### **Interpretability of formulas discovered by our pipeline**\n\n> *Could the authors comment on the interpretability of the symbolic regression formulas discovered by their method?*\n\nTo demonstrate this, we report examples of formulas and discovered model:\n\nFor Keijzer 3, the ground truth formula is $0.3x+\\sin(2πx)$, and the discovered formula is $0.2963x+\\sin(6.2781x)−0.001$, which closely matches the true function, both in structure and parameter values.  \nWe applied our method to the classical physics formula of free fall motion: $y(t)=v_0t-\\frac{1}{2}gt^2$, while $v_0=10$ and $g=9.81$. And the discovered final formula is $-4.9065x^2 + 10.0164x - 0.0042$, effectively showing that our pipeline can successfully recover the interpretable equation from data. \n\nWe also provide the discovered kernel structure interpretation of the Radio dataset (Fig. 2 in our paper), which our discovered model $\\text{PER} \\* (\\text{PER} \\+ \\text{SE})$ revealed two periodic components: 10.25 years for the first period, 1.01 years for the second period, which alignes the data’s characteristics with multiple periods that the dataset has.\n\nThe formulas and models discovered by our pipeline are not only compact but also structurally interpretable, often aligning with forms that are familiar and meaningful to human experts, highlighting that our method is effective in producing expressions that are both accurate and explainable."}, {"review_id": "Cv5z2XyTlT", "reviewer": "Reviewer_MXP2", "summary": "This work focuses on automated model discovery in 1D data. A multi-step pipeline is designed and the pipeline leverages the multimodal information. An interesting part is replacing the role of human in the pipeline in observing results figures with a vision-language model to understand the results via capturing relationships and trends. The evaluation part also benefits from leveraging automatically generated code. Quantitative results are demonstrated to justify the effectiveness of the proposed approach in the discovered model’s performance in downstream datasets. Ablation study also demonstrate the performance of leveraging multimodal information compared to text only information.", "questions": "An example on the resulting model and corresponding parameters would facilitate the understanding. \n\nWhat’s the resolution of the results plot? Any example on the code generated?\n\nHow exactly are the results in Figure 3 for LLM generated? Are text based results fed into the model in pure number series or are they incorporated into python code using matplotlib (or whatever other visualization tools) which are further used to generate visual figures? For a fair comparison I expect the text-form results to be generated from something in code-style. \n\nLine177 typo.", "limitations": "yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nThis work explores the possibility of leveraging multimodal large language models to replace the role of human in automated model architecture design, which is interesting. \nExperiments are conducted in multiple datasets with competitive results.\nThe multi-step pipeline design is reasonable in the form of an agent system.\n\n\n\nWeaknesses:\n\nThe analysis is not comprehensive enough [Figure 2]: How the model processes different visualizations [line 47-48]? For example, human often observe the plots to identify whether a certain model is overfitting, is underfitting, or has a good generalization capability, or the loss might be strongly shaking, or observe some unnormal curves. How does the proposed approach perform in each case? The meaning of the y-axis in a plot could also significantly change the information a plot delivers (e.g., accuracy is increasing or the loss is increasing).\n\nI think the visual quality is indeed important. This component is a clear difference compared to prior work but this component is not comprehensively analyzed regarding its quality, complexity, resolution, etc.\n\n[line25-26] how does this approach balance interpretability and model fit?\n\nWhat code is generated to evaluate what aspects? There are many aspects to evaluate including OOD, robustness, etc. How to automate these aspects via LLM? I understand it might be too much to be included in one single paper, but relevant discussions would be helpful.\n\n[line39-40] what is the tool pool? What tools exist and what does not exist in the searching space? \n\nI like the general idea but more evaluations and implementations details would strengthen the contributions and facilitate the understanding.", "was_revised": true, "final_justification": "The authors have addressed my concerns so my final rating is positive. Considering the work focuses only on 1D data, the impact might be somewhat limited so I don't rate a higher score.", "rebuttal": "We thank reviewer MXP2 for valuable comments, and recognizing our work as:\n\n- **Possibility of leveraging multimodal large language** models to replace the role of human **in automated model architecture design**, which is interesting\n- Replacing the role of human in the pipeline in observing results figures with a vision-language model as i**nteresting**, to understand the results via capturing relationships and trends\n- and also checking our typo\n\nWe have addressed MXP2’s concerns and questions below.\n\n### **Ablation study on different visualizations**\n\n> *How the model processes different visualizations \\[line 47-48\\]?*\n\nWe agree with your comment that to interpret the plot correctly requires not only the visual structure of the visualization, but also the semantic meaning of its axis and different visual quality. \nAs the reviewer suggested, we have also conducted how EvaluatorVLM responds to different axis plot visualizations (loss curves, accuracy) for three different types of cases: 1) model overfitting, 2) model underfitting (unstable training & validation curves), and 3) model well-fitted (stable training & validation curves).\nIn the case, we have found that EvaluatorVLM has evaluated the underfitting or overfitting case with the given loss curve or accuracy, interpreting the case with stable training loss and unstable validation loss as a low generability case, and a relatively higher fitness score. With the shaking loss curve, EvaluatorVLM judged the model as low fitness and low generalizability, assigning low scores.\n\n\n> *I think the visual quality is a clear difference compared to prior work but this component is not comprehensively analyzed regarding its quality, complexity, resolution…*  \n\nWe also conducted an ablation study of the plot components to examine how EvaluatorVLM processes different plot quality, including resolution, color, and thickness. We report the correlation between human annotation and EvaluatorVLM’s answer. As shown, good results were obtained with 1) resolution 800x600, 600x250 2) red and blue line are preferred and 3) linewidths 1, 2 are most preferred. For fitness, thinner lines (1, 2) were preferred for fine-grained comparison, and for generalizability, linewidth 3 is preferred.\nWe thank the reviewer for the important comments, and we will add the ablation study to the final version.\n\n| Resolution | 128x128 | 300x200 | 400x300 | 600x600 | 800x600 | 600x400 | 600x250 | 1200x500 |\n| :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- |\n| visual fitness | 0.293 | 0.635 | 0.595 | 0.410 | 0.612 | 0.647 | **0.725** | 0.595 |\n| visual generalibility | 0.194 | 0.630 | 0.601 | 0.586 | **0.844** | 0.658 | 0.699 | 0.511 |\n| average | 0.244 | 0.632 | 0.598 | 0.498 | **0.728** | 0.653 | 0.712 | 0.553 |\n\n| Color | Red | Blue | Green |\n| :---- | :---- | :---- | :---- |\n| visual fitness | **0.812** | 0.804 | 0.703 |\n| visual generalibility | 0.673 | **0.782** | 0.719 |\n| average | 0.743 | **0.793** | 0.711 |\n\n| Linewidth | linewidth=1 | linewidth=2 | linewidth=3 | linewidth=4 | linewidth=5 |\n| :---- | :---- | :---- | :---- | :---- | :---- |\n| visual fitness | 0.764 | **0.817** | 0.622 | 0.569 | 0.631 |\n| visual generalibility | 0.766 | 0.742 | **0.799** | 0.753 | 0.744 |\n| average | 0.765 | **0.780** | 0.711 | 0.661 | 0.688 |\n\n### **Methods for balancing interpretability and model fit**\n\nTraditional methods for interpretable model selection have utilized a complexity term to regularize the model not to be in a complex form. However, such complexity-based regularization alone is often insufficient for discovering truly meaningful or high-quality models. Our method addresses this limitation by going beyond complexity control, incorporating human-like model suggestion and evaluation to better balance interpretability and model fit.\n\n### **AnalyzerVLM’s generated code analysis**\n\n> What code is generated to evaluate what aspects? There are many aspects to evaluate including OOD, robustness, etc...*\n\nWe have shown the example of AnalyzerVLM's analysis at the Appendix. AnalyzerVLM focuses on suggesting a good model with the analysis, covering OOD analysis and whether the model is robust. We categorize the behaviors of AnalyzerVLM as follows:\n\n1\\. Numerical analysis \n\n- Directly printing out the data or predictions sometimes  \n- Period estimation: using scipy.signal.find\\_peaks() to detect and calculate peak intervals  \n- Linearity estimation: via np.polyfit() to evaluate global linear trends\n\n2\\. Plot-based visualization\n\n- When data & model access code snippet is given, AnalyzerVLM tends to choose visualizing data and model’s prediction using plt.plot(), analyzing the plot whether the overall trend suddenly drops or maintained at OOD regions.  \n- AnalyzerVLM tends to calculate the residual of data and model’s prediction, to check whether current model is fully reflecting the data’s trend. If the residual shows any characteristic, AnalyzerVLM tends to assign a new model with its characteristic.  \n- To check the residual’s characteristics, AnalyzerVLM tends to draw the residual distribution or histogram (to check whether it is skewed), or the correlations between input and residuals.\n\nWe also provide the examples of AnalyzerVLM checking the data’s characteristics and residual distribution visualization below.  \n```python\n# Analyze for periodicity\npeaks, _ = find_peaks(y)\nperiods = np.diff(X[peaks])\nif len(periods) > 0:\n    estimated_period = np.mean(periods)\nelse:\n    estimated_period = None\n\n# Analyze for smoothness (lengthscale)\ndifferences = np.diff(y)\nlengthscale = np.mean(np.abs(differences)) if len(differences) > 0 else None\n\n# Analyze for linearity (slope and offset)\nslope, intercept = np.polyfit(X, y, 1)\nprint(f\"Estimated Period: {estimated_period}\")\nprint(f\"Estimated Lengthscale: {lengthscale}\")\nprint(f\"Estimated Slope: {slope}, Estimated Offset: {intercept}\")\n\n# Visualize residuals distribution\nplt.figure(figsize=(10, 6))\nsns.histplot(residuals, bins=30, kde=True, color='blue')\nplt.axvline(0, color='red', linestyle='--', label='Zero Line')\nplt.savefig('./tmpimgs/residual_distribution_analysis.png')\n```\n\n\n### **What tools exist and what does not exist in the searching space?**\nFor the tool pool, we did not explicitly give the list of tools that AnalyzerVLM can use, but relies on AnalyzerVLM’s internal ability to analyze using Python. But we have checked that AnalyzerVLM mostly relies on such tools: 1) matplotlib and seaborn, 2) numpy, 3) scikit-learn 4) scipy. With such tools, AnalyzerVLM conducts the multi-step analysis for recognizing its data’s characteristics.\n\n### **Example of the resulting model and corresponding parameters**\n\nTo demonstrate this, we provide the example of the searched kernel result of Radio dataset (Fig.2). \n\nOur discovered model PER * (PER + SE) revealed two periodic components: 10.25 years for period 1, 1.01 years for the second period, which aligns the data’s characteristics with the multiple periods that the dataset has. Also SE showed small local structure, smoothed trend (SE lengthscale: 53.8 years).\n\nOverall, this result shows that our pipeline can automatically discover models with interpretable kernel structures and parameters, revealing meaningful characteristics such as periodicities and smooth trends inherent in the data.\n\n### **Clarification of Figure 2 & 3**\n\n> *What’s the resolution of the results plot?*\n\nThe resolution of the results plot (Fig. 2) is 2000x560.\n\n> *How exactly are the results in Figure 3 for LLM generated? Are text based results fed into the model in pure number series or are they incorporated into python code using matplotlib...*\n\nWe understand the concerns about the settings of text-based analysis and our multimodal analysis at Figure 3\\. As you mentioned, for text-based results, we did not feed the pure number series but gave the code snippets for accessing data to be incorporated into python code. Also we restricted it not to use visualization tools, but allowed only numerical analysis results. We have seen that the AnalyzerLLM conducts numerical analysis to understand data’s overall structure: linearity check (to find out its shape is linear), or periodicity check (peak interval variance), or mean/variance check."}]

Author rebuttal:

Meta-review: This paper takes a creative step by using VLMs as both analyzers and evaluators for automated model discovery. The Visual Information Criterion is a particularly interesting idea with human-like intuition. The reviewers agreed the work is technically solid, with promising results. Concerns around scaling to multivariate data, consistency of VLM-based judgments, and interpretability were discussed in the rebuttal with a few new small experiments. The work has clear limitations in scope but it also opens up an exciting direction worth accepting.
Decision: Accept (poster)