Title: Automated Model Discovery via Multi-modal & Multi-step Pipeline
Venue: NeurIPS 2025
Accepted: yes
Keywords: Model Discovery

Time-Series Data

Vision Language Models

Model Selection

LLM Reasoning
Abstract: Automated model discovery is the process of automatically searching and identifying the most appropriate model for a given dataset over a large combinatorial search space. Existing approaches, however, often face challenges in balancing the capture of fine-grained details with ensuring generalizability beyond training data regimes with a reasonable model complexity. In this paper, we present a multi-modal \& multi-step pipeline for effective automated model discovery. Our approach leverages two vision-language-based modules (VLM), AnalyzerVLM and EvaluatorVLM, for effective model proposal and evaluation in an agentic way. AnalyzerVLM autonomously plans and executes multi-step analyses to propose effective candidate models. EvaluatorVLM assesses the candidate models both quantitatively and perceptually, regarding the fitness for local details and the generalibility for overall trends. Our results demonstrate that our pipeline effectively discovers models that capture fine details and ensure strong generalizability. Additionally, extensive ablation studies show that both multi-modality and multi-step reasoning play crucial roles in discovering favorable models.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work focuses on automated model discovery in 1D data. A multi-step pipeline is designed and the pipeline leverages the multimodal information. An interesting part is replacing the role of human in the pipeline in observing results figures with a vision-language model to understand the results via capturing relationships and trends. The evaluation part also benefits from leveraging automatically generated code. Quantitative results are demonstrated to justify the effectiveness of the proposed approach in the discovered model’s performance in downstream datasets. Ablation study also demonstrate the performance of leveraging multimodal information compared to text only information.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThis work explores the possibility of leveraging multimodal large language models to replace the role of human in automated model architecture design, which is interesting. \nExperiments are conducted in multiple datasets with competitive results.\nThe multi-step pipeline design is reasonable in the form of an agent system.\n\n\n\nWeaknesses:\n\nThe analysis is not comprehensive enough [Figure 2]: How the model processes different visualizations [line 47-48]? For example, human often observe the plots to identify whether a certain model is overfitting, is underfitting, or has a good generalization capability, or the loss might be strongly shaking, or observe some unnormal curves. How does the proposed approach perform in each case? The meaning of the y-axis in a plot could also significantly change the information a plot delivers (e.g., accuracy is increasing or the loss is increasing).\n\nI think the visual quality is indeed important. This component is a clear difference compared to prior work but this component is not comprehensively analyzed regarding its quality, complexity, resolution, etc.\n\n[line25-26] how does this approach balance interpretability and model fit?\n\nWhat code is generated to evaluate what aspects? There are many aspects to evaluate including OOD, robustness, etc. How to automate these aspects via LLM? I understand it might be too much to be included in one single paper, but relevant discussions would be helpful.\n\n[line39-40] what is the tool pool? What tools exist and what does not exist in the searching space? \n\nI like the general idea but more evaluations and implementations details would strengthen the contributions and facilitate the understanding.\n\nQUESTIONS:\nAn example on the resulting model and corresponding parameters would facilitate the understanding. \n\nWhat’s the resolution of the results plot? Any example on the code generated?\n\nHow exactly are the results in Figure 3 for LLM generated? Are text based results fed into the model in pure number series or are they incorporated into python code using matplotlib (or whatever other visualization tools) which are further used to generate visual figures? For a fair comparison I expect the text-form results to be generated from something in code-style. \n\nLine177 typo.\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a multi-modal and multi-step pipeline for automated model discovery. The core innovation lies in integrating vision-language models into the model proposal and evaluation process. Specifically, the authors introduce AnalyzerVLM, which conducts iterative analyses by dynamically generating and executing code to understand data trends and propose candidate models, and EvaluatorVLM, which assesses model quality using a novel Visual Information Criterion. VIC combines traditional Bayesian Information Criterion with VLM-based visual evaluations to balance model fit and generalizability. The pipeline operates in four stages: model proposal, parameter fitting, evaluation, and selection, iterating these steps to refine model quality.\nThe authors evaluate their approach on several real-world time series datasets and symbolic regression tasks, showing that it outperforms traditional methods and recent LLM-based systems in both training fit and test generalization. Ablation studies highlight the importance of multi-modal representations and multi-step reasoning for achieving strong performance. Overall, the work aims to mimic human expert behaviors in automated scientific discovery, reducing reliance on predefined grammars and manual interventions.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths：\t\n1.The paper presents a novel multi-modal, multi-step pipeline for automated model discovery, leveraging vision-language models to imitate human expert reasoning and visual judgment.\n2.The introduction of the Visual Information Criterion to combine perceptual assessment with statistical metrics is an original and impactful contribution.\n3.Experiments on time-series forecasting and symbolic regression demonstrate superior generalization performance over state-of-the-art baselines such as BoxLM and Automatic Statistician. \n\nWeaknesses:\n1.The current pipeline is demonstrated only on univariate (1D) datasets. While the approach shows promise, the visualization-based reasoning employed by EvaluatorVLM may not directly generalize to high-dimensional or multivariate data, where visual representation becomes ambiguous or infeasible. In such cases, the system would inevitably need to rely on numerical metrics or alternative modalities, raising questions about scalability.\n2.Using VLMs to judge fitness and structure similarity is a novel and intriguing idea, but it also raises concerns about reliability. This is especially critical for extrapolated regions where human-like intuition from VLMs may not align with numerical accuracy. Appendix A.5 mentions averaging VLM assessments across two runs, but it is unclear how consistent these judgments are. How often do VLMs provide conflicting evaluations, and how does the system handle such cases?\n3.The design of the Visual Information Criterion assumes that combining perceptual assessments from VLMs with BIC yields a balanced evaluation of model fitness and complexity. However, it is unclear how sensitive the final model selection is to the choice of the weighting parameter \\alpha in VIC, or whether the method is robust to different trade-offs between these components. A deeper analysis or justification of this design choice would strengthen confidence in the approach.\n\nQUESTIONS:\n1.The method is currently demonstrated only on univariate datasets. Do the authors have thoughts or preliminary experiments on how their approach could handle multivariate or high-dimensional data, especially given the challenges of visualizing such data?\n2.Appendix A.5 mentions averaging VLM assessments across two runs, but how consistent are these evaluations? Could the authors provide quantitative evidence of agreement between VLM outputs, or discuss how the system handles conflicting evaluations?\n3.In scientific applications of symbolic regression, model interpretability and formula readability are often as critical as predictive accuracy. Simple, human-readable expressions can provide insights and foster understanding of the underlying phenomena. Could the authors comment on the interpretability of the symbolic regression formulas discovered by their method? Are there mechanisms within the proposed pipeline—or could there be extensions—to encourage the discovery of more concise and interpretable expressions?\n\nLIMITATIONS:\nYes. The authors explicitly discuss the current restriction to 1D datasets and the reliance on visualization quality in Section 5. However, they could further elaborate on potential challenges in scaling their approach to high-dimensional or noisy data, and on the implications of using closed-source VLMs for reproducibility.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
This work levrages two Vision-Language Models (AnalyzerVLM + EvaluatorVLM) to automatically discover mathematical models for time series data. AnalyzerVLM iteratively analyzes data through code generation, EvaluatorVLM evaluates models using a novel Visual Information Criterion (VIC) that combines traditional metrics with visual assessment.

STRENGTHS AND WEAKNESSES:
Strengths

+ Intresting to use VLMs for automated model discovery with human-like visual reasoning
+ Outperforms baselines (ARIMA, Prophet, etc.) across different datasets with better generalization
+ Captures human intuition about model reasonableness through visual evaluation

Weaknesses

- Only 1D time series was tested
- Multi-step VLM inference could be expensive, unclear how it scales
- VIC weighting (α) and scoring thresholds lack principled selection

QUESTIONS:
How does this work with high-dimensional data where simple plots aren't possible?
How do you choose α (varies 1000x between applications) without extensive tuning?
When does visual assessment give wrong answers? What about noisy/counter-intuitive data?
How expensive is multi-step VLM inference really? Is this practical at scale?

LIMITATIONS:
See Weaknesses above.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03