MATHMO: Automated Mathematical Modeling Through Adaptive Search
Authors: Tennison Liu
Mihaela van der Schaar
Venue: ICLR 2026 Poster (2026)
Keywords: automated modeling
autoformulation
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Mathematical modeling is the process of understanding and predicting complex real-world phenomena. 
Traditionally, it is a time-intensive effort reliant on deep human expertise and iterative refinement. 
Automating this intricate process, therefore, offers the potential to significantly accelerate discovery and broaden the application of mathematical modeling across diverse domains. Such automation, however, must address inherent challenges, including fundamental modeling uncertainty, balancing multiple conflicting objectives, and incorporating subjective qualities into assessing model utility.
We approach this by conceptualizing mathematical modeling as a sequential decision-making problem under uncertainty. 
In response, we introduce $\texttt{MATHMO}$, a novel adaptive search method designed to automatically navigate the complex decisions in selecting mathematical frameworks, specifying model formulations, and defining algorithmic procedures. 
Specifically, $\texttt{MATHMO}$ employs a principled bi-level search strategy---combining high-level exploration across diverse frameworks and local intra-framework model refinements---leveraging Large Language Models for exploration, surrogate evaluations, and incorporating subjective preferences into the automated process. We demonstrate $\texttt{MATHMO}$'s efficacy on diverse real-world tasks, where it successfully discovers Pareto-efficient frontiers of models that balance varied objectives, including subjective criteria.

Published as a conference paper at ICLR 2026
MATHMO: AUTOMATED MATHEMATICAL MODELING THROUGH ADAPTIVE SEARCH
Tennison Liu, Mihaela der Schaar DAMTP, University of Cambridge Cambridge, UK
t15220 cam alaamaacuur uk
ABSTRACT
Mathematical modeling is the process of understanding and predicting complex real-world phenomena. Traditionally, it is a time-intensive effort reliant on deep hu- man expertise and iterative refinement. Automating this intricate process, therefore, offers the potential to significantly accelerate discovery and broaden the application of mathematical modeling across diverse domains. Such automation, however, must address inherent challenges, including fundamental modeling uncertainty, bal- ancing multiple conflicting objectives, and incorporating subjective qualities into assessing model utility. We approach this by conceptualizing mathematical model- ing as a sequential decision-making problem under uncertainty. response, we introduce MA T HMO, novel adaptive search method designed to automatically navi- gate the complex decisions selecting mathematical frameworks, specifying model formulations, and defining algorithmic procedures. Specifically, MATHMO employs high-level exploration across
a principled bi-level search strategy-combine diverse frameworks and local fntra-framework model refinements-leverree Large Language Models for exploration, surrogate evaluations, and incorporating subjective preferences into the automated process. We demonstrate MATHMO's efficacy on diverse real-world tasks, where it successfully discovers Pareto-efficiees frontiers of models that balance varied objectives, including subjective criteria.
INTRODUCTION
Mathematical modeling is the art and science of translating complex real-world phenomena into precise mathematical language, allowing us to represent, understand, and predict complex situations. This capability is crucial in almost all aspects of life, from natural sciences and engineering to economics and social systems (Turing, 1990; Sugihara and May, 1990; Banwarth- Kuhn and Sindi, 2020). Indeed, the ability to abstract complex realities into mathematical models is often considered a key feature of intelligent civilization, enabling generalized problem-solving knowledge transfer, and accumulation of scientific understanding over time (Simon, 2019).
The time-consune mone-onnuuing and expertise-drine -diivvn nature of modeling, coupled with its inherent complexities, makes its automation highly appealing prospect. Automating this process could democratize access to powerful analytical models and tools, efficiently exploring trade-off's between multiple models, uncovering novel approaches or insights, and enhancing decision-making across diverse domains.
Several key characteristics distinguish the mathematical modeling process. Firstly. it is pursued in the face of fundamental uncertainty: the optimal framework or model specification often unknown priori, demanding iterative exploration involving building, testing, and refining models based on feedback (Jakeman et al., 2006). Secondly, modeling frequently contends with multiple, often conflicting, objectives (c.g., optimization model's solution quality versus runtime (Chandrasckaran and Jordan, 2013)). Thus, the aim is typically not a single 'best' model, but a diverse frontier models representing different trade-off's for the human modeler to investigate. Lastly, subjective qualities like interpretability and domain understanding further influence model utility beyond objective metrics (Dirac, 1963), making it crucial for them to be captured in the automated modeling process.
We conceptualize automated mathematical modeling as a sequential decision- decision-making problem. Here, the modeler makes series of choices: (1) the mathematical framework to employ, (2) the concrete

---

Published as a conference paper at ICLR 2026
model specification including appropriate representations, parameterizations and assumptions, and (3) the appropriate computational algorithms to obtain the desired outputs. The uncertainty lies in not knowing which modeling decisions will yield useful models, thus requiring principled approach of adaptive search that carefully balances exploration and exploitation given uncertainty.
In response, we introduce MATHMO, novel method designed automate key aspects of the mathe- matical modeling pipeline. At a high level, our system takes a modeling problem description and of objective functions, and discovers set of models that represent efficient trade-off's among these objectives. The adaptive search procedure in MATHMO operates under a bi-level structure. The upper-level performs adaptive resource allocation across different mathematical frameworks, while the low-level module initiates local search mechanism to explore each framework's 's model and algorithm space. Crucial to this process are Large language models (LLMss,, which are employed to sample realizations from the search space and to perform surrogate evaluations to improve search effi- ciency. Additionally, they are used as models of subjective evaluations, thus incorporating subjective model preferences into the modeling pipeline. In each round, the generated model and its evaluations are observed, informing and adapting the next iteration of modeling decisions.
Contributions. The primary contributions of this work are threefold: (1) We formally define auto- mated mathematical modeling, conceptualizing it as a sequential rrcision-making problem under uncertainty. (2) We present MATHMО, an adaptive search framework designed for this problem, capable of efficient exploration, balancing multiple objectives, and incorporating subjective modeling preferences. To the best of knowledge, this is the first work address this exciting problem area. (3) We demonstrate the efficacy of MATHMO on four diverse real-world modeling tasks (two prescriptive, two predictive), demonstrating its ability discover Pareto-efficient frontiers of models.
2 PRELIMINARIES
2.1 FORMALISM
Mathematical modeling is primarily a declarative endeavor (Van Roy and Haridi, 2004). The core cognitive task involves translating problem p P (potentially accompanied by dataset D,) into suitable formal representation, where the subsequent derivation of mathematical outputs is delegated to computational tools. We formalize this process of representation and derivation sequential decision-making problem under uncertainty (c.f. "Box's 's Loop" (Box, 1979)):
1. Selecting a high-level approach or framework f E F. where the chosen framework encodes foundational assumptions and provides access to specialized mathematical tools and techniques.
2. Specifying exact model m E M(f) within this framework, which is concrete mathematical formulation representing the system under study.
3. Developing a computational or algorithmic procedure a E A(f, m) to this model to derive the desired mathematical output
mDp)) E O.
4. Evaluating the mathematical output model characteristics to inform subsequent refinements.
This sequence of modeling decisions, denoted by (f, m. a), defines structured search space Ω {(f, m, a) if E F(d), m E M(f), E A(f,, m.)). We evaluate each modeling outcome using vector of k E N objective functions, [(m, a) each objective J.((--) can depend on both the mathematical outputs o (e.g., solution optimality) and characteristics of the model-output pair (e.g., runtime). The goal is then to identify modeling decisions (f, m. a) E Qt that address the following oujti-obeective optimization problem:
[m, a)...., Jo (m, a))²" For generality,
(imimize(,,m.jesr T(m, a) = (m,,), Th (7,ma))
(1)
As there typically does not exist single model-algorithm pair that can minimize all objective functions simultaneously, we are more interested in finding the Pareto optimal models, i.e., models that cannot be improved in any of its objectives without degrading Pareto optimal pair (m, a) is non-dominated, where a pair (m, a) is said to dominate another pair (m' a') if i J,,, a) < J,(m', a') and 3i E [k], (m,, a) J[(mm, (Miettinen, 1999).
least one. Mathematically,
Key challenges. This problem definition emits several noteworthy challenges:
1. Efficient exploration under uncertainty. A defining hallmark of the mathematical modeling process is the fundamental uncertainty. It involves making sequential choices (framework, model,
2

---

Published as a conference paper at ICLR 2026
arg
F(d)
fx
Mif' x ml)
Objective function
Problem descriplion P
Compustation f- a m(
(eC SOT Sponaaii
Set of models S
Dataset Dy
MOSE
jropised
wlettion
www.cotiion
Figure 1: Overview of MATHMO. Given a modeling problem desciption, MAMO employs a bi-level adaptive search strategy to identify Pareto set of models presenting diverse trade-offs.
algorithm) with uncertain outcomes in an interconnected and complex space. Efficient exploration, informed by prior beliefs and feedback, is thus crucial to navigating this search space.
Fundamental trade-offs. Modeling inherently involves balancing conflicting objectives, such as accuracy and interpretability. Different modeling frameworks (c.g., deep learning versus linear models) often embody fundamentally distinct trade-off frontiers, necessitating exploration across frameworks, beyond locally within a framework, to identify a set of Pareto efficient models.
3. Subjective qualities of models. Beyond objective metrics, subjective qualities like Occam's s Razor, interpretability, or alignment with domain understanding are integral model's utility." Although mathematical proxies for these elusive qualities exist (e.g.. sparsity, minimum description length), they are typically cramwwor-speciife and not directly comparable.
2.2 RELATED WORKS
Our work builds upon and extends several lines of work:
AutoML. The field of AutoML aims to automate applying machine learning to real-world problems (Thornton et al., 2013). This broad endeavor encompasses hyperparameter optimization (Snoek et al., 2012; Li et al., 2018), neural architecture search (Zoph and Le, 2016; Pham et al., 201 8), automated feature engineering (Khurana et al., 2016). and discovering loss functions (Real et al., 2020) or optimization algorithms (Chen et al., 2023b). Typically, these approaches predefine custom search space (sometimes referred to as a coomain-speciffe language), and apply search techniques such as Bayesian optimization, evolutionary algorithms, bandit-based search (He et al., 2021). Our work shares this automation goal but differs significantly in scope, focusing on constructing mathematical models and their algorithmic procedures, navigating more complex, open-ended search space than the narrower, pre-defined search spaces targeted by conventional AutoMI..
The advent of L.MMs has presented new opportunities for automated search problems. These large- scale pretrained models function as highly flexible generators, enabling search over problem spaces expressible through natural language (Brown et al., 2020) and overcoming bottlenecks in representa- tion and search. They have been employed as zeroth-order optimizers over numerical spaces (Yang al., 2024) (e.g., for hyperparameter optimization (Liu et al., 2024)). More strikingly, LLMs show remarkable efficacy in symbolic and combinatorial domains, in search spaces of reward functions (Ma et al., 2024), neural architectures (Chen al., 2023a), symbolic expressions (Liiu al., 2025; Shojaee et al., 2025), and algorithms (Romera- (Romera-Paredes et al., 2024).
LLMs for mathematical formulations. Our work most closely related to emerging research using LLMs to generate mathematical formulations. Here, "formulation" means specifying model within a predefined framework. Instances include formulating statistical models (Li et al., 2024), name-theoretic models (Mensfelt et al., 2024), dynamic systems (Holt al., 2024), and convex optimization models (Ahmaditeshnizzi et al., 2024). These typically assume a known modeling approach. Our work generalizes this line of research by not assuming a predefined framework. Instead, it automatically searches across diverse mathematical frameworks, removing a priori assumptions and enabling efficient exploration of diverse trade-offs offered by fundamentally different frameworks (e.g., metaheuristic optimization vs. convex optimization).
3 PROPOSED FRAMEWORK
The space of potential mathematical models for any given problem inherently vast and complex. Navigating this nested, heterogeneous space with flat exploration strategy is prone inefficiencies.
It more important have beauty one't 's equations than have them fit experiment" PPul Dirac
3

---

Published as a conference paper at ICLR 2026
We propose an adaptive search framework that exploits structure in the modeling process to decompose this complexity, with the aim of improving efficiency (Dempe, 2002).
3.1 BI-LEVEL ADAPTIVE SEARCH
Our approach is informed by key observations about mathematical modeling. Firstly, the set of viable high-level modeling frameworks for a given problem is typically much smaller than the vast space of concrete models and algorithmic instantiations. This allows for more readily applicable priors on framework-level performance characteristics and suitability, for instance, incorporating coarse priors on the trade-off between performance and interpretability for deep learning versus mechanistic models. Secondly, performance variations between frameworks generally dominate those within them. The fundamental trade-off's offered by an exact method (e.g., integer programming) versus an approximate one (e.g.. a metaheuristic) are typically more significant than those between different formulations under the same integer programming paradigm.
Based on these insights, we introduce a bi-level separation in the search process. The upper-level search explores different modeling frameworks, while the lower-level search focuses discovering effective model formulations and solver/algorithm designs within that chosen framework. This bi-level separation, which mirrors the cognitive workflow often employed by human modelers, is expected to confer several advantages, Explicitly separating framework-level decisions allows for more effective exploration of model trade-offs. Different frameworks often populate distinct regions of this frontier, and the bi-level formulation helps systematically identify such trade-offs. Furthermore, modeling choices within a given framework tend to be structurally similar. This relative homogeneity means that feedback from one model instance provides stronger signal for guiding improvements to related formulations within the same framework. For instance, insights from incorporating logistic growth term into one dynamical system model are more directly transferable to refining another dynamical system's 's parameters than to designing the neural architecture a deep model.
3.2 FORMAL DESCRIPTION
Mirroring the sequential nature of modeling decisions, our framework is formalized as an adaptive search process. The search proceeds in iterations indexed by = 1.2..... T, decisions are informed by the history of previously explored models and their evaluated performance. Let S,-1 = {m,, de', Te (e))) of < t} denote this history, where, for notational simplicity. we represent the objective value as
At each iteration I,
[mme, oe) ) E k For the history within particular framework f. we define 8/11 S, The iterative process is decomposed into two nested levels:
Upper-level problem. At each iteration t, the upper-level decision involves selecting a modeling framework E F()). This selection is guided by past performance across all explored frameworks:
ft = arg seF(p) S(f;S
(2)
Here, is scalar utility function that estimates the potential value to explore framework at time f, given S, This function, by quantifying preferences over frameworks, is crucial for managing the exploration-exploiattie
trade-off (Jones et al., 1998; Srinivas al., 2010). For instance, might prioritize frameworks that have recently yielded high-performing models (exploitation) or those less explored that could unveil novel regions of the Pareto frontier (exploration).
Lower-level problem. Once a framework ft is selected by the upper level, the lower-level problem focuses identifying new model pair (m4, ax) within the space of M(fi) x A(ft, This local exploration leverages the historical performance of evaluated pairs within the framework:
(m, a.) = arg (m..) Br (m, a; S(L))
Here, 3y. is a framework-speciffc utility function that learns from evaluations of past pairs S/h,, explored within framework f. to estimate the potential value of a candidate pair (m,a). Together, Equations (2) and (3) define an iterative loop that systematically explores the model space, leveraging the bi-level structure to balance broad exploration across frameworks with focused refinement within them. Once (Je, me, ax) are obtained, they then evaluated to obtain rt, and added to the history.
4

---

Published as a conference paper at ICLR 2026
4 MATHMO: AUTOMATED MATHEMATICAL MODELING WITH LLMS
In what follows, we describe the Automated Mathematical Modeler (MATHMO), our specific imple- mentation of the adaptive search framework (for an algorithmic overview, see Section C.I).
4.1 LLM SEARCH OPERATORS
Conventional automated search methods typically necessitate clearly defined search space and a formal solution representation, often through domain-specinne language (DSL) (Hutter et al., 2011; 1; 2019). The domain of general mathematical modeling. however, presents a significant challenge: the space of potential mathematical objects is extraordinarily vast and diverse, rendering the priori definition of a comprehensive DSL or a fully structured search space practically infeasible.
To navigate this expansive and ill-defined landscape, MATHMO leverages Large Language Models (LLMs) as core search operators. Modern LLMs, pre-trained on massive corpora of text and code, encapsulate extensive knowledge across numerous domains, including mathematics, scientific literature, and programming (Brown et al., 2020; Kaplan et al., 2020). This pre-training endows them with strong implicit domain priors, which can be harnessed guide the exploration of plausible and potentially effective mathematical modeling choices, moving beyond the limitations of rigidly defined search spaces. In MATHMO, L.MSs fulfill two crucial roles in the bi-level search process:
1. Generative samplers. LLMs are employed to sample from the space of frameworks, as well as model and algorithmic specifications. In our implementation, specific models and algorithms are represented as executable Python code, while high-level frameworks are represented textual descriptions. Conditioned on the problem description p. LLMs are prompted to sample suitable modeling frameworks, denoted as f ~ p- p). For a selected framework f, the LLM generates specific model and algorithmic instantiations, i.e., (m, a) ~ po(., p. f, S)), conditioned on the problem, framework, and past examples belonging to that framework."
Surrogate models. LLMs also function as surrogate models to estimate the objective value of proposed model-algoritte
pairs and inform the utility functions guide search. Specifically, i ~ Psoo( (m, a) p,, f. S²). where the subscript SM is employed to denote the surrogate model. Additionally, also
LLMS as Surrogate Models Of Subjective Evaluations (MOSE). This surrogate, i.e., f = ProsE ( (m, a) p). predicts subjective quality scores (e.g., numan-percein interpretability) based on model representation and output, which are integrated into the overall evaluation, allowing subjective inductive biases to be incorporated into search.
The operation of LLMs in these roles relies on specific prompts. The details are provided in Section C, but they follow standard "skeleton" structure, incorporating the problem description p. current context (c.g., selected framework f. and history S), and the specific task for the LLM.
4.2 UPPER-LEVEL PROBLEM: FRAMEWORK SELECTION
The upper-level problem addresses the decision of which modeling framework fr to commit to for an additional step of exploration. In MATHMO, we employ the Pareto Upper Confidence Bound (Pareto-UCB) strategy to realize the framework selection utility a(f, (Equation (2)). This method navigates the inherent multi-objective trade-offs by identifying frontier of frameworks that are optimistically non-dominated thus balancing the need to explore new avenues with exploiting proven ones (Drugan and Nowe, 2013; Xu and Klabjan, 2023).
At the beginning of search (t 0), an initial set of candidate frameworks is proposed by employing the L.Le-based sampler f Po p). Each framework initialized with an optimistic estimate of its potential, by setting an infinite upper confidence bound (UCB) value. This ensures that each framework is selected for at least one initial exploration cycle, providing data for subsequent, more informed decisions. Specifically, for each framework f. the historical performance vectors (re (me, ar, rer) E S(_1} are used to estimate the empirical mean jy R and variance ož R This is then used to calculate the UCB vector UCB, E & where each component is computed using
Heee db A are used to denote the prompts that module that sampling distrihution (Sumers et al., 2023).
5

---

Published as a conference paper at ICLR 2026
formula that considers both the estimated mean and its uncertainty, encouraging exploration:
16 In(No-1) + dg
In(N
JCB - مرs + a N8a-1
(4)
Nea-
where f. and 07, are the estimated mean and variance of objective j of framework f. 5 is the number of times framework has been evaluated, and No-1 is the total number of exploration steps (across all frameworks). c and d are hyperparameters that control the exploration bonus.
The set of UCBy vectors, one for each framework, is then used to identify subset of the promising frameworks. A framework f is considered part of the Pareto optimal set, if its UCB, is non-dominated (i.e., JCB UCBp,, j A j E [k] and j E [X] JCB UCB V f', assuming maximization). From this Pareto-UCB set, one framework is randomly selected to be explored in the next iteration.
4.3 LOWER-LEVER PROBLEM: LOCAL EXPLORATION
Once the upper-level process selects a framework ft, the lower-level is concerned with performing local exploration within the chosen framework to identify promising (me, a.) for subsequent eval- uation. MATHMO achieves this through a three-stage process that involves first sampling candidate pairs, performing surrogate evaluations, and finally selecting the most promising one based these predictions, in process akin to Bayesian Optimization (Snoek et al., 2012; Liu et al., 2024).
First, a set of diverse candidate model-algorithm pairs are sampled, which we denote as $ ((m(() a(+) i E [T], where each (i ,,()) ~ po(:: ps Se, 1 For each sampled candidate pair, we then estimate its k-dimensional objective vector using LLMs as surrogate model: f(0 (0) ((() p, ft; 1/-1), which provides low-cost prediction of how each candidate might perform if fully evaluated, with each of the objectives estimated independently.
Given the predicted objectives, MATHMO performs selection based maximizing the estimated hypervolume (Guerreiro et al., 2020). Intuitively, a higher estimated hypervolume means the pair is more likely to dominate larger portion of the k-dimensional objective space, relative a reference point. Formally, (V ()) rıat), where Tiet R is the reference point. To account for potentially different scales of the l objectives, the individual objective values are normalized to [o, 1] before calculation, and Yeer is set as 1 The pair (m4, ax) is then selected the candidate that yields the largest hypervolume: (M,, ax) = arg max (m,ā) Hmm, ã; Tret).
4.4 MOSE: SURROGATE MODEL OF SUBJECTIVE EVALUATIONS
Mathematical modeling is not solely guided by objective performance metries; subjective qualities, such as interpretability and aligmment with domain knowledge, are crucial utility considerations for human modelers. Indeed, models generally reflect how we conceptualize and understand complex situations. Incorporating these aspects into an automated search is thus vital to ensure the generated models are useful, amenable further analysis, and capable of communicating valuable insights. While tramework-speciff metrics like complexity or sparsity penalties can promote such qualities (c.g., in symbolic linear regression), they are often not transferable across different paradigms, limiting their utility when the goal is to compare diverse models spanning multiple frameworks.
To address this, we introduce MOSE generalized, cross-framework mechanism for integrating subjective criteria into model evaluation. We acknowledge that subjective qualities can be highly observer- depeeden:: MOSE therefore aims not to capture perfect objectivity but to provide consistent surrogate approximations. This approach is based on the observation that subjective qualities are often more reliably expressed through comparative evaluations than absolute scores -n insight also exploited in Reinforcement Learning from Human Feedback (Bradley and Terry, 1952; Christiano et al., 2017). Furthermore, we leverage the ability of advanced LLMs to simulate human judgments, enabling scalable preference elicitation without costly annotations, a technique proven effective in domains like alignment and diversity search (Bai et al., 2022; Bradley et al., 2024).
To ensure comparability of subjective evaluations across diverse discovered models, MOSE employs a predefined reference set of models, denoted as Mier- This set, generated at the start of the search, remains fixed consistent frame of reference. When evaluating new model m, on

---

Published as a conference paper at ICLR 2026
dieevia pecoranitae Land
chocrtta Inaaiition
zuthoge
cxpttooss das pine
tekecariig
ndolowd cookd
Jeperal)
560
Tado
meionpan (3)
realseep
snglammten elten %
eaghamsten clast 75
eeglaraten effer %
E
Figure 2: Pareto fronts and adaptive exploration. (Top) Pareto fronts of models produced by MATHMO on TSP, JSS, Ecology, and Epidemiology tasks. (Bottom) Corresponding cumulative % of exploration effort allocated each modeling framework throughout the search process.
specific subjective quality (e.g., interpretability). MOSE performs pairwise comparisons against each reference model. Specifically, it predicts 'I' if the evaluated model is more preferable, and "O' otherwise. The probabilities associated with these predictions are then averaged obtain score: è Maal Σ, Prose (m, > m; p). By averaging over fixed reference set, this approach yields consistent score in [0, 1]. and mitigates potential sensitivities the choice of reference baseline.
EXPERIMENTS
Research questions. In this section, we present an empirical evaluation of MATHMO. Our experiments are designed to investigate the following research questions:
1. How does MATHMO perform on diverse, real-world tasks, particularly its ability to discover a range of models that effectively navigate different trade-offs (Section 5.1).
What are the contributions of specific algorithmic design decisions within MATHMO to search performance, as analyzed through controlled ablation studies (Section 5.2).
3. How effectively does MOSE capture and integrate subjective modeling preferences, such as interpretability, into the automated modeling process (Section 5.3).
Problems. To address these questions, we employ four distinct modeling problems, two prescrip- tive and two predictive, each presenting unique challenges and trade-offs: Job Shop Scheduling (JSS): This problem involves optimally scheduling jobs on machines, subject to precedence/resource constraints. We investigate the trade-off between makespan (total completion time) and runtime, using 10 instances of varying complexities (50-300 ops). Traveling Salesman (TSP): This NP-hard problem seeks the shortest route visiting each location exactly once. We examine the trade-off between route cost (total tour length) and runtime (time to find a tour), utilizing 10 instances of diverse complexities (30-50 locations). Ecology: This involves understanding and predicting population dynamics in ecological environments. We focus on the trade-off between predictive performance (measured by RMSE on unseen data) and interpretability (assessed by MOSE). We use real-world two-species dataset. Epidemiology: The goal is to understand and simulate the spread of infectious diseases. We investigate the trade-off between predictive performance (RMSE) and interpretabilityy employing real-world COVID-19 dataset from Italy.
In these problems, modeling trade-offs are crucial. For instance, operations managers might use slower, more optimal models for long-term planning (JSS, TSP), yet rapid. near-optimal solutions are invaluable for dynamic rescheduling or handling disruptions. Similarly, while ecologists and epidemiologists need high predictive accuracy, model interpretability is vital for gaining scientific insights into underlying mechanisms (e.g., population dynamics, disease transmission) and informing effective interventions like conservation strategies or public health policies.
For all experiments, we run MATHMO for 20 iterations. This process starts with proposing an initial set of 5 frameworks, and each model evaluation is subject time limit of 300 seconds. For MOSE,
7

---

Published as a conference paper at ICLR 2026
Table I: Ablation study. Hypervolume performance and relative improvements achieved by MATHMO.
Ablations
TSP
JSS
Ecology
Epidemiology
% Improvement
MATHMO
0.998
0.994
0.992
0.967
MATHHORAN
0.972
0,948
0.992
0.939
2.552%
MATHMOFLLA
0.987
0.945
1.000
0.792
5.848%
MATEIMONAIVE
0.977
0.973
0.894
0.713
10.096%
Spomsss 4030
Speetse 4125
Spartso 0/90
Inkeppoablll 1x
63 pemmuatioo cznnupy
parsuaeer ocEic
pecuratation enttoopy
Figure 3: Evaluation of MOSE. Correlation analysis of MOSE interpretability scores against structural and functional complexity metries on ecology task (Left) and epidemiology task (Right).
reference set comprising 3 models is employed. We Additional details on datasetssexpprimmetal setup are provided in Section D.
100-4-40222-05-122 as the LLM.
Additional results. In the interest of space, we provide extended experiments and analyses in Section B. We first establish generality beyond our core benchmarks by evaluating MATHMO on two large-scale medical risk prediction domains, NHANES and SEER (Section B.1), which involve expensive model training and a distinct multi-objective trade-off between discriminative performance (AUROC 1) and calibration (ECE l). We next benchmark against strong baselines, including mature Nomain-specializee solvers and AutoML-style symbolic regression systems (Section B.2), as well as recent LLM-based basde heuristic design methods (Section B.3).
We then assess robustness and internal dynamics through framework removal and inter-run consistency analyses (Section B.4), comparisons of discovered Pareto fronts and hypervolume trajectories under ablations, and targeted studies of upper-level framework selection and lower-level surrogate-guided exploration (Sections B.6 and B.7). Finally, we provide additional validation of MOSE (Section B.8), including sensitivity analyses and expert human alignment studies.
5.1 PERFORMANCE ON DIVERSE TASKS
The performance of MA THMO across the four modeling tasks is visualized in Figure 2. Panel (Top) of the figure displays the Pareto fronts of discovered models for each task, while panel (Bottom) illustrates the corresponding framework exploration dynamics (cumulative effort allocation).
Frameworks dominate trade-offs. A consistent finding is that different modeling frameworks tend to excel in different regions of the Pareto frontier, underscoring the importance of framework selection. For the ISS and TSP tasks, exact methods such as mathematical optimization and constraint pro- gramming yield solutions closer optimality but incur significantly higher computational runtimes. Conversely, metahcuristics and custom heuristics provide solutions with much faster runtimes, with trade-off in solution quality. This pattern extends the Ecology and Epidemiology tasks. Here, time-series forecasting methods like vector autoregression achieve strong predictive performance but are assessed by MOSE as less interpretable. contrast, frameworks such as dynamical systems (e.g.. compartmental models) are considered more interpretable, though they exhibit higher RMSE.
Effective -rrmmwrra refinement. MATHМO demonstrates effectively exploration within frame- works to refine solutions along the Pareto front, for instance, refining solver heuristics on JSS/TSP and redesigning simulated annealing- metaheuristics. Similarly, for the Ecology and Epidemiology tasks, when exploring within the dynamical systems framework, MATHMO proposes variations in model specification (c.g., logistic growth terms, interaction terms) to improve predictive accuracy.
Adaptive exploration behavior: The framework exploration dynamics reveal adaptive search be- havior, characteristic of UCB- UCB-family strategies. On JSS, TSP, and Epidemiology, we observe a clear pattern of initial broad exploration followed by focused exploitation on frameworks that contribute to the Pareto-UCB frontier. Interestingly, on the Ecology task, the exploration allocation remains more evenly distributed among several distinct frameworks (time-series, dynamical systems, rule-
8

---

Published as a conference paper at ICLR 2026
based models), which is consistent with the observation that these frameworks contribute unique, non-dominated solutions to different regions of the Pareto frontier.
5.2 CONTROLLED ANALYSIS OF ALGORITHMIC COMPONENTS
Next, we turn understanding the contribution of the design decisions of MATHMO. For these purposes, we evaluate three ablations: (1) MATHMORAN. where the Pareto-UCB framework selection strategy is replaced with random selection. (2) MATHMOFLAT which collapses the bi-level search into flat search space, applying a globalized version of the local exploration mechanism. (3) MATHMONAIVE, which omits the surrogate-guided local exploration, relying solely on direct sampling.
The comparative performance of MATHMO and its ablated versions, detailed by hypervolume in Table I, reveals several key insights. The complete MATHMO generally achieves the best hypervolume, with one notable exception the Ecology problem, where MA THMOFLAA (without the bi-level struc- ture) found Pareto-dominant solutions by concentrating its search on dynamical systems. Employing random framework selection (MATONORAN) resulted an average hypervolume decrease of 2.5%, underscoring the value of adaptive exploration at the framework level. The removal of bi-level search structure had more pronounced negative impact on model diversity; manual examination indicated this ablation tended to overconcentrate (allocating 95% of exploration effort to metaheuris- ties on JSP/TSP and 80% on ecology). MATHMONAIVE: which relies on repeated L.LM sampling, exhibits the poorest performance overall. However, it interestingly achieved better hypervolume than MATHMOFLAT on the JSS task, suggesting that even naive, broad sampling by the LLM could sometimes provide better coverage of the Pareto frontier than overly myopic flat search.
5.3 SUBJECTIVE QUALITY EVALUATIONS
In this final experimental section, we assess how well MOSE captures aspects of model interpretability. Quantifying interpretability directly is challenging the general sense, and the broader utility MOSE lies in its generalizability across different frameworks and problems. Fortunately, for the time-series problems in our benchmark, we can analyze MOSE's scores against two commonly used proxy complexity metries: (I) structural complexity measured by the number of free parameters (fewer parameters often correlate with more understandable models), and (2) functional complexity assessed using permutation entropy of the model's predicted time-series. Permutation entropy quantifies the regularity and predictability of a time-series, with lower entropy suggesting simpler, more regular dynamics, while higher entropy indicates more chaotic patterns (Bandt and Pompe, 2002).
Our analysis, with detailed correlations presented in Figure 3, yields several insights into MOSE's behavior. Firstly, MOSE scores tend to cluster by modeling framework (e.g.. dynamical systems and rulc-based models consistently receive higher interpretability compared to autoregressive forecasting). The scores exhibit statistically significant negative correlation with structural complexity on both -06678 (p = 1.31 x 10 10-$) for Epidemiology and p =00.2000 -0.298 (p 0.0318) for Ecology. The relationship with functional complexity appears more context- dependent. A significant negative correlation is observed on Ecology (p -0.261, p 0.0313). but almost no correlation is identified on Epidemiology. (parameter count and permutation entropy) are themselves indirect measures of interpretability and are primarily employed here for analytical validation within these specific time-serics contexts.
tasks: Spearman correlation
important to note that these proxy metrics
DISCUSSIONS
Automated mathematical modeling represents an exciting frontier in applying artificial intelligence to complex social, scientific, and engineering problems. this work, we advance this frontier by characterizing the process as sequential decision-making problem under uncertainty. We proposed novel adaptive search framework designed to navigate the complex modeling space, featuring mecha- nisms capable of efficient exploration, balancing multiple objectives, and incorporating subjective preferences. Our concrete instantiation, MATHMO, demonstrates the potential of leveraging LLMs as versatile search operators within structured bi-level search architecture. Empirical results across real-world modeling tasks underscore MATHMO's efficacy in discovering diverse frontiers of models, providing decision-maker, with rich set of alternatives offering different trade-offs.

---

Published as a conference paper at ICLR 2026
Future directions. This research opens numerous avenues for future exploration and enhancement. Currently, the set of modeling frameworks is sampled and fixed at the beginning of the search. Future work could explore dynamic framework generation, enabling the system discover construct new modeling paradigms based on accumulated insights, rather than being confined to an initial set. While designed with multi-objective scenarios in mind, the framework can naturally be applied to single-objective modeling tasks, and further investigation could optimize its performance for such cases. Beyond MOSE for subjective preferences, developing richer interactive mechanisms for human experts to influence the search could significantly enhance outcomes. Lastly, the framework 's general nature permits the development of more specialized utility functions and optimizations of LLMs as samplers or surrogate models, potentially yielding further performance gains. We hope that MATHMO and the proposed framework lay useful foundation for future advancements this domain.
10

---

Published as a conference paper at ICLR 2026
ETHICS AND REPRODUCIBILITY STATEMENTS
Ethics. This work evaluates public benchmarks and de-identified real-world datasets (e.g.. NHANES and SEER) used in accordance with their respective data usage policies. No identifiable personal data are accessed or stored. When employing external LLM services, we follow provider guidelines to ensure that submitted data are not retained or used for human review. Our study focuses on methodological development and does not involve real-world deployment or decision-making
Reproducibility. Experimental investigations are described in Section 5 with further details of the method, experimental setup, and datasets included in Sections to D. We provide the code to reproduce our results at https:://// /gghhu....b... cwm/tennttteeeeocom ManniuMMMHTMM
ACKNOWLEDGMENTS
We thank the anonymous ICLR reviewers, members of the van der Schaar lab, and Andrew Rashbass for many insightful comments and suggestions. TL would like to thank AstraZeneca for their sponsorship and support. This work was supported by Microsoft's Accelerate Foundation Models Academic Research initiative.
REFERENCES
Ahmaditeshnizi, A., Gao, W., and Udell, M. (2024). Optimus: Scalable optimization modeling with (mi) 1p solvers and large language models. In International Conference on Machine Learning, pages 577-596. PMLR.
Akinbami, L. J., Chen, T.-C., Davy, O., Ogden, C. L.,, Fink, S., Clark, J., Riddles, M. K., and Mohadjer, L. K. (2022). National health and nutrition examination survey, 2017-march 2020 prepandemic file: sample design, estimation, and analytic guidelines.
Astorga, N., Liu, T., Xiao, Y., and van der Schaar, M. (2024). Autoformulation of mathematical optimization models using Ilms, arXiv preprint 1.1009241.
Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., McKinnon, C., al. (2022). Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073.
Bandt, C. and Pompe, B. (2002). Permutation entropy: a natural complexity measure for time series. Physical review letters, 88117)1174102
Senwarth-Kuh,, M. and Sindi, S. (2020). How and why to build a mathematical model: case study using prion aggregation. Journal of Biological Chemistry, 295(15):502255035
Bellmore, M. and Nemhauser, G. L. (1968). The traveling salesman problem: survey. Operations Research, 16(3):538-555
Blank, J. and Deb, K. (2020). pymoo: Multi-objective optimization in python. IEEE Access, 8:89497-89509.
Box, G. E. (1979). Robustness in the strategy of scientific model building. In Robustness in statistics, pages 201-236. Elsevier.
Bradley, H., Dai, A., Teufel, H. B., Zhang, J., Oostermeijer, K., Bellagente, M., Clune, J., Stanley, K., Schott, G., and Lehman, J. (2024). Quality-diverse through AI feedback. In The Twelfth International Conference on Learning Representations.
Bradley, R. A. and Terry. M. E. (1952), Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/))3224-45.
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901.
11

---

Published as a conference paper at ICLR 2026
Brunton, S. L.,, Proctor, J. L.,, and Kutz, J. N. (2016). Discovering goveniing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the national academy of sciences, 113(15):3932-399
Chandrasekaran, V. and Jordan, M. I. (2013). Computational and statistical tradeoff' via convex relaxation. Proceedings of the National Academy of Sciences, [10(13)B1181-D) 190.
Chen, A., Dohan, D., and So, D. (2023a). Exoprompting: Language models for code-level neural architecture search. Advances in neural information processing systems, 36:7787-7817.
Chen, X., Liang. C., Huang, D., Real, E., Wang, K., Pham, H., Dong, X., Luong, T., Hsieh, C.-J., Lu, Y., et al. (2023b), Symbolic discovery of optimization algorithms. Advances in neural information processing systems, 0444205--42233
Christiano, P. F., Leike, J., Brown, T., Martic, M., Legg, S., and Amodei. D. (2017). Deep rein- forcement learning from human preferences. Advances in neural information processing systems, 30.
Dempe, (2002). Foundations of bilevel programming. Springer Science & Business Media.
Dirac, P. M. (1963). The evolution of the physicist's picture of nature. Scientific American, 208(5):45-555
Dong, E., Du, H., and Gardner, L.. (2020). An interactive web-based dashboard to track covid-19 in real time. The Lancet infectious diseases, 2035):533-534
Drugan, M. M. and Nowe, A. (2013). Designing multi-objective multi-armed bandits algorithms: A study. In The 2013 international joint conference neural networks (IJCNN, pages 1-8. IEEE.
Guerreiro, A. P., Fonseca, C. M., and Paquete, L. (2020). The hypervolume indicator: Problems and algorithms. arXiv preprint arXiv:2005.00515
He, X., Zhao, K., and Chu, X. (2021). Automl: A survey of the state-of-the-ar.. Knowledge-bass systems, 212:106622.
Holt, S., Liu, T., and der Schaar, M. (2024). Automatically learning hybrid digital twins of dynamical systems. In The Thirty-eighth Annual Conference on Neural Information Processing Systems.
Hutter, F., Hoos, H. and Leyton-Brown, K. (2011). Sequential model-based optimization for general algorithm configuration. In Learning and intelligent optimization: 5th international conference, LION 5, rome, Italy, January 17-21, 2011. selected papers 5, pages 507-523. Springer.
Hutter, F., Kotthoff, L., and Vanschoren, J. (2019). Automated machine learning: methods, systems, challenges. Springer Nature.
Hwang, Y., Kim, Y., Koo, J., Kang, T., Bac, H., and Jung, K. (2025). Llms can be easily confused by instructional distractions. arXiv preprint 250.:220000032.22
Jakeman, A. J.. Letcher, R. A., and Norton, P. (2006). Ten iterative steps development and evaluation of environmental models. Environmental Modelling & Soffware, 21(5):602-614,
Jones, D. R., Schonlau, M., and Welch, W. J. (1998). Efficient global optimization of expensive black-box functions. Journal of Global optimization, 13:455-492.
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. (2020). Scaling laws for neural language models. arXiv preprint arXiv:2001.08361
Khurana, U., Turaga, D., Samulowitz, H., and Parthasrathy, S. (2016). Cognito: Automated feature engineering for supervised learning. In 2016 IEEE 16th international conference data mining workshops (ICDMW), pages 1304-1307. IEEE.
Lawrance, S. (1984). Resource constrained project scheduling: an experimental investigation of heuristic scheduling techniques. GSIA, Carnegie Mellon University.
12

---

Published as a conference paper at ICLR 2026
Li, L., Jamieson, K., DeSalvo, G., Rostamizadch, A., and Talwalkar, A. (2018). Hyperband: A novel bandit- based approach to hyperparameter optimization. Journal of Machine Learning Research, 18(18)):1-52.
Li, M. Y., Fox, E.,, and Goodman, N. (2024). Automated statistical model discovery with language models. In International Conference on Machine Learning, pages 27791-27807, PMLR.
Ling, C., Zhao, X., Zhang, X., Cheng. W., Liu, Y., Sun, Y., Oishi, M., Osaki, T., Matsuda, K., Ji, J., et al. (2024). Uncertainty quantification for in-context learning of large language models. arXiv preprint arXiv:2402. 10189.
Liu, T., Astorga, N., Seedat, N., and van der Schaar, M. (2024). Large language models to enhance bayesian optimization. In The Twelfth International Conference Learning Representations.
Liu, T., Huynh, N., and van der Schaar, M. (2025). Decision tree induction through LLMs via remantically-awan evolution. The Thirteenth International Conference on Learning Represen- fations.
Ma, Y. J., Liang, W., Wang. G., Huang, D.-A., Bastani, O., Jayaraman, D., Zhu, Y., Fan, L., and Anandkumar, A. (2024). Eureka: Human-leve reward design via coding large language models. In The Twelfth International Conference on Learning Representations.
Mensfelt, A., Stathis, K., and Trencsenyi, V. (2024). Autoformalization of game descriptions using large language models. arXiv preprint arXiv:2409.
Miettinen, K. (1999). Nonlinear multiobjectiive optimization, volume 12. Springer Science Business Media.
Pham, H., Guan, M., Zoph, B., Le, Q., and Dean, J. (2018). Efficient neural architecture search via parameters sharing. In International conference on machine learning, pages 4095-4 4104. PMLR.
Real, E., Liang, C., So, D., and Le, Q. (2020). Automl-zero zero: Evolving machine learning algorithms from scratch. In International conference machine learming, pages 8007-8019. PMLR.
Requeima. J., Bronskill, J., Choi, D., Turner, R., and Duvenaud, D. K. (2024). Llm processes: Nu- merical predictive distributions conditioned on natural language. Advances Neural Information Processing Systems, 37:109609-109671.
Ries, L. A. G., Eisner, M., Kosary, C., Hankey, B., Miller, B., Clegg, L., Mariotto, A., Feuer, E., and Edwards, B. (1975). Seer cancer statistics review. National Cancer Institute, 2004.
Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J., Ellenberg, J. S., Wang, P., Fawzi, O., al. (2024). Mathematical discoveries from program search with large language models. Nature, 625(7995):468 475.
Shojace, P., Meidani, K., Gupta, S., Farimani, A. B., and Reddy, C. K. (2025). LLM-SR: Scientific equation discovery via programming with large language models. In The Thirteenth International Conference on Learning Representations.
Simon, H. A. (2019). The Sciences of the Artificial, reissue of the third edition with a new introduction by John Laird. MIT press.
Snoek, J., Larochelle, H., and Adams, R. P. (2012). Practical bayesian optimization of machine learning algorithms. Advances in neural information processing systems, 25.
Srinivas, N., Krause, A., Kakade, S., and Seeger, M. (2010). Gaussian process optimization in the bandit setting: No regret and experimental design. In Proceedings of the 27th International Conference on Machine Learning, pages 1015-1022. Omnipress.
Stenseth, N. C., Falck, W., Bjørnstad, O. N., and Krebs, C. J. (1997). Population regulation in snowshoe hare and canadian lynx: asymmetric food web configurations between hare and lynx. Proceedings of the National Academy of Sciences, 94(10):514-51552
Sugihara, G. and May. R. M. (1990). Nonlinear forecasting as a way of distinguishing chaos from measurement error in time series. Nature, 3444468)444662
13

---

Published as a conference paper at ICLR 2026
Sumers, T., Yao, S., Narasimhan, K., and Griffiths, T. (2023). Cognitive architectures for language agents. Transactions on Machine Learning Research.
Thornton, C., Hutter, F,, Hoos, H. H., and Leyton-Brown, K. (2013). Auto-weka: Combined selection and hyperparameter optimization of classification algorithms. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 847-855.
Turing, A. M. (1990). The chemical basis of morphogenesis. Bulletin of mathematical biology, 52:153-197.
van den Berg. N. L., Machado, D.. Santos, S., Rocha, L., Chacón, J., Harcombe, W., Mitri, S., and Patil, K. R. (2022). Ecological modelling approaches for predicting emergent properties in microbial communities. Nature ecology & evolution, 6(7):855-865
Van Roy, P. and Haridi, S. (2004). Concepts, techniques, and models of computer programming. MIT press.
Xiang, Y., Jia, Y., Chen, L.., Guo, L., Shu, B., and Long, E. (2021). Covid-19 epidemic prediction and the impact of public health interventions: A review of covid-19 epidemic models. Infectious Disease Modelling, 6:324-342.
Xiang, Y. Yan, H., Gui, L., and He, Y. (2024). Addressing order sensitivity of in-context demon- stration examples in causal language models. In Findings of the Association for Computational Linguistics ACI. 2024, pages 6467-6481.
Xiong, H., Shi, S., Ren, D., and Hu, J. (2022). A survey of job shop scheduling problem: The types and models. Computers Operations Research, 142:105731.
Xu, M. and Klabjan, D. (2023). Pareto regret analyses in multi-objectivv multi-armed bandit. In International Conference Machine Learning, pages 38499-38517. PMLR.
Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q. V., Zhou, D., and Chen, X. (2024). Large language models as optimizers. In The Twelfth International Conference on Learning Representations.
Yao, S., Liu, F., Lin, X., Lu, Z., Wang, Z., and Zhang, Q. (2025). Multi-objective evolution of heuristic using large language model. In Proceedings of the AAAI Conference on Artificial Intelligence. volume 39, pages 27144-27152.
Zoph, B. and Le, Q. V. (2016). Neural architecture search with reinforcement learning. arXiv preprint 16X.v:1611.01578
14

---

Published as a conference paper at ICLR 2026
Appendix
Table of Contents
A Additional Discussions
16
A.1 Real-World Modeling: Uncertainty, Trade-offs, and Subjective Criteria
16
A.2 Enhancing Components of the Adaptive Search Framework
17
B Extended Experiments and Empirical Analyses
19
B.1 Additional Domains: Large-Scale Medical Risk Prediction
19
B.2 Comparison to Mature Solvers AutoML. Baselines
Comparison Against LLM-Based Heuristic Design Baselines
B.4 Robustness and Sensitivity Analyses
21
B.5 Comparison of Discovered Pareto Fronts
22
B.6 Insights: Upper-Level Framework Selection
23
B.7 Insights: Lower-Level Exploration Efficiency
24
B.8 Additional Evaluations of MOSE...
24
Technical Details
28
C.1 Algorihmm/Pscudooode.
28
C.2 LLM Search Operators
28
C33 Adaptive Search
29
C.4 Miscellaneous Implementation Details
30
D Experimental Details
32
D.1 Datasets
32
D.2 Metrics
15

---

Published as a conference paper at ICLR 2026
A ADDITIONAL DISCUSSIONS
In the following section of the appendix, we offer additional discussion to motivate the need for automated mathematical modeling (MATHMO), reflecting the multi-objective trade-offs, variety of frameworks, and role of subjective criteria inherent in real-world modeling. We then outline potential enhancements to key components of the adaptive search framework.
A. REAL. WORLD MODELING: UNCERTAINTY, TRADE-OFFS, AND SUBJECTIVE CRITERIA Table 2: Overview of problems. Illustrating multi oulti--bjecctve trade-offs, modeling diversity, uncer- tainty, and subjective considerations.
Problem
Objectives
Modeling Approaches
Uncertainty Modeling
Subjective Criteria
TSP
Journey cost, Run-
A ILP. A Metabeuristics,
Instance size and structure affect per- formance: solver behavior is unpre- dictable
time
2 Heuristies
Job Scheduling
Shop
P Makespan, Run- time
A Constraint program-
Solution quality and runtime
ming, é Metaheuristies,
vary with problem characteristics; method choice is non-obvious Complex, noisy dynumics; uncertain model structure; missing or sparse data
Heuristics
Ecology
è Predictive accuracy,
A Differential equations,
Alignment with ecological knowledge and interpretability Interpretabilityy for public health commu- nication
b Interpretability
ARIMA, A Graphical models
Epidemiology
D Forecast accuracy,
A Compartmental mod-
Highly sensitive to data quality and regime changes; difficult validate assumptions
Interpeetability
els, A Stutistical models,
A Time-series models
Medical Di-
b. Accuracy, & Explain-
A Deep learning, A
Variation in populations, equipment, labeling: generalization uncer- tain
Clinical trust and aligument with expert reasoning
agnosis
ability. b. Uncertainty
Rule-based systems, A Prohabilistic models
calibration
Portfollo
b Expected return,
A Classical finance mod- els, & ML models (e.g.. RNNs)
Financial time series are non- stutionary; market conditions shift unpredictably
Transparency, ex- plainubility, and risk aligument
Optimiza- tion
Risk, Robustness
Drug
Re-
D. Predictive accuracy.
A Mechanistic models,
High patient variability; limited, ex- pensive data; strong prior assump- tioms needed
Regulatory approval and clinical inter- pretability
sponse
P Biological inter-
A Multi-omics ML, A Hybrid causal models
Modeling
pretability, > Safety
This work addresses relatively underexplored area: the automation of mathematical modeling under real-world constraints. Our problem formulation and methodological framework are motivated by three core challenges commonly encountered in practice: (1) uncertainty in modeling secisions-such as which frameworks or assumptions are most appropriate; (2) multi-objective trade-off's between competing criteria like accuracy, runtime, and interpretability: and (3) subjective, cuman--centro considerations that are difficult to mathematically formalize but critical to real-world adoption.
Our analysis focuses on domains such job-shop scheduling, vehicle routing, ecology, and epi- demiology, each of which presents unique modeling challenges and trade-offs, such as optimality versus runtime or predictive accuracy versus interpretability. However, these issues are far from domain-specific. In the following discussion, we illustrate how similar concerns arise across wide range of modeling scenarios, further underscoring the importance of a general-purpose, flexible approach to automated modeling. An overview of the discussion is summarized in Table 2.
Medical diagnosis. The task predicting diseases or conditions based on clinical data such as medical images, lab results, or patient history.
1. Trade-offs:
Diagnostic accuracy (essential for minimizing missed incorrect diagnoses) vs. explainability (clinicians need to understand model reasoning to trust and act on predictions) vs. b uncertainty calibration (important for risk-aware decision-making, especially in borderline cases). Possible modeling frameworks: A Deep learning (highly accurate, data-intensive, low inter- pretability); A rule-based expert systems (e.g.. risk scores; interpretable but often underperform, and lack uncertainty handling); A probabilistic models (capture uncertainty but require strong assumptions and are harder to scale).
3. Subjective criteria: Clinicians value interpretability and alignment with domain knowledge- understanding why model makes a diagnosis is as important as the prediction itself.
4. Uncertainty modeling: Variation in patient populations, medical instrumentation (e.g., imaging devices), and comorbidities makes it unclear which modeling assumptions will generalize. Ground truth labels may also noisy or inconsistent across annotators.
Financial portfolio optimization. The process of allocating assets to maximize returns while managing risk in dynamic market environments.
16

---

Published as a conference paper at ICLR 2026
1. Trade-offs; Expected return (central to investor objectives) risk (higher returns generally entail higher volatility); > predictive performance (important for exploiting market inefficiencies) vs. robustness (models may overfit to past data and fail under new market regimes).
2. Possible modeling frameworks: A Classical financial models (e.g., Markowitz, :ittemmitermmn; principled but sensitive to input estimation errors); A machine learning models (e.g.. RNNs; flexible, can learn patterns, but require large, clean datasets and may overfit lack robustness).
3. Uncertainty in modeling: Market dynamics are non-stationary and hard to forecast. Expected re- tums, volatilities, and correlations shift over time, making it unclear which models or assumptions will remain valid. A model that performs well in one regime/time horizon may fail in another.
Drug response modeling. Predicting how individual patients will respond to given drug, often in the context of personalized medicine or drug development.
1. Trade-offs: Predictive accuracy (critical for identifying effective treatments) vs. biological interpretability (important for understanding mechanisms and gaining trust); >> short-term efficacy (desired for immediate outcomes) vs. long-term safety (essential for regulatory approval and patient well-being).
Possible modeling frameworks: Mechanistic models (e.g., PK/PD: grounded in biology. interpretable, but slow and purameter-sensitivel A multi-omics ML models (data-driven and expressive, but opaque and difficult to validate); A hybrid models (e.g., causal or semi-mechanistic; combine strengths, but sensitive misspecification).
3. Subjective criteria: Interpretability and biological plausibility crucial for clinical trust and regulatory acceptance.
Uncertainty in modeling: High variability across patients (e.g., in genetics or metabolism)
complicates model generalization. Data is often limited, expensive obtain, and ethically constrained.
In all these examples, the goal is not to identify a single best model, but rather to present the human user with diverse set of viable modeling options-eaae representing different trade-off's across relevant objectives such as accuracy, interpretability, robustness, and runtime. This is crucial because the optimal modeling choice often depends on context-specific constraints, user preferences, and shifting priorities. Framing this as a multi-objective optimization problem allows us to systematically explore the space of trade-offs and upproximate the Pareto frontier, enabling users to make informed decisions based on their own criteria and operational needs.
A.2 ENHANCING COMPONENTS OF THE ADAPTIVE SEARCH FRAMEWORK
Our work presents general framework for approaching automated mathematical modeling. Specifi- cally, it decomposes the automated modeling process into a bi-level search: the upper level selects among modeling frameworks while managing level performs local search within each framework to identify high-performing model-algorithe pairs. While our specific instantiation of this framework demonstrates effectiveness, we outline key areas where individual components of this framework could be improved to enhance performance, flexibility, and robustness.
trade-offs, and the lower
A.2.1 UPPER-LEVEL UTILITY FUNCTION
We use Pareto-UCE guide framework selection by approximating the Pareto frontier across multiple objectives. This balances exploration and exploitation in principled way. However:
Non-stationarity. Pareto-UCB assumes a stationary reward distribution, which is perhaps un- tenable in our modiinng performance is expected to improve over time due to ongoing low-level cxploration. This results in a shifting reward distribution and suggests that a dynamic or non-stationary approach may be more appropriate.
2. Independence assumption. Frameworks are currently treated as independent in the selection problem, but in reality, their performance often correlated (e.g., if moodel-based control methods perform well, similar model-based RL might too). Ideally, inter- free-frrmework correlations and structure are captured somehow, although modeling or learning this correlation a-priori remains challenging.
17

---

Published as a conference paper at ICLR 2026
3. Alternative utility functions. Optimism-bane basedd acquisition strategies like Pareto-UCE presents a strong initial approach, but Bayesian acquisition functions (c.g., entropy search) may better capture uncertainty and trade-offs in this multii-bjective setting and are worth investigating in future work.
A.2.2 LOWER-LEVER UTILITY FUNCTION
Our lower-level utility function relies on an LLM-based surrogate estimate the objective perfor- mance of candidate models, using expected hypervolume improvement as the acquisition criterion.
1. Surrogate models. Developing surrogate model this non-conventional search space (i.e., space of models expressed in natural language) is very challenging. While our work uses an LLM-based surrogate model and demonstrated improved search efficiency result, this is fruitful area with open challenges in calibration and generalization.
2. Alternative search strategies. Other local search methods, such as evolutionary algorithms or learned reinforcement learning- baasenng-based search policies, could offer more robust and generalizable exploration under uncertainty.
3. Framework-speciffi search policies. More interestingly. specialized framework-speen ex- ploration strategies that exploit the structure unique to each framework could greatly improve efficiency. For instance, leveraging the hierarchical structures in mathematical programming (Astorga et al., 2024) or neural architecture search for deep learning models (Zoph and Le, 2016). However, this requires engineering these framework-specifie strategies beforehand.
A.2.3 LLM GENERATIVE SAMPLERS
LLMss are crucial for open-ended exploration in the space of frameworks, models, and algorithms.
1. Finetuned search operators. In our work we used general purpose LLMs as samplers, although there is no principled reason this could not improved with specific finetuned search operators. Dynamic sampling. When comes to framework sampling, our current approach samples and fixes set of frameworks at the beginning of search. Ideally, this could be improved with dynamic approach, which less constraining, enables more efficient exploration budget allocation, and encourages the emergence of hybrid novel modeling paradigms.
A.2.4 LLM SURROGATE MODELS
LLMs, which operate directly natural language inputs, offer powerful alternative to traditional surrogate models like Gaussian Processes that are limited to well-defined numerical feature spaces. This flexibility enables surrogate modeling over open-ended model descriptions, expanding the expressive range of the search process. However, L.L.M-assed surrogates also come with notable challenges. Their uncertainty estimates tend to be poorly calibrated (Ling et al., 2024), and their outputs can be highly sensitive to prompt design and formatting (Xiang et al., 2024; Hwang et al., 2025). Addressing these -shouuh techniques such prompt ensembling, temperature scaling, and uncertainty-awee aware decoding- remaains an important direction for improving both reliability and performance in LLM-basee surrogate evaluation.
A.2.5 MOSE: MODEL OF SUBJECTIVE EVALUATION
A general-purpose, framewrra model of subjective evaluation introduces significant potential for modeling human-like preferences in automated modeling pipelines. Our initial results showed that MOSE scores are meaningfully correlated with structural and functional properties of candidate models, suggesting alignment with domain-relevant heuristics. Nonetheless, the model remains in an early stage and would benefit from further validation and tuning across diverse problem types. Encouragingly, similar challenges have been tackled in reinforcement learning from human feedback (RLHF), resulting in reliable and robust preference models (Christiano al., 2017). In setting, MOSE could be further improved using paired preference labels gathered from domain experts, enabling data-efficient finetuning and deeper alignment with subjective modeling criteria.
18

---

Published as a conference paper at ICLR 2026
repervied lanzzin
antirmaton opliniintios
supervied leszniss
cusuuble louting survial cambas
Beenion modiling
assalllee aciloch
Baycito uoditag
01173
(3)
01300
03000
0.800
AUTOC
AUROC
Figure 4: NHANES/SEER Pareto fronts. Pareto-efficient models discovered across AUROC- ECE
trade-offs.
B EXTENDED EXPERIMENTS AND EMPIRICAL ANALYSES
B. ADDITIONAL DOMAINS: LARGE-SCALE MEDICAL RISK PREDICTION
We evaluate MATHMO on two further domains (NHANES and SEER) to demonstrate generality beyond the tasks in Section 5:
NHANES (Akinbami et al., 2022): A national health survey containing 86, 000 records of demo- graphic, behavioral, clinical, and environmental covariates, where the goal is predict risk of myocardial infarction.
SEER (Ries et al., 1975): A population-level cancer registry containing 100, 000 patients with incidence, demographic, and survival information, where the goal to predict risk for breast cancer.
These experiments are designed to evaluate: (Z) they involve substantively different problem domains than our earlier tasks, (2) evaluation significantly more expensive due to training large-scale models, and (3) they feature different objective trade-offs: here, discriminative performance (AUROC 1) and probabilistic calibration (ECE l), which are central in clinical modeling. We also use a gpt LLM backbone to highlight that MATHMO is LLM-agnostic.
Analysis. Figure 4 illustrates that MATHМO consistently discovers diverse Pareto-efficient models spanning supervised learners, Bayesian models, survival analysis, and ensemble methods. These experiments show that MATHMO scales to high-cost real-world modeling settings and is able to autonomously identify high-quality trade-offs across heterogenous modeling frameworks, reinforcing generality beyond the initial set of domains.
B.2 COMPARISON TO MATURE SOLVERS AND AUTOML BASELINES
In this subsection, we compare MATHOO discovered models with mature solvers (on TSP/ISSP) and against AutoML baselines (on EoologyEpiidemioloyy)
Mature solvers. For TSP and JSSP, we benchmark against the highly optimized Concorde solver (branch-and-cur) and OR-Tools CP-SAT. These solvers are framework-sppecffc and require strong manual modeling choices (variables, constraints, heuristies, tuning). In contrast, MATHMO automat- ically generates models across heterogencous frameworks, producing full Pareto frontier rather than single optimized point. Including traditional solvers provides the strongest possible reference point, ensuring that the solutions generated by MATHMO are sensible and well- geolngro relative to gold-standard human-engunn enginneeee baselines.
Analysis. Tables and show that MATHMO attains competitive performance while simultaneously exploring solutions across optimization, heuristics, and metaheuristics, which populate distinct regions of the Pareto frontier. These comparisons validate that MA MATHOO produces reasonable solutions near
19

---

Published as a conference paper at ICLR 2026
SINDy
Interpettbllly 611
Interpeet biitit
RAISE (43
(1)
Figure 5: SINDY vs. MATHMO. Pareto fronts on Ecology (left) and Epidemiology (right): MATHMO dominates SINDy across accuracy and interpretability.
expert-tuned solvers, but with the added benefit of automatically discovering multiple qualitatively different trade-offs, which solvers cannot provide.
Table 3: TSP baseline comparison. Concorde vs. MATHMO and other baselines.
Concorde
MATHMO
MEoH
FunSearch
(4.52, 0.0408)
(4.59, 0.6769)
(5.18, 0.1254)
(4.77, 0.1316)
(3.57, 0.0140)
(3.57, 0.1888)
(3.70, 0.1307)
(3.90, 0.1289)
(4.60, 0.0243)
(4.66, 0.6476)
(4.89, 0.1286)
(4.89, 0.1265)
(5.11, 0.0305)
(5,19, 0,1566)
(5,66, 0.1307)
(5,40, 0.1418)
(4.24, 0.0405)
(4.24, 0.1805)
(5.83, 0.1268)
(4.48, 0.1314)
Table 4: JSSP baseline comparison. OR-Tools vs. MATHMO and other baselines.
OR-Tools
MATHMO
MEoH
FunSearch
(1039.00, 0.0599)
(1039.00, 0.0230)
(1096.00, 0.0602)
(1039.00, 0.0536)
(1218.00, 0.2954)
(1421.00, 0.0246)
(1503.00, 0.0640)
(1372.00, 0.1610)
(1235.00, 13.4015)
(1235.00, 15.9644)
(1514.00, 0.0621)
(1644.00, 0747)
(1721.00, 0.6016)
(1721.00. 1.1240)
(2175.00, 0.0615)
(1925.00, 0.1697)
(1888.00, 0.2975)
(2127.00, 0.0233)
(2183.00, 0.0620)
(11487.00, 0.1949)
Comparison to SINDy. For the time-series tasks (Ecology and epidemiology), compare against SINDY (Brunton et al., 2016), a widely used symbolic regression technique to discover interpretable and predictive mechanistic models. As S SINDY requires users to choose basis-function library and sparsity threshold, it lends itself well to AutoML-based model selection. To keep budgets comparable, we allocate SINDY a consistent search budget of 20 configurations (4 libraries sparsity thresholds), matching the per-framework iteration budget in MATHMO.
Analysis. Figure shows that across both Ecology and Epidemiology. SINDy is entirely dominated: its models occupy a narrow interpretability band, while MATHO''s models span richer set of trade- offs through access to multiple modeling frameworks. Takeaway: MATHMO yields strictly better Pareto fronts due to its ability to explore beyond a single symbolic regression paradigm.
B.3 COMPARISON AGAINST LLM-BASED HEURISTIC DESIGN BASELINES
We benchmark MATHMO against two recent and representative systems: MEoH (Yao et al., 2025) and FunSearch (Romera-Paredes al., 2024). Both are state-of-the-art approaches in automated heuristic design using large language models (LLMs). Specifically, MEoH addresses multi-objective search using evolutionary operators and
mechanism for diversity maintenance,
20

---

Published as a conference paper at ICLR 2026
while FunSearch employs genetic programming with island-model evolutionary strategy, tailored primarily for iigle-objective problems. Although neither method is designed for cenerrl-purpose cross-framework mathematical modeling, they provide strong points of reference for evaluating the relative effectiveness MAM.
Experimental setup. All methods evaluated four benchmark problems under fixed budget of 20 algorrihn evaluations. For FunSearch, the multi-objective criteria are scalarized using uniform weights to ensure fair comparison. Table 5 summarizes the results.
Table 5: Performance comparison against baselines. Higher values are better.
Method
TSP
JSSP
Ecology
Epidemiology
MßBН
0.9480
0.8587
0.8360
0.9054
FunSearch
0.9772
0.8130
0.6240
0.6628
MATHMO
0.9877
0.9655
0.9576
0.9793
МАTHМO consistently outperforms both baselines across all domains. We attribute these gains two main factors:
Cross-framework modeling. MATHMO explicitly searches over diverse modeling frame- works (c.g., dynamical systems, symbolic regression, constraint programming), whereas MEoH and FunSearch tend to restrict exploration to or frameworks. For example, in the JSSP domain, MATHMO leveraged constraint programming frameworks, yielding superior solutions not discovered by either baseline.
Surrogate-guided search. MATHMO employs LLM- based surrogate models to guide candi- date selection, improving sample efficiency and focusing evaluations high-potential areas. In contrast, MEoH and FunSearch rely exclusively on direct evaluation evolved models.
B.4 ROBUSTNESS AND SENSITIVITY ANALYSES
We also conducted a post-hoc sensitivity analysis by measuring the average drop in hypervolume (HV) when each framework is removed from the search space. Results are summarized below:
TSP: 7.38% (mathematical optimization). 7.20% (metaheuristics), 1.21% (heuristics)
JSSP: 2.16% (constraint programming). 1.29% (metaheuristics)
Ecology: 16.71% (symbolic regression), 9.62% (time-series forecasting), 0.35% (rule- based)
Epidemiology: 15.00% (rule-based)
These results indicate that certain frameworks contribute uniquely to the Pareto frontier, while others are progressively deprioritized by the Pareto-UCB mechanism.
To further assess robustness, conducted experiments the vehicle routing problem (VRP) with multiple random initializations.
Table Inter-run consistency on VRP (5 runs). HV denotes hypervolume.
Run
2
3
HV
0.989
0.974
0.980
0.989
0.987
Average HV: 0.984 + 0.005.
The results show that hypervolume remains stable across runs (0.984 + 0.005), and framework contributions the Pareto frontier consistent across different initializations.
21

---

Published as a conference paper at ICLR 2026
Table 7: Framework contributions to the Pareto frontier (percentage HV contribution).
Framework
Run
Run
Run 2
Run 3
Run 4
Heuristics
9.02%
2.00%
9.54%
8.78%
10.21%
Mathematical optimization
7.18%
1.82%
5.44%
6.87%
7.34%
Metaheuristics
0.33%
0.14%
0.86%
0.23%
NATHDO
Jemns Cor (1)
Joareyy (J)
WATHNOGEST
MATTINO
Beeatee 14
Kakogen
Makogga MATTDRD
Malugga
MATTIMO
2005X
Figure 6: Pareto front comparisons. Visualizations of the discovered Pareto fronts for MATHMO, MATHMOFLAT, and MATHMONAIVE. denotes non non-dominated models; traces the estimated Pareto front; colors indicate distinct modeling frameworks. From top to bottom: TSP, JSS, Ecology. Epidemiology.
Frotker
Prokken
Naccked me
Denstion
Bovalios
Dresison
Figure 7:
Hypervolume during search. Temporal progression of normalized hypervolume through- the search process for MATHMO and control variants: MATHMOPLAT and NATHMONAIVE. From left to right: TSP. JSS, Ecology. Epidemiology.
B.5 COMPARISON OF DISCOVERED PARETO FRONTS
We begin by analyzing the quality the discovered Pareto frontiers, as well as the rate of hypervolume improvement. The Pareto front reflects the diversity and optimality of the trade-offs discovered during search. while hypervolume progression captures the efficiency with which the method explores the multi-objective space.
22

---

Published as a conference paper at ICLR 2026
Frobles:
Problex Keiier
Pribbes Reieskdar
10
DIV
Figure 8: Upper-level framework selection. Comparison of hypervolume progression between two upper-level selection strategies: MATHMO (adaptive) and MATHMORAN (random). From left right: TSP, JSS, Ecology, Epidemiology.
Profies
Pralieen Todagy
Probiee: Epikeisbyr
Naxter Rocintion
Kurshe Charee
Ntxter Olemned Babatton
Sarbe Endaniess
Faskeiske
5000000
Ntrvie claereed Enlvities
Narder of Cheeve Eraietion
Mtrkee of Dieicoad Botaateee
Narche of Charied Dholcatkee
Figure 9: Lower-level exploration efficiency, (Top) Relative hypervolume improvement as function of the number of model evaluations for MATHMO and RAN. (Bottom) NRMSE of surrogate predictions over time. From left to right: TSP, JSS, Ecology, Epidemiology
We compare MATHMO against two ablations: MATHMOPLAT- which removes the bi-level structure and performs sequential sampling of model-algorithm pairs without adaptive framework selection; and MATHOONAVEE, which further removes the sequential aspect entirely, instead sampling set of model-algorithm pairs in parallel. For all three settings, we allow 20 iterations, with MATHMONAIVE generating parallel samples.
Discovered pareto fronts. In Figure 6, we compare the Pareto frontiers discovered by each method. МАTHМO consistently identifies the most complete and diverse set of Pareto-efficient solutions. For instance, on TSP, it uncovers models that offer range of trade-offs between journey cost and runtime. contrast, MATHMOFLAT explores fewer frameworks and, while identifies some competitive models in terms of journey cost, it fails to find low-runtime alternatives with modest sacrifices in optimality. More generally, we observe that purely sequential approaches like MATHMOPLAT tend to focus locally on just 1-2 frameworks, limiting their coverage of the multi-objective space and resulting in fewer diverse trade-off solutions. This trend is also evident in Ecology and Epidemiology. where MATHMO discovers broader and more densely populated Pareto fronts that dominate across both objectives. An exception arises JSS, where MATHMONAIVE discovers a Pareto front comparable to that MATHMO. This is likely due favorable random coverage and the nature of the problem landscape, where multiple high-performing models can be sampled without requiring adaptive search.
Search efficiency. Figure 7 shows the progression of normalized hypervolume across iterations. On TSP, Ecology, and Epidemiology. MATHMO achieves faster gains in hypervolume compared to the ablations, often requiring significantly fewer iterations identify Pareto-efficient solutions. The case of JSS is again an outlier: although MA MATHMONAIV eventually achieves comparable hypervolume, MATHMO reaches the same level only 6 iterations, highlighting its superior sample efficiency.
B.6 INSIGHTS: LEVER-LEVEL FRAMEWORK SELECTION
We next examine the dynamics of leper-level framework selection, which in MATHMO is guided by the Pareto-UCB utility function. To isolate its impact, compare against control ablation,
23

---

Published as a conference paper at ICLR 2026
MATHMORAN. which is identical to MATHMO except that it replaces utility-guided selection with uniform random sampling over frameworks.
Observations. Figure 8 shows the progression of normalized hypervolume across iterations. On TSP. Ecology, and Epidemiology. we observe similar rates of improvement between MATHMO and MATHMORAN during the initial & 6 iterations, reflecting the exploratory phase of the search. How- ever, after this point, MATHMO exhibits a notably sustained increase in hypervolume, suggesting that it effectively focuses its search budget on frameworks with greater potential for Pareto improvement. This behavior illustrates the utility of guided upper-level selection: Pareto-UCB enables adaptive resource allocation toward promising regions of the search space.
In contrast, on JSS, both methods converge to comparable hypervolume levels at similar rates. This echoes the earlier observation in Pareto front comparisons, where MATHMONAIVE also performed well. These results suggest that the effectiveness of upper-level selection may be For instance, in settings like JSS where the landscape relatively flat or where good models are distributed across many frameworks, random selection may suffice to find competitive solutions.
B.7 INSIGHTS: LOWER-LEVEL EXPLORATION EFFICIENCY
We now turn to the efficiency of local exploration, specifically how well the surrogate model guides the selection of candidate model-algorithm pairs. To isolate this effect, we compare MATHMO against a control variant, RAN, which is identical in every respect except that it selects a candidate at random rather than using a surrogate to estimate and optimize objective performance.
To ensure a fair comparison, both methods are evaluated on the same historical set of models and the same set of candidate proposals at each step. This setup, averaged over 5 random seeds, controls for variation in the history and candidate pool, ensuring that any observed differences can be attributed solely to the srecision-maiing strategy, i.e., surrogate-guide versus random selection.
Local search efficiency. Figure 9 (Top) shows relative hypervolume improvement as function of the number of evaluated models. For each selected point, we compute the gain in hypervolume relative to the historical set alone; a larger value indicates that the newly acquired model improved the current Pareto front. Across all four benchmarks, TSP, JSS, Ecology, and Epidemiology. we observe that MA THMO consistently achieves higher relative hypervolume gains. This indicates that surrogate- surrogate-gudedd selection is more effective at identifying models that advance the Pareto frontier, leading to more sample-efficient exploration compared to random selection.
Surrogate model performance. To better understand the surrogate's 's behavior, we also measure its predictive accuracy in terms of normalized RMSE (NRMSE) on the candidate models, averaged across the two objectives. In TSP, Ecology, and Epidemiology, we find that NRMSE decreases with more historical data, suggesting that the LL.M-bssd surrogate improves with more observations, consistent with the observation in Liu et al. (2024) that LLM-based surrogate estimation generalizes better with more context. In contrast, on JSS, surrogate accuracy worsens over time, with increasing NRMSE across iterations. Manual inspection revealed possible explanation: that the dominant source of error lies in estimating runtime for metaheuristic algorithms- critical for identifying Pareto- improving trade-offs in this problem domain. These runtime behaviors are difficult to predict based solely on surface-leve descriptions, leading to poor surrogate performance. This likely explains why both MA THMO and RAN achieve similar relative hypervolume improvements on JSS: as the surrogate becomes less informative, its selection decisions approach random choice.
B.8 ADDITIONAL EVALUATIONS OF MOSE
Sensitivity analysis. MOSE used during search as a surrogate for subjective human preferences (c.g., model interpretabilityy. Surrogate stability is critical: if small changes to the reference set Mee produced inconsistent preferences, MATHMO's search could be noisy or unreliable. Therefore, evaluating the sensitivity of MOSE directly addresses whether its interpretability judgments robust.
Analysis. We compute MOSE scores for 20 models across four independently sampled reference sets. The distributions in Figure 10 and correlations in Tables 8 and 9 show consistently high agreement (r > 0.93). MOSE produces stable and consistent judgments across reference sets, indicating that is reliable component for guiding subjective-objective trade-off's during search.
24

---

Published as a conference paper at ICLR 2026
MOSE Seom Distellatiin with Different Referencn Sits (Reolagy)
MIGS Seam Distrtlattonn with Difkecnt Reforemco Seta (Roilemioooyy
MOBI Store Henniterrion
MOSE Stote Detiiiatioon
0.53
Modal Trales
Modall Tralex
Figure 10: MOSE sensitivity. Interpretability score distributions across four reference sets (Mret).
Human study, As MOSE is used to approximate human qualitative preferences, it is essential to verify that its judgments align with human experts. The goal is not to perform full-scale study, but to provide evidence that MOSE is directionally consistent with expert reasoning. We collect expert pairwise interpretability judgments 25 randomly sampled model pairs. For each pair, record the MOSE scores and the expert preference of interpretability.
Analysis. Agreement with MOSE 79.2% for Ecology (Table 10) and 76.0% for Epidemiology (Table 11). The high agreement rates indicate that MOSE captures meaningful subjective preferences consistent with expert intuition, supporting its use as scalable interpretability surrogate.
Table Epidemiology
Correlations Between Reference Sets (Maer)
Reference Set Pair
Pearson (p-value)
(3, 2)
0,9695 (1.92e-13 12)
(3, 1)
0.9801 (4.30e-14) 14)
(3, 0)
0.9673 (3.61e-12)
(2, 1)
0.9892 (1.85e-16)
(2, 0)
0.9507 (1.36e-10)
(1, 0)
0.9561 (4.87e-11)
Table Ecology Correlations Between Reference Sets (Mret)
Reference Set Pair
Pearson 1 (p-value)
(3, 2)
0.9823 (1.53e-14)
(3, 1)
0.9525 (9.87e-11)
(3, 0)
0,9923 (8.84e-18)
(2, I)
0.9351 (1.54e-09)
(2, 0)
0.9809 (4,066114
(1, 0)
0.9567 (4.36e- 11)
25

---

Published as a conference paper at ICLR 2026
Table 10: Human interpretability judgments (Ecology). Pairwise expert preferences compared
against MOSE predictions for 25 model pairs. Agreement indicates whether MOSE selects the same model as the human expert.
MOSE(A)
MOSE(B)
Expert
Agree
0.403
0.673
B
0.626
0.645
A
0.600
0.675
B
0.650
0.643
B
0,403
0.679
B
0.679
0.675
A
0.626
0.687
B
0.626
0.635
A
0.563
0.687
B
0.673
0.635
A
0.626
0.650
B
0.591
0.701
B
0.655
0.614
A
B
0.600
0.635
B
0.600
0.638
0,626
0,655
B
AAAAAAAA
0.679
0.650
0.679
0.635
ALLAKA
A
0.673
0.638
0,645
0.403
AAAAK
0.687
0.635
0.614
0.678
0.403
0.701
A
0.614
0.614
A
A Agreement Rate
19/24 (79.2%)
26

---

Published as a conference paper at ICLR 2026
Table 11: Human interpretability judgments (Epidemiology). Expert pairwise evaluations com- pared against MOSE decisions for 25 model pairs. Agreement indicates whether MOSE matches expert preference.
MOSE(A)
MOSE(B)
Expert
Agree
0.464
0.578
B
0.524
0.525
B
0.601
0.500
A
0.663
0.507
A
0,464
0.662
B
0.662
0.500
A
0.524
0.540
0.524
0.502
0.390
0.540
0,578
0,502
B
0.524
0.663
0,550
0.556
B
0.658
0.548
AMMARAR
A
0.601
0.502
0.601
0.551
0.524
0.658
AAAYK
0.662
0.663
0.662
0.502
0.578
0.551
0.525
0.464
0.556
0.540
0.540
0.502
0.548
0.488
A
0,464
0.556
B
0.555
0.548
A
Agreement Rate
19/25 (76.0%)
27

---

Published as a conference paper at ICLR 2026
C
TECHNICAL DETAILS
In this section of the Appendix, we provide additional details on the implementation of MATHMO.
C.I ALCORITHI//SEEUDOCDEE
Algorithm Bi-level Adaptive Search Loop in MATHMO
1: Input: Problem description p, dataset Dp, number of iterations T, number of frameworks F. number of candidates L
2: Sample initial frameworks: f; ~ pol- pp), Vie [P]
3: Initialize histories: 5 = 0, vi E [F]]
4: for = 1 to 7 do
5: Upper-level: Compute utility: a aff) PCreto-UCB(ff: 1((-1)),, AiE [F]
6: Select framework: f. arg max,, , e F a(fi)
7: Lower-level: Sample candidate pairs: (m,, a5 ~ . p. f., ". v. [
Estimate objectives: fg (MM, p. far Sh 11 Vj [Z]
8:
9: Select candidate pair: (m)")a(0)
arg max (m.a) (m.a)
HV HVV (F))
10: Solve and evaluate to obtain (t)
11: Update history: Sta) + S.tt
(m((), al)), r(00]
12: end for
S(
13: Output: Pareto set: p Pareto
C.2 LLM SEARCH OPERATORS
To recap, MATHMO leverages LLMs as search operators, specifically for three distinct roles: sampling realizations, surrogate evaluations of candidate models, and model of subjective evaluation (MOSE).
Generative sampler [frameworks). Conditioned on the problem description p. LLMs are prompted sample suitable modeling frameworks, which we denote ~ pa p). Specifically, the LLM is instructed to return proposed frameworks in JSON structure containing two fields: "modeling framework" (string) and "framework description". The prompt skeleton and output format are described in Figures 11 and 12 respectively. Note that the descriptions enclosed in () represent placeholder values that populated dynamically runtime.
Generative sampler [model and algorithm). Conditioned on the problem description p, selected framework f, and a set of previously evaluated models within that framework S the LLM generates
model-algorithm pair, (m, a) p p. ST)).. The output is returned in a JSON format with the fields; "model" (a Python code string), "dependencies" (a list of package names), and "explanation" (a rationale for the design). Prompt details and output structure are described in Figures 13 and 14.
3. Surrogate evaluations [candidate model]. For each candidate model- algor--m pair (m, a). LLMs are employed as surrogates estimate nulti-objecttiv performance metrics, offering low-cost approximation. This approach is motivated by the unstructured nature of the input space, which differs from traditional numerical or mixed-integer domains in Bayesian Optimization (Snock al., 2012), and is supported by recent successes of LLM-based surrogates in language- driven domains (Liu et al., 2024; Requeima et al., 2024). Each objective fy E R for j E [k] is estimated independently using the LLM, based on input (m, ā, f. f,S"), where f, Pau (m, à p. f.S).. Multiple predictions are sampled in parallel to construct an empirical predictive distribution over the objectives. The prompt to achieve this is described in Figure 15.
4. MOSE Surrogate Model of Subjective Evaluations. LLMs also serve surrogate for subjective human judgment, enabling a generalized, cross cross-frament framework assessment mechanism for qualitative criteria. Prior work has demonstrated that LLMs can effectively model human preferences in alignment, safety, and prose diversity tasks (Bai et al., 2022; Bradley et al., 2024). In our framework, MOSE uses LLM to predict whether proposed model m, is subjectively preferred
28

---

Published as a conference paper at ICLR 2026
over baseline models m, Miet given problem description p. This is formalized Peoss > m, p). A prediction of "1' indicates that my is subjectively superior. The associated token probabilities extracted. This process is repeated for each m, Maes where the preference scores against each reference model are averaged to compute the final MOSE score. The prompt structure depicted in Figure 16.
You are an expert modeling assistant. Your task to help the user create a formal model to solve their problem.
Task:: You will recei description of the problen em and the desired objective (8) of the model. Your job is to propose modeling framework that can be used to solve the problem. You should also provide detailed explanation of your proposed framework, including any assumptions or constraints that making.
*Problem description:**
**Pobblem instance descripticns:** INSTANCE
Outputt format requirement:"
You must output your respense single, valid JSON object. other text should precede or follow the JSON. The JSON object must strictly follow this structure: SOUTFUT_FORMT_BBRUREERRRRC
Figure 11: Prompt structure for framework proposal.
moodeling_ ing_nnameworrre (string) "concise terminclogy to generally describe the modeling framework (e.g., mathematical imiiaitt,on, dynamical systems)", drraaeworkkdescriittae (string): "high-level description of the proposed modeling fzamework"
Figure 12: Output format for proposed frameworks.
C.3 ADAPTIVE SEARCH
Having detailed the implementation of LLM search operators in MATHMO, now cover various implementation details the adaptive search process. Subsequently, we tabulate the key hyperpa- rameters and describe the computational resources.
Upper-level: framework selection. At the beginning the search process (t = 0). an initial set of candidate frameworks is proposed independently using the LLM-based sampler po pp). Then in each iteration:
1. Compute statistics. For each framework, we compute summary statistics using the framework- specific historical performance vectors {ra (m,, a, r:) S specifically the empirical mean Jy E R and variance of E A
2. Compute UCB. As our a..,.) is implemented using the Pareto-UCB policy, we compute the UCB vector UCB, E & for each framework using Equation (2).
Identify Pareto set. Using the set of UCB vectors, the set of Pareto optimal (non-dominated) frameworks is identified.
29

---

Published as a conference paper at ICLR 2026
You are an expert modeling assistant. Your task to help the user create a formal model to solve their problem.
Task:44 You will receive description of the problem. and the desired objective of the model. Your job is to return the model that can generate the required output/solution in the output format speci The model you generate should belong to the modeling framework specified.
**Problem description:**
**Problem instance descripticns:: INSSTANGEIIKSCKIITTOOO
Choseen modeling framework:**
IMOCELING_FRAMEOOR
Outppt format requirements:** BAATUU___FOOAARREEURREEEEE
Figure 13: Prompt structure for model/algorithm proposal.
"model" " (Python code) "comleete Python code of the model and algorithm generated to represent and solve the provided probl em "
cependencies* : "list of external Python package dependency"
"odel explanation (str): "detailed description of the generated model and algorithm"
Figure 14: Output format for proposed model/algorithm.
Selection. If there exists more than one framework in the Pareto-UCB set, one framework is randomly selected to be explored next.
Lower-level:
local exploration. In each iteration of lower-level exploration, the following steps
occur:
1. Proposal. A set of candidate model-algorithh pairs are sampled, denoted as S = (t) a0(()) | [7]]
2. Surrogate estimation. For each candidate pair, we obtain an estimated objective vector p(i)
Pau( (f) a());,;; f, S5).
3. Selection. The pair (m, ax) that yields the largest estimated hypervolume improvement (m, a)
arg max HV(m, a; Trat) is selected undergo evaluation.
4. Evaluation. We execute the model and algorithm as subprocess. If any errors occurred during execution, the error trace is extracted and passed the LLM fix any mistakes and regenerate the model. If the model still does not execute after 3 MAX RETRIES, the process returns to the proposal step. Note that there is also a 300 second TIMEOUT imposed on model execution. model timeout, it is forcefully terminated.
5. Observe objectives. The objective vectors r E R are obtained by evaluating the model and generated outputs, and the triplet are added to the set models Sf + Sf am).
C.4 MISCELLANEOUS IMPLEMENTATION DETAILS
Key hyperparameters. We detail the hyperparameters for implementing MATHMO in Table 12.
30

---

Published as a conference paper at ICLR 2026
You are an expert evaluation assistant. Your task is to help the user evaluate models that were generated for a particular problem.
**sk::* Your role is to help evaluate mathematical models designed for specific problem. You will be given: description of the problem, the objective the models are intended to achieve,
A history of previously generated models along with their
performance metrics,
candidate model for evaluation.
Based on this information, predict the likely performance of the new candidate model.
**Problem description:**
(GENERAL PROBLEM DESCRIPTION)
**Problem instance descripticns:** [INSTIICEJDSSCRPIIOONN
*History of proposed modcl9:44 [MODEL_HISTORY]
**Output format
[OCTPUT_FORMAI_RR) REQUIREMENTS)
Figure 15: Prompt structure for surrogate evaluations.
You are an expert at evaluating the subjective quality of models. Your task is to assess the subjective quality of a model based on its description against the baseline reference provided.
**aakk: You will receive a target model description (in code) and baseline model description (also in code) Your job is to assess the (CRITERION) of the target model compared to the baseline reference. Provide detailed assessment using the specified output format.
*uubjcctive criterion assessed:** CRRTEERIONDDESCRIPTTON
Important instructions:**
Your assesmment/explanaiiie should be grounded in semantic meaning of the target model and the baseline reference. The assessment should based your best intuition and semantic understanding of the models.
Then score the target model, returning if the target model is more VELITERION_VALUE than the baseline reference model, or 0 if net.
**aaget model (TARGET_MODEL)
Bascline reference model:** [BASELINE_REF]
Figure 16: Prompt structure for surrogate evaluations.
LLM. We use gpt- spt--4o-2024-55-2 as the underlying LLM in all experiments.
Computer resources. We all experiments AMD EPYC 7v13 64-Core Processor.
31

---

Published as a conference paper at ICLR 2026
Code and reproducibility. The full implementation of MATHMO, along with the code necessary to reproduce all key results, will be released on GitHub upon acceptance of the paper.
Table 12: Description of key hyperparameters.
Hyperparameter
Description
Value
Number of frameworks
I
Number of candidate model-algorithm pairs proposed in each iteration of local exploration
Number of MC extimates for surrogate evaliations
(c.d)
Pareto-UCB exploration borus hyperparameters
(1. 1) (default)
MMal
Number of baseline models in MCSE ovaluations
Number of scarch iterations
20
LLM hy perparameeerr (sampling temperature)
0.7 (default)
LLM hyperparameter (top-p sampling)
0.9 (default)
CIMBOUT
Max runtime scconds) allowed for each model-algorithhm pair exocute
300
D EXPERIMENTAL DETAILS
In this section of the appendix, we will describe the datasets and metrics employed in our empirical evaluations.
D.I DATASETS
Traveling Salesman (TSP). The Traveling Salesman Problem (TSP) is foundational combinatorial optimization problem where the objective is to find the shortest possible route that visits set of cities exactly once and returns to the starting point. The two primary trade-offs in modeling TSP journey cost (total tour length) and runtime (solution time). Journey cost reflects the quality of the solution and is crucial in applications like logistics and manufacturing, while runtime is vital in scenarios requiring rapid decisions, such as dynamic routing. Common modeling techniques include: (1) exact methods such as Integer Linear Programming (ILP), which guarantee globally optimal solutions but scale poorly with problem size; (2) metaheuristic approaches like Genetic Algorithms, which offer faster approximate solutions at the cost of optimality; and (3) domain-specific heuristics such as nearest-neighhor insertion algorithms, which are simple and computationally efficient (Bellmore and Nemhauser, 1968). For our experiments, we generated 10 random Euclidean instances with 30-50 cities each. Each instance was created by uniformly sampling city coordinates in a 2D unit square, a standard method for generating synthetic TSP datasets.
Job Shop Scheduling (JSS). Job Shop Scheduling a canonical operations research problem that involves assigning a sequence of jobs to set of machines, where each job consists of a series of operations with specific processing requirements. The primary objectives are to minimize the makespan (i.e., the total time required to complete all jobs) and to reduce runtime, which becomes critical in dynamic or large-scale industrial systems. A lower makespan increases throughput, directly impacting productivity, while efficient computation ensures that schedules can be adapted in real time. (1) Constraint Programming (CP), which provides precise encodings but may struggle with scalability; (2) metaheuristic methods such as Tabu Search or Simulated Annealing, which balance exploration and exploitation to find high-quality solutions efficiently; and (3) greedy or rule-based heuristics, which offer speed and interpretability the cost of optimality (Xiong et al., 2022). We use 10 well-known benchmark instances from Lawrance (1984), which are widely adopted in the scheduling literature. These instances span 10 30 jobs, 10 machines, and 50 to 300 operations. The dataset is available through the open-source (https:/ / gighhb..bb.....gggg com/ abloo22/ job__hop_pp_pppppppppppppppopg_a
lib repository
Ecology. Ecological modeling seeks to understand and predict interactions among species and their environments, often involving dynamic systems such as predator-pre relationships. A key modeling trade-off in ecology lies between predictive performance-cann capturing future population dynamics accurately-and interpretability, which is critical for gaining ecological insights and informing conservation efforts. Common modeling approaches include: (1) differential equation systems such as Lotka- Volterra models, which provide interpretable representations of species interactions; (2) classical time-series models like ARIMA, which are effective for chort-term forecasts; and (3) probabilistic graphical models, which capture structured uncertainty and latent ecological processes (van den Berg al., 2022). We use a dataset containing Hare-I-nxx populations (Stenseth et al.,
32

---

Published as a conference paper at ICLR 2026
1997), which records annual observations of the Snowshoe Hare and Canadian Lynx populations multiple decades.
Epidemiological modeling focuses understanding and forecasting the spread of infectious diseases, often under constraints that demand both accurate prediction and clear inter- pretability for public health petisionnmaking Predictive performance ensures that interventions be timed effectively, while interpretability allows stakeholders to understand transmission mecha- nisms and policy implications. Common approaches include: (1) compartmental models (e.g., SIR, SEIR), which capture the flow of individuals through disease states using differential equations; (2) statistical models such as Poisson and negative binomial regressions, which model count data under uncertainly; and (3) time-series forecasting techniques, including autoregressive and neural models, for flexible temporal prediction (Xiang et al., 2021). For our experiments, we use COVID-19 time series data from Italy, sourced from the COVID-19 Data Repository by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (Dong al., 2020). The dataset contains daily counts of confirmed cases, deaths, and recoveries.
Epidemiology.
We note that in all our experiments, the dataset is provided as an input only at runtime- the model has been generated by the LLM. The LLMs themselves do not have access to the dataset contents during model generation; they are only given high-level metadata, such as the number of features, problem size (e.g., number of operations cities), or time series length, to inform their proposals.
D.2 METRICS
Hypervolume. The hypervolume (HV) metric quantifies the volume of the objective space that is dominated by set of solutions, relative to a fixed reference point. It serves as standard measure in multi-oheective optimization, capturing both convergence and diversity of the solution set. Formally, let R = ,' be a set of n k-dimensional objective vectors and let Tee E I& be reference point that dominated by all vectors in R. The hypervolume is defined as:
NV(R; = 2 U (ri, Freta) x .... [Tk, [Kk, Free,K)) PER
(5)
where X denotes the Lebesgue measure in IR All objectives are first normalized to [0, 1]. and we set = 1.1 ensure it lies outside the normalized Pareto front. We compute hypervolume using the pymoo library (Blank and Deb, 2020) (https://pypp wwwp:///p//////poooooo/oo/
Relative Hypervolume Improvement. To assess progress over time, we compute the relative hypervolume improvement, which quantifies the gain hypervolume relative to the best value achieved at previous timestep. Let HV, and HV, denote the HV at iteration and l', where L < t.
HV, HV
RHT,
(6)
HVe
We employ this metric compare the impact of search strategies over the course of search.
Normalized RMSE. Root Mean Squared Error (RMSE) is standard regression metric that measures
the average magnitude prediction error. our context, we use a normalized RMSE (NRMSE) to account for scale differences across objectives. Given set of ground-truth values {y H and predictions {6e H0 RMSE is defined as:
1 R y
RMSE
0.)2
(7)
i=1
We normalize this by the empirical range of the true values og yielding:
NRMSF RMSE
(8)
0g
Permutation entropy. Permutation entropy (PE) is model-free measure of complexity for time series or ordered sequences, capturing the unpredictability of local ordering patterns. Given a time series {a:)_ the sequence is partitioned into overlapping windows of length d (embedding
33

---

Published as a conference paper at ICLR 2026
dimension), and each window is mapped to permutation pattern based on the relative ordering of its elements. Let Ty denote the i-th unique pattern and p(T.)) its empirical frequency. The PE is then defined as:
Ha = - Σ p( log
which is often normalized by log(d!) to yield a value in [0, 1]. We compute PE using the antropy library (attps: / /ttp:/pppp.... orgg ppoppprectttttoooooooooooooooeee / /anyr/ This metric provides lens into the structural complexity of sequences produced by time-series models
34


Reviews:
[{"review_id": "NwMHZaMRjZ", "reviewer": "Reviewer_N3pu", "summary": "This paper introduces MATHMO, a novel system for automated mathematical modeling that leverages Large Language Models (LLMs) within a bi-level adaptive search framework. The work conceptualizes mathematical modeling as a sequential decision-making problem under uncertainty, addressing three key challenges: (1) fundamental modeling uncertainty, (2) balancing multiple conflicting objectives, and (3) incorporating subjective qualities into model evaluation.\n\nThe core contribution is a bi-level search architecture where the upper level performs adaptive framework selection using Pareto-Upper Confidence Bound (Pareto-UCB), while the lower level conducts local exploration within selected frameworks using LLM-based surrogates and Bayesian optimization principles. The system incorporates MOSE (Model of Subjective Evaluations), which uses LLMs to approximate human preferences for subjective criteria like interpretability across different modeling frameworks.\n\nThe experimental evaluation spans four diverse real-world tasks: Traveling Salesman Problem (TSP), Job Shop Scheduling (JSS), Ecology modeling, and Epidemiology forecasting. Results demonstrate MATHMO's ability to discover diverse Pareto-efficient frontiers of models that effectively balance trade-offs between objectives such as solution quality vs. runtime, and predictive accuracy vs. interpretability.", "questions": "1. Scalability: How does the system scale to larger problem instances or longer modeling horizons? The current evaluation is limited to relatively small problems.\n2. LLM Prompt Sensitivity: Given the known sensitivity of LLMs to prompt variations, how robust is the system to different prompt designs or LLM model choices?\n3. Dynamic Framework Generation: The current approach samples and fixes frameworks at the beginning. Could dynamic framework generation further improve performance?\n4. Human-in-the-Loop: How could the system be extended to incorporate real human feedback during the modeling process rather than relying solely on LLM approximations?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. Novel problem formulation:  The paper presents a well-motivated and technically sound approach. Its treatment of automated mathematical modeling as a sequential decision-making problem is interesting.\n2. Practical impact: Demonstrates feasibility on real-world problems with meaningful trade-offs. Provide inspirations for human-AI collaboration in scientific modeling. The four benchmark problems represent genuine challenges in operations research, ecology, and epidemiology. The ability to discover meaningful trade-offs (e.g., accuracy vs. interpretability) has clear practical value.\n3. Technical Quality: The bi-level decomposition is well-motivated and technically sound. The integration of Pareto-UCB for framework selection and LLM-based surrogates for local exploration shows careful consideration of the problem structure.\n4. The paper is generally well-written with clear exposition and good organization. The formalization is accessible, and the figures effectively illustrate the key concepts. The related work section provides good context. Areas for improvement include:", "weaknesses": "1. Limited Scale and Diversity: While the problems are diverse, the evaluation is limited in scope (4 problems, 20 iterations). More extensive evaluation on larger problem sets would strengthen the claims.\n2. Theoretical Analysis: While the approach is intuitive, the paper lacks formal convergence analysis or regret bounds for the proposed adaptive search strategy.The lack of formal convergence guarantees or regret analysis leaves questions about the theoretical properties of the proposed approach.\n3.  Baseline Comparisons: While comparisons against MEoH and FunSearch are informative, comparisons against more traditional AutoML baselines (e.g., Bayesian optimization over predefined search spaces) would provide better context for the novelty claims.  The paper relies solely on general LLMs (gpt-4o) with search algorithms and prompt engineering, without exploring the effectiveness of domain-specific fine-tuned models. This approach may miss potential advantages of specialized models in mathematical modeling tasks. The paper does not adequately justify why the bi-level search framework with general LLMs is optimal. Alternative approaches, such as ensemble methods or hybrid search strategies, are not evaluated, leaving questions about design decision rationale and adaptability boundaries.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sIN3xLs82Y", "reviewer": "Reviewer_VgEc", "summary": "This paper introduces MATHMO, a novel framework designed to automate the process of mathematical modeling. The authors conceptualize mathematical modeling as a sequential decision-making problem under uncertainty, aiming to discover not a single best model, but a Pareto frontier of models that balance multiple, often conflicting, objectives. MATHMO employs a principled bi-level adaptive search strategy: an upper level selects between different high-level modeling frameworks (e.g., mathematical optimization vs. dynamical systems), while a lower level refines specific model formulations and algorithms within the chosen framework. A core technical contribution is the use of Large Language Models (LLMs) as versatile search operators for both generating candidate models (as code) and acting as surrogate models to efficiently estimate performance. Uniquely, the framework introduces MOSE (Model of Subjective Evaluations), an LLM-based component to quantify and incorporate subjective criteria like interpretability into the automated search process. The efficacy of MATHMO is demonstrated on four diverse real-world tasks, showing its ability to identify Pareto-efficient models that navigate complex trade-offs.", "questions": "- How does MATHMO guard against the LLM's inherent biases, which might cause it to completely miss entire families of effective models? Is there a risk that the search becomes trapped within the \"comfort zone\" of the LLM's training data?\n- Could you provide more details on the computational cost of a full run of MATHMO? Specifically, what is the wall-clock time and the approximate number of LLM API calls required for a 20-iteration run?\n- In Section C.3, you briefly mention a retry mechanism for execution errors. Code generation is notoriously brittle. Could you elaborate on the effectiveness of this self-correction loop? What percentage of generated models fail initially, and what types of errors is the LLM-based correction mechanism successful at fixing?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The paper considers heuristic design with multiple framework. \n- More applicable problems are tested compared with the baselines.", "weaknesses": "- The entire framework's success is fundamentally predicated on the quality and diversity of the models the LLM can generate. The search is effectively bounded by the LLM's pre-trained knowledge and potential biases. If the LLM has not been trained on sufficient examples of a certain niche but powerful framework (e.g., specific types of constraint programming or advanced statistical models), it may never propose them, leading to a \"rich-get-richer\" dynamic where common frameworks are over-explored.\n- The paper does not analyze the diversity of the initial framework proposals or the \"mutations\" suggested in the lower-level search. It's unclear if the LLM is genuinely exploring novel variations or just making superficial syntactic changes to the code. The search is only as good as the operator, and the operator here is a black box.\n- The paper validates MOSE's interpretability scores by correlating them with structural/functional complexity proxies (parameter count, permutation entropy). These proxies are not equivalent to interpretability. A model with few parameters can be completely uninterpretable if the parameters lack physical or domain meaning. Conversely, a complex model can be interpretable if its structure is well-understood (e.g., a large but modular compartmental model in epidemiology).\n- The core assumption is that an LLM's preference ($p_\\mathrm{MOSE}(m_t \\succ m_i \\mid p)$) is a faithful proxy for that of a human domain expert. This is a very strong and unproven assumption. A small-scale user study with actual ecologists or epidemiologists would be required to truly validate that MOSE captures what experts consider \"interpretable.\" Without this, MOSE is just optimizing for what a specific LLM thinks is interpretable.\n- The score is dependent on the initial, fixed reference set $M_{\\mathrm{ref}}$. The paper doesn't explore the sensitivity of the results to the composition of this small ($n=3$) reference set.\n- The experimental setup involves 20 iterations with a 300-second timeout per model evaluation. This is already a significant time budget. More importantly, each iteration in the lower-level search involves multiple LLM calls: $l=3$ candidate generations, and for each, $q=3$ surrogate estimations. This suggests at least 12 LLM calls per iteration, using a state-of-the-art model (GPT-4o). This approach seems computationally expensive and potentially slow, which may limit its practical application to problems requiring more extensive search or faster turnaround. The paper lacks a discussion on the computational overhead and how it scales.\n- The formalism $(m, a)$ and its implementation as a single Python script works well for the self-contained problems chosen. However, real-world mathematical modeling is often a multi-stage pipeline (e.g., data cleaning -> feature engineering -> model selection -> calibration -> post-hoc analysis). It is unclear how the current framework would handle such complex workflows where decisions and trade-offs exist at each stage. The current approach seems to conflate the entire solution process into a single generative step.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rpMVeLt5dL", "reviewer": "Reviewer_dMwL", "summary": "The paper introduces MATHMO, a framework for automated mathematical modeling that treats model construction as a sequential decision-making process under uncertainty. It uses a bi-level adaptive search: the upper level selects modeling frameworks (e.g., optimization, dynamical systems, regression) while the lower level refines models and algorithms within each framework. Large Language Models (LLMs) are used as generators, surrogate evaluators, and subjective scorers via the proposed MOSE module, which estimates interpretability and other qualitative factors. The system is tested on four tasks — Traveling Salesman, Job Shop Scheduling, Ecology, and Epidemiology — and shown to produce Pareto-efficient frontiers balancing multiple objectives such as accuracy, runtime, and interpretability. The work combines ideas from AutoML, symbolic discovery, and human-in-the-loop modeling into a unified adaptive system, aiming to automate core aspects of mathematical model design.", "questions": "Got it — here’s a tighter, natural-sounding version that feels like a human reviewer’s “Questions & Suggestions” section.\n\n---\n\n\n1. How general is MATHMO in practice? Are the modeling frameworks fixed at the start or can the system invent new ones? How do you prevent the LLM from proposing nonsense frameworks?\n\n2. Why not include a simple GPT-4 prompt baseline for context? It would help readers see what is gained by adding adaptive feedback. Did you test this internally? Also, do the surrogate and MOSE modules improve the *quality* of models or just the coverage of the Pareto front?\n\n3. The paper cites FunSearch, MEoH, and AutoFormulation. What is concretely new beyond combining their elements? How is the Pareto-UCB exploration different from MEoH’s evolutionary approach?\n\n4. The MOSE idea is interesting but underspecified. How consistent are its scores across prompts or random seeds? Any human comparison to check if MOSE’s “interpretability” aligns with human judgment?\n\n5.  Hypervolume is fine, but would it be possible for the authors to show some discovered models - I think it would make results more convincing. How often do generated models fail to run? What is the compute cost relative to the baselines?\n\n6. Can this approach scale to heavier tasks where each model evaluation takes hours or days? Could smaller open LLMs work as surrogates or is GPT-4-level capability required?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper offers a fresh and ambitious framing of mathematical modeling as a sequential decision process under uncertainty. It creatively combines LLM generation, surrogate modeling, and subjective evaluation within a clear bi-level adaptive search framework. The idea of MOSE for quantifying interpretability is novel and thought-provoking. Experiments across optimization and scientific domains show flexibility and coherence. The paper is clearly written, conceptually rich, and points toward an important new direction in automating the reasoning process behind mathematical modeling.", "weaknesses": "1. The experiments are small-scale (20 iterations, modest problem sizes) and mostly illustrative. They show proof of concept, not scalability. It would help to see at least one larger or higher-cost task to test how the adaptive search behaves when model evaluations are expensive.\n\n2.  The paper argues MATHMONAIVE serves as a GPT baseline, but this isn’t made explicit. A simple “GPT-4 one-shot” or “GPT-4 iterative prompting” baseline would clarify how much improvement truly comes from adaptivity rather than LLM creativity.\n\n3.   No baselines from classical AutoML (Auto-Sklearn, Optuna) or symbolic regression frameworks (PySR, Eureqa). These would help ground the claim that MATHMO goes beyond existing automation tools.\n\n4. The MOSE module is an appealing idea, but there’s no evidence it correlates with actual human judgments of interpretability. Even a small human study or qualitative comparison would make it more credible.\n\n5. The method depends heavily on GPT-4’s internal priors. It’s unclear how stable results are under different prompts or smaller LLMs. This raises questions about reproducibility and accessibility.\n\n6.  The paper reports hypervolume numbers but doesn’t show concrete model examples or code snippets. It’s hard to judge if the discovered models are sensible, interpretable, or novel.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QWLluRaUch", "reviewer": "Reviewer_2x75", "summary": "It proposes MATHMO that automates mathematical modeling using a bi-level search: an upper level selects modeling frameworks via Pareto-UCB, while a lower level refines models/algorithms within a framework. It uses LLMs as generators, surrogates, and for evaluating subjective qualities. The authors demonstrate MATHMO's efficacy on four real-world tasks (TSP, Job Shop Scheduling, Ecology, Epidemiology), showing it can discover Pareto frontiers of models that balance competing objectives like accuracy, runtime, and interpretability.", "questions": "It would be useful to see a comparison and discussion with other related methods for automated modelling, such as Chain-of-Experts、ORLM、LLMOPT and benchmarks such as NL4OPT、MAMO、IndustryOR.\n\nThe manuscript mentions a 300-second timeout for model execution. How were models that timed out or failed to execute incorporated into the history S_t? Were they assigned a penalty value, and if so, how did this affect the utility functions α and β?\n\nFor the MOSE module, how was the fixed reference set M_ref constructed? Was a specific strategy used to ensure diversity and representativeness across different frameworks?\n\nThe results show that on the JSS task, the surrogate's NRMSE worsened over time. Beyond the proposed explanation, were any other strategies attempted to improve surrogate predictions for runtime? For example, was feature engineering from the code or a separate, specialized predictor for this objective explored?\n\nThe hyperparameter l (the number of candidate pairs proposed per iteration) is set to 3. A larger *l* might improve the lower-level selection at a higher computational cost for surrogate evaluations. Was any sensitivity analysis performed on this parameter?\n\nThe framework sampling is done once at the beginning of the search. Did the authors experiment with allowing the upper level to dynamically propose new frameworks mid-search, rather than being restricted to the initial set?\n\nIn the comparison against baselines (MEoH, FunSearch), could you provide the detailed settings used for both baselines? Also, is the budget of 20 model–algorithm evaluations too small? A common practice in LLM-driven algorithm design is to use hundreds of evaluations.\n\nThe manuscript focuses on discovering the Pareto frontier. In a practical deployment, how would MATHMO be used to recommend a single model to an end-user? Does the framework include a mechanism for incorporating user-specific preferences over the objectives to make this final selection?\n\nWhy was GPT-4o chosen, and were other LLMs compared for the sampler/surrogate roles?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "Clearly defines modeling as a sequential, multi-objective decision problem.\n\nUses LLMs as core search operators for generation, surrogate evaluation, and subjective scoring.", "weaknesses": "Performance is highly sensitive to prompt engineering and LLM outputs (e.g., poor surrogate predictions for JSS runtime).\n\nLack of comprehensive comparison with existing baselines and benchmark evaluations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)