MM-Agent: LLM as Agents for Real-world Mathematical Modeling Problem
Authors: Fan Liu
Zhe-Rui Yang
Cancheng Liu
Tianrui SONG
Xiaofeng Gao
Hao Liu
Venue: NeurIPS 2025 poster ()
Keywords: Mathematical Modeling Agent
LLM Agent
LLM
Primary area: deep_learning
TL;DR: We introduce MM-Agent and MM-Bench to enable and evaluate LLM-powered end-to-end mathematical modeling, achieving superior performance on real-world problems and Finalist Award  in MCM/ICM 2025.

Abstract: Mathematical modeling is a cornerstone of scientific discovery and engineering practice, enabling the translation of real-world problems into formal systems across domains such as physics, biology, and economics. Unlike mathematical reasoning, which assumes a predefined formulation, modeling requires open-ended problem analysis, abstraction, and principled formalization. While Large Language Models (LLMs) have shown strong reasoning capabilities, they fall short in rigorous model construction, limiting their utility in real-world problem-solving. To this end, we formalize the task of LLM-powered real-world mathematical modeling, where agents must analyze problems, construct domain-appropriate formulations, and generate complete end-to-end solutions.We introduce MM-Bench, a curated benchmark of 111 problems from the Mathematical Contest in Modeling (MCM/ICM), spanning the years 2000 to 2025 and across ten diverse domains such as physics, biology, and economics.  To tackle this task, we propose MM-Agent, an expert-inspired framework that decomposes mathematical modeling into four stages: open-ended problem analysis, structured model formulation, computational problem solving, and report generation.Experiments on MM-Bench show that MM-Agent significantly outperforms baseline agents, achieving an 11.88\% improvement over human expert solutions while requiring only 15 minutes and \$0.88 per task using GPT-4o. Furthermore, under official MCM/ICM protocols, MM-Agent assisted two undergraduate teams in winning the Finalist Award (\textbf{top 2.0\% among 27,456 teams}) in MCM/ICM 2025, demonstrating its practical effectiveness as a modeling copilot.

MM-Agent: LLM as Agents for Real-world Mathematical Modeling Problem
Fan Liu"!, Zherui Yang*), Cancheng Liu', Tianrui Song', Xiaofeng Gao?, Hao Liu
AI Thrust, The Hong Kong University of Science and Technology (Guangzhou)
Department of Computer Science and Engineering, Shanghai Jiao Tong University
connect. coneoettttttgg...omeett.... edu. en; yangzhr9@outlook fliu2360hkust
(canchengl lu,, (sokstt7 gao-xte pro-xfocs.o. sjtu. edu, en; millluuuut....
connect. edu
Abstract
Mathematical modeling is a cornerstone of scientific discovery and engineering practice, enabling the translation of real-world problems into formal systems across domains such physics, biology, and economics. Unlike mathematical reasoning, which assumes a predefined formulation, modeling requires open-ended problem analysis, abstraction, and principled formalization. While Large Language Models (LLMs) have shown strong reasoning capabilities, they fall short rigorous model construction, limiting their utility in real-world problem-solvingg To this end, we formalize the task --weere real-wor world mathematical modeling, where agents must analyze problems, construct domain comain-approprrate formulations, and generate complete end-to-end solutions. We introduce MM- MMMBBenc,, curated bench- mark of 111 problems from the Mathematical Contest in Modeling (MCM/ICM) |] spanning the years 2000 to 2025 and across ten diverse domains such as physics, biology, and economics. To tackle this task, we propose MM-Agent, inspired framework that decomposes mathematical modeling into four stages: open- ended problem analysis, structured model formulation, computational problem solving, and report generation. Experiments on MM-Bench show that MM-Agent significantly outperforms baseline agents, achieving an 1 1.88% improvement over human expert solutions while requiring only 15 minutes and $0. 88 per task using GPT-4o. Furthermore, under official MCM/ICM protocols, MM-Agent assisted two undergraduate teams in winning the Finalist Award (top 2.0% % among 27, 27,456 teams) in MCM/ICM 2025, demonstrating its practical effectiveness modeling copilot.
expert-
Code available at httppp////gghhaaooooccaaaaaaaaaa
Demo Wetfeaatteelllliilggggg/h/aaattt
is
available
at
http:://uuginngfaae...o//
Introduction
Mathematical modeling serves cornerstone methodology for formulating, analyzing, and solving complex real-world problems, underpinning scientific discovery and technological advancement across applied mathematics, natural sciences, engineering, and the social sciences. In practice, this process often begins by identifying the core problem, abstracting it into mathematical form, constructing appropriate models, and solving them to generate actionable insights. It enables the
"quual contribution. 'Correspondence to Hao Liu.
https::/www.cco contest. wwwapccccc....oo htt.://wwwwo.o...ooooooooooett/a
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

(a) Well-defined Mathematics Problem
(b) weal-world Open-ended Mathematical Modeling Problem
Formulated Problem
Decision Decision-making Scenario
Problem Formulation
Task:
x 2y 10 I y = 6
allocate frrehghting
B. plan evacuation routes
Agent
C: diepotch hellonpters
Agent
Forest Fire Management
D: schecule supply deliverice
Agent
O Agent
((8)
Σ
[v
atd
T
2
1/
Linear Regression
Computational
Solving
Agent
Solution
Modeling
Solving
Report
Figure 1: Traditional well-defined mathematics problem vs LLM-powered open- open-ended mathematical modeling problem. Left: A well-defined mathematical problem, where an agent solves a well-defined problem to obtain a solution. Right: An open- open-ended mathematical modeling problem, where given an abstract application scenario or phenomenon, the agent first needs to formulate the mathematical problem before solving it and providing an end-to-end solution.
transformation of ill-posed challenges, such as epidemic control, energy forecasting, and supply chain management, into mathematical systems that support analysis, prediction, and decision-mann making through abstraction, theoretical formulation, empirical validation, and iterative refinement 2 Un- like mathematical reasoning, which starts from fixed formulations, mathematical modeling demands open-ended problem abstraction, assumption design, and domain -ggonddg interpretation, making context-sensitive and hard to automate, as shown in Figure Recent advances in Large Language Models (LLMs) offer new opportunities to automate parts of this workflow, showing promise in symbolic reasoning, scientific problem-solving, and numerical computation 3 LLM-based modeling agents could unlock scalable, efficient solutions across disciplines. However, current agents often fail to capture essential modeling principles, such as abstraction, constraints, and assumptions, leading to oversimplified and scientifically invalid outputs. As shown in Table they frequently omit key assumptions, producing models with limited real-world validity.
5
Developing
To address this gap, we formally define the task of LLM-powered real-world mathematical modeling, which requires agents to translate complex real-works problems or phenomena into structured and executable modeling pipelines, culminating complete analytical reports. To enable systematic evaluation, we introduce MM-Bench, new benchmark constructed from 111 real- weal-world problems adapted from MCM/ICM, spanning the years 2000 to 2025. MM-Bench covers ten application domains (e.g., physics, biology, and economics) and eight modeling task types (e.g., decision- making, prediction, and evaluation). Each sample includes rich contextual components (e.g., textual descriptions, task goals, dataset information, and variable definitions) and requires agents to conduct problem interpretation, model formulation, and numerical reasoning in an integrated, end-to-end fashion. A detailed breakdown of task types and domain distribution provided in Appendix B.
To address this task, we propose MM-Agent, an end-to-end solution for open-ended real-world modeling problems. Inspired by expert workflows, MM MM-Agent systematically analyzes unstructured problem descriptions, formulates structured mathematical models, derives solutions, and generates analytical reports. Among these stages, the modeling step poses the greatest challenge, it requires abstracting complex scenarios into mathematically coherent formulations grounded in both problem context and solution feasibility. To address this, we introduce the Hierarchical Mathematical Modeling Library (HMML): a tri-level knowledge hierarchy encompassing domains, subdomains, and method nodes. HMML encodes 98 high-level modeling schemas that enable both problem-awand -waree and solution- aware retrieval of modeling strategies, supporting abstraction and method selection. Specifically, MM-Agent first analyzes the problem and decomposes it into subtasks. It then retrieves suitable methods from HMML and refines its modeling plans via an actor-critic mechanism. To solve the models, the agent autonomously generates and iteratively improves code using the MLE-Solver for efficient, accurate execution. Finally, it compiles a structured report summarizing the modeling approach, experimental results, and key insights.
Our contribution can summarized as follows: (1) We develop MM-Bench, a benchmark comprising 111 real-world mathematical modeling problems across 8 problem types and 10 domains, designed to

---

evaluate the mathematical modeling capabilities of LLM agents. This benchmark has been carefully created based on real-world competitions. (2) To enhance the mathematical modeling capabilitiess of L.M agents, we construct the HMMI, three-tiered structure that organizes and retrieves modeling methods through broad domains (e.g., optimization, simulation), specific subdomains (e.g., linear programming, Monte Carlo simulations), and method nodes representing techniques, core ideas, and applications, enabling precise task-specific retrieval. (3) We introduce MM-Agent, an autonomous agent framework to create mathematical representations of real-world scenarios for making predictions or providing insights. (4) We conduct comprehensive experiments on the proposed benchmark and demonstrate that MM-Agent effectively solves mathematical modeling tasks, outperforming baseline approaches. with an average cost of $0.88 and $0.56 per task on GPT-4o [6] and DeepSeck R1 M7. respectively, and achieving an 11.88 % gain over human expert solutions. Furthermore, following official MCM/ICM protocols, MM-Agent helped two undergraduate teams win the Finalist Award (top 2.0% % out of 27,456 teams) in MCM/ICM 2025.
2 Related Works
LLM Agents. Recent advances have led to the development of LLM-based agents that incorporate structured planning, reasoning, and interaction capabilities. By leveraging mechanisms such as memory augmentation, reflective reasoning, and tool usage, these agents enhance task decomposition, iterative refinement, and adaptive problem-solvin
8
9
10
I1
12] As a result, LLM- based agents
have been successfully applied in diverse areas, including software engineering [3] [5] 16], game
playing [7
18
19
20,
21 22], human interaction modeling [23 24], cybersecurity [25] 26 27].
roboties 22
32 data science [33
34
35
36
medical diagnosis (37 38
web
automation 41
and scientific research [5
46
48 49).
LLMs for Autonomous Research. Automated scientific workflows have enabled the integration of LLMs across various research stages, such as literature review, idea generation, experimental design, and scientific writing, Some studies focus on general research tasks [88, while others explore specific domains like Al
45 46 50, 47). biomedical discovery [51], chemical experiments (52,, and traffic research [33] For instance, Agent Laboratory [46] is an autonomous 4LLM-based framework designed to expedite Al research by managing key stages such as literature review, experimentation, and report generation. Similarly, ORGANA (52) is robotic system that automates chemical experiments by integrating decision-making and perception tools. It collaborates with chemists via LLMs to define objectives and generate detailed experiment logs.
LLMs for Mathematics. LLMs has advanced mathematical problem-solving through curated datasets and improved reasoning strategies 4]. The math [54] 55 benchmarks have become key resources for evaluating mathematical competence, especially when paired with prompting techniques such as COT [56]. In formal mathematics. LLMs have been fine-tuned on theorem- perreingp datasets
like M2niF2F 57 and integrated with proof assistants such as Lean [58] and Coq [59]. Program-aie aided reasoning further enhances LLM performance by allowing models to generate and execute code for verification (60). While LLMs perform well on well-defined mathematical tasks [61] 62 63 64]
with clear symbolic goals, mathematical modeling remains an open-ended challenge that requires translating real-world scenarios into formal representations, often without a single correct solution.
To the best of our knowledge, MM-agent is the first work to explore the application of LLMs to real-world mathematical modeling problems. To facilitate autonomous mathematical modeling, we de- velop an automated pipeline encompassing problem analysis, mathematical modeling, computational solving, and solution reporting.
3 Building LLM Agent for Real-World Mathematical Modeling Problems
Section 3.1 introduces the task of real-world mathematical modeling and presents the construction of MM-Bench, the first benchmark designed to enable systematic evaluation of LLM-based modeling agents on open-ended tasks. To further support model construction, also introduce the Hierarchical Mathematical Modeling Library (HMML), which encodes a tri-level knowledge hierarchy spanning domains, subdomains, and method nodes to facilitate structured method selection and abstraction in Section 3.2 In Section 3.3 we present MM-Agent, an expert-inspired framework that decomposes the modeling process into four key stages: open-ended problem analysis, structured model formulation, computational problem solving, and report generation.
3.1 MM-Bench
3

---

Benchmark Construction. Real-world mathematical modeling competitions, such as MCM/ICM, challenge undergraduate students worldwide transform complex real-world phenomena (e.g., risk management, biological dynamics) into mathematical frameworks for prediction, optimization, and decision-making making 1 2). Drawing participation from a large and diverse pool of teams across many countries and regions [65] these prestigious contests require participants to collaboratively interpret open-ended problems, conduct in-depth analyses, and develop comprehensive solutions [66]. These contests offer natural benchmark for evaluating the problem-solving capabilities of LLM agents in complex, real-world scenarios. We collect all competition problems from the MCM and ICM contests held from 2000 to 2025 + which include both the problem descriptions and associated attachments, such as datasets. We then use GPT-4o to extract the following elements from the competition problems: the background information describing the context of the problem. the problem requirements outlining the tasks to be completed, the dataset path indicating the location of the dataset, the dataset description providing details about the dataset, and the variable description explaining the attributes within the dataset. For policy-oriented or decision-focuse tasks, datasets may not be provided, as these problems typically emphasize qualitative reasoning or scenario-based analysis. Finally, we manually review and correct errors in the extracted information, resulting in the creation of the Mathematical Modeling Benchmark, named MM-Bench. The resulting MM-Bench consists of 10 domains, 8 task types (e.g. decision-making, prediction, evaluation el al.), and a total of 111 problem samples. Detailed statistical information is provided in Section B of the Appendix.
MM-Bench evaluates the performance of agents on weal-world mathematical modeling problems, which involve translating real-world phenomena into simplified mathematical forms to analyze, interpret, and predict system behavior and outcomes. Given a mathematical problem F, the agent accesses all relevant content f E F (e.g., background information, problem requirements, dataset path, dataset description, and variable description) to generate a final solution report.
Task Formation.
Since real-world mathematical modeling problems are open-ended and often lack standard solutions, we reference official modeling evaluation criteria to assess agent performance (66) Specifically, we evaluate the final solution report along four key dimensions: (1) Analysis Evaluation. Examines problem definition clarity, identification of key components, and the logical coherence between sub-tasks and overarching objectives. (2) Modeling Rigorousness. Focuses on rigor and rationality, evaluating whether the assumptions are clearly stated and justified, and whether the chosen methods, metrics, and model structure accurately and scientifically represent the real-world problem. (3) Practicality and Scientificity. Evaluates the practicality and scientific validity of the model, ensuring that it is realistically applicable, provides valuable insights for decision-making, and adheres to scientific principles. This stage also verifies whether the model is theoretically sound and considers all relevant scientific factors ensure its validity. (4) Result and Bias Analysis. Measures the clarity, interpretability, and analytical depth of results, with attention to identifying and mitigating data or modeling biases to ensure robustness and transparency. We conduct both LLM-based and -opern-human evaluations to ensure a comprehensive and reliable assessment. For further details, please refer Section ij the Appendix.
Evaluation.
3.2 Hierarchical Mathematical Modeling Library Construction
To enhance the mathematical mod- eling capabilities of LLM agents, we construct the Hierarchical Math- ematical Modeling Library (HMML), three-tiered structure that orga- nizes and retrieves modeling meth- ods through broad domains (e.g., optimization, simulation), specific subdomains (e.g., linear program- ming, Monte Carlo simulations), and method nodes representing tech- niques, core ideas, and applica- tions, enabling precise task-specific re- trieval. As shown in Figure 2 HMML adopts a tree structure. At the top
Hlierurciical Mathematical Modeling Library
Domain
Optimization
Machine Learing
Operations Reserch
Prediction
Evaluation
Graph Theory
Programming Theory
Stochastic Programming Theory
SubDomain
Lincar Programss Programming idra. lincar abjoctive functions and constuints.... capplication> rosumme allocasion,
Nanlincar Programming Score, islaar nonlincar objective functions and constaimms.
Others
Score, ideas Goal Programming ...
Method
Dysimii Programming <appliicttion pesluction schodaling. allocation.
<application-:
engineering design, cocnomic management.
production plaming....
The structure of HMML is organized in three levels: modeling domains, subdomains, and method nodes.
Figure 2:
https::/www http::://wwwccotttttt.omo wwwtest.ctc.oooooooo..oo.... httpe//ww.......eeoe.../e//eeee.

---

Analysis
2 Task I
Solution Reporting
Task Execution Order
Task
Task
Problem
Mathematical Modaling
Preliminary Outline
Problam
S
Hierarchical Modsiang
Subuak
Isection (Abstracc)
Undersiaading
Keowlolge Retrivaal
Openiioos Rewerch
Doman
lscetion (R [Restatmmeett
Problam Decompositin
tep-X
Programenigg
Theocy
Sub-
+
(Assumptions;
domain
Modeling
sscciinn ( (uattffication
Actor
I incar
Quadratic Programming
Mathod
Progmmmming
lscctinn (Solution)
Task
Task
Task
Modeling Criti
X-1.6
5-1.3
Secciinn SecctionnConeeeeennne (Conchusion)
Dependency Amalyyii
Moceiin
3
Computational Solving
Scheeme
Code Clavention
code
Funning
<
Code
Covirvement
Store <
Store Retrieve 3
Tisk
Task
Analysis
Mammrr
Modeling Matiod
Took 2:
Dependency Graph
Experiment Result
Solution Report
Figure 3: Overview of the MM-Agent framework. The workflow consists of four sequential phases: Problem Analysis, Mathematical Modeling, Computational Solving, and Solution Reporting. In the Problem Analysis phase, MM-Agent decomposes the input problem into structured subtasks. In Mathematical Modeling, constructs formal mathematical representations for each subtask. During Computational Solving, MM-Agent applies appropriate computational methods to derive solutions. Finally, in Solution Reporting, synthesizes the results into a comprehensive report, clearly summarizing the solutions and associated insights.
level, the library consists of modeling domains T T= (1(1, T(2) T(m).. Each modeling do- main T()) is subdivided into multiple subdomains: T+) (T(1.)) T(.22) (e,,)), J. Within each subdomain (( specific method nodes A/(5..11) are represented as tuples: A((dd))) (modeling method, core idea, application). Here, the modeling method provides a high-level overview, the core idea explains the underlying principles, and the application describes the method's typical use cases (c.g., resource allocation, production scheduling). For example, in the domain of Operations Research (T()), the subdomain Programming Theory (T(1.1) includes the method node A(LL11),, which involves Linear Programming. The core idea is optimizing linear objectives with constraints, applied to problems like production resource scheduling. The HMML includes modeling domains such as Operations Research, Optimization, Machine Learning, Prediction, and Evaluation, with 17 subdomains and about 98 modeling methods, such as Linear Programming, Ant Colony Optimization, Expectation Maximization, Analytic Hierarchy Process, and Kolmogorov-Smingg Smiiroy Test, For further details, please refer to Section C in the Appendix.
3.3 MM-Agent
This section introduces MM-Agent, LLM-bbsed multi-agent system designed to automate mathe- matical modeling tasks. Its workflow consists of four key phases: Problem Analysis, Mathematical Modeling, Computational Solving. and Solution Reporting. MM-Agent begins by analyzing the given problem and breaking into subtasks. It then constructs formal mathematical models for each subtask, conducts experiments, and generates a solution. Finally, MM-Agent produces comprehensive report summarizing the solution and results. The overall framework is illustrated in Figure
3.3.1 Problem Analysis
This section presents the problem analysis phase of MM-Agent, which transforms complex world problems into mathematical modeling tasks. The process involves abstracting key elements (e.g., background, requirements) and analyzing relationships (e.g., variable dependencies) identify suit- able modeling methods. The problem analysis phase consists of three steps: problem understanding, problem decomposition, and lask dependency analysis.
Problem Undersanding. Given a mathematical modeling problem F. we consider an L.LM y xx; where TT(-) denotes language model parameterized by 0 which autoregressively
5

---

generates output tokens from an imput sequence x under the guidance of an instruction prompt x The prompt x, encodes resk-relevant context such as background information, problem requirements. and dataset descriptions. Conditioned on this input, the analyst agent performs a structured analysis to identify the problem type, core concepts, assumptions, objectives, and other essential factors. Specifically,, this process is represented as U, = To (F(F;,,), where x, represents the profile prompt used for problem undersanding, and Up is the analysis result. To deepen the understanding of the problem, the analyst agent adopts self-reflectinn to iteratively refine its analysis.
Problem Decomposition. After understanding the problem, the coordinator agent decomposes it into a set of subtasks to address its multiple objectives. This process is represented as D = To [F. Xa). where X represents the profile prompt used for problem decomposition, D (D1. D2. Dn denotes the set of subtasks, and each D; corresponds to an individual subtask. Each subtask is associated with a specific objective or component of the problem. As illustrated in Figure 7 in Appendix F..5 for the problem of predicting momentum in tennis matches, the agent decomposes the problem into four key subtasks: Momentum Quantification, Differentiation between Momentum and Randomness, Momentum Prediction, and Model Generalization.
Task Dependency Analysis. Since individual tasks are not independent, dependencies exist among them (e.g.. a model prediction task may rely on the analysis results from a data analysis task). To capture these dependencies and optimally address problem requirements. the task coordinator agent first conducts comprehensive analysis to identify interdependencies among tasks. This process is formulated as U To (D: where U {u, .... _,,.] represents the task-specific dependency analysis with u, denoting the detailed analysis of task D,. The instruction prompt directs the LLM to analyze dependencies among tasks. Subsequently, the task coordinator agent further leverages task analysis results to construct sequential subtasks {D1, D2,, Dn, with depenceny graph represented by G = (V,8) E), where V = {Di, D2, ,D.} represents the set of nodes (tasks), and & = (D:: D,) [P:. D3 E V} denotes the directed edges indicating task dependencies. The sequential subtasks are executed in order, with the outcomes of historical modeling processes stored in memory modules represented as H {(1,, 21). (D.,. 2n)}. where Q (M,, C, O,} denotes the intermediate outputs of subtask Dj. Specifically, Q, contains the mathematical modeling scheme M,, computational code C,, and experimental results O, The coordination agent leverages a dependency graph to manage relationships among subtasks and utilizes the memory module to facilitate information transfer and communication between tasks.
3.3.2 Mathematical Modeling
To efficiently automate solving mathematical modeling, we propose the Hierarchical Actor- Modeling Optimization. The specific mathematical modeling process for each subtask D; involves hierarchical method retrieval from the HMML, followed by actor-critic iterative optimization.
Hierarchical Modeling Knowledge Retrieval. Given subtask D; and a hierarchical modeling library T. Depth-First Search (DFS) traversal is initiated from the root node T(.), to compute the similarity between the subtask and modeling methods. The similarity measure is defined as ||=x|-||e>>] where en and ey represent embeddings of the subtask D and the math-
Sim(D,N)) en-evv
ematical modeling method node N., respectively. In practice, adopt the embedding model mGTE [67] to generate these embeddings. After traversing the entire hierarchical tree of methods, each method node's final score is updated by combining its similarity with the similarity of its parent node, computed as S(D,A) = w Sim(D,NV) scoring function, is hyperparameter, and N parent represents the parent (subdomain) of method node A. Finally, the top-K K method nodes with the highest scores, denoted as Nop K = (N(1), ... .N(x)). are selected and returned to the modeling agent.
-w)- (D, (D,,N D.,Naarrm)) ), where S denotes the
Actor-Crite Iterative Modeling Optimization. While retrieved mathematical modeling knowledge offers foundational methods and ideas, it often lacks the depth needed to address specific problem nuances (e.g., dealing with nonlinear constraints, optimizing multiple conflicting objectives, etc.). To overcome these limitations, we introduce actor-critic iterative optimization framework that progressively refines the modeling scheme, enabling it to effectively manage complex constraints and enhance overall solution quality.
Given a problem D. the task coordinator agent retrieves the relevant dependent resource R, from memory modules H based on the task dependency graph G. Using the retrieved resource R, and

---

the method set set Nop- obtained from the retrieval step, the actor modeling agent generates an initial modeling scheme: M = गH (Di,Noop-e Ra; Xa), where X, is the modeling prompt. Subsequently, the critic agent evaluates the quality of the current modeling scheme MM and provides targeted feedback: F(()) na (D,, M, Rexx),, where X. is the critic feedback prompt. Upon receiving feedback _(",, the actor modeling agent refines the scheme by integrating the critic's suggestions and corrections by M To (M F((; xx,), where x, is the mathematical modeling refine prompt.
This iterative procedure continues until the maximum number of iterations ny is reached.
3.3.3 Computational Solving and Solution Reporting
This section describes the computational solving and solution reporting phase of MM-Agent, which focuses on solving the mathematical model and generating a comprehensive solution report. The agent autonomously writes code to conduct computational experiments using the MLE-Solver [68]. which iteratively generates, tests, and refines code to ensure efficient and accurate execution. Upon
completion of the experiments, the agent formulates structured solution report, summarizing the modeling approach, experimental results, and key findings.
Code Generation and Execution. Given the mathematical modeling scheme M, the modeling programmer agent generates the corresponding code as follows: C, (y; where x, represents the instruction prompt used to direct the LLM to generate the computational code, and C, denotes the mathematical modeling code for task D, After code generation, the program is compiled to check for runtime errors. If it compiles successfully, the experimental results O2 are returned. If the code fails to compile, the agent attempts to repair it over n. iterations by analyzing the last error message and making the necessary corrections. Upon task completion, the task coordinator agent updates the agent's s memory: H < H J (Dr, 2.). In practice, for policy-related modeling problems, where the goal is to provide insights and recommendations based on existing knowledge models, the modeling agent directly offers these insights without generating code.
Preliminary Report Outline. After all tasks have been completed, reporting agent compiles comprehensive summary of the problem-solving process. The first step is to construct a structured outline for the mathematical modeling report. This outline establishes the framework of the report, organizing it into eight key sections: abstract, problem restatement, model assumptions, justification of assumptions, notation and definitions, problem analysis, solution, and conclusion. To ensure clarity and coherence, the outline integrates proper LaTeX formatting, facilitating seamless compilation and further refinements. By structuring the content systematically, it provides a solid foundation for an in-depth and well-organized final report.
Solution Report. Once the outline is established, the reporting agent employs specialized commands to progressively refine the report, drawing on the task coordinator agent's memory H. Prior to incorporating any revisions, the system compiles the LaTeX code to ensure that it functions correctly, preserving the integrity of the document. Through series of iterative edits, the agent guarantees that the report meets the necessary standards for quality, coherence, and academic rigor.
4 Experiments
4.1 Experimental Setup
Baselines. We evaluate MM-Agent against both human- solutions and SOTA LLM agents. As no prior work directly targets mathematical modeling problems, we repurpose existing autonomous research agents for comparison. The baselines include: (I) Human Team: Award-winning solutions (Honorable Mention or above) from real-world modeling competitions, serving as a strong human benchmark; (2) DS-Agent [33]: An LLM agent for automated data science, adapted with its core case-based reasoning framework for modeling tasks; (3) ResearchAgent [50]: : Originally designed to automate experimentation loops for machine learning tasks, adapted with its core framework for modeling problems; and (4) Agent Laboratory [46]: A scientific discovery framework that guides agents through literature review, experimentation, and report writing. We extend it to search arXiv for relevant modeling methods and assemble them into problem-solving pipelines.
7

---

Table Experimental results on the 2021-2024 and 2025 mathematical modeling competitions. AE, MR, PS, and RBA denote Analysis Evaluation, Modeling Rigorousness, Practicality and Scientificity and Result and Bias Analysis, respectively.
2021-2024
2025
Methods
AE
MR
PS
RBA
Overall
AE
MR
PS
RBA 1
Overall
Human
Human Team
9.04
6.20
8.79
7.62
7.91
9.25
7.42
8.92
6.50
8.02
GFT-4o
GPT-4o
7.62
3.86
8.48
5.17
6,28
7.67
3.67
8.90
5.75
6,50
DS-Agent
8.18
7.08
8.72
7.47
7.86
8.25
7.33
8.92
7.10
7.90
Research
7.97
6.80
8.82
7.37
7.74
8.00
7.30
8.60
7.00
7.73
Agent Laboratory
8.56
6.35
8.63
5.56
7.28
8.75
5.58
8.58
5.33
7.13
MM-Agent
9.15
7.28
9.00
8.44
8.85
8,86
7.21
9.00
8.43
8,38
DeepSeek-Ni
1-67 IB
DeepSeek-RI
7.23
4.79
8.69
4.50
6.30
7.42
4,25
8,50
5.25
6.35
DS-Agent
8,25
6.88
8.74
7.19
7.77
7.92
6.33
9.00
7.60
7.71
ResearchAgent
8.13
7.04
8.77
6.92
7.72
8.00
6.75
8.83
7.58
7.79
Agent Laborutory
8.65
5.96
8.70
5.91
7.31
8.83
5.50
8.83
5.58
7.19
MM--gget
9.54
8.25
9.06
8.54
8.85
9,50
8.33
9.25
8.58
8.92
Experimental Implementation. We select subset mathematical modeling problems from the past five years (2021-2025) our test set, ensuring diversity across problem types and domains to support representative evaluation. This subset consists of 32 problems in total. To mitigate potential data leakage from LLM pretraining, 2025. The LLM agents used in this evaluation include GPT-4o and Deepseek-F R1 as base models. For the evaluation, a unified 1-to-10 10 scale. The selected human experts have previously earned at least an Honorable Mention in mathematical modeling competitions. Additional experimental details provided in Appendix D To evaluate annotation quality, we measure inter-annotator agreement, including both human-human and model-human agreements, detailed in Appendix E.4
evaluate problems from 2021-2024 separately from those in
adopt both GPT-4o- based automatic scoring and human expert review, using
4.2 Experimental Results
Main Experiments. Table Thbles that MM-Agent achieves statc-of-the-art (SOTA) performance across all evaluation dimensions. (1) Directly applying foundational models (GPT-40 or DeepSeek- R1-67IB) without agent-level orchestration results in significantly weaker performance, particularly in MR and RBA. This gap underscores the inadequacy of LLMs in handling the open-ended, struc- tured reasoning required for real-world modeling tasks and highlights the necessity of structured agent-based workflows. (2) MM-Agent consistently outperforms all baseline agents, achieving the highest overall scores under both GPT-4o and DeepSeek- RI 1-671B backbones. (3) Agents built on DeepSeek-R 167 IB surpass their GPT-4o counterparts, with MM-Agent demonstrating marked gains in Modeling Rigorousness and Result and Bias Analysis, suggesting stronger reasoning capabilities in the larger model. (4) Human teams remain strong competitors, outperforiigg all I.IM M-based agents except MM-Agent on most metrics, underscoring both the complexity of the task and MM-Agent's near-human modeling proficiency. (5) The 2025 results closely mirror those from 2021-2022 indi- cating strong temporal consistency. This robustness mitigates concerns about potential data leakage (e.g., memorized solutions) and further supports the conclusion that MM-Agent performs genuine modeling rather than overfitting. In addition benchmark results, we developed publicly available modeling copilot system based on MM-Agent, aligned with official MCM/ICM protocols and LLM usage guidelines. This system assisted two undergraduate teams securing the Finalist Award (top 2.0% among 27,456 teams) in the 2025 MCM/ICM competition. This real-world validation illustrates MM-Agent's 's practical effectiveness as modeling copilot, capable of supporting human users in high-stakes, open-ended scientific tasks.
4.3 Ablation Study and Further Analysis
To better understand the design and practical utility of MM-Agent, we present a three-part analysis. First, we conduct an ablation study to quantify the impact of each core module on modeling perfor-

---

mance. Second, we evaluate token usage, cost, and runtime to assess deployment efficiency. Finally, we test MM-Ageet on well-defined, formulated mathematical problems to examine its generalization beyond open-ended modeling.
Contribution of Key Components. We perform an ablation study to as- sess the impact of three core mod- ules in MM-Agent: task dependency analysis (DA), the Hierarchical Math- ematical Modeling Library (HMML), and hierarchical actor-critic model- ing (HACM). Specifically, we (1) re- place DA with a naive task parser (w/o DA), (2) substitute HMMI. with flat retrieval library lacking hierarchical structure (w/o HMML), and (3) re-
10
vi 15GCM
10
Wi
DA
MAAM
Score
9
Score
8
7-
6
MR
PS
RBA
AE
MR
PS
RBA
(a) GPT-4o
(b) DeepSeek-R1-671B
Figure 4: Ablation study of the effect of the problem analysis and mathematical modeling.
These variants allow us to evaluate each module's contribution to structured problem understanding and modeling performance. As shown in Figure 4 MM-Agent consistently outperforms all ablated variants across GPT-4o and PrepSeek-R1-6711
move HACM to disable iterative self-refinement (w/o HACM).
backbones under four evaluation metrics. Removing DA significantly reduces MR, indicating that deep task comprehension is essential for rigorous formulation. The absence of HACM leads to sharp declines in AE and PS, highlighting its critical role in constructing coher- ent, scientifically sound models. Notably, removing HMML causes clear drops in AE and RBA, underscoring the importance of structured retrieval in aligning modeling strategies with both problem context and solution needs. Unlike flat libraries that treat all methods equally, HMML encodes 98 high-level modeling schemas organized hierarchically by problem type, abstraction level, and solution paradigm. This enables problem-aware and solution-aware retrieval that better supports abstraction, constraint reasoning, and method selection, key capabilities for effective modeling.
Cost Efficiency Analysis. We assess the cost ef- ficiency of MM-Agent in solving real-world math- ematical modeling problems, focusing on token usage, monetary cost, and runtime. All evalua- tions are conducted via official APIS provided by model vendors. As shown in Table MM-Agent matches the performance of DS-Agent and Re- searchAgent with comparable computational cost and runtime. Compared to Agent Laboratory, it achieves higher performance while substantially reducing both cost and execution time, highlight- ing its scalability and practical viability. Further results on additional models and a detailed case study are included in Appendix
Table 2: Experimental results on average token consumption, cost, and runtime.
Methods
Token
Cost(S)
Runtime(s)
GPT-40
DS-Agent
198,186
0.77
1.044
Research/gent
170,732
0.67
459
Agent Laboratory
746,159
2.14
1,015
MM-Agent
240,877
0.88
906
DeepSeek-RI
16-1B
DS-Agent
341,432
0.46
7,035
ResearchAgent
222,030
0.28
4,816
Agent Laboratory
974,423
0.89
11,331
MM-Agent
530,363
0.56
7,529
Experiments on Well-defined Mathematical Optimization Problems. To complement MM- Bench's open-ended focus, we evaluate MM-Agent on well-defined mathematical optimization tasks, including both linear and nonlinear programming. In this setting, the agent receives complete problem specifications (e.g., variables, objective function, and constraints and directly outputs the numerical solution. Since these tasks have known ground truth answers, accuracy serves as a direct performance metric. We conduct experiments on the OPTIBENCH dataset (4. with detailed results provided in Table (Appendix E.2). MM-Agent consistently outperforms GPT-4o across all subtasks in sero-shottae setting, demonstrating robust generalization formulated optimization problems.
Conclusion
In this work, we introduce mathematical modeling. By assessing agents across diverse domains and problems, we expose key challenges in bridging real-world phenomena with mathematical formulations. Our findings reveal that existing LLM agents often overlook essential modeling principles, such as abstraction, constraints, and assumptions, resulting in oversimplified and scientifically invalid outputs. To address these issues, we propose MM-Agent, an autonomous pipeline that systematically handles problem analysis, model
a benchmark for evaluating LLM- agents in real- world

---

formulation, solution development, and result interpretation. Comprehensive experiments show that MM-Agent significantly outperforms existing LLM agents, though challenges remain higher-order reasoning and interdisciplinary problem-solving. We hope our benchmark and framework lay foundation for future progress in LL-M-driven mathematical modeling.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (Grant No. 62572417, No.92370204), National Key R&D Program of China (Grant No. 20.22223YFF0225004) the Guangzhou Basic and Applied Basic Research Program under Grant No. 2024A04J3279, Education Bureau of Guangzhou Municipality, and CCF-DiDi GAIA Collaborative Research Funds.
References
[1] Edward A Bender. An introduction to mathematical modeling. Courier Corporation, 2000.
[2] Mark Meerschaert. Mathematical modeling. Academic press, 2013.
[3] Trieu H. Trinh, Yuhuai Wu, Quoe V. Le, He He, and Thang Luong. Solving olympiad geometry without human demonstrations. Nat., 625(7995)447--42,, 2024.
[4] Zhicheng Yang, Yiwei Wang. Yinya Huang. Zhijiang Guo, Wei Shi, Xiongwei Han, Liang Feng, Linqi Song, Xiaodan Liang, and Jing Tang. Optibench meets resocratic: Mcasure and improve LLMs for optimization modeling. In The Thirteenth International Conference Learning Representations, 2025,
[5] Giulio Starace, Oliver Jaffe, Dame Sherburn, James Aung, Jun Shern Chan, Leon Maksin, Rachel Dias, Evan Mays, Benjamin Kinsella, Wyatt Thompson, et al. Paperbench: Evaluating ai's ability to replicate ai research. arXiv preprint arXiv:2504.01 2025.
[6] OpenAT. Gpt-4o system card, 2024.
[7] AL. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li. Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang. Bochao Wu, Bei Feng. Chengda Lu. Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai. Deli Chen, Dongjie Erhang Li. Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao. Guanting Chen. Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L.. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang. Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang. Minghui Tang. Meng Miaojun Wang. Mingming Li. Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang. Qinyu Chen. Qiushi Du, Ruiqi Ge, Ruisong Zhang. Ruizhe Pan, Runji Wang. R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Yc, Shiyu Wang. Shuiping Yu, Shunfeng Zhou, Shuting Pan, and S. S. Li. Deepscck-rl: Incentivizing rcasoning capability in iims reinforcement learning. CoRR, abs/2501. 12948, 2025.
[8] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In ICLR, 2023.
[9] Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian, Yujia Qin, Xin Cong, Ruobing Xie, Zhiyuan Liu, Maosong Sun, and Jie Zhou. Agentverse: Facilitating multi-agent collahoration and exploring emergent hehaviors. In ICLRR 2024.
[10] Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem, CAMEL: communica- tive agents for "mind" exploration of large language model society. In NeurlPS, 2023.
[11] Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. Chatdev: Communicative agents for software development. In ACL (1), pages 15174-15186. 2024.
[12] Qingyun Wu, Gagan Bansal, Jicyu Zhang, Yiran Wu, Shaokun Zhang, Frkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. Autogen: Finaliing next-gen LLM applications via multi-agent conversation framework. CoRR, abs/2308.08 155, 2023.
[13] Carlos E. Jimenez, John Yang. Alexander Wettig, Shunyu Yao. Kexin Pei, Ofir Press, and Karthik R. Narasimhan. Swe-bench: Can language models resolve real-world github issues? In ICLR, 2024.
10

---

[14] Xingyao Wang, Boxuan Li. Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge. Jiayi Pan. Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fugiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang. Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng, Heng Ji. and Graham Neubig. Opendevin: An open platform for Al software developers as generalist agents. CoRR, abs/2407. 16741,, 2024.
[15] John Yang. Carlos E. Jimenez, Alexander Wettig. Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. Swe-agent: Agent-computer interfaces enable automated software engineering. In NeurlPS, 2024.
[16] Junwei Liu, Kaixin Wang, Yixuan Chen, Xin Peng, Zhenpeng Chen, Lingming Zhang, and Yiling Lou. Large language model-based agents for software engineering: A survey. CORR, abs/2409.00977, 2024.
[17] Altera. AI, Andrew Ahn, Nie Becker, Stephanie Carroll, Nico Christie, Manuel Cortes, Arda Demirci, Melissa Du, Frankie Li, Shuying Luo, Peter Y. Wang. Mathew Willows, Feitong Yang. and Guangyu Robert Yang. Project sid: Many-agent simulations toward AI civilization. CoRR, abs/2411 0tt//2111.00111 14, 2024.
[18] Xidong Feng, Yicheng Luo, Ziyan Wang, Hongrui Tang, Mengyue Yang. Kun Shao, David Mguni, Yali Du, and Jun Wang. Chessgpt: Bridging policy learning and language modeling. In NeurlPS, 2023.
[19] Guanzhi Wang. Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke 7hu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended emhodied agent with large language models. Trans, Mach. Learn. Res., 2024, 2024.
[20] Yubo Dong, Xukun Zhu, Zhengzhe Pan, Linchao Zhu, and Yi Yang. Villageragent: A graph-based multi-agent framework for coordinating complex task dependencies in minecraft. In ACL (Findings), pages 16290- 16314, 2024.
[21] Zelai Xu, Chao Yu, Fei Fang. Yu Wang. and Yi Wu. Language agents with reinforvement learning for strategic play in the werewolf game. In ICML, 2024.
[22] Sihao Hu, Tiansheng Huang, Fatih Ilhan, Selim F. Tekin, Gaowen Liu, Ramana Kompella, and Ling Liu. survey on large language model-based game agents. CoRR, abs/2404.022 2024.
[23] Joon Sung Park, Joseph C. O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Perey Liang, and Michael S. Bernstein. Generative agents: Interactive simulacra of human behavior. In UIST, pages 2:1-2:22, 2023.
[24] Joon Sung Park, Carolyn Q. Zou, Aaron Shaw, Benjamin Mako Hill, Carrie J. Cai, Meredith Ringel Morris, Robb Willer, Percy Liang, and Michael S. Bernstcin. Generative agent simulations of 1,000 people. CoRR, abs/2411.10109, 2024.
[25] Talor Ahramovich, Meet Udeshi, Minghao Shao, Kilian Lieret, Haoran Xi, Kimberly Milner, Sofja Jancheska, John Yang, Carlos F. Jimenez, Farshad Khorrami, Prashanth Krishnamurthy, Brendan Dolan- Gavitt, Muhammad Shafique, Karthik Narasimhan, Ramesh Karri, and Ofir Press. Enigma: Enhanced interactive generative model agent for CTF challenges. CoRR, abs/2409. 2024.
[26] Richard Fang, Rohan Bindu, Akul Gupta, Qiusi Zhan, and Daniel Kang. LLM agents can autonomously back websites. CoRR, abx/2402.06664, 2024.
[27] Shengye Wan, Cyrus Nikolaidis, Daniel Song, David Molnar, James Crnkovich, Jayson Grace, Manish Bhatt, Sahana Chennabasappa. Spencer Whitman, Stephanie Ding. Vlad Ionescu, Yue Li, and Joshua Saxe. CYBERSECEVAL 3: Advancing the evaluation of cybersecurity risks and capabilities large language models. CoRR, abs/2408.01660 2024.
[28] Brian Ichter, Authony Brohan, Yevgen Chebotar, Chelsea Finn, Karol Hausman, Alexander Herzog, Daniel Ho, Julian Ibarz, Alex Irpan, Eric Jang, Ryan Julian, Dmitry Kalashnikov, Sergey Levine, Yao Lu, Carolina Parada, Kanishka Rao. Pierre Sermanet, Alexander Toshev. Vincent Vanhoucke, Fei Xia. Ted Xiao, Peng Xu, Mengyuan Yan, Noah Brown, Michael Ahn, Omar Cortes, Nicolas Sievers, Clayton Tan, Sichun Xu, Diego Reyes, Jarek Rettinghouse, Jornell Quiambao, Peter Pastor, Linda Luu, Kuang-Huei Lee, Yuheng Kuang. Sally Jesmonth, Nikhil J. Joshi, Kyle Jeffrre, Rosario Jauregui Ruano, Jasmine Hsu, Keerthana Gopalakrishnan, Byron David, Andy Zeng, and Chuyuan Kelly Fu. Do as I can, not as I say: Grounding language in robotic affordances. In CoRL, volume 205, pages 287-318, 2022.
[29]
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian lehter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li- Billell Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling. Haohuan Wang, and Ury Zhilinsky. To: A vision- lasion-language language-action flow model for general robot control. CoRR, 2et//2211.44444 2024.

---

[30] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alexander Herzog, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan. Tomas Jackson, Sally Jesmonth, Nikhil J. Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Isabel Leal, Kuang- Queig-Huei Lec, Sergey Levine, Yao Lu, Utsav Malla, Decksha Manjumath, Igor Mordatch, Ofir Nachum, Carolina Parada, Jodilyn Peralta, Emily Perez, Karl Pertsch, Jornell Quiambao, Kanishka Rao, Michael S. Ryoo, Grecia Salazar, Pannag R. Sanketi, Kevin Sayed, Jaspiar Singh, Sumedh Sontakke, Austin Stone, Clayton Tan, Huong T. Tran, Vincent Vanhoucke, Steve Vega, Quan Vuong. Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Tianhe Yu, and Brianna Zitkovich, RT-1: robotics transformer for real-world control at scale. In Robotics: Science and Systems, 2023.
[31]
Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, Quan Vuong, Vincent Vanhoucke, Huong T. Tran, Radu Soricut, Anikait Singh, Jaspiar Singh, Pierre Sermanet, Pannag R. Sanketi, Grecia Salazar, Michael S. Ryoo, Krista Reymann, Kanishka Rao, Karl Pertsch, Igor Mordatch, Henryk Michalewski, Yao Lu, Sergey Levine, Lisa Loe, Tsang- Wei Edward Lee, Isabel Leal, Yuheng Kuang, Dmitry Kalashnikov, Ryan Julian, Nikhil J. Joshi, Alex Irpan, Brian Ichter, Jasmine Hsu, Alexander Herzog, Karol Hausman, Keerthana Gopalakrishnan, Chuyuan Fu, Pete Florence, Chelsea Finn, Kumar Avinava Dubey, Danny Driess, Tianli Ding, Krzysztof Marcin Choro- manski, Xi Chen, Yevgen Chebotar, Justice Carbajal, Noah Brown, Anthony Brohan, Montserrat Gonzalez Arenas, and Kehang Han. RT-2: Dision-languag In CoRL, volume 229, pages 2165 2183, 2023.
models transfer web knowledge to robotic control.
[32] Ji Woong Kim, Tony Z. Zhao, Samuel Schmidgall, Anton Deguet, Marin Kobilarov, Chelsea Finn, and Axel Krieger. Surgical robot transformer (SRT): imitation learning for surgical tasks. In CoRL, volume 270, pages 130-144, 2024.
[33] Siyuan Guo, Cheng Deng, Ying Wen, Hechang Chen, Yi Chang, and Jun Wang. Ds-agent: Automated data science by empowering large language models with case-based reasoning. In ICMI, 2024.
[34] Sirui Hong. Yizhang Lin. Bang Liu, Bangbang Liu, Binhao Wu, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, Li Zhang, Lingyao Zhang, Min Yang. Mingchen Zhuge, Taicheng Guo, Tuo Zhou, Wei Tao, Wenyi Wang. Xiangru Tang, Xiangtao Lu. Xiawu Zheng, Xinbing Liang, Yaying Fei, Yuheng Cheng. Zongze Xu, and Chenglin Wu. Data interpreter: An LLM agent for data science. CoRR, abb//2402.18679, 2024.
[35] Ruisheng Cao, Fangyu Lei, Haoyuan Wu, Jixuan Chen, Yeqiao Fu, Hongcheng Gao. Xinzhuang Xiong, Hanchong Zhang, Wenjing Hu, Yuchen Mao, Tianbao Xie, Hongshen Xu, Danyang Zhang, Sida I. Wang, Ruoxi Sun. Pengcheng Yin, Caiming Xiong, Ansong Ni, Qian Liu, Victor Zhong, Lu Chen, Kai Yu, and Tao Yu. Spider2-v: v: How far are multimodal agents from automating data science and engineering workflows?
In NewrlPS, 2024.
[36] Md. Mahadi Hassan, R. Alexander Knipper, and Shubhra Kanti Karmaker Santu. Chatgpt as your personal data scientist. CoRR, abs/2305.13357. 2023.
[37] Daniel McDuff, Mike Schackermann, Tao Tu, Anil Palepu, Amy Wang. Jake Garrison, Karan Singhal, Yash Sharma, Shekoofeh Azizi, Kavita Kulkamni, Le Hou, Yong Cheng, Yun Liu, S. Sara Mahdavi, Sushant Prakash, Anupam Pathak, Christopher Semturs, Shwetak N. Patel, Dale R. Webster, Ewa Dominowska, Juraj Gottweis, Joelle K. Barral, Katherine Chou, Gregory S. Corrado, Yossi Matias, Jake Sunshine, Alan Karthikesalingam, and Vivek Natarajan. Towards accurate differential diagnosis with large language models. CoRR, abs/2312.001664 2023.
[38] Samuel Schmidgall, Rojin Ziaei, Carl Harris, Eduardo Pontes Reis, Jeffrey Jopling, and Michael Moor. Agentelinic: a multimodal agent benchmark to evaluate Al in simulated clinical environments. CoRR, abs/2405.07960, 2024.
[39] Tao Tu, Anil Palepu, Mike Schackermann, Khaled Saab, Jan Freyherg, Ryutaro Tanno, Amy Wang, Brenna Li, Mohamed Amin, Nenad Tomasev, Shekoofeh Azizi, Karan Singhal, Yong Cheng, Hou, Albert Webson, Kavita Kulkarni, S. Sara Mahdavi, Christopher Semturs, Juraj Gottweis, Joelle K. Barral, Katherine Chou, Gregory S. Corrado, Yossi Matias. Alan Karthikesalingam, and Vivek Natarajan. Towards conversational diagnostic AL. CoRR, 0as/24001.0654, 2024.
[40] Jianing Qiu, Kyle Lam, Guohao Li, Amish Acharya, Tien Yin Wong, Ara Darzi, Wu Yuan, and Erie Topol. Llm-based agentic systems medicine and healthcare. Nature Machine Intelligence, 6(12): 141)::1418-1200. 2024.
[41] Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samual Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards generalist agent for the web. In NeurlPS, 2023.
12

---

[42] Izzeddin Gur, Hiroki Furuta, Austin V. Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, and Aleksandra Faust. A real-world webagent with planning, long context understanding, and program synthesis. ICLR, 2024.
[43] Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, and Dong Yu. Webvoyager: Building an end-to-end web agent with large multimodal models. In ACL (1), pages 6864 6890, 2024.
[44] Pranav Putta, Edmund Mills, Naman Garg, Sumeet Motwani, Chelsea Finn, Divyansh Garg, and Rafael Rafailov. Agent Q: advanced reasoning and learning for autonomous Al agents. CoRR, abs/2408.07199 2024.
[45] Yutaro Yamada, Robert Tjarko Lange, Cong Lu, Shengran Hu, Chris Lu, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist-v2: Workshop-lever automated scientific discovery via agentic tree search. arXiv preprint 2rXiv:2504008066 2025.
[46] Samuel Schmidgall, Yusheng Su. Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Zicheng Liu, and Emad Barsoum. Agent laboratory: Using LLM agents research assistants. CoRR, 0e5/22501.42227 2025.
[47] Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob N. Foerster, Jeff Clune, and David Ha. The Al scientist: Towards fully automated open-ended scientific discovery. CoRR, abs/2408.06299 2024.
[48] Jinheon Back, Sujay Kumar Jauhar, Silviu Cucerzan, and Sung Ju Hwang. Rescarchagent: Iterative research idea generation over scientific literature with large language models. CoRR, abs/2404.07738, 2024.
[49] Deepak Nathani, Lovish Madaan, Nicholas Roberts, Nikolay Bashlykov, Ajay Menon, Vincent Moens, Amar Budhiraja, Despoina Magka, Vladislav Vorotilov, Gaurav Chaurasia, Dieuwke Hupkes, Ricardo Sil- veira Cabral, Tatiana Shavrina, Jakob N. Foerster. Yoram Bachrach, William Yang Wang, and Roberta Raileanu. Mlgym: A new framework and benchmark for advancing Al research agents. CoRR. abs/2502. 14499, 2025.
[50] Qian Huang, Jian Vora, Percy Liang, and Jure Leskovec. Benchmarking large language models AI research agents. CoRR, 10.0333, 2023.
[51] Shanghua Gao, Ada Fang, Yepeng Huang. Valentina Giunchiglia, Ayush Noori, Jonathan Richard Schwarz, Yasha Ektefaie, Jovana Kondic, and Marinka Zitnik. Empowering biomedical discovery with ai agents. Cell, 187(22):6125-61) 6151, 2024.
[52] Kourosh Darvish, Marta Skreta, Yuchi Zhao, Naruki Yoshikawa, Sagnik Som, Miroslav Bogdanovic, Yang Cao, Han Hao, Haoping Xu, Alán Aspuru-Guzik, al. Organa: robotic assistant for automated chemistry experimentation and characterization. Matter, 8(2), 2025.
[53] Xusen Guo, Xinxi Yang, Mingxing Peng. Hongliang Lu, Meixin Zhu, and Hai Yang. Automating traffic model enhancement with Al research agent. CoRR, abs/2409. 16876, 2024.
[54] Dan Hendrycks, Collin Burs,, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In NeurIPS Datasets and Benchmarks, 2021.
[55] Karl Cohhe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. CoRR, 1bs/2110.14168 2021.
[56] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoe V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In NeurlPS, 2022.
[57] Kunhao Zheng, Jesse Michael Han, and Stanislas Polu, minif2f: cross-system benchmark for formal olympiad-level mathematics. In ICLR, 2022.
[58] Jesse Michael Han, Jason Rute, Yuhuai Wu, Edward W. Ayers, and Stanislas Polu. Proof artifact co co-training for theorem proving with language models. In ICLR, 2022.
[59] Kaiyu Yang and Jia Deng. Learning to prove theorems via interacting with proof assistants. In ICML, volume 97 of Proceedings of Machine Learning Research, pages
2019.
13

---

[60] Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. Trans. Mach. Learn. Res., 2023, 2023.
[61] Karl Cobbe. Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reichhro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110. 2021.
[62] Ziyang Xiao, Dongxiang Zhang, Yangjun Wu, Lilin Xu, Yuan Jessica Wang, Xiongwei Han, Xiaojin Fu, Tao Zhong, Jia Zeng, Mingli Song, and Gang Chen. -hainttf-eepeers:: When Ilms meet complex operations research problems. In The Tweifth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net net, 2024.
[63] Ali Teshiiii Wenzhi Gao, and Madelcine Udell. Optimus: Scalable optimization modeling with (MI)LP solvers and large language models. In Forty-first International Conference Machine Learning. ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024.
[64] Rindranirina Ramamonjison, Timothy Yu, Raymond Li, Haley Li, Giuseppe Carenini, Bissan Ghaddar, Shiqi He, Mahdi Mostajabdaveh, Amin Sanitalchi-Dehkorre Zirui Zhou, al. NI4opt competition: Formulating optimization problems based on their natural language descriptions. In NeurIPS 2022 Competition Track, pages 189-203. PMLR, 2023.
[65] COMAP. Highlights from comap's ' 2024 mem icm, 2024.
[66] COMAP. Mem/iem contest rules, registration and instructions, 2025.
[67] Xin Zhang. Yanzhao Zhang. Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang. Pengjun Xie, Huang, al. mgte: Generalized long-context text representation and reranking models for multilingual text retrieval. In Proceedings of the 2024 Conference on Empirical Methods Natural Language Processing: Industry Track, pages 1393-1412, 2024.
[68] Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung. Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, Aleksander Madry, and Lilian Weng. MLE-bench: Evaluating machine learning agents on machine learning engineering. In ICLR, 2025.
14

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: yes
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: yes
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification: no
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: yes
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification: yes
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ cpipp/.ccppullc//uuuii wwwiicccguie//eeeeeeeeeii CoodeSubeiiie ssi wsionnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: yes
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: yes
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
17

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: yes
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httip:://neurippee http///urrs.cccupup////e httpp//ssccppp..t.ttttttttteeeea
Answer: [Yes]
Justification: yes
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: yes
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
18

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
Justification: no
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: CC BY-NC
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppswpptseeee whworee.wthhhe....c.. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
19

---

Answer: [Yes]
Justification: we have provided the dataset and open-sourced the code.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification: yes
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: yes
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes]
20

---

We have developed mathematical modeling agent based on a large language model.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non standad components.
Please refer to our LLM policy hattps://neurips 220p://ee////oofeeeeee/2222///// for what should or should be described.
21

---

MM-Agent: LLM as Agents for Real-world
Mathematical Modeling Problem
Supplementary Material
Contents
Introduction
2
Related Works
3
Building LLM Agent for Real- World Mathematical Modeling Problems
MM-Bench
3.2
Hierarchical Mathematical Modeling Library Construction
3.3
MM-Agent
3.3.1
Problem Analysis
3.3.2
Mathematical Modeling
3.3.3
Computational Solving and Solution Reporting
Experiments
4.1
Experimental Setup
4.2
Experimental Results
4.3
Ablation Study and Further Analysis
5
Conclusion
Broader impacts
24
Statistics of MM-Bench
24
Hierarchical Mathematical Modeling Library Construction
25
Experiments Setup
25
Other Experiments
26
Experiments on Other Models
26
Experiments on Well-defined Math Optimization Problems
27
Human Evaluation Results
27
22

---

E,4
Annotation Quality
28
E,5
Case Study
28
Prompts used for MM-Bench and MM-Agent
29
23

---

A Broader impacts
Mathematical modeling serves as a cornerstone methodology for formulating, analyzing, and solving complex real-world problems, underpinning scientific discovery and technological advancement across applied mathematics, natural sciences, engineering. and the social sciences. By automating this process, our work LLM-powered mathematical modeling agents (MM-Agents) has the potential to substantially broaden access to high-quality modeling expertise, accelerate research in data-scarce expert-limited domains, and support decision-making in high-stakes environments such as epidemiology, sustainability, and infrastructure planning.
MM-Agents lower the barrier to mathematical modeling for diverse real-world applications. MM- Agents can democratize the modeling process by enabling non-experts, such students, practitioners, or policymakers, to explore complex systems through structured analytical reasoning. This may significantly enhance STEM education, interdisciplinary collaboration, and rapid response in time-sensitive domains like disaster management or urban systems. Moreover, by encoding expert- level workflows and structured domain knowledge, MM-Agents provide a foundation for scalable, reusable modeling across diverse fields, potentially catalyzing scientific discovery in areas with limited modeling resources.
Contamination & Plagiarism. Because our dataset includes publicly available mathematical modeling competitions, there is a possibility that LLMs may have previously encountered solution reports (e.g.. from websites like Arxiv). This introduces potential risk of contamination, meaning models might memorize solutions or gain insights that artificially boost their performance on MM- bench beyond actual capabilities. To mitigate this risk, we selected competition problems from the most recent year (2025), ensuring the evaluated LLMS had not been trained on these specific solutions. Our experiments (Section(4.2) detected systematic contamination effects in GPT-4o or Deepseek-RI. Furthermore, the distribution of results from the 2025 competition aligns consistently with those from previous years. Nevertheless, we cannot guarantee that future models will remain unaffected. To proactively manage potential contamination risks, we recommend regularly updating the MM-bench with new mathematical modeling problems.
Judge Bias and Accessibility. Although we evaluate annotation quality using inter-annotator agreement metrics, bias still exists in both human and LLM annotators, particularly due to the inherent subjectivity in evaluating different modeling solutions. future work, we aim to develop mathematical modeling judge, which could provide more structured, transparent, and consistent evaluations, thereby mitigating annotator bias. Additionally, running LLM agents on MM-Bench is computationally intensive. In our experiments, GPT-4o and DeepSeek-RI consumed approximately 0.53 million and 0.24 million tokens, respectively.
Misuse. The MM-agent offers substantial opportumities to streamline real-world mathematical model- ing tasks across diverse areas such as engineering optimization, environmental resource management, and epidemic forecasting. By automating complex and labor-intensive processes, this technology enables researchers to focus more effectively on conceptual innovation and experimental design. Nevertheless, the agent's 's robust automation capabilities also introduce significant ethical concerns. Reduced barriers to entry may inadvertently promote the generation of low-quality or misleading scientific outputs. Furthermore, entirely Al-generated reports could potentially be misused in mathe- matical modeling competitions. To address these ethical challenges and safeguard academic integrity, it is imperative to transparently disclose any Al assistance involved in competition submissions.
B Statistics of MM-Bench
MM-Bench consists of 10 domains, 8 task types (e.g., decision, prediction, evaluation et al.), and a total of 111 problem samples, all soureed from undergraduate- (MCM and ICM). Each sample in MM-Bench is based on a mathematical modeling competition problem and includes background information that describes the context of the problem, the problem requirements outlining the tasks to be completed, the dataset puth indicating the location of the dataset, dataset description providing details about the dataset, and variable description explaining the attributes within the dataset. For policy- or decision-focusee tasks, datasets may not be
Mathematical Modeling Contests
24

---

provided. as these problems often emphasize qualitative reasoning or scenario-bar based analysis. The statistical information can be seen Figure
Geosclence
Sports
Medicine
Economic
& Managomen
Simulation
SW
18%
Statishic
Graph 25g
Optimization
Evaluation
25
Physics
8%
Biology
Transportation
Environmental
Dectsion
Scienco
Engineering
Social Sciencs
Prodiction
S Syrstmm
Modeling
14%
20%
Data Domain
(b) Problem Type
Figure 5: Illustrations of problem domain and types.
C Hierarchical Mathematical Modeling Library Construction
To enhance LLM agents' mathematical modeling capabilities, we introduce the Hierarchical Mathematical Modeling Library (HMML), three-level structured hierarchy designed for effi- cient, targeted method retrieval. Unlike conventional flat libraries, HMML explicitly captures method heterogeneity by categorizing them into distinct modeling domains (top layer), ciated subdomains (middle layer), and specific method nodes (bottom layer). This structured design streamlines retrieval through progressively refined searches guided by high-level reason- ing schemas tailored specifically to mathematical modeling tasks. Specifically, HMML adopts structure comprising three abstraction layers, illustrated in Figure The top layer represents distinct mathematical modeling domains, the second layer corresponds to their re- spective subdomains, and the third layer includes specific method nodes. Formally, the hier- archical structure of HMML is represented as follows: at the highest level, the mathematical Each modeling domain sub- tree (t) is further subdivided into multiple subdomains: (()) (T(1))) (+.22) (1.6k) J. Within each subdomain T(+)), specific method nodes ((3..)) are structured explicitly as tuples: N(t.) = {modeling method, core idea, application). Here, modeling method provides a high-level introduction to the mathematical modeling approach, ples underpinning the modeling method, and application indicates typical scenarios and delineates their application scope, such resource allocation optimization and production scheduling. For example, in the domain of operations research (T)) Operations Research), the subdomain of programming theory (T.1) Programming Theory) includes the specific method node A111,.1), which involves the modeling method of linear programming, with the core idea of optimization using linear objectives and constraints, and its application in production resource scheduling. The final mathematical modeling library features five domains (e.g., Operations Research, Optimization, Machine Learning, Prediction and Evaluation), with 17 subdomains (e.g.. Programming Theory, Graph Theory, Clustering, Statistics, etc.). encompassing approximately 98 modeling methods (e.g.. Linear Programming, Ant Colony Optimization, Expectation Maximization, Analytic Hicrarchy Process, Kolmogorov-Smingt Test).
modeling domains are denoted as T = (T()), (2), T(m)).
idea describes the fundamental princi-
D Experiments Setup
25

---

Evalaution. Since real-wordd world mathematical modeling problems are open open-ended and lack standard answers, we follow official Mathematical Modeling evaluation standards to assess the final solution report. We evaluate the agent's mathematical modeling capabilities across four key aspects: (1) Analysis Evaluation. Assesses the clarity of problem definition, the identification of key components, and the coherence of sub-tasks in relation to the overall goal. (2) Modeling Rigorousness. Focuses on rigor and rationality, evaluating whether the assumptions are clearly stated and justified, and whether the chosen methods, metrics, and model structure accurately and scientifically represent the real- werl-world problem. (3) Practicality and Scientificity. Evaluates the practicality and scientific validity of the model, ensuring that it is realistically applicable, provides valuable insights for decision-makinn and adheres to scientific principles. This stage also verifies whether the model is theoretically sound and considers all relevant scientific factors to ensure its validity. (4) Result and Bias Analysis. Assesses the clarity, interpretability, and thoroughness of the results and analysis. Additionally, it evaluates how well potential biases, such data model bias, are identified, analyzed, and mitigated to enhance robustness and acknowledge model limitations.
Baselines. We compare MM-Agent with both human team competition solutions and existing LLM- based agents. Since there is no prior work specifically addressing mathematical modeling problems, we adopt other LLMs agents designed for autonomous research to tackle these problems. Specifically, our baselines include: (1) Human Team, using original solutions from real-world mathematical modeling competitions, where teams obtained at least Honorable Mention. These award winning solutions serve as a benchmark reference; (2) LLM, where a LLM is directly used to generate mathematical modeling solutions; (3) DS-Agent, specialized LLM agent for automating data science tasks. We adapt its core design, based on case-based reasoning, to address mathematical modeling problems; (4) ResearchAgent, an L.M--based agent designed to automate research workflows and generate research ideas. We integrate it with a machine learning agent to enhance its capabilities for mathematical modeling; and (5) Agent Laboratory, an LLM-based framework designed to accelerate scientific discovery by guiding the research process through stages of literature review, experimentation, and report writing. For Agent Laboratory, the agent searches arXiv for related papers to identify mathematical modeling methods, which are then used to construct its pipeline for solving modeling problems.
E
Other Experiments
E.1 Experiments on Other Models
The experimental results in agents, ASeAgent and ResearchAgent, across all evaluation metrics in the 2021-2025 mathematical modeling competitions on Qwen2.5-72B. Since the capabilities of Qwen2.5-72B are weaker than those of GPT-4o and PrepSeck-R166111 the agent's 's laboratory workflow is unable to run efficiently. MM-Agent achieves the highest overall score of 8.37, excelling in Analysis Evaluation (8.61), Modeling Rigorousness (7.59), and Practicality and Scientificity (8.89), reflecting its superior ability in handling complex problems with scientific rigor and practical relevance. The stability of MM- Agent's 's performance across the competition years further supports the robustness of its approach, ensuring that its success is not result of overfitting, but rather reflection of its effective modeling capabilities.
that MM-Agent consistently outperforms the baseline
Table 3: Experiment results on the 2021-2025 mathematical modeling competitions on Qwen2.5- 72B. AE, MR, PS, and RBA denote Analysis Evaluation, Modeling Rigorousness, Practicality and Scientificity, and Result and Bias Analysis, respectively.
2021-2025
Methods
AE
MR
PS 1
RBA
Overall t
Qwen2.5-72B
7,58
3.71
8.35
5.72
6.34
DS-Agent
8.33
7.08
8.53
7.48
7,86
ResearchAgent
8.17
6.94
8.73
7.63
7.87
MM-Agent
8.61
7.39
8.89
8.37
8.37
26

---

Table 4: Experimental results on average token consumption, cost, and runtime using Qwen-2.5 72B.
Methods
Token
Cost($)
Runtime(s)
DS-Agent
264973
0.21
1757
ResearchAgent
313688
0.24
2577
MM-Agem
455610
0.34
2691
Table 5: Experiment on the well-defined optimization problem under zero-shot setting.
Linear
Nonlinear
Model
Code Pass
w/o Table
w/ Table
w/o Table
w/ Table
GPT-4o MM-Agent
77.5% 79.5%
68.8% 70.0% %
48.1% 50.4%
40.0% 42.0%
66.8% 68.8%
89.6%
99.3 %
E.2
Experiments on Well-defined Math Optimization Problems
To further evaluate the capabilities of our MM-Agent, extended the experiments to well-defined mathematical optimization problems, including both linear and nonlinear programming. In this setting, the agent is provided with the variables, objective function, and constraints, and is required to solve the optimization problem by directly producing the numerical solution. Since these problems have well-defined ground-truth solutions, accuracy be directly used the metric evaluate the performance of MM-Agent. Specifically, we conduct experiments on the widely used dataset OPTIBENCH 4 and the results shown in Table As these optimization problems do not require task decomposition, we appropriately modify the agent's configuration to align with the structure of this task. As shown in Table MM-Agent consistently outperforms GPT-4o across all subtasks under the zero-shot setting. In linear programming, MM-Agent achieves 79.5% accuracy without tabular input and 70.0% with table support, surpassing GPT-4o by 2.0% and 1.2%, respectively. Similar trends are observed in nonlinear optimization, where MM-Agent improves performance by 2.3% (w/o table) and 2.0% (w/ table). Notably, MM-Agent also achieves higher overall accuracy (68.8%) and code pass rate (99.3%), indicating better robustness and code reliability. These results highlight MM-Agent's enhanced reasoning capability and robustness solving well-structured mathematical tasks.
E.3 Human Evaluation Results
The experiments evaluated by human experts are summarized Figure 6 We report aver- age scores across four key dimensions: Anal- ysis Evaluation (AE), Modeling Rigorousness (MR), Practicality and Scientificity (PS), and Result and Bias Analysis (RBA). As shown in the figure, the MM-Agent consistently achieves the highest performance across all dimensions, particularly excelling in AE and RBA. AS-Agent and Agent Laboratory ex- hibit comparable performance in AE and PS, though the former shows slight superiority in RBA. Notably, ResearchAgent performs com- petitively in PS but lags behind in MR and RBA, indicating weaker modeling rigor and bias awareness. These results demonstrate the superior overall performance and robustness of MM-Agent under expert evaluation, particularly in producing well-analyzed and unbiased modeling outputs.
10
Agent Laboratory
DS-Agent
Score
ResearchAgent
8
MM-Agent
7
6
5
AE
MR
PS
RBA
Figure 6: Human Evaluation Results.
'The execution failed due the demanding requirements the instruction followwnn capabilities of LLMs.
27

---

Step Problem Understanding
Background: player
advantage often atmibuzed 'momentum.
The problem demands rigerous framework to quaatity momentum- - lasent construct interred from observable point outcomes while disentangling structural factons (e séventaggs stochoricity
Problem requireeet: ".. Develop model that cagtures the flow of points ocear...
path: /WWimbledon featured, masches.sv""
Dataset Destription: ("Wimbleoo, festured matches": "Data set second round")
Variable Description ("match id "match idenificatioo"
clepthh "depth return)
Problem
Anolysis
Step Task Dependency Analysis
Step Problem Decomposition
serves foundational step... The second task dependeas on the first task third
Quantficarion.
Task 2:
Diffinentiaing Momenturn from Randomness,
Task Predictive Modeling Momentant
The fourth tack depends on the first three tasks...
Task 4
Pros--Dmmann Geveraiization Analysis
Task Dependency Graph
Dependency Analysis
Tasks
Figure 7:
The workflow of the problem analysis phase in MM-Agent. Mathematical modeling tasks often involve interdependent objectives and subtasks. MM-Agent addresses this complexity by decomposing the problem into structured subtasks.
E,4 Annotation Quality
To evaluate annotation quality, we measure inter-annotator agreement on the MM-2025 datasets. We first rank the scores provided by each human and LLM annotator, and then compute the Pearson correlation coefficient between the ranked scores of each annotator pair. The results are summarized in Table 6 Human-
Agreements: The agreement between two human annotators varies across evaluation categories. We observe consistently high agreement the four metrics- metrics-Annllsiis Evaluation (AE), Modeling Rigorousness (MR), Practicality and Scientificity (PS), and Result and Bias Analysis (RBA) indicating strong overall consistency among annotators. Model-Human Agreements: To further assess annotation reliability. compare model-generated scores with human evaluations on the same subset. The model demonstrates reasonable alignment with human assessments, particularly in MR and PS, reported in Table 6 While RBA and AE exhibit relatively lower agreement compared to the other metrics, this does not necessarily indicate a shortcoming in the quality of problem and result analysis produced by MM-Agent, demonstrated in Figure and and Table T Instead, we interpret this as reflecting the inherent subjectivity and variability in how such explanations evaluated by different annotators, phenomenon also discussed by (48
Table 6: Results of agreements between two human annotation results and between human and model evaluation results.
Categories
AE
MR
PS
RBA
Human and Human
0.7475
0.4813 .4813
0.7890
0.7625
Model and Human
0.5068
0.7130
0.7860
0.5692
E,5
Case Study
This section provides detailed descriptions of the case study shown in Figures 7 and 8 illustrating how MM-A MM-Agent performs end-to-end problem-solving on real-world mathematical modeling task from the MCM competition.
In the problem analysis phase (Figure 7]. MM-Agent begins with Step 1: Problem Understanding. It extracts essential elements such as the task background, dataset path, and variable descriptions. For example, the agent identifies that the modeling task involves quantifying "momentum" in tennis matches using a dataset named Wimbledon featured_maa aat__matches.ee csv. The agent interprets the modeling goal as constructing a framework that infers momentum from point-level outcomes while accounting for severe advantages and inherent stochasticity. In Step 2: Problem Decomposition, MM- Agent breaks down the overall objective into four coherent subtasks: (1) Momentum Quantification, (2) Differentiating Momentum from Randomness, (3) Predictive Modeling of Momentum S Swings, and (4) Domsin Generalization Analysis. This decomposition transforms the open-ended problem into actionable components. Step 3 involves Task Dependency Analysis, where MM-Agent constructs
28

---

Mathematical Modeling
Step 1: Hierarchical Modeling Knowledge Retrieval
Step Actor-Critic Modeling Optimization
Task Dynamic Mome-
HMMLL
GARCH Model,
To mnudel the momentumm quantification, we begin
The structure oversimplifies
ntum Quantificatin.
Battle Model.
nunlincar momennamm dynamics.
Task
Methods
Modeling Scheme
Feedbock
Step Solution Report
Step Preliminary
Step 1: Code Generation and Execution
Report Outline
ingre
dynamic momentuum quantification in tennis matches using eeglme-switching state- framework.
'section (Abstract)
clans Momrensmkaatee?
'section (Problem Restatement)
Code
Isection [Solation]
wetehoe Llimo 1]
Isection (C (Cocclusion)
Nusuch dicctooy:
Modeling Method
Report
Report Outline
Computational Solving and Solution Reporting
Figure 8: The workflow of the mathematical modeling phase and computational solving and solution reporting phase of MM-Agent.
dependency graph capturing the logical and computational relationships between subtasks. For instance, Task I recognized foundational, while the remaining tasks build its outcomes. This structure ensures that the agent follows semantically grounded modeling order.
In the modeling and reporting phase (Figure 8p. Step is Hierarchical Modeling Knowledge Re- trieval. The agent retrieves candidate methods from the Hierarchical Mathematical Modeling Library (HMML) based the task description, including HMMs, GARCH models, and battle models for momentum estimation. Step features an Actor-Critic Iterative Optimization mechanism. The actor module proposes an initial modeling scheme, such Hidden Markov Model for momentum quantification. The critic then evaluates the model and returns structured feedback, noting that the current approach oversimplifies nonlinear momentum dynamics, prompting revisions toward models. Step 3 is Code Generation and Execution.
more suitable alternatives like regime-switchie MM-Agent generates Python code for the selected modeling pipeline. execution fails due to errors (e.g., PileNotFoundErros the agent diagnoses the issue and refines the code, continuing the process until a functioning version achieved.
The final stage involves constructing the solution report. The agent first drafts a preliminary outline, including sections like Abstract, Problem Restatement, Solution, and Conclusion. It then generates a complete report in human-readable LaTeX format, reflecting the entire modeling workflow. The final solution report can be seen in Figure
F Prompts used for MM-Bench and MM-Agent
This appendix presents the full set of prompts used in the construction of MM-Bench and the implementation of MM-Agent, These prompts are designed to support the automated, modular, and rigorous execution of real-world mathematical modeling workflows using large language models. Each prompt encapsulates a specific functional objective within the overall agent pipeline, including problem understanding, task decomposition, model formulation, code generation, result interpretation, and solution synthesis. The prompts are carefully structured to align with academic writing and reasoning standards, support multi-agent collaboration, and enable traceable, reproducible modeling. We include both instruction-level and response-forman specifications to ensure clarity and operational consistency. Together, these prompts form the foundation of our benchmark and agent framework, enabling end-to-end mathematical modeling automation.
Analysis Evaluation Prompt
Your task evaluate the rationality and overall coherence of the problem decomposition into sub-problems by the modeler, given the backgroud and problem requirement in mathematical modeling.
29

---

Figure 9: An example report generated by MM-Agent for Problem C of the 2024 MCM.
*BBacground** [background)
"*Problem Requirements**
(requirements)
Below is the modeler's task analysis: **Task *
(all task analyses)
30

---

**Evaluation Criteria**:
### 1. Problem Analysis and Understanding
#### 1.1 Problem Definition and Goals
Ensure the model definition is clear, the analysis accurate, and the goals are explicit.
Is the scope and goal of the problem clearly defined?
Are the key components of the problem effectively identified?
Are the actual goals that the model aims to solve clearly stated?
**Sooring Criteria**: 1-2 = Completely unclear; 3-4 Not clear enough; 5-6 = Basically clear; 7-8 Clear; 9-10 Completely clear.
#### 1.2 Relevant Scope and Coverage
Ensure that the core part of the problem is not deviated from, and whether each sub-task is interrelated and completely covers the actual goals.
Do the sub-tasks have dependencies?
Are all sub-tasks and steps directly related and support the final goal?
Are there any key parts missing or deviations from the actual goals?
**Soorng Criteria**;
1-2 Completely deviated from the goal; 3-4 Partially deviated; 5-6 Basically covered; 7-8 Mostly covered: 9-10 = Completely covered.
**Utpput *s: Please put your evaluation reasons and scores in the tags <rea-
son> your_reason </reason>, and <score> your_score </score>. Example:
### 1.1 Problem Definition and Goals: Evanu=tvalttoonn ** MnnnThe modeler has provided a clear definition of the problem and its goals. However, there some areas that need further clarification, such as the specific metrics used to measure success and the assumptions made during the analysis. Overall, the problem definition is mostly clear but could benefit from additional detail.
** **Score:*Me *Mncresoon>>>> The problem definition is mostly clear but lacks some details </reason> \n<score>> </score>
### 1.2 Relevant Scope and Coverage: E
sub-tasks are well-
defined and cover the main aspects of the problem. There a logical flow between the tasks, and each task supports the overall goal. However, sub-tasks could be more detailed to ensure complete coverage of the problem.
**Score: * *n<reason>> The sub-tasks are well-defined but could be more detailed </reason> \n<score> 8 </score>
Please objectively and detailedly evaluate the problem analysis and understanding according to the above evaluation criteria, and give the final score and reason. ### 1.1 Problem Definition and Goals:
Figure 10: The prompt used for evaluating Analysis of Agent.
Modeling Rigorousness Evaluation Prompt
Your task is to evaluate the rigor and rationality the modeling given the backgroud and problem requirement in mathematical modeling, particularly focusing the assumptions and rationality.
** BBckground*** [background]
31

---

**Problem Requirements**
{requirements]
Below is the modeler's modeling analysis: **Modeling Analysis**
fall dall_task_rn analyses)
Evaalattion Criteria**:
### 2. Rigor and Rationality of Modeling
#### 2.1 Assumptions
Clear and explicit. These assumptions are the foundation of the model and need to be rigorously justified.
Are the model assumptions clearly explained?
Are the assumptions reasonable and consistent with the background of the actual problem? Is the rationality and impact of the assumptions considered?
**Sooring Criteria**
= Completely unreasonable; 3-4 Partially reasonable; 5-6 Average; 7-8 Reasonable;
9-10 Very reasonable.
#### 2.2 Rationality
The rationality the model is key to evaluation. Evaluation criteria can include: whether an appropriate model is chosen, whether the model can realistically reflect the problem, etc. Has the model chosen appropriate methods and metrics?
Does the structure of the model scientifically reflect the actual problem?
* *Sooring Criteria**
1-2 = Completely unreasonable; 3-4 Partially reasonable; 5-6 = Average; 7-8 Reasonable; 9-10 Very reasonable.
* #Output Format**;
Example:
assumptions are erucial for model building. but the modeling analysis does not describe the assumptions in sufficient detail. The rationality and impact of the assumptions are not fully justified, lacking detailed explanations of data sources, data distribution, and competition characteristics. For example, the assumption about "serve advantage" is mentioned but not detailed on how quantified and integrated into the model. Additionally, the assumptions are not clearly explained, making the foundation of the model less robust. *
### 2.1 aaaappiiiinsnnn"=EEvvaaattttto
The model assumptions not clear enough and lack sufficient explanation their sources and impacts </reason> \n<score> </score>
#### 2.2 Rationality *Av="lvatatioo" * "nnnThe rationality of the model is average. The modeler chose to evaluate player performance based on match data (such as points won, games won, and sets won), which is reasonable to some extent. However, the specific modeling methods and metrics are not detailed. For example, how to quamtify "performance score", how handle time series data, and whether psychological factors in the competition are considered. Although some possible methods (such as time series analysis, regression, or classification) mentioned, their specific applications and reasons for selection not deeply explained. The structure of the model may have certain limitations in reflecting the actual problem.
**Score: * The rationality of the model is average, with methods and metrics not detailed, and the model structure has limitations </reason> \n<score>> </score>
Please objectively and detailedly evaluate the rigor and rationality of the modeling according to the above evaluation criteria, and give the final score and reason. ### 2.1 Assumptionslnln' ** Evaluation:
32

---

Figure 11: The prompt used for evaluating Modeling Rigorousness of Agent.
Practicality and Scientificity Evaluation Prompt
Your task is to evaluate the practicality and scientificity of the modeling process given the background and problem requirements in mathematical modeling, particularly focusing on whether the model can practically solve the problem and whether it adheres to scientific principles.
"BBackground** (hackground)
**Problem Requirements**
(reuuirements]
Below is the modeler's modeling process:
** **Modeling **
fall task analyses)
*Evaluation Criteria**;
### 3. Practicality and Scientificity
#### 3.1 Practicality
Does the modeling method match the characteristics and requirements of the problem?
Does the model provide meaningful insights beyond mere data fitting? Can its output support decision-making with clear explanations and reliable predictions across different datasets?
Does the approach go beyond standard machine learning data processing? Has it been deeply optimized or extended, potentially integrating interdisciplinary methods like mathematical or physical modeling?
Does the model introduce novel frameworks, constraints, objectives, or data representations? Does it push beyond conventional techniques to propose new theoretical computational approaches?
Is the selected modeling method appropriate for the given problem?
Is the model reasonably constructed?
Can the model solve the actual problem?
Are the application scenarios of the model clear? Is it feasible for practical operation?
Can the model's output provide useful information for deccision-making or exaplaining or predeting?
Does the approach go beyond basic data analysis and machine learning algorithms?
Does the model demonstrate innovation or creativity in its approach to addressing the problem?
Is the modeling approach tailored to the specific problem rather than using generic methods?
**Scoring Criteria**:
Completely impractical; 3-4 Partially practical; 5-6 = Average; 7-8 Practical; 9-10 VVry practical.
#### 3.2 Scientificity
Does the model adhere to scientific principles? Is there a theoretical basis?
Are the assumptions and methods of the model scientifically justified?
Does the model consider all scientific factors to ensure rationality?
Does the approach transcend simple data analysis incorporate deeper mathematical
domain-specific principles?
Is the approach innovative rather than standard application of common techniques?
Does the modeling process demonstrate understanding of the problem's 's unique characteris-
tics?
33

---

**Sooring Criteria**:
1-2 = Completely unscientific; 3-4 Partially scientific; 5-6 Average; 7-8 Scientific; 9-10
Veryy scientific.
* **Output #ormat"":
Example:
The model is somewhat practical, but it lacks several key aspects. The modeling method does not fully match the characteristics and requirements of the problem. Additionally, the model does provide meaningful insights beyond mere data fitting, and its output lacks clear explanations and reliable predictions across different datasets. The approach does not go beyond standard machine learning or data processing, and it has not been deeply optimized or extended to integrate interdisciplinary methods like mathematical or physical modeling, Furthermore, the model does introduce novel frameworks, constraints, objectives, or data representations, and it does not push beyond conventional techniques to propose new theoretical or computational approaches. **Score: In<rreson>> The model lacks several key aspects, including matching the problem characteristics, providing meaningful insights, and introducing novel approaches </reason> \ncscore> 6 </score>
### 3.1
#### 3.2 Scientificity inlieittty/nmmmmmmmmm Evall=Etiioon
model adheres to clear scientific principles and employs reasonable theoretical foundations. The assumptions and methods scientifically justified, and the modeler has thoroughly explained the rationality of the assumptions. Rather than relying solely on basic data analysis techniques, the approach incorporates sophisticated mathematical principles and demonstrates innovative application of theoretical concepts to the specific domain of the problem.
**Score:** Axcooe:*\<<cress0>"" #ncreaoon>>>>> The model adheres to scientific principles, incorporates advanced mathematical concepts, and demonstrates innovative application rather than generic approaches </reason> 'n<score>> 7 </score>
Please objectively and detailedly evaluate the practicality and scientificity of the modeling process according to the above evaluation criteria, and provide the final score and reason.
### 3.1 Practicalitylnnn" "aaallittyyynntEalataaaataaaa EEvaluation:
Figure 12: The prompt used for evaluating Practicality and Scientificity of Agent.
Result and Bias Analysis Evaluation Prompt
Your task is evaluate the result analysis and bias analysis of the given modeling report, particularly focusing on the rationality, interpretability of the model output, and the identification and correction of biases.
** *=Backrround* (background)
**Problem Requirements** (requirements)
Below the modeler's modeling report: **Modeling **
hil__ask_anallsee
* **Evaluation Criteria**;
### 4. Result Analysis and Bias Analysis
#*## 4.1 Result Analysis
34

---

Are the model output results clear and as expected?
Does the result provide sufficient analysis to explain the model's inference process?
Are the model results interpretable and do they help in understanding the essence of the problem?
Does the analysis provide clear conclusions and highlight the strengths and weaknesses of the model?
**Scoring Criteria**; 1-2 Completely unclear; 3-4 Partially clear; 5-6 Average; 7-8 Clear; 9-10 Very clear.
#### 4.2 Bias Analysis
-ooes the model identify and analyze potential biases?
Does it consider data bias, model bias, and other factors?
Does the model appropriately correct biases reduce their impact on the results?
* **Scoring Criteria**:
1-2 Completely ignored biases; 3-4 Partially considered biases; 5-6 Average; 7-8 Considered biases and corrected; 9-10 Very thorough, biases effectively corrected.
**Output Format**:
Example I:
model output results are clear and well explain the model'ss inference process. The modeler has detailed the background and significance of the model results, helping to understand the core of the problem. The results show reasonable inference path, making the entire analysis process more transparent. The analysis also provides clear conclusions and highlights the strengths and weaknesses of the model.
### 4.1 Result
EEvalutttnn
** The result analysis is very clear and effectively supports decision- making </reason> In<score> 9 </score>
model effectively identifies and analyzes biases, particularly potential data biases. The modeler provides correction measures for biases and explains how these corrections affect the model results. Although there are still some biases in certain aspects of the model, overall, comprehensive correction has been made.
### 4.2 Bias Analysistn'n ** Evaluatioon
**Score:** The bias analysis is thorough, and biases have been effectively corrected </reason> \ncscore> </score>
Please objectively and detailedly evaluate the result analysis and bias analysis of the modeling according to the above evaluation criteria, and provide the final score and reason.
### 4.1 Result *
Figure 13: The prompt used for evaluating Result and Bias Analysis of Agent.
Data Description Prompt
Data Description:
dlata_descrrptionn
(variable_descrippteee description)
Your task is to generate a detailed summary of the dataset based on the dataset de- scription provided. It needs to cover comprehensive information, but not explain each field
35

---

by Using plain text to describe in single paragraph, without any Markdown formatting or syntax.
Figure 14: The prompt used describe the dataset.
Problem Template
Problem Background: {problem_backgrouee b
Problem Requirement: (addendum)
Dataset Path: [dataset_path)
Data Description: {data |
Figure 15: The Problem Template prompt.
Problem Understanding Prompt
# Mathematical Modeling Problem: probllemm
You are tasked with analyzing mathematical modeling problem with focus on the underlying concepts, logical reasoning, and assumptions that inform the solution process. Begin by considering the nature of the problem in its broader context. What are the primary objectives of the model, and how do they shape the way you approach the task? Think critically about the assumptions that may be inherently embedded the problem. What implicit beliefs constraints have been set up. either explicitly or implicitly, within the problem's description? Reflect on how these assumptions might influence the interpretation and application of any potential solutions.
Dive deeper into the relationships and interdependencies between the different ponents the problem. What are the potential hidden complexities that may arise from these interconnections? Are there any conflicts tensions between different aspects of the problem that need to be resolved? Explore how these interdependencies might lead to unforeseen challenges and require revisiting initial assumptions or redefining the parameters of the task.
Consider how the complexity of the problem may evolve across different scales or over time. Are there dimendepeneent factors or long-tern consequences that should be accounted for, especially terms of the stability or sustainability of the model's outcomes? Think about how the model's behavior might change under different scenarios, such as variations in input or changes external conditions. Reflect on whether any simplifications or idealizations in the problem might inadvertently obscure key dynamics that are crucial for an accurate representation.
In your analysis, also give attention to possible alternative perspectives on the prob- lem. Are there different ways to frame the issue that could lead to distinct modeling approaches or solution strategies? How would those alternative perspectives impact the overall approach? Additionally, evaluate the potential risks uncertainties inherent in the
36

---

problem, especially when it comes choosing between competing modeling approaches. Consider how the outcomes might vary depending on the choices you make in constructing the model, and how you would manage such trade-offs.
Finally, reflect the dynamic nature of the modeling process itself. How might your understanding the problem evolve as you continue to explore its intricacies? Ensure that your thought process remains flexible, with readiness revise earlier conclusions as new insights emerge. The goal is to maintain a reflective, iterative analysis that adapts to deeper understandings of the task hand, rather than pursuing fixed rigid approach.
{user_ prompp
Respond as comprchensively and in much detail as possible. Do not format your response in Markdown. Using plain text, without any Markdown formatting or syntax. Written as one or more cohesive paragraphs. Avoid structuring your answer in bullet points or numbered lists.
Figure 16: The prompt used in the Problem Understanding step. It guides the agent perform deep conceptual and contextual analysis of the modeling task, encouraging reflection on assumptions, interdependencies, temporal dynamics, uncertainties, and alternative perspectives support rigorous and adaptive problem framing.
Problem Understanding Critique Prompt
# Mathematical Modeling Problem: goodelingg_roblenne
Problem Analysis: (problem_ p
Critically examine the analysis results of the given mathematical modeling problem, focusing on the following aspects:
1. Depth of Thinking: Evaluate whether the analysis demonstrates comprehensive understanding of the underlying problem. Does go beyond surface-level observations? Are the assumptions, limitations, and potential implications of the results carefully considered? Assess whether the analysis adequately addresses both the broader context and specific intricacies of the problem.
Novelty of Perspective: Analyze the originality of the approach taken in the analysis. Does it introduce new insights merely rehash well-established methods solutions? Are alternative perspectives or unconventional techniques explored, or is the analysis constrained by narrow set of assumptions typical approaches?
Critical Evaluation of Results: Consider the extent to which the analysis critically engages with the results. Are the conclusions drawn from the analysis well-supported by the mathematical findings, or do they overlook key uncertainties or counterexamples? Does the analysis acknowledge potential contradictions ambiguities in the data?
Rigor and Precision: Assess the level of rigor applied in the analysis. Are the steps logically consistent and mathematically sound, assumptions that undermine the conclusions? Does the analysis exhibit clear, methodical approach, is it characterized by vague reasoning and imprecision?
there overlooked errors, gaps, or
5. Contextual Awareness: Evaluate how well the analysis situates itself within the broader landscape of mathematical modeling in this area. Does it consider previous work developments the field? Is there any indication of awareness of real-world implications, practical constraints, or ethical concerns. if applicable?
37

---

Critique the analysis without offering any constructive suggestions focus should solely be on highlighting weaknesses, gaps, and limitations within the approach and its execution.
The prompt used for criticizing problem analysis in the Problem Understanding step. It prompts the agent to conduct focused critique of the initial analysis by evaluating its depth, originality. logical rigor, and contextual awareness, helping identify gaps and limitations without providing corrective suggestions.
Figure 17:
Problem Understanding Improvement Prompt
Mathematical Modeling Problem:
boodeling_problemt
# Problem Analysis: iproblem_analysite
# Problem Analysis Critique: wvroblem_analssisccriin
Refine and improve the existing problem analysis based on critique provided to generate insightful analysis.
Provide the improved version directly. DO NOT mention any previous analysis con- tent and deficiencies the improved analysis. Just refer to the above critical suggestions and directly give the new improved analysis. [essr_prompt]
Respond comprehensively and in much detail possible. Do not format your response Markdown. Using plain text, without any Markdown formatting or syntax. Written as one or more cohesive paragraphs. Avoid structuring your answer in bullet points or numbered lists.
IMPROVED PROBLEM ANALYSIS:
Figure 18: The prompt used for improving problem analysis in the Problem Understanding step. It guides the agent to revise its initial analysis by incorporating critical feedback, enabling more rigorous, insightful, and context-aware problem understanding through iterative refinement.
Task Decompose Prompt
# Decompose Principle: dlecomposed_princct
# Mathematical Modeling Problem: Bondeling_problent
Problem Analysis: {problem_analyšin
# Modeling Solution: imodelinggsolution
Please decompose the given modeling solution into (tasknum) distinct and well- defined subtasks that collectively contribute to the overall objective. These subtasks should
38

---

be clearly separated in their focus, each addressing specific aspect of the modeling process. The goal is to break down the solution into key stages or methodologics, ensuring that all components of the solution are covered without redundancy. For each subtask, the approach or technique should be explicitly described, detailing the specific data, algorithms, or models required. The decomposition should reflect a logical and comprehensive path toward completing the task, with each part having clear purpose and contributing to the final result. [user_prompt)
Each subtask should be described as comprehensively and in as much detail as possible within single paragraph using plain text and seperated by "-" for each subtask. All the contents and details of the original solution need to be covered by the [tasknum] subtasks without omission.
Figure 19: The prompt used in the Problem Decomposition step. It instructs the agent to transform holistic modeling solution into coherent set of structured subtasks, each with distinct objectives, methods, and contributions to the overall problem, ensuring logical coverage and methodological clarity.
Task Description Prompt
# Mathematical Modeling Problem: (modeling_probllem)
# Problem Analysis: prroblem_analysis)
# Modeling Solution: |modeling_soliee solution
# Decomposed Subtasks: |decomposed_suteee }
You are tasked with refining and improving the description subtask [task_i) to ensure it is more detailed, clear, and focused. Provide precise and comprehensive explanation the task, specifically elaborating on scope, goals, and methodology without venturing into other subtasks. Make sure the description includes clear and concise language that defines the necessary steps, techniques, or approaches required for this subtask. applicable, specify the data inputs, tools, models to be used, but do not introduce analysis, results, or discussions related to other components of the modeling process. The goal is to enhance the clarity, depth, and precision of this subtask description, ensuring is fully understood on its own without needing further explanation.
The description of subtask (task i] should be comprehensive and in much detail as possible within single paragraph using plain text.
The prompt used for refining and improving task descriptions. It guides the agent to produce precise, self-contained, and detailed explanation of specific subtask, clarifying its scope, objectives, methodology, and required resources while avoiding overlap with other components.
Figure 20:
Task Dependency Analysis Prompt
Understanding the dependencies among different tasks in a mathematical modeling process crucial for ensuring coherent, logically structured, and efficient solution. Given mathematical modeling problem and its solution decomposition into (tasknum) subtasks, analyze the interdependencies among these subtasks.
39

---

## Input Information:
** Mathematical Modeling Problem: sanodeling_probleee
Proobemm Analysis:** **
** Modeling Solution: [modelinggsoluttee ssssstolltion
** *eecomposed Tasks:** (task descriptions]
## Task Dependency Analysis Instructions:
1. **Identify Task Dependencies:** For each task, determine which preceding tasks provide necessary input, data, conditions for its execution. Clearly outline how earlier tasks influence or constrain later ones.
** **Describe Dependency Types:** Specify the nature of the dependencies between tasks.
This includes:
*DDaa Dependency: When one task produces outputs that are required as inputs for another task.
When later task builds upon theoretical framework, assumptions, models established by an earlier task.
*Methodological Dependency:
*Computational Dependency: When a task requires prior computations or optimizations to be completed before proceeding.
*Structural Dependency: When task is logically required be completed before another duc to hicrarchical or sequential constraints.
*CCode Dependency: When one task relies on code structures, functions, or modules that defined executed in preceding task. This includes shared variables, functions, or libraries that must be defined before their use in later tasks.
3. * "Ensuree Completeness: Verify that all tasks in the decomposition accounted for in the dependency analysis and that no essential dependencies are missing.
## Output Format:
Respond comprehensively and in much detail possible. Do not format your response in Markdown. Using plain text, without any Markdown formatting or syntax. Written as tasknum cohesive paragraphs, each paragraph is dependency analysis of task.
The response should be comprehensive and written in clear, well-structured for- mat without bullet points, ensuring logical flow of dependency relationships and their implications.
The prompt used the Task Dependency Analysis step. It guides the agent to identify and describe data, methodological, computational, and structural dependencies among subtasks, ensuring a coherent and executable modeling workflow.
Figure 21:
DAG Construction Prompt
A well-structured Directed Acyclic Graph (DAG) is essential for visualizing and optimizing the dependencies between different tasks in mathematical modeling process. Given a problem and its solution decomposition into tasknum subtasks, construct DAG that accurately represents the dependency relationships among these tasks. The DAG should capture all necessary dependencies while ensuring that no cycles exist in the structure.
## Input Information:
**Mathematical Modeling *roblem:** (modeling_problee
**Problem Analysis:** [problem_analysiss analysis)
**Modeling Solution:** (modeling_soluutee ssouttiolooon
** Decomposeed Tasks:* [task_ cesss_deccrpttons]
** Dependency Analysis:** [eas__depeendnccy_anllyss
## Output Format (STRICT REQUIREMENT):
40

---

You **MUST** return valid JSON- adjacency list *#ithouu*** any additional text, explanations, or comments. **Only** output the JSON object.
### JSON Format (Strictly Follow This Format):
"json
task ID": [dependent__Dpg
## Example Output:
"json
{I
"1": 0
"2": [T]
[T"]
"4": (2' '3
})
Figure 22: The prompt used for constructing the Task Dependency Graph. instructs the agent to generate DAG in strict JSON format, capturing all task-level dependencies derived from prior analysis to enable structured visualization and exccution planning.
Model Formulas Construction Prompt
Reference Modeling Methods:
imodeling_methone
[data_summary]
# Task Description:
Task Analysis:
[task_ama analysis)
The structure of code for Task (task_id) {code_structure)
# The result for Task (task_id)
[task_resull
When formulating the mathematical model for the current task, it is essential to consider how this task depends on other tasks in the overall process.
You are collaborating part of a multi-agent system to solve a complex mathemati- cal modeling problem. Each agent is responsible for a specific task, and some preprocessing or related tasks may have already been completed by other agents. not repeat any steps that have already been addressed** by other agents. Instead, rely on their outputs when necessary and focus solely on the specific aspects the task assigned you.
crucial that you **do
You are tasked with developing a set of precise, insightful, and comprehensive mathematical formulas that effectively model the problem described in the task. Begin by conducting an in-depth analysis of the system, process, or phenomenon outlined, identifying all relevant variables, their interdependencies, and the fundamental principles, laws, constraints that
41

---

govern the behavior of the system, applicable in the relevant field. Clearly define all variables, constants, and parameters, and explicitly state any assumptions, approximations, or simplifications made during the formulation process, including any boundary conditions or initial conditions if necessary.
Ensure the formulation considers the full scope of the problem, and if applicable, incorporate innovative mathematical techniques. Your approach should be well-suited for practical computational implementation, addressing potential numerical challenges, stability concerns, limitations in simulations. Pay careful attention to the dimensional consistency and units of all terms to guarantee physical or conceptual validity, while remaining true to the theoretical foundations of the problem.
In the process of deriving the mathematical models, provide clear, step-by-step explanation of the reasoning behind each formula, highlighting the derivation of key expressions and discussing any assumptions or trade-offs that are made. Identify any potential sources of uncertainty, limitations, approximations inherent in the model, and provide guidance on how to handle these within the modeling framework.
The resulting equations should be both flexible and scalable, allowing for adapta- tion to different scenarios the ability be tested against experimental real-world data. Strive to ensure that your model not only rigorous but also interpretable, balancing complexity with practical applicability. List all modeling equations clearly in LaTeX format, ensuring proper mathematical notation and clarity of presentation. Aim for a model that is both theoretically sound and practically relevant, offering a balanced approach to complexity and tractability in its use. p
Respond as comprehensively and in as much detail as possible, ensuring clarity, depth, and more cohesive
rigor throughout. Using plain text and LaTeX for formulas. Written paragraphs. Avoid structuring your answer bullet points or numbered lists.
The prompt used for constructing the model formulas. It instructs the agent to derive detailed, rigorous, and task-specific mathematical formulations by integrating prior task outputs, domain principles, and computational considerations, ensuring both theoretical soundness and practical applicability.
Figure 23:
Model Formulas Critique Prompt
{data_sumiin summary
# Task Description: [task_desee juis__description
Task Analysis: (task_analysis
# Task Modeling Formulas: (modeling formulas
The goal of this task is to critically evaluate the modeling formulas used to repre- sent given mathematical modeling problem. Your analysis should address the following dimensions: accuracy and rigor, innovation and insight, and the applicability of the models to real-world scenarios.
Accuracy and Rigor:
42

---

Formula Integrity:
Evaluate whether the mathematical models and the corresponding formulas are mathemati- cally sound and consistent with the underlying assumptions of the problem. Are the formulas properly derived, free from logical errors, and reflective of the relevant domain knowledge? Are any simplifications approximations made, and if so, they justifiable within the context of the model'ss s scope?
Examine the assumptions made in formulating the model. Are these assumptions realistic, and how do they affect the model's precision and robustness?
2. Innovation and Insight:
Novelty of Approach:
Critique the originality of the modeling approach. Does the model present new or unconventional way of solving the problem, or does it simply rely on established methodologies without offering new insights?
Consider whether any innovative methods, such as the introduction of novel variables or the use of innovative computational techniques, contribute improving the model.
Theoretical Insight:
Evaluate the depth of the theoretical insights provided by the model. Does it offer fresh perspective or new understanding of the problem? How well does it illuminate the key dynamics and relationships within the system under study?
Does the model reveal previously unnoticed phenomena, or does it suggest new directions for further research?
Integration of Existing Knowledge: Assess the extent to which the model integrates existing mathematical, theoretical, and empirical work. Does it build prior research, and if so, does it do in way that adds substantial value or clarity? Are there gaps where additional diossddisciplinary knowledge could enhance the model?
Applicable:
Real-Waoor World Relevance:
Evaluate the model's practical applicability. How well does it apply to problems, and to what extent does provide actionable insights for decision-making or problem-sole solving in the field?
Critique the analysis without offering any constructive suggestions-youu focus should solely be on highlighting weaknesses, gaps, and limitations within the formulas.
The prompt used for criticizing the model formulas. It guides the agent to identify weaknesses in mathematical soundness, theoretical depth, and real-world applicability of the formulas, fostering rigorous evaluation without offering corrective suggestions.
Figure 24:
Model Formulas Improvement Prompt
dlata_summany
# Task Description: )
# Task Analysis: [task_analyiic a
43

---

# Task Modeling Formulas: [modeling formulas
# Task Modeling Formulas Critique: mmodeling formulas formulas_ccitiquue)
Based the provided critique and analysis, refine the existing modeling formulas to address the identified limitations and gaps.
Respond as comprehensively and in as much detail as possible, ensuring clarity, depth, and rigor throughout. Using plain text and LaTeX for formulas. Written one or more cohesive paragraphs. Avoid structuring your answer in bullet points or numbered lists. {user_ [user_prompt] Provide new version of the task modeling formulas that integrates these improvements directly. DO NOT mention any previous formulas content and deficiencies.
IMPROVED TASK MODELING FORMULAS:
Figure 25: The prompt used for improving the model formulas.
Model Construction Prompt
# Task Description: [task_deste <issskdecrrption)
Task Analysis: (task_analysis,
# Task Modeling Formulas: (modeling formulas)
# The structure of code for Task [task_id] id]: |
The result for Task (task_ide (task_resulty
Please consider the dependencies between the current task and the preceding tasks.
You are collaborating part of a multi-agent system solve a complex mathemati- cal modeling problem. Each agent is responsible for a specific task, and some preprocessing or related tasks may have already been completed by other agents. not repeat any steps that have already been addressed by other agents. Instead, rely on their outputs when necessary and focus solely on the specific aspects the task assigned you.
crucial that you **do
Please continue the modeling formula section by building upon the previous intro- duction to the formula. Provide comprehensive and detailed explanations and instructions that elaborate on each component of the formula. Deseribe the modeling process thoroughly, in- cluding the underlying assumptions, step-by-step derivations, and any necessary instructions for application. Expand on the formula by incorporating relevant mathematical expressions where appropriate, ensuring that each addition enhances the reader's understanding of the model. Make sure to seamlessly integrate the new content with the existing section, maintaining natural flow and avoiding any repetition conflicts with previously covered
44

---

material. Your continuation should offer clear and in-depth exploration of the modeling formula, providing all necessary details to facilitate complete and coherent understanding of the modeling process.
[ossr_prompt]
Respond comprehensively and in much detail possible. Do not format your response in Markdown. Using plain text, without any Markdown formatting or syntax. Written as one cohesive paragraphs. Avoid structuring your answer in bullet points or numbered lists.
The prompt used for improving the model formulas. It instructs the agent to revise existing formulas by directly integrating feedback from prior critique, ensuring the final formulation is more rigorous, complete, and aligned with problem-specific constraints.
Figure 26:
Code Generation Prompt
# Dataset Path: [dataset_path)
# Data Description: )
# Variable Description: variable desebippidscriiin
Other files (Generated by Other Agents);
/dependent_file laatiifflleeeiiiie
# Task Description:
ftask description)
Task Analysis:
# Task Modeling Formulas: [modeling_formulass
Task Modeling Process: mmodeling process]
# Code Template: (code_template)
## Role & Collaboration:
You are an expert programmer working as part of a multi-agent system. Your role to implement the code based on the provided dataset (**refer to the Dataset Path, Dataset Description, and Variable Description **) ** **or preprocessed files generated by other agents** (**refer to "Other **les"**), along with the modeling process and given code template. Other agents will use your results to make decisions, but they will **nor** review your code. Therefore, it is crucial that:
** **Ensure the code is *xecutable** and will successfully run without errors, producing the expected results. **It should be tested to verify it works in the intended environment** **Reuse files from "Other Files" whenever possible** instead redoing tasks that have already been completed by other agents.
*llll data processing steps must save the processed results to local files (CSV, JSON, or pickle) for easy access by other agents,**
45

---

4. *h output should be detailed possible**, including intermediate results and final outputs.
5. ***Ensu transparency*** by logging key computation steps and providing clear outputs.
## Implementation Guidelines:
** **Prioritize using files from "Other Files" before processing raw data to avoid redundant computation.
Follow the provided moddeingg formulas** and ** modeling process** precisely.
The **oode must be executable**; ensure that the Python code you generate runs without errors. Do not just focus on producing the correct output format; **focus on producing working solution ** that can be executed successfully in Python environment.
**Store intermediate and final data processing results to local** in appropriate formats (c.g.,
CSV, JSON, pickle).
Provide **dtailed print/logging outputs ** to ensure that other agents can understand the results without needing read the code. )
## Expected Response Format:
You **MUST** return the Python implementation in the following format:
"python
Here is the Python code.
Figure 27: The prompt used for generating code. instructs the agent to produce fully executable Python implementations aligned with prior modeling outputs, while ensuring correctness, repro- ducibility, and interoperability within a multi-agent system through structured input, logging, and file-based output handling.
Code Debugging Prompt
# Code Template:
# Modeling Process:
(modeling_procest)
Current Code:
(code)
However, there are some bugs in this version. Here is the execution result: Execution Result:
{observation)
You helpful programming expert. Based the provided execution result, please revise the script to fix these bugs. Your task to address the emror indicated in the result, and refine modify the code needed to ensure it works correctly. {user_prompt)
Please respond exactly in the following format:
"python
Provide the corrected python code here.
The prompt used for debugging code. It instructs the agent identify and fix execution errors based on observed outputs, ensuring that the corrected Python script is functional, aligned with the modeling process, and ready for downstream use.
Figure 28:
46

---

Code Structure Extraction Prompt
You programming expert. Please extract the structure from the following code and output it in the following JSON format, please return an empty list if the corresponding item available.:
The code is:
"python {code}
The output format is: "json
{1
"seript_path": (save_path) "class":
{{
"name": class name,
"description": description of class,
"class functions":
{{
"name": function name.
"description". description of class function,
"parameters":
[{
"name": param name.
"type": param type, "description": description of param, }).
"returns": "description": "return of the function."
1) ))
}}
1.
"function": [
{{
"name": function name, "description": description of class function,
"parameters": {(
"name": param name, "type": param type, "description": description of param,
}).
1. "returns:: 11 "description": "return of the function." "
11.
1.
"file_outputs"
[(
"path": "file_path",
"file "edddcdcrition". "description of the file".
["column C cssseeeeee _aaeeeeeeeeeeeeee None"]
47

---

11
1)
Figure 29: The prompt used for extracting the structure of code.
Result Interpretation Prompt
# Task Description:
[ description)
Task Analysis: (task_anade jass__aalysis]
# Task Modeling Formulas: {
# Task Modeling: (task modeling
# Code Execution Result: (execution_restee r
Based on the task description, analysis, modeling framework, and code execution result, present a comprehensive and detailed account intermediate results, calculations, and outcomes generated during the task. Clearly articulate the results of any computations or operations performed, providing numerical values, data trends, or statistical measures necessary. If visual representations such graphs, charts, tables were used to communicate the results, ensure they are clearly labeled and explained, highlighting their relevance the overall task. Discuss the intermediate steps processes that led to the results, including any transformations or assumptions made during calculations. If applicable, compare and contrast these results with expected outcomes previously known results to gauge the task's 's success. Provide a thoughtful interpretation of the findings, considering how they contribute advancing understanding solving the problem hand, and highlight any areas where further investigation or refinement may be needed. (user_prompt)
Respond as comprehensively and in as much detail possible. Do not format your response in Markdown. Using plain text and LaTeX for formulas only, without any Markdown formatting or syntax. Written as one or more cohesive paragraphs. Avoid structuring your answer bullet points or numbered lists.
The prompt used for extracting the structure of code. It guides the agent parse and represent the structural elements of the code, including functions, classes, parameters, and output files, in standardized JSON format support traceability, reuse, and documentation in downstream tasks.
Figure 30:
Solution Formulation Prompt
Task Description: )
# Task Analysis:
48

---

ttask analysis
# Task Modeling Formulas: [task_ f
Task Modeling:
(task_modeling)
# Task Result:
[task_result]
Craft a comprehensive and insightful answer section that synthesizes the findings presented in the results section directly address the research questions and objectives outlined at the outset of the study. Begin by clearly stating the primary conclusions drawn from the analysis, ensuring that each conclusion explicitly linked to specifie aspects of the results. Discuss how these conclusions validate or challenge the initial hypotheses or theoretical expectations, providing a coherent narrative that illustrates the progression from data to insight.
Evaluate the effectiveness and reliability of the mathematical models employed, highlighting strengths such as predictive accuracy, robustness, or computational efficiency. Address any limitations encountered during the modeling process, explaining how they may impact the validity of the conclusions and suggesting potential remedies or alternative approaches. Consider the sensitivity of the model to various parameters and the extent to which the results are generalizable to other contexts or applications.
Analyze potential biases that may have influenced the results, including data bias, model bias, and computational bias. Discuss whether the dataset is representative of the problem space and whether any imbalances, selection biases, or sampling limitations might have affected the conclusions. Examine modeling assumptions, parameter choices, and architectural constraints that could introduce systematic deviations in the results. Assess how numerical precision, algorithmic approximations, or implementation details might influence the stability and faimeess of the model's 's predictions.
Discuss strategies to mitigate identified biases and improve the reliability of the conclusions. Consider adjustments in data preprocessing, such as resampling, normalization, or augmentation, to address distribution imbalances. Explore refinements to the modeling process, including regularization techniques, fairness constraints, and sensitivity analyses, to ensure robustness across different scenarios. Evaluate the impact of alternative modeling approaches and discuss the extent to which the proposed methods can generalize beyond the given dataset problem context.
Explore the broader implications of the findings for the field of study, identifying how they contribute to existing knowledge, inform future research directions, or influence practical applications. Discuss any unexpected outcomes and their significance, offering interpretations that may reveal new avenues for exploration or theoretical development. Reflect the societal, economic, environmental relevance of the results, if applicable, and propose recommendations based on the study's 's insights.
Conclude the section by summarizing the key takeaways, emphasizing the contribu- tion of the research solving the problem at hand, and outlining the next steps for further investigation or implementation. Ensure that the discussion is logically structured, with each paragraph building upon the previous ones form cohesive and persuasive argument that underscores the study's 's value and impact.
49

---

The content of this Task Answer section should be distinct and not merely repeti- tion of the Task Result section. Ensure that there is no duplication.
{user_ puser_prompt)
Respond as comprehensively and in as much detail as possible. Do not format your response in Markdown. Using plain text and LaTeX for formulas only, without any Markdown formatting or syntax. Written as one or more cohesive paragraphs. Avoid structuring your answer in bullet points numbered lists.
The prompt used for formulating the solution. It guides the agent to synthesize modeling results into a coherent, bias-aware, and insight-driven conclusion that addresses research objectives, evaluates model reliability, and reflects on broader implications.
Figure 31:
Chart Guidelines Generation Prompt
## Instruction
Create a highly detailed and comprehensive chart that effectively visualizes the complex mathematical relationships and insights presented in the provided mathematical modeling paper. Begin by selecting the most appropriate type of chart-such as line graph, bar chart, scatter plot, heatmap, or 3D surface plot-basee on the nature of the data and the specific relationships being analyzed. Clearly define the variables involved, including their units and scales, and incorporate any derived metrics that enhance interpretability. Ensure that the axes are labeled accurately and descriptively, with appropriate units and scales, whether linear or logarithmic, to best represent the data distribution and relationships. Include a clear and concise legend that distinguishes between different datasets or variables, using distinct colors patterns that are both aesthetically pleasing and easily distinguishable. Utilize gridlines to aid the accurate reading values, and choose a color scheme that enhances readability while maintaining visual appeal.
Emphasize the purpose of the chart, whether it is highlight trends over time, compare different values, show distributions, illustrate correlations, validate theoretical models, support key arguments within the paper. Articulate the intended message of the chart clearly, ensuring that every design choice-from the type of chart to the specific visual elements used-aligns with the objectives of the mathematical modeling paper. Incorporate multiple lines or bars if comparing different datasets, use shading or contouring for density representation, and add error bars to indicate uncertainty where applicable. Include annotations to highlight significant data points, trends, or anomalies that are critical the analysis, providing context and explanations that guide the viewer's understanding.
Balance aesthetics with functionality by selecting colors and contrasts that not only make the chart visually compelling but also enhance readability and comprehension. Avoid unnecessary complexity by keeping the design clean and focused, ensuring that the chart remains clear and easy to interpret without sacrificing accuracy or depth of information. If beneficial, incorporate supplementary visual aids such as trend lines, regression curves, or overlays of empirical and theoretical results to strengthen the analysis and provide additional layers of insight. The final chart should serve as a precise and compelling visualization that effectively conveys the mathematical insights, facilitates understanding, and robustly supports the overall narrative and conclusions the mathematical modeling paper.
)ossr_prompt)
## Paper Content <paper> (paper_content) </paper>
50

---

## Existing Charts
{existing }
## Create New Chart
Please create a chart that aligns closely with the above paper content while avoid- ing redundancy with existing charts. Follow the markdown format below describe your chart:
**Chart *
[Provide clear and descriptive title for the chart]
**Chart Type**
[Specify the type of chart]
**Purpose**
[Describe the core purpose of the chart a paragraph]
**Data Variables**
[escribe the data or variables used in the chart in paragraph]
**Chart Presentation Guidelines**
[A comprehensive guide on chart presentation, covering data representation, key layout elements, units, axis labels, legends, gridlines, annotations, and other essential considerations for effective visualization.)
**Intended Message**
[Articulate the key message insight the chart is intended to convey in a paragraph)
The prompt used for generating chart creation guidelines. instructs the agent to design detailed, purpose-driven visualizations that align with the mathematical insights of the paper, specifying data, layout, annotation, and interpretability considerations to ensure clarity, relevance, and analytical depth.
Figure 32:
Paper Chapter Creation Prompt
You are tasked with creating publication-quality LaTeX chapter for mathematical modeling research paper. Carefully transform the provided structured draft into a coherent, rigorous, and concise narrative chapter that aligns logically and seamlessly with the previously written content.
## Target Chapter: |chapter_path]
## Structured Draft: <structured draft>
jjson_ context) carnncturedddrffii
## Preceding Chapters (for seamless narrative integration and avoiding repetition):
<preceding content>
_coetetggccotent>>>>
previous _
</preceding_content>>>>>
## Requirements:
Write exclusively in accurate, idiomatic LaTeX; avoid Markdown syntax and symbols
51

---

entirely.
Clearly indicate the chapter content corresponds precisely to the target chapter '(chap-
ter_ per_path]": do not repeat reference explicitly the content of other chapters.
Integrate any mathematical formulas properly using correct LaTeX environments ("eeeiin align)'). Truncate and wrap long formulas and symbols.
Present the chapter as continuous, fluent narrative without section headings, subsections, bullet points, or numbered lists, Response only chapter content, do not include headlines and anything else.
Critically evaluate the structured draft, selecting only most high-quality important and relevant content. Remove all redundancy, eliminate low-value statements, and distill essential information clearly and succinctly.
Maintain rigorous academic style, logical coherence, and clarity throughout, ensuring that the chapter integrates naturally with preceding chapters.
## Output Format:
"latex
CHAPTER CHAPTER_CONTEETTE CONTENT_TEXT
Figure 33: The prompt used for creating a paper chapter. It guides the agent to transform structured draft into fluent, publication-ready LaTeX chapter that integrates rigorously with preceding content while ensuring clarity, conciseness, and academic coherence.
Paper Chapter Creation with Preceding Prompt
You are tasked with generating a publication-qual quallity LaTeX chapter for mathematical modeling paper. Write cohesive, academically rigorous chapter that integrates seamlessly with the preceding content of the paper.
## Chapter to write: (chapter_pattee p
## Preceding Content:
<preceding_ _ontiinn__oonnt>>>tee (previous ) <preceding_ content> <enceding_content>>>>
## Writing Requirements:
Use accurate and proper LaTeX syntax throughout, avoid all Markdown syntax or symbols. Present the content continuous, coherent narrative without using sections, subsections, or bullet points. Response only chapter content, do not include headlines and anything else. Make clear that the section you need to write is "[chapter_path) Do not involve the content of other chapters.
Figure 34:
The prompt used for creating a paper chapter with preceding content. guides the agent to generate cohesive, LaTee-formatted chapter that maintains academic rigor and continuity with prior sections, ensuring seamless narrative flow while adhering strictly to chapter boundaries.
Paper Notation Creation Prompt
You are an Al assistant trained to extract and typeset the Notations table from a mathematical modeling paper in LaTeX format. Your task is to take the input paper and output a properly formatted LaTeX table displaying the notations used in the paper.
Well-structured and easy to read. 2. Properly typeset for LaTeX documents.
52

---

3. Adaptive in size and position to fit neatly into any document. Truncate and wrap long formulas, symbols and text in the table for better readability.
<paper> (previous ]
</paper>
Exmple of Table Format:
"latex
(begin[table (centering
Ienewcommand/ / (arrayyyttttcch
bbegin tabular) (>> (raggedright aaraaabassaaaa //3mm/>
Araggerrihht baggeddhhht.arrrabbckksssss [1cm)]
toprule
[etabf[ & (eexb((Description)
midrule
((x)) ) & description...
'bottomrule
(tabula)))
caption Table of Notations)
[abe
lend [
Response only latex table content, do not include headlines and anything else.
Figure 35: The prompt used for creating paper notations. It instructs the agent extract and format a LaTeX- cooppaccomeeible notations table from the paper, ensuring clarity, structural consistency, and integration within mathematical modeling documents.
Paper Meta Information Prompt
You are an expert academic writer tasked with analyzing paper chapters and generating key metadata for mathematical modeling paper.
# Input Chapters "paper_chapters
Based on the content of these chapters, please generate:
1. A concise, descriptive title that reflects the paper's main focus
2. A comprehensive and detailed summary highlighting key findings and methodology
4-6 relevant keywords that capture the paper's main themes
Returns the Legal JSON Format: "'Json
"title": "A clear, concise title".
"summary": "A well-structured summary covering the following information: {n- Restatement and Clarification of the Problem: Describe the problem to be solved in your own words. nn- Explanation of Assumptions and Their Rationality: Highlight the assumptions made in the modeling process and clearly list all the variables required for the model. nn Model Design and Rationality Argumentation: Specify the type of model used or describe the construction of new model, explain how it was established and the rationale behind its design. nn- Description Model Testing and Sensitivity Analysis: Include error analysis and other testing items.".
53

---

"keywords":
"keyword1; keyword2; keyword3; keyword4..."
}}
Requirements:
Title should specific and academic in tone
Summary should follow standard academic abstract structure and be approximately 400 words
Keywords should be ordered from general to specific
must return a strictly legal JSON
Figure 36: The prompt used for creating paper meta information. It instructs the agent generate publication-ready title, abstract, and keyword set from chapter content, producing a structured and legally formatted JSON summary aligned with academic conventions.
54


Reviews:
[{"review_id": "xmpAICBIU0", "reviewer": "Reviewer_geD1", "summary": "In this paper, the authors introduce an agentic system for open-ended mathematical modeling. Rather than rely on zero-shot LLM capabilities, they break down modeling problems into problem framing, model formulation, solution development, and interpreting and preparing results. By using temporal splits, LLM-as-judge and human expert review, and experiments on optimization problems with ground truth answers, they show that their system outperforms other agentic systems for mathematical modeling.", "questions": "The authors rely on the temporal split (2021-2024) vs (2025) as evidence that the underlying LLMs are performing “genuine modeling” rather than memorization. Are there any metrics of problem similarity between the splits from year to year that could be used to indicate how different the problem statements are?\nCan the authors expand on the results in Table 1 - which metrics are from automated scoring from GPT-4o vs human expert review? How many expert reviewers were consulted and how was expert review performed?\nMM-agent helped two teams achieve Finalist awards in competition - which specific subtasks, problems, or evaluations did the top performing teams outperform the MM-agent augmented teams in? Is there a clear path to highlight and address these limitations in future work?\nTo what degree did the two finalist teams rely on MM-agent? \nAre there problem types and domains that are not currently considered in the MM-Bench evaluation that the framework could be extended to?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\nThe approach and research objectives are clearly laid out.\nThe token consumption results are an interesting evaluation, showing that MM-Agent is not simply consuming more tokens than other methods.\nThe “real-world” applicability of MM-agent as used in the MCM/ICM competitions could be compelling evidence of practical utility.\n\nWeaknesses\nA few points could benefit from increased clarity: the usefulness and rigor of the temporal split as an approximately “OOD” evaluation, the delta in performance between top-performing human experts and teams using MM-agent as a modeling copilot, and the exact use cases and utility of MM-agent to the finalist teams.\nHMML is included as a main contribution, but it seems like a simple tree structure representing hierarchies of modeling approaches. It’s unclear how useful or necessary this is to MM-Agent’s performance.", "was_revised": true, "final_justification": "The authors have clarified all questions and comments raised during the review, and I will increase my score accordingly.", "rebuttal": "### Thanks for reviewer geD1 and global resposne\nWe sincerely thank Reviewer geD1 for their thoughtful and constructive feedback. We appreciate your recognition of our **agentic framework, the token efficiency analysis, and the real-world applicability of MM-Agent in MCM/ICM competitions.**  Thank you also for raising insightful questions on evaluation clarity, temporal split design, HMML’s role, and MM-Agent’s impact in competition settings. These points have helped us further strengthen and clarify our work.\n\n> [W1 & Q1] W1:“ A few points could benefit from increased clarity: the usefulness and rigor of the temporal split as an approximately “OOD” evaluation, the delta in performance between top-performing human experts and teams using MM-agent as a modeling copilot, and the exact use cases and utility of MM-agent to the finalist team” and Q1: ”Similarity Across Temporal Splits”\n\nResponse: Thank you for the thoughtful question. We would like to kindly clarify that our benchmark problems are drawn from the **real-world MCM/ICM competition**, the world’s largest mathematical modeling contest. Its strict annual format ensures **fresh and diverse topics each year**, eliminating content repetition by design.\n\nTo further support this, we compute **semantic similarity** between each 2025 problem and those from 2010–2024 using the widely used **mGTE embedding model** \\[1]. As shown in the table below, most similarity scores fall between **0.5–0.6**, confirming that the 2025 problems are **substantively distinct** from prior years. This supports our claim that the model’s success is **not due to memorization**, but rather genuine generalization to novel problem statements.\n\nTable: Similarity between each problem from 2025 and problems from 2010 to 2024.\n\n|  | 2010 | 2011 | 2012 | 2013 | 2014 | 2015 | 2016 | 2017 | 2018 | 2019 | 2020 | 2021 | 2022 | 2023 | 2024 |\n|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n| 2025A | 0.46 | 0.50 | 0.56 | 0.56 | 0.53 | 0.46 | 0.57 | 0.55 | 0.54 | 0.62 | 0.48 | 0.57 | 0.59 | 0.60 | 0.52 |\n| 2025B | 0.51 | 0.52 | 0.53 | 0.62 | 0.51 | 0.55 | 0.58 | 0.50 | 0.61 | 0.61 | 0.55 | 0.54 | 0.60 | 0.62 | 0.59 |\n| 2025C | 0.59 | 0.56 | 0.52 | 0.61 | 0.55 | 0.51 | 0.61 | 0.58 | 0.62 | 0.57 | 0.57 | 0.54 | 0.61 | 0.59 | 0.58 |\n| 2025D | - | - | - | - | - | 0.61 | 0.63 | 0.63 | 0.63 | 0.62 | 0.59 | 0.55 | 0.66 | 0.64 | 0.63 |\n| 2025E | - | - | - | - | - | - | 0.58 | 0.58 | 0.52 | 0.65 | 0.60 | 0.69 | 0.59 | 0.56 | 0.54 |\n| 2025F | - | - | - | - | - | - | 0.66 | 0.58 | 0.69 | 0.55 | 0.61 | 0.68 | 0.61 | 0.57 | 0.62 |\n\n\n[1] mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval\n\n> **Q2: “Expert Review and Evaluation Details**\n\nResponse: Thank you for the insightful question. The metrics reported in Table 1 reflect automated evaluation using GPT-4o as an LLM-as-judge. To complement this, we conducted expert human evaluation on a representative subset of model outputs, with results presented in Appendix E.3 (Figure 6). Due to the high cost of human annotation, only a subset was reviewed, but the results demonstrate that **MM-Agent consistently achieves the highest performance**, particularly on the **AE** and **RBA** dimensions.\n\nFor the review process, we recruited **three expert evaluators** with prior MCM/ICM awards or extensive experience in mathematical modeling. To ensure evaluation quality, we consulted with an **official MCM/ICM judge** to provide training and establish scoring rubrics aligned with the contest’s official criteria. All outputs were **anonymized** to avoid bias, and each expert conducted the evaluations independently.\n\nTo assess reliability, we measured agreement scores **between human annotators** and **between human and model** evaluations (see Table below). The results indicate **moderate to high consistency** across most categories. While **AE** and **RBA** show slightly lower agreement, reflecting the inherent subjectivity in explanation-based evaluation as discussed in prior work \\[2], this does not undermine the observed superiority of MM-Agent.\n\n**Table: Agreement between Human and Model Evaluations**\n\n| Category                    | Human vs. Human | Model vs. Human |\n| --------------------------- | --------------- | --------------- |\n| AE (Answer Explanation)     | 0.7475          | 0.5068          |\n| MR (Modeling Reasoning)     | 0.4813          | 0.7130          |\n| PS (Problem Solving)        | 0.7890          | 0.7860          |\n| RBA (Result-Based Analysis) | 0.7625          | 0.5692          |\n\nIn future work, we plan to develop **more standardized and robust evaluation protocols** to further improve the reliability of assessing open-ended modeling outputs.\n\n[2]Researchagent: Iterative research idea generation over scientific literature with large language models\n\n> Q3: \"MM-agent helped two teams achieve Finalist awards in competition - which specific subtasks, problems, or evaluations did the top performing teams outperform the MM-agent augmented teams in? Is there a clear path to highlight and address these limitations in future work?\"\n\nThank you for the thoughtful question. We would like to kindly clarify that, due to MCM/ICM rules, AI agents are not allowed to participate independently, but human teams are permitted to use LLMs as assistance. Therefore, our MM-Agent was deployed as a copilot, collaborating with human participants during the competition. Given this setting, a direct comparison with Outstanding Award teams is not strictly objective, as final rankings also reflect subjective judging criteria such as creativity and presentation quality. We acknowledge a performance gap, particularly in modeling originality, explanation rigor, and task structuring. To address these limitations, we plan to enhance MM-Agent’s reasoning and abstraction capabilities (e.g., via reinforcement learning), and further improve its task decomposition strategies for complex, open-ended modeling.\n\n> Q4: “ Degree of MM-Agent Usage by Finalist Teams”\n\nResponse: To better understand how the two finalist teams leveraged MM-Agent, we conducted follow-up interviews and questionnaire surveys. Both teams reported using MM-Agent as a copilot for over 8 hours during the 3-day competition, with particularly high engagement in tasks such as problem analysis, task clarification, modeling guidance, workflow planning, and data visualization. They explicitly affirmed that MM-Agent played a substantial role in structuring their modeling process and accelerating exploratory analysis.\n\n\n> [W2 & Q5] W2:”HMML is included as a main contribution, but it seems like a simple tree structure representing hierarchies of modeling approaches. It’s unclear how useful or necessary this is to MM-Agent’s performance. And ”Q5: \"HMML is included as a main contribution, but it seems like a simple tree structure representing hierarchies of modeling approaches. It’s unclear how useful or necessary this is to MM-Agent’s performance.\"\n\n\nResponse: We appreciate the reviewer’s question. We emphasize that HMML is a **heterogeneous hierarchical design** that fundamentally differs from flat modeling libraries. It supports both **problem-aware** and **solution-aware** retrieval of modeling strategies, enabling more accurate abstraction and principled method selection across diverse modeling scenarios.\nTo rigorously evaluate its effectiveness, we conducted an **ablation study** by replacing HMML with a flat retrieval baseline lacking hierarchical structure (as detailed in **Figure 4 and lines 350–373**). Removing HMML led to consistent performance drops across all evaluation metrics, particularly in **Analysis Evaluation (AE)** and **Modeling Rigorousness (MR)**, demonstrating that HMML plays a **critical role in guiding structural reasoning and improving modeling quality**.\n\n\n**Table: Ablation Study**\n| Method | AE | MR | PS | RBA |\n| ----------------- | ---- | ---- | ---- | ---- |\n| w/o HMML (GPT-4o) | 8.34 | 6.47 | 8.97 | 7.75 |\n| MM-Agent (GPT-4o) | **9.09** | **7.26** | **9.00** | **8.44** |\n| w/o HMML (R1) | 8.26 | 7.70 | 9.00 | 8.36 |\n| MM-Agent (R1) | **9.53** | **8.27** | **9.10** | **8.55** |\n\n> Q6: \"Are there problem types and domains that are not currently considered in the MM-Bench evaluation that the framework could be extended to?\"\n\nResponse: Thank you for the thoughtful question. We would like to kindly clarify that MM-Bench is entirely constructed from real-world MCM/ICM problems spanning the past 25 years, ensuring authentic and diverse coverage of open-ended modeling tasks across a wide range of domains. While we believe the current benchmark offers reasonably comprehensive representation, we acknowledge that certain domains, such as geometry, education, and energy, are currently underrepresented. We view these areas as valuable extensions and plan to incorporate them in future versions of MM-Bench to further enhance its domain coverage and generalizability."}, {"review_id": "ch7LJowD9I", "reviewer": "Reviewer_1dcb", "summary": "This paper investigates the capabilities of large language models (LLMs) in tackling real-world mathematical modeling problems. Unlike traditional mathematical reasoning, which assumes a well-defined problem formulation, mathematical modeling involves: (1) translating complex scenarios described in natural language into precise mathematical formulations, and (2) solving these formulations and producing detailed technical reports. \n\nThis paper introduces MM-Bench, a curated benchmark comprising 111 problems from the Mathematical Contest in Modeling (MCM/ICM) spanning diverse domains in science, engineering, and economics.\nFurthermore, the paper proposes MM-Agent, a multi-agent LLM framework inspired by expert workflows. Experimental results demonstrate that MM-Agent outperforms baseline agents.", "questions": "The authors are encouraged to address the limitations above.", "limitations": "N/A", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. Relevance and scope: Mathematical modeling is essential in applied domains like biology, urban planning, and economics. The paper addresses a critical, underexplored challenge for LLMs: translating realistic, open-ended problems into formal models and executable solutions.\n\nWeaknesses\n1. Benchmark formulation limits novelty. The inclusion of a method-retrieval step from HMML transforms what should be an open-ended modeling challenge into a more constrained retrieval problem. This design may overestimate the LLM’s genuine modeling skill and reduce the benchmark’s ability to assess true modeling creativity and abstraction.\n\n2. Limited architectural innovation. The core MM-Agent architecture largely follows the established LLM-agent architecture. While well-engineered, it lacks unique algorithmic or architectural features that push beyond current agent paradigms.\n\n3. CIaim. Although MM-Agent’s assistance in MCM/ICM success is impressive, the manuscript does not clearly explain how much of this outcome resulted from the agent's contribution versus human creativity. Clarifying the balance of influence would strengthen the claim.", "was_revised": true, "final_justification": "I will raise the score to 4 as my concerns are mostly addressed.", "rebuttal": "### Thanks for reviewer 1dcb and global response\n\nWe sincerely thank Reviewer 1dcb for their thoughtful and constructive review. We appreciate your recognition of the **importance and relevance of mathematical modeling as a real-world LLM challenge**, as well as your acknowledgment of the **scope and potential of our MM-Bench benchmark and MM-Agent framework**.\n\nPlease find our detailed point-by-point responses below.\n\n>  W1: \"Benchmark formulation limits novelty. The inclusion of a method-retrieval step from HMML transforms what should be an open-ended modeling challenge into a more constrained retrieval problem. This design may overestimate the LLM’s genuine modeling skill and reduce the benchmark’s ability to assess true modeling creativity and abstraction.\"\n\n\nResponse:  Thank you for the thoughtful question.  **We would like to kindly clarify a potential misunderstanding: MM-Bench and MM-Agent are two distinct contributions. MM-Bench poses open-ended mathematical modeling tasks that require full end-to-end solutions, not constrained retrieval problems, and the HMML module is part of MM-Agent, not the benchmark.**\n\n\n**MM-Bench is constructed from 25 years of real-world MCM/ICM problems**, which are widely recognized for evaluating **open-ended mathematical modeling skills**. Each task in MM-Bench requires solving an ill-posed, real-world problem through a **complete end-to-end modeling pipeline**, including problem analysis, model formulation, computational solving, and result interpretation. It does **not reduce the task to a constrained retrieval problem**, but instead preserves the full scope and complexity of open-ended modeling challenges.\n\n\nTo address these open-ended challenges, we propose **MM-Agent**, an expert-inspired framework that decomposes mathematical modeling into four stages: **open-ended problem analysis, structured model formulation, computational problem solving, and report generation**. The **HMML module is a component of MM-Agent**, It provides **abstract modeling schemas** to guide agent modeling. Within this pipeline, the modeling stage is especially challenging, requiring the abstraction of complex real-world problems into mathematically coherent formulations. To address this, MM-Agent integrates the Hierarchical Mathematical Modeling Library (HMML), a tri-level schema of 98 abstract modeling strategies, which supports problem- and solution-aware retrieval, principled method selection, and refinement via a critic-actor mechanism. \n\n\n> W2: \"Limited architectural innovation. The core MM-Agent architecture largely follows the established LLM-agent architecture. While well-engineered, it lacks unique algorithmic or architectural features that push beyond current agent paradigms.\"\n\n\nWe would like to kindly clarify that **MM-Agent is not derived from existing LLM-agent paradigms**, but is instead a **purpose-built framework** specifically designed to tackle the unique challenges of real-world mathematical modeling. To the best of our knowledge, it is the **first LLM-agent system systematically developed for this domain**, which demands structured abstraction, rigorous formulation, and context-grounded reasoning.\n\nMM-Agent adopts a **four-stage expert-inspired pipeline**, spanning problem analysis, model formulation, computational solving, and report generation. Central to this framework is the **Hierarchical Mathematical Modeling Library (HMML)**, a tri-level knowledge hierarchy that enables **problem-aware and solution-aware retrieval** of abstract modeling schemas. This design supports principled abstraction, constraint reasoning, and methodological alignment, distinguishing MM-Agent from prior agent architectures.\n\nAs recognized by Reviewers **geD1** and **3X2K**, MM-Agent demonstrates both **strong empirical performance in MCM/ICM competitions** and **architectural innovation**, contributing a novel and practically useful agentic system for complex, open-ended modeling tasks.\n\n\n\n> W3: \"CIaim. Although MM-Agent’s assistance in MCM/ICM success is impressive, the manuscript does not clearly explain how much of this outcome resulted from the agent's contribution versus human creativity. Clarifying the balance of influence would strengthen the claim.\"\n\n\nThank you for the thoughtful question. Due to privacy requests from the participating teams, we did not disclose detailed submission content in the paper. During the contest, MM-Agent was deployed strictly as a copilot, collaborating with human participants in compliance with MCM/ICM rules (which prohibit AI-only participation but allow LLM-assisted modeling).\n\nFollowing the competition, we conducted interviews and surveys with the two finalist teams. Both reported using MM-Agent for over 8 hours during the 3-day contest, with particularly high engagement in problem analysis, task clarification, modeling guidance, workflow planning, and data visualization. They explicitly affirmed that MM-Agent played a substantial role in structuring their modeling pipeline and accelerating exploratory analysis, underscoring its meaningful contribution to their success."}, {"review_id": "jGZrC7fOGT", "reviewer": "Reviewer_badD", "summary": "This paper presents MM-Agent, an expert-inspired framework designed to address real-world mathematical modeling problems by leveraging Large Language Models (LLMs). The authors formalize the task of LLM-powered mathematical modeling, which involves analyzing complex real-world problems, constructing domain-appropriate formulations, and generating end-to-end solutions. They introduce MM-Bench, a benchmark comprising 111 problems from the Mathematical Contest in Modeling (MCM/ICM) spanning 2000 to 2025, covering ten diverse domains such as physics, biology, and economics. The paper also proposes the Hierarchical Mathematical Modeling Library (HMML), a three-tiered knowledge hierarchy that encodes 98 high-level modeling schemas to support structured method selection and abstraction. MM-Agent decomposes the modeling process into four stages: problem analysis, model formulation, computational problem-solving, and report generation. Experimental results demonstrate that MM-Agent significantly outperforms baseline agents, achieving an 11.88% improvement over human expert solutions while requiring only 15 minutes and $0.88 per task using GPT-4o. Furthermore, MM-Agent assisted two undergraduate teams in winning the Finalist Award (top 2.0% among 27,456 teams) in the MCM/ICM 2025 competition.", "questions": "Only one question - The evaluation for agent results is not clear in lines 149-163,  we cannot know the score of Analysis Evaluation, Modeling Rigorousness,  Practicality and Scientificity, Result and Bias Analysis is reliable or not? For me, it's hard to judge that a higher score will mean a better result, It might just seem to have a good effect. For such problems, what seems good may be quite a distance from being truly correct.", "limitations": "The main concern is:\n- Novelty\n- The reliability of the result evaluation", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\n- The authors have curated a comprehensive benchmark of 111 real-world mathematical modeling problems from the MCM/ICM competition.\n- The experimental results demonstrate that MM-Agent significantly outperforms baseline agents and achieves competitive performance compared to human expert solutions. \n\n\nWeaknesses\n- Yes, I agree the motivation is clear and the paper is well written, but my concern is - To handle real-world mathematical modeling problems, the pipeline of  unstructured problem descriptions, structured mathematical models, solutions, and the generation of analytical reports is not novel. The novelty is limited, it's not the first work to address mathematical modeling  in this domain.", "was_revised": true, "final_justification": "My concerns are mostly addressed.", "rebuttal": "### Thanks for Reviewer badD and global response\n\nWe sincerely thank Reviewer badD for their thoughtful comments and constructive suggestions. We appreciate your recognition of the **significance of our work, especially the comprehensive design of MM-Bench and the strong empirical results achieved by MM-Agent.**  Mathematical modeling, unlike traditional mathematical reasoning, requires open-ended abstraction, assumption design, and domain-specific formulation to translate complex real-world problems into formal systems, making it uniquely challenging and largely underexplored in LLM research. To address this, we introduce MM-Bench, **the first benchmark built from 25 years of MCM/ICM problems, and MM-Agent, a purpose-built agentic framework tailored to modeling tasks. As acknowledged by Reviewers geD1 and 3X2K, our contributions lie in both novel methodology and strong empirical validation， including MM-Agent’s real-world success in MCM/ICM competitions.**\n\nWe again thank the reviewer for their feedback. Please find our detailed point-by-point responses below.\n\n>  W1: \"To handle real-world mathematical modeling problems, the pipeline of unstructured problem descriptions, structured mathematical models, solutions, and the generation of analytical reports is not novel. The novelty is limited, it's not the first work to address mathematical modeling in this domain.\"\n\nResponse: We appreciate the reviewer’s observation and would like to clarify a key distinction. While prior work may be **labeled under \"mathematical modeling,\"** they fundamentally focus on **solving well-defined mathematical problems, such as optimization tasks or differential equations, where the problem formulation is already given.** For example, Huang et al. [1] and Zhang and Luo [2] demonstrate the use of LLMs to solve ODEs or linear programs, assuming the mathematical structure is pre-specified.\n\nIn contrast, our work tackles **open-ended and real-world mathematical modeling**, where the central challenge lies not in solving a predefined problem, but in formulating the problem itself. This involves interpreting ambiguous natural language, abstracting key variables, and selecting appropriate mathematical representations, steps that precede any reasoning or solution process. As acknowledged by **Reviewers geD1 and 3X2K, our contribution lies in both the carefully designed benchmarking with strong empirical performance (e.g., in MCM/ICM competitions), and the development of a novel LLM-based agent framework.** MM-Agent bridges the gap between unstructured problem descriptions and formal mathematical models,an underexplored and fundamentally different challenge from conventional mathematical reasoning tasks.\n\n[1]  LLMs for Mathematical Modeling: Towards Bridging the Gap between Natural and Mathematical Languages\n\n[2] Or-llm-agent: Automating modeling and solving of operations research optimization problem with reasoning large language model\n\n> Q1: \"The evaluation for agent results is not clear in lines 149-163, we cannot know the score of Analysis Evaluation, Modeling Rigorousness, Practicality and Scientificity, Result and Bias Analysis is reliable or not? For me, it's hard to judge that a higher score will mean a better result, It might just seem to have a good effect. For such problems, what seems good may be quite a distance from being truly correct.\"\n\nResponse:  We strongly agree with the reviewer that evaluating open-ended problems is inherently challenging, even for human experts, precisely because these tasks, unlike well-formulated mathematical problems, lack clear ground-truth answers or objective correctness criteria. Therefore, to address this issue, we follow a dual evaluation strategy which is widely used in **AI for scientific discovery** \\[3,4]: (1) **LLM-as-judge evaluation** using GPT-4o, and (2) **Human expert evaluation** grounded in domain knowledge. Under both evaluation settings, **MM-Agent consistently outperformed all baselines**, demonstrating its superior capability in handling open-ended modeling tasks. Furthermore, under official MCM/ICM evaluation protocols, MM-Agent assisted two undergraduate teams in **winning the Finalist Award (top 2.0% out of 27,456 teams)** in MCM/ICM 2025, providing compelling evidence of its practical effectiveness as a modeling copilot.\n\nFor the LLM-as-judge evaluation, we carefully followed the official ICM/MCM scoring rubrics and assessed agent responses across four key dimensions: **Analysis Evaluation (AE)**, **Modeling Rigorousness (MR)**, **Practicality and Scientificity (PS)**, and **Result and Bias Analysis (RBA)**. The specific evaluation prompts for each dimension are provided in Figures 7–10 of Appendix F, ensuring transparency and reproducibility.\n\nFor the human evaluation, we recruited **three expert annotators** with prior MCM/ICM awards or substantial mathematical modeling experience. To ensure reliability, we consulted with an **official ICM/MCM judge** to establish standardized scoring rubrics and conducted training. All model outputs were **anonymized** and evaluated independently, as illustrated in Figure 6 of Appendix E.3.\n\nWe further assess the **consistency and reliability** of our evaluation protocol, we computed agreement scores between human annotators as well as between human and model evaluations:\n**Table: Agreement between Human and Model Evaluations**\n\n| Category                    | Human vs. Human | Model vs. Human |\n| --------------------------- | --------------- | --------------- |\n| AE (Answer Explanation)     | 0.7475          | 0.5068          |\n| MR (Modeling Reasoning)     | 0.4813          | 0.7130          |\n| PS (Problem Solving)        | 0.7890          | 0.7860          |\n| RBA (Result-Based Analysis) | 0.7625          | 0.5692          |\n\nAs shown in Table 6 of Appendix E.3, we observe **moderate to high agreement**, especially in core modeling and problem-solving aspects. Categories such as AE and RBA exhibit slightly lower agreement due to their inherently subjective and explanation-based nature, as discussed in prior work \\[3].\n\nWe acknowledge that **designing robust and interpretable evaluation frameworks remains an open challenge in AI-driven scientific discovery**, particularly for complex modeling tasks. We consider this a crucial direction for future research and are actively exploring more rigorous, hybrid evaluation protocols to improve assessment fidelity.\n\n[3] ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models\n\n[4] Agent Laboratory: Using LLM Agents as Research Assistants"}, {"review_id": "CON746zdVg", "reviewer": "Reviewer_3X2K", "summary": "This paper introduces MM-Agent, a framework leveraging LLMs to automate real-world mathematical modeling tasks. The authors formalize the task of LLM-powered mathematical modeling and propose MM-Bench, a benchmark comprising 111 diverse problems from the Mathematical Contest in Modeling (MCM/ICM) spanning 2000–2025. MM-Agent decomposes the modeling process into four stages: problem analysis, structured model formulation, computational problem solving, and report generation. It incorporates the Hierarchical Mathematical Modeling Library (HMML) to facilitate method retrieval and an actor-critic mechanism for iterative optimization. Experiments show that MM-Agent outperforms baseline agents, achieving an 11.88% improvement over human expert solutions while maintaining cost efficiency.", "questions": "1. Observing Figure 9, there are few chartss included in the results generated by LLM. Does the author have any comments on this?\n2. In my personal opinion, due to the limited competition time, the length and presentation quality of modeling competition papers may significantly influence human evaluations. What are the advantages of content generated by LLMs or content assisted by LLMs in these two aspects?\n3. When describing the proposed method, I suggest that more comparisons and discussions be conducted with existing methods to clearly define the incremental contribution of this work. From the existing text, I do not clearly understand the relationship between this work and ResearchAgent or Agent Laboratory.\n4. How would the entries from modeling competitions that have won various awards be scored in the rating system of this paper? I want to know how the model performs without human intervention compared to human contestants.", "limitations": "It seems there is no dedicated paragraph discussing limitations.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n+ The authors submitted the code and a detailed appendix.\n+ This work provides a system for solving problems and also designs a new benchmark.\n\nWeaknesses\n- The paper claims that it \"helped two undergraduate teams win the Finalist Award,\" but I didn't see more details in the main text or appendix.\n- Regarding the overall presentation of the paper, my impression after reading it is that there are a lot of design details, but the conveyed insights are quite limited. Personally, I don't particularly appreciate the writing style of this paper.\n- I have some points that I don't understand, as detailed in the following \"Questions\".", "was_revised": false, "final_justification": "", "rebuttal": "### Thanks for reviewer 3X2K and global response\n\nWe sincerely thank Reviewer 3X2K for your constructive feedback and thoughtful questions. We greatly appreciate your recognition that our work provides **both a practical system (MM-Agent) and a new benchmark (MM-Bench)** for advancing research in LLM-powered mathematical modeling. Your comments have helped us better articulate our motivation, design contributions, and empirical validation.\n\nPlease find our point-by-point responses below.\n\n\n\n> W1: \"The paper claims that it \"helped two undergraduate teams win the Finalist Award,\" but I didn't see more details in the main text or appendix.\"\n\nResponse:  We thank the reviewer for pointing this out and apologize for the lack of details in the original submission. Due to privacy concerns and competition rules, we are unable to disclose identifying information about the two participating teams.\nWe would like to kindly clarify that under official MCM/ICM guidelines, AI agents are not permitted to participate independently; however, human teams are allowed to use LLMs as assistants. In this context, our MM-Agent was deployed as a copilot, working collaboratively with two undergraduate teams during the competition.\n\nTo better understand how MM-Agent supported their efforts, we conducted follow-up interviews and questionnaires after the competition. **Both finalist teams reported using MM-Agent for over 8 hours across the 3-day contest, particularly for tasks such as problem analysis, task clarification, modeling guidance, workflow planning, and data visualization.** They explicitly acknowledged that MM-Agent played a substantial role in structuring their modeling process and accelerating exploratory analysis, ultimately contributing to their success.\n\n> W2: \"Regarding the overall presentation of the paper, my impression after reading it is that there are a lot of design details, but the conveyed insights are quite limited. Personally, I don't particularly appreciate the writing style of this paper.\"\n\nResponse: We sincerely appreciate the reviewer’s comments on the writing and overall presentation. In the next revision, we will refine the writing style to improve readability and ensure that the core insights of our work are conveyed more clearly.\n\n> Q1: \"Observing Figure 9, there are few chartss included in the results generated by LLM. Does the author have any comments on this?\"\n\nResponse: We appreciate the reviewer’s observation regarding the use of charts in the generated outputs (e.g., Figure 9). As our MM-Agent is designed as an end-to-end autonomous agent, we observed that in problems not strongly data-driven, the agent tends to focus more on analytical reasoning, modeling abstraction, and textual explanation rather than generating visual outputs such as charts. This behavior aligns with the agent's prioritization of problem structuring and formal reasoning in the absence of explicit numerical datasets.\n\nWhen MM-Agent is used in a human-in-the-loop copilot setting, users can explicitly prompt the agent to generate charts or visual summaries, which substantially improves the visual component of the solution. In fact, in the MCM/ICM deployment, participating teams reported actively leveraging MM-Agent for data visualization when appropriate. We consider enhancing chart-awareness and integrating adaptive visualization generation as a promising direction for future agent refinement.\n\n\n> Q2: \"In my personal opinion, due to the limited competition time, the length and presentation quality of modeling competition papers may significantly influence human evaluations. What are the advantages of content generated by LLMs or content assisted by LLMs in these two aspects?\"\n\nResponse: We agree with the reviewer that the length and presentation quality of modeling competition papers may influence human evaluations. However, in real-world competitions such as MCM/ICM, the maximum paper length is strictly limited to 25 pages, which helps mitigate the influence of paper length on evaluation outcomes.\n\nRegarding presentation quality, LLM-generated content may indeed demonstrate advantages in aspects such as grammar and fluency. That said, we would like to clarify that, according to the official rules of these competitions, LLMs are not allowed to participate independently, and any use of AI tools must be explicitly disclosed. We only provide LLM-generated solutions as a reference for participating teams. These teams are required to adapt and disclose their use of LLMs as per the competition guidelines. Therefore, while LLMs may contribute to improved presentation quality, their actual impact remains limited under current competition regulations.\n\n> Q3: \"When describing the proposed method, I suggest that more comparisons and discussions be conducted with existing methods to clearly define the incremental contribution of this work. From the existing text, I do not clearly understand the relationship between this work and ResearchAgent or Agent Laboratory.\"\n\nResponse: We thank the reviewer for the insightful comments.  To the best of our knowledge, MM-Agent is the first agentic framework systematically applied to real-world mathematical modeling tasks, a domain requiring structured abstraction, formal method selection, and computational reasoning. We appreciate the opportunity to clarify its distinction from existing systems like ResearchAgent and Agent Laboratory. **While ResearchAgent and Agent Laboratory primarily focus on research ideation or general scientific workflows**, **MM-Agent is specifically designed for end-to-end mathematical modeling and computational problem solving. It uniquely supports structured problem analysis, hierarchical modeling method selection via our HMML with a critic-actor mechanism.**  Importantly, MM-Agent introduces a formal modeling scheme selection mechanism that maps sub-problems to domain-specific mathematical methods (e.g., linear programming, quadratic programming), a unique contribution that is not addressed in the above works. These capabilities, particularly the formal modeling scheme selection and execution of structured optimization workflows, are not addressed in prior agent systems.\n\n> Q4: \"How would the entries from modeling competitions that have won various awards be scored in the rating system of this paper? I want to know how the model performs without human intervention compared to human contestants.\"\n\n\nResponse： Thank you for the insightful question. **We would like to kindly clarify that we included real-world award-winning solutions (Honorable Mention or above) from past MCM/ICM competitions as a baseline in our evaluation (see **Human Team** in Table 1, Lines 324–343).** These entries serve as strong human references for assessing model performance. Results show that while human teams remain strong competitors, often outperforming most LLM-based agents, **MM-Agent achieves near-human performance**, and even surpasses human teams on certain metrics. This underscores both the inherent difficulty of the task and the effectiveness of MM-Agent in open-ended modeling.\n\n> L1 “It seems there is no dedicated paragraph discussing limitations.”\n\nWe thank the reviewer for the suggestion. The limitations of our work are discussed in the Broader Impacts section (see lines 979–1020). We will make this discussion more explicit in the next revision for clarity."}]

Author rebuttal:

Meta-review: This paper discusses the development of a system to perform mathematical modelling. The paper has garnered excitement and appreciation from the reviewers. In a long iterative back-and-forth, I believe the authors have responded to most of the comments and concerns the reviewers had. I looked at the paper to find better validation of the statement in the abstract: "Furthermore, under official MCM/ICM protocols, MM-Agent assisted two undergraduate teams in winning the Finalist Award (\textbf{top 2.0% among 27,456 teams}) in MCM/ICM 2025, demonstrating its practical effectiveness as a modeling copilot." and couldn't quite find this validation - to what extent and how the system helped the team. It is a strong claim, so if it is indeed the case, it would be good to base it.
Decision: Accept (poster)