Title: MM-Agent: LLM as Agents for Real-world Mathematical Modeling Problem
Venue: NeurIPS 2025
Accepted: yes
Keywords: Mathematical Modeling Agent

LLM Agent

LLM
Abstract: Mathematical modeling is a cornerstone of scientific discovery and engineering practice, enabling the translation of real-world problems into formal systems across domains such as physics, biology, and economics. Unlike mathematical reasoning, which assumes a predefined formulation, modeling requires open-ended problem analysis, abstraction, and principled formalization. While Large Language Models (LLMs) have shown strong reasoning capabilities, they fall short in rigorous model construction, limiting their utility in real-world problem-solving. To this end, we formalize the task of LLM-powered real-world mathematical modeling, where agents must analyze problems, construct domain-appropriate formulations, and generate complete end-to-end solutions.We introduce MM-Bench, a curated benchmark of 111 problems from the Mathematical Contest in Modeling (MCM/ICM), spanning the years 2000 to 2025 and across ten diverse domains such as physics, biology, and economics.  To tackle this task, we propose MM-Agent, an expert-inspired framework that decomposes mathematical modeling into four stages: open-ended problem analysis, structured model formulation, computational problem solving, and report generation.Experiments on MM-Bench show that MM-Agent significantly outperforms baseline agents, achieving an 11.88\% improvement over human expert solutions while requiring only 15 minutes and \$0.88 per task using GPT-4o. Furthermore, under official MCM/ICM protocols, MM-Agent assisted two undergraduate teams in winning the Finalist Award (\textbf{top 2.0\% among 27,456 teams}) in MCM/ICM 2025, demonstrating its practical effectiveness as a modeling copilot.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces MM-Agent, a framework leveraging LLMs to automate real-world mathematical modeling tasks. The authors formalize the task of LLM-powered mathematical modeling and propose MM-Bench, a benchmark comprising 111 diverse problems from the Mathematical Contest in Modeling (MCM/ICM) spanning 2000–2025. MM-Agent decomposes the modeling process into four stages: problem analysis, structured model formulation, computational problem solving, and report generation. It incorporates the Hierarchical Mathematical Modeling Library (HMML) to facilitate method retrieval and an actor-critic mechanism for iterative optimization. Experiments show that MM-Agent outperforms baseline agents, achieving an 11.88% improvement over human expert solutions while maintaining cost efficiency.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n+ The authors submitted the code and a detailed appendix.\n+ This work provides a system for solving problems and also designs a new benchmark.\n\nWeaknesses\n- The paper claims that it \"helped two undergraduate teams win the Finalist Award,\" but I didn't see more details in the main text or appendix.\n- Regarding the overall presentation of the paper, my impression after reading it is that there are a lot of design details, but the conveyed insights are quite limited. Personally, I don't particularly appreciate the writing style of this paper.\n- I have some points that I don't understand, as detailed in the following \"Questions\".\n\nQUESTIONS:\n1. Observing Figure 9, there are few chartss included in the results generated by LLM. Does the author have any comments on this?\n2. In my personal opinion, due to the limited competition time, the length and presentation quality of modeling competition papers may significantly influence human evaluations. What are the advantages of content generated by LLMs or content assisted by LLMs in these two aspects?\n3. When describing the proposed method, I suggest that more comparisons and discussions be conducted with existing methods to clearly define the incremental contribution of this work. From the existing text, I do not clearly understand the relationship between this work and ResearchAgent or Agent Laboratory.\n4. How would the entries from modeling competitions that have won various awards be scored in the rating system of this paper? I want to know how the model performs without human intervention compared to human contestants.\n\nLIMITATIONS:\nIt seems there is no dedicated paragraph discussing limitations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents MM-Agent, an expert-inspired framework designed to address real-world mathematical modeling problems by leveraging Large Language Models (LLMs). The authors formalize the task of LLM-powered mathematical modeling, which involves analyzing complex real-world problems, constructing domain-appropriate formulations, and generating end-to-end solutions. They introduce MM-Bench, a benchmark comprising 111 problems from the Mathematical Contest in Modeling (MCM/ICM) spanning 2000 to 2025, covering ten diverse domains such as physics, biology, and economics. The paper also proposes the Hierarchical Mathematical Modeling Library (HMML), a three-tiered knowledge hierarchy that encodes 98 high-level modeling schemas to support structured method selection and abstraction. MM-Agent decomposes the modeling process into four stages: problem analysis, model formulation, computational problem-solving, and report generation. Experimental results demonstrate that MM-Agent significantly outperforms baseline agents, achieving an 11.88% improvement over human expert solutions while requiring only 15 minutes and $0.88 per task using GPT-4o. Furthermore, MM-Agent assisted two undergraduate teams in winning the Finalist Award (top 2.0% among 27,456 teams) in the MCM/ICM 2025 competition.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- The authors have curated a comprehensive benchmark of 111 real-world mathematical modeling problems from the MCM/ICM competition.\n- The experimental results demonstrate that MM-Agent significantly outperforms baseline agents and achieves competitive performance compared to human expert solutions. \n\n\nWeaknesses\n- Yes, I agree the motivation is clear and the paper is well written, but my concern is - To handle real-world mathematical modeling problems, the pipeline of  unstructured problem descriptions, structured mathematical models, solutions, and the generation of analytical reports is not novel. The novelty is limited, it's not the first work to address mathematical modeling  in this domain.\n\nQUESTIONS:\nOnly one question - The evaluation for agent results is not clear in lines 149-163,  we cannot know the score of Analysis Evaluation, Modeling Rigorousness,  Practicality and Scientificity, Result and Bias Analysis is reliable or not? For me, it's hard to judge that a higher score will mean a better result, It might just seem to have a good effect. For such problems, what seems good may be quite a distance from being truly correct.\n\nLIMITATIONS:\nThe main concern is:\n- Novelty\n- The reliability of the result evaluation", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the capabilities of large language models (LLMs) in tackling real-world mathematical modeling problems. Unlike traditional mathematical reasoning, which assumes a well-defined problem formulation, mathematical modeling involves: (1) translating complex scenarios described in natural language into precise mathematical formulations, and (2) solving these formulations and producing detailed technical reports. \n\nThis paper introduces MM-Bench, a curated benchmark comprising 111 problems from the Mathematical Contest in Modeling (MCM/ICM) spanning diverse domains in science, engineering, and economics.\nFurthermore, the paper proposes MM-Agent, a multi-agent LLM framework inspired by expert workflows. Experimental results demonstrate that MM-Agent outperforms baseline agents.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. Relevance and scope: Mathematical modeling is essential in applied domains like biology, urban planning, and economics. The paper addresses a critical, underexplored challenge for LLMs: translating realistic, open-ended problems into formal models and executable solutions.\n\nWeaknesses\n1. Benchmark formulation limits novelty. The inclusion of a method-retrieval step from HMML transforms what should be an open-ended modeling challenge into a more constrained retrieval problem. This design may overestimate the LLM’s genuine modeling skill and reduce the benchmark’s ability to assess true modeling creativity and abstraction.\n\n2. Limited architectural innovation. The core MM-Agent architecture largely follows the established LLM-agent architecture. While well-engineered, it lacks unique algorithmic or architectural features that push beyond current agent paradigms.\n\n3. CIaim. Although MM-Agent’s assistance in MCM/ICM success is impressive, the manuscript does not clearly explain how much of this outcome resulted from the agent's contribution versus human creativity. Clarifying the balance of influence would strengthen the claim.\n\nQUESTIONS:\nThe authors are encouraged to address the limitations above.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors introduce an agentic system for open-ended mathematical modeling. Rather than rely on zero-shot LLM capabilities, they break down modeling problems into problem framing, model formulation, solution development, and interpreting and preparing results. By using temporal splits, LLM-as-judge and human expert review, and experiments on optimization problems with ground truth answers, they show that their system outperforms other agentic systems for mathematical modeling.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\nThe approach and research objectives are clearly laid out.\nThe token consumption results are an interesting evaluation, showing that MM-Agent is not simply consuming more tokens than other methods.\nThe “real-world” applicability of MM-agent as used in the MCM/ICM competitions could be compelling evidence of practical utility.\n\nWeaknesses\nA few points could benefit from increased clarity: the usefulness and rigor of the temporal split as an approximately “OOD” evaluation, the delta in performance between top-performing human experts and teams using MM-agent as a modeling copilot, and the exact use cases and utility of MM-agent to the finalist teams.\nHMML is included as a main contribution, but it seems like a simple tree structure representing hierarchies of modeling approaches. It’s unclear how useful or necessary this is to MM-Agent’s performance.\n\nQUESTIONS:\nThe authors rely on the temporal split (2021-2024) vs (2025) as evidence that the underlying LLMs are performing “genuine modeling” rather than memorization. Are there any metrics of problem similarity between the splits from year to year that could be used to indicate how different the problem statements are?\nCan the authors expand on the results in Table 1 - which metrics are from automated scoring from GPT-4o vs human expert review? How many expert reviewers were consulted and how was expert review performed?\nMM-agent helped two teams achieve Finalist awards in competition - which specific subtasks, problems, or evaluations did the top performing teams outperform the MM-agent augmented teams in? Is there a clear path to highlight and address these limitations in future work?\nTo what degree did the two finalist teams rely on MM-agent? \nAre there problem types and domains that are not currently considered in the MM-Bench evaluation that the framework could be extended to?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}