MURKA: Multi-Reward Reinforcement Learning with Knowledge Alignment for Optimization Tasks
Authors: Wantong Xie
Yi-Xiang Hu
Jieyang Xu
Feng Wu
Xiangyang Li
Venue: NeurIPS 2025 poster ()
Keywords: Large Language Models
Reinforcement Learning
Optimization Modeling
Multi-Agent Systems
Knowledge Distillation
Primary area: deep_learning

Abstract: Optimization plays a central role in Operations Research (OR) and numerous industrial applications, yet automating the end-to-end process of translating natural language descriptions into executable optimization programs remains a formidable challenge. While recent efforts have applied Large Language Models (LLMs) to this task, existing approaches are hindered by high inference costs, limited robustness across domains, and weak verification mechanisms. In this work, we propose MURKA, a reinforcement learning and knowledge distillation-based framework that enhances LLM-driven optimization modeling via collaborative agent alignment. MURKA orchestrates three specialized agents---Extractor, Solver, and Checker---to achieve accurate problem understanding, robust formulation, and verifiable execution. The Extractor is trained using group relative policy optimization with a composite reward function that incorporates semantic correctness and execution fidelity. The Solver benefits from knowledge distillation from a powerful teacher model, yielding structurally valid and executable formulations in AMPL. The Checker iteratively verifies solution correctness via solver feedback. 
We validate MURKA's generalizability through extensive experiments across diverse OR benchmarks, demonstrating its robustness and scalability.
Experimental results on eight diverse OR benchmarks, including NLP4LP, ComplexOR, and NL4Opt, demonstrate that MURKA, built on the LLaMa3-8B backbone, achieves a 5.9\% absolute improvement in solution accuracy and a 5.1\% increase in execution success rate compared to leading baselines. These results establish MURKA as an effective and scalable paradigm for LLM-driven optimization, with strong potential for deployment in real-world OR applications.

MURKA: Multi-Reward Reinforcement Learning with Knowledge Alignment for Optimization Tasks
Wantong Xie', Yi-Xiang Hu², Jieyang Xu², Feng Wu'; Xiang-Yang Li
Institute of Advanced Technology. University of Science and Technology of China School of Computer Science and Technology. University of Science and Technology of China {wantongxie, peattonnxieexxxiinnniiie jienagguieeyyygnuuaaamaa. @mail ustc.edu. cn (wufeng02, (iangyanggl) -ustt. edu. cn
Abstract
Optimization plays central role in Operations Research (OR) and numerous industrial applications, yet automating the end-to-end process of translating natural language descriptions into executable optimization programs remains formidable challenge. While recent efforts have applied Large Language Mod- els (LLMs) this task, existing approaches hindered by high inference costs, limited robustness across domains, and weak verification mechanisms. work, we propose MURKA, reinforcement learning and knowledge distillation- based framework that enhances LLM-driven optimization modeling via collabora- tive agent alignment. MURKA orchestrates three specialized agents-Exttratoor Solver, and Checker-to achieve accurate problem understanding, robust formula- tion, and verifiable execution. The Extractor is trained using group relative policy optimization with composite reward function that incorporates semantic correct- ness and execution fidelity. The Solver benefits from knowledge distillation from a powerful teacher model, yielding structurally valid and executable formulations in AMPI. The Checker iteratively verifies solution correctness via solver feedback. We validate MURKA's generalizability through extensive experiments across di- verse OR benchmarks, demonstrating its robustness and scalability. Experimental results on eight diverse OR benchmarks, including NLP4LP, ComplexOR, and NL40pt, demonstrate that MURKA, built on the LLaMa3-8B backbone, achieves a 5.9% absolute improvement in solution accuracy and 5.1% 5.1% increase in execu- tion success rate compared to leading basclines. These results establish MURKA effective and scalable paradigm for LLM-driven optimization, with strong potential for deployment in real-world OR applications.
this
Introduction
Operations Research (OR) has long served as a cornerstone for solving complex decision-making problems across domains such defense, logistics, supply chain management, and business oper- ations [Trimborn et al., 2020]. However, the traditional OR modeling pipeline is notoriously labor- intensive. requiring substantial expertise to manually translate real-world objectives into structured mathematical formulations. This expert-driven workflow presents significant scalability challenges, especially in fast-evolving industrial environments.
With the rapid advancement of Large Language Models (LLMs), exemplified by ChatGPT- 4 [Achiam et al., 2023] and DeepSeek-RI [Guo et al., 2025], new paradigm has emerged: using LLMs to automate the translation of natural language descriptions into optimization models. Recent
Corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

studies [Xiao et al., 2023, Li et al., 2023, AhmadiTeshhizi et al., 2024, Zhang et al., 2024, Huang al., 2025] have demonstrated the potential of LLMs to reduce expert dependency and accelerate OR modeling. These methods can be broadly categorized into (1) prompt-based approaches, which rely on carefully engineered prompts to guide L.L,s, and (2) learning- -aaee approaches, which fine-tune models via supervised learning or Reinforcement Learning (RL).
Despite their promise, the deployment of such methods in real-world scenarios remains limited. Three critical challenges continue to impede the practical adoption of LLMs for OR modeling: (CI) Scalability and latency- Large models impose high computational costs. making real-time or edge deployment infeasible, particularly in industrial systems requiring low-latency decisions; (C2) Se- mantic extraction and modeling accuracy Existing approaches struggle balance modeling cor- rectness and efficiency. Manual annotation is expensive and non-scalable, while automatic extraction methods often suffer from semantic drift and ambiguity; (C3) Cross-domain generalization Many LLM--basd methods exhibit limited adaptability across heterogeneous optimization domains, such as manufacturing, energy, and logistics.
To address these challenges, we propose MURKA, a collaborative multi-agent framework that aligns LLMs with optimization tasks via RL. MURKA orchestrates three specialized agents- Extractor, Solver, and Checker-to systematically convert natural language problem descriptions into verifiable optimization solutions through an end-to-end, multi-stage pipeline. Our contributions can be summarized as follows:
Modular multi-agent framework. To address (C1), we design a collaborative multi-agent frame- work that decomposes the modeling process into information extraction, model generation, and verification stages, enhancing efficiency, modularity, and interpretability.
Combined multi-reward RL strategy. To tackle (C2), propose composite reward function that integrates format validation, constraint checking, semantic analysis, and similarity checking.
Cross-domain alignment via knowledge distillation. To mitigate (C3), we distill reasoning patterns from high-capacity teacher model into compact solver model, producing executable AMPL code with improved structural and functional accuracy.
Substantial improvements over existing methods. MURKA achieves state-of-the-art perfor- mance on eight diverse OR benchmarks, including NLP4LP, ComplexOR, and NL.Opt, with 5.9% relative improvement in solution accuracy and a 5.1% increase in execution success rate compared to previous methods.
Together, these innovations position MURKA robust, scalable, and generalizable solution for LL.M-driven optimization modeling, with strong applicability to broad spectrum of real-world OR tasks.
2
MURKA Framework
2.1 Overview
Given optimization problem described in natural language Q, our goal is to produce its optimal solution A, as shown in Figure 1. We realize through a multi-agent framework, MURKA, which can be formalized automatic four-stage description-to--ooution pipeline:
S
Optimizer
À C A,
I
(I)
where M typically takes the linear form
min Z
Σ
s.t.
Σgje Sc,, Vj E J
(2)
iET
YE!
but the framework generalises to mixed-integer and non-linear cases.
1. Information Extraction (E). An Extractor agent parses Q and outputs structured tuple (S, P. P)V,C,O) containing sets, parameters, decision variables, constraints, and the objective function.

---

Humam: Both sulfate and ginger need added to shampoe. One unit sulfate takes minutesito to effective while one of ginger takes 0.75 minutes to be effective. The shampoo must confain least 100 units of sulfates and tufal of 400 units both ingredient. Since too much sulfate can damage the hair, there can he at most twice the amount sulfate as ginger in the shampon. How many unils of should added to the shampoo minimise the total amount of time it for mixture be effective? (Note: one must be ded before the other)
Stets> Parameterr
</Sett>> <Parametors>
sulfate effective (0.5 minutes) -E::: Time for one unit of ginger to be effective (0.75 minntess
Time
TimePerUnitSalfate 0.5 minutes
minutes
MinnalffateUiits: minimum units of sulfate required 100
<Paaramcters> <Variables>
-otallinii:: total units both ingredients required = 400
-X_s: Number of units sulfates added
Max CaanSaaffaeediinngnRetiie -2
X X_g: Number units ginger added
</Parameters> <Variables>
</Varíables> <Objective>
SulfateUnits: units of sulfate to be added
Minimize: 0,5 x_5 0_7
GingerUnits:: units of ginger to be added
</Objective> :Constraints>
</Vrriables> <bjjettiv>>
-x_s>- 100: Sulfate must he ut least 100 units
Minimize: T TimePerUnllGingee GingerUniis
-x =0400 400: Total amounnt of both ingredients 400 -
~/hjective :Constraints
Ampl Compile Error
("variables": ("SelfateUnihh" 266.0, "GingerUaits: 134.0) Succeed
Failed
''objective"; 233.5)
The impact of information quality on the solution results. The color purple represents parameter, and violet represents a variable. The left side extracts low-quality information, leading to incorrect results, while the right side extracts high-quality information, resulting in correct out- comes.
Figure 1:
Code Generation (S). A Solver agent transforms Q and I into an executable AMPL model M [Fourer et al.. 2003], which links the problem's mathematical formulation to an external opti- mization engine.
3. Numerical Optimisation. The generated code calls the optimizer, such as Gurobi (Gurobi Opti- mization, LLC, 2025] solver, to obtain a candidate solution Á together with diagnostic informa- tion (status, duals, objective value).
4. Iterative Verification (C). A Checker agent validates Á against the original specification; if infeasibilities or format errors are detected, it returns corrective feedback to the Solver, triggering another generate-solve cycle until feasible, verified solution A is produced.
Illustrative example. For natural- diet problem, the Extractor retrieves the food set, nutritional constraints, and cost coefficients; the Solver converts them into the corresponding AMPL model; the numerical optimizer returns the optimal food quantities, and the Checker verifies nutri- tional feasibility before providing the final answer.
Therefore, as shown in Figure 2 (a), MURKA includes three important agents: Extractor, and Checker. They complete the entire optimization problem-solving task through collaboration. The Extractor is obtained through the training of the Group Relative Policy Optimization (GRPO) [Shao al., 2024] RL with our specially improved and designed efficient multi-dimensional rewards (see $2.2). The Solver first synthesizes data through deep thinking and reasoning model, and then is fine-tuned by knowledge distillation (see $2.3).
2.2 Extractor Alignment via Combined Reward Reinforcement Learning
The information extraction task for optimization problem formulation involves parsing complex natural language descriptions into structured components, making it well-suited for RL due to its sequential decision-making nature and the need to optimize extraction quality through iterative feed- back [Pan et al., 2025]. To model this task, we adapt the GRPO algorithm. The process begins with generating multiple candidate solutions, referred to as completions, to explore diverse extractions of key problem elements. Specifically, at each training step, we sample a batch of optimization problems and generate G' completions (denoted o;) for each problem, representing structured in- formation extractions. Next, we compute the advantage value for each completion to guide policy updates. In the original GRPO, this relies on a reward model, which poses challenges in practical
3

---

(oi.e g, 04,4)) MURKA Framework
Optimizalion
problems
Extracior
Salver
Gurobi
Findl Response
(b) Extruciur Alignment
(c) Solver Aligument
Optimization
Preblem
Infoemation
Rewards
Advantages
1g
Teacher Model
Q-A with Eatracted Inforaattion & AMPL
optiniaation prociems
Filler By Sulver
Mr
P6
Sp.
de equuet revert bomiiin Seceetla Auilyii, Crestraie Chicklng, slabbrity
objective
Student Model
tuacher medil.
Medal Vile condided
Figure 2: MURKA framework pipeline. (a) Multi-agent orchestration of Extractor-Sove Saaerr-oovvv--eeeeeee agents for optimization problem solving. (b) RL training workflow for the Extractor with multi- dimensional reward signals. (c) Knowledge distillation process for Solver alignment through teacher- student model transfer.
settings due to its high data and computational demands and sensitivity to training data quality. To address these limitations, we propose an enhanced GRPO approach that replaces the reward model with composite reward function ro;), detailed in Section 2.4. For each of the G' completions, the reward is calculated using r(o;), enabling the advantage value Å., to better reflect the quality of extractions and provide more effective policy optimization. The advantage is normalized as shown in Equation 3, where mean(r(o)) and std(r(o)) represent the mean and standard deviation of the rewards across completions, respectively.
Â r(o,) mean(r(o))
std(r(o))
In the links of estimating the KL divergence and calculating the loss, follow the methods of the original GRPO algorithm. The KL divergence Drr. is used to measure the difference between two policies, and its definition is shown in Equation 4, where To and Te.e, the current policy (determined by the model parameter 8) and the reference policy, respectively. represents the probability of taking the action o,, at the t-th moment of the i-th sample under the policy no. given the optimization problem q and the previous information extraction 0r By estimating DKL, the magnitude of the policy update punished to ensure that the model does not deviate too much from the reference policy when updating the policy, maintaining the stability of the training process.
Tref (or, q, 0i,ss) log
Tref (or,e q. O1,5t)
DEL [Taa Frot]
(4)
To (0i, 4 0i,
no (0i, q Oi,<r)
When calculating the loss, the loss function is still determined by maximizing the advantage and considering the KL divergence penalty, and its definition is shown in Equation 5. The first term represents the sealed advantage, and the second term punishes the deviation from the reference policy through DEL. Where is hyperparameter used to control the degree of KL divergence penalty, and nograd means that the old policy term does not participate in the gradient calculation.
losl
ma (oi, q. 01,5) Touas (0r,e Oi,<t) grad
((0)
A.,. 3D [Te][Tyes]]
(5)

---

Algorithm Comprehensive Reward Calculation
Require: Candidate response Ci, weights Wformat Weonstr, Weems Wesim
Ensure:
Comprehensive reward R 1: Extract text content & f- c.cootentt
2: Compute format reward Riormaa
(see Algorithm 2, lines 2-6)
3: Compute constraint reward Reonstr
(see Algorithm 2. lines 7-14)
4: Compute semantic reward Rsem
(see Algorithm lines 2-5)
5: Compute similarity reward Raim
(see Algorithm 3, lines 6-13)
6: Compute comprehensive reward:
7: R f Wfformat Roomat + Wconstr
Reonstr
Weem
Reem + Wsim
Rsim
8: return R
2.3 Solver Alignment through Knowledge Distillation
Inspired by Hinton et al. [2015], our Solver model alignment strategy employs knowledge distilla- tion to transfer optimization modeling expertise from a high-capacity teacher model to a compact student model, shown in Figure 2(c). We curate diverse training dataset of 3,602 problem instances, sampled from 20% of the test set and stratified by scenarios and optimization types (Ta- ble 10, Table 11). Using teacher models like DeepSeek-RI [Guo al., 2025], we generate enriched outputs (q. i, m, a)) from question-answe answer pairs (g, a), where is the extracted information and m the AMPL model. Data augmentation via prompt engineering and domain-expert input enhances diversity, while an iterative refinement process using Gurobi [Gurobi Optimization, LLC, 2025] val- idates and adjusts AMPL models until they match the ground-truth a, ensuring high-quality training data.
2.4 Combined Reward Implementation Details
Additional symbol explanations are provided in Appendix A. Algorithm integrates format valida- tion, constraint checking, semantic analysis, and similarity checking, calculating comprehensive reward R through weighted summation to evaluate the overall quality of model-generated results. Specific implementation details for each stage provided in Appendix B. Algorithms 2 and 3 de- scribe our method for comprehensively evaluating model genee-egeerrtte results using multi ditinsiiiasii reward functions, with format validation, semantic analysis, and constraint checking core stages to compute the final reward value.
Our methodology establishes multi-criteria reward mechanism through formal composition of four orthogonal evaluation dimensions. Let e denote the candidate response, the composite reward R(c,) is formulated as:
R(c:)
Σ
wy (x()(c))
(6)
IE (fint, som, sim}
where W/ denotes configurable weight coefficients reflecting dimensional importance. stituent rewards computed follows:
The con-
Format Verification. This component evaluates XML structure compliance through pattern matching and positional analysis:
Rint [patrict (1) + I(poot 4i)
strict match
soft match
Σ Ir I(pos(s) E +.)) axlippre 4post|1
where &, denotes valid positional constraints for tag s, and ||- II1 measures redundant text length.
5

---

Constraint Checking, variable interaction:
This reward evaluates mathematical constraint validity through operator
Reonstr Texist T ++ [Tpp ILLEVUP + [2o] .r
(8)
with clip( Reonstr
Rmins Rmax) ensuring numerical stability.
Semantic Analysis.
Element-level analysis of objective function components:
Ryem clip
Σ LEE
(ra + r2leevur - r3 r=Tunddr(e)) Ramin: Renux
(9)
Similarity Checking. This metric combines cardinality alignment and embedding-basee similar- ity, with embeddings generated by the nomic-embed model [Nussbaum et al., 2024]:
S(A, B) = LA
1
Σ
max cos(Memae ( Memb BE
(10)
aE
sim
clip 3 cand + 8(11 Vaa) + ((Pam: Pac)), Rmin Rmax)
where Acant
Wi
V
Paiml
Pacil measures set cardinality discrepancy.
The complete reward system employs hierarchical boundary constraints and weight calibration to maintain numerical stability while preserving gradient information for policy optimization. Hyper- parameters (a, 33 6, 78,40 w;) are grid-optimized through orthogonal experimental design.
Experiments
We conduct an extensive evaluation of MURKA demonstrate its effectivvnees automating opti- mization problem modeling, comparing it against a diverse set of state-of-the-art baseline methods across multiple dimensions. Our experimental design encompasses several objectives: (I) Assessing MURKA's performance against leading LLM LLM-assisted, multi-agent, domain-specific, and general optimization approaches; (2) Validating the contributions of its Extractor and Solver components
through ablation studies; (3) Evaluating its generalization across diverse tasks beyond optimiza- tion; (4) Analyzing the impact of training hyperparameters on performance stability; (5) Exploring its scalability on smaller-scale models. Detailed implementation specifics, including experimental descriptions, prompt templates, hyperparameter configurations, and computational resources, are provided in Appendix D and Appendix E.
3.1 Experimental Setup
Benchmarks, We evaluated method eight diverse benchmarks Saatenarrkk---OOppp NNL40tt [Ramamonjison et al., 2023], Mamo Easy, Mamo Complex [Huang et al., 2024], NLP4LP [Ahmadi et al., 2024], ComplexOR [Xiao al., 2023], IndustryOR [Huang al., 2025]. OptiBench [Yang al.. 2024], and OptMATH [Lu et al., [025]-comprising 2,224 problem instances. These benchmarks cover over 20 real-world scenarios. including Agriculture. Transportation, and Entertainment, and span seven optimization categories, such as Linear Programming, Mixed-Integer Programming, and Combinatorial Optimization. Details are provided in Appendix C.
Baselines.
To benchmark our method, we compare against four categories of baseline approaches:
1. LLM-Assisted Optimization Methods. Methods integrating advanced large language models, such as GPT-4 [Achiam et al., 2023), Qwen-3 [Team, 2025] and DeepSeek- R1 |Guo et al., 2025], with the Gurobi solver [Gurobi Optimization, LLC., 2025] for optimization tasks.
2. Multi-Agent Optimization Frameworks. Frameworks employing collaborative multi-agent strategies, including Reflexion [Shinn et al., 2023], which enhances decision-makinn through verbal reinforcement learning without model fine-tuning: COE [Xiao et al., 2023], which uses chain of expert agents with forward thinking and backward reflection solve complex oper- ations research problems; and OptiMUS ShmadiTeshnil et al., 2024], which builds and solves

---

Table 1: The SA compared the domain-specitfs optimization models. The best results among previous alignment works are marked with underline, the overall best results are marked in bold, and values in purple indicate that our method outperforms the previous alignment baselines.
Mamo
Mamo Complex
Micro
Macro
Method
Model
NLt
Specific
Easy
Avg
Avg
GPT-4
47.3%
66.5%
14.6%
22.3%
49.19%
37.68%
Directly
DeepSeek-Re
94.8%
95.9%
51.2%
45.2%
82.48 %
71.78%
Qwen2.5-7B
94.7%
86.5%
51.2%
24.4%
75.23%
64.20%
OptMATH
Qwen2.5-32B
95.9%
89.9%
54.1%
34.7%
78.88%
68.65%
Mistral-7B
84.4%
81.4%
32.0%
27.0%
67.56%
56.20%
ORLM
Math-7B
86.5%
82.2%
37.9%
33.0%
70.05%
59.90%
LLaMa3-8B
85.7%
82.3%
37.4%
38.0%
70.42%
60.85%
Ours
I.LMMa-8BB
93.5%
95.9%
55.6%
37.4%
82.18%
70.60%
Specific represents the average of IndustryOR and OpIMATH.
Math-7B represents the MeapSeek-Math-7BBBBe
linear and mixed-integer programming models from natural language using modular multi-agent collaboration.
Domain-specific Customized Optimization Models. Specialized approaches for industrial op- erations research and mathematical optimization, including ORLM [Huang et al., 2025], which trains open-source large language models via IR-Instruct synthetic datasets, and OptMATH [Lu et al., 2025], which generates large-scale datasets using selected seed data and back-translation to fine-tune LLMs for automated optimization modeling.
General Optimization Methods. Methods for generic optimization problems, including NL2OR [Li et al., 2024], which translates natural language into operations research models for accessible solutions, and LLMOPT [JIANG et al., 2024], a learning- based framework that defines and solves diverse optimization tasks with improved generalization.
Metrics. In the experiment, we use three performance metrics to comprehensively evaluate the quality of the generated code of the algorithm, namely, Solution Accuracy (SA), which measures the correctness of the provided solutions, Compilation Accuracy (CA), which assesses the ability of the code to compile without errors, and Execution Rate (ER), which indicates the proportion of successful executions.
3.2 Performance Evaluation of MURKA Against Optimization Baselines
LLM-Assisted Optimization Methods Comparison. As shown in Figure 3a, MURKA achieves macro-average performance of 68.6 %, closely approaching DeepSeek-RI's 73. despite us- ing a significantly smaller LLaMa3-8B backbone. It is critical to note that the DeepSeck-Ri-671B baseline over 80 times larger than our model. Despite this scale disparity, MURKA demon- strates comparable and even superior performance specific benchmarks, underscoring our frame- work's substantial computational efficiency and viability for practical deployment. Our selection of LLaMa3-8B was predicated on its strong, publicly verifiable performance, establishing it as a robust foundation for our experiments. However, we emphasize that MURKA's core technical contribu- tion is its model-agnostic architecture. The framework is designed as modular, "plug-and-play enhancement that can be seamlessly applied to other capable base models, such as Qwen- 7B or fu- ture open-source alternatives. This adaptability ensures that the demonstrated performance gains attributable to our framework's structured approach rather than the intrinsic capabilities of a single base model.
We hypothesize that this competitive performance stems from the nature of NL-to-Optimization challenges. These tasks present two distinct hurdles: (1) complex reasoning chains, which require intricate, multi-step logical deductions, and (2) robust structural extraction, which demands precise parsing of numerical values, relationships, and constraints from dense ambiguous text. While large models with strong general reasoning capabilities, such as DeepSeek-RI naturally excel the former, their generalist approach can be brittle when faced with the latter. In contrast, MURKA A's Ex-
7

---

Table 2: The SA and ER compared the general optimization methods. The best results among previous alignment works are marked with underline, and values in purple indicate that our method outperforms the previous alignment baselines.
Metrics
ER
SA
ER
SA
ER
SA
ER
SA
ER
SA
Method
NL40pt
Mamo Easy
Mamo Complex
NLP4LP
Micro Avg
NL2OR
86.5%
74.8%
70.1%
63.2%
22.8%
18.0%
88.8%
75.6%
59.5%
48.3%
LLMOPT
99.0%
93.0%
100.0%
95.3%
98.0%
68.0%
100.0%
83.8%
92.7%
76.7%
Ours
100.0% % 93.5%
100.0% %
95.9%
100.0% %
55.6%
100.0%
87.6%
97.0%
76.5%
Method
ComplexOR
IndustryOR
OptiBench
OptMATH
Macro Avg
NL2OR
11.1%
5.56%
14.0%
4.0%
62.3%
43.1% 1%
6.63%
2.41 %
45.3%
35.8%
ILMOPT
94.7%
72.7%
92.0%
44.0%
82.3%
66.4%
75.3%
40.0%
92.7% 97.8%
70.4%
Ours
100.0%
72.2%
99.0%
38.0%
91.4%
69.3%
92.2%
36.8%
68.6%
NL2OR uses LLaMa3-70B as its backbone, while LLMOPT uses Qwen as its backbone.
Mano Compplx
GPT-
COE
NL40g
Marm
OpeiMUS
NLMME
Xber
DeopSook-R1
Dar
CompleeOU
NIP41P
Mirr
InduttoR
OstBorch
Complex
(a) LLM- Assisted Methods Comparison
(b) Multi-Agent methods Comparison
Figure 3: The SA compared LLM--ssisted optimization methods and Multi-Agent methods.
tractor agent is specifically optimized via our multi-dimensional RL reward to achieve high-fidelity structural parsing, making it robust against the very ambiguities that can derail a general reason- ing process. In essence, MURKA bridges the performance gap not by mirroring general problem- solving abilities, but by specializing in the systematic and verifiable translation of natural language into structured optimization models.
Multi-Agent Optimization Frameworks Comparison. Our approach delivers state-of-the-ar performance across all benchmarks (Figure 3b), with absolute improvements of 14.8% and 11.93% micro- and macro- macro-average metrics, respectively, over prior multi-agent methods. These gains, consistent across standard and complex scenarios, highlight the strength of our structured task de- composition framework in enhancing collaborative reasoning.
Domain- Tailoree Customized Optimization Models Comparison. Our method outperforms specialized optimization models by 3.4% micro-average and 1.75% in macro-average metrics (Table 1). It achieves 95.9% on Mamo Easy and 55.6 33.6% on Mamo Complex, surpassing task-specific models (OptMATH) and celtmiization-aiinnee LLMS (ORLM). These results demonstrate our frame- work's 's ability to bridge general-purpose purpose LLMs and domain-specific solvers through systematic rea- soning decomposition.
General Optimization Methods Comparison. Table 2 shows our method excels in Execution Rate (ER), achieving near-perfect scores of 100% across benchmarks, including NL40pt, Mamo Easy, Mamo Complex, N.P44P, and ComplexOR. In Solution Accuracy (SA), it outperforms LL- MOPT on NL.P4IP (87.6% vs. 83.8%) and OptiBench (69.3% vs. 66.4%)) but slightly trails on ComplexOR (72.2% vs. 72.7%) and IndustryOR (38.0% vs. 44.0%). With a micro-average ER
8

---

SA
CA
103
NL40
93.5
91.7
66.1
97.8
98.7
88.3
100.0
100.0
100.0
95.9
90.3
73.5
97.7
81.4
100.0
190.0
80
Mamo Complex
44.5
11.4
18.5
100.0
99.0
NLP4LP
87.6
84.7
66.1
99.2
95.7
88.8
100
100 0
100
60
Penenntgga
ComplexOR
72.2
38.9
16.7
81.1
44.4
22.2
100.0
100
100.0
On
21.0
OpeiBeach
69.3
62.6
43.3
89.1
88.1
66.6
91.4
100.0
100.0
OptMath
36.3
11.1
0.6
52.9
18.7
3.6
92.2
88.0
87.3
Figure 4: Heatmap of ablation study comparing Munl, Mg, and Ms across benchmark datasets. Deeper purple indicates poorer performance.
of 97.0% and mvero-average SA of 68.6%, our method demonstrates robust versatility, though im- provements specific SA metries are needed.
Our proposed method showcases significant strengths in optimization problem modeling. It achieves state-of-the-arr SA in multi-agent settings, with top scores on NL40pt (93.5%), NLP4LP (87.6%). and ComplexOR (72.2%). alongside near-perfect ER scores of 100% across most benchmarks. In specific optimization tasks, dominates on NL.4pt, Mamo Easy, and Mamo Complex, though it falls short on the Specific benchmark (37.4%) compared to DeepSeek-RI -1-5eek--16611B (45.2%). For general optimization, its robust ER and competitive SA underscore its versatility, While it excels in many areas, enhancements are needed in specific metrics like IndustryOR and OptMATH, well as broader SA performance, to further strengthen its capabilities.
3.3 Ablation Experiment
The primary ablation study validates the roles of the core architectural components: the extractor (E) and the solver (S). By comparing the full system Manl (integrating & and S), MM (with È replaced by LLaMa3-70B), and Ms (with S replaced by LLaMa3-70B) across eight optimization benchmarks, we assess their contributions using SA, CA, and ER as metrics. As shown in Figure 4, Mui signif- icantly outperforms Mg and Ms across all benchmarks. For example, on the ComplexOR dataset, Man achieves a SA of 72.2%, whereas Mg and Ms only reach 45.6% and 51.1%, respectively. Simi- larly, the ER Mill remains near-perfect at 100.0%, compared to 68.9% 9% for Mç and 77.8% for Ms. This confirms the necessity of the synergistic integration of & and S, with specialized components demonstrating significant advantages over general-purgge -prrpose large models optimization tasks.
To further dissect the sources of these performance gains, we conducted a detailed ablation study on the individual components our composite reward function in Appendix D.1.3. This analysis reveals a clear hierarchy of their importance. The Riormut and Reonstr are foundational; their absence leads structurally malformed text and logically flawed models, causing a catastrophic drop in performance across all benchmarks. The Reem is also critical, guiding the Extractor to identify the correct objective function. Without the model often solves a different problem entirely, rendering the final solution invalid. Finally, the Ram provides fine-grained semantic tuning by preventing subtle logical flaws, such as confusing variable or parameter identities. While its removal results in a less severe performance drop, it is crucial for achieving high accuracy. Collectively, these results confirm that each reward component addresses a unique and critical facet of the modeling challenge- from structural integrity to semantic accuracy- and that their synergy is fundamental to MURKA's high performance. Furthermore, we verified that our alignment process preserves the base model's general capabilities and tested the framework's scalability on smaller models, with detailed results presented in Appendix D.1.5.
To further enhance MURKA, we recommend improving E's information extraction by incorporating domain knowledge boost CA complex tasks and strengthening S's modeling capabilities by exploring advanced optimization algorithms to improve SA. Additionally, motivated by our reward component analysis which reveals a clear hierarchy their importance, integrating dynamic reward weighting in the RL process could adaptively prioritize extraction accuracy for diverse problem

---

Future work could analyze the contributions of & and S across varying task complexities and optimize the framework for ressurce-constrained scenarios, potentially exploring lightweight model architectures for edge deployment.
types.
4 Related Work
The research is positioned at the intersection of three active areas: synthetic data generation for optimization, multi-agent frameworks, and the application of reinforcement learning for LLM align- ment.
Data Synthesis for Optimization Problems. High-quality data is crucial for training LLMs to handle OR tasks. To overcome the scarcity of real-world datasets, recent works have focused on synthetic data generation. Methods include semi-automated synthesis from industrial case studies [Huang et al., 2025], back-translation from mathematical expressions to natural language [Yang et al., 2024], and using rejection sampling to control problem complexity [Lu et al., 2025]. These approaches have been instrumental in creating large-scale datasets for supervised tinettunigg Dis- tinctly, MURKA leverages synthetic data not for direct supervision, but environment train our Extractor agent via reinforcement learning, allowing to learn robust policies from interactive feedback rather than static examples.
Multi-Agent Frameworks. Decomposing complex problem-sole solving into tasks for specialized agents has proven effective. In the OR domain, frameworks like Chain-of of -f-eererts [Xiao et al., 2023] orchestrate agents with forward-solving and crekward-error-coreettnn OptiMUS AAhmad'Teshnizii et al., 2024] utilizes modular collaboration to handle linear and mixed- integer programming. These frameworks typically coordinate agents through sophisticated prompt engineering and predefined workflows. MURKA builds on this collaborative paradigm but enhances agent expertise through direct training. Our agents are not just directed by prompts but are special- ized via reinforcement learning and knowledge distillation, leading to more adaptive and capable collaboration.
mechanisms, while
Reinforcement Learning for LLM Alignment. Reinforcement learning is a powerful technique for aligning LLMs with complex objectives, moving beyond simple instruction following. While foundational methods like PPO [Schulman et al., 2017) and DPO [Rafailov et al., 2023] are widely used, recent advancements have focused enhancing reasoning capabilities. Notably, group rela- tive reward mechanisms, as employed in mathematical reasoning tasks [Shao et al.. 2024, Guo et al., 2025], have shown success by optimizing policies based on the relative quality multiple generated outputs. We adapt this paradigm to the structured domain of optimization. Our key contribution is the design of a composite reward function that provides granular feedback on multiple facets of the extraction taskincluding format, constraints, and semanticsthereby aligning the LLM specifically with the rigorous demands of mathematical optimization modeling.
5 Discussion
The MURKA framework presents significant practical implications for OR automation. By reducing expert dependency and computational costs, it enables rapid prototyping of optimization models across various industries. The released dataset and benchmark facilitate standardized evaluation of OR modeling capabilities in LLMs. Potential societal benefits include more accessible optimization tools for small businesses and accelerated pocision-making in critical domains. However, several limitations remain: (1) MURKA assumes structured natural language problem descriptions, which may not fully capture real-world problem complexity. (2) While our combined reward mechanism improves extraction accuracy, it still relies on predefined templates that may require adaptation for novel problem types. (3) The knowledge distillation process depends on synthetic data quality, which could inherit biases from the teacher model.
10

---

Acknowledgments
The research is partially supported by Innovation Program for Quantum Science and Technology 20212D0302900 and China National Natural Science Foundation with No.62132018, 62231015, "Pioneer" and "Leading Goose" R&D Program of Zhejiang. 2023C01029, and 2023C01143, An- hui Provincial Natural Science Foundation under Grant 2208085MF172 and the USTC Kumpeng- Ascend Scientific and Educational Innovation Excellence Center. We also thank Sijia Zhang, Shuli Zeng, Xiaotian Pan, and the anonymous reviewers for their comments and helpful feedback.
References
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08772 08774, 2023.
Ali AhmadiTeshnizi, Wenzhi Gao, and Madeleine Udell. Optimus: Scalable optimization model- ing with (mi) ip solvers and large language models. In Forty-first International Conference on Machine Learning, 2024.
Robert Fourer, David M. Gay, and Brian W. Kernighan. AMPL: A Modeling Language for Math- ematical Programming. Thomson Brooks/Cole, Pacific Grove, CA, 2nd edition, 2003. ISBN 0-534-38809-4, 517 xxi pages.
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPoñi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, h, Haonan Li, Kyle McDonell, Niklas Muen- nighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang. and Andy Zou. The language model evaluation harness, 07 2024. URL https://zenodo wtw./ecoodd// 12608660.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501.12944 2025.
Gurobi Optimization, LLC. Gurobi Optimizer Reference Manual, 2025. URL https://www gurobi com.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. arXiv preprint arXiv: 02530,2231120 2015.
Chenyu Huang, Zhengyang Tang, Shixi Hu, Ruoqing Jiang, Xin Zheng, Dongdong Ge, Benyou Wang, and Zizhuo Wang. Orlm: customizable framework in training large models for auto- mated optimization modeling. Operations Research, 2025.
Xuhan Huang, Qingming Shen, Yan Hu, Anningzhe Gao. and Benyou Wang. Mamo: mathematical modeling benchmark with solvers. arXiv preprint arXiv:2405. 43144, 2024.
Caigao JIANG, Xiang Shu, Hong Qian, Xingyu Lu, JUN ZHOU, Aimin Zhou, and Yang Yu. Llmopt: Learning to define and solve general optimization problems from scratch. In The Thirteenth International Conference Learning Representations, 2024.
Beibin Li, Konstantina Mellou, Bo Zhang. Jeevan Pathuri, and Ishai Menache. Large language models for supply chain optimization. arXiv preprint arXiv:2307.03875 2023.
Junxuan Li, Ryan Wickman, Sahil Bhatnagar, Raj Kumar Maity, and Arko Mukherjee. NI2or: Solve complex operations research problems using natural language inputs. arXiv preprint 07277 2024.
Hongliang Lu, Zhonglin Xie, Yaoyu Wu, Can Ren, Yuxuan Chen, and Zaiwen Wen. Optmath: scalable bidirectional data synthesis framework for optimization modeling. In Forty-second International Conference Machine Learning, 2025.

---

Zach Nussbaum, John Xavier Morris, Andriy Mulyar, and Brandon Duderstadt. Nomic embed: Training a reproducible long context text embedder. Transactions on Machine Learning Research, 2024.
Xiaotian Pan, Junhao Fang, Feng Wu, Sijia Zhang, Yi-Xiang Hu, Shaoang Li, and Xiang-Yang Li. Guiding large language models in modeling optimization problems via question partitioning. Alnternational Joint Conference Artificial Intelligence, 2025.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728-5333 53741, 2023.
Rindra Ramamonjison, Haley Li, Timothy Yu. Shiqi He, Vishnu Rengan, Amin Sanitalebi-Dehkordi Zirui Zhou, and Yong Zhang. Augmenting operations research with auto- formulfonmlttion of optimiza- tion models from problem descriptions. Methods in Natural Language Processing: Industry Track, pages 29-62, 2022.
Proceedings of the 2022 Conference on Empirical
Rindranirina Ramamonjison, Timothy Yu, Raymond Li. Haley Li, Giuseppe Carenini, Bissan Ghad- dar, Shiqi He, Mahdi Mostajabdaveh. Amin Banitalebi-Dehhee Dehkori,, Zirui Zhou, et al. NI4opt com- petition: Formulating optimization problems based on their natural language descriptions. In NeurIPS 2022 Competition Track, pages 189-203. PMLR, 2023.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint (eX.iv:707.06355 2017.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseckmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 2024.
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634 1448634--652,, 2023.
Qwen Team. Qwen3, April 2025. URL httttt:////eeet http::///gwnnm..itt.bbbb.. UtL. http:/awww..iiitiiiiboo////////e ho/bblog/wwe///
Simon Trimborn, Mingyang Li, and Wolfgang Karl Härdle. Investing with cryptocurrenciesa liquid- ity constrained investment approach. Journal of Financial Econometrics, 18(2):280-306, 2020.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi. Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824 24837, 2022.
Ziyang Xiao, Dongxiang Zhang, Yangjun Wu, Lilin Xu, Yuan Jessica Wang, Xiongwei Han, Xiaojin Fu, Tao Zhong, Jia Zeng, Mingli Song, et al. Chain-of-experts: When llms meet complex oper- ations research problems. The twelfth international conference on learning representations, 2023.
Zhicheng Yang, Yiwei Wang, Yinya Huang, Zhijiang Guo, Wei Shi, Xiongwei Han, Liang Feng, Linqi Song, Xiaodan Liang, and Jing Tang. Optibench meets resocratic: Measure and improve lims for optimization modeling. In The Thirteenth International Conference on Learning Repre- sentations, 2024.
Jihai Zhang, Wei Wang, Siyan Guo, Li Wang. Fangquan Lin, Cheng Yang, and Wotao Yi.. Solv- ing general lanuuaal- Proceedings of the 2024 Conference of the North American Chapter of the Association for Compu- tational Linguistics: Human Language Technologies (Volume 6: Industry Track), pages 483 4900 2024.
optimization problems with large language models. In
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly outine the MURKA framework, its contributions (multi-agent system, GRPO, knowledge distillation, benchmark), and performance improvements, which align with the detailed results Sections 3.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: The paper includes a discussion section that explicitly addresses limitations, such reliance on structured natural language inputs, predefined templates for reward mechanisms, and potential biases in synthetic data from the teacher model.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The au- thors should reflect on how these assumptions might be violated practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-tex system might be used reliably provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations their approach to ad- dress problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
13

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [No]
Justification: The paper focuses on empirical contributions (e.g., framework design, ex- perimental results) rather than theoretical results requiring proofs, so this question is not applicable.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple-
mented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Appendix C and Appendix provide comprehensive details on benchmarks, baselines, metrics, training dataset construction, computational resources, and hyperparam- eters, enabling reproduction of main experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature the contribution. For example
(a) the contribution primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to re- produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
14

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case au- thors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The supplemental material includes the code, README file, training exam- ple data, and log examples, providing sufficient instructions to reproduce the main experi- mental results as described in Appendix D.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
public/guuues//oddSuunnss soloossoolooo for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: /ttg///ddde//eeuuumiieeeeeeeeeee
/ //nips
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: Appendix D details hyperparameters, data splits, and training configurations, ensuring clarity for understanding results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined other appropri- ate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
15

---

Justification: The paper reports standard deviations for rewards in the sensitivity analysis of epochs, clearly stating the variability captured across training steps. The method for calculating standard deviation is implied through the training process, and assumptions of normality are not explicitly violated.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK report I-sigma error bars, but one should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative eтror rates).
If eгror bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Appendix D.2 specifies the use of eight NVIDIA GeForce RTX 4090 GPUs with 24 GB memory each and four AMD EPYC 7763 64-Core Processors 2.45GHz.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn 't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: h/tpurss////uuss..... hetpblic//ccicccuuiileeeeeeeee
Answer: [Yes]
Justification: The paper adheres to ethical research practices, with no indications of viola- tions (e.g., proper citation of assets, no human subject research), aligning with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
16

---

The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The section 5 highlights positive impacts (e.g., accessible optimization tools for small businesses, accelerated decision-making) and acknowledges potential negative impacts indirectly through limitations (e.g., biases synthetic data).
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e.g., gated release of models, providing defenses addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (c.g.. pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
Justification: The paper does not release data models with high risk for misuse, as the MURKA framework and its associated benchmark are designed for optimization problem modeling in operations research, with applications in domains like logistics and supply chain management.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards quiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
allow for controlled use of the model, for example by re-
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
17

---

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Appendix D details the licenses, terms of use, and proper crediting for all code, models, and datasets used, ensuring compliance with their original sources and re- spective licenses.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the pack- age should be provided. For popular datasets, paperswi thcode com/datasets has curated licenses for some datasets. Their licensing guide can help determine the li-
cense of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documenta- tion provided alongside the assets?
Answer: [No]
Justification: The paper does not introduce new assets such datasets, code, models for public release, as it focuses on the MURKA framework and evaluation on existing benchmarks without releasing new resources.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the pa- per include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [No]
Justification: The paper does not involve crowdsourcing or human subject research, this question is not applicable.
Guidelines:
The answer NA means that the paper does involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
18

---

According to the NeurIPS Code of Ethics, workers involved in data collection, cura- tion, or other labor should be paid at least the minimum wage the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [No]
Justification: The paper does not involve human subjects crowdsourcing, making IRB approvals irrelevant.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equiva-
lent) may be required for any human subjects research. you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality the research, declaration is required.
Question:
Answer: [Yes]
The paper explicitly describes the use of LLMs as core components of the MURKA framework, detailing their roles in extraction, solving, and knowledge distillation in Section 2.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components. wwwparippscccCoofeeeeeeeeeeeeeee 2c/Confeerecces/2025////////////
Please refer to our LLM policy for what should or should not be described.
19

---

A Algorithm Symbol Definitions and Explanations
Table 3 lists all the key symbols used in the algorithm along with their explanations. These symbols cover various aspects of the algorithm, such as reward calculation, semantic analysis, and similarity checks. Each row in the table includes a symbol and its detailed explanation, ensuring that readers can accurately understand the meaning of each symbol. We recommend that readers refer to Table as needed while reading the algorithm description to ensure their understanding of the algorithm is accurate and comprehensive.
Table Algorithm Symbol Explanation
Symbol
Explanation
Wi
Weight coefficient for the i-th reward component
r
Basc reward for correct items in format and constraint checks Reward for numerical elements in semantic analysis
r1
12
Reward for defined variables/parameters in semantic analysis
T3
Penalty for undefined elements in semantic analysis Penalty coefficient for redundant text length (pre, ppos))
O
B
Penalty coefficient for set cardinality difference similarity Weight for variable similarity in reward calculation Weight for parameter similarity in reward calculation Scaling factor for similarity reward
71
72
8
Rmin
Lower bound of reward value for boundary constraints
Rmax
Upper bound of reward value for boundary constraints Strict matching mode for segmented structure validation Lenient matching mode for label detection Set of key labels, e.g., {Sets, Parameters.... Set of variables from candidate responses or predefined Set of parameters from candidate responses or predefined Set of objective function elements for semantic analysis Target variable set, predefined standard
Psrict
Poo
S
V
P
E
V Vaim
P aim
Target parameter set, predefined standard
Vact
Actual variable set from candidate responses Actual parameter set from candidate responses Length of preceding redundant text before label Length of trailing redundant text after label Indicator function: 1 if condition met, else 0 Clipping function, constrains to [a, Embedding model, maps text to vector space Cosine similarity between vectors u and v
Pa act
lpe
post
I(-)
clip (z, a, b)
Memb
cos( u, v)
B Algorithm Notations
B.1 Structure Validation Reward Computation
Format validation stage (line 2-6): First, the integrity of the segmented structure is verified by strictly matching the pattern Pstrict- Then, the existence of tags is detected using the loose pattern Psoft- Finally, single-item reward is given for the accurate positioning of the key tag set S (Sets, Parameters, J. Meanwhile, a linear penalty a is imposed on the redundant text length (pwa)) in the tag context.
Constraint checking stage (line 7-14): As shown in line 7, this module first assigns basic reward r to the existing constraint segments. Then, it analyzes the constraint conditions one by one: when a legal operator is detected, r is added (line 10); when defined variable v E P is used, added; when a non-negative constraint exists, an additional r is added. Finally, the total value is constrained within a preset interval through the clip function, and stability control completed as shown in line 14.
20

---

Algorithm 2 Structure Validation Reward
Require: Candidate response C₁
Ensure: Format reward Rormat constraint reward Reonstr
1: Extract text content I +- c..onneett
2: Format Verification Phase:
3: Ratric &- r (mtch(PParicts t.])
4: Raoft f- 11( searcc t4))
5: Rami <- Σ Ereslr I[(s exists with correct position)] alppp + ppla)
6: Riormut Rstrict + Ryoh + Rxml
7: Constraint Checking Phase:
8: Initialize Raoostr -- r- I(Constraints section exists in t;)
9: for each constraint k do
Reonstr + Reonstr + r. [(vald operator)
11: Reonsur +- Reonst + r (uses defined variables)
12:
Reonstr + Reonstr + r. Cerrr-Icoomttt I(contains non-negane non-negative constraint)
13:
end for
14:
Reonstr f- clip Reonstri Rmin Rmax)
15: return Rioeraa Revontir
B.2 Content Validation Reward Computation
Semantic analysis stage (line 2-5): After extracting the element from the objective function, hierarchical reward mechanism is implemented shown in line 4: numerical elements receive a basic reward rı, elements within variable set or the parameter set P receive a compliance reward r2, and undefined elements trigger a penalty r3. Finally, the reward value is ensured to be stable within the effective interval [mmin: Rmax] through the boundary constraint function.
Similarity checking stage (line 6-13): This module achieves semantic alignment of structured ele- ments through embedding space measurement and imposes linear penalty term on the cardinality difference between the variable set V and the parameter set P. Then, the text is mapped to the Rd space through the embedding model Memb, and the bidirectional maximum cosine similarity be- tween the target set and the candidate set is calculated. Finally, a weighted fusion strategy is adopted convert the semantic similarity into reward value, and the clip function used ensure numer- ical stability.
Algorithm 3 Content Validation Reward
Require: Candidate response C
Ensure: Semantic reward Raem- similarity reward Rsim
1: Extract text content & c;conntnt
2: Semantic Analysis Phase:
3: Extract variable set V, parameter set P, objective elements E if is numerical
4: Ryem + EEEE
T2
if E VUP
T3
otherwise
5: Rsem + clip( Ryems Rmin- Rmax)
6: Similarity Checking Phase:
7:
Vaa f (xtractXML(tt, Variables)
8: Put &- ExtractXMLis Parameters)
9:
Rien < :3((Vaim) aim Vinll + |Paiml P
10:
simy [Vinl Σ EvEVum max cos(Menm) embb (v), Memb (u))
11:
simp Punl Epero.. max cos(Memb (p), (v), Memb
Rum Rien 5(m1 simy + simp)
13:
Rum f- dli(Rsim Rmin. Rmax)
return Rggm, Rsim
21

---

C Detailed Benchmark Descriptions
This appendix provides detailed overview of the eight benchmarks used evaluate method: ALt Mamo Easy, Mamo Complex, NLP4LP, ComplexOR, IndustryOR, OptiBench, and Opt- MATH. These benchmarks collectively comprise 2,224 problem instances, covering over 20 real- world scenarios and eight optimization categories. The scenarios and optimization types for each benchmark summarized in Table 10 and Table 11, respectively.
The optimization types are abbreviated as follows: LP denotes Linear Programming, IP denotes Inte- ger Programming, MIP denotes Mixed Integer Programming, NP denotes Nonlinear Programming, CO denotes Combinatorial Optimization, MOP denotes Multi-objective Programming, and SOCP denotes Second-Oree Cone Programming.
D Detailed Experimental Inventory
To comprehensively evaluate the MURKA framework, we elaborate in this section on the experi- mental design, adhering to the MIT License, Llama Community License, and Apache License 2.0 for all existing assets (code, models, datasets) used this study to ensure full compliance with their terms of use.
D.I Experimental Design and Objectives Analysis
D.1.1 Capabilities of Native Small-Scale LLMs in Optimization Tasks
Due to the high deployment costs of large-scale language models in practical scenarios, prelimi- nary work first investigates the capabilities native small-scale LLMs optimization tasks.
This experiment evaluates the performance of native small-scale LLMs optimization tasks, com- paring LLaMa3-8B and LLaMa3-70B on the NL4Opt and Mamo tasks. We employ Zero-shot, Few-shot, and CoT [Wei et al., 2022] prompting strategies, and compare these with method that generates AMPL files followed by Gurobi solving. Results, shown Table 4, indicate that both LLaMa3-8B and LLaMa3-70B perform poorly when directly handling eigh-precisoon optimization problems, highlighting the limitations of native small-scal LLMs in optimization tasks, particularly in scenarios requiring high precision. By generating AMPL files and solving with Gurobi, the opti- mization capabilities of ILLMM338BB and LLaMa33708B improve significantly, by 3.73CE and 6.54(E, respectively. The larger improvement in LLaMa3-703 suggests that larger-scale models have greater potential in optimization modeling and solving, likely due to their enhanced language understanding and generation abilities.
Inspired by Ramamonjison al. [2022], we introduce an information extraction pipeline, using LLaMa3-8B for information extraction or expert-curated information, combined with LLaMa3-70B and Gurobi for modeling and solving. As shown in Figure 5, this approach improves CA by an average of 4.67% and SA by 7.37%, indicating that information extraction significantly enhances the model's understanding and modeling quality for optimization problems. Next work could explore optimizing automated information extraction reduce reliance expert-curated information.
Table Comparison of the SA of the native small- scale LLMs in optimization tasks with and without Gurobi Solver.
Mamo Easy
Mamo Complex
Model
Method
NL4Opt
Zero-shot
0.0%
0.3%
0.0%
Few-shot
0.0%
0.0%
0.0%
LLaMa3-88 8B
CoT
0.8%
0.9%
0.0%
Gurobi
4.4%
2.0%
0.0%
Zero-shot
1.6%
8.3%
3.8%
Few-shot
1.6%
0.2%
0.0%
LLaMa3-70B
CoT
1.6%
9.8%
1.9%
Gurobi
33.5%
52.0%
1.4%
22

---

Uede
Figure 5: Effect of Extracted Information Quality SA and CA.
D.1.2 Performance Evaluation of MURKA Against Optimization Baselines
We propose the MURKA framework (see $2) to enhance the performance of large language mod- els in optimization tasks. The Experimental Setup details the Benchmarks, Baselines, and Metrics used. Performance comparisons between MURKA and selected baselines presented via radar charts $3, with detailed data in Table 5 and Table 6. Results demonstrate that MURKA outper- forms baselines across multiple optimization task benchmarks, exhibiting stronger robustness and generalization.
Table 5: Comparison of the SA metric against LLM-Assisted Methods.
Mamo Complex
Method
NL4Opt
Mamo Easy
NLP4LP
Micro Avg
GPT-4
47.3%
66.5%
14.6%
35,8%
49.34%
Qwen3-32B
83.5%
70.7%
27.0%
76.0%
57.68% %
DeepSeck-R1-671B
94.8%
95.9%
51.2%
93.8%
78.15%
Ours
93.5%
95.9%
55.6 %
87.6%
76.48%
Method
ComplexOR
IndustryOR
OptiBench
OptMATH
Macro Avg
GPT-4
9.5%
28.0%
62.8%
16.6%
35.14%
Qwen3-32B
33.3%
21.0%
56.0%
13.86%
47.67%
DeepSeek-RI
72.2%
47.0%
70.7%
43.4%
71.13%
Ours
72.2%
38.0%
69.3%
36.8%
68.61%
Table 6: Comparison of the SA metric against Multi-Agent Methods.
Method
NL40pt
NI.P4IP ComplexOR
Micro Avg
Macro Avg
Reflexion
53.0%
46.3%
19.1%
48.45%
39.47%
Chain-of Experts
Chapersf-Experts
64.2%
53.1% 11
38.1%
57.76%
51.80%
OptiMUS
78.8%
72.0%
66.7%
75.00%
72.50%
Ours
93.5%
87.6%
72.2%
89.80%
84.43%
D.1.3 Detailed Ablation Studies on Key Components
As detailed in $3.3, our primary ablation study confirmed the critical, synergistic roles of the Ex- tractor (E) and Solver (S). further dissect the framework's performance gains, we conducted an additional ablation study on the individual components of our multi-part reward function.
23

---

Table Ablation study on the impact of re- ward components on SA.
Method
NI.40pt
NLP4LP ComplexOR
Ours (Full)
93.5%
87.6%
72.2%
w/o Raim
78.6%
73.4%
58.5%
w/o Rsem
60.2%
57.9%
39.2%
w/o Reonstr
55.8%
52.1%
34.6%
w/o format
51.3%
48.5%
30.1%
The results, summarized in Table 7, reveal clear hierarchy of importance among the reward com- ponents. The format reward most critical; its absence leads to malformed text and frequent compilation errors, severely undermining performance. The constraint reward and semantic reward are the next most critical, as their removal results in models that are either logically flawed due to incorrect constraints
solve entirely different problem due to wrong objective function. Fi- nally, the similarity reward addresses more subtle logical errors, such as variable confusion; while its removal is less catastrophic, it remains essential for fine-tuning the model's accuracy.
D.1.4 Generalization Assessment Across Diverse Tasks
This experiment evaluates the performance MURKA-aligned LLaMa3-SB (Ours) against native LLaMa3-8B on seven tasks using the eval-harness [Gao et al., 2024], with accuracy as the metric (except for WMT14, which uses BLEU score). The MURKA-alignee, model slightly outperforms na- tive LLaMa3-8B most tasks but shows minor declines XNLI and WMT14. Overall, MURKA does not significantly degrade performance across a wide range of tasks.
Table 8: Performance comparison of MURKA and LLaMa3-8B across seven tasks, measuring accuracy (except WMT14, which uses BLEU score).
Method
GSM8K XNLI TruthfulQA ToxiGen MMLU
QQP
WMT14
LLMMa3-BB
76.48%
44.76%
54.09%
43.19%
68.00%
42.80%
36.81
Ours
78.86%
43.95%
54.56%
43.40%
68.01%
48.76%
36.52
A
+2.38%
-0.81%
+0.47%
+0.21%
+0.01%
+5.96%
-0.29
D.1.5 Evaluation of MURKA on Smaller-Scale Models
This experiment aligns LLaMa-8B and LLaMa-3B models using the MURKA framework and eval- uates their performance on four benchmarks: N1.40tt, Mamo Easy, NI.4IIP, and OptiBench, with ER and SA as metrics. Results, shown in Table 9, indicate that MURKA performs better on LLaMa- 8B. Model scale has a greater impact on SA than ER, particularly in complex tasks. We recommend prioritizing larger models when resources permit to enhance performance.
Table The SA and ER compared to the smaller-scale models.
Metrics
ER
SA
ER
SA
ER
SA
ER
SA
Method
NL40pt
Mamo Easy
NLP4LP
OptiBench
LLaMa-8B
100.0%
93.5%
100.0%
95.9%
100.0%
87.6%
91.4%
69.3%
LLaMa-3B
96.1%
77.0%
96.9%
83.7%
95.0%
76.5%
90.6%
48.1% 1%
A
-3.9%
-16.5%
-3.1%
-12.2%
-5.0%
-11.1%
-0.8%
-21.2%
D.2 Train Dataset Construction and Augmentation
To construct a high-quality dataset for optimization problems, we systematically curated data, en- suring balanced representation of scenarios and problem types, as detailed in Table 10 and Ta- ble From the test set, we randomly sampled 20% of the instances, stratified by scenarios and types, to serve as the foundation for knowledge distillation and data synthesis. Initially, we utilized

---

Table 10: The scenarios of the benchmarks.
Scenarios
N1.40p
Mamo Fasy
Mamo Complex
IndustryOR
NT.P4LP
ComplexOR
OptiBench
OPMATH
Agriculture
30
5
14
0
Energy
33
7
5
22
6
Health
40
53
49
Retail
47
37
Environment
Education
32
0
3
D
Financial Services
46
2
6
Transportation
78
76
48
Public Utilities
Manufacturing
71
68
230
57
Software
10
2
Construction
56
Legal
Customer Service
Entertainment
34
0
0
6
0
Others
100
4
29
37
Sum
230
652
211
242
18
605
166
Table 11: The optimization types of the benchmarks.
Types
NL4Opt
Manno Easy
Maro Complex
IndustryRR
NLP4LP
ComplexOR
OptiBench
OpıMATII
98
59
I
36
422
14
20
MIP
412
48
44
130
113
NP
0
2
0
26
9
65
9
19
0
MOP
SOCP
Others
0
0
Sum
230 652
211
100 242
18 605
166
powerful inference models, such as DeepSeek-RI and Qwen3-235B, as teacher models to generate seed data. These models performed reasoning-based analysis the sampled problems, producing Problem-AMPL. pairs, which map optimization problems to their AMPL formulations. To expand the diversity of scenarios, types, and data, we applied data augmentation techniques, leveraging prompt engineering and domain-exper input to refine and enrich the seed dataset while ensuring correctness of the AMPL formulations.
The augmented Problem-AMPL. pairs were then solved using the Gurobi optimizer. We iteratively refined the AMPL models based on solver logs, adjusting formulations within fixed number of iter- ations until successful convergence was achieved. Finally, information extracted through prompt engineering and domain experts. Valid Problem-AMPL. pairs were retained as the final output of this stage. Through this distillation and augmentation process, we generated a training dataset com- prising 3,602 instances, ensuring complete separation between training and test sets prevent data leakage. This dataset serves as a robust resource for training optimization models, with diverse scenarios and problem types reflective of real-world benchmarks.
D.3 Computational Resources and Training Configuration
To solve the optimization model, we use Gurobi 12.0.1 [Gurobi Optimization, LLC, 2025]. The training of MURKA's extractor (E) and solver (S) components was conducted on single GPU, leveraging its 24 GB of VRAM. The solver's longer training time and higher computational cost are attributed to its larger maximum sequence length distinct hyperparameters, as detailed in Table 12. During inference, MURKA requires approximately 16 GB of VRAM (using FP16 preci- sion), and the Gurobi solver runs on the CPU with a peak RAM usage of around 40 GB for the most complex cases.
25

---

When comparing computational costs, direct latency comparison with baselines like Chain-of- Experts challenging, as they rely on API calls closed-source models (c.g. GPT-4), making their performance subject to network variability and external load. Therefore, we advocate using token consumption more stable and equitable metric for cost, directly reflecting both API ex- penses and computational workload. Under this metric, MURKA is dramatically more efficient, reducing token usage by over 20x compared to Chain-of-Experts and also outperforming other learning-based methods. This provides strong, concrete evidence for the scalability and efficiency of our framework.
Table 12: Hyperparameter Configuration for Extractor and Solver.
Model| Epoch Batch LearningRate LoRA_R Max Length WarmUp Ratio
Weight Decay Adam Beta
3
25
16
5e-6
16
1024
0.1
0.1
0.9
S
15
32
5e-5
8
3072
0
0
0.9
E
Prompt template
E.1
Information Extraction
Prompt
input__ariables = ["question"],
template = -..
You are professional optimization problem analyst, proficient in extracting key ele- ments from optimization problems described in natural language. Your task is to accurately output the sets, parameters, variables, objective function, and constraints specific format.
Ensure that the output is concise, professional, and meets the requirements.
Here is the specific description of the optimization problem: {question]
Please extract the required information from the following optimization problem ac- cording the format below:
<Sets>
List the sets involved in the problem here. such as: set_name: description of the set
</Sets>
<Parameters>
List the parameters involved in the problem here. such as: parameter name: description of the parameter
</Parameters>
<Variables>
List the variables involved in the problem here. such as: variable_name: name: description of the variable
</Variables>
<Objective>
Specify the objective function of the problem here. such
26

---

Muximize/Minmmiz:: objective function </Objective>
<Constraint>>
List the constraints existing in the problem here. such as: constraint_
</Constraints>
E.2
Code Generation
Prompt
input__ariables= variables = ("question", "extracted_inff
template = -..
You are optimization expert.
You should solve question step by step within specified label tags.
Solve the optimization problem and only output the complete AMPL model code within <AMPL>/AMPL>>tgs.
Problem: (question]
Extract Information: cestracted_inoo
Ensure that the output strictly adheres to the correct AMPL, syntax and structure for model files.
E.3
Knowledge Distillation
Prompt
iiria__vriables= = ["Problem")
template = mmn
The following is an operations research problem.
Problem description: [Problem]
Let's solve it step by step:
step 1. Understand the problem
Please extract the key information for the optimization problem from the following natural language description:
Problem description: Provide detailed description of the task problem, including the business background and the specific optimization goal.
Decision variables: Identify the variables that need to be decided (e.g.. quantity, time,
27

---

allocation, etc.).
Objective function: Clearly define the optimization goal (c.g., maximizing profit, minimiz- ing cost, minimizing time, etc.). Constraints: List all the limiting conditions that affect decision-making (e.g.. resource constraints, time limitations, etc.).
step 2. Develop the mathematical model
Based the analysis in step construct the mathematical model:
Decision variable symbols: Define each decision variable with symbol and explain its meaning.
Objective function: Express the optimization goal with a mathematical formula.
Constraints: List all constraints and express them with mathematical formulas, including boundary conditions and other restrictions.
step 3. Implement the model in AMPL
Using the mathematical model developed in step 2, write the AMPL code:
Declare variables: Define each decision variable and its value range.
Objective function: Write the AMPL expression for the objective function.
-Constraint: Write the AMPL expressions for each constraint.
step Output the AMPL code
Output the complete AMPL model code within <AMPL></MPL>ttgs.
28


Reviews:
[{"review_id": "dMZbr0ERkO", "reviewer": "Reviewer_LeaD", "summary": "This paper presents MURKA, a framework that uses reinforcement learning and knowledge distillation to convert natural language descriptions into optimization programs. The system has three agents: an Extractor that parses problem descriptions, a Solver that generates AMPL code, and a Checker that verifies solutions. The Extractor is trained with a modified GRPO algorithm using multi-dimensional rewards. The Solver learns from a teacher model through knowledge distillation. Tests on eight benchmarks show MURKA achieves 5.9% better solution accuracy and 5.1% better execution success compared to existing methods.", "questions": "Can you provide concrete numbers on training time, inference latency, and memory usage? How does MURKA compare to baselines in terms of computational cost?", "limitations": "The authors partially address limitations in Section 5. They mention assumptions about structured input, template dependency, and potential bias from teacher models. However, they don't discuss computational costs, and all that ., so I would recommend the authors to improve this section.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "1.nThe paper presents a well-structured pipeline with three specialized agents. The division of labor between extraction, solving, and checking makes sense.\n2. The authors test on eight different benchmarks covering various optimization problems. Results show consistent improvements across most metrics.\n3.The authors show that both the Extractor and Solver components contribute to performance, with clear evidence in Figure 4.\n\n4.The core ideas (multi-agent systems, RL for LLMs, knowledge distillation) are not new. The contribution is mainly in combining these existing techniques.\n5. While the paper claims to address scalability (C1), there's no analysis of training time, inference latency, or computational requirements compared to baselines.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your time and for providing thoughtful and constructive feedback on our work. We are encouraged by the recognition of our well-structured pipeline, extensive experiments evaluation, and clear ablation studies clear. \nWe appreciate the critical points raised, especially regarding computational cost analysis and novelty. Below, we address each concern in detail and outline the improvements we will incorporate in the final version.\n\n### Point-by-Point Response\n\n#### **Concern: Lack of computational cost analysis**\n\nWe appreciate the reviewer’s request for a more detailed computational cost analysis. While we initially provided this in Appendix D.2 and D.3, we agree that these insights should be brought forward and clarified. Below, we offer a comprehensive summary:\n\n**Training Cost & Data Efficiency:** Our framework is highly efficient in terms of both training resources and data requirements.\n\n- **Hardware and Time:** The Extractor and Solver were trained on a single NVIDIA RTX 4090 GPU (24GB). Training times were approximately 46 hours for the Extractor and 110 hours for the Solver (see Appendix D.3, Table 11).\n- **Data Efficiency:** Crucially, MURKA achieves superior performance while being remarkably data-efficient. Our training dataset contains only 3,602 instances, which is merely **36.7%** of the data used by a strong baseline like LLMOPT.\n\nDespite using significantly less training data, MURKA surpasses LLMOPT's performance, highlighting the effectiveness and efficiency of our alignment strategy. This proves our approach not only achieves state-of-the-art results but does so with substantially lower data and computational overhead for training.\n\n**Inference Cost:**\n\n- **Latency:** In terms of computational efficiency, our method exhibits superior performance. On the NL4Opt, NLP4LP, and ComplexOR benchmarks, MURKA’s average total inference latencies (the sum of LLM generation time and Gurobi solver time) are 37.0s, 36.1s, and 54.5s respectively. This represents a speedup of over 2x compared to OptiMUS, confirming a substantial advantage in efficiency.\n- **Memory:** During inference, MURKA requires approximately 16 GB of VRAM (using FP16 precision). The Gurobi solver runs on CPU with a peak RAM usage of around 40 GB for the most complex cases.\n\n**Cost Comparison with Baselines:** Direct latency comparison with baselines like Chain-of-Experts is challenging, as they rely on API calls to closed-source models (e.g., GPT-4), making their performance subject to network variability and external load. Therefore, we advocate using token consumption as a more stable and equitable metric for computational cost, as it directly reflects both API expenses and computational workload.\n\n| **Method**       | **NL4Opt** | **NLP4LP** | **ComplexOR** |\n| ---------------- | ---------- | ---------- | ------------- |\n| Chain-of-Experts | 38230.2    | 42985.2    | 47740.1       |\n| OptiMUS          | 15937.4    | 12972.0    | 13522.2       |\n| LLMOPT           | 1912.0     | 2078.3     | 4530.7        |\n| **Ours**         | **1602.2** | **1621.0** | **2067.8**    |\n\n*Table: Average Token Count (Prompt + Completion) per Task*\n\nAs the table above clearly shows, MURKA is **dramatically more efficient**, reducing token usage by over 20x compared to Chain-of-Experts and also outperforming other learning-based methods. This provides strong, concrete evidence for our claim of creating a scalable framework (C1).\n\n#### **Concern: Novelty of the work**\n\nWe greatly appreciate you raising the concern about novelty, as it gives us an opportunity to more clearly articulate the core contribution of our work. As you pointed out in the Strengths section, the true novelty of our work lies in the systematic and organic integration of these existing techniques, along with their targeted adaptation, to create a \"well-structured\" and collaborative framework where the \"division of labor makes sense.\" This framework is designed to efficiently solve the complex challenge of converting natural language into verifiable optimization programs.\n\n### Concluding Remarks\n\nIn summary, we hope the responses above have clarified the two main issues you raised. First, we have provided a detailed computational cost analysis, demonstrating that the MURKA framework holds significant advantages in training data efficiency, inference latency, and token consumption. Second, we have reiterated that the core novelty of our work lies in the systematic integration and targeted adaptation of existing techniques to create an efficient, collaborative framework. Based on your valuable feedback, we will incorporate the full cost analysis into the final version of our paper and restructure the introduction to better highlight the systemic innovation of our work."}, {"review_id": "LPCyVIQzmX", "reviewer": "Reviewer_DipB", "summary": "The paper presents MURKA, an end-to-end framework that automates converting natural-language optimization problems into executable models and verified solutions via a three-agent pipeline including Extractor, Solver and Checker.​\n\nKey Contributions​\n* Modular design separates extraction, generation, and verification, boosting transparency and reducing latency.​\n* Multi-dimensional RL reward offers fine-grained feedback on extracted specs' syntax and semantics.​\n* Cross-domain knowledge distillation lets the 8B-parameter model match or outperform larger LLMs in optimization tasks at lower compute costs.​\n* Validated on 8 benchmarks (e.g., NL4Opt, Mamo), with 5.9% higher solution accuracy and 5.1% higher execution success rate than prior methods. Ablation studies confirm Extractor and Solver value.​\n\nMURKA is an effective, scalable alternative to prompt engineering or monolithic fine-tuning for LLM-driven optimization.", "questions": "1. Could you please provide more analysis on the reason why the performance of the proposed method is worse than DeepSeek R1? Is it because long-CoT models with enhanced reasoning capabilities are especially useful in this task? Is it indicate that R1-distilled qwen 7B should be used in this task rather than llama3 8B?", "limitations": "No", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "​One of the key aspects of MURKA is its modular design. By clearly separating the tasks of information extraction, model generation, and verification, the framework enhances transparency. This modularity also has the added benefit of reducing latency, as each component can be optimized independently. The ability to understand and modify individual parts of the system without affecting the others makes MURKA more accessible and easier to maintain.​\n\nThe multi-dimensional RL reward system employed in MURKA provides fine-grained feedback on both the syntactic and semantic aspects of the extracted specifications. This detailed feedback mechanism is essential for improving the quality of the information processed by the framework. It allows for more precise adjustments during the learning process, ensuring that the output models are both accurate and meaningful.​\n\nAnother significant feature of MURKA is its use of cross-domain knowledge distillation. This technique enables an 8B model to match or even outperform larger models in optimization tasks while keeping compute costs low. By leveraging knowledge distillation, MURKA can achieve high-performance results with a relatively smaller model, making it more resource-efficient and scalable.​\n\nHowever, one notable concern regarding MURKA is its performance in comparison to DeepSeek R1. Although MURKA shows strong results on some tasks, there are instances where its performance falls short of that of DeepSeek R1. It is possible that long-CoT models with enhanced reasoning capabilities, such as DeepSeek R1, have an inherent advantage in NL-to-optimization tasks. The performance gap might indicate that for achieving optimal results, alternative models like R1-distilled Qwen 7B, which could potentially combine the benefits of knowledge distillation with more advanced reasoning mechanisms, might be more appropriate than the LLaMA3 8B model used in MURKA.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your thoughtful and encouraging review. We especially appreciate your recognition of our key contributions, including the transparency of the modular design, the fine-grained feedback from the multi-dimensional RL reward, and the efficiency of our cross-domain knowledge distillation. We appreciate your insightful question regarding the performance comparison with DeepSeek-R1, and we hope the following clarifications will address your concern and reinforce the value of our work.\n\n### Point-by-Point Response\n\n#### **Concern about performance compared to DeepSeek-R1.**\n\nThe comparison to DeepSeek-R1 is indeed a crucial point. As noted in Table 7, the baseline DeepSeek-R1 used in our comparison is a **671B-parameter model**, which is over **80× larger** than the LLaMA3-8B model used in MURKA. Despite this disparity, MURKA achieves comparable or even superior performance on several benchmarks (e.g., Mamo Complex: 55.6% vs. 51.2%), underscoring our framework’s effectiveness and computational efficiency. This is a primary contribution of our work: bridging the performance gap with significantly smaller, more practical, and deployable models.\n\nWe agree with your hypothesis that models with enhanced reasoning capabilities are particularly effective for NL-to-optimization tasks. Your insight that performance is tied to the problem's characteristics is excellent and gets to the heart of the matter. We hypothesize the performance differences stem from two distinct types of challenges in NL-to-Optimization:\n\n- **Complex Reasoning Chains:** For problems where the solution requires intricate, multi-step logical deductions and dependencies (e.g., complex conditional logic), a model with strong Long-CoT capabilities like DeepSeek-R1 naturally has an advantage. Its general reasoning prowess allows it to connect disparate pieces of information across a long context. This is likely why it excels on certain benchmarks.\n- **Robust Structural Extraction:** However, many optimization problems hinge on the precise and robust extraction of numerical values, relationships, and constraints from dense or ambiguously worded text. This is where MURKA excels. A generic Long-CoT approach can sometimes be brittle, misinterpreting a single critical value or relationship, leading to an entirely incorrect formulation. Our **Extractor agent**, in contrast, is specifically optimized via a multi-dimensional RL reward to systematically and accurately parse these structural components. It is trained to be robust against the very ambiguities that can derail a general reasoning process.\n\nIn essence, while DeepSeek-R1's strength lies in its general problem-solving, MURKA's advantage is in its specialized, high-fidelity conversion of natural language into a structured, verifiable model. \n\n#### **Concern about the choice of the LLaMA3-8B backbone.**\n\nYou raise an excellent point about our choice of LLaMA3-8B versus other models like Qwen-7B. Our selection of LLaMA3-8B was based on its strong, publicly verifiable performance in English comprehension—as evidenced by its leading position on the Hugging Face Open LLM Leaderboard for models of its class, making it a solid foundation for our experiments. However, the core technical contribution of our work is the **model-agnostic nature of the MURKA framework itself**. Our framework is designed as a plug-and-play enhancement for any capable base model. It can be seamlessly applied to Qwen-7B, LLaMA3-8B, or any future, more advanced open-source models. This adaptability is key because, as you rightly suggest, the ideal solution involves a synergy between different capabilities.\n\n### Concluding Remarks\n\nIn summary, we have addressed your valuable questions by clarifying our design choices. We explained that MURKA's competitive performance against much larger models stems from its specialized strength in **robust structural extraction**, which complements the general reasoning abilities of models like DeepSeek-R1 by targeting a different, yet critical, set of problem characteristics. Furthermore, we highlighted that the MURKA framework is intentionally **model-agnostic**, designed to synergize with and enhance any underlying LLM. Based on this valuable discussion, we will expand the future work section in the camera-ready version to explicitly discuss these complementary strengths and the framework's adaptability. We believe these clarifications strengthen the paper and are grateful for the opportunity to discuss these points."}, {"review_id": "P0cojodHFI", "reviewer": "Reviewer_jiyM", "summary": "This paper introduces MURKA, a framework designed to automate the process of solving Operations Research problems from natural language descriptions. The core of MURKA is a multi-agent system composed of three specialized agents: an Extractor, a Solver, and a Checker. The Extractor, trained using reinforcement learning with a composite reward function, parses the natural language problem to identify key components. The Solver, enhanced through knowledge distillation from a powerful teacher model, then formulates this structured information into executable AMPL code. Finally, the Checker verifies the solution's correctness using feedback from an optimization engine like Gurobi. The authors demonstrate through extensive experiments on 2,224 OR problems that MURKA, using a LLaMa3-8B backbone, outperforms existing methods in solution accuracy and execution success rate.", "questions": "Questions\n1. The composite reward function includes four weighted components.  How sensitive is the Extractor's performance to these weights, and how well would this reward function generalize to a new domain or problem type not represented in the training benchmarks (e.g., Second-Order Cone Programming)? Would introducing a new problem type require a full re-design and re-tuning of the reward function?\n2. The shampoo example in Figure 1 illustrates the extraction of explicitly stated numerical constraints.  How does MURKA handle problems where constraints are implicit or require deeper semantic understanding and logical deduction to formulate correctly? For instance, a problem might describe relationships that need to be translated into a system of equations not explicitly provided in the text. How does the framework push the LLM to perform this kind of multi-step reasoning, rather than just information extraction?\n3. What is AMPL? (A Mathematical Programming Language)?", "limitations": "Yes, the authors have adequately addressed the limitations in Section 5.  They correctly identify the reliance on structured problem descriptions, the potential inflexibility of predefined reward templates, and the risk of inheriting biases from the teacher model during knowledge distillation.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The paper tackles the challenging and highly relevant problem of automating OR modeling from natural language. The proposed MURKA framework is original in its combination of a multi-agent architecture, reinforcement learning with a multi-faceted reward function (GRPO), and knowledge distillation for this specific task. This approach presents a significant step forward from simpler prompt-based or supervised fine-tuning methods.\n2. The design of the MURKA framework is well-structured. Decomposing the complex problem into specialized agent tasks (Extract, Solve, Check) is a logical approach that enhances modularity and interpretability. \n3.  The paper is very well-written, clear, and easy to follow. The framework is explained logically, and Figure 2 provides an excellent visual overview of the different components (Extractor Alignment, Solver Alignment). The extensive appendices provide commendable detail on the experimental setup, hyperparameters, and prompts, ensuring a high degree of reproducibility.\n\nWeaknesses:\n1. The framework's effectiveness, particularly the Solver agent, is heavily dependent on knowledge distillation from a high-capacity teacher model like DeepSeek-R1. This creates a potential bottleneck, as the student model's quality is capped by the teacher's performance, and it requires access to powerful, often proprietary models for training. This reliance is particularly critical because, as the paper's own results show, even the best-performing LLMs struggle with the most complex OR problems; solution accuracy on the Mamo Complex and Specific benchmarks remains low, around 51-55% and 37-45% respectively. This highlights a more fundamental issue: while the framework excels at teaching an LLM to structure explicitly stated information into a correct format (like AMPL code), its ability to infer complex, unstated constraints or formulate entirely new mathematical relationships seems limited. The framework serves more as a powerful method for optimizing an LLM's performance on known problem patterns rather than a system that fundamentally expands its core logical and mathematical reasoning abilities for the OR domain.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your thorough evaluation, constructive feedback, and insightful questions. We are encouraged that you found our MURKA framework to be original, well-structured, and the paper clear and reproducible. We will first address your technical questions before turning to the concerns about model performance and reasoning depth, aiming to comprehensively clarify all points raised.\n\n### Point-by-Point Response\n\n#### **Question: Sensitivity and generalization of the composite reward function.**\n\n1. **Sensitivity to Weights**: To analyze sensitivity, we conducted an ablation study in which each reward component was removed in turn to assess its individual contribution. Results are summarized below:\n\n| **Method**      | **NL4Opt** | **NLP4LP** | **ComplexOR** |\n| --------------- | ---------- | ---------- | ------------- |\n| **Ours (Full)** | **93.5%**  | **87.6%**  | **72.2%**     |\n| $ω_{sim}=0$     | 78.6%      | 73.4%      | 58.5%         |\n| $ω_{sem}=0$     | 60.2%      | 57.9%      | 39.2%         |\n| $ω_{constr}=0$  | 55.8%      | 52.1%      | 34.6%         |\n| $ω_{format}=0$  | 51.3%      | 48.5%      | 30.1%         |\n\n*Table: Ablation study on reward components' impact on SA (%).*\n\nThese results demonstrate that each component is essential. The dramatic drop in performance upon the removal of any single component robustly validates the synergistic effect and soundness of our multi-faceted reward mechanism. In particular, the catastrophic failure when the format ($ω_{format}$) and core logic ($ω_{constr}$, $ω_{sem}$) rewards are removed underscores that the model must first grasp the fundamental structure of a problem to have any chance of solving it effectively.\n\n\n2. **Generalization**: Our reward design is based on general principles of mathematical optimization—extracting variables, parameters, objectives, and constraints—rather than task-specific heuristics. This abstraction allows MURKA to generalize across diverse optimization types, including MIP, LP, and NLP. As shown in Table 5, MURKA achieves consistent results across seven different optimization categories.  For a new optimization type like Second-Order Cone Programming (SOCP), which can be expressed with these same fundamental components, the reward function would generalize well without requiring a full redesign. For tasks outside optimization, we show in Table 9 of our paper that our alignment does not significantly degrade the model's general capabilities.\n\nWe will incorporate this ablation study and discussion into the final version of the paper.\n\n#### **Question: What is AMPL?**\n\nThank you for catching this omission. We will revise the paper to define AMPL (A Mathematical Programming Language) upon first mention, as referenced in line 79 ([Fourer et al., 2003]).\n\nTo elaborate, AMPL is a high-level, algebraic modeling language designed specifically for describing and solving large-scale optimization problems. We chose AMPL for several key reasons that align with our framework's design:\n\n- **Separation of Model and Data**: AMPL allows the logical structure of a problem (the model) to be defined separately from the specific data used. This modularity is a perfect fit for our Extractor-Solver architecture. The Solver agent can focus entirely on generating a syntactically and logically correct model file, while the specific numerical values (parameters) can be handled independently.\n- **Solver Independence**: An AMPL model can be solved by a wide variety of optimization engines (e.g., Gurobi) without changing the model itself. This provides flexibility and allows us to leverage the power of state-of-the-art commercial solvers like Gurobi for the final numerical optimization step.\n- **Readability**: AMPL's syntax closely resembles standard mathematical notation, making the generated models relatively easy for human experts to read, verify, and debug, which is crucial for building trust in automated systems.\n\nWe will clarify this in the final version of the manuscript to better justify our choice of AMPL as the target language.\n\n#### **Concern: Dependency on a powerful teacher model and the performance cap on complex problems.**\n\nThank you for raising this critical point, which touches upon both the practical value of our work and a fundamental challenge for the field. Our response is twofold:\n\n1. **Value Proposition: Enabling Practical, Low-Latency Deployment**: We agree that the student model's performance is influenced by the teacher. However, the primary goal of our knowledge distillation strategy (Sec 2.3) is not simply to mimic the teacher, but to **democratize and operationalize** its capabilities. We transfer the specialized reasoning patterns of a massive, proprietary model like DeepSeek-R1 (a 671B parameter model) into a much smaller, more efficient, and openly accessible student model (LLaMa3-8B).\n\n   This addresses a critical barrier to real-world adoption. Many industrial applications, such as dynamic production scheduling on a factory floor or real-time logistics routing, are highly sensitive to network latency and require local, on-premise deployment. Relying on constant API calls to a massive, cloud-hosted model is often infeasible due to network delays, data privacy concerns, and prohibitive operational costs. By successfully distilling expertise into a compact 8B model, MURKA enables the development of lightweight, locally deployable OR tools, which we see as its primary application value. The fact that our 8B model approaches the performance of a model over 80x larger (Fig 3a) highlights the success of this practical knowledge transfer.\n\n2. **Performance on Complex Problems as a Field-Wide Challenge**: We fully agree with your insightful observation that even powerful models like DeepSeek-R1 struggle with the most complex OR problems. This indeed suggests, as you noted, that current LLMs may be adept at generating syntactically correct formats without achieving full, deep logical comprehension. This is an important, acknowledged limitation of the *current generation of all LLMs*, not a flaw unique to our framework.\n\n   Our results should be viewed in this context. As shown in Table 1 of our paper, DeepSeek-R1 achieves only 51.2% on Mamo Complex. Our 8B model's performance (55.6%) is not only highly competitive but even surpasses the teacher in this instance. This demonstrates that while the absolute performance ceiling is tied to the progress of foundation models, our framework provides a state-of-the-art method for aligning models to this complex domain, making the most of the reasoning capabilities that are currently available.\n\n#### **Concern: The framework is more for formatting than deep reasoning and struggles with implicit constraints.**\n\nWe apologize if the shampoo example in Figure 1 oversimplified our framework's capabilities. We agree that natural language descriptions are often ambiguous. A core, implicit benefit of our framework is that it forces a translation from this ambiguous prose into a structured, formal representation. This process of extraction and structured code generation inherently acts as a disambiguation step, clarifying the problem's logical relationships before solving. \n\nWe will add a more complex example to the appendix in the final version to better illustrate this process.\n\n### Concluding Remarks\n\nIn summary, we have addressed the key points raised in your review. We demonstrated the necessity of each component in our composite reward function with a new ablation study and clarified that its design is generalizable. We also elaborated on our choice of AMPL, highlighting its suitability for our framework. Furthermore, we contextualized our work's primary contribution as enabling practical, low-latency OR modeling. We believe these clarifications and additions will significantly strengthen the paper and are grateful for the opportunity to improve our work."}, {"review_id": "wF2SAvEM5K", "reviewer": "Reviewer_Pqa3", "summary": "The paper introduces MURKA, a reinforcement learning and knowledge distillation-based multi-agent framework for translating natural language descriptions of optimization problems into verifiable, executable solutions. MURKA leverages three specialized agents: Extractor, Solver and Checker. It is trained via a composite reward function and guided by distilled knowledge from a high-capacity teacher model. It achieves strong results on eight OR benchmarks, outperforming several state-of-the-art methods in both solution accuracy and execution robustness, despite using a relatively small 8B-parameter LLM backbone.", "questions": "* How does the composite reward function balance semantic accuracy vs syntactic format?\n* What is the role of each reward component in the final performance gains?\n* How sensitive is performance to the weights of the reward terms?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* Novel architecture.\n* Effective training.\n* Competitive results.\n* I like that it achieves high performance using a smaller LLaMa3-8B model, something that shows cost-efficiency.\n* Thorough evaluation results and ablation study\n\nWeaknesses:\n* It relies on reliance on synthetic data, something that I don't like.\n* Sometimes it slightly underperforms in certain domain-specific cases, for instance, in IndustryOR.\n* Paper difficult to read, but the overall idea is good.", "was_revised": true, "final_justification": "Thank you. I'll keep my score. During the rebuttal, my concern about the reliance on synthetic data has been resolved. However I still believe that the model underpeforms in certain domain-specific cases and the paper still remains difficult to read.", "rebuttal": "We sincerely thank you for your detailed review and constructive feedback. We are encouraged that you recognized the novelty of our architecture, the effectiveness of our training strategy, the competitive results, and the cost-efficiency of achieving high performance with a smaller 8B model. We appreciate the opportunity to address your concerns regarding the reliance on synthetic data, performance on specific benchmarks, and the clarity of the paper. We hope the following clarifications will resolve your concerns.\n\n### Point-by-Point Response\n\n#### **Question: How does the composite reward function balance semantic accuracy vs. syntactic format?**\n\nThe balance is explicitly managed by the weighted combination of the four reward components, as defined in Equation 6. The weights ($ω_{format}$,$ω_{constr}$,$ω_{sem}$,$ω_{sim}$) were carefully tuned (as noted on line 163) to establish a hierarchy. **Syntactic format** ($R_{format}$) is foundational; an ill-formatted model is not executable. Therefore, its weight is prioritized to ensure the agent first learns to generate syntactically valid structures. **Semantic accuracy**, captured by the combination of $R_{sem}$, $R_{constr}$, and $R_{sim}$, is prioritized next. These rewards ensure the model is not only executable but also logically correct—representing the right objective function and constraints. This hierarchical weighting encourages the model to first become “fluent” in the AMPL language, and then “accurate” in its mathematical reasoning. Our ablation studies validate this design (see below).\n\n#### **Question: What is the role of each reward component in the final performance gains?**\n\nTo directly answer this, we conducted an ablation study on the reward components. The results clearly demonstrate that each component plays a critical and distinct role in achieving the final performance.\n\n| **Method**      | **NL4Opt** | **NLP4LP** | **ComplexOR** |\n| --------------- | ---------- | ---------- | ------------- |\n| **Ours (Full)** | **93.5%**  | **87.6%**  | **72.2%**     |\n| $ω_{sim}=0$     | 78.6%      | 73.4%      | 58.5%         |\n| $ω_{sem}=0$     | 60.2%      | 57.9%      | 39.2%         |\n| $ω_{constr}=0$  | 55.8%      | 52.1%      | 34.6%         |\n| $ω_{format}=0$  | 51.3%      | 48.5%      | 30.1%         |\n\n*Table: Ablation study on reward components' impact on SA (%).*\n\n\n- **Removing Format Reward** ($R_{format}$): This is the most critical component. Without it, the Extractor produces malformed text, leading to frequent compilation errors in the downstream Solver. This severely impacts both execution and accuracy, especially on datasets with complex structure like ComplexOR.\n- **Removing Constraint Reward** ($R_{constr}$): This is the next most critical. Without it, the Extractor fails to correctly identify constraints, leading to models that are logically flawed or contain syntax errors. This results in a massive performance drop.\n- **Removing Semantic Reward** ($R_{sem}$): This is similarly vital. If the objective function is wrong, the model solves an entirely different problem. The impact is comparable to getting the constraints wrong, as it corrupts the core logic of the optimization model.\n- **Removing Similarity Reward** ($R_{sim}$): This reward addresses more subtle logical errors. Without it, the model might still produce a structurally sound model but could confuse variables (e.g., use the cost of Product A for Product B). This leads to significant, but less catastrophic, performance degradation compared to the other three components.\n\n#### **Question: How sensitive is performance to the weights of the reward terms?**\n\nOur hyperparameter tuning revealed a clear and logical hierarchy of sensitivity. The model's performance is most sensitive to the weights in the following order: $ω_{format} > ω_{constr} > ω_{sem} > ω_{sim}$. This ranking aligns perfectly with the ablation results above. A non-negotiable foundation of correct formatting is required before the logical components (constraints and objective) can have any meaning. The fine-grained similarity check provides a final polish. This robust hierarchy suggests that the reward design is principled. We will add a note on this sensitivity analysis to the appendix in the final version.\n\n#### **Concern: Reliance on synthetic data**\n\nWe understand and share your concern about synthetic data.  Our primary motivation was to enable our reinforcement learning (RL) and knowledge distillation framework. **Large-scale, real-world datasets with structured annotations for this task are not readily available, particularly because optimization problems in industrial settings often involve sensitive business data, making them private and difficult to obtain for public research.** Therefore, synthetic data serves as a crucial and practical tool to distill reasoning patterns from a powerful teacher model and to provide the diverse feedback signals needed for our multi-reward RL process. Crucially, we demonstrate that this training strategy generalizes effectively to unseen, real-world problems. As shown in Tables 1 & 2 and Figure 3, MURKA achieves state-of-the-art or highly competitive performance across eight diverse, established benchmarks, indicating that the model learns robust, generalizable optimization principles rather than overfitting to synthetic artifacts. In the camera-ready version, we will add a more detailed discussion on this point in Section 5 (Discussion).\n\n#### **Concern: Slight underperformance in specific cases**\n\nWe appreciate you pointing out this specific case. As shown in Table 2, our model (38.0% SA) slightly underperforms LLMOPT (44.0% SA) on the IndustryOR benchmark. We have analyzed our experiment logs and identified a key reason for this discrepancy. Due to hardware constraints during model alignment, we set a `Max_Length` of 3072 for the input sequence (as noted in Table 11). We found that many problems in the IndustryOR benchmark have context lengths that exceed this limit, leading to input truncation and a subsequent drop in performance. This is a technical limitation rather than a fundamental flaw in our method's reasoning capability. Despite this, we wish to emphasize that MURKA demonstrates superior overall performance and robustness. It outperforms all baselines on multiple other benchmarks (e.g., NL4Opt, NLP4LP) and achieves a significantly higher Execution Rate (ER) of 97.0% vs. 92.7% for LLMOPT, which is critical for real-world deployment. We will add a brief analysis of these specific failure cases to the appendix of the final paper.\n\n### Concluding Remarks\n\nIn summary, we have addressed the main concerns raised. We clarified that our use of synthetic data enables our framework and leads to strong generalization on real-world benchmarks, and we explained that the slight underperformance on IndustryOR stems from a technical sequence length limitation. Furthermore, our ablation study demonstrates the critical and distinct role of each component in our composite reward function. In response to your valuable feedback, we will revise the paper for clarity and add further analysis on data limitations, specific failure cases, and reward sensitivity to the appendix of the final version. We believe these clarifications and improvements will strengthen our paper and more clearly highlight our contribution."}]

Author rebuttal:

Meta-review: This paper presents MURKA, a multi-agent framework combining reinforcement learning with a composite multi-dimensional reward and knowledge distillation to translate natural language descriptions of optimization problems into executable, verifiable programs. The modular Extractor–Solver–Checker design improves interpretability and efficiency, with ablation studies showing each component’s necessity. Experiments on eight OR benchmarks demonstrate consistent gains in solution accuracy and execution success over baselines, while achieving low inference cost. Reviewers praised the technical soundness, thorough evaluation, and practical deployability, though they noted moderate conceptual novelty, reliance on synthetic data, and limited performance on the most complex reasoning tasks. The authors’ rebuttal addressed these points with additional cost analysis and clarifications. However, the writing requires significant improvement: the paper is at times difficult to follow and not fully self-contained, which reduces accessibility for the broader community. Taking all factors into account, I find the contributions meaningful and empirically validated, but with moderate novelty and clarity issues. I recommend acceptance as a Poster, but strongly urge the authors to substantially improve clarity and self-containedness in the camera-ready version.
Decision: Accept (poster)