OptMATH: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
Authors: Hongliang Lu
Zhonglin Xie
Yaoyu Wu
Can Ren
Yuxuan Chen
Zaiwen Wen
Venue: ICML 2025 poster (2025)
Keywords: Optimization
LLM
Optimization Modeling
Synthetic Data
Primary area: optimization
TL;DR: We presents a bidirectional data synthesis framework, termed OptMATH, for constructing datasets on optimization modeling tasks. Experiment results show that models trained on OptMATH achieve better performance on various benchmarks.

Abstract: Despite the rapid development of large language models (LLMs), a fundamental challenge persists: the lack of high-quality optimization modeling datasets hampers LLMs' robust modeling of practical optimization problems from natural language descriptions (NL). This data scarcity also contributes to the generalization difficulties experienced by learning-based methods.
To address these challenges, we propose a scalable framework for synthesizing a high-quality dataset, named OptMATH. Starting from curated seed data with mathematical formulations (MF), this framework automatically generates problem data (PD) with controllable complexity. Then, a back-translation step is employed to obtain NL. To verify the correspondence between the NL and the PD, a forward modeling step followed by rejection sampling is used. The accepted pairs constitute the training part of OptMATH. Then a collection of rejected pairs is identified and further filtered. This collection serves as a new benchmark for optimization modeling, containing difficult instances whose lengths are much longer than these of NL4OPT and MAMO.
Through extensive experiments, we demonstrate that models of various sizes (0.5B-32B parameters) trained on OptMATH achieve superior results on multiple modeling benchmarks, thereby validating the effectiveness and scalability of our approach. The OptMATH dataset and related resources are available at \url{https://github.com/optsuite/OptMATH}.

OptMATH:
A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
Hongliang Lu
Zhonglin Xie 2 Yaoyu Wu
Can Ren
3 Yuxuan Chen Zaiwen Wen
Abstract
1. Introduction
Automatic translation of natural language descriptions of optimization problems into solver-ready formats is a criti- cal step in democratizing access to optimization techniques, This capability would enable individuals without expertise in optimization to leverage the power of optimization for solving real-world problems across various domains, includ- ing logisties (Ghiani et al., 2022), finance (Consigli, 2019), and engincering (Rao, 2019). Known as optimization mod- eling, this task has long been challenging due the inherent ambiguity of natural language and the need for deep under- standing of optimization modeling principles. The manual process of formulating an optimization problem typically involves iterative refinement and demands significant mas- tery of the relevant techniques, making time-consuming and inaccessible to many practitioners.
Despite the rapid development of large language models (LLMs), a fundamental challenge per- sists: the lack of high-quality optimization mod- eling datasets hampers L.LMs' robust modeling of practical optimization problems from natural language descriptions (NL). This data scarcity also contributes to the generalization difficulties experienced by learning-based methods. To ad- dress these challenges, we propose a scalable framework for synthesizing a high-quality dataset, named OptMATH. Starting from curated seed data with mathematical formulations (MF), this framework automatically generates problem data (PD) with controllable complexity. Then, back- translation step is employed to obtain NL. To ver- ify the correspondence between the NI. and the PD, forward modeling step followed by rejec- tion sampling used. The accepted pairs consti- tute the training part of OptMATH. Then col- lection of rejected pairs is identified and further filtered. This collection serves as new bench- mark for optimization modeling, containing dif- ficult instances whose lengths are much longer than these of NL4OPT and MAMO. Through ex- tensive experiments, we demonstrate that models of various sizes (0.5B-32B parameters) trained on OptMATH achieve superior results on multiple modeling benchmarks, thereby validating the ef- fectiveness and scalability of our approach. The OptMATH dataset and related resources are avail- able at https:://// / con/ opt suite/ O.HMATT.
Recent advances in Large Language Models (LLMs), such ChatGPT (Brown et al., 2020b), GPT-4 (OpenAI et al., 2023), and OpenAl's ol (OpenAI, 2024), have demonstrated remarkable capabilities in understanding natural language and performing complex reasoning tasks. Notably, the intro- duction of ol has significantly enhanced the performance of LLMs in mathematical reasoning, achieving state-of-the- results on challenging datasets including AIME (2024), GPQA, and CodeForces. However, optimization modeling presents unique challenges. Unlike grade-school mathemat- ics, where problems typically have single correct solu- tion, optimization problems can often be approached using multiple valid models, making the task semi-open-ended. Furthermore, optimization frequently relies on a vast body of empirical knowledge that is less formally structured than purely mathematical concepts. As result, research has shown that directly applying LLMs to optimization model- ing tasks yields suboptimal outcomes (Ramamonjison et al., 2021).
'Equal contribution, alphabetical order 'College of Engineer- ing, Peking University "Beijing International Center for Math- ematical Research, Peking University "School of Mathematics Science, Peking University. Correspondence Zaiwen Wen <wenzw@ pku.edu.cn>>
LLMs for Optimization Modeling. To address this issue, recent efforts have explored various strategies. Research leveraging LLMs for optimization modeling typically follows two main approaches. The first uses prompt engi- neering techniques to guide L.MMs in generating refin- ing optimization models, without modifying the underlying model parameters. Examples include the NL4Opt compe-
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

OptMATH:: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
tition (Ramamonjison al., 2021), which aims to extract optimization formulations from natural language descrip- tions, and OptiMUS (AhmadiTeshnizi et al., 2023), which proposes an agent-based prompt engineering method. More recently, Autoformulation (Astorga et al., 2024) combines Monte Carlo tree search with LLMs to address optimization modeling. While these methods rely heavily on the base capabilities of general-purposs LLMs, they do not yield fundamental advancements beyond refined prompting.
dresses data scarcity by developing generative models that preserve instance hardness and constraints. Examples in- clude the bipartite graph variational autoencoder framework proposed by (Geng et al., 2023), the block decomposition operators for constraint matrices introduced in (Liu et al., 2024b), the duality-driven feasibility guarantees established by (Wang et al., 2023), and the adaptive constraint modi- fication mechanisms developed in (Guo et al., 2024). The rule-based approach leverages instance space analysis tech- niques (Alipour et al., 2022; Strassl & Musliu, 2022: Alipour & Smith-Miles, 2023) to guide the generation of more di- verse instance distributions (Smith-Miles & Bowly, 2015; Bowly, 2019). Alternatively, it may rely on manually se- lected features to control the characteristics of the generated instances (Bowly et al., 2020).
The second approach centers on fine-tuning, wherein model parameters are adapted using specially curated or synthe- sized optimization datasets. ORLM (Tang et al., 2024) intro- duces
a data augmentation framework for opti- mization problems, and demonstrates performance gains via Supervised Fine-Tuning Tuning on a foundation model. LLMOPT (Jiang et al., 2024) likewise applies a multi-instruccie fine- tuning strategy and self-correction mechanisms. Similarly, OptLLM (Zhang et al., 2024) presents framework that fine- tuned a Qwen model using large-scale self-developed op- timization dataset, reporting an accuracy boost over prompt- based models. However, many of these methods still tend to generate synthetic training data that can be limited in quantity, or may lack consistent quality and sufficient com- plexity. Consequently, their ability to generalize to more sophisticated optimization tasks can be constrained.
Our Contributions. We propose scalable bidirectional synthesis framework that addresses the critical challenge of data scarcity in optimization modeling through triplet- aligned (NL, MF, PD) data generation and rigorous vali- dation. Our framework uniquely integrates a closed-loop workflow with optimal value matching, ensuring seman- tic equivalence between NL, MF, and PD. This approach demonstrates exceptional scalability. The framework's 's do- main adaptability is evidenced by coverage of 10+ real- world applications (e.g., logistics, energy, finance) through 53 seed generators, with manual analysis confirming 99.65 9.6% equivalence accuracy across all triplets.
LLM-Based Data Synthesis for Optimization. Data syn- thesis methods have become essential for addressing data searcity and enhancing the performance of LLMs. Accord- ing to (Wang et al., 2024). these methods can be broadly cat- egorized into two main approaches: data augmentation and data synthesis. IR-Instrcct (Tang et al., 2024; Wang al., 2022) exemplifies a data augmentation approach, following the self-instruct framework to expand existing datasets. In (Yang et al., 2025), a framework is proposed for the reverse synthesis of optimization problem data pairs. However, this approach initiates reverse synthesis from formatted demon- strations, which inherently limits its scalability and does not ensure the correspondence between the generated code and the natural language descriptions. MIL.--xoll (Li et al., 2024) introduces an evolutionary framework designed to generate diverse mixed-inted -ixtedinteger linear programming (MILP) problems using LLMs. While MILP-Evvlve and other syn- thesis efforts represent significant steps forward in synthetic data generation, the critical challenge of translating natural language descriptions into robust mathematical optimization models often remains key area for advancement, which our OptMATH framework directly addresses through its scalable, bidirectionally validated approach.
We introduce OptMATH-Train, large scale verified opti- mization modeling dataset containing rigorously validated (NL., MF, PD) triplets. Each triplet undergoes three-stage mathematical consistency checks (MF- to-PD compilation), semantic fidelity validation (PD-to- NI. backtranslation), and solution equivalence verification through solver- rejection sampling. From rejected instances, we curate OpiMATH- -Bnch,, a challenging bench- mark comprising "hard instances" characterized by ex- tended natural language contexts (2.9x longer than MAMO EasyLP) and complex constraints. We further span it us- ing various problems including IP, MILP, IP, NLP, SOCP. This benchmark provides the standardized evaluation for long- long-context optimization modeling.
quality control:
Finally, extensive experiments demonstrate the efficacy of our framework. Models trained on the OptMATH- -Taii dataset achieve ofate-of-the-art performance established modeling benchmarks, including NL.4PT and MAMO. These results definitively validate the effectiveness and scalability of our framework for generating high-quality optimization modeling datasets.
multiple
Instance Generation for Optimization. Recent research in MILP instance generation has evolved along two primary axes: learning-based structural synthesis and rule-based distribution expansion. The learning- based paradigm ad-
2

---

OptMATH:: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
2. Backgrounds & Overview
where prompty is modeling prompt template map- ping NL to MF' and PD'. The quality metric Q evalu- ates the generated (MF'PD') pairs based on the verified (NL, MF, PD) triplet. Constraint (3) formalizes the auto- mated formulation process (Autoformulation), as depicted in the example in Appendix D.I. Optimizing relies on hav- ing a large corpus of high-quality triplets (NL, MF, PD). To address this requirement, we propose a systematic frame- work for generating synthetic training data that maintains mathematical rigor.
In mathematical optimization theory, a canonical optimiza- tion problem can be formulated as:
min subject to
g(x),
(x(x) = 0, i E
(1)
(xxx 2 0, E
where x E IR" denotes the decision vector. The objective function g R" + R assigns a scalar value to each candi- date solution, which we seek to minimize. The constraint functions Q; I2" + R define the feasible region through equality constraints indexed by & and inequality constraints indexed by I.
An Overview of Our Pipeline. The pipeline of our frame- work is presented in Figure 1. In the reverse data gener- ation phase, we collect optimization problems from two sources: (1) LP/MPS files from challenging benchmarks such as MIPLIB 2017 (Gleixner et al., 2021) and netlib (Netlib, 1990; Gay, 1985), and (2) over 50 expert-curated seed problem generators covering diverse optimization sce- narios. Through our carefully designed backtranslation pipeline, we leverage both the LP files and their MIs to gen- crate high-quality NLs of optimization problems. Notably, using an L.LM-based feedback workflow with evaluation, our collected generators can produce tremendous PDs with controllable varying difficulty levels. This enables us to effectively address the data scarcity challenge in training learning- based optimization methods.
To formalize the optimization modeling problem, we define several key concepts and illustrate them using a concrete example in Appendix D.I. For a specific optimization prob- lem, we define NT. as the natural language description, which corresponds to the "Input-Natural Language Descrip- tion" in the example. We represent the LP/MPS file, the concrete mathematical expression (corresponding to the "Output-Instanee Formulation"), and any other solver-ready formats, such as executable Python code with Gurobi shown in Appendix D.I, as problem data (PD). A common char- acteristic of these representations is that they allow us to obtain the optimal value of the problem by invoking a solver based on the PD. Mathematical formulation (MF) refers to formulation where concrete numbers are not yet specified, corresponding to the "Output-Geners Formulation" in the example. We emphasize that, in subsequent sections, we may use different forms of PD. However, these forms es- sentially carry the same information about the problem and can be inferred from the context without ambiguity. We use different forms of PD facilitate their integration into the workflow and to enhance clarity in various contexts.
In the forward modeling and evaluation process, we uti- lize our trained AutoFormulator to translate the generated NLs back into PDs. Specifically, in this phase, all PDs are represented as solver code, which can then be exported as LP files. We then implement rigorous rejection sampling strategy, where only instances whose optimal objective val- ues match between the original and generated LP files are retained. This equivalence-basse sures the high quality of our OptMATH- Train dataset by guaranteeing the semantic consistency of each instance.
filtering mechanism en-
Modern solvers such as Gurobi and Mosek (Gurobi Opti- mization, LLC, 2024; MOSEK ApS, 2025) can efficiently solve problems stored with PDs using algorithms like interior-point methods (Karmarkar, 1984). However, prac- tical challenges remain. In world applications, one of the main difficulties lies in converting informal NLs of prob- lems into precise MFs. Moreover, extracting the PDs from NL. poses an additional significant challenge. Tradition- ally, this process has required deep optimization expertise (Boyd, 2004), but recent advances in LLMs offer promising opportunities to automate this transformation.
Building upon the high-quality instances obtained through rejection sampling, we employ various data augmentation strategies to further enhance the diversity and coverage of our dataset. This enriched collection of training pairs is then utilized to fine-tune a foundation model, leading to AutoFormulator, a specialized model specifically designed for automated mathematical optimization modeling.
3. Feedback-Driven PD Generation
The mature development of the optimization community has provided us with access to many high-quality optimization PDs. These PDs are typically stored in standardized formats like MPS or LP files. To effectively leverage these resources, we began by curating over 50 seed problem classes sourced from a variety of optimization journals and websites (see Appendix A.2 for details). For each i-th problem class, we
Let Ag represent an LLM parameterized by 0. The formula- tion for increasing the modeling capability of the LLM can be expressed as:
max E(NI,MF,D))-DDQ (NI, (MF', .PD)], (2)
s.t. (MF', PD') As (NL)))
3

---

OptMATHI: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
Step1: Reverse Data Generation
Step2: Forward Modeling and Evaluation
Quality Filtering
Rejection Sampling
Benchmark (Hard I.P Files)
Generator
Generated LP Files
I. Files
Reject
min c²x s.t. Ax s b
Backtranslation Pipeline
LLM/Expert
Math Formula
Natural Language Description
AutoFormulator
Step3: Fine-Tuning
0
Augmentation
Fine Tuning
OptMATH-Train
Filter
Augmented Data
Base Model
Figure overview our scalable, bidirectional data synthesizing pipeline.
developed a corresponding instance generator G₁. This gen- erator takes a problem-specific configuration input and outputs a probability distribution over PDs. The distribu- tion is designed to produce PDs with varying scales and complexities, which are controllable through adjustable con- figurations. Before delving into the controlled generation process for the PDs, we first explain how the complexity of PDs can measured.
tribution of each component to the overall complexity. To illustrate it, we provide an example in Appendix B. that considers MIP problem incorporating multiple constraint types to optimize production costs for two products under resource constraints.
Selecting Parameters to Control the Complexity. We now present the workflow in Algorithm 1 for selecting parameter configurations for instance generator G2 to generate PDs fitting the complexity, feasibility, and solving time require- ments. The prompt templates are illustrated in Appendix E.4. As formalized in Algorithm 1, the process begins by specifying target bounds. Then template promptic for initializing the configuration are incorporated. After obtain- ing the configuration, we generate PDs using it. We then evaluate the generated PDs through the complexity score, solving time, and feasibility satisfactory. Then, feedback promptRc created based the statistics of these metries over the N generated PDs. The LLM iteratively adjusts parameters based on feedback from solved instances, ulti- mately converging to configuration that satisfy predefined criteria. This ensures generated PDs remain both expressive and tractable by adhering to runtime thresholds.
Measuring the Modeling Complexity. The complexity of formulating and solving MIP problem depends on mod- eling choices such as the types of variables, the forms of constraints, and auxiliary modeling techniques employed. We introduce a scoring function S defined as:
S(PD) = Obin Noin + Otim Nim + Oreons -
+ Pin-Nin + Bindic-Nindie + Aqquad Nqund
(4)
+ Bpen + "BiggM /BigM + Öexpr Lespe:
where Nbin; Nint, Noont are the number of binary, inte- ger, and continuous variables, respectively. Similarly, Nin: .Vindde Nquud, Ngen represent the number of linear, in- dicator, quadratic, and general nonlinear constraints. The term SBigM is factor reflecting the frequency of Big-M formulations, and Lexpr is the average number of terms per constraint and the objective function, which captures the structural information of the expressions. Lastly, the weights o., (,,, BigM, despe are tunable parameters reflecting the con-
4. The Data Synthesis Framework
This section presents bidirectional scalable data synthe- sis framework. In this section, all of the PDs are in solver

---

OptMATHI A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
Algorithm 1 FredbbckkDrrien Problem Data Generation
Algorithm 2 Bidirectional Data Synthesis Algorithm
Require: Target complexity range [Smin, Smax], time lim-
Require: Instance pair (MF; (MF,,DD,,)) Max Iteration T
its [Tmin: Tmax] instance generator G, feasibility threshold Ftarget, max iterations T
Ensure: (NL,,j. (NL, 2,j. PD 1.3
D.g. OV,,)
1: Initial generation: NT. <- [(prompt (MF,, PDtj))
Ensure: Configuration O such that for PD; G(O): S(PD,) [mmin maxx] (complexity), Ti I Tmax (solv-
Initialize: SC SR Null
3: for k = 1,...., do
ing time), Pr(f: = feasible 2 Ftarget
4: Self-Criticize:
1: Initialize parameters via LLM:
SC < [(prompt (MF;, PDa.. NL))
O + ((promptic (Smin, Smax, Tmins Tmax))
5: Self-Refine:
2: for t = 1 to T do
SR < [prompt (MF,, SC,
3: Generate N PDs: (PD,), t G(O,-1)
if SR is good enough then
4: Compute metries: S(PD.) (Eq. 4), Ti (solving time),
break end if end for
fi (feasibillity)
5: Aggregate statistics: S, N S(PD.) Ft
1 E F. NI = feasible)
10: NL,, + SR
N
6:
if S. E [smin Smax] and i, I max and F, Z
11: AutoFormulation:
Fraret then return 1--1
(MF', PD(,) + Ao(pompp ())
7:
12: OVi, f- Solve PD..j by Gurobi
8: else
13: OV < Solve PD by Gurobi
9:
Refine parameters via feedback:
14: ir i. = OVL,, then
θ, < (promptRc (S,, Fe, Fri Or-1)
15: return (NLgg. MF f.j PD OV.,)
10: end if
16: else
11:
end for
17: return Null
12: return (no valid O found)
18: end if
code form (see subsection 4.2 for more details). Let L represents the LLM employed on the reverse data genera- tion phase, and Ag for our fine-tuned AutoFormulator with weights B. We define prompt,, promptc. promptR as the prompt templates that accept certain inputs for the initial generation, self-critism, and self-refinement stages. The algorithm is formalized in Algorithm 2, with an illustrative example of the backtranslation process shown in Figure 13,
tasks often benefit from iterative refinement approaches rather than direct generation (Madaan et al., 2024). This observation aligns with human problem-solving processes in mathematics, which typically requires multiple attempts and refinements. Building upon this insight, we design a three-phase backtranslation pipeline that systematically im- proves the quality of generated descriptions through iterative refinement. All prompt templates used in this pipeline can be found in E.
The final OptMATH dataset D is constructed by collecting all valid quadruples (NL,,,ME t.j, PD', Dy) that pass the validation process, where MF and PD4, represent the generated mathematical formulation and problem data using Ag. oV is the optimal value obtained by solving the problem specified by PD... By leveraging our instance generators and the iterative refinement process, this algo- rithm enables scalable generation of high-quality data pairs. The mathematical equivalence between the generated for- mulations and the original instances is rigorously validated through rejection sampling, ensuring the reliability of our dataset. A comprehensive discussion of our quality control and rejection sampling can be found in Section 4.3.
Initial Generation. Given the mathematical formulation MF, and the corresponding problem data PD:., of a prob- lem j in i-class, the LLM generates an initial natural lan- guage description NL using the prompt template prompt,. This stage requires the model to comprehend both the math- ematical semantics and the instance parameters to produce preliminary human-readabl description.
Self-Criticism.. Using prompt tromlttemplate prompt the LLM evaluates the current description by examining the math- ematical equivalence with MF,, completeness of the con- straints and objective functions, clarity and comprehensi- bility, and consistency of the parameters with PD;,- The criticism SC in iteration & incorporates feedback from all previous iterations to guide improvements.
4.1. Backtranslation Pipeline
To generate high-quality NLs of optimization problems at scale, we leverage a specific LLM as the foundation of our pipeline. Recent research has demonstrated that complex
Relf-Refinement. Based on the criticism, the model gen- erates refined descriptions SR with the prompt template promptR- The refinement process focuses on improving
5

---

OptMATHI: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
the mathematical accuracy, completeness of the constraints, and clarity of the descriptions.
rate. We acknowledge that determining the exact equiva- lence between two mathematical formulations remains an open research question worthy of further investigation. Nev- ertheless, our current approach provides a practical and highly effective mechanism for ensuring dataset quality.
This process iterates for T rounds until satisfactory de- scription NL,,, is obtained, with each iteration potentially improving the quality of the generated description. Based on our empirical analysis (see Appendix C.2), we set T"= in the final implementation.
5. Fine- Tine-Tunnig
5.1. Data Augmentation
4.2. Forward modeling
To improve the diversity of our dataset, we use data augmen- tation to augment the training data. This method generates more non-standard problems compared to data genera- tor, enhancing the model's generalization performance. We create rules for problem rewriting, semantic substitution, constraint expansion, and numerical augmentation. For each instance, a randomly selected rule is used to prompt the LLMs to generate the corresponding augmented data. The detailed augmentation rules and prompt templates can be found in Appendix E.7.
Building upon the NLs generated in subsection 4.1, we leverage AutoFormulator to transform them back into MFs and PDs in solver code form, enabling rejection sampling for quality validation. Given NL as input, AutoFormu- lator produces two key outputs: MF and corresponding PD in solver code form. While previous works (Tang et al., 2024; Jiang et al., 2024) adopted fixed output formats, our approach is not constrained to any particular format, as our primary goal to obtain correct solver code, with the formulation serving as an intermediate reasoning step. To facilitate genuine mathematical modeling capabilities rather than superficial format mapping, we design diverse Chain- of of-Thought (CoT) prompting strategies (Wei et al., 2022). This approach generates multiple valid reasoning paths and formulation variants for the same problem, enriching our training data with diverse modeling perspectives and en- hancing the model's mathematical reasoning capabilities. Detailed implementation of these CoT strategies is described in Appendix D.I.
For quality control, we employ a specific LLM L to sample each augmented description twice independently. followed by the rejection sampling strategy described in Section 4.3. This process yields approximately 10 qualified augmented datasets for each problem, and this method was applied to augment 50 thousand instances to complement original dataset.
5.2. Training the AutoFormulator
We adopt supervised fine-tuning (SFT) approach en- hance the AutoFormulator's modeling capabilities. Specif- ically, we employ the LoRA algorithm (Hu et al., 2021) for efficient parameter-efficien fine-tuning, which signif- icantly reduces memory requirements while maintaining model performance by updating only a small set of adapter parameters. Using the OptMATH-Train dataset DSFT { (NL, MF, PD,)) ) NTaain we train the model to generate both mathematical formulations and solver code given prob- lem descriptions. For each training sample, the input con- sists of the problem description NL,, while the target output is the concatenation of the formulation and solver code: yi [MF;; PD,], where [:] denotes sequence concatena- tion. The training objective follows the standard sequence- to-sequence loss:
4.3. Rejection Sampling
To ensure the quality and mathematical soundness of our generated optimization problem descriptions, employ rejection sampling strategy (Yuan et al., 2023; Liu et al., 2024c) to filter and select high-quality samples from the generated candidates.
As illustrated in Algorithm 2, our rejection sampling mech- anism relies on solution-based comparison to validate the generated samples. Specifically, for each generated natural language description NLj we use AutoFormulator trans- form it into a mathematical formulation MF',, ", and solver code PD',-- .. obtaining solution OV- This solution is then compared with OV obtained by directly solving the orig- inal instance PD,,- A sample is accepted into our dataset 'D as a validated quadruple (NL,,,.M',, PD(5, OV.,)) if and only if oV.., OV.
M
(SF -B(
Σ
log Po (o
(5)
While this solution- -based validation approach may not guar- antee perfect equivalence (as problems with identical opti-
where ye represents the token at position L in the target sequence, and yer denotes all preceding tokens. This ap- proach allows the model to learn the mapping from nat- ural language problem descriptions to both mathematical formulations and solver code within a unified sequence-to-
mal values may represent different optimization problems), our analysis using an LLM-committee (Zhao et al., 2024) and manual inspection of randomly sampled instances (1% of the total dataset) reveals a remarkable 99.6% accuracy
6

---

OptMATHI: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
sequence framework.
4000
OpiMATH- Train
EssLP
3500
3,315
OpIMATH-
MAMO ComplesLP
6. Experiments
2,974
3000
6.1. Statistics of the OptMATH Dataset
2500
First, using our generators, we generated a quality-filtered dataset containing over 600,000 LP files, which span 53 distinct problem types and are distributed across five hard- ness levels. For more details on the seed data class, please refer Appendix A.2. To ensure computational feasibility, we impose solving time threshold and employ a feedback pipeline that leverages an LLM to regulate both the com- plexity and feasibility of the generated instances. Further details on this process are provided in Appendix B. The distribution of file lengths across these LP files visual- ized in Figure As shown, the lengths range widely from 1,000 25,000 characters, capturing rich variety of prob- lem complexities. The proportions of different lengths are well-balanced, with a concentration on medium difficulty levels (which are already quite challenging compared to other benchmarks) and gradual decline as the problems become harder. Additionally, the distribution confirms the effectiveness of our complexity control mechanism.
1500
1,045
1000
541
Average Question Length (character)
Figure 3. Question length analysis
Type
NL40KT
MAMO ExsyLe
MANO Complast
GpoMATIH-demd
IF
36.0%
1544%
11,0055
LP
51.3%%
481%
82.8%%
101%
MILP
11.4%
48.6%%
21.3%%
STATS
KLP
0.69%
0.90%
600%
5.1%%
0.1%%
0.97%
810%
622%
Figure 4. The proportion of problems in diffcrent datasets.
To visualize the distribution of different benchmarks and OptMATH dataset, we project their high-dimensional em- beddings onto a 2D space using I-SNE(van der Maaten & Hinton, 2008). As shown in Figure 5, the instances from different sources form distinct clusters, suggesting that OptMATH effectively captures the diversity of differ- ent problem families. It can be observed that OptMATH surrounds the area of other benchmarks. This explains the improvement on various benchmarks obtained by training on OptMATH- -rain.
5
Easy
Medium Easy
Medium
Peroentage (%)
Medium Hard
3
Hard
2
1
20
5,000
10,000
15,000
20,000
25,000
Number of Characters
Figure 2. Distribution of I.P file lengths.
20
We further conducted a comparative analysis of problem lengths between OptMATH and other benchmark datasets, with their average lengths shown in Figure 3. The analy- sis reveals that OptMATH presents significantly more com- plex problem descriptions compared to existing benchmarks. This increased complexity, manifested through longer prob- lem descriptions, poses greater challenges for LLMs,, as longer descriptions typically demand enhanced comprehen- sion and reasoning capabilities.
-40
20
40
ComplexL.P OpIMATH Bench
OptMATII
MAMO EasyLP
Figure 5. Visualization of OptMATH and other benchmarks.
6.2. Autoformulation
As shown in Figure 4, - has selected num- ber of representative mathematical optimization problems covering wide range of application scenarios, including LP, MILP, IP, NLP. SOCP and other optimization problems. For details, please refer to Appendix A.
Evaluation Benchmarks and Metrics. We eval- uate our fine-tuned model on five benchmarks: NLAOPTTRammmonnjoon et al., 2021), MAMO(Huang et al., 2024), IndustryOR(Tang et al., 2024), OptiBench Yang et al., 2025) and our newly constructed OptMATH-Ber

---

OptMATH: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
Table I. Performance Comparison of Models Different Benchmarks
MAMO EasyLP
MAMO ComplexLP
OptMATH Bench
Types
Models
NL4OPT
IndustryOR
OptiBench
Macro AVG
Llama3. (omma3.1-8B(pp) Basss @
0.0%
0.2%
0.0%
0.0%
0.0%
0.0%
0.1 0.1%
Owen2.5-7B(pasSe
86. 9%
83.6%
21.8%
1.6%
10.0%
36,22 36.2%
40.0%
Baseline
GPT-3.5-turbopasseee
78.0%
79.3%
33.2%
15.0%
21.0%
47.4%
51.4%
GPT-A(pass@
89.0%
87.3%
49.3%
16.6%
33.3%
68.6%
57.4%
Deepsack-V3ppasse11)
95.9%
88.3%
51.1%
32.6%
37.0%
71.6%
62.8%
OptiMUS
(GP5----0044-0---13
78.8%
77.0%
43.6%
20.2%
31.0%
45.8%
49,4%
Bwve2..533BB(paseee
92.7%
82.2%
44.6%
9.3%
16.0%
47.6%
48.7%
ORLM-Llama-3333 (reported)
85.7%
82.3%
37.4%
38.09
60.9%
Fine-tuning
ORLM-LLama---888 (reproduced)
84.5%
74.9%
34.1%
2.6%
24.0%
51.1 %
45.2%
OpcMATH-LLlmma3.14 1-8B (pass
55.5%
73.9%
40.8%
24.4%
18.0%
55.5%
44.7%
SRMMATH-Qwen2.55 (pass@11
94.7%
86.5%
51.2%
24.4%
20.0%
57.9%
55.8%
OpiMATTHi-QQen225--222 (pass
95.9%
89.9%
54.1 %
34.7%
31.0%
66.1 %
62.0%
OptMATHH-Lamma3.1
97.6%
94.2%
71.6%
51.6%
37.0%
66.6%
69.8%
P
1HAMATH--Wen2.5-75
98.4%
94.5%
72.5%
56.0%
38.0%:
68.1%
71.3%
Opt
HHAMATH-Qwen2.5-3055 Qwen2.5-32B
97.9%
93.9%
75.4%
67.4%
47.0%
76.8%
76.4%
100
80
Model Fineluned Model
80
73.8%
:11.5%
76.0%
18.0%
76.9%
19.8%
Micro Accuracy (%)
-11.8%
59 99,9%
Accuracy (%)
60
50-
49,3%
40-
40
68.0%
%
20
SL407E
0.1%
..25%
0.5B
1.5B
3B
7B
14B
32B
Model Size
0.0
0.2
0.4
0.6
0.8
1.0
Figure 6. Scaling behavior of Qwen2.5 models (0.5B-32B).
Proportion
Figure 7, Accuracy of Qwen2.5-1.55 within one training epoch.
Detailed descriptions of these benchmarks can be found in Appendix A.1. We use pass @1 1 accuracy as the evaluation metric, which specifically measures whether the optimal value obtained by the generated code matches the ground truth provided in the benchmark. The detailed matching criteria are described in Appendix A. Notably, since prompt design significantly impact model performance, we maintain consistency by using the same prompt template across all model evaluations (see Appendix E.2 for details). Additionally, comprehensive details about our fine-tuning procedure are provided in Appendix D.3.
performs ORLM- 3-8- 3-8B, model of comparable size, demonstrates performance only marginally inferior to GPT- 4. Collectively, these results demonstrate that training with OptMATH-Tran significantly enhances the model's opti- mization modeling capabilities.
The results in Table also highlight that different base models possess varying inherent modeling capabilities. For instance, the Llama3. 10ama3.1-880 (Bubey et al., 2024) base model exhibits very poor initial modeling ability, with performance close to 0% on most datasets. However, after fine-tuning with our dataset (OptMATH-Llame)) 1-8),, its modeling capability is substantially improved. Despite this significant enhancement, the fine-tuned OptMATH-LLama3, 1-BB still performs below the level of OptMATTH-QQen22.. derscoring the impact of the base model's architecture or pre-training on final fine- tuned performance. Moreover, the strong pass @ results achieved by the fine-tuned models suggest high upper limit to their modeling capabilities,
Main Results. The primary results are presented Table First, our best-performing model, OpIMATH HHAMATH--wee2.5-330 Qwen2.5-32B, achieves superior performance across all benchmarks, sur- passing proprietary large language models such as GPT-3.5- Turbo(Brow) Brown et al., 2020a) and GPT4(OpenAl et al., 2023), and reaching a level comparable to Deepseek- -33Ls et al.,
un-
2024a), despite these models often having significantly more parameters. Furthermore, our OptMATH--Qwen2.5-1 out-
8

---

OptMATHI A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
indicating potential for further enhancement through re- inforcement learning methods, similar to approaches like Dee et al., 2025).
Impact Statement
This study introduces OptMATH, a dataset for optimization modeling, comprising a large-scale training set (OptMATH- Train) and challenging benchmark (OptMATH--Bnnd)) OptMATH has the potential democratize optimization by enabling those without expertise to translate real-world problems into mathematical formulations. The OpIMATH- Train dataset will significantly improve LLMs' ability to understand and model optimization problems. Further- more, OptMATH's structured data facilitates the integra- tion of optimization with advanced AI techniques like re- inforcement learning, Monte Carlo Tree Search. Addition- ally, OptMATH- Bearc provides standardized benchmark for evaluating optimization modeling systems, pushing the boundaries of LLM capabilities. Ultimately, OptMATH can improve efficiency and decision decision-making across industries.
Ablation Study on Model Size. To investigate the effective- ness of OptMATH training across different model scales, we conducted experiments using Qwen2.5 models ranging from 0.5B to 32B parameters. Due to computational constraints, we used a randomly sampled subset of 100,000 training examples. As shown in Figure 6, all models exhibit sub- stantial performance improvements after fine-tuning with OptMATH-Tran -rain. Notably, we observe that while larger models generally achieve better absolute performance, the relative performance gains from OptMATH- Train training demonstrate diminishing returns as model size increases.
Ablation Study on Data Size. Figure 7 shows how the amount of training data affects Qwen2.5-1.5B's perfor- mance across different benchmarks. We observed significant improvements in the model's 's optimization modeling capabil- ities even with only a small fraction of the OptMATH-Traie dataset. As we gradually increased the size of the training data, the performance gains became less pronounced, ex- hibiting a typical pattern of diminishing returns. Larger mod- els exhibit smoother learning curves, while smaller models demonstrate greater sensitivity to additional training data, indicating higher potential for improvement through data scaling (detailed results across model sizes can be found in the Appendix D.4).
References
Abura, J. Applying integer linear programming to the fleet assignment problem. Interfaces, 19(4):20-28, 1989.
Adams, J., Balas, E., and Zawack, D. The shifting bottleneck procedure for job shop scheduling. Management Science, 34(3):391-401, 1988. ISSN 00251909, 15265501. URL http: /www. j jttorr.....oo org/ http://taaall//eeee 2632051.
AhmadiTeshnizi, A., Gao, W., and Udell, M. OptiMUS: Optimization modeling using MIP solvers and large lan- guage models, 2023. URL http: http:///xxvv..rr////
7. Conclusion
In this paper, we introduce a bidirectional data synthesis framework for optimization modeling. It utilizes a two- step process: reverse data generation, where LLMs refine themselves in a loop to create diverse datasets, and auto- formulation, where a specialized model translates natural language into mathematical representations. Our evalu- ation on NL4OPT, MAMO and OptMAATH--Benchmarks demonstrated AutoFormulator's superior performance in generating accurate and well-formed optimization models compared to baseline approaches.
Alipour, H. and Smith-Miles, K. Instance space analysis for 2d bin packing mathemat- ical models. Discrete Optimization, 2023, URL https://ww..ceeneed..cet.
www.wwseeieccedreett..... com/
Alipour, H., Muñoz, M. A., and Smith-Miles, K. En- hanced instance space analysis for the maximum flow problem. European Journal of Operational Research, 2022. htRhttts: 2022UULLLLppss//www.eeeeeddddddd www.wwwdiricccccceett..... com/
Acknowledgments
Astorga, N., Liu, T., Xiao, Y., and Schaar, M. d. Aut- oformulation of mathematical optimization models us- ing LLMs, 2024. URL http: http///xxvvvoo// /abs/
The computational resources were supported by the Center for Intelligent Computing and Song-Shan Lake HPC Cen- ter (SSL-HPC) in Great Bay University, Dongguan, China. This work was supported in part by National Key Research and Development Program of China under the grant num- bers 2024YFA101229 and 022YFA11012903 and the Na- tional Natural Science Foundation of China under the grant numbers 12331010 and 12288101. We also thank the anony- mous reviewers for their valuable feedback.
Beasley, J. E.,, Krishnamoorthy, M., Sharaiha, Y. M., and Abramson, D. Scheduling aircraft landings-the static case. Transportation Science, 34(2):180-197 2000. doi: 10.12877tus..4422.880 12302.
Bertsekas, D. Network optimization: continuous and dis- crete models, volume Athena Scientific, 1998.

---

OpIMATH: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
Bowly, S. Stress lesting mixed inleger programming solvers through test instance generation methods. PhD thesis, University of Melbourne, Parkville, Victoria, Australia, 2019.
Cooper, L. G. Market-share models. In Handbooks in Op-
erations Research and Management Science, volume 5, pp. 239-314, Elsevier Science Publishers, 1993. URL
Bowly, S., Smith-Miles, K., Baatar, D., and Mittelmann, H. Generation techniques for linear programming instances with controllable properties. Math. Program. Comput.,
3 ofaloo pdd.. [Online; accessed 19-January-0025),
12(3))389-415, 2020. ISSN 1867-2949,1867-9957 1867-2957. doi: 10. 10.11077/113322-01-0000111 htte//..100//1123350 001007//122222-0--0001-00----111 00170-6
Cordeau, J., Pasin, E, and Solomon, M. An integrated model for logistics network design. Annals of Op- erations Research, 144:59-82, 2006. s s10479-006-0001-3
URL https:::...............tt /doi.
doi: 10.1007/
Boyd, S. Convex optimization. Cambridge UP, 2004.
Dantzig, G., Fulkerson, R., and Johnson, S. Solution of
Brown, G. G., Dell, R. F., and Newman, A. M. Op-
large-scale haveling-salesman problem. Journal of the Operations Research Society of America, 2(4):393-410,
timizing military capital planning. Interfaces, 34(6): 415-425, 2004. ISSN 00922102, 1526551X. URL
1954. doi: 10. 10.1287/0pr..2.4.399
http::/veet jww.ssstorr... http:///aaaaaaee/5222222222
Daskin, M.
Network and Discrete Location: Models, Algorithms, and Applications. ley, 1995. URL https:/ wi com/
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are fho-shot learners. Advances in neural information processing systems, 33: 1877-1901, 2020a.
Wi-
en-us en-us htwee//wooe.aaaaoieeeeeeeeeoaaaa 334Hocells22-AAgggritttmsss--aa3
implicationss htt.:/.ooooo---777711111111111 accessed 19-January20255,.
[Online;
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G.,
Drexl, A. and Kimms, A. Lot sizing and scheduling survey and extensions. European Journal of Operational
Askell, A., Agarwal, S., Mossb, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. Language models are few-shot learners, 2020b. URL http:// arxiv. org/ abs 20000005.... 14165.
Research, 99(2):221-236 1997. ISSN 0377-2217, doi: URL sctenne/Articee/////877721990009
Dubey, A., Jauhri, A., Pandey, A., Kadian, Al-Dahle, et al. The llama 3 herd of models. ArXiv, abs/2407.21783,
Burkard, R., Dell' Amico, M., and Martello, S. Assign- ment Problems. Society for Industrial and Applied Mathematics, 2012. doi: 10.1137/1.978116119977
2024. URL https: //api. wwwantisccooleee..............ee http:s///pp...emaattcccoooooooo hegp/CorpsID::21115
URL https: /twpbs/pppppp////// siam. org/doi/abs//0.
Fleischmann, B. The discrete lot-sizing and schedul- ing problem. European Journal of Operational Research, 44(3):337-348 1990. ISSN 0377-2217. doi: https//bbaan/////000005:::119999 URL https://www.scieeneddreeee. com/ sckenee/arriade/p/////7722222202
Caprara, A., Toth, P., and Fischetti, M. Algorithms for the set covering problem. Annals of Operations
Research, 98(1):353-371, 2000. URL https:
//www. dei www.uniii.iii.. wttppppp......i.eepppppeee.eee
survey_ serree__set_covrrrrngee wwwpeeerrinnprrbbbeeeo f.f. [Online; accessed 19- J00aar
Florian, M. and Klein, M. Deterministic production plan- ning with concave costs and capacity constraints. Man- agement Science, 12-22,, 1971.
Chen, G., Li, X., and Ye, Y. An improved analysis of Ip-based control for revenue management. arXiv preprint, 2022. URL https: org/o./ggoo.ooo abs/ 2101. 11092. [Online; accessed 19-Jauary-2025).
Ford, L. R. and Fulkerson, D. R. Maximal flow through a network. Canadian Journal of Mathematics, 8:399-404, 1956. doi: 10.4153/CJM- 1-560000--95---5-555
Consigli, G. Optimization methods in finance. Quan- titative Finance, 19:717 - 719, 2019. URL https:
Garey, M. R. and Johnson, D. S. Approximation algorithms for bin packing problems: A survey. In Analysis and design of algorithms in combinatorial optimization, pp.
//api. semanticacholatt hwtpah/ww....r.ooouueooseeee org/CorpusID: 262220633.
147-172. Springer, 1981.
10

---

OptMATHI: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
Garille, S. G. and Gass, S. I. Stigler's diet problem revis- ited. Operations Research, 49(1):1-13, 2001. doi: 10. 1287/oppre49...1.11158 URL https:/ /tte:://dooooo /doi.org/ 10. 1ww.////e/e.//// 49.1.1..1123.
selection for hardness-preserving milp instance genera- tion. In International Conference Machine Learn-
ing (ICML), 2024. URL https:/ conf / wetiiccmmmo////// htt.tttss//bbb..rrgggg https///blllp.og.... /rcc/
Gay, D. M. Electronic mail distribution of linear program- ming test problems. Mathematical Programming Society COAL Newsletter, 13:10-12, 12, 1985.
Gurobi Optiizzation, L. ing, 2025. URL https://www.qurol//
Optimization model-
com//ooumentationn 5/refman.
optimizatiion caatiion_mode ing. html. accessed 19-January-2025),
[Online;
Gendron, B., Crainic, T. G., and Frangioni, A. Multicom- modity capacitated network design. In Telecommunica- tions network planning, pp. 1-19. Springer, 1999.
Gurobi Optimization, LLC. Gurobi Optimizer Reference Manual, 2024. URL https: htpp:://wwwt /wwww gur.bi. con.
Geng, Z., Li, X., Wang. J., Li, X., Zhang, Y., and Wu, A deep instance generative framework for MILP solvers under limited data availability. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans,
Hazell, P. B. R. and Norton, R. D. Mathematical Programming for Economic Analysis in Agriculture. Macmillan Publishing Co., 1986. URL https:
/ // /www. www.reseacchhate.eeee http:/ttttee///pubiiiaiiooo 256475967. 256475967_Matthene htt.tMMatthtttittmmmmmm icaa haw.eetteeeettttlaaaorrrrmmmmmm
for Yor_Economic
[Online; accessed 19-Jamuary-2025].
LA, USA, December 16, 2023, 2023.
Hermann, J. W. (ed.). Handbook of Production Scheduling. International Series in Operations Research & Manage- ment Science, Springer New York, NY, I edition. 2006. ISBN 978-0-387-33115-77 doi: https://doi.rr//10.. 0-387-33117-4.
Ghiani, G., Laporte, G., and Musmanno, R. Introduction to Logistics Systems Management: With Microsoft Excel and Python Examples. John Wiley & Sons, 2022.
Gilmore, P. C. and Gomory, R. E. A linear program- ming approach to the cutting-stock problem. Operations Research, (()):449 -8984 1961. doi: 10. 10.1287/opre.966 849. URL httph:tt///// http:///.......1.1/....//1// 1ttp///0..107//.....9. 6. 8.9.
Hillier, F. and Lieherman, G. Introduction to Opera- tions Research. McGraw-Hill Education, 10th edition, 2014. URL https::/thuuiiee . vn 19 19-Jamuary-20255
www.hh..e................ edu [Online; accessed
Gleixner, A., Hendel, G., Gamrath, G., Achterberg, T., Bastubbe, M., Berthold, T., Christophel, M., Jarck, K., Koch, T., Linderoth, J.. Lübbecke, M., Mittelmann, H. D., Ozyurt, D., Ralphs, T. K., Salvagnin, D., and Shinano, Y. MIPLIB 2017: Data-Driven Compila- tion of the 6th Mixed-Integer Programming Library. Mathematical Programming Computation, 2021. doi: 10.11077/112232--200-0000444 URL https:://// /oi. org/10. 015//1..1007.0 hrt/10..007/s125532 020 0019--3..
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y.. Wang, S., Wang, L.., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09635 2021.
Huang, X., Shen, Q. Hu, Y., Gao, A., and Wang, B. Mamo: mathematical modeling benchmark with solvers. CoRR, abs/2405. 13144, 2024. doi: 10.48550/ARXIV, 2405.13144, URL https: htt////.iiia///// http://....//11..0/./. arXiv. -xXiv.2405: 2405. 13144,
Golden, B., Raghavan, S., and Wasil, E. (eds.). The Vehi- cle Routing Problem: Latest Advances and New Chal- lenges. Operations Research/Computer Science Inter- faces Series. Springer New York, NY, edition, 2008. ISBN 978-0 978-0-387-77777-- doi: 978-0-387-77778--8
Jiang. C., Shu, X., Qian, H., Lu, X., Zhou, J., Zhou, A., and Yu, Y. LLMOPT: Learning to define and solve general optimization problems from scratch, 2024. URL http: / //arxiv. org/ htabss/241.11333
Karmarkar, N. new polynomial algorithm for linear programming. In Proceedings of the sixteenth annual ACM symposium Theory of computing, pp. 302-311, 1984.
Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma. S., Wang, P., Bi, X., et al. Deepseck-rl: In- centivizing reasoning capability in lims via reinforcement learning. arXiv preprint arXiv:2501. 12948, 2025.
Klein, M. A primal method for minimal cost flows with applications to the assignment and transportation prob- lems. Management Science, 14(3):205-22 -200, 1967. doi: 10. 10.1287/mnse. 14.3.205.
Guo, Z., Li, Y., Liu, C., Ouyang, W., and Yan, J. Acm- milp: Adaptive constraint modification via grouping and

---

OpIMATHI A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
Kuby, M. J. Programming models for facility dis- persion: the p-dispersion and maxisum dispersion problems. Mathematical and Computer Mod-
OpenAI. Learning reason with Ilms: Introduc- ing openai ol. https://opent htwpopaccaoomoooooooooooooocoocc index/ satningg-t--reaae
-eto-rreso-wwthh----e www.wwwwhhhhhhhste 2024. Ac- cessed: 2024-12-21.
elling, 10(10):792, 1988. ISSN 0895-7177. doi:
com/ Kuhn, H. W. The hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2(1-2):83-
URL
OpenAl, Achiam, J., and Adler, e. a. Technical Re- port, 2023, URL http::////aatiteeeeeeeee/ee wtwg/bs/////eooooeo/oooo 23033. 08774.
Padhy, N. Unit commitment-a bibliographical survey, IEEE Transactions on Power Systems, 19(2): (9(2):1196691 196-1105, 2004. doi: 2000///TS...000..22222..
97, 1955. doi: https///ooooog///.00000oas00s100
URL https: https://onnnneeiibaryyywww doi/abs/ 100. www.:////wvv... http:/200..000000000.0
com/
Pinedo, M. L.. Scheduling: Theory, Algorithms, and Systems.
Springer Cham, 6 edition, 2022. ISBN 978-3-031-05920
Li, S., Kulkarni, J., Menache, I., Wu, C., and Li, B. Towards foundation models for mixed integer linear programming. arXiv preprint arXiv:2410.08288. 2024.
9. doi: https://di..or//100 1rt.///..rr//110107997878/88 0000//9778--00--5921111
Rajendran, C. Heuristics for scheduling in flowshop with
multiple objectives. Europan Journal of Operational
Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., et al. Deepseek- v3 technical report. arXiv preprint arXiv:2412.19437, 2024a.
Research, 82(3):540-555, 1995. ISSN 0377-2217.
doi: hutp:///aooo//000000607772111110 URL https://www.sceenceddieee octeenee/articceee/al///22222292
Liu, H., Wang, J., Zhang, W., Geng, Z., Kuang, Y., Li, X., Li, B., Zhang, Y., and Wu, F. Milp-studio: MILP instance generation via block structure decomposition. CoRR, abs/2410. 228020 2024b. doi: 10.48550/ARXIV. 2410.22806. URL https::t//// /tpp::/ddo..ooo http://doo.oog....../ 48550/ arXiv. 2410 22806.
Ramamonjison, R., Yu, T. T. L.,, Li, R., Li, H., Carenini, G., Ghaddar, B., He, S., Mostajabdaveh, M., Banitalcbi-Dehkorea A., Zhou, Z., and Zhang, Y. Ni4opt competition: Formulating optimization prob- lems based on their natural language descriptions. In Ciccone, M., Stolovitzky, G., and Albrecht, J. (eds.), NeurIPS 2022 Competition Track, November 28 De- cember 9, 2022, Online, volume 220 of Proceedings of Machine Learning Research, pp. 189-203. PMLR, 2021. URL. httppp:::pprocceeeiiii .mm.mllmmnmmm..m...m....mm.mmmm. ml wtt.g................. .press/ v220 11200/rammoni ramamonjisson222.11.
Liu, H., Zhang, Y., Luo. Y., and Yao, A. C.- C. Augmenting math word problems via itera- tive question composing. ArXiv, abs/2401.09003, 2024c. URL http://aaap.emmniccccoo.... ort/Coo:puSD::25702277
Rao, S. Engineering optimization: theory and practice.
Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye. S., Yang, Y., et al. Self-refine: Iterative refinement with self- feedback. Advances in Neural Information Processing Systems, 36, 2024.
John Wiley & Sons, 2019.
Schöbel, A. Line planning in public transportation: models and methods. OR Spectrum, 34:491-510, 2012. doi: 10. 00.1007//00291-01--05554
Markowitz, H. Portfolio selection. The Journal of Finance, 7(1):77-91, 1952. ISSN 00221082, 15406261. URL http://wwwr wttp://tttt//ttttalee/////3
Smith, D. Network flows: Theory, algorithms, and ap- plications. J Oper Res Soc, 45:1340, 1994. doi: 10. N0.11057/jor.11990 1994.208.
Martello, S. and Toth, P. Knapsack problems: algorithms and computer implementations. John Wiley & Sons, Inc., USA, 1990. ISBN 0471924202.
Smith-Miles, K. and Bowly, Generating new test in- stances by evolving in instance space. Comput. Oper. Res., 63:102-113, 2015. ISSN 3805-0548.1873-7665 doi: 10.101 20.10016)...r.2015..444 org/10.10166
URL https: htmp::///do /ooi. 2015. 04. 022.
MOSEK ApS. MOSEK Optimization Software, 2025. Ver- sion 1.0.3.
Solomon, M. M. Algorithms for the vehicle routing and scheduling problems with time window constraints. Oper. Res., 35:254-265, 1987. URL https: /api. s semanticscholar.orm
Netlib. Netlib: collection of mathematical software, papers, and databases. Available at http::/netlib. org, 1990.
12

---

OptMATH: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
SteadieSeifi, M., Dellaert, N., Nuijten, W., Van Woensel, T., and Raoufi, R. Multimodal freight transporta- tion planning: A literature review. European Journal of Operational Research, 233(1):1-15, 2014. ISSN 0377-2217. doi: https///ooiorg//oo.0ffffffo...:: 055. URL htt.:ttpes//www.ccceeeeeeeeeeeee com/ cciienee/atiiclee/i////777727771
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Chi, E. H., Xia, F, Le, Q- and Zhou, D. Chain of thought prompting elicits reasoning in large language models. ArXiv, abs/2201.11903, 2022. URL https: /appi. http://ppp...aam...ee.e semant commanttcschha cscholar. arg/CorppsID:2444111
Wikipedia. Supply chain management - Wikipedia, the free encyclopedia. Berg/w/inden wwwptppte=ttppppppppppet 2 20management & wol.ico12111110033... [Online; accessed 19-January-2025).
Strassl, and Musliu, N. Instance space analysis and algorithm selection for the job shop scheduling problem. European Journal of Operational Research, http:////wwwwwcceeeeeeeeeeeeeeee com/
http:://enwwikippeee
2025.
2022. URL / httpan//eaaaiiceeeeeeeeetteeee0/
Winston, W. L. Operations Research: Applications and Algorithms, Duxbury Press, 4th edition, 2004. URL https: //tp::///wwww academia. edu/ wwwpd//ww/ddd/////oo///o//////dd 43587201 /wi nston ip pdł. [Online; accessed 19-Jamuary-20251
Tang, Z., Huang, C., Zheng, X., Hu, S., Wang, Z., Ge, D., and Wang. B. ORLM: Training large language models for optimization modeling, 2024. URL //axiv//rxiv.. org/ org/abs/ 2o0bs/2405.22 17743.
Xiao, Z., Zhang, D., Wu, Y., Xu, I.., Wang. Y.J., Han, X., Fu, X., Zhong, T., Zeng, J., Song, M., and Chen, G. Chain- of-experts: When llms meet complex operations research problems. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL https: / //openreview net
Toregas, C., Swain, R., VeVelee C., and Bergman, L. The location emergency service facilities. Operations Re- search, 1966):1333-1333, 1971. doi: 10. 10.1287/opre.19 6.1363. URL https: /tt////oo..oo/// org/10. 1287 opre. 19. 6.363.
Toth, P. The vehicle routing problem. SIAM Monographs on Discrete Mathematics and Applications, 2002.
Yang, A., Yang. B., Zhang, B., Hui, B., Zheng. B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., et al. Qwen2. technical report. arXiv preprint arXiv:2412.15115. 2024.
van der Maaten, L. and Hinton, G. E. Visualizing data using t-sne. Journal of Machine Learning Re- search, 9:2579-2605, 2008. URL https://apt semanticschol. wttp:orppo//o......c..oo 5855044.
Yang. Z., Wang, Y., Huang, Y., Guo, Z., Shi, W., Han, X., Feng, L., Song, L., Liang, X., and Tang, J. Optibench meets resocratic: Measure and improve lims for optimiza- tion modeling. In The Thirteenth International Confer- ence on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025. URL https: / /poenreveew net fww.moorrrsoorr wtt.:/rrrnnn.-ifasssss....
Wang, H., Liu, J., Chen, X., Wang, X., Li, P., and Yin, W. DIG-MILP: Deep Instance Generator for Mixed- Integer Linear Programming with Feasibility Guaran- tee. arXiv preprint, 2023. URL https://arxiv. org/abs/23100 13261. Code:
Yuan, Z., Yuan, H., Li, C., Dong, G., Tan, C., and Zhou, Scaling relationship on learning mathematical reason- ing with large language models. ArXiv, abs/2308.01825. 2023. URL https: / /ppi. wwwanticscholae..........cc..... org/ /Co///ooI::
com/ com/
MILP.
Wang, K., Zhu, J., Ren, M., Liu, Li, S., Zhang, Z., Zhang, C., Wu, X., Zhan, Q. Liu, Q., et al. A survey on data synthesis and augmentation for large language models. arXiv preprint arXiv:2410. /2866, 2024.
Zhang, J., Wang, W., Guo, S., Wang, L., Lin, F., Yang, C., and Yin, W. Solving general natural natural-language- -agguaag---eeeiiiionn optimization problems with large language models, arXiv preprint
Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N. A., Khashabi, D., and Hajishirzi, H. Self-instruct: Aligning language models with self-generated instructions. arXiv preprint ar arXiv:2212. 10560, 2022.
2024.
Zhao, R., Zhang, W., Chia, Y. K., Xu, W., Zhao, D., and Bing, L. Auto-arena: Automating Ilm evaluations with agent peer battles and committee discussions, 2024. URL httpp://errii..or//aaass 2405. 20267.
Weatherford, L. R. and Bodily, S. E. Forecasting and con- trol of passenger bookings. Journal of Revenue and Pricing Management, 1(1):37-45, 1997. doi: 10.10 malgrave.rpm55170133 URL https: /tttttss://dd/ //w.iiioo//// ///d/o.or//...///////////////111 1057 (1577//algraee..e rpm. 5170134. [Online; accessed 19-January-2022
Zheng, Y., Zhang, R., Zhang, J., Ye, Y., Luo, Z., Feng, Z. and Ma, Y. Llamafactory: Unified efficient fine- tuning of 100+ language models, In Proceedings of the
13

---

OpIMATH: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
62nd Annual Meeting of the Association for Compula- fional Linguistics Volume 3: System Demonstrations), Bangkok, Thailand, 2024, Association for Computational Linguistics. http: / http:///wwxwvw....//aoooo / 2403. 13372.
Zieyel, E. R. Operations research applica- tions and algorithms. Technometrics, 30:361-362, 1988. URL https: / otg//oorustD::127 122790857.
htppppp..../ http:://pp....mmmnccccooooooooo semanticschola.:
14

---

OptMATHI A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
A. Dataset
A.1. An Introduction of Different Benchmarks
We evaluated the modeling capabilities of our trained model on NL4OPT, MAMO, IndustryOR, OptiBench, and our self-constructed dataset, OptMATH-BBnne Both MAMO and OptMATH- Bench have ground truth annotations, while the original NLAOPT dataset lacks ground truth. To address this, we utilized a LLM to generate initial ground truth for NL,OPT, followed by expert validation and correction for each data. As result, obtained the ground truth for the NL4OPT dataset. In addition, we have also analyzed these datasets in terms of problem scenarios and problem model types, and the distribution of scenarios for each dataset is shown in Figure 8, the distribution of problem types for each dataset is shown in 4.
Scenario
NL4OPT
MAMO EasyLP
MAMO ComplexLP
OptMATH-Benchh
Agriculture
6.12%
4.60%
2.37%
0.00%
Aviation
0.00%
0.00%
I
0.95%
7.25%
Construction
0.82%
6.60%
|
0.47%
0.52%
Education
0.82%
4.91%
0.00%
0.00%
Energy
1.22%
4.60%
2.37%
3.63%
Environment
0.00%
5.83%
0.00%
0.00%
Finance
2.04%
10.74%
6.64%
5.70%
Hcalthcare
12.65%
5.06%
3.79%
0.00%
Logisties
6.53%
1.53%
1/4.69%
12.95%
Manufacturing
31.84%
4.91%
5.21%
36.27%
Markcting
1.22%
6.44%
0.00%
0.00%
Military
0,00%
5.67%
0.00%
0.52%
Retail
4.08%
5.06%
3.32%
0.00%
Services
10.20%
10.89%
2.37%
4.15%
Sports
0,00%
4.75%
0.00%
0.00%
Supply Chain
0.82%
5.06%
27.01%
7.77%
Technology
0.00%
0.92%
4.27%
0.52%
Telccommunications
0.00%
3.68%
1.42%
10.88%
Transportation
16.73%
7.67%
19.43%
5.18%
Other
4.90%
1.07%
5.69%
4.66%
Figure 8. Scenarios distribution of the datascts.
(LaOmoTRamamojjiooo al., 2021) curated dataset derived from the NL4OPT Competition, where participants were tasked with developing automated methods convert natural language problem descriptions into solver-ready code. This dataset primarily focuses LP (Linear Programming) problems across various contexts, though the underlying mathematical models are relatively uniform, with more complex MIPS (Mixed Integer Programming and Scheduling) problems notably absent. For our experiments, we selected the test set from this dataset, filtered out low-quality examples, and retained a total of 245 high-quality instances.
MAMO( et al., 2024) introduces novel optimization dataset to assess the modeling capabilities of LLMs. The dataset divided into two main components, Easy_P and Complex containing 652 and 211 instances, respectively, These components cover both LP and MILP problems, capturing wide range of real-life scenarios. However, the dataset does not include any nonlinear programming (NLP) problems.
IndustryOR(Tang et al., 2024) is the first industrial benchmark for operations research, consisting of 100 real-world OR problems collected from eight different industries. This dataset covers five types of optimization problems: linear programming, integer programming, mixed integer programming, non- con-linear programming, and others, with problems distributed across three levels of difficulty.
15

---

OptMATHI: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
OptiBench( Yang al., 2025) introduces comprehensive benchmark designed evaluate LLMs' end-to-end solving capabilities in optimization tasks. The dataset consists of 605 carefully selected and manually verified problems, covering linear and nonlinear programming with both integer and mixed integer variables. Since this benchmark does not directly include an "answer" field, we extracted the optimal values to serve as golden answers using both cule-based methods and L.L--based approaches.
OptMATH-BBnchh As shown in Table I, while our fine-tuned model achieves remarkable performance on NL4OPT and MAMO EasyLP, these datasets alone are insufficient to comprehensively evaluate the model's optimization modeling capabilities. Moreover, both NL4OPT and MAMO datasets are limited to linear programming problems, making them less representative of the broader optimization landscape. To address this limitation, constructed OptMATH- more challenging dataset for large models that also expands the diversity of problem types.
The creation of OptMATH--ench followed two distinct routes. The first pathway began with instances initially rejected by our AutoFormulator due to failing the optimal value (OV) check, indicating their inherent difficulty. A "LLM-Commitee" (inspired by (Zhao et al., 2024), utilizing diverse powerful models such as GPT-4, Claude, Gemini, and DeepSeek) then filtered these problematic instances. PD and NL pairs were retained only if least one, but no more than two, committee members successfully formulated them (i.e., passed the OV check). This process isolated problems that were well-posed yet presented non-trivial modeling challenges. Crucially, human OR experts subsequently validated the correctness of these selected pairs and further refined them based on relevance and clarity. The second pathway involved OR experts directly curating challenging problems from external operations research literature, including journals and textbooks. This approach ensured methodological and source independence in the dataset's 's composition and allowed for the inclusion of known hard problem types, such as NLP and SOCP (as illustrated in Figure 4).
This dataset includes a carefully curated selection of representative mathematical optimization problems that span broad range of application scenarios, covering LP, MILP, IP, NLP, SOCP, and other common optimization problems. Additionally, the problems in OptMATH-Bench Bench are inherently challenging, making them effective in distinguishing the modeling capabilities of the model.
During evaluation, we observed that certain ambiguities in problem statements could cause the LLM to struggle in determining whether a variable is integer or continuous. To address this, we applied a rule-based substitution approach: as long as the optimal solution derived under either assumption (integer or continuous variable) matches the ground truth, we consider it pass. We also implemented several other rule-based substitutions to handle variations in solution formats and representations, ensuring fair evaluation process where all models are assessed using the same standardized evaluation methodology. To determine whether the optimal values are equivalent, we use the following formula:
ypred Maaeel <G, |Waaeel + 1
where is set to le-6.
A.2. Seed Classes
Our seed problem classes were curated by drawing from MIPLIB instances and integrating insights from both Chain-of- Experts(Xiao et al., 2024) and peer-reviewed literature. For each instance, conducted in-depth analysis of its structure, starting from the problem description to identify its broader optimization category and further refining it into specific subclasses. To ensure theoretical accuracy, we consulted literature that provided detailed descriptions of these optimization subclasses. Based on these references, we formulated the mathematical representation of each subclass, systematically outlining sets (where applicable), parameters, decision variables, objective functions, and constraints. This step aimed to establish an abstract mathematical framework rather than focusing on specific instances.
We organized comprehensive metadata for each problem class in structured me t adata. json file, encompassing subclass names, references, reference links, and LaTeX- mathematical expressions. An example of this metadata structure is provided in Appendix F.5. This systematic documentation not only ensures clarity but also facilitates dataset utilization and future extensions.
Next, we focused on generating new problem instances. We implemented a custom Python class, Generator (), in generator. -py, which contained a step-by-step algorithm to create instances of the identified subclasses (an example
16

---

OpIMATH: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
provided in Appendix E.6). The input parameters and outputs explicitly defined, with detailed specifications for each parameter's type and valid range documented in README md. We validated the generator by running test. gene caterator () with default parameters to ensure the produced instances were both mathematically valid and practically meaningful.
Through this systematic and meticulous approach, we constructed a high-quality dataset of Problem Description (PD) generators that lays a solid foundation for generating natural language descriptions of optimization problems through Backtranslation Pipeline. This dataset designed to versatile and scalable, making suitable for a wide range of applications in optimization research and practice.
Main Class
Problem Class
Num
Reference
Assignment and Resource Allocation
Car Selection Problem
(Burkard et al., 2012)
Optimization
Contract Allocation Problem
(Wikipedia, 2025)
Assignment Problem
(Kuhn, 1955)
Structure-Basd -Based Assignment Prob-
(Burkard al., 2012)
lem
Team Formation Problem
(Gurobi Optimization, 2025)
Military Personnel Deployment
(Brown et al., 2004)
Problem
Combinatorial Optimization
Knapsack Problem
(Martello & Toth, 1990)
Market Share Optimization Problem
(Cooper, 1993)
Set Multi-Cover Problem
(Winston, 2004)
Set Cover Problem
(Caprara al., 2000)
Cutting and Packing Optimization
Bin Packing Problem
(Garey & Johnson, 1981)
Blending Problem
(Zieyel, 1988)
Cutting Stock Problem
(Gilmore & Gomory, 1961)
Domain-Specific Optimization
Diet Problem
3
(Garille & Gass, 2001)
Unit Commitment Problem
(Padhy, 2004)
Farm Planning Problem
(Hazell Norton, 1986)
Facility Location Optimization
Facility Location Problem
(Toregas et al., 1971)
Capacitated Facility Location Prob-
(Daskin, 1995)
lem
Transportation Problem, Airline In-
(Toth, 2002)
dustry Resource Allocation
Facility Dispersion Problem
2
(Kuby, 1988)
Financial and Revenue Optimization
Portfolio Optimization Problem
(Markowitz, 1952)
Profit Maximization Problem
(Hillier & Lieberman, 2014)
Revenue Management Problem
(Chen et al., 2022)
Revenue Maximization Problem
(Weatherford & Bodily, 1997)
Network Flow Optimization
Multi-Commodity Capacitated Net-
(Gendron et al., 1999)
work Design Problem
Multi-Commodity Transportation
(Smith, 1994)
Problem
Minimum Cost Flow Problem
(Klein, 1967) (Smith, 1994)
Multi-Commodity Network Flow
Problem
Network Flow Problem
(Ford & Fulkerson, 1956)
Statie Line Planning Problem
(Schöbel, 2012)
Supply Chain Optimization
(Cordeau et al., 2006)
Network Optimization
(Bertsekas, 1998)
Production Planning and Scheduling
Capacitated Lot-Sizing Problem
(Florian Klein, 1971)
Optimization
Factory Planning Problem
(Pinedo, 2022)
Flow Shop Scheduling Problem
(Rajendran, 1995)
17

---

OptMATHI: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
Input
Other
5000
Ouipul
Aviation
4.1%
Pinance
Logistics
4000
4.1%
29.0%
Frequency
4.7%
3000
Transportation
5.0%
2000
Services
1000
18.9%
10.0%
Supply Chain
5000
10000
Manufacturing
Sequence Length
Figure 9. Distribution of Application Scenarios OptMATH- Train
Figure 10. Sequence Length Distribution of OptMATH-Tras
Main Class
Problem Class
Num
Reference
Job Shop Scheduling Problem
(Adams al., 1988)
Discrete Lot-Sizing and Scheduling
(Fleischmann, 1990)
Problem
Production Planning Problem
3
(Herrmann, 2006)
Lot-Sizing Problem
(Drexl Kimms, 1997)
Transportation and Routing Optimiza-
Aircraft Assignment Problem
(Abara, 1989)
tion
Aircraft Landing Problem
(Beasley et al., 2000)
Transportation Problem
(SteadieSeifi al., 2014)
Traveling Salesman Problem
(Dantzig al., 1954)
Operations Optimization
(Golden al., 2008)
Capacitated Vehicle Routing Prob-
(Solomon, 1987)
lem with Time Windows
A.3. OptMATH-Trrin
The OptMATH-Trail dataset consists of over 150k reverse-gente generseee samples and 50k augmented instances, forming comprehensive collection of optimization problems. The dataset encompasses rich variety of real-world application scenarios. As illustrated in Figure 9, the dataset covers 10 major application domains spanning across both business sectors and specialized industries, demonstrating extensive coverage of real-world optimization scenarios. The substantial proportions in logistics, supply chain, and manufacturing ensure robust representation of primary industrial applications, while the balanced inclusion of sectors like transportation, energy, and finance provides comprehensive coverage of specialized use cases. This thoughtful allocation of problems across different domains not only prevents data concentration but also maintains sufficient samples for each sector, enabling effective model training and evaluation.
The sequence length distribution of OptMATH-Train, as shown in Figure 10, exhibits well-balanced profile for both input and output sequences. The distribution approximates normal distribution with mild right-skewness, centered around 5,000 characters, demonstrating natural variation in problem complexity. This balanced distribution pattern is particularly advantageous for model training, as it ensures sufficient context length for complex problem representation while maintaining computational efficiency. Furthermore, the moderate right-skewnes, encompasses challenging cases with extended sequences, which is essential for developing robust models capable of handling sophisticated optimization problems that require comprehensive reasoning and detailed solution steps.
18

---

OpIMATHI A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
B. Details of Instance Generation
B.1. An Example Measuring the Complexity
Let binary variables y1, 92 {0, I indicate whether products 1 and are produced, integer variables 21, 22 E Z represent production quantities, and continuous variable 2 2 denote total cost. The objective function minimizes total operational costs: min 10y1 8y/- The constraints span four categories: First, linear constraints include the resource limitation 2 3r2 100 and market demand bounds x1 S 50, 32 30. Second, indicator constraints using the Big-M method (with M 100) enforce minimum production levels when activated:
= 2 5 is reformulated as #1 2 5 - 100(1 ), and analogously for y/2 = = 22 3. Third, quadratic constraint 2 2 0.5x, + 0.333 integrates inventory costs into the objective. Finally, a general nonlinear constraint Zie"2 < 100 captures efficiency coupling between products.
To compute the complexity score S(PD), we identify: 2 binary variables, integer variables, and continuous variable; linear constraints, 2 indicator constraints, 1 quadratic constraint, and nonlinear constraint. The Big-M factor frequency fBigh 2, while the average number of terms per expression Lexpr 2.71 is derived from structural analysis of constraints and the objective. With all weights set to I, the total complexity score becomes S = 16.71. This example demonstrates how modeling choices (e.g., introducing nonlinear terms, Big-M parameterization) directly influence the score, providing a quantitative framework for assessing model complexity.
B.2. An Overview of the Generated LP Files
The average lengths of LP files for different difficulty levels are illustrated in Figure 11. We define the complexity thresholds for the five difficulty levels-easy, medium_ease easy, medium, medium_hare and hard-as as [25, 75), [50, 100), [75, 125], [100, 150]. and [125, 175], respectively. The results demonstrate that our feedback-driven problem data generation approach effective, The average length of the generated files across the five difficulty levels is presented in Figure 12, categorized by seed data name. All generated LP files are feasible and possess optimal solutions.
25000
24,329
Average Number of Characters
20000
16,098
15000
11,919
10000
8,430
5000
4,092
0
Easy
Medium Easy
Medium
Medium Hard
Hard
Difficulty Level
Figure 11. Distribution of I.P file lengths across generated instances by difficulty levels.
19

---

OptMATHI A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
10
Average Number of Characters
10a
Betcomen Aicorataa suuting
Manl
commtton
coupply
Statid Lnd
Camb Bastriall
Unicqpesta aisulii
TheMall festeratrarrra
Figure 12. Distribution LP lengths across generated instances by problem types.
C. Details of Backtranslation
C.1. Backtranslation Pipeline
In our reverse generation pipeline, we employ Deepseek-V33 V3(Liu et al., 2024a) as our foundation model and configure its temperature parameter to 0.8 to enhance the diversity of generated problems. Furthermore, to achieve rich contextual diversity, we implement a random scenario assignment mechanism during the Initial Generate phase. This mechanism directs the LLM to synthesize problems that optimally integrate the mathematical characteristics with the designated scenario context. The detailed prompt is elaborated in Section E.I.
Through this backtranslation process, we initially generated approximately 120,000 easy optimization problems. As shown in Figure 15, the length distribution of problem descriptions exhibits a right-skewed pattern, with most problems containing 2,000 to 5,000 characters. After applying rejection sampling, around 40% of the generated problems were filtered out while maintaining a similar distribution pattern. This consistency in distribution before and after filtering suggests that our quality control process effectively removes low-quality samples without introducing length-related biases, ensuring the retained problems maintain natural and appropriate descriptive lengths. After multi-stage refinement including semantic verification and difficulty calibration, the pipeline ultimately produced 150,000 rigorously validated optimization problems. Together with 50,000 augmented instances, this curated collection forms our AptMATH-Traa dataset, where each instance demonstrates: (1) Contextual alignment between mathematical formulations and real- real-world scenarios, (2) Controllable complexity levels matching specified difficulty tiers, and (3) Natural language expressions adhering to authentic problem-solvee solving discourse patterns. The hierarchical quality assurance framework ensures the dataset's 's applicability for both educational interventions and benchmarking mathematical reasoning systems.
20

---

OpIMATHI A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
An Example of Backtranslation
Input-General Formulation
Input-LP File
Minimize
Minimize
ci
80000 Selected [] 40000 Sellcted[2]] 20000
Selected[3] 10000Sclected[4]
Subject
Σ R68
kj Vje Elements
+ 80000 Selected[S] 90000 Selected[6]
Subject
Generator
MultiCover Selected| Selected] Selected[ Selected[6] >=
{0,1) Vi E Sets
4 represents the cost coefficient for each set
MultiCover c10: Selected[ Selected[4] Selected[5]
2,18 binary decision variable indicating whether set
Selected[6]
selected
--4
Sg represents set all sets containing element
Bounds
Binaries
ky represents the minimum numher of times element needs to be covered
Selected[[] Selected[[] Selected[3] Selected[4] Selected[S] Selected[6]
End
Backtranslation
Outpui-Natural Language Description
A city is planning the layout emergency medical stations. There are candidate locations for building medical stations, each with different construction costs:
Location I: Construction cost $80,000; Location Construction cost $40,000
The city is divided into 10 districts, each requiring different numbers medical stations for coverage due to population density and emergency medical needs:
Districts and 2: require coverage
least 4 stations;
District 3: requires coverage least 2 stations
Each candidate location can cover specific districts:
Location covers districts: 1, 2, 6, 10; Location 2 covers districts: 3, 5, 6,
The objective is to decide which locations should be selected for building medical stations, minimizing the total construction
while meeting coverage requirements each district. Each location can only selected not selected (binary decision).
Figure An example of backtranslation: transforming mathematical formulations files into natural language descriptions optimization problems. The process transforms formal mathematical notation and concrete data into human-readable problem descriptions.
C.2. Ablation Study on the Impact of Self-Refine Iterations
To validate the effectiveness of each step in our backtranslation pipeline, we conducted comprehensive ablation studies. We first compared the accuracy between using only the Generate step versus implementing the complete pipeline with Generate, Self-criticize, and Self-refine steps. To investigate the impact of parameter 7' on the acceptance rate of rejection sampling, we randomly selected 500 instances for evaluation, with results shown in Figure 14. The results demonstrate that our Self-Refine loop (7 2 1) consistently outperforms direct generation (T = in terms of acceptance rate. While there are some fluctuations in performance across different T' values, possibly due the inherent hallucination tendencies of large language models, we observe that setting T achieves satisfactory acceptance rate of 61.56%. Considering the trade-off between performance and computational efficiency (token usage), we adopt T 1 in our final data synthesis process.
D. Details of AutoFormulation
D.I. CoT Instructions AutoFormulation
To support comprehensive mathematical modeling capabilities, we developed a diverse set of CoT instructions, which are detailed in Section E.3. These instructions vary in their decomposition approaches, intermediate reasoning steps, and
21

---

OptMATHI: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
Original
5000
Afler Rejection
4000
Aocepanncs (5)
Froquency
3000
63
61,56
2000
60.86
1000
5000
10000
Max Iteration of the self-refine loop
Problem Description Length
Figure 14. Acceptance Rate Maximum Iteration of Gelf-Reiine Loop. The bar chart illustrates the acceptance rate achieved at different maximum iteration limits.
Figure 15. Distribution of Natural Language Description Lengths for Easy Problems in OptMATH Train Dataset. The histogram compares the length distribution before and after rejection pling, showing the quality filtering process.
presentation formats, providing multiple pathways for problem formulation. In Figure 16, we present one representative formulation pattern from our instruction set. This format includes three key components: general mathematical formulation with standard notation, detailed instance-specific formulation with complete parameter specifications, and the corresponding Python implementation using Gurobi. However, this represents just one of many possible formulation styles. Other formats in our instruction set may use different ordering of steps, alternative notation systems, or various levels of mathematical abstraction. The diversity in formulation patterns ensures that dataset captures wide range of valid mathematical modeling approaches while maintaining logical coherence and mathematical correctness.
D.2. Ablation Study on Augmentation
As mentioned in the previous section, the purpose of data augmentation is to increase the diversity of the dataset and generate more non-standard problems, which can help the fine-tuned model to solve more difficult problems. We use 50k raw data, augmented data and mixed data (50% raw data and 50% augmented data) for fine-tuning training on the Qwen2.5-7B model, respectively, and the results show in Table 3 that the model fine-tuned with raw data performs better in solving relatively simple problems, augmented data performs better in solving difficult problems, and mixed data combines the ad advantages of the above two very well, being the best in terms of average accuracy across the four types of test sets.
Table 3. Comparison of Original Data and Augmentation Data in Training Models.
MAMO
MAMO ComplexI.P
OptMATH Bench
Micro Avg
Macro Avg
Types
NL4OPT
EasyLP
Without Augmentation
86.9%
88.0%
44.5%
31.1%
72.1%
62.3%
Without Original
82.9%
85.5%
44.7%
23.6%
69.2%
59.2%
Mixture of Augmentation and Original
87.3%
87.7%
48.1%
33.3%
73.1%
64.1 %
D.3. SFT
We employ the LlamaFactory framework for fine-tuning (Zheng et al., 2024). We select the Qwen2.5 series (0.5B~322B) as our base models (Yang et al., 2024), and the hyperparameters are generally set as follows: initial learning rate le-4, 1~3 epochs, LoRA rank of 32, LoRA alpha of 32, and LoRA dropout of 0.1. While there are minor variations hyperparameters
22

---

OptMATHI: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
An Example of AutoFormulation
Input-Natural Language Description
A city planning the layout of emergency medical stations. There are candidate locations for building medical stations, cach with different construction costs:
Location I: Construction
$80,000; Location 2: Construction cost $40,000
The city divided into 10 districts, each requiring different numbers of medical stations for coverage due population density and emergency medical needs:
Districts and require coverage by least stations; District requires coverage by at least stations
Each candidate location can cover specific districts:
Location 1 covers districts: 1, 2. 6, 10; Location covers districts: 3, 6, 9
The objective is to decide which locations should be selected for building medical stations, minimizing the total construction cost while meeting the coverage requirements for each district. Each location can only be selected not selected (binary decision).
AutoFormulation
Output-General Formulation
Output-Python Code with Gurobi
gursbipy import
Minimize
modell
m Nodel (a Station Location")
Subject to
Σ F68/
T4 2 ky Vj E Elements
Parameters
coata $80,
{0,1}
Vi Sets
2: 4,
c; represents the coefficient each
æ;is binary decision variable indicating whether set selected
Coverage requirements for each district coverage_rer 4, 2: 4, 2,
Sy represents the set of all sets containing element j represents the minimum number of times element needs to be covered
Coverage matrix coverage 1: E. 2, 6, 101, 2: 13, 9]r
Output-Instance Formulation
1.Sets
(I. ....66):st potential locations
(1,...10): set of districts
Decision selected - a.addaara toosts. keys 1), vtypo=GRR, RTMARY, name="Saleotad")
Parrameters cfi]: construction for location
e[l] 8, c[2] e[3] {[4] c(5) c[6]
Objective Function m. ae tob Objtiiive (quickeum [costs selected[i] for in costs. ()), GRR,MTNTNT2R)
[]]: coverage requirement for district
r|1 -
[(4] r
[[6] - 1, 2, 3. ([9] - [10) -
Constraints
a[ij]] coverage matrix, if location can cover district otherwise.
district in coverage reg.keyakee
Sum up all locations
3. Variables
m.addCorstiz (peleeste for loc in coverage. keys () district in
x(i] E (0, binary decision variable
coverage (acoee/ll=coovaeeee" aeaaaeeeeeeeeeeeeeeeeeeeeeeeeeee reg.regggaaeee reqidiatriet), 1"MuitiCover e(distrrittt
x(i] if location is selected
x[i] otherwise
Optimize model m. omt ze
4. Objeetive Function
Minimize: E(iEL) c[i]xx[i] (total construction cost)
Print results
5. Constraints
if m.status GR3. OPT1ALL:
print ("Anoptina solution found:")
Coverage requirements: Y(ELL) a[-]]xx[i] <(). vjeD
Binary constraints: x[i] (0,1), vieL
elae:
print ("Ho solution found":
Figure 16. An Example of AutoFormulation
23

---

OptMATH: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
across different experiments, the overall settings remain similar and we omit these details for brevity. Notably, illustrated in our framework diagram the entire AutoFormulator training process is an iterative cycle. The Rejection Sampling in Step 2 relies AutoFormulator's modeling capabilities stronger modeling abilities lead to higher pass rates and better data quality. Similarly, the data augmentation phase depends on AutoFormulator's s modeling competence. Higher quality data, in turn, results in a more capable Formulator through TThoogghhh this systematic model fine-tuning and data augmentation approach, we have developed a dynamically evolving fine-tuning framework. This framework not only accurately transforms natural language descriptions into mathematical formulations and solver code but, more importantly, establishes a ioppoviio data flywheel mechanism. This positive feedback loop enables the AutoFormulator system continuously enhance its capability in handling complex optimization problems through ongoing learning and self-optimization, forming a virtuous growth cycle.
D.4. Detailed Ablation Studies Model Size and Data Size
To investigate the impact of model capacity and training data volume on optimization modeling performance, we conducted two sets of experiments using OptMATH-Train. For the model size study in Figure 6 and Figure 17, we compare the performance of baseline and finetuned Qwen2.5 models ranging from 0.5B to 32B parameters. For the data scaling analysis in Figure 7 and Figure 18, we track the accuracy progression within the first training epoch across different model sizes, using varying proportions of the training data.
The model size experiments reveal distinct scaling patterns across benchmarks. On NLAOPT, the performance improves from 12.7% (0.5B) 96.7% (32B), showing particularly rapid gains in the 0.5B-3B range. For MAMO EasyLP, we observe similar but more moderate improvements, with accuracy increasing from 31.9% to 90.5%. However, more challenging benchmarks like MAMO ComplexLP and OptMATH--Bnch, even the largest models achieve relatively modest gains, reaching 52.6% and 30.6% respectively at 32B parameters.
The comparison between baseline and OptMATH-Train finetuned models reveals interesting patterns across different model scales. For simpler benchmarks like NL4OPT and MAMO EasyLP, while the performance gap narrows with increased model size, OptMATH-Taall finetuning still provides consistent improvements even for the largest models. More notably, on complex benchmarks such as MAMO ComplexLP and OptMATH-Bench, models finetuned on OptMATH-Train demonstrate substantial performance gains across all model sizes, highlighting the effectiveness of our training dataset in enhancing models' capabilities for challenging optimization problems.
The data scaling analysis reveals distinct learning dynamics across model sizes. Smaller models (0.5B, 1.5B, 3B) exhibit higher initial performance variance during training, while larger models (7B, 14B) demonstrate more stable learning curves from the outset. Notably, all model sizes achieve relative performance stability after utilizing approximately 40% of the training data, though the absolute performance levels differ significantly. The 3B model, for instance, maintains consistently higher performance across all benchmarks while requiring a similar proportion of training data reach stability.
This efficient data utilization pattern holds true across all benchmarks, regardless of their complexity levels. Whether for the relatively straightforward tasks in NL4OPT or the more challenging problems in OptMATH-Bende models typically converge to their peak performance using around 40% of the available training data. The remaining 60% of the data contributes primarily to fine-tuning and minor performance adjustments rather than substantial improvements.
24

---

OptMATHI A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
Table 4. Performance comparison of Qwen2.5 models of varying sizes on mathematical optimization tasks. The percentages in parentheses
indicate improvements after fine-tuning.
Models
NLAOPT
MAMO EasyLP
MAMO ComplexLP
OptMATH--Bnch
Micro AVG
Qwen2.5-0.58
0.00%
0.15%
0.00%
0.00%
Dween..540.5B(Fnnctumm 12.65% (112.65%)
31.90% (131.75%)
16.59% (116.59%)
15.03% (115.03%)
23.29% (123.21%)
Qwen2.5-1.5B
0.00%%
2.15%
0.95%
0.00%
1.23%
(ren2255).5B(Finetueed))
46.12% (144611%%)
68. 10% (165.95%)
22.75% (121.80%)
18.65% (118.65%)
49.27% (148.04%)
Qwen2.5-3B
67.35%
65.18%
16.11%
0.52%
48.04%
(wven255-3B(innetunel)
68.57% (11.22%)
80.98% % (115.80%)
25.59% (19.48%)
15.03% (114.51%)
59.88% (11 1.84%)
Qwen2.5-7B
86.94%
83,59%
21.80%
1.55%
62.03%
86.94%
89 42% (15.83%)
48 82% (127.02%)
30.05% (128.50%)
73.56% (1 1.53%)
Qwen2.5-14B
93.47%
82.52%
42.65%
14.51%
68.02%
Qwen2.5- (.0..e1ee(eei
95.51% (12.04%)
90 90.49% (77.97%)
51.18% (18.53%)
29.53% (115,02%)
76.02% (18.00%)
Qwen2.5-322
92.65%
82.21% % 88 00.00%% (15.83%)
44.55%
9.33%
67.26%
Qwen2.5 (eeen.5-32B(Fineeee 32B(Fnnttnedd)
96.73% (14.08%)
56.4% (111)85%
36.27% (126.94%)
76.86% (19.60%)
100
95.5%
12.0%
96.7%
14.1%
100
Baseline Model
89.4%
15.8%
90.5%
BB
Baseline Model
Finetuned Model
7eQvans
68.1%
68.6%
Accuracy (%)
60
146.1%
Accuracy
40
46.1%
86.9%
93.5%
92.7%
40
131,8%
31.9%
83.6%
82.5%
87.3%
65.2%
20
12.7%
0.5B
1.5B
3B
7B
14B
0.5B
1.5B
3B
7B
14B
32B
Model Size
Model Size
(a) NL4OPT
(b) MAMO EasyLP
Baseline Model
Baseline Modal
Finetuned Model
Finetuned
80
80
Accuracy (%)
Accuracy (9%
18.5%
48.8%
126.9%
40
40
116.0%
36.3%
(21.8%
29.
20
116.6%
16.6%
44.5%
20
115.0%
15.0%
18.6%
118.6%
114.5%
15.0%
21.8%
16.1%
0.0%
0.0%%
0.0%
0.0%
1.5B
1.58
Model Size
Model Size
(c) MAMO ComplexLP
(d) OptMATH-Bench
Figure 17. Scaling hehavior of Qwen2.5 models (0.5B-32B) on various benchmarks.
25

---

OptMATHI: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
40
35-
30
60-
15
Accuricy
20-
400
15-
10-
Modon Carpoot?
Proportion
Proportion
(a) Qwen2.5-0.5B
(b) Qwen2.5-3B
100
Accurncy (%)
Accurncy (%)
NLater
AouEe Beck
0.0
0.2
0.4
0.6
0.0
0.2
0.4
0.6
0.8
1.0
Proportion
Proportion
(c) Wwen2.5-BB
(d) Qwen2.5-14B
Figure Scaling behavior Qwen2.5-n0 5.B.5, Qwen2.5-3B. Qwen2.5-7B and Qwen2.5-14B Accuracy Within Training Epoch.
26

---

OpIMATHI: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
E. Prompt Templates
this section, we present all the important prompt templates. Due to space constraints, certain parts are omitted, and only the prompt frameworks are shown.
E.1. Reverse Data Generate Prompt
GENERATE PROMPT
As Operations Research Expert, analyze the given mathematical aptimization expression and LP data.
Input Mathematical Expression: seathamatiaalLexpressinn
Input LP Data:
([lp_data))
Reference Examples:
[eeammless]]
## Required Output:
Provide ONLY clear, detailed natural language description the optimizatinn
problem that:
Describes the complete scenario
States all decisions to be made
Specifies the objective elearly
Incorporates all constraints and conditions naturally
Includes all numerical parameters within narrative
Uses appropriate domain terminolegy
Maintains mathematical accuracy without showing formulation
SELF-CRITICISM PROMPT
As operations Research Expert, evaluate if the generated problem description matches the mathematical optimization problem...
Input LP Data: ((1p_data))
Generated Problem Description: srroblee_descrrpttonn
Analysis steps...
## Required output:
If perfect match: "Complete Instance"
I$ inconsistencies exist: "Incomplete Instance: [Lst specific discrepancies...i
27

---

OptMATHI: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
SELF- SELF-EEFINEMEN PROMPT
As Operations Research Expert, analyze the criticism and refine the problem description needed.
First, check the criticism result: ((criticism))
IE the eriticism shows "Complete Instance";
Output "Nothing need to refine"
Otherwise, follow these steps generate improved description:
1. Review Input Materials:
Mathematical Expression: peathematical_expressse
Data: ((Ip_data))
Initial Descriptien: ([initial_desccritttmnn)
2. Task:
Based the criticism feedback, LP data information, and initial description, generate complete and accurate problem description.
Required Output:
If criticism is "Complete Instance";
Output "Nothing need to refine"
Otherwise:
[Direct natural language description of the optimization problem. No intuotoyy phrases meta-cemmentary section headers separators
Just the complete problem description in clear natural language
Ensure exact match with all uP data parameters
Include all constraints and objectives naturally
Avoid mathemaicaal notation
Note: The output should be ONLY the complete natural language description itself, with no additional text or formatting.
28

---

OpIMATHI: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
E.2. Baseline Prompt
BASELINE PROMPT TEMPLATE FOR OPTIMIZATION MODELING
Below is an operations research questicn. Build a mathematical model and corresponding python code using 'gurobipy" that appropriately addresses the question.
Question: ()
Notes:
Please output Python code starting with the following lines: " 'python koython//nnimmpor gurobipy as gp gurobipy import G
Make sure the model variable is named 'model'.
Avoid using "<" and ">" in Gurobi constraints; instead, use "<=" or ">=" appropriate.
Carefully determine whether the variable is integer continuous variable.
Response:
(Provide your response here, keep the notes above in mind)
E.3. AutoFormulation Instructions
CoT INSTRUCTIONS
instructions Total instructions: entries
Showing 5 representative examples below...
"Below is operations research question. Build mathematical model and corresponding Python code using 'gurobipy' solve it.",
"Create a complete solution that includes: 1) Mathematical formulation 2) Python code using gurobipy 3) Results interpretation. Ensure all variables and constraints are properly defined.",
"Transform this operations research problem into mathematicas model and implement it in Python with gurobipy. Include clear variable definitions and constraint explanations.",
"The following an operations research problem. Let's solve it step by step: 1) Identify the decision variables, objective function, and constraints 2) Formulate the mathematical model 3) Implement the solution using Gurobi in Python 4) Verify and interpret the results.",
"This an operations research problem. Follow this structured approach: Begin with understanding the problem -> Identify the key variables -> Analyze the constraints -> Develop the nathematical model -> Solve it programmatccally using Gurobi in Python. Provide clear reasoning and explanations at each stage." "
29

---

OpIMATHI: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
E.4. Prompts for Configuration Selecting
INITIALIZING THE PARAMETERS
You are an optimization expert helping to tune the parameters in: \begin(verbatiml
[generator code)
\end(verbatim)
CRITICAL REQUIREMENT:
Generated instances MUST be solvable to OPTIMALITY by Gurobi...
Model Complexity Score Calculation:
1. Variable Score:
Binary variables: weight Integer variables: weight (weights int Continuous variables: weight
2. Constraint Score...
3. Additional Complexity Factors...
Total Score Variable Score Constraint Score Big-M Score
Expression Score
Secondary Requirements:
1. Model Complexity: (complexity__score)
to (complextty\_soree..oo Solve Time: (min\_solvei_timee) to (mave_solve's (_tmme) secends
2.
Return parameter values in JSON format matching 'default parranterss structure:
Keep exact same keys
Preserve data types
Use 1ists for tuples
Format with proper indentation
FEEDBACK РROMPT
Based on testing (total_instances, instances with your suggested parameters: (Lastsouggcteccppaaamteeer
Here the detailed results:
1. Solution Status Analysis:
Total Instances: [total_instanees Solvable Instances: (soivable ittaaalleeiittteeessseeee OPTIMAL Solutiens:
instances) (loptimal rate: (1118)
Solution Status Distribution:
Status Percentages:
2. Overall Performance (Only OPTIMAL Sclutions)
-success_rate:.1e al rel_rreeutmmmntt"" out of instal_isttaness) instances met all requirements)
Success Rate: (success paa.rrttteyyy. .ffs ((results (" "reeussreeets_mmet ]]
Note: Only OPTIMAL solutions are considered successful
Satisfaction (Only OPTTMAL Solutions): 3. Complexity Score Distribution: ( ("complexity")) required range: (requirements) Success rate: Inum wtw.attttyiigggeeieeeeeeeeeeeeee htspeemeetesttttaaattttteeeee instances)
Solve Time Distribution: (anputiiiion/aalyeeee.eea
("solve time")
Required range: (requirements)
success rate: (mm.aaaaeaannnnreeuuuueeaaoeaooe
30

---

OptMATHI: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
4. Model Structure Analysis (Only OPTTMAL, Solutions) Variable Distributions: Binary Variables:
Constraint Distributions:
Linear Constraints:
Indicator Constraints:
Quadratic Constraints:
General Constraints:
Complexity Score Components:
5. Distribution Analysis Insights: [throughout analysis of the instances generated by the paraneters by calling statistics package in Python)
Based on these results and distribution analysis, please suggest parameter values that would:
1. MAXIMIZE the proportion of instances that reach OPTIMAL status
2. Adjust the model complexity to meet the target score range (for OPTIMAL instances
3. Maintain solve times within the required range (for OPTIMAL instances)
4. Reduce variability in key metrics where high variance was detected Increase the everall success rate
5.
Return your response in JSON format, strictly following structure of the previous suggestions dictionary. Ensure that:
1. A11 keys remain exactly the same as in the previous suggestions
2. The data types for each value are preserved
3. The JSON should be properly formatted with indentation for readability
4. Do not acd any new keys or remove any existing keys
31

---

OpIMATHI: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
E.5. An Example of Metadata
Metadata
Subclass: Bin Packing
Reference:
Garey, M. R. and Johnson, D. S. "Asproximation Algorithms for
Bin Packing Problems: Survey." Analysis and Design of Algori thms in
Combinatorial Optimization (1981)
Reference URL: http://///...o///00000/0/3333330
Mathematical Formula:
Consider n items, where each item has:
Weight 8; The weight of item i
The problem includes:
Bin Capacity c: The uniform capacity of each bin
Bin Usage Variable fj A binary variable indicating whether bin j is used
Assignment Variable dyf A binary variable indicaring whether item i is assigned bin
Minimize
Σ Vi
j=1
Subject to:
8,3i.j S cyj
=j=1... 1.....,
Σ Fi.
Vi 1.....
5=1
Fiji f) E {0,1}
Vi,j = 1....
32

---

OpIMATHI: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
E. An Example of Generator
Python Code for Bin Packing Generator
import gurobipy as gp from gurobipy import GRB import random
class Generator:
def _init__self., (self, parameters=None, seed=None)
self.problon_tiye "binpacking"
default_parameters
"n_items": (3, 10),
10
"weight_range": (1, 50), "bin_capacity": 100
11
if parameters is None:
14
parameters default parameters for key, value in parameters. .items ()
15
16
staatrrseef,, key, value) self.seed seed if self.seed:
19
random. seed (seed)
20
def Senerate_instanee (self)
self.n_items random. randint wecseff._iittmee items 1ist (range (self.n_items)
23
24
itim_weights (1: random.randint
wwwpswwwwgghhteeeeeeeeeeeee for i in items] )
25
model gp. Model ("HinPacking")
model Params Outputrlag .arramasoootuuutlla
Suppress Gurobi output
28
x model addVars (items, itens, vtype=GRB. BINARY, name="*") y model (items, vtype=GRB. BBINRYY, name="y")
29
Objective: Minimize the nunber of bins used model. setobjective (gp. qup.quicksumm(( [i] for in items), MINIMIEE for j in items: model.addConst..
32
33
34
qp.quicksum (tttem__eights(a) x[i,i] for i in items) <= self.
bin_capacity y[i],
36
name=E"Capacity_!j"
37
38
for i in items:
model (adCCosttr
gp.quicksue (x j] for j items)
41
name=f"AAssigmmett__
42
return model
33

---

OpIMATHI: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
E.7. Augmentation Prompt
GENERATE AUGMENTATION PROBLEM PROMPT
AUGMENTATION ON RULES
Semantic Enhancement
"Rephrase the problem description while maintaining the mathematical structure..."
"Rewrite the problem using different expressions and terminology..." Change the scenario
"Transform the problem into different application scenario while preserving the same structure...",
"Conceive a variant on another scenario for the mathematical model...", Numerical enhancement
"Change the numerical parameters while maintaining the same problem structure
"Scale up or down the proplem size by adjusting parameters proportionally.... Problem variant generation
"Generate a variant by redding/mmvvigg/nee fying constraints....", "Create variation by combining di fifrrnent types of constraints....) Complicating the problem
## Variable Expansion
"Increase the number of decision variables while maintaining similar structure
"Add bounds for adjustment variables...."
## Constraints Expansion
"Add realistic constraints like capacity limitations, budget restriations....,
"Inure cross-variable constraints between components....)
## Data Complexity
"Convert parameters into tabular form with complex data structures...",
## Problem Types
"Generate non-linear proplems by replacing linear relations...",
"Combine with other problem types te generate hybrid problens..."" Multi-objective
"Add new objective functions generate milti-ojectivv problems...",
"Modify objective function to include additional terms...", Problem symnetry
"Generate variants by introcucing symmetries...
"Generate dual problems while medifying paramoters and constraints..."
AUGMENTATION_TEWE TEMPLATE = """Below is an optimization problem, please generate new aptimization problem by following the augmentation rule provided.
Original Problem
The original optimization problem is as follows:
periginal_probless
Augmentation Rule (rule)
# Augmented Problem
Please construct new optimization problem according to the above requirements and the provided question in the following format:
[Write your new problem here]
generated problem should maintain mathematica validity and pract cal feasibi welbility.A.. And just provide the problem descripting without any additional information
34


Reviews:
[{"review_id": "Xv1D2ITuLE", "reviewer": "Reviewer_WcAc", "summary": "The paper presents OptMATH, a scalable bidirectional data synthesis framework designed to address the challenge of data scarcity in optimization modeling. It automatically generates high-quality optimization problem data with controllable complexity, starting from curated seed data with mathematical formulations. The framework employs a backtranslation step to obtain natural language descriptions and uses forward modeling with rejection sampling to verify the correspondence between the descriptions and problem data. The accepted pairs form the OptMATH training dataset, while rejected pairs are filtered to create a challenging benchmark. Extensive experiments demonstrate that models trained on OptMATH achieve superior results on multiple modeling benchmarks, validating the framework's effectiveness and scalability.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review on the potential areas for improvement. We appreciate the opportunity to address these points:\n\n**Essential References Not Discussed:** See “Essential References Not Discussed” section of response to Reviewer PCa3\n\n**Regarding Weakness 1:** Our framework directly addresses this via the **rejection sampling mechanism detailed in Section 4.3 and Algorithm 2**. Each generated NL description is translated back into problem data (PD') using our AutoFormulator. We then rigorously validate this translation by comparing the optimal objective value (OV') obtained from solving PD' against the optimal value (OV) from the original problem data (PD). Only pairs where OV' equals OV are accepted into OptMATH-Train. Consequently, this process **can process NL descriptions that seem to be complex or ambiguous** accurately, ensuring that all included data points feature corresponding NL and PD that are **demonstrably modelable by an LLM**.\n\n**Regarding Weakness 2:** Please refer to the \"Addressing 'In-Domain' Concern\" and \"Distinction Justifies Fairness\" sections of the response to Reviewer LZQX and the table in the response to Reviewer mCmd, where we add new benchmarks. These arguments clearly show that the **benchmarks, even OptMATH-Bench, originating from the same PD distribution, do not overlap with OptMATH-Train**. These results show the models finetuned on OptMATH-Train **generalize to entirely new domains or problem types beyond the curated seed data.**\n\n**Regarding Weakness 3:** Please refer to the \"Scalability for Massive Instances\" a \"Data Generation Efficiency\" sections of the response to Reviewer PCa3. Our pipeline is efficient and can **easily adapt the data-model separation paradigm** to process extremely complex optimization problems.\n\n**Regarding Weakness 4:** Please refer to the \"Scenario/Problem Type Coverage\" section of the response to Reviewer PCa3.\n\n**Regarding Weakness 5:** In **Appendix A.2 and A.3, we have given a detailed description of the diversity** of seed data and OptMATH-Train. For additional metrics, please refer to the table in the response to Reviewer mCmd. We evaluate the performance of finetuned models on IndustryOR and OptiBench. We also reported the results using pass@8; the higher performance at pass@8 compared to pass@1 indicates a **high upper bound on the modeling capability** of the finetuned models, signifying that models trained on OptMATH-Train attain a **high ceiling for their modeling skills**.\n\n**Regarding Weakness 6:** We agree that this is a critical aspect and will revise the paper to elaborate on our methodology. To clarify, our **process is largely automated**: we begin with core problem structures inspired by benchmarks like MIPLIB (e.g., TSP, JSS variants) and **utilize parameterized instance generators** designed around these structures. For instance, our Job Shop Scheduling generator accepts parameters like job/machine counts and operation details to automatically create problem data (PD). Crucially, these **generated PD are validated for solvability (e.g., using Gurobi feasibility checks) before being included in our seed set** and subsequently translated into natural language by LLMs. To further increase automation and reduce human biases, we can easily adapt the methods in addressing **weakness 4** and **reduce the amount of seed data to merely 8 generators**.\n\n**Regarding Weakness 7:** Our empirical validation is crucial here: as reported in Section 4.3, manual checks confirmed that our OV-based rejection sampling **achieves a 99.6% accuracy rate** in capturing the correct problem semantics and ensuring practical equivalence. We find this level of accuracy to be **highly effective and practically sufficient** for ensuring dataset quality for LLM training. Alternatively, checking graph isomorphism for LP problem structures, inspired by related work [1], could be used for stricter MF equivalence; however, we **opted for the OV check due to its operational simplicity and broad applicability** across problem types within our framework, ensuring a practical and scalable validation approach.\n\n**Regarding Weakness 8:** To proactively enhance scalability and mitigate this effect, our framework incorporates specific strategies aimed at maximizing data diversity within OptMATH-Train. As detailed in **Section 5.1, we employ extensive data augmentation techniques** to generate more varied and non-standard problem instances. Additionally, during the forward modeling phase [**Section 4.2**], **we utilize diverse Chain-of-Thought (CoT) prompting strategies** to capture multiple valid reasoning paths and formulation variants. We believe that enriching the training data with such diversity **helps elicit stronger performance improvements even on larger models, thereby counteracting the diminishing returns trend**.\n\n---\n\n**References**\n\n[1] OptiBench: A Large Language Model Benchmark for Optimization Problem Understanding and Formulation.", "claims_and_evidence": "In general, the claims made in the submission are supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria are well-suited for the problem.", "theoretical_claims": "This paper did not provide any proofs for theoretical claims.", "experimental_designs_or_analyses": "See weaknesses below", "relation_to_broader_scientific_literature": "Optimization.", "essential_references_not_discussed": "OptiBench Meets ReSocratic: Measure and Improve LLMs for Optimization Modeling. ICLR 2025", "other_strengths_and_weaknesses": "While the OptMATH framework presents a significant advancement in generating high-quality optimization modeling datasets, there are several potential weaknesses and areas for improvement:\n\n1. Complexity of Natural Language Descriptions: The paper acknowledges that the complexity of natural language descriptions can vary widely. While the framework aims to generate high-quality descriptions, there may still be instances where the descriptions are overly complex or ambiguous, making it difficult for models to accurately translate them into mathematical formulations.\n\n2. Generalization to New Domains: Although the framework covers a wide range of optimization problems, its ability to generalize to entirely new domains or problem types not covered in the seed data is uncertain. The framework relies on curated seed data, which may limit its adaptability to novel optimization scenarios.\n\n3. Computational Resources: The bidirectional synthesis process, including feedback-driven problem data generation and rejection sampling, is computationally intensive. This may limit the scalability of the framework, especially for very large datasets or extremely complex optimization problems.\n\n4. Optimization Problem Diversity: While the framework generates a diverse set of optimization problems, there is a risk that the generated problems may not fully capture the diversity of real-world optimization challenges. The reliance on existing benchmarks and expert-curated seed data might introduce biases or limit the range of problem types generated.\n\n5. Evaluation Metrics: The paper uses accuracy (pass@1) as the primary evaluation metric, which measures whether the optimal value obtained by the generated code matches the ground truth. While this is a relevant metric, it may not fully capture the quality of the generated mathematical formulations or the reasoning capabilities of the models. Additional metrics, such as the diversity of generated problems or the robustness of the models to variations in problem descriptions, could provide a more comprehensive evaluation.\n\n6. Human-in-the-Loop: The framework involves a significant amount of human effort in curating seed data, designing prompt templates, and validating the generated datasets. This reliance on human expertise may limit the framework's ability to be fully automated and could introduce human biases.\n\n7. Solution-Based Validation: The rejection sampling mechanism relies on comparing the optimal values of the original and generated problem instances. While this approach ensures a high degree of semantic equivalence, it may not guarantee the exact equivalence of the mathematical formulations. Further research is needed to develop more sophisticated validation techniques that can ensure the precise correspondence between natural language descriptions and mathematical formulations.\n\n8. Model Size and Data Scaling: The paper demonstrates that larger models generally achieve better performance, but the relative gains from fine-tuning diminish as model size increases. This suggests that there may be diminishing returns in scaling up the model size, and more efficient training strategies or model architectures might be needed to achieve better performance on complex optimization tasks.\n\nOverall, while the OptMATH framework represents a significant step forward in generating high-quality optimization modeling datasets, addressing these weaknesses could further enhance its effectiveness and applicability in real-world scenarios.", "other_comments_or_suggestions": "No", "questions_for_authors": "No", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "J8B03sFDiR", "reviewer": "Reviewer_mCmd", "summary": "The paper proposes a framework named OptMATH for synthesizing high-quality datasets aimed at optimization modeling from natural language descriptions. This framework addresses the scarcity of optimization datasets by generating problem data through mathematical formulations and back-translation into natural language descriptions. The framework includes a rigorous quality control process involving forward modeling and rejection sampling to ensure mathematical consistency. Extensive experiments demonstrate that models trained on the OptMATH dataset outperform existing benchmarks, showcasing the framework's effectiveness and scalability.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback. We appreciate the suggestions for improving the clarity and scope of our work. We address each point below:\n\n**Regarding Weakness 1:** We will revise the **'Our Contributions' section** to more explicitly contrast OptMATH with prior prompting-based methods. Key differentiators that will be emphasized include:\n\n- **Specialized & Efficient Models:** Fine-tuning on OptMATH-Train yields specialized AutoFormulator models demonstrating superior performance. Additionally, complex, multi-step prompting pipelines (e.g., multi-turn interaction) can sometimes confuse models on simpler tasks, potentially degrading performance relative to complex ones (see Table, OptiMUS on MAMO  EasyLP). Furthermore, fine-tuning yields models with low inference costs suitable for large-scale deployment, unlike prompt-based methods requiring continuous, expensive calls to powerful foundation model APIs.\n- **Beyond Prompting:** Unlike methods relying solely on base LLM capabilities via prompting, OptMATH focuses on fine-tuning models using a large-scale, high-quality dataset synthesized specifically for optimization modeling. This fine-tuning approach **is complementary to prompt engineering techniques**, offering a path to enhance foundational model capabilities rather than just leveraging existing ones.\n- **Scalable High-Quality Data Generation:** Our bidirectional framework systematically generates vast amounts of verified (NL, MF, PD) triplets, addressing the data scarcity that limits fine-tuning approaches.\n- **Rigorous Semantic Validation:** We employ Optimal Value (OV) based rejection sampling, ensuring semantic consistency between the NL description and the problem data, which is more rigorous than checks for mere code executability often used implicitly by prompting methods.\n- **Controllable Complexity & Diversity:** The framework allows generating problem data with targeted difficulty via feedback loops and incorporates diverse problem types.\n\n**Regarding Weaknesses 2&3:** We now include results on the IndustryOR [1] and OptiBench [2] benchmarks. Our Qwen2.5-32B model, finetuned on the OptMATH-Train dataset, demonstrates consistently superior performance, achieving results comparable to GPT-4 and DeepSeek-V3. Results for OptiMUS, Llama, and ORLM on these benchmarks are also presented. Furthermore, to illustrate the impact of finetuning, results for the baseline Llama and Qwen models (before finetuning) have been added. This updated table provides a comprehensive comparative analysis. Due to time constraints, the reproduced ORLM results were neither tuned nor run multiple times, which explains the drop in performance.\n\n| Models | NL4OPT | MAMO  EasyLP | MAMO  ComplexLP | OptMATH-Bench | IndustryOR | OptiBench |\n| --- | --- | --- | --- | --- | --- | --- |\n| GPT-3.5-turbo | 78.0%  | 79.3%  | 33.2%  | 15.0% | 21.0% | 58.1% |\n| GPT-4 | 89.0%  | 87.3%  | 49.3%  | 16.6% | 33.3% | 68.6% |\n| DeepSeek-V3-1226 | 95.9%  | 88.3%  | 51.1%  | 32.6% | 37.0% | 71.6% |\n| OptiMUS base on GPT-4o (2024-05-13) | 78.8% | 77.0% | 43.6% | 20.2% | 31.0% | 45.8% |\n| LLama3.1_8B (pass@1) | 0% | 0.2% | 0% | 0% | 0% | 0% |\n| OptMATH_LLama3.1_8B (pass@1) | 55.5% | 73.9% | 40.8% | 24.4% | 18% | 55.5% |\n| OptMATH_LLama3.1_8B (pass@8) | 97.6% | 94.2% | 71.6% | 51.6% | 37% | 66.6% |\n| Qwen2.5_7B (pass@1) | 86.9% | 83.6% | 21.8% | 1.6% | 10% | 36.2% |\n| OptMATH_Qwen2.5_7B (pass@1) | 94.7%  | 86.5%  | 51.2%  | 24.4% | 20% | 57.9% |\n| OptMATH_Qwen2.5_7B (pass@8) | 98.4% | 94.5% | 72.5% | 56.0% | 38.0% | 68.1% |\n| Qwen2.5_32B (pass@1) | 92.7% | 82.2% | 44.6% | 9.3% | 16.0% | 47.6% |\n| **OptMATH_Qwen2.5_32B (pass@1)** | **95.9%** | **89.9%** | **54.1%** | **34.7%** | **31.0%** | **66.1%** |\n| OptMATH_Qwen2.5_32B (pass@8) | 97.9% | 93.9% | 75.4% | 67.4% | 47.0% | 76.8% |\n| ORLM-LLaMA-3-8B (reported) | 85.7% | 82.3% | 37.4% | * | 38.0% | * |\n| ORLM-LLaMA-3-8B (reproduced) | 84.5% | 74.9% | 34.1% | 2.6% | 24.0% | 51.1% |\n\n**Regarding Weaknesses 4:** We agree this is important. The comprehensive methodology, criteria, and structured organization for our seed data generation—including how we utilize benchmark problem structures (e.g., from MIPLIB) to create validated, parameterized instance generators and associated metadata—are detailed in **Appendix A.2 (Seed Classes)**. Recognizing the value of highlighting this in the main text, we will revise **Section 3** to include a concise summary of this systematic process and clearly reference Appendix A.2 for the full description.\n\n---\n\nReferences:\n\n[1] ORLM: Training large language models for optimization modeling.\n\n[2] OptiBench meets ReSocratic: Measure and improve LLMs for optimization modeling.", "claims_and_evidence": "Experimental results have shown the proposed method can achieve strong results compared with previous studies.", "methods_and_evaluation_criteria": "Yes, the proposed method mainly addresses the data scarcity issue in optimization modeling.", "theoretical_claims": "I did not find any flaws currently.", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "I cannot provide an accurate assessment towards the broader scientific literature, as I am not sufficiently familiar with the target literature.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "### Strengths\n1. The proposed bidirectional data synthesis framework is innovative and provides a systematic solution to the issue of data scarcity in optimization modeling.\n2. The use of rejection sampling ensures high-quality data generation, with the framework demonstrating a remarkable 99.6% accuracy in maintaining mathematical consistency.\n3. The framework is versatile, covering over 10 real-world applications with various optimization problems such as LP, MILP, IP, NLP, and SOCP.\n4. Comprehensive experiments validate the framework's effectiveness, with models trained on OptMATH achieving superior performance on multiple established modeling benchmarks, including NL4Opt and MAMO.\n\n### Weaknesses\n1. While the authors highlight limitations in previous prompting-based methods, the paper lacks clear articulation of how OptMATH specifically overcomes these issues. In the 'Our Contributions' section, consider adding concise explanations that directly contrast OptMATH's advancements with those of prior work, providing readers with a clearer understanding of its unique advantages.\n2. The experiments primarily focus on NL4Opt and MAMO benchmarks. Evaluating the framework on a wider range of standard datasets would be better.\n3. To better illustrate the performance gains achieved by OptMATH, consider including results for Qwen2.5-7B and 32B in Table 1. This would provide a more comprehensive comparative analysis.\n4. The selection of seed data is a critical aspect of OptMATH. The paper should dedicate more space to detailing the process of seed data selection, including the criteria and methodologies employed.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LJqlBHxUmt", "reviewer": "Reviewer_LZQX", "summary": "This paper proposes OptMATH, a method for data generation in the field of optimization modeling, which primarily combines the \"Back Translation\" technique from previous work with the \"rejection sampling\" method. Strictly speaking, it falls under the categories of data augmentation and data annotation within data generation.\n\nAdditionally, it introduce a benchmark that is more comprehensive compared to others in the field, capable of varying difficulty levels and providing stricter scoring, thereby better assessing a model's optimization modeling capabilities.\n\nOverall, a substantial amount of experimentation has been conducted, covering all necessary aspects, and the experimental results are promising, even surpassing those of GPT-4.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the detailed feedback. We address the specific questions raised:\n\n**1. Regarding Questions 1 & 4: OptMATH-Bench Curation, Distinctiveness, and In-Domain Evaluation**\n\nWe clarify OptMATH-Bench's curation and distinction from OptMATH-Train to address concerns about evaluating only in-domain capabilities:\n\n- **Dual Curation Pathways:** OptMATH-Bench was created via two distinct routes. **Pathway 1** started with instances **rejected** by our AutoFormulator (failed OV check), indicating initial difficulty. An \"LLM-Committee\" (inspired by [1], using diverse powerful models like GPT-4, Claude, Gemini, DeepSeek) then filtered these: **(PD, NL) pairs** were retained only if **at least one and at most two** committee members successfully formulated them (passed OV check). This isolated well-posed but non-trivial modeling challenges. Crucially, **human OR experts** subsequently **validated the correctness** of these selected pairs and further refined them based on relevance and clarity. **Pathway 2** involved **experts directly curating challenging problems from external OR literature** (journals, textbooks), ensuring methodological/source independence and including known hard problem types (e.g., NLP, SOCP - see Figure 4).\n- **Addressing \"In-Domain\" Concern:** This dual approach ensures distinction. Pathway 2 uses external sources/methods. Pathway 1 involves significant expert validation. Even if underlying PD distributions overlap, the **distribution of (PD, NL) pairs** in OptMATH-Bench is fundamentally different due to curation. The LLM-Committee filtering specifically ensures the benchmark represents the **hard tail of this paired distribution** relative to strong LLM capabilities. The superior performance of our finetuned model on newly added IndustryOR and OptiBench benchmarks (see Table of the response to **Reviewer mCmd**) further contradicts a purely in-domain evaluation and shows generalization ability.\n\n*Proposed Revision:* We will revise Section 4.3/6.1 to detail these distinct curation pathways, emphasizing expert roles, external sourcing, the LLM-committee filtering targeting the (PD, NL) hard tail, and problem diversity.\n\n**2. Regarding Question 2: Controllable Difficulty Mechanism and Seed Data Validation**\n\nWe clarify the difficulty control and validation, addressing a misunderstanding about generator training:\n\n- **Generator Usage Clarification:** We must correct a misunderstanding: we do **not** *train* generators. We use **pre-defined, parameterized code generators** implementing standard MFs from OR literature (e.g., Bin Packing [2], Appendix E.6). Our novelty is controlling their *input parameters*.\n- **Feedback-Driven Parameter Tuning (Alg. 1):** Difficulty is controlled via an iterative LLM feedback loop (inspired by [3]). The LLM suggests/refines generator parameters based on evaluation feedback (complexity score S(PD), solve time, feasibility) from generated instance batches, steering output towards targets.\n- **Further Validation:** AutoFormulator training uses **curriculum learning**. Seed MFs/generators are expert-curated from literature (Appendix A.2) **to ensure the reasonableness and correctness**. PDs are validated for feasibility/solvability (Alg. 1) and OV equivalence (Alg. 2).\n\n*Proposed Revision:* We will revise Sections 3, 5.2, Appendix A/B to detail the LLM-driven *parameter tuning* (not generator training), curriculum learning, seed curation, and validation.\n\n**3. Regarding Question 3: Use of Benchmarks for Seed Inspiration and Experimental Fairness**\n\nWe clarify the use of solving benchmarks like MIPLIB and address fairness:\n\n- **Solving Benchmarks as Inspiration Only:** These were used *solely* to identify **representative OR problem classes** (e.g., TSP, Job Shop). Their specific solver-focused PD (lacking NL) were **not** reused.\n- **Independent Generator Development:** We built **new parameterized generators from scratch** based on MFs from classic OR **literature** for each identified class (e.g., Job Shop [4], Appendix A.2).\n- **Distinction Justifies Fairness:** The clear separation – using solver benchmarks only for class inspiration, building new generators from literature, not reusing instances, and testing on different types of benchmarks (Optimization Modeling vs. Solving) – ensures our setup is reasonable and fair. We argue that **this strikingly contradicts hacking the modeling benchmarks**. We do not use any benchmarks directly.\n\n*Proposed Revision:* We will revise Section 3 and Appendix A.2 to explicitly state this distinction and the generator development process based on literature.\n\n---\n\n**References:**\n\n[1] Auto-Arena: Automating LLM Evaluations with Agent Peer Battles and Committee Discussions.\n\n[2] Analysis and design of algorithms in combinatorial optimization.\n\n[3] Large language models as optimizers.\n\n[4] The shifting bottleneck procedure for job shop scheduling.", "claims_and_evidence": "Please refer to Questions For Authors", "methods_and_evaluation_criteria": "The methods and metrics make sense.", "theoretical_claims": "This is a work of data synthesis, so there is no proof in the article, and many formulas are just explanations or definitions of properties.", "experimental_designs_or_analyses": "The submission contains some claims that are not fully supported by clear and convincing evidence. Specific problematic claims will be addressed in the \"Other Strengths And Weaknesses\" section.", "relation_to_broader_scientific_literature": "How are the key contributions of the paper related to the broader scientific literature? Be specific in terms of prior related findings/results/ideas/etc.\n1. This article falls within the domains of data augmentation and data annotation in data generation, specifically under the broader category of synthetic data generation.  \n2. The techniques it employs, namely \"back translation\" and \"rejection sampling,\" are not uncommon in the field of synthetic data generation.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Advantages:\n1. The paper introduces OptMATH, a method for data generation in optimization modeling integrating \"Back Translation\" with \"rejection sampling\", contributing to solving the data shortage in field of optimization modeling.\n2. Besides, the introduction of OptMATH-Bench provides a more robust assessment of a model's optimization capabilities compared to existing benchmarks.\n3. The extensive experimentation conducted covers many necessary aspects of the study, yielding promising results that even surpass those of GPT-4.\n\nDisadvantages:\n1. The article lacks transparency in explaining how the rejected data was refined to establish the proposed benchmark, leaving a gap in understanding the curation process.\n2. The paper fails to clearly articulate where the controllable difficulty aspect of the research is manifested. While mentioning the definition of difficulty levels and data generation through a generator, the specifics of training, generation, and validation of the produced data remain undisclosed. This lack of detail raises concerns about the robustness and reliability of the generated seed dataset.\n3. The utilization of data from challenging benchmarks as seed data, without a comprehensive explanation or validation process, raises questions about the fairness and reasonableness of the experimental setup. Testing on different benchmarks, even if not identical, could potentially introduce biases or skew results.\n4. Insufficient elaboration on the methodology employed to create OptMATH-Bench limits the clarity on its distinctiveness from the training data generation process. Without a clear distinction in sources, distributions, and methodology, achieving superior performance on OptMATH-Bench may primarily reflect strong in-domain capabilities rather than a comprehensive model evaluation.", "other_comments_or_suggestions": "No", "questions_for_authors": "1. They did not provide a detailed explanation of how the rejected data was further curated to create the proposed benchmark.  \n2. It is not clearly stated where the controllable difficulty of this work is reflected. They simply mentioned defining difficulty levels and using a generator to produce data? My understanding is that they classify the difficulty of existing data, then train a generator to produce corresponding PD, MF, and OV based on difficulty requirements, which are then fed into the data generation pipeline. However, the specific training and generation details were not further disclosed. Additionally, I assume these data serve as the seed dataset. Was there further validation to ensure the reasonableness and correctness of the generated seed data?  \n3. Part of the data source comes from challenging benchmarks. Generating data from other benchmarks and then testing on different benchmarks, even if not the same one, raises questions: Is this setup reasonable? Is it fair?  \n4. The article does not provide further details on the method for creating OptMATH-Bench. My understanding is that if the methods for generating training data and the benchmark are largely similar, and their sources and distributions are consistent, then good performance on OptMATH-Bench may only indicate strong in-domain capabilities. Outperforming other models on this benchmark might not be entirely fair.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mhtuiUl1Rm", "reviewer": "Reviewer_PCa3", "summary": "This paper proposes an automatic data synthesis framework for LLM optimization modeling. The method can control the problem complexity starting with some seed data. Then, the method obtains the natural language description using a backtranslation step. Experiments demonstrate the effectiveness of training various sizes of LLMs using the generated dataset.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the detailed feedback. We address the specific questions:\n\n**Regarding Claims And Evidence:**\n\n- **Problem Length vs. Complexity:** We fully agree that scenario understanding is crucial for LLMs. However, more complex scenarios naturally require longer, more detailed NL descriptions (e.g., ROADEF 2012 [1]), challenging both LLM abstract reasoning and long-context processing. Thus, length reflects certain scenario complexity. Our paper's results on MAMO (Fig 3, Table 1) show a correlation between scale/length and reduced accuracy.\n- **Scenario/Problem Type Coverage:** Our seed dataset features **over 50 expert-curated problem classes**, which we believe provides substantial and representative coverage. As detailed in Appendix A.2, each class is grounded in **referenced literature and reflects practical application scenarios**. While methods like the evolutionary approach in Li et al. [2] can generate synthetic variations, they often start from a limited set of initial classes (reportedly 8 in that case), whereas our foundation of **50+ manually curated classes is significantly more extensive**. Moreover, such synthetic generation typically recombines existing elements rather than creating fundamentally new problem types. We are confident in the breadth of our 50+ classes; further expansion could effectively build upon this rich seed set using augmentation or evolution.\n\n**Regarding Methods And Evaluation Criteria:**\n\n- **Scalability for Massive Instances:** Our current pipeline already handles considerable scales effectively (up to ~25k characters, Fig 2). However, for the ultra-large instances (>10MB) you mentioned, direct LLM processing is indeed problematic due to context limits. To address this, our framework can adapt using a Model-Data Separation Format (similar to OptiBench [3]):\n    - **Format:** Instance = (NL Description + Structured Data File). NL has scenario/data needs; Data File has numerics.\n    - **OptMATH Adaptation:** Generate (NL + Data File) pairs. Train AutoFormulator to generate code reading the Data File, validating via OV or bipartite graph check. The (MF, NL, PD) interpretation shifts: NL includes scenario + data ref; PD is the final solver-ready format. This is a promising direction for ultra-large scales.\n\n**Regarding Experimental Designs Or Analyses:**\n\n- **Missing Results & Harder Datasets:** We added the requested Optimus results on MAMO/OptMATH Bench and other benchmarks. We also added IndustryOR and OptiBench results. See Table on the response to **Reviewer mCmd.**\n- **Regarding Chain-of-Experts and ComplexOR:** We do not show the results for Chain-of-Experts (CoEs) [5] and ComplexOR. The input for the CoEs requires each instance to include a structured code template and detailed descriptions of its parameters. Additionally, the ComplexOR dataset format is not suitable for end-to-end modeling tests, as its data format for each problem includes a natural language description without numerics.\n\n**Regarding Essential References Not Discussed:**\n\n- We agree [4] is pertinent and will add/discuss it in revised Sec 1. While both explore reverse synthesis, OptMATH uniquely emphasizes:\n    - **Generator-based Scalability:** Abstract MF + parameterized generators enable large-scale, diverse PD creation (Sec 3, Alg 1).\n    - **Rigorous Semantic Validation:** Strict Optimal Value (OV) equivalence check (Sec 4.3) ensures high NL-PD correctness (99.6% manual accuracy), beyond just code executability.\n\n**Regarding Weaknesses:**\n\n- **Applicability to Llama:** We fine-tuned Llama 3.1 8B on OptMATH-Train, showing applicability. Table of the response to Reviewer mCmd shows significant improvements over baseline Llama. Performances of checkpoints are uploaded in [6] named llama_checkpoints_perf.\n\n**Regarding Questions For Authors:**\n\n- **Data Generation Efficiency:** Uses feedback-driven tuning (Alg 1) for controllable quality. PD generation success ~50% (see [6], rate_after_feedback_tuning figure). Rejection sampling (OV matching validation) acceptance is 62.14% (Fig 14: T=1 efficient). 200k valid (NL, PD) samples cost ~$1914 (13.35B tokens, DeepSeek-V3, promotional period), much cheaper than manual creation.\n- **Number of Constraints/Variables:** OptMATH-Train covers wide range: up to ~2500 vars / ~1800 cons, including complex instances. Please refer to optmath_train_under500, optmath_train_linear, optmath_train_log, benchmarks_distribution, benchmarks_under100, and benchmarks_box_plot figures in [6] for a more detailed description.\n\n---\n\n**References:**\n\n[1] ROADEF Challenge 2012 Subject.\n\n[2] Towards foundation models for mixed integer linear programming.\n\n[3] OptiBench: A Large Language Model Benchmark for Optimization Problem Understanding and Formulation.\n\n[4] OptiBench Meets ReSocratic: Measure and Improve LLMs for Optimization Modeling.\n\n[5] Chain-of-Experts: When LLMs meet complex operations research problems.\n\n[6] https://anonymous.4open.science/r/OptMATH-Rebuttal-8F5E", "claims_and_evidence": "The paper mentioned \"This increased complexity, manifested through longer problem descriptions, poses greater challenges for LLMs.\" The author may want to investigate the relation between the modeling accuracy and the problem length. In my understanding, the most challenging part of LLM is to understand the scenarios of the problems, not necessarily the description length.", "methods_and_evaluation_criteria": "While using LLMs to analyze the LP files sounds interesting, this paper can be hard to generalize to large-scale instances. In practice, the LP files can be large, even more than 10M, which poses a great challenge for LLM to process such long input.", "theoretical_claims": "This paper does not contain any proof for theoretical claims.", "experimental_designs_or_analyses": "1. Some experimental results are missing. I wonder whether the results of Chain-of-Experts/Optimus on the MAMO EasyLP/MAMO ComplexLP/OptMATH Bench are missing.\n2. The author may want to conduct experiments on the harder dataset, either the IndustryOR or ComplexOR dataset.", "relation_to_broader_scientific_literature": "This paper is related to the LLM automatic formulation for mathematical optimization.", "essential_references_not_discussed": "The author may want to cite the following works on LLM automatic modeling. \n\n[1] OptiBench Meets ReSocratic: Measure and Improve LLMs for Optimization Modeling.", "other_strengths_and_weaknesses": "Strengths:\n\n1. This paper proposes an interesting bidirectional data synthesis framework for optimization modeling.\n2. The experiment results seem promising in LLM finetune.\n\nWeaknesses:\n\n1. The improvement in the Qwen models is significant. However,  to demonstrate the applicability to other LLMs, I suggest the authors provide experiment results on the Llama models.", "other_comments_or_suggestions": "If the authors can address my concerns, I would like to increase my score.", "questions_for_authors": "1. How about the data generation efficiency of the method?\n2. How about the number of constraints and variables in the generated problems?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper presents a bidirectional data synthesis framework, termed  OptMATH, for constructing datasets on optimization modeling tasks. It starts from curated seed data in mathematical formulations, then employs a backtranslation to obtain natural language descriptions. Further, OptMATH uses forward modeling with rejection sampling to verify the correspondence between the descriptions and problem data. The accepted pairs form the training dataset, while the rejected pairs are filtered to create a challenging benchmark. Experiment results show that models trained on OptMATH achieve better performance on various benchmarks.

As the reviewers point out, the proposed method contributes to solving the data shortage challenge in the field of optimization modeling. The idea of bidirectional data synthesis is interesting. The experiment results show that the synthesized data is promising in improving the modeling ability of LLM. All reviewers give positive scores, though no strong support is provided. I recommend acceptance.
Decision: Accept (poster)