LLMOPT: Learning to Define and Solve General Optimization Problems from Scratch

## Abstract

Abstract Optimization problems are prevalent across various scenarios. Formulating and then solving optimization problems described by natural language often requires highly specialized human expertise, which could block the widespread application of optimization-based decision making. To make problem formulating and solving automated, leveraging large language models (LLMs) has emerged as a potential way. However, this kind of way suffers from the issue of optimization generalization. Namely, the accuracy of most current LLM-based methods and the generality of optimization problem types that they can model are still limited. In this paper, we propose a unified learning-based framework called LLMOPT to boost optimization generalization. Starting from the natural language descriptions of optimization problems and a pre-trained LLM, LLMOPT constructs the introduced five-element formulation as a universal model for learning to define diverse optimization problem types. Then, LLMOPT employs the multi-instruction tuning to enhance both problem formalization and solver code generation accuracy and generality. After that, to prevent hallucinations in LLMs, such as sacrificing solving accuracy to avoid execution errors, model alignment and self-correction mechanism are adopted in LLMOPT. We evaluate the optimization generalization ability of LLMOPT and compared methods across six real-world datasets covering roughly 20 fields such as health, environment, energy and manufacturing, etc. Extensive experiment results show that LLMOPT is able to model various optimization problem types such as linear/nonlinear programming, mixed integer programming and combinatorial optimization, and achieves a notable 11.08% average solving accuracy improvement compared with the state-of-the-art methods. The code is available at https://github.com/caigaojiang/LLMOPT .

## 1 Introduction

Optimization problems are widespread across a various range of scenarios, such as job scheduling [], path planing [, ], matching problem [], and revenue management [], etc. Although powerful solvers are available, formally defining domain-specific optimization problems from natural language descriptions and then developing the solver code are highly complex tasks. It requires specialized domain knowledge, expert involvement, and significant time investment. For example, in finance, the portfolio problems [] often managing thousands of variables and complex constraints, such as full investment constraints, cardinality constraints and pre-assignment constraints, where expert intervention is crucial for achieving accurate (i.e., low simple regret) solutions due to the complexity and specialization of optimization problems.

With the rapid development of large language models (LLMs) like ChatGPT [] and GPT-4 [], using LLMs to automatically formulate and solve optimization problems described by natural language is becoming increasingly attractive. Existing work about it can be roughly divided into prompt-based and learning-based methods. Pioneering studies like Chain-of-Expert [] and OptiMUS [] explore prompt-based methods, leveraging LLM interfaces to extract key information and solve optimization problems. Although these methods show promise, recent advancements in learning-based methods, such as LLaMoCo [] and ORLM [], could demonstrate even greater potential. Notably, ORLM [] reveals that a fine-tuned 7B open-source model can outperform larger pre-trained models like GPT-4, highlighting the value of learning-based methods for optimization tasks.

Figure: Figure 1: An example of the proposed five-element formulation, which provides a structured definition for general optimization problems. Using the five-element formulation as an intermediate step can lead to more accurate solver code and solution.
Refer to caption: x1.png

Although previous research has established the use of LLMs for formulating and solving optimization problems described by natural language, their optimization generalization remains limited, restricting broader applicability and deployment across diverse problems. Optimization generalization, which involves both accuracy and generality in assessing whether LLMs can effectively solve general optimization problems, remains a significant challenge. Specifically, accuracy refers to solving problems correctly, and generality refers to the ability to model and solve diverse optimization problem types across different task scenarios.

In this paper, we propose a unified learning-based framework called LLMOPT to significantly boost the optimization generalization. Through focusing on improving both accuracy and generality to effectively define and solve a wide range of optimization problems, this work tries to narrow the gap between methods and practical applications. Specifically, we propose the five-element formulation for defining optimization problems to enhance solving accuracy as illustrated in Figure 1. To ensure the effectiveness of learning, we design a data augmentation process and conduct data labeling, resulting in high-quality datasets. Leveraging these datasets, we implement multi-instruction supervised fine-tuning to improve the LLM’s accuracy in both defining and solving optimization problems. Additionally, we introduce model alignment to further enhance accuracy and reduce the risk of hallucinations. We also develop an auto-testing process with a self-correction mechanism, enabling the accurate and automated resolution of optimization problems. Finally, the optimization generalization ability of LLMOPT is extensively evaluated on six real-world optimization and operation problem datasets involving roughly 20 domains, including health, environment, energy and manufacturing, etc. The results show that LLMOPT effectively handles various optimization problems, such as linear and nonlinear programming, mixed integer programming and combinatorial optimization, etc. Ablation studies validate the contributions of problem definition and model alignment in improving accuracy. Notably, LLMOPT achieves an average accuracy improvement of 11.08% over state-of-the-art methods.

The consequent sections respectively introduce the related work, present the proposed LLMOPT, show and analysis the empirical results, discuss important issues, and finally conclude the paper.

## 2 Related Work

LLMs for Formulating and Solving Optimization Problems.
The NL4Opt competition [] encourages researchers to explore how LLMs can be leveraged to solve optimization problems efficiently [, , , , ]. Related work can be divided into prompt-based and learning-based methods. The prompt-based methods utilize existing LLM interfaces to automate the solving optimization problems. Chain-of-Expert (CoE) [] introduces an agent-based workflow, where specific tasks are assigned to LLM agents at each stage of reasoning, while OptiMUS [] employs tailored prompts designed specifically for solving linear programming and mixed-integer linear programming problems. The learning-based method remains relatively underexplored. LLaMoCo [] introduces an instruction tuning framework in a code-to-code manner and demonstrates its efficacy through model fine-tuning. ORLM [] proposed a semi-automated approach for generating synthetic training data during the instruction tuning phase, addressing the data-hungry nature of optimization modeling. Besides, Mamo [] offers a benchmark for mathematical modeling with two optimization datasets, and introduces a standardized process for generating solver code using LLMs.

LLMs as Components of Optimizers.
Yang et al. [] propose to use LLMs as optimizers, inspired by Bayesian optimization []. One LLM iteratively optimizes prompts, which are then used to guide another LLM in performing specific optimization tasks. LLMs can also serve as components within an optimizer [, , , , ]. Guo et al. [] uses LLMs to as the crossover and mutation operators in genetic algorithms, thereby finding high-quality prompts. When solving the traveling salesman problem, Liu et al. [] uses LLMs to generate new trajectories based on existing ones, iteratively optimizing them to produce higher-quality solutions. Liu et al. [] and Yang and Li [] apply LLMs to specific multi-objective optimization tasks. Liu et al. [] employs LLMs to simulate both the acquisition function and the surrogate model in the Bayesian optimization process.

## 3 Methodology

### 3.1 An Overview of the Proposed LLMOPT Framework

This work aims to achieve a better optimization generalization of defining and solving the optimization problems through a learning-based approach. To this end, this paper proposes the LLMOPT framework, as illustrated in Figure 2, which comprises three key components: data, learning, and auto-testing. The following sections provide a detailed introduction: First, in Section 3.2, we introduce the five-element formulation to define general optimization problems well, and discuss the augmentation and labeling of training data. Next, in Section 3.3, we detail the multi-instruction supervised fine-tuning (SFT) process and model alignment to enhance solving accuracy. Finally, in Section 3.4, we present the auto-testing process with self-correction.

Figure: Figure 2: The framework of LLMOPT. Sub-figure (a) shows the data labeling process, where experts and GPT-4 work together to label both the five-element formulation and solver code. Sub-figure (b) shows the learning process, in which multi-instruction supervised fine-tuning and model alignment are employed to learn to define and generate code. Sub-figure (c) shows the auto-testing process with self-correction mechanism, which can define and solve optimization problems automatically.
Refer to caption: x2.png

### 3.2 Data: Defining and Labeling General Optimization Problems

#### 3.2.1 Universal Formulation to Define General Optimization Problems

To enable LLMs to formulate more general optimization problems described in natural language, we propose five-element, a universal formulation to define optimization problems. Firstly, in mathematics, an optimization problem can be formally represented as the following expression:

$$ $\min_{\boldsymbol{x}\in\mathcal{X}\subseteq\mathbb{R}^{D}}f(\boldsymbol{x})\,, \,\,\,\,{\rm s.t.}\,\,G(\boldsymbol{x})\leq\boldsymbol{c}\,\,,$ (1) $$

where $\boldsymbol{x}=(x_{1},x_{2},\ldots,x_{D})^{\top}$ is the $D$-dimensional decision variable, $\mathcal{X}\subseteq\mathbb{R}^{D}$ is the feasible domain of $\boldsymbol{x}$, and the goal is to minimize the objective function $f:\mathcal{X}\rightarrow\mathbb{R}$. The constraints are represented by the vector-valued function $G(\boldsymbol{x}):\mathbb{R}^{D}\rightarrow\mathbb{R}^{m}$, where $G(\boldsymbol{x})\leq\boldsymbol{c}$ denotes a series of inequality constraints, and $\boldsymbol{c}=(c_{1},c_{2},\ldots,c_{m})^{\top}$ is the vector of upper bounds for these constraints.

In practice, a vast majority of optimization problems can be formulated as shown in Formula 1.
Based on it, five-element formulation provides a structured and intuitive way to define optimization problems, retaining essential descriptions and making them more accessible and understandable, even for LLMs.
As the name suggests, five-element consist of the five key components of an optimization problem: Sets, Parameters, Variables, Objective and Constraints.
As shown in Figure 1, the element Variables, Objective, and Constraints describe the decision variables $\boldsymbol{x}$, the objective function $f(\boldsymbol{x})$, and the constraints $G(\boldsymbol{x})$ in Formula 1, along with concise descriptions for each. Meanwhile, Sets and Parameters provide detailed information on indices and descriptions, as well as the specific numerical values of the parameters involved in the objective function and constraints.
Importantly, five-element provides a more accurate formulation of the problem by mining and preserving key descriptions of the problem. For example, in Figure 1, the positive integer constraint is implicit in the problem but is effectively captured by the five-element formulation.

As a universal formulation, five-element can define various types of optimization problems, thereby enhancing the optimization generalization ability. For instance, integer programming can be modeled by adjusting the feasible region $\mathcal{X}$, while linear and non-linear programming constraints are captured by different forms of $G(\boldsymbol{x})$. More complex problems, such as multi-objective optimization, extend the objective function to a vector-valued form $F(\boldsymbol{x})$. Additional examples illustrating the formulation of various optimization problems are provided in Appendix J.

#### 3.2.2 Data Augmentation and Labeling

The five-element formulation enables the mapping of optimization problems described in natural language to a universal structure, which can then be used to generate solver code. However, the effectiveness of learning is significantly influenced by the quality, quantity, and distribution of training data. Current available datasets are limited in these aspects and lack proper labels of formulations and solver code. Therefore, data augmentation and labeling are employed to address these gaps.

Firstly, we collect almost all existing optimization problem datasets, including NL4Opt [], Mamo (EasyLP and ComplexLP) [], IndustryOR [], NLP4LP [] and ComplexOR [] whose detail informations are introduced in the Appendix A. Subsequently, 100 samples are randomly selected from each dataset as the reserved testing dataset. For datasets with fewer than 100 samples, all data are used for testing. The remaining samples are used for data augmentation, ensuring a clear separation between training and testing data. In the data augmentation process, LLMs effectively generate data through prompt engineering [, ]. To build a high-quality dataset, seven distinct instructions are applied to 1,763 seed problems. These instructions comprehensively extend the original problems from various perspectives, such as modifying constraints, changing scenarios, altering optimization types, and branching out from the original problem, among other approaches. Finally, experts review the generated problems, removing those with unclear descriptions or infeasible solutions to ensure data diversity and quality. Prompts are detailed in Appendix I.

Then, the labeling of optimization problems using five-element and solver code is performed by experts. GPT-4 [] is also used to generate labels with uncertain correctness, which can be used in the model alignment.
As shown in Figure 2(a), for each optimization problem $p$, experts label the five-element formulation $f$ and solver code $s$, and GPT-4 generates the formulation $f^{\prime}$ and code $s^{\prime}$, respectively. Since GPT-4 may produce errors, experts validate it and assign a desirability label $d^{\prime}$ to indicate whether GPT-4’s label is accurate (True) or not (False).
This process produces two types of labeled data: fully accurate formulations and solver codes, and those with potential errors but validated by experts. The two kinds of data are well-suited for the processes of multi-instruction supervised fine-tuning and model alignment, as discussed in the next section.

### 3.3 Learning: Multi-Instruction SFT and Model Alignment

#### 3.3.1 Multi-Instruction SFT

Direct utilization of LLM to address optimization problems described in natural language often results in inaccuracies, primarily due to their inability to comprehensively capture implicit information. To address this issue, we enhance the capability to both define and solve the problems through multi-instruction supervised fine-tuning (SFT).

The multi-instruction dataset $\mathcal{D}_{\textrm{SFT}}=\mathcal{D}_{\textrm{SFT}}^{f}\cup\mathcal{D}_{
\textrm{SFT}}^{s}$, labeled by experts, is designed to improve both the problem formulation and solving code generation abilities of LLM. The formulation dataset $\mathcal{D}_{\textrm{SFT}}^{f}=\{(u_{i},v_{i})\}_{i}^{N_{f}}$ focuses on enhancing the LLM’s ability to accurately define general optimization problems by providing data pairs $(u_{i},v_{i})$, where $u_{i}$ represents a problem $p$ after being applied to a template, and $v_{i}$ is its corresponding five-element formulation $f$.
Meanwhile, the solving dataset $\mathcal{D}_{\textrm{SFT}}^{s}=\{(u_{i},v_{i})\}_{i}^{N_{s}}$ focuses on improving the LLM’s ability to generate solver code. It contains two types of data: in one, $u_{i}$ is the problem $p$ and $v_{i}$ is the solver code $s$; in the other, $u_{i}$ is the formulation $f$ and $v_{i}$ is the solver code $s$. Both are aligned with the appropriate templates and all the labels in $\mathcal{D}_{\textrm{SFT}}$ are correct labeled by experts.
The instruction templates used in the learning process are detailed in Appendix H, which provides a comprehensive explanation, including a more in-depth description of the five-element to facilitate better understanding by the LLM.

Then, given the instruction $u$ and its label $v$, the objective of SFT is to maximize the conditional probability $p(v\mid u)$. Assuming that $p(v\mid u)=\prod_{i=1}p(v_{i}\mid v_{0:i-1},u)$, this leads to minimizing the negative log-likelihood:

$$ $\mathcal{L}_{\textrm{SFT}}(\theta)=-\mathbb{E}_{(u,v)\sim\mathcal{D}_{\textrm{ SFT}}}\sum_{i=1}\log\pi(v_{i}\mid v_{0:i-1},u;\theta)\,.$ (2) $$

where $\mathcal{D}_{\textrm{SFT}}$ denotes the multi-instruction SFT dataset, which combines instructions from $\mathcal{D}_{\textrm{SFT}}^{f}$ and $\mathcal{D}_{\textrm{SFT}}^{s}$ to train the LLM in defining and solving optimization problems. $\pi(\cdot)$ represents the predicted probability distribution of the LLM, and $\theta$ denotes its parameters.

#### 3.3.2 Model Alignment

To mitigate hallucinations, we incorporate model alignment, which is not utilized by other learning-based methods. We use Kahneman-Tversky Optimization (KTO)[] as our alignment method, as it avoids reward model bias and mitigates the data demands of ranking-based approaches like PPO[] and DPO [].

The KTO algorithm aligns the model using (instruction, completion, desirability) data, where desirability reflects the correctness of the completion using a binary label: True or False. Specifically, the dataset $\mathcal{D}_{\textrm{KTO}}=\{(u_{i},v_{i},d_{i})\}_{i}^{N_{\textrm{KTO}}}$, used by KTO, contains GPT-4 completions with expert-assigned desirability labels. The instructions $u_{i}$ in $\mathcal{D}_{\textrm{KTO}}$ match those in $\mathcal{D}_{\textrm{SFT}}$, covering formulation and code generation tasks. However, the completions $v_{i}$, generated by GPT-4, do not always fulfill the instructions $u_{i}$ correctly, and experts assign desirability labels $d_{i}$ to achieve the model alignment.

Building upon the previous work by Rafailov et al. [], it is evident that the LLM can be interpreted as a reward function. Consequently, we can express the optimal reward for KTO as follows:

$$ $r_{\textrm{KTO}}^{*}(u,v)=\beta\log\frac{\pi^{*}(v\mid u)}{\pi_{\textrm{ref}}( v\mid u)}\,.$ (3) $$

In this equation, $r^{*}(\cdot)$ denotes the optimal reward function of the completion $v$ in response to the instruction $u$, $\pi^{*}(\cdot)$ represents the corresponding optimal model, $\pi_{\textrm{ref}}(\cdot)$ indicates the reference model, and $\beta$ serves as the scaling factor. This ratio measures the relative confidence of the optimal model in generating $v$ compared
to the reference model.

Inspired by classic prospect theory [], the value function employs a logistic transformation on the adjusted ratio of log probabilities, thereby assessing the value of each generation in terms of its desirability and the divergence of the policy from the reference. By substituting the exponential function with a sigmoid function, we derive the value function as follows:

$$ $\phi_{\textrm{KTO}}(u,v;\beta)=\left\{\begin{array}[]{l}\sigma(r_{\textrm{KTO} }(u,v)-z_{\textrm{ref}})\ \ \ \textrm{if}\ v\sim v_{\textrm{desirable}}\mid u, \\ \sigma(z_{\textrm{ref}}-r_{\textrm{KTO}}(u,v))\ \ \ \textrm{if}\ v\sim v_{ \textrm{undesirable}}\mid u\,,\end{array}\right.$ (4) $$

where $z_{\textrm{ref}}$ is defined as the reference point, formulated as: $z_{\textrm{ref}}=\textrm{KL}(\pi^{*}(v^{\prime}\mid u)\mid\mid\pi_{\textrm{ref
}}(v^{\prime}\mid u))$,
and $v\sim v_{\textrm{desirable}}$ means the desirability label $d$ is True.

Another essential component of KTO loss is the weight function, which can be expressed as follows:

$$ $w(v)=\left\{\begin{array}[]{l}\lambda_{D}\ \ \ \textrm{if}\ v\sim v_{\textrm{ desirable}}\mid u\,,\\ \lambda_{U}\ \ \ \textrm{if}\ v\sim v_{\textrm{undesirable}}\mid u\,.\end{ array}\right.$ (5) $$

The weight function $w(v)$ assigns weights to the loss depending on the desirability of the outcome $v$, with $\lambda_{D}$ and $\lambda_{U}$ representing the weights for desirable and undesirable outcomes, respectively.

Finally, the KTO loss function is formulated as:

$$ $\mathcal{L}_{\textrm{KTO}}(\pi^{*},\pi_{\textrm{ref}})=\mathbb{E}_{(u,v,d)\sim \mathcal{D}_{\textrm{KTO}}}[w(v)(1-\phi_{\textrm{KTO}}(u,v;\beta))]\,.$ (6) $$

The KTO loss function encourages the optimal model $\pi^{*}$ to produce completions that align more closely with expert-labeled data, improving the overall optimization generalization of both problem formulation and solver generation.

### 3.4 Auto-Testing: Formulation, Solving and Self-Correction

In the auto-testing process, first, the optimization problem is formulated using the five-element framework based on its natural language description; second, the solver code is generated for the five-element formulation and executed; finally, the solver’s running logs, including output results and errors, are analyzed to determine whether self-correction is necessary. The auto-testing process automates the entire workflow of problem definition and solver code generation, while also integrating the self-correction mechanism for continuous improvement.

Inspired by Chen et al. [], to enhance optimization generalization, we implement self-correction to automatically analyze the output results and identify errors arising during the execution of the solver code. Specifically, the instruction is organized around the problem, the five-element formulation, the solver code, and the execution output results, following a template outlined in Appendix H. The self-correction LLM then determines whether further resolution is necessary or whether the optimal solution has been achieved. If resolution is needed, the model generates an analysis with suggestions and decides to return to whether the problem formulation step or the code generation step. The self-correction mechanism ensures a more robust and adaptive optimization process, improving optimization generalization especially in the accuracy of formulations and final solutions.

## 4 Experiment

To analyze the performance of LLMOPT, we conduct SFT and model alignment based on the open-source LLM Qwen1.5-14B [] and compare it with various learning-based and prompt-based methods on extensive datasets. The experiments aim to answer four key questions below.

- (Q1)
Learning-based vs. Prompting-based Methods: What advantages do learning-based methods, such as LLMOPT, have over LLMs that rely solely on prompt engineering?
- (Q2)
Optimization Generalization (Accuracy and Generality): To what extent can LLMOPT improve optimization generalization ability compared with existing methods?
- (Q3)
Importance of Problem Definition in LLMOPT: How does the proposed five-element formulation as an intermediate step contribute to boost accuracy in solving optimization tasks?
- (Q4)
Effectiveness of Model Alignment in LLMOPT: How effective is model alignment in enhancing the solving accuracy of LLMs for optimization tasks?

The four questions are answered in order in the next sections which include a detailed introduction to the experiments, followed by the analysis of results.

### 4.1 Experimental Setup

The experiments are conducted on six real-world optimization and operation task datasets, namely NL4Opt [], Mamo (EasyLP and ComplexLP) [], IndustryOR [], NLP4LP [], ComplexOR []. These datasets encompass about 20 scenario and 7 types of optimization problems. Detailed descriptions of these datasets are provided in Appendix A. Training and testing data are strictly separated. For the NL4Opt, Mamo (EasyLP and ComplexLP) datasets, we shuffle the original datasets and randomly extract 100 data from each dataset as the testing datasets, and the remaining data is used as seed data for data augmentation. The IndustryOR dataset retains its original partitioning. Due to the limited data in NLP4LP and ComplexOR, all data from these datasets are used for testing and excluded from the training process.

In the experiment, we use three performance metrics to comprehensively evaluate the optimization generalization of the algorithm, namely, Execution Rate (ER), Solving Accuracy (SA), and Average Solving Times (AST). Specifically, ER refers to the proportion of solutions whose code can run without any errors and has running results output. SA refers to the proportion of solutions that correctly solve the optimization problem, i.e., find the optimal solution. AST refers to the average number of times the self-correction process is performed during the test. In our experiment, the maximum number of self-correction re-solves is set to 12 times.

### 4.2 Analysis of Optimization Generalization

In this section, we compare LLMOPT with prompt-based methods (Reflexion [], Chain-of-Experts [], OptiMUS []), learning-based methods (ORLM [] built on Mistral-7B, Deepseek-Math-7B-Base, LLaMa3-8B), and GPT-4 [], demonstrating the optimization generalization capability of LLMOPT.

Figure: Figure 3: The results of the SA metric. Sub-figure (a) compares the SA performance of GPT-4o and the Qwen1.5-14B model with only SFT, showing the potential of learning-based methods. Sub-figures (b), (c) and (d) perform ablation on five-element and KTO. The Mamo Easy and Mamo Complex datasets are abbreviated as Mamo E. and Mamo C., respectively, due to space constraints.
Refer to caption: x3.png

**Table 1: Comparison of the SA metric between LLMOPT and learning-based methods. The results for ORLM are cited from Tang et al. []. Underlined results indicate the previous SOTA, while bold results indicate the current SOTA.**
| Datasets | NL4Opt | Mamo Easy | Mamo Complex | IndustryOR |  |
| --- | --- | --- | --- | --- | --- |
| GPT-4 Directly | 47.3% | 66.5% | 14.6% | 28.0% |  |
| ORLM | Mistral-7B | 84.4% | 81.4% | 32.0% | 27.0% |
| Deepseek-Math-7B-Base | 86.5% | 82.2% | 37.9% | 33.0% |  |
| LLaMa3-8B | 85.7% | 82.3% | 37.4% | 38.0% |  |
| LLMOPT (Ours) | Qwen1.5-14B | 93.0% | 97.0% | 68.0% | 46.0% |
| Improvement Rate $\uparrow$ | +6.5% | +14.7% | +30.1% | +8.0% |  |

**Table 2: Comparison of the SA metric between LLMOPT and prompt-based methods. The results for Reflexion, Chain-of-Experts, and OptiMUS are cited from Xiao et al. []. Underlined results indicate the previous SOTA, while bold results indicate the current SOTA.**
| Datasets | NL4Opt | NLP4LP | ComplexOR |
| --- | --- | --- | --- |
| GPT-4 Directly | 47.3% | 35.8% | 9.5% |
| Reflexion | 53.0% | 46.3% | 19.1% |
| Chain-of-Experts | 64.2% | 53.1% | 38.1% |
| OptiMUS | 78.8% | 72.0% | 66.7% |
| LLMOPT (Ours) | 93.0% | 83.8% | 72.7% |
| Improvement Rate $\uparrow$ | +14.2% | +11.8% | +6.0% |

Learning-based vs. Prompting-based Methods (Answer to Q1).
To demonstrate the potential of the learning-based method, we perform SFT on Qwen1.5-14B [] and compare it with GPT-4o using the same prompt (Appendix H). As shown in Figure 3(a), the results indicate that LLM with only SFT can achieve comparable performance to GPT-4o. For the complete LLMOPT (including model alignment and self-correction), as shown in Figure 4, LLMOPT outperforms both GPT-4o and GPT-4-Turbo, achieving higher solving accuracy with fewer solving times across all six datasets. All these results demonstrate the potential of learning-based methods.

Accuracy (Answer to Q2-1).
We compare the solving accuracy (SA) of LLMOPT with prompt-based and learning-based methods as shown in Table 1 and  2, respectively. To ensure consistency in reproduction, we only cite the results from the original paper. LLMOPT achieves state-of-the-art (SOTA) performance in SA across six datasets, outperforming both prompt-based and learning-based methods. Compared to learning-based methods, defining with the five-element formulation and self-correction process improves the accuracy, with an average increase of 14.83% on four datasets. For prompt-based methods, SFT and model alignment enhance the LLM’s ability to solve optimization problems, yielding a 10.67% average improvement across three datasets. Overall, LLMOPT improves SOTA performance by an average of 11.08% across the six datasets.

Generality (Answer to Q2-2).
To analyze the generality of LLMOPT on general optimization problems, we conduct testing on six datasets, each covering diverse optimization types, such as linear programming, nonlinear programming, integer programming, mixed-integer programming, multi-objective optimization, and combinatorial optimization, as detailed in Appendix G. These datasets span over 20 different fields, including agriculture, energy, and healthcare, as detailed in Appendix F. LLMOPT achieves state-of-the-art results across all datasets, demonstrating its generality.
Moreover, despite the absence of NLP4LP and ComplexOR data in the training dataset, LLMOPT still achieves SOTA performance on them, as shown in Table 1, further proving its generality.
Furthermore, seven examples of the five-element formulation applied to various optimization problems are provided in Appendix J to illustrate the generality of this definition.
The high accuracy and generality demonstrate the optimization generalization capability of LLMOPT.

### 4.3 Ablation Study

In this section, we conduct comprehensive ablation experiments, including LLMOPT without five-element formulation, without KTO alignment, and without self-correction. The results of these experiments are shown in Figure 3, Figure 4, and Table 3.

Importance of Problem Definition (Answer to Q3).
In order to explore the importance of problem definition, we evaluate three metrics on the ablation experiments: Execution Rate (ER), Solving Accuracy (SA), and Average Solving Time (AST). As detailed in Table 3 and Figure 3(b), these results show that using the five-element formulation as the problem definition improves SA across all six datasets.
However, this definition can sometimes lower the ER, as the LLM may oversimplify the problem without it, producing error-free but inaccurate code. In contrast, using the five-element approach ensures correct code generation, improving SA at the cost of slightly reduced ER.

Effectiveness of Model Alignment (Answer to Q4).
Alignment typically enhances the efficiency and effectiveness of LLMs on specific tasks.
As shown in Figure 4(b) and  3(c), the ablation results shows that KTO alignment not only significantly boosts SA but also reduces AST across all six datasets. Furthermore, as shown in Figure 4(b),  3(d), and Table 3, the combination of the five-element formulation and KTO alignment improves performance in terms of SA and AST on complex datasets.

Figure: Figure 4: Comparison of SA and AST between LLMOPT, GPT-4, and ablated versions.
Refer to caption: x4.png

**Table 3: The ablation results of LLMOPT without (w/o) five-element formulation, KTO alignment and self-correction, respectively. In the experiments without self-correction, the model is called only once, resulting in an AST of 1.00. Bold indicates the best performance for each metric.**
| Metrics | ER | SA | AST | ER | SA | AST | ER | SA | AST |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Dataset | NL4Opt | Mamo Easy | Mamo Complex |  |  |  |  |  |  |
| LLMOPT | 99.0% | 93.0% | 3.07 | 100.0% | 97.0% | 1.70 | 97.0% | 68.0% | 6.30 |
| w/o five-element | 99.0% | 91.0% | 2.95 | 100.0% | 96.0% | 2.36 | 98.0% | 64.0% | 6.38 |
| w/o KTO | 99.0% | 90.0% | 3.47 | 100.0% | 97.0% | 2.80 | 95.0% | 65.0% | 6.79 |
| w/o self-correction | 79.0% | 57.0% | (1.00) | 85.0% | 71.0% | (1.00) | 52.0% | 32.0% | (1.00) |
| Dataset | IndustryOR | NLP4LP | ComplexOR |  |  |  |  |  |  |
| LLMOPT | 92.0% | 46.0% | 8.35 | 100.0% | 83.8% | 7.00 | 94.7% | 72.7% | 10.27 |
| w/o five-element | 92.0% | 41.0% | 8.50 | 96.9% | 73.0% | 7.43 | 94.7% | 72.7% | 10.05 |
| w/o KTO | 92.0% | 43.0% | 8.88 | 89.2% | 64.9% | 8.32 | 100.0% | 54.6% | 10.84 |
| w/o self-correction | 55.0% | 31.0% | (1.00) | 47.7% | 35.2% | (1.00) | 47.4% | 18.2% | (1.00) |

## 5 Discussion

Attempt on Larger Models.
To explore the potential for further performance improvement of LLMOPT on larger models, we deploy it on Qwen2-72B []. The detailed results are provided in Appendix D. We conduct experiments on two complex task datasets, Mamo Complex and IndustryOR, and the results show that LLMOPT based on Qwen2-72B shows significant improvements in both SA and ER compared with its performance on Qwen1.5-14B. Given the trade-off between performance and the costs of training and deployment, i.e., green computing issue, we opt not to use larger models, since Qwen1.5-14B already achieves state-of-the-art performance.

Compared with OpenAI o1 Model.
OpenAI o1 model [] has recently garnered attention for its strong reasoning ability. However, due to the limited availability of its API and restrictions on the number of weekly trials, our evaluation of the o1 model’s performance is limited. We conduct experiments on 10 easy and 10 complex problems from the Mamo Complex dataset. With a single call, the o1 model successfully generated code to solve 7 easy problems and 5 complex problems. These results suggest that the o1 model is more accurate in solving optimization problems compared with GPT-4 series models. However, due to the lack of open access, the absence of detailed technical specifications and training data description of o1, as well as the limitations on usage and high costs, fine-tuning open-source large models with LLMOPT presents a more cost-effective solution for achieving better optimization generalization in real industrial scenarios.
When the API of o1 is available in future, we plan to comprehensive compare LLMOPT with it.

The Seesaw Issue of LLMs.
The seesaw issue in LLMs refers to the trade-off between improving the performance on specialized tasks and its generalization across diverse tasks, where gains in one area often result in losses in another. To assess whether enhancing the LLM’s ability to define and solve optimization problems affects its performance on other tasks, we compared the model’s performance before and after fine-tuning across 10 general tasks, including math, code, classification, information extraction, open QA, closed QA, text generation, brainstorming, rewriting and summarization. The results indicate that the fine-tuned model showed performance improvements in 6 tasks and declines in 4 tasks, with an average performance increase of 0.3% across all tasks. Importantly, no significant trade-off or seesaw effect is observed. Detailed results are provided in Appendix E.

About High-Quality Training Data.
High-quality data is crucial for fine-tuning LLMs []. Accurate and diverse training data allows the model to improve its task-specific performance. However, optimization problem data described in natural language is relatively scarce. Although we have collected as many optimization problem datasets as possible, which are detailed in Appendix A, these datasets often lack high-quality labels. For instance, in the IndustryOR dataset [], some data have incorrectly labeled optimal solutions, and the NL4Opt dataset [] does not provide optimal solution annotations but only entity labels. This highlights the scarcity of optimization problems training data. While employing prompt engineering for data augmentation in this work, expert labeling remains a time-consuming and labor-intensive process. Efficiently gathering, synthesizing and generating more diverse and well-labeled high-quality data remains an issue that cannot be ignored in this research direction in future.

## 6 Conclusion

This paper focuses on the challenge of optimization generalization in LLMs, including the accuracy in solving optimization problems and the generality of the problem types that LLMs can handle.
We propose a learning-based framework called LLMOPT, which significantly improves LLMs’ ability to define and solve general optimization problems through multi-instruction supervised fine-tuning and model alignment. LLMOPT introduces the five-element formulation as a universal definition for various types of optimization problems to enhance solving accuracy. LLMOPT is extensively evaluated on a wide range of optimization tasks. It achieves the state-of-the-art optimization generalization ability across all of them, i.e., a notable 11.08% average solving accuracy improvement.

## Ethics and Reproducibility Statement

Ethics.
This work does not involve any human subjects, personal data, or sensitive information. All the testing datasets used are publicly available, and no proprietary or confidential information is used. We follow recommendations to use the Azure OpenAI service when using GPT models.

Reproducibility.
Experimental settings are described in Section 4 and Appendix B, and datasets included in Appendix A. The code is available at [https://github.com/caigaojiang/LLMOPT](https://github.com/caigaojiang/LLMOPT).

## Appendix

## Appendix A Datasets

### A.1 The Introduction of Datasets

We conduct experiments on the following real-world optimiation task datasets.

**Table 4: The statistics of the optimization problem datasets.**
| Dateset Name | # of Data | Notes |
| --- | --- | --- |
| NL4Opt [] | 1101 | All data are unlabeled optimal solution. |
| Mamo Complex [] | 211 | - |
| Mamo Easy [] | 652 | - |
| IndustryOR [] | 100 | Another 3000 data without optimal solution. |
| NLP4LP [] | 65 | All data are unlabeled optimal solution. |
| ComplexOR [] | 19 | All data are unlabeled optimal solution. |

NL4Opt []. The NL4Opt Competition curates a dataset comprising 1101 annotated LPWPs across 6 diverse domains. Each set contained LPWPs from source domains like sales, advertising, and investment, ensuring representation across all splits. However, LPWPs from target domains such as production, transportation, and sciences were exclusively reserved for the development and testing sets.

Mamo []. The optimization dataset of Mamo benchmark consists of two parts, Easy LP and Complex LP. Easy LP contains 652 high school-level MILP problems for basic learning of linear and mixed integer linear programming. Complex LP provides 211 undergraduate-level challenges, integrating LP and MILP, suitable for advanced learners and researchers. These problems are more complex, covering different applications and theoretical challenges, suitable for advanced courses and research projects, and provide comprehensive development of optimization skills from basic to complex.

IndustryOR []. IndustryOR is the first industrial dataset designed specifically for optimization modeling. It incorporates data from 13 different industries and covers a variety of real-world scenarios. IndustryOR consists of real operations research (OR) problems from eight different industries. It covers five types of optimization problems: linear programming, integer programming, mixed integer programming, nonlinear programming, and other special problem types. These problems are also divided into three difficulty levels. The training dataset of IndustryOR contains 3000 instances without labeling optimal solution and the test dataset contains 100 instances with optimal solution.

NLP4LP []. NLP4LP (Natural Language Processing for Linear Programming) is a dataset that includes 65 samples we identified from its repository. These problems are sourced from optimization textbooks and lecture notes covering areas such as facility location, network flow, scheduling, and portfolio management. Each instance in NLP4LP includes a description, sample parameter data file, and optimal value derived from textbook solutions or manual solving, offering a range of complex optimization challenges of varying lengths.

ComplexOR []. ComplexOR dataset is developed with the collaboration of three specialists in operations research. We identify 19 samples from its repository, sourced from diverse references such as academic papers, textbooks, and real-world industrial scenarios. These problems encompass a broad spectrum of topics, including supply chain optimization, scheduling problems, and warehousing logistics.

### A.2 Training Datasets for Multi-Instruction SFT and Model Alignment

We introduce the training datasets used for multi-instruction SFT and model alignment, as illustrated in Table 5. After data augmentation, the SFT dataset comprises 9,828 instances, while the KTO alignment dataset contains 19,563 instances. To keep the generalization capability of the language model, we incorporated additional data into the datasets, which is open-source and available at [https://instructions.apps.allenai.org/](https://instructions.apps.allenai.org/) and [https://huggingface.co/datasets/argilla/ultrafeedback-binarized-preferences-cleaned-kto](https://huggingface.co/datasets/argilla/ultrafeedback-binarized-preferences-cleaned-kto). It is important to note that we do not utilize the entire datasets, but instead randomly selected 20,000 and 30,000 instances for the learning process.

**Table 5: The number of samples in the dataset for SFT and model alignment.**
|  | # of Optimization Data | # of Other Data | Total |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| SFT | 9828 | 20000 | 29828 |  |  |  |  |
| KTO | True Label | 9827 | 19563 | 26384 | 30000 | 36212 | 49563 |
| False Label | 9735 | 3616 | 13351 |  |  |  |  |

## Appendix B Detail setting of multi-instruction SFT and model alignment.

We implement all model training using the PyTorch framework and utilize Qwen 1.5 with 14 billion parameters [] as the base model. We utilize NVIDIA 8*A100 Tensor Core GPUs with 80 GB each for model training and employ 1*A100 GPU for model inference. The hyper-parameters of the training are shown in Table 6.

**Table 6: The detail training settings for SFT and KTO alignments.**
|  | LoRA_Dropout | LoRA_R | LoRA_Alpha | LearningRate | WarmUp Ratio | BatchSize | MaxLength | Epochs | (Un-) DesirableWeight | Beta |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| SFT | 0.05 | 64 | 16 | 3.00E-04 | 0.01 | 24 | 2048 | 20 | / | / |
| KTO | 0.05 | 16 | 16 | 5.00E-07 | 0.1 | 4 | 2048 | 20 | 1.0 | 0.1 |

For the SFT, we adopt the code from the public Github
repository at [https://github.com/QwenLM/Qwen](https://github.com/QwenLM/Qwen), and for KTO training, we adopt the code from the public Github repository at [https://github.com/huggingface/trl](https://github.com/huggingface/trl). And our project code is available at [https://github.com/caigaojiang/LLMOPT](https://github.com/caigaojiang/LLMOPT).

## Appendix C Detailed Results

In this section, we present detailed results comparing LLMOPT with the GPT-4 series models, including the Solving Accuracy (SA) shown in Table 7, the Execution Rate (ER) shown in Table 8, and the Average Solving Times (AST) shown in Table 9. All the results LLMOPT is based on Qwen1.5-14B. In all the tables, the values in bold represent the best performance achieved by LLMOPT, while the underlined values indicate the best performance among the GPT-4 series models. Across all six datasets, LLMOPT outperforms GPT-4 in both Solving Accuracy (SA) and Execution Rate (ER) metrics. Additionally, LLMOPT often achieves better performance than GPT-4 in terms of Average Solving Times (AST).

**Table 7: Detailed results of solving accuracy (SA). Bold indicates LLMOPT’s best performance, while underlined indicates the best results from GPT-4 models.**
|  |  | NL4Opt | Mamo Easy | Mamo Complex | IndustryOR | NLP4LP | ComplexOR |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| w/o debug | GPT-4-turbo | Directly | 47.0% | 65.0% | 13.0% | 26.0% | 13.5% | 9.1% |
| + 5-elem | 48.0% | 66.0% | 19.0% | 28.0% | 10.8% | 0.0% |  |  |
| GPT-4o | Directly | 52.0% | 67.0% | 19.0% | 31.0% | 18.9% | 9.1% |  |
| + 5-elem | 62.0% | 67.0% | 23.0% | 27.0% | 16.2% | 18.2% |  |  |
| LLMOPT | w/o five-element & KTO | 56.0% | 65.0% | 29.0% | 22.0% | 16.2% | 9.1% |  |
| w/o KTO | 52.0% | 63.0% | 33.0% | 28.0% | 18.9% | 9.1% |  |  |
| w/o five-element | 60.0% | 72.0% | 31.0% | 27.0% | 16.2% | 9.1% |  |  |
| LLMOPT (w/o debug) | 57.0% | 71.0% | 32.0% | 31.0% | 35.2% | 18.2% |  |  |
| with debug | GPT-4-turbo | + debug | 75.0% | 81.0% | 25.0% | 30.0% | 32.4% | 18.2% |
| + 5-elem + debug | 76.0% | 85.0% | 27.0% | 29.0% | 37.8% | 27.3% |  |  |
| GPT-4o | + debug | 81.0% | 91.0% | 34.0% | 34.0% | 32.4% | 27.3% |  |
| + 5-elem + debug | 83.0% | 90.0% | 38.0% | 34.0% | 35.2% | 36.4% |  |  |
| LLMOPT | w/o five-element & KTO | 88.0% | 89.0% | 46.0% | 33.0% | 54.1% | 45.5% |  |
| w/o KTO | 90.0% | 97.0% | 65.0% | 43.0% | 64.9% | 54.6% |  |  |
| w/o five-element | 91.0% | 96.0% | 64.0% | 41.0% | 73.0% | 72.7% |  |  |
| LLMOPT (Full) | 93.0% | 97.0% | 68.0% | 46.0% | 83.8% | 72.7% |  |  |

**Table 8: Detailed results of execution rate (ER). Bold indicates LLMOPT’s best performance, while underlined indicates the best results from GPT-4 models.**
|  | NL4Opt | Mamo Easy | Mamo Complex | IndustryOR | NLP4LP | ComplexOR |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| w/o debug | GPT-4-turbo | Directly | 64.0% | 71.0% | 24.0% | 34.0% | 21.5% | 15.8% |
| + 5-elem | 57.0% | 68.0% | 23.0% | 33.0% | 20.0% | 15.8% |  |  |
| GPT-4o | Directly | 71.0% | 71.0% | 42.0% | 40.0% | 35.4% | 42.1% |  |
| + 5-elem | 69.0% | 72.0% | 38.0% | 36.0% | 33.9% | 36.8% |  |  |
| LLMOPT | w/o five-element & KTO | 74.0% | 74.0% | 41.0% | 39.0% | 33.9% | 36.8% |  |
| w/o KTO | 73.0% | 72.0% | 39.0% | 38.0% | 33.9% | 26.3% |  |  |
| w/o five-element | 82.0% | 86.0% | 53.0% | 63.0% | 38.5% | 42.1% |  |  |
| LLMOPT (w/o debug) | 79.0% | 85.0% | 52.0% | 55.0% | 47.7% | 47.4% |  |  |
| with debug | GPT-4-turbo | + debug | 95.0% | 99.0% | 95.0% | 87.0% | 81.5% | 79.0% |
| + 5-elem + debug | 93.0% | 98.0% | 91.0% | 83.0% | 78.5% | 79.0% |  |  |
| GPT-4o | + debug | 99.0% | 99.0% | 96.0% | 83.0% | 86.2% | 89.5% |  |
| + 5-elem + debug | 96.0% | 100.0% | 96.0% | 82.0% | 84.6% | 84.2% |  |  |
| LLMOPT | w/o five-element & KTO | 100.0% | 100.0% | 96.0% | 90.0% | 73.9% | 73.7% |  |
| w/o KTO | 99.0% | 100.0% | 95.0% | 92.0% | 89.2% | 100.0% |  |  |
| w/o five-element | 99.0% | 100.0% | 98.0% | 92.0% | 96.9% | 94.7% |  |  |
| LLMOPT (Full) | 99.0% | 100.0% | 97.0% | 92.0% | 100.0% | 94.7% |  |  |

**Table 9: Detailed results of average solving times (AST). Bold indicates LLMOPT’s best performance, while underlined indicates the best results from GPT-4 models.**
|  | NL4Opt | Mamo Easy | Mamo Complex | IndustryOR | NLP4LP | ComplexOR |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| GPT-4-turbo | + debug | 4.66 | 3.87 | 9.76 | 9.13 | 9.97 | 11.32 |
| + 5-elem + debug | 4.75 | 3.59 | 9.81 | 9.47 | 10.08 | 11.32 |  |
| GPT-4o | + debug | 3.89 | 3.26 | 9.15 | 8.96 | 9.89 | 11.13 |
| + 5-elem + debug | 3.93 | 3.61 | 9.06 | 8.97 | 10.40 | 10.95 |  |
| LLMOPT | w/o five-element & KTO | 3.58 | 3.45 | 8.11 | 9.25 | 8.68 | 10.79 |
| w/o KTO | 3.47 | 2.80 | 6.79 | 8.88 | 8.32 | 10.84 |  |
| w/o five-element | 2.95 | 2.36 | 6.38 | 8.50 | 7.43 | 10.05 |  |
| LLMOPT (Full) | 3.07 | 1.70 | 6.30 | 8.35 | 7.00 | 10.27 |  |

## Appendix D Comparasion between Qwen1.5 and Qwen 2

In this section, we compare the performance of LLMOPT based on Qwen1.5-14B and Qwen2-72B [], focusing on two challenging tasks: Mamo Complex and IndustryOR. As illustrated in Figure 5, LLMOPT based on Qwen2-72B demonstrates superior performance in both tasks. Specifically, LLMOPT on Qwen2-72B outperforms Qwen1.5-14B across both the Solving Accuracy (SA) and Execution Rate (ER) metrics, highlighting the enhanced capabilities of the larger model in handling complex optimization problems.

Figure: Figure 5: Comparasion in (a) solving accuracy and (b) execition rate between Qwen1.5 and Qwen2.
Refer to caption: x5.png

## Appendix E Generalization on Other Tasks

In this section, we compare the model’s performance before and after fine-tuning across 10 tasks: Math, Code, Classification, Extraction, Open QA, Closed QA, Text Generation, Brainstorming, Rewriting, and Summarization, as shown in Figure 6. The results indicate that LLMOPT does not significantly degrade performance on a broad range of tasks.

Figure: Figure 6: Results of performance evaluation on other task types.
Refer to caption: x6.png

## Appendix F Scenarios of Datasets

The scenarios of the datasets are presented in Figure 7, covering 20 scenarios approximately.

Figure: Figure 7: Scenarios of the datasets.
Refer to caption: x7.png

## Appendix G Optimization Types of Datasets

The types of optimization problems are presented in Figure 8, categorized into 7 different classes.

Figure: Figure 8: Optimization types of the datasets.
Refer to caption: x8.png

## Appendix H Templates for Instructions on Learning and Auto-Testing

In this section, all instruction templates are introduced. Each template contains instruction constructed by fully filling in the curly braces “$\{\cdot\}$” to specify the required content. The following templates are used both for constructing learning instructions and for prompts during auto-testing.

Figure: Listing 1: Instruction template of define the five-element formulation from the problem description.

Figure: Listing 2: Instruction template of generate the solver code from the five-element formulation.

Figure: Listing 3: Instruction template of generate the solver code from the problem description.

Figure: Listing 4: Instruction template of self-correction.

## Appendix I Templates for Data Augmentation

In this section, a general template for data augmentation is introduced. The same as previous section, the curly braces “$\{\cdot\}$” should be fully filled. The “ONE OF THE RULES BELOW” represents one of the rules of generating new questions, selected randomly from the ones introduced below.

Figure: Listing 5: General template for data augmentation.

Figure: Listing 6: Rules for data augmentation.

## Appendix J Examples of Five-Element Formulation for Different Optimization Problems

This section presents examples of five-element formulations for different optimization problems, including seven examples spanning linear programming, integer programming, mixed-integer programming, nonlinear programming, and combinatorial optimization. By illustrating the five-element formulation across these diverse problem types, we aim to show the broad applicability of this modeling approach for general optimization problem.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 6
presentation: 3
questions: **Questions and Suggestions for the Authors:**

1. **Clarification on Expert Involvement in Data Labeling**:
   - **Question**: Could the authors provide more details on the qualifications, expertise, or at least agreement rates of the human experts involved in the data labeling process? 
   - **Suggestion**: Including a quantitative or qualitative analysis of expert reliability and consistency would strengthen the validity of the data labeling claims. Evidence such as statistics of certifications or relevant experience, and agreement rates among experts would be valuable to better assess the credibility of the labeled data.

2. **Theoretical Justification for the Five-element Formulation**:
   - **Question**: What is the theoretical basis for claiming that the five-element formulation is a universal method for defining optimization problems? 
   - **Suggestion**: Providing additional references or an in-depth theoretical analysis that supports the universality and applicability of the five-element formulation across various optimization scenarios would enhance the argument. Empirical comparisons to alternative problem formulations could also be beneficial.

3. **Comparative Performance Analysis of Self-correction Mechanism**:
   - **Question**: How does the self-correction mechanism in LLMOPT compare with similar correction methods, such as manual debugging or integration with other LLMs or methods like GPT-4 in a correction loop?
   - **Suggestion**: Presenting a detailed comparison of LLMOPT’s self-correction mechanism with existing correction strategies or showing performance metrics of LLMOPT combined with baseline models could provide insights into its relative effectiveness. This would help demonstrate whether the self-correction offers unique advantages or is comparable to simpler, established methods.

4. **Scalability Concerns**:
   - **Question**: How do the authors envision scaling the current framework for large-scale practical deployments given the heavy reliance on expert validation?
   - **Suggestion**: Suggestions for automating or streamlining the expert review process, possibly by incorporating semi-supervised or active learning techniques, could address concerns about scalability and long-term sustainability of the labeling process.

5. **Detailed Explanation of Generalization Capabilities**:
   - **Question**: Can the authors provide more specific examples or case studies where the five-element formulation showed clear advantages in generalization compared to alternative models?
   - **Suggestion**: Adding empirical evidence or a breakdown of performance across different types of optimization problems, particularly ones not included in the training set, would strengthen the claim of its broad applicability and effectiveness.

6. **Addressing Potential Limitations in Self-correction**:
   - **Question**: How does the self-correction mechanism handle potential limitations or biases in repeated corrections, such as overfitting to a specific type of error?
   - **Suggestion**: Discussing safeguards or mechanisms in place to ensure diverse error handling and avoiding biased correction loops would provide more confidence in the robustness of the self-correction feature.
soundness: 3
strengths: - The paper introduces a *novel five-element formulization* to define optimization problems, which significantly enhances the ability of large language models (LLMs) to accurately interpret and transform natural language descriptions into solvable optimization problems. This structured approach captures the essential elements of optimization scenarios, ensuring clearer problem representation and facilitating better code generation. The inclusion of elements such as sets, parameters, variables, objectives, and constraints helps the model produce more precise solutions and reduces the risk of omitting implicit problem aspects.

- Another notable feature is the *self-correction in test-time*, an automated process called *Auto-testing* integrated within LLMOPT to identify and rectify errors in generated solver code during execution. This mechanism analyzes output logs and determines if adjustments are necessary, enhancing the robustness and accuracy of problem-solving without manual intervention. By automatically looping back to problem reformulation or code generation when needed, LLMOPT can iteratively improve the accuracy of its solutions and adapt effectively to complex challenges.

- The paper boasts *strong evaluation results*, demonstrated by extensive testing across six real-world datasets encompassing approximately 20 different fields and various optimization problem types, such as linear and nonlinear programming and combinatorial optimization. LLMOPT shows superior performance, achieving a notable 11.08% average improvement in solving accuracy compared to existing state-of-the-art methods. This result underscores the framework’s generalization capabilities and effectiveness in diverse optimization scenarios, validated by comprehensive comparisons and ablation studies.
summary: The authors of this paper introduced **LLMOPT**, a novel learning-based framework designed to enhance large language models' (LLMs) capability to define and solve general optimization problems described in natural language. The key innovation is the **five-element formulization**, which structures optimization problems into sets, parameters, variables, objectives, and constraints, enabling more accurate problem representation and solver code generation. LLMOPT also incorporates **multi-instruction supervised fine-tuning (SFT)** and **model alignment** using the Kahneman-Tversky Optimization (KTO) method to improve both the accuracy and generalization of solutions.

Another significant contribution is the **self-correction mechanism** in the auto-testing process, which automates error analysis and solution refinement during execution, ensuring robust performance without manual intervention in test-time. The framework was evaluated on six real-world datasets, covering diverse optimization types and fields, achieving an **average accuracy improvement of 11.08%** over state-of-the-art methods. This demonstrates LLMOPT's effectiveness in boosting both the accuracy and generalizability of LLMs in solving complex optimization problems.
weaknesses: 1. **Complexity of Data Labeling**: The proposed five-element formulation and the use of expert labeling for optimization problem formulations and solver code rely significantly on manual validation. While the authors mention the use of GPT-4 to assist in data generation, human experts are still required to verify the correctness of the outputs. This introduces potential scalability issues as extensive human oversight could limit the practicality of the approach, especially in large-scale deployments. Additionally, the authors do not provide a qualitative or quantitative analysis of the labeling quality or reliability performed by the human experts. Evidence demonstrating the expertise and qualifications of these experts should be presented to support the validity of the labeling process.

2. **Insufficient Theoretical Justification for the Five-element Formulation**: The authors claim that the five-element formulation is a universal method for defining optimization problems, but they do not provide sufficient references or theoretical analysis to support this claim of 'universal'. Similarly, in the experimental results, a detailed and decomposed explanation of why the five-element formulation generalizes well and outperforms other models across different types of problems is lacking. Providing theoretical or empirical grounding beyond performance metrics would strengthen the argument for the effectiveness and universality of this approach.

3. **More comparative Analysis of the Self-correction Mechanism**: While the authors compare the performance of LLMOPT with and without the self-correction mechanism, it would be beneficial to include an analysis of the performance gains when self-correction is combined with baseline methods (e.g., GPT-4 directly). Additionally, a longitudinal comparison between the current self-correction design and other correction methods, such as manual debugging, could further elaborate on the advantages and limitations of the self-correction approach.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: I also have some minor concerns:

* **Dataset construction**: In L219, the authors mentioned they had 1763 'seed' problems. Did they label formulations for all 1763 problems? If so, this is a pretty big contribution if the authors choose to open-source their dataset.
* **Augmented data**: My understanding is that additional examples were introduced by mutating the original seed problems. Were these also formulated and labeled by human experts? How many examples in total (original + mutated) were used in training? 
* Can the authors describe the procedure of collecting the labelled formulations (if human experts were indeed used). This seems like a huge undertaking especially if they are required to write out the mathematical models and code.
* **Eq(4) and (5)**: The notation $\nu \sim \cdot$ was slightly confusing, since it is not a random variable, which is where this notation is mainly used.
* **Eq (5)**: Are $\lambda_D$ and $\lambda_U$ learnt parameters or hyperparameters?
* **Solver**: What solver did LLMOPT use? Is it `Pyomo` as shown in the method figure? Are the baselines evaluated with the same solvers?
* ps. this is just a suggestion, but the use of the word `alignment` is slightly misleading, the authors are not aligning the LLM to principles, but doing a different stage of finetuning with a different dataset.
* pps. another minor suggestion, but the authors should make more prominent that KTO is an existing method, I saw the citation, but it is slightly buried, and might convey the impression that KTO is an original contribution of this work.

I am going to start with a conservative rating, but open to revising if the authors address my concerns.
soundness: 2
strengths: The paper addresses an interesting and timely problem. Optimization modeling is a huge topic, and introducing techniques to automate (partially) this task will have significant impact.

To my knowledge, this is also one of the first works that builds/finetunes LLMs specifically for writing optimization models.

The empirical analysis is comprehensive (spanning many benchmarks) and the resulting performance gains are impressive.
summary: This paper proposes LLMOPT, which finetunes an LLM to formulate optimization problems (from problem description -> mathematical model/solver model). The finetuning is performed in two stages: SFT stage (where the LLM is finetuned using MLE on ground truth outputs), and KTO-based alignment (where the LLM is finetuned based on desirability labels on LLM generated outputs).
weaknesses: 1. **Novelty**: The authors mentioned ORLM, which similarly trains an LLM to do optimization modeling, but did not provide a direct comparison. I also read ORLM (not in the most detail), but it appears to do some data augmentation to train an LLM for model formulation. It seems the main difference is the alignment step (using KTO) and the self-reflection step, can the authors explain the novelty of their method compared to ORLM?

2. **KTO alignment**: There are a few comments on this:
* **Writing/clarity**: The writing in S3.3.2 is quite hard to follow, I had to read the original paper to understand what this part is doing. Importantly, Equation (3) is not correct, in the original DPO paper, the optimal reward function has an additional log partition function term. I did not check if this affected the rest of the formulation.
* **KTO dataset**: Based on my understanding: (1) the SFT step does not use the KTO dataset (which contains GPT4 generated responses, and desirability labels), and (2) the alignment step does not use the original dataset (which contains ground truth formulations). Is my understanding correct? If so, what is the motivation for not using the KTO dataset during SFT and original dataset in KTO alignment (where the ground truth are all labeled as 'desirable')? 
* **Purpose of KTO**: The authors state that `KTO loss function encourages the optimal model $\pi^*$ to produce completions that align more closely with expert-labeled data`. **Minor**: this can't be the optimal model, but the learned policy $\pi$? **Major**: This is exactly the same purpose as SFT, so we are back to the above point of why the KTO dataset is not used directly through SFT, it would likely be more stable and have lower variance compared to KTO updates.
* **Ablation study**: Can the authors more comprehensively ablate the importance of the KTO step? I carefully examined the results in Table 3 (ablation study)---which seems to indicate that `w/o KTO` on many benchmarks does not significantly improve SA. I would be interested to also see how LLMOPT performs when the KTO dataset is used directly during SFT.

3. **Questions about results**: 
* **AST**: Can the authors help me understand the big difference in average solution time (AST) plotted in Fig 4? I had a look at some examples in `NLP4LP`, these are very straightforward LP problems. As such, I am very surprised that LLMOPT and GPT4 have almost 2x difference in solution time. Given the simplicity of the problems in this dataset, (1) this difference is unlikely to be explained by clever reformulations that improve solution time, (2) and unlikely to be noise. 
* **Solution accuracy**: Can the authors elaborate on exactly how solution accuracy is calculated? Is this based on some test cases? My intuition is that it is extremely difficult to check the formulation directly, so how is SA computed and is this a robust evaluation method?
* **Performance by problem type**: In L422-L431, the authors claim their method is more general, which I understood as achieving better performance on more types of problems (e.g. LP, MILPs, QPs). To confirm this claim, I would like to see the performance by problem type (similar to the Figure 8 in App G).

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 2
questions: - Five-element formulation: to my knowledge, OPTIMUS [1] also identifies components such as variables, constraints, parameters etc in the optimization description before they translate the optimization problem into code. Can the authors comment on the difference between the two modeling approaches?
- Line 222: “experts review the generated problems, removing those with unclear descriptions or infeasible solutions to ensure data diversity and quality.” Can the authors comment on what are the criteria for the experts to determine the descriptions are “unclear”? How long is the expert labeling process? Can experts make mistakes? Can the authors comment on whether there is anyway to consider the expert mistakes to further improve the learning performance?
- Line 276 equation (3): I find the description of KTO confusing. For example, what is the reference model pi_ref used by the authors? Also, what is the optimal model (is it the same thing as the learning model)?
- Table 3: Can the authors comment on what is the setup w/o KTO? Is there still a training / fine-tuning component? what is the alternative loss?

Additional Feedback:

- line 53: The authors provide Wrong citation for ORLM on page 1.
- line 249: “are correct labeled by experts” → “are correctly labeled by experts”

[1] AhmadiTeshnizi, Ali, Wenzhi Gao, and Madeleine Udell. "OptiMUS: Scalable Optimization Modeling with (MI) LP Solvers and Large Language Models." ICML (2024).
soundness: 3
strengths: - The authors did a good job benchmarking their method on a variety of benchmarks with many competitive baseline methods. The empirical evaluation seems to be comprehensive.
- To my knowledge, there has not been many works that aim to fine-tune the LLM for MILP modeling tasks, so the task is relatively novel.
- I appreciate the detailed discussion provided by the authors, which I think is valuable and can help guide the community thinking about future steps.
summary: This paper proposes to finetune LLMs to improve MILP modeling from natural languages. Specifically, from natural language description of a MILP problem, this paper proposes to formulate the MILP problem as a five element formulation, and then generate solver code from the  formulation. It uses data augmentation to expand the data set, and ask domain experts to label the data set to finetune the LLMs. Through extensive empirical study, it shows performance improvement from a variety of competitive baseline methods.
weaknesses: - The fine-tuning data collection requires expert manual labels, which limit the scalability and applicability of applying this method.
- Despite strength 2 mentioned above, the related work ORLM cited by the paper already took an initial step in fine-tuning LLMs for MILP modeling, and RLHF/DPO/KTO has been commonly used in LLM literature to finetune LLMs, so I’m a bit concerned whether the novelty of this work is sufficient, especially the work requires expert manual labors for constructing the fine-tuning data.
- Table 3: it seems like a majority of performance improvement is from self-correction instead of the five element components and KTO. If I understand correctly, the self-correction is prompting the LLM to correct any error it makes and has been used in previous papers such as [1], and it’s not related to the KTO fine-tuning pipeline. Given this, I’m a bit concerned about the contribution of the two main components (five elements and KTO) in this work.
- I find certain parts of the paper missing details and somewhat confusing (see my questions below).

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: Please refer to the weakness section.
soundness: 3
strengths: 1. The introduction of this manuscript is clear.

2. Some improvement over baseline results.
summary: This paper proposes a learning-based LLM for solving Optimization Problems. The experimental results demonstrate the effectiveness of the proposed LLMOPT compared with prompt-based methods.
weaknesses: 1. It is questionable whether LLMs are capable of solving large-scale optimization problems. If their applicability is limited to small-scale problems, such as the Traveling Salesman Problem with just 10 nodes, the significance of this method is unclear. A problem of this size can even be solved by the Hopfield network proposed in the 1980s. In fact, both traditional heuristics and more recent neural combinatorial optimization methods are capable of handling problems with thousands of nodes [1, 2].

2. In Abstract, the authors mention, "to prevent hallucinations in LLMs, such as sacrificing solving accuracy to avoid execution errors." However, I do not believe this phenomenon should be described as hallucination. Hallucination typically refers to instances where the output of an LLM appears reasonable but is, in fact, fabricated. In this case, reducing solving accuracy to prevent execution errors does not align with the conventional meaning of hallucination. Please provide a more precise explanation or reconsider this terminology.

3. As shown in Table 3, the self-correction mechanism plays a decisive role in the performance of LLMOPT. However, this mechanism is not originally proposed in this manuscript. As the authors describe on Page 6, "Inspired by Chen et al. (2024), to enhance optimization generalization, we implement self-correction to automatically analyze the output results and identify errors arising during the execution of the solver code." Furthermore, the prior study [3] has also employed this mechanism to improve the performance of LLMs in solving vehicle routing problems.

4. In Section 4, the authors do not report the training time of LLMOPT, making it unfair to directly compare it with prompt-based methods.

5. Please provide the code generated by the LLM to solve the optimization problem. I am doubtful that current LLMs are capable of generating code sophisticated enough to solve medium- or large-scale problems.

[1] Fu Luo, et al. Neural combinatorial optimization with heavy decoder: Toward large scale generalization. In Proceedings of the Advances in Neural Information Processing Systems, 2023.

[2] Huigen Ye, et al. GNN&GBDT-guided fast optimizing framework for large-scale integer programming. In Proceedings of International Conference on Machine Learning, pp. 39864–39878. 2023.

[3] Zhehui Huang, et al. Can Large Language Models Solve Robot Routing?. arXiv, 2024.

Meta-review: This paper introduces LLMOPT, which fine-tunes LLMs to enhance their ability to model optimization problems from natural language descriptions. Specifically, the fine-tuning process consists of two stages: (1) the supervised fine-tuning (SFT) stage, where the LLM is trained using MLE on ground truth outputs, and (2) the KTO-based alignment stage, where the LLM is fine-tuned with desirability labels applied to its generated outputs. LLMOPT employs a novel five-element formulation as a universal framework for defining diverse types of optimization problems, subsequently generating solver code based on this formulation. Experiments on six real-world datasets demonstrate LLMOPT's effectiveness in improving the accuracy and generalizability of LLMs in solving complex optimization problems.

Most reviewers agree that this paper makes meaningful contributions. During the rebuttal phase, the authors addressed the majority of the reviewers' concerns, and I suggest that the authors revise the manuscript accordingly in the final version.

In response to the ethical concerns about data labeling, the authors provided detailed explanations on the annotation process and the background of the annotators. Therefore, I believe further ethical review is unnecessary. Nevertheless, I strongly recommend that the authors provide the additional details (including aspects such as the methodology, the reliability, and the annotator compensation) in the final version.

Overall, I recommend accepting this paper.

Decision: Accept (Poster)