LLaMoCo: Instruction Tuning of Large Language Models for Optimization Code Generation

Authors: Zeyuan Ma, Hongshu Guo, Jiacheng Chen, Zhiguang Cao, Yining Ma, Yue-Jiao Gong
Primary area: optimization
Keywords: Large Language Models, Instruction Tuning, Optimization Code Generation

Abstract:
Recent research on optimization using large language models (LLMs) typically involves either iterative next-step solution seeking or directly prompting LLMs to generate critical optimization codes. However, these methods often suffer from low computational efficiency, high sensitivity to prompt design, and a lack of domain-specific knowledge. We introduce LLaMoCo, the first instruction-tuning framework designed to adapt LLMs for solving optimization problems in a code-to-code manner. LLaMoCo features a comprehensive instruction set that includes code-style problem descriptions as input prompts and robust optimization codes from expert optimizers as target outputs. We then develop a novel two-phase learning strategy with a contrastive learning-based warm-up to enhance convergence during instruction tuning. Extensive experiments demonstrate that a CodeGen (350M) model tuned by our LLaMoCo yields a powerful domain-specific model for generating expert-level optimizers, achieving superior performance compared to GPT-4 Turbo and other competitors on both synthetic and realistic problem sets. The trained model and the usage instructions are available online.

Decision: reject
Reviewer scores: 6, 6, 6, 5

Reviews and discussion:
<thread>
creation date:
1729779107529

thread_type:
official_review

<thread_object>
creator: reviewer_efrn

<content>
summary:
This paper introduces LLaMoCo, a new framework for fine-tuning LLMs to solve optimization problems. The contributions of this paper are two-fold: (1) a novel fine-tuning dataset and (2) a new training warm-up strategy for training leveraging contrastive learning. Experimental evaluations demonstrate that LLaMoCo's models perform well on their held-out test set and realistic problems.

strengths:
1. The authors have developed and plan to release a novel dataset designed to teach language models to solve optimization problems. This represents a significant contribution to both researchers and practitioners.

2. The experimental results are compelling. I especially appreciate Table 3, where the proposed method strongly performs on realistic optimization problems (rather than toy problems).

weaknesses:
1. This paper lacks novelty. This paper primarily focuses on fine-tuning OSS LLMs for a specific domain. The main approach is straightforward from this perspective: the authors adjusted prompts (specifically, framing problem descriptions in Python or Latex) and developed a new dataset. Could authors emphasize the unique technical challenges associated with this domain?

2. The contrastive warm-up technique in this paper seems out of place. This technique does not appear to be specifically tailored to optimization problems. Could it be beneficial for fine-tuning in other domains as well? If not, what are the reasons? I would suggest separating this novel technique into a dedicated paper or clarifying how it suits the domain under discussion. The ablation study in Figure 4 is not very convincing, as it was tested with only a single configuration, making the results dependent on that specific setup.

questions:
1. Is the current dataset format truly optimal? For instance, could leveraging CoT enhance performance? Similarly, would implementing multi-turn iterative improvements for optimization code be a promising approach?

2. Could the proposed method be compared with previous non-LLM-based automatic algorithm selection approaches? Automatic algorithm selection for optimization problems is a well-established research area with a rich body of existing work.

3. Could the specific technical challenges unique to this optimization domain be highlighted? (see Weakness 1)

4. Would it be reasonable to separate the contrastive warm-up technique into a standalone paper or clarify that this technique is highly specialized for the domain under consideration? (see Weakness 2)
</content>
</thread_object>

</thread>
<thread>
creation date:
1730030173523

thread_type:
official_review

<thread_object>
creator: reviewer_ugm1

<content>
summary:
This paper presents LLaMoCo, a pioneering framework that maps optimization problem descriptions directly to expert-level optimization code through instruction tuning. By creating a comprehensive dataset of (problem, best-solver) pairs and using a two-phase training strategy, even a small model (350M parameters) can surpass GPT-4 in selecting and generating appropriate optimizers for both synthetic and realistic optimization tasks.

strengths:
1. 350M parameter model achieves 81.8% optimization performance vs GPT-4's 74.2% (without prompting) while using only 2.4K tokens vs 3.5K tokens.
2. Data pipeline converts 6000 problems to 32570 training pairs through systematic benchmarking of 23 optimizers across different configurations.

weaknesses:
1. Zero-shot evaluation tested on only 8 realistic problems, requiring more cases to validate the claims.
2. GPT-4 baseline with vector search not evaluated.
3. Grid search necessity on original problems is subtle, some parameters are hard to set without careful data observation, requiring further validation of selection appropriateness.

questions:
1. Further experiments needed to demonstrate the combined effect of SFT and alignment.
2. Grid search "best" performance criteria not clearly defined, benchmarking process lacks clear evaluation metrics for optimizer selection.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730031620291

thread_type:
official_review

<thread_object>
creator: reviewer_1qnj

<content>
summary:
This paper proposes a data generation and instruction tuning method for optimization-problem-solving LLMs. The authors conduct comprehensive experiments to demonstrate the optimization capabilities of the instruction-tuned LLMs and analyze the contribution of each component of the method.

strengths:
1. This paper introduces the first complete framework for training LLMs to solve optimization problems, including instruction-tuning dataset construction and detailed methods for training. The method is well-described and effective, making a significant contribution to the optimization community. 

2. The experiments demonstrate performance improvements on both synthetic and realistic problem sets. across different scales of LLMs, highlighting the generalization and effectiveness of LLaMoCo.

weaknesses:
1. Lack of sufficient novelty. Several key components of the method follow prior work [1-3], particularly the instruction-tuning approach (Section 3.2), which reduces its originality. Although this paper introduces the first instruction-tuning framework for optimization tasks, it primarily applies standard training techniques. The authors should emphasize their main innovations more clearly in the paper.

2. Writing. Figure 1 does not effectively highlight the main differences between LLaMoCo and previous methods, which is overly simplified. The authors should include more details of the method. There are typos in the caption of Figure 2 (wither -> either). The capitalization of “LaTeX” in the full paper is inconsistent.

[1]Problem definitions and evaluation criteria for the cec 2021 on single objective bound constrained numerical optimization.

[2]Unixcoder: Unified cross-modal pre-training for code representation.

[3]Exploring the limits of transfer learning with a unified text-to-text transformer.

questions:
1. What is the impact of dataset size on the training performance? Will the performance of the models continue to improve when using more data?

2. How to control the quality of the synthesized tasks? Can we ensure that unsolvable optimization problems or problems with only trivial solutions are not synthesized?

3. Why does the computational overhead of the trained models increase? (in Table 1)
</content>
</thread_object>

</thread>
<thread>
creation date:
1730698575093

thread_type:
official_review

<thread_object>
creator: reviewer_mqqr

<content>
summary:
This paper introduces LLaMoCo, a framework for fine-tuning general-purpose Large Language Models (LLMs) to generate optimization code through instruction tuning. The authors construct a specialized code-to-code instruction dataset tailored for optimization tasks. They enhance the training process with techniques such as contrastive warm-up, data augmentation via rephrasing, and balanced sampling. These methods are evaluated across three pre-trained models of different sizes (S, M, L), showing significant performance improvements. An ablation study further validates the effectiveness of the proposed techniques. Overall, the paper presents a promising approach to adapting LLMs for the specialized task of optimization code generation.

strengths:
1. Specialized Dataset Creation: The development of a tailored code-to-code instruction dataset is a significant contribution. It aligns the fine-tuning process closely with the target task and provides a valuable resource for future research in optimization code generation.
2. Innovative Training Enhancements: Implementing contrastive warm-up, data augmentation through rephrasing, and balanced sampling demonstrates a comprehensive strategy to improve model performance. These techniques address common challenges in model training, such as overfitting and data imbalance.
3. Comprehensive Evaluation and Analysis: Evaluating the framework across models of varying sizes offers insights into scalability and the impact of model complexity. The inclusion of an ablation study allows for a deeper understanding of how each training enhancement contributes to the overall performance.

weaknesses:
1. Unexpected Performance Across Model Sizes: Table 1, 2 and 3 show that the performance of LLaMoCo-S, LLaMoCo-M and LLaMoCo-L are very similar. The results also show that LLaMoCo-S sometimes outperforms its larger counterparts (LLaMoCo-M and LLaMoCo-L), despite having significantly fewer parameters. This is counterintuitive and raises concerns about potential inefficiencies in leveraging larger model’s increased capacity.

questions:
1. Investigate Model Performance Discrepancies: It would be beneficial to analyze why the smaller model occasionally outperforms larger ones. This could involve examining the training dynamics, learning rates, or potential overfitting issues in larger models. Providing insights or adjustments based on this analysis would strengthen the validity of the results.
2. Expand Baseline Comparisons: Could the authors add another baseline of ChatGPT o1-mini/o1-preview? Since o1-mini/o1-preview are reasoning/coding/math enhanced models. I expect it to perform better than ChatGPT 4o. These models are designed for coding tasks and would serve as competitive benchmarks to better evaluate LLaMoCo's performance. Incorporating such comparisons would contextualize LLaMoCo's performance within the broader landscape of code generation research. 
3. Enhance Robustness Evaluation: Assessing the models on out-of-distribution samples or real-world optimization problems beyond the dataset used for training could demonstrate the generalization capabilities and practical applicability of LLaMoCo, which could alleviate/address the concern of “Unexpected Performance Across Model Sizes”.
</content>
</thread_object>

</thread>
<thread>
creation date:
1733281356281

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Global Response

comment:
We would like to express our sincere gratitude for the time and effort the reviewers and AC have invested in reviewing our paper. First of all , we are so honored that LLaMoCo has been recognized as a **significant contribution to optimization community** (Reviewer #mqqR, #1QNj and #eFRN) and **code generation community** (Reviewer #mqqR, #1QNj). We are also pleased to see the reviewers have commended LLaMoCo for its **novel framework** (Reviewer #mqqR, #1QNj and #eFRN), **valuable dataset proposal** (all reviewers), **innovative training paradigm** (Reviewer #mqqR, #1QNj and #eFRN), **comprehensive evaluation** (Reviewer #mqqR and #eFRN) and **superior optimization performance** (all reviewers). 

In this global response, we primarily summarize common suggestions shared by the reviewers and provide an overview of the additional discussion with experimental results that address these suggestions, as follows.

---

**[Adding more baselines for comparison, Reviewer #mqqR, #ugM1 and #eFRN]**

1. Reviewer #mqqR suggests adding GPT-4o, o1-mini and o1-preview for a comprehensive comparison, which we have included them in **Section 4.4 and Table 4** (highlighted in blue) of the revised paper. 
2. Reviewer #ugM1 suggets adding GPT-4 Vector Search as an in-context learning baseline, which we have also included them in **Table 4** of the revised paper. These results further validate the superiority of LLaMoCo. 
3. Reviewer #eFRN suggests adding some automatic algorithm selection methods as baselines. We have clarified the fundamental differences between LLaMoCo and algorithm selection methods, noting that direct comparison would be unfair. This explanation has been added in **Section 4.1, lines 349-354** (highlighted in blue) of the revised paper.

**[Exploring LLaMoCo’s scalling law, Reviewer #mqqR and #1QNj]**

Reviewer #mqqR and #iQNj suggest exploring the impact of the model capacity and dataset size on LLaMoCo’s performance respectively. We hence provide an additional scaling law experiment on LLaMoCo in **Section 4.2, Table 2** and **Appendix E.3, Table 9**, where we find that both model capacity and dataset size play key roles in the final performance and the setting we adopt in the paper achieves best results. 

**[Evaluating LLaMoCo on real-world scenarios, Reviewer #mqqR and #ugM1]**

1. Reviewer #mqqR requests the performance of LLaMoCo on realistic problems. We have clarified that results for eight real-world problems from diverse domains are already included in **Section 4.2, Table 3** of the original paper.
2. Reviewer #ugM1 suggests adding more realistic problem instances to further validate the generalization ability of LLaMoCo. Following the suggestion, we have included a comprehensive real-world problem collection of 57 engineering problems and update the results in **Appendix E.2, Table 8** (highlighted in blue) of the revised paper, which provide a clear evidence of LLaMoCo’s superior generalization performance. 

**[Highlighting novelties of LLaMoCo, Reviewer #1QNj and #eFRN]**

Following the reviewers’ suggestions, we have discussed the challenges of fine-tuning general LLMs for optimization domain and the corresponding novelties of LLaMoCo to address these challenges. We have added this dicussion into our revised paper (Section 2, lines 125-128, 142-145, 169-173; Section 3, 181-187; all highlighted in blue) to highlight our novel methodology.

**[Validating the ablation robustness, Reviewer #ugM1 and #eFRN]**

Reviewer #ugM1 and #eFRN suggest adding the ablation results on the proposed contrastive warm-up strategy on all combinations of the foundation LLMs and training problem sets. Following the suggestion, we have added these results in **Appendix E.5, Figure 6** of the revised paper, where the contrastive warm-up strategy consistently improves the instruction-tuning of LLaMoCo under various settings. 

---

We hope the above summary of our discussion with the reviewers could provide convenience for reviewers and AC to grasp all focused issues and all efforts we have made to address them.
</content>
</thread_object>

</thread>


Metareview:
metareview:
### Summary of Claims and Findings
The paper introduces LLaMoCo, a novel instruction-tuning framework for adapting large language models (LLMs) to generate optimization code directly from problem descriptions in Python or LaTeX. The framework features a curated dataset of optimization problems paired with expert-level solutions and a two-phase training strategy that includes a contrastive warm-up phase to align problem-solution representations. Experimental results demonstrate that LLaMoCo significantly outperforms GPT-4 Turbo and other baselines in optimization performance across synthetic and realistic tasks.

### Strengths
1. **Framework Innovation**: LLaMoCo offers a structured approach to instruction tuning in the optimization domain, addressing challenges such as problem representation, model alignment, and data imbalance.
2. **Comprehensive Evaluation**: The paper demonstrates strong results on a diverse set of synthetic and real-world problems, coupled with scaling law analysis and ablation studies.
3. **Practical Impact**: The model's ability to generate efficient and effective optimization code has broad applications in engineering and optimization research.

### Weaknesses
1. **Perceived Incrementality**: Reviewers noted that many components (e.g., instruction tuning, dataset construction, contrastive learning) leverage existing techniques. Despite the authors’ rebuttal, the domain-specific nature of the contributions remains under-emphasized.
2. **Limited Demonstration of Contrastive Warm-Up's Necessity**: While the warm-up strategy appears effective, its justification as domain-specific was unconvincing to some reviewers, with suggestions that it could apply broadly across domains.
3. **Modest Scaling Gains**: Performance improvements from smaller to larger models appeared incremental, raising questions about the optimization knowledge effectively learned.

### Decision
While LLaMoCo represents a commendable effort to adapt LLMs for optimization code generation, the core novelty remains a concern. The primary contributions are seen as applications of existing methods rather than groundbreaking domain-specific innovations. Combined with lingering skepticism about the necessity and specialization of the proposed techniques, this submission falls marginally below the acceptance threshold.