ICLR 2025

Published as a conference paper at ICLR 2025
LLMOPT: LEARNING TO DEFINE AND SOLVE GEN-
ERAL OPTIMIZATION PROBLEMS FROM SCRATCH
Caigao Jiang♥∗,
Xiang Shu♣∗,
Hong Qian♣†,
Xingyu Lu♥†,
Jun Zhou♥,
Aimin Zhou♣,
Yang Yu♦
♣East China Normal University, China
♥Ant Group, China
♦Nanjing University, China
shux@stu.ecnu.edu.cn,
{hqian,amzhou}@cs.ecnu.edu.cn,
{caigao.jcg,sing.lxy,jun.zhoujun}@antgroup.com,
yuy@nju.edu.cn
ABSTRACT
Optimization problems are prevalent across various scenarios. Formulating and
then solving optimization problems described by natural language often requires
highly specialized human expertise, which could block the widespread applica-
tion of optimization-based decision making. To automate problem formulation
and solving, leveraging large language models (LLMs) has emerged as a potential
way. However, this kind of approach suffers from the issue of optimization gener-
alization. Namely, the accuracy of most current LLM-based methods and the gen-
erality of optimization problem types that they can model are still limited. In this
paper, we propose a unified learning-based framework called LLMOPT to boost
optimization generalization. Starting from the natural language descriptions of op-
timization problems and a pre-trained LLM, LLMOPT constructs the introduced
five-element formulation as a universal model for learning to define diverse op-
timization problem types. Then, LLMOPT employs the multi-instruction tuning
to enhance both problem formalization and solver code generation accuracy and
generality. After that, to prevent hallucinations in LLMs, such as sacrificing solv-
ing accuracy to avoid execution errors, the model alignment and self-correction
mechanism are adopted in LLMOPT. We evaluate the optimization generalization
ability of LLMOPT and compared methods across six real-world datasets cov-
ering roughly 20 fields such as health, environment, energy and manufacturing,
etc. Extensive experiment results show that LLMOPT is able to model vario...