Title: Large Language Models as End-to-end Combinatorial Optimization Solvers
Venue: NeurIPS 2025
Accepted: yes
Keywords: Large Language Model; Combinatorial Optimization; Reinforcement Learning
Abstract: Combinatorial optimization (CO) problems, central to decision-making scenarios like logistics and manufacturing, are traditionally solved using problem-specific algorithms requiring significant domain expertise. While large language models (LLMs) have shown promise in automating CO problem solving, existing approaches rely on intermediate steps such as code generation or solver invocation, limiting their generality and accessibility. This paper introduces a novel framework that empowers LLMs to serve as end-to-end CO solvers by directly mapping natural language problem descriptions to solutions. We propose a two-stage training strategy: supervised fine-tuning (SFT) imparts LLMs with solution construction patterns from domain-specific solvers, while a feasibility-and-optimality-aware reinforcement learning (FOARL) process explicitly mitigates constraint violations and refines solution quality. Evaluation across seven NP-hard CO problems shows that our method achieves a high feasibility rate and reduces the average optimality gap to 1.03–8.20% by tuning a 7B-parameter LLM, surpassing both general-purpose LLMs (e.g., GPT-4o), reasoning models (e.g., DeepSeek-R1), and domain-specific heuristics. Our method establishes a unified language-based pipeline for CO without extensive code execution or manual architectural adjustments for different problems, offering a general and language-driven alternative to traditional solver design while maintaining relative feasibility guarantees.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors leverage supervised fine-tuning (SFT) and reinforcement learning (RL) techniques to post-train a Qwen2.5-7B model for end-to-end combinatorial optimization (CO).\nProblem instances with best known solutions are converted into text descriptions using the TAI tool, and the Qwen model is fine-tuned to generate solutions directly from these textual inputs. The RL phase incorporates feasibility and optimality rewards within the GPRO framework. Empirical results demonstrate surprisingly strong performance compared to other LLMs and heuristic baselines.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- Extensive experiments and strong empirical performance\n- Clear and well-written paper, easy to follow.\n\n**Weaknesses**\n- The model with optimality gaps of 1.03-8.20% is not end-to-end to me. \n- Generalizability to OOD instances is unclear (no systematic evaluation).\n- Lack of interpretability—no insights into how the model derives predictions.\n\nQUESTIONS:\n1. Since the paper does not emphasize the LLM’s reasoning process, what explains its strong performance? Have you compared against supervised GNN-based methods to provide further context?\n\n2.  How well does the method generalize to unseen problem classes? Could the TAI tool be extended to other combinatorial optimization tasks?\n\n3. In Table 2, the label \"ours\" is ambiguous—does it refer to the SFT-only model, RL-finetuned version, or an ensemble?\n\nOther comments:\n\n1. Some key messages in the main results could be highlighted more clearly.\n2. In line 269, the authors should provide additional supporting evidence for their claim: \"While such approaches (reasoning models) may work for very small-scale instances (e.g., fewer than 10 nodes), it becomes impractical to reason clearly and effectively as instance size grows.\"\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper fine-tunes a Qwen-2.5 model to act as an *end-to-end* solver for seven standard NP-hard combinatorial-optimization (CO) tasks (TSP, OP, CVRP, …), using two stages: (1) supervised fine-tuning (SFT) on solutions produced by classical solvers and (2) a feasibility-/optimality-aware RL phase (FOARL). A Best-of-N decoder is applied at inference. Training relies onsynthetic instances per problem and achieves 1 – 8 % average optimality gaps while remaining slower than lightweight heuristics.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n* Extensive quantitative tables comparing against large general-purpose LLMs and several prompt-engineering baselines;\n\n**Weaknesses**\n\n* [Crucial]**No practical application scenario.**  I see no real-world application scenario that requires LLMs to solve CO problems. Why don't people work on domain-specific models/heuristics?\n\n* **Only one base LLM is actually trained.** The core results rely on Qwen-2.5-7B; the versatility check with a tiny 8 B LLaMA is relegated to an appendix. Generality across architectures and scales therefore remains unproven;\n\n\n* **Efficiency gap acknowledged by the authors.** Even after LoRA and BoN sampling, the method is “not yet on par with traditional lightweight heuristics”, undermining its value as a drop-in solver.;\n\n* **Marginal contribution of RL.** Table 1 shows RL changes feasibility but barely improves optimality; ablations suggest most quality comes from brute-force BoN decoding, not from the proposed FOARL.;\n\n* **Limited insight that SFT helps.** It is unsurprising that imitating expert solutions boosts an LLM; the paper does not illuminate *why* language modelling is preferable to standard neural CO architectures.\n\nQUESTIONS:\n1. What is the wall-clock latency of SFT+RL+BoN on problems with 1 000+ nodes, and how does that compare with OR-Tools?\n\n\n2. FOARL appears to give small gains; did the authors try simply weighting feasibility in the loss during SFT?\n\nLIMITATIONS:\nI think the biggest limitation is that there exists no real-world application scenario that pushes experts/practitioners to apply LLM to solve a CO problem. I don't see the benefit of motivation of the research. Please feel free to correct me if the claim above makes a mistake.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 1, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel framework that employs LLMs as end-to-end solvers for combinatorial optimization problems. The method involves a two-stage training process: (1) supervised fine-tuning to predict solutions, followed by (2) a reinforcement learning phase that is explicitly aware of feasibility and optimality to reduce constraint violations and improve performance. The approach demonstrates strong results across seven benchmark datasets, outperforming the considered baselines.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n1. The approach enables LLMs to solve COPs directly from natural language problem descriptions, eliminating the need for custom solvers or intermediate code-generation pipelines.\n\n2. The method achieves strong performance across seven COP datasets, compared to the considered baselines.\n\n3. It has a comprehensive experimental scope, including ablations, sensitivity, and generalization.\n\n\n\n\n\n**Weaknesses:**\n\n1. Unfair comparison: It appears that the LLM-based baselines do not utilize prior information from the generated COP datasets, whereas the proposed method does (by fine-tuning). The authors may clarify on the fairness and rationale of this comparison.\n\n2. Limited baselines: Among LLM-based methods, only OPRO is included as a baseline, despite the existence of other relevant approaches treating LLMs as end-to-end solvers (lines 87-90). The authors should justify the exclusion of these methods. Additionally, for domain-specific baselines, it is unclear why state-of-the-art solvers like Gurobi and LKH3 (which are used for label generation) are not included for comparison.\n\nQUESTIONS:\n1. For fair comparison, should all LLM-based methods enforce equal sampling with the same N? It is better to explicitly report how many samples are generated per instance, whether BoN is used, and if multi-round prompting (e.g., OPRO, PHP) is applied.\n2. In Appendix E.7, do the baseline methods leverage any prior information from the generated seven COP datasets?\n3. Only 5 instances from TSPLIB are used for out-of-distribution experiments in Appendix E.7. These are relatively small-scale, while the paper claims to address scalability up to 100 nodes. Should all TSPLIB instances that meet certain criteria be considered for testing, instead of using only selected examples?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a two-stage fine-tuning framework that turns a 7B-parameter LLM into an end-to-end solver for combinatorial optimization problems. First, SFT teaches solution patterns from solver-generated examples. Second, a feasibility-and-optimality-aware reinforcement learning (FOARL) stage uses a combined constraint-satisfaction and optimality reward to correct infeasible outputs. At test time, Best-of-N sampling further improves solution quality. Empirically, the method achieves near-100% feasibility and optimality gaps of 1–8% across seven CO problems on small to medium sized instances, outperforming both general-purpose, reasoning LLMs and previous LLM as optimizer works.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\t\n- First to directly map natural-language CO descriptions to solutions via SFT + RL without intermediate code execution.\n- Solid integration of SFT and RL with a clear mathematical formulation. Introduces FOARL to explicitly enforce constraints in language outputs.\n- Thorough experiments: seven diverse CO tasks, 3 different problem sizes, BoN sampling study, ablations on heuristic features, and runtime breakdown.\t\n- Methodology clearly described; prompt templates, reward definitions, LoRA settings, and problem-instance conversions are fully documented (Sections 4, Appendices A-C).\n- Demonstrates that a relatively small LLM can match or exceed much larger LLMs on CO tasks in feasibility and solution quality.\n\n\nWeakness: \n- Claims of outperforming “problem-specific heuristics” rest on comparisons to simple methods (NN, FI, PS, ACO) rather than modern state-of-the-art metaheuristics (e.g., HGS, ALNS). No justification is given for excluding these stronger baselines.\n-The description of “heuristic features” (e.g., top-k nearest neighbors) is brief and lacks detail on selection rationale or how different feature choices affect performance and generalization. \n- The paper does not benchmark against exact solvers (e.g., Gurobi, CPLEX) on any task, leaving it unclear when one would prefer this approach over off-the-shelf MIP solvers.\n- For the problem sizes addressed, the computational efficiency might still be lower than both classical heuristics and MIP solvers, raising doubts about practical value. Further experiments are needed.\n\nQUESTIONS:\n1- You compare primarily to NN, FI, sweep, ACO, etc. Why not include modern VRP solvers such as HGS or ALNS, which are widely regarded as state-of-the-art for CVRP? (Similarly for other problems.) Without them, claims of “outperforming problem-specific heuristics” are overstated.\n\n2- Appendix E.1 reports LLM inference times, but you do not compare to MIP solvers like Gurobi/CPLEX on the same tasks. For your instance sizes the exact solutions should be tractable (at least given a target gap@k). Can you provide both solution quality and runtime from a standard solver to justify the benefit of your pipeline?\n\n3- The heuristic features are expert-crafted and problem-specific. How sensitive is your method to the choice or number of such features? How do they affect generalization across problem types if they must be designed by an expert? Could heuristic-feature selection itself be treated as parameter tuning?\n\n4- FOARL improves feasibility, but how does the number or type of constraints affect infeasibility rates? For instance, if you add more constraints (such as time windows or precedence constraints in VRP), does performance degrade gracefully, or not?\n\n5- You evaluate on small/medium/large graphs (Table 2) but, if I understand correctly, always train and test on the same scale. Have you tried training on one size or distribution (e.g., uniform coordinates) and testing on another (e.g., clustered or larger instances) to assess out-of-distribution robustness?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}