Title: $\texttt{STRCMP}$: Integrating Graph Structural Priors with Language Models for Combinatorial Optimization
Venue: NeurIPS 2025
Accepted: yes
Keywords: Learning to Optimize

LLM Code Generation

Combinatorial Optimization
Abstract: Combinatorial optimization (CO) problems, central to operation research and theoretical computer science, present significant computational challenges due to their $\mathcal{NP}$-hard nature. While large language models (LLMs) have emerged as promising tools for CO—either by directly generating solutions or synthesizing solver-specific codes—existing approaches often $\textit{neglect critical structural priors inherent to CO problems}$, leading to suboptimality and iterative inefficiency. Inspired by human experts’ success in leveraging CO structures for algorithm design, we propose $\texttt{STRCMP}$, a novel structure-aware LLM-based algorithm discovery framework that systematically integrates structure priors to enhance solution quality and solving efficiency. Our framework combines a graph neural network (GNN) for extracting structural embeddings from CO instances with an LLM conditioned on these embeddings to identify high-performed algorithms in the form of solver-specific codes. This composite architecture ensures syntactic correctness, preserves problem topology, and aligns with natural language objectives, while an evolutionary refinement process iteratively optimizes generated algorithm. Extensive evaluations across Mixed Integer Linear Programming and Boolean Satisfiability problems, using nine benchmark datasets, demonstrate that our proposed $\texttt{STRCMP}$ outperforms five strong neural and LLM-based methods by a large margin, in terms of both solution optimality and computational efficiency. The code is publicly available in the repository: https://github.com/Y-Palver/L2O-STRCMP.


Peer reviews:
comments: SUMMARY:
This paper proposed a framework that combines GNN and LLM for CO problems. First, a GNN is pretrained to encode structural information of CO problems. Second, a composition model is trained so that LLM generates better code to solve the CO problem. The authored proved that a GNN prior does not hurt the performance, and exhibited promising empirical results.

STRENGTHS AND WEAKNESSES:
Strength:
- The paper is well motivated and well written.
- The background of current research is well explained. 
- The details of GNN and LLM implementations are clear, even to readers that have little background of LLMs. 

Weakness:
- The diagrams in experiments are good, but I still expect some tables with actual numbers. 
- Some empirical results are not consistently better, e.g. CNP solving time in table 5 and 6.
- The theory is a bit weak, but that is very minor point.

QUESTIONS:
- How does the natural language description of the CO problem look like?
- You train the GNN with classification loss, but is it justified that different problems subsume different structures? Some graph based CO problems can be similar, e.g. what if I want to model the maximum independent set and minimum vertex cover problems, they are complementary. 
- Is the solving code generated by the LLM guaranteed to be executable without bugs or syntax error?
- Could you explain more about the sup(P) in definition 1? It is not clear to me why it takes the form of that.

LIMITATIONS:
The limitations are discussed in appendix F, which already address my concerns as well.
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-04

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes to add structural priors for LLM-based algorithmic discovery to improve the performance of the generated code snippet in optimization solvers. Optimization problems are transformed into graphs based on their structures, and a GNN is adopted to encode the graph information into vectors, which are concatenated with the text embedding of the prompts. Experiments are conducted on SAT (bump_var function) and MILP (cut plane selection function).\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- The general idea of improving LLM with structural information is promising.\n- Good writing and sufficient details. The theoretical analysis provides some insights into the role of priors for LLM-based algorithmic discovery.\n- Experiments conducted on both SAT and MILP, two representative CO problems.\n\n## Weaknesses\nWhile the structure of an optimization problem is surely important, the specific method proposed in this paper is questionable. Generally speaking, algorithmic discovery should be problem-irrelevant. That is, LLMs are prompted to perform _general_ improvement on the code snippet, and an improved code snippet should work well on a wide variety of problem instances with different structural information. However, it seems that this paper proposes to fuse the structural information (embedding vectors) of _a specific problem instance_ with a prompt that asks for general improvement, which is not aligned. Consider the following simplified prompt: \n> \"I have the graph information of a Boolean formula $ q = (x₁ ∨ x₂) ∧ (¬x₁) ∧ (¬x₁ ∨ x₂) $ as (...). You are a SAT solver researcher, your goal is to improve bump_var for the SAT solver, ...\"\n\nFrom the perspective of a SAT solver researcher, I don't feel that the previous sentence informing graph information (i.e., the problem feature vector $ h_q $ in the paper) is super useful for me, especially considering that my aim is to improve bump_var in a general sense, rather than just for the specific problem q = (x₁ ∨ x₂) ∧ (¬x₁) ∧ (¬x₁ ∨ x₂). At least, the intuition behind fusing a specific $h_q$ with a general prompt is not immediately clear to me.\n\nTo investigate the issue, I looked into the theoretical analysis and experiential evaluation of the paper, but the issue remains unsolved:\n- For theoretical analysis, the paper proposes some very general theorems regarding \"performance-enhancing prior\". However, it remains unclear why the structural information $h_q$ of a specific problem $q$ could theoretically be a \"performance-enhancing prior\" for generating generally improved code snippets.\n- For experiential evaluation, the improvement of the proposed method is not very significant either. For example, for the four datasets in Table 1 and 5,  comparing STRCMP with AutoSAT, the performance on CNP and CGD is generally the same, while the performance on PRP is much slower. Considering that algorithmic discovery could be a very randomized process with great variance, the current result does not seem to be significant enough to show the effectiveness of $h_q$'s inclusion. It is also worth noting that the baseline approach (AutoSAT) does not adopt any post-training process. Is it possible that the performance gain comes from the post-training process rather than the inclusion of $h_q$?\n\n## Suggestions\nI feel that this paper would make more sense if the problem setting could be adjusted a bit. Instead of searching for a general improvement for a code snippet, targeting a problem-specific improvement (i.e., improvement w.r.t some prior information of the problem) seems to be more promising (the evaluation function should also be adjusted to be problem-specific). Given a problem instance $q$, an LLM could incorporate the structure of the problem $h_q$ by the proposed method, to generate a _problem-specific_ code snippet, which is \"tailored\" to solve the specific problem $q$ efficiently.\n\nQUESTIONS:\nSee \"weaknesses\".\n\nLIMITATIONS:\nSee \"weaknesses\".", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nSummary: \nThis works falls into the line of work of using LLMs to generate programs to solve combinatorial optimization models. The authors incorporate additional constraints / priors in the generation process by converting the input CO problem into a bipartite graph as is commonly done. This graph is passed through a GNN to generate a “structure embedding” which is also passed as input to the final LLM.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n1.\tThe work is well motivated. Adding additional constraints to the generation process to improve quality of generation seems like a good avenue to explore. \n2.\tPaper is well written and structured \n\nWeaknesses: \n1.\tResults are not convincing enough and a bit confusing in places. Across all results – Figs. 4, 5b, 8, 9, 10, 11 and Tables 5 and 6, the ablated variants of the STRCMP often outperform STRCMP. While the authors acknowledge this issue, I feel the paper is incomplete without an explanation as to why, or a way of picking which ablated variant to use in which case, especially given that in a lot of cases, the differences are quite drastic. \n2.\tAlong the same lines, results on MILP instances shown in Figure 10 are not very convincing with LLM4Solver performing identical to STRCMP in most cases.\n\nPlease also consider positioning this work (and comparing where adequate) with recent LLM-based Optimization Co-Pilots. These seem omitted in the current version. Naming only a few here and plz see references therein to better position the work\n- OptiMUS: Optimization Modeling Using MIP Solvers and large language models, A. AhmadiTeshnizi et. al. \n- Constraint modelling with LLMs using in-context learning, K Michailidis, CP 2024 \n- Ner4Opt: Named entity recognition for optimization, Kadioglu et. al. Constraints 2024 \n- Enhancing decision making through the integration of large language models and operations research optimization, Wasserkrug et. al. AAAI'25\n- Holy Grail 2.0: From Natural Language to Constraint Models, D. Tsouros et. al.\n\nQUESTIONS:\n1.\tHow much does the performance of STRCMP depend on the underlying LLM? It would be interesting to see how performance changes with changing LLMs. This would help give an understanding of how much the structural embedding from the GNN contributes to the final result.  \n\n2.\tPerozzi et al. [1] dive deeper into generating tokens from graphs and merging a GNN and LLM to improve downstream task performance. I would recommend giving their paper a read to look for potential avenues of improvement. \n\n[1] Perozzi, B., Fatemi, B., Zelle, D., Tsitsulin, A., Kazemi, M., Al-Rfou, R. and Halcrow, J., 2024. Let your graph do the talking: Encoding structured data for llms. arXiv preprint arXiv:2402.05862.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper provides a framework for solving combinatorial problems that integrates structure priors on the problems themselves to enhance solution quality and solving efficiency. This is achieved through combining a GNN to extract the structural embeddings from CO instances, which are then fed to a LLM that produces the solver-specific code.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n- STRCMP consistently matches or exceeds all baseline performance in terms of solution optimality and computational efficiency. It shows significant reductions in timeouts and solving time for SAT problems, and maintaining strong performance parity for MILP\n- The paper highlights that existing LLM-based approaches for CO problems \"often neglect critical structural priors\". STRCMP directly tackles this significant gap by explicitly integrating these structural priors.\n- The paper also provides a theoretical analysis based on information theory and multi-modal co-learning, showing how integrating priors does not lower the upper bound on model performance.\n\n**Weaknesses:**\n\n- The ablations results are counterintuitive. STRCMP does not uniformly outperform its ablations (SFT Only and DPO Only) across all benchmarks. While acknowledged as a limitation and an area for future research, it indicates an incomplete understanding or unresolved issue in the current training protocol, which slightly diminishes the overall quality of the \"full\" proposed solution.\n- The paper adopts a two-stage training procedure (GNN first, then a frozen GNN with LLM) due to \"significant optimization challenges\" with end-to-end joint training. This suggests a practical workaround rather than an ideal integrated solution. It might potentially limit the synergistic learning that could occur with true end-to-end training.\n- The interpretability part is not convincing. While code generation offers interpretability over black-box NCO methods, the complexity of generated code for highly intricate CO problems or novel heuristics might still pose interpretability challenges in practice. The generated code, even if syntactically correct, might not always be easily digestible or modifiable by human experts without significant effort.\n\nQUESTIONS:\n- This quote from Yang's analysis (lines 69-72) indeed points to a fundamental limitation of LLMs pre-trained primarily on text for structured domains. While fine-tuning is crucial, the initial pre-training distribution shapes the model's inherent priors. The STRCMP framework effectively addresses this by introducing structural priors via a GNN. Could you elaborate on how your \"multi-modal co-learning\" (line 223) specifically helps the LLM overcome these \"weaker priors for structured domains\"? Is it primarily the GNN providing the structured information that the LLM then learns to condition on, or is there a deeper, emergent understanding of structure fostered within the LLM itself due to this multi-modal input that goes beyond just adapting to the fine-tuning data?\n- The paper mentions that the full STRCMP model (with SFT + DPO) doesn't uniformly outperform its ablations, suggesting \"underlying conflicts within the post-training data distribution\". Could you elaborate on what these \"domain-specific distributional discrepancies\" might entail?\n- The paper has adopted a two-stage training due to \"significant optimization challenges\" with end-to-end joint training. Could you provide more specific details on these challenges? For instance, were they related to gradient propagation issues across the GNN-LLM boundary, computational resource limitations, or difficulties in aligning the learning objectives of the two very different modalities?\n- The paper claims superior interpretability due to generating solver-executable code. However, CO problems can be highly complex, and the generated code snippets (e.g., hybrid_diversity_selector in Appendix I) can be quite intricate. How does the interpretability of these generated code functions scale with the increasing complexity of the CO problem or the sophistication of the heuristics discovered? Is the generated code always easily understandable and modifiable by a human domain expert, or can it become opaque in its own right, albeit in a \"code\" format?\n\nLIMITATIONS:\nThe limitation of two stage process has been mentioned in the limitation. But this is also a major weakness of this paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}