Title: StruDiCO: Structured Denoising Diffusion with Gradient-free Inference-stage Boosting for Memory and Time Efficient Combinatorial Optimization
Venue: NeurIPS 2025
Accepted: yes
Keywords: Neural Combinatorial Optimization

Generative Modeling
Abstract: Diffusion models have recently emerged as powerful neural solvers for combinatorial optimization (CO). However, existing approaches fail to reveal how variables are progressively determined during inference, making the final solution opaque until the last step. To address this limitation, we propose a structured denoising diffusion model, StruDiCO, which incrementally constructs solutions through step-wise variable selection. This is achieved via a variable-absorption noising model, wherein the forward process simulates gradual variable deactivation, converging to an empty solution, while the reverse process incrementally selects variables to reconstruct the final solution. This design induces structural continuity across intermediate states, enabling interpretable and trajectory-consistent partial solutions throughout inference. To further improve the reliability of reverse inference, we introduce a constrained consistency sampling strategy, which suppresses low-confidence variable selection at each step to stabilize the reverse process. Leveraging the structure-preserving reverse process, we further propose a lightweight, gradient-free, objective-aware refinement framework, which iteratively improves solution quality by applying structure-aware perturbations to the current solution, performing reverse inference through the constraint consistency model, and decoding with an objective-guided scoring scheme. Extensive experiments on two canonical CO tasks, the Traveling Salesman Problem (TSP) and Maximal Independent Set (MIS), show that StruDiCO outperforms state-of-the-art diffusion-based solvers, achieving up to $3.5\times$ faster inference, 70\% lower GPU memory usage, and significantly improved solution quality, with up to 37.7\% drop reduction on TSP and an average 38.1\% improvement on MIS.
The codes are publicly available at https://github.com/yuuuuwang/StruDiCO.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work focuses on designing diffusion-based solvers for combinatorial optimization (CO) problems. Compared to recent diffusion solvers, its main innovation lies in a novel structure-erasing noising strategy: instead of adding discrete noise directly to label data during the forward process, the model randomly masks a subset of edges in the solution graph. During the reverse denoising process, the model starts from an empty graph and progressively predicts which edges should be recovered to reconstruct a complete solution.\n\nBuilding on this structured modeling approach, the authors further introduce a gradient-free post-optimization strategy, which repeatedly masks parts of the current solution and refines it through a guided search process using a custom-designed scoring function (confidence divided by task objective). The authors claim their method achieves state-of-the-art (SOTA) performance in both solution quality and inference speed.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The reviewer appreciates the idea of structurally designing the diffusion process. While diffusion models have recently shown strong potential as general-purpose function approximators in CO, prior diffusion solvers typically add discrete noise directly to the initial solution, which leads to a lack of interpretability in both the noising and denoising stages, limiting the integration of search-based enhancements. The structured design proposed here alleviates this issue.\n2. The proposed solver outperforms recent neural CO solvers in both solution quality and inference speed, validating the effectiveness of the structured solver design tailored for combinatorial optimization.\n3. Compared to gradient-based search strategies, the gradient-free post-optimization approach introduced here is more convenient to implement and apply, highlighting StruDiCO's practical advantage in broader applications.\n\nWeaknesses:\n1. The reviewer is curious about the contribution of the two key diffusion design components, Variable-Absorption and Constrained Consistency Sampling, to the overall performance. Conducting a thorough ablation study would help clarify their individual effects.\n2. Whether the inference cost of StruDiCO increases significantly with problem size, such as TSP-10000. While this does not affect the rating, clarifying this point would be helpful.\n\nQUESTIONS:\nSome recent diffusion-based solvers have been overlooked in the comparison, such as DISCO: Efficient Diffusion Solver for Large-Scale Combinatorial Optimization Problems, which also explores improving both solution quality and inference speed for CO problems. A technical discussion of the differences from these works, along with public performance comparisons, would further strengthen the completeness and positioning of this work.\n\nLIMITATIONS:\nThe reviewer finds the idea of a structured denoising process compelling and is generally supportive of the paper. Ablation studies should be included to clarify the contributions of key components. The current score may be raised depending on the authors’ responses in the rebuttal phase.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a structured discrete diffusion framework, StruDiCO, tailored for combinatorial optimization (CO) problems. It employs Variable-Absorption as the forward noising strategy during training and Constrained Consistency Sampling for reverse inference during testing. Leveraging the structural consistency throughout the inference process, the authors further introduce a gradient-free refinement method to enhance solution quality. While the paper presents a novel and promising approach, the reviewer has some concerns regarding the completeness of the experimental analysis and the clarity of the objective-aware decoding mechanism.\n\nSTRENGTHS AND WEAKNESSES:\n1. The paper proposes a structured diffusion solver with a novel variable-absorption mechanism. It further integrates a gradient-free local refinement strategy that improves performance over recent advances while offering a faster inference advantage.\n2. Compared to gradient-based methods, the proposed gradient-free refinement appears easier to implement and more applicable in practice. This increases the potential applicability of StruDiCO.\n3. The paper provides clear methodological details and supports the algorithmic design with theoretical analysis.\n\nQUESTIONS:\n1. The proposed Variable-Absorption Diffusion and Constrained Consistency Sampling seem to be the core innovations. However, the experimental section lacks clear ablation studies isolating their contributions. This makes it difficult to assess which component contributes most to the performance gains.\n2. The decoding strategy is guided by a multiplicative score between model confidence and task-specific penalties. This formulation seems simplistic and may bias the search toward either confidence or task cost. The authors are encouraged to elaborate on how this trade-off is balanced.\n3. The reviewer is surprised that the gradient-free refinement achieves significant quality improvements in only 3–5 steps. Typically, gradient-free methods (e.g., 2-opt) require more iterations to converge. The authors should provide a more detailed explanation or empirical evidence to justify this rapid convergence.\n4. Gradient-free methods often scale poorly to large problem instances, which raises concerns from the reviewer about the applicability of StruDiCO in such settings. The authors are encouraged to compare against recent CO solvers. In particular, the reviewer notes that recent work has explored the use of diffusion solvers for tackling large-scale combinatorial optimization problems—such as ‘DISCO: Efficient Diffusion Solver for Large-Scale Combinatorial Optimization Problems’. Comparing StruDiCO with these approaches would further strengthen the contribution of this paper.\n5. Given the fast evolution of recent combinatorial optimization research, the reproducibility of results is crucial for future comparison and follow-up work. The reviewer recommends that the authors consider releasing their codebase to strengthen the impact and reproducibility of their contributions.\n\nLIMITATIONS:\nThis paper provides an insightful approach to designing diffusion solvers for combinatorial optimization. It outperforms recent methods in terms of both solution quality and inference efficiency. However, the reviewer believes that additional ablation studies and clarifications regarding the decoding process would substantially improve the clarity and credibility of the work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-28"}

comments: SUMMARY:
This paper proposes StruDiCO, a structured denoising framework for combinatorial optimization (CO). Its two main novelties are:
- A structured noising process that enforces monotonicity in the forward and reverse directions, aiming to make the denoising path more efficient.
- A gradient-free decoding approach, with no dependency on the gradients from objectives.
It claims significant improvements in speed (3.5x) and memory saving (70%), and improved solution quality as well

STRENGTHS AND WEAKNESSES:
Strengths:
- The gradient-free refinement framework with eliminated back-propagation is a practical contribution
- The paper proposes a threshold-based constrained sampling strategy with theoretical grounding

Weaknesses:
- Marginal empirical improvements: while the method is cleanly designed, its gains over strong baselines are often small or inconsistent, making it difficult to justify the additional complexity of introducing the structured process.
- Gradient-free approach is not critical in tested examples: the tested tasks TSP and MIS do not present known issues with gradient use, so it’s unclear when this feature would be critical.
- The results are based on TSP and MIS which are two common tasks in CO. More diverse tasks would show broader applicability of the proposed method

QUESTIONS:
- The improvements in TSP and MIS looks minimal. Can you elaborate on why the performance gap is small, and under what settings the proposed method provides more substantial benefits?
- Can you provide a more detailed ablation that showcase the contribution of the gradient-free refinement? How much does each component in StruDiCO contribute to the final performance improvement?
- Could StruDiCO be generalized well to more CO problems?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces StruDiCO, a structured denoising diffusion framework designed to solve combinatorial optimization (CO) problems efficiently and interpretably. StruDiCO uses Structured Forward Diffusion to ensure that every intermediate state remains a structurally valid partial solution. This “structure-preserving” approach allows the model to maintain interpretability and stability across the denoising trajectory. In the reverse process, StruDiCO selectively includes variables based on a confidence threshold, avoiding low-confidence predictions. In addition, StruDiCO uses an objective-aware refinement framework that efficiently incorporates\n task objectives without requiring backward gradient computation. Empirical results on standard benchmarks - including the Traveling Salesman Problem (TSP) and Maximum Independent Set (MIS) - demonstrate that StruDiCO outperforms prior diffusion models in both solution quality and inference speed, while using significantly less memory.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The paper proposes a new solution for CO to bridge the gap between step-wise interpretability and global distribution modeling. Their solution, StruDiCO introduces a variable absorption noising process that gradually deactivates variables during the forward diffusion stage. Unlike prior methods (e.g., T2T, DIFUSCO) that apply random or uniform noise, this approach ensures that each intermediate state remains a structurally valid partial solution. In the reverse process, StruDiCO only selects variables whose predicted confidence exceeds a set threshold (δ), discarding low-confidence choices. \n\n- StruDiCO achieves a significantly faster inference and lower memory usage than prior diffusion-based models.\n\nWeaknesses:\n\n- It lacks a valid explanation on why their method is empirically strong compared to other diffusion-based methods\n\nThe refinement step in StruDiCO is based on heuristics. It is unclear how their method is efficient for the new CO problem\n\n- Like other diffusion-based methods, their solution needs a substantial number of labeled optimal or near-optimal solutions.\n\nQUESTIONS:\n- The authors can provide a more detailed explanation of why their method outperforms T2T and DIFUSCO in terms of accuracy.\n\n- How much labeled data is enough for your method?\n\nLIMITATIONS:\nYes, the paper addressed the limitations at Appendix G", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-08"}