Title: Enforcing convex constraints in Graph Neural Networks
Venue: NeurIPS 2025
Accepted: yes
Keywords: Graph Neural Networks

Constrained optimisation

Dykstra's projection algorithm
Abstract: Many machine learning applications require outputs that satisfy complex, dynamic constraints. This task is particularly challenging in Graph Neural Network models due to the variable output sizes of graph-structured data. In this paper, we introduce ProjNet, a Graph Neural Network framework which satisfies input-dependant constraints. ProjNet combines a sparse vector clipping method with the Component-Averaged Dykstra (CAD) algorithm, an iterative scheme for solving the best-approximation problem. We establish a convergence result for CAD and develop a GPU-accelerated implementation capable of handling large-scale inputs efficiently. To enable end-to-end training, we introduce a surrogate gradient for CAD that is both computationally efficient and better suited for optimization than the exact gradient. We validate ProjNet on four classes of constrained optimisation problems: linear programming, two classes of non-convex quadratic programs, and radio transmit power optimization, demonstrating its effectiveness across diverse problem settings.


Peer reviews:
comments: SUMMARY:
This work proposed a projection and search framework, using CAD and SVC. The architecture first ouputs an unconstrained output, then projects it to feasible region, and performs a search in the feaible region. The authors provided convergence analysis, designed a surrogate gradient for the CAD and exhibited promising experiment results.

STRENGTHS AND WEAKNESSES:
Strength:
- The paper is well motivated. Even readers with little background knowledge can understand the research direction and target.
- The paper is well written.
- The project then search idea is interesting and clear, and seems easily implementable in practice.
- The practical consideration for CAD convergence and SVC step size is nice and very convincing. 
- The experiment results are promising.

Weakness:
- Some implementation details are unclear, see questions below
- There is a concurrent work https://arxiv.org/abs/2502.02446, which also uses projection and search method for feasible solutions. Maybe consider citing it and add comparisons or discussions to their method.

QUESTIONS:
- Is it a supervised learning loss to predict the objective value?
- How is the SVC part trained? I mean how is the loss designed?
- For LPs, the numbers in the table are ratio of reaching optimal value? But for continuous optimization, how can you reach optimality? There must be some numerical error. 
- What is the runtime bottleneck of the ProjNet? The CAD or MPNN or what?
- Each CAD step calculation seems cheap, yes, but if it takes a lot of steps then GPU cannot accelerate it.

LIMITATIONS:
The authors discussed the limitation in the paper.  
The frameworks applies to linear constraints, which is a limitation.
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces ProjNet, a GNN architecture designed to enforce input-dependent constraints. To implement this architecture, Sparse Vector Clipping and Component-Averaged Dykstra are proposed for refined solution qualities. A surrogate gradient for CAD enables end-to-end training without unrolling. The authors provide a theoretical convergence guarantee for CAD and implement it with GPU acceleration. ProjNet is evaluated on several synthetic datasets involving both convex and non-convex problems\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. The paper provides a convergence guarantee for the CAD algorithm and introduces a well-motivated surrogate gradient for training.\n2. The GPU-accelerated implementation and sparse methods enable ProjNet to handle large-scale, input-dependent constrained problems efficiently.\n3. ProjNet achieves surprisingly good results on non-convex optimization tasks, showing its practical utility.\n\n**Weaknesses**\n1. The main results are established on randomly generated instances, which is as convincing as using public datasets, especially considering.\n2. The performance can vary significantly by adjusting the hyperparameter $c_h$, and not much details are provided.\n3. While the paper claims a GPU-accelerated implementation of the CAD algorithm, it does not explicitly report whether the experimental results are based on the GPU version. If so, comparing against CPU-only baselines like PDLP may be unfair, as PDLP also supports GPU acceleration. Clarifying this would improve the fairness and transparency of the comparisons.\n\nQUESTIONS:\nIn addition to the points raised in the weaknesses section, another concern is the omission of a potentially relevant baseline. The overall structure of ProjNet is to predict an initial solution, followed by post-processing to improve feasibility and optimality, which bears resemblance to [1] that follows a similar approach for integer programs.\nWhile [1] focuses on discrete optimization, its method can be adapted to linear problems by relaxing the integrality constraints. A brief empirical or conceptual comparison would strengthen the paper. Moreover, this raises an important question about the individual contributions of the CAD and SVC components: could the predicted solutions from ProjNet serve as warm starts for established solvers like PDLP, potentially achieving comparable performance with less architectural complexity?\n\n[1] Qingyu Han, et. al., A gnn-guided predict-and-search framework for mixed-integer linear programming. arXiv preprint arXiv:2302.05636, 2023.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-22"}

comments: SUMMARY:
The paper introduces ProjNet, a novel GNN framework for enforcing input-dependent convex constraints via a CAD algorithm and sparse vector clipping. A theoretical result on the convergence of the CAD algorithm is provided, and numerical comparisons with classical baselines are presented.

STRENGTHS AND WEAKNESSES:
__Strengths:__

1. This paper is in general well-written and is easy to follow. The main ideas are smooth and reasonable.

2. A theoretical result on the convergence of the CAD algorithm is provided.

3. Surrogate gradients are proposed as a computationally efficient alternative to exact gradients, with some theoretical properties.

4. The numerical results look nice compared to classical baselines.

__Weaknesses:__

1. I do not think that projecting the neural network output onto a feasible region is a completely novel idea. There have been some existing works, for example: Wang, R., Zhang, Y., Guo, Z., Chen, T., Yang, X., & Yan, J. (2023, July). LinSATNet: the positive linear satisfiability neural networks. In International Conference on Machine Learning (pp. 36605-36625). PMLR.

2. There are only numerical comparisons with classical baselines. Given the claimed scope of this paper, I would expect to see how this approach can contribute to the "learning-to-optimize" community. In particular, I think the authors should include comparisons with existing GNN methods for optimization. This might be a major flaw.

QUESTIONS:
No.

LIMITATIONS:
The authors present limitations using numerical experiments.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-06-24

comments: SUMMARY:
The authors propose a new framework for learning GNNs that satisfy complicated convex constraints. The key idea of the framework is to develop and integrate two kinds of layers: 1) a GPU-friendly iterative algorithm, the Component-Averaged Dykstra (CAD) algorithm and 2) sparse vector clipping layers, with a base GNN. The authors demonstrate convergence results for CAD and propose a numerical scheme to compute a surrogate gradient of the CAD derivative. The method is validated on a range of convex and nonconvex constrained optimization problems.

STRENGTHS AND WEAKNESSES:
**Strengths**

The paper is well-written and the relevant works are sufficiently described. The algorithm is novel and comprehensively analyzed- including runtime and convergence properties. Numerically efficient variants of the framework are introduced and evaluated empirically. Experimentally, the framework exhibits promising results on several benchmark problems.

**Weaknesses**

The comparative experiments are lacking. Although some classical methods are compared to, no comparison is made to existing ML-based optimization methodologies, e.g. those outlined in the related work section (differentiable optimization layers or other iterative methods). The comparison is primarily based on ablative experiments for the various components of ProjNet. I think it would be interesting and important to demonstrate ProjNet’s performance in comparison to these kinds of algorithms.

QUESTIONS:
How does your method compare to existing ml / GNN-based optimization frameworks?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-02