Title: FSNet: Feasibility-Seeking Neural Network for Constrained Optimization with Guarantees
Venue: NeurIPS 2025
Accepted: yes
Keywords: deep learning

optimization

constraint enforcement
Abstract: Efficiently solving constrained optimization problems is crucial for numerous real-world applications, yet traditional solvers are often computationally prohibitive for real-time use. Machine learning-based approaches have emerged as a promising alternative to provide approximate solutions at faster speeds, but they struggle to strictly enforce constraints, leading to infeasible solutions in practice. To address this, we propose the Feasibility-Seeking Neural Network (FSNet), which integrates a feasibility-seeking step directly into its solution procedure to ensure constraint satisfaction. This feasibility-seeking step solves an unconstrained optimization problem that minimizes constraint violations in a differentiable manner, enabling end-to-end training and providing guarantees on feasibility and convergence. Our experiments across a range of different optimization problems, including both smooth/nonsmooth and convex/nonconvex problems, demonstrate that FSNet can provide feasible solutions with solution quality comparable to (or in some cases better than) traditional solvers, at significantly faster speeds.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces FSNet, a novel framework that integrates feasibility-seeking procedures into neural network pipelines to solve constrained optimization problems efficiently and with theoretical guarantees. Unlike traditional neural solvers or NN-based surrogate models that often produce infeasible solutions, FSNet ensures constraint satisfaction by solving a differentiable, unconstrained optimization subproblem that minimizes constraint violations. This feasibility-seeking step is integrated into both the training and inference stages. The authors provide theoretical convergence guarantees under smoothness and PL conditions and demonstrate that FSNet achieves near-zero constraint violations with competitive or superior objective values, while being significantly faster than traditional solvers across convex, nonconvex, and nonsmooth problem settings.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper proposes FSNet, a neural network framework that integrates a feasibility-seeking step for solving constrained optimization problems. The key innovation lies in formulating an unconstrained, differentiable subproblem that enforces feasibility, enabling end-to-end training while guaranteeing convergence under certain assumptions. The method performs well empirically across a variety of convex, nonconvex, and nonsmooth problems, achieving near-zero constraint violation and competitive or superior objective values, often with large runtime improvements over traditional solvers.\n\nHowever, the presentation of the problem setup is unclear early in the paper. Equations (1) and (2), which are central to the formulation, are introduced only later, making it difficult to understand the context of the framework during the initial discussion. Furthermore, the theoretical analysis relies on strong assumptions such as the PL condition and boundedness of derivatives. These conditions are difficult to justify in the context of neural networks and general nonconvex optimization, especially given that even strong convexity does not necessarily imply PL. The connection between the general applications discussed in the introduction and the formulation in (1) also lacks clarity. Additionally, the assumption that the input distribution is finite (x ~ D) may be restrictive in practice, and it is unclear how the method would perform or be affected in infinite or continuous distributions. Finally, the scalability of FSNet with respect to problem size and dataset cardinality is not analyzed theoretically or validated through large-scale experiments.\n\nQUESTIONS:\n1. The introduction lacks clarity on the specific problem the paper addresses. It would help to present equations (1) and (2) earlier. The high-level idea in lines 29–36 is difficult to follow without knowing the underlying problem and assumptions.\n\n2. The feasibility and convergence guarantees are derived under the PL condition (line 43 and Assumption 1). However, how can the loss landscape induced by a neural network satisfy the PL condition? This assumption seems strong, particularly since PL does not generally hold even for some strongly convex functions.\n\n3. In line 111, it is stated that x represents parameters. It is unclear whether the optimization problems mentioned earlier (e.g., in energy systems, robotics) conform to the structure in equation (1). Clarifying this connection would be helpful.\n\n4. The assumption that x \\~ D is drawn from a finite distribution is strong. What happens if D is continuous or infinite? How would this affect the theoretical results and practical performance?\n\n5. Theorem 3 mentions convergence of a sequence to limit points. However, does the convergence of the underlying optimization procedure itself hold? Under what conditions?\n\n6. How does the scalability of FSNet depend on the number of variables, constraints, and the size of distribution D (x~D)? It would be valuable to analyze this both theoretically and through experiments on larger-scale problems.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the Feasibility-Seeking-Integrated Neural Network (FSNet) is proposed to solve optimization problems with complex constraints. Core idea of this method is to introduce a differentiable feasibility search step based on neural network predictions, ensuring that the output solutions satisfy the constraints while enabling end-to-end training. This design not only provides theoretical guarantees of convergence (with exponential convergence under certain conditions), but also demonstrates excellent empirical performance across various types of problems (convex/non-convex, smooth/non-smooth).\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. Theoretical Rigor and Clarity: A detailed convergence analysis is provided, including exponential convergence for feasibility search and guarantees for end-to-end training, which enhances the credibility of the approach.\n\n2. Sufficient Empirical Evidence: Extensive experiments have been conducted on a variety of complex problems, comparing the performance with traditional solvers as well as other learning methods, and the results are convincing.\n\n3. Good Presentation: The paper is well-written, main body of the paper and the level of details in the appendix are appropriate.\n\n**Weaknesses**\n1. Assumption Dependency for Non-Convex Problems: Current convergence guarantees hold under certain assumptions (e.g., L-smoothness, PL condition). However, in practical applications of non-convex problems, these conditions might difficult to ensure. It is suggested to clearly define the scope of applicability of these conditions in the discussion (or limitation) and to look forward to future relaxation schemes.\n\n2. Algorithm Complexity and Practicality: Although the method performs excellently in several problems, the computational cost of the feasibility search step in high-dimensional, large-scale problems still needs attention. I suggest supplementing an analysis of its complexity and the discussion on its application potential in ultra-large-scale problems.\n\n3. Experimental Design and Comparison: The experiments primarily focused on smaller-scale problems (e.g., 100 decision variables). In the future, the scalability of the method could be validated in higher-dimensional, more complex real-world application scenarios.\n\nOverall, I consider this is an interesting paper. If the authors could address the concerns during the rebuttal process, I would be happy to increase my score.\n\nQUESTIONS:\nPlease refer to the **Weaknesses** part.\n\nLIMITATIONS:\nPlease refer to the **Weaknesses** part.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces FSNet, a Feasibility-Seeking Neural Network framework aimed at solving constrained optimization problems efficiently while providing guarantees on constraint satisfaction and convergence. The approach integrates a differentiable feasibility-seeking step into both the training and inference pipelines of standard neural networks, minimizing constraint violations via a separate unconstrained optimization problem, which is solved iteratively starting from the NN predictions. Theoretical analyses demonstrate guarantees on exponential convergence to feasible points under standard smoothness and PL conditions, and convergence of the NN parameter optimization under SGD.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1. FSNet is not limited to a specific problem class; it is applicable to both convex and nonconvex, smooth and nonsmooth constrained optimization problems. The feasibility-seeking module is conceptually general and practically flexible.\n2. FSNet achieves large speedups over solvers (up to 1000x in some cases), showing practical value for real-time or large-scale deployments.\n\n**Weaknesses:**\n\n1. The benchmarks are synthetic, following protocols from previous works, but there is a lack of real, application-driven case studies (e.g., energy systems, supply chains, robotics), which would better validate the practical impact and generalizability.\n2. The current method does not exploit specific structures of certain problem classes, which leads to FSNet being outperformed by highly optimized solvers. There is limited discussion on how to adapt FSNet to leverage such structure for further efficiency gains.\n3. The method’s performance and guarantees depend on tuning parameters such as penalty $\\rho$, convergence thresholds, and number of unrolled iterations in the feasibility-seeking step.\n\nQUESTIONS:\nPlease list up and carefully describe questions and suggestions for the authors, which should focus on key points (ideally around 3–5) that are actionable with clear guidance. Think of the things where a response from the author can change your opinion, clarify a confusion or address a limitation. You are strongly encouraged to state the clear criteria under which your evaluation score could increase or decrease. This can be very important for a productive rebuttal and discussion phase with the authors.\n\n1. Can the authors provide quantitative analysis or further comments on FSNet’s performance when the smoothness or PL conditions are significantly violated—are there clear failure cases, or does the method degrade gracefully?\n2. How sensitive are the results (constraint violation and optimality gap) to the penalty weight $\\rho$ and unrolled step count $K$ in both training and inference?\n3. Are there foreseeable bottlenecks or challenges when scaling FSNet to thousands or tens of thousands of decision variables? Is the feasibility-seeking optimization step a limiting factor?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
This paper proposes FSNet, a neural network framework that integrates a differentiable feasibility-seeking mechanism to solve constrained optimization problems efficiently. A feasibility-seeking procedure is performed via vanilla gradient update. Unlike traditional solvers that are computationally intensive or learning-based methods that often produce infeasible solutions, FSNet explicitly minimizes constraint violations during inference, ensuring solution feasibility. The approach supports end-to-end training and offers theoretical guarantees on convergence and feasibility.

STRENGTHS AND WEAKNESSES:
Strengths: The authors present their ideas clearly, with well-written content and detailed mathematical derivations.

Weaknesses: The proposed gradient step-based method is similar to optimizer-based constraint satisfiability layers, such as CvxpyLayers, OptNet, SATNet, GLinSAT, etc. Some of these layers use interior point method to solve the projection problem while some of them use gradient-based first-order methods. Actually, the proposed method FSNet can be regarded as a projection layer whose objective is to minimize the constraint violation and the resulting (convex/non-convex) optimization problem is solved by the gradient-based method. One potential gap between the theorem in this paper and the reality is that, it is usually hard to decide Lipschitz constant in the actual computation. Instead of everytime tuning the stepsize by hand, will it be better to automatically approximate the Lipschitz constant especially in convex cases? Besides, if there is some potential way to tackle the stepsize in non-convex cases, that will be much better.

QUESTIONS:
See weaknesses.

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03