Enforcing Hard Linear Constraints in Deep Learning Models with Decision Rules
Authors: Gonzalo E. Constante Flores
Hao Chen
Can Li
Venue: NeurIPS 2025 poster ()
Keywords: constrained deep learning
physics-informed neural networks
safe machine learning
Primary area: optimization
TL;DR: This paper introduces a model-agnostic neural network framework that guarantees satisfaction of input-dependent linear constraints using decision rules.

Abstract: Deep learning models are increasingly deployed in safety-critical tasks where predictions must satisfy hard constraints, such as physical laws, fairness requirements, or safety limits. However, standard architectures lack built-in mechanisms to enforce such constraints, and existing approaches based on regularization or projection are often limited to simple constraints, computationally expensive, or lack feasibility guarantees. This paper proposes a model-agnostic framework for enforcing input-dependent linear equality and inequality constraints on neural network outputs. The architecture combines a task network trained for prediction accuracy with a safe network trained using decision rules from the stochastic and robust optimization literature to ensure feasibility across the entire input space. The final prediction is a convex combination of the two subnetworks, guaranteeing constraint satisfaction during both training and inference without iterative procedures or runtime optimization. We prove that the architecture is a universal approximator of constrained functions and derive computationally tractable formulations based on linear decision rules. Empirical results on benchmark regression tasks show that our method consistently satisfies constraints while maintaining competitive accuracy and low inference latency.

Enforcing Hard Linear Constraints in Deep Learning Models with Decision Rules
Gonzalo E. Constante
Hao Chen
Davidson School of Chemical Engineering
Davidson School of Chemical Engineering
Purdue University geconstaßpurdue .dd
Purdue University
chen4433ppurdue edu
Can Li
Davidson School of Chemical Engineering
Purdue University
canli@purdue. edu
Abstract
Deep learning models are increasingly deployed in safety-critical tasks where predictions must satisfy hard constraints, such physical laws, fairness require- ments, or safety limits. However, standard architectures lack built- mechanisms enforce such constraints, and existing approaches based regularization projection are often limited to simple constraints, computationally expensive, or lack feasibility guarantees. This paper proposes model-agnostic framework for enforcing input-dependent linear equality and inequality constraints on neural network outputs. The architecture combines task network trained for prediction accuracy with safe network trained using decision rules from the stochastic and robust optimization literature ensure feasibility across the entire input space. The final prediction is a convex combination of the two subnetworks, guaranteeing con- straint satisfaction during both training and inference without iterative procedures or runtime optimization. We prove that the architecture is a universal approximator of constrained functions and derive computationally tractable formulations based on linear decision rules. Empirical results on benchmark regression tasks show that our method consistently satisfies constraints while maintaining competitive accuracy and low inference latency.
Introduction
Failing encode safety, fairness, physical constraints has become of the most fundamental limitations of standard deep learning models, hindering their widespread deployment in safety-critical domains, such energy systems, autonomous vehicles, and medical diagnostics (1 However, strictly guaranteeing constraint satisfaction on a neural network poses a significant challenge, particu- larly in time-sensitive settings and when the constraints are nigh-dimensional or input-dependent.
Popular approaches to promote constraint satisfaction use penalty- based regularization, where the constraint violations penalized in the loss function reward; however, these approaches, known as soft methods, are unable to provide zero constraint violation during the training and inference stages and their penalty coefficients are difficult to determine. Recent work has explored projection- based strategies either as post-hoc methods or incorporated into training, which guarantees hard constraint satisfaction. Nevertheless, these methods often face scalability and run-time limitations, as they generally require iterative procedures or solving optimization problems during inference. For detailed discussions methods for satisfying hard constraints, the reader is referred Section
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Safe Network
SSN (zz)
Decision
Input parameters
Rule
Task Network
falx) - (s) (z)
f7 (x)
f8
Equality projection
Figure 1: Proposed framework.
In this paper, we propose a framework for embedding input-output linear constraints directly into neural networks with low run-time complexity without any iterative procedure. The proposed architecture has two subnetworks: task network, trained to minimize prediction loss, and safe network, designed ensure feasibility with respect to the constraint set. The proposed framework is agnostic to the task network, which is trained using stochastic gradient-descent a algorithms, while the safe network computed using a decision rule approach independent of the training of the task subnetwork. The final output is computed as a convex combination of the two subnetworks. This structure, illustrated in Figure [ allows for seamless integration into end to-e training pipelines and provides constraint satisfaction during training and inference stages.
The main contributions of this work
follows:
Framework for input-output linearly constrained predictions. We propose a framework for embedding hard linear and input-dependent constraints directly into neural networks. The framework produces constraint-see constraint-satiifying outputs in a single forward pass, and can be incorporated into any standard deep learning models.
Theoretical guarantees and inference complexity analysis. We provide a universal approximation theorem that characterizes the expressive power of the proposed architecture and proves that it can approximate any continuous, constraint-san -aattainnt-aiiifyinn function arbitrarily well. Moreover, we provide a detailed analysis of the inference complexity, showing that the proposed framework introduces negli- gible overhead compared to the task network forward pass, particularly when exploiting constraint sparsity when the left-hand constraint coefficients are input-independent.
Benchmarking against Hlate-of-the-aee methods. We compare our method existing projection- based and penalty-based approaches on multiple tasks, showing superior constraint satisfaction, and competitive inference time and accuracy.
The remainder of the paper is organized as follows: Section 2 presents the related work on hard methods to enforce constraints. Sectionniitrrduuee the notation and problem statement, describes the proposed framework, and presents the theoretical analysis of the proposed framework and complexity analysis. Section details the training of linear decision rules. Section 5 shows the numerical experiments to evaluate the performance of the proposed methodology. Finally, Section concludes the paper and outlines our future work.
2 Related work
Activation functions Simple constraints can be enforced using activation functions. For example, softmax can enforce probabilistic simplex constraints, while box constraints can be addressed using sigmoid or clipped ReLU activations followed by post-scaling match the desired bounds.
Projection approaches These methods enforce constraints by projecting the output of the neural network orthogonally into the constraint set using differentiable optimization layers. In most cases, such methods require solving optimization problems during training and inference stages 3 | 5 In special cases, there exist closed- foosed-fomm... solutions for such optimization problems, represented as differentiable layers, bypassing the need to solve them during inference 6 7.. For linear constraints, ( propose a fast projection method for input-dependent constraints, based generalization of

---

closed-form projections for single constraints. However, their approach assumes that the number of constraints is less than the number of variables-an assumption that excludes wide range of realistic problems in the constrained optimization literature. Other procedures involve iterative procedures, including alternating projection methods and their extensions I [1, and unrolling differentiable gradient-based corrections [1]. However, iterative and optimization-baseee projection methods can be computationally expensive and slow to converge, particularly for problems with complex or large numbers of constraints.
Sampling approaches Inner approximation of convex constraint sets can be constructed by sampling the constraint set. A feasible point can be characterized as convex combination of the vertices and rays obtained during the sampling step. These approaches have been proposed for homogeneous linear constraints [T3] and convex polytopes [4]. However, this approach shows two limitations: (i) the number of feasible samples needed grows exponentially with the dimension of the output space, and (ii) it is restricted to constraint sets that are invariant with respect to the input.
Preventive learning This approach aims to enforce constraint satisfaction by shaping the training process so that the network learns to avoid infeasible regions rather than correcting outputs post hoc. 15 propose framework that integrates constraint information into the loss function via preventive penalties and feasible region sampling, encouraging the model to produce constraint-satee cesstainttsatisfying outputs throughout training. However, these approaches rely on carefully balancing penalty terms and may still suffer from feasibility violations.
Gauge functions generalization of norms [16] Tabas and Zhhang 17 [8] propose gauge mapping, which maps a hypercube to a polytope, ensuring that the network output remains within the target polytope. One of the limiting assumptions of this method is that a feasible point of the target polytope is known. Li et al. [99 address this issue by solving an optimization problem during training and inference to find the feasible point. However, such approach is impractical for input-dependent target polytopes. For family of convex input-independent constraints, Tordesillas at. 200 propose RAYEN, which utilizes analytic expressions for the gauge functions, to project infeasible predictions onto the target constraint set. Recently, Liang et al. (21] proposed Homeomorphic Projection. which uses an invertible neural network to map the constraint set to a unit ball, enabling efficient projection via bisection but requiring a known interior point and adding inference overhead.
These works, which non-iterative, are based on gauge functions that are
Our framework, inspired by decision rules from stochastic and robust optimization 22 generalizes the gauge mapping approach enforce input-dependent linear constraints in neural networks through a non-iterative, closed-form correction. In contrast to existing state-of-the-art methods, it guarantees hard constraint satisfaction without requiring feasible points, iterative solvers, or specialized deep learning architectures at inference time.
Proposed framework
Notation We model the input space as a probability space (IR* B(TR*), P), where ((R+*) denotes the Borel o-algebra over R and IP is a probability measure with support X' C R& The space of all Borel-measurable, square-integrable functions from R to R is denoted by C The space of continuous functions from 2' to R" is denoted by C.). We denote by & the set of symmetric matrices in * Given a matrix A E m AT denotes its transpose. The identity matrix is denoted by I with appropriate size inferred from the context. The standard basis vector with in the first component and zeros elsewhere is denoted by e E R The notation M ž 0 denotes that matrix M is symmetric positive semidefinite. All incqualities involving vectors are understood elementwise. nnz(.) denotes the number of non-zero elements.
3.1 Problem setup
For a parameterized function fø : X' C R + R", we aim at enforcing the following constraint set C(x) := ty E R² G(z)y= g(z), H Hx)y I h(z)), where C(x) C R denotes the constraint set associated with input x X. We assume that C(x) is a nonempty and convex set for each x E X. The matrices G(x)) E m x and H(x) E mingg xn and
(I)
vectors g(x) E Te and h(x)) E R are linearly dependent on x and define meg equality and Mineq incquality constraints, respectively.
3

---

Without loss of generality, we represent as
+ + - b(x), s(x) 2 0, Vr E A.
(2)
where Meg Mineq denotes the number of constraints, A(x) R denotes the left-hand side coefficient whose first Meq rows correspond to the equality constraints. b(x) E IRm denotes the
right-hand side coefficient b(z) Br for some B E RT**, and s(x) denotes the slack variable whose first mee elements equal to 0. The 9th row of the left-hand side coefficient A(x) is
representable as x Ag for some matrix Ag R where & I,...,m.
3.2 Proposed framework
We propose an architecture composed of two subnetworks: a safe network, which is designed to produce outputs that satisfy the linear constraints in (2) for all inputs X, and lask network, which focuses on optimizing task-specific performance, such as minimizing a loss function. The outputs of these two networks are combined form the final prediction of the overall architecture.
Specifically, given an input z, the final output is defined as a convex combination of the outputs of the task and safe networks: (x) = (1 (x)())) (z))) (x),
where 13 (I) and S (x) denote the outputs of the task and safe networks parameterized by 0 and 6, respectively. The overall parameter vector is denoted by vb == (0, Ø). The scalar (x) E [o, I] an input-dependent scalar that determines the relative contribution of the safe network to ensure feasibility of the final output.
Task network The goal of the task network is to provide output that minimizes the loss function. The proposed framework is agnostic to the specific architecture of the task network, i.e., it can accommodate any neural network model and whose parameters can be learned using gradient-based methods. The output of the task network is corrected by an equality correction module reported in [6), which orthogonally projects the task network output onto the affine subspace defined by the set of equality constraints. The equality constraint satisfaction module derived in closed-form from the KKT-conditions. The details can be found in the Appendix.
Safe network The role of the safe network is to provide feasible point satisfying the equality and inequality constraints. A key challenge in enforcing input-dependent constraints is to ensure that the predicted output remains feasible for all realizations of the input space x E X. To this end, propose find the parameters of the safe network using decision rule approach, a technique with a strong foundation in optimization under uncertainty (22]. Decision rules, also known as policies in the context of stochastic and robust optimization, provide tractable approach modeling adjustable decisions under uncertainty. A decision rule defines a functional mapping from uncertain parameters to decisions that are immune the uncertainty set, i.e., they satisfy the constraints for all the realizations of the uncertainty set X. Formally, if æ E X denotes an uncertain input drawn from known uncertainty set, decision rule specifies function y(x) that determines the decision corresponding to each realization of x. This formulation closely resembles the prediction task in supervised and reinforcement learning, where the goal is learn an input-output mapping.
Neural network output The output of the task network, fT'N (x). is projected to satisfy the equality constraints, while the output of the safe network, f (x)) is constructed to satisfy both equality and inequality constraints for all E X. We define the final network output as in (3). To ensure that the final output satisfies the inequality constraints, we compute the smallest value of (x) E [0, 1] such that
(x))((()) h(x),
i.e., such that the convex combination is feasible. The equality constraints are satisfied by construction, since both ff FTN (x)) and ssN (x) lie in the affine subspace defined by the equality constraints.
To compute an (2), we define the residual slack vectors:
TN (T(x) = h(x) ((x)fT((x)
h(x) -()))
$9
which quantify the pointwise satisfaction margin of the inequality constraints for each network output.

---

TN (x) < 0} denote the set of inequality constraints violated by the Let fi {l...., Mineg 8,
task network. For each such constraint, we determine the minimum value of (xp(2) E [0, 1] such that the corresponding convex combination becomes feasible. The tightest such value across all violated constraints yields:
TN
a (2) == max IEZ 8
This construction ensures that the final output f (z) satisfies all constraints in C(z), while staying as close as possible to the task network output within the feasible set. Note that although (zp(z) is computed from the slack vectors of (xx) and SNN (z), which depend on the outputs of the task and safe networks parameterized by 0 and Φ, respectively, it is evaluated using max operation that is subdifferentiable almost everywhere. This allows gradients to propagate only through the constraint index that achieves the maximum, while all other terms treated as constant.
To make this explicit, consider the loss L (x), y). By the chain rule,
Oc =
Oc 00 (z)
( (o
050 Ofv(z) 00
2xp(2) 00
(x.(z))
In this expression, the term aa a is nonzero only for the constraint selected by the max, leading to sparse gradients with respect to 0. As result, the effective update to the task network is modulated by both the selected constraint and the value of ar (1)) reflecting the degree feasibility violation in the task output.
3.3 Computing the safe network
The safe network is computed offline, separately from the training of the task network. We formulate the computation of the safe network as the following optimization problem:
maximize saN E BE
subject to
A(x) (x) + +(xx) = b(x) s(x) 2t
(5)
P-a.s
where the slack variable s(x) introduced express inequality constraints in standard form. The objective in (5) encourages the safe network output to lie deep in the interior of the feasible region. The larger the slacks in the safe output, the more flexibility have to interpolate while preserving feasibility. As a result, the final output remains closer to the task network prediction, reducing conservativeness. Note that problem (5) an infinite-die dimensionna optimization problem since the variables, f and s, live in function spaces and the constraints must hold for all æ E X' almost surely.
While problem (5) guarantees feasibility, intractable in general since finding its optimal value is known to be #P-hard (23 To obtain tractable surrogate, we restrict the functional form of the decision rules [22]. Prior efforts include forms of decision rules to be affine (24), segregated affine 25), piecewise affine 226 22), and trigonometric polynomial (28) functions. In Section 4 we provide
computationally tractable inner approximations of problem (5). which can be efficiently solved using state-of-the-art constrained optimization solvers.
3.4 Theoretical guarantees
We prove that, under mild conditions on the task and safe network function classes, the proposed architecture universal approximator of continuous functions that satisfy input-dependent linear constraints.
Assumption 1 (Task Network Function Class), Let X C R be compact. The task network feetorrfuunctio class Frask C C(X, IRr) satisfies the universal approximation property: for any continuous function f" X + R and any E > 0, there exists function ff E Frask such that
sup 115 (x)) 5 f" (x)ll <. x
5

---

Assumption 2 (Safe Network Function Class), Let ((x) C I= be a convex constraint set for each X. The safe network function class Fsato - C(X, R") such that for all f E F
(2) C(x), Vx E.
Moreover. for every a E X, the convex hull of the set { (x) | feN Feate) is dense in C(x).
Theorem (Universal Approximation with Constrained Output). Let X C R& be compact, and let
I + R be a continuous function such that (x) C(x) for all x E X, where C(x) C RR is a convex, deut-deeenddnt constraint set. Suppose Assumptions [ 2land2holdd
Then, for any O, there exist functions JTN E Fiasks foN sade, and scalar E [0, 1] such that the convex combination
fx(x) =1 (1- +af (x)
satisfies fo(a) E for all x E X, and
sup sEx
[lf((z)) ff <E.
3.5 Inference complexity
Equality Projection of Task Network Output When G(z) is input-independent the projection can be efficiently implemented by exploiting the closed-form structure in (T6) without explicitly forming the dense matrix I - GG, where G == G (GGT)- denotes the left pseudo-inverse of G IR ""ee xn. used project onto the affine set defined by the equality constraints. The projection is computed as x ((Gx) + Gg, requiring two matrix-vector products with G and G. both costing On M, and lightweight vector additions at (n). Thus, the overall inference complexity is O(n meg)) regardless of the density of G. When (zx) is input-dependent, the projection must be recomputed for each input. In the dense case, the dominant costs are forming G(x) G()) with On mag) and inverting it with Om,))) resulting in an overall complexity of O(n m m²g)) ). In the sparse case, matrix-vector products cost O Onnz (Gx))) while forming G(x)G(a) costs ((nnz(G(z)) meg). The inversion remains ( leading to a total cost of O O(nz G(x)) Mey + my + n)).
Safe Network, Slack, and Convex Combination Complexity The safe network forward pass consists of dense matrix-vector product Fx, resulting in computational cost of OOn k). The slack computation, defined as sxx) = h/(x)) (x))(xx) requires computing ( ) In the dense case, this operation incurs cost of ( n.), while in the sparse case, it reduces to ( (H (x))), depending on the sparsity of H (xx) The calculation of the blending parameter a involves evaluating Mineq scalar inequalities and thus has cost of (minnnn) Finally, the convex combination of the task and safe network outputs requires (n:) operations.
Total Inference Complexity Overall, the proposed framework achieves low inference complexity, summarized in Table [] The dominant cost arises from the task network forward pass, denoted by CTN. which refers to the computational complexity of a single forward pass through the task network. The remaining components introduce negligible overhead, especially when exploiting sparsity in the constraint matrices.
Table 1: Total inference complexity of the proposed framework. nnz(-) denotes the number of non-zero elements.
Constraints
Coefficients G(x)
Total Complexity
Input-dependent
O (Cin + m + m3 + k + Miney n)
Dense
Input-independent
O (CIN + meq + k. + Mineq n2)
Input-dependent enpttinndependent
(CIn + nnz G(x)) meq + m3 + n nnz + n)
Sparse
O (CTN + n meg + k mnz(H) + n)

---

Offline computation of safe networks with decision rules
4.1 Linear decision rules for linear constraints
To reduce the computational tractability of problem network to linear functions of the input, as follows:
we restrict the functional form of the safe
(x) = Fx,
(6)
where F Rn*k, Additionally, we assume that the input space X' F be a nonempty, compact set defined as:
x== I c]=== P, 2 0, -.......
(7)
for some symmetric matrices P3 3th Note that set X' corresponds the intersection of I ellipsoids intersected with an affine hyperplane constraint e and without loss of generality, we assume that the first component of any E X' is equal to 1. Furthermore, we assume that the linear hull of X' spans R* Under these assumptions, the slack variable s()) becomes a quadratic function of the input z and problem (5)
1. For the sake of notational simplicity
is approximated as:
maxinize F.9.t
subject to
F E Bank S (Si,.... Sm) E (8
E
x A,Fx I + Sez = b/ x, l 1,.... m
PPa.s..
x S5 2 t, Ł= meg
m
We that problem (8) is semi-infinite problem due to the finite number of variables, but an infinite number of quadratic constraints. To make the above problem tractable, we first handle the equality constraints. Since all constraints continuous in x, can symmetrize and rewrite the 9th quadratic equality constraint as follows:
HE 0, Ve X,
where H, (A&F+ ea - beel) Se is symmetric matrix.
Since the constraint is quadratic and homogeneous X, it equivalently holds over the conic hull of X, denoted cone(X). Given that X' spans & by assumption, cone (X) has non-empty interior. The Hessian of the mapping + x. H3æ is given by 2H, Consequently, if the quadratic form vanishes over the interior of cone(X', follows that the Hessian must be He 29 Thus, the original semi-infinite equality constraint admits the equivalent finite-dimensional representation H2 0. This reformulation substantially enhances tractability, as replaces the intractable semi-infinite constraint with linear matrix equality.
Next, we can use Proposition to approximate the semi-infinite constraints in simplifying it into the following semidefinite program (SDP):
maximize F,S,t
subject to
F Rank 5= 5 (S (5K) . mix! te R
1 be + bbee)), =p= 1,..,...,
(A(F + FT ATAA) + St
(10)
S Σ
AejP, t1 2 0, Ve = meg 1,, 1,...,
1=1
A20. 0.
The above SDP constitutes an inner approximation of the original problem in (8) in the general case. This means that the feasible set of the SDP is contained within that of the original problem, ensuring tractability at the cost of some conservatism. However, when and the SDP formulation recovers the original feasible set without loss of generality. This property follows from classical results on the tightness of SDP relaxations for single quadratic constraints. A key advantage of problem (10) is that it yields SDP whose size grows polynomially with the dimensions of the input and output spaces, as well as with the number of constraints defining X. This ensures that the problem can be solved efficiently using tftte-of-the-ar interior-poin methods.
this approximation becomes exact,
7

---

4.2 Linear decision rules for jointly linear constraints
special case of the previously presented framework arises when the left-hand side matrix is input-
independent, ie.. A(x) A. while the right-hand side remains input-dependent as b(x) = Bx,
where B E IR=* is fixed. Under this assumption, the constraints of problem 5 reduce to the following jointly linear form:
Al SN (x) s(x) Bx, s(x)
Vo E X.
(11)
To gain further tractability, we assume the imput space A' C IR to nonempty, compact polyhedron defined
X = {2 E E
Px 2 p).
(12)
for some matrix P E R and vector e R'. Without loss of generality, we assume that the first component of any X' is equal to Furthermore, Under these assumptions, the slack variable s(x) becomes a linear function of the input æ. We note that the input space defined in (12 is special case of the general input space in (7).
assume that the linear hull of X spans R
With the above assumptions, problem (5) is approximated as:
maximize F,S,t
subject Lo
F E Raxk ,S Rmnn TRmxK,LERR LER
(13)
AFæ + Sx Bu, Sz 2 to,
P-a.s.
where v = (I (Lssmm)))) = enable tractable reformulation of problem 13 we can use Proposi-
tion to reformulate the semi-infinite constraints, simplifying it into the following linear program:
maximize F,A,8
subject to
FE E Baxk A mx , ,
AF + +AP= B, Ap 2 lv, A 0.
(14)
The key advantage of the reformulated problem (14) is that a linear program whose size grows polynomially with the dimensions of the input and output spaces, as well with the number of constraints defining X.. This makes the problem efficiently solvable using state-of-the-aar linear programming solvers.
Limitations, extensions, and scalability considerations While the proposed framework demon- strated regression tasks, it naturally extends to classification and actor-critic reinforcement learning methods such as DDPG, where state-dependent action constraints can be enforced. In classification, constraints can be applied in logit space directly the softmax outputs. Many distributional constraints (e.g., total variation distance) can be expressed as linear constraints and are therefore supported by method. Extending stochastic policy gradient methods like PPO is more involved, as projection alters the action distribution and affects proobbbbyyyyityy computation.
The architecture remains highly efficient inference, relying on simple projection and convex combination operations. The main computational cost arises during training, when determining the safe network parameters involves solving optimization problems that scale polynomially with problem size. Scalability can be improved by exploiting sparsity, decomposability, and GPU acceleration, significantly reducing training overhead for large-scale applications.
5 Numerical experiments
Tasks
We evaluate our framework on two end-to-end constrained optimization learning tasks:
DC Optimal Power Flow (DC-OPF): The task is to predict the optimal generation output for varying demands. The demands are assumed to be upper and lower bounded around a nominal value [30].
This task corresponds case of jointly lincar constraints. We consider both the

---

linear and quadratic formulations of the DC-OPF problem to capture different objective structures commonly used in practice. The uncertainty levels are set to +40% for the 14-bus and 57-bus cases, +100 for the 30-bus and 200-bus cases, and +300% for the 118-bus case. Due to space limitations, we present a subset of the quadratic formulation results in the main paper; the remaining results, along with those for the linear formulation, are provided in Section C.3 of the Appendices.
Portfolio Optimization: The task is predict optimal portfolio allocations under uncertain maturity and rating of bonds BT, which are upper and lower bounded +40% around a nominal value. This task corresponds to a case of input-dependent left-hand side uncertainty.
The specific architecture and training configurations of the neural networks used in our experiments are detailed in Section C.2 of the Appendices.
Baselines
We compare our framework with the following baselines:
We employ Gurobi and OSQP (32, two state-of-the-ar solvers for constrained opti- mization problems, to obtain the optimal solutions for both tasks. To enhance convergence speed, both solvers are warm-started using the outputs of a neural network trained to learn the mapping from demands to (i) generation outputs for the DC-OPF problem and (ii) maturity and rating of bonds for the portfolio optimization problem.
Optimizers:
Post projection: A neural network is first trained to learn the same mapping as above. Its predictions are projected onto the feasible set by solving a quadratic optimization problem that minimizes the Euclidean distance between the network output and the feasible set defined by the constraints.
Alternating projection method (APM): An iterative feasibility correction approach that alternates projections onto the equality and inequality constraint sets until feasible point is reached or maximum number of iterations is exceeded.
A variant of APM that accelerates convergence by introducing an extrapolation factor based on the previous iterate, effectively reducing the number of projections required to reach feasibility. This method maintains the same feasibility guarantees as APM but achieves faster convergence in practice.
Extrapolated alternating projection method (EAPM) 33
Deep Constraint Completion and Correction (DC3) [2]] This approach uses equality completion and unrolled projected gradient corrections for satisfying constraints.
Linear Decision Rule (LDR): We assess the performance of the linear decision rule used in the safe network without relying on the task network.
The implementation of all the methods can be found this GitHub repository htttps///github com htw.:iigrrop/Decisiionn....o
Evaluation Metries We assess the performance of the proposed framework and baselines using the following metrics: (i) Optimality Gap: to measure the percentage deviation of the predicted solution's 's objective value from the optimal value computed by the optimizer (Gurobi), (ii) Feasibility: to quantify the degree to which the predicted solution violates the problem's 's inequality and equality constraints, and (iii) Inference speed: to capture the average computation time in miliscconds required to generate a prediction for a given input. The details can be found in the Appendix.
Analysis of Results Table shows that the proposed method consistently achieves zero constraint violations across all tasks, matching the optimizers and outperforming DC3, which exhibits inequality violations that increase with problem size. In terms of optimality, the proposed method maintains gaps below 2.5% in all cases, significantly improving over DC3 and LDR in the portfolio task, where gaps exceed 29% and 48%, respectively. APM achieves low violations but is substantially slower, requiring up to 225 ms and hundreds of iterations for large systems. EAPM improves on APM's runtime but remains slower than the proposed method. Post-projection approaches more accurate than DC3 and faster than APM/EAPM but still lag behind the proposed method in both speed and optimality. The proposed method achieves inference times under ms across all tasks, offering an order-of-magnitue speedup over iterative methods and even outperforming the optimizers, whose runtime grows with problem size. Overall, these results demonstrate that the proposed method provides the best balance between feasibility, optimality, and efficiency for both the portfolio optimization and large-scale quadratic DC-OPF problems, scaling effectively while maintaining high solution quality.

---

Table 2: Results on both tasks over the held out test dataset. The optimality gaps, equality and inequality violations are shown as mean (worst) values, and the values of time are indicated as the average solve time milliseconds (average iterations required by the method).
Method
Optimality Gap Equality Violation Incquality Violation Time
Portfolio optimization problem: a = 16, Mfle
2, Mineg = 99 k = 10
Proposed
2.35(6.37)
0.000r0.000)
0.000(0,000) 000)
2.8(1)
Optimizer (Gurobi)
0.00(0.00)
0.000(0.000)
0.000(0.000)
0.2
APM
0.098)
0.000(0.000)
0.000(0.001)
78.9(109)
EAPM
3.55(8.91)
0.000(0.000)
0.000(0.001)
10.2(5)
DC3
29.6(36.7)
0.000(0.000)
0.053(0.123)
319.2(300)
LDR
48.56(59.52)
0.000(0.000)
0.000(0.000)
1.2(1)
DC-OPF (QP) 118-bus system: n
1126, meg 340, Mineq
768, A = 118
Proposed
1.10(2.13)
0,000(0.000)
0.000(0.000)
2.5(1)
Optimizer (Gurobi)
0,00(0.00)
0.000(0.000)
0.000(0.000)
4.1
Optimizer (OSQP)
0.00(0.00)
0.000(0.000)
0.000(0.000
6.3
Post projection (Gurobi)
0.11(1.88)
0.00000.000)
0.000(0.
10.1
Post projection (OSQP)
0,51(1.88)
0.000(0.000)
0.000(0.000)
5.5
APM
0.05(0.08)
0.000(0.000)
0.001(0.013)
117.6(156)
EAPM
1.50(2.25)
0.000(0.000)
0.000(0.000)
18.8(11)
DC3
5.44(7.19)
0.000(0.000)
0.004(0.088)
335.3(276)
DC-OPF (QP) b00-bus system:
1527, Meq 452, Mineq
1044, A 200
Proposed
1.07(1.71)
0.000(0.000)
0.000(0.000)
2.6(1)
Optimizer (Gurobi)
0.00(0.00)
0.000(0.000)
0.000(0.000)
4.6
Optimizer (OSQP)
0.00(0.00)
0.000(0.000)
0.000(0.000)
15.1
Post projection (Gurobi)
0.23(1.49)
0.000(0.000)
0.000(0.000)
13.2
Post projection (OSQP)
0.22(1.49)
0.000(0.000)
0.000(0.000)
7.4
APM
8.55(9.45)
0.000(0.000)
0.166 181)
225.3(300)
EAPM
3.88(7.95)
0.000(0.000)
0.000(0.000)
78.7(50)
DC3
18.74(20.61)
0.000(0.000)
0.164(0.189)
360.5(300)
6 Concluding remarks and future work
In this work, proposed framework to enforce hard linear equality and inequality constraints on the input-output mapping of neural networks. The framework leverages linear decision rules and convex combinations of task and safe networks provide feasibility guarantees over the entire input space while preserving low run-time complexity. We provided tractable formulations based on robust optimization duality and demonstrated their effectiveness on benchmark tasks.
Despite these promising results, our work presents limitations that warrant future research. First, while linear decision rules offer simplicity and scalability, they may limit the expressiveness of the safe network in highly nonlinear settings. Extending the framework to incorporate more flexible function classes could improve approximation capabilities and reduce conservativeness. Second, the current framework assumes that the input space and constraint sets are polyhedral and convex. Investigating how to efficiently handle equality constraints and other families of sets, including non-convex or time-varying, would broaden the applicability of the method to more realistic scenarios, such as safety- critical control systems and autonomous decision-making Third, although the proposed methods computationally efficient, their scalability could be further enhanced by exploiting problem structure. When the constraints exhibit decomposability, sparsity, or low-dimensional manifolds, customized algorithms could be designed to reduce computational overhead during training. Fourth, the feasibility coefficient (x)) is computed using a piecewise differentiable maximum operator. While this allows gradients to flow through the most violated constraint, it results in sparse and potentially unstable updates. A promising direction is to use a straight- -through estimator, preserving the exact maximum in the forward pass to ensure feasibility while substituting a differentiable surrogate (e.g., softmax over slack ratios) in the backward pass to improve gradient quality. Finally, from a theoretical perspective, providing formal guarantees on the approximation error and feasibility margins introduced by the convex combination of task and safe networks would offer deeper insights into the robustness and generalization capabilities of the proposed architecture.
10

---

Acknowledgments and Disclosure of Funding
C.I. acknowledges the financial support from the Office of Naval Research (Grant No. N000142440 42411641),, the National Science Foundation (Grant No. CBET-2441184), and the startup funding provided by the Davidson School of Chemical Engineering and the College of Engineering at Purdue University.
References
[1] Javier García and Fernando Fernández. A comprehensive survey on safe reinforcement learning. Journal of Machine Leurning Research, 16(42):1437-148) 2015.
[2] James Kotary, Ferdinando Fioretto, Pascal Van Hentenryck, and Bryan Wilder. End-to-end constrained optimization learning: A survey. In International Joint Conference on Artificial Intelligence, pages 4475 4482. ijcai.org, 2021.
[3] Brandon Amos and J Zico Kolter. Optnet: Differentiable optimization as layer in neural networks. Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference Machine Learning. volume 70, pages 136-145. PMLR, 4 2017.
[4] Akshay Agrawal, Brandon Amos, Shane Barratt, Stephen Boyd, Steven Diamond, and J Zico Kolter. Differentiable convex optimization layers. In H Wallach, H Larochelle, A Beygelzimer, d Alché-Buc, E Fox, and R Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[5] Bingqing Chen, Priya L. Donti, Kyri Baker, Zico Kolter, and Mario Bergés. Enforcing policy feasibility constraints through differentiable projection for energy optimization. In Proceedings of the Twelfth ACM International Conference on Future Energy Systems, e-Energy '21, page 199-210, New York, NY. USA, 2021. Association for Computing Machinery.
[6] Hao Chen, Gonzalo E. Constante Flores, and Can Li. Physics-informed neural networks with hard linear equality constraints. Computers & Chemical Engineering, 189:108764, 10 2024.
[7] Wenbo Chen, Mathieu Tanneau, and Pascal Van Hentenryek. End-to-end feasible optimization proxies for large-scale economic dispatch. IEEE Transactions on Power Systems, 39(2):4723- 4734, 2024.
[8] Youngjae Min, Anoopkumar Sonar, and Navid Azizan. Hard-constrained neural networks with universal approximation guarantees. 2025.
[9] Heinz H. Bauschke and Jonathan M. Borwein. On projection algorithms for solving convex feasibility problems. SIAM Review, 38(3):367-422 1996.
[10] Steven Chen, Kelsey Saulnier, Nikolay Atanasov, Daniel D. Lee, Vijay Kumar, George J. Pappas, and Manfred Morari. Approximating explicit model predictive control using constrained neural networks. In 2018 Anmual American Control Conference (ACC), pages 1520-1527, 2018.
[11] Giacomo Lastrucci and Artur M Schweidtmann. Enforce: Exact nonlinear constrained learning with adaptive-depth neural projection. arXiv preprint arXiv:2502.06674, 2025.
[12] Priya Donti, David Rolnick, and J Zico Kolter. DC3: A learning method for optimization with hard constraints. In International Conference on Learning Representations, 2021.
[13] Thomas Frerix, Matthias Niesner, and Daniel Cremers. Homogeneous linear inequality con- straints for neural network activations. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 9229-3234 IEEE, 2020.
[14] Liyuan Zheng, Yuanyuan Shi, Lillian J. Ratliff, and Baosen Zhang. Safe reinforcement learning of control-affine systems with vertex networks. In Ali Jadbabaie, John Lygeros, George J. Pappas, Pablo Parrilo, Benjamin Recht, Claire J. Tomlin, and Melanie N. Zeilinger, editors, Proceedings of the 3rd Conference on Learning for Dynamics and Control, volume 144 of Proceedings of Machine Learning Research, pages 336-347. PMLR, 07 08 June 2021.

---

[15] Tianyu Zhao, Xiang Pan, Minghua Chen, and Steven Low. Ensuring DNN solution feasibility for optimization problems with linear constraints. In The Eleventh International Conference on Learning Representations, 2023.
[16] Franco Blanchini and Stefano Miani. Set-Theoretic Methods in Control. Springer International Publishing, 2015.
[17] Daniel Tabas and Baosen Zhang. Computationally efficient safe reinforcement learning for power systems. In 2022 American Control Conference (ACC), pages 3303-3310. IEEE, 6 2022.
[18] Daniel Tabas and Baosen Zhang. Safe and efficient model predictive control using neural networks: An interior point approach. In 2022 IEEE 61st Conference on Decision and Control (CDC), pages 1142-1147, IEEE, 12 2022.
[19] Meiyi Li, Soheil Kolouri, and Javad Mohammadi. Learning to solve optimization problems with hard linear constraints. IEEE Access, 11:59995-600044 2023.
[20] Jesus Tordesillas, Jonathan P. How, and Marco Hutter. Rayen: Imposition of hard convex constraints on neural networks, 2023.
[21] Enming Liang, Minghua Chen, and Steven H Low. Homeomorphic projection to ensure neural-network solution feasibility for constrained optimization. Journal of Machine Learning Research, 25:1-55, 2024.
[22] A. Ben-Tal, A. Goryashko, E. Guslitzer, and A. Nemirovski. Adjustable robust solutions of uncertain linear programs. Mathematical Programming. 99:351-376, 2004.
[23] Martin Dyer and Leen Stougie. Computational complexity of stochastic programming problems. Mathematical Programming, 106:423-433 12 2006.
[24] Angelos Georghiou, Daniel Kuhn, and Wolfram Wiesemann. The decision rule approach to optimization under uncertainty: methodology and applications. Computational Management Science, 16:545-576, 10 2019.
[25] Xin Chen, Melvyn Sim, Peng Sun, and Jiawei Zhang. A linear decision-basee approximation approach to stochastic programming. Operations Research, 56:344-357, 3 2008.
[26] Angelos Georghiou, Wolfram Wiesemann, and Daniel Kuhn. Generalized decision rule approx- imations for stochastic programming via liftings. Mathematical Programming, 152:301-338, 2015.
[27] Angelos Georghiou, Angelos Tsoukalas, and Wolfram Wiesemann. A primal-dual lifting scheme for two-stage robust optimization. Operations Research, 68:572-590, 3 2020.
[28] Dimitris Bertsimas, Dan Andrei Iancu, and Pablo A. Parrilo. A hierarchy of near-optimal policies for multistage adaptive optimization. IEEE Transactions on Automatic Control, 56:2809-2824,
12 2011.
[29] Daniel Kuhn, Wolfram Wiesemann, and Angelos Georghiou. Primal and dual linear decision rules stochastic and robust optimization. Mathematical Programming, 130:177 130:177-209, 2011.
[30] Sogol Babaeennjjassarookoleee, Adam Birchfield, Richard D. Christie, Carleton Coffrin, Christo- pher DeMarco, Ruisheng Diao, Michael Ferris, Stephane Fliscounakis, Scott Greene, Renke Huang, Cedric Josz, Roman Korab, Bernard Lesieutre, Jean Maeght, Terrence W. K. Mak, Daniel K. Molzahn, Thomas J. Overbye, Patrick Panciatici, Byungkwon Park, Jonathan Snod- grass, Ahmad Tbaileh, Pascal Van Hentenryck, and Ray Zimmerman. The Power Grid Library for benchmarking AC optimal power flow algorithms, 2021.
[31] Mark Rubinstein. Markowitz's" portfolio selection"; A fifty-year retrospective. The Journal of finance, 57(3):1041-1045. 2002.
[32] Bartolomeo Stellato, Goran Banjac, Paul Goulart, Alberto Bemporad, and Stephen Boyd. Osqp: An operator splitting solver for quadratic programs. Mathematical Programming Computation, 12(4))637-672, 2020.
12

---

[33] Heinz H Bauschke, Patrick L Combettes, and Serge G Kruk. Extrapolation algorithm for affine-convex feasibility problems. Numerical Algorithms, 41:239-274, 2006.
[34] A. Ben-Tal and A. Nemirovski. Robust convex optimization. Mathematics of Operations Research, 23:769-805 -059 11 1998.
13

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims the document are properly justified in Sections 3 to which include the framework description, training of linear decision rules, and numerical experiments.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer:
[Yes]
The limitations of the proposed framework are detailed in Section future lines of research.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
14

---

Answer: [Yes]
Justification: The paper provides several theoretical results with their corresponding assump- tions and proofs in the appropriate section and the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The proposed framework is clearly described. All the information for the numerical experiments is stated in the main document while the selection of hyperparameters is presented the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
15

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: The data and code provided for the reproduction of the main experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co.
www.abimmsiiooooooiiiioo.. for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
7 //nips. wwtpccpppppppppu..ccccc htte://wiissseeeeeee wwwweeewwisssoonnooooooooooooeee
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
The paper provides detailed descriptions of datasets, uncertainty levels, and problem parameters. The experimental protocols, including data splits, training hyperparam- eters, and optimizer settings, specified in the main document and Appendix.
Justification:
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
The experimental results report both mean and worst-case metrics, providing insights into variability. While error bars are not shown, the worst-case results complement mean values, ensuring robust reporting of performance across test instances.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
16

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
The paper specifies the computer resources, and average per-instance inference times are reported. The total runtime and iteration counts for other methods are stated.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The research complies with the NeurIPS Code of Ethics. The work involves numerical experiments on publicly available problems without human subjects or sensitive data. Ethical considerations such fairnsss, privacy, and misuse applicable in this context.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
17

---

Answer: [Yes]
Justification: The paper discusses the relevance of constraint satisfaction in safety-critical applications such energy systems, where ensuring feasible and physically consistent outputs is essential. While the work has no direct societal deployment, it contributes positively by enhancing reliability in machine learning models.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question:
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification:
The paper does not use existing assets.
Guidelines:
The answer NA means that the paper does not existing assets.
18

---

The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the datasets.paperrsst paperswi thcode. cow datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors are encouraged to reach out to the asset's ss creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [NA]
The paper does not introduce new datasets or pretrained models. All assets used are either synthetic data generated under specified uncertainty levels standard benchmark systems.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The research does not involve crowdsourcing, surveys, or experiments with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for rescarch with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
19

---

Answer:
[NA]
Justification:
No IRB approval is needed as the study does not involve human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: No large language models (LLMS) were used in the core methodology. ex- periments, scientific analysis of the research. Any assistance from LLMs limited to non-substantive editing.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy http://errrippeceoooo//eeeeeee/e for what should or should not be described.
20

---

A Handling Equality Constraints
To enforce equality constraints in the oulput of the task network, we orthogonally project its uncor- rected output onto the affine subspace defined by the constraints. This projection can be formulated the following quadratic program 6
2 10 (x)
$122
122
minimize
(15)
subject to G(z)y- g(x), (A)
where (T is the uncorrected task network output, and À denotes the Lagrange multipliers associated with the equality constraints.
This problem admits the following closed-form solution:
(x) Nx G(x)) (TN)),
(16)
where the optimal Lagrange multipliers are given by:
XTN (x) (G(=)G(=)))
(G(x) "G())) 9()))
(17)
Alternatvely,, using the projection matrix G(x), the projected output can be written as:
(x) (1 G(x)G(x)) (x) G(z)g(=),
(18)
where:
G(x) = G(=)==(()) (G())))))
Thus, G(z) acts projection onto the row space G(z), ensuring that the output strictly (x) (x) g(x) for any input x.
satisfies G(r) TN
B Proofs
B.1
Additional Technical Results
Lemma 1 (Homogeneous S-Lemma 34 Lemma 3.1]). Let P,S E 3 be synmetric matrices and suppose there exists R such that Sa > 0. Then, the following statements are equivalent:
TP 1. For all x E R if Pr 2 0, then x Sx 2 0.
2. There exists X 2 0 such that S 2 XP.
Proposition 1 29 Proposition 6)). Consider the following two statements for some fixed Sk;
(i) YE R with À 2 0 and - - A3P; - 11 0.
(ii) x Sa & P-a.s.
For any E 1, (i) implies (ii). The converse implication holds if = 1.
Proposition 29 Proposition 1]). Let X = {r E R Px p} be nonempty, compact polyhedron defined by P e Blik and p E Rd. Let t R be given. Then, for any z E RR the following statements are equivalent:
(i) z TTx for all X.
(ii) 3X R², such that PT) X p. A2
B.2 Proofs
Proof of Theorem
By Assumption
for any 0, there exists ff E Frask such that
sup |1 (x) f" (x)ll <.
x
21

---

Let r(x) = (x) (x) denote the residual. Since (2) E C(x) and C(x) is convex, there exists (x) E Fsate such that the point
[x(x) = (1 - ao)00 (x) f
SN (x)
remains in C(z), for a sufficiently small a > 0, due to the convexity of C(x) and the density of the convex hull of safe outputs in C(x) (Assumption 2
Since convex combinations are continuous and preserve continuity, fu E C(K, R"), and by choosing sufficiently small, the deviation from f* remains bounded by Hence,
sup [+()) - (x))l =EX
and L (x) C(x) for all X, completing the proof.
Proof of Proposition ! have:
(i) + (ii): Select any I E IR". Under the assumptions of statement (i), we
0 05x
S Σ j=1
A3Ps 11 x ST Σ
A5x Pje - 6||2|1?
5=1
Rearranging:
x Sv Xyx Pfæ: + t||=||?
Since A3 2 0 and assuming that x. Pys: 2 0 P-a.s., it follows that:
S +||=||
Since the choice of was arbitrary, this establishes statement (it).
(ii) (1) ifl = When 1 1, statement (W) implies:
x T Sr 2 / for all æ E Re with x T P1x 2 0.
This can be rewritten as:
x T =
Assuming that the set xx:x" Pix 0} is nonempty, closed, and has nonempty relative interior (as implied by the support of P), the homogeneous S-Iemma Tapplies. Thus, there exists A 2 such that:
$ P1 - t 0.
(ii) (i) ifl 1: When > the result does not hold in general, since the set:
{o R Rd
P,z 2 0, Vj 1.....
may be nonconvex, and thus the S-lemma fails to extend to multiple inequalities. In particular, universal X 2 0 exists in general such that the LMI in (1) is implied by (it).
Proof of Proposition We aim prove the equivalence between the two statements.
(i) + (ii): Assume that = x 2 for all X. By the definition of X, this is equivalent to:
min
{ x Px P222)21 2t.
WER
This is linear program, whose dual is given by:
(p T A x= 2= A2
max
AER
Since X' is nonempty and compact (and thus the primal is feasible and bounded), strong duality holds, yielding:
min 2 T
max X.
ZEX
X20, PT A=
22

---

Thus, the primal optimal value is at least t if and only if there exists À 2 0 such that PT A = 2ndd This proves statement (ii).
p p P' 2
(ii) (i) Assume there exists À 2 0 such that PT X and p TT À 2 t. Then, for any E N:
x X Px App p 1t.
Thus, axx 2 ł for all E X.
This proves the equivalence.
C Details on Numerical Experiments
C.1 Evaluation Metries
We evaluate the performance of all methods using the following metrics:
Optimality Gap Measures the percentage deviation of the predicted solution ý from the optimal solution y" obtained by the optimizer (Gurobi). Defined as:
c. c--
Optimality Gap = 100
(19)
cTy² 1"
where e is the objective cost vector. A lower gap indicates closer-to-optimal prediction.
Equality Violation Quantifies the relative violation the equality constraints, normalized by the magnitude of the right-hand side:
Equality Violation G( g(0))l2 l1g(2)]12
(20)
Inequality Violation Quantifies the degree to which the predicted solution violates the incquality constraints, measured using the Euclidean norm of the positive part of the violation:
Inequality Violation |max (H h(1),
(21)
+ [1(x)
Inference Time Measures the average runtime in milliseconds required to generate prediction for given input. For methods using iterative procedures (e.g., DC3, APM, EAPM), the inference time includes both the forward pass and the correction steps. For the optimizer, it corresponds to the solve time reported by Gurobi. All times were measured over the held-out test set, averaging across all samples.
Dataset Generation For each problem, we generated 100 samples of the input uniformly sampled from the uncertainty set X. The reported metrics are averaged over these test samples. For all methods, hyperparameters were selected based on validation set and fixed across problem sizes to ensure fairness.
C.2 Details Hyperparameter Tuning
The ground truths for all instances were solved using the Gurobi solver on an Apple M2 Pro CPU and 32GB of RAM. All neural networks were trained on NVIDIA T4 Tensor Core GPU with parallelization using Py Torch. The solve time reported in Table ?? was averaged by passing test instances separately, rather than dividing the solve time of the batched instances by batch size, as the GPU execution time does not scale linearly with batch size.
The following configurations and hyperparameters fixed throughout all experiments and all methods, based on preliminary experimentation to confirm the proper convergence of training.
Epochs: 300 for DCOPF, 500 for portfolio optimization
Optimizer: Adam
Learning rate: 10
23

---

Batch size: 64
Hidden layer number: 2
Hidden layer size: 256
Activation: ReLU
Batch normalization: True
Feasibility method stopping tolerance: 10
Feasibility method maximum iterations: 300
DC3 correction procedure momentum: 0.5
DC3 correction learning rate: 10
The proposed method does not require any additional byperparameters. However, we found empiri- cally that pre-training the task network to minimize the mean squared error (MSE) with respect to the safe network output f (z)) serves as an effective initialization strategy. This approach reduces the risk of the model getting stuck in poor local minima in specific instances, without degrading performance in other cases, shown in Table To ensure fair comparisons with other methods and with the proposed method trained from scratch, we adopt 100 pre-training epochs followed by 200 training epochs for the DC-OPF experiments, and 150 pre-training epochs followed by 350 training epochs for the portfolio optimization task.
Table 3: Results on both tasks over the held out test datasct. The optimality gaps, equality and inequality violations are shown mean (worst) values, and the values of time are indicated as the average solve time milliseconds (average iterations required by the method).
Method
Optimality Gap Equality Violation Inequality Violation Time
Portfolio optimization problem: 16, meg Mineg 9, k
10
w/ pre-training
2.35(6.37)
0.000X0.000)
0.000(0.000 0.000(0.000)
2.8(1)
w/o pre-training
3.14(6.36)
0.000X0.000)
2.8(1)
DC-OPF 14-bus system: 123, meg = 38, Mineq
84, A: 14
w/ pre-training
0.00(0.00)
0.000X0.000) 0.000X0.000)
0.000(0.000)
2.1(1)
w/o pre-training
0.00(0.00)
0.000(0.000)
2.1(1)
DC-OPF 30-bus system:
245, may
76, Mineq
168, k = 30
w/ pre-training
0.00(0.00)
0.000X0.000)
0.000(0.000)
2.2(1)
w/o pre-training
10.60(17.63)
0.00000.000)
0.000(0.000)
2.2(1)
DC-OPF 57-bus system:
468, may 141, Miney 324, k 57
w/ pre-training
0.21(0.68)
0.000X0.000) 0.00X0.0000
0.000(0.000)
2.2(1)
w/o pre-training
0.23().72)
0.000(0.000)
2.2(1)
DC-OPF 118-bus system:
1126, may
340, Mineg
768,
118
w/ pre-training
1.27(2.00)
0.00X00.000)
0.000(0.000)
2.3(1)
w/o pre-training
1.15(2.65)
0.000000000
0.000(0.000)
2.3(1)
DC-OPF 200-bus system:
1527, miy 452, mineg 1044,
200
w/ pre-training
0.99(1.78)
0.000X0.000)
0.000(0.000)
2.5(1)
w/o pre-training
(1.35)
0.000X0.000)
0.000(0.000)
2.5(1)
The ALM method and the DC3 method were trained using a soft loss function with additional penalty factor, which penalizes the violation of inequality constraints when the maximum number of iterations insufficient to reduce the violation below the specified tolerance. Table we present the following tuning ranges for these methods' hyperparameters with the final selected values in bold.

---

Table 4: Penalty factor selection for ALM and DC3
Dataset
DC3
ALM
DC-OPF 14-bus
0, 5000
0. 5000
DC-OPF 30-bus
0, 5000
0, 5000
DC-OPF 57-hus
0, 5000
0, 5000
DC-OPF 118-hus
0, 500, 5000
0, 5000
DC-OPF 200-hus
0, 500, 5000
0, 5000
Portfolio optimization problem
0, 0.5, 5
0. 0.5
Table 5: Results the linear formulation of the DC-OPF problem over the held out test dataset. The optimality gaps, equality and inequality violations are shown as mean (worst) values, and the values of time are indicated the average solve time milliseconds (average iterations required by the method).
Method
Optimality Gap
Equality Violation Inequality Violation
Time
DC-OPF (LP) 14-bus system: 123, meg
38, Mine
84, k 14
Optimizer
0.00(0.00)
0.000(0.000)
0.000(0.000)
0.6
Proposed
0.00(0.00)
0.000(0.000)
0.000(0.000)
2.1(1)
APM
0.00(0.00)
0.000(0.000)
0.000(0.000)
42.8(61)
DC3
0.27(3.71)
0.000(0.000)
0.004(0.053)
315.6(291)
LDR
31.15(38.39)
0.000(0.000)
0.000(0.000)
0.9(1)
DC-OPF (L.P 30-bus system: n 245, meg
76, Mineg = 168, & = 30
Optimizer
0.00(0.00)
0.000(0.000)
0.000(0.000)
0.98
Proposed
0.00(0.00)
0.000(0.000)
0.000(0.000)
2.2(1)
APM
0.00(0.00)
0.000(0.000)
0.000(0.000)
65.0(93)
DC3
0.72(2.34)
0.000(0.000)
0.014(0.045)
321.5(295)
LDR
10.20(17.19)
0.000(0.000)
0.000(0.000)
0,9(1)
DC-OPF (LP) 57-bus system: ı = -4446,
141, Wiinnn 324, k 57
Optimizer
0.00(0.00)
0.000(0.000)
0.000(0.000)
2.06
Proposed
0.21(0.68)
0.000(0.000)
0.000(0.000)
2.2(1)
APM
0.00(0.65)
0.000(0,000)
0.010(0.041)
10555(154)
DC3
0.04(1.04)
0.000(0.000)
0.026(0.093)
321.2(292)
LDR
8.86(1 1.78)
0.000(0.000)
0.000(0.000)
0.9(1)
DC-OPF (LP) 118-bus system: = 1126, meq
340, Miney 768, & 118
Optimizer
0.00(0.00)
0,000(0.000)
0.000(0.000)
4.43
Proposed
1.27(2.00)
0.000(0.000)
0.000(0.000)
2.3(1)
APM
0.07(0.16)
0.000(0.000)
0.003(0.018)
123.9(172)
DC3
1.95(8.09)
0.000(0.000)
0.999(1.513)
335.6(300)
LDR
21.51(22.79)
0.000(0.000)
0.000(0.000)
0.9(1)
DC-OPF (LP) 200-bus system: = 1527, meg
452, mineq 1044, A 200
Optimizer
0.00(0.00)
0.000(0.000)
0.000(0.000)
5.47
Proposed
0.99(1.78)
0,000(0.000)
0.000(0.000)
2.5(1)
APM
0.72(4.98)
0.000(0.000)
0.017(0.057)
217.9(300)
DC3
18.89(21.10)
0.000(0.000)
0.210(0.235)
338.8(300)
LDR
11.54(12.91)
0.000(0.000)
0.000(0.000)
0.9(1)
C.3
Additional results for the DC-OPF problem task
C.3.1
Linear programming model results
Analysis of Results DC-OPF (LP) Jhoos that the proposed method consistently achieves zero constraint violations across all tasks, matching the optimizer and outperforming DC3, which exhibits increasing inequality violations as problem size grows. In terms of optimality, the proposed method significantly improves over LDR, maintaining gaps below 1% in large-scale DC-OPF cases, while LDR incurs gaps above 10% and up to 48% in the portfolio task. This highlights the benefit of
25

---

combining the task and safe networks to balance accuracy and feasibility. In terms of runtime, the proposed method achieves inference times under 3 ms across all tasks, providing orders-of-man mangnitude speedups over DC3 and APM, which require hundreds of iterations and runtimes exceeding 300 ms per instance. While L.DR remains the fastest approach (< ms), does so at the expense of large optimality gaps. Importantly, the proposed method is also significantly faster than the optimizer, whose runtime grows with problem size. These results confirm that the proposed method offers the best trade-off between feasibility, accuracy, and efficiency.
C.3.2 Quadratic programming model additional results
Table Remaining results on the quadratic formulation of the DC-OPF problem over the held out test dataset. The optimality gaps, equality and inequality violations are shown mean (worst) values, and the values of time are indicated as the average solve time milliseconds (average iterations required by the method).
Method
Optimality Gap Equality Violation Inequality Violation Time
DC-OPF (QP) 14-bus system: n 123, meq
38, Mineg = 84, k:
14
Proposed
0.00(0.00)
0.000(0.000)
0.000(0.000
2.3(1)
Optimizer (Gurobi)
0.00(0.00)
0.000(0.000)
0.000(0.000)
0.5
Optimizer (OSQP)
0,00(0.00)
0.000(0.000)
0.000(0.000)
0,6
Post projection (Gurobi)
0.37(1.84)
0.000(0.000)
0.000(0.000)
1.6
Post projection (OSQP)
0.37().84)
0.000(0.000)
0.000(0.000)
1.2
APM
0.00(0.00)
0.000(0.000)
0.000(0.000)
45.8(62)
EAPM
0.00(0.00)
0.000(0.000)
0.000(0.000)
4.8(2)
DC3
0.13(1.64)
0.000(0.000)
0.003(0.022)
331.6(291)
DC-OPF (QP) 30-bus system: n
245, meq
76, Mineg
168, A:
30
Proposed
0.00(0.00)
0.000(0.000)
0.000(0.000)
2.3(1)
Optimizer (Gurobi)
0,00(0.00)
0,000(0.000)
0.000(0.00)
1.0
Optimizer (OSQP)
0.00(0.00)
0.000(0.000)
0.000(0.
1.1
Post projection (Gurobi)
0.26(1.16)
0.000(0.000)
0,000(0.000)
2.3
Post projection (OSQP)
0,26(1.16)
0.000(0.000)
0.000(0.000)
1.0
APM
0.00C0.00)
0.000(0.000)
0.000(0.000)
69.0(95)
EAPM
11.13(16.19)
0.000(0.000)
0.000(0.000)
4.4(2)
DC3
0.57(3.73)
0.000(0.000)
0.011(0.067)
337.3(293)
DC-OPF (QP) 57-bus system:
168, 141, mine 324, A 57
Proposed
0.10(0.36)
0.000(0.000)
0.000(0.000)
2.5(1)
Optimizer (Gurobi)
0.00(0.00)
0.000(0.000)
0.000(0.000)
2.1
Optimizer (OSQP)
0.00(0.00)
0.000(0.000)
0.000(0.000)
2.6
Post projection (Gurobi)
0.11(0.65)
0.000(0.000)
0.000(0.000)
4.0
Post projection (OSQP)
0.11(0.65)
0.000(0.000)
0.000(0.000)
2.9
APM
0.03(0.15)
0.000(0.000)
0.000(0.003)
73.1(99)
EAPM
0.09(0.73)
0.000(0.000)
0.000(0.003)
9.2(5)
DC3
0.04(1.17)
0.000(0.000)
0.000(0.004)
306.7(264)
Analysis of Results DC-OPF (QP) Table 6 shows that the proposed method achieves zero constraint violations across all systems, matching the optimizer and outperforming DC3, which exhibits small but non-negligible violations. In terms of optimality, it maintains gaps below 0.1%. closely matching Gurobi and OSQP and improving over post-proy post-projection baselines. APM and EAPM also ensure feasibility but require significantly more iterations and runtime- especially APM, which exceeds 70 ms on larger systems. In contrast, the proposed method achieves inference times under 3 ms, offering an prder-of-magnituue methods are faster than APM/EAPM but still slower and slightly less accurate than the proposed method. DC3 remains the slowest, requiring over 300 iterations. Overall, the proposed method offers the best trade-off between fcasibility, optimality, and efficiency, scaling effectively with system size while maintaining high solution quality.
speedup while retaining near-optimality. Post-projection
26


Reviews:
[{"review_id": "orrbYbheZ3", "reviewer": "Reviewer_Xhps", "summary": "This paper presents a framework to enforce linear input-output constraints during neural neural network training. The idea is to use two separate components: the task network, trained to minimize prediction loss, and a \"safe\" network, designed to ensure feasibility of the constraints. The output is a convex combination of the two. The approach is evaluated on benchmark regression tasks.", "questions": "Please comment on the sizes of the neural network.\n\nHow woudl the approach apply to ACAS-Xu (https://arxiv.org/abs/1702.01135) which has a set of input-output constraints.", "limitations": "no", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Pros:\n\n* Interesting paper that achieves safety constraints at a seemingly low cost.\n\n* Interesting theoretical analysis of the properties of the framework.\n\n* Practical evaluation on regression tasks.\n\nCons:\n\n* Although the authors claim scalability, it is unclear if the approach would scale to large networks and large input domains. Also it is unclear if it extends beyond regression.\n\n* It is unclear if the proposed approach does indeed guarantee absence of violations of constraints.", "was_revised": false, "final_justification": "", "rebuttal": "### Comments on Weaknesses\n\n> **Reviewer Comment**\n> Although the authors claim scalability, it is unclear if the approach would scale to large networks and large input domains. Also it is unclear if it extends beyond regression.\n\n**Response**\n**Scalability.** Our architecture is scalable at inference time, as constraint enforcement relies only on efficient matrix-vector operations, specifically, projection and convex combination steps, as shown in Section 3.5 and Table 1. The main computational cost arises in determining the weights of the safe network, which must satisfy hard input-dependent constraints over $\\mathcal{X}$. However, the size of the associated optimization problems scales *polynomially* with the dimensions of the input and output spaces, ensuring tractability in principle.\nTo further improve scalability in practice, we outline in Section 5 several promising directions, including exploiting sparsity in the constraint matrices and leveraging decomposable or separable structure in the constraints. Additionally, solving large-scale safe network optimization problems could benefit from modern GPU-based solvers, which have demonstrated significant speedups for solving exa-scale optimization problems. We will clarify in the manuscript that the scalability challenge lies primarily in determining the parameters of the safe network, and that both algorithmic structure and hardware acceleration can mitigate this limitation.\n\n**Beyond regression.** While the current experiments focus on regression tasks, the proposed framework naturally extends to both classification and reinforcement learning (RL) settings.\nIn classification settings, the proposed architecture can be applied either in logit space or by imposing constraints directly on the softmax outputs. Constraints involving probability distributions or statistical distances, such as total variation distance, can often be expressed as linear constraints and are therefore naturally compatible with our framework.\nIn reinforcement learning, the proposed architecture is directly applicable to enforcing hard, state-dependent constraints on the action space. In actor-critic algorithms such as DDPG, where the policy is deterministic and outputs a continuous action, the projection and blending architecture can be incorporated into the actor network with minimal overhead, as long as the state space is representable in the structured form described in our paper (e.g., a polytope or ellipsoidal set). This allows the agent to output actions that satisfy state-dependent linear constraints exactly at every time step.\nFor stochastic policy gradient methods such as PPO, incorporating our architecture is more involved. In particular, the projection step alters the support and shape of the action distribution, making it nontrivial to compute log probabilities required for policy optimization and entropy regularization. Addressing this issue would require adapting the policy parametrization or defining a constrained distribution class over the feasible set—an important and challenging direction for future work.\n\nTo summarize these points clearly in the paper, we will add the following to the end of Section 4:\n\n> *Limitations and extensions.* While the proposed framework is demonstrated on regression tasks, it extends naturally to classification (e.g., via constrained logits or softmax outputs) and actor-critic reinforcement learning algorithms such as DDPG, where constrained actions can be enforced in a state-dependent manner. In classification settings, the architecture can operate in logit space or enforce constraints directly on the softmax outputs. Constraints involving distributions or statistical distances, such as total variation distance, can often be written as linear constraints and are therefore naturally supported by our method. Extension to stochastic policy gradient methods like PPO is more involved, as the projection step alters the action distribution and complicates log-probability computation. Although solving the safe network problem scales polynomially with the problem size, its solution can be made more efficient and scalable by exploiting sparsity, decomposability, and GPU-based optimization solvers.\n\n---\n\n> **Reviewer Comment**\n> It is unclear if the proposed approach does indeed guarantee absence of violations of constraints.\n\n**Response**\nWe appreciate the reviewer for raising this important point. Yes, the proposed method guarantees strict satisfaction of all input-dependent constraints—both equalities and inequalities—at training and inference time, by construction.\n\n**Equality constraints** are enforced through a closed-form projection of the task network output onto the affine subspace defined by $G(x) f = g(x)$, as described in equation (3). This projection ensures that the task network output satisfies all equality constraints exactly for any input $x \\in \\mathcal{X}$. Moreover, by construction, the safe network output satisfies both equality and inequality constraints for all $x \\in \\mathcal{X}$. As a result, the outputs of both networks, as well as any convex combination between them, lie in the affine subspace defined by the equality constraints $G(x) f = g(x)$.\n\n**Inequality constraints** are enforced via a convex combination of the projected task network output and the output of a safe network that is feasible by design. The interpolation parameter $\\alpha$ is computed using equation (4) to be the smallest value in $[0,1]$ that ensures all inequalities $H(x) f(x) \\leq h(x)$ are satisfied. This follows directly from the convexity of the constraint set $\\mathcal{C}(x)$ and the fact that both endpoints of the interpolation lie within or outside it in a controlled way.\nWe have clarified this point more explicitly in Section 3.2 to make the guarantee of hard constraint satisfaction more prominent.\n\n---\n\n### Questions\n\n> **Reviewer Question**\n> Please comment on the sizes of the neural network.\n\n**Response**\nThank you for the suggestion. In the supplementary material (see Section C.2), we provided the architecture details, including the number of layers, hidden units, and activation functions. We chose this configuration based on validation performance. If the paper is accepted, we will include a brief summary of the architecture in the main paper to improve clarity.\n\n---\n\n> **Reviewer Question**\n> How would the approach apply to ACAS-Xu ([https://arxiv.org/abs/1702.01135](https://arxiv.org/abs/1702.01135)) which has a set of input-output constraints?\n\n**Response**\nWe thank the reviewer for highlighting the ACAS Xu case study. While this benchmark provides a compelling application of constrained neural networks, our approach is not directly applicable to the properties $\\phi_1$–$\\phi_{10}$ described in the Reluplex paper. These properties are formulated as logical implications, i.e., \"if the input lies within a certain region, then a desired property must hold for the output\", which cannot be encoded as direct linear constraints on inputs and outputs.\nIn contrast, our method is designed to enforce linear equalities and inequalities on network outputs, either unconditionally or over convex subsets of the input space. Therefore, while we can enforce fixed output preferences in specific regions, we cannot capture implication-based properties without resorting to additional logic-based mechanisms."}, {"review_id": "J444xILo7J", "reviewer": "Reviewer_nG53", "summary": "This paper proposes a framework to enforce linear equality and inequality constraints in neural networks. It combines a task network for accuracy with a safe network trained using optimization principles, producing feasible predictions through convex combination.\n\nFor the task network, the authors use the Euclidean projection to project the neural network outputs onto the region of linear constraints. Since only linear constraints are involved in this part, closed-form expressions can be derived. In addition, the authors reformulate the training of the safe network as an optimization problem and reformulate the corresponding problem as an SDP or LP when the safe network are confined to linear functions.\n\nCase studies on DCOPF and portfolio optimization demonstrate certain advantages of the proposed method over some existing approaches.", "questions": "Q1. Separating the constraint set into equality and inequality subsets for individual treatment is a common approach in optimization algorithms. For example, OSQP employs the Alternating Direction Method of Multipliers (ADMM), where equality constraints are transformed into a quadratic equality-constrained problem to obtain a closed-form solution (similar to the method described in Appendix A of the paper). Since this step involves solving a complex system of linear equations, both approaches have a computational complexity of $O(m^3)$. For inequality constraints, OSQP uses Euclidean projection. Through alternating iterations between these two steps, OSQP gradually converges from an initial point toward feasibility and optimality. The reviewer wants to point out that the APM used in the numerical experiments in Section 5 appears to differ significantly from ADMM as used in OSQP, because ADMM also includes additional Lagrangian multipliers and quadratic penalty terms that enhance convergence performance. Therefore, it is possible that the authors may have employed a relatively weak baseline.\n\nQ2. Considering that a linear system with complexity $O(m^3)$ needs to be solved anyway, instead of using the proposed method, would it be faster to use an unconstrained neural network to predict an initial value and then perform a warm start in Gurobi Simplex by setting its `PStart` or in OSQP by setting its initial values?\n\nQ3. DCOPF on small-scale systems can be easily solved by commercial solvers, so applying the proposed method offers limited value, especially considering that the method may also introduce optimality gaps. How does the proposed method perform on large-scale systems, such as power grids with several thousands or even ten thousand buses?\n\nQ4. The solution times for APM and DC3 shown in Table 2 are quite long, which seems difficult to understand for a small-scale DCOPF problem. Have the authors compared with other optimization-based constrained learning methods, such as CvxpyLayers or OptNet?\n\nQ5. Based on Eq. (4), it seems that $\\alpha$ is related to $s_i^{\\rm{TN}}(x)$ and $s_i^{\\rm{SN}}(x)$, and $s_i^{\\rm{TN}}(x)$ and $s_i^{\\rm{SN}}(x)$ are related to $\\theta$ and $\\phi$, namely the parameters of the task network and safe network. Should $\\alpha$ be denoted as $\\alpha(\\theta,\\phi)$ and participate in backpropagation? It seems that currently $\\alpha$ participates in backpropagation according to `alpha = torch.max(masked_alphas, dim=-1).values` in line 331 of `models.py`.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\nThe authors present their ideas clearly, with well-written content and detailed mathematical derivations.\n\nWeaknesses:\nThe authors used a small-scale case study and compared with weak baselines, making it difficult to fully demonstrate the advantages of the proposed method. The results in Table 2 do not show an order-of-magnitude improvement in solution time for the proposed method compared to Gurobi. It seems that using the proposed method does not provide much value.", "was_revised": true, "final_justification": "The authors' response has addressed my concerns, and therefore I have increased my score.", "rebuttal": "**Comment**  \n> The authors used a small-scale case...\n\n**Response**  \nWe appreciate the reviewer’s concerns. In response, we have expanded the experiments to include stronger baselines: (i) Gurobi and OSQP warm-started with unconstrained neural network outputs, (ii) projection-based methods using QP solvers, and (iii) the extended alternating projection method (EAPM), a state-of-the-art iterative approach.\n\nWhile runtime gains may seem modest on small problems, our method enables millisecond-level, one-shot inference with guaranteed constraint satisfaction, avoiding iterative overhead and making it well-suited for real-time and embedded applications. We also evaluate problems with up to 1,527 variables and 1,496 constraints, sizes comparable to or larger than those in related work, such as [3], [5], [6], [8], [10], [12], [15], [17], [18], and [21], demonstrating the method’s practicality for domains like robotics, power systems, and constrained control.\n\n\n### Questions\n\n**Q1**\n\nWe thank the reviewer for highlighting the distinction between the APM used in our original experiments and ADMM-based solvers such as OSQP. We agree that ADMM introduces dual variables and penalty terms that can enhance convergence properties. However, our proposed method is not an iterative solver and does not rely on convergence guarantees. Instead, it performs one-shot inference via a convex combination of two networks, ensuring feasibility by construction. \n\nTo strengthen our evaluation, we have added experiments comparing against a wider set of baselines. These include (1) solving the original optimization problems using Gurobi and OSQP with warm-starts from a trained unconstrained neural network, (2) training an unconstrained neural network and projecting its outputs onto the feasible set via a QP solved using Gurobi and OSQP, and (3) the state-of-the-art Extrapolated Alternating Projection Method (EAPM). As shown in the new experiments, our proposed approach consistently achieves zero constraint violations and competitive optimality gaps while maintaining millisecond-level inference times. It outperforms NN + post-projection methods in both optimality and speed, and achieves comparable or faster inference than OSQP and Gurobi for systems with hundreds to over one thousand variables. These results confirm that our architecture offers a practical, efficient, and constraint-satisfying alternative to traditional and projection-based baselines.\n\n**Q2**\n\nWe thank the reviewer for the suggestion. We implemented the proposed warm-start baseline by training an unconstrained neural network (NN) to predict a solution, which is then projected using Gurobi or OSQP, warm-started with that prediction. We also compared against directly solving the original optimization problem with warm starts. Due to space limitations, the results for the largest instance are included at the end of this rebuttal, whereas the full results of the new experiments are included in our response to Reviewer `kQGn`.\n\nThe key findings of the numerical experiments are as follows:\n\n* Our proposed method achieves low optimality gaps across all cases, with an almost-fixed-cost inference time (1 forward pass and one convex combination) of 2-3 ms per instance.\n* Warm-starting solvers with a neural network prediction followed by post-projection yields feasible solutions, but results in higher optimality gaps (e.g., up to 1.88 in the 118-bus case), and incurs inference times that increase with system size (e.g., up to 13.2 ms with Gurobi in the 200-bus case versus 2.6 ms incurred by the proposed approach).\n* Solving the original QP with warm-started Gurobi or OSQP does achieve optimality, but comes at the cost of slower inference.\n\nFinally, we emphasize that the cost of solving a linear system arises *only if* the equality constraint matrix \\$G(x)\\$ is input-dependent. In the common case where \\$G\\$ is fixed, we compute the projection matrix \\$\\bar{G}\\$ once before training, and reuse it throughout, further reducing the inference complexity.\n\n**Q3**\n\n\nWe thank the reviewer for this question. Our method is not intended to replace optimization solvers in centralized or offline workflows, but rather to enable fast and feasible inference in settings where runtime solver calls are impractical.\n\nThe benefits of our approach are particularly pronounced when the objective is nonconvex and global solvers are intractable. For example, in the nonconvex, linearly constrained problem included in our new experiments, our method achieves a 6x speedup over Ipopt even though the problem involves only 100 variables, 50 equality constraints, and 50 inequality constraints. While commercial solvers efficiently solve small linear programs, our method demonstrates superior speed in finding feasible solutions for problems with over 1,000 variables, and its advantage becomes increasingly significant as problem size grows.\n\nWe also evaluate our method on instances involving over 1,500 variables and nearly 1,500 constraints. Our approach consistently achieves zero constraint violations and inference times in the millisecond range. These problem sizes are comparable to or exceed those studied in prior work, including \\[3], \\[5], \\[6], \\[8], \\[10], \\[12], \\[15], \\[17], \\[18], and \\[21], and are representative of real-world applications in power systems, robotics, and constrained control.\n\nThe primary scalability challenge lies in determining the parameters of the safe network. We explicitly acknowledge this in our conclusion and propose several strategies to address it, including exploiting sparsity in the constraint matrices, leveraging decomposable constraint structure, and utilizing GPU-accelerated solvers for scalable LDR training.\n\nLastly, we would like to note that although our case studies focus on parametric LPs, the proposed method is broadly applicable to any setting where linear constraints need to be enforced in deep learning models. For example, it can be used in reinforcement learning problems to ensure that state-action pairs satisfy specified linear constraints, or in applications where a deep learning model approximates an expensive simulator while maintaining feasibility with respect to linear constraints. In these cases, since the problem itself is not an optimization problem, solvers like Gurobi are not applicable. Our method provides a significantly faster alternative to existing approaches for enforcing linear constraints in neural networks, as demonstrated by our experimental results. We have left these additional case studies for future work.\n\n**Q4**\n\nWe thank the reviewer for this constructive suggestion. We did experiment with `Cvxpylayers`, but we observed several issues that made it unsuitable for inclusion in the final results. In particular, the inference times were significantly higher than other approaches, often by one to two orders of magnitude. Moreover, `Cvxpylayers` does not strictly enforce inequality constraints during inference, due to the relatively high default solver tolerances. Attempts to tighten these tolerances often led to numerical instability or solver failures, especially for larger instances. For these reasons, we opted not to include `Cvxpylayers` in our results.  \nHowever, we have incorporated several strong optimization-based baselines as suggested. These include training an unconstrained neural network followed by post-projection onto the linear constraint set using state-of-the-art QP solvers such as OSQP and Gurobi (with warm starts), as well as EAPM, a recent variant of alternating projections designed for improved convergence. These methods are described in detail in the revised manuscript and evaluated comprehensively in the new experiments.\n\n**Q5**\n\nWe thank the reviewer for raising this insightful question. As noted, $\\alpha(x) $ is computed from the slack residuals $s_i^{\\mathrm{TN}}(x)$ and $s_i^{\\mathrm{SN}}(x)$, which depend on the outputs of the task and safe networks parameterized by $\\theta$ and $\\phi$, respectively. Therefore, $\\alpha(x)$ is indeed a function of these parameters. In this case, gradients do propagate through $\\alpha(x)$ in our implementation: PyTorch's `torch.max` operation supports automatic differentiation, and as confirmed by our computational graph, backpropagation flows through the selected (i.e., maximal) index in the slack ratio expression. This gradient, however, is sparse and non-smooth—only the constraint that achieves the maximum receives a nonzero gradient, while all others are effectively ignored. Consequently, the signal reaching the task network is limited in expressiveness, especially when many constraints are near-active. To clarify this in the manuscript, we have updated the notation of $\\alpha$ and the slack vectors to reflect their dependence on the network parameters, and we will add a clarifying paragraph in Section 3.2 and Section 6.\n\n### Results over the held out test dataset: DC-OPF (QP) 200-bus system\n\n| Method               | Optimality Gap | Equality Violation | Inequality Violation | Time       |\n|----------------------|----------------|---------------------|-----------------------|------------|\n| Proposed             | 1.07 (1.71)    | 0.000 (0.000)       | 0.000 (0.000)         | 2.6 (1)    |\n| Optimizer (Gurobi)   | 0.00 (0.00)    | 0.000 (0.000)       | 0.000 (0.000)         | 4.6        |\n| Optimizer (OSQP)     | 0.00 (0.00)    | 0.000 (0.000)       | 0.000 (0.000)         | 15.1       |\n| Post proj. (Gurobi)  | 0.23 (1.49)    | 0.000 (0.000)       | 0.000 (0.000)         | 13.2       |\n| Post proj. (OSQP)    | 0.22 (1.49)    | 0.000 (0.000)       | 0.000 (0.000)         | 7.4        |\n| APM                  | 8.55 (9.45)    | 0.000 (0.000)       | 0.166 (0.181)         | 225.3 (300)|\n| EAPM                 | 3.88 (7.95)    | 0.000 (0.000)       | 0.000 (0.000)         | 78.7 (50)  |\n| DC3                  | 18.74 (20.61)  | 0.000 (0.000)       | 0.164 (0.189)         | 360.5 (300)|"}, {"review_id": "TDzTLWxamN", "reviewer": "Reviewer_kQGn", "summary": "The paper proposes a new method to enforce linear input-output (i.e. input dependent) constraints for ML models. The key idea is to split the task of prediction subject to constraints into two halves, one half is represented by a task network which tries to output the best prediction, and the other half is represented by a “safe” network which always outputs a feasible point. The key insight is that the “best” (i.e. closest to the task network output) feasible point on the line segment between the task and safe outputs can be calculated in closed form, which is cheaper than e.g. projecting the task network output onto the feasible region. Compared to existing work, the inference and training costs are very low while maintaining good accuracy and zero constraint violations.", "questions": "1. What would happen if the constraints are infeasible? (Potentially for specific inputs)", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths \n\n1. The method achieves a favorable tradeoff between optimality gap and inference speed. Inference is very fast while producing high quality solutions. Competitors are either 1-2 orders of magnitude slower to achieve better results or are 2-3x faster while achieving significantly worse results. \n2. The evaluation is thorough with four baselines, including an exact solution obtained with Gurobi.\n3. The paper also includes an approximation theorem showing that the proposed architecture is a universal approximate of constrained functions, which is a neat result. \n\n## Weaknesses\n\n1. Training may be expensive. Training requires solving a semidefinite program to form the safe network. SDPs are harder in general than projection onto the feasible set of linear constraints, so it is not clear if training is more or less expensive than the alternatives. \n2. The functional form restrictions in the decision rules formulation may be too strong. It is possible that the linear functional class may not be expressive enough to solve more complex problems, but it is difficult to tell without experimental results. (This is acknowledges in the conclusion)\n3. Relatively few datasets are tested. Results are given for portfolio optimization and DC-OPF, but more variety would be ideal. I understand it may be difficult to find good datasets in this area. Perhaps the synthetic instances used in the DC3 paper could be used to give further evidence of the method’s effectiveness?", "was_revised": true, "final_justification": "The proposed method is elegant and theoretically sound. The evaluation is thorough, given the limited datasets available, and the proposed method strikes a highly desirable balance between inference speed and solution quality.", "rebuttal": "> Training may be expensive...\n\nWe thank the reviewer for raising this important concern. We clarify that a semidefinite program (SDP) is only required when the left-hand-side constraint matrices $G(x)$ and $H(x)$ are input-dependent. In this case, the SDP is solved once before training the task network, in order to construct a linear decision rule (LDR). When only the right-hand side $g(x)$ or $ h(x) $ is input-dependent (and the left-hand-side matrices are fixed), the LDR can be determined by solving a single linear program (LP). In both cases, the resulting SDP or LP grows polynomially with the input and output dimensions, and is solved offline, prior to training the task network. While this offline step can be computationally intensive in large-scale settings, it ensures that inference remains fast, deterministic, and constraint-satisfying without the need for runtime optimization. Moreover, this step can be significantly accelerated by exploiting sparsity, decomposable structure in the constraints, and by using modern GPU-accelerated solvers.\n\n> The functional form...\n\nWe agree that LDRs may be restrictive in expressive power when modeling highly nonlinear or nonconvex feasible sets. That said, it is important to emphasize that the use of LDRs is confined to the safe network, which acts as a fallback to guarantee feasibility. The task network, by contrast, is a standard deep neural network trained to optimize task performance, and is free to approximate highly expressive mappings. In this way, the overall architecture maintains high capacity while ensuring hard constraint satisfaction through a lightweight convex combination. Despite the conservativeness of the LDRs, our experiments demonstrate that the proposed architecture achieves competitive performance in terms of both accuracy and feasibility across all benchmarks, with optimality gaps often below 1\\% and zero constraint violations.\n\n> Relatively few datasets...\n\nWe thank the reviewer for this valuable suggestion. We agree that broader empirical coverage is important. In the revised version of the manuscript, we have significantly extended our experimental evaluation. In particular, we now include: (i) new results for DC-OPF with a quadratic objective function; (ii) comparisons against commercial and open-source optimizers (Gurobi and OSQP), both warm-started with a trained unconstrained neural network; (iii) post-projection of unconstrained neural network outputs onto the feasible set using the same warm starts; and (iv) an implementation of EAPM, a state-of-the-art baseline for hard constraint enforcement. These additions offer a comprehensive and rigorous assessment of the proposed method’s performance and scalability. Regarding the DC3 synthetic instances, we appreciate the suggestion and carefully examined their suitability. However, as described in Section 4.1 of the DC3 paper, these problems are constructed such that any variable projected onto the equality constraints automatically satisfies the inequality constraints. As a result, the safe network becomes inactive during training and the projection architecture reduces to a degenerate case. This makes those benchmarks ineffective for testing the benefits of our full framework, especially the safe network’s role in ensuring strict feasibility under inequality constraints.\nThe results of the new experiments are presented at the end of this rebuttal.\n\n## Questions\n\n**Q1** \n\nWe thank the reviewer for raising this important point. Our approach assumes that the linear constraint set is feasible for all $x \\in \\mathcal{X}$. This is explicitly addressed during the formulation of the safe network, where the parameters of the linear decision rule are obtained by solving an optimization problem, either an SDP or LP, depending on the structure of the constraints, that ensures feasibility holds almost surely over the input domain $\\mathcal{X}$. If, for some reason, the constraints are infeasible for a given input within the input space, then the finite-dimensional reformulations used to compute the linear decision rule (problems (10) or (14)) become infeasible, and no valid parameters can be found. In such cases, one practical remedy is to compute a conservative inner approximation of the input space such that the constraints are feasible for all $x \\in \\mathcal{X}$. Alternatively, one could partition the input space and learn separate linear decision rules for each region, ensuring feasibility holds locally within each partition. These strategies allow the method to retain feasibility guarantees even when global feasibility is not possible by a single linear decision rule.\n\n\n### Results on both tasks over the held out test dataset\n\n#### DC-OPF (QP) 14-bus system:\n\n| Method               | Optimality Gap | Equality Violation | Inequality Violation | Time       |\n|----------------------|----------------|---------------------|-----------------------|------------|\n| Proposed             | 0.00 (0.00)    | 0.000 (0.000)       | 0.000 (0.000)         | 2.3 (1)    |\n| Optimizer (Gurobi)   | 0.00 (0.00)    | 0.000 (0.000)       | 0.000 (0.000)         | 0.5        |\n| Optimizer (OSQP)     | 0.00 (0.00)    | 0.000 (0.000)       | 0.000 (0.000)         | 0.6        |\n| Post proj. (Gurobi)  | 0.37 (1.84)    | 0.000 (0.000)       | 0.000 (0.000)         | 1.6        |\n| Post proj. (OSQP)    | 0.37 (1.84)    | 0.000 (0.000)       | 0.000 (0.000)         | 1.2        |\n| APM                  | 0.00 (0.00)    | 0.000 (0.000)       | 0.000 (0.000)         | 45.8 (62)  |\n| EAPM                 | 0.00 (0.00)    | 0.000 (0.000)       | 0.000 (0.000)         | 4.8 (2)    |\n| DC3                  | 0.13 (1.64)    | 0.000 (0.000)       | 0.003 (0.022)         | 331.6 (291)|\n\n#### DC-OPF (QP) 30-bus system:\n\n| Method               | Optimality Gap | Equality Violation | Inequality Violation | Time       |\n|----------------------|----------------|---------------------|-----------------------|------------|\n| Proposed             | 0.00 (0.00)    | 0.000 (0.000)       | 0.000 (0.000)         | 2.3 (1)    |\n| Optimizer (Gurobi)   | 0.00 (0.00)    | 0.000 (0.000)       | 0.000 (0.000)         | 1.0        |\n| Optimizer (OSQP)     | 0.00 (0.00)    | 0.000 (0.000)       | 0.000 (0.000)         | 1.1        |\n| Post proj. (Gurobi)  | 0.26 (1.16)    | 0.000 (0.000)       | 0.000 (0.000)         | 2.3        |\n| Post proj. (OSQP)    | 0.26 (1.16)    | 0.000 (0.000)       | 0.000 (0.000)         | 1.0        |\n| APM                  | 0.00 (0.00)    | 0.000 (0.000)       | 0.000 (0.000)         | 69.0 (95)  |\n| EAPM                 | 11.13 (16.19)  | 0.000 (0.000)       | 0.000 (0.000)         | 4.4 (2)    |\n| DC3                  | 0.57 (3.73)    | 0.000 (0.000)       | 0.011 (0.067)         | 337.3 (293)|\n\n#### DC-OPF (QP) 57-bus system: \n\n| Method               | Optimality Gap | Equality Violation | Inequality Violation | Time       |\n|----------------------|----------------|---------------------|-----------------------|------------|\n| Proposed             | 0.10 (0.36)    | 0.000 (0.000)       | 0.000 (0.000)         | 2.5 (1)    |\n| Optimizer (Gurobi)   | 0.00 (0.00)    | 0.000 (0.000)       | 0.000 (0.000)         | 2.1        |\n| Optimizer (OSQP)     | 0.00 (0.00)    | 0.000 (0.000)       | 0.000 (0.000)         | 2.6        |\n| Post proj. (Gurobi)  | 0.11 (0.65)    | 0.000 (0.000)       | 0.000 (0.000)         | 4.0        |\n| Post proj. (OSQP)    | 0.11 (0.65)    | 0.000 (0.000)       | 0.000 (0.000)         | 2.9        |\n| APM                  | 0.03 (0.15)    | 0.000 (0.000)       | 0.000 (0.003)         | 73.1 (99)  |\n| EAPM                 | 0.09 (0.73)    | 0.000 (0.000)       | 0.000 (0.003)         | 9.2 (5)    |\n| DC3                  | 0.04 (1.17)    | 0.000 (0.000)       | 0.000 (0.004)         | 306.7 (264)|\n\n#### DC-OPF (QP) 118-bus system: \n\n| Method               | Optimality Gap | Equality Violation | Inequality Violation | Time       |\n|----------------------|----------------|---------------------|-----------------------|------------|\n| Proposed             | 1.10 (2.13)    | 0.000 (0.000)       | 0.000 (0.000)         | 2.5 (1)    |\n| Optimizer (Gurobi)   | 0.00 (0.00)    | 0.000 (0.000)       | 0.000 (0.000)         | 4.1        |\n| Optimizer (OSQP)     | 0.00 (0.00)    | 0.000 (0.000)       | 0.000 (0.000)         | 6.3        |\n| Post proj. (Gurobi)  | 0.11 (1.88)    | 0.000 (0.000)       | 0.000 (0.000)         | 10.1       |\n| Post proj. (OSQP)    | 0.51 (1.88)    | 0.000 (0.000)       | 0.000 (0.000)         | 5.5        |\n| APM                  | 0.05 (0.08)    | 0.000 (0.000)       | 0.001 (0.013)         | 117.6 (156)|\n| EAPM                 | 1.50 (2.25)    | 0.000 (0.000)       | 0.000 (0.000)         | 18.8 (11)  |\n| DC3                  | 5.44 (7.19)    | 0.000 (0.000)       | 0.004 (0.080)         | 335.3 (276)|\n\n#### DC-OPF (QP) 200-bus system: \n\n| Method               | Optimality Gap | Equality Violation | Inequality Violation | Time       |\n|----------------------|----------------|---------------------|-----------------------|------------|\n| Proposed             | 1.07 (1.71)    | 0.000 (0.000)       | 0.000 (0.000)         | 2.6 (1)    |\n| Optimizer (Gurobi)   | 0.00 (0.00)    | 0.000 (0.000)       | 0.000 (0.000)         | 4.6        |\n| Optimizer (OSQP)     | 0.00 (0.00)    | 0.000 (0.000)       | 0.000 (0.000)         | 15.1       |\n| Post proj. (Gurobi)  | 0.23 (1.49)    | 0.000 (0.000)       | 0.000 (0.000)         | 13.2       |\n| Post proj. (OSQP)    | 0.22 (1.49)    | 0.000 (0.000)       | 0.000 (0.000)         | 7.4        |\n| APM                  | 8.55 (9.45)    | 0.000 (0.000)       | 0.166 (0.181)         | 225.3 (300)|\n| EAPM                 | 3.88 (7.95)    | 0.000 (0.000)       | 0.000 (0.000)         | 78.7 (50)  |\n| DC3                  | 18.74 (20.61)  | 0.000 (0.000)       | 0.164 (0.189)         | 360.5 (300)|"}, {"review_id": "FhAI1hdoc0", "reviewer": "Reviewer_bPkV", "summary": "This paper proposes to enforce linear constraints on the outputs of ML models by forming the convex combination of the predictions of a pre-trained \"task network\" with a learned \"safe network.\" The role of the task network is to optimize prediction accuracy, whereas the role of the safe network is to ensure that the final predictions satisfy the problem's underlying constraints. The infinite-dimensional safe network training problem is shown to reduce to either an SDP or an LP, in the special cases where the support of the data distribution is either defined by an intersection of ellipsoids, or by a polytope. Numerical experiments on optimal power flow and portfolio optimization benchmark tasks are conducted, and it is shown that the proposed method yields predictions that always satisfy the desired constraints, while achieving prediction performance on par with state-of-the-art constrained optimization solvers.", "questions": "1. Last sentence of Section 3.1: In the notations section, you mentioned that $A_\\ell$ is used to represent the $\\ell$th row of a matrix $A$, but here, you are using $A_\\ell$ to represent a matrix. This might be a little bit confusing, so I'd suggest using consistent notation choices throughout the paper.\n2. \"Let $s^\\text{TN}(x)$ and $s^\\text{SN}(x)$ denote the predicted slack variables\" These slack variables don't seem to be very well defined. Up until this point, the only slack variables mentioned were in $s(x)$, appearing in (2), to combine the inequality constraints into an equivalent equality constrint. Those slack variables don't appear to be \"predicted,\" but rather they are prespecified by the problem at hand. So, what exactly are these \"predicted slack variables\"?\n3. \"Thus, the optimal $\\alpha$ can be computed as:\" Can you clarify precisely what this $\\alpha$ is optimizing?\n4. Is there a reason why you formulate the safe network training in the specific form of (5) in which you are essentially maximizing the amount of slack in the constraints? The intuition behind the particular design of the optimization problem (5) would be good to include in the manuscript.\n5. Minor formatting issue: In the heading of Section 3.4 (and in other sections), you capitalize the second word, whereas elsewhere in the paper, you only capitalize the first word. It would be best to remain consistent.\n6. In Section 3.5, you use the overline notation $\\bar{G}$, but you haven't defined it.\n7. What is $C_\\text{TN}$ in Table 1?\n8. \"Note that set $\\mathcal{X}$ corresponds to the intersection of $l$ ellipsoids...\" Wouldn't you need to assume that all $P_j$ are also negative (semi-)definite.\n9. Notation in (8): Do you mean to say that $S\\in (\\mathbb{S}^k)^m$ rather than $S\\in\\mathbb{S}^m$? Also, why must the slack variable function $s(\\cdot)$ be purely quadratic? It seems to me that the assumptions you outlined result in the constraint becoming $s_l(x) = b_l^\\top x- x^\\top A_l F x$ for $\\mathbb{P}$-almost every $x$, indicating that $s_l$ should have both a quadratic and a linear term.\n10. Notation in (10): Shouldn't $\\Lambda$ also be listed as a decision variable under \"maximize\"?\n11. Proposition 1: I think you mean $S\\in\\mathbb{S}^k$ again, rather than $S\\in\\mathbb{S}$.\n12. Proposition 1: It would help the reader if you clarified exactly what $W_j$ are here. Are these the same as the $P_j$ matrices in the main body of the paper, i.e., the matrices defining the quadratic constraints of the support set $\\mathcal{X}$ over which the semi-infinite quadratic inequality constraints is to hold almost surely? Also, when you employ this result to rewrite (8) into the SDP (10), it seems to me like you are actually tightening the feasible set (i.e., obtaining a lower bound on (8)), assuming that $l\\ne 1$ in general. The way the development is currently worded makes seems to suggest that (8) and (10) are equivalent reformulations, which might be misleading if in fact (10) only lower-bounds (8) in general.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: The paper is well-written, and proposes an interesting idea. The proposed method is theoretically justified by a universal approximation result with constraint satisfaction guarantees. The reformulation of the paper's primary training problem into finite-dimensional forms is also rigorously justified. The performance in the numerical experiments shows very promising results compared to the baseline methods compared to.\n\nWeaknesses: See my \"Questions\" below.", "was_revised": true, "final_justification": "The authors have thoroughly responded to my questions and suggestions, including revisions to clarify certain aspects in the manuscript. All of my concerns have been addressed, and therefore I maintain my positive evaluation of the paper.", "rebuttal": "**Q1**  \n \nYou are correct that in the notation section, we stated that $ A_\\ell $ denotes the $ \\ell $th row of a matrix $ A $. However, throughout the main text, we consistently use $ A_\\ell $ to represent the $ \\ell $th matrix of a set of matrices.  \nWe have revised the notation section and removed the earlier row-vector definition to accurately reflect the intended meaning of $ A_\\ell $ as a matrix rather than a row vector, ensuring consistency throughout the paper. \n\n---\n\n**Q2**  \n\nWe thank the reviewer for this insightful comment. The term “predicted slack variables” was not clearly defined in this context. The slack variable $s(x)$ introduced in equation (2) is a modeling construct used to express the input-dependent inequality constraints $H(x) f_\\psi(x) \\leq h(x)$ in standard form, and it is not learned or predicted by the model.  \nIn the paragraph following equation (3), the quantities denoted by $ s^{\\mathrm{TN}}(x) $ and $ s^{\\mathrm{SN}}(x) $ refer instead to *constraint residuals*, that is, the pointwise difference $ h(x) - H(x) f(x) $ that quantifies how much the output of the task or safe network satisfies (or violates) the inequality constraints. These residuals are not slack variables in the optimization sense, nor are they predicted directly by the model.  \nTo avoid confusion, we removed the term “predicted” in the revised manuscript and rephrase the surrounding text for clarity, as detailed in the next response.\n\n---\n\n**Q3**  \n\nWe appreciate the reviewer’s request for clarification. The purpose of $ \\alpha $ is not to optimize a task loss or performance metric, but rather to determine the smallest value in the interval $ [0,1] $ such that the convex combination of the task and safe network outputs satisfies the input-dependent inequality constraints.  \n\nFormally, for any input $ x \\in \\mathcal{X} $, we seek the minimal $ \\alpha \\in [0,1] $ such that $f_\\psi(x) = (1 - \\alpha) f_\\theta^{\\mathrm{TN}}(x) + \\alpha f_\\phi^{\\mathrm{SN}} (x)  $\nsatisfies $ H(x) f_\\psi(x) \\leq h(x) $. The equality constraints are already satisfied by construction: the safe network output is feasible by design, and the task network output is projected onto the affine equality set before combination.\n\nTo clarify this in the manuscript, we will revise the text as follows:\n\n> **Neural network output**  \n> The output of the task network, $ f_\\theta^{\\mathrm{TN}}(x) $, is projected to satisfy the equality constraints, while the output of the safe network, $ f_\\phi^{\\mathrm{SN}}(x) $, is constructed to satisfy both equality and inequality constraints for all $ x \\in \\mathcal{X} $. We define the final network output as in (3). To ensure that the final output satisfies the inequality constraints, we compute the smallest value of $ \\alpha_\\psi(x) \\in [0,1] $ such that\n$$\nH(x) f_\\psi(x) \\leq h(x),\n$$\ni.e., such that the convex combination is feasible. The equality constraints are satisfied by construction, since both $ f_\\theta^{\\mathrm{TN}}(x) $ and $ f_\\phi^{\\mathrm{SN}}(x) $ lie in the affine subspace defined by the equality constraints.\n> To compute $ \\alpha_\\psi(x) $, we define the residual slack vectors:\n$$\ns_\\theta^{\\mathrm{TN}}(x) := h(x) - H(x) f_\\theta^{\\mathrm{TN}}(x), \\quad\ns_\\phi^{\\mathrm{SN}}(x) := h(x) - H(x) f_\\phi^{\\mathrm{SN}}(x),\n$$\nwhich quantify the pointwise satisfaction margin of the inequality constraints for each network output.\n> Let $ \\mathcal{I} := \\\\{ i \\in \\\\{1, \\ldots, m_{\\mathrm{ineq}} \\\\} \\mid s_{\\theta,i}^{\\mathrm{TN}}(x) < 0 \\\\} $ denote the set of inequality constraints violated by the task network. For each such constraint, we determine the minimum value of $ \\alpha_\\psi(x) \\in [0,1] $ such that the corresponding convex combination becomes feasible. The tightest such value across all violated constraints yields:\n\\begin{equation}\n\\alpha_\\psi(x) := \\max_{i \\in \\mathcal{I}} \\frac{ -s_{\\theta,i}^{\\mathrm{TN}}(x) }{ s_{\\phi,i}^{\\mathrm{SN}}(x) - s_{\\theta,i}^{\\mathrm{TN}}(x) }.\n\\end{equation}\n> \n> This construction ensures that the final output $ f_\\psi(x) $ satisfies all constraints in $ \\mathcal{C}(x) $, while staying as close as possible to the task network output within the feasible set.\n\n---\n\n**Q4**  \n\nThe objective in equation (5) is designed to not only ensure feasibility of the safe network output, but also to encourage it to lie deep in the interior of the feasible set by maximizing the total slack in the inequality constraints.\n\nThis design choice is motivated by the role of the safe network in the final architecture: since we form a convex combination between the task and safe network outputs to ensure feasibility, a safe network output that lies farther from the boundary provides more \"buffer\" margin. In particular, it allows a smaller $\\alpha$ to be used in equation (4), thereby keeping the final output closer to the task network prediction. If the safe network output were close to the constraint boundary, the convex combination might require a larger $\\alpha$, potentially degrading performance.\n\nTo clarify this in the manuscript, we will add the following explanation after equation (5):\n\n> The objective in (5) encourages the safe network output to lie deep in the interior of the feasible region. This promotes robustness during the convex combination with the task network: the larger the slacks in the safe output, the more flexibility we have to interpolate while preserving feasibility. As a result, the final output remains closer to the task network prediction, reducing conservativeness.\n\n---\n\n**Q5**  \n\nThanks for your observation. In the revised version, we have only capitalized the first word of the section headings.\n\n---\n\n**Q6**  \n\nWe appreciate the reviewer for catching this omission. The notation $\\bar{G}$ in Section 3.5 denotes the left pseudo-inverse of the equality constraint matrix $G \\in \\mathbb{R}^{m_{\\mathrm{eq}} \\times n}$, defined as $\\bar{G} := G^\\top (G G^\\top)^{-1}$. This matrix appears in the closed-form projection formula used to enforce the affine equality constraints $Gx = g$, as given in equation (3).\n\nWe will revise the text to define $\\bar{G}$ explicitly.\n\n---\n\n**Q7**  \n\nWe thank the reviewer for pointing this out. $C_{\\mathrm{TN}}$ refers to the computational complexity of a single forward pass through the task network. It is used in Table 1 to quantify the inference cost associated with evaluating $f^\\mathrm{TN}_\\theta(x)$. This is consistent with the description in Section 3.5, where we state that the overall inference complexity is dominated by this operation. We have clarified this in the revised manuscript.\n\n---\n\n**Q8**  \n \nWe appreciate the reviewer’s question. However, it is not necessary to assume that the matrices \\( P_j \\) are negative (semi-)definite. In our setting, the vector \\( x \\in \\mathbb{R}^k \\) satisfies \\( x_1 = 1 \\) almost surely, so the first row and column of each matrix \\( P_j \\) correspond to this constant term. If we remove the first row and column of \\( P_j \\), the resulting submatrix (associated with the truly uncertain components) is indeed negative semidefinite. However, our theoretical results do not rely on this structure. Specifically, Lemma 1 and Proposition 1 in our manuscript hold without requiring definiteness of \\( P_j \\). We will clarify this point in the revised manuscript.\n\n---\n\n**Q9**  \n\nIn the manuscript, we make the modeling assumption that $x_1 = 1$ almost surely. This trick allows us to represent any \\textit{affine function} of the remaining components $x_{2:}$ (i.e., $x_2, x_3, \\ldots, x_k $) as a \\textit{linear function} of the full vector $x$. Specifically, any affine function of the form $A x_{2:} + b$ can be expressed as:\n$$A x_{2:} + b = \\begin{bmatrix} b & A \\end{bmatrix} \\begin{bmatrix} 1 \\\\\\\\ x_{2:} \\end{bmatrix} = \\hat{A} x,$$\nwhere $\\hat{A} := \\begin{bmatrix} b & A \\end{bmatrix} $ and $ x := \\begin{bmatrix} 1 \\\\\\\\ x_{2:} \\end{bmatrix} $.\nLikewise, any \\emph{quadratic plus linear} function of $x_{2:}$, such as\n$$\nx_{2:}^\\top Q x_{2:} + q^\\top x_{2:} + r,\n$$\ncan be rewritten as a \\emph{symmetric quadratic form} in $x$, $x^\\top S x$,\nwhere $S \\in \\mathbb{S}^k$ is a symmetric matrix constructed to capture the original $Q, q, r$ terms. This modeling convention simplifies the optimization formulation while preserving generality.\n\n---\n\n**Q10**  \n\nWe thank the reviewer for pointing this out. $\\Lambda$ is a decision variable in our formulation and should be explicitly listed under the ``maximize'' operator. We have updated the manuscript accordingly.\n\n---\n\n**Q11**  \n\nThanks for pointing this abuse of notation out. Yes, we meant $S\\in\\mathbb{S}^k$. We have updated the manuscript accordingly.\n\n---\n\n**Q12**  \n\nWe thank the reviewer for pointing out this important clarification. First, regarding the notation: the matrices $W_j$ in Proposition 1 indeed correspond to the matrices $P_j$ introduced in the definition of the support set $\\mathcal{X}$ in the main body of the paper. Specifically, each quadratic constraint defining $\\mathcal{X}$ is of the form $x^\\top P_j x \\geq 0$, and these are the same matrices referred to as $W_j$ in Proposition 1. We will update the notation in the appendix to maintain consistency.\n\nRegarding the second point, we agree that Proposition 1 provides a sufficient condition for the satisfaction of a semi-infinite quadratic inequality over a set defined by  $l \\geq 1$ quadratic constraints. Therefore, when $l > 1$, the resulting SDP (10) derived from Proposition 1 yields a conservative inner approximation of the feasible set of (8), and hence the optimal value of (10) can be interpreted as a lower bound on the optimal value of (8). However, when $l = 1$, i.e., the support set $\\mathcal{X}$ is defined by a single quadratic constraint, equivalence holds. We will revise the manuscript accordingly to clarify that the SDP-based reformulation in (10) is an inner approximation of (8) in the general case and an exact reformulation for $l = 1$."}]

Author rebuttal:

Meta-review: The paper provides a method for adding input-output constraints on networks. This is achieved by splitting the task in between two networks, the main network that aims to reduce the loss, and a safe network that imposes the constraints. The main concerns raised by the reviewers involve the scalability of the method, which was addressed by the reviewers during the rebuttal period. I recommend acceptance, but suggest adding notes on scalability and the cost of solving SDPs.
Decision: Accept (poster)