Pinet: Optimizing hard-constrained neural networks with orthogonal projection layers
Authors: Panagiotis D. Grontas
Antonio Terpin
Efe C. Balta
Raffaello D'Andrea
John Lygeros
Venue: ICLR 2026 Oral (2026)
Keywords: hard constrained neural networks
network architecture
implicit layers
operator splitting
optimization
Primary area: optimization

Abstract: We introduce an output layer for neural networks that ensures satisfaction of convex constraints. Our approach, $\Pi$net, leverages operator splitting for rapid and reliable projections in the forward pass, and the implicit function theorem for backpropagation. We deploy $\Pi$net as a feasible-by-design optimization proxy for parametric constrained optimization problems and obtain modest-accuracy
solutions faster than traditional solvers when solving a single problem, and significantly faster for a batch of problems. 
We surpass state-of-the-art learning approaches by orders of magnitude in terms of training time, solution quality, and robustness to hyperparameter tuning, while maintaining similar inference times. Finally, we tackle multi-vehicle motion planning with non-convex trajectory preferences and provide $\Pi$net as a GPU-ready package implemented in JAX.

Published as a conference paper at ICLR 2026
INNET: OPTIMIZING HARD-CONSTRAINED NEURAL NETWORKS WITH ORTHOGONAL PROJECTION LAYERS
Panagiotis D. Grontas" ETH Zürich Pgrontassethh.ch
Antonio Terpin" ETH Zürich
Efe C. Balta
inspire AG & ETH Zürich
sterpin&ethz.ch
efe. afe.baltar inspire ch
Raffaello D'Andrea
John Lygeros
ETH Zürich
ETH Zürich
rdandrealethz.ch
jlygeros@athz.ch
ABSTRACT
We introduce an output layer for neural networks that ensures satisfaction of con- vex constraints. Our approach, Inett, leverages operator splitting for rapid and reliable projections the forward pass, and the implicit function theorem for backpropagation. We deploy Ilnet as Jeasible-by-designn optimization proxy for parametric constrained optimization problems and obtain modest-accuracy solutions faster than traditional solvers when solving single problem, and sig- nificantly faster for a batch problems. We surpass sfate-of-the-art learning approaches by orders of magnitude in terms of training time, solution quality, and robustness to hyperparameter tuning, while maintaining similar inference times. Finally, we tackle multi-vehicle motion planning with non-convex trajec- tory preferences and provide llnet as GPU-ready package implemented in JAX.
https://github: com/ antom/antoun http:///aanoooooorpaaa//iiee.eee
pip install pinet-henn
Backpop
Backprop
f(x)
Loss ((y.x)
Backbone network
Projection layer
Optimization
Figure 1: Illustration of the Inet architecture. The infcasible output of the backbone network is projected onto the feasible set through an operator splitting scheme. To train the backbone network, we use the implicit function theorem to backpropagate the loss through the projection layer.
INTRODUCTION
In this work, deploy neural networks (NNs) to generate feasible-by-design candidate solutions (see Figure 1) for the parametric constrained optimization problem:
minimize (1y, subject to y C(x),
P(x)
where y Rdd is the decision variable, x E Rp is the context (or parameter) of the problem instance, : IRd R + is the objective function, and C(x) C Rdd a non-empty, closed, convex set for all We provide pedagogical example to explain this formulation in Appendix A.
Constrained optimization has universal applicability, from safety-critica applications such as the op- timal power flow in electrical grids (Nellikkath & Chatzivasileiadis, 2022). logistics and schedul- ing (Bengio et al., 2021), and even biology, where enforcing priors on the solution can enhance its
"Equal contribution.

---

Published as a conference paper at ICLR 2026
interpretability (Balcerak et al., 2022; Terpin et al., 2024b). In many applications, optimization pro- grams are solved repeatedly, given different contexts: in logistics, the demands and forecasts vary (Baptiste et al., 2001); in model predictive control (MPC) (Chen et al., 2018; Tabas & Zhang, 2022), the initial conditions; in motion planning, the position of obstacles (Marcueci et al., 2023); in trust- region policy optimization, the advantage function and the trust-region center (Terpin et al., 2022b). This task often becomes computationally challenging when, for example, is high-dimensional, is non-convex, or new solutions are required at high frequency. Rather than solving each problem in- stance from scratch, the mapping from contexts to solutions can be learned with NNs. Despite their successes, NNs typically lack inherent mechanisms to ensure satisfaction of explicit constraints, a limitation that motivates large body of existing work:
Soft- contrrnnstr NNs. Soft penalty terms for constraint violations in the objective function are one approach to incorporate constraints in NNs (Márquez-Neila et al., 2017), and have been adopted to solve parametric constrained optimization problems (Tuor et al., 2021), and partial differential equations through mhysics-informer NNs (Erichson et al., 2019; Raissi et al., 2019). Despite their ability to handle general constraints, these approaches offer no guarantees at inference time. Beside requiring manual tuning the penalty parameters, which is challenging yet critical for good perfor- mance, the use of soft constraints is discouraged for the following reasons. First, the structure of the constraints set can be exploited to design more efficient algorithms; see, e.g., the simplex algorithm (Dantzig, 2002). Second, treating constraints softly may significantly alter the problem solution regardless of tuning (Grontas et al., 2024). Third, certain constrained optimization problems (e.g.. linear programs) may not admit solution all when constraints are treated softly.
Hard-constrained NNs. To circumvent the shortcomings of soft constraints, hard-constrained neural networks (HCNNs) aim to enforce constraints on the NN output by design. Frerix et al. (2020) address linear homogeneous inequality constraints by parameterizing the feasible set. Simi- larly, RAYEN (Tordesillas et al., 2023) enforces various convex constraints by scaling the line seg- ment between infeasible points and a fixed point in the feasible set's interior. While these methods enjoy rapid inference, they require expensive offline preprocessing and are not directly applicable to constraints that depend on the NN input, i.e., they consider feasible sets C and not C(x) in P P(x). Differently, Min et al. (2024) propose closed-form expression to recover feasibility for polyhedral constraints and employs evxpy 1 ayers (Agrawal et al., 2019) for more general convex sets. Chen et al. (2018); Cristian et al. (2023) orthogonally project the NN output or intermediate layers us- ing Dykstra's algorithm (Boyle & Dykstra, 1986), but rely on loop unrolling for backpropagation, which can be prohibitive in terms of memory and computation. Departing from convex sets, DC3 (Donti et al., 2021) introduces an equality completion and inequality correction procedure akin to soft-constrained approaches, but applied during inference. Lastrucci & Schweidtmann (2025) im- pose non- non-linear equality constraints by recursively linearizing them. Lagrangian and augmented Lagrangian approaches considered by Fioretto et al. (2021); Park & Van Hentenryck (2023) for general non-convex constraints, with drawbacks similar to soft constraints, whereas Kratsios et al. (2021) consider (probabilistic) sampling approach. Recently, LinSATNet (Wang et al., 2023) has been proposed to impose non-negative linear constraints, which is restrictive constraint class that renders none of the problems of interest for this work amenable LTnSATNet. This limitation is partially relaxed by GLinSAT (Zeng et al., 2024), which however requires bounded constraints, an assumption not satisfied by, e.g., epigraph reformulations (Stellato et al., 2020, Appendix A.5-A.7).
Implicit layers. Implicit layers embed optimization problems such as quadratic programs (QPs) (Amos & Kolter, 2017; Butler & Kwon, 2023), conic programs (Agrawal et al., 2019), non-linear least-squares (Pineda et al., 2022), or fixed-point equations (Bai et al., 2019; Winston & Kolter, 2020) as NN layers, and apply the implicit function theorem (Dontchev & Rockafellar, 2009) to various measures of optimality. Our work is an instance of implicit layers; see Appendix C.5.
ML for optimization. Using constrained NN architectures to learn solution mappings is among many successful efforts to exploit ML techniques to accelerate (King et al., 2024; Sambharya et al., 2024) or replace optimization solvers (Bertsimas & Stellato, 2022; Zamzam & Baker, 2020). These approaches are referred to as amortized optimization, learning to optimize or optimization learning, and the surveys (Amos, 2023; Van Hentenryck, 2025) cover multiple aspects of the topic.
2

---

Published as a conference paper at ICLR 2026
Contributions. We propose an NN architecture, Ilnet, that generates cessible-b-desipe solutions for Pxx). Given context x, deploy backbone NN to produce a raw output Vraw that we orthogonally project onto C(x). y = argiiin || -Yrww 1 In particular:
We use an operator splitting scheme to compute the projection in the forward pass, and back- propagate through it via the implicit function theorem. Our work is instance of implicit layers (see Appendix C.5), but specializes in projection problems whose structure can be significantly exploited, achieving rapid training and improved inference speed. Our sim- ple idea yields a learning architecture that surpasses state-of-the-att learning approaches by orders of magnitude in terms of training time, solution quality, and robustness hyperpa- rameter tuning, while maintaining similar inference times.
We implement a hyperparameter tuning and matrix equilibration strategy that boosts Inne t's performance and robustifies it against data scaling. As result, IInet solves challenging benchmarks on which existing methods struggle, and is substantially less sensitive to hyper- parameter tuning. In addition, we deploy Inee real-world application in multi-vehicle motion planning with non-convex trajectory costs.
We provide an efficient and GPU-ready implementation of Ilnet in JAX. We make our code available https: / / thub. com/ http://ttooiiiioooooooo httm///annonottrpin//poo..
The indicator function of set K. is Ix(y) = 0 if y E K, and too otherwise. The proximal operator of proper, closed and convex function f with parameter a 0 is prox,f(x) argmin, (f() + 25 lly
Notation.
- =11"). The projection onto a non-empty, closed, and convex set C [π(() = proxz, (x). For a differentiable mapping F R - IR",, its Jacobian evaluated at & 1 is denoted by [F(=)
=== Tx use d instead of 0 when referring total derivatives.
Given vector Th.", the vector-Jacobian product (VJP) of F & with is v OF())
IR..
==
TRAINING HARD- CON--CNNSSTREEE NEURAL NETWORKS
We develop an NN layer that projects the output of any backbone NN onto C(x), and discuss how Ilnet can be trained to generate solutions of P(x), for any given
2.1 PROJECTION LAYER
Given context x, the backbone network produces the output Braw = fxx: 0), where 0 are the network weights, We enforce feasibility by projecting Vraw onto C(x):
y [[(x)) (Braw) argmin
Braw
highlight two benefits of this approach:
Constraint satisfaction. By design, the output y the projection layer always lies C(x).
Decomposition of specifications. The hard constraints prescribe the required behavior of the output, while the objective prescribes the desired behavior. Contrary to soft-constrained NNs, no tradeaff between the two behaviors is introduced in our proposed framework.
d, are closed, convex sets that we design, and IT is the projection onto the first d coordinates. We omit the dependence on x for brevity, but stress that our method readily handles context-dependent constraints. Notice that we work in the possibly variable Vaux T n This provides us the flexibility to choose A and I. such that their respective In particular,
We consider constraints that can be expressed C I K), where A,K A.C.K.
R", by introducing an auxiliary
projections A and ITx admit closed-form expression are numerically-effiieen A will be a hyperplane defined by the coefficient matrix A and the offset vector b, and IC a Cartesian product of the form K. Ki K2 C 2d x nn This representation can describe many constraints of practical interest and we instantiate it with an example next (see Appendix E for more). Consider polytopic sets that often employed in robotics (Chen et al., 2018), numerical solutions to partial differential equations (PDE) (Raissi et al., 2019), and non-convex relaxations for trajectory planning (Malyuta al., 2022), among others. They expressed {y E Rd Fy q. < Cy S u), for
3

---

Published as a conference paper at ICLR 2026
some E, L, C, u of appropriate dimensions. We introduce the auxiliary variable yaux Cy "lony with dimension n d Tineq.
Then, we define A, K _ I= as the following hyperplane and box
E C
0
A
K
y y VERR
IS Maux S
Vaux
1
Vaux
Maux
Importantly. C Il K)) and both II. and Ig be evaluated in closed form. More gen- erally, the proposed decomposition, with A and π admitting closed-form expressions or being numerically efficient, is applicable to any constraint set of the form:
AoqUl bog
(Equality)
loox S I Ubox
(Box)
linog= I Ancqq' S Vinog
(Inequality)
A| asocsill2 S fuc, 3 + bsoci
i == 1, Nuse
(SOC)
|| Ae iill + arsiilll 111 S Se + beyis
1, I....,,,,, No.
(ei ball)
A + alxe S f ,2 bpo i I....,,, Ne
(oo ball)
Us, 2 0, llus,
= lus,,1==1,
for index sets Sj.
(Simplex)
Remark. We stress that the decomposition = T=((AA)) is not an assumption. One can always decompose a convex set in this way, egg., by considering the trivial decomposition A and K. = Rdd Instead, determining A and K. is design choice which we leverage to make the projections A and Ilg computationally efficient. note two important points regarding this design choice:
The only assumption is that A and Tg and their VJP are computable. Being computationally efficient is added benefit of our decomposition, but is not necessary. In this work, focus is on the sets listed as those are the ones that are most interesting in practice.
We show in Appendix E that many practically relevvant constraints C, such as the ones listed in (2), as well as their intersections and Cartesian products all admit efficient decompositions. fact, this list is not exhaustive; see, e.g., the work of Condat (2016) and Boyd & Vandenberghe (2004).
2.1.1 FORWARD PASS
To compute the projection y = [l((x) (Braw) we employ the Douglas-Rachton algorithm (Bauschke & Combettes, 2017, Sec. 28.3), which solves optimization problems of the form min, g(2) h(2), where g and are proper, closed, convex functions. To rewrite (1) in this composite form, we use the auxiliary variable Yaux Rn-d and the indicator function of A and K. obtain:
(
(Ilc(Mtw)) Vaux) argmin || Mraw
IA
1/
Ix
1/
(3)
|Jaxx
3laux
Then, split the objective function follows
([
yaux )) 11y Vrawll Ixc
((rmn)
IA
and h
|laux
[3aux
|Jaux
By applying the Douglas-Rachfin algorithm we obtain the fixed-point iteration:
2k+1 prox,9 (8k)
(4a)
ÅLgoRIthM 1. Operator splitting for projection
4+1 prox (2z 88)
(4b)
Inputs: Vrow K E N, ar Initialization: so E I
8k+1 = 54 + w(1++1 - =x+1)
(4c)
where o is scaling and w (0, a re- laxation parameter. The proximal operators in (4a) and (4b) can evaluated explicitly, see Appendix D.1, allowing us to implement (4) as in Algorithm 1. Note that we write
For kt = 0 to K 1:
Zk+1 IT ATx
(x)) 236 84,1 2
1++1
1+20
[xxx(x)) (22x-1,2 Sk.2)
8k+1 <- 8k w((++] 2k+11
24 [ =x,2] where žk,1 E Rd and 2k.2 R correspond to y and Bax.
Output: 2K,1. SK
4

---

Published as a conference paper at ICLR 2026
Under mild conditions, namely strict feasibility of (3), show in Appendix D.2 that the iterates 2n and ty converge to a solution of (3). We denote the limits &oo (Baw) lim 2 and & (3tww = limg--oo Sg. and highlight their dependence on the point-to-be- projected Braw In particular, we have Roc,1 (Braw) = (Braw)). )- In practice, we run a finite number of iterations K E N of (4), which set K
during testing, and take 2K,1 as the output of the projection layer. We detail our hyperparameters in Section 2.4. We note that, although 2K will not necessarily lie on A K, because K N is finite, Algorithm guarantees that 2 E A. By our choice of A, this implies that 21, satisfies any equality constraints the problem. The feasibility- -eediign comes from the convergence rates, which we derive and empirically demonstrate in Appendix D.3: for a sufficiently high number of iterations, the output of our projection layer is arbitrarily close to the true projection.
during training and K nniterr_teeee
2.1.2 BACKWARD PASS
To train the backbone network using backpropagation, we need to efficiently differentiate the loss L (which in general depends on the projected output of the network and on the input; see Section 2.2) with respect to the backbone network parameters 6. The chain rule gives
(fxx)c(((x0))). d0
dC(y.x)
Olle(yan
fx U)
(5)
Oy
yraw
08
I= (fx:))
(x:0)
1=
Since the first and last terms are standard and typically computed with automatic differentiation, we only need to provide an efficient computational routine for the VJP
( (UTa))))))
e +
T
(6)
8yea
Mean f(x:0))
Rather than differentiating through all the iterations of Algorithm I by loop unrolling, we exploit the implicit function theorem (Dontchev & Rockafellar, 2009) to efficiently evaluate (6). To do so, we first recall that (Braw) = [la Odx(n-d)) [s (Braw) which implies that:
TT ( (ollc( Nyea
)
(s)
Os (Braw)
Ös
Oyaa
Mun fxx:0)
Your fxx:0)
and note that the first VJP is straightforward to compute since IIA is an affine mapping. Therefore, to evaluate (7) we need to backpropagate through 8:x0 (Jruw). Since 800 (Bhaa) is the fixed point of
iteration (4), it satisfies the equation 8 (Braw) + (8 (Draw), ), Mraw), where O represents one iteration
of (4a)-(4c), namely, Sk+1 (Skk, yraw), ). The implicit function theorem applied to Soo (yow)
(5 (Braw), Braw) yields the VJP, see Appendix D.4,
@80(800 (yraw) ), Braw)
+ ((rrw, u)
(8)
Byraw
Yran (x:88
where (Vhrws v) is a solution of the linear system
| (s, Braw)
(( U) U.
Is
(shmn
The matrix in (9) may not be invertible (Agrawal et al., 2019); even if it is, constructing it and computing its inverse may be prohibitively expensive in high dimensions. This difficulty can be cir- cumvented by computing a heuristic quantity. In this vein, we deploy the JAX (Bradbury et al.. 2018) implementation of the bi-conjugate gradient stable iteration (b Dicctatb, van der Vorst (1992)), an indirect linear system solver that requires only matrix-vector products. Therefore, we implement (8) and (9) using VJPs involving ( (Braw), Braw)) / /0 and 0D(s, h/ww)/Os, respectively. We efficiently do this using JAX VJP routines and note that each step of the solver for (9) has essentially the same computational cost as one step of (4). The maximum number of bi cgssstaa steps for (9), n_iter Jwwd, is hyperparameter. In both (8) and (9), we use the output of the forward pass, i.e., Sy in place of the intractable s We discuss eerost-eeeyywhere differentiability of our projection layer and the applicability of the implicit function theorem in Appendix F.
5

---

Published as a conference paper at ICLR 2026
2.2 LOss
The training loss ([[[(()) (x: f)). which a function of the context and the constrained NN output, can be crafted according the specific requirements of the problem. In experiments, directly minimize the objective of (xx) using the network's output by setting L(y, x) = (y,, x). In our framework we can, loosely speaking, interpret training as performing projected gradient descent on the raw NN output space, akin to the work of Cristian et al. (2023).
2.3 NETWORK ARCHITECTURE
Inne flexible: the projection layer can appended to any NN; see Figure We summarize the forward and backward pass of IInet in Alg. 2, while its training and testing outlined in Alg. 3.
ALGORITHIM 2. Ilnet
ALGORITIMM 3. Training/Testing of Ilnet
Forward:
Inputs: Chosen/Tuned hyperparameters
Inputs: 0, K,
(see Section 2.4)
Frow <- [(x; 0)
Training:
y. 4 Call Algorithm [xx, Braw, K, o,
0, <- Initialize weights
Output: y.
For f = to n_epochs:
x + Sample from training set
Backward:
y. Forward(x, nww.ern.nnteiiiii.r o,w)
Inputs: Soo, Rd
C(y. x). aC(y. x) + Compute loss
+ u 0) (AIla())/ss) 1=00
0C/00 t Backward (s (L(y,x)00g)
(1haw, 11) t Solve (9) with as RHS
8,t1 + Optimizer update
02 + ((1raw v1) (ab (s, Yraw) 1/0y)a)) smaw da (0f(x; f)) 0=0
Output
: O, epochs
Testing:
Output:
f Sample from test set
y. t Forward(x, 8, n.iter.test,
Output
2.4 THE SHARP BITS (SHORT VERSION)
To push the performance of Inet, we adopt two important numerical techniques. First, we im- prove the conditioning of the matrix A(x), that defines the set A(x), by implementing the Ruiz cquilibration algorithm (Wathen, 2015); see Appendix C.I. Second, we exploit the fact that, com- pared to existing methods, Inet relies only few hyperparameters. Specifically, nei.er.e...r.. w.iterfffw fwd (number of iterations for the forward pass during training), n_iter_test (number of iterations for the forward pass during inference), omega, sigma (standard Douglas-Rachfort parameters), and n_iter .wwd (number of iterations in the bi cgstab procedure). We describe an auto-tuning procedure that recommends hyperparameters by evaluating the projection on a subset of the vali- dation set in Appendix C.2, and assess its effectiveness in Appendix C.3. Finally, we highlight our choice of enforeing constraints during training, opposed training an unconstrained network and introducing the projection layer only afterwards. We do so because the latter approach may result in training instabilities or suboptimal performance; see Appendix C.4.
NUMERICAL EXPERIMENTS
The code is made available at htttt:://githbb wwt.///ttonootepppi//piiiee The empir- ical data was collected on an Ubuntu 22. 04 machine equipped with an AMD Ryzen Threadripper PRO 5995WX processor and an Nvidia RTX 4090 GPU. For experiments with second-order cone constraints, see Appendix B.5.
3.1 BENCHMARKS AND COMPARISONS WITH STATE-OF OF-THE-ART
We consider set of standard convex and non-convex problems classically used compare HCNNs.

---

Published as a conference paper at ICLR 2026
JAXopt
Iast
Nou-coavex
Small
Large
10
10'-
22
10:1
10"4
Figure 2: Scatter plots of RS and CV on the small and large non-convex problems on the test set. The red dashed lines show the thresholds to consider a candidate solution optimal.
Experimental setup.
We consider the benchmark problems introduced by Donti et al. (2021):
minimize yER
(yy subject to Ay =x Cy
where J(.) is defined as J(v) = Qy + q for the convex problem setup, and as J(y) = y Ju 9 sin(y) for the non-convex, and Q E Raxd d > O, q Rd A E R xd R C Riinc xd u E R"incu, In particular, in the work of Donti et al. (2021), Q is diagonal with positive generated
fixed matrices/vector and the contexts stances are guaranteed to be feasible. Donti et al. (2021) consider problems with d = 100. Here, we include larger problem dimensions, (d, Begy Tineg) {(100, 50, 50), (1000, 500, 500)). which generated with the same scheme. We refer to these datasets as small and large. For each dataset, we generated 10000 contexts split as 7952/1024/11
entries, A, C,
that all problem in-
among soaning/validation/eee sets.
Baselines. We compare Ilnet to DC3 (Donti et al., 2021) and a traditional Solver. For the con- vex objective the Solver is the QP solver OSQP (Stellato et al., 2020), while for the non-convex objective is IPOPT (Wächter Biegler. 2006). Further, we compare an implicit layer approach, where instead of computing the projection (1) with Algorithm 1, we use the JAXopt (Blondel al., 2022) GPU-friendly implementation of OSOP that employs implicit differentiation. Both Inet, DC3, and the JAXopt approach use self-supervised loss (i.e., L. = J) and as backbone multi-layer perceptron (MIP) with hidden layers of 200 neurons each and ReLU activations. Additionally, DC3 includes batch normalization and drop out, well as soft penalty terms in the loss. We use the default parameters of DC3 unless otherwise stated. particular, the DC3 algo- rithm with default parameters diverged during training the large datasets, an effect observed by Tordesillas et al. (2023). To rectify this, we tuned the learning rate of DC3's correction process for the large dataset, and found that 10-8 enables the network to learn. In Appendix B.2, we investigate DC3's performance can be improved by adapting hyperparameters. For Inet we use only 50 training epochs, while for DC3 and 12 epochs, in the interest of training time. On both convex and non-convex benchmarks, we use JAXopt as replacement for our custom projection layer after the backbone NN. The training times reported are, thus, the ones of the backbone network. We omit comparisons with evxpylayy Sayyllyyrss (Agrawal al., 2019) since JAXopt is more recent and stronger baseline: it implements similar functionalities, but it is executable on the GPU; see also Appendix B.
the default 1000. For JAXopt
tolerance of 10-3
Metrics.
We compare methods in terms of the following metrics on the test set:
Relative suboptimality (RS): The suboptimality of a candidate solution compared to the optimal objective J(y"), ), computed by the Solver ver. Since methods may violate constraints and obtain better solution we clip this value, RS == max (0, (J(() (()()))(())))
Constraint violation (CV): We define CV = max(]] - x xl |ma (Cğ u, 0)
Learning curves: Progress on RS and CV over wall-clock time on the validation set.
Single inference time: The time required solve one instance test time.
Batch inference time: The time required to solve batch of 1024 instances at test time.

---

Published as a conference paper at ICLR 2026
OAXopt
Non-convex
Small
304
10
10
10"1
10-4
10-1
1073
40
50
100
120
140
30
100
140
100
180
200
Time
Large
107
13"4
10
10-1
10m
100
150
200
230
200
350
400
450
50
100
150
510
250
200
350
000
450
Time
Figure 3: Comparison of the learning curves in terms of average RS and CV on the validation set, on the small and large non-convex problems. The solid lines denote the mean and the shaded area the standard deviation across seeds. The learning curves for JAXopt on the large dataset are reported only in Appendix B because of the orders of magnitude longer training times.
Next, we report and discuss the results on the non-convex datasets. In the interest of space, the results on the convex problems are given in Appendix B.2.
The RS and CV for each problem instance the test set are reported in Figure 2. We consider a candidate solution be optimal the condition CV 10-3 and RS S 5% is satisfied. These prerequisites for low accuracy solutions are similar to, though somewhat looser from, those employed by numerical solvers (O (O'donoghue et al., 2016; Stellato al., 2020). fact, Inee clears these thresholds by margin. In practice, any solver achieving a CV below 10-5 5 is considered high- accuracy (Stellato et al., 2020) and there is little benefit to go below that. Instead, when methods have sufficiently low CV. having low RS is better.
Results.
Compared to DC3, we correctly solve the vast majority of test problems. Importantly, the very low and consistent constraint violation across all problem instances significantly facilitates the tuning of the number of iterations. By contrast, on the large problems, DC3 exhibits CV and RS that are unacceptably large for any meaningful application. We conjecture that IInet significantly outper- forms DC3 in RS due to the absence of soft penalties in our training loss and the orthogonality of the projection. The JAXopt approach performs better than DC3 but significantly worse than Inne..
The learning curves are shown in Figure 3. Ilnet achieves better performance training time. Crucially, our scheme attains satisfactory CV throughout training, implying that Inet with tiny training budget. Note that training curves
fraction of the
reliably compute feasible solutions include the setup time for Inne (the matrix equilibration, the calculation the pseudo-inverse for the projection onto the affine subspace, and just-in-time compilation). We omit the JAXopt results on the large dataset as it requires roughly 14 hours to complete training. The substantial difference in training times between Innet and JAXopt underscores the importance of the specialized splitting we employed exploiting the structure of the projection problem, and our specialized implementation.
We report inference times in Table 2 in Appendix B. All approaches significantly outperform the Solver. Although DC3 is slightly faster than Inet,, it generates solutions of much lower quality (in RS and CV), hence demonstrating that the advantages of Inne come with only a minor runtime trade-off compared to DC3.
8

---

Published as a conference paper at ICLR 2026
OPTIMIZATION
DECOUPLED CONSTRAINT BATISFACTION
will ...
w012 1
Inoiaal and lixel coneiguuation bX]
lapal injjoclocoo
Ran whkkde tneectoiis (mo))
llaci
Fleet trjjotoriis I. after unbitehigg
peojection
Figure 4: (Top) The Inet approach constrained multi-vehicle motion planning with arbitrary differentiable objective functions (Bottom) From left to right, we show examples of the synthe- sized trajectories for 3 different objectives: Pien effort, Pmid Plef + wwwpprrreeence (right imii + coverage. We refer the reader to Appendix B.3 for formal definitions and additional plots.
To summarize our findings, we showed that existing HCNN methods suffer either from long train- ing/inference times or low quality solutions. Inee address both shortcomings, while retaining modularity and simplicity.
3.2 INET APPLIED: MULTI-VEHICLE MOTION PLANNING
We present an approach to synthesize transition trajectories between multi-vehicle configurations that optimize some non-linear, fleet-level objective subject to dynamics, state and input constraints. We feed an NN with the initial and terminal fleet configurations (the context x), obtain the raw input trajectories and use the vehicle dynamics infer the full state-input trajectories that serves as the raw output Vrow, which are then projected for ensured constraint satisfaction; see Figure 4.
Experimental setup. We follow the formulation in the work of Augugliaro al. (2012). Specif- ically, we denote with pt[t] E Rm the generalized coordinates vehicle i at the discrete times t E fl...., T), and with ft[t] and a,[t] its generalized velocity and acceleration. Its simple dis- cretized dynamics read vz + +| vy [t[{] + hag[t), pi(t + 1] = pi[(] + hee[4 + A aa, [t]. We formulate the motion planning task for the fleet as parametric program in the form of Pxx), where C(x) in- cludes box constraints on positions (workspace constraints), velocities, and accelerations (physical limits), affine inequality constraints for jerk limits, and equality constraints for the dynamics and initial/final configuration for each vehicle. The objective function encapsulates a fleettlevel objec- tive; here consider weighted of workspace coverage, input effort, and trajectory preference given by a potential function; see Appendix B.3 for a rigorous definition.
Qualitative results. We display some of the resulting trajectories for different weights in the ob- jective in Figure and report more visualizations and analysis for larger fleets and longer horizons in Appendix B.3. Crucially, both convex and non-co con-convex objectives are handled effectively by Inet, resulting trajectories that adhere to the goals prescribed by the different objective functions.
Practical Relevance. Multi-vehicle motion planning has received substantial attention for prac- tical applications (Terpin al., 2022a; 2024a), and we highlight three benefits of our approach:
/ Constraint satisfaction. We ensure dynamics, state, and input constraint satisfaction, similar to optimization-based trajectory generation methods (Augugliaro et al., 2012).
Parallelizaaility. Our approach is parallelizable in two ways. First, enables multiple problem instances (different initial and final configurations) to be solved in batches. Second, since the

---

Published as a conference paper at ICLR 2026
constraints consider decoupled between the vehicles, can jointly predict the input trajectories (to enable the network to minimize the joint objective), while solving the projections for each vehicle separately; see Figure 4.
Arbitrary objective optimization. Our framework handle any almost everywhere differen- tiable objective, encoded in Ilnet's is loss L 4. Importantly, we see this example as a proof of concept towards constrained human- prefeeeeeeeeeecee optimization (e.g., using the approach of Chris- tiano et al. (2017)). Deploying traditional solver for this problem is very challenging because the objective functions considered are not available in closed form and are highly non-linear.
We implement this application in a separate codebase atttps://githae com/ antonioterpin/glite
demonstrating also the little overhead required to integrate Ilnet into specific applications. We also explore trajectory planning on a longer horizon (up to 750 steps, amounting to about 9000 optimization variables and constraints), well high-dimensional contexts (mono-channel, 1024 x 1024 images) in Appendix B.4. Our current formulation focuses only on convex, decoupled-amoon-veliin vehicles constraints. Future works could address collision avoidance constraints through sequential convexification techniques (Augugliaro et al., 2012; Malyuta et al., 2022).
4 CONCLUSIONS
Contributions. We introduced output layer that enforces convex constraints satisfaction on the output of an any backbone NN via an operator splitting scheme. The backpropagation is achieved via the implicit function theorem, enabling efficient training. Our work focused on the gritty details of optimizing IInet, introducing also simple yet effective techniques such as hyperparameter tuning and matrix equilibration procedures. We showed through extensive benchmarks that IInet suc- ceeds where existing learning methods fail. We provide GPU-ready implementation in JAX, and showcase how our layer can be embedded an example application, multi-vehicle motion planning.
Limitations. One limitation of our work is the requirement convex constraint sets. Despite the numerous applications involving only convex constraints (Boyd & Vandenberghe, 2004), and the nu- merous applications that can be losslessly convexified (Malyuta et al., 2022). we acknowledge that future work should investigate how to relax this structural assumption. One potential approach could involve sequential convexification of non-convex constraints, similar to the algorithm in (I (Lastrucci & Schweidtmann, 2025) that addresses non-linear equality constraints, or by employing homeomor- phisms (Liang et al., 2024).
Outlook. We believe that Ilnet holds the potential to substantially impact wide range of ma- chine learning domains where constraint satisfaction is crucial. Relevant examples include neural PDE solvers (Raissi et al., 2019), scheduling (Baptiste al., 2001), and robotics (Malyuta et al., 2022), among others. We demonstrated the potential of Inne in some these applications in Sec- tion and we believe that applying method to new applications represents exciting avenue for future work. We expect the integration of hard constraints into large-scale models to result in more robust performance and more trustworthy machine learning systems.
ACKNOWLEDGMENTS AND DISCLOSURE OF FUNDING
We would like thank the reviewers for their constructive comments. This work was supported as a part of NCCR Automation, a National Centre of Competence in Research, funded by the Swiss National Science Foundation (grant number 51NF40 225155).
REFERENCES
Akshay Agrawal, Brandon Amos, Shane Barratt, Stephen Boyd, Steven Diamond, and J Zico Kolter. Differentiable convex optimization layers. Advances in neural information processing systems, 2019.
Brandon Amos. Tutorial on amortized optimization. Foundations and Trends@ in Machine Leam- ing, 2023.
10

---

Published as a conference paper at ICLR 2026
Brandon Amos and J Zico Kolter. Optnet: Differentiable optimization a layer in neural networks. In International Conference on Machine Learning, 2017.
Brandon Amos, Lei Xu, and Zico Kolter. Input convex neural networks. In International Confer- on Machine Learning. 2017.
Federico Augugliaro, Angela P Schoellig, and Raffaello D' Andrea. Generation of collision-free trajectories for quadrocopter fleet: A sequential convex programming approach. In IEEE/RSJ international conference on Intelligent Robots and Systems, 2012.
Shaojie Bai, Zico Kolter, and Vladlen Koltun. Deep equilibrium models. Advances in neural information processing systems, 2019.
Michal Balcerak, Tamaz Amiranashvili, Antonio Terpin, Suprosanna Shit, Lea Bogensperger, Se- bastian Kaltenbach, Petros Koumoutsakos, and Bjoern Menze. Energy matching: Unifying flow matching and energy-based based models for generative modeling. Advances in neural information processing systems, 2022.
Philippe Baptiste, Claude Le Pape, and Wim Nuijten. Constraint-based scheduling: applying con- straini programming to scheduling problems. Springer New York, 2001.
H.H. Bauschke and P.L. Combettes. Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer International Publishing, 2017.
Yoshua Bengio, Andrea Lodi, and Antoine Prouvost. Machine learning for combinatorial optimiza-
tion: methodological tour d' d'horizon. European Journal of Operational Research, 2021.
Dimitris Bertsimas and Bartolomeo Stellato. Online mixed-integer optimization in milliseconds. INFORMS Journal on Computing, 2022.
Mathieu Blondel, Quentin Berthet, Marco Cuturi, Roy Frostig, Stephan Hoyer, Felipe Llinares-
López, Fabian Pedregosa, and Jean-Philippe Vert. Efficient and modular implicit differentiation. Advances in neural information processing systems, 2022.
Jérôme Bolte and Edouard Pauwels. Conservative set valued fields, automatic differentiation, stochastic gradient methods and deep learning. Mathematical Programming, 188(1):19-5)) 2021.
Jérôme Bolte, Tam Le, Edouard Pauwels, and Tony Silveti-Falls. Nonsmooth implicit differentiation for machine-learming and optimization. Advances neural information processing systems, 34: 13537-13549, 2021.
Jérôme Bolte, Edouard Pauwels, and Antonio Silveti-Falls. Differentiating nonsmooth solutions to
parametric monotone inclusion problems. SIAM Journal on Optimization, 34(1):71-97, 2024.
Stephen P Boyd and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004.
James P. Boyle and Richard L. Dykstra. A method for finding projections onto the intersection of convex sets Hilbert spaces. In Advances Order Restricted Statistical Inference, 1986.
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018.
Joshua Brodie, Ingrid Daubechies, Christine De Mol, Domenico Giannone, and Ignace Loris. Sparse and stable Markowitz portfolios. Proceedings of the National Academy of Sciences. 2009.
Enzo Busseti, Walaa M Moursi, and Stephen Boyd. Solution refinement at regular points of conic problems. Computational Optimization and Applications, 74(3)x227643,, 2019..
Andrew Butler and Roy H Kwon. Efficient differentiable quadratic programming layers: an ADMM approach. Computational Optimization and Applications, 2023.
Steven Chen, Kelsey Saulnier, Nikolay Atanasov, Daniel D Lee, Vijay Kumar, George J Pappas, and Manfred Morari. Approximating explicit model predictive control using constrained neural networks. In American Control Conference, 2018.

---

Published as a conference paper at ICLR 2026
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. Advances in neural information processing sys- tems, 2017.
Laurent Condat. Fast projection onto the simplex and the ball. Mathematical Programming, 2016.
Rares Cristian, Pavithra Harsha, Georgia Perakis, Brian L Quanz, and Ioannis Spantidakis. End-to- end learning for optimization via constraint-enforcing approximators. In Proceedings of the AAAI Conference on Artificial Intelligence, 2023.
George B Dantzig. Linear programming. Operations research, 2002.
Damek Davis and Wotao Yin. Faster convergence rates relaxed Gaaceman-Rachfoce and ADMM under regularity assumptions. Mathematics of Operations Research, 2017.
Steven Diamond and Stephen Boyd. CVXPY: A python- modeling language for convex optimization. Journal of Machine Learning Research, 2016.
Asen L Dontchev and R Tyrrell Rockafellar. Implicit functions and solution mappings. Springer, 2009.
Priya I. Donti, David Rolnick, and Zico Kolter. DC3: A learning method for optimization with hard constraints. In International Conference on Learning Representations, 2021.
N Benjamin Erichson, Michael Muchlebach, and Michael W Mahoney. Physics-informed autoen- coders for Lyapunov-stable fluid flow prediction. arXiv preprint arXiv:1905. 10866, 2019.
Ferdinando Fioretto, Pascal Van Hentenryck, Terrence W. K. Mak, Cuong Tran, Federico Baldo, and Michele Lombardi. Lagrangian duality for constrained deep learning. In Machine Learn- ing and Knowledge Discovery in Databases. Applied Data Science and Demo Track. Springer International Publishing, 2021.
Thomas Frerix, Matthias Nießner, and Daniel Cremers. Homogeneous linear inequality constraints for neural network activations. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020.
Panagiotis D. Grontas, Anastasios Tsiamis, and John Lygeros. Operator splitting for convex con- strained Markov decision processes. arXiv preprint a2X /4l.:2222./4002,, 2024.
Mingyi Hong and Zhi-Quan Luo. On the linear convergence of the alternating direction method of multipliers. Mathematical Programming, 2017.
Ethan King, James Kotary, Ferdinando Fioretto, and Ján Drgoña. Metric learning to accelerate convergence of operator splitting methods. In IEEE 63rd Conference on Decision and Control, 2024.
Anastasis Kratsios, Behnoosh Zamanlooy, Tianlin Liu, and Ivan Dokmanić. Universal approxima- tion under constraints is possible with transformers. arXiv preprint arXtv:2110.03300, 2021.
Giacomo Lastrucci and Artur M Schweidtmann. ENFORCE: Exact nonlinear constrained learning with adaptive-depth neural projection. arXiv preprint arXiv:2502.0677: 2025.
Enming Liang, Minghua Chen, and Steven H Low. Homeomorphic projection ensure neural- network solution feasibility for constrained optimization. Journal of Machine Learning Research, 25(329):1-55, 2024.
Sebastián Maldonado and Julio López. Imbalanced data classification using second-order cone programming support vector machines. Puttern Recognition, 2014.
Danylo Malyuta, Taylor P Reynolds, Michael Szmuk, Thomas Lew, Riccardo Bonalli, Marco Pavone, and Behçet Açıkmeşe. Convex optimization for trajectory generation: A tutorial gen- erating dynamically feasible trajectories reliably and efficiently. IEEE Control Systems Magazine, 2022.
12

---

Published as a conference paper at ICLR 2026
Tobia Marcucci, Mark Petersen, David von Wrangel, and Russ Tedrake. Motion planning around obstacles with convex optimization. Science robotics, 2023.
Pablo Márquez-Neila, Mathieu Salzmann, and Pascal Fua. Imposing hard constraints on deep net- works: Promises and limitations. arXiv preprint arXiv: 2017.
Youngjae Min, Anoopkumar Sonar, and Navid Azizan. Hard-constrained neural networks with universal approximation guarantees. arXiv preprint arXiv:2410.1080) 2024.
Rahul Nellikkath and Spyros Chatzivasilciadis. Physics-informed neural networks for AC optimal power flow. Electric Power Systems Research, 2022.
Brendan O' O'donoghue, Eric Chu, Neal Parikh, and Stephen Boyd. Conic optimization via operator splitting and homogeneous self-dual embedding. Journal of Optimization Theory and Applica- tions, 2016.
Seonho Park and Pascal Van Hentenryck. Self-supervised primal-dual learning for constrained op- timization. In Proceedings of the AAAI Conference on Artificial Intelligence, 2023.
Javier Peña, Juan C Vera, and Luis Zuluaga. Linear convergence of the Douglas-Radee Rachford algo- rithm via generic error bound condition. arXiiv preprini arXiv:2111. 1..00711,,, 2021.
Luis Pineda, Taosha Fan, Maurizio Monge, Shobha Venkataraman, Paloma Sodhi, Ricky TQ Chen, Joseph Ortiz, Daniel DeTTone, Austin Wang, Stuart Anderson, et al. Theseus: A library for differ- entiable nonlinear optimization. Advances in Neural Information Prucessing Systems, 2022.
Maziar Raissi, Paris Perdikaris, and George Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 2019.
Rajiv Sambharya, Georgina Hall, Brandon Amos, and Bartolomeo Stellato. Learning to warm-start fixed-point optimization algorithms. Journal of Machine Learning Research, 2024.
Leon Simon al. Lectures on geometric measure theory. Centre for Mathematical Analysis, Aus- tralian National University Canberra, 1984.
Bartolomeo Stellato, Goran Banjac, Paul Goulart, Alberto Bemporad, and Stephen Boyd. OSQP: an operator splitting solver for quadratic programs. Mathematical Programming Computation, 2020.
Daniel Tabas and Baosen Zhang. Safe and efficient model predictive control using neural networks: An interior point approach. In 2022 IEEE 61 Conference Decision and Control (CDC), pp- 1142-1147. IEEE, 2022.
Antonio Terpin, Sylvain Fricker, Michel Perez, Mathias Hudoba de Badyn, and Florian Dörfler. Dis- tributed feedback optimisation for robotic coordination. In 2022 American Control Conference, 2022a.
Antonio Terpin. Nicolas Lanzetti, Batuhan Yardim, Florian Dorfler, and Giorgia Ramponi. Trust region policy optimization with optimal transport discrepancies: Duality and algorithm for con- tinuous actions. Advances in Neural Information Processing Systems, 2022b.
Antonio Terpin, Nicolas Lanzetti, and Florian Dörfler. Dynamic programming in probability spaces via optimal transport. SIAM Journal on Control and Optimization, 2024a.
Antonio Terpin, Nicolas Lanzetti, Martín Gadea, and Florian Dörfler. Learning diffusion at light- speed. In Advances in Neural Information Processing Systems, 2024b.
Jesus Tordesillas, Jonathan How, and Marco Hutter. Rayen: Imposition of hard convex constraints on neural networks. arXiv preprint arXiv:2307.08336, 2023.
Aaron Tuor, Jan Drgona, Elliot Skomski, Koch, Z Chen, Dembach, CM Legaard, and D Vrabie. NeuroMANCER: Neural modules with adaptive nonlinear constraints and efficient regulariza- tions, 2021. URL htw.t:tt////// http:::::////tt....ooo.ooo////// pnn1 / /neuromancer.
13

---

Published as a conference paper at ICLR 2026
H. A. van der Vorst. Bi-CGSTAB: A fast and smoothly converging variant of Bi-CG for the solution of nonsymmetric linear systems. SIAM Journal on Scientific and Statistical Computing, 1992.
Pascal Van Hentenryck. Optimization learning. arXiv preprint arXiv:2501.0344. 2025.
Andreas Wächter and Lorenz T Biegler. On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical programming. 2006.
Runzhong Wang. Yunhao Zhang, Ziao Guo, Tianyi Chen, Xiaokang Yang. and Junchi Yan. LinSAT- Net: the positive lincar satisfiability neural networks. In International Conference Machine Learning. 2023.
Andrew J Wathen. Preconditioning. Acta Numerica, 2015.
Ezra Winston and J Zico Kolter. Monotone operator equilibrium networks. Advances in neural information processing systems, 2020.
Ahmed Zamzam and Kyri Baker. Learning optimal solutions for extremely fast AC optimal power flow. In IEEE International Conference Communications, Control, and Computing Technolo- gies for Smart Grids, 2020.
Hongtai Zeng, Chao Yang, Yanzhen Zhou, Cheng Yang, and Qinglai Guo. GLinSAT: The general linear satisfiability neural network layer by accelerated gradient descent. Advances in Neural Information Processing Systems, 2024.
14

---

Published as a conference paper at ICLR 2026
CONTENTS
1
Introduction
2
Training Hard-Constrained Neural Networks
2.1
Projection layer
2.1.1
Forward pass
2.1.2
Backward pass
2.2
Loss
2.3
Network architecture
2.4
The sharp bits (short version)
3
Numerical experiments
3.1
Benchmarks and comparisons with state-of-the-art
3.2
Inne applied: Multi-vehicle motion planning
Conclusions
10
Toy visual example
19
Additional results
20
B.1 A comparison with cvxpylayers.
20
B.2
Additional analyses for Section 3.1.
20
B.3
Additional details and results for Section 3.2
24
B.4
Contextual coverage and long-horizon planning
29
B.5
Second order cone constraints.
29
The sharp bits
33
C.I Matrix equilibration
................................
C.2 Auto-tune procedure
35
................................
C.3 Ablations.
35
C,4 Why enforcing constraints during training?
37
C.5 Inet as implicit layer
39
D Derivation details
41
D.I
Derivation Algorithm 1.
D.2
Convergence proof of (4).
D.3 Convergence rates of (4).
41
D.4 Derivation of the backward pass
41
D.5 Computational complexity of Ihnet..
42
More examples constraint sets
44
On the differentiability of the projection layer and the applicability of the implicit func-
tion theorem
46
F.I
Almost everywhere differentiability
46
F22 Applicability of the implicit function theorem
46
F3 Effectiveness of the backward pass in approximating the gradient
46
15

---

Published as a conference paper at ICLR 2026
LIST OF FIGURES
1 Illustration of the Ilnet architecture. The infeasible output of the backbone network is projected onto the feasible set through an operator splitting scheme. To train the backbone network, we use the implicit function theorem to backpropagate the loss through projection layer.
2 Scatter plots of and on the small and large non-convex problems on the test set. The red dashed lines show the thresholds to consider a candidate solution optimal.
3 Comparison of the learning curves in terms of average RS and CV the validation set, on the small and large non-convex problems. The solid lines denote the mean and the shaded JAXopt on the large dataset are reported only in Appendix B because of the orders of magnitude longer training times.
the standard deviation across seeds. The learning curves for
4 (Top) The Ilnet approach constrained multi- motion planning with bitrary differentiable objective functions ş. (Bottom) From left to right, we show examples of the synthesized trajectories for different objectives: Plen e effort, immid = Gleft + preference, fright = Pmid + coverage. We refer the reader Appendix B.3 for formal definitions and additional plots.
5 Scatter plots of RS and CV on the small and large convex problems on the test set. The red dashed lines show the thresholds to consider a candidate solution optimal.
20
6 Comparison of the learning curves in terms of average RS and CV on the validation set, the small and large convex problems. The solid lines denote the mean and the shaded area the standard deviation across seeds. The learning curves for JAXopt the large dataset are reported in Figure 7 because of the orders of magnitude longer training times.
22
7 Comparison of the training times in terms of and for the different methods in the small and large convex problem setting. The solid lines denote the mean and the shaded area the standard deviation across 10 seeds. Note the different timescale on the large dataset results, due to the large training time of JAXopt.
23
8 Comparison of the learning curves in terms of average RS and CV on the validation set, on the small and large non-convex problems. The solid lines denote the mean and the shaded area the standard deviation across 5 seeds.
24
9 Scatter plots of RS and CV on the small and large non-convex problems on the test set. The red dashed lines show the thresholds to consider a candidate solution optimal. We superimpose the results of the main text with the ones obtained using more correction steps
smaller soft penalty in the DC3 algorithm.
24
10 Preference and coverage. On the left depict the landscape of the potential (x) with the working space delimited by the red, dashed rectangle. In the middle we report the image used compute the coverage score of the trajectory on the right.
26
11 Collection of synthesized 5-vehicles trajectories. Each row relates to different instance initial and final configurations, and from left to right we report the gen- erated trajectory with network trained with (A, (), (A.v) (1, and (X, (1, respectively.
27
12 Collection synthesized 15-vehicles trajectories. Each row relates to different instance of initial and final configurations, and from left to right report the gen- erated trajectory with a network trained with (A, v) = (0,0), (A,v) (1, 0) and (A, 1) (1, respectively.
28
13
Examples of resulting trajectories for the experiments in Appendix B.4.
29
14 Scatter plots of RS and CV on the second-order cone programs on the test set. The red dashed lines show the thresholds consider candidate solution optimal.
15 Comparison of the learning curves in terms of average RSS and CV the validation set, on the second-order cone programs. The solid lines denote the mean and the shaded area the standard deviation across seeds.
32
16

---

Published as a conference paper at ICLR 2026
16 Visualization of the ablation results in Appendix C.3. Scatter plots of RS and CV for the methods on the large non-convex problems on the test set. The red dashed lines show the thresholds that require consider candidate solution optimal.
36
17 Comparison of the training times in terms of RS and CV for the different ablation configuration on the large non-convex problem setting. For simplicity, we report the learning curves only for a single seed.
36
18 Scatter plot of RS and CV on the small non non-convex test problems for a Inne net- work trained with 50 iterations and evaluated with different numbers of iterations at test time. The red dashed lines indicate the thresholds used to consider a candidate solution optimal.
37
19 Scatter plot of RS and CV on the small non-convex test problems for Inet net- work trained with different values of and (top) 50, (bottom) 100 forward iterations. The red dashed lines indicate the thresholds used to consider candidate solution optimal.
38
20 Scatter plot of RS and on the small non-convex test problems for Ilnet net- work trained with different values of w. The red dashed lines indicate the thresholds used consider a candidate solution optimal.
38
21 Constraint violation for 100 problem instances (light blue) in the test set of the small non-convex benchmark as the number of iterations in the forward pass n. ieeer fwd (i.e., n.iter seieerttriin increases, well the maximum CV among all instances (dark blue).
42
22 VJP estimation error (measured via the form of the difference and the cosine sim- ilarity) for 100 different vectors e; instances (light blue) for two different instances of the small non-convex benchmark as the number of iterations in the backward pass n_ieer bwd increases, as well as the maximum ez-error and the minimum cosine similarity among all instances (dark blue).
17

---

Published as a conference paper at ICLR 2026
LIST OF TABLES
Comparison with cvxpy layers on the small, non-convex benchmark. The RS and CV values reported are the averages over the test set.
20
2 Inference time comparison for single-instance and batch (1024 contexts) settings, evaluated on the small and large non-convex problems. The table reports the median, lower quartile (I.Q. 25th percentile), upper quartile (UQ, 75th percentile), min and max of the runtime.
21
3 Inference time comparison for single-instance and batch (1024 contexts) settings, evaluated the small and large convex problems. The table reports the median, lower quartile (LQ, 25th percentile), upper quartile (UQ, 75th percentile), min and of the runtime. We report results of the Solver (i.e., OSQP) in two modes, normal and parametric labeled with Solver and Solver respectively. Paramet- ric mode means that inform OSQP that the same structure, which speeds up solution time by reusing calculations across consecutive calls, We note that this is feature of OSQP that may or may not be available in other solvers.
repeatedly solving problems with
21
4 Inference time comparison single-instance and batch-instance (1024 problems) settings across different methods, evaluated small and large non-convex prob- lems. The table reports median runtime along with statistical descriptors: lower quartile (LQ, 25th percentile), upper quartile (UQ. 75th percentile), min and max of the runtime. DC3 uses more than the default correction steps.
25
5 Inference time comparison for single-instance and batch-instance (1024 problems) settings across different methods, evaluated on the second-order cone programs. The table reports median runtime along with statistical descriptors: lower quartile (I.Q, 25th percentile), upper quartile (UQ, 75th percentile), min and max of the runtime.
31
6 Inference time comparison for single-instance and batch-instance (1024 problems) settings across different ablation configurations, evaluated the large non-convex problem. The table reports median runtime along with statistical descriptors: lower quartile (LQ, 25th percentile), upper quartile (UQ, 75th percentile), min and max of the runtime.
36
18

---

Published as a conference paper at ICLR 2026
A TOY VISUAL EXAMPLE
In this section, we instantiate our method very simple and interpretable toy problem: learning to predict the outputs a model predictive control (MPC) policy (Chen et al., 2018). In high- dimensional scenarios and for long horizons, running MPC in real-time often becomes compu- tational bottleneck, so learning to predict its outputs-even with minor suboptimality, as long as constraints are still satisfied-can bring substantial benefits (Chen et al., 2018).
Experimental setting. We consider a two-dimensional single integrator system, and define an MPC control law given by the solution
N-1
minimize 1,E
Σ 1a +1 1|ux1 subject to To = x, xx+1 3kk Uk Tk E [-10, 10] (10) K=
Here, denotes the system state, ug the control input, and 2 [3, -12 is target state. We sample the context (in this case, the initial condition for the MPC), x zu, uniformly in [-10, 10]². We deploy Innet to learn the solution of (10) as function of x. We use a self-supervised loss and the backbone NN is an MLP with two hidden layers of 200 neurons and ReLU activations.
Results. We superimpose IInet prediction and the solution of (10) given by Solver. Further, to illustrate the importance of hard constraints, we also plot the prediction of the same MLP trained with soft constraints, i.e., adding to the loss the term INEQCV)) where EQCV and INEQCV is the maximum violation of the equality and inequality constraints, respectively, for two values of A.
We represent with a star, x with square, and state constraints with a dashed rectangle. We tested the trained network in 1000 instances generated by sampling x uniformly in [-10, 10)2. Inne achieves an average relative suboptimality of approximately 0.02%, and constraints satisfied in 100% of problem instances within tolerance 10-3 parent that different values of À induce different be- haviors: Larger values enforce the constraints at the expense of optimality, smaller values do not enforce constraints.
llnet
Solver
SoftMLP SoftMLP 0.001
also ap-
*T
19

---

Published as a conference paper at ICLR 2026
Method
CV
Single inference time [s]
Batch inference time [s]
cvxpylayers
0.0036
0.00005
0.0120
2.5917
Ilnet (CPU)
0.0035
0.00000
0.0052
0.0957
Ilnet (GPU)
0.0035
0.00000
0.0065
0.0135
Table I: Comparison with evxpylayers on the small, non-convex benchmark. The RS and CV values reported are the averages over the test set.
D03
TAXapt
Ilnet
Convex
Small
Large
10
10-1
10
10-2
2
2
10-1
10
10-14
10-4
10-14
10-12
10-10
10
10
20
10
10P
10
10-12
10
10
10
10
10-7
10P
Figure 5: Scatter plots of RS and CV on the small and large convex problems on the test set. The red dashed lines show the thresholds consider a candidate solution optimal.
B ADDITIONAL RESULTS
In this section, collect the additional results for Section 3 and the ablation studies DC3 hyperparameters.
B. A COMPARISON WITH CVXPYLAYERS.
We compare Innet with cvxpylayers on the small non-convex benchmark, and report the sults in Table 1. We use cvxpy layers as alternative to our custom projection layer, in an analogous manner to the comparison with JAXopt Section 3.1. Since evxpylayee layers runs exclusively the CPU, report the results of llnet both CPU and GPU. In all cases, provide training budget of 25 epochs. For cvxpylayers this corresponds to roughly 10 minutes of wall-clock time, whereas for Inee this corresponds to 4 and 28 seconds on the GPU and CPU, respectively. We observe that Inee attains similar performance in terms of RS and CV, but signifi- cantly outperforms evxpy 1 ayers in terms of inference and training time. We omit further results with cvxpy ayers, since JAXopt is more recent and stronger baseline: it implements similar functionalities, but it is executable the GPU (see Figures and 3, and Table 2).
B.2 ADDITIONAL ANALYSES FOR SECTION 3.1.
Additional results. We report the inference times on the non-convex datasets in Table 2. We report the omitted benchmark results on small and large convex datasets of Section 3.1 Figures and 6 and Table 3, which further substantiate our claims. We report the omitted learning curves for JAXopt on the large convex and non-convex datasets in Figure 7 and Figure 8, respectively.
DC3: Hyperparameter ablations. In our benchmarks of Section 3.1, we highlighted two weak- nesses in the performance of DC3: (i) in both small and large non-convex problems the RS is un- satisfactory; (ii) in the large non-convex problem the CV is unsatisfactory. We investigated whether these shortcomings can be mitigated by choosing hyperparameters which are different from de- tried to address (i) by decreasing the soft penalty parameter for CV in DC3's
faults. In particular, loss function from the default 10.0 to 2.0; (ii) by increasing the number of correction steps from the default 10 50. We superimpose the obtained RS and CV in Figure 9, and report the inference times in Table 4 (only for the case of more correction steps, since changing the soft penalty does not affect inference times). Importantly, these results show both the lack of clear decoupling between
20

---

Published as a conference paper at ICLR 2026
Single inference [s]
Batch inference [s]
Method
median
LQ
UQ
min
median
TQ
UQ
min
Non-convex small
DC3
0.00T9
0.0018
0.0019
0,0018
0.00226
0,0020
0.0019
0.0020
0.0019
0.0023
Solver
0.0334
0.0298
0.0497
0.0207
0.1213
41.748
40.200
43.198
36.764
47.149
JAXopt
0.0134
0.0131
0.0136
0.0122
0.0145
0.1371
0.1364
0.1532
0.1357
0.1540
Inet
0.0056
0.0055
0.0056
0.0054
0.0072
0.0130
0.0128
0.0131
0.0123
0.0147
Non-convex
large
DC3
0.0016
0.0016
0.0017
0,0015
0.0046
0.0248
0.0248
0.0248
0.0247
0.0255
Solver
10.159
9.4739
10.807
7.4350
12.612
10720
9538.4
10800
9417.8
16502
JAXopt 0.0578
0.0575
0.0590
0.0559
0.0596
19.430
19.430
19.430
19.429
19.430
lInet
0.0063
0.0063
0.0064
0.0061
0.0092
0.2804
0.2799
0.2807
0.2794
0.2912
Table 2: Inference time comparison for single-instance and batch (1024 contexts) settings, evaluated on the small and large non-convex problems. The table reports the median, lower quartile (LQ, 25th percentile), upper quartile (UQ, 75th percentile), min and max of the runtime.
Single inference
Batch inference [s]
Method
median
LQ
UQ
min
max
median
LQ
UQ
min
max
Convex small
CC3
0.0033
0.0033
0.0033
0.0031
0.0050
0,0033
0.0033
0.0034
0.0032
0.0044
Solver
0.0019
0.0018
0.0019
0.0011
0.0083
1,9350
1.9264
1.9441
1.8931
2.0282
Solver
0.0006
0.0006
0.0006
0.0006
0.0024
0.6190
0.6168
0.6217
0.6100
0.6746
JAXopt
0.0142
0.0138
0.0150
0.0124
0.0165
0.1603
0.1590
0.1648
0.1581
0.1794
Inne 0.0055
0.0055
0.0056
0.0053
0.0060
0,0130
0.0128
0.0130
0.0125
0.0136
Convex large
DC3
0.0072
0.0072
0.0073
0.0071
0.0115
0.0349
0.0349
0.0351
0.0348
0.0369
Solver
0.6660
0.6613
0.6716
0.3159
0.7079
680.59
676.24
682.52
662.29
687.84
Solverf
0.0603
0.0589
0.0620
0.0466
0.9501
62.022
61.591
62.558
60.064
70,987
JAXopt
0.0630
0.0623
0.0641
0.0601
0.0668
21.504
21.504
21.505
21,504
21.505
Iine
0.0063
0,0063
0.0064
0.0061
0.0145
0,2800
0.2797
0.2804
0,2794
0.2851
Table 3: Inference time comparison for single-instance and batch (1024 contexts) settings, evaluated on the small and large convex problems. The table reports the median, lower quartile (LQ. 25th percentile), upper quartile (UQ, 75th percentile), min and max of the runtime. We report results of the ver ver (i.e., OSQP) in two modes, normal and parametric labeled with Solver ver and Sol vert, respectively. Parametric mode means that inform OSQP that with the same structure, which speeds up solution time by reusing calculations across consecutive calls. We note that this feature of OSQP that may or may be available other solvers.
repeatedly solving problems
21

---

Published as a conference paper at ICLR 2026
DC3
JAXopt
Inne
Convex
Small
10
10"
10P
10-7
10
10-2
50
100
150
200
250
300
100
150
200
250
300
Time [s]
Time
Large
10a
10a
101
10-1
10-3
10
10-7
200
3.0
503
700
100
300
600
Time
Figure 6: Comparison of the learning curves terms of average RS and CV on the validation set, on the small and large convex problems. The solid lines denote the mean and the shaded area the standard deviation across seeds. The learning curves for JAXopt on the large dataset are reported in Figure 7 because of the orders of magnitude longer training times.
22

---

Published as a conference paper at ICLR 2026
DC3
JAXopt
Inne
Convex
Small
10
10"
10P
10-7
10
10-2
50
100
150
200
250
300
100
150
200
250
300
Time [s]
Time
Large
10a
10a
101
10-1
10--
10-7
5
-109
Time
100
Figure 7: Comparison the training times in terms of and CV for the different methods in the small and large convex problem setting. The solid lines denote the mean and the shaded area the standard deviation across 10 seeds. Note the different timescale on the large dataset results, due to the large training time of JAXopt.
23

---

Published as a conference paper at ICLR 2026
DC3
JAXopt
Inne
Non-convex
Small
10
10F
172
20-2
2
10"4
10
10-1
20
60
50
1000
120
140
100
190
200
20
10
80
100
120
1.00
160
180
200
Timn
[+]
Large
109
10'
10
10-2
10
6
Time
-10
Time [H]
1201
Figure 8: Comparison of the learning curves in terms of average RS and CV on the validation set, on the small and large non-convex problems. The solid lines denote the mean and the shaded area the standard deviation across 5 seeds.
more corrections
DC3 less soft penalty
llaet
Non-couvex
Small
Large
10
10-1
10
12
10-4
10
10
10
Figure 9: Scatter plots of and CV on the small and large non-convex problems on the test The red dashed lines show the thresholds to consider candidate solution optimal. We superimpose the results of the main with the ones obtained using more correction steps and smaller soft penalty in the DC3 algorithm.
the various parameters of the DC3 algorithm and the advantages of our projection scheme; also Appendix C.
B.3 ADDITIONAL DETAILS AND RESULTS FOR SECTION 3.2
We detail the problem setup for the multi-vehicle motion planning application in Section 3.2 and present additional results. Recall that denote with px(t] RT the generalized coordinates of vehicle at the discrete times E (1...., T + 1}. and with 1/[t] and a, its generalized velocity and acceleration. Its simple discretized dynamics read vz(t 1] wa(() har(t), pe(t 1]
24

---

Published as a conference paper at ICLR 2026
Single inference [s]
Batch inference [s]
Method
median
LQ
UQ
min
max
median
LQ
UQ
min
max
Non-convex small
0.0080
0.0080
0.0080
0.0079
0.0087
0.0081
0.0081
0.0082
0.0081
0.0083
Solver
0.9724
0.6925
1.4587
0.3562
3.6359
1258.8
1243.8
1264.3
1185.2
1299.0
Unnet 0.0056
0.0055
0.0056
0.0054
0.0072
0.0130
0.0128
0.0131
0.0123
0.0147
Non-convex
large
DC3
0.0070
0.0069
0.0073
0.0068
0.0105
0.1177
0.1177
0.1177
0.1176
0.1180
Solver 6.8153
6.7161
6.9636
6.2371
12.046
6991.4
6986.1
7498.7
6964.0
7523.3
llnet 0.0063
0.0063
0.0064
0.0061
0.0092
0.2804
0.2799
0.2807
0.2794
0.2912
Table Inference time comparison for single-instance and batch-instance (1024 problems) settings across different methods, evaluated small and large non-convex problems. The table reports median runtime along with statistical descriptors: lower quartile (LQ, 25th percentile), upper quartile (UQ, 75th percentile), min and max of the runtime. DC3 uses more than the default correction steps.
pb[d] + [tv[t] + 22 a.(]] We have constraints on each of these variables and, thus, we consider as optimization variable [pT
e
a
with
p [P1[0]
PN [0]
P1 [T]
PN [T]
[e [O]
UN [0] T
[T]
UN
[a, [0]
[N
a1 [ 1] aN 1- 1]
where N is the number of vehicles. Below, trajectories that go from a given set of initial positions yi ... to a set of final positions P1[T+ 1] ..N [T 1]. This the context of the optimization problem, i.c.
2. We want the network to generate
x p [0]T
PNJOTT
[T]]
PN + 1) 17]
The constraints on the system are:
Dynamic constraints. The optimization variables p, v, a are related by the system dynamics, via equality constraint of the type Asya'/ = 0.
Initial and final positions constraints. We ensure that the optimal y satisfies the given initial and terminal position constraints via the equality constraint Aiy = b(x). Importantly, we observe that the context affects the constraint via the vector b(x), which in this case is simply
Workspace, velocity and acceleration constraints. These constraints impose box constraints 1p S p S 4p S S up. la I S Ua.
Jerk constraints. Jerk constraints limit how abrupt the change in accelerations can be. These are affine inequality constraints the type S (a,[1 ( 1] (4(4)/ S u, which we can compactly write as lienk v Cy & Ujerk for appropriate Cjek, Ujjrr-
The objective is to minimize
( effort(y) + preference(y) w- coverage(1)
where effort(y) (y) describes the input effort of the solution y. preference(y) describes the fit- ness of y with respect to a spatial potential vh and coverage(y) describes the fraction of the tuning parameters. In experiment,
workspace that the agents cover we use them as binary variables to show the effects of adding certain terms to the objective function. The different terms are defined follows:
time, and X,
Input effort (effort). The input effort is
N T
effort(y)
Σ
|
0=0
25

---

Published as a conference paper at ICLR 2026
Figure 10: Preference and coverage. On the left depict the landscape of the potential v²()) with the working space delimited by the red, dashed rectangle. In the middle we report the image used to compute the coverage score of the trajectory on the right.
Individual contribution (preference). Each vehicle tries to minimize the cumulative value along the path of the scaled Ishigami potential (Terpin et al., 2024b)
4
0.05
sin(21) sin (2))
21
22
sin(z1)
1.25
depicted the left of Figure 10. That is,
preference())
ΣΣ (p.(t)).
Fleet contribution (coverage). We define the coverage over time as the fraction of the space (( (Pmaa))1 )1] x [(Pmiu) 22 (Pmax)2] the vehicles sweep over during their trajectory. We map the position of the jth vehicle into continuous pixel-space of size H W 16 x 16 via
= (p:)t)) (Pmin) W
(p. [t)) (Puni)) H. (Pmax)2 (Pmin)2
= H
(Pmax): (Pmia)1
On each pixel (u, v) we place a bivariate Gaussian
1
(u );)²
(v (v=)))²
2p(u- us)(v 1z) )
Gi(u, v) = A Axxp
2(1 p2)
OzOg
where A 200, J2 obtain a coverage image,
By 1,p 0 are hyperparameters.. We sum (and clip) the N kernels to
I(u, v) min Σ
We display / for a sampled trajectory in Figure 10. Finally, the coverage score is computed as coverage(y)
(p.
HW
255
measuring the fraction of pixels covered by the Gaussian footprints differentiable manner.
Overall, the optimization problem reads:
Air
y = b(x)
Upp
minimize yE
(p)) subject to
Adyn
S
Uhy Mua
lock S Cy S Ujerk-
In Figures 11 and 12 report various trajectories generated with Inne with different number of vehicles N E {5, 15] and horizon T = 25. These additional qualitative results show the effective- ness of Inet in synthesizing trajectories that optimize non-convex, fleet-level preferences also for large fleets and long horizons: in the largest setting reported here, there are n 3030 optimization variables (d 2280).
26

---

Published as a conference paper at ICLR 2026
San
Figure 11: Collection of synthesized 5-vehicles trajectories. Each row relates to different instance of initial and final configurations, and from left right report the generated trajectory with network trained with (A,v) = (0, 0), (A,v) = (1, 0) and (A, v) = (1, 1), respectively.
27

---

Published as a conference paper at ICLR 2026
Figure 12: Collection of synthesized 15-vehicles trajectories. Each row relates to a different instance of initial and final configurations, and from left right report the generated trajectory with network trained with (A,v) = (0, 0), (A,v) = (1, 0) and (A, v) = (1, 1), respectively.
28

---

Published as a conference paper at ICLR 2026
H- 100, res = 1024
750, res 64
Figure 13: Examples of resulting trajectories for the experiments in Appendix B.4.
B.4 CONTEXTUAL COVERAGE AND LONG-IIORIZON PLANNING
Building on top of the trajectory planning application, we provide some preliminary evidence that Inee capable of handling:
1. very high-dimensional context size (in the millions of variables); and
2. a large number optimization variables and constraints (in the tens of thousands).
For this, we focus on single vehicle with the same constraints as in the previous section, increase the horizon length, and consider as cost function the average along the trajectory of the value as- signed (via bilinear interpolation) res x res. Thus, the context x corresponds to the initial and final location well as the map We generate the map by sampling uniformly at random one of four locations
the position the vehicle by a discrete map m of resolution
c E {( [--2., 2.5,- 2.5).. (-2.5, 2.5), (2.5, 22.5,, (2.5, 2.5)]
Then, the res x res pixels comprising the map m correspond to a uniform discretization of the grid [-5,5] [-5, 5]. with values resulting from a Gaussian with center e and standard deviation of 2. This simple construction (in the sense that the single parameter controlling the map m is which was used) allows to test how Innet deals with backbone networks that have as input high- dimeemsinnii contexts, while keeping fixed the number of optimization variables.
For this, we use a convolutional neural network with three layers, 16 features and downsampling factor D for each layer, followed appended to the vector input (the initial and final positions) as input to the same MLP used in the previous experiments. We note that the convolutional neural network and the MI.P trained jointly as the backbone of llnet. We consider two cases:
linear layer and softmax, resulting in 4 logits. These
4. this case, we evaluate if Innt enables training end-to- end of neural networks with constraints comparison, Donti et al. (2021) consider a context of dimension 50).
H = 100, res = 1024,
when the context is very high-dimensional (for
H 750, res 64, to that typical of, e.g., reinforcement learning settings and a number of optimization variables and constraints
In this case, evaluate Ilnet for context of size similar
orders of magnitude larger than state-of of-the-art parametric optimization al- gorithms (for instance, Donti et al. (2021) consider 100 optimization variables, and 50 equality and inequality constraints).
In both settings, Inet successfully learns solve the planning task, qualitatively shown in Figure 13. Despite the simplicity of the maps used, we believe that these examples show the promise of using Inet for high-dimensional optimization problems with very large contexts.
B.5 SECOND ORDER CONE CONSTRAINTS.
In this section, provide evidence that Inee can be used to solve parametric problems with second-order cone constraints. By doing so, we also elaborate on how one can introduce additional constraint types into Innet effortlessly.
Experimental setup.
We focus problems of the form:
minimize (v)
c cT subject to Ay + = =b, 3/2 E Kg
(11)
29

---

Published as a conference paper at ICLR 2026
V2. b Rs K2 {uz R I112,--1l2 S y22-1 is second- order cone, and y2.--1 denotes the first d2 entries of y2, and y2,-1 the last one. We make the following two observations:
where A E Rd V1 Rd
First, the feasible set is non-linear and possibly unbounded. Thus, methods like GLinSAT (Zeng et al., 2024) are not applicable.
Second, one cannot train an unconstrained network and then apply the projection layer, since the optimization objective is linear and, thus, the network would diverge. See also Appendix C.4.
To generate the problem data, we follow the procedure described by O'donoghue et al. (2016, Sec- tion 6.6), reported here for completeness and to clarify the elements of the random parametric prob- lems we consider. First, we generate a random matrix A E Rd sampling uniformly at random in [-1,1]daxd, The matrix is fixed for all the problems of a training run, i.e., it is not a con- text/parameter of the problem. In other words, A is context-independent. Given A, we generate batch of B problems as follows:
1. We sample B random vectors z uniformly in [-1, 1jd and random primal solutions yf IRd: We project z onto the second-order cone obtain 1/2 Hxx (z).
3. We set b Ayi + y/2 and A (vă - =).
The optimal value of the problem is then given by c yī. and the context x (i.c., the input to the NN), is x cb. For Ilnet, we employ the same MLP used in Appendix A followed by the Inet projection layer, so that the optimal output should yield ÿ (01. 02) with e in R c yi- For the experiments in this section, we use (d1, d2) E {(25, 25). (500, 500)} (we refer to these configurations as small and large, respectively) and B 128. We train the network for 1000 epochs with a different batch at each epoch, and then generate an additional one for evaluation at the end of the training, which we calculate the statistics. We compare the performance of our method against:
cvxpylayers Agrawal et al. (2019), which we use as a replacement of our custom projection layer. This is the same procedure we adopted in Section 3.1 for JAXopt. Here, we consider evxpylayers because JAXopt does not support second-orde cone constraints.
sCs O'donoghue et al. (2016), traditional first-order solver for second-order cone programs. To make our comparison with SCS as fair and comprehensive as possible, we perform two bench- marks. On one, we interface SCS through the very popular parser CVXPY (Diamond & Boyd, 2016) using its Disciplined Parametrized Programming functionality, many users would do. On the other benchmark, we interface SCS directly and exploited its native parametric programming functionality, more advanced users would.
Remark. It is important to note that with these benchmarks we are exploiting the full paramet- ric capabilities of existing solvers, which is not always the case by end-users or even other hard- constrained NN benchmarks.
Similarly to the constracon benchmarks in Section 3.1, we consider learning curves, RS, CV, and inference times.
Integrating (11) into Inet. Equivalently, we can write (11) as:
minimize
[cT 0] subject to [A yy==b. y K
(12)
with K,, Rd x K2, K2 being the second-order cone, and without K₂ since we not use any auxiliary variable (i.e., = d = d, - d2). We recall here that K., and K₂ refer to the problem formulation in Section 2.1. In view of Appendix E, this amounts to lifted formulation of second- order cones and is thus without loss of generality. Now, (12) is in our standard formulation and we can apply Algorithm 1. In particular, with the notation used Algorithm 1, we only need to define the projection
t++1 = II,, 222+1.1 - Sk,1 + 20
1 20
Denoting with 1.41,ed 2,,11,1,,d $k,1,eds Braw,ed, the first d entries of the vectors
4K-1- 1, 1,4,,1, 8k,1, Dla.11 and with 1,411, Fk 1,1, 1,41, dr the remaining da entries,
30

---

Published as a conference paper at ICLR 2026
exxpylayer
llbet
Suall
Large
10
10-1
10-
10-14
10-13
10-10
10-4
20-4
10-4
10-10
10-14
10-12
10-10
10
10-6
10-2
Figure 14: Scatter plots of RS and CV on the second-order cone programs on the test set. The red dashed lines show the thresholds to consider a candidate solution optimal.
Single inference
inference [s]
Method
median
LQ
UQ
Inin
median
LQ
UQ
Riin
Small scond-order cone programs
CVXPY
0.00076
0.00274
0.00078
0.00071
0.00084
0.59085
0,58784
0.59399
0.57668
0.70665
SCS
0.00005
0.00004
0.00005
0.04612
0.04580
0.04652
0.04483
poxpylayers
0.00857
0.00813
0.012700
0.00706
0.06567
1.82533
1.8570
4.47413
166853
5.73859
Thet
0,00905
0.00884
0.00928
0.02245
0.02292
Large second-order cone programs
CVXPY
0.06721
0.00658
0.65788
62,70789
62.65441
62.76826
62.46146
63.31168
SCS
0.01072
0.01049
0.010788
0.00976
0.01155
10.97265
10.80431
11,10839
10.68313
26, 18852
cvxpylayers
2.36724
llaet
0.01219
0.01187
0.01356
0.01113
0.04383
0,90982
0.90932
0.91134
0.90844
0.94499
Table Inference time comparison for single-instance and batch-instance (1024 problems) settings across different methods, evaluated on the second-order cone programs. 'he table reports median runtime along with statistical descriptors: lower quartile (LQ, 25th percentile), upper quartile (UQ, 75th percentile), min and max of the runtime.
have
2 1,1.1,
-k,11:d; 20 1.:w.
1x+1,
and
1 2a
22x4122d 8k,2,d, + 20 20w 2,d
1x41,11 = 11x;
2a
The projection 14, (-) onto the second-order cone admits a closed form expression; see, e.g., the work of Busseti àl. (2019).
Results. In Figure 14 report the RS and CV for each problem instance in the test set. Analo- gously to the linearly constrained benchmarks in Section 3.1, we consider a candidate solution to be optimal if the condition CV 10-3 and RS 5% is satisfied. We report training curves in Figure 15 and inference times in Table 5.
Compared to cvxpylaye ayers, Inee can train and perform inference significantly faster, even or- ders of magnitude for large problems. Moreover, Inne provides solutions with both lower RS and
CV. Compared to SCS, Inne is faster at inference for a batch of small problems, and both a single and batch of large problems. Instead, SCS is faster for single small problems.
31

---

Published as a conference paper at ICLR 2026
cvxpylayers
Ilne
SOC
Small
.0P
100
10-1
8 20-6
10-1
10
0
50
100
160
200
230
300
100
150
200
250
300
Time 6]
Time
1.0
10-1
10-3
MC
10
10"9
10-12
10-12
1,000 2,000 3.000
4.000
5,000 6.000
7,000
5,000
9,000
1,000 2,000
3,000
4,000
5,00
6,000
7,000 8,000 9.000
Time b]
Time [s]
Figure 15: Comparison of the learning curves in terms of average RS and CV on the validation set, on the second-order cone programs. The solid lines denote the mean and the shaded area the standard deviation across seeds.
32

---

Published as a conference paper at ICLR 2026
C THE SHARP BITS
Numerical optimization is art of details, and this section describes the sharp bits of Ilnet, which are fundamental to make HCNNs work reliably.
C.I MATRIX EQUILIBRATION
The precision and convergence rates of both the forward (Section 2.1.1) and backward (Sec- tion 2.1.2) passes in our projection layer are heavily influenced, via (4a) and (9), by the condition
number of the matrix A(x) in the equality constraint A(x)
y b(x: (Wathen, 2015). The con-
Yaux
dition number of a matrix A (we drop the dependency on for clarity of exposition) is defined as the ratio of its maximum singular value its minimum singular value, and the procedure of decreasing this ratio for improving numerical performance is called preconditioning or equilibration.
To achieve this, we seek a diagonal scaling (Wathen, 2015). The idea is that to solve
b,
Yaux
ÿ Yaux
one can instead solve D,
D,b for invertible matrices D, E mx De E Raxx
=
and recover the solution as
De
We implement a modified version of Ruiz's 's equi-
Maux
Maux
libration (Wathen, 2015) which outlined in Algorithm In our numerical experiments we use K = 25, E = 10-3 and the Gauss-Seidel update mode.
ALGORITHM 4. Modified Ruiz's equilibration
Inputs: Rmxn, maximum iterations K, tolerance e, update mode (Gauss-Sein) or Jacobi)
D, 4 Im Do x In
Ascaled + A
For k 1 to K:
If Gauss-Seidel update:
Compute row norms dr,5 = [|(Aacalod) 12 for all
D, - diag(1/ (dr,,..... 1/ /d..m) Dr
Ascated f- diag(1/ (1/ 1/1d/// /d......... 1/ dr.m) Ascated
Compute column norms de,, (Aggated) d for all
D. - D. diag(1 ///e/e,11... Vee.i...., 1/v den))
Ascato Ascaled diag(1/ Vdc.l.....
/de,.)
Else:
Compute row norms d = |[(Accalod) 2.: for all
Compute column norms de,s = [Aseated)) alle for all
D, + diag(1/ / Id..,...,,11
dr.)) Dr
D D. - diag(1/ (llag(///e.I,...... /d..l....,1, /on)) Ascated + diag 1/v / dr....... 1/v dr.m) Ascaled diag(L/ / l.....,,////dd. Compute row norms dr = [[(Ascatod)
1/ don)
for all
Compute column norms de, = [I (Ascatod) ..ul2 for all [1 min(d,.)) (dr,s) max(d, ()) < e and min(de.)) mmcm///ma)l)))))) < Return D,, De
Return Dr, D.
Next, we describe how the matrix equilibration affects the Douglas-Rachford Rachford algorithm in the case of polytopic constraint sets. To start, we observe that K. is box constraint and, thus, can be written as
K K1 K2 K1,1 x... Ki.d K2,1 ... K2,n-d-
Then, rewrite (3) in terms of the new coordinates:
D
D
8 Yaux
Faux
D
yaux
c,2
33

---

Published as a conference paper at ICLR 2026
That is, since
y/
E A (ul D,AD, D,b) - =b]
A >
Maux
Maux
EK
-
11 Hlaxx
E Kol X ... dedKia
Blaux
dod+i K2,1 x... den
K1,1 x x... Ri R2 ... x R2
Ri R2,
we seek to solve
(
1 ylux
1)
(Ilc( Manx) De argmin
|D Vraw +
Ize
(13)
Then, we split the objective function as follows
)
Ií
and
|aaux (
I|Do, V/uw
2 Ix
Vaux
Vaux
and by applying the Douglas-Rachford Rachford algorithm we obtain the fixed-point iteration
[ = proxxog(88 = (5ka) [2
Ex+1
(14a)
k == 0, K-1
8441 = prox on (28x+1 - 5z) [Txx (28x+12 5n.2)
(14b)
[kt11,1
8k+1
3x w(tx+ - ZR+1)
(14c)
$k+1.2
where 4 = =.... Oi ...]". with (using the notation (v); to indicate the ith entry of the vector v)
4 = KR.. 20do, + 22(x++1,1) .)- (Sx,
1 2
The equilibration effectively only changes (14b). To prove the update in (14b), we note that equili-
bration does not affect the solution of the proximal in (14b) with respect to the auxiliary variables:
1
5
prox (25x+1 5x) = argmin [D=, Vraw 112
+ Ig
2
+
Taux
20
Taux
argmin DD Braw 1|2 vek,
1
2
20
argmin 2o Uaux 25x+1.2 + k2.21
Gum ER2
Ix (22x- 1,2 8x.2)
In particular, since
S Vaux Su
D-11 & Douy S D D S Dmx S D
the projection onto 2. remains a projection onto a box, but we modify the upper and lower bounds of the box according to Da.- We can write O by completing the square, as
argmin [I + 20 Dz, 8 De. + +8 5x. 1) oeR,
34

---

Published as a conference paper at ICLR 2026
and, thus,
4 arginn [11 + 20d 0 -2odd,, + -2(2++1,1) (3)1
20de, + +2(2x+1,2); (5k,1)1
argmin aeKi,
+ 201e
20de + 2( (B0,1)8 + 2
R,,
Although we described the case with box constraints, our changes can be adapted for other constraint classes such as second-order cone constraints by equilibrating equally variables that are coupled by the constraints. We leave these modifications for future work.
C.2 AUTO-TUNE PROCEDURE
During the auto-tuning procedure, we will tune the hyperparameters sigma and n_iter_ffd. For the remaining ones, we set omega and n_iter to their respective default values 1.7 and 25. For simplicity, we n_iter_test
be equal n.iter_fwd.
For the auto-tuning, we consider a subset of the validation set consisting of 150 contexts out of the 1024 used in the benchmark problems of Section 3.1. Then, we generate 150 corresponding cotts-to-be-projecce from standard normal distribution N(0, I). which will serve as proxy for the infeasible output of the backbone NN. First, to tune s sigma we generate 100 logarithmically- spaced number between 10 and 5.05. Then, compute the projection for each si sigma the 150 context-infeasible point pairs, by running 100 iterations of the projection layer. We evaluate the quality of each s gma by computing the resulting maximum constraint violation and average relative suboptimality of the projection, i.e., ||= Braw TTcxx) (Fraw) Vrow ||- We consider any candidate hyperparameters. We choose among the candidate hyperparameters the one with the minimum constraint violation.
s sigma for which both metries are below certain thresholds
We use the same procedure to choose n.tter_fwe from the set (50, 100,.... 350, 400), where the projection is evaluated using the previously determined sigma.
C.3 ABLATIONS.
We evaluate the effectiveness of the matrix equilibration and auto-tuning by comparing the perfor- mance of our method with and without different components on the large non-convex problem of Section 1.1. We test 3 different configurations: default hyperparameters and no equilibration; auto- tuned hyperparameters and no equilibration: auto-tuned hyperparameters and equilibration. We respectively refer to these configurations Default, Aut and Ilnet. On this problem setting, performing auto-tuning takes roughly minutes, while the equilibration takes less than second. The (tuned) hyperparameters for each configuration
follows:
Default + sigma 1.0, iter
100
Auto + s sigma = 3.28, n Ewd = 350
ITnet <- sigma = 0.161, niieer
and the remaining ones are chosen as discussed previously. We note that the large difference in the value of sigma between Auto and Innet is due the equilibration that changes the problem scaling.
We report in Figure 16 the RS and CV by all 3 configurations, training curves are shown in Figure 17 for 50 training epochs and for single seed, and inference times are given in Table 6. From Figure 16 we can readily deduce the substantial improvement bration offers. Additionally, Figure 17 and Table 6 highlight the clear benefits of our preprocessing steps in terms both training time and inference time. These benefits are due primarily to the fact that well-tuned and equilibrated projection layer requires significantly fewer iterations achieve a satisfactory CV, specifically 50 iterations instead of 100 and 350 for default and only auto-tuned, respectively.
terms of CV that both auto-tuning and equili-
35

---

Published as a conference paper at ICLR 2026
llnet
Default
Auto
Large
10-2
10-4
14
DV
Figure 16: Visualization of the ablation results Appendix C.3. Scatter plots of RS and CV for the methods on the large non-convex problems on the test set. The red dashed lines show the thresholds that we require to consider candidate solution optimal.
Ilnet
Default
Auto
Large
10-2
10-1
10-4
10-4
10-7
1001
200
300
400
500
600
7001
900
900
100
2000
300
400
500
610
700
800
5000
Time
Figure 17: Comparison the training times in terms RS and CV for the different ablation config- uration on the large non-convex problem setting. For simplicity, we report the learning curves only for a single seed.
Single inference [s]
Batch inference [s]
Method
median
LQ
UQ
min
max
median
LQ
UQ
min
max
Non-convex
large
Default
0.0080
0.0080
0.0081
0.00775
0.0090
0.5491
0.5490
0.5493
0.5487
0.5495
Auto
0.0141
0.0139
0.0143
0.0135
0.0150
1.8900
1.8899
1.8901
1.8896
1.8903
IInet
0.0063
0.0063
0.0064
0.0061
0.0092
0.2804
0.2799
0.2807
0.2794
0.2912
Table 6: Inference time comparison for single- inggle-nnttnnc and Batch-instance (1024 problems) settings across different ablation configurations, evaluated on the large non-convex problem. The table re- ports median runtime along with statistical descriptors: lower quartile (LQQ 25th percentile), upper quartile (UQ, 75th percentile), min and max of the runtime.
36

---

Published as a conference paper at ICLR 2026
Small
teat
10
test 25
50
10"22
test 100
10
10 10
10
10"4 10
10"
10"7
10
Figure 18: Scatter plot of RS and CV on the small non-convex test problems for a Inee network trained with 50 iterations and evaluated with different numbers of iterations at test time, The red dashed lines indicate the thresholds used to consider candidate solution optimal.
In our experiments, we have extensively tested the effects of using a different number of forward iterations during testing (n_itt (n_iter_ (et__test) compared the training (n_iter_fwi) with both more and fewer iterations at test time. Using more iterations works well and stably, and is meaningful to achieve reduced constraint violation. Using less iterations needs more care. Slightly less iterations is often possible and can improve inference time. Significantly less may cause issues since the iterates might not be close fixed-point anymore. Of course, "slightly" and "significantly" here depend on the problem at hand. For this, in Figure 18 we report the RS and CV obtained by training Ihet with iter fwd 50, and then deploying with different number of iterations during testing. Perhaps interestingly, note that the RS is unaffected by the number of iterations during testing, whereas the CV gradually decreases. This true as long as we have enough iterations: as the number of iterations during test becomes too small (e.g., 2) also the RS is affected: The output is not close to the projection anymore.
Finally, in Figures 19 and 20 report an ablation study on the parameters and w. These results, together with the ones on the number of forward and backward iterations (cf. Figure 18 and Appen- dices D.3 and F), substantiate the claim of little sensitivity of Innet to hyperparameter tuning. In particular, Figures 19 and 20 show that different values of the parameters and w yield qualitatively similar behaviors. w, we see clearly that different values effectively only change the convergence rates: With more iterations, all values of w achieve sufficiently low CV.
C.4
WHY ENFORCING CONSTRAINTS DURING TRAINING?
Thee enforces the constraints during training, which warrants discussion: What changes if one trains an unconstrained network and enforces the constraints only during inference? We answer this question with illustrative examples.
Some optimization problems may cause divergence of the network if trained unconstrained. The first example shows that, without constraints, the training process may even diverge, as the optimization problems we are interested in may not even be meaningful in the absence of constraints. Consider the parametric (in x) optimization problem:
minimize vER
xy subject to 0 0SSI1
v"(x] for
The optimal solution is
>" (0) E [9.1]
(x0 for
0 1
if <
(x))
if
any [0, 1] otherwise.
However, if we would train an unconstrained network to predict (xx), its values would result in (x) - +oo for > 0 and (xx -00 for < 0.
37

---

Published as a conference paper at ICLR 2026
n.iter.fwd 50
Small
0.28
0.14
10-1
0.017
10"2
0.035
0.07
10"5
10-4
10-14
10-12
10
10
10
10-4
10
10a
n.iter fwd 100
0.28
Small
0.14
10
0.017
10
0.035
0.07
10--
10-9
10-14
10-12
10-19
10-6
10
Figure 19: Scatter plot of RS and CV on the small non-convex test problems for a Inet network trained with different values and (top) 50, (bottom) 100 forward iterations. The red dashed lines indicate the thresholds used to consider candidate solution optimal.
1.3 W
Small
1.5
1.7
1
2
10
10
10-14
10
10-10
10
10
10
10
Figure 20: Scatter plot RS and CV on the small non-convex test problems for a Inee network trained with different values of w. The red dashed lines indicate the thresholds used to consider candidate solution optimal.
38

---

Published as a conference paper at ICLR 2026
It is easy see how may build highee-dimensionee examples along the same lines of the pre- sented one. Importantly, this example shows that, without constraints, the problem may be altered significantly to the point that it is not even possible to train unconstrained network.
The projection of the unconstrained optimizer is often suboptimal. Our second example shows that applying the projection layer to the solution of the unconstrained problem does not yield, in general, the solution to the constrained problem.
As a consequence, one can expect a strong decrease in per- formance when training the network unconstrained (i.c., to solve the unconstrained optimization problem) and only at inference adding projection layer (i.e., projecting the solu- tion of the unconstrained problem onto the feasible set). Consider the parametric (in x) optimization problem de- picted on the right. Here, one may train an unconstrained network to optimize the objective function, finding the op- timizer (xx) = x + 2. However, projecting this onto the constraints y I x would result in y x, which sub- optimal compared to the constrained optimizer y" (x) x- x-2. 2. Importantly, this simple example shows an instance in which training without constraints may downgrade the per- formances of the network.
Feasible
Unconstrained optiminer
Constrained optimiter
Projection of the unconstrained optimizer
p6y. x]
-2
x
An empirical case-study. In the third example, we train an unconstrained network on the small non-convex benchmark (using soft penalty terms) and add the projection layer only during inference on the test set (we refer to this as Inet-in The resulting average RS on the test set for ITnet- inf 0.02178, whereas for Ilnet is 0.00216 using roughly the same computational budget (8 seconds of wall-clock time). The constraint violation is the same for both methods, since they use our projection layer during inference. Training with Inet (i.e., enforcing constraints during training) achieves one order of magnitude better RS. In fact, if we increase the computational budget, Ilnet continues reduce the RS down 0.0007 for 50 seconds of training. On the contrary, Inettin cannot reduce the RS further even with more training (or more hyperparameter tuning)- Therefore, we observe in practice what our previous examples outlined: enforcing the constraints during training allows the network to anticipate the projection and further improve its predictions.
Constraints as an inductive bias. There is subtle difference between constraining the archi- tecture neural network (e.g., by ensuring that the neural network is a convex function for any values of the weights, done by Amos et al. (2017)) and constraining the output to satisfy some properties without altering the "backbone" network architecture. In Innet, one can use the best existing architecture unaltered, but training it with the inductive bias that the output will be modified so to satisfy the constraints. For this, one may argue that enforcing the constraints during training improves the performance of the network at inference time, rather than downgrading it. We thus believe that an approach like llnet may, in fact, change the common perspective on the difficulty of training hard-constrained networks.
Remark. It perhaps worth clarifying the distinction between our work and the work of Amos et al. (2017):
1. Input convex neural networks learn a sealar-valued function, which is guaranteed to be a convex function of its input for any appropriate choices of the network weights (see Amos et al. (2017, Propositions 1, 2)). Then, during inference, a minimum of this function is computed. Recall that the set of minimizers of a convex function a convex set, and in this sense Inet and imput convex neural networks are similar.
On the contrary, Inet learns a vector-valued mapping which is guaranteed lie on a convex set chosen a-priori for any choice of the weights.
C.5 TINE AS AN IMPLICIT LAYER
The techniques used implement Inet csperator-splitiing and backpropagation via the implicit function theorem) are fundamental and widely adopted in the literature. this sense, Inne is a special case of implicit layers (Agrawal et al., 2019; Butler & Kwon, 2023). However, key ideas
39

---

Published as a conference paper at ICLR 2026
related to the type of problem are solving (a projection) and how the structure of this problem can be exploited to derive an efficient formulation of the optimization algorithm. The improvement over the state of the art (as exemplified by our results in Section 3) is achieved by focusing problem setting that is sufficiently general yet rich structure, and by adopting the right optimiza- tion techniques (e.g., which split perform deploy the Douglas- Rachford algorithm). The key ideas, in this sense, are:
1. Using a single additional layer instead of multiple ones.
Using a projection layer.
Adapting the Douglas-Rachford Rachford algorithm to best exploit the resulting problem structure.
40

---

Published as a conference paper at ICLR 2026
D DERIVATION DETAILS
In this section, we report the derivations omitted in the text.
D.1 DERIVATION OF ALGORITHM 1.
To derive Algorithm/ from (4), we will explicitly write the proximal operators in (4). For the z-update, we recall that g ZA and 0, from which immediately follows that prox,g II For the t-update, recall that:
h (1 Mamxx
(F2mm)
1lv
Yoww
Ix:
Manux
and using the definition of the proximal operator yields:
p (2zk+1
8K) argmin
||y Vraw 112 Ix
y Blux
1
y/ V/axx
2zk+ +
8k
2o
M-3
argmin ly Fraw
11 112
22
sk,2112
yEK,
argmin 1
Vaaxx 22k+1,2 + sk,211
27
Von EXiz
Txg
2 1-8-1.1---22 yon
1+20
Mc (2 5x,2)
D.2 CONVERGENCE PROOF OF (4).
In this subsection, we show that (4) converges to solution of (3). To do so, we first introduce the following mild assumption the feasibility of the problem. Assumption 1. A miri * where ri is the relative interior.
This assumption corresponds strict feasibility of (3). In fact, if polyhedron, then we can relax Assumption 1 to An K. / 0.
To obtain our convergence result, we simply note that iteration (4) is the Douglas-Racee Rachford algo- rithm applied to problem (3) and, under Assumption 1, we invoke the Corollaries 27.6 and 28.3 in (Bauschke & Combettes, 2017) which yield the desired result.
D.3 CONVERGENCE RATES OF (4).
The convergence rate of the Douglas-Rachford algorithm is a well-studied topic in the literature, which report here for completeness, For the case of a convex QP, it has been shown that the Bouglas-Rachfos algorithm attains a linear convergence rate (Peña et al., 2021, Subsection 3.2), namely an e-optimal solution is computed in (log(1/E)) iterations. This result applies to our algo- rithm in the case of polyhedral constraints, and indeed we have observed these rates in our numer- ical experiments (see also Figure 21). Other technical conditions for linear convergence are given in (Hong & Luo, 2017, Assumption A); their applicability to setting depends the specific constraint set C. In the general setting, the Douglas-Rachford algorithm typically attains a ((1kk) convergence rate (where k is the number of iterations), hence requiring ((1/s) iterations (Davis & Yin, 2017).
We visualize these convergence rates in the plot in Figure 21, showing how for increasing iterations we get rapidly decreasing CV. Specifically, for one of the benchmarks, we provide the CV for 100 instances in the test for an increasing number of iterations. In particular, we observe that within very few iterations the constraints are satisfied up to high accuracy (e.g.. 10-5 as in (Stellato et al., 2020)).
D.4 DERIVATION OF THE BACKWARD PASS
Applying the implicit function theorem to 800 (3/mw) = $8.00 (3rra)), Mraw) yields the linear equation:
04(s, Braw)
Os (Braw)
av.(s Braw)
(15)
Is
Draw
Uraw
9= (Bran)
Ven Vem= (x:0)
Vhn (x==1(x:0)
41

---

Published as a conference paper at ICLR 2026
10
10-2
10-2
10
10-11
10-14
40
p_1ter rvd
Constraint violation for 100 problem instances (light blue) in the test set of the small non-convex benchmark as the number of iterations in the forward pass n m.iter.fwd (i.e., n_iter_tral in increases, well the maximum CV among all instances (dark blue).
Figure 21:
whose unknown is the Jacobian matrix Now, compute the VJP
as (Niw))
dynn
Viu = (x=0)'
Osoo (Braw)
v
Ven ffm==((e (xx00
where e E Rn++d we assume that the linear system (9) admits a solution, namely E(yraw t). Next, we pre-multiply (15) by [(hraw
(s((s, Fraw)
as (yhaw)
{(yraw, a yraw)
(yttwr, u)
Ōs
yyaa
N
ao (Mran)
Phan ((x:0)
Munn = (xx:8)
Os (Braw)
E.(taw 1i) O (s, yraw)
v
Dirow
Vhaw $(x:0)
Mhan (xx:8)
where the implication follows by the definition of E(Uraw, v). This gives rise to our backward pass outlined in (8) and (9).
D.5 COMPUTATIONAL COMPLEXITY OF INET.
Here we derive the computational complexity of llnet for both the forward and backward pass.
Forward. The complexity of the projection operations in Algorithm I are determined by the sets A(x), (x) and (x),,, by the cost of instantiating them given the context x, which may depend on the dimensionality of and by the dimensions of the input n, d and n d. We denote the complexity of these operations as gx (x, p), I. (n), 9K, (d) and gk, (n d). The overall complexity is:
(gx(n,p) + ((gA(n) + gk, (d) + gKg, (n d) + n)),
where = n iter fwd is the number of forward iterations, and the term n is due to the sum? in the governing update (4c).
Next, we analyze the complexity of (x,nn, (.) (
gx, (d) and gx, (n d):
ge (n,,p) is problem specific. In the examples presented in this work. the complexity is linear in p and amounts stacking the vectors describing the problem instance (e-g.. the initial and final position of the vehicles in Section 3.2) into the equality constraints. In general, one may expect Ix (x(n,,) p) to be dominated by the other contributions. If the matrix A is context dependent, an additional cost is the calculation of the pseudo-inverse, which
to compute the projection
'The complexity also depends on hatch size, the operations can he fully parallelized over batch and, thus, we focus on the per-element complexity.
OOn GPU, this entry-wise vector he parallelized, but here focus the number of operations.
42

---

Published as a conference paper at ICLR 2026
onto the hyperplane: (x(n. p) n²; see also the next item. When the matrix A is independent of the context, the pseudo-inverse can computed only once at the instantiation of the method, and its computational complexity does not affect the forward pass.
(A(n) is the cost of the projection on the hyperplane identified by a matrix A E (d++"))) and vector E Id+d"" The parameter is problem-dependemt but usually proportional to the number of inequality constraints; see Appendix E. For this reason, carry out the complexity analysis for E Rax= For the projection, we need the pseudo-inverse of A, which we compute once for all iterations. Thus, account for this complexity in 9x (x, p). Then, the complexity of computing the projection is dominated by matrix-vecto multiplications: 9A = n²
gk. (d) and 9K:z (n - d) are the cost the projection onto box after a linear combination of the input. Thus, 9K,, (d) OK2 (n d)
Overall, the complexity of the forward pass is O for context-independent A and O(n²) for context-dependent A. On a GPU and for context-independent A matrices, the effective compute time is possibly linear in
Backward. For the backward pass, need to:
Compute the for the linear system in (9), for which need only the first VJP in (7). The complexity Oddnn + d)) = (n²))
Solve linear system in (9) using bi cisstbb (van der Vorst, 1992). The complexity this operation is O(K" h + d)d)
O(K"n²), where K' wwiter.backee..... is the number of
iterations used for bicgstab.
We compute the VJP in (8). By direct inspection of the computational graph of (4a)-(4c), com- plexity of this step is (dd(d + n)) O
Overall, the backpropagation through our projection layer has complexity O(n?), with the constant dominated by the number of iterations used for bicgstab.
43

---

Published as a conference paper at ICLR 2026
E MORE EXAMPLES OF CONSTRAINT SETS
In this section, describe how several classes of constraints that be described within framework admit an efficient projection. We stress that the decomposition C = [Ta[A A n K) is not assumption. One always decompose convex in this way, e.g- by considering the trivial decomposition A = C' and K = IRd Instead, determining A and Æ is a design choice which we leverage make the projections II and 1lę computationally efficient. We note two important points regarding this design choice:
The only assumption is that A and I and their VJP are computable. Being computationally efficient an added benefit of our decomposition, but not necessary.
Below, we show that many practically-relevant constraints can be decomposed a computationaally-effiiii manner: polyhedra, second-order cones, sparsity constraints, simplices, and the intersections and Cartesian products all admit efficient decompositions. In fact, this list is exhaustive; see, e.g., (Condat, 2016; Boyd Vandenberghe, 2004).
We believe the implementation and adoption of these constraints in practical applications represents an important direction for future work.
often employed to enforce constraints in robotics (Augugliaro al., 2012), numerical solutions to PDE (Raissi et al., 2019), and non-convex relaxations for trajectory planning (Malyuta et al., 2022), among others. They expressed as
Polytopic sets
{y E Rd
Ey = cy
for some E, g, C, u of appropriate dimensions. We introduce the auxiliary variable Maux = Cy E Ru with dimension d Tineg. Then, respectively define A, K. R the following affine subspace and box
A
y Jaux
E C
y Maux
q 0
K
y
Rd, 15 Vaux S {
yaux
Importantly, C = (T n K.) and both I and ITx can be evaluated in closed form (Bauschke & Combettes, 2017, Chapter 29).
Second-order cones are employed in portfolio optimization (Brodie et al., 2009), robust control (Chen et al., 2018), and support vector machines (Maldonado & López, 2014), among others. They involve constraints of the form:
ty E Rd [ly + 5 +J..
Introducing the auxiliary variables V = Cy + Rd and Yaux,2 f yyee R, of dimension d = ne + 1, we obtain the representation:
1
fT
C
-
yaux,
ylux,1 2
|| 112 S youx,2
2_
for quantities of the appropriate dimensions. Again, II and II admit a closed form solution (Boyd & Vandenherghe, 2004).
Sparsity constraints encourage solutions with fewer active variables, such as the number of open positions in portfolio optimization (Brodie al., 2009), the "mass splitting" in optimal transport (Dantzig, 2002). Explicitly, they read
{u R |Cy + cli s TT +
Analogously to second-order cones constraints, employing slack variable they can be reduced to a projection onto the ball, which can be done efficiently (Condat, 2016).
44

---

Published as a conference paper at ICLR 2026
Cartesian products of simplices formulations, since they encode the set of admissible coupling measures. While these constraints can also be expressed generic polytopic sets, alternative representation using proposed form can potentially yield more efficient solutions. For instance, for some 01;-... Un. wi,...y Wm 2 0, consider the following constraint
the standard constraints in optimal transport (Dantzig, 2002)
{ U 1R xd2 Vii
C =
2 O,
Mig
wjs
Vij
Vi
==1
1
m Σ j=1
Rdiida
Σ
Vij wj
y E
12 xda
Vij 0,
Vij 0%
AnB
Notice, that C = Ila(An)) as required by our constraint decomposition, and lla(-) is redundant since and IC are of the same dimension C. The projections II and Il can be evaluated efficiently since A is hyperplane and K. is cartesian product of simplices (Condat, 2016).
The intersection of previous sets can readily be expressed in our representation. For instance, the intersection of polytope and second-order cone can expressed the following set:
(v E Rd Ey = q, S Cy S u, [yy + cll2 S f + e).
We introduce the auxiliary variables Vaux, and obtain the representation:
Cy E Timn, Vaux. Fy E 1R Vaux. = fT ER
y
E
0
0
y
q 0
A
Faux, Maux, 2
C
F
T
-1
0
0
0
Yauxx
Mlaxx.
C
Vaux, 3
S
0
-1
Mauxx 3
K Rd x {Haux, E IR Thren ||
S
Maux,
u) x
Faux.
E
+1
||
2/1 S Vaux
Blux, 3
As before, the projections 1A and 1l admit closed-form expressions.
45

---

Published as a conference paper at ICLR 2026
F ON THE DIFFERENTIABILITY OF THE PROJECTION LAYER AND THE APPLICABILITY OF THE IMPLICIT FUNCTION THEOREM
In this section, we discuss the theoretical aspects of the differentiability of the proposed projection layer, as well as the applicability of the implicit function theorem.
F.1 ALMOST EVERYWHERE DIFFERENTIABILITY
To start, we note that since the layers of the backbone network will be at most almost everywhere differentiable (e.g., for ReLU activations), all one may be interested in is to have almost everywhere differentiability of the projection layer. In fact, our experiments show that in practice this is sufficient for markedly surpassing the state of the art. On Euclidean spaces (more generally, on Hilbert spaces) the projection operator is globally I-Lipschitz (see, e.g., (Bauschke & Combettes, 2017, Proposition 4.16), and recall that firmly non-expansive implies I-Lipschitz). Then, with Rademacher's 's theo- rem (Simon et al., 1984, Theorem 1.4) conclude that the projection layer is almost everywhere differentiable (in the Lebesgue measure sense).
F.2 APPLICABILITY OF THE IMPLICIT FUNCTION THEOREM
Providing a full theoretical analysis is beyond the scope of the paper, and we believe it would dis- tract the reader from the main message of the paper. Nonetheless, we (i) believe this to be a very interesting mathematical quest for future work, and (ii) that the outline presented here can give a sense the reader of why the proposed projection layer theoretically sound.
The implicit function theorem is applicable under local differentiability conditions. The way that lated works handle non-smooth points to either (i) show that if the linear system (9) is non-singular, then multiple solutions exist and they belong to the subdifferential (see, e.g., (Amos & Kolter, 2017, Appendix C.I)) or (ii) they resort to heuristics, e.g., by solving least-squares approximation rather than the linear system itself (see, e.g., (Agrawal et al., 2019, Section 4.3)). We similarly run finitely many bi cgstab iterations, which is very computationally attractive as it avoids run-time checks and, in our numerical experience, is quite stable; see Appendix F.3.
Perhaps surprisingly, one can define an implicit function theorem even on non-smooth points by employing conservative gradients, a recent generalization of the subdifferential (Bolte & Pauwels, 2021). This formulation is valid for non-smooth locally Lipschitz mappings, to which projection layer belongs. We refer to the seminal works of Bolte & Pauwels (2021); Bolte et al. (2021) for the precise statement of the non-smooth variant of the theorem. In this framework, the implicit function theorem formula and the chain rule used in Section 2.1 hold under very mild conditions. Roughly speaking, this would correspond to our operator ( y) being semialgebraic and contractive with respect to (Bolte et al., 2024). With this, we could conclude that our derivation in Section 2.1 holds everywhere.
F.3 EFFECTIVENESS OF THE BACKWARD PASS IN APPROXIMATING THE GRADIENT
In this section, we empirically assess how well the proposed backward pass approximates the true gradient induced by the projection operator. Concretely, we compare the VJP obtained from the implicit function aecorem-basee backward pass with finite-difference estimates on a representative problem instance.
Experimental setup We fix two problem instances in the non-convex small benchmark (see Sec- tion 3.1) by fixing the right-hand side of the equality constraints. We then generate set of 100 points-to-be, prooee--roee _m
100 by sampling each coordinate from a normal distribution with standard deviation 10. For each I,, we also sample 100 normally distributed vectors for comparing the VJP. As a reference "ground truth", we compute the VJP using central finite differences with step size 10 which yields an approximation with Oe=2) truncation error. We then compute the VJP using our backward pass for varying numbers of backward iterations n-iter lowd. For each configuration, we report (i) the & norm of the difference and (ii) the cosine similarity, thereby quantifying both the absolute and directional accuracy of the proposed backward pass. For both methods, we iter fwd 200 forward iterations for computing the projection accurately.
46

---

Published as a conference paper at ICLR 2026
10-1
Cbeiue Similarity
Fa etron
0.9
10
10
10-3
2
6
8
10
12
14
16
20
10
12
14
16
18
20
D_ieer Cer Jowd
b.i.err.ba
10
0.9
Cosine Siulliity
12 erTOr
10-2
0.8
10
0.7
10-3
0.6
10-6
D_iter bard
n.iterr bard
Figure 22: VJP estimation error (measured via the fg-norm of the difference and the cosine sim- ilarity) for 100 different vectors v, instances (light blue) for two different instances of the small non-convex benchmark the number of iterations in the backward pass iter bwd increases, as well as the maximum eg-error and the minimum cosine similarity among all instances (dark blue).
Results As shown in Figure 22, the discrepancy between the VJP computed with our backward pass and the finiteddifference reference decreases rapidly as the number of backward iterations in- creases. These results indicate that the proposed backward pass yields highly accurate approxima- tion the true gradient even with a relatively small number of iterations.
47


Reviews:
[{"review_id": "C00dvSqg8n", "reviewer": "Reviewer_o3mf", "summary": "This paper proposes PiNet, a differentiable layer for solving projection problems over convex constraints to ensure neural network output feasibility. The key contribution is a decomposition approach for certain constraints that can be represented as the intersection of hyperplanes and boxes, where each component admits efficient projection. The method applies operator splitting algorithms to the projection problem in this decomposed form, with the backward pass implemented via implicit gradients. Experiments demonstrate improved efficiency and reduced optimality gaps compared to baseline methods on benchmark problems and multi-vehicle motion planning tasks.", "questions": "Please refer to the weakness section. \n\nOverall, this work is well-written. I will adjust my rating upon clarification of the above points and possible additional experiments addressing the weaknesses mentioned.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Ensuring neural network output feasibility is important for real-world applications, and this work addresses a practical and relevant problem.\n2. The combination of constraint decomposition and operator splitting algorithms appears to be novel, and significantly improves projection efficiency for certain types of constraints.", "weaknesses": "1. The range of constraints that admit the decomposition form of hyperplanes and boxes is unclear. Besides the examples provided in the appendix, can the authors provide a clear definition or characterization of the types of constraints that fall within this framework?\n2. The experiments mainly focus on linear constraints. Evaluation of non-linear constraints would further strengthen the contribution and demonstrate the broader applicability of this work.\n3. The authors mention that Min et al. (2024) propose a closed-form expression to recover feasibility for polyhedral constraints. It would be valuable to include a comparison with this recent work to better position the proposed approach.\n4. The related work section on hard-constrained neural networks could be strengthened by including other approaches, such as reparameterization approaches [1-2] and sampling-based approaches [3].\n\n- [1] Tabas D, Zhang B. Safe and efficient model predictive control using neural networks: An interior point approach. CDC 2022.\n- [2] Liang E, Chen M, Low SH. Homeomorphic projection to ensure neural-network solution feasibility for constrained optimization. JMLR 2024.\n- [3] Kratsios A, Zamanlooy B, Liu T, Dokmanić I. Universal approximation under constraints is possible with transformers. ICLR 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "279Z8SeRGp", "reviewer": "Reviewer_hExW", "summary": "The paper introduces a differentiable projection network (ΠNet) for handling hard constraints within end-to-end neural network training. The key idea is to embed a projection operator directly as a network layer, ensuring that outputs always satisfy predefined linear equality and inequality constraints. The projection is computed via a Douglas–Rachford splitting scheme, which alternates between projections onto an affine set and a box set until convergence to their intersection. For backpropagation, ΠNet avoids differentiating through the iterative steps. Instead, it applies the Implicit Function Theorem (IFT) at the fixed point of the DR operator, computing the vector–Jacobian product by solving a linear system. Overall, the method provides a principled, computationally efficient, and differentiable mechanism for enforcing hard linear constraints in neural networks.", "questions": "1. The choice of $\\omega = 1.7$ is mentioned as default but not justified. Did the authors observe sensitivity in convergence or gradient stability with different values of $\\omega$? Providing even brief empirical guidance would be useful for practitioners.\n2. The method uses a fixed number of Douglas–Rachford iterations at inference equal to that during training. Have the authors verified that performance remains stable if more (or fewer) iterations or early-stopping are used at test time? This could reveal whether ΠNet truly learns a stable fixed-point operator or overfits to the training iteration budget.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well organized and clearly written, with consistent notation and a coherent presentation.\n\nIt introduces a conceptually clean and mathematically principled approach to making projection-based constraint enforcement differentiable. By combining the Douglas–Rachford splitting scheme with the Implicit Function Theorem at the fixed point, the authors design a projection layer that is both efficient and end-to-end trainable. This integration of classical operator-splitting methods with modern differentiable programming is novel and results in a solver-free alternative to existing differentiable optimization layers such as cvxpylayers, DC3, and JAXopt.\n\nThe technical development is sound and internally consistent. The backward pass through the fixed-point system is derived correctly, and the implicit differentiation is implemented carefully via the solution of a linear system. The experimental evaluation is thorough: benchmarks include both convex and nonconvex problems across multiple scales, and comparisons are made against strong baselines. ΠNet achieves comparable or superior constraint satisfaction while reducing computation time by one to two orders of magnitude, demonstrating clear numerical advantages.\n\nΠNet provides a practical and general framework for incorporating hard constraints into neural networks without compromising differentiability. Its efficiency and stability make it particularly relevant for constrained learning tasks in control, trajectory optimization, and physics-informed modeling, representing a meaningful contribution to the literature.", "weaknesses": "While the paper is technically solid, several aspects could be clarified or strengthened.\n\nFirst, the claim of handling convex constraint sets is somewhat overstated. All derivations and experiments are limited to linear (affine) equality and inequality constraints. Although these are convex in the geometric sense, the current method does not address more general convex sets such as norm balls, SOCs, or PSD cones. The projections and corresponding Jacobians for these sets are nontrivial, and it remains unclear whether the Douglas–Rachford formulation or the implicit differentiation strategy would remain computationally tractable in those cases. Clarifying this limitation and outlining potential extensions would make the contribution more precise and complete.\n\nIn addition, the experimental comparisons, though comprehensive, could be made fairer and more reproducible. JAXopt and DC3 are trained under different budgets and tolerances. Reporting results under equal wall-clock time or identical training epochs would offer a more balanced assessment.\n\nFinally, although the backward pass based on the Implicit Function Theorem is elegant, the paper does not discuss the conditions ensuring the stability or invertibility of the associated linear system. A short clarification or empirical remark on this aspect would strengthen the completeness of the work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AxltqvdzEg", "reviewer": "Reviewer_yrzg", "summary": "This paper introduces Πnet, a neural network architecture that generates provably feasible solutions for parametric constrained optimization problems. The core innovation is a projection layer that maps a backbone network's raw output onto a convex feasible set using the Douglas-Rachford algorithm, guaranteeing hard constraint satisfaction by design. A key advantage is its efficient training via the implicit function theorem for backpropagation, avoiding the computational cost of unrolling the optimization.", "questions": "Given the significant challenge of enforcing the strict constraints in large-scale or parametric Optimal Transport (OT) problems, how does the Πnet architecture with its hard-constrained compare to state-of-the-art OT solvers like the Sinkhorn algorithm and Adaptive Primal-Dual Accelerated Gradient Descent (APDAGD)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1)  А mathematically sound appendable layer with a robust Forward pass formulation and locally correct and heuristically consistent Backward pass. \n2)  Empirical results are meaningful and appear to be reproducible. The ablation study is sufficient. \n3)  Even with approximate or subgradient differentiation, the forward pass is a valid projected inference scheme. Douglas–Rachford ensures convergence under mild conditions.\n4)  The backward pass gives a practical, stable gradient proxy. Empirically, autodiff through a few DR steps often works and trains faster than unrolled solvers.\n5)  The extra projection layer adds no extra parameters, and only a moderate computational overhead—roughly 2–3x vs an unconstrained MLP, far less than unrolled solvers or cvxpylayers.", "weaknesses": "My main concern is the differentiability during training passes, which may make the Pinet functional only locally.  Please review the mathematical assumptions required for the method to work:\n-- C(x) convex, closed, and with a fixed active constraint set around each training point (no switching during differentiation);\n-- A(x) full row rank;\n-- Φ(s,y) differentiable in a neighborhood of the fixed point (piecewise but locally nonsingular (local contraction);\nIt is my understanding that only then the implicit-function theorem can apply locally, and the gradient formula (Eq. 8) would become correct within that region. \nLikewise, in appendix, no derivations of the explicit Jacobians are provided. For the IFT to apply, the map $\\Phi(s, y_{\\text{raw}})$ must be differentiable. The authors never show $\\partial_s \\Pi_A$ or $\\partial_s \\Pi_K$.\n\nThe authors also start from $s_\\infty = \\Phi(s_\\infty, y_{\\text{raw}})$ and derive $$(I - \\partial_s \\Phi)^\\top \\xi = v, \\quad v^\\top \\partial_{y_{\\text{raw}}} \\Phi$$, which is algebraically correct; however, the solution might diverge if $I - \\partial_s \\Phi$ is singular (non-invertible), and the numerical solution via Bi-CGSTAB may be wrong. Also, if $\\Phi$ is built from non-smooth proximal operators it is obviously not $C^1$. Please clarify whether the solution is stable and unique. \n\nLines 161–163: The projection map is non-differentiable at points where the active constraint set changes. The paper later differentiates this function as if it were $C^1$. So, one needs to clarify that it is Lipschitz but not differentiable everywhere; and the gradients are defined only almost everywhere or as subgradients.\n\nLines 173–179: Projection commutativity is assumed without proof. Derive or restrict A,K so that projection decomposition preserves convex geometry; otherwise (2) does not yield the same minimizer as (1).\n\nEqs. (3a–3c): replace “strict feasibility” with “non-empty intersection and standard monotonicity assumptions.”, because DR requires only non-empty intersection of closed convex sets. Also, include the σ scaling in prox definitions (currently inconsistent with (3a, 3b)). \n\nLines 218–228: IFT application will be valid only where projection is differentiable (interior or non-degenerate active set). This assumption is implicit and needs to be discussed.\n\nRecommendations: \n\n•  Replace global claims (“guaranteed differentiability”) with local or empirical ones. Framing results as empirical efficiency rather than mathematical guarantee is advised.\n•  Include a small proof or citation showing IFT validity for smooth convex sets with full-rank equality constraints.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "j5SUseFgEo", "reviewer": "Reviewer_1VJT", "summary": "This paper introduces \\PiNet, a neural network architecture that guarantees hard constraint satisfaction by projecting the network’s output onto a convex feasible set using operator splitting (Douglas–Rachford iterations) in the forward pass and applying the implicit function theorem (IFT) for efficient differentiation in the backward pass. ΠNet acts as a feasible-by-design implicit layer, enabling neural networks to directly solve parametric constrained optimization problems. The method is further applied to multi-vehicle motion planning, showcasing constraint satisfaction, parallelizability, and flexibility to optimize arbitrary differentiable objectives. A JAX implementation is provided for GPU-ready deployment.", "questions": "How does ΠNet perform relative to OptNet or Theseus, which also differentiate through optimization problems?\n\nHow many Douglas–Rachford iterations are typically needed for practical convergence? Is there a risk of slow progress or oscillation for poorly conditioned problems?\n\nIn multi-vehicle planning, each vehicle’s constraints are decoupled. How would ΠNet behave when inter-agent constraints (e.g., collision avoidance) are included?\n\nThe paper claims strong insensitivity to tuning—could you provide more quantitative evidence or stress tests across random seeds and scaling factors?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "ΠNet introduces a principled mechanism for embedding orthogonal projection layers into neural networks, rigorously ensuring constraint satisfaction during both training and inference.\n\nThe layer can be seamlessly attached to any backbone network, transforming unconstrained predictors into feasible solution generators for convex programs, PDEs, and motion-planning tasks.", "weaknesses": "ΠNet cannot directly handle non-convex or mixed-integer constraints; while the authors mention sequential convexification as future work, this remains a strong limitation for broader adoption.\n\nWhile convergence of the projection layer is discussed, formal proofs of differentiability stability and error propagation across finite iterations are missing.\n\nThe multi-vehicle experiment is insightful but limited. No comparison is made with structured planners (e.g., QP-based MPC, or differentiable MPC variants).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)