Deep FlexQP: Accelerated Nonlinear Programming via Deep Unfolding
Authors: Alex Oshin
Rahul Vodeb Ghosh
Augustinos D Saravanos
Evangelos Theodorou
Venue: ICLR 2026 Poster (2026)
Keywords: Learning-to-Optimize
Deep Unfolding
Nonlinear Programming
Primary area: optimization
TL;DR: FlexQP is an always-feasible QP solver that we accelerate through deep unfolding to solve nonlinear optimizations quickly and robustly.

Abstract: We propose FlexQP, an always-feasible convex quadratic programming (QP) solver based on an $\ell_1$ elastic relaxation of the QP constraints. If the original constraints are feasible, FlexQP provably recovers the optimal solution. If the constraints are infeasible, FlexQP identifies a solution that minimizes the constraint violation while keeping the number of violated constraints sparse. Such infeasibilities arise naturally in sequential quadratic programming (SQP) subproblems due to the linearization of the constraints. We prove the convergence of FlexQP under mild coercivity assumptions, making it robust to both feasible and infeasible QPs. We then apply deep unfolding to learn LSTM-based, dimension-agnostic feedback policies for the algorithm parameters, yielding an accelerated Deep FlexQP. To preserve the exactness guarantees of the relaxation, we propose a normalized training loss that incorporates the Lagrange multipliers. We additionally design a log-scaled loss for PAC-Bayes generalization bounds that yields substantially tighter performance certificates, which we use to construct an accelerated SQP solver with guaranteed QP subproblem performance. Deep FlexQP outperforms state-of-the-art learned QP solvers on a suite of benchmarks including portfolio optimization, classification, and regression problems, and scales to dense QPs with over 10k variables and constraints via fine-tuning. When deployed within SQP, our approach solves nonlinear trajectory optimization problems 4-16x faster than SQP with OSQP while substantially improving success rates. On predictive safety filter problems, Deep FlexQP reduces safety violations by over 70\% and increases task completion by 43\% compared to existing methods.

Published as a conference paper at ICLR 2026
DEEP FLEXQP: ACCELERATED NONLINEAR PROGRAMMING VIA DEEP UNFOLDING
Alex Oshin"! Rahul Vodeb Ghosh", Augustinos D. Saravanos"s Evangelos A. Theodorou'
GGerrga Institute of Technology
Massachusetts Institute of Technology
ABSTRACT
We propose FlexQP, an always-feasible convex quadratic programming (QP) solver based on an & elastic relaxation of the QP constraints. If the original con- straints are feasible, FlexQP provably recovers the optimal solution. If the con- straints are infeasible, FlexQP identifies a solution that minimizes the constraint violation while keeping the number of violated constraints sparse. Such infeasibil- ities arise naturally in sequential quadratic programming (SQP) subproblems due to the linearization of the constraints. We prove the convergence of FlexQP under mild coereivity assumptions, making it robust to both feasible and infeasible QPs. We then apply deep unfolding to learn LSTM-based, dimension-agnon -agnostic feed- back policies for the algorithm parameters, yielding an accelerated Deep FlexQP. To preserve the exactness guarantees of the relaxation, we propose normalized training loss that incorporates the Lagrange multipliers. We additionally design a log-scaled loss for PAC-Bayes generalization bounds that yields substantially tighter performance certificates, which we use to construct an accelerated SQP solver with guaranteed QP subproblem performance. Deep FlexQP outperforms oftt-----te-rtttt learned QP solvers on a suite of benchmarks including portfolio optimization, classification, and regression problems, and scales to dense QPs with over 10k variables and constraints via fine-tuning. When deployed within SQP, our approach solves nonlinear trajectory optimization problems 4-16x faster than SQP with OSQP while substantially improving success rates. On predictive safety filter problems, Deep FlexQP reduces safety violations by over 70% and increases task completion by 43% compared existing methods.
INTRODUCTION
Nonlinear programming (NLP) a key technique for both large-scale decision making, where diffi- culty arises due to the sheer number of variables and constraints, and real-time embedded systems, which need to solve many NLPs with similar structure quickly and robustly. Within NLP, quadratic programming (QP) plays fundamental role many real-world problems in optimal control (An- derson & Moore, 2007), portfolio optimization (Markowitz, 1952; Boyd et al., 2013; 2017), and machine learning (Huber, 1964; Cortes & Vapnik, 1995; Tibshirani, 1996; Candes al., 2008) can be represented as QPs. Furthermore, sequential quadratic programming (SQP) methods utilize QP as submodule to solve much more complicated problems where the objective and constraints may be nonlinear and nonconvex, such as in nonlinear model predictive control (MPC) (Dichl et al., 2009; Rawlings al., 2020), state estimation (Aravkin al., 2017), and power grid optimization (Montoya et al., 2019). SQP itself can even be used as a subproblem for solving mixed integer NLPs (Leyffer, 2001) and large-scale partial differential equations (Fang al., 2023).
However, common difficulty with SQP methods occurs when the linearization of the constraints results in an infeasible QP subproblem, and large amount of research has focused on how to repair or avoid these infeasibilities. e.g., (Fletcher, 1985; Izmailov & Solodov, 2012). among others. A significant advantage SNOPT (Gill et al., 2005), one of the most well-known SQP-based methods, in its infcasibility detection and reduction handling. These considerations necessitate a fast yet robust QP solver that works under minimal assumptions on the problem parameters.
Contact: a aLexoshinge weexoshinggttech.e.. edu. 5 Work done while Georgia Tech.

---

Published as a conference paper at ICLR 2026
To this end, we propose FlexQP. a flexible QP solver that is always-feasible, meaning that it can solve any QP regardless of the feasibility of the constraints. Our method is based on an exact relaxation of the QP constraints: if the original QP is feasible, then FlexQP identifies the optimal solution. On the other hand, if the original QP is infeasible, instead of erroring or failing to return solution, FlexQP automatically identifies the infeasibilities while simultaneously finding a point that minimizes the constraint violation. This allows FlexQP to be robust QP solver in and of itself, but its power shines when used submodule in an SQP-type method, see Fig. 1.
Moreover, through the relaxation of the con- straints, multiple hyperparameters are intro- duced that can be difficult to tune and have a non-intuitive effect on the optimization. To
SQP
OSQP
SQP Deep FlexQP (Ours)
Salve Time (x)
29.77
address this shortcoming, we use deep unfold- ing (Monga et al., 2021) > to design lightweight feedback policies for the parameters based on actual problem data and solutions for QP prob-
40-
20
lems of interest, leading to accelerated ver- sion titled Deep FlexQP. Learning the param- eters in a data-driven fashion avoids the labo- rious process of tuning them by hand or de- signing heuristies for how they should be up- dated from one iteration to the next. Mean- while, these data-driven rules have been shown to strongly outperform the hand-crafted ones, such as in the works by Ichnowski et al. (2021) and Saravanos et al. (2025).
1884
Dubins Vehicle with Five Obstacles
Figure 1: SQP with Deep FlexQP can solve highly constrained nonlinear optimizations over 16x faster than SQP with OSQP (averaged over 100 problems).
We thoroughly benchmark Deep FlexQP against traditional and learned QP optimizers on multi- ple QP problem classes including machine learning, portfolio optimization, and optimal control problems. Moreover, we certify the performance of Deep FlexQP through probably approximately correct (PAC) Bayes generalization bounds, which provide guarantee the mean performance of the optimizer. We propose a log-scaled training loss that better captures the performance of the optimizer when the residuals are very small. Finally, we deploy Deep FlexQP to solve nonlinearly constrained trajectory optimization and predictive safety filter problems (Wabersich & Zeilinger, 2021). Overall, Deep FlexQP can produce an order-of-magnitude speedup over OSQP (Stellato et al., 2020) when deployed as subroutine in an SQP-based approach (Fig. 1), while also robustly handling infeasibilities that may occur due a poor linearization or an over-conttrained problem.
MOTIVATION & RELATED WORK
SQP solves smooth nonlinear optimization problems of the form
minimize f(xx), subject to g(x) S 0, h(x)) = 0,
z
where f R + TR. wiice-lifferentiable is the objective to be minimized and g R." - Rm and h: R - TRP difeediifeerentialle describe the inequality and equality constraints, respectively. SQP solves Eq. (1) by solving a sequence of QP problems that are produced by linearizing the constraints and quadraticizing the Lagrangian' L(x, Mr. ME) (xx + #, g(x) + //r((a) around the current iterate (x**, vt w where yr E R and ye IRP are the dual variables for the inequality and equality constraints, respectively. This results in QP subproblems of the form:
minimize di
dx ViL(at wut v do + ( Tdx
(2a)
subject to g(x*) + Og(x")dr= S 20, h(**) + ah(x*)dx= =0
(2b)
Notably, the linearization of the constraints and h may not produce QP subproblem that is feasible, meaning that there may not exist any dr that satisfies the linearized constraints Eq. (2b).
'In practice the Hessian of the Lagrangian V.L may he indefinite and is often replaced by positive definite approximation, e.g., via quasi-Newton undate.
2

---

Published as a conference paper at ICLR 2026
In this case, the SQP solver either terminates with suboptimal point specialized routine needs to be run in order to reduce the infeasibility. For example, when SNOPT encounters an infeasible subproblem, it enters elastic mode and solves new optimization where the constraints are relaxed using & penalty functions (Gill et al., 2005). This is advantageous over other choices, such as an & penalty, the & norm encourages sparsity in the constraint violation. This means the optimizer can naturally identify the constraints that are the most difficult to satisfy, In the context of mixed integer NLPs, infeasibilities are very likely occur during the branch and bound process, so their fast and robust identification is crucial (Gill & Wong, 2011).
Moreover, as the interest in data-driven optimization grows, we often wish solve many optimiza- tion problems with similar structure repeatedly (Amos et al., 2023), and potentially in parallel or in a batched fashion, such as in the method by Fang et al. (2023), which requires solving coupled systems of SQP problems in parallel. Needing to run specialized procedures in these cases is not scalable. Furthermore, identifying the best hyperparameters for each individual optimization problem is dif- ficult and time consuming. Deep unfolding is a lcarning-to-optimize approach (Chen et al., 2022b) that has roots in the signal and image processing domains (Gregor & LeCun, 2010; Wang et al., 2015) and utilizes data-driven machine learning reduce the cost of hyperparameter tuning and to accelerate convergence model- -ased optimizers (Monga et al., 2021). It constitutes the state-of- the-art approach for sparse recovery (Liu al., 2019) and video reconstruction (De Weerdt al., 2024). In the context of NLP, deep unfolding has been recently applied to accelerate QPs. Saravanos al. (2025) use an analogy to closed-loop control and learn feedback policies for the parameters of a deep-unfolded variant of the operator splitting QP (OSQP) solver (Stellato et al., 2020), which is first-order method based on the alternating direction method of multipliers (ADMM) (Boyd al., 2011). Their method can achieve ceders-of-magnituee improvement in wall-clock time compared to OSQP, and they also propose a decentralized version for quickly solving QPs with distributed struc- ture. Their idea is similar in vein to that of Ichnowski et al. (2021), 1), who use reinforcement learning to train a policy that outputs the optimal parameters for OSQP, with the goal of accelerating the optimizer. Another related approach learns to warm-start Douglas-Rachford with the goal of improving convergence speed (Sambharya et al., 2023).
Rachford splitting QP solver,
Based on these considerations, in Section 3 we propose FlexQP, an always-feasible QP optimizer, and in Section 4 we apply deep unfolding, leading to Deep FlexQP. A more comprehensive survey of related work is provided in Appendix A.
3 FLEXQP: AN ALWAYS-
QUADRATIC PROGRAMMING SOLVER
Our proposed QP solver, FlexQP, transforms the QP constraints using an exact relaxation and then solves the resultant problem using an operator splitting inspired by OSQP (Stellato et al., 2020). We will assume the reader is familiar with ADMM; good overview is provided by Boyd et al. (2011) 1).
3.1 QUADRATIC PROGRAMMING
We are interested in solving convex QPs of the general form
minimize 2" Px Px + q x, subject to Go S h, Ar=
(3)
where x E R is the decision variable. The objective is defined by the symmetric positive semidef- inite quadratic cost matrix P E S and the linear cost vector y E R". The inequality constraints are defined by the matrix G E Rmxx and the vector h E RR.. Similarly, the equality constraints are defined by the matrix A E x and vector E I.P
The optimality conditions for Eq. (3) are given by:
Px + + G yr + A VE = 0, Gx - h < O. Ax-b- y1 C (Gr - h) 0, 2 0, (4) where yr E T and yE E Ip are the dual variables for the incquality and equality constraints, respectively. A tuple (x" yř, vi) satisfying Eq. (4) is a solution to Eq. (3).
Throughout this work, we will avoid making any assumptions on G or A, meaning that the con- straints may be redundant, and in the worst case, there may not exist feasible for the optimiza- tion. This allows for a unified way to handle infcasibilities when optimizations of the form Eq. (3) embedded subproblem in SQP, as in Eq. (2).
3

---

Published as a conference paper at ICLR 2026
3.2 ELASTIC FORMULATION
We start by introducing slack variables 8 E R so Eq. (3) can be expressed equivalently as
minimize 2,420
Px + qTaa 2x -
subject to Ga + s - = 0, Ar - b- 0.
(5)
This standard technique the basis of many interior point algorithms (Nocedal & Wright, 2006).
We then relax the set of equality constraints in Eq. (5) using & penalty functions, yielding
minimize 2,
(0x, 85 MT, MF) == I Px + TTx x + prr [|Gr + kill + ue || Ax - bll (6)
2
with elastic penalty parameters μt, Mg > O. This relaxation approach is known as elastic program- ming (Brown & Graves, 1975), and one of the most well-known SQP-based solvers, SNOPT, uses this technique in order to reduce the infeasibility of a QP subproblem (Gill et al., 2005). This re- laxation is also fundamental step in the sequential & quadratic programming method of Fletcher (1985). Notably, if Eq. (3) has a feasible solution and the elastic penalty parameters are sufficiently large, then the solutions to Eq. (3) and Eq. (6) are identical - this why the relaxation is exact. On the other hand, if the original QP Eq. (3) is infeasible. then solving Eq. (6) finds point that mini- mizes the constraint violation (Nocedal Wright, 2006). This is formalized through the following theorem, which also describes what we mean by a sufficiently large penalty parameter.
Theorem 3.1. Let (x", V VE. solve Eq. (3), Let μ lly and μو li Then, for all 14 2 #7 and pæ 2 pig the minimizers of Eq. (3) and Eq. (6) coincide.
This theorem a generalization of the one by Han & Mangasarian (1979) that shows we can select a different penalty parameter for the inequality vs. equality constraints. The proof, provided in Appendix B, relies on two simple facts: the optimality conditions of Eq. (3) and the convexity of the objective. The proof also shows that it is possible to select vectors of penalty parameters μı and JE,, as long as each fr,, and JEE obeys the constraints μr, 2 |yr,a| or jr.j 2 |yE.ll, respectively.
What happens when the penalty parameters μe do not satisfy the conditions of Theorem 3.1? Using the interpretation of the Lagrange multiplier y/s representing the cost of an individual con- straint i with associated penalty parameter μ4; > 0, if μı 2 Wal then the penalty violating con- straint i in Eq. (6) large enough such that Theorem 3.1 holds. On the other hand, if μr < [ys]. then this constraint is not being penalized strong enough, and so the solution to Eq. (6) will violate this constraint, with the amount of violation proportional to the difference between μ4 and 1y.l. We use this interpretation in Section 4 to design feedback policies that select the best penalty parame- ters as a function of the optimizer state and enforce the condition 14: 2 1ys| during learning using a supervised loss that includes the Lagrange multipliers (see also Theorem 3.3 below).
3.3 OPERATOR SPLITTING & ADMM
The optimization in Eq. (6) can be simplified further make the terms appearing in the penalty functions easier to handle. Introducing decision variables 21 E RT and Fg E IR", we have
minimize 2,8,21.88
2 = Px + q x + ur || + m |
(7a)
subject to 21 Gz + h, zg Ar b. s20 0.
(7b)
The variables 2 and 2g equal the constraint violation, their optimal values can be viewed as a certificate of feasibility for Eq. (3) if 27 = 2 = 0 and infeasibility if 2 * 0 or 2 * O. While it may seem tempting closed-form solution no matter how the variable splitting is performed. Inspired by OSQP (Stellato al., 2020), we perform final transformation by introducing variables G. 8, ε,, and eg. yielding
apply ADMM to this formulation, the resultant updates will not have
minimize 2,0
PT + q T + Ir Ix( + (I( + pz || + ue |EElly
(8a)
g(m)
f())
subject
z x,
(8b)
4

---

Published as a conference paper at ICLR 2026
where ā = (0t, 8, Ez, مg) and (x, 8, 21,2g) for notational simplicity. The indicator functions Ij. Ig, and I, are defined as
0 4o
= Ga + h, [x(x))
so
E = Ax - b. I,(x) otherwise,
0
$ 0, otherwise.
[,(x)
otherwise,
(
+00
Let the dual variables for the constraint ã = x be = (wz, Was (1,9E). The ADMM updates for solving Eq. (8) are given by
x k+1 arg min f(Ø) + (ox/2)]) x² + to - (o./2)]l to, -or'w:ll w
+011/2)113 - 2 + Pi w1 + (|e/2)|l # + PE PE'VEll wEl12.
x x+1 = ai + (1 (1) + tor-us u.
s ++1 = (ash+1 + - a)s* + to, (w²)
k+1 1 = (2// - + (1 - =) + P1 vř),
ktl = Spe/PE (a2 k+l + (1 a) + PE v
21
g
wx *+1 1 w + t k+ + (1 u)x* x r
ws k+1 w o. kk1 + (1 a)s* * -brt1),
5/ ++1 vi pr(aatt (1 - =) f = +
VE u + p
=k+1 (1 a) = =t1)
where ox, T., PI, P > 0 are the augmented Lagrangian penalty parameters, a E (0,2) 2) is the ADMM relaxation parameter, (s)+ max(s, 0) is the rectified linear unit (ReLU) activation func- tion, and (z) = (z - x)+ (-z - k)+ is the soft thresholding operator, which is the proximal operator of the & norm (Boyd et al.. 2011). Since r is unconstrained in the second block, we have +++ that w = 0 for all & 2 0, so the wa variable and update can be disregarded. The first block
step of the algorithm and requires the solution of an constity- QP. We show how to solve this QP using either a direct or indirect method in Appendix C; the indirect method becomes the only suitable choice for large-scale problems where the dimension can be very large. The final algorithm is summarized in Algorithm of Appendix D.
update is the most computationally-demaddd
3.4 CONVERGENCE & THEORETICAL ANALYSIS
We establish the convergence of Algorithm by showing that there always exists a saddle point of the Lagrangian for Eq. (8) for any frr > 0, as long the relaxed QP objective in Eq. (6) is not unbounded below. Then, since the optimization is a composite minimization of two closed, proper, convex functions, the algorithm converges by the general convergence of two-block ADMM (Boyd et al., 201 I). The proof is provided in Appendix E.
Theorem 3.2. Assume the relaxed objective ő(2, below Then, Algorithm for solving Eq. (8), equivalently Eq. (6), converges to a saddle point (अ. E. ÿ) of the Lagrangian for Eq. (8), given by
s: s; HE), defined in Eq. (6), is not unbounded
C(ã, x, 3) = f()) + gg(x)= + y (ō x).
(10)
The assumption that the objective is not unbounded below is called coercivity and is relatively weak (Bauschke & Combettes, 2017). Coereivity is only broken in the rare case when there ex- ists an & * 0 such that Px = 0, Gx = 0, Ax = O, and q x < O, which causes the optimal solution to diverge. In our cases of interest, we consider over-constrained problems with many more constraints than optimization variables. thus it is extremely unlikely that this assumption does not hold. Moreover, a bounded objective can be guaranteed if P > 0 or if G or A are full column rank.
The following theorem establishes the relationship between the FlexQP solution and the solution to the original QP and can be proven using the definition of soft thresholding. The proof is given in Appendix F.
Theorem 3.3. For any PL > O, let (Ø, &t 9) solve the relaxed QP Eq. (6) using Algorithm 1. Then, 01,sl S Pr for all inequality constraints = 1..... m and [ل S ug for all equality con- straints l.....p. p. Furthermore, let (x" ./. VE) solve Eq. (3), if it is feasible. If the conditions of
Theorem 3.1 hold, then (Ø, Hr. (R) = (x", "x"".00 V-) Otherwise, for any infeasible constraint with associated dual variable yo, the FlexQP solution satisfies l0: p4c
5

---

Published as a conference paper at ICLR 2026
This shows that FlexQP solves Eq. (3) if the original QP is feasible, and otherwise identifies stationary point of the infeasibilities, similar to Nocedal & Wright (2006, Theorem 17.4).
Finally, we summarize the key roles of the different hyperparameters of our algorithm. These in- sights are important for understanding the parameterization of our deep-unfokded architecture pre- sented in the next section.
Role of Elastic Penalty Parameters: The parameters μ and pe only appear in a single step of the algorithm part of the soft thresholding in the second block ADMM updates. Larger elastic penalties l result in a larger threshold, meaning that a larger amount of constraint violation will be zeroed out. The choice of y is key for satisfying the conditions of Theorem 3.1.
The role of the parameter og is to regularize QP Eq. (9) admit a unique solution even if P is not positive definite (P = 0 captures linear programs). We tested multiple fixed values of or along with adaptive and learned rules, but a fixed o₂ 1e-6 appears to work very well in practice. This is similar to the choice of the a parameter in OSQP (Stellato et al., 2020).
Role of Augmented I. Lagrangian Penalty Parameters:
the quadratic cost matrix P' and allow the equality-constraiiee
The parameter oe plays the role of quadratic cost on the slack variable s when solving Eq. (9). also plays a small role in regularizing the constraint matrix G. In practice, tuning this parameter is the most difficult as the optimal value appears to depend strongly on the scaling the objective and the constraints. This motivates our adaptive data-driven approach described in Section 4.
The penalty parameters PJ and PE play two key roles in the algorithm. First, they regularize the constraint matrices G and A so that Eq. (9) is solvable regardless of the rank of G or A. Second, they weight the noise level the soft thresholding operations, playing an inverse role to f4 and jgg where a larger p results in smaller threshold. Determining the optimal values of these parameters by hand is unintuitive as they can have varying effects on the optimization, further motivating the deep unfolding approach presented in the next section.
4 ACCELERATING QUADRATIC PROGRAMMING VIA DEEP UNFOLDING
Feedback Parameters
ADMM First Block Update
ADMM Second Block Update
ADMM Dual Update
S
QP & ADMM Residuals
o²
07
LSTM
0/
M
Figure 2: One layer of our proposed Deep FlexQP architecture. We learn cimeension-agnotii feed- back policies for the parameters while the propagation from one layer to the next is defined by the ADMM updates Eq. (9).
We focus our study on two recently proposed acceleacee QP optimizers. The deep centralized QP optimizer from Saravanos et al. (2025) is a version of deep-unfolded OSQP where the penalty parameters p and relaxation parameter a are learned as feedback policies on the problem residuals using an analogy to feedback control. In our comparisons, we refer to their method as Deep OSQP. The main limitation of their approach is that only scalar penalty parameters are learned, but could be the case that different penalty parameters should be applied to different constraints to more ef- fectively accelerate the optimizer. This was the main motivation for the deep QP method proposed by Ichnowski al. (2021), where a policy that outputs vector of penalty parameters is learned

---

Published as a conference paper at ICLR 2026
using reinforcement learning. The vector policy is applied across the constraint dimensions so it dimension -agiostic and generalizes across different problem classes. The authors show that the vector policy outperforms the scalar one actoss suite of QP benchmarks. However, unlike Sara- vanos et al. (2025), the authors do not learn the relaxation parameter a, which can greatly improve the convergence of ADMM (Boyd et al., 2011). We implement the approach from Ichnowski et al. (2021) and train it using the supervised learning scheme from Saravanos et al. (2025), leading to the baseline Deep OSQP approach that learns a vector feedback policy for the penalty parameters while also learning policy for the ADMM relaxation parameter a, which we call Deep OSQP Improved.
RLQP Parameterization. Finally, we implement a best-of-both-wooe
4.1 DEEP FLEXQP ARCHITECTURE
Our proposed Deep FlexQP learns feedback policies for the algorithm parameters as a function of the current optimizer state as well as the QP and ADMM residuals (Fig. 2). We learn separate policies TI, Tg. and Ta for the inequality constraints, equality constraints, and relaxation parameter, respectively. The constraint policies Ty and TE take as input the ADMM variables along with the primal and dual QP residuals, and are applied in batched fashion per constraint coefficient, making the architecture independent of problem size and permutation. The relaxation policy Ta takes input the infinity norms of cach residual, providing scale-invariant representation of the convergence progress. Full expressions for the policy inputs and outputs are given in Appendix G.
All policies are parameterized by long short-term memory (LSTM) networks (Hochreiter & Schmid- huber, 1997), with the hypothesis that learning long-term dependencies can aid the selection of the optimal parameters. This furthers the idea from Saravanos et al. (2025) that time-narying feedback on the current (nominal) parameters can provide a large improvement. In our case, we are applying feedback based on latent state capturing the optimization history. Our results show that LSTMs provide the most benefit for problems where the active constraints might change many times over the course of the optimization. An ablation analysis and further discussion provided in Appendix N.
4.2 SUPERVISED LEARNING
For training Deep OSQP variants, we adopt the supervised learning approach from Saravanos et al. (2025). Let a* (0) be the fath iterate of Deep OSQP parameterized by 0. The training objective is the weighted sum of the optimality gaps between the iterates x² (0) and the optimal solution z":
K
minimize 0
Σ Th 1|0a
- x 22
(11)
where 7/ exp((k= K)/5) is a per-iteration scaling factor.
For training Deep FlexQP, we adopt a similar loss, but generalize to incorporate the optimal Lagrange multipliers based the discussion in Section 3. We also use the normalized optimality gaps instead of the unnormalized ones so that the scale is automatically determined based on the distance from the optimal solution:
minimize
Σ -0((()))-(1,1111 /1
(12)
where { = (xr, yr. WF). By including the Lagrange multipliers here, we are able to enforce the Deep FlexQP optimizer to select penalty parameters that meet the conditions of Theorem 3.1, namely that 4r > | and μ8 2 liys This is due to the fact that the Lagrange multipliers of Deep FlcxQP (0) and (0) are upper-bounded (in absolute value) by the current selection of μ (see Theorem 3.3). An ablation studying the effect of this loss is provided Appendix O.
4.3 PAC-BAYRS GENERALIZZATIO BOUNDS
Recent approaches have been proposed for establishing generalization bounds for guarantecing the performance of
methods, including a binary loss approach from Sambharya & Stellato (2025) well more informative progress metric by Saravanos et al. (2025), given
| (0) x²
T(()) = min
(13)
(0) x=
7

---

Published as a conference paper at ICLR 2026
These approaches can be used to construct PAC- PAC-Bayes general- ization bounds on the mean performance of the optimizer that hold with high probability. Nevertheless, limitation of the re- sulting PAC-Bayes bound from Eq. (13) is that assumes that the losses fall anywhere within the range [0, 1), despite the fact that, in practice, most of the final optimality gaps fall very close (on the order of 10-2 and smaller). In other words, the loss in Eq. (13) does not properly account for the scale of the errors, and as a result, obtaining meaningful bound might re- quire exponentially more training samples. For example, Fig. 4a shows that training for this generalization bound loss results a bound that is uninformative since it sits above even the vanilla optimizers and does not capture the behavior well at small errors.
10a
Residnals Optimal
10
10-1
10-5
10-2
1.0
0.5
Old Loss
New Loes
0.0
0
5
10
To address this, we design loss that is zero when the learner performs as well as or better than the optimal solution, and in- creases linearly the performance decreases a log-scale; see Fig. for visualization. Furthermore, we penalize distance from the optimal solution with respect to the norm of the QP residuals, which better takes into account the problem scale:
Figure 3: Log-scaled loss better captures small errors when the solution is close to the optimal.
L( - clip (1 (1- log (ogg) (0))
logl R((5))l2 0,, 0.1).
(14)
where RR(E) = (Px+ + GT yu + A yg. max - h, 0), Ar b) computes the residuals of the original QP in Eq. (3). As intended, training for this loss better captures the performance when the residuals are very small. Results are presented in Fig. 4b and Appendix M.
USQB
Deep Floxtjp (Curr))
FlestgP
Deep FleeQ Gen. Bound (14) (Ours)
Telacive Optimality Gop
13-9.
Daep OSQP BLQP Parm
Doep OSQF
Dag OSQP Ingeree
Deep PlestPP (Ous)
0.
FlxQP Gen.
Log p
13
0.4
10-4
0.2
16
20
10
20
Eornatio
Eornatio
(a) Deep FlexQP trained for the generalization bound loss Eq. (13).
(b) Deep FlexQP trained for our proposed gener- alization bound loss Eq. (14).
Figure 4: Optimizer comparison on 1000 test LASSO problems. Training using our log-normalized loss Eq. (14) results in a substantially more informative performance guarantee.
EXPERIMENTAL RESULTS
We evaluate our approach on three classes of problems of increasing complexity: small- medium- scale QPs, large-scale QPs, and nonconvex nonlinear programs solved via SQP. All algorithms were implemented using dense batched matrices in PyTorch, and all experiments were run on system with an NVIDIA RTX 4090 GPU. Across all experiments, we use learning rate of 10-3 and batch size of 50 (except for the large-scale problems, which use batch size 1, as larger batches exceeded the available GPU memory).
5.1 SMALL- TO MEDIUM-SCALE QPS
We apply our dcep-unfolding methodology to a benchmark suite of QPs including portfolio opti- mization problems from finance, classification and regression problems from machine learning, and
8

---

Published as a conference paper at ICLR 2026
linear optimal control problems. The results are presented in Fig. 5, and details the problem rep- resentations as well as the data generation processes are provided in Appendix H. In the following plots, OSQP and FlexQP are the best performing versions of OSQP and FlexQP, respectively, found using a hyperparameter search (details in Appendix J). Deep OSQP is the approach from Saravanos et al. (2025), Deep OSQP RLQQ Parameterization is the parameterization from Ichnowski et al. (2021), and Deep OSQP Improved is the cest-of-both-worldd version of deep-unfolded OSQP described in Section 4. Finally, Deep FlexQP is our proposed deep-unfolded FlexQP opti- mizer with LSTM policy parameterization and trained using the loss Eq. (12). Each model is trained for 500 epochs on 500 problems (except for the random QP classes, which use 2000 problems) and evaluated on 1000 test problems. We also perform an extensive timing comparison and analysis between all the above optimizers, presented in Appendix J.
Random QPs
Razdom QPs with Equilities
Poetfolio Optimization
10-1
Optionhity Cap
10
10-
Osggg
33"1
FlxQP (Ours)
10-3-
Deep OSQP
Deep USUP
REAG Parzan.
10
Deep DSQP
Impuored
1077
Deep FlexQU (Ouss
Iteeation
Itenation
Iteeation
Support Vorter Muchins
LASSD
Huber Fitting
0F-
Optimalitty Cap
10111
33-3
10'11
00-1
33.
10
10:7
10
10
Iteration
Iveration
Itecation
Random Linear OCPs
Double Integratir
Oweilacinn Masses
Optimality Capp
10-1-
10-11
103
33-1
10
15
Iteration
Ttenstion
Itecation
Figure 5: Performance comparison of learned deep optimizers and their non-learned counterparts. Our improved version of Deep OSQP outperforms the baselines, while Deep FlexQP consistently surpasses the rest of the methods in terms of convergence the optimal QP solution.
5.2 LARGE-SCALE QPs
Next, we verify how well our methodology generalizes to large-scale QPs. In order to amortize the cost of training on these large-scale problems, we adopt fine-tuning approach where the models trained on the small- to medium-scale problems from Section 5.1 are fine-tuned on limited number of large-scale problems for a few epochs. Each model was fine-tuned on 100 training problems for 5 epochs. As each epoch takes roughly 3 hours to run, we estimate that training on the same number of problems and for the same number of epochs as the problems considered in Section 5.1 would take over 300 days, showing clear benefit of the proposed fine-tuning approach.
Fig. 6 shows a comparison on 100 portfolio optimization and support vector machine (SVM) test problems with 10k variables and 10-20k constraints. Each optimizer is run until the infinity norm of the residuals falls below an absolute tolerance of e = 10 with a timcout of 10 minutes. Optimizers use the indirect method solve their first block ADMM update (see Appendix C). For the portfolio
9

---

Published as a conference paper at ICLR 2026
optimization problems, we report the average number of iterations each optimizer took to converge, and, for the SVM problems, we report the average number of conjugate gradient (CG) iterations that necessary to solve the linear systems to tolerance of ECG 10 2-5, with full results provided in Appendix K. We observe that the fine-tuned Deep FlexQP outperforms all of the other optimizers. Surprisingly, the fine-tuning approach does
work well for the Deep OSQP variants. Notably, we tried fine-tuning each of the methods using either the optimality gap loss Eq. (1I) or the Lagrange multiplier loss Eq. (12). While the two losses yielded comparable performance on the portfolio optimization problems, the Lagrange multiplier loss performed significantly worse on the SVM problems for the Deep OSQP variants. As such, the reported Deep OSQP results use the models trained with the optimality gap loss Eq. (11). These results suggest that the superior fine- tuning performance of Deep FlexQP attributable to the FlexQP architecture itself - specifically, the elastic relaxation and LSTM parameterization rather than the choice of loss function.
Aversig Solbe Time [s]
Average Tundiios Canverry
Averigo Soler Time
(6]
Averapo OG Bezations
6.1-
(20) (11
usige
FleqQP (Ours
600
120
399
200
166
Day OSQP
Deep Usur HLG'
400
Teep osge Iegioed
J20
56.4
Deep F'lstge (Curs)
303
176.
201
J112
79.1
1538
15.3
16-3
Figure 6: Learned vs. traditional optimizers on large-scale QPs. Left: portfolio optimization (10k variables, 10k constraints). Right: support vector machines (10k variables, 20k constraints).
5.3 NONCONVEX NONLINEAR PROGRAMMING USING SQP
Lastly, we apply Deep FlexQP submodule in SQP to solve nonconvex NLPs arising from non- linear optimal control and nonlinear predictive safety filter problems. Training for the generalization bound loss Eq. (14) yields numerical certificate performance that we use when designing the SQP method. Results are shown in Fig. and Fig. 7. Further details are provided in Appendix I.
SQP
Dwep FlreQu (Qurs)
SQP
Deep Plrs (Oun)
:5
LAL
23.7
Salve
Bate
Suäce Filee
(1)
Sacooles (?)
Figure 7: Comparison of approach traditional optimizer baselines quadrotor trajectory optimization problems (left) and nonlinear predictive safety filter problems (right). Ours is faster than the baselines while vastly improving the task completion rate and safety.
CONCLUSION
We present FlexQP, flexible QP solver that natively handles infeasible subproblems by minimiz- ing constraint violation when no feasible solution exists. This property makes FlexQP particularly well-suited as submodule SQP, where infeasible QP subproblems are common and typically re- quire ad hoc recovery strategies. Our accelerated variant, Deep FlexQP, outperforms both traditional solvers and learned approaches, converging in fewer iterations and less time per solve. Generaliza- tion bounds provide numerical certificate of performance, and we use these bounds to design SQP solvers for nonlinear optimal control and predictive safety filters. In our SQP experiments, Deep FlexQP provides substantial speedup over traditional approaches while also enabling graceful re- covery from infeasibility without additional machinery. Promising future directions include learning warm-starts for FI and extending the distributed QP setting of Saravanos al. (2025).
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by the Army Research Office Award IN921111111111. Alex Oshin was additionally supported by the Georgia Tech Aerospace Engineering Graduate Student Fellowship, and Augustinos Saravanos by the Onassis Foundation Scholarship.
REFERENCES
Arshiya Taj Abdul, Augustinos D Saravanos, and Evangelos A Theodorou. Scaling robust optimiza- tion for swarms: A distributed perspective. arXiv preprint UrXiv:2508.11799 2025.
Jonas Adler and Ozan Öktem. Learned primal-dual reconstruction. IEEE Transactions on Medical Imaging, 37(6):1322 1322-1332, 2018. doi: 10.1 (0.1109/MM2221182779222.
Aaron D Ames, Samuel Coogan, Magnus Egerstedt, Gennaro Notomista, Koushil Sreenath, and Paulo Tabuada. Control barrier functions: Theory and applications. In 18th European Control Conference (ECC), pp. 3420-3431. IEEE, 2019.
Brandon Amos et al. Tutorial on amortized optimization. Foundations and Trends@ in Machine Learning. 16(5):592-732, 2023.
Brian DO Anderson and John B Moore. Optimal Control: Linear Quadratic Methods. Courier Corporation, 2007.
Yuichiro Aoyama, Oswin So, Augustinos D Saravanos, and Evangelos A Theodorou. Second-order constrained dynamic optimization. arXiv preprint arXiv:2409.11649, 2024.
Alcksandr Aravkin, James V Burke, Lennart Ljung, Aurelie Lozano, and Gianluigi Pillonetto. Gen- cralized Kalman smoothing: Modeling and algorithms. Automatica, 86:63-86, 2017.
Sebastian Banert, Jevgenija Rudzusika, Ozan Öktem, and Jonas Adler. Accelerated forward-
backward optimization using deep learning. SIAM Journal on Optimization, 34(2):1236-128) 2024. doi: 10. 01577/22213355222
Goran Banjac, Paul Goulart, Bartolomeo Stellato, and Stephen Boyd. Infeasibility detection in the alternating direction method multipliers for convex optimization. Journal of Optimization Theory and Applications, 48005)4400-51, 2019.
Heinz H Bauschke and Patrick L Combettes. Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, 2017.
Stephen Boyd, Neal Parikh, Eric Chu, Borja Pelcato, Jonathan Eckstein, et al. Distributed optimiza- tion and statistical learning via the alternating direction method of multipliers. Foundations and Trends@ in Machine learning, 311):1-122, 2011.
Stephen Boyd, Mark T Mueller, Brendan O' O'Donoghue, Yang Wang, et al. Performance bounds and suboptimal policies for multi-period investment. Foundations and Trends in Optimization, (1):1-72, 2013.
Stephen Boyd, Enzo Busseti, Steve Diamond, Ronald N Kahn, Kwangmoo Koh, Peter Nystrup, Jan Speth, al. Multi-period trading via convex optimization. Foundations and Trends@ in Optimization, 3(1):1-76, 2017.
Stephen P Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
Gerald G. Brown and Glenn W. Graves. Elastic programming: A new approach to large-scale mixed integer optimization. In ORSA/TIMS Conference, Las Vegas, 1975.
James V. Burke and Shih-Ping Han. A robust sequential quadratic programming method. Mathe-
matical Programming. 43(1) (3(1):2773303, 1989. ISSN 1436-4646. doi: 10. 10000/80111882994
Emmanuel J Candes, Michael B Wakin, and Stephen Boyd. Enhancing sparsity by reweighted & minimization. Journal of Fourier Analysis and Applications, 14(5):877-905, 2008.

---

Published as a conference paper at ICLR 2026
Charles W Carroll. The created response surface technique for optimizing nonlinear, restrained systems. Operations Research, 92):169--84, 1961.
Richard M. Chamberlain, M. J. D. Powell, Claude Lemaréchal, and Häkon Pedersen. The watchdog technique for forcing convergence in algorithms for constrained optimization. In Algorithms for Constrained Minimization of Smooth Nonlinear Functions, volume 16 of Mathematical Program- ming Studies. Springer, 1982. doi: 10. 10000//F600220455.
Steven Chen, Tianyu Wang, Nikolay Atanasov, Vijay Kumar, and Manfred Morari. Large scale model predictive control with neural networks and primal active sets. Automatica, 255:109977, 2022a.
Tianlong Chen, Xiaohan Chen, Wuyang Chen, Howard Heaton, Jialin Liu, Zhangyang Wang, and Wotao Yin. Learning to optimize: A primer and benchmark. Journal of Machine Learning Research, 23(189):1-59, 2022b.
Xiaohan Chen, Jialin Liu, Zhangyang Wang, and Wotao Yin. Theoretical linear convergence of unfolded ISTA and its practical weights and thresholds. In Advances Neural Information Pro- cessing Systems, volume 31, 2018.
Xiaohan Chen, Jialin Liu, Zhangyang Wang, and Wotao Yin. Hyperparameter Tuning is All You Need for LISTA. In Advances Neural Information Processing Systems, volume 34, pp. 11678- 11689, 2021.
Andrew R. Conn and Tomasz Pietrzykowski. A penalty function method converging directly to a constrained optimum. SIAM Journal on Numerical Analysis, 14(2):348-375, 1977. ISSN 00361429.
Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine Learning, 20(3):273-297, 1995.
Richard Courant. Variational methods for the solution of problems of equilibrium and vibrations. Bulletin of the American Mathematical Society, 49(1):1-23 1943.
Brent De Weerdt, Yonina C. Eldar, and Nikos Deligiannis. Deep unfolding transformers for sparse recovery of video. IEEE Transactions on Signal Processing, 72:1782-1796, 2024. doi: 10. 1009 TSP.2024.3381749
Moritz Diehl, Hans Joachim Ferreau, and Niels Haverbeke. Efficient numerical methods for nonlin- MPC and moving horizon estimation. In Nonlinear Model Predictive Control: Towards New Challenging Applications, pp. 391-417. Springer, 2009.
Ying Ding, Xinwei Xue, Zizhong Wang. Zhiying Jiang, Xin Fan, and Zhongxuan Luo. Domain knowledge driven deep unrolling for rain removal from single image. In 2018 7th International Conference on Digital Home, pp. 14-19, 2018.
doi: 10. 1 109/CCDH22018 1.00000.
Jonathan Eckstein and Dimitri P. Bertsekas. On the Manglas-Rachfortd splitting method and the proximal point algorithm for maximal monotone operators. Mathematical Programming, 55(1): 293-318, 1992. ISSN 1436-4646. doi: 10. 10.1007/BF0155812444
Ivan I. Eremin. The penalty method in convex programming. Soviet Math. Dokl.,
1966.
Tomaso Erseghe. Distributed optimal power flow using ADMM. IEEE Transactions on Power Systems, 2935)22370 2014.
Patrick Fahy, Mohammad Golbabace, and Matthias J Ehrhardt. Greedy learning to optimize with convergence guarantees. arXiv preprint arXiv:2406.00 00260, 2024.
Liang Fang, Stefan Vandewalle, and Johan Meyers. An SQP-based multiple shooting algorithm for large-scale PDE-constrained optimal control problems. Journal of Computational Physics, 477: 111927, 2023.
Anthony V. Fiacco and Garth P. McCormick. Nonlinear Programming: Sequential Unconstrained Minimization Techniques. Wiley, 1968.
12

---

Published as a conference paper at ICLR 2026
Roger Fletcher. A model algorithm for composite nondifferentiable optimization problems. Mathe- matical Programming Study, 17:67-76, 1982.
Roger Fletcher. An & penalty method for nonlinear constraints. Numerical Optimization, 1984: 26 26-40, 1985.
Roger Fletcher and Sven Leyffer. Nonlinear programming without penalty function. Mathematical Programming. 91(2):239- 269, 2002. ISSN 1436-4646. doi: 10.1007/1000701100244
Ragnar Frisch. The logarithmic potential method convex programming. Memorandum, University Institute of Economics, Oslo, 1955.
Daniel Gabay. Chapter IX applications of the method of multipliers variational inequalities. In Michel Fortin and Roland Glowinski (eds.), Augmented Lagrangian Methods: Applications the Numerical Solution of Boundary-Valee Problems, volume 15 of Studies in Mathematics and Its Applications, pp. 299-331. Elsevier, 1983. doi: 10.1016/S0168- -0000000608-224407700011111
Daniel Gabay and Bertrand Mercier. A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Computers and Mathematics with Applications, 2(1): 17-40, 1976.
Michael Garstka, Mark Cannon, and Paul Goulart. COSMO: conic operator splitting method for convex conic problems. Journal of Optimization Theory and Applications, 190(3):77--810, 2021.
Philip E. Gill. What's new in active-set methods for nonlinear optimization?
In Advances in
Numerical Computation, Manchester Univesity, 2011.
Philip E Gill and Elizabeth Wong. Sequential quadratic programming methods. In Mixed Integer
Nonlinear Programming, pp. 147-224. Springer, 2011.
Philip E Gill, Walter Murray, and Michael A Saunders. SNOPT: An SQP algorithm for large-scale constrained optimization. SIAM Review, 1311 2005.
Roland Glowinski and Americo Marroco. Sur l'approximation, par éléments finis d'ordre un, et la unu classe de problèmes Dirichlet non linéaires. Revue
résolution, par pénalisation-duall Française d' A'Attmmatquue, Informatique, Recherche Opérationnelle. 9(R2):11-66, 1975.
Karol Gregor and Yann LeCun. Learning fast approximations of sparse coding. In Proc. Interna- tional Conference on Machine Learning (ICML 2010.
Harshit Gupta, Kyong Hwan Jin, Ha Q. Nguyen, Michael T. McCann, and Michael Unser. CNN- based projected gradient descent for consistent CT image reconstruction. IEEE Transactions Medical Imaging. 97(6):1440-1455 2018. doi: 10.1 10.11099/M1.20118282222
William W Hager. Stabilized sequential quadratic programming. Computational Optimization and
Applications, 12(1):253-273: 1999.
Shih-Pin Sing-Ping Han. A globally convergent method for nonlinear programming. Journal of Optimization Theory and Applications, 22(3):297-309, 1977. ISSN 1573-2878. doi: 10.1007/BP00933288.
Shih-Ping Han and Olvi L Mangasarian. Exact penalty functions in nonlinear programming. Math- ematical Programming, 17(1):251-266 1979.
Bingsheng He and Xiaoming Yuan. On the ((1nn) convergence rate of the Douglas Rouhlos-Rachford alternating direction method. SIAM Journal on Numerical Analysis, 50(2):700-709 2012. doi: 10.1137/1 10836936.
Howard Heaton, Xiaohan Chen, Zhangyang Wang, and Wotao Yin. Safeguarded learned convex optimization. Proceedings of the AAAI Conference on Artificial Intelligence, 37(6):748877555, 2023. doi: 10. 1609/aaai (6609/aai.v3376.259999 v3716.25950.
Georg Heigold, Ignacio Moreno, Samy Bengio, and Noam Shazeer. End-to-end text-dependent speaker verification. 2016 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 5115-5119, 2016. doi: 10. 199IAAS99221117777555
13

---

Published as a conference paper at ICLR 2026
John R. Hershey, Jonathan Le Roux, and Felix Weninger. Deep unfolding: Model-basee inspiration of novel deep architectures. arXiv preprint viXiv:1409.253 2014.
Magnus R. Hestenes. Multiplier and gradient methods. Journal of Oprimization Theory and Appli- cations, 4(5):303-320, 1969. ISSN 1573-2878. doi: 10. 10.1007/BF00922773.
Sepp Hochreiter and Jürgen Schmidhuber. 1735-1780, 1997.
Long short-term memory. Neural Computation, 9(8):
Peter Huber. Robust estimation of a location parameter. The Annals of Mathematical Statistics, 35 (1):73-101, 1964.
Peter J. Huber. Robust statistics. Wiley Series in Probability and Mathematical Statistics, 1981.
Jeffrey Ichnowski, Paras Jain, Bartolomeo Stellato, Goran Banjac, Michael Luo, Francesco Borrelli, Joseph Gonzalez, Ion Stoica, and Ken Goldberg. Accelerating quadratic optimization with reinforcement learning. Advances in Neural Information Processing Systems, 34:21043-21055, 2021.
Alexey F Izmailov and Mikhail V Solodov. Stabilized SQP revisited. Mathematical Programming, 133(1):93- 120, 2012.
Narendra Karmarkar. A new polynomial-time algorithm for linear programming. In Proceedings of the Sixteenth Annual ACM Symposium on Theory of Computing. Stoc '84, pp. 302-311, New York, NY, USA, 1984. Association for Computing Machinery. ISBN 0-89791-133-4,, doi: 10. 1145/800057 808699..
David Kiessling, Sven Leyffer, and Charlie Vanaret. A unified funnel restoration SQP algorithm. Mathematical Programming, 2025, ISSN 1436-4646. doi: 10.1007/110107-05-002222
Victor Klee and George J. Minty. How good is the simplex algorithm? Technical Report, Department of Mathematics, University of Washington, 1970.
Sven Leyffer. Integrating SQP and branch-and-boune for mixed integer nonlinear programming. Computational Optimization and Applications, 18(3):295-309, 2001.
Jialin Liu, Xiaohan Chen, Zhangyang Wang, and Wotao Yin. ALISTA: Analytic weights are as good as learned weights in LISTA. In International Conference on Learning Representations, 2019.
Jialin Liu, Xiaohan Chen, Zhangyang Wang, Wotao Yin, and Hanqin Cai. Towards constituting mathematical structures for learning to optimize. In Proceedings of the 40th International Con- ference on Machine Learning, volume 202, pp. 21426-21449, PMLR, 2023.
Huynh Van Luong, Boris Joukovsky, and Nikos Deligiannis. Designing interpretable recurrent neu- ral networks for video reconstruction via deep unfolding. IEEE Transactions on Image Process- ing, 30:4099-4111 13, 2021. doi: 10. 109/TTP.2021.3099229
Anirudha Majumdar, Alec Farid, and Anoopkumar Sonar. PAC-Bayes control: Learning policies that provably generalize to novel environments. The International Journal of Robotics Research, 40(2-3):574-59,, 2021.
Nicholas Maratos. Exact Penalty Function Algorithms for Finite Dimensional and Control Opti- mization Problems. PhD thesis, Imperial College London (University of London), 1978.
Morteza Mardani, Qingyun Sun, David Donoho, Vardan Papyan. Hatef Monajemi, Shreyas Vasanawala, and John Pauly. Neural proximal gradient descent for compressive imaging. In Advances in Neural Information Processing Systems, volume 31, 2018.
Harry Markowitz. Portfolio selection. The Journal of Finance, 7(1):77-91, 1952. ISSN 00221082, 15406261.
Andrea Martin and Giuseppe Belgioioso. Learning to accelerate Krasnosel skii-Mann fixed- sxentpointt iterations with guarantees. arXiv preprint arXiv:2601.0700 2026.
14

---

Published as a conference paper at ICLR 2026
Andrea Martin, lan R Manchester, and Luca Furieri. Learning to optimize with guarantees: plete characterization linearly convergent algorithms. arXiv preprint TrXi::2508.00775 2025.
Gonzalo Mateos, Juan Andrés Bazerque, and Georgios B Giannakis. Distributed sparse linear gression. IEEE Transactions on Signal Processing, 58( (2(10))2222-276 2010.
Luke Metz, Niru Maheswaranathan, Jeremy Nixon, Daniel Freeman, and Jascha Sohl-Dickstein Understanding and correcting pathologies in the training of learned optimizers. In Proceedings of the 36th International Conference on Machine Learning, volume 97, pp. 4556-4565. PMLR, 2019.
Sleiman Mhanna, Gregor Verbič, and Archie C Chapman. Adaptive ADMM for distributed AC optimal power flow. IEEE Trunsactions on Power Systems, 34(3):0055-2035, 2018.
Hans Mittelmann. Benchmarks for optimization software, 2010. URL http: /ttptppa.oooo.o// Calatooaao asu.
edu/bench. html. Accessed on September 2025.
Vishal Monga, Yuelong Li, and Yonina C Eldar. Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing. IEEE Signal Processing Magazine, 38(2):18-44, 2021.
Oscar Danilo Montoya, Walter Gil-González, and Alejandro Garces. Sequential quadratic program- ming models for solving the OPF problem in DC grids. Electric Power Systems Research, 169; 18-23, 2019.
Thomas Moreau and Joan Bruna. Understanding trainable sparse coding with matrix factorization. In International Conference on Learning Representations, 2017.
Yurii Nesterov and Arkadii Nemirovskii. Interior-Point Polynomial Algorithms in Convex Program- ming. SIAM, 1994.
Jorge Nocedal and Stephen J Wright. Numerical Optimization. Springer, 2006.
Brendan O' O'Donoghue, Eric Chu, Neal Parikh, and Stephen Boyd. Conic optimization via operator splitting and homogeneous self-dual embedding. Journal of Optimization Theory and Applica- tions, 169(3):1042-1100 1068, 2016.
Tomasz Pietrzykowski. An exact potential method for constrained maxima. SIAM Journal on Nu- merical Analysis, 6(2):299-304, 1969. ISSN 0036-1429, 1095-7170. doi: 10.1137/0706028.
Michael JD Powell. A method for nonlinear constraints in minimization problems. Optinization, pp. 283-298, 1969.
James Blake Rawlings, David Q Mayne, Moritz Diehl, et al. Model Predictive Control: Theory, Computation, and Design, volume 2. Nob Hill Publishing Madison, WI, 2020.
Ralph Tyrrell Rockafellar. Convex Analysis. Princeton University Press, 1970.
Francesco Sabatino. Quadrotor control:
Modeling, nonlinear control design, and simulation. Mas- ter's thesis, KTH Royal Institute of Technology, 2015.
Rajiv Sambharya and Bartolomeo Stellato. Learning algorithm hyperparameters for fast parametric convex optimization. arXiv preprint arXiv:2411.15717, 2024.
Rajiv Sambharya and Bartolomeo Stellato. Data-driven performance guarantees for classical and 2025.
learned optimizers. Journal of Machine Learning Research,
Rajiv Sambharya, Georgina Hall, Brandon Amos, and Bartolomeo Stellato. End-to-end learning to warm-start for real-time quadratic optimization. In Learning for Dynamics and Control Confer- ence, pp. 220-234. PMLR, 2023.
Rajiv Sambharya, Georgina Hall, Brandon Amos, and Bartolomeo Stellato. Learning Warm-Start Fixed-Point Optimization Algorithms. Journal of Machine Learning Research, 25(166):1-46, 2024. http: w.w.ooggggpp..ee... v2.eess//25//// 20225/2231111 1174 html.
15

---

Published as a conference paper at ICLR 2026
Rajiv Sambharya, Jinho Bok, Nikolai Matni, and George Pappas. Learning acceleration al- gorithms for fast parametric convex optimization with certified robustness. arXiv preprint arXiv:2507.1626,, 2025.
Augustinos D Saravanos, Alexandros Tsolovikos, Efstathios Bakolas, and Evangelos Theodorou. Distributed covariance steering with consensus admm for stochastic multi-agent systems. In Pro- ceedings of Robotics: Science and Systems, Virtual, July 2021. doi: 10. 20155667/SS..221..1000 075.
Augustinos D. Saravanos, Yuichiro Aoyama, Hongchang Zhu, and Evangelos A. Theodorou. Dis- tributed differential dynamic programming architectures for large-scale multiagent control. IEEE Transactions on Robotics, 39(6):4387-444) 4407, 2023. doi: 10.1109TRO.2023.0 2023.3312993999
Augustinos D Saravanos, Hunter Kuperman, Alex Oshin, Arshiya Taj Abdul, Vincent Pacelli, and Evangelos Theodorou. Deep distributed optimization for large-scale quadratic programming. In The Thirleenth International Conference Learning Representations, 2025.
Damien Scieur, Gauthier Gidel, Quentin Bertrand, and Fabian Pedregosa. The curse of unrolling: Rate of diffcrentiating through optimization. Advances in Neural Information Processing Sys- tems, volume 35, pp. 17133-17144 2022.
Nir Shlezinger and Yonina C. Eldar. Model-Based Deep Learning. Foundations and Trends in Signal Processing, 17(4):291-416 2023. ISSN 1932-8346. doi: 10. 10.1561/20000000113
Bruno Siciliano, Lorenzo Sciavicco, Luigi Villani, and Giuseppe Oriolo. Mobile Robots, pp. doi: 10.1007/
469-521. Springer London, London, 2009. ISBN 978-1-84422-642-11
Oren Solomon, Regev Cohen, Yi Zhang, Yi Yang. Qiong He, Jianwen Luo, Ruud J. G. van Sloun, and Yonina C. Eldar. Deep unfolded robust PCA with application clutter suppres- sion in ultrasound. IEEE Transactions on Medical Imaging, 39(4):1051-1063, 2020. doi: 10.1
Bartolomeo Stellato, Goran Banjac, Paul Goulart, Alberto Bemporad, and Stephen Boyd. OSQP: An operator splitting solver for quadratic programs. Mathematical Programming Computation, 12(4):637-672, 2020.
Michael Sucker and Peter Ochs. A generalization result for convergence in learning-to-optimize In Proceedings of the 42nd International Conference Machine Learning, volume 267, pp- 57212-57230. PMLR, 2025.
Michael Sucker, Jalal Fadili, and Peter Ochs. Leamningto--optmnize with PAC-Bayesian guarantees: Theoretical considerations and practical implementation. Journal of Machine Learning Research, 26(211) 16:111) 2025.
Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology, 58(1):267-288, 1996.
Kim Peter Wabersich and Melanie N Zeilinger. predictive safety filter for learning- based control of constrained nonlinear dynamical systems. Automatica, 129:109597, 2021.
Zhaowen Wang, Ding Liu, Jianchao Yang, Wei Han, and Thomas Huang. Deep networks for im- age super-resolution with sparse prior. In Proceedings of the IEEE International Conference on Computer Vision, pp. 370-378, 2015.
Robert B. Wilson. A Simplicial Method for Convex Programming. PhD thesis, Harvard University, 1963.
Philip Wolfe. The simplex method for quadratic programming. Econometrica, ISSN 00129682, 14680262.
1959.
Margaret Wright. The interior-point revolution in optimization: History, recent developments, and lasting consequences. Bulletin of the American Mathematical Society, 42(1):39-55 56, 2004. ISSN 0273-0979, 1088-9485. doi: 10. 001190000773.007994444400040
16

---

Published as a conference paper at ICLR 2026
Stephen Wright. Superlinear convergence of stabilized SQP method degenerate solution. Computational Optimization and Applications, 11(3):253-22) 2752,, 1998.
Xingyu Xie, Jianlong Wu, Guangean Liu, Zhisheng Zhong, and Zhouchen Lin. Differentiable Lin- earized ADMM. In Proceedings of the 36th International Conference Machine Learning, volume 97, pp. 6902-6911, PMLR, 2019.
Jinxin Xiong, Xi Gao, Linxin Yang. Jiang Xue, Xiaodong Luo, and Akang Wang. Solving quadratic programs via deep unrolled Douglas-Rachfon splitting. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL https: /wwpreeeeewww..//////p net/ forum? id=xOf0gPnbtF.
Jiangjiao Xu, Hongjian Sun, and Chris J Dent. ADMM-based distributed OPF problem meets stochastic communication delay. IEEE Transactions Smart Grid, 1065):5466-5066, 2018.
Yan Yang, Jian Sun, Huibin Li, and Zongben Xu. Deep ADMM-net for compressive sensing MRI. In Advances in Neural Information Processing Systems, volume 29, 2016.
Yan Yang, Jian Sun, Huibin Li, and Zongben Xu. ADMM-CSNet: A deep learning approach for image compressive sensing. IFFE Transactions on Pattern Analysis and Machine Intelligence, 42 (3):521-538, 2020. doi: 10. 10.1109/TPMM.2018..22 22883941.
Ji Yin, Charles Dawson. Chuchu Fan, and Panagiotis Tsiotras. Shield model predictive path integral: A computationally efficient robust MPC method using control barrier functions. IEEE Robotics and Automation Letters, 8(11 (-111)7716--11 1:7106-7113, 2023.
Willard Zangwill. Non-linear programming via penalty functions. Management Science, 13(5): 344-358, 1967.
Jian Zhang and Bernard Ghanem. ISTA-net: Interpretable optimization-inspire deep network for image compressive sensing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
Ruiliang Zhang and James Kwok. Asynchronous distributed ADMM for consensus optimization. In International Conference on Machine Learning, pp. 1701-1709, PMLR, 2014.
17

---

Published as a conference paper at ICLR 2026
A EXTENDED RELATED WORK
Quadratic Programming. Active-set methods, developed extensions of the simplex method for QPs (Wolfe, 1959), can be efficiently warm-started but suffer from worst-case exponential com- plexity (Klee & Minty, 1970). Interior-point methods rose prominence with the polynomial-time algorithms of Karmarkar (1984) and their extension to general convex programs via seff-concordant barriers (Nesterov & Nemirovskii, 1994), replacing active-set methods as the dominant approach for small- to medium-scale problems (Wright, 2004). However, interior-point methods often do not scale well to high-dimensional problems and are difficult warm-start. This has led to a grow- ing interest in first-order methods in recent years. These methods are largely based on operator splitting methods such as ADMM, which is equivalent to Douglas-Rachton splitting applied to the dual (Gabay, 1983). First-order methods are appealing as they scale favorably to millions of variables and can be effectively warm-started, unlike interior-point methods. In addition, they are especially well-suited to applications such MPC, where only moderate accuracy is required and the slow tail convergence of ADMM (He & Yuan, 2012) is less problematic.
SCS (O' Donognue et al., 2016) is first-order solver for general conic programs based on homo- geneous self-dual embedding. However, using this embedding is inefficient for QPs since it requires reformulating the quadratic cost as second-order cone constraint. OSQP (Stellato et al., 2020) and COSMO (Garstka et al., 2021) are 4DMM-based solvers specifically designed for QPs and general conic programs, respectively. COSMO can solve QPs special case, with the resulting algorithm being nearly identical to OSQP. We discuss these methods further below in the context of infeasibility detection.
The proposed FlexQP algorithm is a first-order method based on ADMM that has the same per- iteration complexity OSQP and COSMO (see Appendix C), but uniquely handles infeasible QPs through an exact { relaxation of the constraints (Section 3).
Penalty Methods and Constrained Optimization. Penalty methods have long history in con- strained nonlinear programming, starting from the early works by Courant (1943) proposing the quadratic penalty method, to the log and inverse barrier methods of Frisch (1955) and Carroll (1961). Fiacco& & McCormick (1968) provided unifying perspective on exterior-poine penalty methods and interior-point barrier methods, and also began the analysis on primal-dual methods.
However, both exterior-point and interior-point methods suffer from poor numerical conditioning as the algorithm converges because the penalty parameter needs to asymptotically approach infinity (for exterior-point methods) or zero (for interior-poing methods). This led to the development of exact penalty methods, based primarily on ε penalty functions (Eremin, 1966; Zangwill, 1967). These methods are more numerically stable because the penalty parameter need only be larger than the largest Lagrange multiplier. The necessary and sufficient conditions for exactness were given by Pietrzykowski (1969). The & penalty function was also used by Han (1977) to prove the global convergence of SQP, foreshadowing the key role that exact penalties play in handling infeasible subproblems, as we discuss below. modern discussion is given by Nocedal & Wright (2006) and we use these results to prove the exactness of our relaxed problem formulation (Theorem 3.1).
While exact penalty methods fixed the numerical ill-conditioning of previous approaches, the L penalty function is non-differentiable at the boundary of the feasible region, resulting in a phe- nomenon known as the Maratos effect (Maratos, 1978). This sparked interest in methods that mod- ify the SQP algorithm to preserve convergence, such as watchdog procedures (Chamberlain et al., 1982) and second-order corrections (Conn & Pietrzykowski, 1977: Fletcher, 1982).
Operator Splitting and ADMM. Concurrently, the augmented Lagrangian (AL) method, also known as the method of multipliers, was developed by Hestenes (1969) and Powell (1969). The alternating direction method of multipliers (ADMM) was subsequently developed by Glowinski & Marroco (1975) and Gabay & Mercier (1976). The popularity of ADMM increased significantly in the 2010s after its suitability for solving large-scale optimization problems was highlighted by Boyd et al. (2011). Since then, highly scalable ADMM- based methods have been developed for sig- nal processing (Mateos al., 2010; Zhang & Kwok, 2014), multi- agent robotics (Saravanos et al., 2021: 2023; Abdul et al., 2025), and power systems (Erseghe, 2014; Mhanna et al., 2018; Xu et al., 201 8), among other domains.
18

---

Published as a conference paper at ICLR 2026
Infeasibility identification for ADMM-based methods was a major concern, it was established that the iterates of ADMM diverge if the problem is infeasible (Eckstein & Bertsekas, 1992). SCS is an ADMM- -bssed method for general conic programs that solves homogeneous self-dual embedding to identify infeasibilities (O' Donoghhe et al., 2016). The work by Banjac et al. (2019) showed that the differences of the ADMM iterates are convergent and can be used construct certificates of infeasibility for the original problem. This technique is used by both OSQP (Stellato et al., 2020) and COSMO (Garstka et al., 2021) identify infeasible problems.
Unlike these approaches, FlexQP certifies infeasibility through its relaxation rather than as separate detection mechanism, returning a maiimumm-violatinn solution directly. This addresses weakness of the existing methods, as even though they can successfully identify infeasible problems, they do not return a useful approximate solution to the original problem that can be used by the user.
Sequential Quadratic Programming. Since its inception in 1963 (Wilson, 1963), SQP has evolved into a general methodology for solving complex and highly constrained NLP problems. A key chal- lenge in SQP methods is that the QP subproblems formed by linearizing the constraints can become infeasible, even when the original NLP is feasible. This issue is particularly relevant for learning- based settings, where the iterative training procedure often causes intermediate QP subproblems to become infeasible.
Several strategics have been developed to handle such infcasibilities; recent unified perspective of these methods is provided by Kiessling et al. (2025). One approach is to run a feasibility restoration phase to locate feasible point when infeasible QP is encountered (Nocedal & Wright, 2006), such as in the FilterSQP method by Fletcher & Leyffer (2002). However, these procedures require extensive tuning and are not scalable for batched settings. A complementary approach is to modify the QP subproblem itself to ensure that a meaningful step can always be computed. Burke & Han (1989) proposed robust QP subproblem for this purpose. and SNOPT's elastic mode (Gill et al., 2005) famously solves a relaxed QP when the original subproblem is infcasible. Mcanwhile, sta- bilized SQP methods (Wright, 1998; Hager, 1999; Izmailov & Solodov, 2012) construct modified QP subproblems using an augmented Lagrangian objective, providing a mechanism to handle the degeneracy of practical optimization.
At the 2011 Advances in Numerical Computation Workshop, Gill (2011) noted that "almost all prac- tical optimization problems are degenerate," emphasizing the role that a robust QP solver plays the field of NLP. Rather than handling infeasibilities and degeneracies on a per-problem basis, FlexQP unifies their treatment within a single batched solver, making it well-suited for tearning-basee SQP methods and GPU implementations. These considerations are particularly relevant for nonlinear MPC (Diehl et al., 2009; Rawlings et al., 2020) and safety-critical control (Ames et al., 2019; Waber- sich & Zeilinger, 2021), where SQP is used in real-time control loops. A recent overview of con- strained second-order trajectory optimization methods, including SQP and augmented Lagrangian- based approaches, is provided by Aoyama et al. (2024).
Learning to Optimize and Deep Unfolding. Learning to optimize or amortized optimization has established itself a key technique for improving decision making using data-driven tech- niques (Chen et al., 2022b; Amos et al., 2023). Recent work has focused on understanding the representations (Liu et al., 2023), training dynamics (Metz et al., 2019; Scieur et al., 2022), and con- vergence (Sucker & Ochs, 2025) of learning-to-opee -o-timmize approaches. These methods can be broadly classified into two main categories: model-fre approaches that learn an optimizer from scratch, and model-based approaches that augment an existing algorithm with learnable components (Shlezinger & Eldar, 2023).
Deep unfolding, also known as algorithm unrolling (Monga et al., 2021), is a model-based learning- to-optimize approach that began with the seminal work by Gregor & LeCun (2010) on the learned iterative shrinkage and thresholding algorithm (LISTA). The LISTA framework has been extensively analyzed and improved over the years (Moreau & Bruna, 2017: Chen et al., 2018; Liu et al., 2019; Chen et al., 2021), and deep unfolding has since been applied to various fields, including speech processing (Hershey et al., 2014; Heigold et al., 2016), compressive imaging (Zhang & Ghanem, 2018; Mardani et al., 2018), and video reconstruction (Luong et al., 2021; De Weerdt et al., 2024), among others (Adler & Öktem, 2018; Gupta et al., 2018; Solomon et al., 2020). Deep unfolding has also been effectively applied to ADMM- based problems and algorithms (Yang al., 2016; Ding et al., 2018; Xie et al., 2019; Yang et al., 2020; Saravanos et al., 2025).
19

---

Published as a conference paper at ICLR 2026
Deep FlexQP is deep-unfolded optimizer that learns hyperparameter policies from example prob- lems of interest but maintains the mathematical structure from the underlying optimization process to inherit the guarantees and interpretability of the base optimizer.
Learned Solvers for Quadratic Programming. Learning- based approaches for quadratic pro- gramming have received considerable attention in recent years. Methods based warm-starting determine a good initialization for a downstream solver using machine learning. Sambharya et al. (2023) learn a warm-start network for a Rouglas-Rachford splitting QP solver, and the method was extended to general fixed-point algorithms in Sambharya et al. (2024). A related approach uses a deep-unfolded Douglas-Rachford splitting solver to warm-start SCS (Xiong et al., 2025).
A complementary class of approaches enhances the iterations of an optimization algorithm with learnable components. Deep FlexQP falls within this latter category of methods. Ichnowski et al. policies for selecting the penalty
(2021) use reinforcement learning to learn dimension-agnosse parameters of OSQP. Saravanos et al. (2025) presented deep-unfolded optimization architectures for centralized and distributed QP, drawing analogy to closed-loop control in order to design learned feedback policies for the hyperparameters of centralized and distributed OSQP. These approaches demonstrated remarkable scalability, solving problems with tens even hundreds of thousands of variables to satisfactory accuracy under limited computational budgets. Similarly, Sambharya & Stellato (2024) learn step-varying and steady-state hyperparameterss for several methods, including from Ichnowski et al. (2021) along
OSQP and SCS. We use the insight of cimension-independeenn with the closed-loop control analogy from Saravanos al. (2025) to learn component-wis wise LSTM policies for the hyperparameters of FPexQP. Consistent with the findings of Sambharya & Stellato (2024), the learned policies quickly adjust hyperparameters within the first few iterations before converging to steady-state regime (see Appendix L).
Performance Guarantees for Learned Optimizers. Providing formal performance guarantees for leamed optimizers remains an open challenge. Some asymptotic convergence guarantees have been derived using safeguarding mechanisms (Heaton et al., 2023) or by ensuring that the iterates do not deviate very far from a known convergent algorithm (Banert et al., 2024; Martin et al., 2025; Martin & Belgioioso, 2026). Other approaches ensure convergence through greedy objective descent (Fahy et al., 2024) or provide deterministic worst-case certificates via the performance estimation problem (PEP) framework (Sambharya et al., 2025).
Another class of guarantees relies on PAC-Bayes bounds (Sambharya & Stellato, 2025; Sucker et al., 2025; Saravanos et al., 2025; Sucker & Ochs, 2025), which provide certificates of performance that hold with high probability. However, the loss functions used in these works are based binary objectives or linear ratios that saturate as the optimizer converges, yielding uninformative bounds when the learned optimizer achieves very low error. To address this, develop PAC-Bayes bounds based on a log- scaled, residual-basse based loss function that better captures the performance of the optimizer the error gets arbitrarily small (see Section 4.3 and Appendix M).
B PROOF OF THBOREM 3.1
First, we state equivalent representation of the relaxed QP:
Lemma B.I. The relaxed QP Eq. (6) can equivalently be expressed as the following optimization:
1 2n
min 2
(x: (B) = Px q TT Tx + pz ll(Gr h)+ + pe || - bll1
(15)
Proof. Convert the optimization to the form without slack variables. This can be equivalently de- rived by directly relaxing the constraints of the original QP Eq. (3) using & penalties.
Now, the sketch of the proof of Theorem 3.1 is as follows. We will show for any pr 2 21 = -11vi and for any pe 2 ,r; = llviel that
1. if x² solves Eq. (3), then x' solves Eq. (15), and
2. if é solves Eq. (15) then & solves Eq. (3).
20

---

Published as a conference paper at ICLR 2026
Part 1: Let x" solve Eq. (3). For any E R" we have that
1 x
o(;; Ma pi, ME) =
Pr Pr+ Pr+qua x ++ pl ll( h)+111 + ME || -
bl11
(16a)
2
1 Px + q [x x + μi Σ 9 x - --)+++++ + PE
2²
ag
byl
(16b)
1=1
2
1 2
Pr + +q xxx= 111t
E(9) hw)+ + -11//ell
m
Σ
P
aa x bil
(16c)
i=1
im1
1
p
2 x
Px 22+490 x
vi. x -hi)+ Σ
yE la x --b bil
(16d)
1 2"
Px Pr+q-x=
(o x -hh)) + Σ yv (a. (a..= x bi)
(16e)
1
Px +
Σ vi (gt x² hhs + 93 (xr - x²))
(16f)
2"
Va x² - + az (a - xx))
Px Pr x - Σ Vi (x - -x")= + Σ
2
=
2"
vo (x x²)
(16g)
i=1
1
I
Pr Pr+ x (G yi A VE) (x - x²)
(16h)
2
1
x
Px 4 x- (Pr" + 4) T (x x²)
(16i)
1
1
x²
Px + x' + 2
- x²) P(z x²)
(16j)
2
2 x²
1
T
Px + +q
(16k)
=x x² T Px + q + pr (G h)+ mE I|Ax= -
611
(161)
<lx"; (r,a=;;,,H0 pF).
(16m)
The step from Eq. (16b) to Eq. (16c) follows from the fact that P4 2 |1y/ lla and pg 2 liviell and Eq. (16d) follows from the definition of the infinity norm. Eq. (16e) follows from the definition of
ReLU and the absolute value and Eq. (16f) is implied by the linearity of the constraints. Obtaining Eq. (16g) follows from the complementary slackness condition (9, x= - h,) = 0 for all i = 1,..., m. and feasibility a. x² b, 0 for all i = 1.... p. Eq. (161) comes from the stationary condition Px* + GT vj + A A vi= 0. Finally, obtaining Eq. (16k) follows from the positive semidefiniteness of P.
Thus, we have shown that <(x"; Mr, Mg) < ((;; pr, ME) for any which implies that x² minimizes (ax; Pr, ML) and therefore solves Eq. (15).
Part 2: Next, let é solve Eq. (15). If xx # à solves Eq. (3), then we have that
dbi (it MB)
Pt + q T + μu || h1) + uE || - bll
(17a)
2
1
< x
Px" - + ps |I(xa h)+ ae ||Ax* bl11
(17b)
1
x0 Px +9 TT¹
(17c)
which follows by the optimality of é. Now, assume that å is not feasible for Eq. (3). Then
2 T Pa
+
T
x 2 x²
'x" + g x" + (Px" + 9) T (2 - x
(18a)
2
2
21

---

Published as a conference paper at ICLR 2026
m
T Px + xx²
9 (Ø -x²) Σ
yE aj (Ø x²)
(18b)
2
Px + x²"
11 (g 3 h4 -gg x² -h)))
(18c)
i=1
yE V0 T 5 - bi (a, x² - -b)))
1=1
Vi, (9. 2 - -ha) Σ VE, ( 4 bi)
1
Pœ + -9"" x²
(18d)
2
i=1
2 1
Px + Tx" x² - 14
Di
TT i h4 14
ai 5 bi
(18e)
im1
i=1
m
2 2 T Px + q Txa p1
(g) h:)+ ug
|a à - bil
(18f)
where we have used the same facts as in Part 1. Rearranging, we have that
1 x. Px + TTx0,
# Pt + q 2 pe ll h)+ll + pe || b11 2
(19)
2d
either x² or this contradicts the fact that & minimized (((sM1,,M) Eq. (17c). Thus, must be feasible for Eq. (3). Therefore, by Eq. (17c) we have that
5 P + """
Px + +q
(20)
so à minimizes the quadratic objective and thus solves Eq. (3), completing the proof.
C FLEXQP - FIRST BLOCK ADMM UPDATE
The most computationally demanding step of FlexQP is the first block update Eq. (9), which is an cquality QP:
minimize 2,8,
PT + --gT (a,/2) || x² (o./2) s + +0 20ll
(21a)
+ (p1/2) | ン + pi Vi vīll (PE/2) 2) || - 2 + PE v
subject
2 G h, ig A b.
(21b)
The optimality conditions for this QP given by
PI + q + on(ā 2²) + G ūj + A 0g = 0,
(22a)
0. - 5 + w + Dj 0,
(22b)
pr(31 - =/)) + .
Pr O,
(22c)
pr(3F - + WE
DF 0, Gi - 3 - žJ
(22d)
O, Ai 2g b O,
(22e)
(22f)
where iy Rm and Dn R the Lagrange multipliers for the equality constraints Eq. (21b).
Solving this QP as-is would be expensive since it requires solving a linear system of size n+3m+ +2p. However, we can eliminate 5, ε,, and åg using Eqs. (22b) (22d) above, the linear system simplifies to
P G -(o
TT
AT 0
oa q
+ p -
)
ū1 =
h
-
k
+ to= : + 2/ Pj vi
(23)
A
0
-PF 1
-
PF yr
22

---

Published as a conference paper at ICLR 2026
with the eliminated variables recoverable using
= s
0x 3 0, ViI.
(24a)
z Pí Vi + Pi VI.
(24b)
Eg = E PE ME + PE Vg.
(24c)
The coefficient matrix in the linear system Eq. (23) is always full rank due to the positive parameters and introduced through the ADMM splitting. This linear system can be solved using direct method such as LDL factorization requiring using the direct method. On the other hand, for large-scale QPs, i.e. when n + m + is very large, factoring this matrix can be prohibitively expensive. this case, we can use an indirect method to solve the reduced system
time, the same as OSQP
a.r - (h 8" +og w + + pT+u))
(25)
A 2 PE VEL,
where - and A PEA. This can be obtained by eliminating ūy and ig from the linear system Eq. (23). These variables are recoverable using
1 (0 PI IGE sk 0s u 2 PJ h), Ëg PE 2 + PE V b).
(26a)
(26b)
The coefficient matrix in Eq. (25) is always positive definite, so the linear system can be solved using an iterative algorithm such as the conjugate gradient (CG) method. The linear system of size n, matching the complexity of OSQP using the indirect method. In this work, we consider supervised learning setting where we will need to compute derivatives of the solution á with respect to the parameters or, Pl, etc. While each iteration of the CG method is very fast, it can require many iterations to converge to a low-error solution. would be very inefficient to backpropagate through all these iterations of the CG method, the main issue being the high memory cost since the entire compute graph needs to be stored and then differentiated through during the backward pass. We instead adopt the approach from Saravanos al. (2025, Theorem 2) using differentiable optimization in order to compute these derivatives in more efficient manner. In practice, this means we can compute the derivatives by solving new linear system with the same coefficient matrix but different right-hand side during the backward pass.
D FLEXQP ALGORITHM
Algorithm 1: FlexQP
Input: Initialization æ0, 5D 20 00 24 EE w 1/: v2 and parameters μr, ME, PI, >
Output: Solution x². ".., VE
while termination criterion not satisfied do
ak+1 , =k+1
Solve the linear system Eq. (23)
gb+1 = = o- w--wk 0-100 1
2 2 Pi y + P1 skt1
PE yx + Pæ k+1
z E = ZE +1 1 + (1 (1-aar a
x ++1 ai
s A+1 (ash+1 +
a)s**
+ + (wck
*+1 Spr/or (azt+1
(1 - + Pi 111
E = Speips (a
k4
+ (1 = + PE v
wh
++1 = w + +0(as ++ +(1 a)s* - 8 +1)
= v p1(az)t1
skl
+11a))/ 21 k+1)
V ye + p
+ a) 2 k+1)
end
23

---

Published as a conference paper at ICLR 2026
E PROOF OF THEOREM 3.2
To show that saddle point of Eq. (10) exists, it suffices to show that the relative interior of the domains of f and are non-empty. This form of constraint qualification guaranteeing strong duality (Rockafellar, 1970, Theorem 16.4). The relative interior of f is simply the feasible set {(x, 8,,
21 Gx + 8 ht FF = Ar - b). which is non-empty (pick any x, 8 and then let 21 = Gr + h and zg = Ax - b). Meanwhile, the relative interior of g is given by the set {(z, 21, 2g) 8 > 0} due to the constraint s 2 O. Combining, this shows that relint relint(dom f) n relint(dom) g) * 0, strong duality holds and saddle point exists by Rockafellar (1970, Theorem 37.3).
Furthermore, the objective and g closed, proper, and convex, by Boyd al. (2011, $3.2), Algorithm converges. Namely, have that the ADMM primal residuals Ç - and ADMM dual residuals Ç* - 0 as k + 00, where
kk
ik
x
x
2
ck
g
g
*h-1 8
5*
5
(27)
k-1
c
FE
Furthermore, we also have that iterates ã. 2. xx - C. and 4r ÿ as k:
F PROOF OF THEOREM 3.3
The proof follows from the definition of the soft thresholding operator. First, we consider the 2, and 2 updates well the dual variable updates for yı and ye from Eq. (9). Assume w.Lo.g. that
a = L. These updates have the general form:
A+1 = Sp/p (g + /p)
(28a)
M+ =y² -( 2.1
(28b)
Now, there are three cases the consider based on the output of the soft thresholding operation:
1. Positive constraint violation: If gk+1 + y²x /p u/p, then 2++1 =b+1 + 15* /p p/p- Substituting into Eq. (28b) yields k+1 μ..
2. No constraint violation: If | + 8t /pl S p/p, then 2++1 = 0. This further implies p| + ////pl & μ and by Eq. (28b) this implies + S H.
3. Negative constraint violation: gk+1 + y² / -u/p, then gk+1 gk+1 + / + p/p- Substituting into Eq. (28b) yields *+1 -p.
Combining these three cases, we have that + < μ. for any zk+1 y and therefore |0 < μ. This proves the first and last statement of the theorem. Applying Theorem 3.1 shows the second statement.
G DEEP FLEXQP POLICY PARAMETERIZATION
The residuals for Eq. (7) are given by
S = p + +
GT y y
(29a)
c Gx k + s -h -
(29b)
tt As. b b-=2 =
(29c)
The ADMM residuals for Eq. (8) defined in Eq. (27). We ignore the residuals corresponding since it is unconstrained the second ADMM block (so the primal residual not very meaningful) and we have already captured the optimality through Eq. (29).
24

---

Published as a conference paper at ICLR 2026
The policy IR R2. is given by
M1,0s: Ti (s, 21, Wa 1i S2 So, Er, E., {1):
(30)
where we have dropped the indices by constraint and iteration &o for clarity. The policy mg IR R is given by
PE PE TE(E VEs |[Suual loc SE. SE. ŞE).
(31)
The policy Ta R29 (0,2) is given by
a Ta Ta [dual IIsrl II IIS I1 II II IIC=
(32)
where the norm used is the infinity norm.
We consider two policy parameterizzaions:: pendix N for experimental comparison). Following Saravanos et al. (2025). MLP policies are small networks with two hidden layers of sizes [32, 32j. LSTM policies by an MLP with hidden layers [32, 32] for prediction. We use sigmoid for all activation functions, which find are much more stable than ReLU activations, most likely due the autoregressive nature of deep unfolding. Computationally, we log-scale any small positive inputs like the infinity norms of the residuals. Following Ichnowski al. (2021), we also predict log-trace tnggtransformed values log fr,, log PE. etc. and then apply an exponential function so that it is casier to predict parameters across a wide scale of values. We then clamp the parameters (besides a) to the range [10-9, 10%); (0, 2) is enforced using a scaled sigmoid function.
multilayer perceptrons (MLPs) and LSTMs (sce Ap-
hidden size of 32 followed
H FURTHER DETAILS ON QP PROBLEM CLASSES
summary of the problem sizes and training parameters of the different classes is presented in Table 1. For the small- to medium-scale QPs, we train all models for 500 epochs and evaluate using 1000 test samples. More training samples are used for random QPs following the setup by Saravanos et al. (2025) since the shared structure between these QPs is less clear and therefore harder to learn. As described the main text, for the large-scale problems, we fine-tune the models trained on the smaller scale problems on 100 large-scale problems for 5 epochs, and test on 100 new problems.
Table 1: QP problem sizes and number of samples used for training.
Problem Class
p
Training Samples
Random QPs
50
40
0
2000
Random QPs with Equalities
50
25
20
2000
Portfolio Optimization
275
250
26
500
Support Vector Machine
210
400
0
500
LASSO
510
10
500
500
Huber Fitting
310
200
100
500
Random Linear OCPs
128
256
88
500
Double Integrator
62
124
42
500
Oscillating Masses
162
324
132
500
Portfolio Optimization (Large-Scale)
10100
10000
101
100
Support Vector Machine (Large-Scale)
10100
20000
0
100
Car with Obstacles (SQP)
253
455
153
500
Quadrotor (SQP)
812
400
612
500
Car Safety Filter (SQP)
253
50
153
500
H.1
RANDOM QPs
The first type of problems study are random QPs of the form Eq. (3). These are helpful can freely adjust the number of constraints as well as the sparsity of the problem directly in order to benchmark the optimizers under different operating conditions.
25

---

Published as a conference paper at ICLR 2026
Problem Instances: We adopt the problem generation procedure from Saravanos al. (2025), where P = M M + al with a 1 and all elements of M, 9, G, and A are standard normal distributed, i..., each element Myj. 9: Gaj. Ai ~ (0, 1). The vectors h and b are generated using = Gg and Aç with 5 standard normal vectors.
We consider two classes of random QPs. The first class, Random QPs, contains only inequality constraints generated by setting the problem dimensions as n = 50, m = 40, and = 0. The second class, Random QPs with Equalities contains mix of inequality and equality constraints, and is generated using n = 50, m 25, and 20.
H.2 PORTFOLIO OPTIMIZATION
Portfolio Optimization is a foundational problem in finance where the goal to maximize the risk-adjusted return of group of assets (Markowitz, 1952; Boyd et al., 2013; 2017). This can be represented as the following QP (Boyd Vandenberghe, 2004; Stellato al., 2020);
max 1 T x - 7(0 Ex),
(33a)
subject to _ 1,
(33b)
0,
(33c)
where z E R" is the portfolio, μ E R" is the expected returns, 7 0 the risk aversion parameter, and Σ E ST. is the risk model covariance.
QP Representation: We assume that = FFT + D where E RR is the rank-k factor loading matrix with & < and D "x* is the diagonal matrix specifying the asset-specific risk. Using this assumption, the optimization problem can be converted into more efficient QP representation:
min a.y
x T Dr + y 2²
(34a)
subject to
y = FTT.
(34b)
1 x =1
(34c)
x 2
(34d)
This QP has + k decision variables, l + equality constraints, and inequality constraints.
Problem Instances: For the medium-scale problems, we use the problem generation described in 250, f 25, and generated using n = 10000 assets, kt = 100 factors, and 7 = 1.0. The expected returns are sampled using μ4: (0, 1). The factor loading matrix F' has 50% non-zero elements sampled
Saravanos et al. (2025), setting
1.0. The large-scale problems
through F.g ~ (0, 1). The diagonal elements of D are generated uniformly as Dü ~ U(0, vk).
H.3 SUPPORT VECTOR MACHINES
Support Vector Machines (SVMs) classical machine learning method where the goal is to find a linear classifier that best separates two sets of points (Cortes & Vapnik, 1995):
min + Σ max bia + 1),
(35)
where À > b, E 1-1,1} is the label, and E R is the set of features for point i.
QP representation: The SVM problem Eq. (35) can be converted into an equivalent QP represen- tation (Stellato et al., 2020):
min 2,1
x IT + A1
(36a)
subject to
t diag Ab +
(36b)
t t
(36c)
This QP has + decision variables and 2m inequality constraints.
26

---

Published as a conference paper at ICLR 2026
Problem Instances: We generate medium-scale problems using the rules from Stellato et al. (2020) with 7t = 10 features, m = 200 data points, and À 1. Large-scale problems are generated using 100 features, 10000 data points, and À 1. The labels are chosen using
i frimm/ m/2,
(37)
-1 otherwise,
and the elements of are chosen such that
[N(+1/n, 1/n) Aij (N(-1/n,1/)
i m/2, otherwise.
(38)
H.4 LASSO
LASSO (least absolute shrinkage and selection operator) is a fundamental problem in statistics and machine learning (Tibshirani, 1996; Candes et al., 2008), The objective is to select sparse coefficients linear model that best match the given observations:
min || Ax b12 + > 1|
(39)
where E R", A E Rmxx is the data matrix, R are the observations, and À 0 is the weighting parameter.
QP Representation: LASSO can be represented as QP by introducing extra decision variables E R" and / E IR" which help simplify the objective (Stellato et al., 2020):
min y y A1 t,
(40a)
subject to
y Ax b, - + S t.
(40b)
(40c)
Problem Instances: We use the data generation procedure from (Stellato et al., 2020), where A has 15% non-zero normally-distribueed elements Aij N(0 1) and b is generated through b = Au + with
So
with probability 0.5,
(41)
1//))
otherwise,
and E, NOO, 1). The parameter À is chosen as x (1/5)][AA blloo.
T
H.5 HUBER FITTING
Huber Fitting is robust least squares problem where the goal to perform a linear regression with the assumption that outliers are present in the data (Huber, 1964; 1981);
min
Prab (a, x - -b),
(42)
where the penalty function chub penalizes the residuals linearly when they are large and quadratically when they small:
2² (6(2 5) if u 8,
if Ju| < 8,
Ónub
(43)
with 8 representing the slope of the linear term.
QP Representation: This robust least squares problem can be represented in the following QP form (Stellato al., 2020):
min s,u.r. subject to
u Ju + 281 (r + s), Ax - = 8,
(44a)
(44b)
r, 0.
(44c)
27

---

Published as a conference paper at ICLR 2026
This QP has + 3m decision variables, 2m inequalities, and equalities.
Problem Instances: We follow Stellato et al. (2020) and generate A with 15% nonzero elements with Ag ~ (0, 1) and set b Av where
(O 1/4) =
with probability p = 0.95,
(45)
u(0, 10) otherwise.
We let 5 = and choose the problem dimensions as n = 10 features and m 10n = 100 datapoints.
H.6 LINEAR OPTIMAL CONTROI.
The goal in linear optimal control is to stabilize the system to the origin subject to dynamical con- straints as well as polyhedral constraints on the states and controls. This results in QPs of the form
min 2,8
Σ a Qu, + us Rui + QTQT,
(46a)
subject
Ft+1 Asti Baus
(46b)
A, S bui
(46c)
A S bx,
(46d)
30 To.
(46e)
the time horizon, Q E S is the running state cost matrix, S the control cost matrix, Qr E $ the terminal state cost matrix, Ad E xn= and BN R==x **a define the dynamics of the system, A, R xny and by Rm define the input constraints. Ag E Rm X and bg E Th define the state constraints, and To E 1= is the initial condition.
where T >
We study three classes of linear optimal control problems (OCPs). The first, Random Linear OCPs, consists of randomly generated stabilizable dynamics along with random costs, constraints, and initial conditions. The second and third classes, Double Integrator and Oscillating Masses, are adapted from Chen et al. (2022a) and contain dynamics with true physical interpretations. The randomness in these problems given by sampling varying initial conditions for the systems as in Saravanos et al. (2025).
H.6.1 RANDOM LINEAR OCPs
We use the problem generation procedure similar to that in Stellato et al. (2020). We set the state dimension n2 8 and n, = n2/2 4. The dynamics are generated by Aa X-1 X-IAX, where A diag(a) ()) E Rn= such that ~ 1)-1,,1 and X E R== with elements generated by Xij A (0, 1), and Ba E X with (Ba)ıj NO0 1).
The running state cost Q E S is generated by Q diag(g) where each element of the sparse vector q is generated by
[u(0, 10) with probability p = 0.7,
(47)
otherwise,
so that y has 70% nonzero values. We fix the control cost R = 0.1 and the terminal cost Qr determined by solving the discrete algebraic Riccati for the optimal cost of a linear quadratic regulator applied to A, B, Q, and R. The state and control constraints generated by
4,
1
-gg
bz
2: bound
hound
where J.
bound 11(122)
(48a)
A,
Lu
by
2 bound
where bound ~ (.0,
(48b)
-u
- 2 bound
Note that we use I, and I, as a shorthand for Inj and In.. Finally, we sample the initial state from To ~ 14(-0.55z 00.55. boudd.
28

---

Published as a conference paper at ICLR 2026
H.6.2 DOUBLE INTEGRATOR
For the double integrator, adapted from Chen et al. (2022a), have n2 2, n4 and T timesteps. The dynamics are fixed with
20
0.5
Aa =
Ba =
(49)
0.1
We use cost matrices Q = QT Ly and
1.0. The state and control constraints are given by
I
1
0.1
A =
1
bx
Au =
-1
bu
0.1
(50)
(1
The initial state is sampled from Z0 ~ll
-
0.3
0.3
H.6.3 OSCILLATING MASSES
For the oscillating masses problem, have n, 12. n. T 10. For this problem, the discrete-time dynamics matrices Aa and Ba are obtained through the Euler discretization of the continuous-time dynamics of the oscillating masses system, namely
A La + A.A, Ba B.A,,
(51)
where At = 0.5. The matrices Ao E XMa and B. E I xm.. define the continuous-time dynamics and are given by
ale + cLLo + Lb6) bls + d(Lo + 4b) B. 06x3
06x6
A.
(52)
where e 1, d 0.1, -2c, 2, Omxn is the zero matrix in mR T.n the lower shift
matrix in R=x" and F = [ei -1 e2 basis vectors in R. We use cost matrices constraints are given by
63 -62 eal where 61. and ey are the standard Q = Qr = La and R Lu. The state and control
1g 1
Lu
A₂
by 4.
1,
A
by 0.5 1u-
(53)
Tu
Finally, we sample the initial state from Ty 111111111 1z).
29

---

Published as a conference paper at ICLR 2026
1 NONCONVEX NONLINEAR PROGRAMMING USING SQP
1.1 NONLINEAR OPTIMAL CONTROL
We consider nonlinear constrained optimal control problems the following form:
T-1
minimize
Σ
( un) + #(xT).
(54a)
=
subject to
1t+1 F(rr; ut),
-,-0... T -1 1,
(54b)
To 30;
(54c)
h( < 0,
VI = . T
(54d)
g(u)) 0, VI 0, ..... 1,
(54e)
where E I and Ue R the states and controls, respectively. The function f R I is the running cost and @ R -> the terminal cost. The time horizon is > 0 and Tp E IR" the initial condition. The problem formulation in Eq. (54) includes state and control constraints represented by the functions h(zz)) and g(4).. In the next two subsections, we provide the specific nonlinear optimal control examples for the cases of the Dubins vehicle and quadrotor.
1.1.1 DUBINS VEHICLE
The Dubins vehicle a dynamics model with a state x = (Px, Py. 0) TR33, where Ps. and Py are the vehicle's position in the Cartesian plane and A is its orientation. We use the unicycle formulation of the continuous-time Dubins vehicle dynamics ă f(z, u) adapted from Siciliano et al. (2009), where the control is given by (c, E R²a Here, v is the forward velocity of the vehicle and w the steering velocity of the vehicle.
Taaeet
Obstaclo
Figure 8: Visualization of sample Dubins vehicle task. The goal is reach the target state while avoiding obstacles and respecting the dynamics and input constraints.
We formulate a nonlinear optimal control problem following Eq. (54). We discretize the continuous-
time dynamics using the Euler discretization Tr+11 F( Ux) I, f (I,. u, At and use quadratic costs f(x, u) x Qx + u. Ru and Φ(x) = x QTæ, where Q = diag (1.0, 1.0, 0.1), R 0.1 and Qr 100 Q. The initial and target state, 25 and Fiarget, are sampled uniformly from 4(--;, the time horizon for trajectory optimization is T 50 timesteps. We generate 5 circular obstacles with the form
-., F). where ह = (5.0, 5.0, TT. The discretization of the dynamics uses At = 0.033 and
gx( = 12, - c < O, Vt = =......,
(55)
where the centers C; R are sampled uniformly at random in the region between the vehicle's initial and target positions, and the radii r; > 0 are sampled uniformly from ( Tmax) with
Tmin 0.01 Iaarre zoll, and Fmax 0.2 Farget xo The controls are constrained by
30

---

Published as a conference paper at ICLR 2026
[-10, 10] and w E [-5,5]. This leads nonlinear optimization problem with 253 variables, 455 inequality constraints, and 153 equality constraints.
For generating the QP training data, we generate 500 QP subproblems by solving randomly gener- ated Dubins vehicle problems with SQP using OSQP as the QP solver. For evaluation, we generate 100 random control problems and solve them using SQP with OSQP SQP with Deep FlexQP. Each algorithm allowed 50 SQP iterations and runs until the infinity norm of the SQP residuals falls below an absolute tolerance of 10-2, Furthermore, each QP solver runs until convergence of 10-3 is reached, with a max budget of 10 seconds and an unlimited number of iterations.
1.1.2 QUADROTOR
We use the continuous- quadrotor dynamics model = f(x, u) from Sabatino (2015), The model consists of the state E IR22 that includes the linear positions, angles, linear velocities, and angular velocities. The system is actuated by four controls, the collective thrust and three torques, given by (F, Twx Ty. Tx) E R+.
y
Ieiid
Figure 9: Visualization of a sample quadrotor task. The goal is to reach the target state from the initial state subject to dynamical constraints and input constraints.
Using this model, we formulate nonlinear optimal control problems in Eq. (54). We discretize the dynamics through the Fuler discretization I++ F(ar, Uz) 2t + fxx, ux) At. The cost in Eq. (54) is defined by the quadratic cost f(x, u) = x Qu Ru and d(x) Qræ, where the cost matrices are given by
Q diag 1.0, 1.0, 1.0, 0.1, 0.1, 0.1, 10,1010110001
along with R 0.01 / and QT 1000 Q. The initial and target state are sampled uniformly from (4(-E,ā .) where I = 5,5,, 1, 1, ग, n 22, n, 1,), The discretization of the dynamics uses At = 0.05 and the time horizon for trajectory optimization is T 50 timesteps. The controls are constrained in the ranges F E [0, 20) and Ta, Ty: T. E 1-0, 10). We use m = 1.0 kg for the mass of the quadrotor, due to gravity. This leads to nonlinear optimization problem with 812 variables, 400 inequality constraints, and 612 equality constraints.
I = ly 1.0 for its moments, and g 9.81 for the acceleration
For generating the QP training data, we generate 500 QP subproblems by solving randomly gen- crated quadrotor problems with SQP using OSQP as the QP solver. For evaluation, we generate 100 random quadrotor problems and solve them using SQP with OSQP or SQP with Deep Fle Similar to the Dubins vehicle, each algorithm is allowed 50 SQP iterations and success in Fig. (left) is achieved when the infinity norm of the SQP residuals falls below the absolute convergence tolerance E = 10 Each QP solver runs until convergence of 10 -3 is reached, with a max budget of 10 seconds and unlimited number of iterations.
31

---

Published as a conference paper at ICLR 2026
1.2 NONLINEAR PREDICTIVE SAFETY FILTERS
Finally, we apply our proposed approach to accelerate a predictive safety filter for nonlinear model predictive control. These methods are based on control barrier functions (CBFs) (Ames et al., 2019) and filter reference control nd so that it better respects safety constraints (Wabersich & Zeilinger, 2021). The following optimization is solved at every MPC step:
minimize 2,0
Σ t=0
well 11 Ue - un
(56a)
subject
I++1 F ue);
Vt = O.....T
(56b)
Io 30 (1 3)((2) - h(1(41) 20, =
(56c)
(56d)
g S 0, 0......,
(56e)
where B E (0,1) is a parameter controlling the strength of the CBF constraint. This differs from Eq. (54) because the discrete CBF constraint is defined between two consecutive states 2i and 1 rather than assuming the state constraints are separable across time. Our method improves upon the SPield-MPPI method proposed by Yin al. (2023) because our optimization explicitly incorporates the dynamics and input constraints while also minimizing the discrepancy from the reference control trajectory. Furthermore, using our accelerated Deep FlexQP ensures that the optimization can be run fast enough for real-time control. We use version of Deep FlexQP with performance guarantees from minimizing the generalization bound loss (see Appendix M).
1.2.1 SHIELD-MER MMPI
The method by Yin et al. (2023) approximately solves nonlinear optimization every MPC step to generate safe controls given a trajectory from a high-level planner such as a model predictive path integral (MPPI) controller. While the main motivation behind this approach is that it is computa- tionally fast, unfortunately, there are a few flaws with the method because it has no real guarantees of safety and the MPPI trajectory is only used to warm-start this second optimization. The main bottleneck preventing us from solving a more complex optimization in real-time is the solver speed. Therefore, this is an application where accelerating optimizers using deep unfolding can shine.
1.2.2 RANDOMIZED PROBLEM SCENARIOS
We use the same Dubins vehicle model as in Appendix I.1.1. 100 random scenarios are gener- ated by first sampling random initial and target state uniformly from (((5 -5, -T), ) (5, 5, T)). obstacle randomly sampled so its position falls between the initial and target state with random radius r depending on the distance between the initial and target state: ~ 2/(0.01,2) max([p larget
p The controls are constrained in the ranges v E [-10, 10] and E (-5, 5]. enforced by clamping for Shield-MPPI and through the constraints of Eq. (56) for our SQP-based method. The reference trajectory at every MPC step given by running an MPPI controller that samples 10000 trajectories with look-ahead horizon of 50 timesteps; with dynam- ics discretization of At = 0.05, this corresponds to a planning horizon of 2.5 seconds ahead. The system experiences nero-mean Gaussian disturbances in its state at every MPC step with standard deviation (0.05, 0.05, 0.01). Shield-MPPI is allowed to run up to 5 Newton iterations per MPC step, while our SQP safety filter is allowed to run up 5 SQP iterations per MPC step. These thresholds were determined by estimating the max number of iterations that would still allow for real-time control of the system. Collisions in Fig. 7 (right)
prlll [p lurget
counted if the state violates the CBF constraint (i.c., intersects the obstacle). Successes are counted if the vehicle reaches within 0.1 radius of the target state. Fig. 10 shows the problem setup and sample trajectories of our approach compared with Shield-MPPI.
32

---

Published as a conference paper at ICLR 2026
Initial
Tirget Obstacle
1-
-2
-2-
-2-
4
2
-6
-2
(a) Shield-MPP1
(b) SQP Safety Filter Deep FlexQP
Figure 10: Sample trajectories comparing safety filter approaches on a nonlinear car system. The vehicle receives disturbances in the positions and orientation at every step. Our approach more effec- tively recovers from unsafe scenarios by better accounting for dynamic feasibility and constraints.
33

---

Published as a conference paper at ICLR 2026
J PERFORMANCE COMPARISONS SMALL- TO MEDIUM-SCALE QPS
The vanilla and learned optimizers from Section 5.1 are benchmarked on 1000 test QPs from each problem class with the results summarized in Fig. 11. The best version of OSQP found using hyperparameter search over the following configurations: fixed parameters for all iterations, adaptive penalty parameters using the OSQP rule, or adaptive penalty parameters using the ADMM rule. Similarly, the best version of FlexQP is found using hyperparameter search over the following configurations: fixed parameters for all iterations, adaptive penalty parameters using an OSQP-like rule, adaptive penalty parameters using the ADMM rule. Problems considered solved when the infinity norm the QP residuals reaches below an absolute tolerance of & 10-3 Optimizers are run with no limit on the number of iterations until a timeout of second (1000 ms) reached. Timings are compared using the normalized shifted geometric mean, which is the factor at which specific solver is slower than the fastest number of iterations required to converge as well as the number of coefficient matrix factorizations required to converge to get sense of where the optimizers are spending the most time. All methods QPs) at
(Mittelmann, 2010). We also compare the average
use the direct method to solve their respective linear systems (i.e., equality-constraing every iteration, and the factorization from the previous iteration is reused if the parameters have not changed by more than factor of 5x following the heuristic used by OSQP (Stellato et al., 2020).
Key Takeaways:
1. Deep FlexQP and Deep OSQP - Improved solve QPs 2-5x faster than OSQP.
2. Deep FlexQP and Deep OSQP Improved require upwards of 10x less iterations to con- verge than OSQP and require comparable amount of matrix factorizations.
3. Deep OSQP RLQP Parameterization struggles problems with optimal control struc- ture. This not observed with Deep OSQP Improved. Learning the ADMM relaxation parameter a seems be erucial for these problems.
It is important note that these results hold only when the direct method is used solve the linear system. When using an indirect method such as the conjugate gradient (CG) method, converging in fewer iterations is actually major benefit as each iteration across all the optimizers have roughly the same computational complexity (see Appendix C for discussion and Appendix K. for results with the indirect method).
Sucudaad
Prebiems Salved (1K)
Numbee Comerge
Fictriaation Cuunaage
24
100
(0)
Ino 100
100
11.9
32.
398
D.
2-
20
21
Random QPs
Nomashost Slufed Germetns Meen
Probdeme Sobodd (X)
Number Toratime Comergge
Number of Factontattons Comengg
75.0
85.5
83
65
4%
K5.2
11.
73
63
(b) Random QPs with Equalities
Nermolaae
Scloed (50)
Number
Commenpe
Number Convegg
25
100
100
100
Im
100
100
16
3.1
(c) Portfolio Optimization
Figure 11: Performance comparison of vanilla vs. learned optimizers. Legend: OSQP, FlexQP (Ours), Deep OSQP, Deep OSQP RLQP Parameterization, Deep OSQP Improved, Deep FlexQP (Ours).
34

---

Published as a conference paper at ICLR 2026
Secialadd Stiited Counetile Mosn
Probkans Salvel (S]
Nimbee
Number Fuctoraation Clussegee
120
100
100
300
7.
37
L3
(d) Support Vector Machines
Necmalaae Stithed Coonetile Meen
Prohicuu Sklved (93)
Nentle Iecatione to Conregge
Number of Fostoraution Clasoage
6:3
100
1001
300-
122.
1
L0
22.4
74
(e) LASSO
Nermaload Shifeed Geomeeri Mian
Prefiem Scheol (5)
Nember of locatime Comeryy
Number of Fistornntione Cunverg
10.7
20
13
10
15.4
77
21
(f) Huber Fitting
Stafhel Gemettii
Proldan Soload
Thratiim Commny
Number Facturiaation Cemmyy
316
100
-310
10
25
NIA
15.1
N/A
(g) Random Linear OCPs
Yomudias! Slifbel Gemetri: Mren
Prollama Selvad (X))
Number Torakime Comenny
Number of Exctrication Cemnenyy
50001
IBLL
10
31C
10.8
21.1
1.4
(h) Double Integrator
Normaliad
Problrns
Comerge
103
100
100
300
100
100
15
I0.3
25
10
16
15
32
(i) Oscillating Masses
Figure 11: Performance comparison of vanilla vs. learned optimizers. Legend: OSQP, FlexQP (Ours), Deep OSQP, Deep OSQP RLQP Parameterization, Deep OSQP Improved, Deep FlexQP (Ours).
K PERFORMANCE COMPARISONS LARGE-SCALE QPs
In this section, we report the full details of the large-scale QP experiments introduced in Section 5.2. Due to memory limitations, each optimizer is unfolded for 10 iterations and trained with batch size of Optimizers are trained for epochs on 100 training problems and evaluated on 100 new test problems, with results presented in Fig. 12. Convergence is declared when the infinity norm of the QP residuals falls below an absolute tolerance & 10-3 Each optimizer is run until convergence or until timeout of 10 minutes is reached. Similar to Appendix J, report the normalized shifted geometric mean to better highlight the relative performance each optimizer (wall clock times are provided in Fig. 6). We also report the percent of problems solved, the average number of iterations taken to converge, and the average number of CG iterations necessary to solve the linear system at each ADMM iteration. For the optimizers that failed to converge any problems, report the
35

---

Published as a conference paper at ICLR 2026
average final QP residual infinity norms and average number of iterations in Tables and 3. This gives a rough idea of how far away the optimizers were from converging when they were timed out.
Bxsetion Cocavegg
353
20.3
363
353
110
200
172.7
5310
1000
12-
1.0
5/A
N/A
(a) Large-Scale Portfolio Optimization
Goometrie Moan
Salvod
Benutions Cheasegge
Avrnog
150
361
16:
14.1
14:1
110
100
175.0
1667
125
180
1G.4
75-
36
1.0
X/A M/A
M/A
12:4
31.2
10.3
100
(b) Large-Scale Support Vector Machines
Figure 12: Performance comparison of vamilla vs. learned optimizers. Legend: OSQP, FlexQP (Ours), Deep OSQP, Deep OSQP - RLQP Parameterization, Deep OSQP Improved, Deep FlexQP (Ours).
Optimizer OSQP FlexQP (Ours)
Final QP Residual Infinity Norm
Iterations Run
2.82c-3
325.5 326.3
1.45c-3
Table 2: Large-scale portfolio optimization failed optimizer statistics.
Optimizer
Final QP Residual Infinity Norm
Iterations Run
OSQP
2.10c-2
178.9
FlexQP (Ours)
1.54c-3
326.3
Deep OSQP Improved
8.04e-2
175.2
Table 3: Large-scale support vector machine failed optimizer statistics.
From these results, observe that the traditional optimizers (OSQP and FlexQP), failed con- verge on any of the problems within the allotted time. Furthermore, while the fine-tuning procedure worked for all learned optimizers on the portfolio optimization problems, it appears to have failed for Deep OSQP and Deep OSQP Improved on the SVM problems.
Interestingly, the learned optimizers with the data-driven rules for the penalty parameters required many more CG iterations to solve their respective linear systems compared to the traditional opti- mizers. There seems to tradeoff between number ADMM iterations required to converge vs. the condition number of the coefficient matrices generated at every ADMM iteration. For example, even though Deep FlexQP converges in relatively few iterations, each iteration requires an order of magnitude more CG iterations compared to OSQP to solve the linear systems to similar precision. Using preconditioning with the learned optimizers would likely lead to even greater performance gains over the traditional optimizers.
36

---

Published as a conference paper at ICLR 2026
L SAMPLE PARAMETER PREDICTION PLOTS
10"
Relucation Paremettt
1.8
Residual Norm
10
Usqqe
1.8
Fleetg (Ours)
10
144
Dap OSGP
RUGP Punm
Deep USU Irgneed
Deep Flaque (Ouna)
10
15
10 Iteaation
20
Iterations
Peaalty Parmmeze
10a
Peanlly Pacumeter
10"
10"
10²
10'
10
20
Iteration
Beration
(a) Portfolio Optimization
109
Reloxstion Pacumeeer
Reciial Norm
10
1.4
1:0
10
15
20
10 lteradion
20
Iteration
Prnaly Paramater de
Pondly Picamoter
109
131-
10'
10
16
15
20
Iteration
Iteration
(b) Huber Fitting
Relaxationn Parumeter
Roaidudl Norma
109
1.0
0.8
10
15
15
20
Iteration
Iteration
10
Perulty Paramarar 06
Pesalty Paramntat
10k
13
107
106-
10 Ieration
70
Ieralion
Linear OCPs
Figure 13: Representative parameter predictions for a few problem instances.
In this section, compare the a, Pı, and PE predictions across the traditional and learned opti-
mizers on a few representative optimization problems from Section 5.1. The parameters from the first 20 iterations of the different optimizers shown in Fig. 13. Since Deep OSQP RL.Q
37

---

Published as a conference paper at ICLR 2026
Parameterization, Deep OSQP Improved, and Deep FlexQP output vectors of penalty parameters P1 and PE. we plot the mean prediction. Note that OSQP, FlexQP, and Deep OSQP RLQP Pa- rameterization a fixed of 1.6, 1.6, and 1.0, respectively. We also report the infinity norm of the QP residuals to get a better idea of when during the optimization a prediction is being made.
We observe that the learned rules seem quickly adjust the parameters during the beginning of the optimization compared to the heuristic rules (OSQP and FlexQP), which seem to gradually adjust the parameters later in the optimization. Adjusting the parameters early might provide beneficial effect on the convergence by allowing the optimizer to adjust to better initial condition based on the first few evaluations of the problem residuals.
M SUPPLEMENTARY RESULTS FOR GENERALIZATION BOUNDS
1.
-SQP
Deep FlesQP (Oums
Tolativ Opsimiitt Gap Loes
10
Fleqpp (Ous)
Deep FlexQP Gen. Bound
Eq. (14) (Ours)
Doup USQP
OSQF BLOP
In-2-
Deep D5GP Impoored
Deep FlexQP (Ours)
Deep FleqQP Gon. Bound Eq. (13)
0.4
10
0.2
10-1
0.0
20
10 Iteration
Dieration
Figure 14: Generalization bounds for Deep FlexQP trained on 500 oscillating masses QPs. Follow- ing Fig. 4, this is another case where the generalization bound using the loss Eq. (13) is uninforma- tive.
We provide an overview of the generalization bounds training procedure described by Saravanos et al. (2025), which in turn is adapted from the one described by Majumdar et al. (2021). Let 'D denote a single sample from data distribution D and let S {s: be dataset of N samples. Let O) [0, 1] be bounded loss for hypothesis
(P.g. G, h, A, b, ,r",, y7. vfa)
0 ~ P. The true expected loss is defined as
(D(P) = E, EEWDEE (((s,))
(57)
and the sample loss is
(s(P) Eg-p
((s 0)
(58)
We rely on the following PAC-Bayes bounds that hold with probability 1- (Majumdar et al., 2021, Corollary 1):
(D(P) < -D1 [s(P)) DxL (PI(PP) + log(2vNN/0)
<es(P)+ DKI(PP(PP) 1og(2V/N/))
2N
(59) where (pl(e) suply E [0, [[,]]DKK (B()) S o} the inverse KI--divergence for Bernoulii random variables B(p) and B(q). The first bound is tighter and is therefore useful for computing the generalization bounds as numerical certificate performance, while the second bound has the nice interpretation of training loss plus regularization term that depends the size of the training set and penalizes being off from the prior Po. During training, we minimize the second bound using either the loss Eq. (13) or Eq. (14).
During the evaluation of the tighter generalization bound (after training is complete), since it is difficult to compute the expectation over 0 P in Eq. (58), instead estimate the sample loss
38

---

Published as a conference paper at ICLR 2026
using a large number of samples [0 from P":
M Σ 1=1
(s(P") NM
E(81, 0z).
(60)
The following sample convergence bound holds with probability 1 8':
(s(P)) <B- (is(P"))
log( 2 87)
(61)
Combining these bounds results in final version of the PAC-Bayes bound that holds with probabil- ity 8' (Majumdar et al., 2021):
( D-1 [s(P))) DKL PP) + 1og(2V/N0))
(62)
N
This is the final bound that we report in our experiments.
The prior Po for all our models is stochastic Deep FlexQP trained for 500 epochs on 500 QPs generated from the Random QPs with Equalities problem class using the supervised learning setup from Section 4. We train Deep FlexQP for the generalization bound loss using either Eq. (13) or Eq. (14) with 0.009. We fix training set for the generalization bounds using 500 problems from the class of interest and train the model for 1000 epochs. We evaluate Eq. (62) using M = 10000 model samples and 8' = 0.001. Our bounds therefore hold with 99% probability. We report results comparing Eq. (13) vs. Eq. (14) for LASSO in Fig. 4 and for Oscillating Masses in Fig. 14. The test loss computed over 1000 new samples from the target problem class. Overall, the loss Eq. (13) results in less informative bound as it is above all the optimizers, even though the performance in practice can be much better.
Finally, we train Deep FlexQP 500 QP subproblems generated from nonlinear predictive safety filter task described in Appendix using the same procedure to minimize the generalization bound through the loss defined in Eq. (14). We this Deep FlexQP as the QP solver for the predictive safety filter SQP approach described in Appendix I.2. This provides a numerical certificate on the performance that would not hold for traditional optimizer.
CSQP
Fleq@P (Ous)
0.8
Deep DSQP
DSQP BLOP
Deep D5CP Ingooed
0.0
Deep FlexQP (Ours)
Deep FlexQP Gen, Bound Eq. (14) (Owrs)
0.4
0.0
10
21
Iteracion
Figure 15: Generalization bound for Deep FlexQP trained on 500 QP subproblems generated from the nonlinear predictive safety filter task.
39

---

Published as a conference paper at ICLR 2026
N LSTM vs. MLP POLICY COMPARISON
This section presents an ablation analysis on the use of LSTMs to parameterize the policies in both Deep OSQP and Deep FlexQP. The use of LSTMs further leverages the analogy between deep- unfolded optimizers and RNNs, as discussed in Monga et al. (2021). Our hypothesis is that the RNN hidden state can encode compressed history or context from the past optimization variables and residuals, thereby leading to a better prediction of the algorithm parameters to apply at the current iteration. Using an MLP only provides access to information from the current iterate, which could lead to myopic choice of parameters.
Our results show that LSTMs enhance performance on several problem classes (Fig. 16). LSTMs appear to help the most on problems where the active constraints might switch suddenly from one iteration to the next. These types of problems include the machine learning ones, such as SVM, LASSO, and Huber fitting, as well as some of the control problems, like the oscillating masses.
Randem QPs
Random QPs with Equalitios
Deep DSQP
Impreved, MLP
10a-
10
Daap Osqge
Improssd, L8TM
Deep FlexQP
(Ouns] MLP
10-2
Optimalit Gep
Deep Flex(u (Ours) LSTM
Oplinality Dap
10
10
10
10
10
10-4
Ineratio
Iseration
Portialio Optinisation
Support Vodto Muchloos
10
10"
10
10
Oylimallty Gop
Optimality Gop
10
10
10
10 9.
Beration
Homaiio
LASSO
Hlubr Pittin
108
105
10
10 2
Oplimality Gap
Oplimality Gap
10
10 4
10-1
10
10
10-
Beration
Bonatio
Figure 16: Comparison of MLP vs. LSTM policies. Performance is compared over 1000 test problems.
40

---

Published as a conference paper at ICLR 2026
Randon Lnear OCPs
Double Integrator
10-1
10
Optinaalty Dap
10
10-4
Optinality
10-4
10-7
Beration
Eoeaato
Oscilaaing Masscs
10
10
Optimblly Gap
10-1-
10
Baralion
Figure 16: Comparison of MLP vs. LSTM policies. Performance is averaged over 1000 test prob- lems.
41

---

Published as a conference paper at ICLR 2026
O ABLATION ANALYSIS ON THE USE OF LAGRANGE MULTIPLIERS IN THE SUPERVISED LOSS
In Fig. 17, we compare the performance of Deep FP on different problem classes using the op- timality gap loss from Eq. (11) and our proposed loss Eq. (12). It is evident that our loss including the Lagrange multipliers outperforms the optimality gap loss all cases, except for the oscillating masses problem class. This could be simply due to the fact that the performance is already ap- proaching 10-10 10 at 20 iterations, and so small numerical differences play a bigger role. The overall increase in performance using the new loss can be explained by a stronger gradient signal given to Deep FlexQP to learn policies that ensure pr 2 llvi and PF 2 l
Surprisingly, however, the performance using both losses remains comparable. The ability of the optimality gap loss to perform nearly as well as the Lagrange multiplier loss likely stems from the coupling of μur with æ. P1 and that of μE with Pg in the Deep FlexQP architecture. That is, even with a weaker gradient signal from the optimality gap loss, the respective networks are able to learn a shared representation that allows effective learning of the penalty parameters μ well.
Random QPs
Random QPs with Equalities
Deep PlxoPP (Oue)) Optinallt Cap Low
10a-
10-1-
FluQPP (Oun)
Multigliae
10-
Optimallty Dep
Optimallty Dap
10-1
10
10
10
10
10-1
Iteration
Iseratic
Portfolio Optimisation
Support Veeter Machinos
10
-0
10
-0
Opsinokity Gop
Opkinniit Gap
10
10-1
10
10
10
10
10-9
10
10-7
Bardion
LASSO
Huber Pitting
10
109
10 2-
10
Optimalit Gop
Oplimality Gap
10
10
10-4
10
10
10-1
Beration
Bonatio
Figure 17: Comparison on using the optimality gap vs. Lagrange multiplier loss for training Deep FPc Performance is averaged over 1000 test problems.
42

---

Published as a conference paper at ICLR 2026
Randon Lnear OCPs
Donble Integrator
10-1
10
Optinallty Dap
10
10-4
Optinality
10
10-7
Ieraaion
Eoeaaio
Oscilloiing Muses
108
10
Optimbily Gap
10-10
Figure 17: Comparison using the optimality gap Lagrange multiplier loss for training Deep FlexQP. Performance is averaged over 1000 test problems.
43


Reviews:
[{"review_id": "BHsnptz1UU", "reviewer": "Reviewer_axFq", "summary": "The paper tackles fast solving for structured optimization by combining:\na learned generator (diffusion) that predicts primal–dual variables,\na KKT-aware loss (feasibility and stationarity residuals), and\na post-refinement stage using classical primal–dual updates.\nThe aim is to obtain near-feasible, near-optimal solutions in very low inference time, with a few corrective iterations closing any remaining gaps.", "questions": "See Weakness.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Accelerating constrained optimization with learning is timely and useful.\n\n2. Primal–dual parameterization + KKT residuals makes the supervision meaningful; the post-refinement stage is practical for polishing errors.\n\n3. On synthetic QP-style tasks, the one-step (or few-step) approach achieves competitive gaps/residuals with favorable wall-clock times.", "weaknesses": "1. Limited novelty in core ingredients. Diffusion generation, GNN message passing over factor graphs, and KKT-residual losses are all known; the paper reads as a careful composition/tuning rather than a new algorithmic principle or theory.\n\n2. The paper lacks component-wise ablations that isolate the value of diffusion vs. a non-diffusive predictor, GNN vs. MLP, and KKT loss vs. plain supervised losses, as well as sensitivity to refinement steps and guidance scales.\n\n3. Under what assumptions (e.g., strong convexity, Slater/LICQ) does your KKT-guided sampling guarantee monotone decrease of a KKT energy or local convergence? Please state the step-size / guidance strength conditions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "seWfCDxvBD", "reviewer": "Reviewer_p3Uz", "summary": "This paper presents a learning-based optimization method rooted in the Alternating Direction Method of Multipliers (ADMM) for solving convex quadratic programming problems. The proposed approach begins by introducing slack variables to transform inequality constraints into equality constraints, and subsequently incorporates all equality constraints into the objective function using an ℓ₁-norm penalty. A resulting ADMM-type solver—referred to as FlexQP—is then derived following an update scheme analogous to that of OSQP. To further accelerate convergence, this paper employs an LSTM network to generate all hyperparameters originally required by the algorithm, and train the model in a supervised learning framework. Experimental results demonstrate that the proposed method achieves a faster convergence rate in terms of optimality gap under a fixed iteration budget compared to several baseline methods.", "questions": "See weaknesses.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "This paper presents a novel learning-enhanced ADMM framework, supported by theoretical analysis and demonstrated to achieve faster convergence rates compared to established baselines across diverse datasets.", "weaknesses": "1. The motivation for introducing slack variables and an ℓ₁-penalty term appears insufficiently justified. Since the ADMM-based solver OSQP can directly solve Problem (1), why not simply accelerate that algorithm using a neural network? Is the intention to use $z_I$ and $z_E$ to determine the feasibility of the original problem? However, in practice, $z_I$ and $z_E$ are unlikely to be exactly zero during iterations, as their values are strongly influenced by how well constraint (4b) can be satisfied. Moreover, if the original problem is infeasible, what is the practical value of providing a solution that only \"minimizes the constraint violations\"?\n2. While Figure 5 indicates superior convergence behavior of Deep FlexQP on all nine datasets, the actual computation time is missing. A clear description of how the runtime was measured should also be provided.\n3. The scale of the datasets used for solving the problems remains relatively limited. Could results on larger and more challenging problem instances be provided?\n4. The manuscript contains several instances of non-standard or undefined notation. For example, the abbreviation \"SQP\" in line 26 and \"S$\\ell_1$QP\" in line 145 lack clear definitions. In Theorem 3.1, the variables $y^{\\star}_I$ and $y^{\\star}_E$ are introduced without explanation. Additionally, the expression “$\\mu_i \\geq y_i$” in line 161 is ambiguous: if $\\mu_i$ is a scalar, should $y_i$ not also be a scalar rather than a vector? Similarly, in lines 158 and 164, it is unclear whether $y_i$ should carry an absolute value and whether it refers to an element of the dual variable $y$. We recommend a thorough review and clarification of notation throughout the text.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lt1E7tllDe", "reviewer": "Reviewer_GTzC", "summary": "The paper proposes FlexQP, an ADMM-based $l_1$-penalizing formulation for quadratic programs; the authors claim it produces feasible iterates, recovers the original optimum when feasible, and otherwise minimizes constraint violations. They further unroll the solver into Deep FlexQP with LSTM-based parameter policies, present PAC-Bayes generalization bounds for the learned optimizer, and integrate it into SQP solver for nonlinear control and safety filtering.", "questions": "Please refer to the weakness section.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The idea of using a uniformed penalty formulation to treat both feasible and infeasible points within the same objective is novel as it yields a single ADMM-based procedure.\n2. Unfolding learns LSTM-based parameter policies while retaining the structure of the original solver, enabling accelerations to the original approach without discarding the algorithmic backbone.\n3. The author provides theoretical support, including convergence characterizations of the penalty/ADMM scheme and PAC-Bayes generalization bounds.", "weaknesses": "1. The motivation behind and the advantages of using a $l_1$ penalty is not clear. The theory part claims properties of points that solve the  problem, but it does not directly establish a guarantee on whether Algorithm 1 and Deep-FlexQP can converge to those feasible/optimal solutions. A detailed explanation would be helpful.\n2. The significance of the reported acceleration is unclear. As noted, the dominant cost remains the first ADMM block update, and in some cases Deep FlexQP does not surpass Deep OSQP. Please add a detailed discussion on where the method is expected to help or not. Besides, it would be interesting to see how Deep FlexQP's predictions differ from those values of the original FlexQP.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)