Paper: Leveraging augmented-Lagrangian techniques for differentiating over infeasible quadratic programs in machine learning (ICLR-2024)

Reviewer comment (Writing):
This paper is very technical and the presentation should be improved. For example, the non-linear map (G) is introduced without any background information. The authors should add more text to motivate this non-linear map and explain how it is derived.

Author response:
We thank a lot the reviewer for this feedback. We are going to highlight the key ideas of our work in order to improve the presentation and readability of the results.

The map G is found via a change of variable from the KKT conditions for QP-H (detailed in equation 12 of the appendix, before the proof of Lemma 1). We will add this context to give more intuition about the origins of this map G.

Reviewer comment (Experiments):
Limited scope of applications. The experiment only shows training QP layers solving Sudoku problems. However, I am interested to see if the improved QP layer enables new applications.

Author response:
Again we thank a lot the reviewer for his feedback. The results of this work also apply to the classical examples where QPs layers need to be trained, in particular those in the appendix (e.g., cartpole, denoising and image classification). We will mention them more explicitly in the core of our work.

We also plan to consider more practical applications involving infeasible QPs notably in the context of control-based and robotic experiments.

Reviewer comment (Reproducibility):
How is the hierarchical quadratic program (QP-H) solved numerically? Is it similar to phase-I method in interior point methods?

Author response:
We thank a lot the reviewer for this relevant question. We will add more details to amphisize more precisely this practical numerical aspect. 

QP-H is solved numerically through an Augmented Lagrangian (AL) algorithm (in practice we use the ProxQP solver). More precisely, when applied to a QP, an AL algorithm has the converging property of outputting a solution to QP-H, if the original QP is primally infeasible. We make use of this feature to solve QP-H. More details can be found in the following reference [1].

Reviewer comment (Novelty):
The approach behind CvxpyLayers seems to use a similar least-squares relaxation (as also stated in the paper). How does the presented approach differ from this?

Author response:
We first would like to thank the reviewer for the time spent evaluating our work and for the constructive feedback.

## Question

***The approach behind CvxpyLayers seems to use a similar least-squares relaxation (as also stated in the paper). How does the presented approach differ from this?***

We thank a lot the reviewer for this relevant question. 

Fundamentally the least square relaxation is not applied to the same problem. 

In our work, we define the broader notion of the closest feasible QP solution and try differentiating through it via the Implicit Function Theorem (IFT). The least-square relaxation is then applied in order to output an "informative" Conservative Jacobian (named Extended Conservative Jacobian) when the IFT cannot be applied in this context. Note, that the least-square relaxation outputs also the IFT solution when it applies. 

CvxpyLayer considers differentiating via the IFT through a specific class of convex cones (named "Disciplined Parametrized Programs"). It applies directly the least-square relaxation to this specific problem.

Reviewer comment (Novelty):
The approach behind CvxpyLayers seems to use a similar least-squares relaxation (as also stated in the paper). How does the presented approach differ from this?

Author response:
We first would like to thank the reviewer for the time spent evaluating our work and for the constructive feedback.

## Question

***The approach behind CvxpyLayers seems to use a similar least-squares relaxation (as also stated in the paper). How does the presented approach differ from this?***

We thank a lot the reviewer for this relevant question. 

Fundamentally the least square relaxation is not applied to the same problem. 

In our work, we define the broader notion of the closest feasible QP solution and try differentiating through it via the Implicit Function Theorem (IFT). The least-square relaxation is then applied in order to output an "informative" Conservative Jacobian (named Extended Conservative Jacobian) when the IFT cannot be applied in this context. Note, that the least-square relaxation outputs also the IFT solution when it applies. 

CvxpyLayer considers differentiating via the IFT through a specific class of convex cones (named "Disciplined Parametrized Programs"). It applies directly the least-square relaxation to this specific problem.

Reviewer comment (Experiments):
To address the infeasibility issue for the QP layers, one straightforward way is to introduce slack variables and add penalties about the constraint violation in the objective function. The authors are suggested to show the advantage of the proposed approach as compared to this straightforward approach.

Author response:
We thank again the reviewer for this excellent question. 

Indeed, as pointed out by the reviewer, one straightforward way to address the infeasibility issue for the QP layers is to introduce slack variables and add penalties for the constraint violation in the objective function. It corresponds to a reformulation of QP-H($\theta$) as a convex QP. 

It turns out that when applied to a QP, an Augmented Lagrangian (AL) algorithm has the implicit converging property of outputting a solution to QP-H, if the original QP is primally infeasible. We make use of this feature to solve QP-H. More details can be found in the following reference [1].

Regarding the comparison asked by the reviewer, we've already carried out similar tests (see section 4.1.2), and even implemented them in other frameworks. The results of the reformulation of QP-H as a QP have been disappointing. This is actually one of the reasons which motivated this work, and the use of our AL-based techniques.

Reviewer comment (Theory):
One application of the QP layers is to use it as a building block for existing iterative algorithms (similar to the approach in [R1] and [R2]). The authors are suggested to analyze the convergence performance of the iterative algorithms when using the proposed conservative Jacobian as a building block.

Author response:
We thank also the reviewer for this question. We will add these two very relevant references in our related work section.

The performance of the iterative algorithm using conservative Jacobian as a building block is evaluated in our experiments in Section 4.1.2, and also in our appendix (see the different "pedagogical examples" of Appendix C.1). Other works also carry out this type of convergence analysis [2,3]. 

We would like to point out that the relatively recent notion of Conservative Jacobian (CJ) has been introduced to help explain better the behavior of Stochastic Gradient Descent-Based algorithms applied to nonsmooth functions in Deep Learning frameworks. For example, the CD3 algorithm detailed in [R1] may already use implicitly CJs. Indeed, a ReLU function is path differentiable, hence, the gradient step proposed in Section 3.2 for the inequality correction may actually be a CJ, if the functions considered in the composition (i.e., $g_x$ and $\phi_x$) are also path differentiable. This may explain the current performance of this algorithm.