Title: A Primal-Dual-Assisted Penalty Approach to Bilevel Optimization with Coupled Constraints
Venue: NeurIPS 2024
Accepted: yes
Keywords: Bilevel Optimization

Constrained Optimization

Hessian-free

Convergence Analysis

Penalty Based

Primal Dual
Abstract: Interest in bilevel optimization has grown in recent years, partially due to its relevance for challenging machine-learning problems. Several exciting recent works have been centered around developing efficient gradient-based algorithms that can solve bilevel optimization problems with provable guarantees. However, the existing literature mainly focuses on bilevel problems either without constraints, or featuring only simple constraints that do not couple variables across the upper and lower levels, excluding a range of complex applications. Our paper studies this challenging but less explored scenario and develops a (fully) first-order algorithm, which we term BLOCC, to tackle BiLevel Optimization problems with Coupled  Constraints.  We establish rigorous convergence theory for the proposed algorithm and demonstrate its effectiveness on two well-known real-world applications - support vector machine (SVM) - based model training and infrastructure planning in transportation networks.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis works addresses a bilevel problem where the lower-level has constraints that couple both the upper- and lower-level variables. The proposed method leverages a penalty approach and Lagrangian duality theory that transform the original bilevel problem into a single-level minimization problem where the x and y variables are decoupled. A first-order algorithm is developed which relies on the solution of two min-max subproblems and a rigorous convergence theory is developed. The effectiveness of the proposed method is validated on two real-world applications: infrastructure planning in transportation networks and support vector machine model training.\n\nSTRENGTHS:\n* This works deals with a challenging bilevel problem, as not only the lower-level is constrained, but in fact these constraints couple both the upper- and lower-level variables. \n* The proposed algorithm is first-order, even though the problem is a challenging one due to the presence of coupled constraints in the lower-level. In contrast, methods relying on implicit gradients, regardless if the problem is constrained or not, usually require access to second-order information.\n* Evaluation of the BLOCC algorithm on both toy examples and real-world applications (SVM model training and Transportation network design).\n\nWEAKNESSES:\n* Lemmas 2 and 3, which derive some key properties of the inner min-max problems, assume that the Lagrangian multipliers are bounded for every $x \\in X$. This is not a common assumption in literature and thus I am not sure why it is seen as mild. The authors cite Theorem 1.6 from [30] as a justification, however it is not immediately clear how the bounded property follows from that theorem. In fact, Theorem 1.6 is written for a more general (abstract) setting. As this is an important point, I think the authors should explain this in more detail. They might even include the theorem (or perhaps a different version of the theorem, tailored to the specific setting of this problem) in the Appendix and provide a clearer explanation.\n* Requires the solution of two min-max subproblems where the solution of each subproblem requires a double-loop algorithm. As a result, the proposed BLOCC method is a triple loop algorithm. On the contrary, most hypergradient methods rely only on the solution of a minimization problem (the lower problem).\n* The convergence result requires additional assumptions (assumption 5) beyond the ones concerning the objectives ($f,g$) and the constraints ($g_{c}$) of the original bilevel problem. It is unclear how restrictive assumption 5 is, as it is not expressed directly over $f,g,g_c$, but rather over some dual function. On the other hand, at least from the perspective of the dual function, the assumption is mild as the dual function is already concave.\n\nQUESTIONS:\n* Can you elaborate on Assumption 3? What do you mean by saying that $g^{c}$ satisfies the LICQ in $y$? Does this mean that for a given x the LICQ holds: a) for every $y \\in Y(x)$ or b) for every KKT point $y^{*}(x)$ of the lower-level problem?\n* Do the Assumptions 1-5 and the conditions within the theorems hold in the bilevel problems of the experiments (SVM model training and Transportation network design)?\n* Typos\n  * In equation (7): the + symbol is misplaced\n  * In the proof of Lemma 10 in page 17: in the formula of $\\nabla u_{h}(x)$ we should have $\\nabla h^{c}$ instead of $h^{c}$\n\nLIMITATIONS:\nNo negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an algorithm named BLOCC for solving a particular class of bilevel optimization problems where so-called coupled constraints are present. Coupled constraints are constraints w.r.t. the upper and lower level variables which are applied in addition to simple constraints like bound/ball constraints which are easy to solve for. Bilevel problems with coupled constraints have not been addressed in prior works in this literature, which rely on solving the inner problem and using the implicit function theorem (IFT) to pass gradients through the inner problem.\n\nBLOCC is particular suited for extremely large-scale optimization problems where other existing approaches which use IFT are intractable to solve due to the size of the KKT systems of the inner problems. Notably, BLOCC avoids solving the inner problem directly, instead replacing the inner problem with a simpler problem which only involves the simple constraints (and can be solved with a fast projection instead of requiring an iterative method). The coupled constraint is moved to the objective function of the outer problem, which is also modified as a result to be a saddle point problem. A custom first-order min-max solver is proposed to solve the outer problem and is an additional contribution of the paper. \n\nThe convergence properties of the algorithm is established, along with equivalence of the penalty reformulation to the original bilevel problem and BLOCC is evaluated on some interesting bilevel optimization problems against related methods. Particularly appealing are the extremely large-scale transportation network design experiments.\n\nSTRENGTHS:\nThe proposed method handles a broader range of bilevel problems compared to prior works in this literature, i.e., large-scale problems with coupled constraints in the inner problem. The characteristics of the algorithm itself (projection free, first-order only) are appealing.\n\nFurthermore, a reasonably detailed convergence analysis for all components of the algorithm are provided. The technical writing overall is of a reasonably high quality.\n\nFinally, the large-scale experiments for the network planning setting help demonstrate that BLOCC is useful to solve large-scale problems in practice which are currently unattainable by methods which use implicit function theorem.\n\nWEAKNESSES:\nThe main weakness in the paper in my mind is the statement of contribution, it was not immediately obvious why BLOCC is necessary on the first read-through. Only after reading the experimental section did I realize the importance of the algorithm design for large-scale problems. \n\nAs a more concrete suggestion, I think contextualising this work in the broader bi-level optimization literature would be beneficial. For instance, the value function reformulation (see [1] for example) is another way to solve an arguably broader class of optimization problems. However, I can see why directly solving the resultant single level problem may be computationally intractable for very large-scale problems using standard solvers, which motivates BLOCC. I would suggest referencing that literature better and emphasizing why the algorithmic properties of BLOCC enable application to large-scale bi-level problems earlier in the paper.\n\nI also think more analysis around the comparison methods LV-HBA and GAM for the large-scale problem in 4.3 is required. For instance, line 330-331 states that for GAM a Hessian must be inverted (presumably as part of using the IFT and differentiating through the inner problem). However, the actual linear system of equations may be solved using something more scalable such as conjugate gradient, which may permit the IFT methods to perform reasonably on this task. Some discussion or a direct comparison would greatly improve the experimental section in the paper.\n\nFinally, a more detailed description of GAM and LV-HBA would be helpful to better understand the differences to BLOCC. For instance, it is claimed that LV-HBA projection steps are expensive, however looking at their paper it seems that the projection is a fairly simple operation (Proj_Z defined in section 2.3 in their paper)? Adding this information will make the paper more self-contained.\n\n[1] Sinha et al. Bilevel Optimization based on Kriging Approximations of Lower Level Optimal Value Function. (2018 IEEE Congress on Evolutionary Computation)\n\nQUESTIONS:\nHow easy/robust is the tuning of the penalty parameters?\n\nTypically you just have a set of constraints for a problem. What is the significance of decoupling the inner problem into two sets of constraints, i.e., g^c and \\mathcal{Y}(x)? This appears to be a quirk of this literature and not say something like [1].\n\n[1] Sinha et al. Bilevel Optimization based on Kriging Approximations of Lower Level Optimal Value Function. (2018 IEEE Congress on Evolutionary Computation)\n\nLIMITATIONS:\nI believe there is a general lack of discussion of limitations in this paper. It would be nice to understand at the least which parameters are difficult to tune.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposed a fully first-order algorithm named BLOCC for the bilevel optimization problems with lower level coupled constraints. They provide convergence theory for the algorithm and demonstrate its effectiveness through numerical experiments in SVM-based model training and infrastructure planning in transportation networks.\n\nSTRENGTHS:\nThis paper designs a first-order algorithm for bilevel programming problems with coupling constraints in the lower level, and it does not require joint projection operations. The effectiveness of the algorithm is validated through extensive experiments. The writing is clear and well-structured.\n\nWEAKNESSES:\n1. This paper relies on some restrictive assumptions compared to other works, such as the strong convexity of both lower level objective $g$ and constraint $g^c$ with respect to $y$ (Assumption 2) and the Linear Independence Constraint Qualification (LICQ) condition in lower level (Assumption 4). These assumptions reduce the challenge of the bilevel problem considered. It is unclear whether the application problems tested in the numerical experiments satisfy these assumptions. The authors should clearly state in the introduction that their work depends on the strong convexity of the lower level problem.\n\n2. The proposed algorithm requires solving two minimax problems at each iteration, which can be computationally expensive for practical applications.\n\n3. The theoretical results do not clearly state how to choose the step size \\eta.\n\n4. The experiments in the paper lack details on the chosen hyperparameters of the algorithm.\n\n5. I have checked the code provided in the Supplementary Material. For the SVM problem, the implementation does not match the proposed algorithm; the minimax problems at each iteration are solved by different solvers than those stated in the paper.\n\nQUESTIONS:\n1. There are some typos in the paper. For example, in line 165, it should be \\mathbb{R}^{d_x}_{+}, in line 125, there is an extra “and”. \n2. How did the authors choose the iteration number T for the two inner minimax solvers in the experiments?\n\nLIMITATIONS:\nRelies on the strong convexity of the lower level problem.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}

comments: SUMMARY:
This paper presents an algorithm to solve bilevel optimization problems with coupled constraints using a primal-dual-assisted penalty reformulation. The study establishes rigorous convergence theory and demonstrates the algorithm's effectiveness through real-world applications, including SVM model training and transportation network planning.

STRENGTHS:
Strength:
- The paper is well-written with clear introduction of the problem, assumptions and technical presentation for their methodology.
- The primal-dual-assisted penalty reformulation and the main optimization algorithm effectively addresses the challenges of this class of problems.
- Comprehensive theoretical analysis are provided to show the theoretical convergency. 
- Application examples including SVM and transportation network design, are provided to demonstrate the effectiveness of the proposed algorithm.

WEAKNESSES:
Weakness:
- It would be great if more discussion on the computational complexity and scalability of the proposed method can be provided.
- In the experiments part, the compared baselines are very limited.

QUESTIONS:
How can the proposed algorithm be seamlessly integrated with other solvers?

LIMITATIONS:
NA
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-18