Title: An Alternating Optimization Method for Bilevel Problems under the Polyak-Łojasiewicz Condition
Venue: NeurIPS 2023
Accepted: yes
Keywords: Bilevel optimization

nonconvex constrained optimization

convergence analysis
Abstract: Bilevel optimization has recently regained interest owing to its applications in emerging machine learning fields such as hyperparameter optimization, meta-learning, and reinforcement learning. Recent results have shown that simple alternating (implicit) gradient-based algorithms can match the convergence rate of single-level gradient descent (GD) when addressing bilevel problems with a strongly convex lower-level objective. However, it remains unclear whether this result can be generalized to bilevel problems beyond this basic setting. In this paper, we first introduce a stationary metric for the considered bilevel problems, which generalizes the existing metric, for a nonconvex lower-level objective that satisfies the Polyak-Łojasiewicz (PL) condition. We then propose a Generalized ALternating mEthod for bilevel opTimization (GALET) tailored to BLO with convex PL LL problem and establish that GALET achieves an $\epsilon$-stationary point for the considered problem within $\tilde{\cal O}(\epsilon^{-1})$ iterations, which matches the iteration complexity of GD for single-level smooth nonconvex problems.

A Generalized Alternating Method for Bilevel Optimization under the Polyak-Łojasiewicz Condition

Abstract

Bilevel optimization has recently regained interest owing to its applications in emerging machine learning fields such as hyperparameter optimization, metalearning, and reinforcement learning. Recent results have shown that simple alternating (implicit) gradient-based algorithms can match the convergence rate of single-level gradient descent (GD) when addressing bilevel problems with a strongly convex lower-level objective. However, it remains unclear whether this result can be generalized to bilevel problems beyond this basic setting. In this paper, we first introduce a stationary metric for the considered bilevel problems, which generalizes the existing metric, for a nonconvex lower-level objective that satisfies the Polyak-Łojasiewicz (PL) condition. We then propose a G eneralized AL ternating m E thod for bilevel op T imization ( GALET ) tailored to BLO with convex PL LL problem and establish that GALET achieves an ϵ -stationary point for the considered problem within ˜ O ( ϵ -1 ) iterations, which matches the iteration complexity of GD for single-level smooth nonconvex problems.

1 Introduction

Bilevel optimization (BLO) is a hierarchical optimization framework that aims to minimize the upper-level (UL) objective, which depends on the optimal solutions of the lower-level (LL) problem. Since its introduction in the 1970s [5], BLO has been extensively studied in operations research, mathematics, engineering, and economics communities [14], and has found applications in image processing [12] and wireless communications [8]. Recently, BLO has regained interests as a unified framework of modern machine-learning applications, including hyperparameter optimization [49, 21, 22, 54], meta-learning [20], representation learning [3], reinforcement learning [62, 59], continual learning [55, 4], adversarial learning [73] and neural architecture search [39]; see [41].

In this paper, we consider BLO in the following form

where both the UL objective f ( x, y ) and LL objective g ( x, y ) are differentiable, and the LL solution set S ( x ) is not necessarily a singleton. For ease of notation, we denote the optimal function value of the LL objective as g ∗ ( x ) := min y g ( x, y ) and call it value function .

Although BLO is powerful for modeling hierarchical decision-making processes, solving generic BLO problems is known to be NP-hard due to their nested structure [31]. As a result, the majority of recent works in optimization theory of BLO algorithms are centered on nonconvex UL problems with strongly convex LL problems (nonconvex-strongly-convex), which permit the development of efficient algorithms; see e.g. [24, 27, 32, 9, 35, 10, 34, 67, 66]. The strong convexity assumption for LL ensures the uniqueness of the minimizer and a simple loss landscape of the LL problem, but it

Tianyi Chen

excludes many intriguing applications of BLO. In the context of machine learning, the LL objective might represent the training loss of a neural network, which can be non-strongly-convex [65].

To measure the efficiency of solving nonconvex BLO problems, it is essential to define its stationarity, which involves identifying a necessary condition for the optimality of BLO. In cases where the LL problem exhibits strongly convexity, the solution set S ( x ) becomes a singleton, leading to a natural definition of stationarity as the stationary point of the overall objective f ( x, S ( x )) , i.e., ∇ f ( x, S ( x )) = 0 . However, for BLO problems with non-strongly-convex LL problems, S ( x ) may have multiple solutions, rendering ∇ f ( x, S ( x )) ill-posed. This motivates an intriguing question:

Q1: What is a good metric of stationarity for BLO with nonconvex LL problems?

To address this question, we focus on the setting of BLO with the LL objective that satisfies the PL condition (nonconvex-PL). The PL condition not only encompasses the strongly convexity condition [24, 27, 32, 9, 35, 10, 34, 67, 66] and the Morse-Bott condition [2], but also covers many applications such as overparameterized neural networks [38], learning LQR models [19], and phase retrieval [61].

By reformulating the LL problem by its equivalent conditions, one can convert the BLO problem to a constrained optimization problem. Then with certain constraint qualification (CQ) conditions, a natural definition of the stationarity of the constrained optimization problem is the Karush-Kuhn-Tucker (KKT) point [11]. For example, constant rank CQ (CRCQ) was assumed in [37], and linear independence CQ (LICQ) was assumed or implied in [44, 28]. However, it is possible that none of these conditions hold for nonconvex-PL BLO (Section 2.2). In Section 2, we study different CQs on two constrained reformulations of (1) and then identify the best combination. Based on the right CQ on the right constrained reformulation, we prove the inherent CQ and propose a new notion of stationarity for the nonconvex-PL BLO, which strictly extends the existing measures in nonconvex-strongly-convex BLO [24, 27, 32, 9] and nonconvex-nonconvex BLO [2, 37, 28] without relaxing the problem [36, 57]. We emphasize the importance of defining new metric in Section 2.3.

Given a stationary metric, while ϵ -stationary point can be found efficiently in O ( ϵ -1 ) iterations for nonconvex and smooth single-level problem [6], existing works on the BLO with non-strongly-convex LL problem either lack complexity guarantee [2, 43, 36, 42], or occur slower rate [57, 37, 58, 7]. Moreover, most existing algorithms update the UL variable x after obtaining the LL parameter y sufficiently close to the optimal set S ( x ) by running GD from scratch, which is computationally expensive [37, 57]. In contrast, the most efficient algorithm for nonconvex-strongly-convex BLO updates x and y in an alternating manner, meaning that x is updated after a constant number of y updates from their previous values [9, 32]. Then another question arises:

Q2: Can alternating methods achieve the O ( ϵ -1 ) complexity for non-strongly-convex BLO?

Addressing this question is far from trivial. First, we need to characterize the drifting error of S ( x ) induced by the alternating strategy, which involves the change in the LL solution sets between two consecutive UL iterations. However, we need to generalize the analysis in nonconvex-strongly-convex BLO [9, 32, 24, 27] because S ( x ) is not a singleton. Moreover, we need to select an appropriate Lyapunov function to characterize the UL descent, as the nature candidate f ( x, S ( x )) is ill-posed without a unique LL minimizer. Finally, since the Lyapunov function we choose for UL contains both x and y , it is crucial to account for the drifting error of y as well.

By exploiting the smoothness of the value function g ∗ ( x ) and with the proper design of the Lyapunov function, we demonstrate the ˜ O ( ϵ -1 ) iteration complexity of our algorithm, which is optimal in terms

of ϵ . This result not only generalizes the convergence analysis in nonconvex-strongly-convex BLO [24, 27, 32, 9, 35, 10, 34, 67, 66] to the broader problem class, but also improves the complexity of existing works on nonconvex-non-strongly-convex BLO, specifically ˜ O ( ϵ -1 . 5 ) in [57] and ˜ O ( ϵ -4 ) in [37]; see Table 1. We present our algorithm in Section 3 and analyze its convergence in Section 4, followed by the simulations and conclusions in Section 5.

1.1 Related works

Nonconvex-strongly-convex BLO. The interest in developing efficient gradient-based methods and their nonasymptotic analysis for nonconvex-strongly-convex BLO has been invigorated by recent works [24, 32, 27, 9]. Based on the different UL gradient approximation techniques they use, these algorithms can be categorized into iterative differentiation and approximate implicit differentiationbased approaches. The iterative differentiation-based methods relax the LL problem by a dynamical system and use the automatic differentiation to approximate the UL gradient [21, 22, 25], while the approximate implicit differentiation-based methods utilize the implicit function theory and approximate the UL gradient by the Hessian inverse (e.g. Neumann series [9, 24, 27]; kernel-based methods [26]) or Hessian-vector production approximation methods (e.g. conjugate gradient descent [32, 54], gradient descent [35, 1]). Recent advances include variance reduction and momentum based methods [34, 67, 13]; warm-started BLO algorithms [1, 35]; distributed BLO approaches [63, 46, 68]; and algorithms solving BLO with constraints [64, 66]. Nevertheless, none of these attempts tackle the BLO beyond the strongly convex LL problem.

Nonconvex-nonconvex BLO. While nonconvex-strongly-convex BLO has been extensively studied in the literature, efficient algorithms for BLO with nonconvex LL problem remain under-explored. Among them, Liu et al. [43] developed a BLO method with initialization auxiliary and truncation of pessimistic trajectory; and Arbel and Mairal [2] generalized the implicit function theorem to a class of nonconvex LL functions and introduced a heuristic algorithm. However, these works primarily focus on analyzing the asymptotic performance of their algorithms, without providing finite-time convergence guarantees. Recently, Liu et al. [37] proposed a first-order method and established the first nonasymptotic analysis for non-strongly-convex BLO. Nonetheless, the assumptions such as CRCQ and bounded | f | , | g | are relatively restrictive. Huang [28] has proposed a momentumbased BLO algorithm, but the assumptions imply strongly convexity. Another research direction has addressed the nonconvex BLO problem by relaxation, such as adding regularization in the LL [42, 50], or replacing the LL optimal solution set with its ϵ -optimal solutions [36, 57]. Although this relaxation strategy overcomes the CQ-invalid issues, it introduces errors in the original bilevel problem [7]. To the best of our knowledge, none of these BLO algorithms handling multiple LL solutions can achieve the optimal iteration complexity in terms of ϵ .

Nonconvex-convex BLO. Another line of research focuses on the BLO with convex LL problem, which can be traced back to [48, 18]. Convex LL problems pose additional challenges of multiple LL solutions which hinder from using implicit-based approaches for nonconvex-strongly-convex BLO. To tackle multiple LL solutions, an aggregation approach was proposed in [56, 40]; a primal-dual algorithm was considered in [58]; a difference-of-convex constrained reformulated problem was explored in [72, 23]; an averaged multiplier method was proposed in [45]. Recently, Chen et al. [7] has pointed out that the objective of the non-strongly-convex LL problem can be discontinuous and proposed a zeroth-order smoothing-based method; Lu and Mei [47] have solved it by penalized min-max optimization. However, none of these attempts achieve the iteration complexity of ˜ O ( ϵ -1 ) . Moreover, although some works adopted KKT related concept as stationary measure, they did not find the inherent CQ condition, so the necessity of their measure to the optimality of BLO is unknown [45, 47]. In this sense, our work is complementary to them. The comparison with closely related works is summarized in Table 1.

Notations. For any given matrix A ∈ R d × d , we list the singular values of A in the increasing order as 0 ≤ σ 1 ( A ) ≤ σ 2 ( A ) ≤ · · · ≤ σ d ( A ) and denote the smallest positive singular value of A as σ + min ( A ) . We also denote A -1 , A † , A 1 / 2 and A -1 / 2 as the inverse of A , the Moore-Penrose inverse of A [29], the square root of A and the square root of the inverse of A , respectively. Ker( A ) = { x : Ax = 0 } , Ran( A ) = { Ax } denotes the null space and range space of A .

2 Stationarity Metric of Nonconvex-PL BLO

We will first introduce the equivalent constraint-reformulation of the nonconvex-PL BLO and then introduce our stationarity metric, followed by a section highlighting the importance of our results.

2.1 Equivalent constraint-reformulation of BLO

By viewing the LL problem as a constraint to the UL problem, BLO can be reformulated as a single-level nonlinear constrained optimization problem. Based on different equivalent characteristics of the LL problem, two major reformulations are commonly used in the literature [16]. The first approach is called value function-based reformulation, that is

Clearly, g ( x, y ) -g ∗ ( x ) = 0 is equivalent to y ∈ S ( x ) so that (2) is equivalent to (1).

On the other hand, recall the definition of PL function below, which is not necessarily strongly convex or even convex [33].

Definition 1 ( PL condition). The function g ( x, · ) satisfies the PL condition if there exists µ g > 0 such that for any given x , it holds that ∥∇ y g ( x, y ) ∥ 2 ≥ 2 µ g ( g ( x, y ) -g ∗ ( x )) , ∀ y.

According to Definition 1, for PL functions, ∇ y g ( x, y ) = 0 implies g ( x, y ) = g ∗ ( x ) . Therefore, the second approach replaces the LL problem with its stationary condition, that is

We call (3) the gradient-based reformulation. The formal equivalence of (2) and (3) with (1) is established in Theorem 3 in Appendix.

For constrained optimization, the commonly used metric of quantifying the stationarity of the solutions are the KKT conditions. However, the local (resp. global) minimizers do not necessarily satisfy the KKT conditions [11]. To ensure the KKT conditions hold at local (resp. global) minimizer, one needs to assume CQ conditions, e.g., the Slater condition, LICQ, Mangasarian-Fromovitz constraint qualification (MFCQ), and the CRCQ [30]. Nevertheless, Ye and Zhu [70] have shown that none of these standard CQs are valid to the reformulation (2) for all types of BLO.

2.2 Stationarity metric

We will next establish the necessary condition for nonconvex-PL BLO via the calmness condition [11, Definition 6.4.1], which is weaker than the Slater condition, LICQ, MFCQ and CRCQ.

Definition 2 ( Calmness). Let ( x ∗ , y ∗ ) be the global minimizer of the constrained problem

where h : R d x + d y → R d and d ≥ 1 . If there exist positive ϵ and M such that for any q ∈ R d with ∥ q ∥ ≤ ϵ and any ∥ ( x ′ , y ′ ) -( x ∗ , y ∗ ) ∥ ≤ ϵ which satisfies h ( x ′ , y ′ ) + q = 0 , one has

then the problem (4) is said to be calm with M .

Optimality gap v.s. iterations

Optimality gap

6

1

Initialization1

-·Initialization 2

4

2

0

0

5

10

15

20

25

Iteration

Stationary scorev.s.iterations

Stationary score

10

I1

Valuefunctionbased,Initialization1

I1

Ourstationarymetric,Initialization1

-·Valuefunctionbased,Initialization2

5

-Ourstationary metric,Initialization2

0

0

5

10

15

20

25

Iteration

x, y

The calmness of a problem quantifies the sensitivity of the objective to the constraints. Specifically, the calmness conditions of reformulations (2) and (3) are defined by setting h ( x, y ) = g ( x, y ) -g ∗ ( x ) and h (

) =

∇

y

g

(

x, y

)

, respectively.

The calmness condition is the weakest CQ which can be implied by Slater condition, LICQ, MFCQ and CRCQ [11]. However, as we will show, even the calmness condition does not hold for the nonconvex-PL BLO when employing the value function-based reformulation (2).

Example 1. Considering x ∈ R , y = [ y 1 , y 2 ] ⊤ ∈ R 2 , and the BLO problem as

Lemma 1. Considering the BLO problem in Example 1, the LL objective satisfies the PL condition and the global minimizers of it are within the set

but the calmness condition of reformulation (2) is not satisfied under on all of the global minimizers.

Figure 1 illustrates this fact by showing that the KKT score on the value function-based reformulation does not approach 0 as the distance to the optimal set decreases. This observation implies that the KKT conditions associated with the value function-based reformulation (2) do not constitute a set of necessary conditions for global minimizers for the nonconvex-PL BLO problems. As a result, the KKT point of (2) is unable to serve as the stationary point for the nonconvex-PL BLO problems.

Therefore, we adopt the gradient-based reformulation (3). Unfortunately, it is still possible that the standard CQs do not hold for some of the nonconvex-PL BLO problems associated with the gradient-based reformulation (3). To see this, let ( x ∗ , y ∗ ) be the global minimizer of (3). By denoting the matrix concatenating the Hessian and Jacobian as [ ∇ 2 yy g ( x ∗ , y ∗ ) , ∇ 2 yx g ( x ∗ , y ∗ )] , the generic CQ conditions are instantiated in the gradient-based reformulation (3) by

LICQ, MFCQ: The rows of [ ∇ 2 yy g ( x ∗ , y ∗ ) , ∇ 2 yx g ( x ∗ , y ∗ )] are linearly independent; and,

CRCQ: ∃ neighborhood of ( x ∗ , y ∗ ) such that [ ∇ 2 yy g ( x, y ) , ∇ 2 yx g ( x, y )] is of constant rank.

If g ( x, y ) is strongly convex over y , then ∇ 2 yy g ( x, y ) is of full rank for any x and y , which ensures the LICQ, MFCQ and CRCQ of the gradient-based reformulation (3). However, if g ( x, y ) merely satisfies the PL condition like Example 1, none of the standard CQs hold, which is established next.

Lemma 2. Example 1 violates Slater condition, LICQ, MFCQ and CRCQ conditions of (3) .

Remarkably, the following lemma demonstrates that the gradient-based reformulation of the nonconvex-PL BLO inherits the calmness condition.

Lemma 3 ( Calmness of nonconvex-PL BLO). If g ( x, · ) satisfies the PL condition and is smooth, and f ( x, · ) is Lipschitz continuous, then (3) is calm at its global minimizer ( x ∗ , y ∗ ) .

The Lipschitz smoothness of g ( x, y ) and Lipschitz continuity of f ( x, y ) over y are standard in BLO [9, 27, 24, 32, 35, 34, 1, 13, 37]. In this sense, nonconvex-PL BLO associated with the gradient-based reformulation (3) is naturally calm so that the KKT conditions are necessary conditions for its optimality [11]. A summary and illustration of our theory is shown in Figure 2.

To benefit the algorithm design (Section 3.2), we establish the necessary conditions of the optimality for nonconvex-PL BLO by slightly modifying the KKT conditions of (3) in the next theorem.

̸

Theorem 1 ( Necessary condition in nonconvex-PL BLO). If g ( x, · ) satisfies the PL condition and is smooth, and f ( x, · ) is Lipschitz continuous, then ∃ w ∗ = 0 such that

hold at the global minimizer ( x ∗ , y ∗ ) of (1) .

This necessary condition is tight in the sense that it is a generalization of stationary measures in the existing literature for BLO with LL problem exhibiting strongly convexity [9, 27, 24, 32], PL with CRCQ [37], invertible Hessian and singleton solution [28] and Morse-Bott functions [2]. Thanks to the inherent calmness of PL BLO, our result eliminates the CRCQ condition in [37]. We next show the connections of our results with other works.

Nonconvex-strongly-convex BLO or PL with invertible Hessian and singleton solution. As S ( x ) is singleton and ∇ yy g ( x, y ) is always non-singular, the solution to (8b) is uniquely given by

Therefore, the necessary condition in (8) is equivalent to

where the first equality is obtained by the implicit function theorem. Therefore, (8) recovers the necessary condition ∇ f ( x ∗ , S ( x ∗ )) = 0 for nonconvex-strongly-convex BLO.

Nonconvex-Morse-Bott BLO. Morse-Bott functions are special cases of PL functions [2]. In this case, ∇ yy g ( x, y ) can be singular so that (8b) may have infinite many solutions, which are given by

According to [2, Proposition 6], for any y ∗ ∈ S ( x ∗ ) , ∇ 2 xy g ( x ∗ , y ∗ ) ⊂ Ran( ∇ 2 yy g ( x ∗ , y ∗ )) , which is orthogonal to Ker( ∇ 2 yy g ( x ∗ , y ∗ )) . As a result, although the solution to (8b) is not unique, all of possible solutions yield the unique left hand side value of (8a). i.e. ∀ w ∗ ∈ W ∗ ,

Plugging into (8a), we arrive at

where ϕ ( x ∗ , y ∗ ) is the same as the degenerated implicit differentiation in [2].

Based on (8), we can define the ϵ - stationary point of the original BLO problem (1) as follows.

Definition 3 ( ϵ - stationary point). A point (¯ x, ¯ y ) is called ϵ -stationary point of (1) if ∃ ¯ w such that R x (¯ x, ¯ y, ¯ w ) ≤ ϵ, R w (¯ x, ¯ y, ¯ w ) ≤ ϵ and R y (¯ x, ¯ y ) ≤ ϵ .

2.3 The importance of necessary conditions for BLO without additional CQs

Next, we emphasize the importance of deriving the necessary condition for the optimality of the nonconvex-PL BLO without additional CQs.

On the one hand, the necessary condition for the optimality of BLO is fundamental to the algorithm design and has been investigated for a long time in the optimization community [70, 71, 15, 17], but has not yet been fully understood [69]. One of the main challenges is that traditional CQs for constrained optimization are hard to check or do not hold in BLO [16, 70]. As a result, the development of mild CQs and the identification of BLO classes that inherently satisfy these CQs are considered significant contributions to the field [70, 71, 15, 17, 69]. Among those, the calmness condition is the weakest CQ [11].

On the other hand, recent BLO applications in machine learning often involve nonconvex LL problems,

Gradient-based

Calmness

strongly-

convex-BLO

reformulation

Gradient-basedPL-BLO

reformulation

LICQ

MFCQ

CRCQ

Value-functionbased

PL-BLO

the necessary condition for the solution of which is far less explored in either optimization or machine learning community. In the optimization community, most works focus on proving linear bilevel and sequential min-max optimization satisfy certain CQs [70, 16]. In the machine learning community, works on nonconvex-nonconvex BLO either relax the LL problem that introduces error [43, 57] or impose assumptions that are hard to verify in practice such as CRCQ [37] or Morse-Bott condition [2]. To the best of our knowledge, we are the first to propose the necessary condition for the optimality of a class of nonconvex-nonconvex BLO problems with checkable assumptions. Moreover, algorithms in constrained optimization are always CQ-dependent, e.g. the convergence of an algorithm depends on whether a particular CQ condition is satisfied. As we prove that standard CQs are invalid for nonconvex-PL BLO, several existing BLO may become theoretically less grounded [37, 28].

3 An Alternating Method for Bilevel Problems under the PL Condition

In this section, we introduce our Generalized ALternating mEthod for bilevel opTimization with convex LL problem, GALET for short, and then elucidate its relations with the existing algorithms.

3.1 Algorithm development

To attain the ϵ -stationary point of (1) in the sense of Definition 3, we alternately update x and y to reduce computational costs. At iteration k , we update y k +1 by N -step GD on g ( x k , y ) with y k, 0 = y k and y k +1 = y k,N as

Algorithm 1 GALET for nonconvex-PL BLO

While setting N = 1 is possible, we retain N for generality. We then update w via the fixed-point equation derived from (8b) and employ ∇ yy g ( x, y ) ( ∇ y f ( x, y ) + ∇ yy g ( x, y ) w ) as the increment for w . This increment can be viewed as the gradient of L ( x, y, w ) defined as

which is quadratic w.r.t. w , given x k and y k +1 .

However, unlike the LL objective g ( x, y ) , the objective L ( x, y, w ) is Lipschitz smooth with respect to x and y only for bounded w , which makes it difficult to control the change of solution (11) under different x and y . Hence, we update w k +1 via T -step GD on with w k, 0 = 0 and w k +1 = w k,T as

After obtaining the updated y k +1 and w k +1 , we update x k by the fixed point equation of (8a) as

̸

We summarize our algorithm in Algorithm 1. We choose w k, 0 = 0 for simplicity, but w k, 0 = w 0 = 0 is also valid. Same convergence statement can hold since the boundedness and Lipschitz continuity of limit points lim t →∞ w k,t are still guaranteed. From the algorithm perspective, [2] shares similarities with us. However, without recognizing the property of GD converging to the minimal norm solution [52], Arbel and Mairal [2] introduces an additional Hessian into the objective (11), resulting in the calculation of fourth-order Hessian operation, which is more complex than GALET.

3.2 Relation with algorithms in nonconvex-strongly-convex BLO

We explain the relation between GALET and methods in the nonconvex-strongly-convex BLO.

First, if g ( x, y ) is strongly convex in y , minimizing L ( x, y, w ) over w yields the unique solution w ∗ ( x, y ) = -( ∇ 2 yy g ( x, y ) ) -1 ∇ y f ( x, y ) . This means that optimizing L ( x, y, w ) corresponds to implicit differentiation in the nonconvex-strongly-convex BLO [9, 27, 24, 32]. Therefore, we refer the optimization of w as the shadow implicit gradient level.

On the other hand, if ∇ 2 yy g ( x, y ) is positive definite, the problem (11) over w is equivalent to

which can be verified by their first-order necessary conditions. Thus, one can update w by GD on (14) via

nonconvex-strongly-convex does not work well for nonconvex-PL BLO. Note that the increment in (15) eliminates the extra Hessian in (12b) and recovers the updates for nonconvexstrongly-convex BLO in [35, 27, 32, 9, 66]. Actually, the additional Hessian in (12b) is inevitable for an algorithm to find the stationary point of the nonconvex-PL BLO. Figure 3 shows the comparative results of our algorithm with the nonconvex-strongly-convex BLO algorithm using (15) instead of (12b) on Example 1, where the global optimality is measured by the distance to the global optimal set (7) with the explicit form stated in (46). We observe that the update (15) fails to find the global optimizer of the example in Lemma 1 so that the additional Hessian in (12b) is unavoidable.

4 Convergence Analysis

In this section, we provide the convergence rate analysis under convex PL LL settings. We first introduce the assumptions and the challenges. Next, we analyze the descent property and drifting error in LL, the bias of w k +1 to the optimal solution of min w L ( x k , y k +1 , w ) and the descent property of UL. Finally, we define a new Lyapunov function and state the iteration complexity of GALET.

Assumption 1 (Lipschitz continuity) . Assume that ∇ f, ∇ g and ∇ 2 g are Lipschitz continuous with ℓ f, 1 , ℓ g, 1 and ℓ g, 2 , respectively. Additionally, we assume f ( x, y ) is ℓ f, 0 -Lipschitz continuous over y g x,y min yy

. Assumption 2 (Landscape of LL objective) . Assume that g ( x, y ) is µ g -PL over y . Moreover, let σ g > 0 be the lower-bound of the positive singular values of Hessian, i.e. σ = inf { σ + ( ∇ 2 g ( x, y )) } .

Remark 1. By definition, singular values are always nonnegative. We use σ g to denote the lower bound of the non-zero singular values of ∇ 2 yy g ( x, y ) . Assumption 2 means that the non-zero singular values are bounded away from 0 on the entire domain. Given Assumption 1 that the Hessian is globally Lipschitz, they together potentially rule out negative eigenvalues of the Hessian. However, different from strong convexity [9, 27, 24, 32, 28], this assumption still permits ∇ 2 yy g ( x, y ) to possess zero eigenvalues and includes BLO problems of multiple LL solutions. As an example, this includes the loss of the (overparameterized) generalized linear model [52]. In addition, Assumption 2 is needed only in the convergence rate analysis along the optimization path to ensure the sequence { w k } is well-behaved. Therefore, it is possible to narrow down either the Hessian Lipschitz condition or the lower bound of singular values to a bounded region, when the local optimal sets are bounded.

Challenges of analyzing GALET. The absence of strong convexity of the LL brings challenges to characterize the convergence of GALET. To see this, recall that in recent analysis of BLO such as [9], when g ( x, · ) is strongly convex, the minimizer of LL is unique. Therefore, to quantify the convergence of Algorithm 1, one can use the Lyapunov function V k SC := f ( x k , S ( x k )) + ∥ y k -S ( x k ) ∥ 2 where f ( x k , S ( x k )) and ∥ y k -S ( x k ) ∥ 2 are used to account for the UL descent and LL error caused by the alternating update and the inexactness, respectively. However, V SC is not well-defined when S ( x ) is not unique. Therefore, it is necessary to seek for a new Lyapunov function.

4.1 Descent of each sequence in GALET

Anature alternative of ∥ y k -S ( x k ) ∥ 2 under the PL condition is the LL optimality residual R y ( x k , y k ) , the evolution of which between two steps can be quantified by

where the first term characterizes the drifting LL optimality gap after updating x , and the second term shows the descent amount by one-step GD on y . Unlike the strongly convex case where S ( x ) is Lipschitz continuous, both g ∗ ( x ) and g ( x, y ) are not Lipschitz continuous, so we cannot directly bound the the first difference term in (16) by the drifting of update ∥ x k +1 -x k ∥ . Owing to the opposite sign of g ( x, y ) and g ∗ ( x ) , we bound the first term in (16) by the smoothness of g ( x, y ) and g ∗ ( x ) . The second term in (16) can be easily bounded by the fact that running GD on PL function ensures the contraction of function value distance.

Lemma 4. Under Assumption 1-2, let y k +1 be the point generated by (10) given x k and the stepsize β ≤ 1 ℓ g, 1 . Then it holds that

where L g := ℓ g, 1 (1 + ℓ g, 1 / 2 µ g ) and η 1 = O (1) is a constant that will be chosen in the final theorem.

Rearranging the terms in (17a) and using the fact that g ( x, y ) ≥ g ∗ ( x ) , the second term in (16) can be upper bounded by a negative term and O ( β 2 ) . Adding (17a) and (17b), letting α, β is the same order and choosing η 1 = O (1) properly yield

When we choose η 1 large enough, the second term will be dominated by the negative term -O ( α ) ∥ d k x ∥ 2 given by the UL descent, so it will be canceled out.

(a) UL level:

α

(b) Shadow implicit gradient level:

T, γ

(c) LL level:

N,β

20

α=0.1

7000

0

T=50,=0.1

3000

0

N=1,3=3

3000

0

Q=0.3

Q=0.5

Q=0.7

6000

5000

-20

T=5,=0.1

T=1,=0.1

T=50,=0.5

2500

2000

-20

N=1,β=1

N=1,3=0.5

N=1,3=0.1

2500

2000

x-20

0=0.9

4000

T=5，=0.5

T=1,=0.5

1500

40

N=5,3=1

N=5,3=0.5

1500

-40

3000

2000

-40

1000

-60

1000

-60

1000

-60

500

500

-80

-40

-20

0

20

40

60

0

20

40

60

0

20

40

60

X

X

X

With the iterates generated by GD initialized at 0 converging to the minimal norm solution of the least squares problem [52], we can characterize the drifting error of w using the Lipschitz continuity of its minimal norm solution. The following lemma characterizes the error of w k +1 to the minimal-norm solution of the shadow implicit gradient level w † ( x, y ) := -( ∇ 2 yy g ( x, y ) ) † ∇ y f ( x, y ) .

Lemma5. Under Assumption 1-2, we let w k +1 denote the iterate generated by (12a) given ( x k , y k +1 ) and w k, 0 = 0 , and denote b k := ∥ w k +1 -w † ( x k , y k +1 ) ∥ . If we choose the stepsize ρ ≤ 1 ℓ 2 g, 1 , then the shadow implicit gradient error can be bounded by b 2 k ≤ 2(1 -ρσ 2 g ) T ℓ 2 f, 0 /µ g .

Instead of using f ( x, S ( x )) which is ill-posed, we integrate the increment of x to get F ( x, y ; w ) = f ( x, y ) + w ⊤ ∇ y g ( x, y ) . Then by plugging the optimal minimal norm solution w † ( x, y ) , we choose F ( x k , y k ; w † ( x k , y k )) as the counterpart of f ( x k , S ( x k )) in V k SC . We can quantify its difference between two adjacent steps by the smoothness of F ( x, y ; w ) .

Lemma 6. Under Assumption 1-2, by choosing α ≤ min { 1 4 L F +8 L w ℓ g, 1 , βη 2 2 L 2 w } , one has

where L w is the constant defined in Appendix and constant η 2 will be chosen in the final theorem.

4.2 A new Lyapunov function

Based on (18) and (19), we can define a new Lyapunov function for the nonconvex-PL BLO as

where c is chosen to balance the coefficient of the term R y ( x k , y k ) and ∥ d k x ∥ 2 such that

Telescoping to (21) results in the convergence of both ∥ d k x ∥ 2 and R y ( x k , y k ) . By definition, ∥ d k x ∥ 2 = R x ( x k , y k +1 , w k +1 ) , so the convergence of R x ( x k , y k , w k ) is implied by the Lipschitz continuity of increments and the fact that ∥ y k +1 -y k ∥ , ∥ w k +1 -w k ∥ → 0 . Likewise, the convergence of R w ( x k , y k , w k ) can be established by recognizing that b 2 k is sufficiently small when selecting a large T , ensuring that R w ( x k , y k +1 , w k +1 ) is also small, and consequently, R w ( x k , y k , w k ) . The convergence results of GALET can be formally stated as follows.

From Definition 3, one need O ( KT ) = O ( ϵ -1 log( ϵ -1 )) =: ˜ O ( ϵ -1 ) iterations in average to achieve ϵ -stationary point of the BLO problem (1). The next remark shows that this complexity is optimal.

Remark 2 ( Lower bound). The iteration complexity ˜ O ( ϵ -1 ) in Theorem 2 is optimal in terms of ϵ , up to logarithmic factors. This is because when f ( x, y ) is inde-

BOME

GALET

1.5

V-PBGD

measure

10

2

(1/K)

IAPTT-GM

Test loss

GALET

10

1.0

Stationary

10

4

0.5

10

5

0

500

1000

1500

2000

2500

30

0

1000

2000

3000

Iteration

Iteration

pendent of y and g ( x, y ) = 0 , Assumptions 1 and 2 reduce to the smoothness assumption of f on x , and thus the O ( ϵ -1 ) lower bound of nonconvex and smooth minimization in [6] also applies to BLO under Assumptions 1 and 2.

5 Preliminary Simulations and Conclusions

The goal of our simulations is to validate our theories and test the performance of GALET on actual learning applications. The experimental setting and parameter choices are included in the Appendix.

Our stationary measure is a necessary condition of the global optimality. As shown in Figure 1, GALET approaches the global optimal set of Example 1 and our stationary measure also converges to 0 , while the value-function based KKT score does not.

UL stepsize is the most sensitive parameter in our algorithm. As shown in Figure 4, varying UL stepsize α leads to different trajectories of GALET, while varying parameters in the LL and shadow implicit gradient level only cause small perturbations. This means that α is the most sensitive parameter. The fact that GALET is robust to small values of T and N makes it computationally appealing for practical applications, eliminating the need to tune them extensively. This phenomenon also occurs in the hyper-cleaning task.

Our method converges fast on the actual machine learning application. We compare GALET with BOME [37], IAPTT-GM [43] and V-PBGD [57] in the hyper-cleaning task on the MNIST dataset. As shown in Figure 5, GALET converges faster than other methods and the convergence rate of GALET is O (1 /K ) , which matches Theorem 2. Table 2 shows that the test accuracy of GALET is comparable to other methods.

Conclusions and limitations. In this paper, we study BLO with lower-level objectives satisfying the PL condition. We first establish the stationary measure for nonconvex-PL BLO without additional CQs and then propose an alternating optimization algorithm that generalizes the existing alternating (implicit) gradient-based algorithms for bilevel problems with a strongly convex lower-level objective. Our algorithm termed GALET achieves the ˜ O ( ϵ -1 ) iteration complexity

for BLO with convex PL LL problems. Numerical experiments are provided to verify our theories.

One potential limitation is that we only consider the deterministic BLO problem and require secondorder information, and it is unclear whether our analysis can be generalized to handle the stochastic case and whether the same iteration complexity can be guaranteed with full first-order information. Another limitation is Assumption 2. This stipulates that the singular value of the LL Hessian must be bounded away from 0 to ensure the stability of the { w k } .

References

Michael Arbel and Julien Mairal. Amortized implicit differentiation for stochastic bilevel optimization. In Proc. International Conference on Learning Representations , virtual, 2022.

Michael Arbel and Julien Mairal. Non-convex bilevel games with critical point selection maps. In Proc. Advances in Neural Information Processing Systems , New Orleans, LA, 2022.

Sanjeev Arora, Simon Du, Sham Kakade, Yuping Luo, and Nikunj Saunshi. Provable representation learning for imitation learning via bi-level optimization. In Proc. International Conference on Machine Learning , virtual, 2020.

Zalán Borsos, Mojmír Mutn` y, and Andreas Krause. Coresets via bilevel optimization for continual learning and streaming. In Proc. Advances in Neural Information Processing Systems , virtual, 2020.

Jerome Bracken and James T McGill. Mathematical programs with optimization problems in the constraints. Operations Research , 21(1):37-44, 1973.

Yair Carmon, John C Duchi, Oliver Hinder, and Aaron Sidford. Lower bounds for finding stationary points i. Mathematical Programming , 184(1-2):71-120, 2020.

Lesi Chen, Jing Xu, and Jingzhao Zhang. On bilevel optimization without lower-level strong convexity. arXiv preprint arXiv:2301.00712 , 2023.

Lisha Chen, Sharu Theresa Jose, Ivana Nikoloska, Sangwoo Park, Tianyi Chen, and Osvaldo Simeone. Learning with limited samples: Meta-learning and applications to communication systems. Foundations and Trends® in Signal Processing , 17(2):79-208, 2023.

Tianyi Chen, Yuejiao Sun, and Wotao Yin. Closing the gap: Tighter analysis of alternating stochastic gradient methods for bilevel problems. In Proc. Advances in Neural Information Processing Systems , virtual, 2021.

Tianyi Chen, Yuejiao Sun, Quan Xiao, and Wotao Yin. A single-timescale method for stochastic bilevel optimization. In Proc. International Conference on Artificial Intelligence and Statistics , virtual, 2022.

Frank H Clarke. Optimization and nonsmooth analysis . SIAM, 1990.

Caroline Crockett and Jeffrey Fessler. Bilevel methods for image reconstruction. Foundations and Trends® in Signal Processing , 15(2-3):121-289, 2022.

Mathieu Dagréou, Pierre Ablin, Samuel Vaiter, and Thomas Moreau. A framework for bilevel optimization that enables stochastic and global variance reduction algorithms. In Proc. Advances in Neural Information Processing Systems , New Orleans, LA, 2022.

Stephan Dempe. Foundations of bilevel programming . Springer Science & Business Media, 2002.

Stephan Dempe and Alain B Zemkoho. On the karush-kuhn-tucker reformulation of the bilevel optimization problem. Nonlinear Analysis: Theory, Methods & Applications , 75(3):1202-1218, 2012.

Stephan Dempe and Alain B Zemkoho. The bilevel programming problem: reformulations, constraint qualifications and optimality conditions. Mathematical Programming , 138(1):447473, 2013.

Stephan Dempe, Vyatcheslav V Kalashnikov, and Nataliya Kalashnykova. Optimality conditions for bilevel programming problems. Optimization with Multivalued Mappings: Theory, Applications, and Algorithms , pages 3-28, 2006.

Joydeep Dutta and Stephan Dempe. Bilevel programming with convex lower level problems. Optimization with Multivalued Mappings: Theory, Applications, and Algorithms , pages 51-71, 2006.

Maryam Fazel, Rong Ge, Sham Kakade, and Mehran Mesbahi. Global convergence of policy gradient methods for the linear quadratic regulator. In Proc. International Conference on Machine Learning , Stockholm, Sweden, 2018.

Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proc. International Conference on Machine Learning , Sydney, Australia, 2017.

Luca Franceschi, Michele Donini, Paolo Frasconi, and Massimiliano Pontil. Forward and reverse gradient-based hyperparameter optimization. In Proc. International Conference on Machine Learning , Sydney, Australia, 2017.

Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi, and Massimilano Pontil. Bilevel programming for hyperparameter optimization and meta-learning. In Proc. International Conference on Machine Learning , Stockholm, Sweden, 2018.

Lucy L Gao, Jane Ye, Haian Yin, Shangzhi Zeng, and Jin Zhang. Value function based differenceof-convex algorithm for bilevel hyperparameter selection problems. In Proc. International Conference on Machine Learning , Baltimore, MD, 2022.

Saeed Ghadimi and Mengdi Wang. Approximation methods for bilevel programming. arXiv preprint arXiv:1802.02246 , 2018.

Riccardo Grazzi, Luca Franceschi, Massimiliano Pontil, and Saverio Salzo. On the iteration complexity of hypergradient computation. In Proc. International Conference on Machine Learning , virtual, 2020.

Ryuichiro Hataya and Makoto Yamada. Nyström method for accurate and scalable implicit differentiation. In Proc. International Conference on Artificial Intelligence and Statistics , Valencia, Spain, 2023.

Mingyi Hong, Hoi-To Wai, Zhaoran Wang, and Zhuoran Yang. A two-timescale stochastic algorithm framework for bilevel optimization: Complexity analysis and application to actorcritic. SIAM Journal on Optimization , 33(1):147-180, 2023.

Feihu Huang. On momentum-based gradient methods for bilevel optimization with nonconvex lower-level. arXiv preprint arXiv:2303.03944 , 2023.

MJames. The generalised inverse. The Mathematical Gazette , 62(420):109-114, 1978.

Robert Janin. Directional derivative of the marginal function in nonlinear programming . Springer, 1984.

Robert G Jeroslow. The polynomial hierarchy and a simple model for competitive analysis. Mathematical programming , 32(2):146-164, 1985.

Kaiyi Ji, Junjie Yang, and Yingbin Liang. Bilevel optimization: Convergence analysis and enhanced design. In Proc. International Conference on Machine Learning , virtual, 2021.

Hamed Karimi, Julie Nutini, and Mark Schmidt. Linear convergence of gradient and proximalgradient methods under the polyak-łojasiewicz condition. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases , Riva del Garda, Italy, 2016.

Prashant Khanduri, Siliang Zeng, Mingyi Hong, Hoi-To Wai, Zhaoran Wang, and Zhuoran Yang. A near-optimal algorithm for stochastic bilevel optimization via double-momentum. In Proc. Advances in Neural Information Processing Systems , virtual, 2021.

Junyi Li, Bin Gu, and Heng Huang. A fully single loop algorithm for bilevel optimization without hessian inverse. In Proc. Association for the Advancement of Artificial Intelligence , virtual, 2022.

Gui-Hua Lin, Mengwei Xu, and Jane J Ye. On solving simple bilevel programs with a nonconvex lower level program. Mathematical Programming , 144(1-2):277-305, 2014.

Bo Liu, Mao Ye, Stephen Wright, Peter Stone, et al. Bome! bilevel optimization made easy: A simple first-order approach. In Proc. Advances in Neural Information Processing Systems , New Orleans, LA, 2022.

Chaoyue Liu, Libin Zhu, and Mikhail Belkin. Loss landscapes and optimization in overparameterized non-linear systems and neural networks. Applied and Computational Harmonic Analysis , 59:85-116, 2022.

Hanxiao Liu, Karen Simonyan, and Yiming Yang. DARTS: Differentiable architecture search. In Proc. International Conference on Learning Representations , New Orleans, LA, 2019.

Risheng Liu, Pan Mu, Xiaoming Yuan, Shangzhi Zeng, and Jin Zhang. A generic firstorder algorithmic framework for bi-level programming beyond lower-level singleton. In Proc. International Conference on Machine Learning , virtual, 2020.

Risheng Liu, Jiaxin Gao, Jin Zhang, Deyu Meng, and Zhouchen Lin. Investigating bi-level optimization for learning and vision from a unified perspective: A survey and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021.

Risheng Liu, Xuan Liu, Xiaoming Yuan, Shangzhi Zeng, and Jin Zhang. A value-function-based interior-point method for non-convex bi-level optimization. In Proc. International Conference on Machine Learning , pages 6882-6892, virtual, 2021.

Risheng Liu, Yaohua Liu, Shangzhi Zeng, and Jin Zhang. Towards gradient-based bilevel optimization with non-convex followers and beyond. Proc. Advances in Neural Information Processing Systems , 2021.

Risheng Liu, Xuan Liu, Wei Yao, Shangzhi Zeng, and Jin Zhang. Towards extremely fast bilevel optimization with self-governed convergence guarantees. arXiv preprint arXiv:2205.10054 , 2022.

Risheng Liu, Yaohua Liu, Wei Yao, Shangzhi Zeng, and Jin Zhang. Averaged method of multipliers for bi-level optimization without lower-level strong convexity. In Proc. International Conference on Machine Learning , Honolulu, HI, 2023.

Songtao Lu, Siliang Zeng, Xiaodong Cui, Mark Squillante, Lior Horesh, Brian Kingsbury, Jia Liu, and Mingyi Hong. A stochastic linearized augmented lagrangian method for decentralized bilevel optimization. In Proc. Advances in Neural Information Processing Systems , New Orleans, LA, 2022.

Zhaosong Lu and Sanyou Mei. First-order penalty methods for bilevel optimization. arXiv preprint arXiv:2301.01716 , 2023.

Zhi-Quan Luo, Jong-Shi Pang, and Daniel Ralph. Mathematical Programs with Equilibrium Constraints . Cambridge University Press, 1996.

Dougal Maclaurin, David Duvenaud, and Ryan Adams. Gradient-based hyperparameter optimization through reversible learning. In Proc. International Conference on Machine Learning , Lille, France, 2015.

Akshay Mehra and Jihun Hamm. Penalty method for inversion-free deep bilevel optimization. In Asian Conference on Machine Learning , virtual, 2021.

Maher Nouiehed, Maziar Sanjabi, Tianjian Huang, Jason D Lee, and Meisam Razaviyayn. Solving a class of non-convex min-max games using iterative first order methods. In Proc. Advances in Neural Information Processing Systems , Vancouver, Canada, 2019.

Samet Oymak and Mahdi Soltanolkotabi. Overparameterized nonlinear learning: Gradient descent takes the shortest path? In Proc. International Conference on Machine Learning , Long Beach, CA, 2019.

Barak A Pearlmutter. Fast exact multiplication by the hessian. Neural computation , 6(1): 147-160, 1994.

Fabian Pedregosa. Hyperparameter optimization with approximate gradient. In Proc. International Conference on Machine Learning , New York City, NY, 2016.

Quang Pham, Chenghao Liu, Doyen Sahoo, and HOI Steven. Contextual transformation networks for online continual learning. In Proc. International Conference on Learning Representations , virtual, 2021.

Shoham Sabach and Shimrit Shtern. A first order method for solving convex bilevel optimization problems. SIAM Journal on Optimization , 27(2):640-660, 2017.

Han Shen, Quan Xiao, and Tianyi Chen. On penalty-based bilevel gradient descent method. arXiv preprint:2302.05185 , 2023.

Daouda Sow, Kaiyi Ji, Ziwei Guan, and Yingbin Liang. A constrained optimization approach to bilevel optimization with multiple inner minima. arXiv preprint arXiv:2203.01123 , 2022.

Bradly Stadie, Lunjun Zhang, and Jimmy Ba. Learning intrinsic rewards as a bi-level optimization problem. In Conference on Uncertainty in Artificial Intelligence , virtual, 2020.

Gilbert W Stewart. On the perturbation of pseudo-inverses, projections and linear least squares problems. SIAM review , 19(4):634-662, 1977.

Ju Sun, Qing Qu, and John Wright. A geometric analysis of phase retrieval. Foundations of Computational Mathematics , 18:1131-1198, 2018.

Richard S Sutton and Andrew G Barto. Reinforcement Learning: An Introduction . MIT press, 2018.

Davoud Ataee Tarzanagh, Mingchen Li, Christos Thrampoulidis, and Samet Oymak. FEDNEST: Federated bilevel, minimax, and compositional optimization. In Proc. International Conference on Machine Learning , Baltimore, MD, 2022.

Ioannis Tsaknakis, Prashant Khanduri, and Mingyi Hong. An implicit gradient-type method for linearly constrained bilevel problems. In IEEE International Conference on Acoustics, Speech and Signal Processing , Singapore, 2022.

Paul Vicol, Jonathan P Lorraine, Fabian Pedregosa, David Duvenaud, and Roger B Grosse. On implicit bias in overparameterized bilevel optimization. In Proc. International Conference on Machine Learning , Baltimore, MD, 2022.

Quan Xiao, Han Shen, Wotao Yin, and Tianyi Chen. Alternating implicit projected sgd and its efficient variants for equality-constrained bilevel optimization. In Proc. International Conference on Artificial Intelligence and Statistics , Valencia, Spain, 2023.

Junjie Yang, Kaiyi Ji, and Yingbin Liang. Provably faster algorithms for bilevel optimization. In Proc. Advances in Neural Information Processing Systems , virtual, 2021.

Shuoguang Yang, Xuezhou Zhang, and Mengdi Wang. Decentralized gossip-based stochastic bilevel optimization over communication networks. In Proc. Advances in Neural Information Processing Systems , New Orleans, LA, 2022.

Jane J Ye. Constraint qualifications and optimality conditions in bilevel optimization. Bilevel Optimization: Advances and Next Challenges , pages 227-251, 2020.

Jane J Ye and Daoli Zhu. Optimality conditions for bilevel programming problems. Optimization , 33(1):9-27, 1995.

Jane J Ye and Daoli Zhu. New necessary optimality conditions for bilevel programs by combining the MPEC and value function approaches. SIAM Journal on Optimization , 20(4): 1885-1905, 2010.

Jane J Ye, Xiaoming Yuan, Shangzhi Zeng, and Jin Zhang. Difference of convex algorithms for bilevel programs with applications in hyperparameter selection. Mathematical Programming , pages 1-34, 2022.

Yihua Zhang, Guanhua Zhang, Prashant Khanduri, Mingyi Hong, Shiyu Chang, and Sijia Liu. Revisiting and advancing fast adversarial training through the lens of bi-level optimization. In Proc. International Conference on Machine Learning , Baltimore, MD, 2022.

Supplementary Material for 'A Generalized Alternating Method for Bilevel Optimization under the Polyak-Łojasiewicz Condition"

Table of Contents

A Auxiliary lemmas

We use ∥ · ∥ to be the Euclidean norm and X be the metric space with the Euclidean distance metric. Let x ∈ X be a point and A ⊂ X be a set, we define the distance of x and A as

The following lemma relates the PL condition to the error bound (EB) and the quadratic growth (QG) condition.

Lemma 7 ([33, Theorem 2]) . If g ( x, y ) is ℓ g, 1 -Lipschitz smooth and PL in y with µ g , then it satisfies the EB condition with µ g , i.e.

Moreover, it also satisfies the QG condition with µ g , i.e.

Conversely, if g ( x, y ) is ℓ g, 1 -Lipschitz smooth and satisfies EB with µ g , then it is PL in y with µ g /ℓ g, 1 .

The following lemma shows that the calmness condition ensures the KKT conditions hold at the global minimizer.

Proposition 1 ([11, Proposition 6.4.4]) . Let ( x ∗ , y ∗ ) be the global minimzer of a constrained optimization problem and assume that this problem is calm at ( x ∗ , y ∗ ) . Then the KKT conditions hold at ( x ∗ , y ∗ ) .

The following lemma gives the gradient of the value function.

Lemma 8 ( [51, Lemma A.5]). Under Assumption 2 and assuming g ( x, y ) is ℓ g, 1 - Lipschitz smooth, g ∗ ( x ) is differentiable with the gradient

Moreover, g ∗ ( x ) is L g -smooth with L g := ℓ g, 1 (1 + ℓ g, 1 / 2 µ g ) .

B Proof of Stationary Metric

B.1 Theorem 3 and its proof

Theorem 3. For nonconvex-PL BLO, A point ( x ∗ , y ∗ ) is a local (resp. global) optimal solution of (1) if and only if it is a local (resp. global) optimal solution of (3) . The same argument holds for (2) .

Proof. First, we prove the equivalence of (1) and reformulation (3). According to PL condition,

Therefore, if ∇ y g ( x, y ) = 0 , it holds that 0 ≥ g ( x, y ) -g ∗ ( x ) ≥ 0 which yields g ( x, y ) = g ∗ ( x ) . On the other hand, if g ( x, y ) = g ∗ ( x ) , the first order necessary condition says ∇ y g ( x, y ) = 0 . As a consequence, we know that

In this way, for any local (resp. global) optimal solution ( x ∗ , y ∗ ) of (3), there exists a neighborhood U such that for any ( x, y ) ∈ U with ∇ y g ( x, y ) = 0 , it holds that f ( x ∗ , y ∗ ) ≤ f ( x, y ) . Since ∇ y g ( x, y ) = 0 ⇔ y ∈ S ( x ) , we know ( x ∗ , y ∗ ) is also a local (resp. global) optimal solution of (2). The reverse holds true for the same reason.

The equivalence of Problem (1) and reformulation (2) can be proven in the same way.

B.2 Proof of Lemma 1

First, the gradient of g ( x, y ) can be calculated by

Since ∥ [1 , -cos( y 2 )] ⊤ ∥ ≥ 1 and g ∗ ( x ) = 0 , we know

Thus g ( x, y ) satisfies the PL condition with µ g = 1 .

Next, the LL problem in Example 1 is equal to x + y 1 -sin( y 2 ) = 0 so that the BLO of

is equivalent to

According to the constraint, we know y 1 -sin( y 2 ) = -x so that solving the BLO amounts to solving min { x 2 -x } with the constraint x + y 1 -sin( y 2 ) = 0 . Therefore, the global minimizers of BLO are those points ( x ∗ , y ∗ ) where x ∗ = 0 . 5 and y ∗ = ( y ∗ 1 , y ∗ 2 ) ⊤ with y ∗ 1 -sin( y ∗ 2 ) + 0 . 5 = 0 .

On the other hand, reformulating (24) using (2) yields

̸

If the calmness holds for (25), then according to Proposition 1, there exists w ∗ = 0 such that

However, for any w ∗ = 0 , g ∗ ( x ) = 0 and

̸

̸

which contradicts (26). Therefore, the calmness condition fails for (24) with reformulation (2).

B.3 Proof of Lemma 2

Figure 6 shows the landscape of g ( x, y ) when x = 1 and the red line denotes its solution set. The figure reveals that g ( x, y ) is nonconvex in y with multiple LL solutions, and the solution set is closed but nonconvex.

Proof. It is easy to see that S ( x ) = { y | x + y 1 = sin( y 2 ) } and g ∗ ( x ) = 0 .

First, the Slater condition is invalid because ∇ y g ( x, y ) = 0 is an equality constraint.

Next, we can compute the Hessian and Jacobian of g ( x, y ) as

Since ∇ 2 yx g ( x, y ) coincides with the first column of ∇ 2 yy g ( x, y ) , we know that

For any global minimizer ( x ∗ , y ∗ ) of the bilevel problem, it should satisfy the LL optimality, i.e. y ∗ ∈ S ( x ∗ ) , and thus x ∗ + y ∗ -sin( y ∗ ) = 0 . In this way, we have

1 2

̸

Thus, rank( ∇ 2 yy g ( x ∗ , y ∗ )) = 1 = 2 which contradicts the LICQ and MFCQ conditions.

Moreover, it follows that rank( ∇ 2 yy g ( x, y )) = 1 if and only if

Thus, the CRCQ condition is equivalent to that there exists a neighborhood of ( x ∗ , y ∗ ) such that either sin( y 2 ) = 0 or x + y 1 -sin( y 2 ) = 0 holds for any ( x, y ) in that neighborhood. We fix x ∗ and search such neighborhood for y first. It is clear that finding an area of y such that sin( y 2 ) ≡ 0 is impossible due to the nature of sin function. On the other hand, x ∗ + y 1 -sin( y 2 ) = 0 is equivalent to y ∈ S ( x ∗ ) so that making x ∗ + y 1 -sin( y 2 ) ≡ 0 in an neighborhood of y ∗ is also unrealistic since otherwise, the solution set S ( x ∗ ) should be open, which contradicts that S ( x ) is closed. Thus, (6) violates CRCQ condition.

B.4 Proof of Lemma 3

Assume the PL and the Lipschitz smoothness constants of g ( x, · ) are µ g and ℓ g, 1 , and the Lipschitz continuity constant of f ( x, · ) is ℓ f, 0 . Consider ∀ q , and ∀ ( x ′ , y ′ ) , s.t. ∇ y g ( x ′ , y ′ )+ q = 0 , then letting y q ∈ Proj S ( x ′ ) ( y ′ ) and according to Lemma 7, one has

Since ( x ∗ , y ∗ ) solves (3) and ( x ′ , y q ) is also feasible to (3), one has f ( x ∗ , y ∗ ) ≤ f ( x ′ , y q ) . Thus,

This justifies the calmness definition in (5) with M := ℓ f, 0 µ g .

B.5 Proof of Theorem 1

Proof. By applying Lemma 3 and Proposition 1, we know that KKT conditions are necessary, i.e.

On the other hand, since

C Proof of Lower-level Error

In this section, we give the proof of Lemma 4. To prove (17a) in Lemma 4, by the smoothness of g ( x, y ) and the update rule, we have

where (a) results from Assumption 2. Then subtracting g ∗ ( x k ) from both sides of (28), we get

By plugging in β ≤ 1 ℓ g, 1 and applying N times, we get the conclusion in (17a).

To prove (17b), we decompose the error by

To obtain the upper bound of J 1 , we first notice that g ( x, y k +1 ) -g ∗ ( x ) is ( ℓ g, 1 + L g ) -Lipschitz smooth over x according to the smoothness of g ( x, y k +1 ) in Assumption 1 and g ∗ ( x ) from Lemma 8. Therefore, we have

where (a) is derived from Lipschitz continuity of ∇ g ( x, y ) and Lemma 8, (b) comes from CauchySwartz inequality, (c) is due to Lemma 7. Plugging (30) into (29) yields the conclusion in (17b).

D Proof of Shadow Implicit Gradient Error

We first present some properties of L ( x, y, w ) and then prove Lemma 5. We define

D.1 Propositions of L ( x, y, w ) and L ∗ ( x, y )

Proposition 2. Under Assumption 1-2, L ( x, y, w ) is σ 2 g -PL and ℓ 2 g, 1 - Lipschitz smooth over w .

Proof. According to the definition of L ( x, y, w ) , we can calculate the Hessian of it as

Since ∇ ww L ( x, y, w ) ⪯ ℓ 2 g, 1 I , we know that L ( x, y.w ) is ℓ 2 g, 1 - Lipschitz smooth with respect to w .

On the other hand, L ( x, y, w ) is σ 2 g -PL since 1 2 ∥ Aw + b ∥ 2 is a strongly convex function composited with a linear function which is σ 2 min ( A ) -PL according to [33].

D.2 Proof of Lemma 5

According to Proposition 2 and following the same proof of Lemma (17a), we know that

where the last inequality results from L ( x k , y k +1 , w 0 ) = ∥∇ y f ( x k , y k +1 ) ∥ 2 / 2 ≤ ℓ f, 0 / 2 and L ∗ ( x k , y k +1 ) ≥ 0 . Then according to the EB condition, we know

On the other hand, since w k,t ∈ Ran( ∇ 2 yy g ( x k , y k +1 )) according to the update, then w k +1 ∈ Ran( ∇ 2 yy g ( x k , y k +1 )) and thus

As a result, the bias can be bounded by

E Proof of Upper-level Error

E.1 Propositions of w † ( x, y )

Before proving the UL descent, we first present some property of w † ( x, y ) .

Lemma 9 (Lipschitz continuity and boundedness of w † ( x, y ) ) . Under Assumption 1-2, for any x, x 1 , x 2 and y, y 1 , y 2 , it holds that

Proof. First, ( ∇ 2 yy g ( x, y ) ) † is bounded since according to Assumption 2,

On the other hand, according to [60, Theorem 3.3],

which says ( ∇ 2 yy g ( x, y ) ) † is ( √ 2 ℓ g, 2 /σ 2 g ) - Lipschitz continuous.

By the definition (32), w † ( x, y ) = -( ∇ 2 yy g ( x, y ) ) † ∇ y f ( x, y ) . Therefore, the boundedness of w † ( x, y ) is given by

Besides, for any x 1 , x 2 and y 1 , y 2 , we have

where (a) is due to C 1 D 1 -C 2 D 2 = C 1 ( D 1 -D 2 ) + D 2 ( C 1 -C 2 ) and Cauchy-Schwartz inequality and (b) comes from Lemma 9.

E.2 Proof of Lemma 6

We aim to prove the descent of upper-level by

For ease of use, we derive the gradient of F ( x, y ; w † ( x, y )) below.

Lemma 10. Under Assumption 1-2, F ( x, y ; w ) is ( ℓ f, 1 + ℓ g, 2 ∥ w ∥ ) -smooth over x and y .

Proof. This is implied by the Lipschitz smoothness of f ( x, y ) and ∇ y g ( x, y ) .

Lemma 11. Under Assumption 1-2, we have

where η 2 is a constant that will be chosen in Theorem 2.

Proof. According to the definition of F ( x, y ) = f ( x, y ) + w ⊤ ∇ y g ( x, y ) , it holds that

where (a) is earned by Young's inequality and the Lipschitz continuity of w † ( x, y ) and ∇ y g , (b) is resulting from letting y ∗ = arg min y ∈ S ( x k ) ∥ y -y k ∥ and Lemma 7, that is

Lemma 12. Under Assumption 1-2, we have

where b k := ∥ w k +1 -w † ( x k , y k +1 ) ∥ , and the constant L F is defined as L F = ℓ f, 0 ( ℓ f, 1 + ℓ g, 2 ) /σ g .

Proof. Since F ( x, y ; w ) is ( ℓ f, 1 + ℓ g, 2 ∥ w ∥ ) Lipschitz smooth with respect to x , then according to Lemma 9, we have

where (a) is due to ⟨ A,B ⟩ = ∥ A ∥ 2 2 + ∥ B ∥ 2 2 -∥ A -B ∥ 2 2 , (b) results from

Rearranging terms yields the conclusion.

Lemma 13. Under Assumption 1-2, we have

Proof. Letting y ∗ = Proj ( y k,i ) , we first bound ∥ y k +1 -y k ∥ 2 by

where (a) comes from ∇ y g ( x k , y ∗ i ) = 0 , (b) is due to quadratic growth condition in Lemma 7, (c) is derived from the contraction in (17a).

Then, according to the definition of F ( x, y ; w ) = f ( x, y ) + w ⊤ ∇ y g ( x, y ) , it holds that

where (a) is resulting from (34) and (37).

Lemma 14. Under Assumption 1-2, we have

Proof. We can expand F ( x, y ; w ) with respect to y by

where (a) results from (37). Letting y ∗ = arg min y ∈ S ( x k ) ∥ y -y k ∥ , the bound of J 2 is derived by

where (a) is due to ∇ y g ( x k , y ∗ ) = 0 , (b) is due to the mean value theorem and there exists t ∈ [0 , 1] such that y ′ = ty k +(1 -t ) y ∗ , (c) is due to ∥ I -AA † ∥ ≤ 1 and

and (d) comes from w † ( x, y ) = -( ∇ 2 yy g ( x k , y k ) ) ∇ y f ( x, y ) such that

Here, I -∇ 2 yy g ( x k , y k ) ( ∇ 2 yy g ( x k , y k ) ) † is the orthogonal projector onto Ker( ∇ 2 yy g ( x k , y k )) .

Finally, according to (31), Lemma 6 can be proved in the following way.

Proof. Adding (33), (35), (36) and (38), we get

Then by choosing α ≤ min { 1 4 L F +8 L w ℓ g, 1 , βη 2 2 L 2 w } , we yield the conclusion.

F Proof of Overall Descent of Lyapunov Function

Proof. According to (17a), (17b) and (29), we know

Then since V k = F ( x k , y k ; w † ( x k , y k )) + c R y ( x k , y k ) , then adding (40) and (41) yields

where (a) comes from setting η 1 = 2 cℓ g, 1 , (b) is achieved by c ≥ max { 8 ℓ g, 2 ℓ f, 0 µ 2 g , 8 NL w ℓ 2 g, 1 µ 2 g } , (c) is gained by letting cα β ≤ µ 2 g 8 ℓ 2 g, 1 , (d) is achieved by

and (e) is earned by

In a word, a sufficient condition for (42) is

Then rearranging the terms and telescoping yield

where (a) comes from Lemma 5. Then by choosing T = log( K ) , we know that

Moreover, since ∇ w L ( x k , y k , w ) is ℓ 2 g, 1 -Lipschitz smooth over w according to Proposition 2 and

then it holds that

Therefore, averaging the left-hand-side and plugging in the gradient of L yield,

where the last inequality is due to (43) and the boundedness of ∥∇ w L ( x 0 , y 0 , w 0 ) ∥ 2 . Similarly, we can bound

where (a) comes from ∥ w k ∥ 2 ≤ 2 ∥ w † ( x k -1 , y k ) ∥ 2 +2 b 2 k -1 ≤ 2 ℓ 2 f, 0 σ 2 g + O (1 /K ) . Therefore, using (43), we know

G Additional Experiments and Details

Synthetic experiments. We test our algorithm GALET on Example 1 with different initialization. To see whether our stationary metric is a necessary condition of the optimality of BLO, we need to

define a value directly reflects whether the algorithm achieves the global optimality or not. By the explicit form of the solution set in (7), the global optimality gap can be measured by

where the first term correpsonds the distance to the global optimal x ∗ = 0 . 5 and the second term represents LL optimality. GALET achieves the global minimizers in (7) regardless of the initialization; see the top plot in Figure 1. The bottom plot in Figure 1 shows the stationary score of reformulation (2) and (3). If the KKT conditions of (2) hold, then it is equivalent to say the following

whose proof is attached below.

Proof. First, the KKT conditions of (2) can be written as there exists σ ∗ such that

(48c) implies y ∗ ∈ S ( x ∗ ) . For PL function g ( x, y ) , we know that if y ∗ ∈ S ( x ∗ ) , then ∇ y g ( x ∗ , y ∗ ) = 0 and according to Lemma 8, we have ∇ x g ( x ∗ , y ∗ ) -∇ x g ∗ ( x ∗ ) = 0 . Therefore, (48a) and (48b) are reduced to

which completes the proof.

Therefore, we can use the stationary score defined by ∥∇ x f ( x, y ) ∥ 2 + ∥∇ x f ( x, y ) ∥ 2 +( g ( x, y ) -g ∗ ( x )) to measure whether the point achieves the KKT point of (2). For reformulation (3), our stationary measure is simply defined as the summation of the residual in (8). For both initializations, the stationary score of the value function-based reformulation (2) does not reach 0 , whereas our measure based on the gradient-based reformulation (3) does. As the optimality gap of our algorithm converges to 0 , this implies the KKT condition for the value function-based reformulation (2) does not serve as a necessary condition for optimality. In contrast, our stationary metric does, thereby validating our theoretical findings in Section 2.

We also test the sensitivity of the parameters of our method and show the trajectories starting from the same initialization in Figure 4. We conduct experiments on different parameter and the search grid is: UL stepsize α ∈ { 0 . 1 , 0 . 3 , 0 . 5 , 0 . 7 , 0 . 9 } ; shadow implicit gradient loops and stepsizes T ∈ { 1 , 5 , 50 } , γ ∈ { 0 . 1 , 0 . 5 } and LL stepsizes and loops N ∈ { 1 , 5 } , β ∈ { 0 . 1 , 0 . 5 , 1 , 3 } . The default choice of parameter is α = 0 . 3 , K = 30 , β = 1 , N = 1 , γ = 0 . 1 , T = 1 . We observe that our algorithm is least sensitive to the shadow implicit gradient level parameters since all of the combinations of T and γ lead to almost the same trajectory. For the LL parameters β and N , our algorithm is also robust as long as the learning rate β is neither set too large ( β = 3 , divergence) nor too small ( β = 0 . 1 , slow convergence). The UL stepsize plays an important role on the convergence of GALET as different choices of α lead to distinct paths of GALET. The choices of α = 0 . 1 , 0 . 3 , 0 . 5 all lead to the convergence of GALET, while a slightly larger value ( α = 0 . 5 ) causes GALET to prioritize optimizing the UL variable x first.

Real-data experiments. We compare our method with the existing methods on the data hypercleaning task using the MNIST and the FashionMNIST dataset [21]. Data hyper-cleaning is to train a classifier in a corrupted setting where each label of training data is replaced by a random class number with a corruption rate p c that can generalize well to the unseen clean data. Let x be a vector being trained to label the noisy data and y be the model weight and bias, the objective function is given by

where CE denotes the cross entropy loss and σ denotes sigmoid function. We are given 5000 training data with corruption rate 0 . 5 , 5000 clean validation data and 10000 clean testing data. The existing

methods for nonconvex-strongly-convex BLO [21, 32] often adopts a single fully-connected layer with a regularization as the LL problem, but the regularization and the simple network structure always degenerate the model performance. As our algorithm is able to tackle the nonconvex LL problem with multiple solutions, we can consider more complex neural network structure such as two layer MLP without any regularization. Since training samples is fewer than the model parameters, the LL training problem is an overparameterized neural network so that satisfies the PL condition.

Algorithm implementation details. Instead of calculating Hessian matrix explicitly which is time-consuming, we compute the Hessian-vector product via efficient method [53]. Specifically, we can auto differentiate ∇ y g ( x, y ) ⊤ w with respect to x and y to obtain ∇ 2 xy g ( x, y ) w and ∇ 2 yy g ( x, y ) w . Also, let v = ∇ y f ( x, y ) + ∇ 2 yy g ( x, y ) w which detaches the dependency of v over x and y , the Hessian-vector product ∇ 2 yy g ( x, y ) v can be calculated by auto differentiation of ∇ y g ( x, y ) ⊤ v .

Parameter choices. The dimension of the hidden layer of MLP model is set as 50 . We select the stepsize from α ∈ { 1 , 10 , 50 , 100 , 200 , 500 } , γ ∈ { 0 . 1 , 0 . 3 , 0 . 5 , 0 . 8 } and β ∈ { 0 . 001 , 0 . 005 , 0 . 01 , 0 . 05 , 0 . 1 } , while the number of loops is chosen from T ∈ { 5 , 10 , 20 , 30 , 50 } and N ∈ { 5 , 10 , 30 , 50 , 80 } .

2.0

2.0

BOME

BOME

BOME

1.5

V-PBGD

V-PBGD

V-PBGD

IAPTT-GM

1.5

IAPTT-GM

1.5

IAPTT-GM

Test loss

GALET

Test loss

GALET

Test loss

GALET

1.0

1.0

1.0

0.5

0.5

0.5

0

500

1000

Iteration

1500

2000

2500

300(

0

1000

2000

Iteration

3000

4000

5000

0

1000

2000

Iteration

3000

4000

5000

(a) MNIST, linear model

The test loss v.s. iteration of different methods is shown in Figure 7, it can be seen that GALET converges fast in all of regimes. Moreover, we report the test accuracy of different methods in Table 3. It shows that GALET achieves comparable test accuracy with other nonconvex-nonconvex bilevel methods, which improves that of nonconvex-strongly-convex bilevel method RHG.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposed a generalized alternating method for nonconvex bilevel optimization with PL condition in lower level. Meanwhile, it provided convergence analysis for the proposed method under a new stationary metric. Some experimental results demonstrate efficiency of the proposed method.\n\nSTRENGTHS:\nThis paper proposed a generalized alternating method for nonconvex bilevel optimization with PL condition in lower level. Meanwhile, it provided convergence analysis for the proposed method under a new stationary metric. Some experimental results demonstrate efficiency of the proposed method.\n\nWEAKNESSES:\nIn the paper, some incomprehensible words make it difficult for me to judge whether the conclusion in the paper is correct ? \n\nFor example, clearly, the lines 276-277 of the paper (‘’ Note that $\\lambda_g>0$ allows $\\nabla^2_{yy}g(x,y)$ to have either negative eigenvalues or zero eigenvalues … .’’) are not incorrect. \n\nFrom Table 1, the authors said BOME [33] and MGBiO [24] only studied a singleton on $S(x)$. From [33] and [24], they studied the bilevel optimization problem:\n\n$$  \\min_{x\\in \\mathbb{R}^{d_x}} f(x,y^*), \\quad s.t., \\quad y^* \\in \\arg\\min_{y\\in \\mathbb{R}^{d_y}} g(x,y),$$\n\nwhich equals the bilevel problem (1) given in this paper. Thus, I think the BOME [33] and MGBiO [24] also studied non-singleton on $S(x)$.\n\nForm this paper, the authors point that they introduce and use a new stationary metric in convergence analysis. It is better? Please detail differences between it and other metrics used in [48,33,24,2,38].\n\nQUESTIONS:\n1)\tIn Assumption 2, since $\\lambda_g>0$, clearly, $\\nabla^2_{yy} g(x,y)$ only have positive eigenvalues ?\n\n2)\tIn the equality (11b), can we use \n$$\\nabla^2_{yy} g(x^k,y^{k+1})\\big(\\nabla_y f(x^k,y^{k+1})+\\nabla^2_{yy} g(x^k,y^{k+1})w^{k,t}\\big)$$ instead of \n$$\\nabla_y f(x^k,y^{k+1})+\\nabla^2_{yy} g(x^k,y^{k+1})w^{k,t}$$ ?\n\n3)\tIn the proposed GALET algorithm, since using the multi-step gradient descent to update variable $y$ and introduced variable $w$, why still compute Hessian matrix $\\nabla^2_{yy} g(x,y)$ and Jacobian matrix $\\nabla^2_{xy} g(x,y)$ ?\n\n4)\tIn Theorem 2, how to choose the learning rates $\\alpha$, $\\rho$ and $\\beta$ ? Please detail them.\n\n5)\tIn Theorem 2, why give $S=O(1)$ and $T=O(\\log(K))$ ? In other words, why $T$ is larger than $S$ ?\n\n\n\n---------------------------------------After Rebuttals ------------------------------------------------------------------\n\n\nThanks for authors' responses. However, the authors' responses did not address my main concern: Assumption 2 is very strict. \n\nAdding Assumption 1, i.e. $ \\nabla^2_g$ is Lipschitz continuity, the authors claim that Hessian matrix maybe includes zero eigenvalue (singular value). Clearly, $\\lambda_g$ is very small even approach zero. Thus, the gradient complexity is infinite, **Please the table given in Rebuttal by Authors to Reviewer swWK**. \n\nWhen Hessian matrix does not include zero eigenvalue (singular value), i.e. $\\lambda_g >0$, \nclearly, assumptions 2  and 1 are exactly strong convexity, defeating the whole purpose of the paper.\n\nThus I lower my score.\n\nLIMITATIONS:\nNot potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies an important problem of bilevel optimization with non-convex lower level function. The authors reformulate the problem as a constraint optimization and prove a constraint qualification (CQ) for the reformulated problem under mild assumptions, and show that other stronger CQ do not hold under the same assumptions. Next, authors introduce the stationarity measure for this problem and propose an algorithm which converges in this measure with minimax optimal rate under the given assumptions. Numerical simulations on synthetic and real data are provided to test the proposed algorithm and compare to other baselines.\n\nSTRENGTHS:\nThe paper is well written and easy to follow. The limitations of previous works are well explained and the goal of the paper is clearly stated. The key contribution of the paper is substantially relaxing the assumptions and proving improved iteration complexity for bilevel problems, which matches with the optimal complexity (even for single level problems) in terms of $\\epsilon$.\n\nWEAKNESSES:\n1. The mathematical clarity of some statements in the main part can be certainly improved. For instance, in Definition 2 (calmness) it is unclear to me if h(x, y) is a function or a map to $\\mathbb R^d$, is $q$ a vector or a scalar? I also assume that the statement should hold for any pair (x', y'), not only for one. Although one can eventually infer the answers to these questions from the proofs in the appendix, it is better to be more rigorous. \n\n2. In Figure 2, I do not understand what is the difference between the blue and the red boxes (Gradient-based PL-BLO reformulation). \n\n3. I suggest to remove the right plot from Figure 5. It does not really make sense to compare the actual convergence rate with the asymptotic theoretical sublunar rate. How the exact constant was chosen? Why GALET is actually faster than theoretically proven 1/K. \n\n4. The intuition for convergence of terms $\\mathcal R_x$ and $\\mathcal R_y$ are well explained before stating Theorem 2. However, why $\\mathcal R_w$ converges is not immediately unclear from (20). I suggest to clarify this in the revision.\n\nQUESTIONS:\n1. In the proof of Lemma 3, do we need smoothness of $f(x, \\cdot)$? It does not seem to be used in the proof. \n\n2. In theorem 1 (8b), it is unclear to me where the term $\\nabla_{yy}^2 g(x^*, y^*)$ comes from. Is it necessarily positive definite? It would be good to explain why this term is needed. Same question for $\\nabla_{yy}^2 g(x^k, y^{k+1})$ term in (11b). The authors claim \"the additional Hessian in (11b) is inevitable\". Why? \n\n3. Why do we necessarily have $\\lambda_g >0$? for any $x, y$ the Hessian might have negative eigenvalues for PL functions.\n\nLIMITATIONS:\nThe paper makes the fist steps in relaxing assumptions and proving the optimal rate for the fundamental setting of bilevel nonconvex optimization. Authors admit that the extension of the developed techniques to the stochastic setting might require different techniques. \n\n---\nAcknowledgement of the authors' rebuttal. \n\nI thank the authors for the rebuttal and addressing my concerns. It is a little disappointing that in the discussion phase it turned out that the lower level problem cannot be non-convex PL while it was one of the key claims of the paper (it's of course great that other reviewers could detect this during the review period). I believe the key results of this work are still valuable even if they merely apply for convex lower level problems and the condition on the eigenvalues does not seem very natural. I still recommend the acceptance of this work, but am less certain about the decision after the rebuttal/discussion.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies unconstrained bilevel optimization under the PL condition at the lower level. All previous work is limited to strongly-convex lower-level problems. The authors propose a new convergence criterion for this setting based on stationary-point seeking reformulation, supporting it with the calmness and constraint-qualification arguments. The algorithm is a slight variant of recent Hessian-inverse-free methods for strongly-convex lower-level cases. Convergence analysis is performed with standard smoothness assumptions in deterministic settings (with access to true gradients and Hessians). The algorithm is demonstrated via the MNIST data-hyperclearning experiment.\n\nSTRENGTHS:\nThe paper extends the existing literature to problems with PL lower level. Bilevel optimization is a fast-moving area, and this extension is a timely contribution to the literature. I haven't seen yet the PL condition considered in the context of Bilevel optimization (even though it appeared several times in min-max literature, and the underlying idea is not quite different). \n\n- The convergence criterion is well-supported by calmness and constraint-qualification arguments. \n\n- Presentation is clear and easy to follow.\n\nWEAKNESSES:\nOverall, the algorithmic idea is essentially the same as previously developed algorithms. \n\n- For instance, the work in [11] considered the same algorithm with strongly-convex lower-level objectives, to avoid Hessian-inverse estimation. The idea is essentially to solve a linear equation efficiently, which can be done in various ways (e.g., by running the GD on a quadratic function). This work does the same, the only difference is to force the initializer $w_0$ to be at 0, so that GD converges to the minimum-norm solution (the pseudo-inverse solution). However, the same argument does not seem to be applicable to SGD or non-zero initialization. Furthermore, this seems why Assumption 2 is required: lower-bound for non-zero singular values for all $y$. Given that the lower-level objective is already $\\mu$-PL, another assumption seems redundant. \n\n- The paper only considers deterministic cases, but most previous work can also handle stochastic gradients and Hessian oracles. Though this paper seems to be hard to be extended to stochastic analysis. \n\n- I think there are several technical issues that need more justification. Please see the questions below.\n\nQUESTIONS:\n- Can a PL function have multiple optimal solutions, and at the same time Hessian-Lipschitz? Because at suboptimal points that is a neighborhood of optimal points, the gradient-growth has to be at least $\\mu$. \n\nOtherwise, which one should be given up? Analysis might be intact if we change the condition to have a unique lower-level solution, but if we give up Hessian-Lipschitzness, I am a little not sure if everything still works. \n\n\n- In the inner loop, w_0 already has to start from 0 to ensure the minimum-norm solution. What can be said if the gradient is not exact or if $w_0$ starts from other points? \n\n\n- Can you at least have Assumption 2 only at the optimal points, and not on all y? \n\n- For (8b) in KKT condition, why not just $||\\nabla f(x^*, y^*) + \\nabla^2_{yy} g(x^*, y^*) w^*||^2 \\le \\epsilon$, instead of having additional $\\nabla^2_{yy} g(x^*,y^*)$?\n\nLIMITATIONS:\nThis paper is theoretical and has no concerns about societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the bilevel optimization problem, which may feature a non-convex lower-level problem with non-unique optimal function values. The lower-level problem can be transformed into a constraint optimization problem using function value or gradient-based constraints. The paper establishes necessary conditions via calmness to evaluate the optimality of bilevel optimization problems where the inner problem satisfies the PL condition. Based on these conditions, the authors develop the GALET method, which alternately updates the inner and outer variables. The effectiveness of GALET is demonstrated through an experiment involving hyper-data cleaning, which shows that GALET performs comparably to V-PBGD.\n\nSTRENGTHS:\nThe main strength of this paper is the proposal of a stationary metric to evaluate the optimality of BLO problems with non-convex lower-level problems. This metric enables to achieve optimal iteration complexity.\n\nWEAKNESSES:\nOverall, I have the following concerns and comments about this paper:\n1.  While formulation (2) is directly equivalent to BLO problem (1), the equivalence of the gradient-based formulation (3) only holds under PL conditions. As a result, it is unclear whether the proposed stationary condition is applicable in cases where the inner function of BLO does not satisfy the PL condition. Do you think it is feasible to extend the proposed stationary condition to the more general non-convex setting?\n\n2. As checked, it appears that the values of $\\beta$, $N$ $\\alpha$, and $\\eta_1$  are closely related in the analysis. However, in Theorem 2 and Lemma 4,  the authors merely simplify these parameters to the order of $\\mathcal{O}(1)$. As a result, the theoretical results do not elucidate the relationship between these parameters. It would be helpful if the authors could provide more details on how these parameters are related to each other in the main results.\n\n3. Figure 1 shows the stationary score of function value-based and gradient-based formulations (2) and (3) under different initializations. It is unclear which algorithm is adopted to solve the function value-based problems. It may affect the value of the KKT measure over iteration. Could you provide more information regarding the details of this result?\n\n4.  The main results presented by the authors focus solely on iteration complexity and do not provide information on the gradient oracle complexity or Hessian-vector complexity. Could you provide the complexity of the gradient and Hessian vector? It would be beneficial to include information on how these complexities depend on various Lipschitz constants and the PL constant $\\mu_g$. In addition, the strongly convex function is a special example of the PL function. It would be helpful to compare the results of GALET for strongly convex inner functions to those in the existing literature.\n\n5. The proposed method, GALET, is a second-order method, while the V-PBGD method is a fully first-order method. The authors have claimed complexity improvements over V-PBGD. But, from the numerical experiments, I did not observe any advantages over the first-order V-PBGD method, as expected.  It raised the question of whether the second-order information helped in your experiments. It would be helpful if you could provide additional evidence to clarify the performance of the experiments.\n\nQUESTIONS:\nAs mentioned in weaknesses.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-18"}