Enhanced Bilevel Optimization via Bregman Distance
Abstract
Bilevel optimization has been recently used in many machine learning problems such as hyperparameter optimization, policy optimization, and meta learning.
Although many bilevel optimization methods have been proposed, they still suffer from the high computational complexities and do not consider the more general bilevel problems with nonsmooth regularization.
In the paper, thus, we propose a class of enhanced bilevel optimization methods with using Bregman distance to solve bilevel optimization problems, where the outer subproblem is nonconvex and possibly nonsmooth, and the inner subproblem is strongly convex.
Specifically, we propose a bilevel optimization method based on Bregman distance (BiO-BreD) to solve deterministic bilevel problems, which achieves a lower computational complexity than the best known results.
Meanwhile, we also propose a stochastic bilevel optimization method (SBiO-BreD) to solve stochastic bilevel problems based on stochastic approximated gradients and Bregman distance.
Moreover, we further propose an accelerated version of SBiO-BreD method (ASBiO-BreD) using the variance-reduced technique, which can achieve a lower computational complexity than the best known computational complexity with respect to condition number κ and target accuracy ϵ for finding an ϵ -stationary point.
We conduct data hyper-cleaning task and hyper-representation learning task to demonstrate that our new algorithms outperform related bilevel optimization approaches.
Introduction
Bilevel optimization can effectively solve the problems with a hierarchical structure, thus it recently has been widely used in many machine learning tasks such as hyper-parameter optimization [36, 19, 9, 37], meta learning [9, 30, 21], neural network architecture search [29], reinforcement learning [15] and image processing [30].
In the paper, we consider solving the following nonconvex-stronglyconvex bilevel optimization problem:
[Equation]
Inner
[Equation]
where function F ( x ) =
f
( x, y ∗ ( x )) :
X → R is smooth and possibly nonconvex, and function h ( x ) is convex and possibly nonsmooth, and function g ( x, y ) :
X × R d 2 → R is µ -strongly convex in y ∈ R d 2 .
The constraint set X ⊆ R
d 1 is compact and convex.
The Problem (1) covers a rich class of nonconvex objective functions with nonsmooth regularization, and is more general than the existing nonconvex bilevel optimization formulation in [11, 21] that does not consider any nonsmooth regularization.
Here the function h ( x ) can be the nonsmooth regularization term such as h ( x ) = λ ∥ x ∥ 1 .
[Table]
[Table] [Table]
STABLE [5]
[Table] [Table]
SUSTAIN
[Table]
[Table] [Table]
[Table] [Table]
[Equation]
O
[Equation]
O
[Equation]
√
Many recent machine learning research problems utilize the stochastic loss functions.
Thus, we also consider the following stochastic bilevel optimization problem:
[Equation]
Inner
[Equation]
where function F ( x ) =
E ξ (cid:2) F ( x ; ξ ) (cid:3) =
E ξ (cid:2) f
( x, y ∗ ( x ); ξ ) (cid:3) is smooth and possibly nonconvex, and function h ( x ) is convex and possibly nonsmooth, and function g ( x, y )
=
E ζ (cid:2) g ( x, y ; ζ ) (cid:3) :
X × R d 2 →
R is µ -strongly convex in y ∈ R d 2 .
ξ and ζ are random variables following unknown distributions D and D ′ , respectively.
Both Problem (1) and Problem (2) have been used in many machine learning tasks with a hierarchical structure, such as hyper-parameter meta-learning [9, 21] and neural network architecture search [29].
Many bilevel optimization methods recently have been developed to solve these problems.
For example, [11, 21] introduced a class of effective methods to solve the above deterministic Problem (1) and stochastic Problem (2) with h ( x ) = 0 .
However, these methods suffer from high computational complexity issue.
More recently, multiple accelerated methods were designed for stochastic Problem (2) with h ( x ) = 0 .
Specifically, [5, 22, 14, 40] proposed accelerated bilevel optimization algorithms via using the variance reduced techniques of SARAH/SPIDER/SNVRG [35, 8, 39, 42] and STORM [6].
However, these accelerated methods obtain a lower computational complexity without considering the condition number, which also accounts for an important part of the computational complexity (please see Tables 1 and 2).
Meanwhile, these accelerated methods only focus on the special case of the stochastic bilevel optimization Problem (2) with h ( x ) = 0 .
To fill in the gaps, in the paper, we propose a class of efficient bilevel optimization methods with lower computational complexity to solve the bilevel optimization Problems (1) and (2), where the outer subproblem is nonconvex and possibly nonsmooth, and the inner subproblem is strongly convex.
Specifically, we use the mirror decent iteration to update the variable x based on the Bregman distance.
Our main contributions are summarized as follows:
(i)
We propose a class of enhanced bilevel optimization methods for nonconvex bilevel optimization problems based on Bregman distance.
Moreover, we provide a comprehensive convergence analysis framework for our proposed bilevel methods.
(ii)
An efficient bilevel optimization method based on Bregman distances (BiO-BreD) is proposed to solve the deterministic bilevel Problem (1).
We prove that our BiO-BreD achieves a lower sample complexity than the best known result (please see Table 1).
(iii)
We introduce an efficient bilevel optimization method based on adaptive Bregman distances (SBiO-BreD) to solve the stochastic bilevel Problem (2).
Moreover, we design an accelerated version of SBiO-BreD algorithm (ASBiO-BreD) via using the variance reduced technique, which achieves a lower sample complexity than the best known result (please see Table 2).
Note that our methods can solve the constrained bilevel optimization with nonsmooth regularization but not rely on any form of constraint set and nonsmooth regularization.
In the other words, our methods can solve the unconstrained bilevel optimization without nonsmooth regularization studied in [11, 21].
Naturally, our convergence analysis can be applied to both the constrained bilevel optimization with nonsmooth regularization and the unconstrained bilevel optimization without nonsmooth regularization.
Related Works
In this section, we will revisit the existing bilevel optimization algorithms and Bregman distance based methods.
2. Bilevel Optimization Methods
Bilevel optimization recently has attracted increasing interest in many machine learning applications such as model-agnostic meta-learning, neural network architecture search, and policy optimization.
Thus, recently many algorithms [9, 11, 15, 33, 34, 21, 27] have been proposed to solve the bilevel optimization problems.
Specifically, [11] proposed a class of approximation methods for bilevel optimization and studied convergence properties of the proposed methods under convexity assumption.
[33, 34] developed the gradient-based descent aggregation methods for convex bilevel optimization.
[36] presented a nonlinear primal–dual algorithm for nonsmooth convex bilevel optimization in parameter learning problems.
In parallel, [15] introduced a two-timescale stochastic algorithm framework for nonconvex stochastic bilevel optimization in reinforcement learning.
Multiple accelerated bilevel approximation methods were developed later.
Specifically, [21] proposed faster bilevel optimization methods based on the approximated implicit differentiation (AID) and iterative differentiation (ITD), respectively.
[5, 22, 14, 40] presented several accelerated bilevel methods for the stochastic bilevel problems using variance-reduced techniques.
More recently, [17] proposed a class of efficient adaptive methods for nonconvex-strongly-convex bilevel optimization problems.
At the same time, the lower bound of bilevel optimization methods has been studied in [20] for these nonconvex-strongly-convex bilevel optimization problems.
In addition, [33, 26, 31, 32] designed a class of value-function-based and gradient-based bilevel methods for nonconvex bilevel optimization problems and studied asymptotic convergence properties of these methods.
[37] analyzed a class of special nonconvex nonsmooth bilevel optimization methods for selecting the best hyperparameter value for the nonsmooth ℓ p regularization with 0 < p ≤ 1 .
2.2 Bregman Distance-Based Methods
Bregman distance-based method ( a.k.a , mirror descent method) [4, 1] is a powerful optimization tool because it uses the Bregman distance to fit the geometry of optimization problems.
Bregman distance was first proposed in [2], and later extended in [3].
[4] introduced the first proximal minimization algorithm with Bregman function.
[1] studied the mirror descent for convex optimization.
[7] presented an effective variant of mirror descent, i.e. composite objective mirror descent, for regularized convex optimization.
More recently, [25] integrated the variance reduced technique to the mirror descent algorithm for stochastic convex optimization.
[41] studied the convergence properties of mirror descent algorithm for solving nonsmooth nonconvex problems.
The variance-reduced adaptive stochastic mirror descent algorithm [28] was proposed to solve the nonsmooth nonconvex finite-sum optimization.
Preliminaries 3.1 Notations
Let I d denote a d -dimensional identity matrix.
U{ 1 , 2 , · · · , K } denotes a uniform distribution over a discrete set { 1 , 2 , · · · , K } .
∥ · ∥ denotes the ℓ 2 -norm for vectors and spectral norm for matrices, respectively.
For two vectors x and y , ⟨ x, y ⟩ denotes their inner product.
∇ x f ( x, y ) and ∇ y f ( x, y ) are the partial derivatives w.r.t.
variables x and y .
Given the mini-batch samples B = { ξ i } bi =1 , we define ∇ f ( x ; B ) = 1 b (cid:80) bi =1 ∇ f ( x ; ξ i ) .
For two sequences { a n , b n } ni =1 , a n = O ( b n ) denotes that a n ≤ Cb n for some constant C > 0 .
The notation ˜ O ( · ) hides logarithmic terms.
Given a convex closed set X , we define a projection operation P X ( x 0 ) =
arg min x ∈X ∥ x − x 0 ∥ 2 .
∂h ( x ) is the subgradient set of function h ( x ) .
3.2 Some Mild Assumptions
Assumption 1.
Function F ( x ) =
f ( x, y ∗ ( x )) is possibly nonconvex w.r.t.
x , and function g ( x, y ) is µ -strongly convex w.r.t.
y .
For stochastic case, the same assumptions hold for f ( x, y ∗ ( x ); ξ ) and g ( x, y ; ζ ) , respectively.
Assumption 2.
Functions f
( x, y ) and g ( x, y ) satisfy 1) ∥∇ y f
( x, y ) ∥ ≤ C fy and ∥∇ 2 xy g ( x, y )
∥ ≤ C gxy for any x ∈ X and y ∈ R d 2 ; 2)
The partial derivatives ∇ x f ( x, y ) , ∇ y f ( x, y ) , ∇ x g ( x, y ) and ∇ y g ( x, y ) are L-Lipschitz, e.g., for x, x 1 , x 2 ∈ X and y, y 1 , y
2 ∈ R d 2 , ∥∇ x f ( x 1 , y ) − ∇ x f ( x 2 , y )
∥
≤ L ∥ x 1 − x 2 ∥ , ∥∇ x f ( x, y 1 ) − ∇ x f ( x, y 2 ) ∥ ≤ L ∥
y 1 − y 2 ∥ .
For stochastic case, the same assumptions hold for f ( x, y ; ξ ) and g ( x, y ; ζ ) for any ξ and ζ .
Assumption 3.
The partial derivatives ∇ 2 xy g ( x, y ) and ∇ 2 yy g ( x, y ) are L gxy -Lipschitz and L gyyLipschitz, e.g., for all x, x 1 , x 2 ∈ X and y, y 1 , y 2 ∈ R d∥∇
xy g ( x 1 , y ) − ∇
xy
g
( x 2 , y ) ∥ ≤ L gxy ∥ x 1 − x 2 ∥ , ∥∇ 2 xy g ( x, y 1 ) − ∇ 2 xy g ( x, y 2 )
∥ ≤ L gxy ∥
y 1 − y 2 ∥ .
For stochastic case, the same assumptions hold for ∇ 2 xy g ( x, y ; ζ ) and ∇ 2 y g ( x, y ; ζ ) for any ζ .
Assumption 4.
Function h
( x ) for any x ∈ X is convex but possibly nonsmooth.
Assumption 5.
Function Φ( x ) =
F ( x ) + h ( x ) is bounded below, i.e., Φ ∗ =
inf x ∈X Φ( x ) >
−∞ .
Assumptions 1-3 are commonly used in bilevel optimization methods [11, 21, 22].
According to
Assumption 1, ∥ f ( x, y 1 ) − f ( x, y 2 ) ∥
= ∥∇ y f
( x, y τ )( y 1 − y 2 ) ∥ ≤ ∥∇ y f ( x, y τ ) ∥∥ y 1 − y 2 ∥ ≤
C fy ∥
y 1 − y 2 ∥ , where y τ = τy 1 + (1 − τ ) y 2 and τ ∈ [0 , 1] .
Thus ∥∇ y f
( x, y ) ∥ ≤ C fy is similar to the assumption that the function f is M -Lipschitz in [21].
From the proofs in [21], we can find that they still use the norm bounded partial derivative ∥∇ y f
( x, y ) ∥ ≤ M .
Similarly, according to Assumption 1, we have ∥∇ y g
x 1 , y ) − ∇ y g
( x 2 , y ) ∥ ≤ L ∥ x 1 − x 2 ∥ .
Since ∥∇ y g ( x 1 , y ) − ∇ y g ( x 2 , y ) ∥
=
∥∇ 2 xy g ( x τ ′ , y )
( x 1 − x 2 ) ∥ ≤ ∥∇ 2 xy g
( x τ ′ , y )
∥∥ x 1 − x 2 ∥ ≤ C gxy ∥ x 1 − x 2 ∥ , where x τ ′ =
τ ′ x 1 + (1 − τ ′ )
x 2 and τ ′ ∈ [0 , 1] , we can let C gxy =
L as in [21].
From the proofs in [21], we can find that they still use the norm bounded partial derivative ∥∇ 2 xy g ( x, y )
∥ ≤ L for all x, y .
Throughout the paper, we let C gxy =
L .
Assumption 4 is generally used for regularization such as h ( x ) =
∥ x ∥ 1 .
Assumption 5 ensures the feasibility of the Problems (1) and (2).
When we use the first-order methods to solve the above bilevel optimization Problems (1) and (2), we can easily obtain the partial (stochastic) derivative ∇ y g
( x, y ) or ∇ y g ( x, y ; ζ ) to update variable y .
However, it is hard to get the (stochastic) gradient ∇ F ( x ) =
∂f ( x,y ∗ ( x )) ∂x
[Equation]
, when there is no closed form solution for the inner problem of Problems (1) and (2).
Thus, a key point of solving the Problems (1) and (2) is to estimate the gradient ∇ F ( x ) .
The following lemma provides one gradient estimator of ∇ F ( x ) .
Lemma 1.
(Lemma 2.1 in [11])
Under the above Assumptions (1, 2, 3), we have, for any x ∈ X
[Equation]
Lemma 1 provides a natural estimator of ∇ F ( x ) , defined as, for all x ∈ X , y ∈ R d
[Equation]
Next, we show some properties of ∇ F ( x ) , y ∗ ( x ) and ¯ ∇ f ( x, y ) in the following lemma:
∇ g
Compute partial derivative w t =
∂f ( x t ,y Kt ) ∂x via backpropagation w.r.t x t ; 9:
Given a ρ strongly convex mirror function ψ t ; 10:
Update x t
1 = arg x
⟨ w t ,
+ h
+ 1 γ D ψ t x, x t )
; 11: end for 12: Output: Uniformly randomly choose from { t
Tt =1 .
Lemma 2.
(Lemma 2.2 in [11])
Under the Assumptions (1, 2, 3), for all x, x 1 , x 2 ∈ X and y ∈ R d 2 , we have ∥ ¯ ∇ f ( x, y ) − ∇ F ( x ) ∥
≤ L y ∥
y ∗ ( x ) − y ∥
∥
y ∗ ( x 1 ) − y ∗ ( x 2 ) ∥ ≤ κ ∥ x 1 − x 2 ∥ , ∥∇ F ( x 1 ) − ∇ F ( x 2 ) ∥
≤ L F ∥ x 1 − x 2 ∥ , where L y =
L + L 2 µ + C fy L gxy µ + L gyy C fy L µ, κ = Lµ , and L F = L + 2 L 2 + L gxy C 2 fy µ +
L gyy C fy L + L 3 + L gxy C fy L µ+ L gyy C fy L 2 µ.
Bilevel Optimization via Bregman Distance Methods
In this section, we propose a class of enhanced bilevel optimization methods based on Bregman distance to solve the deterministic Problem (1) and the stochastic Problem (2), respectively.
4. Deterministic BiO-BreD Algorithm
In this subsection, we propose an efficient deterministic bilevel optimization method via Bregman distances (BiO-BreD) to solve the deterministic Problem (1).
Algorithm 1 summarizes the algorithmic framework of our BiO-BreD method.
Given a ρ -strongly convex and continuously-differentiable function ψ ( x ) , i.e. , ⟨ x 1 − x 2 , ∇ ψ ( x 1 ) − ∇ ψ ( x 2 ) ⟩ ≥ ρ ∥ x 1 − x 2 ∥ 2 , we define a Bregman distance [3, 4] for any x 1 , x 2 ∈ X :
[Equation]
In Algorithm 1, we use the mirror descent iteration to update the variable x at t + 1 -th step:
[Equation]
(5) where γ > 0 is stepsize, and w t is an estimator of ∇ F ( x t ) .
Here the mirror function ψ t can be dynamic as the algorithm is running.
Let ψ t ( x ) = 12 ∥ x ∥ 2 , we have D ψ t ( x, x t ) =
12 ∥
x − x t ∥ 2 .
When X = R d 1 , the above subproblem (5) is equivalent to the proximal gradient descent.
When
X ⊆
R d 1 and h ( x ) = 0 , the above subproblem (5) is equivalent to the projection gradient descent.
Let ψ t ( x ) = 12 x T H t x , we have D ψ t ( x, x t )
= 12 ( x − x t ) T H t ( x − x t ) .
When H t is an approximated
Hessian matrix, the above subproblem (5) is equivalent to the proximal quasi-Newton decent.
When
H t is an adaptive matrix as used in [18], the above subproblem (5) is equivalent to the proximal adaptive gradient decent.
In Algorithm 1, we use gradient estimator w t =
∂f ( x t ,y Kt ) ∂x to estimate ∇ F ( x t ) , where the partial derivative w t =
∂f ( x t ,y Kt ) ∂x is obtained by the backpropagation w.r.t.
x t .
4. SBiO-BreD Algorithm
In this subsection, we introduce an efficient stochastic bilevel optimization method via Bregman distance (SBiO-BreD) to solve the stochastic bilevel optimization Problem (2).
Algorithm 2 describes the algorithmic framework of our SBiO-BreD method.
compute
Given K ≥ 1 and draw K + 1 independent samples ¯ ξ = { ξ, ζ 0 , · · · , ζ K − 1 } , as in [15, 22], we definite a stochastic gradient estimator:
[Equation]
where k ∼ U{ 0 , 1 , · · · , K − 1 } is a uniform random variable independent on ¯ ξ .
It is easy to verify that ¯ ∇ f ( x, y, ¯ ξ ) is a biased estimator of ¯ ∇ f ( x, y ) ,
i.e. E ¯ ξ (cid:2) ¯ ∇ f ( x, y ; ¯ ξ ) (cid:3) ̸
= ¯ ∇ f ( x, y ) .
For the gradient estimator (6), thus we define a bias R ( x, y )
=
¯ ∇ f ( x, y )
− E
¯ ξ (cid:2) ¯ ∇ f ( x, y ; ¯ ξ ) (cid:3) :
X × R d 2 → R .
Lemma 3.
( Lemma 11 in [15] )
Under the about Assumptions (1, 2, 3), for any K ≥ 1 , the gradient estimator in (6) satisfies
[Equation]
Lemma 5 shows that the bias R ( x, y ) decays exponentially fast with number K , and with choosing
K =
Lµ log( LC fy T/µ ) , we have ∥ R ( x, y )
∥ ≤ 1 T .
Let LC fy µ (cid:0) 1 − µL (cid:1) K ≤ 1 T , we have K log(1 − µL )
≤ log( µ LC fy T ) .
Due to µ < L , we have K ≥ log( C fy LTµ ) / log( LL − µ ) .
Further due to µL ≤ log( LL − µ ) , let K
=
Lµ log( LC fy T/µ ) , we have ∥ R ( x, y )
∥ ≤ 1 T .
Note that here we use C gxy = L .
To simplify notations, let ¯ ξ it = { ξ t,i , ζ 0 t,i · · · , ζ K − 1 t,i } .
In Algorithm 2, we use mini-batch stochastic gradient estimator w t = ¯ ∇ f ( x t , y t ; ¯ B t ) = 1 b (cid:80) bi =1 ¯ ∇ f ( x t , y t ; ¯ ξ it ) , where ¯ ∇ f ( x t , y t ; ¯ ξ it ) =
∇ x f ( x t , y t ; ξ t,i ) − ∇ 2 xy g
( x t , y t ; ζ 0 t,i ) (cid:20) K
L
[Equation]
(cid:89)
j =
[Equation]
with k ∼ U{ 0 , 1 , · · · , K − 1 } .
Let R ( x t , y t ) =
w t − ¯ ∇ f ( x t , y t ) =
¯ ∇ f ( x t , y t ; ¯ B t ) − ¯ ∇ f ( x t , y t ) , we have E [ ¯ ∇ f ( x t , y t ; ¯ B t )]
= R ( x t , y t )
+ ¯ ∇ f ( x t , y t ) .
According to the above Lemma 5, it is easy to verify that ∥ R ( x t , y t ) ∥ ≤ LC fy µ (cid:0) 1 − µL (cid:1) K .
4. ASBiO-BreD Algorithm
In this subsection, we propose an accelerated version of SBiO-BreD method (ASBiO-BreD) to solve the stochastic bilevel optimization Problem (2) via using variance reduced technique of
SARAH/SPIDER/SNVRG [35, 8, 39, 42].
Algorithm 3 shows the algorithmic framework of the
ASBiO-BreD method.
In Algorithm 3, we use the variance reduced technique of SPIDER to accelerate SBiO-BreD algorithm.
When mod ( t, q ) = 0 , we draw a relative large batch samples B t = { ζ it } bi =1 and ¯ B t = { ¯ ξ it } bi = to estimate our stochastic partial derivatives v t and w t , respectively.
When mod ( t, q ) ̸ = 0 , we draw a mini-batch samples I t = { ξ it } b 1 i =1 and ¯ I t = { ¯ ξ it } b 1 i =1 ( b > b 1 ) to estimate v t and w t , respectively.
Let R ( x t , y t ) =
¯ ∇ f ( x t , y t ; ¯ I t ) − ¯ ∇ f ( x t , y t ) when mod ( t, q ) ̸ = 0 , we have
E [ ¯ ∇ f ( x t , y t ; ¯ I t )]
= R ( x t , y t )
+ ¯ ∇ f ( x t , y t ) and ∥ R ( x t , y t )
∥ ≤ LC fy µ (cid:0) 1 − µL (cid:1) K .
1: Input: T, K ≥ 1 , q , stepsizes γ > 0 , λ > 0 , { η t } Tt =1 , mini-batch sizes b and b 1 ; 2: initialize: x 0 ∈ X and y 0 ∈ R d 2 ; 3: for t = 0 , 1 , · · · , T − 1 do 4: if mod ( t, q ) = 0 then 5:
Draw randomly b independent samples B t = { ζ it } bi =1 , and compute stochastic partial derivative v t = ∇ y g
( x t , y t ; B t ) ; 6:
Draw randomly b ( K +1) independent samples ¯ B t = { ξ t,i , ζ 0 t,i · · · , ζ K − 1 t,i } bi =1 , and compute stochastic partial derivative w t = ¯ ∇ f ( x t , y t ; ¯ B t ) ; 7: else 8:
Generate randomly b 1 independent samples I t = { ζ it } b 1 i =1 , and compute stochastic partial derivative v t = ∇ y g
( x t , y t ; I t ) − ∇ y g
( x t − 1 , y t − 1 ; I t )
+ v t − 1 ; 9:
Generate randomly b 1 ( K + 1) independent samples ¯ I t = { ξ t,i , ζ 0 t,i · · · , ζ K − 1 t,i } b 1 i =1 , and compute stochastic partial derivative w t = ¯ ∇ f ( x t , y t ; ¯ I t ) − ¯ ∇ f ( x t − 1 , y t − 1 ; ¯ I t ) + w t − 1 ; 10: end if 11:
Update y t
+1 =
y t − λη t v t ; 12:
Given a ρ -strongly convex mirror function ψ t ; 13:
Update x t +1 = arg
min
x
∈X (cid:8) ⟨ w t , x ⟩ + h ( x ) + 1 γ D ψ t ( x, x t ) (cid:9) ; 14: end for 15: Output: Uniformly and randomly choose from { x t , y t } Tt =1 .
Convergence Analysis
In this section, we study the convergence properties of our new algorithms ( i.e. , BiO-BreD, SBiOBreD, and ASBiO-BreD) under mild conditions.
All proofs are provided in the Appendi B.
We begin with introducing a useful convergence metric ∥G t ∥ 2 or E ∥G t ∥ 2 to measure convergence properties of our algorithms.
Given the generated parameters x t at iteration t in our algorithms, as in [10, 28], we define the generalized gradient at iteration t as:
[Equation]
where F ( x ) =
f ( x, y ∗ ( x )) or F ( x ) =
E ξ [ f ( x, y ∗ ( x ); ξ )] .
When ψ t ( x ) = 12 ∥ x ∥ 2 , X = R d
and h ( x ) =
c is a constant, we have ∥G t ∥ 2 = ∥∇ F ( x t ) ∥ 2 , which is a common convergence metric used in [11, 21].
When ψ ( x ) = 12 ∥ x ∥ 2 , X ⊆
R
d 1 and h ( x ) =
c is a constant, our convergence metric ∥G t ∥ 2 = ∥ 1 γ ( x t − P X ( x t − γ ∇ F ( x t ))
∥ 2 which was used in [15].
Next, we provide some useful lemmas and some mild assumptions.
Lemma 4.
(Lemma B.2 in [22])
Under the above Assumptions (1, 2, 3), stochastic gradient estimator ¯ ∇ f ( x, y ; ¯ ξ ) is L K -Lipschitz continuous, e.g., for x, x 1 , x 2 ∈ X and y ∈ R d 2 ,
E ¯ ξ ∥ ¯ ∇ f ( x 1 , y ; ¯ ξ ) − ¯ ∇ f ( x 2 , y ; ¯ [Equation] ) ∥ 2 ≤ L 2 K ∥ x 1 − x 2 ∥ 2 , where L 2 K = 2 L 2 + 6 L
[Equation]
Lemma 5.
Suppose the sequence { x t , y t } Tt =1 be generated from Algorithms 2 and 3.
Under the above assumptions, given 0 < η t ≤ 1 for all t ≥ 1 and 0 < λ ≤ 16 L , we have
[Equation]
The above lemma 5 basically follows the Lemma 28 of [16] used for minimax optimization.
Assumption 6.
The stochastic partial derivative ∇ y g
( x, y ; ζ ) satisfies E [ ∇ y g ( x, y ; ζ )]
= ∇ y g
( x, y ) and E ∥∇ y g ( x, y ; ζ ) − ∇ y g
( x, y ) ∥ 2 ≤ σ 2 .
The estimated stochastic partial derivative ¯ ∇ f ( x, y ; ¯ ξ ) defined in (6) satisfies E ¯ ξ (cid:2) ¯ ∇ f ( x, y ; ¯ ξ ) (cid:3)
=
¯ ∇ f ( x, y )
+ R ( x, y )
and E ¯ ξ ∥ ¯ ∇ f ( x, y ; ¯ ξ ) − ¯ ∇ f ( x, y ) −
R ( x, y ) ∥ 2 ≤ σ 2 .
Assumption 7.
The mirror functions { ψ t ( x ) } Tt =0 are ρ -strongly convex, where ρ > 0 .
Assumption 6 is commonly used in stochastic bilevel optimization methods [15, 22].
Assumption 7 shows that the constant ρ can be seen as a lower bound of the strong convexity of all the mirror functions ψ t ( x ) for all t ≥ 0 , which is widely used in mirror descent algorithms [28] and adaptive gradient algorithms [18].
5. Convergence Analysis of BiO-BreD Algorithm
In this subsection, we provide the convergence properties of our BiO-BreD algorithm.
Theorem 1.
Suppose the sequence { x t , y t } Tt =1 be generated from Algorithm 1.
Let 0 < γ ≤ 3 ρ 4 L F , 0 < λ < 1 L , K = log( T ) / log( 1 1 − λµ )
+ 1 and ∥ y 0 t − y ∗ ( x t )
∥ 2 ≤ ∆ for all t ≥ 0 , we have
[Equation]
where κ =
Lµ , L 1 =
L ( L + µ ) µ , L 2 = 2 C fy (
[Equation]
gxy
[Equation]
LL gyy
[Equation]
and L 3 = LC fy µ .
Remark 1.
Without loss of generality, let L ≥ 1 µ , λ = 12 L , γ = 3 ρ 4 L F and ρ = O ( L ) .
It is easy to verify that our BiO-BreD algorithm has a convergence rate of O (cid:0) κ 2 T (cid:1) .
Let κ 2 T
=
ϵ , we have
T = κ 2 ϵ − 1 .
Due to K = log( T ) / log( 1 1 − λµ ) + 1 , we choose K
= O ( κ log( 1 ϵ )) for finding ϵstationary point of the problem (1) , we need the gradient complexity: Gc ( f, ϵ ) =
2 T =
O ( κ 2 ϵ − 1 ) and Gc ( g, ϵ ) =
KT =
˜ O ( κ 3 ϵ − 1 ) , and the Jacobian-vector and Hessian-vector product complexities:
JV ( g, ϵ ) = KT =
˜ O ( κ 3 ϵ − 1 ) and HV ( g, ϵ ) =
KT =
˜ O ( κ 3 ϵ − 1 ) .
5. Convergence Analysis of SBiO-BreD Algorithm
In this subsection, we provide the convergence properties of our SBiO-BreD algorithm.
Theorem 2.
Suppose the sequence { x t , y t } Tt =1 be generated from Algorithm 2.
Let ∆ =
∥
y 0 − y ∗ ( x 0 ) ∥ 2 , K =
Lµ log( LC fy T µ ) , 0 < η = η t ≤ 1 , 0 < γ ≤ min (cid:0) 3 ρ 4 L F , 9 ηρµλ 800 κ 2 , ηµρλ
L 2 y (cid:1) and 0 < λ ≤ 16 L , we have
[Equation]
Remark 2.
Without loss of generality, let L ≥ 1 µ , λ = 16 L , γ = min (cid:0) 3 ρ 4 L F , 9 ηρµλ 800 κ 2 , ηµρλ
L 2 y (cid:1) and ρ = O ( L ) , we have γρ
= O ( 1 κ 3 ) It is easily verified that our SBiO-BreD algorithm has a convergence rate of O (cid:0) κ 3 T + κ 2 b (cid:1) .
Let κ 3 T
=
ϵ 2 and κ 2 b
= ϵ 2 , we have T = 2 κ 3 ϵ − 1 and b = 2 κ 2 ϵ − 1 .
Due to K =
Lµ log( LC fy T µ ) , we have K
= O ( κ log( κ 4 ϵ ))
=
˜ O
( κ ) .
For finding ϵ -stationary point of the problem (2) , we need the gradient complexity: Gc ( f, ϵ ) = 2 bT
=
κ
5 ϵ − 2 and Gc ( g, ϵ ) = bT
=
O ( κ 5 ϵ − 2 ) , and the Jacobian-vector and Hessian-vector product complexities: JV ( g, ϵ ) =
bT =
O ( κ 5 ϵ − 2 ) and
HV ( g, ϵ ) =
KbT =
˜ O ( κ 6 ϵ − 2 ) .
5.3 Convergence Analysis of ASBiO-BreD Algorithm
In this subsection, we provide the convergence properties of our ASBiO-BreD algorithm.
Theorem 3.
Suppose the sequence { x t , y t } Tt =1 be generated from Algorithm 3.
Let ∆ =
∥
y 0 − y ∗ ( x 0 ) ∥ 2 , b 1 = q , K =
Lµ log( LC fy T
[Equation]
) , 0 < η = η t ≤ 1 , 0 < γ ≤ min (cid:0) 3 ρ 38 L 2 K η , 3 ρ 4 L F ,
[Equation]
19 L 2 y , ρη 8 , 9 ρηµλ 400 κ 2 (cid:1) and 0 < λ ≤ min (cid:0) 16 L ,
[Equation] [Equation]
, we have
[Equation]
Remark 3.
Without loss of generality, let L ≥
[Equation]
, γ = min (cid:0) 3 ρ
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
400 κ 2 (cid:1) and ρ = O ( L ) , we have γρ
= O ( 1 κ 4 ) .
It is easily verified that our ASBiO-BreD algorithm has a convergence rate of O (cid:0) κ 4 T + κ 2 b (cid:1) .
Let κ 4 T =
ϵ 2 and κ 2 b
= ϵ 2 , we have
T = 2 κ 4 ϵ − 1 and b = 2 κ 2 ϵ − 1 .
Due to K =
Lµ log( LC fy T µ ) , we have K
= O ( κ log( κ 4 ϵ ))
=
˜ O
( κ ) .
Let b 1 = q
=
κϵ − 0 .
For finding ϵ -stationary point of the problem (2) , we need the gradient complexity: Gc ( f, ϵ ) =
( bTq + 2 b 1 T ) =
O ( κ 5 ϵ − 1 . 5 ) and Gc ( g, ϵ ) =
bTq + 2 b 1 T
=
O
( κ 5 ϵ − 1 . 5 ) , and the
Jacobian-vector and Hessian-vector product complexities: JV ( g, ϵ ) =
bTq + 2 b 1 T
=
O
( κ 5 ϵ − 1 . 5 ) and HV ( g, ϵ )
=
K ( bTq + 2 b 1 T ) =
˜ O ( κ 6 ϵ − 1 . 5 ) .
Numerical Experiments
In this section, we perform two tasks to demonstrate the efficiency of our algorithms: 1) data hyper-cleaning task [38] over the MNIST dataset [24]; 2) hyper-representation learning task over the Omniglot dataset.
In the experiment, we compare our algorithms ( i.e. , BiO-BreD, SBiO-BreD, and ASBiO-BreD) with the following bilevel optimization algorithms: reverse [9]/AID-BiO [11, 21],
AID-CG [12], AID-FP [12], stocBiO [21]), MRBO [20], VRBO [20], FSLA [27], SUSTAIN [22], and VR-saBiAdam [17].
All experiments are averaged over 5 runs and we use a server with AMD
EPYC 7763 64-Core CPU and 1 NVIDIA RTX A5000.
6.1 Data Hyper-cleaning
In this task, we perform data hyper-cleaning over the MNIST dataset [24].
The formulation of this problem is as follows:
[Equation]
where l ( · ) denotes the cross entropy loss, D T and D V are training and validation dataset, respectively.
Here λ = { λ i } i ∈D T are hyper-parameters and C ≥ 0 is a tuning parameter, σ ( · ) denotes the sigmoid function.
In experiment, we set C = 0 .
The dataset includes a training set and a validation set where each contains 5000 images.
A portion of the training data are corrupted by randomly changing their labels, and we denote the portion of corrupted images as ρ .
We use Bregman function ψ t ( x ) = 12 x T H t x to generate the Bregman distance in all our algorithms, where H t is the adaptive matrix as used in [18], i.e. the exponential moving average of the square
[Table]
[Table]
of the gradient
and we use coefficient 0.99 in experiments.
For hyper-parameters, we perform grid search for our algorithms and other baselines to choose the best setting.
The detailed experimental setup is described in the Appendix A.
[Table]
The experimental results are summarized in Figure 1.
As shown by the figure, BiO-BreD outperforms the reverse algorithm; SBiO-BreD outperforms AID-FP/stocBiO and AID-CG methods, and ASBiOBreD outperforms the other SPIDER based algorithm MRBO and several momentum-based variance reduction methods: MRBO, SUSTAIN, FSLA, and VR-saBiAdam.
6.2 Hyper-representation Learning
In this task, we perform the hyper-representation learning task over the Omniglot dataset [23].
The formulation of this problem is as follows:
[Equation]
where l ( · ) denotes the cross entropy loss, D T ,ξ and D V ,ξ are training and validation dataset for a randomly sampled meta task.
Here λ = { λ i } i ∈D T are hyper-representations and C ≥ 0 is a tuning parameter to guarantee the inner problem to be strongly convex.
In the experiment, we set α = 0 .and C = 0 .
The detailed experimental setup is described in the Appendix A.2.
The results are summarized in
Table 3 and 4.
From these results, our AsBiO-BreD algorithm outperforms other baselines in the non-smooth case.
We also consider the smooth case, where the upper level problem is not added the L 1 regularization w.r.t the outer variable.
The results without L 1 regularization are given in the
Appendix A.2.
Conclusions
In the paper, we proposed a class of enhanced bilevel optimization methods based on the Bregman distance to solve the nonconvex-strongly-convex bilevel optimization problems possibly with nonsmooth regularization.
Moreover, we provided a comprehensive theoretical analysis framework to analyze our methods.
The theoretical results show that our methods outperform the best known computational complexities with respect to the condition number κ and the target accuracy ϵ for finding an ϵ -stationary point.
Acknowledgments and Disclosure of Funding
This work was partially supported by NSF IIS 1838627, 1837956, 1956002, 2211492, CNS 2213701, CCF 2217003, DBI 2225775. FH was a postdoctoral researcher at the University of Pittsburgh. FH and HH are the corresponding authors.