STOCHASTIC BILEVEL PROJECTION-FREE OPTIMIZATION *
Abstract
Bi-level optimization is a powerful framework to solve a rich class of problems such as hyper-parameter optimization, model-agnostic meta-learning, data distillation, and matrix completion.
The existing first-order solutions to bi-level problems exhibit scalability limitations (for example, in matrix completion) because of the requirement of projecting solutions onto the feasible set.
In this work, we propose a novel S tochastic B i-level F rank- W olfe (SBFW) algorithm to solve the stochastic bi-level optimization problems in a projection- free manner.
We utilize a momentum-based gradient tracker that results in a sample complexity of O ( ϵ − 3 ) for convex outer objectives with strongly convex inner objectives.
We formulate the matrix completion problem with denoising as a stochastic bilevel problem and show that SBFW outperforms the state-of-the-art methods for the problem of matrix completion with denoising and achieves improvements of up to 82% in terms of the wall-clock time required to achieve the same level of accuracy.
Introduction
We consider the two-level hierarchical optimization problem
[Equation]
Here, the outer problem involves minimizing the objective function F ( x , y ⋆ ( x )) :
=
E θ [ f ( y ⋆ ( x ); θ )] with respect to x over the convex compact constraint set X ⊂ R m .
Here y ⋆ ( x ) is a unique solution of the inner optimization problem, which for a given x , entails minimizing the strongly convex function G ( y , x ) :=
E ξ [ g ( y , x ; ξ )] with respect to optimization variable y .
The function F ( · ) and G ( · ) are the expected values of continuous and proper closed functions f : R m
→ R and g : R n × R m → R with respect to independent random variables θ ∈ Θ and ξ ∈ Ξ , respectively.
Observe that for bilevel problems of type (1), the inner and outer problems are interdependent and cannot be solved in isolation.
Yet, these problems arise in a number of areas, such as meta-learning (Rajeswaran et al. 2019), continual learning (Borsos, Mutny, and Krause 2020), reinforcement learning (Zhang et al. 2020a), and hyper-parameter optimization (Franceschi et al. 2018).
To solve such problems, the first-order stochastic approximation algorithms have been recently proposed (Yang, Ji, and Liang 2021;
Chen, Sun, and Yin 2021).
In some works, such as (Khanduri et al. 2021b), the constraint set X in the outer optimization problem is taken to be X
= R m , resulting in a simpler unconstrained outer optimization problem.
However, in applications such as meta-learning, personalized federated learning, and corsets (Borsos, Mutny, and Krause 2020), the constraint set X is a strict subset of X ⊂ R m .
The standard approach to deal with such constraint sets is to project the updates of the outer optimization problem onto X at every iteration.
Though popular and widely used, the projected gradient approaches may not necessarily be practical, for instance, in cases where the projection sub-problem is too expensive to be solved at every iteration.
The difficulties surrounding projection-based methods have motivated the development of projection-free algorithms that use the FrankWolfe (FW) updates.
These FW-based algorithms only require solving a linear program over X , which could be significantly cheaper than solving a non-linear projection problem, as in the case of ℓ 1 -norm or nuclear norm ball constraints.
Projection-free algorithms for single-level stochastic optimization algorithms are well-known, and state-ofthe-art algorithms achieve a sample complexity of O ( ϵ − 2 )
(Xie et al. 2020; Akhtar and Rajawat 2022).
These algorithms rely on a recursive gradient tracking approach that allows the samples to be processed sequentially and achieves variance reduction without the use of checkpoints or large batches.
Motivated by these developments, we ask the following question: “ Is it possible to develop efficient projection-free algo-
rithms for
bi
level stochastic optimization
problems? ” This work puts forth the S tochastic B i-level F rank- W olfe (SBFW) algorithm, which is the first projection-free algorithm for bi-level problems.
Our main contributions are:
• We propose a novel projection-free SBFW algorithm, utilizing the idea of momentum-based gradient update (Cutkosky and Orabona 2019) to track the gradient of the outer objective function.
The proposed algorithm is able
[Table]
to achieve a sample complexity of O ( ϵ
• We test the proposed algorithm on matrix completion and establish the efficacy of the proposed techniques compared to state-of-the-art algorithms (cf.
Sec. 4).
We achieve an improvement of up to 82% in the computation time for the proposed algorithm as compared to state-ofthe-art methods.
Related Works
A series of works proposed to solve the problem of the form (1) has appeared recently (Yang, Ji, and Liang 2021;
Chen, Sun, and Yin 2021; Khanduri et al. 2021b; Huang and Huang 2021).
The seminal works in (Ghadimi and Wang 2018; Yang, Ji, and Liang 2021) proposed a class of doubleloop approximation algorithms to iteratively approximate the stochastic gradient of the outer objective and incurred a sample complexity of O ( ϵ − 2 ) in order to achieve the ϵ -stationary point.
The double loop structure of these approaches made them impractical for large-scale problems; (Ghadimi and Wang 2018) required solving an inner optimization problem to a predefined accuracy, while (Yang,
Ji, and Liang 2021) required a large batch size of O ( ϵ − 1 ) at each iteration.
To address this issue, various single-loop methods involving simultaneous updates of inner and outer optimization variables have been developed (Chen, Sun, and
Yin 2021; Khanduri et al. 2021b;
Yang, Ji, and Liang 2021).
A single-loop two-time scale stochastic algorithm proposed in (Hong et al. 2020)
incurred a sub-optimal sample complexity of O ( ϵ − 2 . 5 ) .
This is further improved recently in (Chen, Sun, and Yin 2021;
Khanduri et al. 2021b;
Yang,
Ji, and Liang 2021), in which the authors have utilized the momentum-based variance reduction technique from (Cutkosky and Orabona 2019) to obtain optimal convergence rates.
While all of the above-mentioned works seek to solve (1), they all require a projection onto X at every iteration.
In this work, we are interested in developing projectionfree stochastic optimization algorithms for bi-level problems, which is still an open problem and the subject of the work in this paper.
A comprehensive list of all existing related works is provided in Table 1.
Motivating Example
In general, for noise-free data, the data matrix in the matrix completion problem can be modeled as a low-rank matrix motivating the use of the nuclear norm constraint.
Low-rank matrix completion problem arises in various applications such as image processing, multi-task learning, and collaborative filtering.
However, under noisy observations, directly solving the matrix completion problem with just the nuclear norm constraints can result in sub-optimal performance (McRae and Davenport 2021).
Further, noise is present in many vision applications, and using only low-rank priors is insufficient to recover the underlying matrix.
A common approach to tackle the noise is to apply a denoising algorithm as a pre-processing step.
In general, however, it is necessary to apply some heuristics since denoising algorithms require access to the full matrix, which is not available in the pre-processing stage.
Denoising is also impractical in online settings, where a random subset of the matrix entries is observed at every iteration.
The bilevel optimization framework provides a way out, allowing the incorporation of denoising steps within the inner-level sub-problem.
Mathematically, the bi-level matrix completion with denoising problem can be written as
[Equation]
where M ∈ R n × m is the given incomplete noisy matrix, ∥ V ∥ 1 := (cid:80) i,j | V i,j | is the sum-absolute-value ( ℓ 1 ) norm, λ
1 and λ 2 are regularization parameters, and Ω 1 and Ω 2 represents the set of available entries at outer and inner level respectively.
Note that the regularization over the discrepancy between X and denoised matrix Y results in bilevel formulation (2).
A similar technique in deterministic settings is utilized in various other applications in machine learning and signal processing problems (Crockett and Fessler 2021).
The problem in (2) is a special case of general formulation in (1) with f ( x , y ⋆ ) :
= (cid:80) i,j ( X i,j − Y i,j ) 2 with x :=
X , y ⋆ :=
Y and g :=
∥
X − V ∥ 2 F with y := V .
However, when the entries are revealed in the form of randomly selected subsets Ω t 1 ⊂ Ω 1 and Ω t 2 ⊂ Ω 2 at every iteration, it becomes stochastic in nature.
The main challenge here is due to the nuclear norm constraint, which makes it quite computationally expensive (sometimes even impractical) to solve (2) using projection-based bilevel algorithms.
In Sec. 4, we will show experimentally that the proposed algorithm SBFW is best suited to address such challenges.
Algorithm Development
We note that solving the bi-level optimization problem in (1) is NP-hard in general, but we restrict our focus to problems where the inner objective is continuously twice differentiable in ( x , y ) and also strongly convex w.r.t y with parameter µ g > 0 .
Such an assumption is common in the related works (Ghadimi and Wang 2018; Chen, Sun, and Yin 2021; Yang, Ji, and Liang 2021) and ensures that y ⋆ ( x ) is unique for any x ∈ X .
A stochastic projected gradient descent update to solve (1) can be written as
[Equation]
where we can write the biased estimate h ( x t , y t ; θ t , ξ t ) as
[Equation]
x t , y
[Equation]
where ∇ x f ( x t , y t ; θ t ) is an unbiased estimate of ∇ x F ( x t , y t ) , ∇ y f ( x t , y t ; θ t ) is an unbiased estimate of ∇ y F ( x t , y t ) , and M ( x t , y t ; ˜ ξ t ) is a biased estimate of product ∇ 2 xy G ( y t , x t ) · [ ∇ 2 yy G ( y t , x t )] − 1 .
Here, we have used ξ t in LHS to highlight the fact that the hessian of the function g ( · ) is also random in nature.
The term
M ( x t , y t ; ˜ ξ t ) is a biased estimation of [ ∇ 2 yy G ( y t , x t )] −with bounded variance.
The explicit form of M ( x t , y t ; ˜ ξ t ) is
[Equation]
here, ˜ ξ t is a collection of k + 1 i.i.d.
samples i.e. ˜ ξ t := { ξ t,i : i ∈ { 0 , 1 , · · · , k }} , with ξ t, 0 being the sample of ∇ 2 xy g ( y t , x t ) and ξ t, 1 , · · · , ξ t,k being the i.i.d.
samples of ∇ 2 yy g
( y t , x t ) .
Further, the parameter l is selected uniformly from { 1 , · · · , k } and for l =0 , we use the convention Π li =1 (cid:16) I − 1 L g ∇ 2 yy g
( x t , y t ; ξ t,i ) (cid:17)
=
I .
Here, I is the identity matrix, and L g is the Lipschitz parameter for gradient ∇ y g
( x , y ) .
Similar to the update in (3), a significant challenge that remains unaddressed to date in the literature for the bi-level problems is associated with the projection operator in (3).
The projection is easy to evaluate when the constraint set is a simple convex set (onto which projection operation is computationally cheap such as probability simplex) or has a closed-form solution (set of unit-ball).
However, the projection step is often computationally costly (e.g., nuclear norm constraint), and its complexity could be comparable to the problem at hand (Jaggi 2013).
In this work, we alleviate this issue by proposing projection-free algorithms for bi-level optimization problems, which is the key novel aspect of our work.
Stochastic Projection-Free Bi-level Optimization
Before proceeding, we discuss the particular choice of y t in (3).
A popular choice (see (Chen, Sun, and Yin 2021; Hong
Input: x 1 ∈ X , y 1 ∈ R m , η t , δ t , ρ t , β t , and d 1 = h 1 ( θ 1 ; ξ 1 ) using (4) 1 for t = 2 to T do
Update approximate inner optimization solution
[Equation]
Gradient tracking evaluate h ( x t , y t ; θ t , ξ t ) and h ( x t − 1 , y t − 1 ; θ t , ξ t ) using (4) and compute d t
[Equation]
Evaluate feasible direction
[Equation]
Update solution x
[Equation]
+1 =
(1 − η t ) x
[Equation]
η
[Equation]
4 Output: x T +1 or ˆ x selected uniformly from { x i } Ti = et al. 2020; Ji, Yang, and Liang 2020)) for y t is the stochastic gradient descent update for the inner optimization problem given by y t =
y t − 1 − δ t ∇ y g ( y t − 1 , x t − 1 ; ξ t ) , where ∇ y g ( y t − 1 , x t − 1 ; ξ t ) is the unbiased estimate of the gradient E ξ [ ∇ y g
( y t − 1 , x t − 1 ; ξ )] , and δ t > 0 denotes the step size.
Now we are ready to propose the first projection-free algorithm for bi-level stochastic optimization problems.
We propose to use a conditional gradient method (CGM) based approach (Jaggi 2013; Hazan and Luo 2016) instead of calculating the projection in (3).
That is, we solve a linear minimization problem to find a feasible direction s t ∈ X for a given stochastic gradient direction h ( x t , y t ; θ t , ξ t ) , given by, s t :=
arg min s ∈X ⟨ s , h ( x t , y t ; θ t , ξ t ) ⟩ .
This reduces the optimization problem of evaluating the projection operator in (3) to solving a linear program which is easier to solve in practice.
Hence, the iterate in (3) gets modified to
[Equation]
where η t > 0 is the step size.
To this end, we would like to emphasize that naive use of h ( x t , y t ; θ t , ξ t ) in (6) for the evaluation of s t which is then used in (7) can result in the iterate divergence due to the non-vanishing variance of the gradient estimate (Mokhtari, Hassani, and Karbasi 2020).
The standard approach to deal with this issue is to use a biased gradient estimate with low variance instead of an unbiased one.
For example, a mini-batch approximation is proposed in (Hazan and Luo 2016; Reddi et al. 2016) with linearly increasing batch size with iteration index.
Such an approach runs into memory issues when utilized in practice.
To address the issue of memory and iterate divergence, we took motivation from the momentum-based approach in (Cutkosky and Orabona 2019) and propose to use the following gradient tracking scheme given by
[Equation]
We remark that such a tracking technique is recently utilized in (Khanduri et al. 2021b) for projection-based bi-level op- timization problems.
However, in this work, our focus lies in developing projection-free algorithms, and hence analysis is significantly different from (Khanduri et al. 2021b).
Our proposed algorithm is summarised in Algorithm 1.
Convergence Analysis: SBFW
We will start with the assumptions required to perform the analysis in this work that is similar to the assumptions considered in the existing literature (Hong et al. 2020; Khanduri et al. 2021a).
Assumption 1 For some σ 2 x > 0 , σ 2 y > 0 , σ 2 xy > 0 , and σ 2 g > 0 we have E [ {∇ x f ( x , y ) − ∇ x f ( x , y , θ ) } 2 ] ≤ σ 2 x ,
E [ {∇ y f
( x , y ) − ∇ y f
( x , y , θ ) } 2 ] ≤ σ 2 y ,
E [ {∇ 2 xy g
( x , y ) − ∇ 2 xy g
( x , y , θ ) } 2 ] ≤ σ 2 xy ,
E [ {∇ y g ( x , y ) − ∇ y g
x , y , ξ ) } 2 ] ≤ σ 2 g .
Assumption 2 For any given x ∈ X , the terms ∇ x f ( x , y ) , ∇ y f ( x , y ) , ∇ y g ( x , y ) , ∇ 2 xy g ( x , y ) and ∇ 2 yy g ( x , y ) are
Lipschitz continuous with respect to y with Lipschitz parameter L f x , L f y , L g , L g xy and L g yy , respectively.
Similarly, for any given y ∈ R n , the terms ∇ y f ( x , y ) , ∇ 2 xy g ( x , y ) and ∇ 2 yy g ( x , y ) are Lipschitz continuous with respect to x with positive constants L f y , L g xy and L g yy , respectively.
Note that for the sake of simplicity, here we slightly abused the notation and used the same constants.
Assumption 3 For all x ∈ X and y ∈ R n , it holds that E [ ∥∇ y f ( x , y ) ∥ ] ≤ C y and E [ ∥∇ 2 xy g
( x , y ) ∥ ] ≤ C xy for some for constants C y > 0 and C xy > 0 .
Assumption 4 The inner function g ( x , y ) is µ g -strongly convex in y for any x ∈ X .
We start the analysis by presenting intermediate Lemmas 12 and Corollary 1 which eventually leads to the main result of this section presented in Theorem 1.
Lemma 1 Consider the proposed Algorithm 1 and x t ∈ N + be the iterates generated by it, then for the algorithm parameter δ t ≤ min { 23 µ g , µ g
[Equation]
1+
[Equation]
2 g )
L 2 g } and step size η t , it holds that (i)the optimality gap of the lower level problem satisfies
[Equation]
(ii) Also, for the constant b 1 defined as b
[Equation] [Equation]
0 − 1) } , it holds that
[Equation]
The proof Lemma 1 is provided in Appendix 4.
Lemmaquantifies how close y t is from the optimal solution of the inner problem at x t and establishes the progress of the innerlevel update.
[Figure]
Lemma 2 Consider the proposed Algorithm 1 and x t ∈ N + be the iterates generated by it, then for the algorithm parameter δ t , ρ t , and η t , we have
[Equation]
where the bias B t is defined as B t :
= E [ h ( x t , y t ; θ t , ξ t )] − ∇ S ( x t , y t ) .
The proof of Lemma 2 is provided in Appendix 4, which is based on the proof of Lemma C.3 of (Khanduri et al. 2021a).
However, different from (Khanduri et al. 2021a), our proof bounds the term ∥ x t − x t − 1 ∥ 2 using the update equation x t +
=
(1 − η t ) x t + η t s t from Algorithm 1 along with compactness assumption of the domain X .
Lemma 2 describe the tracking error in the gradient approximation ∇ S ( x , y ) at point x t and y t .
The presence of (1 − ρ t ) 2 term in RHS of (11) shows that the variance of the tracking error reduces with iteration.
Next, we utilize Lemma 1-(2) to establish a bound on the gradient estimation error for ∇ Q as Corollary 1.
Corollary 1 For the proposed Algorithm 1, with δ t = 2 a 0 t q , where a 0 = min { 13 µ g ,
[Equation]
[Equation] [Equation]
and η t ≤ 2 ( t +1) 3 q/ 2 for 0 < q ≤ 1 , the gradient approximation error E ∥∇ Q ( x t ) − d t ∥ 2 converges to zero at the following rate
[Equation]
where
[Equation]
The proof of Corollary 1 is provided in Appendix 4.
The result in Corollary 1 is presented in general form and indicates that for properly chosen parameters q , the gradient approximation error in expectation decreases at each iteration and approaches zero asymptotically.
We will use this upper bound to prove the convergence of the proposed algorithm SBFW for different types of objective functions in
[Table]
the following theorem.
Note that in the analysis of Corollary 1 we have set β t ≤ C xy C y µ g ( t +1) q .
To satisfy this condition, the number of samples k at iteration t needed to approximate the Hessian inverse in (5) is k
= O (log((1 + t ) q )) .
Now we are ready to present the first main result of this work as Theorem 1.
Theorem 1 Consider the proposed Algorithm 1 with δ t = at 2 / 3 , where a 0 = min { 23 µ g ,
[Equation]
2 t +1 and k
= 2 L g 3 µ g (log(1 + t ))
, then the output is feasible x T +1 ∈ X and satisfies
E
[Equation]
here L Q =
( L fy + L ) C xy µ g
+
[Equation]
+
[Equation]
(cid:104)
[Equation]
g
[Equation]
g +
[Equation]
(cid:105) and C 1 = 3(max { 2 ∥
y 1 − y ⋆ ( x 1 ) ∥ 2 , (2( C xy /µ g ) 2 D 2 + 16 a 20 σ 2 g ) / (2 a 0 − 1) } + C xy C
y µ g + 8(
L k L 2 g + L k D 2 + σ 2 f )) .
The proof of Theorem 1 is provided in Appendix 4.
Theorem 1 shows that the optimality gap for SBFW decays as
O ( T − 1 / 3 ) for general convex objectives, where T is the total number of iterations.
We note that for at each iteration,
SBFW requires 2 k + 1 gradient samples to obtain gradient estimate: 2 k samples for outer gradient estimate (8) and one sample for the inner variable update.
Further, we have set k
≈ O (log( t )) .
Hence, the SFO complexity of SBFW for the outer objective is O (log( ϵ − 1 ) ϵ − 3 )
≈ O
( ϵ − 3 ) .
Similarly, observe that E [ ∥ y t − y ⋆ ( x t ) ∥ ] ≤ O (( t +1) − q ) , where q = 2 / 3 (as δ t
=
O ( t − 2 / 3 ) ).
Hence, the SFO complexity of the inner objective for SBFW is O ( ϵ − 1 . 5 ) .
It can be seen that complexity for the inner level objective of the proposed algorithm SBFW is comparable to the projection-based stateof-the-art methods (Ji, Yang, and Liang 2020; Chen, Sun, and Yin 2021;
Khanduri et al. 2021b); however, it shows slightly worse performance in terms of the outer level complexity.
This is not surprising as we are tackling the outer level in a projection-free manner.
Numerical Experiments
In this section, first, we consider the problem of low-rank matrix completion formulated in (2) to illustrate the performance of our proposed SBFW algorithm.
All the experiments are performed in MATLAB R2018a with Intel(R)
Core(TM) i7-8550U CPU @ 1.80GHz.
To test the scalability of our proposed projection-free algorithm, we run an experiment over large-size matrices of MovieLens 1 datasets, movielens/ which contain user ratings of movies ranging from 0 to 5 .
We start with Movielens 100 k dataset of 10 5 ratings fromusers for 1700 movies.
This dataset is denoted by observation matrix M of size 1000 × 1700 .
For the simulations, we define the set of observed entries Ω by sampling the matrix uniformly at random from M with a batch size of b = 500 .
Note that as SBFW is a single loop algorithm, we compare the performance of SBFW with other state-of-the-art single loops projection-based bilevel algorithms such as SUSTAIN (Khanduri et al. 2021b), TTSA (Hong et al. 2020), and MSTSA (Khanduri et al. 2021a) .
Fig.
1 plots the evolution of normalized error for 2500 iterations for all the algorithms.
We note (Fig. 1(left)) that the proposed algorithm is not the best in terms of the convergence rate when compared to projection-based schemes, which is expected from the slower theoretical convergence rates.
However, when compared in terms of the amount of clock time required to achieve the same level of normalized error ( 0 . 68 × 10 − 1 ), the proposed scheme outperforms the other state-of-the-art methods as shown in the bar plot of Fig. 1(left).
To further highlight the importance of the projection-free bilevel algorithm in practice, we perform additional experiments on a larger dataset (of MovieLens 1 M), which contains 1 million ratings from 6000 users and for 4000 movies.
We plot the evolution of normalized error with time in
Fig.
1(right), where we only compare SBFW against SUSTAIN, which is the state-of-the-art projection-based bilevel algorithm.
It is interesting to note that even though SUSTAIN has a better theoretical convergence rate, it shows inferior performance in actual computation time (due to the projection operation) compared to SBFW, as evident from
Fig. 1(right).
In Table 2, we provide computation time comparisons (to complete 10 3 iteration) of both the algorithms (under same settings) over different real datasets.
Observe that for large data sets, SBFW is approximately 10 × faster than the SUSTAIN and exhibits an improvement upto 82% in the computation time.
This performance gain in terms of computation time comes from the fact that other methods require performing projections over the nuclear norm at each iteration which is computationally expensive due to the computation of full singular value decomposition.
In contrast,
SBFW solves only a single linear program at each iteration, which only requires the computation of singular vectors corresponding to the highest singular value.
Conclusion
This paper presents the first projection-free algorithm for stochastic bi-level optimization problems with a strongly convex inner objective function.
We utilize the concept of momentum-based tracking to track the stochastic gradient estimate and establish the oracle complexities of the proposed SBFW algorithm for the convex outer objective functions.
Numerical results show that the proposed projectionfree variant has a significantly reduced wall-clock time as compared to its projection-based counterparts.
Appendix
We will start with deriving an upper bound on the expected estimation error when the momentum-based method is employed to track the function or gradient.
Lemma 3 Let us estimate function Ψ( x ) =
E ξ [Ψ( x , ξ )] by y t using step size δ t as follows
[Equation]
Then the expected tracking error E t [ ∥ y t − Ψ( x t ) ∥ 2 ] satisfies
[Equation]
Proof: Consider the update equation in (14), add/subtract the term (1 − δ t )Ψ( x t − 1 ) in the right hand side of (14) to obtain
[Equation]
Subtract Ψ( x t ) from both sides in (16) and take norm square:
[Equation]
Now, expand the square and calculate conditional expectation E t =
E
[( · ) |F t ] to obtain
E t [ ∥ y
[Equation]
Ψ
[Equation]
x
[Equation]
∥
[Equation]
− 2 ⟨
[Equation]
1 − δ t )
( y t −
[Equation]
x t − 1 )) , (1 −
[Equation]
t )( E t [Ψ
[Equation]
x t
[Equation]
− Ψ( x t − 1 , ξ t )])
+ E t [Ψ( x t ) − Ψ
( x t , ξ t )] ⟩ + E t ∥
[Equation]
1 − δ t )
(Ψ( x t − 1 ,
[Equation]
t )
[Equation]
Ψ( x t
[Equation] [Equation]
)+Ψ( x
[Equation] [Equation]
Ψ
[Equation]
x t
[Equation]
Note that E t [Ψ( x t − 1 ) − Ψ
( x t − 1 , ξ t )]
= 0 and E t
[Ψ( x t ) − Ψ
( x t , ξ t )]
= 0 , which implies that
[Equation]
where the last inequality holds due to the fact that
[Equation]
for any two random variables X and Y .
Before proceeding, we will discuss some of the existing results which are useful for the analysis in this paper.
Lemma 4 [(Ghadimi and Wang 2018), Lemma 2.2] Under
Assumption1, the following statements hold.
(a) For any x ∈ X and y ∈ R n
[Equation]
where L :=
L
[Equation]
and all the constants are as defined in Assumption1.
(b) The inner optimal solution y ⋆ ( x ) is C xy µ g -Lipschitz continuous in x , which implies that for any x 1 , x 2 ∈ X , it holds that
[Equation]
y
[Equation]
( x
− y
[Equation]
(c) The gradient of outer objective ∇ Q is L Q -Lipschitz continuous in x , which implies that for any x 1 , x 2 ∈ X , it holds that ∥ Q ( x 1 ) − Q ( x
∥ ≤ L Q ∥ x 2 − x 1 ∥ where
[Equation]
Lemma
[(Khanduri et al. 2021b), Lemma 4.1] Suppose
Assumption
holds, and the gradient estimate h ( x , y ; θ, ξ
is constructed with k number of samples using (4) , then (a) for any x ∈ X and y 1 , y 2 ∈ R n , we have
[Equation]
(b) for any y ∈ R n and x 1 , x 2 ∈ X , we have
[Equation]
C 2 g xy L 2 f y + C 2 f y L 2 g xy
[Equation]
+ k 2 C 2 g xy C 2 f y L 2 g yy ] µ g
[Equation]
2 L g −
[Equation]
Lemma
[Lemma 2 (Akhtar and Rajawat 2021)] Let ψ t be a sequence of real numbers which satisfy
[Equation]
for some r 1 ∈ (0 , 1] such that r 1 ≤ r 2 ≤ 2 r 1 , c 1 > 1 , and c 2 ≥ 0 .
Then, for c = max { ψ 1 ( t 0 + 1) r 2 − r 1 , c 2 c 1 − 1 } , ψ t + would converge to zero at the following rate
[Equation]
Lemma 7 Under Assumption 1, consider the estimator defined in (4) , then (i) define bias B t := E [ h ( x t , y t ; θ t , ξ t )] −∇ S ( x t , y t ) , it holds that we have,
[Equation]
where σ
[Equation]
= σ
[Equation]
(ii) For t ≥ 0 , it is possible select k (required to approximate the Hessian inverse in (5) )
such that ∥ B t ∥ ≤ β t where β t ≤ ct a for some constant c and a > 0 .
Proof: For proof of Lemma (7)(i) see [Lemma 11, (Hong et al. 2020)].
The proof Lemma (7)(ii) is straightforward.
From (25) we have β t = (cid:0) O
(1 − µ g /L g ) k (cid:1) .
Now on setting k = O ( log ( t ))
we can get the required condition as β t ≤ ct a .
It shows that with proper selection of k , we can make the bias to decay polynomially to zero.
Proof of Lemma
From the update step 2 of Algorithm 1, we can write
[Equation] [Equation]
By expanding the square and taking conditional expectation term inside the inner product terms, we obtain
[Equation]
here (28) comes from the fact that E t [ ∇ y g
[Equation]
, while (29) comes from using the strong convexity property of function g .
Now consider the last term E t ∥∇ y g ( x
[Equation]
− 1 , y t − 1 , ξ
[Equation]
) ∥ 2 of
[Equation] [Equation]
where we use the inequality ∥ a + b ∥ 2 ≤ 2 ∥ a ∥ 2 + 2 ∥ b ∥ 2 .
From Assumption 1, we can further upper bound (31) as
E t [ ∥∇ y g ( x t − 1 , y t − 1 ,
[Equation]
t
[Equation] [Equation]
[Equation]
where we used the fact that ∇ y g ( x t − 1 , y ⋆ ( x t − 1 ))
= 0 .
Substituting the upper bound in (33) in (30) we obtain
E t [ ∥ y
[Equation]
The last inequality in (35) is obtained by selecting δ t such that 2 δ t (1 + σ 2 g )
L 2 g ≤ µ g .
To proceed next, we use Young’s inequality to bound the term E t [ ∥ y t − y ⋆ ( x t ) ∥ 2 ] in (35) as
[Equation]
[Equation]
where the second inequality comes from Lemma 4(b)
[Equation]
and the last inequality comes from the update equation and the compactness of the domain X .
Utilizing (35) into (36)
[Equation]
we get
[Equation]
To proceed next
[Equation]
we substitute α = 2(1 − δ t µ g ) δ t µ g which also implies that (cid:0)
+ 1 α (cid:1) (1 − δ t µ g ) =
1 − µ g δ t 2 :
[Equation]
where the second inequality comes from the fact that 2 − δ t µ g δ t µ g < 2 δ t µ g while in the last inequality, we have assumed that δ t is chosen such that δ t ≤ 23 µ g giving 1 + 1 α ≤ 2 .
In
Corollary 1 we will see that our choice of step sizes satisfies these conditions.
To prove part (ii), we start with writing Lemma(i) for t =
t + 1 and set δ t = 2 a 0 t q where a 0 = min { 1 3 µ g , µ g 2(1+
[Equation]
2 g )
L 2 g } and η t =
( t +1) 3 q 2 , which gives
[Equation]
Note such selection of δ t ensures that the conditions 2 δ t (1+ σ 2 g )
L 2 g ≤ µ g and δ t ≤ 23 µ g required in Lemma 1 are satisfied.
Now taking full expectation and using Lemma 6 we get
[Equation]
where b 1 = max { 2 q ∥ y 1 − y ⋆ ( x 1 ) ∥ 2 , (2( C xy /µ g ) 2 D 2 +
[Equation]
σ
[Equation] [Equation]
− 1) } .
Proof of Lemma
Starting with update equation (8) and employing Lemmawe can write
[Equation]
here the last inequality is obtained using (26) and the fact that (1 − ρ 2 t ) ≤ 1 .
Now we introduce h ( x t , y t − 1 ; θ t , ξ t ) and bound the second term of RHS of (41) as
[Equation]
here (a) comes from simple norm property, (b) comes from
Lemma 5, (c) comes from the update equation while (d) comes from Assumption 1 and from the compactness of the set.
Using (42) in (41), we get the desired expression
[Equation]
Proof Corollary
We start with setting δ t =
[Equation]
in Lemma 2 to obtain
[Equation]
here the last inequality is obtained using the fact 1 / ( t + 1) 3 q ≤ 1 / ( t + 1) 2 p .
Application of Lemma 6 gives
[Equation]
where b 2 = max { 2 q ∥ d 1 − ∇ S ( x
[Equation]
L
[Equation] [Equation]
As, we have initialize
[Equation]
we can use the bound
[Equation]
we can bound the term E ∥∇ Q ( x t ) − d t ∥ 2 as follows
[Equation]
here second inequality comes from simple norm property, while the third inequality is obtained using Lemma (4)(a) on the first term, Lemma 7 on the second term, and (45) on the third term.
The last inequality comes from (40) and using β t ≤
[Equation]
g
[Equation]
t +1)
q :=
b 3 ( t +1)
q
and the constant C= 3( b 1 + b 2 + b 3 ) is defined as
[Equation]
C xy /µ g )
2 D 2 + 16 a 20 σ 2 g ) / (2 a 0 − 1) } + 8(
L k L 2 g + L k D 2 + σ 2 f ) + C xy
C
y µ g ) .
Proof of Theorem
From the initialization of variable x , we have x 1 ∈ X .
Also since we obtain s t solving a linear minimization problem over the set X , we have s t ∈ X .
Thus, x t +1 which is a convex combination of x t and s t , i.e. x t +1 =
(1 − η t +1 ) x t + η t
+
s t will also lie in the set X .
Hence x T +1 ∈ X and ˆ x ∈ X .
Now, starting with definition of Q ( · ) , we have Q ( x ) =
E θ [ f ( x , y ⋆ ( x ); θ )] .
Also note that we have set k = qL g µ g (log(1 + t )) , this ensures that the condition β t ≤ C xy C y µ g ( t +1) q required in the analysis of Corollary (1) is satisfied.
Hence, we can use results from Corollary (1) with q = 2 / 3 for convex case.
Using the smoothness assumption of Q we can write
[Equation]
48) where L Q =
( L f y + L ) C xy µ g +
[Equation]
(cid:105) (see Lemma 4).
Here, in the last expression we have replace term x t +1 − x t = η t ( s t − x t ) .
Now adding and subtracting η t ⟨ d t , s t − x t ⟩ in (48) we get
[Equation]
here in last the inequality is obtained using optimality of s t .
Now introducing η t ⟨ ∇ Q ( x t ) , x ⋆ − x t ⟩ in RHS of (49) and regrouping the terms we obtain
[Equation]
here in the second inequality we use bound η t ⟨ ∇ Q ( x t ) − d t , s t − x ⋆ ⟩
[Equation]
η t D ∥∇ Q ( x t ) − d t ∥ and in last inequality we used the bound ⟨ ∇ Q ( x t ) , x ⋆ − x t ⟩ ≤ Q ( x ⋆ ) − Q ( x t ) .
Subtracting
Q ( x ⋆ ) , taking expectation and using E
∥ X ∥ ≤ (cid:113) E ∥
X ∥we get
[Equation]
Further, setting q = 2 / 3 hence, η t = 2 t +1 and using Corollary 1, we can bound the second term of (51) η t D (cid:113) E ∥∇ Q ( x t ) − d t ∥ 2 ≤ 2 D √ C
[Equation]
t +1) 4 / 3 . which gives
[Equation]
( t + 1) 2 .
Multiplying both side by t ( t + 1) we can write t ( t + 1) E
[ Q
[Equation]
x
[Equation]
+
[Equation]
x ⋆ )] (53) ≤ t ( t − 1) E
[ Q
[Equation]
x t
[Equation]
+
[Equation]
+
[Equation]
≤ t ( t − 1) E
[ Q ( x t ) − Q ( x ⋆ )]
+ 2 D (cid:112) C 1 ( t + 1) 23 + 2 L Q D 2 ,
Summing for t = 1 , 2 , · · · , T and rearranging we get
E [ Q
[Equation]
x T +
[Equation]
−
[Equation]
here we use the fact that (cid:80) Tt =1 ( t + 1) 2 / 3 ≤ 35 ( T + 1) 5 / 3 .