A Projection-free Algorithm for Constrained Stochastic Multi-level Composition Optimization
Abstract
We propose a projection-free conditional gradient-type algorithm for smooth stochastic multi-level composition optimization, where the objective function is a nested composition of T functions and the constraint set is a closed convex set.
Our algorithm assumes access to noisy evaluations of the functions and their gradients, through a stochastic first-order oracle satisfying certain standard unbiasedness and second-moment assumptions.
We show that the number of calls to the stochastic first-order oracle and the linear-minimization oracle required by the proposed algorithm, to obtain an ϵ -stationary solution, are of order O T ( ϵ − 2 ) and O T ( ϵ − 3 ) respectively, where O T hides constants in T .
Notably, the dependence of these complexity bounds on ϵ and T are separate in the sense that changing one does not impact the dependence of the bounds on the other.
For the case of T = 1 , we also provide a high-probability convergence result that depends poly-logarithmically on the inverse confidence level.
Moreover, our algorithm is parameter-free and does not require any (increasing) order of mini-batches to converge unlike the common practice in the analysis of stochastic conditional gradient-type algorithms.
Introduction
We study projection-free algorithms for solving the following stochastic multi-level composition optimization problem
[Equation]
(1) where f i : R d i → R d i − 1 , i = 1 , · · · , T ( d 0 = 1) are continuously differentiable functions, the composite function F is bounded below by F ⋆ > −∞ and X
⊂ R d is a closed convex set.
We assume that the exact function values and derivatives of f i ’s are not available.
In particular, we assume that f i ( y ) =
E ξ i [ G i ( y, ξ i )] for some random variable ξ i .
Our goal is to solve the above optimization problem, given access to noisy evaluations of ∇ f i ’s and f i ’s.
There are two main challenges to address in developing efficient projection-free algorithms for solving (1).
First, note that denoting the transpose of the Jacobian of f i by ∇ f i , the gradient of the objective function F ( x ) in (1), is given by ∇ F ( x )
= ∇ f T ( y T ) ∇ f T − 1 ( y T − 1 ) · · · ∇ f 1 ( y 1 ) , where y i =
f i +1 ◦ · · · ◦ f T ( x ) for 1 ≤ i < T , and y T
= x .
Because of the nested nature of the gradient ∇ F ( x ) , obtaining an unbiased gradient estimator in the stochastic first-order setting, with controlled moments, becomes non-trivial.
Using naive stochastic gradient estimators lead to oracle complexities that depend exponentially on T (in terms of the accuracy parameter).
Next, even when T = 1 in the stochastic setting, projection-free algorithms like the conditional gradient method or its sliding variants invariably require increasing order of mini-batches 1 [28, 40, 24, 39, 52], which make their practical implementation infeasible.
In this work, we propose a projection-free conditional gradient-type algorithm that achieves levelindependent oracle complexities (i.e., the dependence of the complexities on the target accuracy is independent of T ) using only one sample of ( ξ i ) 1 ≤ i ≤ T in each iteration, thereby addressing both of the above challenges.
Our algorithm uses moving-average based stochastic estimators of the gradient and function values, also used recently by [19] and [4], along with an inexact conditional gradient method used by [3] (which in turn is inspired by the sliding method by [28]).
In order to establish our oracle complexity results, we use a novel merit function based convergence analysis.
To the best of our knowledge, such an analysis technique is used for the first time in the context of analyzing stochastic conditional gradient-type algorithms.
1.1 Preliminaries and Main Contributions
We now introduce the technical preliminaries required to state and highlight the main contributions of this work.
Throughout this work, ∥·∥ denotes the Euclidean norm for vectors and the Frobenius norm for matrices.
We first describe the set of assumptions on the objective functions and the constraint set.
Assumption 1 (Constraint set) .
The set X ⊂ R d is convex and closed with max x,y ∈X ∥ x − y ∥
≤ D X .
Assumption 2 (Smoothness) .
All functions f 1 , . . .
, f T and their derivatives are Lipschitz continuous with Lipschitz constants L f i and L ∇ f i , respectively.
The above assumptions on the constraint set and the objective function are standard in the literature on stochastic optimization, and in particular in the analysis of conditional gradient algorithms and multi-level optimization; see, for example, [28], [50] and [4].
We emphasize here that the above smoothness assumptions are made only on the functions ( f i ) 1 ≤ i ≤ T and not on the stochastic functions ( G i ) 1 ≤ i ≤ T (which would be a stronger assumption).
Moreover, the Lipschitz continuity of f i ’s are implied by the Assumption 1 and assuming the functions are continuously differentiable.
However, for sake of illustration, we state both assumptions separately.
In addition to these assumptions, we also make unbiasedness and bounded-variance assumptions on the stochastic first-order oracle.
Due to its technical nature, we state the precise details later in Section 3 (see Assumption 3).
We next turn our attention to the convergence criterion that we use in this work to evaluate the performance of the proposed algorithm.
Gradient-based algorithms iteratively solve sub-problems in the form of
[Equation]
for some β > 0 , g ∈ R d and x ∈ X .
Denoting the optimal solution of the above problem by
P X ( x, g, β ) and noting its optimality condition, one can easily show that
[Equation]
where N X (¯ x ) is the normal cone to X at ¯ x and B (0 , r ) denotes a ball centered at the origin with radius r .
Thus, reducing the radius of the ball in the above relation will result in finding an approximate firstorder stationary point of the problem for non-convex constrained minimization problems.
Motivated by this fact, we can define the gradient mapping at point ¯ x ∈ X as
[Equation]
where Π X ( y ) denotes the Euclidean projection of the vector y onto the set X .
The gradient mapping is a classical measure has been widely used in the literature as a convergence criterion when solving nonconvex constrained problems [35].
It plays an analogous role of the gradient for constrained optimization problems; in fact when the set X ≡ R d the gradient mapping just reduces to ∇ F (¯ x ) .
It should be emphasized that while the gradient mapping cannot be computed in the stochastic setting,
it
[Table]
still serves as a measure of convergence.
Our main goal in this work is to find an ϵ -stationary solution to (1), in the sense described below.
Definition 1.
A point ¯ x ∈ X generated by an algorithm for solving (1) is called an ϵ -stationary point, if we have E [ ∥G X (¯ x, ∇ F (¯ x ) , β ) ∥ 2 ] ≤ ϵ , where the expectation is taken over all the randomness involved in the problem.
In the literature on conditional gradient methods for the nonconvex setting, the so-called Frank-Wolfe gap is also widely used to provide convergence analysis.
The Frank-Wolfe Gap is defined formally as
[Equation]
As pointed out by [3], the gradient mapping criterion and the Frank-Wolfe gap are related to each other in the following sense.
Proposition 1.
[3] Let g X ( · ) be the Frank-Wolfe gap defined in (4) and G X ( · ) be the gradient mapping defined in (3) .
Then, we have ∥G X ( x, ∇ F ( x ) , β ) ∥ 2 ≤ g X ( x, ∇ F ( x )) , ∀ x ∈ X .
Moreover, under Assumption 1, 2, we have g X ( x, ∇ F ( x )) ≤ (cid:104) (1 /β ) (cid:81)
Ti =
L f i + D X (cid:105)
∥G
X ( x
[Equation]
F (
[Equation]
β ) ∥ .
For stochastic conditional gradient-type algorithms, the oracle complexity is measured in terms of number of calls to the Stochastic First-order Oracle (SFO) and the Linear Minimization Oracle (LMO) used to the solve the sub-problems (that are of the form of minimizing a linear function over the convex feasible set) arising in the algorithm.
In this work, we hence measure the number of calls to SFO and LMO required by the proposed algorithm to obtain an ϵ -stationary solution in the sense of Definition 1.
We now highlight our main contributions :
• We propose a projection-free conditional gradient-type method (Algorithm 1) for solving (1).
In Theorem 2, we show that the SFO and LMO complexities of this algorithm, in order to achieve an ϵ -stationary solution in the sense of Definition 1, are of order O ( ϵ − 2 ) and O ( ϵ − 3 ) , respectively.
• The above SFO and LMO complexities are in particular level-independent
(i.e., the dependence of the complexities on the target accuracy is independent of T ).
The proposed algorithm is parameter-free and does not require any mini-batches, making it applicable for the online setting.
• When considering the case of T ≤ 2 , we present a simplified method (
Algorithm 3 and 4) to obtain the same oracle complexities.
Intriguingly, while this simplified method is still parameterfree for T = 1 , it is not when T = 2 (see Theorem 3 and Remark 3.1).
Furthermore, for the case of T = 1 , we also establish high-probability bounds (see Theorem 5).
A summary of oracle complexities for stochastic conditional gradient-type algorithms is in Table 1.
1.2 Related Work
Conditional Gradient-Type Method.
The conditional gradient algorithm [15, 29], has had a renewed interest in the machine learning and optimization communities in the past decade; see [33,
26, 20, 27, 5, 17] for a partial list of recent works.
Considering the stochastic convex setup, [22, 24] provided expected oracle complexity results for the stochastic conditional gradient algorithm.
The complexities were further improved by a sliding procedure in [28], based on Nesterov’s acceleration method.
[40, 52, 24] considered variance reduced stochastic conditional gradient algorithms, and provided expected oracle complexities in the non-convex setting.
[39] analyzed the sliding algorithm in the non-convex setting and provided results for the gradient mapping criterion.
All of the above works require increasing orders of mini-batches to obtain their oracle complexity results .
[34] and [54] proposed a stochastic conditional gradient-type algorithm with moving-average gradient estimator for the convex and non-convex setting that uses only one-sample in each iteration.
However, [34] and [54] require several restrictive assumptions, which we explain next (focusing on [54] which considers the nonconvex case).
Specifically, [54] requires that the stochastic function
G 1 ( x, ξ 1 ) has uniformly bounded function value, gradient-norm, and Hessian spectral-norm, and the distribution of the random vector ξ 1 has an absolutely continuous density p such that the norm of the gradient of log p and spectral norm of the Hessian of log p has finite fourth and second-moments respectively.
In contrasts, we do not require such stringent assumptions.
Next, all of the above works focus only on the case of T = 1 .
[1] considered stochastic conditional gradient algorithm for solving (1), with T = 2 .
However, [1] also makes stringent assumptions: (i) the stochastic objective functions G 1 ( x, ξ 1 ) and G 2 ( x, ξ 1 ) themselves have Lipschitz gradients almost surely and (ii) for a given instance of random vectors ξ 1 and ξ 2 , one could query the oracle at the current and previous iterations, which makes the algorithm not to be truly online.
See Table 1 for a summary.
Stochastic Multi-level Composition Optimization.
Compositional optimization problems of the form in (1) have been considered as early as 1970s by [12].
Recently, there has been a renewed interest on this problem.
[13] and [10] considered a sample-average approximation approach for solving (1) and established several asymptotic results.
For the case of T = 2 , [48], [49] and [6] proposed and analyzed stochastic gradient descent-type algorithms in the smooth setting.
[9] and [11] considered the non-smooth setting and established oracle complexity results.
Furthermore, [25] proposed algorithms when the randomness between the two levels are not necessarily independent.
For the general case of T ≥ 1 , [50] proposed stochastic gradient descent-type algorithms and established oracle complexities established that depend exponentially on T and are hence sub-optimal.
Indeed, large deviation and Central Limit Theorem results established in [13, 10], respectively, show that in the sample-average approximation setting, the arg min of the problem in (1) based on n samples, converges at a level-independent rate (i.e., dependence of the convergence rate on the target accuracy is independent of T ) to the true minimizer, under suitable regularity conditions.
[19] proposed a single time-scale Nested Averaged Stochastic Approximation (NASA) algorithm and established optimal rates for the cases of T = 1 , 2 .
For the general case of T ≥ 1 , [4] proposed a linearized NASA algorithm and established level-independent and optimal convergence rates.
Concurrently, [43] considered the case when the function f
i are non-smooth and established asymptotic convergence results.
[53] also established non-asymptotic level-independent oracle complexities, however, under stronger assumptions than that in [4].
Firstly, they assumed that for a fixed batch of samples, one could query the oracle on different points, which is not suited for the general online stochastic optimization setup.
Next, they assume a much stronger mean-square
Lipschitz smoothness assumption on the individual functions f i and their gradients.
Finally, they required mini-batches sizes that depend exponentially on T , which makes their method impractical.
Concurrent to [4], level-independent rates were also obtained for unconstrained problems by [7], albeit, under the stronger assumption that the stochastic functions G i ( x, ξ i ) are Lipschitz, almost surely.
It is also worth mentioning that while some of the above papers considered constrained problems, the algorithms proposed and analyzed in the above works are not projection-free, which is the main focus of this work.
Methodology
In this section, we present our projection-free algorithm for solving problem (1).
The method generates three random sequences, namely, approximate solutions { x k } , average gradients { z k } , and average function values { u k } , defined on a certain probability space (Ω , F , P ) .
We let F k to be the σ -algebra generated by { x 0 , . . .
, x k , z 0 , . . .
, z k , u 01 , . . .
, u k 1 , . . .
, u 0 T , . . .
, u kT } .
The overall method is given in Algorithm 1.
In (7), the stochastic Jacobians J k +1 i ∈
R
d i × d i − 1 , and the product
Input: x
[Equation]
X , z 0 =
[Equation]
R
[Equation]
, u
[Equation]
R
[Equation]
, i
[Equation]
, T , β k > 0 , t k > 0 , τ k ∈ (0 , 1] , δ ≥ 0 .
for k = 0 ,
. , N do
[Equation]
i at ki
[Equation] [Equation]
8) end for
Output: ( x R , z R , u R 1 , · · · , u RT ) , where R is uniformly distributed over { 1 ,
. , N }
Input: ( x, z, β, M, δ )
Set w 0 = x . for t = 0 , 1 , 2 , . . .
, M do
Find v t ∈ X with a quantity δ ≥ 0 such that
[Equation]
Set w t +
=
(1 − µ t ) w t + µ t v t with µ t = min (cid:26) 1 , ⟨ β ( x − w t ) − z,v t −
w t ⟩ β ∥ v t − w t ∥(cid:27) .
end for
Output: w M (cid:81)
Ti =1 J k +1 T
+1 − i is calculated as J k +1 T
J k +1 T − 1 · · · J k + ∈ R d T × d 1 ≡ R d T × 1 .
In (8), we use the notation ⟨· , ·⟩ to represent both matrix-vector multiplication and vector-vector inner product.
There are two aspects of the algorithm that we highlight specifically: ( i )
In addition to estimating the gradient of
F , we also estimate a stochastic linear approximation of the inner functions
f i by a moving-average technique.
In the multi-level setting we consider, it helps us to avoid the accumulation of bias, when estimating the f i directly.
Linearization techniques were used in the stochastic optimization since the work of [42].
A similar approach was used in [4] in the context of projected-based methods for solving (1).
It is also worth mentioning that other linearization techniques have been used in [9] and [11] for estimating the stochastic inner function values for weakly convex two-level composition problems, and ( ii )
The ICG method given in Algorithm 2 is essentially applying deterministic conditional gradient method with the exact line search for solving the quadratic minimization subproblem in (2) with the estimated gradient z k in (7).
It was also used in [3] as a sub-routine and is motivated by the sliding approach of [28].
Main Results
In this section, we present our main result on the oracle complexity of Algorithm 1.
Before we proceed, we present our assumptions on the stochastic first-order oracle.
Assumption 3 (Stochastic First-Order Oracle) .
Denote u kT +1 ≡ x k .
For each k , u ki +1 being the input, the stochastic oracle outputs G k +
i ∈
R
d i and J k
+1 i such that given F k and for any i ∈ { 1 , . . .
, T }
[Equation]
+
i
[Equation]
F
[Equation] [Equation]
[Equation] [Equation]
(c) The outputs of the stochastic oracle at level i ,
G k +1 i and J
k
+
i , are independent.
The outputs of the stochastic oracle are independent between levels, i.e., {
G k +1 i } i
=1 ,...,T are independent and so are {
J k +1 i } i =1 ,...,T .
Parts (a) and (b) in Assumption 3 are standard unbiasedness and bounded variance assumptions on the stochastic gradient, common in the literature.
Part (c) is essential to establish the convergence results in the multi-level case.
Similar assumptions have been made, for example, in [50] and [4].
We also emphasize that unlike some prior works (see e.g., [54]), Assumption 3 allows the case of endogenous uncertainty, and we do not require the distribution of the random variables ( ξ i ) 1 ≤ i ≤ T to be independent of the distribution of the decision variables ( u i ) 1 ≤ i ≤ T .
Remark.
Under Assumption 2, and 3, we can immediately conclude that E [ ∥ J k +1 i ∥ 2 | F k ] =
E [ ∥ J k +
i
[Equation]
In the sequel, ˆ σ
[Equation]
i will be used to simplify the presentation.
We start with the merit function used in this work and its connection to the gradient mapping criterion.
Our proof leverages the following merit function:
[Equation]
) where α, { γ i } 1 ≤ i ≤ T are positive constants and
[Equation]
Compared to [4], we require the additional term ∥∇ F ( x ) − z ∥ 2 , which turns out to be essential in our proof due to the ICG routine.
The following proposition relates the merit function above to the gradient mapping.
Proposition 2.
Let G X ( · ) be the gradient mapping defined in (3) and η ( · , · ) be defined in (10) .
For any pair of ( x, z ) and β > 0 , we have ∥G X ( x, ∇ F ( x ) , β ) ∥ 2 ≤ − 4 βη ( x, z ) + 2 ∥∇ F ( x ) − z ∥ 2 .
Proof.
By expanding the square, and using the properties of projection operation, we have
[Equation]
Thus, we have η ( x, z ) ≤ − β 2 ∥
Π X ( x − 1 β z ) − x ∥ 2 .
The proof is completed immediately by noting that ∥G ( x, ∇
[Equation]
We now present out main result on the oracle complexity of Algorithm1.
Theorem 2.
Under Assumption 1, 2, 3, let { x k , z k , { u ki } 1 ≤ i ≤ T } k ≥ 0 be the sequence generated by
Algorithm 1 with N ≥ 1 and
[Equation]
where β is an arbitrary positive constant.
Provided that the merit function W α,γ ( x, z, u ) is defined as (9) with
[Equation]
we have,
[Equation] [Equation]
[Equation] [Equation]
where u T +1 = x, B ( β, σ 2 , L, D X , T, δ )
= 4ˆ σ 2 +32 βD 2 X
(1+ δ ) (cid:16) 35 + 5 L 2 ∇ F β 2 (cid:17) , and ˆ σ 2 is a constant depending on the parameters ( β, σ 2 , L, D X , T ) given in (42) .
The expectation is taken with respect to all random sequences generated by the method and an independent random integer number R uniformly distributed over { 1 , . . .
, N } .
That is to say, the number of calls to SFO and LMO to get an ϵ -stationary point is upper bounded by O T ( ϵ − 2 ) , O T ( ϵ − 3 ) respectively.
Remark.
The constant B ( β, σ 2 , L, D X , T, δ ) is O ( T ) given the definition of ˆ σ 2 and the value of γ j in (12) , which further implies that the total number of calls to SFO and LMO of Algorithm 1 for finding an ϵ -stationary point of (1) , are bounded by O ( T 2 ϵ − 2 )
=
O T ( ϵ − 2 ) and O ( T 3 ϵ − 3 ) =
O T ( ϵ − 3 ) respectively.
Furthermore, it is worth noting that this complexity bound for Algorithm 1 is obtained without any dependence of the parameter β k on Lipschitz constants due to the choice of arbitrary positive constant β in (11) , and τ k , t k depend only on the number of iterations N and k respectively.
This makes Algorithm 1 parameter-free and easy to implement.
Remark.
As discussed in Section 2, the ICG routine given in Algorithm 2 is a deterministic method with the estimated gradient z k in (7) .
The number of iterations, t k , required to run Algorithmis given by t k = ⌈ √ k ⌉ .
That is, we require more precise solutions for the ICG routine, only for later outer iterations.
Furthermore, due to the deterministic nature of the ICG routine, further advances in the analysis of deterministic conditional gradient methods under additional assumptions on the constraint set X (see, for example, [16, 18]) could be leveraged to improve the overall LMO complexity.
3.1 The special cases of T = 1 and T =
We now discuss several intriguing points regarding the choice of tuning parameter β , for the case of T = 2 , and the more standard case of T = 1 .
Specifically, the linearization technique used in Algorithm 1 turns out to be not necessary for the case of T = 2 and T = 1 to obtain similar rates.
However, without linearization, the choice of β is dependent on the problem parameters for
T = 2 .
Whereas it turns out to be independent of the problem parameters (similar to Algorithm 1 and
Theorem 2 which holds for all T ≥ 1 ) for T = 1 .
As the outer function value estimates (i.e., u k + sequence) are not required for the convergence analysis, we remove them in Algorithms 3 and 4.
Algorithm 3 NASA with Inexact Conditional Gradient Method ( NASA+ICG ) for T =
Replace Step 2 of Algorithm 1 with the following:
Update the average gradient z and the function value estimate u 2 respectively as
[Equation]
Algorithm 4 ASA with Inexact Conditional Gradient Method ( ASA+ICG ) for T =
Replace Step 2 of Algorithm 1 with the following:
Update the average gradient z as: z k +1 =
(1 − τ k )
z k + τ k J k + .
Theorem 3.
Let Assumptions 1, 2, 3 be satisfied by the optimization problem (1) .
Let C 1 , C 2 and Cbe some constants depending on the parameters ( β, σ 2 , L, D X , δ ) , as defined in (54) and (62) .
Let
[Equation]
k ≥ 1 , where N is the total number of iterations.
(a) Let T = 2 , and let { x k , z k , u k 2 } k ≥ 0 be the sequence generated by Algorithm 3 with
[Equation]
Then, we have ∀ N ≥ 1 ,
[Equation]
(b) Let T = 1 and let { x k , z k } k ≥ 0 be the sequence generated by Algorithm 4 with β k ≡ β > 0 .
Then, we have ∀ N ≥ 1 ,
[Equation]
All expectations are taken with respect to all random sequences generated by the respective algorithms and an independent random integer number R uniformly distributed over { 1 , . . .
, N } .
In both cases, the number of calls to SFO and LMO to get an ϵ -stationary point is upper bounded by O ( ϵ − 2 ) , O ( ϵ − 3 ) respectively.
Remark.
While we can obtain the same complexities without using the linear approximation of the inner function for T = 2 , it seems necessary to have a parameter-free algorithm as the choice of β in (15) depends on the knowledge of the problem parameters.
Indeed, the linearization term in (8) helps use to better exploit the Lipschitz smoothness of the gradients get an error bound in the order of τ 2 k ∥
d k ∥ 2 for estimating the inner function values.
Without this term, we are only able to use the Lipschitz continuity of the inner functions and so the error estimate will increase to the order of τ k ∥
d k ∥ .
Hence, we need to choose a larger beta (as in (15) ) to reduce ∥ d k ∥ and handle the error term without compromising the complexities.
However, this is not the case for T = 1 as it can be seen as a two-level problem whose inner function is exactly known (the identity map).
In this case, the choice of β is independent of the problem parameters with or without the linearization term.
3.2 High-Probability Convergence for T =
In this subsection, we establish an oracle complexity result with high-probability for the case of
T = 1 .
We first provide a notion of ( ϵ, δ ) -stationary point and a related tail assumption on the stochastic first-order oracle below.
Definition 4.
A point ¯ x ∈ X generated by an algorithm for solving (1) is called an ( ϵ, δ ) -stationary point, if we have ∥G X (¯ x, ∇ F (¯ x ) , β ) ∥ 2 ≤ ϵ with probability 1 − δ .
Assumption 4.
Let ∆ k +1 =
∇ F ( x k ) − J k + for k ≥ 0 .
For each k , given F k we have
E [∆ k +1 | F k ] = 0 and ∥ ∆ k +1 ∥ (cid:12)(cid:12)
F k is K -sub-Gaussian.
The above assumption is commonly used in the literature; see [23, 21, 30, 55].
We also refer to [45] and Appendix E for additional details.
The high-probability bound for solving non-convex constrained problems by Algorithm 4 is given below.
Theorem 5.
Let Assumptions 1, 2, 4 be satisfied by the optimization problem (1) with T
= 1 .
Let τ 0 = 1 , t 0 = 0 , τ k = 1 √ N , t k = ⌈√
k ⌉ , ∀ k ≥ 1 , where N is the total number of iterations.
Let T =and let { x k , z k } k ≥ 0 be the sequence generated by Algorithm 4 with β k ≡ β > 0 .
Then, we have ∀ N ≥ 1 , δ > 0 , with probability at least 1 − δ ,
[Equation]
Therefore, the number of calls to SFO and LMO to get an ( ϵ, δ ) -stationary point is upper bounded by
O ( ϵ − 2 log 2 (1 /δ )) , O ( ϵ − 3 log 3 (1 /δ )) respectively.
Remark.
To the best of our knowledge, the above result is (i) the first high-probability bound for one-sample stochastic conditional gradient-type algorithm for the case of T = 1 , and (ii) the first high-probability bound for constrained stochastic optimization algorithms in the non-convex setting; see Appendix J of [32].
Proof Sketch of Main Results
In this section, we only present the proof sketch.
The complete proofs are provided in the appendix.
For convenience, let u T +
= x , and we denote H k as the function value of the subproblem at step k , y k as the optimal solution of the subproblem i.e.,
[Equation]
Then, the proof of Theorem 2 proceeds via the following steps:
We first leverage the merit function W k :=
W α,γ ( x k , z k , u k ) defined in (9) with appropriate choices of α, γ for any β > 0 to obtain
[Equation]
where R k is the residual term (see (31)) and E [ R k | F k ] ≤ ˆ σ 2 τ 2 k , as shown in Proposition 3.
Telescoping the above inequality, in Lemma 11 we obtain the following:
[Equation]
To further control the error term H k (˜ y k ) − H k ( y k ) introduced by the ICG method, we set t k , the number of iterations in ICG method at step k , to ⌈ √ k ⌉ .
By Lemma 8, we therefore have
[Equation]
Also, with the choice of τ k = 1 √ N and z 0 =
[Equation]
we can conclude that
[Equation]
Then, taking expectation of both sides and by the definition of random integer R
[Equation]
we have
[Equation]
∀ N ≥ 1 , where B is a constant depending on the problem parameters ( β, σ 2 , L, D X , T, δ ) .
As a result, we can obtain (13) and (14) by noting that ∀ k ≥∥G
[Equation]
where the second inequality follows the non-expansiveness of the projection operator.
The proofs of Theorems 3 and 5 follow the same argument with appropriate modifications.
The high-probability convergence proof of Theorem 5 mainly consists of controlling the tail probability of the residual term R k being large.
Discussion
In this work, we propose and analyze projection-free conditional gradient-type algorithms for constrained stochastic multi-level composition optimization of the form in (1).
We show that the oracle complexity of the proposed algorithms is level-independent in terms of the target accuracy.
Furthermore, our algorithm does not require any increasing order of mini-batches under standard unbiasedness and bounded second-moment assumptions on the stochastic first-order oracle, and is parameter-free.
Some open questions for future research: ( i ) Considering the one-sample setting, either improving the LMO complexity from O ( ϵ − 3 ) to O ( ϵ − 2 ) for general closed convex constraint sets or establishing lower bounds showing that O ( ϵ − 3 ) is necessary while keeping the SFO in the order of O ( ϵ − 2 ) , is extremely interesting; and ( ii ) Providing high-probability bounds for stochastic multi-level composition problems ( T > 1 ) and under sub-Gaussian or heavy-tail assumptions (as in [32, 31]) is interesting to explore.
Acknowledgment
The authors are grateful to anonymous reviewers for their constructive comments that greatly improved the presentation of this paper.
TX was partially supported by a seed grant from the Center for
Data Science and Artificial Intelligence Research, UC Davis and National Science Foundation (NSF) grant CCF-1934568.
KB was partially supported by a seed grant from the Center for Data Science and Artificial Intelligence Research, UC Davis and NSF grant DMS-2053918.
[-and let u T +
-]
[-Let { x k } k ≥ 0 and { u ki } k ≥ 0 be generated by Algorithm 1 and u T +
-]
[-+
-]
[-+
-]
[-e ki + f i ( u ki +1 ) + ∇ f i ( u ki +1 ) ⊤ ( u k +
-]
[-τ k G k +
-]
[-i − J k +1 i ⊤ ( u k +
-]
[-e ki + ∆ k +1 J i ⊤ ( u k +
-]
[-u k +
-]
[-+
-]
[-u k +
-]
[-+
-]
[-where the inequality comes from the fact that ∥ J k +
-]
[-+
-]
[-= e ki + τ k ∆ k +1 G i + ∆ k +1 J i ⊤ ( u k +
-]
[-+
-]
[-W α,γ ( · , · , · ) be defined in (9) with positive constants { α, { γ i } 1 ≤ i ≤ T } , and u T +
-]
[-+ βd k , x − y k (cid:11) ≥ 
-]
[-we have E [ ∥ u k +
-]
[-y 
-]
SG was partially supported by an NSERC Discovery Grant.
[-= 
-]
[-= 
-]
[-= 
-]
[-where α i,k > 0 and (cid:80) k − 1 i =
-]
[-Noting that α = β 4 L 2 ∇ F , we obtain that for all a > 0 with probability at least 1 − δ 3 , (cid:80) Ni =
-]