Stochastic Adaptive Regularization Method with Cubics: A High Probability Complexity Bound
Abstract
We present a high probability complexity bound for a stochastic adaptive regularization method with cubics, also known as regularized Newton method.
The method makes use of stochastic zeroth, ﬁrst and second-order oracles that satisfy certain accuracy and reliability assumptions.
Such oracles have been used in the literature by other adaptive stochastic methods, such as trust region and line search.
These oracles capture many settings, such as expected risk minimization, stochastic zeroth order optimization, and others.
In this paper, we give the ﬁrst high-probability iteration bound for stochastic cubic regularization and show that just as in the deterministic case, it is superior to other adaptive methods.
1. Introduction
We consider the unconstrained optimization problem
[Equation]
where the objective function φ is assumed to be sufﬁciently smooth, but whose value and gradients are not computable exactly.
Instead we assume that we have access to stochastic zeroth, ﬁrst and second order oracles that, given x , produce random estimates of φ ( x ) , ∇ φ ( x ) and ∇ 2 φ ( x ) respectively.
These oracles are as follows.
Stochastic zeroth order oracle ( SZO ( (cid:15) f , ν, b ) ).
Given a point x , the oracle computes f ( x, Ξ( x )) , a (random) estimate of the function value φ ( x ) .
Ξ( x ) is a random variable (whose distribution is deﬁned, given x , (cid:15) f , ν and b ).
We assume the absolute value of the estimation error E ( x ) =
| f
( x, Ξ( x )) − φ ( x ) | (we omit the dependence on Ξ for brevity) to be a “one-sided” sub-exponentiallike random variable 1 with parameters ( ν, b ) , whose mean is bounded by some constant (cid:15) f > 0 .
Speciﬁcally,
[Equation]
We view x as the input to the oracle, f ( x, Ξ( x )) as the output and the values ( (cid:15) f , ν, b ) as values intrinsic to the oracle.
To simplify notation, we use f ( x ) instead of f ( x, Ξ( x )) in the paper.
Stochastic ﬁrst order oracle ( SFO ( κ g ) .
Given a point x and constants M 1 > 0 , δ 1 ∈ [0 , 12 ) , the oracle computes g ( x, Ξ 1 ( x )) , a (random) estimate of the gradient ∇ φ ( x ) , such that
[Equation]
The distribution of Ξ 1 ( x ) is deﬁned, given x , M 1 , δ 1 and κ g .
Here we view x , M 1 and δ 1 as inputs input to the oracle, while κ g is intrinsic to the oracle.
Similarly we deﬁne
Stochastic second order oracle ( SSO ( κ H ) ).
Given a point x and constants M 2 > 0 , δ 2 ∈ [0 , 12 ) , the oracle computes H ( x, Ξ 2 ( x )) , a (random) estimate of the Hessian ∇ 2 φ ( x ) , such that
[Equation]
The distribution of Ξ 2 ( x ) is deﬁned, given x , M 2 , δ 2 and κ H .
Here we view x , M 2 and δ 2 as inputs to the oracle, while κ H is intrinsic to the oracle.
Similar oracles have been introduced and used recently in [17] and [8] for the analysis of high probability iteration complexity bounds for a stochastic adaptive step search (line search) method and a stochastic trust region method, respectively. Speciﬁc examples and settings under which the assumptions are satisﬁed can be found in [17] and [8].
Related work.
In [17], the authors derived high probability iteration complexity results for a step search method that relies on stochastic, possibly biased zeroth and ﬁrst order oracles.
That paper extends results in [11], [6] and [21] in two key ways - allowing the use of possibly biased oracles and the derivation of high probability complexity bound.
On the other hand, [8] provides a similar extension to biased oracles for high probability iteration complexity for stochastic trust region method, compared to previous work in [1], [16], and [7, 15]
Adaptive regularization with cubics (ARC) method is known to have optimal complexity for ﬁnding (cid:15) stationary points of deterministic smooth functions [10, 13].
In that sense the method is superior to deterministic line search and trust region methods.
In [11] the expected complexity analysis is derived for the case of stochastic ﬁrst and second order oracles, but under the assumption that the values φ ( x ) can be computed exactly.
Moreover, the deﬁnitions of the ﬁrst and second order oracles imposed stronger conditions on those oracles than we require here.
There are many other variants of adaptive cubic regularization methods under various assumptions and requirements on the function value, gradient, and Hessian estimates.
For example, in [2, 4, 5], bounds on expected complexity are provided under the assumption that function estimates are either exact or have a deterministically bounded error while the gradient and Hessian estimates are probabilistic.
In [3, 14, 19, 20, 22, 23], the function, gradient, and Hessian estimates are assumed to have deterministically bounded errors whose magnitude can be adaptively changed in the course of the algorithm.
Our contributions.
In this work we provide the ﬁrst analysis of a stochastic ARC method (SARC) that allows 1. stochastic function estimates that can have arbitrarily large errors, and 2.
stochastic gradient and Hessian approximations whose accuracy is bounded by an adaptive quantity with sufﬁciently high probability, but otherwise can be arbitrarily bad.
To the best of our knowledge, our
Initialization: Choose parameters γ ∈ (0 , 1) , θ ∈ (0 , 1) , δ ∈ [0 , 12 ) σ min > 0 and κ θ ∈ (0 , 1) .
Pick initial x 0 , κ g > 0 , κ H > 0 , ¯ (cid:15) ≥ 0 , (cid:15) (cid:48) f > 0 and σ 0 > σ min .
Repeat for k = 0 , 1 , . . .
trial s
Algorithm parameters κ θ
¯ (cid:15) , δ σ k at x k . sufﬁcient decrease Let + =
+ s
Compute function value estimations f ( x k ) =
f ( x k , k ) and f ( x + k ) =
+ k
+ k the SZO set
[Equation]
(2) where
[Equation] [Equation]
[Equation]
otherwise
[Equation]
[Equation] [Equation]
work is the ﬁrst to derive a complexity bound in this setting with an overwhelmingly high probability. We show that our variant of stochastic ARC, while more general than those in prior literature, still maintains its optimal iteration complexity.
The analysis presented here extends the stochastic settings and high probability results in [17] and [8] to the framework in [11]. However, this extension is far from trivial, as it requires careful modiﬁcation of most of the elements of the existing analysis. We point out these modiﬁcations in the appropriate places in the paper. For space reasons, all proofs are in the Appendix.
2. Adaptive regularization method with cubics (ARC) with probabilistic second-order models
The Stochastic Adaptive Regularization with Cubics (SARC) algorithm is presented below as Algorithm 1, with its subroutine presented as Algorithm 2.
Remark 1 Algorithm 2 will always terminate in a ﬁnite number of iterations, if ¯ (cid:15) > 0 .
In particular,
M 1 takes at most log (cid:16)
Mα k ¯ (cid:15) (cid:17) + 1 iterations to be less or equal to the required precision lower bound [Equation]
[Equation] [Equation] [Equation] (cid:26) (cid:107)
s k,i (cid:107) , ¯ (cid:15) σ k (cid:107) s k,i (cid:107) (cid:27) , end procedure and return g k,i , H k,i and s k,i .
3b. Unsuccessful step: Otherwise, set M 1 ,i +1 ← max { M 1 ,i 2 , ¯ (cid:15)σ k } , M 2 ,i +1 ← max { M 2 ,i 2 , (cid:113) ¯ (cid:15)σ k } and go to step 1. α k ¯ (cid:15) , and M 2 takes at most log (cid:16) M √ α k ¯ (cid:15) (cid:17) + 1 iterations to be less or equal to the required precision lower bound √ α k ¯ (cid:15) .
Note √ α k ¯ (cid:15) is the lower bound for max (cid:26) (cid:107)
s k,i (cid:107) , ¯ (cid:15) σ k (cid:107) s k,i (cid:107) (cid:27) for any (cid:107) s k,i (cid:107) .
Remark 2 Step 2 in Algorithm 2 can be replaced by the simple requirement that s k,i is an exact global optimizer of m k,i , which is stronger than the requirement listed in Step 2.
3. Elements of stochastic analysis
Algorithms 1 and 2 together generates a random stochastic process.
Let i k be the total number of iterations the loop in Algorithm 2 executes during iteration k of Algorithm 1.
Let M k denote (cid:110)
Ξ k , Ξ + k , Ξ 1 k,i k , Ξ 2 k,i k (cid:111) , whose realizations are (cid:110) ξ k , ξ + k , ξ 1 k,i k , ξ 2 k,i k (cid:111) , where Ξ k , Ξ
+ k dictate the randomness of the function estimations at x k and x + k .
A stochastic process (cid:110)(cid:16) G k , H k , S k , f
( X k , Ξ k ) , f ( X + k , Ξ + k ) , X k , A k = 1Σ k (cid:17)(cid:111) is generated by the algorithm, with realization (cid:110)(cid:16) g k , H k , s k , f ( x k , ξ k ) , f ( x + k , ξ + k ) , x k , α k = 1 σ k (cid:17)(cid:111) .
It is adapted to {F k : k ≥ 0 } , where F k = σ ( M 0 , M 1 , . . .
, M k ) .
At iteration k , X k denotes the (random) iterate,
G k is the gradient apprpximation, H k is the Hessian approximation, A k = 1Σ k is the inverse of the model regularization parameter.
S k is the step computed by the Algorithm 2.
f ( X k , Ξ k ) and f ( X + k , Ξ + k ) are the function estimates at the current point and the candidate point, respectively.
Note that conditioned on X k and M 1 ,i k , G k is dictated by Ξ 1 k,i k .
Similarly, conditioned on X k and M 2 ,i k , H k is dictated by Ξ 2 k,i k .
The function estimates are dictated by Ξ k , Ξ + k in the zeroth order oracle.
We further deﬁne E k :=
| f
( X k , Ξ k ) − φ ( X k ) | and E + k :=
| f
( X + k , Ξ + k ) − φ ( X + k ) | , with realizations e k and e + k .
Let Θ k := 1 { iteration k is successful } .
The random variable Θ k is clearly measurable with respect to the ﬁltration F k .
By the construction of Algorithm 2, the stochastic model m k at iteration k is “sufﬁciently accurate” with probability at least 1 − δ .
Speciﬁcally, we have the following lemma.
Lemma
By construction of Algorithm 2, given iterate x k at iteration k , the indicator variable
J k (cid:0) Ξ 1 k ( x k ) , Ξ 2 k ( x k ) (cid:1) = 1 (cid:8) (cid:107)∇ φ ( x k ) − g ( x k , Ξ 1 k ( x k ))
(cid:107) ≤ κ g max (cid:8) (cid:107) S k (cid:107) 2 , α k ¯ (cid:15) (cid:9) , and (cid:107) ( ∇ 2 φ ( x k ) − H ( x k , Ξ 2 k ( x k )))
S k (cid:107) ≤ κ H max (cid:8) (cid:107) S k (cid:107) 2 , α k ¯ (cid:15) (cid:9)
[Equation]
satisﬁes the following submartingale-like condition
[Equation]
A key concept that will be used in the analysis is the concept of a true iteration .
Deﬁnition 4 (True iteration)
We say that iteration k is true if
[Equation]
and | f ( x k )
[Equation]
and is false otherwise.
I k is the indicator random variable that iteration k is true.
The stopping time of the algorithm is deﬁned as follows.
Deﬁnition 5 (Stopping time)
For (cid:15) > 0 , T (cid:15) :
= min (cid:8) k : (cid:13)(cid:13) ∇ φ ( X + k ) (cid:13)(cid:13) ≤ (cid:15) (cid:9) + 1 , the iteration complexity of the algorithm for reaching a (cid:15) -stationary point.
We will refer to T (cid:15) as the stopping time of the algorithm.
It is easy to see that T (cid:15) is a stopping time of the stochastic process with respect to F k .
Given a level of accuracy (cid:15) , we aim to derive a bound on the iterations complexity T (cid:15) with high probability.
In particular, we will show the number of iterations until the stopping time T (cid:15) is a sub-exponential random variable itself.
The random variable Z k is deﬁned to measure the progress towards optimality.
Deﬁnition 6 (Measure of Progress)
For each k ≥ 0 , let Z k ≥ 0 be a random variable measuring the progress of the algorithm at step k :
Z k = φ ( X k ) − φ ∗ , where φ ∗ is a lower bound of φ .
We make the following assumptions on the nonconvex objective φ and the algorithm input (cid:15) (cid:48) f .
Assumptionφ is bounded below by a constant φ ∗ , φ ∈ C 2 ( R n ) and has globally L -Lipschitz continuous gradient and L H -Lipschitz continuous Hessian.
Assumption
[Equation]
Since the stochastic oracles are noisy and possibly biased, the algorithm cannot be expected to converge to a stationary point.
Instead, it can only converge to an (cid:15) -stationary point where (cid:15) is dictated by ¯ (cid:15) and (cid:15) (cid:48)
f as follows.
Inequality 1 (Lower bound on the size of convergence neighborhood)
[Equation]
where κ h
= θ 6 (1 − κ θ ) 3 / [Equation] , κ s = 2 κ g + κ H + L + L H , and [Equation]
(cid:111)(cid:17) , where u = inf x (cid:110) (cid:15) (cid:48)
f − E
[ E ( x )] (cid:111) .
The stochastic process generated by the algorithm has the following properties.
Proposition 7 (Properties of the stochastic process) Let Assumptions 1 and 2 hold, and suppose (cid:15) satisﬁes Inequality 1.
For ¯ α = 1 σ c > 0 and the following non-decreasing function h : R → R :
[Equation]
the following holds for all k < T ε − 1 :
(i) h (¯ α ) > 4 (cid:15) (cid:48)
f p − 12 .
(The lower bound of potential progress an iteration with step size ¯ α can make.)
(ii) P ( I k = 1 | F k − 1 ) ≥ p for all k .
(Conditioning on the past, the next iteration is true with probability at least p .)
(iii)
If I k Θ k = 1 then Z k +1 ≤ Z k − h
( A k ) + 4 (cid:15) (cid:48)
f .
(True, successful iterations make progress.)
(iv) If A k ≤ ¯ α and I k
= 1 then Θ k
= 1 .
(Small and true iterations are also successful.)
(v) Z k +1 ≤ Z k + 2 (cid:15) (cid:48)
f
+ E k
+ E + k for all k .
(The “damage” at each iteration is bounded.)
4. High probability iteration complexity
We can now use essentially the same analysis as in [18] to obtain a high probability iteration bound.
The only difference remains is the failure event is now t + 1 < T (cid:15) instead of t < T (cid:15) with the new stopping time.
Theorem 8 Suppose Assumptions 1 and 2 hold for Algorithm 1, and the Inequality 1 on (cid:15) is satisﬁed.
Then we have the following bound on the iteration complexity: For any s ≥ 0 , ˆ p ∈ (cid:16)
[Equation]
, p (cid:17) , and t ≥
R
[Equation]
, we have
[Equation]
min
[Equation]
(cid:27)(cid:19) ,
where R = φ ( x 0 ) − φ ∗ κ h (cid:15) 3 / 2 + max (cid:110)
[Equation]
, 0 (cid:111) , with κ h , p , σ c and κ s as deﬁned previously.
Remark
Theorem 8 essentially shows the iteration complexity of the algorithm is O ( (cid:15) − 3 / 2 ) with overwhelmingly high probability, which matches its deterministic counterpart.
If ¯ (cid:15) = 0 , the stopping time can be also deﬁned as:
[Equation]
By [17], with an appropriately chosen γ with respect to p , α k = 1 σ k will remain sufﬁciently large with high probability.
As a result, the accuracy requirements for the ﬁrst and second order oracles will remain reasonable to satisfy with high probability.
Appendix A. Proof of Lemma
Proof We show P ( J k = 1 | F k − 1 ) ≥
− δ by showing
[Equation]
and
[Equation]
By deﬁnition of the oracles and construction of Algorithm 2 we have:
[Equation]
and
[Equation]
Inequality (11) implies
[Equation]
Inequality (10) and (12) together gives
[Equation]
Using the fact that conditioning on a set of x k ’s, (Ξ 1 k ( x k ) , Ξ 2 k ( x k )) ’s are independent of each other, we obtain
[Equation]
Appendix B. Lemmas and the proof of Proposition
The following lemmas provide useful properties of the stochastic process generated by Algorithm 1, which is essential for the convergence analysis.
Lemma 10 Consider any realization of Algorithm 1, on each iteration k we have
[Equation]
On every successful iteration k , we have
[Equation]
14) or
[Equation]
Proof The proof is similar to the proof of Lemma 3.3 in [12].
Clearly, (14) follows from (13) and the sufﬁcient decrease condition (2)-(4):
[Equation]
and (15) follows from the deﬁnition of e k and e + k .
It remains to prove (13).
Combining the ﬁrst condition on step s k in (6), with the model expression for s = s k , we can write
[Equation]
The second condition on s k in (6) implies (
s k ) T H
k s k ≥ − σ k (cid:107) s k (cid:107) 3 .
Together with the above equation, we obtain (13).
Lemma 11 Let Assumption 1 hold.
For any realization of Algorithm 1, if iteration k is true (i.e.,
I k = 1 ), and if
[Equation]
then iteration k is either successful or produces s k such that (cid:107) s k (cid:107) 2 < ¯
(cid:15)σ k .
Proof
Clearly, if ρ k − 1 ≥ 0 , then k is successful by deﬁnition.
Let us consider the case when ρ k < 1 ; then if 1 − ρ k ≤ 1 − θ , k is successful.
We have from (2), that
[Equation]
Notice that:
[Equation]
The second last inequality is by deﬁnition of e k and e + k [Equation] and the last inequality is by deﬁnition of the iteration being true
[Equation] Taylor expansion and triangle inequalities give, for some ξ k ∈ [ x k , x + k ] ,
[Equation]
[Table] where the last inequality follows from the fact that the iteration is true and hence (8) holds: (cid:107)∇ φ ( x k ) − g k
[Equation]
≤
[Equation]
g max
[Equation]
[Equation] and (cid:107)
[Equation]
[Equation]
, and from Assumption 1.
So as long as (cid:107) s k (cid:107) 2 ≥ ¯ (cid:15)σ k , we have
[Equation]
which together with (13) give that 1 − ρ k ≤ 1 − θ when σ k satisﬁes (20).
Note that for the above lemma to hold σ c does not have to depend on L .
However, in what follows we will need another condition on σ c , which will involve L ; hence for simplicity of notation we introduced σ c above to satisfy all necessary bounds.
Lemma 12 Let Assumption 1 hold.
Consider any realization of Algorithm 1.
On each true iteration k we have
[Equation]
where κ s = 2 κ g
+ κ H + L + L H .
Proof
Triangle inequality, equality
[Equation]
s and condition (7) on s k together give
[Equation]
Recalling Taylor expansion of ∇ φ ( x + k ) :
[Equation]
and applying triangle inequality, again
[Equation]
we have
[Equation]
where to get the second inequality, we also used (8) and Assumption 1.
We can bound (cid:107) g k (cid:107) as follows
[Equation]
[Table] Thus ﬁnally, we can bound all the terms on the right hand side of (18) in terms of (cid:107) s k (cid:107) 2 and using the fact that κ θ ∈ (0 , 1) we can write
[Equation]
which is equivalent to (17).
Lemma 13 Let Assumption 1 hold.
Consider any realization of Algorithm 1.
Let ¯ (cid:15) satisfy
[Equation]
Then on each true iteration k [Equation] with k < T (cid:15) − 1 we have
[Equation]
Proof
If iteration k is true and (cid:107)∇ φ ( x + k ) (cid:107) > (cid:15) (since k < T (cid:15) − 1 ) then by Lemma 12:
[Equation]
but since
[Equation]
so
[Equation]
Hence, we must have
[Equation]
Thus, (cid:107) s k (cid:107) 2 ≥ ¯ (cid:15) σ k .
Corollary 14 Let Assumption 1 hold.
Consider any realization of Algorithm 1.
Let ¯ (cid:15) satisfy (19) , if k < T (cid:15) − 1 and iteration k is true (i.e., I k
= 1 ), then if
[Equation]
the iteration k is successful.
Proof
The result is straightforward by applying Lemma 11 and 13.
Lemma 15 Let Assumption 1 hold.
Consider any realization of Algorithm 1.
Let ¯ (cid:15) satisfy (19) and k < T (cid:15) − 1 , then on each true and successful iteration k , we have
[Equation]
where κ f := θ 12 √ 2 (1 − κ θ ) 3 / 2 σ min and σ c is deﬁned in (20) .
Proof
Combining Lemma 12, 13, inequality (15) from Lemma 10 and the deﬁnition of successful iteration in Algorithm 1 we have, for all true and successful iterations k ,
[Equation]
Using that σ k ≥ σ min and that κ s ≤ σ c , the result follows.
Hence, if k < T (cid:15) − 1 , and ¯ (cid:15) satisﬁes (19), any true and successful iteration that has σ k ≤ σ c provides O ( (cid:15) 3 / 2 ) reduction in φ ( x ) .
B.1. Proof of PropositionProof
Part (i) follows easily from the deﬁnitions of ¯ α , h ( α ) and inequality 1.
Part (ii) has exactly the same proof as that of Proposition 1 part (ii) in [18].
Part(iii) follows directly from Lemma 15.
Part(iv) follows directly from Corollary 14.
Part (v) has exactly the same proof as that of Proposition 1 part (v) in [18].