E SCAPING SADDLE POINTS IN ZEROTH - ORDER OPTIMIZATION : TWO FUNCTION EVALUATIONS SUFFICE
Anonymous authors
Paper under double-blind review
A BSTRACT
Two-point zeroth order methods are important in many applications of zeroth-order optimization arising in robotics, wind farms, power systems, online optimization, and adversariable robustness to black-box attacks in deep neural networks, where the problem can be high-dimensional and/or time-varying. Furthermore, such problems may be nonconvex and contain saddle points.
While existing works have shown that zeroth-order methods utilizing Ω( d ) function valuations per iteration (with d denoting the problem dimension) can escape saddle points efﬁciently, it remains an open question if zeroth-order methods based on two-point estimators can escape saddle points.
In this paper, we show that by adding an appropriate isotropic perturbation at each iteration, a zeroth-order algorithm based on 2 m (for any 1 ≤ m ≤ d ) function evaluations per iteration can not only ﬁnd (cid:15) -second order stationary points polynomially fast, but do so using only ˜ O ( d / (cid:15) 2 . 5 ) function evaluations.
I NTRODUCTION
Two-point (or in general 2 m -point, where 1 ≤ m < d with d being the problem dimension) estimators, which approximate the gradient using two (or 2 m ) function evaluations per iteration, have been widely studied by researchers in the zeroth-order optimization literature, in convex (Nesterov and
Spokoiny, 2017; Duchi et al., 2015; Shamir, 2017), nonconvex (Nesterov and Spokoiny, 2017), online (Shamir, 2017), as well as distributed settings (Tang et al., 2019). A key reason for doing so is that for applications of zeroth-order optimization arising in robotics (Li et al., 2022), wind farms (Tang et al., 2020a), power systems (Chen et al., 2020), online (time-varying) optimization (Shamir, 2017), learning-based control (Malik et al., 2019;
Li et al., 2021), and improving adversarial robustness to black-box attacks in deep neural networks (Chen et al., 2017), it may be costly or impractical to wait for Ω( d ) (where d denotes the problem dimension) function evaluations per iteration to make a step. This is especially true for high-dimensional and/or time-varying problems.
See Appendix A for more discussion. However, despite the advantages of zeroth-order methods with two-point estimators, there has been a lack of existing work studying the ability of two-point estimators to escape saddle points in nonconvex optimization problems. Since nonconvex problems arise often in practice, it is crucial to know if two-point algorithms can efﬁciently escape saddle points of nonconvex functions and converge to second-order stationary points.
To motivate the challenges of escaping saddle points using two-point zeroth-order methods, we begin with a review of escaping saddle points using ﬁrst-order methods.
The problem of efﬁciently escaping saddle points in deterministic ﬁrst-order optimization (with exact gradients) has been carefully studied in several earlier works (Jin et al., 2017; 2018).
A key idea in these works is the injection of an isotropic perturbation whenever the gradient is small, facilitating escape from a saddle if a negative curvature direction exists even without actively identifying the direction.
However, the analysis of efﬁcient saddle point escape for stochastic gradient methods is often more complicated.
In general, the behavior of the stochastic gradient near the saddle point can be difﬁcult to characterize.
Hence, strong concentration assumptions are typically made on the stochastic gradients being used, such as subGaussianity, boundedness of the variance or a bounded gradient estimator (Ge et al., 2015;
Daneshmand et al., 2018; Xu et al., 2018; Fang et al., 2019; Roy et al., 2020; Vlaski and Sayed, 2021b), creating an analytical issue when such idealized assumptions fail to hold.
Indeed, though zeroth-order methods can be viewed as stochastic gradient methods, common zeroth order estimators, such as two-point estimators (Nesterov and Spokoiny, 2017), are not subGaussian, and can have unbounded variance.
For instance, it can be shown that the variance of the two-point estimator is on the order of Ω( d (cid:107)∇ f ( x ) (cid:107) 2 ) (Nesterov and Spokoiny, 2017), with both a dependence on the problem dimension d as well as on the norm of the gradient, which can be unbounded.
Due to non-subGaussianity and unboundedness, it is tricky to bound the effect of such zeroth-order estimators and establish tight concentration inequalities that facilitate its escape near saddle points.
In addition, the large variance of the zeroth-order estimator is also an issue away from saddles when the gradient is large.
While this is not an issue to show function improvement in expectation, as we discuss later, this becomes an issue when guaranteeing high probability bounds.
Due to these difﬁculties, previous works on escaping saddle points in zeroth-order optimization have exclusively focused on approaches requiring Ω( d ) function evaluations per iteration to accurately estimate the gradient (Bai et al., 2020; Vlatakis-Gkaragkounis et al., 2019), or in some cases negative curvature directions (Zhang et al., 2022; Lucchi et al., 2021) or the Hessian itself (Balasubramanian and Ghadimi, 2022), reducing in a sense the zeroth-order problem back to a ﬁrst-order one.
However, as explained earlier, two-point or 2 m -point zeroth-order algorithms are important for high-dimensional and/or time-varying problems in many applications areas.
This raises an important question: Can two-point zeroth-order methods escape saddle points and reach approximate second order stationary points efﬁciently?
Our Contribution.
In this work, we show that by adding an appropriate isotropic perturbation at each iteration, a zeroth-order algorithm based on any number m of pairs ( m ranging from 1 to d ) of function evaluations per iteration can not only ﬁnd ( (cid:15), √ (cid:15) ) -second order stationary points (cf.
the deﬁnition later in Deﬁnition 1) polynomially fast, but do so using only ˜ O ( polylog( 1 δ ) d / (cid:15) 2 . 5 ) function evaluations, with a probability of at least 1 − δ .
In particular, this proves that using a single two-point zeroth-order estimator at each iteration (with appropriate perturbation) sufﬁces to efﬁciently escape saddle points in zeroth-order optimization, with high probability.
Moreover, for functions that are ( (cid:15), ψ ) strict-saddle (see Deﬁnition 3 for a deﬁnition of strict saddle functions), our results become ˜
O ( polylog( 1 δ ) d / ψ(cid:15) 2 ) , which is a signiﬁcant improvement when ψ (cid:29) (cid:15) ; strict saddle functions have been identiﬁed as an important class of functions in nonconvex optimization, with several well-known examples such as tensor decomposition (Ge et al., 2015), dictionary learning and phase retrieval (Sun et al., 2015).
A comparison of our results with existing zeroth-order and ﬁrst-order methods is shown in Table 1.
We also provide numerical results in Appendix G showing that our proposed two-point algorithm requires fewer total function evaluations to converge than zeroth order methods that use 2 d function evaluations per iteration, for a nonconvex test function proposed in Du et al. To overcome the theoretical challenges that were discussed earlier, we i) ﬁrst show, via a careful analysis, that zeroth order methods can make function value improvement across iterates with large gradients with high probability, even when only a single two-point estimator (which can have signiﬁcant variance at large gradients) is used per iteration.
ii) Second, near saddle points, we overcome issues caused by the unbounded variance and non-subGaussinity of zeroth-order gradient estimators by developing new technical tools, including novel martingale concentration inequalities involving Gaussian vectors, to tightly bound such terms.
In turn, this allows us to show that the noise emanating from the zeroth-order estimators will not overwhelm the effect of the additional isotropic perturbative noise, facilitating escape along negative curvature directions.
To the best of our knowledge, both analyses are novel, and may be independent contributions on their own.
Related Work.
Due to space considerations, we defer a full discussion of related work to Appendix A.
P ROBLEM S ETUP
We make the following assumptions on the class of functions f : R d → R which we consider.
Assumption 1 (Properties of f ) .
We suppose that f : R d → R satisﬁes the following properties:
f is twice-differentiable and lower bounded, i.e. f ∗ :=
min x f ( x ) >
−∞ . 2.
f is L -Lipschitz, i e. (cid:107)∇ f ( x ) − ∇ f ( y ) (cid:107) ≤ L (cid:107) x − y (cid:107) ∀ x, y ∈ R d 3.
f is ρ -Hessian Lipschitz, i
e. (cid:13)(cid:13) ∇ 2 f ( x ) − ∇ 2 f ( y ) (cid:13)(cid:13) ≤ ρ (cid:107) x − y (cid:107) ∀ x, y ∈ R d .
In our work, we focus on ﬁnding approximate second order stationary points, deﬁned below.
Deﬁnition 1.
A point x ∈ R d is an ( (cid:15), ϕ ) -second order stationary point if (cid:107)∇ f ( x ) (cid:107) < (cid:15), and λ min ( ∇ 2 f ( x )) > − ϕ.
We deﬁne an ( (cid:15), ϕ ) -approximate saddle point as follows.
[Table]
2 m and (ii) O ( √ (cid:15) ) if otherwise.
Deﬁnition 2.
A point x ∈ R d is an ( (cid:15), ϕ ) -approximate saddle point, if (cid:107)∇ f ( x ) (cid:107) < (cid:15), and λ min ( ∇ 2 f ( x )) < − ϕ.
Following past convention (Jin et al., 2019a), we will focus in particular on escaping ( (cid:15), √ ρ(cid:15) )
-saddle points. For notational simplicity, in following text, we refer to ( (cid:15), √ ρ(cid:15) ) -saddle points simply as (cid:15) - saddle points and ( (cid:15), √ ρ(cid:15) ) -second order stationary points as (cid:15) -second order stationary points.
Beyond the deﬁnition of (cid:15) -approximate saddle points above, it is known that many nonconvex functions with saddle points, such as orthogonal tensor decomposition (Ge et al., 2015), phase retrieval and dictionary learning (Sun et al., 2015), satisfy what is known as a strict saddle condition (Ge et al., 2015).
For the Hessians of the saddle points of such functions, there is always a strict negative eigenvalue whose magnitude is bounded from below.
We provide a precise deﬁnition below.
Deﬁnition 3.
A twice-differential function f ( x ) is ( (cid:15), ψ ) -strict saddle, if for any point x , at least one of the following is true:
[Equation]
As a corollary of Theorem 1 (to be stated later), for functions f which are ( (cid:15), ψ ) strict saddle, assuming that ψ ≥ √ ρ(cid:15) , the sample complexity of our algorithm scales as ˜Ω (cid:16) d max { L 2 ,L } ( f ( x 0 ) − f ∗ ) m(cid:15) 2 ψ (cid:17) , which scales as ˜Ω (cid:0) dm(cid:15) 2 (cid:1) when ψ is of size Ω(1) .
Thus, in this setting, for two-point estimators, where m = 1 , the dependence on d and (cid:15) in our sample complexity (as measured by function evaluations) matches that achieved by the algorithms in Vlatakis-Gkaragkounis et al.
; Zhang et al.
(2022), which have to use 2 d function evaluations per iteration to estimate the gradient.
In our work, we consider the following batch symmetric two-point zeroth-order estimator.
Deﬁnition 4 ((Batch) two-point zeroth-order estimator with perturbation) .
We deﬁne a m -batch two-point zeroth order estimator as follows:
[Equation]
where Z i i.i.d ∼ N (0 , I ) , and u > 0 is a smoothing radius.
Such 2 m zeroth-order gradient estimators have frequently been studied in zeroth-order optimization works (see e.g. Nesterov and Spokoiny (2017)).
To facilitate efﬁcient escape from saddle points, our proposed Algorithm 1 adds isotropic perturbation at each iteration.
Algorithm input : x 0 , horizon T , step-size η , smoothing radius u , perturbation radius r , batch size m for step t = 0 , . .
, T 1 do
Sample Z ( m ) =
{ Z t,i } mi =1 ∼ N (0 , I ) to compute g ( m ) u
( x t )) .
Update x t +1 = x
t − η (cid:16) g ( m ) u ( x t )
+ Y t (cid:17) , where Y t ∼ N (0 , r 2 d I )
We now state an informal version of our main result, and follow that with a few remarks.
Theorem 1 (Main result, informal version of Theorem 2) .
Consider running Algorithm 1.
Let ˜ O hide polylogarithmic terms in δ and other parameters.
Suppose δ ∈ (0 , 1 /e ] .
Suppose √ ρ(cid:15) ≤ min { 1 , L } 1 , such that ¯ ψ ≤ min { 1 , L } , where ¯ ψ := (cid:26) min { ψ, 1 , L } if f ( · ) is ( (cid:15), ψ ) -strict saddle for any ψ > √ ρ(cid:15) √ ρ(cid:15) otherwise .
Suppose u = ˜
O (cid:18) min { √ (cid:15), √ r } √ ρd (cid:19) , r = ˜ O
( (cid:15) ) , η = ˜ O (cid:16) m ¯ ψ d max { L,L 2 } (cid:17) , Then, in
T = ˜Ω (cid:18) ( f ( x 0 ) − f ∗ ) η(cid:15)+ ρ 2 ( f ( x 0 ) − f ∗ ) η ¯ ψ(cid:19)
[Equation]
m
[Equation]
iterations (with each iteration using 2 m function evaluations), with probability at least 1 − δ , at least half the iterates are (cid:15) -approximate
second-order stationary points.
Remark 1 .
As the choice of η in Proposition 4 (Appendix D) and Theorem 2 (Appendix F) respectively imply, the ˜Ω (cid:16) f ( x 0 ) − f ∗ η(cid:15)(cid:17) term in the sample complexity comes from the large gradient iterations (Proposition 4), whereas the ˜Ω (cid:16) ρ 2 ( f ( x 0 ) − f ∗ ) η ¯ ψ(cid:17) term comes from the escape saddle point phase.
Comparison to gradient-based methods.
For ﬁrst-order escape saddle point algorithms, standard perturbation-based methods (without acceleration) can ﬁnd a ( (cid:15), O ( √ (cid:15) )) -second-order stationary point using ˜ O
(1 /(cid:15) 2 ) iterations for deterministic GD (Jin et al., 2019a), while for standard SGD the best-known rates are slower at ˜ O
(1 /(cid:15) 3 . 5 )
(Fang et al., 2019).
In contrast, our sample complexity (as measured by the total number of function evaluations) is ˜ O (cid:16) d(cid:15) 2 ¯ ψ (cid:17) , where ¯ ψ is deﬁned in Eq.
The extra (linear) dependence on d is typical for zeroth-order algorithms (see e.g. Nesterov and Spokoiny (2017)); intuitively, gradient calculation for d -dimensional functions requires O ( d ) calculations agnostically, so it makes sense that zeroth-order algorithms requires d times more iterations.
For general non strict-saddle functions, our dependence on (cid:15) sits between that of the deterministic methods and SGD methods, and suggests the beneﬁt of a specialized treatment of zeroth-order methods over considering them simply as a subclass of SGD methods.
Moreover, for ( (cid:15), ψ ) - strict-saddle functions where ψ = Ω(1) , our sample complexity becomes ˜ O ( d(cid:15) 2 ) , with an (cid:15) dependence that matches that of the best existing sample complexity for non-accelerated ﬁrst-order escape saddle point methods Jin et al.
Comparison to existing zeroth-order methods.
As Table 1 suggests, our sample complexity signiﬁcantly outperforms that of Bai et al.
and Balasubramanian and Ghadimi (2022), and also that in Lucchi et al.
(2021), which is a random search method.
The sample complexity in VlatakisGkaragkounis et al.
; Zhang et al. (2022) outperform our method, with a function evaluation complexity of ˜ O (cid:0) d(cid:15) 2 (cid:1) .
However, for for ( (cid:15), ψ ) - strict-saddle functions where ψ = Ω(1) , our sample complexity becomes ˜ O ( d(cid:15) 2 ) , which matches the sample complexity in Vlatakis-Gkaragkounis et al.
; Zhang et al.
Moreover, a key limitation of their methods is a requirement to use Ω( d ) function evaluations to estimate the gradient at each iteration, which may not be practical in realistic applications when d is large.
In contrast, our method supports any number of function evaluations at each iteration between 1 to d . Moreover, numerically, we found that for a test nonconvex function proposed in Du et al.
(2017), our method (with two-point estimators) takes fewer function evaluations to escape saddle points and converge to the global minimum than the methods in
Vlatakis-Gkaragkounis et al. ; Zhang et al.
P ROOF STRATEGY AND KEY CHALLENGES IN THE ZEROTH - ORDER SETTING
Broadly speaking, our proof include two major parts, i) characterizing the progress made in iterations when the gradient is large (which we can deﬁne to be iterations t where (cid:107)∇ f ( x t ) (cid:107) ≥ (cid:15) ) (Section 3.1), ii) and iterations when we are at an (cid:15) -approximate saddle point (where progress may be made along the negative eigendirection of the Hessian matrix) (Section 3.2).
While the approach is similar to the ﬁrst-order case (e.g. Jin et al. (2019a)), the zeroth-order setting brings forth several challenges as we explain later in the individual subsections.
In the rest of this section, we explain these challenges at a high level, propose our solutions, and provide statements of our main technical results.
Note that due to the space limit, the full proof is provided in the Appendix. 3. S HOWING FUNCTION DECREASE WHEN GRADIENTS ARE LARGE Challenge. Due to the noise in two-point (or 2 m where m is a small constant) zeroth-order gradient, even when the gradient is large, it may not always be possible to make progress at each iteration, especially when m < d is used in the gradient estimation equation in Eq.
While it is tempting to use an expectation-based argument to handle this issue, it is known that expectation-based function decrease arguments are insufﬁcient for escape saddle point purposes (see e.g. Proposition 1 in Ziyin et al.
We tackle this issue by using high-probability arguments instead; we note that achieving these high-probability bounds is highly nontrivial due to the large variance of the two-point zeroth-order estimator (scaling with d times the squared norm of the gradient). Hence, any single iteration of the zeroth-order method may in fact lead to a function increase rather than decrease. High-level proof outline.
(i)
We ﬁrst characterize the function value change for our proposed algorithm (Lemma 1).
(ii) Next, we tackle the issue of the possibility that the function value might increase for any given iteration. The key idea here is that across any small consecutive number of iterations, there will be one iteration where the zeroth-order estimator is sufﬁciently aligned with the gradient direction (Lemma 14 in Appendix D).
(iii)
Along with a series of other technical results in Appendix D, we then show that the function makes sufﬁcient progress across the duration of the algorithm, with high probability (Proposition 1). To more concretely illustrate the key analytical challenge, we next introduce the following function decrease lemma, proved in Appendix D.
Lemma 1 (Function decrease for batch zero-order optimization) .
Suppose at each time t , the algorithm performs the update step (with batch-size parameter 1 ≤ m ≤ d )
[Equation]
where
[Equation]
where each Z t,i is drawn i.i.d from N (0 , I ) , u > 0 is the smoothing radius, and Y t ∼ N (0 , r 2 d I ) with r > 0 denoting the perturbation radius.
Then, there exist absolute constants c 1 > 0 , C 1 ≥ 1 such that, for any T ∈ Z + and T ≥ τ > 0 , α > 0 and δ ∈ (0 , 1 /e ] , upon deﬁning H 0 ,τ ( δ ) to be the event on which the inequality [Equation]
[Equation]
is satisﬁed (where χ := log( C 1 dmT/δ ) ), we have
[Equation]
for any 0 ≤ τ (cid:48) ≤ T . Our goal is to show that we can arrive at a contradiction f ( x T ) <
min x f ( x ) when there is a large number of steps at which (cid:107)∇ f ( x t ) (cid:107) ≥ (cid:15) (Proposition 1).
As we can see from Eq.
(3), this implies that we need to prove a lower bound of the form
[Equation]
for some α which is not too large (an example would be picking α such that it only scales logarithmically in the problem parameters).
However, it is tricky to prove such a lower-bound in the zeroth-order setting.
In particular, for small batch-sizes m ,
[Equation]
2 could be small even as (cid:107)∇ f ( x t ) (cid:107) 2 is large; this is because for each i ∈ [ m ] , Z t,i could have a negligible component in the ∇ f ( x t ) direction.
This necessitates a more delicate analysis to prove a bound similar to Eq.
Due to space reasons, we defer our more detailed proof approach outline to Appendix D (see the discussion immediately following Lemma 1)
The results in Appendix D culminates in the following result which limits the number of large-gradient.
Proposition 1 (Bound on number of iterates with large gradients, informal version of Proposition 4) .
Let δ ∈ (0 , 1 /e ] be arbitrary.
Letting ˜
O hide polylogarithmic dependencies on δ (and other parameters), consider choosing u , r , η and T such that
[Equation]
Then, with probability at least 1 − O ( δ ) , there are at most T/ 4 iterations for which (cid:107)∇ f ( x t [Equation] (cid:107) ≥ (cid:15) [Equation]
3. M AKING PROGRESS NEAR SADDLE POINTS
Challenge.
The noise in two-point zeroth-order estimators makes the analysis around (cid:15) − approximate saddle points challenging, because the concentration properties of the (non-subGaussian) noise are hard to characterize.
Intuitively, a noisier estimator might facilitate easier escape from saddle point.
However, without an appropriate concentration bound, the noise may behave in unpredictable ways, preventing escape from saddle regions.
Previous analysis of saddle point escape using stochastic estimators typically requires these estimators to satisfy subGaussian properties (Jin et al., 2019a; Fang et al., 2019), which zeroth-order estimators do not satisfy. High-level proof outline. (i)
We ﬁrst prove a technical result showing that the travelling distance of the iterates can be bounded in terms of the function value decrease (i.e., Improve or Localize,
Lemma 2).
(ii) Next, at any (cid:15) -saddle point, we consider a coupling argument and deﬁne two sequences running near-identical zeroth-order dynamics, differing only in the sign of their perturbative term along the minimum eigendirection of H , which denotes the Hessian of the saddle (Lemma 3). Using Lemma 2 in point (i), if we assume for contradiction that the two sequences both “get stuck” and make little function value progress, the dynamics of the difference between the two sequences will remain small as both sequences remain close to the saddle point. iii)
However, since the perturbation vectors of the two sequences differ in the (most) negative direction of H , the norm of the the difference of the two sequences will grow exponentially so long as a).
the sequences remain close to the saddle point (and thus the Hessian has a negative curvature direction) and b).
the effect of the zeroth-order stochastic noise can be controlled.
This leads to a contradiction, implying that sufﬁcient function decrease must have been made (Proposition 5 in Appendix E.3).
(iv) To show that the zerothorder stochastic noise can be controlled, we prove one technical result (Proposition 2), providing a concentration bound for the product of (possibly unbounded) subGaussian random vectors that scales linearly with the dimension d .
This enables us to control the effect of the zeroth-order noise near saddle points, and is essential in showing that the eventual sample complexity scales linearly with d .
We provide a more detailed proof sketch below, where we elaborate more on our analytical challenges and ideas.
We ﬁrst introduce an informal statement of a key technical result that bounds, with high probability, the travelling distance of the iterates in terms of the function value decrease.
Lemma 2 (Improve or Localize, informal version of Lemma 23) .
Consider the perturbed zerothorder update Algorithm 1.
Let δ ∈ (0 , 1 /e ] be arbitrary.
Consider any T s = ˜Ω (cid:0) 1 m log(1 /δ )
(cid:1) , and any t 0 ≥ 0 .
For any F > 0 , suppose f ( x T s + [Equation]
[Equation] x
[Equation]
Letting ˜
O hide polylogarithmic terms involving δ , suppose
[Equation]
Then, with probability at least 1 − O (cid:0) T s δT (cid:1) (here T ≥ T s denotes the total number of iterations), for each τ ∈ { 0 , 1 , . . .
, T s } , we have that
[Equation]
where φ T s ( δ, F
[Equation]
= ˜
[Equation]
Intuitively, the above result shows that if little function value improvement has been made, then the algorithm’s iterates have not moved much, such that it remains approximately in a saddle region if it started out in a saddle region.
Next, Lemma 3 formally introduces the coupling we have mentioned, setting the stage for the rest of our arguments.
For notational convenience, in this section, unless otherwise speciﬁed, we will assume that the initial iterate x 0 is an (cid:15) -saddle point.
Lemma 3.
Suppose x 0 is an (cid:15) -approximate saddle point.
Without loss of generality, suppose that the minimum eigendirection of H := ∇ 2 f ( x 0 ) is the e 1 direction, and let γ to denote − λ min ( ∇ 2 f ( x 0 ))
(note γ ≥ √ ρ(cid:15) ).
Consider the following coupling mechanism, where we run the zeroth-order gradient dynamics, starting with x 0 , with two isotropic noise sequences, Y t and Y (cid:48) t respectively, where ( Y t ) 1 = − ( Y t ) (cid:48) 1 , and ( Y t )
j = ( Y t ) (cid:48)
j for all other j (cid:54)
= 1 .
Suppose that the sequence { Z t,i } t ∈ T,i ∈ [ m ] is the same for both sequences.
Let { x t } denote the sequence with the { Y t } noise sequence, and let the { x (cid:48) t } denote the sequence with the { Y (cid:48) t } noise sequence, where
[Equation]
0 = x 0 , and ˜ H (cid:48) t,i :=
H (cid:48) t,i, + − H (cid:48) t,i, −, with H (cid:48)
t,i, + = ∇ 2 f (
x (cid:48) t + α (cid:48) t,i, + uZ (cid:48) i ) for some α (cid:48) t,i, + ∈ [0 , 1] , and
H (cid:48) t,i, − = ∇ 2 f ( x t − α (cid:48) t,i, − uZ (cid:48) i ) for some α (cid:48) t,i, − ∈ [0 , 1] .
Then, for any t ≥ 0 , ˆ x t +1 :=
x t +1 − x (cid:48) t +1 = − η t (cid:88) τ
= ( I − ηH ) t − τ ˆ ξ g 0 ( τ ) (cid:124) (cid:123)(cid:122) (cid:125) W g 0 ( t +1) − η t (cid:88) τ
= ( I − ηH ) t − τ ( ¯ H τ − H )
ˆ x τ (cid:124) (cid:123)(cid:122) (cid:125) W H ( t +1) − η t (cid:88) τ
= ( I − ηH ) t − τ ˆ ξ u ( τ ) (cid:124) (cid:123)(cid:122) (cid:125) W u
( t +1) − η t (cid:88) τ
= ( I − ηH ) t − τ ˆ Y τ (cid:124) (cid:123)(cid:122) (cid:125) W p
( t +1)
where
[Equation]
[Table]
Our goal is to show that the dominating term in the evolution of the difference dynamics comes from the W p term involving the additional perturbation.
To this end, we need to bound the remaining terms,
W g 0 , W H , W u .
A key technical challenge is to ﬁnd a precise concentration bound for the W g 0 ( t + 1) term, where
[Equation]
For the simplicity of discussion, we assume for the time being that m = 1 , and drop the i index in the subscript of Z τ,i .
Since E [ Z τ Z (cid:62) τ ] = I , heuristically, assuming that Z τ Z (cid:62) τ − I satisﬁes “nice” concentration properties, utilizing the independence of the Z τ ’s across time and the fact that ( I − ηH )
[Equation]
1 + ηγ ) I , we would like to show that with high probability,
[Equation]
where F τ − 1 is a sigma-algebra containing all randomness up to and including iteration τ − 1 , such that x τ and x (cid:48)
τ are both in F τ − 1 , but Z τ is not.
Then, assuming that Eq.
(5) holds, since
E
[Equation]
it follows that
[Equation]
With this bound on (cid:107) W g 0 ( t ) (cid:107) , we eventually prove in Proposition 5 in Appendix E.3 that our algorithm escapes any (cid:15) − saddle point with constant probability and that the O ( d ) term appearing in the square root term above will eventually lead to an O ( d ) dependence in the sample complexity 2 .
We note that the O ( d ) dimension dependence matches that of the best-known existing upper bound for ﬁnding ﬁrst-order stationary points in smooth nonconvex zeroth-order optimization (Nesterov and Spokoiny, 2017), and has been conjectured to be the best possible dimension dependence for general smooth nonconvex zeroth-order optimization (Balasubramanian and Ghadimi, 2022).
Key technical challenge The key challenge in the above argument is to show that an equation in the form of Eq.
(5) could in fact hold.
At ﬁrst glance, that an inequality such as Eq.
(5) should hold is rather non-obvious — this is because while the variable ( Z τ Z τ − I )( ∇ f ( x τ ) − ∇ f ( x (cid:48) τ ))
| F τ − 1 is mean-zero, it is subExponential rather than subGaussian.
In fact, even in the subGaussian case, given a sequence of random vectors x 0 , . . .
, x t − 1 , such that each E [ x τ | F τ − 1 ] = 0 , and that each x τ | F τ − 1 is norm-subGaussian with parameter σ τ ∈ F τ − 1 (which is an appropriate generalization of subGaussianity for vectors, proposed in Jin et al.
(2019b))
, proving a concentration inequality of the form (cid:13) (cid:13)(cid:13) (cid:80) t − 1 τ =0 x τ (cid:13) (cid:13)(cid:13)
≈ ˜
O (cid:18)(cid:113)(cid:80) t − 1 τ {+=
+}
σ 2 τ (cid:19) is a very delicate matter.
In our case, the analogue of x τ is ( I − ηH ) t − 1 − τ ( Z τ Z τ − I )( ∇ f ( x τ ) − ∇ f ( x (cid:48) τ )) , while the analogue of σ 2 τ is (1 + ηγ ) 2( t − 1 − τ ) E (cid:104) (cid:107) ( Z τ Z τ − I ) ( ∇ f ( x τ ) − ∇ f ( x (cid:48) τ ))
(cid:107) 2 | F τ − 1 (cid:105) .
Existing techniques (cf.
Tropp et al.
; Jin et al.
(2019b)) rely crucially on subGaussian properties that allow for each τ
the moment-generating function E
[ e θ Y τ | F τ − 1 ] to be deﬁned for any ﬁxed (and non-random) θ > 0 , where Y τ takes the form
[Equation]
such that E
[ Y τ | F τ − 1 ] =
(since E [ x τ | F τ − 1 ] = 0 ), and the eigenvalues of Y τ are ±(cid:107) x τ (cid:107) .
In the case when x τ is merely subExponential, the Moment Generating Function (MGF), E [ e θ Y τ | F τ − 1 ] , will no longer be well-deﬁned at any ﬁxed (and non-random) θ > 0 .
This poses a challenge in our setting, since x τ takes the form ( I − ηH ) t − 1 − τ
( Z τ Z (cid:62) τ − I )( ∇ f ( x τ ) − ∇ f
x (cid:48) τ )) , which is subExponential rather than subGaussian.
While it may be possible to force ( I − ηH ) t − 1 − τ ( Z τ Z (cid:62) τ −
I )( ∇ f ( x τ ) − ∇ f ( x (cid:48) τ )) to be sub-Gaussian, say by normalizing Z τ to have norm √ d note any bounded random vector is also subGaussian), such that (cid:13)(cid:13) ( Z τ Z (cid:62) τ − I ) g
(cid:13)(cid:13) 2 ≤ d 2 (cid:107) g (cid:107) 2 for any vector g ∈ R d , a careful examination of the argument in Proposition 5 would show that this results in a
O ( d 2 ) rather than O ( d ) dependence in the sample complexity, incurring a heavy price on the overall sample complexity (extra factor of d ) if d is large.
Our solution To overcome the issue, we build on the following observation: with high probability, for any vector g ∈ R d , (cid:12)(cid:12)
Z (cid:62) τ g (cid:12)(cid:12) is bounded within some log factor of (cid:107) g
(cid:107) .
On the event { (cid:12)(cid:12) Z (cid:62) τ g (cid:12)(cid:12) =
˜ O ( (cid:107) g (cid:107) ) } , the variable
[Equation]
behaves approximately like a subGaussian random vector since Z τ ∼ N (0 , I d ) .
Based on this intuition, after some careful analysis, we can show that ( Z τ Z (cid:62) τ − I )( ∇ f ( x τ ) − ∇ f ( x [Equation]
τ )) | F τ − 1 is subGaussian on the event that (cid:12)(cid:12) Z (cid:62) τ ∇ f ( x τ ) (cid:12)(cid:12) is bounded within some log factor of (cid:107)∇ f ( x τ ) (cid:107) , which happens with high probability.
This then allows us to show that on this event, the corresponding MGF is well-deﬁned for all ﬁxed θ > 0 , enabling us to prove a concentration inequality of the form Eq.
This intuition is crystallized in the following proposition, which proves a more general bound than what we strictly need.
For notational simplicity, we introduce the function lr( x ) :
= log ( x log( x )) .
Proposition 2.
Let F t , t ≥ − 1 be a ﬁltration.
Let ( Z t ) t ≥ 0 be a sequence of random vectors following the distribution N (0 , I ) such that Z t ∈ F t and is independent of F t − 1 , and let ( v t ) t ≥ 0 be a sequence of random vectors such that v t ∈ F t − 1 .
For each τ ≥ 0 , let
[Equation]
where each M t is a deterministic matrix of appropriate dimension.
Then, there exist some absolute constants c (cid:48) , C > 0 such that for any τ ∈ Z + and δ ∈ (0 , 1 /e ] , the following statements hold:
For any θ > 0 , with probability at least 1 − δ , we have
[Equation]
For any B > b > 0 , with probability at least 1 − δ , either τ
[Equation]
d (lr( Cτ/δ )) 2 (cid:107) v t (cid:107) 2 ≥ B or (cid:107) [Equation] τ (cid:107) [Equation]
(cid:48)
[Equation]
[Equation]
2 , b (cid:41) (log( Cτd/δ ) + log(log( B/b ) + 1)) Moreover, as is clear from the bounds above, we may pick C ≥ 1 such that log (cid:0) Cδ (cid:1) ≥ 1 , ∀ δ ∈ (0 , 1 e ] .
With this result, along with a series of other technical results in Appendix E.3, we can show that the algorithm makes a function decrease of F with Ω(1) probability near an (cid:15) -saddle point (Proposition 5in Appendix E.3).
Armed with Proposition 5, as well as Proposition 1, the main result in Theorem 1 then follows.
The complete detailed analysis can be found in Appendix E (Escaping saddle point) and Appendix F (main result).
C ONCLUSION
In this paper, we proved that using two function evaluations per iteration sufﬁces to escape saddle points and reach approximate second order stationary points efﬁciently in zeroth-order optimization.
Along the way, we gave the ﬁrst analysis of high-probability function change using two(or more)-point zeroth-order gradient estimators, as well as a novel concentration bound for sums of subExponential (but not subGaussian) vectors which are each the products of Gaussian vectors.
These technical contributions may be of independent interest to researchers working in zeroth-order optimization as well as general stochastic optimization.
There are a few limits of the current results which lead to several interesting future directions, such as extension to noisy function evaluations, as well as studying if some zeroth-order estimators such as asymmetric two-point estimators (Nesterov and Spokoiny, 2017) or single-point estimators (Flaxman et al., 2005) could actually escape saddle points without additional perturbation noise.
A R ELATED W ORK
Two-point methods in zeroth-order optimization.
Two-point (or in general 2 m -point, where 1 ≤ m < d with d being the problem dimension) estimators, which approximate the gradient using two (or 2 m ) function evaluations per iteration, have been widely studied by researchers in the zerothorder optimization literature, in convex (Nesterov and Spokoiny, 2017; Duchi et al., 2015; Shamir, 2017), nonconvex (Nesterov and Spokoiny, 2017), online (Shamir, 2017), as well as distributed settings (Tang et al., 2019).
A key reason for doing so is that for applications of zeroth-order optimization arising in robotics (Li et al., 2022), wind farms (Tang et al., 2020a), power systems (Chen et al., 2020), online (time-varying) optimization (Shamir, 2017), learning-based control (Malik et al., 2019;
Li et al., 2021), and improving adversarial robustness to black-box attacks in deep neural networks (Chen et al., 2017), it may be costly or impractical to wait for Ω( d ) (where d denotes the problem dimension) function evaluations per iteration to make a step. This is especially true for high-dimensional and/or time-varying problems. Indeed, for high-dimensional problems, two-point estimators can make swift progress even in the initial stage compared to 2 d -point estimator, and can reach a higher-quality solution if computation is limited (Tang et al., 2020b; Chen et al., 2017). For instance, consider the work in (Chen et al., 2017), which studies the use of zeroth-order estimators to perform black-box attacks on deep neural networks, in order to identify (and then defend against) adversarial images that may lead to misclassiﬁcation.
In the paper, the authors employed two-point zeroth-order estimators, due to the high computational cost of using 2 d function evaluations per iteration for hundreds of iterations (here d is the dimension of an image, which in this case is over 20000).
The authors showed empirically that their two-point estimators worked well; however there over no accompanying theoretical results.
For online or time-varying environments, two-points estimators also often preferable.
Since zerothorder methods are often used in physical systems whose environment drifts or changes over time, this leads naturally to a time-varying or online optimization.
For these problems, 2 d -point estimators will not produce a good estimation because the underlying function can drift to a very different problem while waiting for the 2 d function evaluations.
Indeed, the fewer function evaluations an optimization procedure needs, the faster it can catch up with the time-varying environment.
In fact, for online optimization, it has been shown that two points estimator is optimal for convex
Lipschitz functions (Shamir, 2017). Thus, two-point estimators are a natural ﬁt for time-varying online optimization problems. Saddle point escape with access to deterministic gradient. While standard gradient descent can escape saddle points asymptotically (Lee et al., 2019; Panageas et al., 2019), it is known that standard gradient descent may take exponential time to escape saddle points (Du et al., 2017).
Hence, when access to deterministic gradient is available, research has centered on escaping saddle points with adding perturbation (Jin et al., 2017), momentum/acceleration based methods (Jin et al., 2018; Sun et al., 2019a; Staib et al., 2019), or gradient-based robust Hessian power/curvature exploitation methods (Zhang and Li, 2021; Adolphs et al., 2019).
In addition, there has also been work on escaping saddle points devoted to speciﬁc optimization settings, such as constrained optimization (Mokhtari et al., 2018; Avdiukhin et al., 2019), optimization of weakly convex functions (Huang, 2021), bilevel optimization (Huang et al., 2022), as well as on general manifolds (Sun et al., 2019b;
Criscitiello and Boumal, 2019; Han and Gao, 2020).
Saddle point escape in stochastic gradient descent (SGD).
In practice, only stochastic gradient estimators are available in many problems.
While SGD may converge to local maxima in worst-case scenarios (Ziyin et al., 2021), under assumptions such as bounded variance or subGaussian noise, there have been many works that have studied the problem of saddle point escape in SGD (Ge et al., 2015; Daneshmand et al., 2018; Xu et al., 2018; Jin et al., 2019a; Vlaski and Sayed, 2021b).
The best existing rate (without considering momentum/variance reduction techniques) appears to belong to that of Fang et al.
(2019), which converges to (cid:15) -second order stationary points using ˜ O
(1 /(cid:15) 3 . 5 ) stochastic gradients.
While zeroth-order gradient estimators may also be viewed as stochastic gradients, they typically do not satisfy the bounded/subGaussian noise assumptions that are assumed in these works, making a direct comparison inappropriate.
Escaping saddle point via momentum methods in SGD has also been studied (Wang et al., 2021;
Antonakopoulos et al., 2022); while we do not consider incorporating momentum in our works, this may be interesting future work.
A number of papers has also considered the specialized setting of escaping saddle points in nonconvex ﬁnite-sum optimization (Reddi et al., 2018; Liang et al., 2021), with many considering the case where variance-reduction is used (Ge et al., 2019; Li, 2019).
While the ﬁnite-sum problem is quite different from our problem, the variance reduction approach considered in these works may be a relevant future direction.
The saddle point escape problem has also been studied in other speciﬁc settings such as compressed optimization (Avdiukhin and Yaroslavtsev, 2021), distributed optimization (Vlaski and Sayed, 2021a), or in the overparameterization case (Roy et al., 2020).
Saddle point escape with zeroth-order information.
The problem of escaping saddle points in zeroth-order optimization has been studied less often, and we have already listed all known works comparable to our work in the introduction (Bai et al., 2020; Vlatakis-Gkaragkounis et al., 2019; Balasubramanian and Ghadimi, 2022); a more detailed comparison of these works with our results has been provided in the discussion following the statement of our main result Theorem 1.
We would like to mention that Roy et al.
(2020) also includes a convergence result of ˜ O (cid:16) d
[Table] (cid:15) [Table] 5 (cid:17) for the case with noisy function evaluations, which is incomparable to our existing work which focuses on the case with exact function evaluation.
In addition, Roy et al.
(2020) also makes a subGaussian assumption on the estimator noise, which zeroth-order estimators in our paper do not satisfy.
Nonetheless, considering the extension to noisy function evaluations will make for important future work.
Zeroth-order optimization.
Our work rests on a line of research in zeroth-order optimization which focuses on constructing gradient estimators using zeroth-order function values (Flaxman et al., 2005;
Duchi et al., 2015; Nesterov and Spokoiny, 2017; Shamir, 2017; Larson et al., 2019).
As we have discussed, for smooth nonconvex functions, it is known that two-point zeroth-order estimators sufﬁce to ﬁnd ﬁrst-order (cid:15) -stationary points using ˜ O ( d/(cid:15) 2 ) function evaluations (Nesterov and Spokoiny, 2017).
Our work studies the more complicated problem of reaching (cid:15) -second order stationary points, attaining a rate of ˜ O ( d/(cid:15) 2 . 5 ) .
B P ROOF R OADMAP
We begin by introducing several key concentration inequalities in Appendix C which we will frequently use in our proofs.
We then describe in detail (and prove) the sequence of results that lead up to Proposition 4 in Appendix D, showing that there cannot be too many iterations with large gradients.
Next, we describe the saddle point argument in detail, and prove Propositionin Appendix E.3.
Finally, we combine these results and prove our main result Theorem 2 (whose informal version is Theorem 1) in Appendix F
Throughout our proofs, absolute constants, as denoted by e.g. ( c, c (cid:48) , C ) , may change from line to line.
However, within the same proof, for clarity, we try to index different constants differently.
We assume d ≥ 2 and m ≤ d .
Notations.
We shall denote the conditional expectation and conditional probability by E F [ · ] = E [ · | F ] and P F ( · )
=
P ( · | F ) where F is a sigma-algebra.
C C ONCENTRATION INEQUALITIES
This section serves to introduce several probabilistic results which will be useful for our main proofs in subsequent sections. We ﬁrst introduce subGaussian, subExponential and norm-subGaussian random vectors in Appendix C.1. Next, in Appendix C.2, we provide concentration bounds for normsubGaussian and subExponential random vectors.
We then prove a novel concentration inequality involving products of subGaussian random vectors in Appendix C.3.
We conclude by stating some concentration bounds for Appendix C.4 random variables.
C. SUB G AUSSIAN , SUB E XPONENTIAL AND NORM - SUB G AUSSIAN RANDOM VECTORS
We ﬁrst deﬁne subGaussian and subExponential random vectors.
A detailed reference for these concepts can be found in Vershynin (2018).
Deﬁnition 5 (subGaussian and subExponential random vectors) .
( λ (cid:104) g
− E [
(cid:105) ) ≤ exp( λ 2 σ 2 / 2) ∀ λ ∈ R
Meanwhile, a random vector x ∈ R d is σ -subExponential (SE( σ )), if there exists σ > 0 such that for any unit vector g ∈ S
d − 1 ,
[Equation]
An alternative concentration property for random vectors revolving around its norm, known as norm-subGaussianity (Jin et al., 2019b), is also relevant.
Deﬁnition 6 (norm-subGaussian random vectors) .
A random vector x ∈ R d is σ -norm-subGaussian (nSG( σ )), there exists σ > 0 such that
[Equation]
We recall the following result which provides several examples of nSG random vectors.
In particular, it tells us a random vector x ∈ R d that is ( σ/ √ d ) − subGaussian is also σ -subGaussian.
Lemma 4 (Lemma 1 in Jin et al. (2019b)) .
There exists absolute constant c such that the following random vectors are all nSG( cσ ).
A bounded random vector x ∈ R d so that (cid:107) x (cid:107) ≤ σ . 2.
A random vector x ∈ R d , where x = ξ e 1 and the random variable ξ ∈ R is σ -subGaussian.
A random vector x ∈ R d that is ( σ/ √ d ) − subGaussian
In addition, if x ∈ R d is zero-mean nSG( σ ), its component along a single direction is also subGaussian.
Lemma 5.
Suppose x ∈ R d is zero-mean nSG( σ ).
Then, for any ﬁxed vector v ∈ R d , (cid:104) v , x (cid:105) is zero-mean (cid:107) v (cid:107) σ -subGaussian.
Proof.
Without loss of generality, we assume that v ∈ S
d − 1 is a unit vector.
That (cid:104) v , x (cid:105) is zero-mean follows directly from x being zero-mean and v being ﬁxed.
Meanwhile, since |(cid:104) v , x (cid:105)| ≤ (cid:107) v (cid:107)(cid:107) x (cid:107) =
(cid:107) x (cid:107) , for any s ≥ 0 , it follows that
[Equation]
where the last inequality follows from the fact that x is zero-mean and also nSG( σ ).
Hence (cid:104) v , x (cid:105) is zero-mean SG( σ ), as desired.
C. C ONCENTRATION BOUNDS FOR NORM - SUB G AUSSIAN AND SUB E XPONENTIAL RANDOM VECTORS
We begin by giving some concentration bounds for norm-subGaussian random vectors.
To do so, we introduce the following condition.
Condition 1. Consider random vectors x 1 , . .
, x n ∈ R d , and corresponding ﬁltrations F i generated by ( x 1 , . . .
, x i ) .
We assume x i | F i − 1 is zero-mean, nSG( σ i ), with σ i ∈ F i − 1 , i.e, E
[Equation]
i
[Equation]
F i − 1 ] = 0 , and
[Equation]
where σ i is a measurable function of ( x 1 , . .
, x i − 1 ) for each i .
For norm subGaussian random vectors satisfying Condition 1, we ﬁrst have the following bound.
Lemma 6.
Suppose ( x 1 , . . .
, x n ) ∈ R
d satisfy Condition 1, i.e. each x i | F i − 1 is mean-zero, nSG( σ i ) with σ i ∈ F i − 1 .
Let { u i } denote a sequence of random vectors such that u i ∈ F i − 1 for every i ∈ [ n ] .
Then, there exists an absolute constant c , such that for any δ ∈ (0 , 1) and λ > 0 , with probability at least 1 − δ ,
[Equation]
Proof.
We note that if x i is mean-zero and nSG( σ i ), then by Lemma 5, (cid:104) u
i , x i (cid:105) | F i − 1 is zero-mean and (cid:107) u
i (cid:107) σ i -subGaussian.
The rest of the proof follows from the proof of Lemma 39 in Jin et al.
(2019a) (key idea is exponentiate and then apply Markov’s inequality).
For completeness, we restate the proof here.
Observe that for any i , since (cid:104) u i , x i (cid:105) is (cid:107)
u i (cid:107) σ i -subGaussian, for any λ > 0 , we have that
[Equation]
For any λ > 0 and s ≥ 0 , observe that
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table] [Equation] [Table] Above, (i) follows from the fact that (cid:104) u i , x i (cid:105) | F i − 1 is zero-mean and (cid:107)
u i (cid:107) σ i -subGaussian for each i ∈ [ n ] .
The ﬁnal result then follows by picking c = 12 and s = log(1 /δ ) .
Assuming Condition 1, the following concentration result also holds for a sequence of nSG random vectors.
Lemma 7 (Lemma 6, Corollary 7 and Corollary 8 in Jin et al. (2019b) combined) .
Suppose ( x 1 , . . .
, x n ) ∈ R
d satisfy Condition 1.
Then, there exists an absolute constant c such that for any ﬁxed δ ∈ (0 , 1) , θ > 0 , with probability at least 1 − δ ,
[Equation]
Moreover, there are two corollaries.
((Jin et al. 2019b, Corollary 7))
When { σ i } is deterministic, there exists an absolute constant c such that for any ﬁxed δ ∈ (0 , 1) , with probability at least 1 − δ .
[Equation]
((Jin et al., 2019b, Corollary 8))
Suppose that the { σ i } sequence is random.
Then, there exists an absolute constant c such that for any ﬁxed δ ∈ (0 , 1) and B > b > 0 , with probability at least 1 − δ : either
[Equation]
We state here a Bernstein-type concentration inequality for sub-exponential random variables, which we also need.
Lemma 8 (Bernstein concentration inequality) .
Consider a sequence of independently distributed σ -subexponential variables x 1 , . .
, x n ∈ R , with mean E [ x i ] ≤ c (cid:48)
σ for some c (cid:48) > 0 and each i ∈ [ n ] .
Then, there exists an absolute constant C > 0 , such that for any δ ∈ (0 , 1) , with probability at least 1 − δ , n (cid:88)
[Equation]
Proof.
The result of Eq.
(6) follows by applying Bernstein’s inequality to (cid:80)
ni
=1 x i − E
[ x i ] (so each summand is mean-zero).
Per Bernstein’s inequality, (cf.
Theorem 2.8.1 in Vershynin (2018)), there exists an absolute constant c > 0 such that for any s ≥ 0 ,
[Equation]
Pick s = σ (cid:16) n + log(1 /δ )
c (cid:17) .
Then, min (cid:26) s
[Equation]
Continuing, we have that
[Equation]
(cid:27)(cid:19) ≤
[Equation]
(cid:18) − c (cid:18) n + log(1 /δ )
c (cid:19)(cid:19) ≤ δ.
Thus, it follows that with probability at least 1 − δ ,
[Equation]
where implication holds since by assumption, E [ x i ] ≤ c (cid:48) σ for some c (cid:48) > 0 .
Then, by setting
C = max { 1 + c (cid:48) , 1 /c } , the desired result follows.
C. A NOVEL CONCENTRATION INEQUALITY FOR THE ZEROTH - ORDER SETTING
In the zeroth-order setting, we will frequently have to bound the norms of terms of the form
[Equation]
where M t is a known and ﬁxed quantity, while Z t is random, and v t depends on x 0 and the history of previous { Z j } t − 1 j =0 ’s, and is hence F t − 1 -measurable.
For our purposes, it sufﬁces to consider
Z t ∼ N (0 , I ) .
To see why such a bound will be useful, as mentioned in the main text and as we will see again later in the full proofs, in the analysis of escaping saddle points, we need to bound a term of the form
[Equation]
where H = ∇ 2 f ( x 0 ) (assuming that x 0 is an (cid:15) -saddle point), and x t and x (cid:48) t are two coupled sequences.
Comparing with Eq.
(7), we see that for the equation above, we can deﬁne M t =
η ( I − ηH )
τ − 1 − t (a ﬁxed and known quantity) and v t = ∇ f ( x t ) − ∇ f ( x (cid:48) t ) (clearly, ∇ f ( x t ) − ∇ f ( x (cid:48) t ) is F t − 1measurable).
This motivates why we wish to bound terms of the form Eq.
Observe that each ( Z t Z (cid:62) t − I ) v t | F t − 1 term is subExponential rather than subGaussian.
While it is possible to deﬁne norm-subExponential vectors in analogous way to norm-subGaussian vectors, the corresponding moment generating function (MGF) for subExponential random variables is not deﬁned on the entirety of R .
When bounding a sum in the form of (cid:80) τ − 1 t =
( Z t Z (cid:62) t − I ) v t , this creates a subtle but challenging technical issue.
Following the intuition outlined in the main text, we bypass this difﬁculty by proving the following result.
For notational simplicity, we introduce the function
[Equation]
We now recall Proposition 2 which we ﬁrst introduced in the main text.
Proposition 2.
Let F t , t ≥ − 1 be a ﬁltration.
Let ( Z t ) t ≥ 0 be a sequence of random vectors following the distribution N (0 , I ) such that Z t ∈ F t and is independent of F t − 1 , and let ( v t ) t ≥ 0 be a sequence of random vectors such that v t [Equation] F t [Equation]
For each τ ≥ 0 , let
[Equation]
where each M t is a deterministic matrix of appropriate dimension.
Then, there exist some absolute constants c (cid:48) , C > 0 such that for any τ ∈ Z + and δ ∈ (0 , 1 /e ] , the following statements hold:
For any θ > 0 , with probability at least 1 − δ , we have
[Equation]
τ
[Equation]
≤
[Equation]
For any B > b > 0 , with probability at least 1 − δ , either
[Equation]
lr
[Equation]
Cτ
[Equation]
[Equation]
Moreover, as is clear from the bounds above, we may pick C ≥ 1 such that log (cid:0) Cδ (cid:1) ≥ 1 , ∀ δ ∈ (0 , 1 e ] .
Proof.
We will focus on proving the ﬁrst point, since the second follows as a natural corollary of our proof of the ﬁrst part and the proof of (Jin et al., 2019b, Corollary 8).
For simplicity, we shall assume v t (cid:54) = 0 in the intermediate steps; extension to the general case is straightforward.
First of all, for 0 ≤ α < 1 , let
[Equation]
It’s not hard to see that for a ﬁxed δ ∈ (0 , 1 /e ] , g ( α ; δ ) is continuous and strictly increasing over α ∈ [0 , 1) .
Then, since log xx + 1 ≤ x for x ≥ 1 , by plugging in x = log(1 /δ ) , we get lr(1 /δ )
[Equation]
which leads to
[Equation]
for δ ∈ (0 , 1 /e ] .
Furthermore [Equation] we obviously have g (0; δ ) < 0 .
Therefore g ( α ; δ ) = 0 has a unique solution in (0 , 2 δ ) , which we denote by α ( δ ) .
These results imply that, for a random variable Z following the standard normal distribution [Equation] we have [Equation]
[Equation]
[Equation]
and
[Equation]
for any δ ∈ (0 , 1 /e ] , where we deﬁne the absolute constant C :
= 2(1 / 2 + 2 / √ 2 π ) .
Now we let A t denote the event
[Equation]
Since Z (cid:62) t v t / (cid:107) v t (cid:107) conditioned on F t − 1 follows the standard normal distribution, we have
[Equation]
and
[Equation]
Moreover, for any random vector u ∈ F t − 1 that is orthogonal to v t , we have
[Equation]
where we used the fact that Z (cid:62) t u is independent of Z (cid:62) t v t conditioned on F t − 1 .
Therefore
[Equation]
Consider deﬁning then the random variable Q t by
[Equation]
We now show that Q t | F t − 1 is norm-subGaussian.
Let u ∈ R d with (cid:107) u (cid:107) = 1 be arbitrary.
We have
[Equation]
where we denote u ⊥ = (cid:16) I − v t v (cid:62) t (cid:107) v t (cid:107) 2 (cid:17) u .
Since (cid:12)(cid:12)(cid:12)(cid:12)
[Equation]
we see that u (cid:62) v t (cid:16) | Z (cid:62) t v t | 2 (cid:107) v t (cid:107) 2 − 1 (cid:17) · 1 A t conditioned on F t − 1 is | u (cid:62) v t | (2 lr(1 /δ )
− 1) -subGaussian.
Furthermore, since | u (cid:62)⊥ Z t Z (cid:62) t v t · 1 A t | ≤ | Z (cid:62) t u ⊥ | (cid:112) 2 lr(1 /δ )
(cid:107) v t (cid:107) , we have
P F t − 1 (cid:0) | u (cid:62)⊥ Z t Z (cid:62) t v t · 1 A t | ≥ s (cid:1) ≤ P F t − 1 (cid:16) | Z (cid:62) t u ⊥ | (cid:112) 2 lr(1 /δ )
(cid:107) v t (cid:107) ≥ s (cid:17) , and since Z t u ⊥ / (cid:107) u ⊥ (cid:107) | F t − 1 follows the standard normal distribution, we see that u (cid:62) ⊥ Z t Z (cid:62) t v t · 1 A t is a (cid:112) 2 lr(1 /δ )
(cid:107) u ⊥
cid:107)(cid:107) v t (cid:107) -subGaussian variable.
Note that u (cid:62) Q t is just the sum of u (cid:62) v t (cid:16)
| Z (cid:62) t v t
| 2 (cid:107) v t (cid:107) 2 − 1 (cid:17) · 1 A t and u (cid:62)⊥ Z t Z (cid:62) t v t · 1 A t , we can conclude that u (cid:62) Q t is subGaussian with parameter [Equation]
[Equation]
, whenever δ ∈ (0 , 1 /e ] .
Consequently, by (Jin et al., 2019b, Lemma 1), we see that Q t | F t − 1 is 8 lr [Equation] √ [Equation]
(cid:107) -norm-subGaussian.
It follows easily that M t Q t | F t − 1 is mean-zero and 8 lr(1 /δ )
(cid:107) M t (cid:107) 2 (cid:107) v t (cid:107)√ d -norm-subGaussian.
Hence, by (Jin et al., 2019a, Lemma 6), we know that there exists an absolute constant c > 0 such that for any θ > 0 and δ > 0 , we have that with probability at least 1 − δ , (cid:13)(cid:13)(cid:13)(cid:13)(cid:13)
[Equation]
Now, consider denoting the event
[Equation]
(cid:12)(cid:12) ∈ (cid:16) α
( δ ) (cid:107) v
[Equation]
(cid:107)
[Equation]
(cid:112) 2 lr(1 /δ ))
(cid:107) v
[Equation]
(cid:107) (cid:17) , ∀ t ∈ { 0 , . . .
, τ − 1 } (cid:111)
By the union bound and Eq.
(9), we note that
P
[Equation]
A ) ≥ 1 − τCδ. Moreover, note that on the event A , (cid:80) τ − 1 t =
M t Q t = (cid:80)
τ − 1 t =
M t ( Z t Z (cid:62) t − I ) v t .
Hence,
P
[Equation]
=
[Equation]
(cid:33) ≥ P [Equation] [Table] t =
[Equation]
M
[Equation]
τ − [Equation] (cid:88) t = d (lr(1 /δ ))
[Equation]
(cid:107) M t (cid:107) 22 (cid:107) v t (cid:107) 2 + 1 θ log(2 d/δ ) , and A happens (cid:33) ≥ 1 − (cid:32)
P (cid:32)(cid:13)(cid:13)(cid:13)(cid:13)(cid:13)
[Table]
≥ 1 − ( δ + τCδ )
[Table]
Now, by rescaling δ to δ/ ( Cτ + 1) , we get the desired result.
Note this C is different from the C in the statement of the lemma by an absolute multiplicative factor.
C. SUB -W EIBULL RANDOM VARIABLES
In our work, we occasionally require bounding sums of heavy-tailed distribution, e.g. higher powers of (cid:107) Z (cid:107) where Z ∼ N (0 , I ) .
To this end, we consider the following deﬁnition of sub-Weibull random variables.
Deﬁnition 7.
We say that a random variable X ∈ R is sub-Weibull( K, α ) for some K, α > 0 ,
[Equation]
For instance, the standard normal distribution is sub-Weibull( 1 , 12 ).
From the way we deﬁne the tail parameter α , the larger the α , the heavier the tail of the distribution.
In our work, we need to show that the sum of sub-Weibull random variables is again sub-Weibull, which is ensured by the following result
Lemma 9.
Suppose X and Y are sub-Weibull( K X , α ) and sub-Weibull
( K Y , α ) respectively.
Then,
XY is sub-Weibull( C ( K X · K Y ) , 2 α ) and X + Y is sub-Weibull
( C ( K X + K Y ) , α ) for some absolute constant C > 0 .
A helpful result is the following, which bounds the sum of identically distributed sub-Weibull random variables.
Lemma 10 (Corollary 3.1 in Vladimirova et al.
Suppose X 1 , . . .
, X n are identically distributed ( K (cid:48) , α ) sub-Weibull random variables.
Then, for some absolute constant c > 0 , for all s ≥ ncK (cid:48) , we have
[Equation]
In our work, we frequently need to bound sums of the k -th power of the norm of a standard ddimensional Gaussian.
We do so using Lemma 10.
Lemma 11.
Suppose X i i.i.d ∼ N (0 , I d ) for i ∈ [ n ] .
Then, for any k ∈ Z + , there exists absolute constants c, C > 0 such that for any δ ∈ (0 , 1) , with probability at least 1 − δ ,
[Equation]
i
[Equation]
In particular, for any δ ∈ (0 , 1 /e ) such that log(1 /δ ) ≥ 1 , it follows that [Equation] n (cid:88) i = [Equation]
(cid:107)
[Equation]
k (cid:12) (cid:12)
[Equation]
Proof.
First, observe that for any j ∈ [ d ] , ( X i ) 2 j , being subExponential, is (1 , 1) -subWeibull.
Then, by Lemma 9, (cid:107) X
i (cid:107) 2 = (cid:80) dj
=1 ( X i ) 2 j is ( cd, 1) for some absolute constant c .
Now, it follows from deﬁnition of sub-Weibullness in Deﬁnition
that (cid:107) X
i (cid:107) 2 k is ( c k d k , k ) -subWeibull.
Hence, applying
Lemma 10, we have that there exists absolute constant C > 0 such that for any s ≥ nCc k d k ,
[Equation]
Choosing s =
(1 + (log(1 /δ ))
k )
nCc k d k , we arrive then at the desired result.
C. S UPERMARTINGALE CONCENTRATION INEQUALITIES
We ﬁrst state and prove a supermartingale-type concentration inequality of the form we later require.
Lemma 12.
Consider a ﬁltration of sigma-algebras F 0 ⊂ F 1 ⊂ · · · ⊂ F n − 1 ⊂ F n and a sequence of random variables X 1 , . .
, X n such that X i ∈ F i .
Suppose that
[Equation]
and
[Equation]
for some a, b > 0 and 0 < p ≤ 1 2 .
Then, for any 0 < µ ≤ b such that |− b + µ | ≥ 1 − p p ( a + µ ) , we have
[Equation]
Proof.
Observe that by Markov’s inequality, for any λ > 0 ,
[Equation]
Now, observe that
[Equation]
Let us now compute E F n − 1 [exp( λ ( X n + µ ))] :
[Equation]
Then observe that by our choice of µ , − b + µ <
[Equation]
and that |− b + µ | ≥ ( a + µ ) 1 − pp .
Since we assumed p ≤ 12 , this means that 1 − pp ≥ 1 and so for any k ≥ 1 ,
[Equation]
Consequently, by Taylor expansion,
[Equation]
k [Equation] = exp [Equation]
+
[Equation]
[Equation]
) , which leads to E
[Equation]
Now, continuing from Eq.
(11), we have that
[Equation]
Thus, for any λ > 0 and s ≥ 0 ,
[Equation]
[Equation]
By ﬁnding the minimizing λ , we ﬁnd that
[Equation]
which completes the proof.
We will later require a weakened form of a supermartingale concentration inequality, as stated and proven below.
Proposition 3 (Weakened supermartingale concentration inequality) .
Consider a ﬁltration of sigmaalgebras F 0 ⊂ F 1 · · · ⊂ F n and a sequence of random variables X 1 , . .
, X n such that X i ∈ F i .
Consider for each i ∈ { 1 , . . . , n } a bad set B i where 1 B i ∈ F i − 1 , and suppose P
[Equation]
and
P
[Equation]
for some a, b > 0 and 0 ≤ p ≤ 1 / 2 .
Then, for any 0 < µ ≤ b such that |− b + µ | ≥ 1 − pp ( a + µ ) , we have
[Equation]
Proof.
We deﬁne Q i :=
X i 1 B ci .
We can then apply Lemma 12 and get
[Equation]
Since P ( X i (cid:54) =
Q i for some i ∈ [ n ]) ≤ (cid:80)
i P ( X i ∈
B i )
, it follows that
[Equation]
which completes the proof.
D F UNCTION DECREASE IN LARGE GRADIENT REGIME
In this section, we show that sufﬁcient function decrease can be made across the iterations with large gradients. We ﬁrst restate and prove the function decrease lemma (Lemma 1), ﬁrst introduced in the main text. We then provide a detailed roadmap of our proof in the subsequent discussion following the proof of Lemma 1.
Lemma 1 (Function decrease for batch zero-order optimization) .
Suppose at each time t , the algorithm performs the update step (with batch-size parameter 1 ≤ m ≤ d )
[Equation]
where
[Equation]
where each Z t,i is drawn i.i.d from N (0 , I ) , u > 0 is the smoothing radius [Equation] and Y t ∼ N (0 , r 2 d I ) with r > 0 denoting the perturbation radius
[Equation] Then, there exist absolute constants c 1 > 0 , C 1 ≥ 1 such that, for any T ∈ Z + and T ≥ τ > 0 , α > 0 and δ ∈ (0 , 1 /e ] , upon deﬁning H 0 ,τ ( δ ) to be the event on which the inequality
[Equation]
is satisﬁed (where χ := log( C 1 dmT/δ ) ), we have
[Equation]
for any 0 ≤ τ (cid:48) ≤ T .
Proof.
First, for each t ∈ {− 1 , . .
. , τ } , we deﬁne F t to be the sigma-algebra generated by
[Equation]
Note that F − 1 is the sigma-algebra generated only by x 0 .
By Taylor expansion, for any x, y ∈ R d , there exists α ∈ [0 , 1] such that f ( x + y )
=
f ( x ) + (cid:104)∇ f ( x ) , y (cid:105) + 12 y (cid:62) ∇ 2 f ( x + αy ) y .
Therefore
[Equation]
with
[Equation]
for some α i, ± ∈ [0 , 1] , and
[Equation]
By the ρ -Hessian Lipschitz property of f , it follows that (cid:13)(cid:13)(cid:13)
[Equation]
t
[Equation]
i
[Equation]
Observe that
[Equation]
[Table]
[Equation]

 (13)
Above, to derive (i), we used the L -smoothness of f .
To derive (ii), we used the expression for ( x t +1 − x t ) shown in Eq.
To derive (iii), we used the fact that ab ≤ ( a 2 + b 2 ) / 2 for any a, b ∈ R ≥ 0 , as well as two applications of the fact that (cid:107) a + b (cid:107) 2 ≤ 2( (cid:107) a (cid:107) 2 + (cid:107) b (cid:107) 2 ) for any two vectors a, b ∈ R d .
To derive (iv), we used the fact that (cid:13)(cid:13)(cid:13)
˜
H t,i (cid:13)(cid:13)(cid:13) ≤ ρu (cid:107) Z t,i (cid:107) .
To continue from Eq.
(13), we ﬁrst observe that we can rewrite
[Equation]
x
[Equation]
) , so that
[Equation]
m
[Equation]
i =
Z
[Equation] (cid:13) [Equation] Observe that we can apply the bound in Proposition 2 to (cid:13)(cid:13)(cid:80) mi =1 ( Z t,i Z (cid:62) t,i − I ) ∇ f ( x t ) (cid:13)(cid:13) , and since
Z t,i is independent of F t − 1 for all i , we know there exist absolute constants c 1 > 0 , C 1 ≥ 1 such that for any δ ∈ (0 , 1 /e ] and θ > 0 , with probability at least 1 − δ conditioned on F t − 1 ,
[Equation]
Moreover, since C 1 ≥ 1 , log( C 1 dm/δ ) and lr( C 1 m/δ ) both are at least 1 as long as δ ∈ (0 , 1 /e ] .
Observe that conditioned on F t − 1 , ∇ f ( x t ) is ﬁxed.
Hence, we can pick
[Equation]
which is F t − 1 -measurable, and plug it into Eq.
(14) to ﬁnd that the probability conditioned on F t −of the following event
[Equation]
is at least 1 − δ .
By taking the total expectation, it follows that the event has a total probability at least 1 − δ .
Thus, with probability at least 1 − δ ,
[Equation]
where the last inequality comes from the fact that lr( C 1 dm/δ ) ≥ 1 , our assumption at the outset of the appendix that d ≥ m , and denoting c 2 := 4 c 1 + 2 .
Denote the event ˜
H 0 ,τ ( δ ) as the event that f
[Equation]
holds.
Now, continuing from Eq.
(13), and using the bound in Eq.
(16), summing over the iterations from t = 0 to τ − 1 , we ﬁnd using the union bound that P ( ∩ τ
[Equation]
τ =1 ˜ H 0 ,τ ( δ )) ≥ 1 − τ (cid:48)
δ , P ( ˜ H 0 ,τ ( δ )) ≥ 1 − τδ.
Now, by Lemma 6, for any δ ∈ (0 , 1) , α > 0 , with probability at least 1 − δ , there exists an absolute constant c 3 > 0 such that
[Equation]
Meanwhile, since Y t ∼ N (0 , ( r 2 /d ) I ) , (cid:107) Y t (cid:107) 2 is sub-exponential with sub-exponential norm cr 2 for some absolute constant c > 0 , and by Bernstein’s inequality (Lemma 8), there exists some absolute constant c 4 > 0 such that
[Equation]
[Equation]
with probability at least 1 − δ .
To bound (cid:80) τ − 1 t =0 1 m (cid:80) mi =1 (cid:107) Z t,i (cid:107) 6 and (cid:80) τ − 1 t =0 1 m (cid:80) mi =1 (cid:107) Z t,i (cid:107) 8 , both sums of heavy tailed Gaussian moments, we use Lemma 11, which states that for any k ∈ Z + and δ ∈ (0 , 1) , with probability at least 1 − δ ,
[Equation]
for some absolute constants c 5 , c 6 > 0 .
As in the statement of the proof, using χ := lr( C 1 dm/δ ) to ease the notation, denote the event that
[Equation]
holds as H 0 ,τ ( δ ) .
Plugging Eq.
(18), Eq.
(19), and Eq.
(20) into Eq.
(17), by union bound, we see that
[Equation]
The ﬁnal result then follows by rescaling δ to δT and denoting c 1 := max { c 2 , c 3 , 2 c 4 , c 5 c 36 / 2 , c 5 c 46 } .
Outline of proof approach.
Similar to the ﬁrst-order setting, our goal is to show that we can arrive at a contradiction f ( x T ) <
min x f ( x ) when there is a large number of steps at which (cid:107)∇ f ( x t ) (cid:107) ≥ (cid:15) .
Roughly speaking, as Eq.
(3) shows, we need to prove a lower bound of the form
[Equation]
for some α which is not too large (an example would be picking α such that it only scales logarithmically in the problem parameters).
However, it is tricky to prove such a lower-bound in the zeroth-order setting.
In particular, for small batch-sizes m , 1 m (cid:80) mi =1 (cid:13)(cid:13)
Z (cid:62) t,i ∇ f ( x t ) (cid:13)(cid:13) 2 could be small even as (cid:107)∇ f ( x t ) (cid:107) 2 is large; this is because for each i ∈ [ m ] , Z t,i could have a negligible component in the ∇ f ( x t ) direction.
This necessitates a more careful analysis to prove a bound similar to Eq.
We do so using the following approach.
Intuitively, whilst for each individual iteration t , 1 m (cid:80) mi =1 (cid:13)(cid:13)
Z (cid:62) t,i ∇ f ( x t ) (cid:13)(cid:13) 2 could be small even as (cid:107)∇ f ( x t ) (cid:107) 2 is large, in a small number of (consecutive) iterations { t 0 , . . .
, t 0 + t f } , with high probability, there will be at least one iteration t within { t 0 , . . .
, t 0 + t f − 1 } , such that 1 m (cid:80) mi =1 (cid:13)(cid:13)
Z (cid:62) t,i ∇ f ( x t ) (cid:13)(cid:13) 2 = Ω( (cid:107)∇ f ( x t ) (cid:107) 2 ) .
We formalize this intuition in Lemma 14.
Thus, we consider breaking the time-steps into chunks where each chunk has t f consecutive iterations.
Consider any such interval { t 0 , . .
, t 0 + t f − 1 } .
There are two cases to consider.
(a)
The ﬁrst case is when the gradient throughout all t f iterations is large enough to dominate the perturbation terms.
Intuitively, in this case, it is not hard to see that given appropriate parameter choices, the gradient will change little throughout the t f iterations.
In fact, as we formalize in Lemma 16, for an appropriate choice of t f and η , we can show that
[Equation]
As a result, combined with point 1, we see that
[Equation]
Thus, by choosing α and η judiciously, for such intervals, it is possible to show that
[Equation]
Thus, in these intervals, it is possible to obtain function improvement on the order of
[Equation]
(b) The remaining case is when the gradient is small and dominated by the perturbation terms in any one of the t f iterations.
In this case, as we show in Lemma 17, for each of the t f iterations, the gradient will be small and on the same scale as the perturbation terms.
In turn, by choosing r, u and η appropriately, we can make the perturbation terms small.
Thus, whilst these intervals may not contribute to function decrease, they also contribute little in the way of function increase.
When there are at least T/ 4 iterations with large gradient (i e. (cid:107)∇ f ( x t ) (cid:107) ≥ (cid:15) ), assuming t f divides T , it follows that there are at least T/ (4 t f ) intervals of length t f where one iteration in the interval contains a large gradient.
By choosing u, r and η appropriately such they are dominated by (cid:15) , it is possible to show that with high probability, such an interval cannot belong to the second case above, and must instead be from the ﬁrst case.
Since (cid:107)∇ f ( x t ) (cid:107) ≈
(cid:107)∇ f ( x t 0 ) (cid:107) for each t ∈ { t 0 , . . .
, t 0 + t f − 1 } in this case, and we know that one of the iterations has a gradient with size at least (cid:15) , it follows that we make function decrease progress of at least η Ω( (cid:15) 2 ) for such intervals.
By appropriately choosing η, u and r to limit the effects of the intervals of the second form, we can then show a contradiction of the form f ( x T ) <
f ∗ .
We demonstrate this formally in Proposition
We formalize our approach in the following series of results.
First, for analytical convenience, we prove the following result showing that for any t , the perturbation terms (cid:107) Y t (cid:107) and 1 m (cid:80) mi =1 (cid:107) Z t,i (cid:107)are bounded with high probability.
Lemma 13.
There exists an absolute constant c 3 > 0 such that, for any t ∈ N , the event
G t ( δ ) :
= (cid:40) (cid:107) Y t (cid:107) 2 ≤ c 23 r 2 (cid:18) 1 + log( T/δ ) d (cid:19) andm m (cid:88) i
= (cid:107) Z t,i (cid:107) 4 ≤ 2 c 3 d(cid:18) log T δ (cid:19)
[Equation]
has probability at least 1 − 2 δ/T for any δ ∈ (0 , 1 /e ] .
Proof.
Noting that Y t ∼ N (0 , ( r 2 /d ) I ) , by applying Bernstein’s inequality (Lemma 8), it can be shown that with probability at least δ/T ,
[Equation]
where c 3 > 0 is some absolute constant.
Then by using Lemma 11, applying the union bound, and redeﬁning the constant c 3 , we complete the proof.
Next, in Lemma 14, we show that in a small number of iterations, with high probability, there exists some iteration t such that
[Equation]
Lemma 14.
There exists an absolute constant c 2 ≥ 1 such that, upon deﬁning
[Equation]
and deﬁning the event
[Equation]
we have
[Equation]
for any δ ∈ (0 , 1) , t 0 ∈ N and k ≥ t f ( δ ) [Equation]
Proof.
Denote the event
[Equation]
Observe that, conditioned on F t − [Equation] the set of random variables
[Equation]
f
[Equation]
x t [Equation] [Equation] x t
[Equation] | [Equation] m i = are independent, mean-zero, and subexponential with subexponential norm ≤ c (cid:107)∇ f ( x t ) (cid:107) 2 for some absolute constant c > 0 .
Hence
[Equation]
t,i ∇ f ( x t ) (cid:12)(cid:12) 2 (cid:17) > m
[Equation]
(cid:107)∇ f ( x t
[Equation]
(cid:107)(cid:33) ≤ exp ( − c (cid:48) m ) , where c (cid:48) is some positive absolute constant.
Then, for any t 0 , k ∈ N ,
P (cid:32)m m (cid:88) i = (cid:12)(cid:12) Z (cid:62) t,i ∇ f ( x t ) (cid:12)(cid:12) 2 < 1 2 (cid:107)∇ f ( x t ) (cid:107) 2 for every t ∈ [ t 0 , t 0 + k ) (cid:33) =
E (cid:34)
[Equation]
(cid:89)
[Equation]
[Equation]
(cid:35) =
E (cid:34)
[Equation]
(cid:3)(cid:35) ≤ exp( − c (cid:48) m ) · E (cid:34) t 0 + k − 2 (cid:89) t = t1 E t (cid:35) ≤ · · · ≤ exp( − c (cid:48) mk ) .
Therefore, by letting c 2 = max { 1 , 1 /c (cid:48) } and k ≥ t f ( δ ) =
(cid:24) cm log
T δ (cid:25) , we get
P (cid:32)m m (cid:88) i = (cid:12)(cid:12) Z (cid:62) t,i ∇ f ( x t ) (cid:12)(cid:12) 2 < 1 2 (cid:107)∇ f ( x t ) (cid:107) 2 for every t ∈ [ t 0 , t 0 + k ) (cid:33) ≤ δ
T , which completes the proof.
The term t f ( δ ) will frequently appear in the proofs to come; in the sequel we denote
[Equation]
where c 2 ≥ 1 is the absolute constant deﬁned in Lemma 14.
We next show that with high probability, the norm difference term (cid:107)∇ f ( x t +1 ) − ∇ f ( x t ) (cid:107) can be bounded in terms of (cid:107)∇ f ( x t ) (cid:107) and the perturbation terms (cid:13)(cid:13)(cid:13) u 2 m (cid:80) mi =1 Z t,i Z (cid:62) t,i ˜ H t,i Z t,i (cid:13)(cid:13)(cid:13) as well as (cid:107) Y t (cid:107) .
Lemma 15.
Deﬁne
[Equation]
where t f ( δ ) is deﬁned in Eq.
[Equation] and let C 1 ≥ 1 be the corresponding absolute constants deﬁned in
Lemma 1.
Then there exists an absolute constant c 4 > 0 such that, whenever η satisﬁes
[Equation]
we have
[Equation]
for any δ ∈ (0 , 1 /e ] and t ∈ Z + .
Proof.
Since ∇ f is L -Lipschitz, following the zeroth-order update step, we see that
[Equation]
Now, it follows from Eq.
(16) (with a slight modiﬁcation in the absolute constant terms since here the norm is not squared) that there exists some absolute constant c 4 > 0 such that for any δ ∈ (0 , 1 /e ] , we have that with probability at least 1 − δ/T , the event
[Equation]
Hence, continuing from Eq.
(26), it follows that with probability at least 1 − δ/T ,
[Equation]
and by plugging in the condition Eq.
(24), we see that the event A
[Equation]
has probability at least 1 − δ/T .
We show now that if the norm of the gradient dominates the norm of the perturbation terms, and we choose the step-size η sufﬁciently small, then in a small number of iterations, the norm of the gradient does not change very much.
For notational simplicity, we denote the event
E [Equation] :=
[Equation]
Lemma 16.
Let δ ∈ (0 , 1 /e ] and
T ∈ Z + be such that T > 2 t f ( δ ) + 1 .
Consider any positive integer t (cid:48)
f ≤ 2 t f ( δ ) , and any t 0 ∈ { 0 , . .
, T − 1 − t (cid:48) f } .
Suppose η satisﬁes the condition Eq.
Then, on the event
E ( t
[Equation]
t (cid:48)
f
[Equation]
δ ) ∩   t 0 + t (cid:48)
f − 1 (cid:92) t = t
A t ( δ ) 
 , we have
[Equation]
for all t ∈ { t 0 , . . .
, t 0 + t (cid:48)
f − 1 } .
Proof.
By plugging
[Equation]
into the deﬁnition of A t ( δ ) , we see that, on the event E ( t 0 , t (cid:48) f , δ ) ∩ (cid:16)(cid:84) t
+ t (cid:48)
f − 1 t = t
A t ( δ ) (cid:17) , we have (cid:107)∇ f ( x t +1 ) − ∇ f ( x t ) (cid:107) ≤ (cid:107)∇ f ( x t ) (cid:107) 4 t f ( δ ) , and consequently, (cid:18) 1 − 4 t f ( δ ) (cid:19) (cid:107)∇ f ( x t ) (cid:107) ≤ (cid:107)∇ f ( x t +1 )
(cid:107) ≤ (cid:18) 1 + 4 t
f
( δ ) (cid:19) (cid:107)∇ f ( x t ) (cid:107) , which leads to (cid:18) 1 − 4 t f ( δ ) (cid:19) t − t(cid:107)∇ f
( x 0 ) (cid:107) ≤ (cid:107)∇ f ( x t ) (cid:107) ≤ (cid:18) 1 + 4 t f ( δ ) (cid:19) t − t(cid:107)∇ f
( x 0 ) (cid:107) for all t ∈ { t 0 , . . .
, t 0 + t (cid:48) f } .
Then, since (1 + 1 / (4 x )) 2 x ≤ 2 and (1 − 1 / (4 x )) 2 x ≥ 1 / 2 for any x ≥ 1 , noting that t (cid:48) f ≤ 2 t f ( δ ) , we get the desired result.
Conversely, in the following result, we show that in a small number of consecutive iterations, if the gradient is smaller than the perturbation terms in any one of the iterations, then for each of the iterations in this range, the gradient will be small and be on the same scale as the size of the perturbation terms.
Lemma 17.
Let δ ∈ (0 , 1 /e ] and
T ∈ Z + be such that T > 2 t f ( δ ) + 1 .
Consider any positive integer t (cid:48)
f ≤ 2 t f ( δ ) , and any t 0 ∈ { 0 , . .
, T − 1 − t (cid:48) f } .
Suppose η satisﬁes the condition Eq.
Then, on the event
[Equation]
we have
[Equation]
where c 5 is some absolute constant.
Proof.
Let t (cid:48) be the ﬁrst iteration in { t 0 , t 0 + 1 , . . .
, t 0 + t (cid:48)
f − 1 } such that
[Equation]
Since we are working on an event which is a subset of E c ( t 0 , t (cid:48) f , δ ) , t (cid:48) is well-deﬁned.
By (cid:107) ˜
H t (cid:48) ,i (cid:107) ≤ ρu (cid:107) Z t (cid:48) ,i (cid:107) , we see that
[Equation]
where we used the deﬁnition of G t ( δ ) .
Recall that t (cid:48) is the ﬁrst time step such that Eq.
(27) holds.
By deriving similarly as in the proof of
Lemma 16, we can show that for any j ∈ { t 0 , t 0 + 1 , . . .
, t (cid:48) − 1 } ,
[Equation]
[-Meanwhile, for iterations t ∈ [ t (cid:48) , t 0 + t (cid:48) f ) , by using the deﬁnitions of A t ( δ ) and G t ( δ ) , we have (cid:107)∇ f ( x t +1 ) (cid:107) ≤ (cid:18) 1 + 8 t f ( δ ) (cid:19) (cid:107)∇ f ( x t ) (cid:107) + ηLc(cid:32) u 2 d 2 ρ (cid:18) log T δ (cid:19) 
-]
[-+ (cid:114) 
-]
[-( δ ) (cid:19) t − t (cid:48) − i ηLc(cid:32) u 2 d 2 ρ (cid:18) log T δ (cid:19) 
-]
[-+ (cid:114) 
-]
Meanwhile, for iterations t ∈ [ t (cid:48) , t 0 + t (cid:48) f ) , by using the deﬁnitions of A t ( δ ) and G t ( δ ) , we have
[-u 2 d 2 ρ (cid:18) log T δ (cid:19) 
-]
[-+ (cid:114) 
-]
[Equation]
where we used t (cid:48)
f ≤ 2 t f ( δ ) and the fact that (1 − 1 / (8 x )) 2 x ≤ e 1 / 4 for all x > 0 .
By deﬁning c 5 := 16 c 3 , we complete the proof.
We next derive a useful result showing that the function change f ( x τ ) − f ( x 0 ) can be decomposed into one component arising from intervals when the gradient dominates noise (which improves function value) and another component arising from intervals with small gradient which may add to function value but whose contributions are bounded in terms of η, u and r .
For now, we focus on the case τ ≥ t f ( δ ) , since it will be useful to us in proving that there cannot be more than T/ 4 iterations with large gradient.
Lemma 18 (Function change for large τ ) .
Let c 1 > 0 , c 4 > 0 , c 5 > 0 , C 1 ≥ 1 be the absolute constants deﬁned in the statements of the previous lemmas.
Let δ ∈ (0 , 1 /e ] , and let τ ≥ t f ( δ ) ) be arbitrary.
Consider splitting { 0 , 1 . .
, τ − 1 } into K := (cid:98) τ/t f ( δ ) (cid:99) intervals:
[Equation]
Let I 1 denote the set of indices k such that for every time-step t in the interval J k , the gradient dominates the noise terms as
[Equation]
Suppose we choose η such that
[Equation]
Then, on the event E
[Equation]
we have the following upper bound on function value change:
[Equation]
Moreover, P ( E τ ( δ )) ≥ 1 − (5 τ +4) δ T .
Proof.
Without loss of generality, we may assume that τ is a multiple of t f ( δ ) .
Then, any interval
J k = { t 0 , . .
, t 0 + t f ( δ ) − 1 } belongs to one of the following two cases:
Case 1 ) (Gradient dominates noise): Recall that this means that for every t ∈ J k , we have [Equation] [Equation] t
[Equation]
By our choice of η in Eq.
(29), we can apply Lemma 16 to get
[Equation]
We now consider the two cases when J has fewer than t f ( δ ) iterations and when J =
J k f
Note also that on the event B kt f ( δ ) ( δ ; t f ( δ )) , there exists some t ∈ J k such that
[Equation]
This implies then that
[Equation]
Thus by setting α = 128 t f ( δ ) in Eq.
(3) and by choosing η such that
[Equation]
it follows that
[Equation]
[Equation] Case 2 ) (Gradient does not dominate noise): there exists some t ∈ J k such that
[Equation]
By our choice of η in Eq.
[Equation]
we can apply Lemma 17 to get
[Equation]
Hence
[Equation]
by setting α = 128 t f ( δ ) in Eq.
(3) and choosing η such that
[Equation]
it follows that
[Equation]
Without loss of generality [Equation] we may assume that τ is a multiple of t f ( δ ) .
Then, any interval
J k = { t 0 , . .
, t 0 + t f ( δ ) − 1 } belongs to one of the following two cases:
Having studied the two cases [Equation] we may now proceed to use them to complete the proof.
Let I c 1 denote the complement of I 1 in { 0 , 1 , . .
, K − 1 } .
Then,
[Equation]
and so by Eq.
[Equation]
Note that we choose α = 128 t f ( δ ) .
In addition, observe that by our choice of δ (such that δ ≤ 1 e ), it follows that (cid:113)
[Equation]
We can now complete our proof by using the union bound (suppressing the dependence of some of the events on δ for notational simplicity) to derive
[Equation]
We are now ready to show that if sufﬁciently many iterations have a large gradient, then with high probability, the function value of the last iterate f ( x T ) , will be less than min x f ( x ) , a contradiction.
Hence this limits the number of iterations that can have a large gradient.
Proposition 4.
Let c 1 > 0 , c 2 ≥ 1 , c 4 > 0 , c 5 > 0 , C 1 ≥ 1 be the absolute constants deﬁned in the statements of the previous lemmas, and let δ ∈ (0 , 1 /e ] be arbitrary.
Suppose we choose u , r , η and
T such that
[Equation]
Then, with probability at least 1 − 6 δ , there are at most T/ 4 iterations for which (cid:107)∇ f ( x t ) (cid:107) ≥ (cid:15) .
Proof.
Without loss of generality, we assume that T is a multiple of t f ( δ ) , and we similarly split { 0 , 1 , . . .
, T } into
K = (cid:98) T/t f ( δ ) (cid:99) intervals J 0 , . .
, J K − 1 .
Let I 1 denote the set of indices k such that for every t ∈ J k ,
[Equation]
We let I c 1 denote the complement of I 1 in { 0 , 1 , . . .
, K − 1 } .
We denote
E T ( δ ) :
= H T ( δ ) ∩
[Equation]
In the remaining part of the proof, unless otherwise stated, we shall always assume that we are working on the event E T ( δ ) .
By Lemma 18 with τ = T and our choices of η and δ in the statement of the lemma, we have f ( x T ) − f ( x 0 ) ≤ − (cid:88)
[Equation]
[Equation] Suppose that there are at least T/ 4 iterations where (cid:107)∇ f ( x t ) (cid:107) ≥ (cid:15) .
Let I (cid:15) denote the set of indices k for which there exists some t ∈ J k with (cid:107)∇ f ( x t ) (cid:107) ≥ (cid:15) .
Then, by the pigeonhole principle, the set
I (cid:15) has at least (cid:100) T/ (4 t f ( δ ))
(cid:101) members.
Note that, by our choices of the parameters η, u, r , it can be shown that
[Equation]
while by Lemma [Equation] if k is in I c
[Equation]
we have
[Equation]
This implies that I (cid:15) ⊆
I 1 .
Observe that by Lemma 16, for any k ∈ I 1 , we have
[Equation]
This implies in particular that for any k ∈ I (cid:15) , we have min t ∈
J k
(cid:107)∇ f ( x t ) [Equation] 2 ≥ [Equation]
2 , and consequently −
[Equation]
Hence, by Eq.
[Equation]
Now, by our choices of u
[Equation]
r and η , we have
[Equation]
where we used log( T/δ ) ≥ 1 and 2 c 2 log( T/δ ) ≥ t f ( δ ) .
We also have
[Equation]
where we used c 2 d log( T/δ ) ≥ t f ( δ ) , c 2 ≥ 1 and log( T/δ ) ≥ 1 .
Finally,
[Equation]
By plugging these bounds into Eq.
(38), we get
[Equation]
Therefore, as long as
[Equation]
we will get f ( x T ) < f ∗ , which is a contradiction.
Thus, we can conclude that on the event E T ( δ ) , there are at most T/ 4 iterations for which (cid:107)∇ f ( x t ) (cid:107) ≥ (cid:15) .
We can now complete our proof by using the union bound (suppressing the dependence of some of the events on δ for notational simplicity) to derive
[Equation]
E E SCAPING SADDLE POINT
In this section, we ﬁrst show that the travelling distance of the iterates can be bounded in terms of the function value improvement (Appendix E.2).
Utilizing this result, as well as Proposition 2 in
Appendix C.3 which provides a concentration bound on the the zeroth-order noise, we then prove that sufﬁcient function value decrease can be made near a saddle point in Appendix E.3.
E. K EY QUANTITIES AND NOTATION
We will use γ to denote − λ min ( ∇ 2 f ( x 0 )) , where we know that γ ≥ √ ρ(cid:15).
E. I MPROVE OR L OCALIZE
In this subsection [Equation] we aim to bound the movement of the iterates across a number of steps in terms of the function value improvement made during these number of steps.
We ﬁrst state a simple result separating the norm of the difference between x t
+ τ and x t 0 into a few different terms [Equation]
Lemma 19.
Consider the perturbed zeroth-order update Algorithm [Equation]
Then, for any t 0 ∈ N and τ ∈ N ,
[Equation]
) where
[Equation]
Proof.
For notational convenience, let t 0 :
= 0 .
Then, applying the form of the perturbed zeroth-order update in Algorithm 1, we get
[Equation]
[Table]
V 2 (
[Table] ,τ ) + [Table]
η
[Table] We now proceed to bound the terms V 1 ( t 0 , τ ) , V 2 ( t 0 , τ ) , V 3 ( t 0 , τ ) and V 4 ( t 0 , τ ) .
First, we have the following result bounding V 1 ( t 0 , τ ) .
Lemma 20.
Let c 1 > 0 , c 2 ≥ 1 , c 4 > 0 , c 5 > 0 , C 1 ≥ 1 be the absolute constants deﬁned in the statements of the previous lemmas, and let δ ∈ (0 , 1 /e ] be arbitrary.
Suppose we choose η such that
[Equation]
There are two cases to consider.
The ﬁrst is when τ ≥ t f ( δ ) .
In this case, split { t 0 , t 0 +1 , . . .
, t [-
-]0 +
τ − 1 } into K := (cid:98) τ/t f ( δ ) (cid:99) intervals:
[Equation]
Then, on the event
E {+t 0 ,τ ( δ ):= H t 0 ,τ ( δ ) ∩ (cid:32) t 
 + τ − 1 (cid:92)+}
[Equation]
=
[Equation] [Equation]
[-(cid:33) ∩ (cid:32) t 
 + τ − 1 (cid:92) t = t G t (-] δ [Equation]
=
t [-
-]G
[Equation]
( δ ) [-( δ ; t f ( δ )) (cid:33) ∩B t 
-][Equation] ∩ [Equation]
−
[Equation]
[Equation]
δ
[Equation]
[Equation]
[Equation]
− 1) [Equation] f
[Equation]
δ )
[Equation]
δ
[Equation]
τ − (
[Equation] − 1) [Equation] ( δ )) , we have that
V
[Equation]
where
[Equation]
The second is when τ < t f ( δ ) .
Suppose we choose u and r such that
[Equation]
Suppose the event ∩ t 0 + τ − 1 t =
t
[Equation] A t ( δ ) ∩ G t ( δ ) holds.
Suppose also that (cid:107)∇ f ( x t 0 ) (cid:107) ≤ (cid:15).
Then,
[Equation]
Proof.
We ﬁrst consider the case where τ ≥ t f ( δ ) .
Let I 1 denote the set of indices k such that for every time-step t in the interval J k , the gradient dominates the noise terms as
[Equation]
WLOG, we may assume that t 0 := 0 , and denote V 1 ( τ ) :
= V 1 (0 , τ ) .
WLOG, we also assume that τ is a multiple of t f ( δ ) .
From Lemma 18, on the event that E τ ( δ ) holds and by our choice of η , we have
[Equation]
By Lemma 16 (and our choice of η )
[Equation]
it follows that for any k ∈ I 1 , on the event ∩ t ∈
J k A t [Equation] , we have
[Equation]
Thus, on the event that E τ ( δ ) holds
[Equation]
for our choice of η
[Equation]
we have η
[Equation]
Similarly, for any k ∈ I c 1 (where I c 1 denotes the complement of I 1 in { 0 , 1 , . . .
, K − 1 } , i.e. intervals where the gradient is smaller than than the perturbation terms in some iteration), on the event ( ∩ t ∈
J k
A t ( δ ))
∩ ( ∩ t ∈ J k G t ( δ )) , by Lemma 17 (and our choice of η ), we have
[Equation]
On the event that E τ ( δ ) holds, this gives us then η
[Equation]
Hence, on the event that E τ ( δ ) holds, we have that
[Equation]
This yields the ﬁnal result for the case τ ≥ t f ( δ ) .
We next consider the case where 1 ≤ τ < t f ( δ ) .
Recall the notation that
[Equation]
There are two cases to consider.
(a) On the event E ( t 0 , t 0 + τ, δ ) ∩ (cid:0)
∩ t 0 + τ − 1 t = tA t ( δ ) (cid:1) , we have by Lemma
that (cid:107)∇ f ( x t ) (cid:107) ≤ 2 (cid:107)∇ f ( x 0 ) (cid:107) for each t ∈ { 0 , 1 , . . .
, τ − 1 } .
Then, V
[Equation]
t 0 , τ ) = 8 η
[Equation]
where the ﬁnal inequality uses the assumption that (cid:107)∇ f ( x 0 ) (cid:107) ≤ (cid:15).
(b) Suppose the event E c ( t 0 , t 0 + τ, δ ) ∩ (cid:0) ∩ t 0 + τ − 1 t = tA [Equation] δ )
[Equation]
∩ t [Equation] + τ − 1 t = tG t ( δ ) (cid:1) holds.
In this case, by Lemma 17, we have that for each t ∈ { t 0 , t 0 + 1 , . . .
, t
+ τ − 1 } (cid:107)∇ f ( x t ) (cid:107)
[Equation]
≤ (cid:15), where the ﬁnal inequality follows by our choice of η, u and r (cf.
Eq.
Hence,
[Equation]
The ﬁnal result for the case τ < t f ( δ ) then follows.
We proceed to bound V 2 ( t 0 , τ ) .
Lemma 21.
Let c 1 > 0 , c 2 ≥ 1 , c 4 > 0 , c 5 > 0 , C 1 ≥ 1 be the absolute constants deﬁned in the statements of the previous lemmas, and let δ ∈ (0 , 1 /e ] be arbitrary and τ > 0 be arbitrary.
Suppose we choose η such that
[Equation]
Let T s denote an integer such that T s ≥ max { τ, t f ( δ ) } , and for any F > 0 , deﬁne B
[Equation]
δ ;
[Equation]
[Equation]
t
[Equation]
δ )
[Equation]
F + N u,
[Equation] [Equation] [Equation] Let c (cid:48) , C > 0 denote the same constants as in the statement of Proposition 2.
Denote the event that either
[Equation]
or
[Equation]
holds as L t 0 ,τ ( δ ; F ) 6 .
We show that P ( L t 0 ,τ ( δ ; F )) ≥ 1 − δT .
Finally, denote the event M t 0 ,T s ( F ) as the event that f ( x t 0 ) − f ( x t
+ T s ) <
F .
Then, on the event L t 0 ,τ ( δ ) ∩ E t 0 ,T s ( δ ) ∩ M t 0 ,
T s ( F ) (where E 0 ,T s ( δ ) is as deﬁned in Lemma 20),
[Equation]
where
[Equation]
Proof.
We note that P ( L t 0 ,τ ( δ ; F )) ≥ 1 − δT .
is a direct consequence of Proposition 2.
In the rest of the proof, without loss of generality, we assume that t 0 = 0 for notational simplicity.
On the event
L 0 ,τ ( δ ; F ) ∩ E 0 ,T s ( δ ) ∩ M t 0 ,T s ( F ) , suppose that
[Equation]
= ⇒ η τ − 1 (cid:88) t =
(cid:107)∇ f ( x t ) (cid:107) 2 ≥ 8 t f ( δ )( F + N u,r
( T s , δ ))
=
⇒ η
T s − 1 (cid:88) t =
(cid:107)∇ f ( x t ) (cid:107) 2 ≥ 8 t f ( δ )( F + N u,r
( T s , δ ))
=
⇒ 8 η 2 T s
T s − 1 (cid:88) t =
(cid:107)∇ f ( x t ) (cid:107)
[Equation]
≥ 64 ηT
[Equation] t f ( δ [Equation] ( F + N
[Equation]
r ( T
[Equation]
, δ )) = ⇒ 8 η 2 T s T s − 1 (cid:88) t =
(cid:107)∇ f ( x t ) (cid:107) 2 ≥ 64 ηT s t f ( δ )( f ( x 0 ) − f ( x T s )
+ N u,r ( T s , δ )) , since f ( x 0 ) − f ( x T s ) ≤ F ⇐⇒ V 1 (0 , T s ) ≥ 64 ηT s t f ( δ )( f ( x 0 )
− f ( x T s )
+ N u,r
( T s , δ )) , where we note the last equation contradicts Lemma 20.
For notational simplicity, denote
[Equation]
+(cid:19) .
Observe that β 1 is larger than β τ for every τ ≥ 1 .
Since L t 0 ,τ ( δ ; F ) holds, we must have then that
[Equation]
[Table]
[Equation]
We note that (i) is a consequence of Lemma 20, while (ii) comes from our assumption that the event
M t 0 ,T s ( F ) holds,
i.e. f ( x t 0 ) − f ( x t
+ T s ) ≤ F .
We next bound V 3 ( t 0 , τ ) and V 4 ( t 0 , τ ) .
Lemma 22.
Let c > 0 denote the same constant in Lemma 7.
Consider any arbitrary 0 < δ ≤ 1 /e, and let τ ≥ t f ( δ ) be arbitrary.
Let N t 0 ,τ ( δ ) denote the event that
[Equation]
where c 6 > 0 is an absolute constant.
Then, by Lemma 7, P ( N t 0 ,τ ( δ )) ≥ 1 − δT .
Denote the event
[Equation]
4 (cid:41) ,
where c 7 > 0 is an absolute constant.
Then, on the event ∩ t 0 + τ − 1 t = t
O t ( δ ) , we have
[Equation]
Moreover, for each t , P ( O t ( δ )) ≥ 1 − δT .
Proof.
The proof for V 3 ( t 0 , τ ) follows directly from Lemma 7, by picking c 6 to be the c that appears in the statement of Lemma 7.
Meanwhile, observe that
[Equation]
Above, to derive (iii), we used the bound that (cid:13)(cid:13)(cid:13)
˜
H t,i (cid:13)(cid:13)(cid:13) ≤ ρu (cid:107) Z t,i (cid:107) .
The ﬁnal inequality is a consequence of our assumption that ∩ t 0 + τ − 1 t = t
O t ( δ ) holds.
Finally, the result that P ( O t ( δ )) ≥ 1 − δT holds due to Lemma 11, where we note that we may pick the absolute constant c 7 to be equal to 2 Cc 4 , where c, C > 0 are the absolute constants that appear in the statement of Lemma 11.
Finally, combining the earlier results, we have the following technical result, which bounds the travelling distance of the iterates in terms of the decrease in function value decrease.
Lemma 23 (Improve or Localize) .
Consider the perturbed zeroth-order update Algorithm 1.
Let c (cid:48) > 0 , c 1 > 0 , c 2 ≥ 1 , c 4 > 0 , c 5 > 0 , c 6 > 0 , c 7 > 0 , C 1 ≥ 1 be the absolute constants deﬁned in the statements of the previous lemmas, and let δ ∈ (0 , 1 /e ] be arbitrary.
Consider any T s ≥ t f ( δ ) .
For any F > 0 , suppose f ( x T s ) − f ( x 0 ) >
− F, i.e. f ( x 0 ) − f ( x T s ) < F .
Suppose that the event
P {+t 0 ,T s ( δ, F ) : = ∩ T s τ =1 ( L t 0 ,τ ( δ ; F ) ∩ N t 0 ,τ ( δ )) ∩ (cid:16) ∩ t 
+}
+ T s − 1 t = t
O t ( δ ) ∩ A t ( δ ) ∩ G t ( δ ) (cid:17) ∩ (cid:16) ∩ T s − 1 τ = t f ( δ ) E t 0 ,τ ( δ ) (cid:17) holds, where the events E t 0 ,τ ( δ ) , L t 0 ,τ ( δ ) , N t 0 ,τ ( δ ) , O t ( δ ) are as deﬁned in Lemma 20, Lemmaand Lemma 22, and G t ( δ ) and A t ( δ ) are as deﬁned in Lemma 13 and Lemma 15.
Suppose we choose u , r and η such that
[Equation]
Suppose η ≤ min (cid:110) 1 , 1 t f ( δ ) , 1 t f δL (cid:111) .
Suppose also we pick u and r small enough such that u ≤
[Equation]

[Equation]
Then, for each τ ∈ { 0 , 1 , . .
, T s } , we have that
[Equation]
, where
[Equation]
{+) max (cid:26) 
 d m (lr( CT 2 /δ )) 2 F, T s F (cid:27) + T s ηt f ( δ ) F,+} where β 1 ( δ ; F ) is deﬁned as in Lemma 21.
Moreover, P ( P t 0 ,T s ( δ, F )) ≥ 1 − 12 T s δ T .
Proof.
We recall that
[Equation]
t
[Equation]
(cid:88) t
[Equation]
m
[Equation]
(cid:88)
i
[Equation]
[Equation]
(cid:13)(cid:13)(cid:13)(cid:13)(cid:13)
[Equation]
t
[Equation]
t
[Equation]
m m
[Equation]
i = uZ
[Equation]
i Z (cid:62)
[Table] [Table] [Table] By Lemma 20, Lemma 21, and Lemma 22, which bound V 1 ( t 0 , τ ) , V 2 ( t 0 , τ ) , and V 3 ( t 0 , τ ) , V 4 ( t 0 , τ ) respectively, on the event P t 0 ,T s ( δ, F ) , we have, for any 0 ≤ τ ≤ T s , (cid:107) x
[Equation]
− x
[Equation]
2 ≤ V 1 (0 , τ ) + V
[Equation]
0 , τ
[Equation]
+ V
[Equation]
[Equation]
τ
[Equation]
+ V
[Equation]
[Equation]
τ ) ≤ max
[Equation]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
(cid:15)
[Equation] (cid:9) [Equation] where N u,r ( τ ; δ ) is deﬁned as in Lemma 20.
For the simpliﬁed bound (which does not contain N u,r ( τ ; δ ) ), it remains for us to show that our choice of u and r ensures that N u,r ( T s , δ ) ≤ F and
[Equation]
First, our choice of u ensures that
[Equation]
Next, recall that
[Equation]
Recalling our choice of η such that
[Equation]
[Equation]
[Equation]
it follows that
[Equation]
where the last inequality follows choosing r such that r 2 ≤
[Equation]
Similarly, we have
[Equation]
By choosing r such that
[Equation]
, it follows that
[Equation]
as desired.
We next lower bound the probability of
P
[Equation]
Observe that ∩
[Equation]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
Note[-this implies that ∩ T s τ = t f ( δ ) E t 0 ,τ ( δ ) ∩ (cid:16) ∩ T s − 1 t = t 
 A t ( δ ) ∩ G t ( δ ) (cid:17)-]
[Table]
We note that by Lemma 1,
[Equation]
[Table]
Meanwhile, we note that
[Equation]
[Table]
[Equation]
[Table] [Equation] [Table] [Equation]
Hence, by Lemma
[Equation]
we have that
[Equation]
Meanwhile, by Lemma 13 and Lemma [Equation] we may bound
[Equation]
Hence, it follows that
[Equation]
Meanwhile, it follows from our results in the preceding lemmas that
[Equation]
Hence, it follows that P ( P t 0 ,T s ( δ, F )) ≥ 1 − 12 T s δ T .
E. P ROVING FUNCTION VALUE DECREASE NEAR SADDLE POINT
We next build on the technical result earlier to prove that each time we are near the saddle point, there is a constant probability of making signiﬁcant function value decrease.
We brieﬂy provide a high-level proof outline below.
In our proof, we introduce a coupling argument connecting two closely-related sequences both starting from the saddle, differing only in the sign of their perturbative term along the minimum eigendirection of the Hessian at the saddle.
Speciﬁcally, when function decrease from a saddle is not sufﬁciently large, due to the earlier technical result, we know that the coupled sequences will remain within a radius φ of the original saddle for a large number (which we will denote as T s ) of iterations.
We then utilize this fact to show that the difference of the coupled sequence will (with some constant probability) grow exponentially large, eventually moving out of their speciﬁed radius φ within T s iterations, leading to a contradiction.
Our ﬁrst result formally introduces the coupling, setting the stage for the rest of our arguments.
For notational convenience, in this section, unless otherwise speciﬁed, we will often assume that the initial iterate x 0 is an (cid:15) -saddle point.
Lemma 3.
Suppose x 0 is an (cid:15) -approximate saddle point.
Without loss of generality, suppose that the minimum eigendirection of H := ∇ 2 f ( x 0 ) is the e 1 direction, and let γ to denote − λ min ( ∇ 2 f ( x 0 ))
(note γ ≥ √ ρ(cid:15) ).
Consider the following coupling mechanism, where we run the zeroth-order gradient dynamics, starting with x 0 , with two isotropic noise sequences, Y t and Y (cid:48) t respectively, where ( Y t ) 1 = − ( Y t ) (cid:48) 1 , and ( Y t )
j = ( Y t ) (cid:48)
j for all other j (cid:54)
= 1 .
Suppose that the sequence { Z t,i } t ∈ T,i ∈ [ m ] is the same for both sequences.
Let { x t } denote the sequence with the { Y t } noise sequence, and let the { x (cid:48) t } denote the sequence with the { Y (cid:48) t } noise sequence, where
[Equation]
and ˜ H (cid:48) t,i := H (cid:48)
t,i, + − H (cid:48) t,i, −, with H (cid:48)
t,i, + = ∇ 2 f (
x (cid:48) t + α (cid:48) t,i, + uZ (cid:48) i ) for some α (cid:48) t,i, + ∈ [0 , 1] , and
H (cid:48) t,i, − = ∇ 2 f ( x t − α (cid:48) t,i, − uZ (cid:48) i ) for some α (cid:48) t,i, − ∈ [0 , 1] .
Then, for any t ≥ 0 , ˆ x t +1 :=
x t +1 − x (cid:48) t +1 = − η t (cid:88) τ
= ( I − ηH ) t − τ ˆ ξ g 0 ( τ ) (cid:124) (cid:123)(cid:122) (cid:125) W g 0 ( t +1) − η t (cid:88) τ
= ( I − ηH ) t − τ ( ¯ H τ − H )
ˆ x τ (cid:124) (cid:123)(cid:122) (cid:125) W H ( t +1) − η t (cid:88) τ
= ( I − ηH ) t − τ ˆ ξ u ( τ ) (cid:124) (cid:123)(cid:122) (cid:125) W u
( t +1) − η t (cid:88) τ
= ( I − ηH ) t − τ ˆ Y τ (cid:124) (cid:123)(cid:122) (cid:125) W p ( t +1) where
[Equation]
(cid:48)
[Equation]
(cid:88) i [Equation] ( Z t,
[Equation]
[Equation]
(cid:62)
[Equation]
[Equation]
(cid:48) t
[Equation]
[Equation]
[Equation]
(cid:48)
[Equation]
[Equation]
[Equation] 1 m m (cid:88) i = u 2 Z t,i Z t,i ˜ H (cid:48) t,i Z t,i , ˆ ξ u ( t ) = ξ u ( t ) − ξ (cid:48) u ( t ) , ˆ Y t =
Y t − Y (cid:48) t , ¯ H t = (cid:90)0 ∇ 2 f ( ax t + (1 − a ) x (cid:48) t ) da.
Proof.
Observe that
[Equation]
t +1) − η t (cid:88) τ
= ( I − ηH ) t
[Equation]
( τ ) (cid:124) (cid:123)(cid:122) (cid:125) W u ( t +1) − η t (cid:88) τ
= ( I − ηH ) t − τ ˆ Y τ (cid:124) (cid:123)(cid:122) (cid:125) W p
( t +1)
where
[Equation]
To derive the ﬁnal equality, we utilized the fact that x (cid:48)
= x 0 .
This completes our proof.
Suppose x 0 is an (cid:15) -saddle point.
Recall that γ > 0 denotes − λ min ( ∇ 2 f ( x 0 )) , where we know that γ ≥ √ ρ(cid:15) . [Equation] L
[Equation]
if f ( · ) is ( (cid:15), ψ ) -strict saddle for any ψ > √ ρ(cid:15)
[Equation]
otherwise . In the sequel, for any t ≥ 0 , it is helpful to deﬁne the quantities
[Equation]
We next introduce some probabilistic events (and their implications) which, if true, can be used to bound the sizes of (cid:107) W g 0 ( t + 1) (cid:107) , (cid:107) W u ( t + 1) (cid:107) , (cid:107) W u ( t + 1) (cid:107) (and as we will see in the next result, indirectly bound (cid:107) W H ( t + 1) (cid:107) .
These bounds will be useful in the ﬁnal proof of making function value progress near a saddle point.
Lemma 24.
We assume δ ∈ (0 , 1 /e ] throughout the lemma.
Suppose that we pick u, r and η as speciﬁed in Lemma 23.
Suppose T s ≥ t f ( δ ) .
Suppose also that
[Equation]
Then, we have the following results.
Let S φ ( δ ) denote the event
[Equation]
, F ) ,
[Equation]
In addition
[Equation]
let S u ( δ ) denote the event
[Equation]
where c 3 is the same absolute constant as the c 3 in the preceding lemmas.
Then,
[Equation]
Consider deﬁning the event R t ( δ ) , which is the event where either
[Equation]
normalsize holds.
Above, c (cid:48) , C refer to the same constants as in Proposition 2, and
G T s ( δ, F ) :=
[Equation]
Then, P ( R t ( δ )) ≥ 1 − δT .
Suppose the event
[Equation]
∩
[Equation]
R
[Equation]
δ
[Equation]
∩ S
[Equation]
δ ) holds. Then, the event S g 0 ( δ ) holds, where
[Equation]
δ
[Equation]
, and S g 0 ,t ( δ ) is deﬁned as S g 0 , [Equation] =
[Equation]
 where
[Equation]
In addition, let S p ( δ ) denote the event
[Equation]
Then, P ( S p ( δ )) ≥ 1 − T s δT .
Proof.
We consider the three claims separately.
Note that our assumptions satisfy the conditions required in Lemma 23.
Hence, by
Lemma 23, on the event P 0 ,T s ( δ, F ) , we have that (cid:107) x
τ − x 0 (cid:107) 2 ≤ φ T s ( δ, F ) .
Simultaneously, on the event P 0 ,T s ( δ, F ) , we know that ∩ T s [Equation] 1 t =
G t ( δ [Equation] holds, i.e.
[Equation]
≤
[Equation]
Thus, for W u ( t + 1) , we have that
[Equation]
[Table] [Equation] [Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
where the inequality in (iv) holds due to Eq.
(45), the equality in (v) holds due to the deﬁnition of β ( t + 1) , and the inequality in (vi) used the fact that γ ≥ ¯ ψ .
Hence the event
[Equation]
δ, [Equation] ) and (cid:111) ∩ S
[Equation]
( δ ) holds with probability at least 1 − 12 T s δ T .
Note that by the coupling, the distribution of x (cid:48) τ is the same as that of x τ .
Thus, by the assumption f
( x (cid:48) T s ) − f ( x 0 ) > − F , it follows by a similar argument that the bound (cid:107) x (cid:48) τ − x 0 (cid:107) 2 ≤
φ T s ( δ, F ) also holds with probability at least 1 − 12 T s δ T .
The claim then follows by an application of the union bound.
For the second claim, observe ﬁrst that the claim P ( R t ( δ )) ≥ 1 − δT is a consequence of
Proposition 2.
Suppose next that f ( x T s ) − f ( x 0 ) >
− F .
Then, by deﬁnition of the event
S φ ( δ ) , we know that
[Equation]
where φ T s ( δ, F ) is as deﬁned in Lemma 23.
Suppose now that R t ( δ ) holds true, and suppose for contradiction
that
[Equation]
This implies that there exists some 0 ≤ τ ≤ t ≤ T s such that (cid:107) x τ − x (cid:48) [Equation] δ [Equation]
However, we also know that on the event S φ ( δ ) ,
[Equation]
This leads to a contradiction.
We must then have that
[Equation]
where
[Equation]
Observe that
[Equation]
which means that W p ( t + 1) is a 1-dimensional Gaussian with variance η 2 t
[Equation]
Since α ( t + 1) ≤ β ( t + 1) , using the subGaussianity of a Gaussian distribution, it follows that for any t , with probability at least 1 − δ/T , [Equation] d
[Equation]
For any F > 0 , we are now ready to show that the algorithm makes a function decrease of F with Ω(1) probability near an (cid:15) -saddle point.
Proposition 5.
Suppose that x t 0 is an (cid:15) -approximate saddle point.
Let c (cid:48) > 0 , c 1 > 0 , c 2 ≥ 1 , c 4 > 0 , c 5 > 0 , c 6 > 0 , c 7 > 0 , C 1 ≥ 1 be the absolute constants deﬁned in the statements of the previous lemmas, and let δ ∈ (0 , 1 /e ] be arbitrary.
Consider any F > 0 .
As in the statement of Lemma 23, suppose we choose u , r and η such that
[Equation]
[Equation]
Suppose we pick
[Equation]
where
[Equation]
=
[Equation]
} if f ( · ) is ( (cid:15), ψ ) -strict saddle for any ψ > √ ρ(cid:15) √
[Equation]
otherwise .
Suppose in addition that u, η also satisfy the conditions
[Equation]
where ζ 1 ( δ, F ) is as deﬁned in Lemma 23, c (cid:48) , c 3 , C > 0 are the same constants as in the previous results, and c 9 = 2 √ 2 + 120 .
Suppose also that φ T s ( δ, F ) satisﬁes the bound
[Equation]
[Equation] Then, with probability at least 13 − 13 T s δ T , f ( x t
+ T s )
− f ( x t 0 ) ≤ − F .
Proof of Proposition 5.
Without loss of generality, we assume that t 0 = 0 .
By Lemma 3, we have ˆ x t + := x
t +1 − x (cid:48) t + = − η t (cid:88) τ
= t
( I − ηH ) t − τ ˆ ξ g 0 ( τ ) (cid:124) (cid:123)(cid:122) (cid:125) W g 0 ( t +1) − η t (cid:88) τ
= t
( I − ηH ) t − τ ( ¯ H τ − H )
ˆ x τ (cid:124) (cid:123)(cid:122) (cid:125) W H ( t +1) − η t (cid:88) τ
= t
( I − ηH ) t − τ ˆ ξ u ( τ ) (cid:124) (cid:123)(cid:122) (cid:125) W u
( t +1) − η t (cid:88) τ
= t
( I − ηH ) t − τ ˆ Y τ (cid:124) (cid:123)(cid:122) (cid:125) W p ( t +1)
where
[Equation]
[Equation]
m m
[Equation]
Recall that we deﬁne for t ≥ 0 ,
[Equation]
Throughout the proof, we suppose for contradiction that
[Equation]
and assume the event
[Equation]
holds, where the events intersected are deﬁned in Lemma 24.
Then, by Lemma 24, the event S g 0 ( δ ) (also deﬁned in Lemma 24) holds 7 .
Consider the following induction argument, where we seek to show that there exists an absolute constant c 9 > 0 such that for every t ∈ { 0 , 1 , . .
, T s } ,
[Equation]
Combined with a lower bound on (cid:107) W p ( t + 1) (cid:107) (which makes use of the property that W p ( t + 1) is a 1-dimensional Gaussian), we will then use the inductive claim in Eq.
(49) to show that
[Equation]
Since W p ( t + 1) is a 1-dimensional Gaussian random variable with a standard deviation that grows exponentially with t , by our choice of T s , we will see that (cid:13)(cid:13) x T s − x (cid:48)
T s (cid:13)(cid:13) is larger than what expect (since our assumptions imply that max (cid:110) (cid:107) x
[Equation] F )
[Equation] i.e. x
[Equation]
s and x (cid:48)
[Equation]
both remain close to x 0 and hence close to each other).
This yields a contradiction, implying that on the event we assumed to hold, i.e.
[Equation]
the assumption
[Equation]
is not true, i.e. one of the sequences must have made function value progress of at least F .
We proceed to prove Eq.
Observe that the claim holds for the base case t = 0 ; this is true since x 0 = x (cid:48) 0 .
Now suppose that this holds for all τ ≤ t .
We will seek to show that Eq.
(49) holds for t
+ 1 as well.
We do so by bounding the norms of W g 0 ( t + 1) , W H ( t + 1) , W u ( t + 1) and W p ( t + 1) respectively.
(Bounding (cid:107) W g 0 ( t + 1) (cid:107) )
Since the event S g
( δ ) holds, it follows that for each 0 ≤ t ≤
T s − 1 , we have that
[Equation]
where ζ
[Equation]
and the terms G T s ( δ, F ) and g (1) are deﬁned as in Lemma 24.
Recall by the inductive claim in Eq.
(49) that there exists c 9 > 0 such that
[Equation]
Hence, it follows that
[Equation]
Hence, noting the choice of T s in Eq.
(47), by choosing η such that
[Equation]
it follows that
[Equation]
Meanwhile, the term W H ( t + 1) can be bounded as follows.
By the inductive assumption in
Eq.
(49), we have that
[Equation]
Moreover, on the event our proof assumes, we know that
[Equation]
Thus, using the ρ -Hessian Lipschitz property
[Equation]
we have
[Equation]
Given our choice of T s in Eq.
(47), if
[Equation]
it follows that
[Equation]
Meanwhile, for W u ( t + 1) , since the event S u ( δ ) holds, we have that
[Equation]
Now, by picking
[Equation]
+ [Equation] it follows that with probability 1 − δ/T , (cid:107) W u ( t + 1) (cid:107) ≤ β ( t +1) ηr 60 √ d [Equation]
Meanwhile, observe that since S p ( δ ) holds, it follows that [Equation] d
[Equation]
Combining the bounds for W g 0 , W p , W H and W u , it follows that
[Equation]
where the ﬁnal inequality uses the fact that 0 < δ ≤ 1 /e (which implies log( T/δ ) ≥ 1 ).
Hence, we see that the ﬁrst part of the inductive claim of Eq.
(49) holds with the constant c 9 := 120 + 2 √ 2 , and the second part follows naturally as a consequence of our argument above.
Meanwhile, observe that for any η such that η ¯ ψ ≤ 12 , we have that (1 + ηγ ) {+
+} η ¯ ψ ≥ 2 .
Thus, by choosing η such that η ¯ ψ ≤ 12 , we have that for any t ≥ 1 η ¯ ψ ,
[Equation]
Hence, following Eq.
(46), by choosing T s ≥ 1 η ¯ ψ , W p ( T s ) is a 1-dimensional Gaussian with variance at least 2 η 2 r 2 β
( T s ) d , such that with probability at least 2/3,
[Equation]
Simultaneously, we know that on the event
[Equation]
S p ( δ ) , we have
[Equation]
We note that by Lemma 24, we have
[Equation]
Thus, with probability at least 2 / 3 − 26 T s δ T , we have
[Equation]
Thus, choosing T s ≥ ιη ¯ ψ , where
[Equation]
= max
[Equation]
, noting that if η ¯ ψ ≤ 1 / 2 , then (1 + ηγ ) 1 η ¯ ψ ≥ (1 + η ¯ ψ ) 1 η ¯ ψ ≥ 2 , we have that with probability at least 2 / 3 − 26 T s δ T ,
[Equation]
Thus, at least one of (cid:107) x T s − x 0 (cid:107) and (cid:13)(cid:13) x (cid:48)
T s − x 0 (cid:13)(cid:13) is larger than (cid:112) φ ( T s , δ ) , a contradiction.
Since the two sequences have the same distribution, it follows that with probability at least 1 / 3 − 13 T s δ T , f ( x T s ) − f ( x 0 ) ≤ − F .
In the result above, we require an upper bound on the norm of φ T s ( δ, F ) to hold (i.e. equation 48), which in turn necessitates an upper bound on F , the function value improvement we can expect to make.
Below, we show how to choose F to be as large as possible (up to constants and logarithmic factors) whilst still satisfying equation 48, assuming that u, r and η are chosen appropriately small such that the dominant term of (cid:107) φ T s ( δ, F ) (cid:107) scales with F .
Lemma 25.
Consider choosing F such that
[Equation]
Suppose η ≤ min (cid:110) 1 , 1 t f ( δ ) , 1 t f δL (cid:111) .
Suppose we pick u and r small enough such that
[Equation]
Then, N u,r ( T s , δ ) ≤ F
[Equation]
and that
[Equation]
Suppose in addition η is small enough
so that
[Equation]
Suppose also that ¯ ψ ≤ 1 8 and η ≤ md , so that T s ≥ ιη ¯ ψ ≥ dm .
Then, the condition in Eq.
(48) will be satisﬁed.
Proof.
We note that since ι ¯ ψ ≤ T s ≤ 2 ι ¯ ψ , it follows by our choice of r that r also satisﬁes the condition
[Equation]
Hence, our choice of η [Equation] u and r satisﬁes the conditions in Lemma
[Equation]
and it follows then that
[Equation]
where β 1 ( δ ; F ) is as deﬁned in Lemma 21.
The condition in Eq.
(48) requires that
[Equation]
By our choice of η such that
[Equation]
it sufﬁces for us to show that
[Equation]
By our assumption, we know that T s ≥ dm .
Thus, further simplifying indicates that it sufﬁces for us to show
[Equation]
By choosing F such that
[Equation]
we see that Eq.
(51) is satisﬁed.
Remark 2 .
Suppose without loss of generality that T s = ιη ¯ ψ .
Then, as a consequence of Lemma 25, we note that the amortized function value progress of decreasing function value by F over T s iterations is
[Equation]
In this section, we prove our main result. First, we need an additional result (Lemma 26) showing that with high probability, we can bound the function value increase if a saddle appears within t f ( δ ) iterations immediately after we have had T s iterations after the previous saddle.
We note that such a bound is necessary because our earlier result upper bounding function increase in τ iterations (see Lemma 18) focused on the case where τ ≥ t f ( δ ) .
Next, we state and prove Theorem 2, which is the precise version of Theorem 1 in the main text.
Lemma 26 (Function change for small τ ) .
Let c 1 > 0 , c 4 > 0 , c 5 > 0 , C 1 ≥ 1 be the absolute constants deﬁned in the statements of the previous lemmas.
Let δ ∈ (0 , 1 /e ] , and suppose τ < t f ( δ ) .
Let J denote the interval { 0 , 1 . . .
, τ − 1 } where τ < t f ( δ ) .
Suppose we choose η such that
[Equation]
Suppose also we pick u, r and η as prescribed in the statement of Proposition 4.
Suppose that min t ∈ J (cid:107)∇ f ( x t ) (cid:107) ≤ (cid:15) .
Then, on the event
[Equation]
, we have the following upper bound on function value change:
[Equation]
Moreover, P ( D τ ( δ )) ≥ 1 − (4 t f ( δ )+4)
δ T .
Proof.
Throughout the proof, we assume that the event D τ ( δ ) holds.
Let J denote { 0 , 1 . . .
, τ − 1 } where τ < t f ( δ ) .
Then, J belongs to one of the two following cases.
Case 1 ) (Gradient dominates noise): Recall that this means that for every t ∈ J , we have
[Equation]
By our choice of η in Eq.
(29), we can apply Lemma 16 to get
[Equation]
Thus by setting α = 128 t f ( δ ) in Eq.
(3) and by choosing η such that
[Equation]
it follows that
[Equation]
where the ﬁnal bound holds since we assumed min t ∈ J (cid:107)∇ f ( x t ) (cid:107) ≤ (cid:15) .
Case 2 ) (Gradient does not dominate noise): there exists some t ∈ J such that
[Equation]
By our choice of η in Eq.
[Equation]
we can apply Lemma 17 to get
[Equation]
Note that, by our choices of the parameters η
[Equation]
u
[Equation]
r
[Equation]
it can be shown that
[Equation]
Hence, by setting α = 128 t f ( δ ) in Eq.
(3) and choosing η such that
[Equation]
it follows that
[Equation] [Equation]
Combining both cases above (Eq.
(53) and Eq.
[Equation]
we see that for the choice α = 128 t f ( δ ) , the bound
[Equation] [Equation]
always holds.
Recall by Eq.
(3) that we have
[Equation]
By plugging in Eq.
(55) above, as well as the choice α = 128 t f ( δ ) , we see that
[Equation]
We can now complete our proof by using the union bound (suppressing the dependence of some of the events on δ for notational simplicity) to derive [Equation]
Armed with Proposition 5 and Lemma
[Equation]
we are now ready to show for T sufﬁciently large [Equation] with high probability, there can be no more than T/ 4 (cid:15) -saddle points.
Combined with Proposition 4, this yields the following result.
Theorem 2.
Suppose we pick u, r, η such that they satisfy the conditions in Proposition 5 and
Lemma 25.
Suppose F is chosen as prescribed in Lemma 25.
Suppose that ¯ ψ ≤ 1 , so that
T s ≥ ιη ¯ ψ ≥ dmL 9 .
Suppose we pick T s as prescribed in Proposition 5.
Suppose in addition we pick r such that
[Equation]

[Equation]
Suppose also that we choose η such that
[Equation]
Suppose
[Equation]
where
[Equation]
Then, with probability at least 1 − 22 δ
[Equation]
there are at least T/ 2 (cid:15) -approximate second order stationary points.
Proof.
Consider deﬁning the following sequence of stopping times:
[Equation]
We note that if τ i = T , then τ j = T for any j > i .
Let N s denote the (random) number of saddle points encountered in T iterations.
We observe that we can decompose the function change as
[Equation]
We ﬁrst consider U 1 .
Letting x j := x T for any j ≥ T , we have that N s
[Equation]
Now, by Eq.
(30), observe that with probability at least 1 − (5 T s +4)
δ T ≥ [Equation] −
[Equation]
(note T s ≥ 4 ), for any 1 ≤ i ≤ T/T s , we have that
[Equation]
Suppose we pick u, r such that M u,r,T s ≤ 0 .
1 F .
Recall from Proposition 5 that with probability at least 1 / 3 − 13 T s δ T , ( f ( x τ i + T s ) − f ( x τ i ))
1 τ i <T ≤ − F .
Choosing δ such that 1 / 3 − 13 T s δ T ≥ 0 .
3 , and letting µ = 0 .
1 F , we note that |− F + µ | = 0 .
9 F ≥ 0 .
2 F ≥ 0 .
3 ( M u,r,T s + µ ) .
Now, let E τ
i denote the bad event on which neither ( f ( x τ i + T s ) − f ( x τ i ))
1 τ i <T ≤ − F, nor (
[Equation] [Equation]
−
[Equation]
M
[Equation]
≤
[Equation]
1 F.
We know that
E τ i has probability at most 6 T s δ T .
Let E τ := ∪ (cid:98) T/T s (cid:99) i =
E τ i , such that P ( E τ ) ≤ 6 δ .
Then, by applying the weakened supermartingale inequality in Proposition 3, we have
P
[Equation]
(cid:18)
[Equation]
Now, pick s = 2 F (cid:113)(cid:112) log(1 /δ )
T/T s , then
P  
T/
[Equation]
(cid:88) i =
( f ( x
[Equation]
i + T s ) − f
[Equation]
Note that supposing for contradiction that there are at least T/ 4 saddles, we must then have that
N s ≥ T/ (4 T s ) , such that
[Equation]
where we may ensure the last inequality by picking T/T s such that
[Equation]
Note that our choice of T ensures this.
Thus, with probability at least 1 − 7 δ ,
[Equation]
Next, we bound the summand U 2 .
Recall that
[Equation]
Without loss of generality, we may analyze each of the summands f
( x τ i +1 ) − f ( x τ i + T s ) in the same way as we treat ( f ( x τ 1 ) − f ( x 0 )) .
Let us then consider the summand f ( x τ 1 ) − f ( x 0 ) .
There are two cases to consider.
The ﬁrst is when τ 1 < t f ( δ ) .
In this case, since we know that (cid:107)∇ f
( x τ 1 ) (cid:107) ≤ (cid:15) (as x τ 1 is an (cid:15) -saddle point), it follows by Lemma
that
[Equation]
with probability at least 1 − (
4 t f δ )+4) δ T .
The second case is when τ 1 ≥ t f ( δ ) .
In this case, by
Lemma
we
have that
[Equation] [Equation]
with probability at least 1 − (5 τ 1 +
[Equation]
T .
By our choice of u
[Equation]
we know that
[Equation]
Hence
[Equation]
by picking r such that
[Equation]
it follows that
[Equation]
Then, if τ 1 < t f ( δ ) , with probability at least 1 − (5 t f ( δ [Equation] +4)
δ
[Equation]
Suppose also that we pick r such that
[Equation]
Then
[Equation]
it can be veriﬁed that
[Equation]
Then
[Equation]
by a union bound
[Equation]
it follows that with probability at least 1 − 9 δ ,
[Equation]
Therefore
[Equation]
by the union bound
[Equation]
with probability at least 1 − 16 δ ,
[Equation]
By recalling our choice of F in Lemma
[Equation]
by choosing η such that
[Equation]
[Table]
[Equation]
it follows that with probability at least 1 − 16 δ ,
[Equation]
Choose T such that
[Equation]
yields a contradiction, where
[Equation]
Hence, with probability at least 1 − 16 δ , there cannot be more than T/ 4 saddle points.
In addition, with probability at least 1 − 6 δ , by Proposition 4, there cannot be more than T/ 4 iterates with (cid:107)∇ f ( x t ) (cid:107) ≥ (cid:15) .
Hence, with probability at least 1 − 22 δ , there are at least T/ 2 (cid:15) -approximate second order stationary points. G S IMULATIONS
We test the performance of our proposed algorithm with two-point estimators (ZOPGD-2pt) against existing zeroth-order benchmarks using the octopus function (proposed in Du et al.
(2017)) of varying dimensions.
It is known that the octopus function deﬁned on R d , which chains d saddle points sequentially, takes exponential (in d ) time for exact gradient descent to escape; it has thus emerged as a popular benchmark to evaluate and compare the performance of algorithms that seek to escape saddle points.
In our experiments, we compare the performance of our two-point estimator algorithm (ZOPGD-2pt) with PAGD (Algorithm 1 in Vlatakis-Gkaragkounis et al.
and ZO-GD-NCF (see Zhang et al.
(2022)), which are the only two existing zeroth-order algorithms that have (a) a ˜
O ( d / (cid:15) 2 ) sample complexity for escaping saddle points (with the latter algorithm yielding the tightest bounds), and (b) performed the best empirically on escaping saddle points (see the simulation results in Zhang et al.
We note that both PAGD and ZO-GD-NCF have to use 2 d function evaluations per iteration to estimate the gradient while our algorithm only needs to use 2 function evaluations.
In our plots, we plot the function value against the number of function evaluations.
For completeness, we also plot the performance of exact gradient descent (normalized such that its x -axis is also the number of function queries).
We tested the algorithms for d = 10 and d = 30 .
To account for the stochasticity in the algorithms, for each algorithm, we computed the average and standard deviation over 30 trials, and plotted the mean trajectory with an additional band that represents 1 .
5 times the standard deviation.
For our algorithm´s hyperparameters, we picked
[Equation]
For PAGD, we used the hyperparameters listed in their paper, and for ZO-GD-NCF, we used the code from their Neurips submission.
We note in particular that both methods used the step-size 14 L .
For initialization, we chose a random x 0 near the saddle point at the origin, drawn from N (0 , 10 − 3 I d × d )(ﬁxed for all trials and all algorithms).
As we can see in Fig.
1, in both cases, our algorithm reaches the global minimum of the octopus function in signiﬁcantly fewer function evaluations than PAGD and ZO-GD-NCF (approximately 2.
times faster than ZO-GD-NCF, and approximately 3 times faster than PAGD), despite our algorithm only using 2 function evaluations per iteration compared to 2 d function evaluations per iteration for both PAGD and ZO-GD-NCF.
As a sanity check, we note that the number of function evaluations required for PAGD and ZO-GD-NCF to reach the global minimum approximately matches that in
Figure 1 of Zhang et al.
; here the correspondence is only approximate since Zhang et al.
(2022) only plots one trial while we compute the mean and standard deviation of 30 trials.
This result suggests that in addition to the theoretical convergence guarantees, there might also be empirical beneﬁts to using two-point estimators versus existing 2 d -point estimators in the zeroth-order escaping saddle point literature.
[Figure]