Zeroth-Order Negative Curvature Finding: Escaping Saddle Points without Gradients
Anonymous Author(s)
AffiliationAddress email
Abstract
We consider escaping saddle points of nonconvex problems where only the functionevaluations can be accessed.
Although a variety of works have been proposed, themajority of them require either second or first-order information, and only a few ofthem have exploited zeroth-order methods, particularly the technique of negativecurvature finding with zeroth-order methods which has been proven to be the mostefficient method for escaping saddle points.
To fill this gap, in this paper, wepropose two zeroth-order negative curvature finding frameworks that can replace
Hessian-vector product computations without increasing the iteration complexity.
We apply the proposed frameworks to ZO-GD, ZO-SGD, ZO-SCSG, ZO-SPIDERand prove that these ZO algorithms can converge to ( ϵ, δ ) -approximate secondorder stationary points with less query complexity compared with prior zeroth-orderworks for finding local minima
Introduction
Nonconvex optimization has received wide attention in recent years due to its popularity in modernmachine learning (ML) and deep learning (DL) tasks.
Specifically, in this paper, we study thefollowing unconstrained optimization problem:
[Equation]
where both f i ( · ) and f ( · ) can be nonconvex.
In general, finding the global optima of nonconvexfunctions is NP-hard.
Fortunately, finding local optima is an alternative because it has been shown intheory and practice that local optima have comparable performance capabilities to global optima inmany machine learning problems [14, 15, 26, 17, 16, 19, 27].
Gradient-based methods have beenshown to be able to find an ϵ -approximate first-order stationary point ( ∥∇ f ( x ) ∥ ≤ ϵ ) efficiently, bothin the deterministic setting ( e.g. , gradient descent [32]; accelerated gradient descent [8, 29]) andstochastic setting ( e.g. , stochastic gradient descent [32, 37]; SCSG [28]; SPIDER [13]).
However, innonconvex settings, first-order stationary points can be local minima, global minima, or even saddlepoints.
Converging to saddle points will lead to highly suboptimal solutions [20, 39] and destroy themodel’s performance.
Thus, escaping saddle points has recently become an important research topicin nonconvex optimization.
Several classical results have shown that, for ρ -Hessian Lipschitz functions (see Definition 1),using the second-order information like computing the Hessian [33] or Hessian-vector products[1, 9, 2], one can find an ϵ -approximate second-order stationary point (SOSP, ∥∇ f ( x ) ∥ ≤ ϵ and∇ 2 f ( x ) ⪰ −√ ρϵ I ).
However, when the dimension of x is large, even once access to the Hessianis computationally infeasible.
A recent line of work shows that, by adding uniform random perturbations, first-order (FO) methods can efficiently escape saddle points and converge to SOSP.
In the deterministic setting, [22] proposed the perturbed gradient descent (PGD) algorithm with gradientquery complexity ˜ O
(log 4 d/ϵ 2 ) by adding uniform random perturbation into the standard gradientdescent algorithm.
This complexity is later improved to ˜ O (log 6 d/ϵ 1 . 75 ) by the perturbed acceleratedgradient descent [24] which replaces the gradient descent step in PGD by Nesterov’s acceleratedgradient descent.
[Table]
PAGD [41] Deterministic
[Table]
Theorem.
[Table]
ZO-SCRN [5]
[Table]
Theorem.
Stochastic
[Table]
Theorem.
Stochastic
CoordGE + (RandGE)
[Table]
[Equation]
Theorem.
Stochastic
CoordGE
[Equation]
† guarantees ( ϵ, O ( √ ϵ ))
-approximate SOSP, and ‡ guarantees ( ϵ, ϵ 2 / 3 ) -approximate SOSP.
Another line of work for escaping saddle points is to utilize the negative curvature finding (NCF),which can be combined with ϵ -approximate first-order stationary point (FOSP) finding algorithmsto find an ( ϵ, δ )-approximate SOSP.
The main task of NCF is to calculate the approximate smallesteigenvector of the Hessian for a given point.
Classical methods for solving NCF like the powermethod and Oja’s method need the computation of Hessian-vector products.
Based on the fact the
Hessian-vector product can be approximated by the finite difference between two gradients, [42, 4]proposed the FO NCF frameworks Neon+ and Neon2, respectively.
In general, adding perturbationsin the negative curvature direction can escape saddle points more efficiently than adding randomperturbations by a factor of ˜ O (poly(log d )) in theory.
Specifically, in the deterministic setting,
CDHS [9] combined with Neon2 can find an ( ϵ, δ )-approximate SOSP in gradient query
complexity˜ O (log d/ϵ 1 . 75 ) .
Recently, the same result was achieved by a simple single-loop algorithm [44], whichcombined the techniques of perturbed accelerated gradient descent and accelerated negative curvaturefinding.
In the online stochastic setting, the best gradient query complexity result ˜ O
(1 /ϵ 3 ) is achievedby SPIDER-SFO + [13], which combined the near-optimal ϵ -approximate FOSP finding algorithm
SPIDER and the NCF framework Neon2 to find an ( ϵ, δ ) -approximate
SOSP.
However, the gradient information is not always accessible.
Many machine learning and deep learningapplications often encounter situations where the calculation of explicit gradients is expensive oreven infeasible, such as black-box adversarial attack on deep neural networks [36, 31, 10, 6, 40] andpolicy search in reinforcement learning [38, 11, 25].
Thus, zeroth-order (ZO) optimization, whichuses function values to estimate the explicit gradients as an important gradient-based black-boxmethod, is one of the best options for solving this type of ML/DL problem.
A considerable bodyof work has shown that ZO algorithms based on gradient estimation have comparable convergencerates to their gradient-based counterparts.
Although many gradient estimation-based ZO algorithmshave been proposed in recent years, most of them focus on the performance of converging to FOSPs[34, 18, 21, 13], and only a few of them on SOSPs [23, 41, 30, 5].
As mentioned above, although there have been several works of finding local minima via ZOmethods, they utilized the techniques of random perturbations [23, 41], random search [30], andcubic regularization [5], as shown in Table 1, which are not the most efficient ones of escaping saddlepoints as discussed before.
Specifically, in the deterministic setting, [23] proposed the ZO perturbedstochastic gradient (ZPSGD) method, which uses a batch of Gaussian smoothing based stochastic ZOgradient estimators and adds a random perturbation in each iteration.
As a result, ZPSGD can find an ϵ -approximate SOSP using ˜ O (cid:0) d 2 /ϵ 5 (cid:1) function queries.
[41] proposed the perturbed approximategradient descent (PAGD) method which iteratively conducts the gradient descent steps by utilizingthe forward difference version of the coordinate-wise gradient estimators until it reaches a point witha small gradient.
Then, PAGD adds a uniform perturbation and continues the gradient descent steps.
The total function queries of PAGD to find an ϵ -approximate SOSP is ˜ O (cid:0) d log 4 d/ϵ 2 (cid:1) .
Recently,[30] proposed the random search power iteration (RSPI) method, which alternately performs randomsearch steps and power iteration steps.
The power iteration step contains an inexact power iterationsubroutine using only the ZO oracle to conduct the NCF, and the core idea is to use a finite differenceapproach to approximate the Hessian-vector product.
In the stochastic setting, [5] proposed a zerothorder stochastic cubic regularization newton (ZO-SCRN) method with function query complexity˜ O (cid:0) d/ϵ 7 / 2 (cid:1) using Gaussian sampling-based gradient estimator and Hessian estimator.
Unfortunately,each iteration of ZO-SCRN needs to solve a cubic minimization subproblem, which does not have aclosed-form solution.
Typically, inexact solvers for solving the cubic minimization subproblem needadditional computations of the Hessian-vector product [1] or the gradient [7].
Thus, it is then natural to explore faster ZO negative curvature finding based algorithms to makeescaping saddle points more efficient.
To the best of our knowledge, negative curvature findingalgorithms with access only to ZO oracle is still a vacancy in the stochastic setting.
Inspired bythe fact that the gradient can be approximated by the finite difference of function queries with highaccuracy, a natural question is: Can we turn FO NCF methods (especially the state-of-the-art Neon2)into ZO methods without increasing the iteration complexity and turn ZO algorithms of finding FOSPsinto the ones of finding SOSPs?
Contributions.
We summarize our main contributions as follows:
• We give an affirmative answer to the above question.
We propose two ZO negative curvaturefinding frameworks, which use only function queries and can detect whether there is a negativecurvature direction at a given point x on a smooth, Hessian-Lipschitz function f : R d → R inoffline deterministic and online stochastic settings, respectively.•
We apply the proposed frameworks to four ZO algorithms and prove that these ZO algorithms canconverge to ( ϵ, δ )-approximate SOSPs, which are ZO-GD, ZO-SGD, ZO-SCSG, and ZO-SPIDER.•
In the deterministic setting, compared with the classical setting where δ =
O
( √ ϵ )
[22, 24, 23, 41],or the special case δ =
ϵ 2 / 3 [30], our Theorem 4 is always not worse than other algorithms inTable 1.
In the online stochastic setting, all of our algorithms don’t need to solve the cubicsubproblem as in ZO-SCRN and our Theorem 6 improves the best function query complexity bya factor of ˜ O
(1 / √ ϵ ) .
Preliminaries
Throughout this paper, we use ∥ · ∥ to denote the Euclidean norm of a vector and the spectralnorm of a matrix.
We use ˜ O ( · ) to hide the poly-logarithmic terms.
For a given set S drawn from[ n ] := { 1 , 2 , . . . , n } , define f S ( · ) := 1 |S| (cid:80) i ∈S
f i ( · ) .
Definition 1.
For a twice differentiable nonconvex function f : R d → R ,
• f is ℓ -Lipschitz smooth if ∀ x, y ∈ R d , ∥∇ f ( x ) − ∇ f ( y )
∥ ≤ ℓ ∥ x − y ∥ .
• f is ρ -Hessian Lipschitz if ∀ x, y ∈ R d , ∥∇ 2 f ( x ) − ∇ 2 f ( y )
∥ ≤ ρ ∥ x − y ∥ .
Definition 2.
For a twice differentiable nonconvex function f : R d → R , we say
• x ∈ R d is an ϵ -approximate first-order stationary point if ∥∇ f ( x ) ∥ ≤ ϵ .
• x ∈ R d is an ( ϵ, δ ) -approximate second-order stationary point if ∥∇ f ( x ) ∥ ≤ ϵ, ∇ 2 f ( x ) ⪰ − δ I .
We need the following assumptions which are standard in the literature of finding SOSPs [4, 13, 44].
Assumption 1.
We assume that f ( · ) in (1) satisfies:
• ∆ f := f
( x 0 ) − f ( x ∗ ) < ∞ where x ∗ := argmin x f ( x ) .
• Each component function f i ( x ) is ℓ -Lipschitz smooth and ρ -Hessian Lipschitz.
• (For online case only)
The variance of the stochastic gradient is bounded: ∀ x ∈ R d , E ∥∇ f i
x ) −∇ [Equation]
We’ll also need the following more stringent assumption to get high-probability convergence resultsof ZO-SPIDER.
Assumption 2.
We assume that Assumption 1 holds, and in addition, the gradient of each componentfunction f i ( x ) satisfies ∀ i, x ∈ R d , ∥∇ f i ( x ) − ∇ f ( x [Equation] ∥ 2 ≤ σ 2 .
2.1 ZO Gradient Estimators
Given a smooth, Hessian Lipschitz function f , a central difference version of the deterministiccoordinate-wise gradient estimator is defined by
[Equation]
where e i denotes a standard basis vector with 1 at its i -th coordinate and 0 otherwise; µ is thesmoothing parameter, which is a sufficient small positive constant.
A central difference version of therandom gradient estimator is defined by
[Equation]
where u ∈ R d is a random direction drawn from a uniform distribution over the unit sphere; µ is thesmoothing parameter, which is a sufficient small positive constant.
Remark 1.
Deterministic vs. Random : CoordGradEst needs d times more function queries than
RandGradEst.
However, as will be discussed in section 4, it has a lower approximation error andthus can reduce the iteration complexity.
Central Difference vs. Forward Difference (please refer to
Appendix A.1): Under the assumption of Hessian Lipschitz, a smaller approximation error bound canbe obtained by the central difference version of both CoordGradEst and RandGradEst.
2.2 ZO Hessian-Vector Product Estimator
By the definition of derivative: ∇ 2 f ( x ) · v = lim µ → [Equation] ( x [Equation]
) −∇ f ( x ) µ , we have ∇ 2 f ( x ) · v can beapproximated by the difference of two gradients ∇ f ( x + v ) −∇ f ( x ) for some v with small magnitude.
On the other hand, ∇ f ( x + v ) , ∇ f ( x ) can be approximated by ˆ ∇ coord f
( x + v )
, ˆ ∇ coord f ( x ) withhigh accuracy, respectively.
Then the coordinate-wise Hessian-vector product estimator is defined by:
[Equation]
Note that we do not need to know the explicit representation of H f ( x ) .
It is merely used as a notationfor a virtual matrix and can be viewed as the Hessian ∇ 2 f ( x 0 ) with minor perturbations.
As statedin the following lemma, the approximation error is efficiently upper bounded.
Lemma 1.
Assume that f is ρ -Hessian Lipschitz, then for any smoothing parameter µ and x ∈ R d ,we have
[Equation]
The ZO Hessian-vector product estimator was previously studied in [43, 30], but we provide a tighterbound than that in Lemma 6 in [30].
This is because we utilize properties of the central differenceversion of the coordinate-wise gradient estimator under the Hessian Lipschitz assumption.
It is thendirectly concluded that, if f ( · ) is quadratic, we have ρ = 0 and ∥H f ( x ) v − ∇ 2 f ( x ) v ∥ = 0 .
Zeroth-Order Negative Curvature Finding
In this section, we introduce how to find the negative curvature direction near the saddle point usingzeroth-order methods.
Recently, based on the fact that the ∇ 2 f (
) · v can
1: η ← δ C 20 ℓ 2 log(100 d ) , T ← C 20 log(100 d ) ηδ , σ ← η 2 δ 3 (100 d ) 3 C 0 ρ , r ← (100 d ) C 0 σ 2: ξ ← σ ξ ′ ∥
ξ ′ ∥ , with ξ ′ ∼ N (0 , I ) 3: x 1 ← x 0 + ξ 4: for t = 1 , . .
, T do 5: µ t ← ∥ x t − x 0 ∥ 6: x t +1 = x
t − η H
f i ( x 0 )( x t − x 0 ) with µ = µ t and i ∈ [ n ] 7: if ∥ x t +1 − x
∥ ≥ r then return v = x s − x∥ x s − x 0 ∥ for a uniformly random s ∈ [ t ]
Return v = ⊥
Input: f ( · ) = 1 n (cid:80) ni
=1 f i ( · ) , x 0 , δ > 0 , p ∈ (0 , 1] .
1: for j = 1 , 2 , · · · , Θ(log(1 /p )) do 2: v j ← ZO-NCF-Online-Weak
( f, x 0 , δ ) 3: if v j ̸ = ⊥
then 4: m ← Θ( ℓ 2 log(1 /p )
δ) , v ′ ← Θ( δdρ ) v j 5:
Draw i 1 , . .
, i m uniformly randomly from [ n ] 6: z j = 1 m (cid:80) mk =1 ( v ′ )
T H fik ( x 0 ) v ′ ∥ v ′
∥7: if z j ≤ − 3 δ 4 then return v = v j
Return v = ⊥ be approximated by ∇ f ( x + v ) − ∇ f ( x ) with approximation error up to O ( ∥ v ∥ 2 ) , [4] proposeda FO framework named Neon2 that can replace the Hessian-vector product computations in NCFsubroutine with gradient computations and thus can turn a FO algorithm for finding FOSPs into a FOalgorithm for finding SOSPs. Enlightened by Neon2, we propose two zeroth-order NCF frameworks( i.e. , ZO-NCF-Online and ZO-NCF-Deterministic ) using only function queries to solve nonconvexproblems in the online stochastic setting and offline deterministic setting, respectively.
3.1 Stochastic Setting
In this subsection, we focus on solving the NCF problem with zeroth-order methods under theonline stochastic setting and propose ZO-NCF-Online .
Before introducing ZO-NCF-Online , we firstintroduce ZO-NCF-Online-Weak with weak confidence of 2 / 3 for solving the NCF problem.
We summarize ZO-NCF-Online-Weak in Algorithm 1.
Specifically, ZO-NCF-Online-Weak consistsof at most T =
O ( log 2 d δ 2 ) iterations and works as follows:
Given a detection point x 0 , add arandom perturbation with small magnitude σ as the starting point.
At the t -th iteration wheret = 1 , . . .
, T , set µ t =
∥ x t − x 0 ∥ to be the smoothing parameter µ in (2).
Then we keep updatingx t
+1 = x
t − η H
f i ( x 0 )( x t − x 0 ) where H f i ( x 0 )( x t − x 0 ) is the ZO Hessian-vector product estimatorand stops whenever ∥ x t +1 − x 0 ∥ ≥ r or the maximum iteration number T is reached.
Thus aslong as Algorithm 1 does not terminate, we have that the approximation error
∥H f i ( x 0 )( x t − x 0 ) −∇
2 f i ( x 0 )( x t − x 0 ) ∥ can be bounded by O ( √ dr 2 ) according to Lemma 1.
Note that, althoughthe error bound is poorer by a factor of O ( √ d ) as compared to Neon onlineweak in [4] which used thedifference of two gradients to approximate the Hessian-vector product and achieve an approximationerror up to O ( r 2 ) , with our choice of r in Algorithm 1, the error term is still efficiently upper bounded.
Other than the additional error term caused by ZO approximation, the motivation of ZO-NCF-OnlineWeak is almost the same as Neon online weak .
That is, under reasonable control of the approximationerror of the Hessian-vector product, using the update rule of Oja’s method [35] to approximatelycalculate the eigenvector corresponding to the minimum eigenvalue of ∇ 2 f ( x 0 )
= 1 n (cid:80) ni =1 ∇
f i ( x 0 ) .
Under similar analysis, we conclude that as long as the minimum eigenvalue of ∇ 2 f ( x 0 ) satisfiesλ min ( ∇ 2 f ( x 0 ))
≤ − δ , ZO-NCF-Online-Weak will stop before T and find a negative curvaturedirection that aligns well with the eigenvector corresponding to the minimum eigenvalue of ∇ f 2 ( x 0 ) .
Then we have the following lemma:
Lemma 2 (ZO-NCF-Online-Weak) .
The output v of Algorithm 1 satisfies:
If λ min ( ∇ 2 f ( x 0 )) ≤ − δ ,then with probability at least 2 / 3 , v ̸ = ⊥ and v T ∇ 2 f ( x 0 ) v ≤ − 34 δ .
We summarize ZO-NCF-Online in Algorithm 2.
Specifically, ZO-NCF-Online repeatedly calls ZONCF-Online-Weak for Θ(log(1 /p ))
times to boost the confidence of solving the NCF problem from2 / 3 to 1 − p .
We have the following results:
Lemma 3.
In the same setting as in Algorithm 2, define z = 1 m (cid:80) mj
=1 v T ( H f ij ( x 0 )) v .
Then, if∥ v ∥ ≤ δ 16 dρ and m =
Θ( ℓ 2 δ 2 ) , with probability at least 1 − p , we have (cid:12)(cid:12)(cid:12) z
∥ v ∥ 2 − v T ∇ 2 f ( x ) v ∥ v
[Table]
(cid:12)(cid:12)(cid:12) [Table]
Theorem 1.
Let f ( x ) = 1 n (cid:80) ni
=
f i ( x ) where each f i is ℓ -smooth and ρ -Hessian Lipschitz.
Forevery point x 0 ∈ R d , every δ ∈ (0 , ℓ ] , the output of Algorithm 2 v satisfies that, with probability atleast 1 − p : If v = ⊥ , then ∇ 2 f ( x 0 ) ⪰ − δ I ; If v ̸ = ⊥ , then ∥ v ∥ = 1 and v T ∇ 2 f ( x 0 ) v ≤ − δ 2 .
Thetotal function query complexity is O (cid:16) d log 2 ( d/p ) ℓ 2 δ
[Equation]
3.2 Deterministic Setting
In this subsection, we focus on solving the NCF problem with zeroth-order methods under the offlinedeterministic setting and propose ZO-NCF-Deterministic .
We summarize ZO-NCF-Deterministic in
Algorithm 3.
Since we want to compute the eigenvector corresponding to the most negative eigenvalueof ∇ 2 f ( x 0 ) approximately, one can convert it into an approximated top eigenvector computationproblem of M := − 1 ℓ ∇ 2 f ( x 0 )
+ (1 − 3 δ 4 ℓ ) I .
This is because all eigenvalues of ∇ 2 f ( x 0 ) in [ − 3 δ 4 , ℓ ]will be mapped to eigenvalues of M in [ − 1 , 1] , and all eigenvalues of ∇ 2 f ( x 0 ) smaller than − δ willbe mapped to eigenvalues of M greater than 1 + δ 4 ℓ .
[Table]
y 1 ← ξ 4: for t = 1 , . . .
, T do 5: µ t = ∥
y t ∥ 6: y t +1 = 2 M ( y t ) − y t − 1 where M ( y ) =
( − 1 ℓ H f ( x 0 )
+ (1 − 3 δ 4 ℓ ))
y 7: x t +1 = x 0 + y t +1 − M ( y t ) 8: if ∥ x t +1 − x 0 ∥ ≥ r then return v = x
t +1 − x 0 ∥ x t +1 − x 0 ∥
Return v = ⊥ .
Similar to ZO-NCF-Online-Weak , ZO-NCF-Deterministic starts by adding a random perturbationξ to the detection point x 0 .
To find the negative curvature direction v of ∇ 2 f ( x 0 ) such thatv T ∇ 2 f ( x 0 ) v ≤ − δ 2 , the classical power method which updates through x T +
= x 0 + M T ξ[30] will take T ≥ ˜Ω( ℓδ ) number of iterations since eigenvalues of M greater than 1 + δ 4 ℓ grows in aspeed (1 + δ/ℓ ) T .
To reduce the iteration complexity T , we can replace the matrix polynomial M Twith the matrix Chebyshev polynomial T T ( M ) and virtually update x T +
= x 0 + T T ( M ) ξ .
Definition 3.
Chebyshev polynomial {T n ( x ) } n ≥ 0 of the first kind is
T 0 ( x ) = 1 , T 1 ( x ) =
x,
T n +1 ( x ) = 2 x · T n ( x ) − T n − 1 ( x ) , then it satisfies T t ( x ) = (cid:26) cos( n arccos( x )) , x ∈ [ − 1 , 1]
[Equation]
[Equation]
>.
In the matrix case, we have the so-called matrix Chebyshev polynomial T t ( M ) [3], which satisfies:
T t +1 ( M ) ξ = 2 M T t ( M ) ξ − T t − 1 ( M ) ξ .
Thus, eigenvalues of M greater than 1 + δ 4 ℓ will grow to(cid:16)
+ δ/ 4 ℓ + (cid:112) ( δ/ 4 ℓ ) 2 + δ/ 2 ℓ (cid:17) T ≈ (cid:16) 1 + (cid:112) δ/ℓ (cid:17) T , so we only need to choose T ≥ (cid:112) ℓ/δ .
On the other hand, since we only have access to the zeroth-order information, we need to stablycompute the matrix Chebyshev polynomial.
In algorithm 3, we set µ t = ∥
y t ∥ and use M ( y t ) =( − 1 ℓ H f ( x 0 ) + (1 − 3 δ 4 ℓ ))
y t to approximate M y t with approximation error up to 2 ρ √ drtℓ ∥ y t ∥ .
Withproper choice of r , it allows us to use the inexact backward recurrence [3] to ensure a stablecomputation of matrix Chebyshev polynomial:
[Equation]
Then the output x T +1 = x 0 + y T
+
− M ( y T ) is close to x 0 + T T ( M ) ξ with a small approximationerror.
Finally, we have the following theorem:
Theorem 2.
Let f ( x ) = 1 n (cid:80) {+ni =
+}
f i ( x ) where each f i is ℓ -smooth and ρ -Hessian Lipschitz.
Forevery point x 0 ∈ R d , every δ ∈ (0 , ℓ ] , the output of Algorithm 3 v satisfies that, with probability atleast 1 − p : If v = ⊥ , then ∇ 2 f ( x 0 ) ⪰ − δ
I ; If v ̸ = ⊥ , then ∥ v ∥ = 1 and v T ∇ 2 f ( x 0 ) v ≤ − δ 2 .
Thefunction query complexity is O
( d log dp √ ℓ √ δ ) .
Applications of Zeroth-Order Negative Curvature Finding
In this section, we focus on applying the zeroth-order negative curvature frameworks to the following
ZO algorithms: ZO-GD, ZO-SGD, ZO-SCSG, and ZO-SPIDER.
The following result shows that onecan verify if a point x is an ϵ -approximate FOSP using CoordGradEst.
Proposition 1.
In the online setting, using CoordGradEst with a batch size of O (cid:16)(cid:16) 128 σ 2 ϵ+ 1 (cid:17) log 1 p (cid:17)and smoothing parameter µ ≤ (cid:113) ϵ 4 ρ √ d , we can verify with probability at least 1 − p , either∥∇ f ( x ) ∥ ≥ ϵ/ 2 or ∥∇ f ( x ) ∥ ≤ ϵ .
In the deterministic setting, using once computation of CoordGradEst with smoothing parameter µ ≤ (cid:113) 3 ϵ 2 ρ √ d , we can verify with probability 1 , either∥∇ f ( x ) ∥ ≥ ϵ/ 2 or ∥∇ f ( x ) ∥ ≤ ϵ .
4.1 Applying Zeroth-Order Negative Curvature Finding to ZO-GD and ZO-SGD
We apply ZO-NCF-Online to ZO-SGD to turn it into a local minima finding algorithm, and propose
ZO-SGD-NCF in Algorithm 4.
At each iteration, we use a batch size of O (cid:16) σ 2 ϵ 2 log (cid:16) 2 Kp (cid:17)(cid:17) CoordGradEst to verify if x t is an ϵ -approximate stationary point.
If not, ZO-SGD-NCF either estimatesthe gradient ∇ f S ( x t ) = 1 | S | (cid:80)
i ∈ S ∇ f i ( x t ) by CoordGradEst ( Option I ) or RandGradEst ( Option II ) with both mini-batch size O ( σ 2 ϵ 2 )
; If so, we call the ZO-NCF-Online subroutine.
Then, If we findan approximate negative curvature direction v around x t , then we update x t +1 by moving from x t inthe direction v with step-size δ/ρ .
We have the following theorem:
Algorithm
Input: Function f , starting point x 0 , confidence p ∈ (0 , 1) , ϵ > 0 and δ > 0 . 1: for t = 0 , . . . , K − 1 do 2: uniformly randomly choose a set B with batch size O ( σ 2 ϵ 2 log(2 K/p )) 3: if ∥ ˆ ∇ coord f B ( x t ) ∥ ≥ 3 ϵ 4 then 4: uniformly randomly choose S ⊆ [ n ] 5:
Option I : x t +1 ← x t − η ˆ ∇ coord f S ( x t ) 6: Option II : x t +1 ← x t − η ˆ ∇ rand f S ( x t ) 7: else 8: v ← ZO-NCF-Online ( f, x t , δ, p 2 K ) 9: if v = ⊥ then return x t 10: else x t +1 = x t ± δρ v
Theorem 3. Under Assumption 1, we set µ 1 = (cid:113) 3 ϵ 2 ρ √ d and other parameters as follows, Option
[Equation]
Option II: | S | = max { 8 σ 2 ϵ 2 , 1 } [Equation] where µ 1 and µ 2 are only used in Line 3 and Line 5 (or Line 6) of Algorithm 4, respectively.
Withprobability at least 1 − p , Algorithm 4 outputs an ( ϵ, δ ) -approximate local minimum in function querycomplexity [Equation] Input: ϵ > δ > 0 . do 2: if
Option η
∇ 5: else 6: v ← ZO Deterministic ( , pK ) 7: if v
⊥ then return
8: else
{++
+}
= x t ± δρ v
Remark 2.
Note that the dominant term of the function query complexity in Option I is ˜ O ( dϵ 4 ) , whilein Option II is ˜ O
( d 2 ϵ 4 ) .
This is because CoordGradEst has a lower approximation error and thus canreduce the iteration complexity by a factor of d .
Then the function query complexity of Option II isdominated by evaluating the magnitude of the gradient (Line 3 in Algorithm 4).
In the Deterministic setting, we apply ZO-NCF-Deterministic to ZO-GD to turn it into a local minimafinding algorithm and propose ZO-NCF-GD in Algorithm 5.
The update rule of ZO-NCF-GD issimilar to that in ZO-NCF-SGD, the only difference is that we don’t need to use mini-batch samplingof the stochastic gradient.
Similarly, we have the following theorem:
Theorem 4.
Under Assumption 1, we set µ 1 = (cid:113) 3 ϵ 2 ρ √ d and other parameters as follows,
[Equation]
3 ϵ 4 ρd, ϵ 16 √ dℓ (cid:27) , where µ 1 and µ 2 are only used in Line 2 and Line 3 (or Line 4) of Algorithm 5, respectively.
Withprobability at least 1 − p , Algorithm 5 outputs an ( ϵ, δ ) -approximate local minimum in function querycomplexity
Option I: [Equation]
4.2 Applying Zeroth-Order Negative Curvature Finding to ZO-SCSG and ZO-SPIDER
In the stochastic setting, we can also apply the zeroth-order negative curvature finding to the variancereduction-based algorithms: SCSG [28] and SPIDER [13].
Due to space limitation, We defer thedetailed discussions of these applications to Appendix E and F.
To apply ZO-NCF-Online to SCSG, we first propose a zeroth-order variant of the SCSG [28] methodin Algorithm 6.
At the beginning of the j -th epoch, we estimate the gradient ∇ f I j (˜ x j − 1 ) by
CoordGradEst over a batch sampling set I j with size B .
In the inner loop iterations, the stochasticgradient estimator v jk − 1 is either constructed by CoordGradEst or by RandGradEst over a mini-batchsampling set I jk − 1 with size b .
Then we apply ZO-NCF-Online to ZO-SCSG and propose the
ZO-SCSG-NCF method (see Algorithm 7).
Theorem 5 (informal, full version deferred to Appendix E) .
With probability at least 23 , for both
Option I and Option II , Algorithm 7 outputs an ( ϵ, δ ) -approximate local minimum in function querycomplexity
[Equation]
[-The problem described in Remark 2 doesn’t exist in ZO-SCSG-NCF as we only evaluatethe magnitude of the gradient after each epoch ( i.e. , Line 4 in Algorithm 7), and the function querycomplexity is almost the same in the inner loop for both Option I and Option II .4.
-]
We apply ZO-NCF-Online to ZO-SPIDER to turn it into a local minima finding algorithm and propose ZO-SPIDER-NCF in Algorithm 8.
As a by-product, we also propose a zeroth-order variant of the SPIDER method in Appendix G that can converge to an ϵ -approximate FOSP with high probability rather than expectation.
Using the same technique as in SPIDER-SFO + [13], that is, instead ofmoving in a large single step with size δ/ρ along the approximate negative curvature direction asin ZO-SGD-NCF and ZO-SCSG-NCF, we can split it into δ/ ( ρη ) equal length mini-steps with sizeη .
As a result, we can maintain the SPIDER estimates and improve the so-called non-improvablecoupling term 1 δ 3 ϵ 2 by a fact of δ .
Theorem 6 (informal, full version deferred to Appendix F) .
With probability at least 34 , Algorithm 8outputs an ( ϵ, δ ) -approximate local minimum in function query complexity
[Equation]
Remark 3.
[-Under Assumption 2, if we set ˜ ϵ = 10 ϵ log(
 ( K 0 +1)) , |S-]We can boost the confidence the of Theorem 5 and 6 to 1 − p by running log(1 /p ) copiesof Algorithm 7 and 8.  Numerical Experiments
Octopus Function.
We first consider the octopus function proposed by Du et al. The octopusfunction has only one local optimum: x ∗ = (4 τ, . . . , 4 τ ) T and d saddle points:(0 , . . . , 0) T , (4 τ, 0 , . .
. , 0) T , . . . , (4 τ, . . , 4 τ, 0) T . We compare ZO-GD-NCF, ZPSGD, PAGD, and RSPI on the octopus function with growing dimensions.
The parameters corresponding to the octopus function are set with τ = [-10 ϵ log(
-]e, L = e, γ = 1 .
Allalgorithms are initialized at point (0 , . . . , 0) T , which is a strict saddle point and the one farthest fromthe optimal point among the d saddle points. We set ϵ = 1 e − 4 , δ = √ ρϵ for all experiments and report the function value v.s. the number offunction queries in Figure 1. For RSPI, we follow the hyperparameter update strategy as described in([30], Appendix, Section F): We keep σ 2 constant and update σ 1 = ρ σ 1 σ 1 every T σ 1 iterations.
Weconduct a grid search for T σ 1 and ρ σ 1 . [Figure] The results in Figure 1 illustrate that all algorithms are able to escape saddle points. With the increaseof the dimension of the octopus function, more function queries are needed for each algorithm toconverge to the local minimum.
Note that in all experiments, RSPI performs worse than PAGD and ZO-GD-NCF. This is because RSPI is not a gradient based algorithm. Although it can efficientlyescape from the saddle point using the negative curvature finding, it converges very slowly when thecurrent point is far from the saddle point due to the random search.
Due to space limitation, we defer more experimental results to Appendix G.  Conclusion
In this paper, we analyse two types of ZO negative curvature finding frameworks, which can beused to find the negative curvature directions near a saddle point in the deterministic setting andstochastic setting, respectively. We apply the two frameworks to four ZO algorithms and analyse thecomplexities for converging to ( ϵ, δ ) -approximate SOSPs. Finally, we conduct several numericalexperiments to verify the effectiveness of the proposed method in escaping saddle points. As a future work, it would be interesting to study the (zeroth-order) unified negative curvature findingframeworks with generic analysis that can be applied to any FOSPs finding algorithms.
Checklist
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] In the conclusion, we mention away that could improve the current work.
(c) Did you discuss any potential negative societal impacts of your work?
[No](d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] Please refer to the Appendix for complete proofs of the theoretical results.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [Yes]
(b) Did you specify all the training details (e g., data splits, hyperparameters, how they were chosen)?
[Yes](c)
Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?
[No]
(d) Did you include the total amount of compute and the type of resources used (e g., typeof GPUs, internal cluster, or cloud provider)?
[No]4.
If you are using existing assets (e g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re [Table]
using/curating?
[N/A]
[Table]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
[Table]
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
[Table]
(a) Did you include the full text of instructions given to participants and screenshots, if
[Table]
applicable?
[N/A]
[Table]
) Did you describe any potential participant risks, with links to Institutional Review
[Table]
Board (IRB) approvals, if applicable?
[N/A]
[Table]
(c) Did you include the estimated hourly wage paid to participants and the total amount [Table] spent on participant compensation?
[N/A] [Table]