Zeroth-Order Negative Curvature Finding: Escaping Saddle Points without Gradients
Anonymous Author(s)
AffiliationAddress email
Abstract
We consider escaping saddle points of nonconvex problems where only the functionevaluations can be accessed.
Although a variety of works have been proposed, themajority of them require either second or first-order information, and only a few ofthem have exploited zeroth-order methods, particularly the technique of negativecurvature finding with zeroth-order methods which has been proven to be the mostefficient method for escaping saddle points.
To fill this gap, in this paper, wepropose two zeroth-order negative curvature finding frameworks that can replace
Hessian-vector product computations without increasing the iteration complexity.
We apply the proposed frameworks to ZO-GD, ZO-SGD, ZO-SCSG, ZO-SPIDERand prove that these ZO algorithms can converge to ( ϵ, δ ) -approximate secondorder stationary points with less query complexity compared with prior zeroth-orderworks for finding local minima
Introduction
Nonconvex optimization has received wide attention in recent years due to its popularity in modernmachine learning (ML) and deep learning (DL) tasks.
Specifically, in this paper, we study thefollowing unconstrained optimization problem:
[Equation]
where both f i ( · ) and f ( · ) can be nonconvex.
In general, finding the global optima of nonconvexfunctions is NP-hard.
Fortunately, finding local optima is an alternative because it has been shown intheory and practice that local optima have comparable performance capabilities to global optima inmany machine learning problems [13, 14, 25, 16, 15, 18, 26].
Gradient-based methods have beenshown to be able to find an ϵ -approximate first-order stationary point ( ∥∇ f ( x ) ∥ ≤ ϵ ) efficiently, bothin the deterministic setting ( e.g. , gradient descent [31]; accelerated gradient descent [8, 28]) andstochastic setting ( e.g. , stochastic gradient descent [31, 36]; SCSG [27]; SPIDER [12]).
However, innonconvex settings, first-order stationary points can be local minima, global minima, or even saddlepoints.
Converging to saddle points will lead to highly suboptimal solutions [19, 38] and destroy themodel’s performance.
Thus, escaping saddle points has recently become an important research topicin nonconvex optimization.
Several classical results have shown that using the second-order information like computing the Hessian [32] or Hessian-vector products [1, 9, 2], one can find an ϵ -approximate second-orderstationary point (SOSP, ∥∇ f ( x ) ∥ ≤ ϵ and ∇ 2 f ( x ) ⪰ −√ ρϵ I ).
However, when the dimension ofx is large, even once access to the Hessian is computationally infeasible.
A recent line of workshows that, by adding uniform random perturbations, first-order (FO) methods can efficiently escapesaddle points and converge to SOSP.
In the deterministic setting, [21] proposed the perturbed gradient descent (PGD) algorithm with gradient query complexity ˜ O
(log 4 d/ϵ 2 ) by adding uniform randomperturbation into the standard gradient descent algorithm.
This complexity is later improved to˜ O (log 6 d/ϵ 1 . 75 ) by the perturbed accelerated gradient descent [23] which replaces the gradientdescent step in PGD by Nesterov’s accelerated gradient descent.
[Table]
[Table]
[Table]
Theorem.
Deterministic
[Table]
ZO-SCRN [5]
Stochastic
[Table]
Theorem.
Stochastic
[Table]
Theorem.
Stochastic
CoordGE + (RandGE)
[Table]
Theorem.
Stochastic
CoordGE
[Equation]
† guarantees ( ϵ, O ( √ ϵ ))
-approximate SOSP, and ‡ guarantees ( ϵ, ϵ 2 / 3 ) -approximate SOSP.
Another line of work for escaping saddle points is to utilize the negative curvature finding (NCF),which can be combined with ϵ -approximate first-order stationary point (FOSP) finding algorithmsto find an ( ϵ, δ )-approximate SOSP.
The main task of NCF is to calculate the approximate smallesteigenvector of the Hessian for a given point.
Classical methods for solving NCF like the powermethod and Oja’s method need the computation of Hessian-vector products.
Based on the fact the
Hessian-vector product can be approximated by the finite difference between two gradients, [41, 4]proposed the FO NCF frameworks Neon+ and Neon2, respectively.
In general, adding perturbationsin the negative curvature direction can escape saddle points more efficiently than adding randomperturbations by a factor of ˜ O (poly(log d )) in theory.
Specifically, in the deterministic setting,
CDHS [9] combined with Neon2 can find an ( ϵ, δ )-approximate SOSP in gradient query
complexity˜ O (log d/ϵ 1 . 75 ) .
Recently, the same result was achieved by a simple single-loop algorithm [43], whichcombined the techniques of perturbed accelerated gradient descent and accelerated negative curvaturefinding.
In the online stochastic setting, the best gradient query complexity result ˜ O
(1 /ϵ 3 ) is achievedby SPIDER-SFO + [12], which combined the near-optimal ϵ -approximate FOSP finding algorithm
SPIDER and the NCF framework Neon2 to find an ( ϵ, δ ) -approximate
SOSP.
However, the gradient information is not always accessible.
Many machine learning and deep learningapplications often encounter situations where the calculation of explicit gradients is expensive oreven infeasible, such as black-box adversarial attack on deep neural networks [35, 30, 10, 6, 39] andpolicy search in reinforcement learning [37, 11, 24].
Thus, zeroth-order (ZO) optimization, whichuses function values to estimate the explicit gradients as an important gradient-based black-boxmethod, is one of the best options for solving this type of ML/DL problem.
A considerable bodyof work has shown that ZO algorithms based on gradient estimation have comparable convergencerates to their gradient-based counterparts.
Although many gradient estimation-based ZO algorithmshave been proposed in recent years, most of them focus on the performance of converging to FOSPs[33, 17, 20, 12], and only a few of them on SOSPs [22, 40, 29, 5].
As mentioned above, although there have been several works of finding local minima via ZOmethods, they utilized the techniques of random perturbations [22, 40], random search [29], andcubic regularization [5], as shown in Table 1, which are not the most efficient ones of escaping saddlepoints as discussed before.
Specifically, in the deterministic setting, [22] proposed the ZO perturbedstochastic gradient (ZPSGD) method, which uses a batch of Gaussian smoothing based stochastic ZOgradient estimators and adds a random perturbation in each iteration.
As a result, ZPSGD can find anϵ -approximate SOSP using ˜ O (cid:0) d 2 /ϵ 5 (cid:1) function queries.
[40] proposed the perturbed approximate gradient descent (PAGD) method which iteratively conducts the gradient descent steps by utilizingthe forward difference version of the coordinate-wise gradient estimators until it reaches a point witha small gradient.
Then, PAGD adds a uniform perturbation and continues the gradient descent steps.
The total function queries of PAGD to find an ϵ -approximate SOSP is ˜ O (cid:0) d log 4 d/ϵ 2 (cid:1) .
Recently,[29] proposed the random search power iteration (RSPI) method, which alternately performs randomsearch steps and power iteration steps.
The power iteration step contains an inexact power iterationsubroutine using only the ZO oracle to conduct the NCF, and the core idea is to use a finite differenceapproach to approximate the Hessian-vector product.
In the stochastic setting, [5] proposed a zerothorder stochastic cubic regularization newton (ZO-SCRN) method with function query complexity˜ O (cid:0) d/ϵ 7 / 2 (cid:1) using Gaussian sampling-based gradient estimator and Hessian estimator.
Unfortunately,each iteration of ZO-SCRN needs to solve a cubic minimization subproblem, which does not have aclosed-form solution.
Typically, inexact solvers for solving the cubic minimization subproblem needadditional computations of the Hessian-vector product [1] or the gradient [7].
Thus, it is then natural to explore faster ZO negative curvature finding based algorithms to makeescaping saddle points more efficient.
To the best of our knowledge, negative curvature findingalgorithms with access only to ZO oracle is still a vacancy in the stochastic setting.
Inspired bythe fact that the gradient can be approximated by the finite difference of function queries with highaccuracy, a natural question is: Can we turn FO NCF methods (especially the state-of-the-art Neon2)into ZO methods without increasing the iteration complexity and turn ZO algorithms of finding FOSPsinto the ones of finding SOSPs?
Contributions.
We summarize our main contributions as follows:
• We give an affirmative answer to the above question.
We propose two ZO negative curvaturefinding frameworks, which use only function queries and can detect whether there is a negativecurvature direction at a given point x on a smooth, Hessian-Lipschitz function f : R d → R inoffline deterministic and online stochastic settings, respectively.•
We apply the proposed frameworks to four ZO algorithms and prove that these ZO algorithms canconverge to ( ϵ, δ )-approximate SOSPs, which are ZO-GD, ZO-SGD, ZO-SCSG, and ZO-SPIDER.•
In the deterministic setting, compared with the classical setting where δ =
O
( √ ϵ )
[21, 23, 22, 40],or the special case δ =
ϵ 2 / 3 [29], our Theorem 4 is always not worse than other algorithms inTable 1.
In the online stochastic setting, all of our algorithms don’t need to solve the cubicsubproblem as in ZO-SCRN and our Theorem 6 improves the best function query complexity bya factor of ˜ O
(1 / √ ϵ ) .
Preliminaries
Throughout this paper, we use ∥ · ∥ to denote the Euclidean norm of a vector and the spectralnorm of a matrix.
We use ˜ O ( · ) to hide the poly-logarithmic terms.
For a given set S drawn from[ n ] := { 1 , 2 , . . . , n } , define f S ( · ) := 1 |S| (cid:80) i ∈S
f i ( · ) .
Definition 1.
For a twice differentiable nonconvex function f : R d → R ,
• f is ℓ -Lipschitz smooth if ∀ x, y ∈ R d , ∥∇ f ( x ) − ∇ f ( y )
∥ ≤ ℓ ∥ x − y ∥ .
• f is ρ -Hessian Lipschitz if ∀ x, y ∈ R d , ∥∇ 2 f ( x ) − ∇ 2 f ( y )
∥ ≤ ρ ∥ x − y ∥ .
Definition 2.
For a twice differentiable nonconvex function f : R d → R , we say
• x ∈ R d is an ϵ -approximate first-order stationary point if ∥∇ f ( x ) ∥ ≤ ϵ .
• x ∈ R d is an ( ϵ, δ ) -approximate second-order stationary point if ∥∇ f ( x ) ∥ ≤ ϵ, ∇ 2 f ( x ) ⪰ − δ I .
We need the following assumptions which are standard in the literature of finding SOSPs [4, 12, 43].
Assumption 1.
We assume that f ( · ) in (1) satisfies:
• ∆ f := f
( x 0 ) − f ( x ∗ ) < ∞ where x ∗ := argmin x f ( x ) .
• Each component function f i ( x ) is ℓ -Lipschitz smooth and ρ -Hessian Lipschitz.
• (For online case only)
The variance of the stochastic gradient is bounded: ∀ x ∈ R d , E ∥∇ f i ( x ) −
∇ f ( x ) ∥ 2 ≤ σ 2 .
We’ll also need the following more stringent assumption to get high-probability convergence resultsof ZO-SPIDER in subsection 4.3.
Assumption 2.
We assume that Assumption 1 holds, and in addition, the gradient of each componentfunction f i ( x ) satisfies ∀ i, x ∈ R d , ∥∇ f i ( x ) − ∇ f ( x ) ∥ 2 ≤ σ 2 .
2.1 ZO Gradient Estimators
Given a smooth, Hessian Lipschitz function f , a central difference version of the deterministiccoordinate-wise gradient estimator is defined by
[Equation]
where e i denotes a standard basis vector with 1 at its i -th coordinate and 0 otherwise.
A centraldifference version of the random gradient estimator is defined by
[Equation]
where u ∈ R d is a random direction drawn from a uniform distribution over the unit sphere.
Remark 1.
Deterministic vs. Random : CoordGradEst needs d times more function queries than
RandGradEst.
However, as will be discussed in section 4, it has a lower approximation error andthus can reduce the iteration complexity.
Central Difference vs. Forward Difference (please refer to
Appendix B.1): Under the assumption of Hessian Lipschitz, a smaller approximation error bound canbe obtained by the central difference version of both CoordGradEst and RandGradEst.
2.2 ZO Hessian-Vector Product Estimator
By the definition of derivative: ∇ 2 f ( x ) · v = lim µ → 0 ∇ f ( x + µv ) −∇ f
( x ) µ , we have ∇ 2 f ( x ) · v can beapproximated by the difference of two gradients ∇ f ( x + v ) −∇ f ( x ) for some v with small magnitude.
On the other hand, ∇ f ( x + v ) , ∇ f ( x ) can be approximated by ˆ ∇ coord f
( x + v )
, ˆ ∇ coord f ( x ) withhigh accuracy, respectively.
Then the coordinate-wise Hessian-vector product estimator is defined by:
[Equation]
Note that we do not need to know the explicit representation of H f ( x ) .
It is merely used as a notationfor a virtual matrix and can be viewed as the Hessian ∇ 2 f ( x 0 ) with minor perturbations.
As statedin the following lemma, the approximation error is efficiently upper bounded.
Lemma 1.
If f is assumed to be ρ -Hessian Lipschitz, then for any smoothing parameter µ andx ∈ R d , we have
[Equation]
The ZO Hessian-vector product estimator was previously studied in [42, 29], but we provide a tighterbound than that in Lemma 6 in [29].
This is because we utilize properties of the central differenceversion of the coordinate-wise gradient estimator under the Hessian Lipschitz assumption.
It is thendirectly concluded that, if f ( · ) is quadratic, we have ρ = 0 and ∥H f ( x ) v − ∇ 2 f ( x ) v ∥ = 0 .
Zeroth-Order Negative Curvature Finding
In this section, we introduce how to find the negative curvature direction near the saddle point usingzeroth-order methods.
Recently, based on the fact that the Hessian-vector product ∇ 2 f ( x ) · v canbe approximated by ∇ f ( x + v ) − ∇ f ( x ) with approximation error up to O ( ∥ v ∥ 2 ) , [4] proposeda FO framework named Neon2 that can replace the Hessian-vector product computations in NCFsubroutine with gradient computations and thus can turn a FO algorithm for finding FOSPs into a FOalgorithm for finding SOSPs.
Enlightened by Neon2, we propose two zeroth-order NCF frameworks( i.e. , ZO-NCF-Online and ZO-NCF-Deterministic ) using only function queries to solve nonconvexproblems in the online stochastic setting and offline deterministic setting, respectively.
1: η ← δ C 20 ℓ 2 log(100 d ) , T ← C 20 log(100 d ) ηδ , σ ← η 2 δ 3 (100 d ) 3 C 0 ρ , r ← (100 d ) C 0 σ 2: ξ ← σ ξ ′ ∥
ξ ′ ∥ , with ξ ′ ∼ N (0 , I ) 3: x 1 ← x 0 + ξ 4: for t = 1 , . .
, T do 5: µ t ← ∥ x t − x 0 ∥ 6: x t +1 = x
t − η H
f i ( x 0 )( x t − x 0 ) with µ = µ t and i ∈ [ n ] 7: if ∥ x t +1 − x
∥ ≥ r then return v = x s − x∥ x s − x 0 ∥ for a uniformly random s ∈ [ t ]
Return v = ⊥
Input: f ( · ) = 1 n (cid:80) ni
=1 f i ( · ) , x 0 , δ > 0 , p ∈ (0 , 1] .
1: for j = 1 , 2 , · · · , Θ(log(1 /p )) do 2: v j ← ZO-NCF-Online-Weak
( f, x 0 , δ ) 3: if v j ̸ = ⊥
then 4: m ← Θ( ℓ 2 log(1 /p )
δ) , v ′ ← Θ( δdρ ) v j 5:
Draw i 1 , . .
, i m uniformly randomly from [ n ] 6: z j = 1 m (cid:80) mj =1 ( v ′ )
T H fij ( x 0 ) v ′ ∥ v ′
∥7: if z j ≤ − 3 δ 4 then return v = v j
Return v = ⊥
3.1 Stochastic Setting
In this subsection, we focus on solving the NCF problem with zeroth-order methods under theonline stochastic setting and propose ZO-NCF-Online .
Before introducing ZO-NCF-Online , we firstintroduce ZO-NCF-Online-Weak with weak confidence of 2 / 3 for solving the NCF problem.
We summarize ZO-NCF-Online-Weak in Algorithm 1.
Specifically, ZO-NCF-Online-Weak consistsof at most T =
O ( log 2 d δ 2 ) iterations and works as follows:
Given a detection point x 0 , add arandom perturbation with small magnitude σ as the starting point.
At the t -th iteration wheret = 1 , . . .
, T , set µ t =
∥ x t − x 0 ∥ to be the smoothing parameter µ in (2).
Then we keep updatingx t
+1 = x
t − η H
f i ( x 0 )( x t − x 0 ) where H f i ( x 0 )( x t − x 0 ) is the ZO Hessian-vector product estimatorand stops whenever ∥ x t +1 − x 0 ∥ ≥ r or the maximum iteration number T is reached.
Thus aslong as Algorithm 1 does not terminate, we have that the approximation error
∥H f i ( x 0 )( x t − x 0 ) −∇
2 f i ( x 0 )( x t − x 0 ) ∥ can be bounded by O ( √ dr 2 ) according to Lemma 1.
Note that, although theerror bound is poor by a factor of O ( √ d ) as compared to Neon onlineweak in [4] which used the differenceof two gradients to approximate the Hessian-vector product and achieve an approximation error up to O ( r 2 ) . With proper choice of r in Algorithm 1, our error term is still efficiently upper bounded.
Other than the additional error term caused by ZO approximation, the motivation of ZO-NCF-OnlineWeak is almost the same as Neon onlineweak .
That is, under reasonable control of the approximationerror of the Hessian-vector product, using the update rule of Oja’s method [34] to approximatelycalculate the eigenvector corresponding to the minimum eigenvalue of ∇ 2 f ( x 0 )
= 1 n (cid:80) ni =1 ∇
f i ( x 0 ) .
Under similar analysis, we conclude that as long as the minimum eigenvalue of ∇ 2 f ( x 0 ) satisfiesλ min ( ∇ 2 f ( x 0 ))
≤ − δ , ZO-NCF-Online-Weak will stop before T and find a negative curvaturedirection that aligns well with the eigenvector corresponding to the minimum eigenvalue of ∇ f 2 ( x 0 ) .
Then we have the following lemma:
Lemma 2 (ZO-NCF-Online-Weak) .
The output v of Algorithm 1 satisfies:
If λ min ( ∇ 2 f ( x 0 )) ≤ − δ ,then with probability at least 2 / 3 , v ̸ = ⊥ and v T ∇ 2 f ( x 0 ) v ≤ − 34 δ .
We summarize ZO-NCF-Online in Algorithm 2.
Specifically, ZO-NCF-Online repeatedly calls ZONCF-Online-Weak for Θ(log(1 /p ))
times to boost the confidence of solving the NCF problem from2 / 3 to 1 − p .
We have the following results:
Lemma 3.
In the same setting as in Algorithm 2, define z = 1 m (cid:80) mj
=1 v T ( H f ij ( x 0 )) v .
Then, if∥ v ∥ ≤ δ 16 dρ and m =
Θ( ℓ 2 δ 2 ) , with probability at least 1 − p , we have (cid:12)(cid:12)(cid:12)
[Equation]
(cid:12)(cid:12)(cid:12) ≤ δ 4 .
Theorem 1.
Let f ( x ) = 1 n (cid:80) ni
=
f i ( x ) where each f i is ℓ -smooth and ρ -Hessian Lipschitz.
Forevery point x 0 ∈ R d , every δ ∈ (0 , ℓ ] , the output of Algorithm 2 v satisfies that, with probability atleast 1 − p : If v = ⊥ , then ∇ 2 f ( x 0 ) ⪰ − δ I ; If v ̸ = ⊥ , then ∥ v ∥ = 1 and v T ∇ 2 f ( x 0 ) v ≤ − δ 2 .
Thetotal function query complexity is O (cid:16) d
[Equation]
[Table]
[Figure]
[Table]
(cid:17) .
3.2 Deterministic Setting
In this subsection, we focus on solving the NCF problem with zeroth-order methods under the offlinedeterministic setting and propose ZO-NCF-Deterministic .
We summarize ZO-NCF-Deterministic in
Algorithm 3.
Since we want to compute the eigenvector corresponding to the most negative eigenvalueof ∇ 2 f ( x 0 ) approximately, one can convert it into an approximated top eigenvector computationproblem of M := − 1 ℓ ∇ 2 f ( x 0 )
+ (1 − 3 δ 4 ℓ ) I .
This is because all eigenvalues of ∇ 2 f ( x 0 ) in [ − 3 δℓ , ℓ ]will be mapped to eigenvalues of M in [ − 1 , 1] , and all eigenvalues of ∇ 2 f ( x 0 ) smaller than − δ willbe mapped to eigenvalues of M greater than 1 + δ 4 ℓ .
Algorithm
Input: Function f ( · ) , point x 0 , negative curvature δ > 0 , confidence p ∈ (0 , 1] . 1: T ← C 21 log dp √ ℓ √ δ , σ ≜ ( d/p ) − 2 C 1 δT 4 ρ , r ≜ ( d/p ) C 1 σ 2: ξ ← σ ξ ′ ∥ ξ ′ ∥ , with ξ ′ ∼ N (0 , I ) 3: x 1 ← x 0 + ξ , y 0 ← 0 , y 1 ← ξ 4: for t = 1 , . . .
, T do 5: µ t = ∥
y t ∥ 6: y t +1 = 2 M ( y t ) − y t − 1 where M ( y ) =
( − 1 ℓ H f ( x 0 )
+ (1 − 3 δ 4 ℓ ))
y 7: x t +1 = x 0 + y t +1 − M ( y t ) 8: if ∥ x t +1 − x 0 ∥ ≥ r then return v = x
t +1 − x 0 ∥ x t +1 − x 0 ∥
Return v = ⊥ .
Similar to ZO-NCF-Online-Weak , ZO-NCF-Deterministic starts by adding a random perturbationξ to the detection point x 0 .
To find the negative curvature direction v of ∇ 2 f ( x 0 ) such thatv T ∇ 2 f ( x 0 ) v ≤ − δ 2 , the classical power method which updates through x T +
= x 0 + M T ξ[29] will take T ≥ ˜Ω( ℓδ ) number of iterations since eigenvalues of M greater than 1 + δ 4 ℓ grows in aspeed (1 + δ/ℓ ) T .
To reduce the iteration complexity T , we can replace the matrix polynomial M Twith the matrix Chebyshev polynomial T T ( M ) and virtually update x T +
= x 0 + T T ( M ) ξ .
Definition 3.
Chebyshev polynomial {T n ( x ) } n ≥ 0 of the first kind is
[Equation] then it satisfies T t ( x ) = [Equation]
cos
[Equation] n arccos
[Equation]
x
[Equation]
, x ∈ [Equation]
In the matrix case, we have the so-called matrix Chebyshev polynomial T t ( M ) [3], which satisfies:
T t +1 ( M ) ξ = 2 M T t ( M ) ξ − T t − 1 ( M ) ξ .
Thus, eigenvalues of M greater than 1 + δ 4 ℓ will grow to(cid:16)
+ δ/ 4 ℓ + (cid:112) ( δ/ 4 ℓ ) 2 + δ/ 2 ℓ (cid:17) T ≈ (cid:16) 1 + (cid:112) δ/ℓ (cid:17) T , so we only need to choose T ≥ (cid:112) ℓ/δ .
On the other hand, since we only have access to the zeroth-order information, we need to stablycompute the matrix Chebyshev polynomial.
In algorithm 3, we set µ t = ∥
y t ∥ and use M ( y t ) =( − 1 ℓ H f ( x 0 ) + (1 − 3 δ 4 ℓ ))
y t to approximate M y t with approximation error up to 2 ρ √ drtℓ ∥ y t ∥ .
Withproper choice of r , it allows us to use the inexact backward recurrence [3] to ensure a stablecomputation of matrix Chebyshev polynomial:
[Equation]
Then the output x T +1 = x 0 + y T
+
− M ( y T ) is close to x 0 + T T ( M ) ξ with a small approximationerror.
Finally, we have the following theorem:
Theorem 2.
Let f ( x ) = 1 n (cid:80) [-ni =
-]n i =
1 f i ( x ) where each f i is ℓ -smooth and ρ -Hessian Lipschitz.
Forevery point x 0 ∈ R d , every δ ∈ (0 , ℓ ] , the output of Algorithm 3 v satisfies that, with probability atleast 1 − p : If v = ⊥ , then ∇ 2 f ( x 0 ) ⪰ − δ
I ; If v ̸ = ⊥ , then ∥ v ∥ = 1 and v T ∇ 2 f ( x 0 ) v ≤ − δ 2 .
Thefunction query complexity is O
( d log dp √ ℓ √ δ ) .
Input: Function f , starting point x 0 , confidence p ∈ (
1) , ϵ > 0 and δ > 0 .
1: for t = 0 , . . .
, K − 1 do 2: uniformly randomly choose S ⊆
[ n ] 3:
Option I : ¯ x t +1 ← x t − η ˆ ∇ coord f S ( x t ) 4: Option II : ¯ x t +1 ← x t − η ˆ ∇ rand f S ( x t ) 5: if ∥∇ f ( x t ) ∥ ≥ ϵ 2 then 6: x t +1 = ¯ x t +1 7: else 8: v ← ZO-NCF-Online ( f, x t , δ, p 2 K ) 9: if v = ⊥ then return x t 10: else x t +1 = x t ± δρ v
Algorithm
Input: Function f , starting point x 0 , confidence p ∈ (0 , 1) , ϵ > 0 and δ > 0 .
1: for t = 0 , . . .
, K − 1 do 2:
Option I : ¯ x t +1 ← x t − η ˆ ∇ coord f ( x t ) 3:
Option II : ¯ x t +1 ← x t − η ˆ ∇ rand f ( x t ) 4: if ∥ ˆ ∇ coord f ( x t ) ∥ ≥ 3 ϵ 4 then 5: x t +1 =
¯ x t +1 6: else 7: v ←
ZO-NCF-Deterministic ( f, x t , δ, pK ) 8: if v = ⊥
then return x t 9: else x t +1 = x
t ± δρ v
Applications of Zeroth-Order Negative Curvature Finding
In this section, we focus on applying the zeroth-order negative curvature frameworks to the following
ZO algorithms: ZO-GD, ZO-SGD, ZO-SCSG, and ZO-SPIDER.
The following result shows that onecan verify if a point x is an ϵ -approximate FOSP using CoordGradEst.
Proposition 1.
In the online setting, using O (cid:16)(cid:16) 128 σ 2 ϵ+ 1 (cid:17) log 1 p (cid:17) CoordGradEst with smoothingparameter µ ≤ (cid:113) 3 ϵ 4 ρ √ d , we can verify with probability at least 1 − p , either ∥∇ f ( x ) ∥ ≥ ϵ/ 2 or∥∇ f ( x ) ∥ ≤ ϵ .
In the deterministic setting, using once computation of CoordGradEst with smoothingparameter µ ≤ (cid:113) 3 ϵ 2 ρ √ d , we can verify with probability 1 , either ∥∇ f ( x ) ∥ ≥ ϵ/ 2 or ∥∇ f ( x ) ∥ ≤ ϵ .
4.1 Applying Zeroth-Order Negative Curvature Finding to ZO-GD and ZO-SGD
We apply ZO-NCF-Online to ZO-SGD to turn it into a local minima finding algorithm, and propose
ZO-SGD-NCF in Algorithm 4.
At each iteration, ZO-SGD-NCF either estimates the gradient∇ f S ( x t ) = 1 | S | (cid:80) i
∈ S ∇ f i ( x t ) by CoordGradEst ( Option I ) or RandGradEst ( Option II ) withboth mini-batch size O ( σ 2 ϵ 2 ) .
Then, we use O (cid:16) σ 2 ϵ 2 log (cid:16) 2 Kp (cid:17)(cid:17) CoordGradEst to verify if x t is an ϵapproximate stationary point, if so, we call the ZO-NCF-Online subroutine.
If we find an approximatenegative curvature direction v around x t , then we update x t +1 by moving from x t in the direction vwith step-size δ/ρ .
We have the following theorem:
Theorem 3.
Under Assumption 1, set the parameters as follows, Option I: | S | = max { 32 σ 2 ϵ 2 , 1 } , K =
O ( ρ 2 ∆ f δ+ ℓ ∆ f ϵ 2 ) , η = 14 ℓ , µ 1 = (cid:113) 3 ϵ 4 ρ √ d , µ 2 = (cid:113) 3 ϵ 4 ρ √ d ; Option II: | S | = max { 32 σ 2 ϵ 2 , 1 } , K =
O ( ρ 2 ∆ f δ 3 + dℓ ∆ f ϵ 2 ) , η = 18 dℓ , µ 1 = min (cid:110)(cid:113)
[Equation]
4 ρd , ϵ 16 √ dℓ (cid:111) , µ 2 = (cid:113) 3 ϵ 4 ρ √ d , where µ 1 and µ 2 are onlyused in Line 4 and Line 5 of Algorithm 4, respectively.
With probability at least 1 − p , Algorithm 4outputs an ( ϵ, δ ) -approximate local minimum in function query complexity
Option I
[Equation]
[-+
-]
Remark 2.
Note that the dominant term of the function query complexity in Option I is ˜ O ( dϵ 2 ) , whilein Option II is ˜ O
( d 2 ϵ 2 ) .
This is because CoordGradEst has a lower approximation error and thus canreduce the iteration complexity by a fact of d .
Then the function query complexity of Option II isdominated by evaluating the magnitude of the gradient (Line 5 in Algorithm 4).
In the Deterministic setting, we apply ZO-NCF-Deterministic to ZO-GD to turn it into a local minimafinding algorithm and propose ZO-NCF-GD in Algorithm 5.
The update rule of ZO-NCF-GD issimilar to that in ZO-NCF-SGD, the only difference is that we don’t need to use mini-batch samplingof the stochastic gradient.
Similarly, we have the following theorem:
Theorem 4.
Under Assumption 1, set the parameters as follows, Option I: η = 14 ℓ , µ 1 = 250 (cid:113)
[Equation]
3 ϵ 4 ρd , ϵ 16 √ dℓ (cid:111) , µ 2 = (cid:113) 3 ϵ 2 ρ √ d , where µ 1 and µ 2 are only used in Line 3 and Line 4 of Algorithm 5, respectively.
With probability at least1 − p , Algorithm 5 outputs an ( ϵ, δ ) -approximate local minimum in function query complexity
Option I: ˜ O ( dℓ ∆ f ϵ+ d √ ℓ √ δ ρ 2 ∆ f δ 3 ); Option II: ˜ O ( d 2 ℓ ∆ f ϵ+ d √ ℓ √ δ ρ 2 ∆ f δ 3 ) .
4.2 Applying Zeroth-Order Negative Curvature Finding to ZO-SCSG
In this subsection, we first propose a zeroth-order variant of the SCSG [27] method in Algorithm 6.
At the beginning of the j -th epoch, we estimate the gradient ∇ f I j (˜ x j − 1 ) by CoordGradEst over abatch sampling set I j with size B .
In the inner loop iterations, the stochastic gradient estimator v jk − 1is either constructed by CoordGradEst or by RandGradEst over a mini-batch sampling set I jk − 1 withsize b .
After running ZO-SCSG for one epoch, we have the following lemma:
II: v jk − 1 = ˆ ∇ rand f I jk − 1 ( x jk − 1 ) − ˆ ∇ rand f I jk − 1 ( x j 0 ) + v j 10: x jk = x jk − 1 − ηv jk −11:
˜ x j = x jN j
Input: Function f , starting point x 0 , batch size B , mini-batch size b , K , ϵ > 0 and δ > 0 .
1: if b > B then return ZO-SGD( f, x 0 , 23 , ϵ, δ ) 2: for t = 0 , . . .
, K − 1 do 3: ¯ x t +1 ← apply ZO-SCSG on x t for one epoch with batch size B and mini-batch size b 4: if ∥∇ f ( x t ) ∥ ≥ ϵ 2 then 5: x t +1 = ¯ x t +1 6: else 7: v ← ZO-NCF-Online ( f, x t , δ, 1 20 K ) 8: if v = ⊥ then return x t 9: else x t +1 = x t ± δρ v
Lemma 4 (One epoch analysis) . Under Assumption 1, Option I: Let ηℓ = γ (cid:0) bB (cid:1) 23 . Suppose γ ≤ 14 , B ≥ 8 b and b ≥ 1 , then after running ZO-SCSG for one epoch, we have(cid:18)
B
[Equation]
(4) where c is a sufficiently large constant.
Option II: Let ηℓ = γ (cid:16) b/dB (cid:17) 23 .
Suppose γ ≤ 18 , B ≥ 8 b/dand b ≥ d , then after running ZO-SCSG for one epoch, we have(cid:18) B b/d (cid:19)
13 E ∥∇ f [Equation] where c is a sufficiently large constant. Remark 3.
The epoch size N j obeys the Geometric distribution, i.e. , N j ∼ Geom ( BB + b ) in Option Iand N j ∼ Geom ( B B + b/d ) in Option II . Since in expectation we have E ( N j ) N j ∼ Geom ( θ ) =
θ 1 − θ 27],then for both Option I and Option II , the function query complexity in each epoch is O ( d · B ) . Theorem 5. Under Assumption 1, if we set the parameters as follows, Option I: B =max
[Equation]
[Equation]
2 ϵ 2 ,
[Equation]
[Equation]
=
[Equation]
1 , Θ
[Equation]
; Option
II
[Equation] (cid:113) 3 ϵ 4 ρ √ d , where µ 1 and µ 2 are only use in Line 3 and Line 4 of Algorithm 7, respectively. Withprobability at least 23 , for both Option I and Option II , Algorithm 7 outputs an ( ϵ, δ ) -approximatelocal minimum in function query complexity [Equation]
{+Remark 4. The problem described in Remark 2 doesn’t exist in ZO-SCSG-NCF as we only evaluatethe magnitude of the gradient after each epoch ( i.e. , Line 4 in Algorithm 7), and the function querycomplexity is almost the same in the inner loop for both Option I and Option II .4.
 Applying Zeroth-Order Negative Curvature Finding to ZO-SPIDER In this subsection, we apply ZO-NCF-Online to ZO-SPIDER to turn it into a local minima findingalgorithm and propose+} ZO-SPIDER-NCF in Algorithm 8.
As a by-product, we also propose azeroth-order variant of the SPIDER method in Appendix G that can converge to an ϵ -approximate FOSP with high probability rather than expectation.
Using the same technique as in SPIDER-SFO +[12], that is, instead of moving in a large single step with size δ/ρ along the approximate negativecurvature direction as in ZO-SGD-NCF and ZO-SCSG-NCF, we can split it into δ/ ( ρη ) equal lengthmini-steps with size η .
As a result, we can maintain the SPIDER estimates and improve the so-callednon-improvable coupling term 1 δ 3 ϵ 2 by a fact of δ .
9: v k coord S 2 ( x − ˆ ∇ coord f S 2 ( x k − 1 ) + v k − then 11: x k +1 = x k −
12: else 13: 14: return 15: x k +1 =
x k − η ( v k / ∥ v k
Theorem 6.
{+Under Assumption 2, if we set ˜ ϵ = 10 ϵ log(
 ( K 0 +1)) , |S+} 1 | = 16 σ 2 ϵ 2 , |S 2 | = 16 σ ϵn 0 , η =ϵℓn 0 , q = σn 0 ϵ , µ = (cid:16) ϵ · ˜ ϵ 8 q 2 ρ 2 d (cid:17) 14 =
˜ O
( ϵ d 1 / 4 ) , K = δℓn 0 ρϵ , where n 0 ∈ [1 , 2 σϵ ] , then with probability atleast 3 4 , Algorithm 8 outputs x k with j ≤ J = 8 (cid:16)(cid:106) max (cid:16) 12 ρ 2 ∆ f δ, 4 ρ ∆ f δϵ (cid:17)(cid:107) + 1 (cid:17) , k ≤ K 0 =
J Ksatisfying ∥∇ f
( x k ) ∥ ≤ 3˜ ϵ, λ min ( ∇ 2 f ( x k )) ≥ − 2 δ with ˜ ϵ
= {+10 ϵ log(
+}
( K 0 + 1))
=
˜ O
( ϵ ) . Thetotal function query complexity is bounded by
[Equation]
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes]
(b) Did you describe the limitations of your work
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[No] [Table] (d) Have you read the ethics review guidelines and ensured that your paper conforms to [Table]
them?
[Yes]
[Table]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes](b) Did you include complete proofs of all theoretical results?
[Yes] Please refer to theAppendix for complete proofs of the theoretical results.3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes](b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] [Table]
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[No]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[No]4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes](b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
[Table] (d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
[Table]
(a) Did you include the full text of instructions given to participants and screenshots, if
[Table]
applicable?
[N/A]
[Table]
(b) Did you describe any potential participant risks, with links to Institutional Review
[Table]
Board ) approvals, if applicable
[N/A]
[Table]
you include the estimated hourly wage paid to participants and the total amountspent on participant compensation
[N/A] [Table]