On the Second-order Convergence Properties of Random Search Methods

1 Introduction . We consider solving the non-convex optimization problem minx∈Rd f ( x ) , where f ( · ) is differentiable but its derivatives are not directly accessible , or can only be approximated at a high computational cost . This setting recently gained attention in machine learning , in areas such as black-box adversarial attacks [ 11 ] , reinforcement learning [ 51 ] , meta-learning [ 7 ] , online learning [ 9 ] , and conditional optimization [ 57 ] . We focus our attention on a popular class of derivative-free methods known as random direct-search methods of directional type 1 . These methods optimize f ( · ) by evaluating the objective function over a number of ( fixed or randomized ) directions , to ensure descent using a sufficiently small stepsize . Direct-search algorithms date to the 1960 ’ s , including e.g . [ 44 , 46 ] . More recent variants include deterministic direct search [ 15 ] , random direct search ( e.g . [ 54 ] , or the Stochastic Three Points ( STP ) method [ 6 ] ) , which randomly sample a direction and accept a step in this direction if it decreases the function f ( · ) . As discussed in [ 40 ] , direct-search methods have remained popular over the years for a number of reasons , including their good performance and known global convergence guarantees [ 54 ] , as well as their straightforward implementation that makes them suitable for many problems . We refer the reader to [ 40 , 15 ] for a survey . In machine learning , objective functions of interest are often non-convex , which poses additional challenges due to the presence of saddle points and potentially suboptimal local minima [ 33 ] . Instead of aiming for a global minimizer , one often seeks a second-order stationary point ( SOSP ) : i.e . a solution where the gradient vanishes and the Hessian is positive definite . Indeed , as shown by [ 14 , 36 , 22 , 21 ] , many machine learning problems have no spurious local minimizers , yet have many saddle points which yield suboptimal solutions and are often hard to escape from [ 19 ] . While convergence to SOSPs and saddle escape times have been extensively studied in the context of ∗Alphabetical ordering , all authors contributed equally . 1In this manuscript , we will use the terms “ random direct-search ” and “ random search ” interchangeably . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . gradient-based methods [ 32 , 16 , 10 , 56 ] , prior analyses for ( random ) direct search have mainly focused on convergence to first-order stationary points [ 6 , 54 ] . One exception is the Approximate Hessian Direct Search ( AHDS ) method [ 26 ] , that explicitly computes the Hessian of the objective to obtain second-order worst-case guarantees . However , computing or storing a full Hessian is prohibitively expensive in high dimensions . Towards a better understanding of the complexity of finding second-order stationary points with random search methods , we make the following contributions : • We study the complexity of a simple random search similar to STP ( Algorithm 1 ) to reach SOSPs . We find that the ( worst-case ) complexity requires a number of function evaluations that scales exponentially in terms of the problem dimension d. As we will see , the exponential scaling is not an artefact of the analysis . This is intuitive , indeed , if we are at a saddle point where we just have one direction of negative curvature , finding good alignment with a random direction becomes exponentially difficult as the dimension of the space increases . • To solve this issue , we design a variant of random search ( RSPI , Algorithm 2 ) that , instead of randomly sampling directions from the sphere , relies on an approximate derivative-free power iteration routine to extract negative curvature direction candidates ( unlike [ 26 ] that requires an approximation to the full Hessian ) . This approach is inspired from recent work on gradient-based methods for finding SOSPs [ 10 , 56 ] and effectively decouples negative curvature estimation from progress in the large gradient setting . As a result , RSPI does not suffer from the exponential scaling of the vanilla random search approach : we show that the overall complexity of finding a SOSP in terms of function evaluations becomes linear in the problem dimension . • Finally , we verify our results empirically and compare our novel algorithm to standard randomsearch methods . Our results show improvements of RSPI both in terms of algorithm iterations and ( crucially ) wall-clock time . 2 Related work . Direct-search vs methods that approximate gradients A wide variety of algorithms enter the class of DFO methods . One common distinction is made according to whether or not the algorithm explicitly computes a gradient approximation . Direct-search ( DS ) and pattern-search ( PS ) methods only rely on function evaluations to validate a step along a direction , sampled according to some scheme . In contrast , a second group of methods explicitly compute a gradient approximation [ 48 , 23 , 12 ] . Most methods in the latter group rely only on first-order information , except [ 58 ] that incorporates some second-order information to compute an estimate of the gradient . However , their approach focuses on convex optimization and therefore does not discuss second-order stationarity . Instead , [ 55 ] showed thatO ( −2 ) approximate gradient computations are enough to reach an −SOSP . Unlike DS , the gradient-free method analyzed in [ 55 ] computes an approximation of the gradient and , as a result , it matches the convergence rate guarantees of their exact gradient-based counterparts ( up to constants ) . However , existing lower bounds clearly show that DS methods have a worst rate of convergence . Of special interest to us is the complexity w.r.t . the dimension of the problem , which is exponential , see paragraph “ Lower bounds ” below . First-order guarantees of DS . A recently proposed variant of DS is the stochastic-threepoints ( STP ) approach proposed in [ 6 ] that simply samples a direction at random on the sphere and accepts the step if it decreases the function . This is , in some sense , a simpler variant of direct search compared to adaptive methods such as [ 54 ] that also include an adaptive mechanism of the step-size selection to ensure convergence without having to rely on knowing certain quantities such as the smoothness constant of the objective . As shown in [ 6 ] , finding a point x such that ‖∇f ( x ) ‖ ≤ with STP requires O ( d −2 ) function evaluations . If instead the perturbation directions are sampled from a fixed basis and not from the unit sphere , the best known complexity increases to O ( d2 −2 ) [ 54 , 6 ] . Ignoring the dependency on the dimension , these results match the iteration complexity of steepest descent [ 47 ] . Second-order guarantees for PS and DS . Second-order convergence guarantees have been developed in the context of ( generalized ) pattern search ( GPS ) methods , which share similarities with direct search methods . These methods sample update directions from a positive spanning set D. The work of [ 1 ] proved a type of “ pseudo-second-order ” stationarity condition for GPS , where the Hessian is positive semidefinite in the directions of the basis D ( but not with respect to all vectors in the space ) . A variant of GPS that explicitly constructs an approximate Hessian was shown by [ 2 ] to converge to second-order stationary points under some assumptions on the quality of the approximation . The results discussed so far typically consider the properties of the limit points of the sequence of iterates and do not provide worst-case complexity rates of convergence . In contrast , [ 26 ] proved convergence to a second-order stationary point as well as derived worst-case guarantees ( upper bounds only ) for a variant of a direct search approach that constructs an approximate Hessian matrix using a finite difference approach . Model-based approaches . Model-based methods construct a function approximation which is used to compute the next step and updated at every iteration . The literature on such methods is broad and we refer the reader to [ 38 ] for a survey of the relevant work . To the best of our knowledge , these methods also require an approximation of the Hessian to obtain second-order guarantees . Lower bounds . [ 59 , 24 ] show that random search suffers from an exponential dependency to the dimension of the problem . Similar results are discussed in [ 20 ] for derivative-free methods that use two function evaluations to approximate derivatives . These results serve as a motivation for this work to improve the complexity of random search methods in terms of the input dimension . We mostly focus on designing a new type of random search that achieves better worst-case guarantees than existing lower bounds for vanilla random search . Inexact Power Iteration . Computing the largest eigenvector of a matrix has many applications in statistics and data analysis . Iterative methods such as the power iteration and the Lanczos algorithm are commonly used to solve this problem [ 25 ] . Perhaps the most complete and up-to-date analysis of convergence of inexact power methods for eigenspace estimation can be found in [ 5 , 29 ] . Crucially , these convergence rates depend on the eigenvalue distribution . If one instead only seeks a direction aligned with a suboptimal but large eigenvalue , the rate becomes independent of the eigenvalue distribution [ 37 ] . Alternatively to the power method , stochastic methods such as Oja ’ s algorithm [ 49 ] benefit from a cheaper iteration cost . Improvements have been proposed in [ 52 ] that analyzes a variance reduced method that achieves a linear rate of convergence . An accelerated variant of stochastic PCA is also proposed in [ 18 ] . 3 Analysis . We work in Rd with the standard Euclidean norm ‖ · ‖ . Our goal is to optimize a twice continuously differentiable non-convex function f ( · ) : Rd → R without having access to gradients . We need the following assumption , standard in the literature on SOSPs [ 33 ] , also in the DFO setting [ 55 ] . Assumption 1 . The function f ( · ) is lower bounded , L1-smooth and L2-Hessian Lipschitz . In line with many recent works on non-convex optimization [ 32 , 35 , 34 , 16 , 10 , 55 , 4 ] , our goal is to find an ( , γ ) -second-order stationary point ( SOSP ) , i.e . a point x such that : ‖∇f ( x ) ‖ ≤ and λmin ( ∇2f ( x ) ) ≥ −γ , ( 1 ) with , γ > 0 . We analyze two different DFO algorithms : ( 1 ) a two-step random search method ( RS , Algorithm 1 ) – similar to STP [ 6 ] and ( 2 ) a novel random search method ( RSPI , Algorithm 2 ) that extracts negative curvature via Algorithm 3 . We show that , while the approximation of negative eigendirections requires extra computation , the overall complexity of RSPI to find a SOSP is much lower than RS in terms of number of function evaluations . Theorem 1 ( Main result , informal ) . The complexity — in terms of number of function evaluations — of vanilla Random Search ( similar to STP , Algorithm 1 ) for finding second-order stationary points depends exponentially on the problem dimension ( see Lemma 3 and Theorem 5 ) . This dependency can be reduced to linear by computing an approximate negative curvature direction at each step ( i.e . Algorithm 2 , see Theorem 10 ) . We will use O ( · ) , Θ ( · ) , Ω ( · ) to hide constants which do not depend on any problem parameter .

Summary: This work studies the problem of optimizing a non-convex function in a deterministic setting, where the gradient of the function is not accessible. The authors provide analysis of convergence to a second order stationary point, where the gradient of the function is zero and the Hessian matrix is positive definite.     Assumptions: Let $f$ to be the objective function:  1. $f$ is lower bounded. 2. The gradient and Hessian of $f$ are Lipschitz 3. The smallest eigenvalue of the Hessian matrix has a uniform lower bound.  First the authors proposed a decent algorithm (Algorithm 1, page 4), which is a variant of vanilla random search method. Their analysis for this algorithm is divided into two parts: The case when the gradient is large (Lemma 2, page 4), and the case when the gradient is small but the smallest eigenvalue of Hessian is negative (Lemma 4, page 5). The latter is the case where they are close to a strict saddle point. Aggregating Lemmas 4 and 5 yields one of their main results, Theorem 5 (page 5). This theorem outlines the fact that convergence to a second order stationary point grows exponentially as a function of $d$ (dimension). To overcome this issue, they proposed Algorithms 2 and 3. The main idea of these algorithms is that instead of a random direction search (which causes an exponential dependency on $d$), they estimate negative curvature directions for escaping a saddle point. This new approach requires $4d$ additional function evaluations at each iteration. As they initially claimed, they could overcome the curse of dimensionality and obtain a linear dependency with respect to $d$. This result is marked out in Theorem 10 (page 7).