On the Second-order Convergence Properties of Random Search Methods

1 Introduction . We consider solving the non-convex optimization problem minx∈Rd f ( x ) , where f ( · ) is differentiable but its derivatives are not directly accessible , or can only be approximated at a high computational cost . This setting recently gained attention in machine learning , in areas such as black-box adversarial attacks [ 11 ] , reinforcement learning [ 51 ] , meta-learning [ 7 ] , online learning [ 9 ] , and conditional optimization [ 57 ] . We focus our attention on a popular class of derivative-free methods known as random direct-search methods of directional type 1 . These methods optimize f ( · ) by evaluating the objective function over a number of ( fixed or randomized ) directions , to ensure descent using a sufficiently small stepsize . Direct-search algorithms date to the 1960 ’ s , including e.g . [ 44 , 46 ] . More recent variants include deterministic direct search [ 15 ] , random direct search ( e.g . [ 54 ] , or the Stochastic Three Points ( STP ) method [ 6 ] ) , which randomly sample a direction and accept a step in this direction if it decreases the function f ( · ) . As discussed in [ 40 ] , direct-search methods have remained popular over the years for a number of reasons , including their good performance and known global convergence guarantees [ 54 ] , as well as their straightforward implementation that makes them suitable for many problems . We refer the reader to [ 40 , 15 ] for a survey . In machine learning , objective functions of interest are often non-convex , which poses additional challenges due to the presence of saddle points and potentially suboptimal local minima [ 33 ] . Instead of aiming for a global minimizer , one often seeks a second-order stationary point ( SOSP ) : i.e . a solution where the gradient vanishes and the Hessian is positive definite . Indeed , as shown by [ 14 , 36 , 22 , 21 ] , many machine learning problems have no spurious local minimizers , yet have many saddle points which yield suboptimal solutions and are often hard to escape from [ 19 ] . While convergence to SOSPs and saddle escape times have been extensively studied in the context of ∗Alphabetical ordering , all authors contributed equally . 1In this manuscript , we will use the terms “ random direct-search ” and “ random search ” interchangeably . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . gradient-based methods [ 32 , 16 , 10 , 56 ] , prior analyses for ( random ) direct search have mainly focused on convergence to first-order stationary points [ 6 , 54 ] . One exception is the Approximate Hessian Direct Search ( AHDS ) method [ 26 ] , that explicitly computes the Hessian of the objective to obtain second-order worst-case guarantees . However , computing or storing a full Hessian is prohibitively expensive in high dimensions . Towards a better understanding of the complexity of finding second-order stationary points with random search methods , we make the following contributions : • We study the complexity of a simple random search similar to STP ( Algorithm 1 ) to reach SOSPs . We find that the ( worst-case ) complexity requires a number of function evaluations that scales exponentially in terms of the problem dimension d. As we will see , the exponential scaling is not an artefact of the analysis . This is intuitive , indeed , if we are at a saddle point where we just have one direction of negative curvature , finding good alignment with a random direction becomes exponentially difficult as the dimension of the space increases . • To solve this issue , we design a variant of random search ( RSPI , Algorithm 2 ) that , instead of randomly sampling directions from the sphere , relies on an approximate derivative-free power iteration routine to extract negative curvature direction candidates ( unlike [ 26 ] that requires an approximation to the full Hessian ) . This approach is inspired from recent work on gradient-based methods for finding SOSPs [ 10 , 56 ] and effectively decouples negative curvature estimation from progress in the large gradient setting . As a result , RSPI does not suffer from the exponential scaling of the vanilla random search approach : we show that the overall complexity of finding a SOSP in terms of function evaluations becomes linear in the problem dimension . • Finally , we verify our results empirically and compare our novel algorithm to standard randomsearch methods . Our results show improvements of RSPI both in terms of algorithm iterations and ( crucially ) wall-clock time . 2 Related work . Direct-search vs methods that approximate gradients A wide variety of algorithms enter the class of DFO methods . One common distinction is made according to whether or not the algorithm explicitly computes a gradient approximation . Direct-search ( DS ) and pattern-search ( PS ) methods only rely on function evaluations to validate a step along a direction , sampled according to some scheme . In contrast , a second group of methods explicitly compute a gradient approximation [ 48 , 23 , 12 ] . Most methods in the latter group rely only on first-order information , except [ 58 ] that incorporates some second-order information to compute an estimate of the gradient . However , their approach focuses on convex optimization and therefore does not discuss second-order stationarity . Instead , [ 55 ] showed thatO ( −2 ) approximate gradient computations are enough to reach an −SOSP . Unlike DS , the gradient-free method analyzed in [ 55 ] computes an approximation of the gradient and , as a result , it matches the convergence rate guarantees of their exact gradient-based counterparts ( up to constants ) . However , existing lower bounds clearly show that DS methods have a worst rate of convergence . Of special interest to us is the complexity w.r.t . the dimension of the problem , which is exponential , see paragraph “ Lower bounds ” below . First-order guarantees of DS . A recently proposed variant of DS is the stochastic-threepoints ( STP ) approach proposed in [ 6 ] that simply samples a direction at random on the sphere and accepts the step if it decreases the function . This is , in some sense , a simpler variant of direct search compared to adaptive methods such as [ 54 ] that also include an adaptive mechanism of the step-size selection to ensure convergence without having to rely on knowing certain quantities such as the smoothness constant of the objective . As shown in [ 6 ] , finding a point x such that ‖∇f ( x ) ‖ ≤ with STP requires O ( d −2 ) function evaluations . If instead the perturbation directions are sampled from a fixed basis and not from the unit sphere , the best known complexity increases to O ( d2 −2 ) [ 54 , 6 ] . Ignoring the dependency on the dimension , these results match the iteration complexity of steepest descent [ 47 ] . Second-order guarantees for PS and DS . Second-order convergence guarantees have been developed in the context of ( generalized ) pattern search ( GPS ) methods , which share similarities with direct search methods . These methods sample update directions from a positive spanning set D. The work of [ 1 ] proved a type of “ pseudo-second-order ” stationarity condition for GPS , where the Hessian is positive semidefinite in the directions of the basis D ( but not with respect to all vectors in the space ) . A variant of GPS that explicitly constructs an approximate Hessian was shown by [ 2 ] to converge to second-order stationary points under some assumptions on the quality of the approximation . The results discussed so far typically consider the properties of the limit points of the sequence of iterates and do not provide worst-case complexity rates of convergence . In contrast , [ 26 ] proved convergence to a second-order stationary point as well as derived worst-case guarantees ( upper bounds only ) for a variant of a direct search approach that constructs an approximate Hessian matrix using a finite difference approach . Model-based approaches . Model-based methods construct a function approximation which is used to compute the next step and updated at every iteration . The literature on such methods is broad and we refer the reader to [ 38 ] for a survey of the relevant work . To the best of our knowledge , these methods also require an approximation of the Hessian to obtain second-order guarantees . Lower bounds . [ 59 , 24 ] show that random search suffers from an exponential dependency to the dimension of the problem . Similar results are discussed in [ 20 ] for derivative-free methods that use two function evaluations to approximate derivatives . These results serve as a motivation for this work to improve the complexity of random search methods in terms of the input dimension . We mostly focus on designing a new type of random search that achieves better worst-case guarantees than existing lower bounds for vanilla random search . Inexact Power Iteration . Computing the largest eigenvector of a matrix has many applications in statistics and data analysis . Iterative methods such as the power iteration and the Lanczos algorithm are commonly used to solve this problem [ 25 ] . Perhaps the most complete and up-to-date analysis of convergence of inexact power methods for eigenspace estimation can be found in [ 5 , 29 ] . Crucially , these convergence rates depend on the eigenvalue distribution . If one instead only seeks a direction aligned with a suboptimal but large eigenvalue , the rate becomes independent of the eigenvalue distribution [ 37 ] . Alternatively to the power method , stochastic methods such as Oja ’ s algorithm [ 49 ] benefit from a cheaper iteration cost . Improvements have been proposed in [ 52 ] that analyzes a variance reduced method that achieves a linear rate of convergence . An accelerated variant of stochastic PCA is also proposed in [ 18 ] . 3 Analysis . We work in Rd with the standard Euclidean norm ‖ · ‖ . Our goal is to optimize a twice continuously differentiable non-convex function f ( · ) : Rd → R without having access to gradients . We need the following assumption , standard in the literature on SOSPs [ 33 ] , also in the DFO setting [ 55 ] . Assumption 1 . The function f ( · ) is lower bounded , L1-smooth and L2-Hessian Lipschitz . In line with many recent works on non-convex optimization [ 32 , 35 , 34 , 16 , 10 , 55 , 4 ] , our goal is to find an ( , γ ) -second-order stationary point ( SOSP ) , i.e . a point x such that : ‖∇f ( x ) ‖ ≤ and λmin ( ∇2f ( x ) ) ≥ −γ , ( 1 ) with , γ > 0 . We analyze two different DFO algorithms : ( 1 ) a two-step random search method ( RS , Algorithm 1 ) – similar to STP [ 6 ] and ( 2 ) a novel random search method ( RSPI , Algorithm 2 ) that extracts negative curvature via Algorithm 3 . We show that , while the approximation of negative eigendirections requires extra computation , the overall complexity of RSPI to find a SOSP is much lower than RS in terms of number of function evaluations . Theorem 1 ( Main result , informal ) . The complexity — in terms of number of function evaluations — of vanilla Random Search ( similar to STP , Algorithm 1 ) for finding second-order stationary points depends exponentially on the problem dimension ( see Lemma 3 and Theorem 5 ) . This dependency can be reduced to linear by computing an approximate negative curvature direction at each step ( i.e . Algorithm 2 , see Theorem 10 ) . We will use O ( · ) , Θ ( · ) , Ω ( · ) to hide constants which do not depend on any problem parameter .

Summary: This paper is related to random-search methods when optimizing non-convex objective functions without having access to derivatives. The authors provided convergence guarantees to a second-order stationary with only access to the function evaluations. Further, they showed some numerical results for highlighting their theoretical results.