High Probability Complexity Bounds for Line Search Based on Stochastic Oracles

1 Introduction . In this paper , we analyze a line-search method when applied to the problem of minimizing an unconstrained , differentiable , possibly non-convex function : Rn ! R. The goal is to find a `` -stationary point for ; that is , a point x with kr ( x ) k `` . We make the standard assumption that r is LLipschitz , but the knowledge of L is not assumed by the algorithm . We consider a setting where neither the function value ( x ) nor the gradient r ( x ) are directly computable . Instead , the algorithm is given black-box access to the following probabilistic oracles : • Probabilistic zeroth order oracle . Given a point x , the oracle computes f ( x , ⇠ ) , a ( random ) estimate of the function value ( x ) . ⇠ is a random variable ( whose distribution may depend on x ) , with probability space ( ⌦ , F⌦ , P ) . We assume the absolute value of the estimation error e ( x ) = |f ( x , ⇠ ( x ) ) ( x ) | ( we omit the dependence on ⇠ for brevity ) to be a “ one-sided ” sub-exponential-like random variable1 with parameters ( ⌫ , b ) , whose mean is bounded by some constant ✏f > 0 . Specifically , E⇠ [ e ( x ) ] ✏f and E⇠ [ exp { ( e ( x ) E [ e ( x ) ] ) } ] exp ✓ 2⌫2 2 ◆ , 8 2 0 , 1 b . ( 1 ) • Probabilistic first order oracle . Given a point x and a constant ↵ > 0 , the oracle computes g ( x , ⇠0 ) , a ( random ) estimate of the gradient r ( x ) , such that P⇠0 ( kg ( x , ⇠0 ) r ( x ) k max { ✏g , ↵kg ( x , ⇠0 ) k } ) 1 . ( 2 ) Here , ⇠0 is a random variable ( whose distribution may depend on x ) , with associated probability space ( ⌦0 , F⌦0 , P 0 ) . ( 1 ) 2 ( 0 , 1 ) is the probability , intrinsic to the oracle , that the 1This is a weaker requirement than assuming e ( x ) to be sub-exponential , as one only needs to guard against the possibility of the error e ( x ) being too large . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . gradient estimate is “ sufficiently accurate ” with respect to ✏g , , and ↵ . Lastly , , ✏g 0 are constants , intrinsic to the oracle , which represent the precision the oracle can achieve . Note that ✏g allows the gradient estimate to be bounded away from the true gradient by a constant distance . Remark We will analyze a line search algorithm that relies on these two oracles . In the zeroth order oracle , the constants ✏f and ( ⌫ , b ) are intrinsic . In the first order oracle , , ✏g , and are intrinsic . These values can not be controlled . On the other hand , ↵ is an input to the first order oracle that can be chosen by the algorithm . In fact , as we shall see in Section 3 , ↵ will be the step size of the line search method . These two oracles cover several settings , including • Standard supervised learning , where gradients and values of the loss function are computed based on a mini-batch . Here , the random variables ⇠ and ⇠0 in the zeroth and first order oracles represent the random set of samples in the mini-batch . • Zeroth order optimization , where gradients are estimated via randomized finite differences using ( possibly noisy ) function values . This arises in policy gradients in reinforcement learning , as is used in [ SHC+16 ] and analyzed in [ BCCS21 ] . • A variety of other settings , where the gradients and function estimates may be biased stochastic estimates of the true gradients and function values . The constants in the oracles determine the precision of the function and gradient estimates . These constants will also dictate the accuracy achievable by the line search method we analyze . Specifically , if ✏f = 0 and ✏g = 0 , then the algorithm converges to a stationary point . Otherwise , a precise lower bound is derived for the smallest kr ( x ) k the algorithm can achieve , in terms of the constants in the oracles . It is worth noting that the oracles can be biased . Indeed , the zeroth order oracle can incur arbitrarily large error , as long as it satisfies 1 . Moreover , the first order oracle only requires g ( x , ⇠0 ( x ) ) to be a “ sufficiently accurate ” estimate ofr ( x ) with probability 1 . Thus g ( x , ⇠0 ( x ) ) can be an arbitrary vector with probability , so it in principle can have an arbitrarily large bias . The line-search algorithm is given in Section 3 . It is a modification of the standard Armijo-based line search algorithm [ NW06 ] , with access to the zeroth and first order oracles . The two small modifications are : 1 ) The Armijo condition is relaxed by an additive constant 2✏f , to account for the inexact function evaluations , and 2 ) The first order oracle is called in each iteration , and a new search direction is generated whenever the step size changes . This allows the method to progress to near-stationary points without assuming the gradient estimates ( e.g . the mini-batch gradients in supervised learning ) to be Lipschitz continuous . Our framework and analysis are based on results in [ CS17 ] , [ GRVZ18 ] and [ BCS19 ] . However , there are several key differences . In [ CS17 ] and [ BCS19 ] the line search has access to stronger oracles , with ✏g = 0 and |f ( x , ⇠ ) ( x ) | ✏f deterministically . Under these assumptions , [ CS17 ] and [ BCS19 ] derive an expected iteration complexity bound . In this paper , we provide a high probability tail bound on the iteration complexity , showing that the algorithm is very likely to succeed in a number of iterations on the order of its expected iteration complexity . Moreover , we consider more general oracles , with arbitrary ✏g and possibly unbounded |f ( x , ⇠ ) ( x ) | . Thus , we significantly strengthen the results in [ CS17 ] and [ BCS19 ] . To the best of our knowledge , the only other high probability complexity bound of this kind is derived in [ GRVZ18 ] for a trust region algorithm under the assumption ✏g = 0 and |f ( x , ⇠ ) ( x ) |= 0 deterministically , which are much stronger oracles . Stochastic line search has also been analyzed in [ PS20 ] and [ VML+19 ] . In [ PS20 ] the assumptions on |f ( x , ⇠ ) ( x ) | are different . On the one hand , they allow for more general distributions than sub-exponential . On the other hand , it is assumed that |f ( x , ⇠ ) ( x ) | can be made arbitrarily small with some fixed probability . An expected iteration complexity bound is then derived for arbitrarily small `` . In contrast , we do not assume this , and analyze the iteration complexity of reaching an `` -stationary point , with `` lower-bounded by a function of the constants in the oracles . Moreover , our analysis and results are much simpler than those in [ PS20 ] and we derive an iteration complexity bound in high probability , not just in expectation . In [ VML+19 ] , the traditional line search is analyzed for empirical loss minimization , where the function oracles are implemented using a random mini-batch of a fixed size . The mini-batch remains fixed during backtracking until a standard Armijo condition is satisfied . Thus the search direction remains the same until a step is taken . While good computational performance has been reported in [ VML+19 ] , its theoretical analysis requires several very restrictive assumptions , especially for nonconvex functions . Also , they bound the expected sum of squared gradient norms , while we bound the iteration complexity with high probability . We note that using similar techniques as in [ BCS19 ] , our analysis can be extended to the convex and strongly convex cases . In summary , we present an analysis of an adaptive line search algorithm under very general conditions on the gradient and function estimates . The results not only subsume most results in the prior literature , but also substantially extend the framework . Moreover , high probability tail bounds on iteration complexity are derived , instead of only expected iteration complexity . 2 Oracles . In this section , we discuss a couple of settings , and show how they are captured by our framework . All norms used are 2-norm . 2.1 Expected loss minimization . Let us first discuss how the oracle definitions apply to expected loss minimization . In this setting , ( x ) = Ed⇠D [ ` ( x , d ) ] . Here , x is the model parameters , d is a data sample following distribution D , and ` ( x , d ) is the loss when the model parameterized by x is evaluated on data point d. In this case , the zeroth and first order oracles can be as follows , where S is a mini-batch sampled from D : f ( x , S ) = 1 |S| X d2S ` ( x , d ) , g ( x , S ) = 1 |S| X d2S rx ` ( x , d ) . ( 3 ) In general , S can be chosen to depend on x . We now show how our zeroth and first order oracle conditions are satisfied by selecting an appropriate sample size |S| . Proposition 1 . Let ê ( x , d ) : = ` ( x , d ) ( x ) be a ( ⌫̂ ( x ) , b̂ ( x ) ) -subexponential random variable and Vard⇠D [ ` ( x , d ) ] ✏̂ ( x ) 2 , for some ⌫̂ ( x ) , b̂ ( x ) , ✏̂ ( x ) . Let e ( x , S ) = |f ( x , S ) ( x ) | and N = |S| , then ES [ e ( x , S ) ] 1 p N ✏̂ ( x ) and e ( x , S ) is ( ⌫ ( x ) , b ( x ) ) -subexponential , with ⌫ ( x ) = b ( x ) = 8e2 max n ⌫̂ ( x ) p N , b̂ ( x ) o . In the case when the support of D is bounded , ` is Lipschitz , and the set of x we consider is bounded , the assumption of Proposition 1 is satisfied . Thus , f ( x , S ) is a zeroth order oracle with ✏f = supx 1p N ✏̂ ( x ) , ⌫ = supx ⌫ ( x ) , and b = supx b ( x ) , and ✏f can be made arbitrarily small by taking a large enough sample . Under standard assumptions on r ` ( x , d ) , for instance , suppose Assumption 4.3 in [ BCN18 ] holds : for some Mc , Mv 0 and for all x , Ed⇠D kr ` ( x , d ) r ( x ) k2 Mc +Mv kr ( x ) k2 , ( 4 ) one can show g ( x , S ) is a first order oracle with a large enough sample size . Proposition 2 . Let g = g ( x , S ) . Assuming Ed⇠Dr ` ( x , d ) = r ( x ) , then |S| Mc +Mv kr ( x ) k 2 min ( 1 ✏2g , ( 1 + ↵ ) 2 2↵2 kr ( x ) k2 ) implies P ( kg r ( x ) k max { ✏g , ↵kgk } ) 1 . This bound implies a looser bound of : |S| max ( 2Mc ✏2g , 2Mv ( 1 + ↵ ) 2 2↵2 ) . Remark Let us discuss what is required from the minibatch size in this setting . Unlike standard SGD , the minibatch size is chosen dynamically . When convergence to a stationary point is desired , ✏g has to be zero , and gradient estimate gk tends to zero . Thus , kg r ( x ) k also has to tend to zero . If Mc = 0 , then fixing the mini-batch size to be at least Mv ( 1+↵ ) 2 2↵2 provides a valid first order oracle . Thus , unless ↵ tends to zero , the minibatch size remains bounded from below . This is similar to the interpolation condition used in e.g . [ BM11 , VML+19 ] . On the other hand , when Mc > 0 , the minibatch size has to grow in order to approach a stationary point . This is similar to dynamic minibatch size selection , discussed , e.g . in [ BCNW12 ] . The difference between our results and those in [ BCNW12 ] is that our batch bound is implementable and guarantees convergence , while the one in [ BCNW12 ] is implementable only as a heuristic . As our computational results show , however , a fixed and small mini-batch size appears to work very well , perhaps because Mc is small .

Summary: Consider unconstrained smooth optimization with probabilistic zeroth- and first-order oracles. This paper proposes a new gradient descent with line search method and provides a high-probability upper bound of the iteration complexity. The authors highlight that their assumptions on the noisy zeroth- and first-order oracles are weaker than those in literature.