High Probability Complexity Bounds for Line Search Based on Stochastic Oracles

1 Introduction . In this paper , we analyze a line-search method when applied to the problem of minimizing an unconstrained , differentiable , possibly non-convex function : Rn ! R. The goal is to find a `` -stationary point for ; that is , a point x with kr ( x ) k `` . We make the standard assumption that r is LLipschitz , but the knowledge of L is not assumed by the algorithm . We consider a setting where neither the function value ( x ) nor the gradient r ( x ) are directly computable . Instead , the algorithm is given black-box access to the following probabilistic oracles : • Probabilistic zeroth order oracle . Given a point x , the oracle computes f ( x , ⇠ ) , a ( random ) estimate of the function value ( x ) . ⇠ is a random variable ( whose distribution may depend on x ) , with probability space ( ⌦ , F⌦ , P ) . We assume the absolute value of the estimation error e ( x ) = |f ( x , ⇠ ( x ) ) ( x ) | ( we omit the dependence on ⇠ for brevity ) to be a “ one-sided ” sub-exponential-like random variable1 with parameters ( ⌫ , b ) , whose mean is bounded by some constant ✏f > 0 . Specifically , E⇠ [ e ( x ) ] ✏f and E⇠ [ exp { ( e ( x ) E [ e ( x ) ] ) } ] exp ✓ 2⌫2 2 ◆ , 8 2 0 , 1 b . ( 1 ) • Probabilistic first order oracle . Given a point x and a constant ↵ > 0 , the oracle computes g ( x , ⇠0 ) , a ( random ) estimate of the gradient r ( x ) , such that P⇠0 ( kg ( x , ⇠0 ) r ( x ) k max { ✏g , ↵kg ( x , ⇠0 ) k } ) 1 . ( 2 ) Here , ⇠0 is a random variable ( whose distribution may depend on x ) , with associated probability space ( ⌦0 , F⌦0 , P 0 ) . ( 1 ) 2 ( 0 , 1 ) is the probability , intrinsic to the oracle , that the 1This is a weaker requirement than assuming e ( x ) to be sub-exponential , as one only needs to guard against the possibility of the error e ( x ) being too large . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . gradient estimate is “ sufficiently accurate ” with respect to ✏g , , and ↵ . Lastly , , ✏g 0 are constants , intrinsic to the oracle , which represent the precision the oracle can achieve . Note that ✏g allows the gradient estimate to be bounded away from the true gradient by a constant distance . Remark We will analyze a line search algorithm that relies on these two oracles . In the zeroth order oracle , the constants ✏f and ( ⌫ , b ) are intrinsic . In the first order oracle , , ✏g , and are intrinsic . These values can not be controlled . On the other hand , ↵ is an input to the first order oracle that can be chosen by the algorithm . In fact , as we shall see in Section 3 , ↵ will be the step size of the line search method . These two oracles cover several settings , including • Standard supervised learning , where gradients and values of the loss function are computed based on a mini-batch . Here , the random variables ⇠ and ⇠0 in the zeroth and first order oracles represent the random set of samples in the mini-batch . • Zeroth order optimization , where gradients are estimated via randomized finite differences using ( possibly noisy ) function values . This arises in policy gradients in reinforcement learning , as is used in [ SHC+16 ] and analyzed in [ BCCS21 ] . • A variety of other settings , where the gradients and function estimates may be biased stochastic estimates of the true gradients and function values . The constants in the oracles determine the precision of the function and gradient estimates . These constants will also dictate the accuracy achievable by the line search method we analyze . Specifically , if ✏f = 0 and ✏g = 0 , then the algorithm converges to a stationary point . Otherwise , a precise lower bound is derived for the smallest kr ( x ) k the algorithm can achieve , in terms of the constants in the oracles . It is worth noting that the oracles can be biased . Indeed , the zeroth order oracle can incur arbitrarily large error , as long as it satisfies 1 . Moreover , the first order oracle only requires g ( x , ⇠0 ( x ) ) to be a “ sufficiently accurate ” estimate ofr ( x ) with probability 1 . Thus g ( x , ⇠0 ( x ) ) can be an arbitrary vector with probability , so it in principle can have an arbitrarily large bias . The line-search algorithm is given in Section 3 . It is a modification of the standard Armijo-based line search algorithm [ NW06 ] , with access to the zeroth and first order oracles . The two small modifications are : 1 ) The Armijo condition is relaxed by an additive constant 2✏f , to account for the inexact function evaluations , and 2 ) The first order oracle is called in each iteration , and a new search direction is generated whenever the step size changes . This allows the method to progress to near-stationary points without assuming the gradient estimates ( e.g . the mini-batch gradients in supervised learning ) to be Lipschitz continuous . Our framework and analysis are based on results in [ CS17 ] , [ GRVZ18 ] and [ BCS19 ] . However , there are several key differences . In [ CS17 ] and [ BCS19 ] the line search has access to stronger oracles , with ✏g = 0 and |f ( x , ⇠ ) ( x ) | ✏f deterministically . Under these assumptions , [ CS17 ] and [ BCS19 ] derive an expected iteration complexity bound . In this paper , we provide a high probability tail bound on the iteration complexity , showing that the algorithm is very likely to succeed in a number of iterations on the order of its expected iteration complexity . Moreover , we consider more general oracles , with arbitrary ✏g and possibly unbounded |f ( x , ⇠ ) ( x ) | . Thus , we significantly strengthen the results in [ CS17 ] and [ BCS19 ] . To the best of our knowledge , the only other high probability complexity bound of this kind is derived in [ GRVZ18 ] for a trust region algorithm under the assumption ✏g = 0 and |f ( x , ⇠ ) ( x ) |= 0 deterministically , which are much stronger oracles . Stochastic line search has also been analyzed in [ PS20 ] and [ VML+19 ] . In [ PS20 ] the assumptions on |f ( x , ⇠ ) ( x ) | are different . On the one hand , they allow for more general distributions than sub-exponential . On the other hand , it is assumed that |f ( x , ⇠ ) ( x ) | can be made arbitrarily small with some fixed probability . An expected iteration complexity bound is then derived for arbitrarily small `` . In contrast , we do not assume this , and analyze the iteration complexity of reaching an `` -stationary point , with `` lower-bounded by a function of the constants in the oracles . Moreover , our analysis and results are much simpler than those in [ PS20 ] and we derive an iteration complexity bound in high probability , not just in expectation . In [ VML+19 ] , the traditional line search is analyzed for empirical loss minimization , where the function oracles are implemented using a random mini-batch of a fixed size . The mini-batch remains fixed during backtracking until a standard Armijo condition is satisfied . Thus the search direction remains the same until a step is taken . While good computational performance has been reported in [ VML+19 ] , its theoretical analysis requires several very restrictive assumptions , especially for nonconvex functions . Also , they bound the expected sum of squared gradient norms , while we bound the iteration complexity with high probability . We note that using similar techniques as in [ BCS19 ] , our analysis can be extended to the convex and strongly convex cases . In summary , we present an analysis of an adaptive line search algorithm under very general conditions on the gradient and function estimates . The results not only subsume most results in the prior literature , but also substantially extend the framework . Moreover , high probability tail bounds on iteration complexity are derived , instead of only expected iteration complexity . 2 Oracles . In this section , we discuss a couple of settings , and show how they are captured by our framework . All norms used are 2-norm . 2.1 Expected loss minimization . Let us first discuss how the oracle definitions apply to expected loss minimization . In this setting , ( x ) = Ed⇠D [ ` ( x , d ) ] . Here , x is the model parameters , d is a data sample following distribution D , and ` ( x , d ) is the loss when the model parameterized by x is evaluated on data point d. In this case , the zeroth and first order oracles can be as follows , where S is a mini-batch sampled from D : f ( x , S ) = 1 |S| X d2S ` ( x , d ) , g ( x , S ) = 1 |S| X d2S rx ` ( x , d ) . ( 3 ) In general , S can be chosen to depend on x . We now show how our zeroth and first order oracle conditions are satisfied by selecting an appropriate sample size |S| . Proposition 1 . Let ê ( x , d ) : = ` ( x , d ) ( x ) be a ( ⌫̂ ( x ) , b̂ ( x ) ) -subexponential random variable and Vard⇠D [ ` ( x , d ) ] ✏̂ ( x ) 2 , for some ⌫̂ ( x ) , b̂ ( x ) , ✏̂ ( x ) . Let e ( x , S ) = |f ( x , S ) ( x ) | and N = |S| , then ES [ e ( x , S ) ] 1 p N ✏̂ ( x ) and e ( x , S ) is ( ⌫ ( x ) , b ( x ) ) -subexponential , with ⌫ ( x ) = b ( x ) = 8e2 max n ⌫̂ ( x ) p N , b̂ ( x ) o . In the case when the support of D is bounded , ` is Lipschitz , and the set of x we consider is bounded , the assumption of Proposition 1 is satisfied . Thus , f ( x , S ) is a zeroth order oracle with ✏f = supx 1p N ✏̂ ( x ) , ⌫ = supx ⌫ ( x ) , and b = supx b ( x ) , and ✏f can be made arbitrarily small by taking a large enough sample . Under standard assumptions on r ` ( x , d ) , for instance , suppose Assumption 4.3 in [ BCN18 ] holds : for some Mc , Mv 0 and for all x , Ed⇠D kr ` ( x , d ) r ( x ) k2 Mc +Mv kr ( x ) k2 , ( 4 ) one can show g ( x , S ) is a first order oracle with a large enough sample size . Proposition 2 . Let g = g ( x , S ) . Assuming Ed⇠Dr ` ( x , d ) = r ( x ) , then |S| Mc +Mv kr ( x ) k 2 min ( 1 ✏2g , ( 1 + ↵ ) 2 2↵2 kr ( x ) k2 ) implies P ( kg r ( x ) k max { ✏g , ↵kgk } ) 1 . This bound implies a looser bound of : |S| max ( 2Mc ✏2g , 2Mv ( 1 + ↵ ) 2 2↵2 ) . Remark Let us discuss what is required from the minibatch size in this setting . Unlike standard SGD , the minibatch size is chosen dynamically . When convergence to a stationary point is desired , ✏g has to be zero , and gradient estimate gk tends to zero . Thus , kg r ( x ) k also has to tend to zero . If Mc = 0 , then fixing the mini-batch size to be at least Mv ( 1+↵ ) 2 2↵2 provides a valid first order oracle . Thus , unless ↵ tends to zero , the minibatch size remains bounded from below . This is similar to the interpolation condition used in e.g . [ BM11 , VML+19 ] . On the other hand , when Mc > 0 , the minibatch size has to grow in order to approach a stationary point . This is similar to dynamic minibatch size selection , discussed , e.g . in [ BCNW12 ] . The difference between our results and those in [ BCNW12 ] is that our batch bound is implementable and guarantees convergence , while the one in [ BCNW12 ] is implementable only as a heuristic . As our computational results show , however , a fixed and small mini-batch size appears to work very well , perhaps because Mc is small .

Summary: The paper considers a new line search variant for minimizing functions with access to function value (zero order) and gradient (first order) estimates only through probabilistic oracles. For both oracles the output is a function of the input point and a random variable, and with probability 1-delta the output is close to the actual value. For the remaining delta probability, the first order oracle assumes nothing while the zero order oracle assumes that large errors are increasingly unlikely, making the requirements for the two oracles different.  The random variable in the oracle, could for instance represent the mini-batch used to estimate the required quantity. The authors propose a variant of Armijo Line search that  - Adds some slack on the Armijo condition depending on the quality of the zeroth order oracle  - Recompute gradient and function values in each step in the line search instead of using the same one.  For this line search method the authors show high probability iteration bounds for minimizing a Lipshitz bounded smooth function, that converge to an epsilon stationary point (norm gradient less than epsilon) where epsilon is lower bounded by the quality of the oracles.  The bounds are more general in the sense that less is assumed about the oracles, and that the paper shows high probability iteration bounds, where the usual is a bound on the expected number of iterations.