Stochastic Bias-Reduced Gradient Methods

1 Introduction . Consider the fundamental problem of minimizing a µ-strongly convex function F : X ! R given access to a stochastic ( sub- ) gradient estimator r̂F satisfying E r̂F ( x ) 2 @ F ( x ) and Ekr̂F ( x ) k2 G 2 for every x 2 X . Is it possible to transform the unbiased estimator r̂F into a ( nearly ) unbiased estimator of the minimizer x ? : = argminx2X F ( x ) ? In particular , can we improve upon the O ( G/ ( µ p T ) ) bias achieved by T iterations of stochastic gradient descent ( SGD ) ? In this paper , we answer this question in the affirmative , proposing an optimum estimator x̂ ? , which ( for any fixed > 0 ) has bias kEx̂ ? x ? k = O ( ) and variance Ekx̂ ? Ex̂ ? k2 = O ✓ G 2 µ2 log ✓ G µ ◆◆ , and , in expectation , costs O ( log ( Gµ ) ) evaluations of r̂F . 3 Setting = G/ ( µ p T ) , we obtain the same bias bound as T iterations of SGD , but with expected cost of only O ( log T ) stochastic gradient evaluations ( the worst-case cost is T ) . Further , the bias can be made arbitrarily small with only logarithmic increase in the variance and the stochastic gradient evaluations of our estimator , and therefore—paralleling the term “ nearly linear-time ” [ 27 ] —we call x̂ ? nearly unbiased . Our estimator is an instance of the multilevel Monte Carlo technique for de-biasing estimator sequences [ 25 ] and more specifically the method of Blanchet and Glynn [ 8 ] . Our key observation is that this method is readily applicable to strongly-convex variants of SGD , or indeed any stochastic optimization method with the same ( optimal ) rate of convergence . ⇤Stanford University , { asi , jmblpati , yujiajin , sidford } @ stanford.edu †Tel Aviv University , ycarmon @ tauex.tau.ac.il 3When X = BR ( x0 ) ⇢ Rd , F ( x ) = 1n P i2 [ n ] F̂ ( x ; i ) , and r̂F is the subgradient of a uniformly random F̂ ( x ; i ) we can also get an estimator with bias 0 and expected cost O ( log ( nd ) ) . See Appendix A.1 for details . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . 1.1 Estimating proximal points and Moreau-Yoshida envelope gradients . Given a convex function f and regularization level , the proximal point of y is Pf , ( y ) : = argminx2Rd f ( x ) + 2 kx yk 2 . Since computing Pf , amounts to solving a -strongly-convex problem , our technique provides low-bias and cheap proximal point estimators . Proximal points are ubiquitous in optimization [ 43 , 19 , 52 , 38 ] and estimating them efficiently with low bias opens up new algorithmic possibilities . One of these possibilities is estimating the gradient of the Moreau-Yoshida envelope f ( y ) : = minx2Rd f ( x ) + 2 kx yk 2 , which is a -smooth , G2/ ( 2 ) -accurate approximation of any G-Lipschitz f ( see , e.g. , [ 43 , 29 ] and Appendix B.3 ) . Since rf ( y ) = ( y Pf , ( y ) ) , our optimum estimator provides a low-bias estimator for rf ( y ) with second moment and expected cost greater than those of r̂f by only a logarithmic factor . Thus , for any non-smooth f we can turn r̂f into a gradient estimator for the smooth surrogate f , whose smoothness is independent of the problem dimension , allowing us to perform dimension-free randomized smoothing [ 20 ] . 1.2 Applications via accelerated bias-reduced methods . Our optimum estimator is a new primitive in stochastic convex optimization and we expect it to find multiple applications . We now describe three such applications : the first improves on previously known complexity bounds while the latter two recover existing bounds straightforwardly . For simplicity of presentation we assume ( in the introduction only ) Ekr̂fk2 1 and unit domain size . In each application , we wish to minimize an objective function given access to a cheap subgradient estimator r̂f as well as an expensive application-specific operation O ( e.g. , a projection to a complicated set ) . Direct use of the standard stochastic gradient method finds an ✏-accurate solution using O ( ✏ 2 ) computations of both r̂f and O , and our goal is to improve the O complexity without hurting the r̂f complexity . To that end , we design stochastic accelerated methods consisting of T iterations , each one involving only a constant number of O and proximal point computations , which we approximate by averaging copies of our optimum estimator.4 Its low bias allows us to bound T ⌧ ✏ 2 as though our proximal points were exact , while maintaining an eO ( ✏ 2 ) bound on the total expected number of r̂f calls.5 Thus , we save expensive operations without substantially increasing the gradient estimation cost . Table 1 summarizes each application , and we briefly describe them below . Minimizing the maximal loss ( Section 4 ) . Given N convex , 1-Lipschitz functions f ( 1 ) , . . . , f ( N ) we would like to find an ✏-approximate minimizer of their maximum fmax ( x ) = maxi2 [ N ] f ( i ) ( x ) . This problem naturally arises when optimizing worst-case behavior , as in maximum margin classification and robust optimization [ 53 , 15 , 45 , 6 ] . We measure complexity by the number of individual function and subgradient evaluations , so that the expensive operation of evaluating f ( 1 ) , . . . , f ( N ) at a single point has complexity O ( N ) and the subgradient method solves this problem with complexity O ( N✏ 2 ) . Carmon et al . [ 13 ] develop an algorithm for minimizing fmax with complexity eO ( N✏ 2/3 + ✏ 8/3 ) , improving on the subgradient method for sufficiently large N . Using our bias-reduced Moreau gradient envelope estimator in a Monteiro-Svaiter-type accelerated proximal point method [ 12 , 11 , 38 ] , we obtain improved complexity eO ( N✏ 2/3 + ✏ 2 ) . This matches ( up to logarithmic factors ) a lower bound shown in [ 13 ] , settling the complexity of minimizing the maximum 4While averaging is parallelizable , our optimum estimator itself is sequential . Consequently , our approach does not yield improve parallelism ; see Appendix A.2 for further discussion 5It is easy to turn expected complexity bounds into deterministic ones ; see Appendix A.3 . of N non-smooth functions . Our result reveals a surprising fact : for N ⌧ ( GR/✏ ) 4/3 , minimizing the maximum of N functions is no harder than minimizing their average . Projection-efficient optimization via dimension-free randomized smoothing ( Section 3 ) . Consider the problem of minimizing a convex function f using an unbiased gradient estimator r̂f over convex set X for which Euclidean projections are expensive to compute ( for example , the cone of PSD matrices ) . When f is L-smooth , a stochastic version of Nesterov ’ s accelerated gradient descent ( AGD ) [ 16 ] performs only O ( p L/✏ ) projections . For non-smooth f we instead apply AGD to the Moreau envelope smoothing of f ( with appropriate = O ( ✏ 1 ) ) using our nearly-unbiased stochastic estimator for rf . This yields a solution in O ( ✏ 1 ) projections and eO ( ✏ 2 ) evaluations of r̂f . Our algorithm provides a simple alternative to the recent work of Thekumparampil et al . [ 51 ] whose performance guarantees are identical up to a logarithmic factor . Gradient-efficient composite optimization ( Section 5 ) . We would like to minimize ( x ) = ⇤ ( x ) + f ( x ) , where ⇤ is convex and L-smooth but we can access it only via computing ( expensive ) exact gradients , while f is a non-smooth convex functions for which we have a ( cheap ) unbiased subgradient estimator r̂f . Problems of this type include inverse problems with sparsity constraints and regularized loss minimization in machine learning [ 34 ] . To save r⇤ computations , it is possible to use composite AGD [ 41 ] which solves O ( p L/✏ ) subproblems of the form minimizex hr⇤ ( y ) , xi+ f ( x ) + 2 kx x 0 k 2 . Lan [ 34 ] designed a specialized method , gradient sliding , for which the total subproblem solution cost is O ( ✏ 2 ) evaluations of r̂f . We show that a simple alternative—estimating the subproblem solutions via our low-bias optimum estimator—recovers its guarantees up to logarithmic factors . 1.3 Non-smooth differentially private stochastic convex optimization . We now discuss a potential application of our technique that is conditional on the existence of an improved optimum estimator . In it , we minimize the population objective function f ( x ) = ES⇠P f̂ ( x ; S ) under the well-known constraint of differential privacy [ 22 ] . Given n i.i.d . samples Si ⇠ P and assuming that each f̂ is 1-Lipschitz , convex and sufficiently smooth , Feldman et al . [ 23 ] develop algorithms that obtain the optimal error and compute O ( n ) subgradients of f̂ . The nonsmooth case is more challenging and the best existing bound is O ( n11/8 ) for the high-dimensional setting d = n [ 32 , 3 ] . In Section 6 we show that our optimum estimator , combined with recent localization techniques [ 23 ] , reduces the problem to private mean estimation . Unfortunately , our estimator is heavy-tailed , leading to insufficient utility . Nevertheless , assuming a version of our estimator that has bounded outputs , we give an algorithm that queries eO ( n ) subgradients for nonsmooth functions , solving a longstanding open problem in private optimization [ 14 , 4 ] . This motivates the study of improved versions of our estimators that have constant sensitivity . 1.4 Related work . Multilevel Monte-Carlo ( MLMC ) techniques originate from the literature on parametric integration for solving integral and differential equations [ 25 ] . Our approach is based on an MLMC variant put forth by Blanchet and Glynn [ 8 ] for estimating functionals of expectations . Among several applications , they propose [ 8 , Section 5.2 ] an estimator for argminx ES⇠P f̂ ( x ; S ) where f̂ ( · ; s ) is convex for all s and assuming access to minimizers of empirical objectives of the form P i2 [ N ] f̂ ( x ; si ) . The authors provide a preliminary analysis of the estimator ’ s variance ( later elaborated in [ 9 ] ) using an asymptotic Taylor expansion around the population minimizer . In comparison , we study the more general setting of stochastic gradient estimators and provide a complete algorithm based on SGD , along with a non-asymptotic analysis and concrete settings where our estimator is beneficial . A number of works have used the Blanchet-Glynn estimator in the context of optimization and machine learning . These applications include estimating the ratio of expectations for semisupervised learning [ 7 ] , estimating gradients of distributionally robust optimization objectives [ 35 ] , and estimating gradients in deep latent variable models [ 47 ] . Our estimator is similar to that of Levy et al . [ 35 ] in that we also have to pick a “ critical ” doubling probability for the ( random ) computational budget , which makes the expected cost and variance of our estimators depend logarithmically on the bias .

Summary: This paper proposes a procedure that converts any optimal stochastic gradient method into an estimator of the optimum $x^*$ using the multilevel Monte-Carlo approach. The estimator has a smaller expected cost of stochastic gradient computation compared with the original stochastic gradient method. The authors demonstrate that the proposed estimator can be combined with existing methods to improve the computation complexity and provide alternative algorithms to other methods in different applications.