Stochastic Bias-Reduced Gradient Methods

1 Introduction . Consider the fundamental problem of minimizing a µ-strongly convex function F : X ! R given access to a stochastic ( sub- ) gradient estimator r̂F satisfying E r̂F ( x ) 2 @ F ( x ) and Ekr̂F ( x ) k2 G 2 for every x 2 X . Is it possible to transform the unbiased estimator r̂F into a ( nearly ) unbiased estimator of the minimizer x ? : = argminx2X F ( x ) ? In particular , can we improve upon the O ( G/ ( µ p T ) ) bias achieved by T iterations of stochastic gradient descent ( SGD ) ? In this paper , we answer this question in the affirmative , proposing an optimum estimator x̂ ? , which ( for any fixed > 0 ) has bias kEx̂ ? x ? k = O ( ) and variance Ekx̂ ? Ex̂ ? k2 = O ✓ G 2 µ2 log ✓ G µ ◆◆ , and , in expectation , costs O ( log ( Gµ ) ) evaluations of r̂F . 3 Setting = G/ ( µ p T ) , we obtain the same bias bound as T iterations of SGD , but with expected cost of only O ( log T ) stochastic gradient evaluations ( the worst-case cost is T ) . Further , the bias can be made arbitrarily small with only logarithmic increase in the variance and the stochastic gradient evaluations of our estimator , and therefore—paralleling the term “ nearly linear-time ” [ 27 ] —we call x̂ ? nearly unbiased . Our estimator is an instance of the multilevel Monte Carlo technique for de-biasing estimator sequences [ 25 ] and more specifically the method of Blanchet and Glynn [ 8 ] . Our key observation is that this method is readily applicable to strongly-convex variants of SGD , or indeed any stochastic optimization method with the same ( optimal ) rate of convergence . ⇤Stanford University , { asi , jmblpati , yujiajin , sidford } @ stanford.edu †Tel Aviv University , ycarmon @ tauex.tau.ac.il 3When X = BR ( x0 ) ⇢ Rd , F ( x ) = 1n P i2 [ n ] F̂ ( x ; i ) , and r̂F is the subgradient of a uniformly random F̂ ( x ; i ) we can also get an estimator with bias 0 and expected cost O ( log ( nd ) ) . See Appendix A.1 for details . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . 1.1 Estimating proximal points and Moreau-Yoshida envelope gradients . Given a convex function f and regularization level , the proximal point of y is Pf , ( y ) : = argminx2Rd f ( x ) + 2 kx yk 2 . Since computing Pf , amounts to solving a -strongly-convex problem , our technique provides low-bias and cheap proximal point estimators . Proximal points are ubiquitous in optimization [ 43 , 19 , 52 , 38 ] and estimating them efficiently with low bias opens up new algorithmic possibilities . One of these possibilities is estimating the gradient of the Moreau-Yoshida envelope f ( y ) : = minx2Rd f ( x ) + 2 kx yk 2 , which is a -smooth , G2/ ( 2 ) -accurate approximation of any G-Lipschitz f ( see , e.g. , [ 43 , 29 ] and Appendix B.3 ) . Since rf ( y ) = ( y Pf , ( y ) ) , our optimum estimator provides a low-bias estimator for rf ( y ) with second moment and expected cost greater than those of r̂f by only a logarithmic factor . Thus , for any non-smooth f we can turn r̂f into a gradient estimator for the smooth surrogate f , whose smoothness is independent of the problem dimension , allowing us to perform dimension-free randomized smoothing [ 20 ] . 1.2 Applications via accelerated bias-reduced methods . Our optimum estimator is a new primitive in stochastic convex optimization and we expect it to find multiple applications . We now describe three such applications : the first improves on previously known complexity bounds while the latter two recover existing bounds straightforwardly . For simplicity of presentation we assume ( in the introduction only ) Ekr̂fk2 1 and unit domain size . In each application , we wish to minimize an objective function given access to a cheap subgradient estimator r̂f as well as an expensive application-specific operation O ( e.g. , a projection to a complicated set ) . Direct use of the standard stochastic gradient method finds an ✏-accurate solution using O ( ✏ 2 ) computations of both r̂f and O , and our goal is to improve the O complexity without hurting the r̂f complexity . To that end , we design stochastic accelerated methods consisting of T iterations , each one involving only a constant number of O and proximal point computations , which we approximate by averaging copies of our optimum estimator.4 Its low bias allows us to bound T ⌧ ✏ 2 as though our proximal points were exact , while maintaining an eO ( ✏ 2 ) bound on the total expected number of r̂f calls.5 Thus , we save expensive operations without substantially increasing the gradient estimation cost . Table 1 summarizes each application , and we briefly describe them below . Minimizing the maximal loss ( Section 4 ) . Given N convex , 1-Lipschitz functions f ( 1 ) , . . . , f ( N ) we would like to find an ✏-approximate minimizer of their maximum fmax ( x ) = maxi2 [ N ] f ( i ) ( x ) . This problem naturally arises when optimizing worst-case behavior , as in maximum margin classification and robust optimization [ 53 , 15 , 45 , 6 ] . We measure complexity by the number of individual function and subgradient evaluations , so that the expensive operation of evaluating f ( 1 ) , . . . , f ( N ) at a single point has complexity O ( N ) and the subgradient method solves this problem with complexity O ( N✏ 2 ) . Carmon et al . [ 13 ] develop an algorithm for minimizing fmax with complexity eO ( N✏ 2/3 + ✏ 8/3 ) , improving on the subgradient method for sufficiently large N . Using our bias-reduced Moreau gradient envelope estimator in a Monteiro-Svaiter-type accelerated proximal point method [ 12 , 11 , 38 ] , we obtain improved complexity eO ( N✏ 2/3 + ✏ 2 ) . This matches ( up to logarithmic factors ) a lower bound shown in [ 13 ] , settling the complexity of minimizing the maximum 4While averaging is parallelizable , our optimum estimator itself is sequential . Consequently , our approach does not yield improve parallelism ; see Appendix A.2 for further discussion 5It is easy to turn expected complexity bounds into deterministic ones ; see Appendix A.3 . of N non-smooth functions . Our result reveals a surprising fact : for N ⌧ ( GR/✏ ) 4/3 , minimizing the maximum of N functions is no harder than minimizing their average . Projection-efficient optimization via dimension-free randomized smoothing ( Section 3 ) . Consider the problem of minimizing a convex function f using an unbiased gradient estimator r̂f over convex set X for which Euclidean projections are expensive to compute ( for example , the cone of PSD matrices ) . When f is L-smooth , a stochastic version of Nesterov ’ s accelerated gradient descent ( AGD ) [ 16 ] performs only O ( p L/✏ ) projections . For non-smooth f we instead apply AGD to the Moreau envelope smoothing of f ( with appropriate = O ( ✏ 1 ) ) using our nearly-unbiased stochastic estimator for rf . This yields a solution in O ( ✏ 1 ) projections and eO ( ✏ 2 ) evaluations of r̂f . Our algorithm provides a simple alternative to the recent work of Thekumparampil et al . [ 51 ] whose performance guarantees are identical up to a logarithmic factor . Gradient-efficient composite optimization ( Section 5 ) . We would like to minimize ( x ) = ⇤ ( x ) + f ( x ) , where ⇤ is convex and L-smooth but we can access it only via computing ( expensive ) exact gradients , while f is a non-smooth convex functions for which we have a ( cheap ) unbiased subgradient estimator r̂f . Problems of this type include inverse problems with sparsity constraints and regularized loss minimization in machine learning [ 34 ] . To save r⇤ computations , it is possible to use composite AGD [ 41 ] which solves O ( p L/✏ ) subproblems of the form minimizex hr⇤ ( y ) , xi+ f ( x ) + 2 kx x 0 k 2 . Lan [ 34 ] designed a specialized method , gradient sliding , for which the total subproblem solution cost is O ( ✏ 2 ) evaluations of r̂f . We show that a simple alternative—estimating the subproblem solutions via our low-bias optimum estimator—recovers its guarantees up to logarithmic factors . 1.3 Non-smooth differentially private stochastic convex optimization . We now discuss a potential application of our technique that is conditional on the existence of an improved optimum estimator . In it , we minimize the population objective function f ( x ) = ES⇠P f̂ ( x ; S ) under the well-known constraint of differential privacy [ 22 ] . Given n i.i.d . samples Si ⇠ P and assuming that each f̂ is 1-Lipschitz , convex and sufficiently smooth , Feldman et al . [ 23 ] develop algorithms that obtain the optimal error and compute O ( n ) subgradients of f̂ . The nonsmooth case is more challenging and the best existing bound is O ( n11/8 ) for the high-dimensional setting d = n [ 32 , 3 ] . In Section 6 we show that our optimum estimator , combined with recent localization techniques [ 23 ] , reduces the problem to private mean estimation . Unfortunately , our estimator is heavy-tailed , leading to insufficient utility . Nevertheless , assuming a version of our estimator that has bounded outputs , we give an algorithm that queries eO ( n ) subgradients for nonsmooth functions , solving a longstanding open problem in private optimization [ 14 , 4 ] . This motivates the study of improved versions of our estimators that have constant sensitivity . 1.4 Related work . Multilevel Monte-Carlo ( MLMC ) techniques originate from the literature on parametric integration for solving integral and differential equations [ 25 ] . Our approach is based on an MLMC variant put forth by Blanchet and Glynn [ 8 ] for estimating functionals of expectations . Among several applications , they propose [ 8 , Section 5.2 ] an estimator for argminx ES⇠P f̂ ( x ; S ) where f̂ ( · ; s ) is convex for all s and assuming access to minimizers of empirical objectives of the form P i2 [ N ] f̂ ( x ; si ) . The authors provide a preliminary analysis of the estimator ’ s variance ( later elaborated in [ 9 ] ) using an asymptotic Taylor expansion around the population minimizer . In comparison , we study the more general setting of stochastic gradient estimators and provide a complete algorithm based on SGD , along with a non-asymptotic analysis and concrete settings where our estimator is beneficial . A number of works have used the Blanchet-Glynn estimator in the context of optimization and machine learning . These applications include estimating the ratio of expectations for semisupervised learning [ 7 ] , estimating gradients of distributionally robust optimization objectives [ 35 ] , and estimating gradients in deep latent variable models [ 47 ] . Our estimator is similar to that of Levy et al . [ 35 ] in that we also have to pick a “ critical ” doubling probability for the ( random ) computational budget , which makes the expected cost and variance of our estimators depend logarithmically on the bias .

Summary: The paper develops a new primitive for stochastic optimization – a low-biased and low-cost estimator the minimizer x* for any Lipschitz strongly-convex function. Specially, the author(s) use a multi-level Monte-Carlo method to turn any optimal stochastic gradient method into an estimator x*. The paper also shows the potential of the estimator through four applications.