ZeroSARAH: Efficient Nonconvex Finite-Sum Optimization with Zero Full Gradient Computations

∑n i=1 fi ( x ) . To the best of our knowledge , in this noncon- vex finite-sum regime , all existing variance-reduced methods , including SARAH , SVRG , SAGA and their variants , need to compute the full gradient over all n data samples at the initial point x0 , and then periodically compute the full gradient once every few iterations ( for SVRG , SARAH and their variants ) . Note that SVRG , SAGA and their variants typically achieve weaker convergence results than variants of SARAH : n2/3/ 2 vs. n1/2/ 2 . Thus we focus on the variant of SARAH . The proposed ZeroSARAH and its distributed variant D-ZeroSARAH are the first variance-reduced algorithms which do not require any full gradient computations , not even for the initial point . Moreover , for both standard and distributed settings , we show that ZeroSARAH and D-ZeroSARAH obtain new state-of-the-art convergence results , which can improve the previous best-known result ( given by e.g. , SPIDER , SARAH , and PAGE ) in certain regimes . Avoiding any full gradient computations ( which are time-consuming steps ) is important in many applications as the number of data samples n usually is very large . Especially in the distributed setting , periodic computation of full gradient over all data samples needs to periodically synchronize all clients/devices/machines , which may be impossible or unaffordable . Thus , we expect that ZeroSARAH/D-ZeroSARAH will have a practical impact in distributed and federated learning where full device participation is impractical . 1 INTRODUCTION . Nonconvex optimization is ubiquitous across many domains of machine learning ( Jain & Kar , 2017 ) , especially in training deep neural networks . In this paper , we consider the nonconvex finite-sum problems of the form min x∈Rd { f ( x ) : = 1 n n∑ i=1 fi ( x ) } , ( 1 ) where f : Rd → R is a differentiable and possibly nonconvex function . Problem ( 1 ) captures the standard empirical risk minimization problems in machine learning ( Shalev-Shwartz & Ben-David , 2014 ) . There are n data samples and fi denotes the loss associated with i-th data sample . We assume the functions fi : Rd → R for all i ∈ [ n ] : = { 1 , 2 , . . . , n } are also differentiable and possibly nonconvex functions . Beyond the standard/centralized problem ( 1 ) , we further consider the distributed/federated nonconvex problems : min x∈Rd { f ( x ) : = 1 n n∑ i=1 fi ( x ) } , fi ( x ) : = 1 m m∑ j=1 fi , j ( x ) , ( 2 ) where n denotes the number of clients/devices/machines , fi denotes the loss associated with m data samples stored on client i , and all functions are differentiable and can be nonconvex . Avoiding any full gradient computations is important especially in this distributed setting ( 2 ) , periodic computation of full gradient over all data samples needs to periodically synchronize all clients , which may be impossible or very hard to achieve . There has been extensive research in designing first-order ( gradient-type ) methods for solving centralized/distributed nonconvex problems ( 1 ) and ( 2 ) such as SGD , SVRG , SAGA , SCSG , SARAH and their variants , e.g. , ( Ghadimi & Lan , 2013 ; Ghadimi et al. , 2016 ; Allen-Zhu & Hazan , 2016 ; Reddi et al. , 2016 ; Lei et al. , 2017 ; Li & Li , 2018 ; Zhou et al. , 2018 ; Fang et al. , 2018 ; Wang et al. , 2018 ; Ge et al. , 2019 ; Pham et al. , 2019 ; Li , 2019 ; Li & Richtárik , 2020 ; Horváth et al. , 2020 ; Li et al. , 2021 ) . Note that SVRG and SAGA variants typically achieve weaker convergence results than SARAH variants , i.e. , n2/3/ 2 vs. √ n/ 2 . Thus the current best convergence results are achieved by SARAH variants such as SPIDER ( Fang et al. , 2018 ) , SARAH ( Pham et al. , 2019 ) and PAGE ( Li et al. , 2021 ; Li , 2021 ) . However , all of these variance-reduced algorithms ( no matter based on SVRG , SAGA or SARAH ) require full gradient computations ( i.e. , compute ∇f ( x ) = 1n ∑n i=1∇fi ( x ) ) without assuming additional assumptions except standard L-smoothness assumptions . We would like to point out that under an additional bounded variance assumption ( e.g. , Ei [ ‖∇fi ( x ) −∇f ( x ) ‖2 ] ≤ σ2 , ∀x ∈ Rd ) , some of them ( such as SCSG ( Lei et al. , 2017 ) , SVRG+ ( Li & Li , 2018 ) , PAGE ( Li et al. , 2021 ) ) may avoid full gradient computations by using a large minibatch of stochastic gradients instead ( usually the minibatch size is O ( σ2/ 2 ) ) . Clearly , there exist some drawbacks : i ) σ2 usually is not known ; ii ) if the target error is very small ( defined as E [ ‖∇f ( x̂ ) ‖2 ] ≤ 2 in Definition 1 ) or σ is very large , then the minibatch size O ( σ2/ 2 ) is still very large for replacing full gradient computations . In this paper , we only consider algorithms under the standard L-smoothness assumptions , without assuming any other additional assumptions ( such as bounded variance assumption mentioned above ) . Thus , all existing variance-reduced methods , including SARAH , SVRG , SAGA and their variants , need to compute the full gradient over all n data samples at the initial point x0 , and then periodically compute the full gradient once every few iterations ( for SVRG , SARAH and their variants ) . However , full gradient computations are time-consuming steps in many applications as the number of data samples n usually is very large . Especially in the distributed setting , periodic computation of full gradient needs to periodically synchronize all clients/devices , which usually is impractical . Motivated by this , we focus on designing new algorithms which do not require any full gradient computations for solving standard and distributed nonconvex problems ( 1 ) – ( 2 ) . 2 OUR CONTRIBUTIONS . In this paper , we propose the first variance-reduced algorithm ZeroSARAH ( and also its distributed variant D-ZeroSARAH ) without computing any full gradients for solving both standard and distributed nonconvex finite-sum problems ( 1 ) – ( 2 ) . Moreover , ZeroSARAH and Distributed D-ZeroSARAH can obtain new state-of-the-art convergence results which improve previous bestknown results ( given by e.g. , SPIDER , SARAH and PAGE ) in certain regimes ( see Tables 1–2 for the comparison with previous algorithms ) . ZeroSARAH is formally described in Algorithm 2 , which is a variant of SARAH ( Nguyen et al. , 2017 ) . See Section 4 for more details and comparisons between ZeroSARAH and SARAH . Then , D-ZeroSARAH is formally described in Algorithm 3 of Section 5 , which is a distributed variant of our ZeroSARAH . Now , we highlight the following results achieved by ZeroSARAH and D-ZeroSARAH : • ZeroSARAH and D-ZeroSARAH are the first variance-reduced algorithms which do not require any full gradient computations , not even for the initial point ( see Algorithms 2–3 or Tables 1–2 ) . Avoiding any full gradient computations is important in many applications as the number of data samples n usually is very large . Especially in the distributed setting , periodic computation of full gradient over all data samples stored in all clients/devices may be impossible or very hard to achieve . We expect that ZeroSARAH/D-ZeroSARAH will have a practical impact in distributed and federated learning where full device participation is impractical . • Moreover , ZeroSARAH can recover the previous best-known convergence result O ( n + √ nL∆0 2 ) ( see Table 1 or Corollary 1 ) , and also provide new state-of-the-art convergence results without any full gradient computations ( see Table 1 or Corollary 2 ) which can improve the previous best result in certain regimes . • Besides , for the distributed nonconvex setting ( 2 ) , the distributed D-ZeroSARAH ( Algorithm 3 ) enjoys similar benefits as our ZeroSARAH , i.e. , D-ZeroSARAH does not need to periodically synchronize all n clients to compute any full gradients , and also provides new state-of-the-art convergence results . See Table 2 and Section 5 for more details . • Finally , the experimental results in Section 6 show that ZeroSARAH is slightly better than the previous state-of-the-art SARAH . However , we should point out that ZeroSARAH does not compute any full gradients while SARAH needs to periodically compute the full gradients for every l iterations ( here l = √ n ) . Thus the experiments validate our theoretical results ( can be slightly better than SARAH ( see Table 1 ) ) and confirm the practical superiority of ZeroSARAH ( avoid any full gradient computations ) . Similar experimental results of D-ZeroSARAH for the distributed setting are provided in Appendix A.2 . 3 PRELIMINARIES . Notation : Let [ n ] denote the set { 1 , 2 , · · · , n } and ‖ · ‖ denote the Euclidean norm for a vector and the spectral norm for a matrix . Let 〈u , v〉 denote the inner product of two vectors u and v. We use O ( · ) and Ω ( · ) to hide the absolute constant , and Õ ( · ) to hide the logarithmic factor . We will write ∆0 : = f ( x0 ) − f∗ , f∗ : = minx∈Rd f ( x ) , G0 : = 1n ∑n i=1 ‖∇fi ( x0 ) ‖2 , ∆̂0 : = f ( x0 ) − f̂∗ , f̂∗ : = 1n ∑n i=1 minx∈Rd fi ( x ) and G ′ 0 : = 1 nm ∑n , m i , j=1,1 ‖∇fi , j ( x0 ) ‖2 . Definition 1 A point x̂ is called an -approximate solution for nonconvex problems ( 1 ) and ( 2 ) if E [ ‖∇f ( x̂ ) ‖2 ] ≤ 2 . To show the convergence results , we assume the following standard smoothness assumption for nonconvex problems ( 1 ) . Assumption 1 ( L-smoothness ) A function fi : Rd → R is L-smooth if ∃L > 0 , such that ‖∇fi ( x ) −∇fi ( y ) ‖ ≤ L‖x− y‖ , ∀x , y ∈ Rd . ( 3 ) It is easy to see that f ( x ) = 1n ∑n i=1 fi ( x ) is also L-smooth under Assumption 1 . We can also relax Assumption 1 by defining Li-smoothness for each fi . Then if we further define the average L2 : = 1n ∑n i=1 L 2 i , we know that f ( x ) = 1 n ∑n i=1 fi ( x ) is also L-smooth . Here we use the same L just for simple representation . For the distributed nonconvex problems ( 2 ) , we use the following Assumption 2 instead of Assumption 1 . Similarly , we can also relax it by defining Li , j-smoothness for different fi , j . Here we use the same L just for simple representation . Assumption 2 ( L-smoothness ) A function fi , j : Rd → R is L-smooth if ∃L > 0 , such that ‖∇fi , j ( x ) −∇fi , j ( y ) ‖ ≤ L‖x− y‖ , ∀x , y ∈ Rd . ( 4 ) Algorithm 1 SARAH ( Nguyen et al. , 2017 ; Pham et al. , 2019 ) Input : initial point x0 , epoch length l , stepsize η , minibatch size b 1 : x̃ = x0 2 : for s = 0 , 1 , 2 , . . . do 3 : x0 = x̃ 4 : v0 = ∇f ( x0 ) = 1n n∑ i=1 ∇fi ( x0 ) // compute the full gradient once for every l iterations 5 : x1 = x0 − ηv0 6 : for k = 1 , 2 , . . . , l do 7 : Randomly sample a minibatch data samples Ib with |Ib| = b 8 : vk = 1b ∑ i∈Ib ( ∇fi ( xk ) −∇fi ( xk−1 ) ) + vk−1 9 : xk+1 = xk − ηvk 10 : end for 11 : x̃ randomly chosen from { xk } k∈ [ l ] or x̃ = xl+1 12 : end for Algorithm 2 SARAH without full gradient computations ( ZeroSARAH ) Input : initial point x0 , stepsize { ηk } , minibatch size { bk } , parameter { λk } 1 : x−1 = x0 2 : v−1 = 0 , y−11 = y −1 2 = · · · = y−1n = 0 // no full gradient computation 3 : for k = 0 , 1 , 2 , . . . do 4 : Randomly sample a minibatch data samples Ikb with |Ikb | = bk 5 : vk = 1bk ∑ i∈Ikb ( ∇fi ( xk ) −∇fi ( xk−1 ) ) + ( 1− λk ) vk−1 + λk ( 1 bk ∑ i∈Ikb ( ∇fi ( xk−1 ) − yk−1i ) + 1n n∑ j=1 yk−1j ) // no full gradient computations for vks 6 : xk+1 = xk − ηkvk 7 : yki = { ∇fi ( xk ) for i ∈ Ikb yk−1i for i /∈ Ikb // the update of { yki } directly follows from the stochastic gradients computed in Line 5 8 : end for 4 ZeroSARAH ALGORITHM AND ITS CONVERGENCE RESULTS In this section , we consider the standard/centralized nonconvex problems ( 1 ) . The distributed setting ( 2 ) is considered in the following Section 5 . 4.1 ZeroSARAH ALGORITHM We first describe the proposed ZeroSARAH in Algorithm 2 , which is a variant of SARAH ( Nguyen et al. , 2017 ) . To better compare with SARAH and ZeroSARAH , we also recall the original SARAH in Algorithm 1 . Now , we highlight some points for the difference between SARAH and our ZeroSARAH : • SARAH requires the full gradient computations for every epoch ( see Line 4 of Algorithm 1 ) . However , ZeroSARAH combines the past gradient estimator vk−1 with another estimator to avoid periodically computing the full gradient . See the difference between Line 8 of Algorithm 1 and Line 5 of Algorithm 2 ( also highlighted with blue color ) . • The gradient estimator vk in ZeroSARAH ( Line 5 of Algorithm 2 ) does not require more stochastic gradient computations compared with vk in SARAH ( Line 8 of Algorithm 1 ) if the minibatch size bk = b . • The new gradient estimator vk of ZeroSARAH also leads to simpler algorithmic structure , i.e. , single-loop in ZeroSARAH vs. double-loop in SARAH . •Moreover , the difference of gradient estimator vk also leads to different results in expectation , i.e. , 1 ) for SARAH : Ek [ vk −∇f ( xk ) ] = vk−1 −∇f ( xk−1 ) ; 2 ) for ZeroSARAH : Ek [ vk −∇f ( xk ) ] = ( 1− λk ) ( vk−1 −∇f ( xk−1 ) ) . 4.2 CONVERGENCE RESULTS FOR ZeroSARAH Now , we present the main convergence theorem ( Theorem 1 ) of ZeroSARAH ( Algorithm 2 ) for solving nonconvex finite-sum problems ( 1 ) . Subsequently , we formulate two corollaries which present the detailed convergence results by specifying the choice of parameters . In particular , we list the results of these two Corollaries 1–2 in Table 1 for comparing with convergence results of previous works . Theorem 1 Suppose that Assumption 1 holds . Choose stepsize ηk ≤ 1 L ( 1+ √ Mk+1 ) for any k ≥ 0 , where Mk+1 : = 2λk+1bk+1 + 8λk+1n 2 b3k+1 . Moreover , let λ0 = 1 , γ0 ≥ η02λ1 and α0 ≥ 2nλ1η0 b21 . Then the following equation holds for ZeroSARAH ( Algorithm 2 ) for solving problem ( 1 ) , for any iteration K ≥ 0 : E [ ‖∇f ( x̂K ) ‖2 ] ≤ 2∆0∑K−1 k=0 ηk + ( n− b0 ) ( 4γ0 + 2α0b0 ) G0 nb0 ∑K−1 k=0 ηk . ( 5 ) Remark : Note that we can upper bound both terms on the right-hand side of ( 5 ) . It means that there is no convergence neighborhood of ZeroSARAH and hence , ZeroSARAH can find an -approximate solution for any > 0 . In the following , we provide two detailed convergence results in Corollaries 1 and 2 by specifying two kinds of parameter settings . Note that the algorithm computes full gradient in iteration k if the minibatch bk = n. Our convergence results show that without computing any full gradients actually does not hurt the convergence performance of algorithms ( see Table 1 ) . In particular , we note that the second term of ( 5 ) will be deleted if we choose minibatch size b0 = n for the initial point x0 ( see Corollary 1 for more details ) . Here Corollary 1 only needs to compute the full gradient once for the initialization , and does not compute any full gradients later ( i.e. , bk ≡ √ n for all k > 0 ) . Also note that even if we choose b0 < n , we can also upper bound the second term of ( 5 ) . It means that ZeroSARAH can find an -approximate solution without computing any full gradients even for the initial point , i.e. , minibatch size bk < n for all iterations k ≥ 0 . For instance , we choose bk ≡ √ n for all k ≥ 0 in Corollary 2 , i.e. , ZeroSARAH never computes any full gradients even for the initial point . Corollary 1 Suppose that Assumption 1 holds . Choose stepsize ηk ≤ 1 ( 1+√8 ) L for any k ≥ 0 , minibatch size bk ≡ √ n and parameter λk = bk2n for any k ≥ 1 . Moreover , let b0 = n and λ0 = 1 . Then ZeroSARAH ( Algorithm 2 ) can find an -approximate solution for problem ( 1 ) such that E [ ‖∇f ( x̂K ) ‖2 ] ≤ 2 and the number of stochastic gradient computations can be bounded by # grad : = K−1∑ k=0 bk ≤ n+ 2 ( 1 + √ 8 ) √ nL∆0 2 = O ( n+ √ nL∆0 2 ) . Remark : In Corollary 1 , ZeroSARAH only computes the full gradient ∇f ( x0 ) = 1n ∑n i=1∇fi ( x0 ) once for the initial point x0 , i.e. , minibatch size b0 = n , and then bk ≡ √ n for all iterations k ≥ 1 in Algorithm 2 . In the following Corollary 2 , we show that ZeroSARAH without computing any full gradients even for the initial point does not hurt its convergence performance . Corollary 2 Suppose that Assumption 1 holds . Choose stepsize ηk ≤ 1 ( 1+√8 ) L for any k ≥ 0 , minibatch size bk ≡ √ n for any k ≥ 0 , and parameter λ0 = 1 and λk = bk2n for any k ≥ 1 . Then ZeroSARAH ( Algorithm 2 ) can find an -approximate solution for problem ( 1 ) such that E [ ‖∇f ( x̂K ) ‖2 ] ≤ 2 and the number of stochastic gradient computations can be bounded by # grad = O ( √ n ( L∆0 +G0 ) 2 ) . Note that G0 can be bounded by G0 ≤ 2L∆̂0 via L-smoothness Assumption 1 , then we also have # grad = O ( √ n ( L∆0 + L∆̂0 ) 2 ) . Remark : In Corollary 2 , ZeroSARAH never computes any full gradients even for the initial point , i.e. , minibatch size bk ≡ √ n for all iterations k ≥ 0 in Algorithm 2 . If we consider L , ∆0 , G0 or ∆̂0 as constant values then the stochastic gradient complexity in Corollary 2 is # grad = O ( √ n 2 ) , i.e. , full gradient computations do not appear in ZeroSARAH ( Algorithm 2 ) and the term ‘ n ’ also does not appear in its convergence result . Also note that the parameter settings ( i.e. , { ηk } , { bk } and { λk } in Algorithm 2 ) of Corollaries 1 and 2 are exactly the same except for b0 = n ( in Corollary 1 ) and b0 = √ n ( in Corollary 2 ) . Moreover , the parameter settings ( i.e. , { ηk } , { bk } and { λk } ) for Corollaries 1 and 2 only require the values of L and n , which is the same as all previous algorithms . If one further allows other values , e.g. , , G0 or ∆̂0 , for setting the initial b0 , then the gradient complexity can be further improved ( see Appendix D for more details ) . 5 D-ZeroSARAH ALGORITHM AND ITS CONVERGENCE RESULTS Now , we consider the distributed nonconvex problems ( 2 ) , i.e. , minx∈Rd { f ( x ) : = 1n ∑n i=1 fi ( x ) } with fi ( x ) : = 1m ∑m j=1 fi , j ( x ) , where n denotes the number of clients/devices/machines , fi denotes the loss associated with m data samples stored on client i . 5.1 D-ZeroSARAH ALGORITHM To solve distributed nonconvex problems ( 2 ) , we propose a distributed variant of ZeroSARAH ( called D-ZeroSARAH ) and describe it in Algorithm 3 . Same as our ZeroSARAH , D-ZeroSARAH also does not need to compute any full gradients at all . Avoiding any full gradient computations is important especially in this distributed setting , periodic computation of full gradient across all n clients may be impossible or unaffordable . Thus , we expect the proposed D-ZeroSARAH ( Algorithm 3 ) will have a practical impact in distributed and federated learning where full device participation is impractical . 5.2 CONVERGENCE RESULTS FOR D-ZeroSARAH Similar to ZeroSARAH in Section 4.2 , we also first present the main convergence theorem ( Theorem 2 ) of D-ZeroSARAH ( Algorithm 3 ) for solving distributed nonconvex problems ( 2 ) . Subsequently , we formulate two corollaries which present the detailed convergence results by specifying the choice of parameters . In particular , we list the results of these two Corollaries 3–4 in Table 2 for comparing with convergence results of previous works . Note that here we use the smoothness Assumption 2 instead of Assumption 1 for this distributed setting ( 2 ) . Theorem 2 Suppose that Assumption 2 holds . Choose stepsize ηk ≤ 1 L ( 1+ √ Wk+1 ) for any k ≥ 0 , whereWk+1 : = 2λk+1sk+1bk+1 + 8λk+1n 2m2 s3k+1b 3 k+1 . Moreover , let λ0 = 1 and θ0 : = nm ( nm−1 ) λ1 + 4nmλ1s0b0 s21b 2 1 . Then the following equation holds for D-ZeroSARAH ( Algorithm 3 ) for solving distributed problem ( 2 ) , for any iteration K ≥ 0 : E [ ‖∇f ( x̂K ) ‖2 ] ≤ 2∆0∑K−1 k=0 ηk + ( nm− s0b0 ) η0θ0G′0 nms0b0 ∑K−1 k=0 ηk . ( 6 ) Algorithm 3 Distributed ZeroSARAH ( D-ZeroSARAH ) Input : initial point x0 , parameters { ηk } , { sk } , { bk } , { λk } 1 : x−1 = x0 2 : v−1 = 0 , y−11 = y −1 2 = · · · = y−1n = 0 // no full gradient computation 3 : for k = 0 , 1 , 2 , . . . do 4 : Randomly sample a subset of clients Sk from n clients with size |Sk| = sk 5 : for each client i ∈ Sk do 6 : Sample the data minibatch Ikbi ( with size |I k bi | = bk ) from the m data samples in client i 7 : Compute its local minibatch gradient information : gki , curr = 1 bk ∑ j∈Ikbi ∇fi , j ( xk ) , gki , prev = 1bk ∑ j∈Ikbi ∇fi , j ( xk−1 ) , yki , prev = 1bk ∑ j∈Ikbi yk−1i , j yki , j = { ∇fi , j ( xk ) for j ∈ Ikbi yk−1i , j for j /∈ Ikbi , yki = 1 m m∑ j=1 yki , j 8 : end for 9 : vk = 1sk ∑ i∈Sk ( gki , curr − gki , prev ) + ( 1− λk ) vk−1 + λk 1sk ∑ i∈Sk ( gki , prev − yki , prev ) + λky k−1 // no full gradient computations for vks 10 : xk+1 = xk − ηkvk 11 : yk = 1n n∑ i=1 yki // here y k i = y k−1 i for client i /∈ S k 12 : end for Corollary 3 Suppose that Assumption 2 holds . Choose stepsize ηk ≤ 1 ( 1+√8 ) L for any k ≥ 0 , clients subset size sk ≡ √ n , minibatch size bk ≡ √ m and parameter λk = skbk2nm for any k ≥ 1 . Moreover , let s0 = n , b0 = m , and λ0 = 1 . Then D-ZeroSARAH ( Algorithm 3 ) can find an -approximate solution for distributed problem ( 2 ) such that E [ ‖∇f ( x̂K ) ‖2 ] ≤ 2 and the number of stochastic gradient computations for each client can be bounded by # grad = O ( m+ √ m n L∆0 2 ) . Corollary 4 Suppose that Assumption 2 holds . Choose stepsize ηk ≤ 1 ( 1+√8 ) L for any k ≥ 0 , clients subset size sk ≡ √ n and minibatch size bk ≡ √ m for any k ≥ 0 , and parameter λ0 = 1 and λk = skbk 2nm for any k ≥ 1 . Then D-ZeroSARAH ( Algorithm 3 ) can find an -approximate solution for distributed problem ( 2 ) such that E [ ‖∇f ( x̂K ) ‖2 ] ≤ 2 and the number of stochastic gradient computations for each client can be bounded by # grad = O ( √ m n L∆0 +G ′ 0 2 ) . Remark : Similar discussions and remarks of Theorem 1 and Corollaries 1–2 for ZeroSARAH in Section 4.2 also hold for the results of D-ZeroSARAH ( i.e. , Theorem 2 and Corollaries 3–4 ) .

Summary: This paper introduce ZeroSARAH which is a variant of variance reduced SARAH algorithm, for solving $\min_x 1/n \sum_{i=1}^n f_i(x)$ with nonconvex and smooth $f_i$. ZeroSARAH and its distributed version are the first algorithms that do not require any full gradient computations. Most algorithms for this setting either required computing the full gradient once in the beginning or periodically. The paper also argues that in distributed optimization, computation of full gradients can be a bottleneck and therefore avoiding it is important.