Linear Convergent Decentralized Optimization with Compression

1 INTRODUCTION . Distributed optimization solves the following optimization problem x∗ : = argmin x∈Rd [ f ( x ) : = 1 n n∑ i=1 fi ( x ) ] ( 1 ) with n computing agents and a communication network . Each fi ( x ) : Rd → R is a local objective function of agent i and typically defined on the data Di settled at that agent . The data distributions { Di } can be heterogeneous depending on the applications such as in federated learning . The variable x ∈ Rd often represents model parameters in machine learning . A distributed optimization algorithm seeks an optimal solution that minimizes the overall objective function f ( x ) collectively . According to the communication topology , existing algorithms can be conceptually categorized into centralized and decentralized ones . Specifically , centralized algorithms require global communication between agents ( through central agents or parameter servers ) . While decentralized algorithms only require local communication between connected agents and are more widely applicable than centralized ones . In both paradigms , the computation can be relatively fast with powerful computing devices ; efficient communication is the key to improve algorithm efficiency and system scalability , especially when the network bandwidth is limited . In recent years , various communication compression techniques , such as quantization and sparsification , have been developed to reduce communication costs . Notably , extensive studies ( Seide et al. , 2014 ; Alistarh et al. , 2017 ; Bernstein et al. , 2018 ; Stich et al. , 2018 ; Karimireddy et al. , 2019 ; Mishchenko et al. , 2019 ; Tang et al. , 2019b ; Liu et al. , 2020 ) have utilized gradient compression to significantly boost communication efficiency for centralized optimization . They enable efficient large-scale optimization while maintaining comparable convergence rates and practical performance with their non-compressed counterparts . This great success has suggested the potential and significance of communication compression in decentralized algorithms . While extensive attention has been paid to centralized optimization , communication compression is relatively less studied in decentralized algorithms because the algorithm design and analysis are more challenging in order to cover general communication topologies . There are recent efforts trying to push this research direction . For instance , DCD-SGD and ECD-SGD ( Tang et al. , 2018a ) introduce difference compression and extrapolation compression to reduce model compression error . ( Reisizadeh et al. , 2019a ; b ) introduce QDGD and QuanTimed-DSGD to achieve exact convergence with small stepsize . DeepSqueeze ( Tang et al. , 2019a ) directly compresses the local model and compensates the compression error in the next iteration . CHOCO-SGD ( Koloskova et al. , 2019 ; 2020 ) presents a novel quantized gossip algorithm that reduces compression error by difference compression and preserves the model average . Nevertheless , most existing works focus on the compression of primal-only algorithms , i.e. , reduce to DGD ( Nedic & Ozdaglar , 2009 ; Yuan et al. , 2016 ) or P-DSGD ( Lian et al. , 2017 ) . They are unsatisfying in terms of convergence rate , stability , and the capability to handle heterogeneous data . Part of the reason is that they inherit the drawback of DGD-type algorithms , whose convergence rate is slow in heterogeneous data scenarios where the data distributions are significantly different from agent to agent . In the literature of decentralized optimization , it has been proved that primal-dual algorithms can achieve faster converge rates and better support heterogeneous data ( Ling et al. , 2015 ; Shi et al. , 2015 ; Li et al. , 2019 ; Yuan et al. , 2020 ) . However , it is unknown whether communication compression is feasible for primal-dual algorithms and how fast the convergence can be with compression . In this paper , we attempt to bridge this gap by investigating the communication compression for primal-dual decentralized algorithms . Our major contributions can be summarized as : • We delineate two key challenges in the algorithm design for communication compression in decentralized optimization , i.e. , data heterogeneity and compression error , and motivated by primal-dual algorithms , we propose a novel decentralized algorithm with compression , LEAD . • We prove that for LEAD , a constant stepsize in the range ( 0 , 2/ ( µ + L ) ] is sufficient to ensure linear convergence for strongly convex and smooth objective functions . To the best of our knowledge , LEAD is the first linear convergent decentralized algorithm with compression . Moreover , LEAD provably works with unbiased compression of arbitrary precision . • We further prove that if the stochastic gradient is used , LEAD converges linearly to the O ( σ2 ) neighborhood of the optimum with constant stepsize . LEAD is also able to achieve exact convergence to the optimum with diminishing stepsize . • Extensive experiments on convex problems validate our theoretical analyses , and the empirical study on training deep neural nets shows that LEAD is applicable for nonconvex problems . LEAD achieves state-of-art computation and communication efficiency in all experiments and significantly outperforms the baselines on heterogeneous data . Moreover , LEAD is robust to parameter settings and needs minor effort for parameter tuning . 2 RELATED WORKS . Decentralized optimization can be traced back to the work by Tsitsiklis et al . ( 1986 ) . DGD ( Nedic & Ozdaglar , 2009 ) is the most classical decentralized algorithm . It is intuitive and simple but converges slowly due to the diminishing stepsize that is needed to obtain the optimal solution ( Yuan et al. , 2016 ) . Its stochastic version D-PSGD ( Lian et al. , 2017 ) has been shown effective for training nonconvex deep learning models . Algorithms based on primal-dual formulations or gradient tracking are proposed to eliminate the convergence bias in DGD-type algorithms and improve the convergence rate , such as D-ADMM ( Mota et al. , 2013 ) , DLM ( Ling et al. , 2015 ) , EXTRA ( Shi et al. , 2015 ) , NIDS ( Li et al. , 2019 ) , D2 ( Tang et al. , 2018b ) , Exact Diffusion ( Yuan et al. , 2018 ) , OPTRA ( Xu et al. , 2020 ) , DIGing ( Nedic et al. , 2017 ) , GSGT ( Pu & Nedić , 2020 ) , etc . Recently , communication compression is applied to decentralized settings by Tang et al . ( 2018a ) . It proposes two algorithms , i.e. , DCD-SGD and ECD-SGD , which require compression of high accuracy and are not stable with aggressive compression . Reisizadeh et al . ( 2019a ; b ) introduce QDGD and QuanTimed-DSGD to achieve exact convergence with small stepsize and the convergence is slow . DeepSqueeze ( Tang et al. , 2019a ) compensates the compression error to the compression in the next iteration . Motivated by the quantized average consensus algorithms , such as ( Carli et al. , 2010 ) , the quantized gossip algorithm CHOCO-Gossip ( Koloskova et al. , 2019 ) converges linearly to the consensual solution . Combining CHOCO-Gossip and D-PSGD leads to a decentralized algorithm with compression , CHOCO-SGD , which converges sublinearly under the strong convexity and gradient boundedness assumptions . Its nonconvex variant is further analyzed in ( Koloskova et al. , 2020 ) . A new compression scheme using the modulo operation is introduced in ( Lu & De Sa , 2020 ) for decentralized optimization . A general algorithmic framework aiming to maintain the linear convergence of distributed optimization under compressed communication is considered in ( Magnússon et al. , 2020 ) . It requires a contractive property that is not satisfied by many decentralized algorithms including the algorithm in this paper . 3 ALGORITHM . We first introduce notations and definitions used in this work . We use bold upper-case letters such as X to define matrices and bold lower-case letters such as x to define vectors . Let 1 and 0 be vectors with all ones and zeros , respectively . Their dimensions will be provided when necessary . Given two matrices X , Y ∈ Rn×d , we define their inner product as 〈X , Y〉 = tr ( X > Y ) and the norm as ‖X‖ = √ 〈X , X〉 . We further define 〈X , Y〉P = tr ( X > PY ) and ‖X‖P = √ 〈X , X〉 P for any given symmetric positive semidefinite matrix P ∈ Rn×n . For simplicity , we will majorly use the matrix notation in this work . For instance , each agent i holds an individual estimate xi ∈ Rd of the global variable x ∈ Rd . Let Xk and∇F ( Xk ) be the collections of { xki } ni=1 and { ∇fi ( xki ) } ni=1 which are defined below : Xk = [ xk1 , . . . , x k n ] > ∈ Rn×d , ∇F ( Xk ) = [ ∇f1 ( xk1 ) , . . . , ∇fn ( xkn ) ] > ∈ Rn×d . ( 2 ) We use ∇F ( Xk ; ξk ) to denote the stochastic approximation of ∇F ( Xk ) . With these notations , the update Xk+1 = Xk − η∇F ( Xk ; ξk ) means that xk+1i = xki − η∇fi ( xki ; ξki ) for all i . In this paper , we need the average of all rows in Xk and ∇F ( Xk ) , so we define Xk = ( 1 > Xk ) /n and ∇F ( Xk ) = ( 1 > ∇F ( Xk ) ) /n . They are row vectors , and we will take a transpose if we need a column vector . The pseudoinverse of a matrix M is denoted as M† . The largest , ith-largest , and smallest nonzero eigenvalues of a symmetric matrix M are λmax ( M ) , λi ( M ) , and λmin ( M ) . Assumption 1 ( Mixing matrix ) . The connected network G = { V , E } consists of a node set V = { 1 , 2 , . . . , n } and an undirected edge set E . The primitive symmetric doubly-stochastic matrix W = [ wij ] ∈ Rn×n encodes the network structure such that wij = 0 if nodes i and j are not connected and can not exchange information . Assumption 1 implies that −1 < λn ( W ) ≤ λn−1 ( W ) ≤ · · ·λ2 ( W ) < λ1 ( W ) = 1 and W1 = 1 ( Xiao & Boyd , 2004 ; Shi et al. , 2015 ) . The matrix multiplication Xk+1 = WXk describes that agent i takes a weighted sum from its neighbors and itself , i.e. , xk+1i = ∑ j∈Ni∪ { i } wijx k j , where Ni denotes the neighbors of agent i . 3.1 THE PROPOSED ALGORITHM . The proposed algorithm LEAD to solve problem ( 1 ) is showed in Alg . 1 with matrix notations for conciseness . We will refer to the line number in the analysis . A complete algorithm description from the agent ’ s perspective can be found in Appendix A . The motivation behind Alg . 1 is to achieve two goals : ( a ) consensus ( xki − ( Xk ) > → 0 ) and ( b ) convergence ( ( Xk ) > → x∗ ) . We first discuss how goal ( a ) leads to goal ( b ) and then explain how LEAD fulfills goal ( a ) . In essence , LEAD runs the approximate SGD globally and reduces to the exact SGD under consensus . One key property for LEAD is 1 > n×1D k = 0 , regardless of the compression error in Ŷk . It holds because that for the initialization , we require D1 = ( I −W ) Z for some Z ∈ Rn×d , e.g. , D1 = 0n×d , and that the update of Dk ensures Dk ∈ Range ( I − W ) for all k and 1 > n×1 ( I −W ) = 0 as we will explain later . Therefore , multiplying ( 1/n ) 1 > n×1 on both sides of Line 7 leads to a global average view of Alg . 1 : Xk+1 = Xk − η∇F ( Xk ; ξk ) , ( 3 ) which doesn ’ t contain the compression error . Note that this is an approximate SGD step because , as shown in ( 2 ) , the gradient ∇F ( Xk ; ξk ) is not evaluated on a global synchronized model Xk . However , if the solution converges to the consensus solution , i.e. , xki − ( Xk ) > → 0 , then Eξk [ ∇F ( Xk ; ξk ) −∇f ( Xk ; ξk ) ] → 0 and ( 3 ) gradually reduces to exact SGD . Algorithm 1 LEAD Input : Stepsize η , parameter ( α , γ ) , X0 , H1 , D1 = ( I−W ) Z for any Z Output : XK or 1/n ∑n i=1 X K i 1 : H1w = WH 1 2 : X1 = X0 − η∇F ( X0 ; ξ0 ) 3 : for k = 1 , 2 , · · · , K − 1 do 4 : Yk = Xk − η∇F ( Xk ; ξk ) − ηDk 5 : Ŷk , Ŷkw , H k+1 , Hk+1w = COMM ( Y k , Hk , Hkw ) 6 : Dk+1 = Dk + γ2η ( Ŷ k − Ŷkw ) 7 : Xk+1 = Xk − η∇F ( Xk ; ξk ) − ηDk+1 8 : end for 9 : procedure COMM ( Y , H , Hw ) 10 : Q = COMPRESS ( Y −H ) 11 : Ŷ = H+Q 12 : Ŷw = Hw +WQ 13 : H = ( 1− α ) H+ αŶ 14 : Hw = ( 1− α ) Hw + αŶw 15 : Return : Ŷ , Ŷw , H , Hw 16 : end procedure With the establishment of how consensus leads to convergence , the obstacle becomes how to achieve consensus under local communication and compression challenges . It requires addressing two issues , i.e. , data heterogeneity and compression error . To deal with these issues , existing algorithms , such as DCD-SGD , ECD-SGD , QDGD , DeepSqueeze , Moniqua , and CHOCO-SGD , need a diminishing or constant but small stepsize depending on the total number of iterations . However , these choices unavoidably cause slower convergence and bring in the difficulty of parameter tuning . In contrast , LEAD takes a different way to solve these issues , as explained below . Data heterogeneity . It is common in distributed settings that there exists data heterogeneity among agents , especially in real-world applications where different agents collect data from different scenarios . In other words , we generally have fi ( x ) 6= fj ( x ) for i 6= j . The optimality condition of problem ( 1 ) gives 1 > n×1∇F ( X∗ ) = 0 , where X∗ = [ x∗ , · · · , x∗ ] is a consensual and optimal solution . The data heterogeneity and optimality condition imply that there exist at least two agents i and j such that ∇fi ( x∗ ) 6= 0 and ∇fj ( x∗ ) 6= 0 . As a result , a simple D-PSGD algorithm can not converge to the consensual and optimal solution as X∗ 6= WX∗ − ηEξ∇F ( X∗ ; ξ ) even when the stochastic gradient variance is zero . Gradient correction . Primal-dual algorithms or gradient tracking algorithms are able to convergence much faster than DGD-type algorithms by handling the data heterogeneity issue , as introduced in Section 2 . Specifically , LEAD is motivated by the design of primal-dual algorithm NIDS ( Li et al. , 2019 ) and the relation becomes clear if we consider the two-step reformulation of NIDS adopted in ( Li & Yan , 2019 ) : Dk+1 = Dk + I−W 2η ( Xk − η∇F ( Xk ) − ηDk ) , ( 4 ) Xk+1 = Xk − η∇F ( Xk ) − ηDk+1 , ( 5 ) where Xk and Dk represent the primal and dual variables respectively . The dual variable Dk plays the role of gradient correction . As k → ∞ , we expect Dk → −∇F ( X∗ ) and Xk will converge to X∗ via the update in ( 5 ) since Dk+1 corrects the nonzero gradient ∇F ( Xk ) asymptotically . The key design of Alg . 1 is to provide compression for the auxiliary variable defined as Yk = Xk − η∇F ( Xk ) − ηDk . Such design ensures that the dual variable Dk lies in Range ( I −W ) , which is essential for convergence . Moreover , it achieves the implicit error compression as we will explain later . To stabilize the algorithm with inexact dual update , we introduce a parameter γ to control the stepsize in the dual update . Therefore , if we ignore the details of the compression , Alg . 1 can be concisely written as Yk = Xk − η∇F ( Xk ; ξk ) − ηDk ( 6 ) Dk+1 = Dk + γ 2η ( I−W ) Ŷk ( 7 ) Xk+1 = Xk − η∇F ( Xk ; ξk ) − ηDk+1 ( 8 ) where Ŷk represents the compression of Yk and F ( Xk ; ξk ) denote the stochastic gradients . Nevertheless , how to compress the communication and how fast the convergence we can attain with compression error are unknown . In the following , we propose to carefully control the compression error by difference compression and error compensation such that the inexact dual update ( Line 6 ) and primal update ( Line 7 ) can still guarantee the convergence as proved in Section 4 . Compression error . Different from existing works , which typically compress the primal variable Xk or its difference , LEAD first construct an intermediate variable Yk and apply compression to obtain its coarse representation Ŷk as shown in the procedure COMM ( Y , H , Hw ) : • Compress the difference between Y and the state variable H as Q ; • Q is encoded into the low-bit representation , which enables the efficient local communication step Ŷw = Hw +WQ . It is the only communication step in each iteration . • Each agent recovers its estimate Ŷ by Ŷ = H+Q and we have Ŷw = WŶ . • States H and Hw are updated based on Ŷ and Ŷw , respectively . We have Hw = WH . By this procedure , we expect when both Yk and Hk converge to X∗ , the compression error vanishes asymptotically due to the assumption we make for the compression operator in Assumption 2 . Remark 1 . Note that difference compression is also applied in DCD-PSGD ( Tang et al. , 2018a ) and CHOCO-SGD ( Koloskova et al. , 2019 ) , but their state update is the simple integration of the compressed difference . We find this update is usually too aggressive and cause instability as showed in our experiments . Therefore , we adopt a momentum update H = ( 1−α ) H+αŶ motivated from DIANA ( Mishchenko et al. , 2019 ) , which reduces the compression error for gradient compression in centralized optimization . Implicit error compensation . On the other hand , even if the compression error exists , LEAD essentially compensates for the error in the inexact dual update ( Line 6 ) , making the algorithm more stable and robust . To illustrate how it works , let Ek = Ŷk −Yk denote the compression error and eki be its i-th row . The update of D k gives Dk+1 = Dk + γ 2η ( Ŷk − Ŷkw ) = Dk + γ 2η ( I−W ) Yk + γ 2η ( Ek −WEk ) where −WEk indicates that agent i spreads total compression error − ∑ j∈Ni∪ { i } wjie k i = −eki to all agents and Ek indicates that each agent compensates this error locally by adding eki back . This error compensation also explains why the global view in ( 3 ) doesn ’ t involve compression error . Remark 2 . Note that in LEAD , the compression error is compensated into the model Xk+1 through Line 6 and Line 7 such that the gradient computation in the next iteration is aware of the compression error . This has some subtle but important difference from the error compensation or error feedback in ( Seide et al. , 2014 ; Wu et al. , 2018 ; Stich et al. , 2018 ; Karimireddy et al. , 2019 ; Tang et al. , 2019b ; Liu et al. , 2020 ; Tang et al. , 2019a ) , where the error is stored in the memory and only compensated after gradient computation and before the compression . Remark 3 . The proposed algorithm , LEAD in Alg . 1 , recovers NIDS ( Li et al. , 2019 ) , D2 ( Tang et al. , 2018b ) , Exact Diffusion ( Yuan et al. , 2018 ) . These connections are established in Appendix B .

Summary: The paper introduces a novel decentralized algorithm (LEAD) incorporated with compression that achieves linear convergence rate in strongly convex setting. The main idea is to apply and communicate the compression of an auxiliary variable instead of the primal or dual iterates.  Convergence analysis is provided for both deterministic and stochastic variants. Experiments shows the state-of-the-art performance.