BASGD: Buffered Asynchronous SGD for Byzantine Learning

1 INTRODUCTION . Due to the wide application in cluster-based large-scale learning , federated learning ( Konevcnỳ et al. , 2016 ; Kairouz et al. , 2019 ) , edge computing ( Shi et al. , 2016 ) and so on , distributed learning has recently become a hot research topic ( Zinkevich et al. , 2010 ; Yang , 2013 ; Jaggi et al. , 2014 ; Shamir et al. , 2014 ; Zhang & Kwok , 2014 ; Ma et al. , 2015 ; Lee et al. , 2017 ; Lian et al. , 2017 ; Zhao et al. , 2017 ; Sun et al. , 2018 ; Wangni et al. , 2018 ; Zhao et al. , 2018 ; Zhou et al. , 2018 ; Yu et al. , 2019a ; b ; Haddadpour et al. , 2019 ) . Most traditional distributed learning methods are based on stochastic gradient descent ( SGD ) and its variants ( Bottou , 2010 ; Xiao , 2010 ; Duchi et al. , 2011 ; Johnson & Zhang , 2013 ; Shalev-Shwartz & Zhang , 2013 ; Zhang et al. , 2013 ; Lin et al. , 2014 ; Schmidt et al. , 2017 ; Zheng et al. , 2017 ; Zhao et al. , 2018 ) , and typically assume no failure or attack on workers . However , in real distributed learning applications with multiple networked machines ( nodes ) , different kinds of hardware or software failure may happen . Representative failure include bit-flipping in the communication media and the memory of some workers ( Xie et al. , 2019 ) . In this case , a small failure on some machines ( workers ) might cause a distributed learning method to fail . In addition , malicious attack should not be neglected in an open network where the manager ( or server ) generally has not much control on the workers , such as the cases of edge computing and federated learning . Some malicious workers may behave arbitrarily or even adversarially . Hence , Byzantine learning ( BL ) , which refers to distributed learning with failure or attack , has recently attracted much attention ( Diakonikolas et al. , 2017 ; Chen et al. , 2017 ; Blanchard et al. , 2017 ; Alistarh et al. , 2018 ; Damaskinos et al. , 2018 ; Xie et al. , 2019 ; Baruch et al. , 2019 ; Diakonikolas & Kane , 2019 ) . Existing BL methods can be divided into two main categories : synchronous BL ( SBL ) methods and asynchronous BL ( ABL ) methods . In SBL methods , the learning information , such as the gradient in SGD , of all workers will be aggregated in a synchronous way . On the contrary , in ABL methods the learning information of workers will be aggregated in an asynchronous way . Existing SBL methods mainly take two different ways to achieve resilience against Byzantine workers which refer to those workers with failure or attack . One way is to replace the simple averaging aggregation operation with some more robust aggregation operations , such as median and trimmed-mean ( Yin et al. , 2018 ) . Krum ( Blanchard et al. , 2017 ) and ByzantinePGD ( Yin et al. , 2019 ) take this way . The other way is to filter the suspicious learning information ( gradients ) before averaging . Representative examples include ByzantineSGD ( Alistarh et al. , 2018 ) and Zeno ( Xie et al. , 2019 ) . The advantage of SBL methods is that they are relatively simple and easy to be implemented . But SBL methods will result in slow convergence when there exist heterogeneous workers . Furthermore , in some applications like federated learning and edge computing , synchronization can not even be performed most of the time due to the offline workers ( clients or edge servers ) . Hence , ABL is preferred in these cases . To the best of our knowledge , there exist only two ABL methods : Kardam ( Damaskinos et al. , 2018 ) and Zeno++ ( Xie et al. , 2020 ) . Kardam introduces two filters to drop out suspicious learning information ( gradients ) , which can still achieve good performance when the communication delay is heavy . However , when in face of malicious attack , some work finds that Kardam also drops out most correct gradients in order to filter all faulty ( failure ) gradients . Hence , Kardam can not resist malicious attack ( Xie et al. , 2020 ) . Zeno++ scores each received gradient , and determines whether to accept it according to the score . But Zeno++ needs to store some training instances on server for scoring . In practical applications , storing data on server will increase the risk of privacy leakage or even face legal risk . Therefore , under the general setting where server has no access to any training instances , there have not existed ABL methods to resist malicious attack . In this paper , we propose a novel method , called buffered asynchronous stochastic gradient descent ( BASGD ) , for ABL . The main contributions of BASGD are listed as follows : • To the best of our knowledge , BASGD is the first ABL method that can resist malicious attack without storing any instances on server . Compared with those methods which need to store instances on server , BASGD takes less risk of privacy leakage . • BASGD is theoretically proved to be convergent , and be able to resist failure or attack . • Empirical results show that BASGD significantly outperforms vanilla ASGD and other ABL baselines when there exist failure or malicious attack on workers . In particular , BASGD can still converge under malicious attack , when ASGD and other ABL methods fail . 2 PRELIMINARY . This section presents the preliminary of this paper , including the distributed learning framework used in this paper and the definition of Byzantine worker . 2.1 DISTRIBUTED LEARNING FRAMEWORK . Many machine learning models , such as logistic regression and deep neural networks , can be formulated as the following finite sum optimization problem : min w∈Rd F ( w ) = 1 n n∑ i=1 f ( w ; zi ) , ( 1 ) where w is the parameter to learn , d is the dimension of parameter , n is the number of training instances , f ( w ; zi ) is the empirical loss on the training instance zi . The goal of distributed learning is to solve the problem in ( 1 ) by designing learning algorithms based on multiple networked machines . Although there have appeared many distributed learning frameworks , in this paper we focus on the widely used Parameter Server ( PS ) framework ( Li et al. , 2014 ) . In a PS framework , there are several workers and one or more servers . Each worker can only communicate with server ( s ) . There may exist more than one server in a PS framework , but for the problem of this paper servers can be logically conceived as a unity . Without loss of generality , we will assume there is only one server in this paper . Training instances are disjointedly distributed across m workers . Let Dk denote the index set of training instances on worker k , we have ∪mk=1Dk = { 1 , 2 , . . . , n } and Dk ∩ Dk′ = ∅ if k 6= k′ . In this paper , we assume that server has no access to any training instances . If two instances have the same value , they are still deemed as two distinct instances . Namely , zi may equal zi′ ( i 6= i′ ) . One popular asynchronous method to solve the problem in ( 1 ) under the PS framework is ASGD ( Dean et al. , 2012 ) ( see Algorithm 1 in Appendix A ) . In this paper , we assume each worker samples one instance for gradient computation each time , and do not separately discuss the mini-batch case . In PS based ASGD , server is responsible for updating and maintaining the latest parameter . The number of iterations that server has already executed is used as the global logical clock of server . At the beginning , iteration number t = 0 . Each time a SGD step is executed , t will increase by 1 immediately . The parameter after t iterations is denoted as wt . If server sends parameters to worker k at iteration t′ , some SGD steps may have been excuted before server receives gradient from worker k next time at iteration t. Thus , we define the delay of worker k at iteration t as τ tk = t− t′ . Worker k is heavily delayed at iteration t if τ tk > τmax , where τmax is a pre-defined non-negative constant . 2.2 BYZANTINE WORKER . For workers that have sent gradients ( one or more ) to server at iteration t , we call worker k loyal worker if it has finished all the tasks without any fault and each sent gradient is correctly received by the server . Otherwise , worker k is called Byzantine worker . If worker k is a Byzantine worker , it means the received gradient from worker k is not credible , which can be an arbitrary value . In ASGD , there is one received gradient at a time . Formally , we denote the gradient received from worker k at iteration t as gtk . Then , we have : gtk = { ∇f ( wt ′ ; zi ) , if worker k is loyal at iteration t ; arbitrary value , if worker k is Byzantine at iteration t , where 0 ≤ t′ ≤ t , and i is randomly sampled from Dk . Our definition of Byzantine worker is consistent with most previous works ( Blanchard et al. , 2017 ; Xie et al. , 2019 ; 2020 ) . Either accidental failure or malicious attack will result in Byzantine workers . 3 BUFFERED ASYNCHRONOUS SGD . In synchronous BL , gradients from all workers are received at each iteration . During this process , we can compare the gradients with each other , and then filter suspicious ones , or use more robust aggregation rules such as median and trimmed-mean for updating . However , in asynchronous BL , only one gradient is received by the server at a time . Without any training instances stored on server , it is difficult for server to identify whether a received gradient is credible or not . In order to deal with this problem in asynchronous BL , we propose a novel method called buffered asynchronous SGD ( BASGD ) . BASGD introduces B buffers ( 0 < B ≤ m ) on server , and the gradient used for updating parameters will be aggregated from these buffers . The detail of the learning procedure of BASGD is presented in Algorithm 2 in Appendix A . In this section , we will introduce the details of the two key components of BASGD : buffer and aggregation function . 3.1 BUFFER . In BASGD , the m workers do the same job as that in ASGD , while the updating rule on server is modified . More specifically , there are B buffers ( 0 < B ≤ m ) on server . When a gradient g from worker s is received , it will be temporarily stored in buffer b , where b = s mod B , as illustrated in Figure 1 . Only when each buffer has stored at least one gradient , a new SGD step will be executed . Please note that no matter whether a SGD step is executed or not , the server will immediately send the latest parameters back to the worker after receiving a gradient . Hence , BASGD introduces no barrier , and is an asynchronous algorithm . For each buffer b , more than one gradient may have been received at iteration t. We will store the average of these gradients ( denoted by hb ) in buffer b . Assume that there are already ( N − 1 ) gradients g1 , g2 , . . . , gN−1 which should be stored in buffer b , and hb ( old ) = 1N−1 ∑N−1 i=1 gi . When the N -th gradient gN is received , the new average value in buffer b should be : hb ( new ) = 1 N ∑N i=1 gi = N−1 N · hb ( old ) + 1 N · gN . This is the updating rule for each buffer b when a gradient is received . We use N tb to denote the total number of gradients stored in buffer b at the t-th iteration . After the parameter w is updated , all buffers will be zeroed out at once . With the benefit of buffers , server has access to B candidate gradients when updating parameter . Thus , a more reliable ( robust ) gradient can be aggregated from the B gradients of buffers , if a proper aggregation function Aggr ( · ) is chosen .

Summary: Review: This paper proposes BASGD which uses buffers to perform asynchronous Byzantine learning. In each SGD step, all workers compute gradients and send them to the server where their ad buffer is updated. When all of the buffers are updated, the server performs an model update. When a worker send a gradient to the server, it also pulls the latest model and compute the gradient on it no matter the server update the model or not. The main contribution in this paper is to introduce a new approach to do asynchronous Byzantine learning without storing training samples on the server like zeno++.