DBA: Distributed Backdoor Attacks against Federated Learning

1 INTRODUCTION . Federated learning ( FL ) has been recently proposed to address the problems for training machine learning models without direct access to diverse training data , especially for privacy-sensitive tasks ( Smith et al. , 2017 ; McMahan et al. , 2017 ; Zhao et al. , 2018 ) . Utilizing local training data of participants ( i.e. , parties ) , FL helps train a shared global model with improved performance . There have been prominent applications and ever-growing trends in deploying FL in practice , such as loan status prediction , health situation assessment ( e.g . potential cancer risk assessment ) , and next-word prediction while typing ( Hard et al. , 2018 ; Yang et al. , 2018 ; 2019 ) . Although FL is capable of aggregating dispersed ( and often restricted ) information provided by different parties to train a better model , its distributed learning methodology as well as inherently heterogeneous ( i.e. , non-i.i.d . ) data distribution across different parties may unintentionally provide a venue to new attacks . In particular , the fact of limiting access to individual party ’ s data due to privacy concerns or regulation constraints may facilitate backdoor attacks on the shared model trained with FL . Backdoor attack is a type of data poisoning attacks that aim to manipulate a subset of training data such that machine learning models trained on the tampered dataset will be vulnerable to the test set with similar trigger embedded ( Gu et al. , 2019 ) . Backdoor attacks on FL have been recently studied in ( Bagdasaryan et al. , 2018 ; Bhagoji et al. , 2019 ) . However , current attacks do not fully exploit the distributed learning methodology of FL , as they embed the same global trigger pattern to all adversarial parties . We call such attacking scheme centralized backdoor attack . Leveraging the power of FL in aggregating dispersed information from local parties to train a shared model , in this paper we propose distributed backdoor attack ( DBA ) against FL . Given the same global trigger pattern as the centralized attack , DBA decomposes it into local patterns and embed them to different adversarial parties respectively . A schematic comparison between the centralized and distributed backdoor attacks is illustrated in Fig.1 . Through extensive experiments on several financial and image datasets and in-depth analysis , we summarize our main contributions and findings as follows . •We propose a novel distributed backdoor attack strategy DBA on FL and show that DBA is more persistent and effective than centralized backdoor attack . Based on extensive experiments , we report a prominent phenomenon that although each adversarial party is only implanted with a local trigger pattern via DBA , their assembled pattern ( i.e. , global trigger ) attains significantly better attack performance on the global model compared with the centralized attack . The results are consistent across datasets and under different attacking scenarios such as one-time ( single-shot ) and continuous ( multiple-shot ) poisoning settings . To the best of our knowledge , this paper is the first work studying distributed backdoor attacks . • When evaluating the robustness of two recent robust FL methods against centralized backdoor attack ( Fung et al. , 2018 ; Pillutla et al. , 2019 ) , we find that DBA is more effective and stealthy , as its local trigger pattern is more insidious and hence easier to bypass the robust aggregation rules . •We provide in-depth explanations for the effectiveness of DBA from different perspectives , including feature visual interpretation and feature importance ranking . •We perform comprehensive analysis and ablation studies on several trigger factors in DBA , including the size , gap , and location of local triggers , scaling effect in FL , poisoning interval , data poisoning ratio , and data distribution . 2 DISTRIBUTED BACKDOOR ATTACK AGAINST FEDERATED LEARNING . 2.1 GENERAL FRAMEWORK . The training objective of FL can be cast as a finite-sum optimization : minw∈Rd [ F ( w ) : = 1 N ∑N i=1 fi ( w ) ] . There are N parties individually processing N local models , each of whom trains with the local objective fi : Rd 7→ R based on a private datasetDi = { { xij , yij } ai j=1 } , where ai = |Di| and { xij , yij } represents each data sample and its corresponding label . In supervised FL setting , each local function fi is computed as fi ( wi ) = l ( { xij , yij } j∈Di , wi ) where l stands for a loss of prediction using the local parameters wi . The goal of FL is to obtain a global model which can generalize well on test data Dtest after aggregating over the distributed training results from N parties . Specifically , at round t , the central server sends the current shared model Gt to n ∈ [ N ] selected parties , where [ N ] denotes the integer set { 1 , 2 , . . . , N } . The selected party i locally computes the function fi by running an optimization algorithm such as stochastic gradient descent ( SGD ) for E local epochs with its own dataset Di and learning rate lr to obtain a new local model Lt+1i . The local party then sends model update Lt+1i −Gt back to the central server , who will averages over all updates with its own learning rate η to generate a new global model Gt+1 : Gt+1 = Gt + η n n∑ i=1 ( Lt+1i −G t ) ( 1 ) This aggregation process will be iterated until FL finds the final global model . Unless specified otherwise , we use Gt ( Lti ) to denote the model parameters of the global ( local ) model at round t. Attacker ability . Based on the Kerckhoffs ’ s theory ( Shannon , 1949 ) , we consider the strong attacker here who has full control of their local training process , such as backdoor data injection and updating local training hyperparameters including E and lr . This scenario is quite practical since each local dataset is usually owned by one of the local parties . However , attackers do not have the ability to influence the privilege of central server such as changing aggregation rules , nor tampering the training process and model updates of other parties . Objective of backdoor attack . Backdoor attack is designed to mislead the trained model to predict a target label τ on any input data that has an attacker-chosen pattern ( i.e. , a trigger ) embedded . Instead of preventing the convergence in accuracy as Byzantine attacks ( Blanchard et al. , 2017 ) , the purpose of backdoor attacks in FL is to manipulate local models and simultaneously fit the main task and backdoor task , so that the global model would behave normally on untampered data samples while achieving high attack success rate on backdoored data samples . The adversarial objective1 for attacker i in round t with local datatset Di and target label τ is : w∗i = argmax wi ( ∑ j∈Sipoi P [ Gt+1 ( R ( xij , φ ) ) = τ ] + ∑ j∈Si cln P [ Gt+1 ( xij ) = y i j ] ) . ( 2 ) Here , the poisoned dataset Sipoi and clean dataset S i cln satisfy S i poi ∩Sicln = ∅ and Sipoi ∪Sicln = Di . The function R transforms clean data in any class into backdoored data that have an attacker-chosen trigger pattern using a set of parameters φ . For example , for image data , φ is factored into trigger location TL , trigger size TS and trigger gap TG ( φ = { TS , TG , TL } ) , which are shown in Fig.2 . The attacker can design his own trigger pattern and choose an optimal poison ratio r to result in a better model parameter w∗i , with which G t+1 can both assign the highest probability to target label τ for backdoored data R ( xij , φ ) and the ground truth label y i j′ for benign data x i j′ . 2.2 DISTRIBUTED BACKDOOR ATTACK ( DBA ) . We again use Fig.1 to illustrate our proposed DBA in details . Recall that current centralized attack embeds the same global trigger for all local attackers2 ( Bagdasaryan et al. , 2018 ) . For example , the attacker in Fig.1 . ( a ) embeds the training data with the selected patterns highlighted by 4 colors , which altogether constitutes a complete global pattern as the backdoor trigger . In our DBA , as illustrated in Fig.1 . ( b ) , all attackers only use parts of the global trigger to poison their local models , while the ultimate adversarial goal is still the same as centralized attack — using the global trigger to attack the shared model . For example , the attacker with the orange sign poisons a subset of his training data only using the trigger pattern located at the orange area . Similar attacking methodology applies to green , yellow and blue signs . We define each DBA attacker ’ s trigger as the local trigger and the combined whole trigger as the global trigger . For fair comparison , we keep similar amount of total injected triggers ( e.g. , modified pixels ) for both centralized attack and DBA . In centralized attack , the attacker tries to solve the optimization problem in Eq.2 without any coordination and distributed processing . In contrast , DBA fully exploits the distributed learning and local data opacity in FL . Considering M attackers in DBA with M small local triggers . Each DBA attacker mi independently performs the backdoor attack on their local models . This novel mechanism breaks a centralized attack formulation into M distributed sub-attack problems aiming to solve3 w∗i = argmax wi ( ∑ j∈Sipoi P [ Gt+1 ( R ( xij , φ ∗ i ) ) = τ ; γ ; I ] + ∑ j∈Sicln P [ Gt+1 ( xij ) = y i j ] ) , ∀ i ∈ [ M ] ( 3 ) 1In our implementation , we use cross entropy as training objective . 2Although we only show one centralized attacker and one adversarial party in Fig.1 , in practice centralized attack can poison multiple parties with the same global trigger , as discussed in ( Bagdasaryan et al. , 2018 ) . 3In our implementation , we use cross entropy as training objective . Size ( b ) Trigger Gap ( c ) Trigger Location Gapy Gapx Shiftx Shifty ( a ) Trigger Size Figure 2 : Trigger factors ( size , gap and location ) in backdoored images . where φ∗i = { φ , O ( i ) } is the geometric decomposing strategy for the local trigger pattern of attacker mi and O ( i ) entails the trigger decomposition rule for mi based on the global trigger φ. DBA attackers will poison with the poison round interval I and use the scale factor γ to manipulate their updates before submitting to the aggregator . We will explain the related trigger factors in the next subsection . We note that although none of the adversarial party has ever been poisoned by the global trigger under DBA , we find that DBA indeed outperforms centralized attack significantly when evaluated with the global trigger . 2.3 FACTORS IN DISTRIBUTED BACKDOOR ATTACK . With the framework of DBA on FL , there are multiple new factors to be explored . Here we introduce a set of trigger factors that we find to be critical . Fig.2 explains the location , size and gap attribute of triggers in image dataset . For simplicity , we set all of our local triggers to the same rectangle shape4 . Fig.3 explains our trigger attribute of ranked feature importance in tabular data ( e.g. , the loan dataset ) . Trigger Size TS : the number of pixel columns ( i.e. , the width ) of a local distributed trigger . Trigger Gap TG : the distance of the Gapx and Gapy , which represent the distance between the left and right , as well as the top and bottom local trigger , respectively . Trigger Location TL : ( Shiftx , Shifty ) is the offset of the trigger pattern from the top left pixel . Scale γ : the scaling parameter γ = η/N defined in ( Bagdasaryan et al. , 2018 ) is used by the attacker to scale up the malicious model weights.5 For instance , assume the ith malicious local model is X . The new local model Lt+1i that will be submitted is calculated as L t+1 i = γ ( X −Gt ) +Gt . Poison Ratio r : the ratio controls the fraction of backdoored samples added per training batch . Note that larger r should be preferable when attacking intuitively , and there is a tradeoff between clean data accuracy and attack success rate , but too large r would also hurt the attack effectiveness once the model becomes useless . Poison Interval I : the round intervals between two poison steps . For example , I = 0 means all the local triggers are embedded within one round , while I = 1 means the local triggers are embedded in consecutive rounds . Data Distribution : FL often presumes non-i.i.d . data distribution across parties . Here , we use a Dirichlet distribution ( Minka , 2000 ) with different hyperparameter α to generate different data distribution following the setups in ( Bagdasaryan et al. , 2018 ) .

Summary: The authors introduce the idea of distributed backdoor attacks in the FL framework, in which the dishonest participants in FL add local triggers to their training data to influence the global model to classify triggered images in a desired way. They show empirically that the learned models then are more likely to be successfully forced to misclassified images in which all the local triggers are present at test time, than are models learned using centralized backdoor attacks, where all attackers use the same trigger pattern (one of the same size as the concatenation of the local triggers, to be fair in the comparison). They then demonstrate that because the local triggers cause smaller corruptions in the model coefficients, these distributed attacks survive robust FL training algorithms (namely FoolsGold, and a recent robust regression based method) more often than centralized attacks. Similar experiments are conducted on the Loan text dataset, using appropriate analogs of local triggers, with similar results.