BAFFLE: TOWARDS RESOLVING FEDERATED LEARNING’S DILEMMA - THWARTING BACKDOOR  AND INFERENCE ATTACKS

1 INTRODUCTION . Federated learning ( FL ) is an emerging collaborative machine learning trend with many applications such as next word prediction for mobile keyboards ( McMahan & Ramage , 2017 ) , medical imaging ( Sheller et al. , 2018a ) , and intrusion detection for IoT ( Nguyen et al. , 2019 ) . In FL , clients locally train model updates using private data and provide these to a central aggregator who combines them to a global model that is sent back to clients for the next training iteration . FL offers efficiency and scalability as the training is distributed among many clients and executed in parallel ( Bonawitz et al. , 2019 ) . In particular , FL improves privacy by enabling clients to keep their training data locally ( McMahan et al. , 2017 ) . This is not only relevant for compliance to legal obligations such as the GDPR ( 2018 ) , but also in general when processing personal and sensitive data . Despite its benefits , FL is vulnerable to backdoor ( Bagdasaryan et al. , 2020 ; Nguyen et al. , 2020 ; Xie et al. , 2020 ) and inference attacks ( Pyrgelis et al. , 2018 ; Shokri et al. , 2017 ; Ganju et al. , 2018 ) . In the former , the adversary stealthily manipulates the global model so that attacker-chosen inputs result in wrong predictions chosen by the adversary . Existing backdoor defenses , e.g. , ( Shen et al. , 2016 ; Blanchard et al. , 2017 ) fail to effectively protect against state-of-the-art backdoor attacks , e.g. , constrain-and-scale ( Bagdasaryan et al. , 2020 ) and DBA ( Xie et al. , 2020 ) . In inference attacks , the adversary aims at learning information about the clients ’ local data by analyzing their model updates . Mitigating both attack types at the same time is highly challenging due to a dilemma : Backdoor defenses require access to the clients ’ model updates , whereas inference mitigation strategies prohibit this to avoid information leakage . No solution currently exists that defends against both attacks at the same time ( §6 ) . Our Goals and Contributions . In this paper , we provide the following contributions : 1 . BAFFLE , a novel generic FL defense system that simultaneously protects both the security and the data privacy of FL by effectively preventing backdoor and inference attacks . To the best of our knowledge , this is the first work that discusses and tackles this dilemma , i.e. , no existing defense against backdoor attacks preserves the privacy of the clients ’ data ( §4 ) . 2 . To the best of our knowledge , we are the first to point out that combining clustering , clipping , and noising can prevent the adversary to trade-off between attack impact and attack stealthiness . However , the naı̈ve combination of these two classes of defenses is not effective to defend against sophisticated backdoor attacks . Therefore , we introduce a novel backdoor defense ( cf . Alg . 1 ) that has three-folds of novelty : ( 1 ) a novel two-layer defense , ( 2 ) a new dynamic clustering approach ( §3.1 ) , and ( 3 ) a new adaptive threshold tuning scheme for clipping and noising ( §3.2 ) . The clustering component filters out malicious model updates with high attack impact while adaptive smoothing , clipping , and noising eliminate potentially remaining malicious model contributions . Moreover , BAFFLE is able to mitigate more complex attack scenarios like the simultaneous injection of different backdoors by several adversaries that can not be handled in existing defenses ( §3 ) . 3 . We design tailored efficient secure ( two-party ) computation protocols for BAFFLE resulting in private BAFFLE , the first privacy-preserving backdoor defense that also inhibits inference attacks ( §4 ) . To the best of our knowledge , no existing defense against backdoor attacks preserves the privacy of the clients ’ data ( §6 ) . 4 . We demonstrate BAFFLE ’ s effectiveness against backdoor attacks through an extensive evaluation on various datasets and applications ( §5 ) . Beyond mitigating state-of-the-art backdoor attacks , we also show that BAFFLE succeeds to thwart adaptive attacks that optimize the attack strategy to circumvent BAFFLE ( §5.1 ) . 5 . We evaluate the overhead of applying secure two-party computation to demonstrate the efficiency of private BAFFLE . A training iteration of private BAFFLE for a neural network with 2.7 million parameters and 50 clients on CIFAR-10 takes less than 13 minutes ( §5.3 ) . 2 BACKGROUND AND PROBLEM SETTING . Federated learning ( FL ) is a concept for distributed machine learning where K clients and an aggregator A collaboratively build a global model G ( McMahan et al. , 2017 ) . In training round t ∈ [ 1 , T ] , each client i ∈ [ 1 , K ] locally trains a local modelWi ( with p parameters/weightsw1i , . . . , w p i ) based on the previous global model Gt−1 using its local data Di and sends Wi to A . Then , A aggregates the received models Wi into the new global model Gt by averaging the local models ( weighted by the number of training samples used to train it ) : Gt = ΣKi=1 ni×Wi n , where ni = ‖Di‖ , n = ΣKi=1ni ( cf . Alg . 2 and Alg . 3 in §A for details ) . In practice , previous works employ equal weights ( ni = n/K ) for the contributions of all clients ( Bagdasaryan et al. , 2020 ; Xie et al. , 2020 ) . We adopt this approach , i.e. , we set Gt = ΣKi=1 Wi K . Adversary model : In typical FL settings , there are two adversaries : malicious clients that try to inject backdoors into the global model and honest-but-curious ( a.k.a . semi-honest ) aggregators that correctly compute and follow the training protocols , but aim at ( passively ) gaining information about the training data of the clients through inference attacks ( Bonawitz et al. , 2017 ) . The former type of adversary Ac has full control over K ′ ( K ′ < K2 ) clients and their training data , processes , and parameters ( Bagdasaryan et al. , 2020 ) . Ac also has full knowledge of the aggregator ’ s operations , including potentially applied backdooring defenses and can arbitrarily adapt its attack strategy at any time during the training like simultaneously injecting none , one , or several backdoors . However , Ac has no control over any processes executed at the aggregator nor over the honest clients . The second adversary type , the honest-but-curious aggregatorAs , has access to all local model updates Wi , and can thus perform model inference attacks on each local model Wi to extract information about the corresponding participant ’ s data Di used for training Wi . Backdoor attacks . The goals of Ac are two-fold : ( 1 ) Impact : Ac aims at manipulating the global model Gt such that the modified model G′t provides incorrect predictions G′t ( x ) = c ′ 6= Gt ( x ) , ∀x ∈ IAc , where IAc is a trigger set specific adversary-chosen inputs . ( 2 ) Stealthiness : In addition , Ac seeks to make poisoned models and benign models indistinguishable to avoid detection . Model G′t should therefore perform normally on all other inputs that are not in the trigger set , i.e. , G′t ( x ) = Gt ( x ) , ∀x 6∈ IAc , and the dissimilarity ( e.g. , Euclidean distance ) between a poisoned model W ′ and a benign model W must be smaller than a threshold ε : ‖W ′ −W‖ < ε . Inference Attacks . The honest-but-curious aggregator As attempts to infer sensitive information about clients ’ data Di from their model updates Wi ( Pyrgelis et al. , 2018 ; Shokri et al. , 2017 ; Ganju et al. , 2018 ; Carlini et al. , 2019 ; Melis et al. , 2019 ) by maximising the information φi = Infer ( Wi ) that As gains about the data Di of client i by inferring from its corresponding model Wi . 3 BACKDOOR-RESILIENT FEDERATED LEARNING . We introduce BAFFLE , a novel defense against backdoor attacks preventing adversary Ac from achieving attack stealthiness and impact ( cf . §2 ) . Ac can control the attack impact by , e.g. , adjusting the poisoned data rate PDR , i.e. , the fraction of poisoned data DAc in the training data D ( Eq . 3 ) , or , by tuning the loss-control parameter α that controls the trade-off between backdoor task learning and similarity with the global model ( Eq . 4 ) , see §D for details . On one hand , by increasing attack impact , poisoned models become more dissimilar to benign ones , i.e. , easier to be detected . One the other hand , if poisoned updates are not well trained on the backdoor to remain undetected , the backdoor can be eliminated more easily . BAFFLE exploits this conflict to realize a multilayer backdoor defense shown in Fig . 1 and Alg . 1 . The first layer , called Model Filtering ( §3.1 ) , uses dynamic clustering to identify and remove potentially poisoned model updates having high attack impact . The second layer , called Poison Elimination ( §3.2 ) , leverages an adaptive threshold tuning scheme to clip model weights in combination with appropriate noising to smooth out and remove the backdoor impact of potentially surviving poisoned model updates . 3.1 FILTERING POISONED MODELS . The Model Filtering layer utilizes a new dynamic clustering approach aiming at excluding models with high attack impact . It overcomes several limitations of existing defenses as ( 1 ) it can handle dynamic attack scenarios such as simultaneous injection of multiple backdoors , and ( 2 ) it minimizes false positives . Existing defenses ( Blanchard et al. , 2017 ; Shen et al. , 2016 ) cluster updates into two groups where the smaller group is always considered potentially malicious and removed , leading to false positives and reduced accuracy when no attack is taking place . More importantly , Ac may also split compromised clients into several groups injecting different backdoors . A fixed number of clusters bares the risk that poisoned and benign models end up in the same cluster , in particular , if models with different backdoors differ significantly . This is shown in Fig . 2 depicting different clusterings of model updates1 . Fig . 2a shows the ground truth where Ac uses two groups of clients : 20 clients inject a backdoor and five provide random models to fool the deployed clustering-based defense . Fig . 2b shows how K-means ( as used by Shen et al . ( 2016 ) ) fails to separate benign and poisoned models so that all poisoned ones end up in the same cluster with the benign models . 1The models were trained for an FL-based Network Intrusion Detection System ( NIDS ) , cf . §E . Algorithm 1 BAFFLE 1 : Input : K , G0 , T . K is the number of clients , G0 is the initial global model , T is the number of training iterations 2 : Output : GT . GT is the updated global model after T iterations 3 : for each training iteration t in [ 1 , T ] do 4 : for each client i in [ 1 , K ] do 5 : Wi ← CLIENTUPDATE ( Gt−1 ) . The aggregator sendsGt−1 to Client i who trainsGt−1 using its dataDi locally to achieve local modalWi and sendsWi back to the aggregator . 6 : ( c11 , . . . , cKK ) ← COSINEDISTANCE ( W1 , . . . , WK ) . ∀i , j ∈ ( 1 , . . . , K ) , cij is the Cosine distance betweenWi andWj 7 : ( b1 , . . . , bL ) ← CLUSTERING ( c11 , . . . , cKK ) . L is the number of admitted models , bl are the indices of the admitted models 8 : ( e1 , . . . , eK ) ← EUCLIDEANDISTISTANCES ( Gt−1 , ( W1 , . . . , WK ) ) . ei is the Euclidean distance betweenGt−1 andWi 9 : St ← MEDIAN ( e1 , . . . , eK ) . St is the adaptive clipping bound at round t 10 : for each client l in [ 1 , L ] do 11 : W∗bl ← Wbl ∗ MIN ( 1 , St/ebl ) . W ∗ bl is the admitted model after clipped by the adaptive clipping bound St 12 : G∗t ← ∑L l=1W ∗ bl /L . Aggregating , G∗t is the plain global model before adding noise 13 : σ ← λ ∗ St . Adaptive noising level 14 : Gt ← G∗t +N ( 0 , σ ) . Adaptive noising Dynamic Clustering . We overcome both challenges by calculating the pairwise Cosine distances measuring the angular differences between all model updates and applying the HDBSCAN clustering algorithm ( Campello et al. , 2013 ) . The Cosine distance is not affected by attacks that scale updates to boost their impact as this does not change the angle between the updates . While Ac can easily manipulate the L2-norms of updates , reducing the Cosine distances decreases the attack impact ( Fung et al. , 2018 ) . HDBSCAN clusters the models based on their density and dynamically determines the required number of clusters . This can also be a single cluster , preventing false positives in the absence of attacks . Additionally , HDBSCAN labels models as noise if they do not fit into any cluster . This allows BAFFLE to efficiently handle multiple poisoned models with different backdoors by labeling them as noise to be excluded . We select the minimum cluster size to be at least 50 % of the clients , i.e. , K2 + 1 , s.t . it contains the majority of the updates ( which we assume to be benign , cf . §2 ) . All remaining ( potentially poisoned ) models are marked as outliers . This behavior is depicted in Fig . 2d where the two benign clusters C and D from Fig . 2c are merged into one cluster while both malicious and random contributions are labeled as outliers . Hence , to the best of our knowledge , our clustering is the first FL backdoor defense for dynamic attacks where the number of injected backdoors varies . The clustering step is shown in Lines 6-7 of Alg . 1 where L models ( Wb1 , . . . , WbL ) are accepted .

Summary: In the paper, the authors proposed a novel privacy-preserving defense approach BAFFLE for federated learning which could simultaneously impede backdoor and inference attacks. To impede backdoor attacks, the Model Filtering layer (i.e., by dynamic clustering) and Poison Elimination layer (i.e., by noising and clipping) were presented respectively for the malicious updates and the weak manipulations of the model. To thwart inference attacks, private BAFFLE was built to evaluate the BAFFLE algorithm under encryption using secure computation techniques.