An Equivalence Between Data Poisoning and Byzantine Gradient Attacks

To study the resilience of distributed learning , the “ Byzantine ” literature considers a strong threat model where workers can report arbitrary gradients to the parameter server . While this model helped generate several fundamental results , it has however sometimes been considered unrealistic , when the workers are mostly trustworthy machines . In this paper , we show a surprising equivalence between this model and data poisoning , a threat considered much more realistic . More specifically , we prove that any gradient attack can be reduced to data poisoning in a personalized federated learning system that provides PAC guarantees ( which we show are both desirable and realistic in various personalized federated learning contexts such as linear regression and classification ) . Maybe most importantly , we derive a simple and practical attack that may be constructed against classical personalized federated learning models , and we show both theoretically and empirically the effectiveness of this attack . 1 INTRODUCTION . Learning algorithms now typically leverage data generated by a large number of users ( Smith et al. , 2013 ; Wang et al. , 2019a ; b ) to often learn a common model that fits a large population ( Konecný et al. , 2015 ) , but also sometimes to construct a personalized model for each individual ( Ricci et al. , 2011 ) . Autocompletion ( Lehmann & Buschek , 2021 ) , conversational ( Shum et al. , 2018 ) and recommendation ( Ie et al. , 2019 ) algorithms are examples of such algorithms deployed at scale . To be effective , besides huge amounts of ( distributed ) data ( Brown et al. , 2020 ; Fedus et al. , 2021 ) , these algorithms require a high level of customization . This has motivated research into personalized federated learning ( Fallah et al. , 2020 ; Hanzely et al. , 2020 ; Dinh et al. , 2020 ) . However , in applications such as content recommendation , activists , companies , and politicians have strong incentives to promote certain views , products or ideologies ( Hoang , 2020 ; Hoang et al. , 2021 ) . Remember for instance that , on YouTube , two views out of three result from algorithmic recommendations ( Solsman , 2018 ) . Perhaps unsurprisingly , this has led to vast amounts of fabricated activities to bias algorithms ( Bradshaw & Howard , 2019 ; Neudert et al. , 2019 ) , like “ fake reviews ” ( Wu et al. , 2020 ) . The scale of this phenomenon is well illustrated by the case of Facebook which , in 2019 alone , reported the removal of around 6 billion fake accounts from its platform ( Fung & Garcia , 2019 ) . This is particularly concerning in the era of “ stochastic parrots ” ( Bender et al. , 2021 ) : climate denialists are incentivized to pollute textual datasets with claims like “ climate change is a hoax ” , as autocompletion , conversational and recommendation algorithms trained on such data will more likely spread these views ( McGuffie & Newhouse , 2020 ) . This raises serious concerns about the vulnerability of personalized federated learning to such misleading data . Data poisoning attacks clearly constitute now a major machine learning security issue ( Kumar et al. , 2020 ) . Overall , in highly adversarial environments like social media , given the advent of deep fakes ( Johnson & Diakopoulos , 2021 ) , we should expect that most data are strategically crafted and labeled . In this context , the authentication of the data provider seems critical . In particular , the safety of learning algorithms arguably demands that they be trained solely on cryptographically signed data , that is , data that provably come from a known source . But even signed data can not be wholeheartedly trusted since users usually have preferences over what ought to be recommended to others . They thus have incentives to behave strategically in order to promote certain views or products . To address data poisoning , the Byzantine learning literature usually considers that each federated learning worker may behave arbitrarily ( Blanchard et al. , 2017 ; Mhamdi et al. , 2018 ; El-Mhamdi et al. , 2021 ) . Recall that at each iteration of the federated learning stochastic gradient descent , each worker is given the updated model , and is asked to compute the gradient of the loss function with respect to ( a batch of ) its local data . Byzantine learning usually assumes that a malicious ( Byzantine ) worker may report any gradient ; without having to justify whether such a gradient could have been generated through data poisoning . In fact , the gradient attack threat model has sometimes been claimed to be unrealistic in practical federated learning ( Shejwalkar et al. , 2021 ) , especially when the workers are machines owned by trusted entities ( cross-silo FL ( Kairouz et al. , 2021 ) ) . We prove in this paper an equivalence between gradients attacks and fabricated data injection , in a general and desirable collaborative learning framework . Thereby , our paper provides the first practically compelling argument for the necessity to protect federated learning against gradient attacks . Contributions . As a preamble of our main result , we formalize local PAC * learning1 ( Valiant , 1984 ) for personalized learning , and prove that a simple and general solution to personalized federated linear regression and classification is indeed locally PAC * learning . Our proof leverages a new concept called gradient-PAC * learning , which is of independent interest . We prove that it is sufficient to guarantee local PAC * learning , and that it is verified by basic learning algorithms , like linear and logistic regression . This is an important and highly nontrivial contribution of this paper . Our main contribution is then to prove that local PAC * learning in personalized federated learning essentially implies an equivalence between data poisoning and gradient attacks . More precisely , we show how any ( converging ) gradient attack can be turned into a data poisoning attack , with the same resulting harm . Given how easy it generally is to create fake accounts on web platforms and to inject data poisoning by generating fake activities , this result should arguably greatly increase the concerns over the vulnerabilities of federated learning with user-generated data . Finally , we propose a simple but very general strategic gradient attack , called the counter-gradient attack ( CGA ) , which any participant to federated learning can deploy to bias the global model towards any target model that better suits their interest . We prove the effectiveness of this attack under fairly general assumptions , which apply to many proposed personalized learning frameworks including Hanzely et al . ( 2020 ) ; Dinh et al . ( 2020 ) . We then show empirically how this gradient attack can be turned into a devastating data poisoning attack , with remarkably few data . Related work . Byzantine learning has provided both negative and positive results in Byzantine resilience ( Blanchard et al. , 2017 ; Mhamdi et al. , 2018 ; Baruch et al. , 2019 ; Xie et al. , 2019 ) , some of which apply almost straightforwardly to personalized federated learning ( El-Mhamdi et al. , 2020 ; 2021 ) . Such results study the resilience against a minority of adversarial users . Our paper however focuses on a different kind of malicious users . Namely , like Suya et al . ( 2021 ) , we study the resilience against strategic users , who aim to bias the learned models towards a specific target model . The study of the resilience against strategic users is part of the research on strategyproof learning . Many special cases of strategyproofness have been tackled , including regression ( Chen et al. , 2018b ; Dekel et al. , 2010 ; Perote & Perote-Peña , 2004 ; Ben-Porat & Tennenholtz , 2017 ) , classification ( Meir et al. , 2012 ; Chen et al. , 2020 ; Meir et al. , 2011 ; Hardt et al. , 2016 ) , statistical estimation ( Cai et al. , 2015 ) , and clustering ( Perote & Sevilla , 2003 ) . However , none of these papers tackles a general personalized federated learning scheme . Typically , for linear regression , Chen et al . ( 2018b ) and Perote & Perote-Peña ( 2004 ) assume that each user can only provide a single data point . This greatly restricts the users ’ ability to contribute to the learning model . And while Dekel et al . ( 2010 ) allows multiple contributions , they either require payments , which might not be possible ( e.g. , due to ethical reasons ) , or they restrict the model to one dimension or a constant function in Rd . Conversely , Suya et al . ( 2021 ) show how to arbitrarily manipulate convex learning models through multiple data injections , in the case where a single model is learned from all data at once . A large literature has focused on data poisoning , with either a focus on backdoor ( Dai et al. , 2019 ; Zhao et al. , 2020 ; Severi et al. , 2021 ; Truong et al. , 2020 ; Schwarzschild et al. , 2021 ) or triggerless attacks ( Biggio et al. , 2012 ; Muñoz-González et al. , 2017 ; Shafahi et al. , 2018 ; Zhu et al. , 2019 ; 1We omit complexity considerations for the sake of generality . We define PAC * to be PAC without such considerations . Huang et al. , 2020 ; Barreno et al. , 2006 ; Aghakhani et al. , 2021 ; Geiping et al. , 2021 ) . However , most of this research analyzed data poisoning without signed data . One noteworthy exception is Mahloujifar et al . ( 2019 ) , whose universal attack amplifies the probability of a ( bad ) property . Collaborative PAC learning was previously introduced by Blum et al . ( 2017 ) , and then extensively studied ( Chen et al. , 2018a ; Nguyen & Zakynthinou , 2018 ) , sometimes with the presence of Byzantine collaborating users ( Qiao , 2018 ; Jain & Orlitsky , 2020 ; Konstantinov et al. , 2020 ) . We stress however that this line of work assumes that all honest users have the same labeling function . In other words , given any query , they agree on how the query should be answered . This is a very unrealistic assumption in many critical applications , like content moderation or language processing . In fact , in such applications , removing outliers can be argued to amount to ignoring minorities ’ views , which would be highly unethical . The very definition of PAC learning must then be adapted , which is what we do in this paper ( also , we adapt it to parameterized models ) . Structure of the paper . The rest of the paper is organized as follows . Section 2 presents a general model of personalized learning , formalizes local PAC * learning and defines a general federated gradient descent algorithm . Section 3 proves the equivalence between data poisoning and gradient attacks , under local PAC * learning . Section 4 proves the local PAC * learning properties for federated linear regression and classification . Section 5 then describes a simple and general data poisoning attack , whose effectiveness against ` 22 is proved theoretically and empirically . Section 6 concludes . 2 A GENERAL PERSONALIZED FEDERATED LEARNING FRAMEWORK . We consider a set [ N ] = { 1 , . . . , N } of users . Each user n ∈ [ N ] has a local signed dataset Dn , and aims to learn a local model θn ∈ Rd . Users may collaborate to improve their models . Personalized learning must then input a tuple of users ’ local datasets ~D , ( D1 , . . . , DN ) , and output a tuple of local models ~θ∗ , ( θ∗1 , . . . , θ ∗ N ) . Like many others , we assume that the users perform federated learning to do so , by leveraging the computation of a common global model ρ ∈ Rd . Intuitively , the global model is an aggregate of all users ’ local models , which users can leverage to improve their local models . The common global model will typically allow users with too few data to obtain an effective local model , while it may be mostly discarded by users whose local datasets are large . More formally , we consider a very general personalized learning framework which generalizes the models proposed by Dinh et al . ( 2020 ) and Hanzely et al . ( 2020 ) . Namely , we consider that the personalized learning algorithm outputs a global minimum ( ρ∗ , ~θ∗ ) of a global loss given by LOSS ( ρ , ~θ , ~D ) , ∑ n∈ [ N ] Ln ( θn , Dn ) + ∑ n∈ [ N ] R ( ρ , θn ) , ( 1 ) whereR is a regularization , typically with a minimum at θn = ρ . For instance , Hanzely et al . ( 2020 ) and Dinh et al . ( 2020 ) define R ( ρ , θn ) , λ ‖ρ− θn‖22 , which we shall call the ` 22 regularization . But other regularizations may be considered , like the ` 2 regularization R ( ρ , θn ) , λ ‖ρ− θn‖2 , or the smooth- ` 2 regularizationR ( ρ , θn ) , λ √ 1 + ‖ρ− θn‖22 . Note that , for all such regularizations , the limit λ →∞ essentially yields the classical non-personalized federated learning framework .

Summary: The paper studies data and model poisoning attacks in federated learning (FL). It argues that there is an equivalence between data poisoning and model poisoning attacks, by restricting attention to linear and logistic regression. The main technique is to leverage ideas from PAC-learning. Another key contribution is to propose a gradient poisoning attack.