An Equivalence Between Data Poisoning and Byzantine Gradient Attacks

To study the resilience of distributed learning , the “ Byzantine ” literature considers a strong threat model where workers can report arbitrary gradients to the parameter server . While this model helped generate several fundamental results , it has however sometimes been considered unrealistic , when the workers are mostly trustworthy machines . In this paper , we show a surprising equivalence between this model and data poisoning , a threat considered much more realistic . More specifically , we prove that any gradient attack can be reduced to data poisoning in a personalized federated learning system that provides PAC guarantees ( which we show are both desirable and realistic in various personalized federated learning contexts such as linear regression and classification ) . Maybe most importantly , we derive a simple and practical attack that may be constructed against classical personalized federated learning models , and we show both theoretically and empirically the effectiveness of this attack . 1 INTRODUCTION . Learning algorithms now typically leverage data generated by a large number of users ( Smith et al. , 2013 ; Wang et al. , 2019a ; b ) to often learn a common model that fits a large population ( Konecný et al. , 2015 ) , but also sometimes to construct a personalized model for each individual ( Ricci et al. , 2011 ) . Autocompletion ( Lehmann & Buschek , 2021 ) , conversational ( Shum et al. , 2018 ) and recommendation ( Ie et al. , 2019 ) algorithms are examples of such algorithms deployed at scale . To be effective , besides huge amounts of ( distributed ) data ( Brown et al. , 2020 ; Fedus et al. , 2021 ) , these algorithms require a high level of customization . This has motivated research into personalized federated learning ( Fallah et al. , 2020 ; Hanzely et al. , 2020 ; Dinh et al. , 2020 ) . However , in applications such as content recommendation , activists , companies , and politicians have strong incentives to promote certain views , products or ideologies ( Hoang , 2020 ; Hoang et al. , 2021 ) . Remember for instance that , on YouTube , two views out of three result from algorithmic recommendations ( Solsman , 2018 ) . Perhaps unsurprisingly , this has led to vast amounts of fabricated activities to bias algorithms ( Bradshaw & Howard , 2019 ; Neudert et al. , 2019 ) , like “ fake reviews ” ( Wu et al. , 2020 ) . The scale of this phenomenon is well illustrated by the case of Facebook which , in 2019 alone , reported the removal of around 6 billion fake accounts from its platform ( Fung & Garcia , 2019 ) . This is particularly concerning in the era of “ stochastic parrots ” ( Bender et al. , 2021 ) : climate denialists are incentivized to pollute textual datasets with claims like “ climate change is a hoax ” , as autocompletion , conversational and recommendation algorithms trained on such data will more likely spread these views ( McGuffie & Newhouse , 2020 ) . This raises serious concerns about the vulnerability of personalized federated learning to such misleading data . Data poisoning attacks clearly constitute now a major machine learning security issue ( Kumar et al. , 2020 ) . Overall , in highly adversarial environments like social media , given the advent of deep fakes ( Johnson & Diakopoulos , 2021 ) , we should expect that most data are strategically crafted and labeled . In this context , the authentication of the data provider seems critical . In particular , the safety of learning algorithms arguably demands that they be trained solely on cryptographically signed data , that is , data that provably come from a known source . But even signed data can not be wholeheartedly trusted since users usually have preferences over what ought to be recommended to others . They thus have incentives to behave strategically in order to promote certain views or products . To address data poisoning , the Byzantine learning literature usually considers that each federated learning worker may behave arbitrarily ( Blanchard et al. , 2017 ; Mhamdi et al. , 2018 ; El-Mhamdi et al. , 2021 ) . Recall that at each iteration of the federated learning stochastic gradient descent , each worker is given the updated model , and is asked to compute the gradient of the loss function with respect to ( a batch of ) its local data . Byzantine learning usually assumes that a malicious ( Byzantine ) worker may report any gradient ; without having to justify whether such a gradient could have been generated through data poisoning . In fact , the gradient attack threat model has sometimes been claimed to be unrealistic in practical federated learning ( Shejwalkar et al. , 2021 ) , especially when the workers are machines owned by trusted entities ( cross-silo FL ( Kairouz et al. , 2021 ) ) . We prove in this paper an equivalence between gradients attacks and fabricated data injection , in a general and desirable collaborative learning framework . Thereby , our paper provides the first practically compelling argument for the necessity to protect federated learning against gradient attacks . Contributions . As a preamble of our main result , we formalize local PAC * learning1 ( Valiant , 1984 ) for personalized learning , and prove that a simple and general solution to personalized federated linear regression and classification is indeed locally PAC * learning . Our proof leverages a new concept called gradient-PAC * learning , which is of independent interest . We prove that it is sufficient to guarantee local PAC * learning , and that it is verified by basic learning algorithms , like linear and logistic regression . This is an important and highly nontrivial contribution of this paper . Our main contribution is then to prove that local PAC * learning in personalized federated learning essentially implies an equivalence between data poisoning and gradient attacks . More precisely , we show how any ( converging ) gradient attack can be turned into a data poisoning attack , with the same resulting harm . Given how easy it generally is to create fake accounts on web platforms and to inject data poisoning by generating fake activities , this result should arguably greatly increase the concerns over the vulnerabilities of federated learning with user-generated data . Finally , we propose a simple but very general strategic gradient attack , called the counter-gradient attack ( CGA ) , which any participant to federated learning can deploy to bias the global model towards any target model that better suits their interest . We prove the effectiveness of this attack under fairly general assumptions , which apply to many proposed personalized learning frameworks including Hanzely et al . ( 2020 ) ; Dinh et al . ( 2020 ) . We then show empirically how this gradient attack can be turned into a devastating data poisoning attack , with remarkably few data . Related work . Byzantine learning has provided both negative and positive results in Byzantine resilience ( Blanchard et al. , 2017 ; Mhamdi et al. , 2018 ; Baruch et al. , 2019 ; Xie et al. , 2019 ) , some of which apply almost straightforwardly to personalized federated learning ( El-Mhamdi et al. , 2020 ; 2021 ) . Such results study the resilience against a minority of adversarial users . Our paper however focuses on a different kind of malicious users . Namely , like Suya et al . ( 2021 ) , we study the resilience against strategic users , who aim to bias the learned models towards a specific target model . The study of the resilience against strategic users is part of the research on strategyproof learning . Many special cases of strategyproofness have been tackled , including regression ( Chen et al. , 2018b ; Dekel et al. , 2010 ; Perote & Perote-Peña , 2004 ; Ben-Porat & Tennenholtz , 2017 ) , classification ( Meir et al. , 2012 ; Chen et al. , 2020 ; Meir et al. , 2011 ; Hardt et al. , 2016 ) , statistical estimation ( Cai et al. , 2015 ) , and clustering ( Perote & Sevilla , 2003 ) . However , none of these papers tackles a general personalized federated learning scheme . Typically , for linear regression , Chen et al . ( 2018b ) and Perote & Perote-Peña ( 2004 ) assume that each user can only provide a single data point . This greatly restricts the users ’ ability to contribute to the learning model . And while Dekel et al . ( 2010 ) allows multiple contributions , they either require payments , which might not be possible ( e.g. , due to ethical reasons ) , or they restrict the model to one dimension or a constant function in Rd . Conversely , Suya et al . ( 2021 ) show how to arbitrarily manipulate convex learning models through multiple data injections , in the case where a single model is learned from all data at once . A large literature has focused on data poisoning , with either a focus on backdoor ( Dai et al. , 2019 ; Zhao et al. , 2020 ; Severi et al. , 2021 ; Truong et al. , 2020 ; Schwarzschild et al. , 2021 ) or triggerless attacks ( Biggio et al. , 2012 ; Muñoz-González et al. , 2017 ; Shafahi et al. , 2018 ; Zhu et al. , 2019 ; 1We omit complexity considerations for the sake of generality . We define PAC * to be PAC without such considerations . Huang et al. , 2020 ; Barreno et al. , 2006 ; Aghakhani et al. , 2021 ; Geiping et al. , 2021 ) . However , most of this research analyzed data poisoning without signed data . One noteworthy exception is Mahloujifar et al . ( 2019 ) , whose universal attack amplifies the probability of a ( bad ) property . Collaborative PAC learning was previously introduced by Blum et al . ( 2017 ) , and then extensively studied ( Chen et al. , 2018a ; Nguyen & Zakynthinou , 2018 ) , sometimes with the presence of Byzantine collaborating users ( Qiao , 2018 ; Jain & Orlitsky , 2020 ; Konstantinov et al. , 2020 ) . We stress however that this line of work assumes that all honest users have the same labeling function . In other words , given any query , they agree on how the query should be answered . This is a very unrealistic assumption in many critical applications , like content moderation or language processing . In fact , in such applications , removing outliers can be argued to amount to ignoring minorities ’ views , which would be highly unethical . The very definition of PAC learning must then be adapted , which is what we do in this paper ( also , we adapt it to parameterized models ) . Structure of the paper . The rest of the paper is organized as follows . Section 2 presents a general model of personalized learning , formalizes local PAC * learning and defines a general federated gradient descent algorithm . Section 3 proves the equivalence between data poisoning and gradient attacks , under local PAC * learning . Section 4 proves the local PAC * learning properties for federated linear regression and classification . Section 5 then describes a simple and general data poisoning attack , whose effectiveness against ` 22 is proved theoretically and empirically . Section 6 concludes . 2 A GENERAL PERSONALIZED FEDERATED LEARNING FRAMEWORK . We consider a set [ N ] = { 1 , . . . , N } of users . Each user n ∈ [ N ] has a local signed dataset Dn , and aims to learn a local model θn ∈ Rd . Users may collaborate to improve their models . Personalized learning must then input a tuple of users ’ local datasets ~D , ( D1 , . . . , DN ) , and output a tuple of local models ~θ∗ , ( θ∗1 , . . . , θ ∗ N ) . Like many others , we assume that the users perform federated learning to do so , by leveraging the computation of a common global model ρ ∈ Rd . Intuitively , the global model is an aggregate of all users ’ local models , which users can leverage to improve their local models . The common global model will typically allow users with too few data to obtain an effective local model , while it may be mostly discarded by users whose local datasets are large . More formally , we consider a very general personalized learning framework which generalizes the models proposed by Dinh et al . ( 2020 ) and Hanzely et al . ( 2020 ) . Namely , we consider that the personalized learning algorithm outputs a global minimum ( ρ∗ , ~θ∗ ) of a global loss given by LOSS ( ρ , ~θ , ~D ) , ∑ n∈ [ N ] Ln ( θn , Dn ) + ∑ n∈ [ N ] R ( ρ , θn ) , ( 1 ) whereR is a regularization , typically with a minimum at θn = ρ . For instance , Hanzely et al . ( 2020 ) and Dinh et al . ( 2020 ) define R ( ρ , θn ) , λ ‖ρ− θn‖22 , which we shall call the ` 22 regularization . But other regularizations may be considered , like the ` 2 regularization R ( ρ , θn ) , λ ‖ρ− θn‖2 , or the smooth- ` 2 regularizationR ( ρ , θn ) , λ √ 1 + ‖ρ− θn‖22 . Note that , for all such regularizations , the limit λ →∞ essentially yields the classical non-personalized federated learning framework .

Summary: The paper formalizes the concept of PAC*-learnability for a generalized personalized federated learning model and provides a sufficient condition. Under this model, converging gradient attack is equivalent to data poisoning. The paper also proposes the counter-gradient attack that is effective and data-efficient.