Voting-based Approaches For Differentially Private Federated Learning

1 INTRODUCTION . With increasing ethical and legal concerns on leveraging private data , federated learning ( McMahan et al. , 2017 ) ( FL ) has emerged as a paradigm that allows agents to collaboratively train a centralized model without sharing local data . In this work , we consider two typical settings of federated learning : ( 1 ) Local agents are in large number , i.e. , learning user behavior over many mobile devices ( Hard et al. , 2018 ) . ( 2 ) Local agents are in small number with sufficient instances , i.e. , learning a health related model across multiple hospitals without sharing patients ’ data ( Huang et al. , 2019 ) . When implemented using secure multi-party computation ( SMC ) ( Bonawitz et al. , 2017 ) , federated learning eliminates the need for any agent to share its local data . However , it does not protect the agents or their users from inference attacks that combine the learned model with side information . Extensive studies have established that these attacks could lead to blatant reconstruction of the proprietary datasets ( Dinur & Nissim , 2003 ) and identification of individuals ( a legal liability for the participating agents ) ( Shokri et al. , 2017 ) . Motivated by this challenge , there had been a number of recent efforts ( Truex et al. , 2019b ; Geyer et al. , 2017 ; McMahan et al. , 2018 ) in developing federated learning methods with differential privacy ( DP ) , which is a well-established definition of privacy that provably prevents such attacks . Among the efforts , DP-FedAvg ( Geyer et al. , 2017 ; McMahan et al. , 2018 ) extends the NoisySGD method ( Song et al. , 2013 ; Abadi et al. , 2016 ) to the federated learning setting by adding Gaussian noise to the clipped accumulated gradient . The recent state-of-the-art DP-FedSGD ( Truex et al. , 2019b ) is under the same framework but with per-sample gradient clipping . A notable limitation for these gradient-based methods is that they require clipping the magnitude of gradients to τ and adding noise proportional to τ to every coordinate of the shared global model with d parameters . The clipping and perturbation steps introduce either large bias ( when τ is small ) or large variance ( when τ is large ) , which interferes the SGD convergence and makes it hard to scale up to largecapacity models . In Sec . 3 , we concretely demonstrate these limitations with examples and theory . Particularly , we show that the FedAvg may fail to decrease the loss function together with gradient clipping , and DP-FedAvg requires many outer-loop iterations ( i.e. , many rounds of communication to synchronize model parameters ) to converge under differential privacy . To avoid the gradient clipping , we propose to conduct the aggregation over the label space , as shown to be an effective approach in standard ( non-federated ) learning settings , i.e. , voting-based modelagnostic approaches ( Papernot et al. , 2017 ; 2018 ; Zhu et al. , 2020 ) . To achieve it , we relax the traditional federated learning setting to allow unlabeled public data at the server side . We also consider a more complete scenario for federated learning , where there are a large number of local agents or a limited number of local agents . The agent-level privacy as introduced in DP-FedAvg , works seamlessly with our setting having many agents . However , when there are few agents , hiding each data belonging to one specific agent becomes burdensome or unnecessary . To this end , we provide a more complete privacy notion , i.e. , agent-level and instance-level . Under each of the setting , we theoretically and empirically show that the proposed label aggregation method effectively removes the sensitivity issue caused by gradient clipping or noise addition , and achieves favorable privacy-utility trade-off compared to other DPFL algorithms . Our contributions are summarized as the following : 1 . We propose two voting-based DPFL algorithms via label aggregation ( PATE-FL and PrivateKNN-FL ) and demonstrate their clear advantages over gradient aggregation based DPFL methods ( e.g. , DP-FedAvg ) in terms of communication cost and scalability to high-capacity models . 2 . We provide provable differential privacy guarantees under two levels of granularity : agentlevel DP and instance-level DP . Each is natural in a particular regime of FL depending on the number of agents and the size of their data . 3 . Extensive evaluation demonstrates that our method improves the privacy-utility trade-off over randomized gradient-based approaches in both agent-level and instance-level cases . A remark of our novelty . Though PATE-FL and Private-kNN-FL are algorithmically similar to the original PATE ( Papernot et al. , 2018 ) and Private-KNN ( Zhu et al. , 2020 ) , they are not the same and we are adapting them to a new problem — federated learning . The adaptation itself is nontrivial and requires substantial technical innovations . We highlight three challenges below . • Several key DP techniques that contributed to the success of PATE and Private-KNN in the standard setting are no longer applicable ( e.g. , Privacy amplification by Sampling and Noisy Screening ) . This is partly due to that in standard private learning , the attacker only sees the final models ; but in FL , the attacker can eavesdrop in all network traffic . • Moreover , PATE and Private-kNN only provide instance-level DP . We show PATE-FL and Private-kNN-FL also satisfy the stronger agent-level DP . PATE-FL ’ s agent-level DP parameter is , surprisingly , a factor of 2 better than its instance-level DP parameter . And PrivatekNN-FL in addition enjoys a factor of k amplification for the instance-level DP . • A key challenge of FL is the data heterogeneity of individual agents , while PATE randomly splits the dataset so each teacher is identically distributed . The heterogeneity does not affect our privacy analysis but does make it unclear whether PATE would work . We are the first to report strong empirical evidence that the PATE-style DP algorithms remain highly effective in the non-iid case . 2 PRELIMINARY . In this section , we start with introducing the typical notations of federated learning and differential privacy . Then , two randomized gradient-based baselines , DP-FedAvg and DP-FedSGD , are introduced as the DPFL background . 2.1 FEDERATED LEARNING . Federated learning ( McMahan et al. , 2017 ; Bonawitz et al. , 2017 ; Mohassel & Zhang , 2017 ; Smith et al. , 2017 ) is a distributed machine learning framework that allows clients to collaboratively train a global model without sharing local data . We considerN agents , each agent i has ni data kept locally and privately from a party-specific domain distributionDi . C is the number of classes . The objective is to output a global model that performs well on the target ( server ) distribution . Most prior works consider the target distribution as a uniform distribution over the union of all local data , which is restrictive in practice . Here we consider an agnostic federated learning scenario ( Mohri et al. , 2019 ; Peng et al. , 2019c ) , where the server distributionDG can be different from all agent distributions . In light of this , we assume each agent has access to part of unlabeled server data drawn from the target distribution DG . FedAvg ( McMahan et al. , 2017 ) is a vanilla federated learning algorithm that we consider as a non-DP baseline . In this algorithm , a fraction of agents is sampled at each communication round with a probability q . Each selected agent downloads the shared global model and improves it by learning from local data using E iterations of stochastic gradient descent ( SGD ) . We denote this local update process as an inner loop . Only the gradient is sent to the server , where it is averaged with other selected agents ’ gradient to improve the global model . The global model is learned after T communication rounds , where each communication round is denoted as one outer loop . 2.2 DIFFERENTIAL PRIVACY FOR FEDERATED LEARNING . Differential privacy ( Dwork et al. , 2006 ) is a quantifiable and composable definition of privacy that provides provable guarantees against identification of individuals in a private dataset . Definition 1 . A randomized mechanism M : D → R with a domain D and range R satisfies ( , δ ) -differential privacy , if for any two adjacent datasets D , D′ ∈ D and for any subset of outputs S ⊆ R , it holds that Pr [ M ( D ) ∈ S ] ≤ e Pr [ M ( D′ ) ∈ S ] + δ . The definition applies to a variety of different granularity , depending on how the adjacent datasets are defined , i.e. , if we are to protect whether one agent participates into training , the neighboring datasets are defined by adding or removing the entire local data within that agent . It is known as agent-level ( user-level ) differential privacy , which has been investigated in DP-FedAvg ( Geyer et al. , 2017 ; McMahan et al. , 2018 ) . Compared to FedAvg , DP-FedAvg ( Figure 1 ) enforces clipping of peragent model gradient to a threshold S and adds noise to the scaled gradient before it is averaged at the server . Note that this DP notion is favored when data samples within one agent reveal the same sensitive information , e.g. , cell phone agents send the same message . However , when there are only a few agents , hiding the entire dataset from one agent becomes difficult and inappropriate . We then consider the instance-level DP , where the adjacent dataset is defined by differing one single training example . This definition is consistent with the standard non-federated learning differential privacy ( Abadi et al. , 2016 ; Bassily et al. , 2014 ; Chaudhuri et al. , 2011 ) . Model training with instance-level DP restricts the adversary ’ s power in detecting a specific training instance ’ s presence or absence . DP-FedSGD ( Truex et al. , 2019a ; Peterson et al. , 2019 ) , one such state-of-the-art for the instance-level DP , performs NoisySGD ( Abadi et al. , 2016 ) for a fixed number of iterations at each agent . The gradient updates are averaged on each communication round at the server , as shown in Figure 1 . SMC is a cryptographic technique that securely aggregates local updates before the server receives it . While SMC does not have a differential privacy guarantee , it can be combined with DP to amplify the privacy guarantee ( Bhowmick et al. , 2018 ; Agarwal et al. , 2018 ; Truex et al. , 2019b ) against attackers that eavesdrop what sent out by each agent . In our experiment , we assume that the aggregation is conducted by SMC for all privacy-preserving algorithms that we consider . 3 CHALLENGES FOR GRADIENT-BASED FEDERATED LEARNING . In this section , we highlight the main challenges of the conventional DPFL frameworks in terms of accuracy , convergence and communication cost . For other challenges , we refer the readers to a survey ( Kairouz et al. , 2019 ) . The details of DP-FedAvg are summarized in appendix algorithm section . 3.1 CHALLENGE 1 : BIASED GRADIENT ESTIMATION . Recent works ( Li et al. , 2018 ) have shown that the FedAvg may not converge well under heterogeneity ( e.g. , non-identical distributions ) . Here , we provide a simple example to show that the clipping step of DP-FedAvg may raise additional challenge . Example 2 ( clipping ) . Let N = 2 , each agent i ’ s local update is 4i ( E iterations of SGD ) . We enforce clipping of per-agent update4i by performing4i/max ( 1 , ||4i||2τ ) , where τ is the clipping threshold . Consider the special case when ||41||2 = τ+α and ||42||2 ≤ τ . Then the global update will be 12 ( τ41 ||41||2 +42 ) , which is biased . The unbiased global update shall be 12 ( 41 +42 ) . Such a simple example can be embedded in more realistic problems , causing substantial bias that leads to non-convergence .

Summary: The paper proposes two approaches (i.e., PATE-FL and Private-kNN-FL) to train a differentially private global model in a federated setting based on [1] and [2]. In PATE-FL, each client first trains a teacher model using their local dataset. The teacher models are used to make noisy predictions on a public dataset. Then, the public dataset with predicted labels is used to train a final model. In Private-kNN-FL, instead of training a teacher model, the prediction depends on the labels of the nearest neighbors. The experiments show that PATE-FL and Private-kNN-FL can outperform DP-FedAvg in terms of agent-level DP and instance-level DP, respectively.