Distributed Skellam Mechanism: a Novel Approach to Federated Learning with Differential Privacy

1 INTRODUCTION . Deep neural networks , especially large-scale ones such as GPT-3 ( Brown et al. , 2020 ) , are known for their excellent memorization capabilities ( Song et al. , 2017 ; Feldman , 2020 ; Zhang et al. , 2021 ) . However , it is rather difficult to control what exactly the neural net memorizes , and unintended data memorization can be a serious concern when the underlying training data contains sensitive information ( Carlini et al. , 2019 ) . For instance , consider a bank that trains a GPT-like language model on call center transcripts . Due to data memorization , it is possible to extract sensitive information by letting the model auto-complete a prefix , e.g. , “ my account number is : ” . Clearly , if such a model ( or its API ) is ever exposed to the adversary , it becomes a ligation machine as attackers can attempt with various prefixes to extract sensitive data , and subsequently sue the bank for privacy violations . Shokri et al . ( 2017 ) report that simple and intuitive measures often fail to provide sufficient protection , and the only way found to completely address the issue is to train the model with the rigorous guarantees of differential privacy ( DP ) ( Dwork et al. , 2006 ) . This paper focuses on the scenario that multiple individual participants jointly train a machine learning model using federated learning ( FL ) ( McMahan et al. , 2017 ) through distributed stochastic gradient descent ( SGD ) ( McDonald et al. , 2010 ; Dean et al. , 2012 ; Coates et al. , 2013 ; Abadi et al. , 2016a ) . Specifically , in every iteration , each individual computes the gradients with respect to the current model weights based on her own data ; then , gradients from all participants are aggregated to update the model . Note that the gradients from each individual may reveal sensitive information about her private dataset ( Shokri et al. , 2017 ; Pyrgelis et al. , 2018 ; Yeom et al. , 2018 ; Nasr et al. , 2019 ; Melis et al. , 2019 ) . A common approach to addressing this problem is by employing a secure multiparty computation ( MPC ) protocol ( Yao , 1986 ; Chaum et al. , 1987 ; Gennaro et al. , 2002 ; Ishai et al. , 2010 ; Beimel et al. , 2014 ; Cramer et al. , 2015 ; Ananth et al. , 2018 ) , which computes the aggregate gradients while preserving the confidentiality of the gradients from each individual participant . One advantage of MPC is that it is a decentralized approach that does not require a trusted third party , which can be difficult to establish in some applications , e.g. , in finance and healthcare . Note that although MPC protects individuals ’ privacy in the gradient update process by concealing the gradient values of each participant , it does not provide any protection against data extraction attacks caused by unintended data memorization ( Dwork et al. , 2015 ; Song & Shmatikov , 2019 ; Melis et al. , 2019 ; Song & Shmatikov , 2020 ) . As mentioned earlier , an effective methodology to defend against such attacks is to perturb the gradients to satisfy differential privacy ( Shokri et al. , 2017 ) . Since there is no trusted third-party in our setting , such gradient perturbations need to done in a decentralized fashion , i.e. , each FL participant adds noise to her own gradients , such that the aggregated gradients over all participants satisfies DP , which is referred to as distributed differential privacy ( Goryczka et al. , 2013 ; Kairouz et al. , 2021 ) . Although gradient perturbation under DP has been studied in previous work ( notably , DPSGD ( Abadi et al. , 2016b ) ) , it is far from trivial to adapt centralized DP solutions to our setting , due to a fundamental problem : that the MPC protocol requires gradients to be represented as integers ( more precisely , finite field elements ( Paillier , 1999 ; Bonawitz et al. , 2017 ; Bell et al. , 2020 ) ) . DPSGD , on the other hand , injects real-valued Gaussian noise to the gradients . Although real numbers can be quantized and ( approximately ) represented using large integers , the quantized random noise have rather different mathematical properties , which render a tight privacy cost analysis much more difficult , especially under the decentralized setting of FL . For instance , a nice property of the continuous Gaussian distribution is that summing up n continuous noise values following i.i.d . unit-variance Gaussian distribution results in an amplified continuous Gaussian noise of variance n. This property does not hold , however , if the Gaussian noise values are first quantized before aggregated . Further , the privacy analysis ( specifically , the moment accountant analysis technique ) of the DPSGD algorithm also replies on other important properties of the continuous Gaussian distribution , which do not hold when the noise is quantized . Hence , DPSGD does not directly apply to our setting . This issue has been neglected by many existing distributed DP solutions , e.g. , ( Goryczka et al. , 2013 ; Valovich & Aldà , 2017 ; Truex et al. , 2019 ) . Existing Solutions . Agarwal et al . ( 2018 ) propose cpSGD , which injects binomial noise ( i.e. , the sum of multiple binary values drawn from independent Bernoulli trials ) to the discretized gradients at each participant of FL , to satisfy DP . Similar to Gaussian noise in the continuous domain , binomial noise can also be aggregated , i.e. , the sum of multiple i.i.d . binomial noise values also follows a binomial distribution . However , compared to the continuous Gaussian distribution , existing theoretical tools for analyzing binomial noise aggregation leads to rather loose bounds ; further , the bionomial distribution is incompatible with the moment accountant analysis technique in DPSGD ( Kairouz et al. , 2021 ) . Consequently , cpSGD leads to poor utility , as demonstrated in Section 4 . Recently , the distributed discrete Gaussian mechanism ( DDG ) ( Kairouz et al. , 2021 ) addresses the above issues by injecting independent discrete Gaussian noise ( Canonne et al. , 2020 ) to the gradients at each participant . Similar to the binomial distribution , the discrete Gaussian distribution is also defined over an integer domain ; meanwhile , DDG is fully compatible with the moment accountant analysis technique in DP-SGD , and , thus , enjoys the tight privacy cost analysis . However , the discrete Gaussian distribution is not aggregatable , meaning that the sum of noise drawn from multiple i.i.d . discrete Gaussian distributions does not follow another discrete Guassian distribution , which renders analysis difficult under in the decentralized setting of FL , and leads to looser bounds in the privacy analysis . Further , the privacy guarantee of the aggregated noise in DDG degrades linearly with the dimensionality d of the gradients , leading to poor scalablity to large neural networks . Our contribution . In this work , we propose a new mechanism for enforcing distributed differential privacy for federated learning : the distributed Skellam mechanism ( DSM ) , which injects random noise drawn from the symmetric Skellam distribution . Although the Skellam distribution has been used before in the DP literature ( Valovich & Aldà , 2017 ) , the privacy analysis therein does not cover the decentralized setting of FL , or the iterative , sampling-based SGD algorithm , both of which require highly non-trivial mathematical analysis . Specifically , we prove that DSM satisfies both Rényi-DP and ( , δ ) -DP , defined in Section 2 . Similar to our competitor DDG described above , DSM is compatible with the DPSGD framework and its moment accountant analysis technique , leading to tight bounds on the privacy loss analysis . Meanwhile , unlike DDG , the privacy guarantees of DSM are independent of the dimensionality of the gradients , which scales well to large models . Further , similar to the continuous Gaussian distribution ( and unlike the discrete Gaussian distribution in DDG ) , i.i.d . Skellam noise values can be aggregated to form an amplified noise that still follows the Skellam distribution , which leads to clean and elegant proofs in the decentralized setting of FL , and tight bounds in the privacy analysis . We apply DSM to federated learning with distributed SGD , with quantized gradients , e.g. , as required by the MPC protocol , and present the complete training algorithm . Extensive experiments using benchmark datasets show that our solution leads to consistent and significant utility gains over its competitors , under a variety of settings with different privacy and communication constraints . 2 PRELIMINARIES . A random variable Y follows a Poisson distribution of parameter λ if its probability distribution is Pr [ Y = k ] = exp ( −λ ) λ k k ! , k = 0 , 1 , 2 , . . .. Both the mean and variance of Y is λ . A random variable Z follows a Skellam distribution if it is the difference between two independent Poisson variables Y1 and Y2 . In this work , we restrict our attention to the case where Y1 and Y2 have the same parameter λ . In that case , the probability distribution of Z is Pr [ Z = k ] = exp ( −2λ ) I|k| ( 2λ ) , k = 0 , ±1 , ±2 , . . . , ( 1 ) where Iv ( u ) , ∑∞ h=0 1 h ! Γ ( h+v+1 ) ( u 2 ) 2h+v is the modified Bessel function of the first kind . We write that Z ∼ Sk ( λ , λ ) . By linearity of expectation , Z has mean 0 and variance 2λ . We say that two datasets X and X ′ are neighboring if one can be obtained by adding or removing one tuple from the other . The main idea of differential privacy ( DP ) is to ensure that the outcomes of a randomized mechanism on neighboring datasets are always similar ; intuitively , this provides plausible deniability on whether a given data record x belongs to the dataset X or not , and , thus , protects the privacy of the individual whose record is x . A classic definition of differential privacy is ( , δ ) -DP ( Dwork et al. , 2006 ) , as follows . Definition 1 ( ( , δ ) -Differential Privacy ( Dwork et al. , 2006 ) ) . A randomized mechanismM satisfies ( , δ ) -differential privacy ( DP ) if Pr [ M ( X ) ∈ O ] ≤ exp ( ) · Pr [ M ( X ′ ) ∈ O ] + δ , ( 2 ) for any set of output O ⊆ Range ( M ) and any neighboring datasets X and X ′ . Note that ( , δ ) -DP can be considered as a worst-case privacy guarantee for a mechanism , as it enforces an upper bound on the probability ratio of all possible outcomes . An alternative definition called Rényi-Differential Privacy ( RDP ) ( Mironov , 2017 ) , which is built upon the concept of Rényi Divergence , considers the average case privacy guarantee instead . Definition 2 ( Rényi Divergence ( van Erven & Harremoës , 2014 ) ) . Assuming that distributions P and Q are defined over the same domain , and P is absolute continuous with respect to Q , then the Rényi divergence of P from Q of finite order α ∈ ( 0 , 1 ) ∪ ( 1 , ∞ ) is defined as : Dα ( P‖Q ) = 1 α− 1 logEX∼P [ ( P ( X ) Q ( X ) ) α−1 ] , ( 3 ) where we adopt the convention that 00 = 0 and y 0 = ∞ for any y > 0 , and the logarithm is with base e. Definition 3 ( Rényi Differential Privacy ( Mironov , 2017 ) ) . A randomized mechanismM satisfies ( α , τ ) -Rényi differential privacy ( RDP ) if Dα ( M ( X ) ‖M ( X ′ ) ) ≤ τ for all neighboring datasets X and X ′ . Given a function of interest , the canonical way make it differentially private is to perturb its outcome through noise injection . Roughly speaking , the scale of the noise should be calibrated to the sensitivity of the function of interest ( Dwork et al. , 2006 ) , formally defined as follows . Definition 4 ( Sensitivity ) . The sensitivity S ( F ) of a function F : D → Rd , denoted as S ( F ) , is defined as S ( F ) = max X∼X′ ‖F ( X ) − F ( X ′ ) ‖ , where X ∼ X ′ denotes that X and X ′ are neighboring datasets , and ‖·‖ is a norm . In particular , injecting continuous Gaussian noise sampled from N ( 0 , σ2 ) to each dimension of function F satisfies ( α , αS 2 ( F ) 2σ2 ) -RDP ( Mironov , 2017 ) , where S ( F ) stands for the L2 sensitivity of function F . In many applications , we also need to analyze the overall privacy guarantee of a mechanism consisting of multiple components ( e.g. , training neural networks with SGD ) . We have the following composition and sub-sampling lemmata for RDP mechanisms . Lemma 1 ( Composition Lemma ( Mironov , 2017 ) ) . If mechanisms M1 , . . . , MT satisfies ( α , τ1 ) , . . . , ( α , τT ) -RDP , respectively , thenM1 ◦ . . . ◦MT satisfies ( α , ∑T t=1 τi ) -RDP . Lemma 2 ( Subsampling Lemma ( Zhu & Wang , 2019 ; Mironov et al. , 2019 ) ) . LetM be a mechanism that satisfies ( l , τ ( l ) ) -RDP for l = 2 , . . . , α ( α ∈ Z , α > 2 ) , and Sq be a procedure that uniformly sample each record of the input data with probability q . Then M ◦ Sq satisfies ( α , τ ) RDP with τ = 1 α− 1 log ( ( 1− q ) α−1 ( αq − q − 1 ) + α∑ l=2 ( α l ) ( 1− q ) α−lqle ( l−1 ) τ ( l ) ) . Finally , any mechanism that satisfies ( α , τ ) -RDP also satisfies ( , δ ) -DP , for values of and δ as follows . Lemma 3 ( Converting ( α , τ ) -RDP to ( , δ ) -DP ( Canonne et al. , 2020 ) ) . For any α ∈ ( 1 , ∞ ) , if Dα ( M ( X ) ‖M ( X ′ ) ) ≤ τ for any neighboring databases X and X ′ , thenM satisfies ( , δ ) -DP for = τ + log ( 1/δ ) + ( α− 1 ) log ( 1− 1/α ) − log ( α ) α− 1 . ( 4 )

Summary: ### update after the discussions  I am satisfied with the responses from the authors' and the updated paper. I therefore recommend accepting the paper.  ---  The paper considers the problem of distributed differentially private (DP) learning using black-box secure multi-party computation (MPC) for aggregating gradients for learning the model.  In principle, using e.g. secure aggregation allows each party to add a small amount of noise scaled so that the noise level after aggregation matches what one needs with a trusted central party. However, since the existing tools (mainly Gaussian mechanism) typically assume continuous space, while MPC works with discrete values, this does not work in practice: the main problem is that the discretised noise is not infinitely divisible so the sum is not guaranteed to follow the same distribution as the individual contributions.  To remedy the problem, the authors propose the distributed Skellam mechanism, which is both dicrete and infinitely divisible. The authors show that the Skellam mechanism privacy cost can be calculated using Rényi DP (RDP), and continue to show that it performs significantly better than the existing methods based on binomial noise and discrete Gaussian noise using MNIST and Fashion MNIST data for testing.