D ISTRIBUTED S KELLAM M ECHANISM : A N OVEL A PPROACH TO F EDERATED L EARNING WITH D IFFERENTIAL P RIVACY
Anonymous authors
Paper under double-blind review
A BSTRACT
Deep neural networks have strong capabilities of memorizing the underlying training data; on the ﬂip side, unintended data memorization can be a serious privacy concern.
An effective and rigorous approach to addressing this problem is to train models with differential privacy ( DP ), which provides information-theoretic privacy guarantees by injecting random noise to the gradients.
This paper focuses on the scenario where sensitive data are distributed among individual participants, who jointly train a model through federated learning , using both secure multiparty computation ( MPC ) to ensure the conﬁdentiality of individual gradients, and differential privacy to avoid data leakage in the resulting model.
A major challenge in this setting is that common mechanisms for enforcing DP in deep learning, which inject real-valued noise , are fundamentally incompatible with MPC, which exchanges ﬁnite-ﬁeld integers among the participants.
Consequently, most existing
DP mechanisms require rather high noise levels, leading to poor model utility.
Motivated by this, we design and develop distributed Skellam mechanism ( DSM ), a novel solution for enforcing differential privacy on models built through an
MPC-based federated learning process.
Compared to existing approaches, DSM has the advantage that its privacy guarantee is independent of the dimensionality of the gradients; further, DSM allows tight privacy accounting due to the nice composition and sub-sampling properties of the Skellam distribution, which are key to accurate deep learning with DP.
The theoretical analysis of DSM is highly non-trivial, especially considering (i) the complicated math of differentially private deep learning in general and (ii) the fact that the Skellam distribution is rather complex, and to our knowledge, has not been applied to an iterative and samplingbased process, i.e., stochastic gradient descent.
Meanwhile, through extensive experiments on various practical settings, we demonstrate that DSM consistently outperforms existing solutions in terms of model utility by a large margin.
I NTRODUCTION
Deep neural networks, especially large-scale ones such as GPT-3 (Brown et al., 2020), are known for their excellent memorization capabilities (Song et al., 2017; Feldman, 2020; Zhang et al., 2021).
However, it is rather difﬁcult to control what exactly the neural net memorizes, and unintended data memorization can be a serious concern when the underlying training data contains sensitive information (Carlini et al., 2019).
For instance, consider a bank that trains a GPT-like language model on call center transcripts.
Due to data memorization, it is possible to extract sensitive information by letting the model auto-complete a preﬁx, e.g., “my account number is: ”.
Clearly, if such a model (or its API) is ever exposed to the adversary, it becomes a ligation machine as attackers can attempt with various preﬁxes to extract sensitive data, and subsequently sue the bank for privacy violations.
Shokri et al.
(2017) report that simple and intuitive measures often fail to provide sufﬁcient protection, and the only way found to completely address the issue is to train the model with the rigorous guarantees of differential privacy ( DP ) (Dwork et al., 2006).
This paper focuses on the scenario that multiple individual participants jointly train a machine learning model using federated learning ( FL ) (McMahan et al., 2017) through distributed stochastic gradient descent (SGD) (McDonald et al., 2010; Dean et al., 2012;
Coates et al., 2013; Abadi et al.,
2016a).
Speciﬁcally, in every iteration, each individual computes the gradients with respect to the current model weights based on her own data; then, gradients from all participants are aggregated to update the model.
Note that the gradients from each individual may reveal sensitive information about her private dataset (Shokri et al., 2017;
Pyrgelis et al., 2018; Yeom et al., 2018;
Nasr et al., 2019; Melis et al., 2019).
A common approach to addressing this problem is by employing a secure multiparty computation ( MPC ) protocol (Yao, 1986;
Chaum et al., 1987;
Gennaro et al., 2002; Ishai et al., 2010; Beimel et al., 2014; Cramer et al., 2015; Ananth et al., 2018), which computes the aggregate gradients while preserving the conﬁdentiality of the gradients from each individual participant.
One advantage of MPC is that it is a decentralized approach that does not require a trusted third party, which can be difﬁcult to establish in some applications, e.g., in ﬁnance and healthcare.
Note that although MPC protects individuals’ privacy in the gradient update process by concealing the gradient values of each participant, it does not provide any protection against data extraction attacks caused by unintended data memorization (Dwork et al., 2015; Song & Shmatikov, 2019;
Melis et al., 2019; Song & Shmatikov, 2020).
As mentioned earlier, an effective methodology to defend against such attacks is to perturb the gradients to satisfy differential privacy (Shokri et al., 2017).
Since there is no trusted third-party in our setting, such gradient perturbations need to done in a decentralized fashion, i.e., each FL participant adds noise to her own gradients, such that the aggregated gradients over all participants satisﬁes DP, which is referred to as distributed differential privacy (Goryczka et al., 2013; Kairouz et al., 2021).
Although gradient perturbation under DP has been studied in previous work (notably,
DPSGD (Abadi et al., 2016b)), it is far from trivial to adapt centralized DP solutions to our setting, due to a fundamental problem: that the MPC protocol requires gradients to be represented as integers (more precisely, ﬁnite ﬁeld elements (Paillier, 1999; Bonawitz et al., 2017; Bell et al., 2020)).
DPSGD , on the other hand, injects real-valued Gaussian noise to the gradients.
Although real numbers can be quantized and (approximately) represented using large integers, the quantized random noise have rather different mathematical properties, which render a tight privacy cost analysis much more difﬁcult, especially under the decentralized setting of FL.
For instance, a nice property of the continuous Gaussian distribution is that summing up n continuous noise values following i.i.d.
unit-variance Gaussian distribution results in an ampliﬁed continuous Gaussian noise of variance n .
This property does not hold, however, if the Gaussian noise values are ﬁrst quantized before aggregated.
Further, the privacy analysis (speciﬁcally, the moment accountant analysis technique) of the
DPSGD algorithm also replies on other important properties of the continuous Gaussian distribution, which do not hold when the noise is quantized.
Hence, DPSGD does not directly apply to our setting.
This issue has been neglected by many existing distributed DP solutions, e.g., (Goryczka et al., 2013; Valovich & Ald ` a, 2017; Truex et al., 2019).
Existing Solutions.
Agarwal et al.
(2018) propose cpSGD , which injects binomial noise (i.e., the sum of multiple binary values drawn from independent Bernoulli trials) to the discretized gradients at each participant of FL, to satisfy DP.
Similar to Gaussian noise in the continuous domain, binomial noise can also be aggregated, i.e., the sum of multiple i.i.d.
binomial noise values also follows a binomial distribution.
However, compared to the continuous Gaussian distribution, existing theoretical tools for analyzing binomial noise aggregation leads to rather loose bounds; further, the bionomial distribution is incompatible with the moment accountant analysis technique in DPSGD (Kairouz et al., 2021).
Consequently, cpSGD leads to poor utility, as demonstrated in Section 4.
Recently, the distributed discrete Gaussian mechanism ( DDG ) (Kairouz et al., 2021) addresses the above issues by injecting independent discrete Gaussian noise (Canonne et al., 2020) to the gradients at each participant.
Similar to the binomial distribution, the discrete Gaussian distribution is also deﬁned over an integer domain; meanwhile, DDG is fully compatible with the moment accountant analysis technique in DP-SGD, and, thus, enjoys the tight privacy cost analysis.
However, the discrete Gaussian distribution is not aggregatable, meaning that the sum of noise drawn from multiple i.i.d.
discrete Gaussian distributions does not follow another discrete Guassian distribution, which renders analysis difﬁcult under in the decentralized setting of FL, and leads to looser bounds in the privacy analysis.
Further, the privacy guarantee of the aggregated noise in DDG degrades linearly with the dimensionality d of the gradients, leading to poor scalablity to large neural networks.
Our contribution.
In this work, we propose a new mechanism for enforcing distributed differential privacy for federated learning: the distributed Skellam mechanism ( DSM ), which injects random noise drawn from the symmetric Skellam distribution.
Although the Skellam distribution has been used before in the DP literature (Valovich & Ald`a, 2017), the privacy analysis therein does not cover the decentralized setting of FL, or the iterative, sampling-based SGD algorithm, both of which require highly non-trivial mathematical analysis.
Speciﬁcally, we prove that DSM satisﬁes both R´enyi-DP and ( (cid:15) , δ )-DP, deﬁned in Section 2.
Similar to our competitor DDG described above, DSM is compatible with the DPSGD framework and its moment accountant analysis technique, leading to tight bounds on the privacy loss analysis.
Meanwhile, unlike DDG , the privacy guarantees of DSM are independent of the dimensionality of the gradients, which scales well to large models.
Further, similar to the continuous Gaussian distribution (and unlike the discrete Gaussian distribution in DDG ), i.i.d.
Skellam noise values can be aggregated to form an ampliﬁed noise that still follows the Skellam distribution, which leads to clean and elegant proofs in the decentralized setting of FL, and tight bounds in the privacy analysis.
We apply DSM to federated learning with distributed SGD, with quantized gradients, e.g., as required by the MPC protocol, and present the complete training algorithm.
Extensive experiments using benchmark datasets show that our solution leads to consistent and signiﬁcant utility gains over its competitors, under a variety of settings with different privacy and communication constraints.
P RELIMINARIES
A random variable Y follows a Poisson distribution of parameter λ if its probability distribution is
Pr[ Y = k ] = exp( − λ )
λ k k ! ,
k = 0 , 1 , 2 , . . . .
Both the mean and variance of Y is λ .
A random variable
Z follows a Skellam distribution if it is the difference between two independent Poisson variables Yand Y 2 .
In this work, we restrict our attention to the case where Y 1 and Y 2 have the same parameter λ .
In that case, the probability distribution of Z is
[Equation]
where I v ( u ) (cid:44) (cid:80) ∞ h = 1 h !
Γ( h + v +1) (cid:0) u 2 (cid:1) 2 h + v is the modiﬁed Bessel function of the ﬁrst kind.
We write that Z ∼ Sk( λ, λ ) .
By linearity of expectation, Z has mean 0 and variance 2 λ .
We say that two datasets X and X (cid:48) are neighboring if one can be obtained by adding or removing one tuple from the other.
The main idea of differential privacy (DP) is to ensure that the outcomes of a randomized mechanism on neighboring datasets are always similar; intuitively, this provides plausible deniability on whether a given data record x belongs to the dataset X or not, and, thus, protects the privacy of the individual whose record is x .
A classic deﬁnition of differential privacy is ( (cid:15), δ ) -DP
(Dwork et al., 2006), as follows.
Deﬁnition 1 ( ( (cid:15), δ ) -Differential
Privacy (Dwork et al., 2006)) .
A randomized mechanism M satisﬁes ( (cid:15), δ )-differential privacy (DP) if
[Equation]
for any set of output O ⊆
Range ( M ) and any neighboring datasets X and X (cid:48) .
Note that ( (cid:15), δ ) -DP can be considered as a worst-case privacy guarantee for a mechanism, as it enforces an upper bound on the probability ratio of all possible outcomes.
An alternative deﬁnition called R´enyi-Differential Privacy (RDP) (Mironov, 2017), which is built upon the concept of R´enyi
Divergence, considers the average case privacy guarantee instead.
Deﬁnition 2 (R´enyi Divergence (van Erven & Harremo¨es, 2014)) .
Assuming that distributions P and Q are deﬁned over the same domain, and P is absolute continuous with respect to Q , then the
R´enyi divergence of P from Q of ﬁnite order α ∈ (0 , 1) ∪ (1 , ∞ ) is deﬁned as:
[Equation]
where we adopt the convention that
=
and y 0 = ∞ for any y > 0 , and the logarithm is with base e .
Deﬁnition 3 (R´enyi Differential Privacy (Mironov, 2017)) .
A randomized mechanism M satisﬁes ( α, τ ) -R´enyi
differential privacy (RDP) if D α ( M ( X ) (cid:107)M ( X (cid:48) )) ≤ τ for all neighboring datasets X and X (cid:48) .
Given a function of interest, the canonical way make it differentially private is to perturb its outcome through noise injection.
Roughly speaking, the scale of the noise should be calibrated to the sensitivity of the function of interest (Dwork et al., 2006), formally deﬁned as follows.
Deﬁnition 4 (Sensitivity) .
The sensitivity S ( F ) of a function F : D → R d , denoted as S ( F ) , is deﬁned as
[Equation]
where X ∼ X (cid:48) denotes that X and X (cid:48) are neighboring datasets, and (cid:107)·(cid:107) is a norm.
In particular, injecting continuous Gaussian noise sampled from N (0 , σ 2 ) to each dimension of function F satisﬁes ( α, αS 2 ( F ) 2 σ 2 ) -RDP (Mironov, 2017), where S ( F ) stands for the L 2 sensitivity of function F .
In many applications, we also need to analyze the overall privacy guarantee of a mechanism consisting of multiple components (e.g., training neural networks with SGD).
We have the following composition and sub-sampling lemmata for RDP mechanisms.
Lemma 1 (Composition Lemma (Mironov, 2017)) .
If mechanisms M 1 , . .
, M T satisﬁes ( α, τ 1 ) , . . .
, ( α, τ T ) -RDP, respectively, then M 1 ◦ . . .
◦ M T satisﬁes ( α, (cid:80) Tt =1 τ i ) -RDP.
Lemma 2 (Subsampling Lemma (Zhu & Wang, 2019; Mironov et al., 2019)) .
Let M be a mechanism that satisﬁes ( l, τ ( l )) -RDP for l = 2 , . .
, α ( α ∈ Z , α > 2) , and S q be a procedure that uniformly sample each record of the input data with probability q .
Then M ◦ S q satisﬁes ( α, τ ) -
RDP with
[Equation]
Finally, any mechanism that satisﬁes ( α, τ ) -RDP also satisﬁes ( (cid:15), δ ) -DP, for values of (cid:15) and δ as follows.
Lemma 3 (Converting ( α, τ ) -RDP to ( (cid:15), δ ) -DP
(Canonne et al., 2020)) .
For any α ∈ (1 , ∞ ) , if
D α ( M ( X ) (cid:107)M ( X (cid:48) )) ≤ τ for any neighboring databases X and X (cid:48) , then M satisﬁes ( (cid:15), δ ) -DP for (cid:15)
[Equation]
D ISTRIBUTED S KELLAM M ECHANISM
Section 3.1 ﬁrst formalizes the problem of distributed sum estimation, and presents the proposed distributed Skellam mechanism ( DSM ) for this problem.
Section 3.2 establishes the privacy guarantees of DSM .
Then, in Section 3.3, we apply DSM to our main problem setting: differentially private federated learning.
3. F ORMAL D ESCRIPTION
Suppose that an integer-valued multi-dimensional dataset X = ( x 1 , . . .
, x n ) is distributed to n individuals (referred to as clients in the following), where client i possesses data point x i ∈ Z d , for i = 1 , . . . , n .
Without loss of generality, we assume that the L 2 and L ∞ norms of each x i is bounded by constants ∆ 2 and ∆ ∞ , respectively, i.e., (cid:107)
x i (cid:107) 2 ≤ ∆ 2 and (cid:107) x i (cid:107) ∞ ≤ ∆ ∞ for any x i ∈ X .
An untrusted server aims to compute the (approximate) sum of the dataset, i.e., ¯ x = (cid:80) ni
=1 x i , from the clients.
The computation of ¯ x must satisfy differential privacy.
In particular, we focus on the
RDP deﬁnition (Deﬁnition 3), which can be converted to the classic ( (cid:15) , δ )-DP
(Deﬁnition 1) through
Lemma 3.
Speciﬁcally, we aim to design a private mechanism M , such that for all neighboring datasets X, X (cid:48) ,
[Equation]
for some α > 1 .
We measure the error of M by
[Equation]
where the expectation is taken over the randomness in M .
Next, we present the proposed solution DSM for the above distributed sum estimation problem, outlined in Algorithm 1.
Each client i ﬁrst independently samples a noise vector following the
Skellam distribution, and injects this noise to her data (Line 2 in Algorithm 1).
In particular, the client samples a random d -dimensional noise vector Z =
( Z 1 , . . .
, Z d ) , where each element is identically and independently distributed, and follows the Skellam distribution of parameter λ , i.e.,
[Equation]
We assume the clients have black-box access to a secure aggregation protocol A , following the convention in (Agarwal et al., 2018; Kairouz et al., 2021).
The clients then compute the sum of their perturbed data using A , and release the sum to the server (Line 3 in Algorithm 1).
Modular Arithmetic with communication constraints.
When each element x i in X is represented using a limited number B of bits, each client needs to perform an extra step of modulo operation depending on B after injecting Skellam noise (Line 2), so as to avoid overﬂows.
Accordingly, after obtaining the sum (Line 3), the server also needs to unwrap the modulo operation to recover the original noisy sum.
We refer the reader to (Kairouz et al., 2021) for a detailed discussion on this issue.
In our experiments in Section 4, we demonstrate that the proposed solution DSM achieves signiﬁcant accuracy gains compared to existing approaches with various settings of the bit limit B .
3. T HEORETICAL A NALYSIS
The following theorem states the privacy guarantee for Algorithm 1, which is the main theoretical result of this paper.
Theorem 1 (Privacy guarantee of the distributed Skellam mechanism) .
When 1 < α < 2 nλ ∆ ∞ + 1 ,
Algorithm 1 with noise parameter λ satisﬁes (
[Equation]
, τ ) -R´enyi
Differential Privacy with
[Equation]
Proof.
To prove the privacy guarantee of Algorithm 1, it sufﬁces to derive the privacy guarantee of the sum of { Sk λ ( x i ) } ni =1 .
Note that the secure aggregation protocol A (used as a black-box) ensures that only the sum of the perturbed data is revealed to the server.
In addition, according to properties of the Skellam distribution, aggregating n independent Skellam noise vectors Z ∼ Sk ( λ, λ ) results in a single Skellam noise ¯ Z ∼ Sk ( nλ, nλ ) (Skellam, 1946).
Hence, we only need to focus on the privacy guarantee achieved by ¯ Z .
Further, in what follows, we focus on the one dimensional case of the problem, i.e., d = 1 .
The proof in the general case where d > 1 then follows from the additivity of R´enyi divergence for product distributions (van Erven & Harremo¨es, 2014).
Without loss of generality, we let s > 0 , Z ∼ Sk( λ, λ ) and compute Φ =
exp(( α − 1) D α
( Z + s (cid:107) Z )) .
We ﬁrst present several useful inequalities, whose proofs can be found in the appendix.
Let
[Equation] [Equation]
According to Lemma 4 in the appendix, we have:
[Equation]
In addition, for any 0 < a ≤ w , according to Lemmata 5-7 in the appendix, we have
[Equation] [Equation]
and (cid:16)
[Equation]
(cid:112)
[Equation]
(cid:112)
[Equation]
By the deﬁnition of Skellam distributions and Eq.
[Equation]
We compute
[Equation]
, where
[Equation]
By Eq.
(11), when z ≥ i ,
[Equation]
Therefore, when z ≥ s , we have Q s, 2 λ ( z ) Q s, 2 λ ( − z ) <
exp (cid:0) szλ (cid:1) , and R s, 2 λ
( z ) < exp (cid:16) s ( α − 1)
z 2 λ (cid:17) .
Since the function v (cid:55)→ v + 1 v is increasing,
[Equation]
By Eq.
(12), when z ≥ i ,
[Equation]
Hence, when z ≥ s , we have Q s, 2 λ ( z ) · Q s, 2 λ ( − z ) < exp (cid:16) s ( s +1)2 λ (cid:17) .
Meanwhile, when − s <
z < s , according to Lemma 8 in the appendix, we have:
[Equation]
Therefore,
[Equation]
The moment generating function of Sk( λ, λ ) is E [ e tZ ] =
e λ ( e t + e − t − 2) .
When 0 < t < 1 , we have e t + e − t − 2 < 1 . 09 t 2 .
Thus, when λ > s ( α − 1),
E
[Equation]
As a result
[Equation]
when λ > s ( α − 1), we have Φ ≤ exp (cid:16) s 2 ( α − 1) 2 λ
[Equation]
Note that the privacy guarantee of DSM in Theorem 1 is independent of the data dimensionality d .
Next we analyze the error incurred by Algorithm 1, which has the following bound:
Corollary 1 (Error bound of the distributed Skellam mechanism) .
With 1 < α < 2 nλ ∆ ∞ + 1 , the error incurred by Algorithm 1 that satisﬁes ( α, τ ) -RDP is bounded by
[Equation]
The proof for the above corollary is omitted for brevity, which directly follows from Theorem 1 and the linearity of expectation.
Note that the error bound in the above corollary is comparable with (i.e., within a constant factor) that of adding continuous Gaussian noise in the centralized setting, which is ( α − 1) · d · ∆ 22 / (2 τ ) (Mironov, 2017).
Meanwhile, since Algorithm 1 satisﬁes RDP, by Lemmataand 2, it allows the tight privacy accounting in DPSGD through composition and subsampling.
Note that although the above analysis restricts the value of α to 1 < α < 2 nλ/ ∆ ∞ + 1 , this constraint only affects the utility of DSM , not its privacy guarantees.
Further, in practice, n (i.e, the number of training data points) is usually much larger than ∆ ∞ , leading to a large range of values for α .
In addition, the optimal α (order of RDP) is often relatively small (e.g. less than 10) in our experiments, and the above range represents a sufﬁciently large space for tuning α .
Comparing the theoretical results DSM with those of existing solutions described in Section 1, cpSGD (Agarwal et al., 2018) does not satisfy RDP, and does not have the corresponding composition and sub-sampling properties that are crucial for tight privacy analysis.
Meanwhile, DDG (Kairouz et al., 2021) ensures ( α, τ (cid:48) )
-RDP with τ (cid:48)
[Equation]
(cid:17)
[Equation]
(cid:19) when each client independently injects discrete
Gaussian noise of variance σ 2 to her data, where ρ is deﬁned as ρ := 10 · (cid:80) n − 1 k
=1 exp( − 2 π 2 σ 2 · kk +1 ) .
Note that the above τ (cid:48) term increases with the data dimensionality d , which is not scalable for large d .
In contrast, the privacy guarantee of DSM is independent of d .
These advantages of DSM lead to signiﬁcant utility gains in our target application:
FL under differential privacy, described next.
3. F EDERATE L EARNING WITH D ISTRIBUTED S KELLAM M ECHANISM
We apply the proposed DSM described in Algorithm 1 to enforce DP on federated learning with distributed SGD, assuming that the clients have access to a black-box secure aggregation protocol.
Algorithm 2 outlines the training process.
In each iteration, the server releases the current model parameters to all clients (Line 2).
Then, DSM randomly selects a subset of clients, whose identities are not known to the server (Line 3).
Each client in the selected subset then computes the gradients based on the current model weights and her own data (Line 5), and performs gradient clipping (Line 6), which is a standard step introduced in DPSGD to bound the sensitivity of deep learning with
DP.
After that, the client quantizes the gradients (Line 7), as required by typical MPC protocols for gradient aggregation.
Here, a parameter γ controls the quantization granularity: a smaller γ leads to more precise quantization, and vice versa.
Subsequently, the client randomly rounds the gradient to the integer grid Z d (Line 8, explained soon).
Next, we apply the distributed Skellam mechanism (Algorithm 1) to the integer-valued gradients (Line 9).
Finally, the server re-scales the approximate gradient sum by a factor of γ (Line 10) and updates the shared model accordingly (Line 11).
We omit additional details on the updating process (e.g., learning rate schedule, weight decay, etc.) as they do not affect the general framework or the privacy guarantees.
After repeating the above process for
T iterations, the training terminates, and the server obtains the ﬁnal model weights θ .
Next we clarify the conditional rounding step (Line 8 of Algorithm 2).
This step is necessary since after gradient clipping (Line 6) and quantization (Line 7), we still have real-valued gradients, which are not compatible with the MPC protocol (i.e., the black-box secure aggregation protocol A used in
DSM ).
To explain the conditional rounding in Line 8, we ﬁrst deﬁne an element-wise unconditional rounding process as follows: for input x ∈ R , x is rounded to floor ( x ) with probability ceil ( x ) − x , and to ceil ( x ) otherwise.
The problem with this unconditional gradient rounding process is that it introduces an additional O ( d ) term on the L 2 sensitivity of the resulting integer gradients.
To mitigate this issue, Kairouz et al.
(2021) propose to repeatedly run the unconditional rounding algorithm until the L 2 norm of the rounded result satisﬁes a pre-deﬁned condition.
The stopping condition in (Kairouz et al., 2021) is rather complex, and increases the sensitivity of the resulting gradients by a factor of O ( √ d ) .
Algorithm 2 adopts this general idea, but sets a much simpler stopping condition than the one in DDG , as follows: the rounded gradient’s L 2 norm must not exceed a pre-deﬁned constant k (e.g., 5 in our experiments) times C/γ , where C is the L 2 clipping bound in Line 6, and γ is the quantization granularity in Line 7.
With this stopping condition, the increase in L 2 sensitivity in Line 8 is a constant independent of the gradient dimensionality d .
Regarding privacy guarantees of Algorithm 2, observe that each iteration of the training process can be seen as running the distributed Skellam mechanism on a random subset of rounded gradients.
This is because the model sharing (Line 2), gradient sum re-scaling (Line 10), and model updating (Line 11) do not incur any additional privacy loss, as the updated model can be reconstructed by the re-scaled perturbed gradient sum, which, in turn, can be computed from the perturbed gradient sum.
In addition, since the identities of the random subset of clients are not known to the server, the privacy guarantee beneﬁts from ampliﬁcation by subsampling.
(We refer the reader to (Kairouz et al., 2021) for a detailed discussion on this issue.)
Hence, the privacy guarantee of Algorithmfollows by applying the composition lemma (Lemma 1) and the ampliﬁcation lemma (Lemma 2) on the privacy analysis of DSM (Theorem 1).
A formal statement of the privacy guarantees of
Algorithm 2 is presented in Theorem 2 in the appendix.
E XPERIMENTS
We evaluate the performance of the proposed solution DSM (Algorithm 2) on two classic benchmark datasets: MNIST (Lecun et al., 1998) and Fashion MNIST (Xiao et al., 2017), which contains grayscale images of handwritten digits and clothing, respectively.
Both datasets represent 10 -class classiﬁcation tasks with 60 , 000 training data records.
We regard each data record in the training data as a client.
For both tasks, we train a three-layer neural network with fully connected layers and ReLu activation, following previous work (Agarwal et al., 2018).
We set the number of neurons per layers to 80 , resulting in a model with d = 63 , 610 weights.
For DDG and DSM , we use the same L 2 clipping norm ( C = 1 ) and quantization granularity ( γ = 0 . 1 ). For cpSGD , we apply the tighter privacy analysis presented in (Koskela et al., 2021). For all experiments, we use the Adam optimizer (Kingma & Ba, 2015) with learning rate η = 0 .
We do not tune the hyperparameters in favor of any particular solution and omit additional experiments on hyper parameter tuning.
We remark that our approach is compatible with existing differentially private parameter tuning techniques (Gupta et al., 2010; Liu & Talwar, 2019), which is an orthogonal topic to this paper.
Our evaluation uses the ( (cid:15), δ ) -DP
(Deﬁnition 1) deﬁnition instead of RDP (Deﬁnition 3, since a competitor cpSGD supports the former but not the latter.
We ﬁx δ to 10 − 5 , and vary the privacy parameter (cid:15) from { 1 , 2 , 3 , 4 , 5 } , batch size m from { 120 , 240 , 480 , 960 } .
The model is trained for 1 epoch, i.e., when m equals to 120 , 240 , 480 , and 960 , we train the model for 500 , 250 , 125 , and
[Figure] [Figure]
m t e s t acc u r ac y % Fashion MNIST, (cid:15) = 3 ,B ∈ { 12 , 32 } .
nonoiseDPSGD(centralized) Skellam ( B =12) DDG ( B =12) cpSGD ( B =32) (c) Varying m 62 rounds, respectively. For ( DDG ) and our solution DSM , we also vary the bit limit per weight B from { 6 , 8 , 10 , 12 } bits; for cpSGD , we ﬁx B to 32 .
This is because the amount of noise injected by cpSGD is rather large, and smaller values of B lead to very poor model utility.
Note that a large B leads to higher communication costs between the clients.
We report the average test accuracy over 5 runs.
The results are shown in Figures 1 and 2.
In addition, we have also included the non-private baseline and the centralized-DP baseline DPSGD (Abadi et al., 2016b) in the ﬁgures.
Overall, the proposed solution DSM (marked as “Skellam” in the ﬁgures) signiﬁcantly outperforms its competitors cpSGD and DDG on all settings.
In particular, the accuracy improvement over
DDG is consistently around 15% and 10% for MNIST and Fashion MNIST, respectively.
From Figures 1(a) and 2(a), we observe that the performance gap expands as (cid:15) decreases, which corresponds to stronger privacy protection and requires higher noise levels.
According to Figures 1(b) and 2(b), our solution is the only mechanism that achieves acceptable accuracy when the number of bits B is as low as 8 per weight, which corresponds to a stringent constraint on the communication costs in the underlying MPC protocol.
The test accuracy obtained with cpSGD is rather low (no more than 30% ), even though it is given a generous value of B = 32 .
Finally, according to Figures 1(c) and 2(c), the accuracy for all methods remains stable with varying batch size m .
C ONCLUSION
In this work, we propose the distributed Skellam mechanism ( DSM ), a novel solution for enforcing differential privacy on machine learning models built through an MPC-based federated learning process using distributed stochastic gradient descent.
Compared to existing solutions, DSM achieves privacy guarantee that is independent of the dimensionality of the weight vector, while allowing tight privacy accounting due to its nice composition and sub-sampling properties.
We conduct extensive experiments on two classic benchmark datasets and various practical settings, and the result demonstrate the consistent and signiﬁcant accuracy gains DSM over existing solutions.
Regarding future work, we plan to further reduce the constant factor in the privacy analysis for DSM to improve model utility under the same level of privacy protection.
Another promising direction is to open up the black box of the MPC protocol and perform careful privacy analysis with considerations for the details of MPC protocol, which might help lower the noise level further, leading to a more favorable privacy-utility trade-off for federated learning.
R EPRODUCIBILITY S TATEMENT
We provide our source code for our distributed Skellam mechanism implementation for reviewing purposes, available at https://anonymous.4open.science/r/Distributed_
Skellam_Mechanism .
Our implementation simulates federated learning with a single machine.
A A PPENDIX A. P ROOFS
Lemma 4.
Let Q s,t ( v ) =
(cid:81) si =1 v + i + √ ( v + i ) 2 + tt .
Then, I | v | ( t ) I | v + s | ( t ) ≤ Q s,t ( v ) .
Proof.
By properties of the modiﬁed Bessel function of the ﬁrst kind (Andrea & Pierpaolo, 2010), we have inequality
[Equation]
and recurrence relation
[Equation]
which leads to
[Equation]
When v < 0 , substituting v for − v − 1 in the previous formula,
[Equation]
When v ≥ 0 , we also have
[Equation]
Therefore, for every v ,
I | v
[Equation]
Lemma 5.
For any w ≥ 0 ,
[Equation]
Proof.
The inequality holds when w = 0 .
By taking the derivative, it can be veriﬁed that the function e w − ( w + √ w 2 + 1) is increasing with respect to w .
Lemma 6.
For any 0 < a ≤ w ,
[Equation]
Proof.
By Lemma 5, w + √ w 2 + 1 ≤ e w .
Then, by Lemma 5 again,
[Equation]
Lemma 7.
For any 0 < a ≤ w ,
[Equation]
Proof.
By Lemma 5, the inequality holds when w = a .
By taking derivative, it can be veriﬁed that the left hand side is decreasing with respect to w .
Lemma 8.
When − s <
z < s , we have
Q s, 2 λ ( z ) ≤ exp (cid:18) s ( s + z ) 2 λ (cid:19) .
Proof.
Use Lemma 5.
When 0 ≤ z < s ,
Q s, 2 λ ( z ) ≤ s (cid:89) i = exp (cid:18) z
+ i 2 λ (cid:19) =
exp (cid:18) s ( s + 1 + 2 z ) 4 λ (cid:19) ≤ exp (cid:18) s ( s + z ) 2 λ (cid:19) .
When − s 2 ≤ z < 0 , using the equality b + (cid:112) b 2 + (2 λ )2 λ · − b + (cid:112) ( − b ) 2 + (2 λ )2 λ = 1 , we have
Q s, 2 λ ( z ) =
s (cid:89) i =
z
+ i + (cid:112) ( z + i )
+ (2 λ )2 λ = s (cid:89) i = −
z z + i + (cid:112) (
z + i ) 2 + (2 λ )2 λ ≤ s (cid:89) i = − 2 z exp (cid:18)
z
+ i 2 λ (cid:19) =
exp (cid:18) s ( s + 2 z − 1) 4 λ (cid:19)
≤ exp (cid:18) s ( s + z ) 2 λ (cid:19) .
When − s <
z < − s 2 , we have
Q s, 2 λ ( z ) = − 2 z − s − 1 (cid:89) i =
z
+ i + (cid:112) ( z + i )
+ (2 λ )2 λ ≤≤ exp (cid:18) s ( s + z ) 2 λ (cid:19) .
A.
P RIVACY GUARANTEE OF A LGORITHM
Theorem 2 (Privacy guarantee of Algorithm 2) .
Let α ∈ Z , 2 < α < 2 nλγCk + 1 .
Then for L 2 norm clipping bound C , noise parameter λ , quantization granularity γ , conditional rounding parameter k , sampling parameter q , and iterations T , Algorithm 2 satisﬁes ( α, τ ) -R ´enyi Differential Privacy with
[Equation]
where τ l is deﬁned as τ l := 1 . 09 l +0 .
[Equation]
We omit the proof for brevity, which follows from our main theoretical result Theorem 1, as well as subsampling (Lemma 2) and composition (Lemma 1).
A. O VERFLOWING IN M ODULAR A RITHMETIC
The key to avoid overﬂowing is to restrain the perturbed gradient weights within the range of the ﬁnite integer ﬁled, namely [ − 2 B − 1 , 2 B − 1 − 1] .
We ﬁrst review the concentration property of the symmetric Skellam distribution Z ∼
Sk ( λ, λ ) of mean 0 .
Note that Z has MGF exp( λ (exp( λ ) + exp( − λ ) − 2)) .
In addition, since x · (exp( x ) + exp( − x ) − 2) ≤ 1 . 09 x 2 for 0 ≤ x < 1 .
We have that Z is sub-exponential with parameters ( √ 2 . 18 , 1) .
Namely,
[Equation]
for
[Equation]
[Equation]
From here we obtain the tail bound for Z ,
[Equation]
Hence, to avoid overﬂowing due to the Skellam noise, it sufﬁces to set 2 B ≥ h log d , where d is the dimension of the gradient weights, and h is some constant to balance the communication cost and the error due to overﬂowing.
A relatively large h limits the error due to overﬂowing while incurring a large communication cost; and vice versa.
Note that a more precise reasoning can be obtained by observing that Z approaches a normal distribution with mean 0 and variance 2 λ , when
Z is of order √ 2 λ .
In fact, in our experiments, we observe that overﬂows are rare under a reasonable communication constraint.
To be more speciﬁc, no overﬂow was observed when the number of bits per dimension reaches 10 or above, and very few overﬂows occured when the number of bits per dimension is 8 (once every a thousand weights). A. C ONCURRENT WORK Agarwal et al.
(2021) also suggest using the Skellam noise in the distributed DP setting.
Different from our work, their privacy analysis for the Skellam noise is dependent on both the L2 norm and
L1 norm of the private input vector.
Since the L1 norm of a high dimensional vector can be much larger than its L2 norm, the privacy analysis we present is tighter.
Further tightening the analysis is a promising future direction.