Shielding Federated Learning: Aligned Dual Gradient Pruning Against  Gradient Leakage
Authors: Shengshan Hu
Lulu Xue
Ruizhi Zhao
Leo Yu Zhang
Chaowei Xiao
Lichao Sun
Minghui Li
Hai Jin
Venue: NeurIPS 2022 Submitted (2022)

Abstract: Federated learning (FL) is a distributed learning framework that claims to protect user privacy. However, gradient inversion attacks (GIAs) reveal severe privacy threats to FL, which can recover the users' training data from outsourced gradients. Existing defense methods adopt different techniques, e.g., differential privacy, cryptography, and gradient perturbation, to against the GIAs. Nevertheless, all current state-of-the-art defense methods suffer from a trade-off between privacy, utility, and efficiency in FL. To address the weaknesses of existing solutions, we propose a novel defense method, Aligned Dual Gradient Pruning (ADGP), based on gradient sparsification, which can improve communication efficiency while preserving the utility and privacy of the federated training. Specifically, ADGP slightly changes gradient sparsification with a stronger privacy guarantee. Through primary gradient parameter selection strategies during training, ADGP can also significantly improve communication efficiency with a theoretical analysis of its convergence and generalization. Our extensive experiments show that ADGP can effectively defend against the most powerful GIAs and significantly reduce the communication overhead without sacrificing the model's utility.

FL claims Shielding Federated Learning: Aligned Dual Gradient Pruning Against Gradient Leakage
Anonymous Author(s)
Affiliation
Address email
Abstract
Federated learning (FL) is distributed learning framework that claims to protect user privacy. However, gradient inversion attacks (GIAs) reveal severe privacy threats FL, which recover the users' training data from outsourced gradients. Existing defense methods adopt different techniques, e.g., differential privacy, cryptography, and gradient perturbation, against the GlAs. Nevertheless, all current state-of-the-art defense methods suffer from trade-off between privacy, utility, and efficiency in FI.. To address the weaknesses of existing solutions, we propose a novel defense method, Aligned Dual Gradient Pruning (ADGP), based on gradient sparsification, which can improve communication efficiency while preserving the utility and privacy of the federated training. Specifically, ADGP slightly changes gradient sparsification with stronger privacy guarantee. Through primary gradient parameter selection strategies during training. ADGP can also significantly improve communication efficiency with theoretical analysis of its convergence and generalization. Our extensive experiments show that ADGP can effectively defend against the most powerful GIAs and significantly reduce the communication overhead without sacrificing the model's utility.
2
4
6
10
11
12
13
14
15
Introduction
18 Federated learning (FL) [ is distributed learning framework, where multiple users train and send
19 their gradients of the local models the server without sharing their local data 1 3.
20 to protect the users' training data since the users do not need to share local data with the server directly. However, recent studies reveal that gradients can be used recover the original training data information via gradient inversion attacks (GIAs) 4 5 To against GIAs, large number of studies have been proposed, where they leverage the advanced privacy protection techniques, such as
21
23
24 differential privacy (DP) (6]. cryptography 7
and gradient perturbation [TO, [1 [2]. However, 25 none of the cxisting defense methods could take care of all privacy, utility, and efficiency difficulties in FL. For example, DP and cryptography-basee based methods could effectively defend GIAs, but sacrifice
8 9
27 either the utility or efficiency respectively [6] 7 g 99. In order to achieve better utility and efficiency
in FL. perturbation-bee basedd methods design various gradient perturbations [10] 12), but all existing perturbation- based methods could only defend one or two kinds of GLAs in practice.
29
30 Fox example, recent perturbation-baut
defense methods (i.e., Precode 112, Soteria [10], and 13 14 15), but fail work against the active GIAs [16, [7 [8] On the contrary. the classic Top-k based gradient sparisification
31 ATS (TI) can effectively defend against optimization-aaseeee GIAs [5]
33 method [19 20 is generally considered as a bad privacy protection solution on optimization-batt based GLAs, but in fact performs much better than recent defense methods under the active attack from our experiments as shown in Table 2 The new findings inspire us to seek for a more practical perturbation-based defense against both optimization-bai based and active GIAs.
35
Submitted to 36th Conference on Neural Information Processing Systems (NeurIPS 2022). Do not distribute.

---

Table I: Comparison of our method with existing privacy- FL methods. Note: represents
the scheme has high guarantee for the property, while X represents otherwise.
Privacy
Defense
Analytical attack
Opumization attack
Active server attack
Utility
Efficiency
R-GAP 22
DLG
IVG
Curious [66
Rob [2]
Precode 12
K
ATS [
X
X
XXX
X
Soteria 10
X
X
DP 23
Top-k 19
X
ADGP (Ours)
37 In this paper. we propose new gradient pruning based method, Aligned Dual Gradient Pruning (ADGP). Specifically, ADGP consists of two components: dual gradient pruning (DGP) and gradient location hounding. Dual gradient pruning is novel gradient sparsification technique, which removes top-kı largest gradient parameters and the bottom-k2 smallest gradient parameters from the local model. DGP leads to strong privacy protection against both optimization-basee and active GIAs.
38
39
40
41
42 To further reduce the expensive download costs caused by the asymmetric gradient selection among different users, we propose gradient location bounding strategy to make the aggregated global gradient stay the same sparsified region. In summary, ADGP achieves a better utility and privacy trade-off, increases FL system efficiency, and is robust against active attacks.
44
46
Furthermore, we give the theoretical analysis of ADGP, which proves the reconstruction error is proportional to gradient distance. So removing larger gradient parameters could enlarge the gradient distance resulting in low reconstruction error. However, removing larger gradients will significantly impact the model's utillty. Thus, to improve the sparsification ratio, which is essential to robustness against active attack way, our method could significantly mitigate GIAs without affecting the model's utility.
47
48
49
50
we also remove the model parameters with smaller gradients. In such a
51
52
We conduct extensive experiments over multiple datasets and models to evaluate our method. The quantitative and visualized results show that our design can effectively make recovered images recognizable under different attacks, and reduce nearly half of the communication costs. Our contributions are as follows: 1) We revisit gradient sparsification to show its potential on mitigating GIA; 2) We propose an improved gradient pruning strategy provide sufficient privacy guarantee while balancing the model accuracy and the system efficiency; 3) We conduct extensive experiments
53
54
55
56
57
58 to show that our design outperforms perturbation-ben defense methods w.r.r privacy protection, model accuracy, and system efficiency.
59
2
Related work
61
Federated learning || B
is considered to be a privacy--reserving framework for distributed machine g as the training data is not directly outsourced. However, the emerging of GIAs [4 5 21 16
62
learning
22
26 27 shatters this conception. It has been proved that the attacker (e.g., curious server) can easily recover the private training data from gradient information to a great extent. The privacy guarantee of federated learning urgently needs to be strengthened.
24
25
64
86
Cryptographic Defense. Traditionally, there are two approaches to construct
28 29 RD) or using cryptographic tools to perform secure aggregation 77 & (1. DP 66 is a popular and effective privacy protection mechanism by adding random noise the data, but it is well known that the noises intro- duced by DP can greatly degrade the model accuracy when meaningful privacy is enforced [33]. Eryptographic-basee secure aggregation can guarantee both privacy and accuracy simultaneously, but incurs expensive computation and communication costs [4). Using the shuffle model [35] 36 can only provide anonymity. Moreover, it totally changes the system model of FL since an additional semi-trusted third party is introduced to work cooperatively with the server.
federated learning: using DP to disturb gradients 66 23,
88
70
71
72
74
Gradient Perturbation Defense. Recently, researchers have begun to explore the possibility of constructing new gradient perturbation mechanisms to better balancing privacy and accuracy. Sun et
76
al. [10] proposed Soteria, scheme that perturbs the representation of training data by pruning the

---

78
gradients of a single fully connected layer. Gao et al. proposed ATS, training data augmentation policy by transforming original sensitive images into alternative inputs, reduce the visibility of reconstructed images. Scheliga et al. [12] presented Precode to extend the model architecture by using variational bottleneck (VB) [37 to prevent attackers from obtaining optimal solutions to reconstructed
79
81
data. These defenses work well against GIAs in the semi-honest setting [4] 5 38 [3], but fail to
88
protect privacy when an active server modifies the model to launch GIAs [21 [6]. Moreover, these works suffer from high computation costs or huge communication burden.
84
85
Gradient Sparsification Defense. From an independent research domain, gradient sparsification has been commonly used for saving communication bandwidth. The most common sparsification strategy Top-k selection, which selects top k gradient parameters with the largest absolute values It has been widely proved that gradient sparsification provides very limited privacy protection
86
87
88
19 20
89
ability T0 1 12 39 unless high pruning ratio (e.g., removing 99% of the gradients) used
90
at the cost of 10% accuracy drop [39]. However, we emphasize that this is misunderstood as they only consider the Top-k sparsification strategy that has never received an in-depth investigation the field of security. It is originally designed for improving system efficiency, thus a direct application inherently suffers from many weaknesses. As shown in Section 4 a slight modification can unleash the potential of gradient sparsification to provide strong privacy guarantee.
91
93
3 Threat Model and Attacks
96 In this work, we consider a strong threat scenario, where an active server, after receiving gradients from users, tries to reconstruct the local training data and is motivated to modify model parameters in each iteration to strengthen the attack performance. Note that the server also wants to obtain a high-quality global model with high accuracy. More specifically, consider the following three kinds of GIAs:
96
100
101
Analyticn tical attack. Analytical attack exploits the structure of the gradients to recover the input, such as using gradient bias terms [40] Recently proposed R-GAP attack [22] exploits the recursive relationship between gradient layers to solve the input. An effective analytical attack depends on the
102
103
104
specific structure and parameters of gradients.
105
Optimization attack. Optimization attack is first proposed using L-BFGS optimizer to solve
min (((x))) 01(x" 122 and gets dummy data x" and dummy label y where y is the label of WW
107
x
44)
The state-of-th of-the-art optimization attack method IVG [5) uses Adam to optimize the cosine
distance and has been widely used to evaluate defense works [10 [2).
109
15]. the existing attacks are all measured by the distance between the generated gradients VW* and the original gradients V. We therefore propose a general definition for optimization attack to better evaluate
Despite different optimizers can be used achieve better attack quality 1
110
111
112
its performance. As shown in Definition [ smaller indicates a stronger optimization attack.
113
Definition 1. An optimization attack is a (e, -a-ttackk,, if it satisfies:
P(E ((((D(w,,w-)) <) 1- 8.
(1)
where IP represents the probability, E represents the expectation, D is the distance function commonly instantiated with Euclidean or cosine distance.
115
116
Active server attack. In this kind of attack, the server can actively modify the global model to
117
realize a better attack result rather than honestly executing the protocols [16 17 18). Recently proposed Rob attack [2I] adds imprint modules to the model and uses the difference between the gradient parameters in adjacent rows of the imprint module to recover the data, achieving the best attack effect in the literature.
119
4
Aligned Dual Gradient Pruning
122
4.1
Analysis of Gradient Sparsification
123 We owe the failure of common Top-k gradient sparsification methods to two reasons: 1) the distance between the Top-k sparsified (i.e., perturbed) gradient g and the real gradient VW is small; and 2) large gradient parameters in V W also reveal label information about user data.
124
3

---

128 To explain the first reason, we investigate the relationship between the reconstruction error of user data and distance of perturbed gradient v.s. real gradient, as shown in Proposition !
127
128
Proposition 1. For any given input x and shared model W. the distance between the recovered data
129
x' and the real data x is bounded by:
VW gi2
XX Xx-x122 1122 110p(xx W)/Ox
(2)
130
where up is the mapping from x to VW, ie., the reconstruction quality is limited by VW
g
131
The proof of the above proposition is moved to the supplementary due space limit (the same hereinafter). From this Proposition, it is clear that the reconstruction error is proportional to the
132
133
gradient distance ||VW- gll2, i.e., effective defense methods should enlarge the gradient distance
134
as much as possible. However, for the Top-k based gradient sparsification [19 20), the & largest parameters are retained, making the gradient distance small by nature.
135
136
To explain the second reason, we consider a L-layer perceptron model trained with cross-entropy loss for classification. Let column vector = [ra, r2..... I'm be the logits (the output of the L-th linear layer) that input to the softmax layer, the confidence score probability vector is thus
138
139
Eye's En e'
e"n { Σje²
and the succinct form of the cross-entropy loss becomes {(x, y)
log( Eje") (, ). Focus on the L-th layer W x + b² -r, it is easy to find
140
01(x,))
dl(x. v) Ôr: 06(x,)) 8) Ōr;
exc
Limys
(3)
bb;
Ob.
Ori
Ejer
141 and
al(x. v)
al(x. v)
Ol(x, y)
VW
xT =
Xx.
(4)
Ör
dr1
Or.
For given x (and so x" is fixed), the magnitude of certain elements of the gradient matrix VWL (i.e., the i-th row) is particularly large if i is the true label of the training data x due to reason that 06(x.y) dr.
142
143
E Ei
06(x.y)
Dr3
To summarize, due to the above two reasons, conclude that common Top-k gradient sparsification cannot provide sufficient protection for user data against passive optimization attacks. From another point of view, a sufficient gradient sparsification ratio also plays an important role in defending against active server attacks. As mentioned in Section 3 active attackers can exploit the correspondence of partial gradient parameters recover the real data. So, the gradient sparsity rate will directly destroy the relationship among gradient parameters constructed by the active attacker. Intuitively, the higher the sparsity rate, the more severe the impact. As will be validated in Section 6 higher sparsity rate can prevent the attacker from obtaining useful gradient information.
146
148
150
151
152
153
4.2
Dual Gradient Pruning
154
Generally speaking, large gradient parameters of local model need to be removed to make the gradient difference larger, but the difference should also be appropriately bounded to maintain high model accuracy. Moreover, it is also necessary to delete small gradient parameters to achieve a high sparsification ratio. With these observations, we propose dual gradients pruning (DGP), new parameter selection strategy for gradient sparsification.
155
156
157
158
The users first sort the absolute values of all Size (VW) local gradient parameters in the descending order. Let Tx, (VW) represent the set of top-k, elements of V W,, BR, (VW) represent the set of its bottom-kg elements. Then the users remove Tx, (V W) and Bx (VW) from VW for gradient sparsification. Note that we set = h as a hyperparameter to regulate the trade-off between privacy and accuracy. Clearly, even with fixed value p. different user will have different sets of
160
162
164 Te, (.) and Br, (-) because their respective local models could be different from each other.
165
We emphasize that although such dual gradients pruning strategy is very simple, it can significantly mitigate GIAs without affecting the model accuracy. A rigorous security proof is shown in Section 5 and experimental results can be found in Section
186

---

Algorithm 1: Aligned Dual Gradient Pruning (ADGP)
Input :Original gradient matrix VW, location binary matrix I, values of kn and l: Output :Spassifeed gradient matrix = (g":""""")
for I < to L do
Remove parameters in Thy (VW") from VW
Keep parameters in VW when location Z, and discard all other parameters Upload g' Tk ( w') to the server
Algorithm 2: A Complete Illustration of our Defense
Input Initial global model w value k and k1, total rounds T, total users N :Shared :Shared global model V
Set e 0
for / < 0 to T do Randomly select user broadeast the location matrix I' of its parameter Tex
for i + to N do
P W + of
ADGP ky k, I.
P g
Sever side aggregation: W W
x
168
4.3
A Complete Illustration of Our Method
169
Although dual gradient pruning provides sufficient privacy guarantee well reduces upload cost of users, users' download costs could still be expensive. This is because different users have different sets of Tk, (-) and B&, (+) when sparsifying their own local gradients, which will ultimately make the global gradient become dense after aggregation.
170
171
172
We thus propose aligned dual gradient pruning (ADGP), improved scheme align the selected gradients across different users. Similar to DGP, for best privacy, each user will still firstly identify his top-kı gradients location set Tk Different from DGP, ADGP also wants save users' download cost by ensuring that all users' uploaded sparsified gradients reside in the same location set. This is achieved by randomly selecting user, who identifies top-2k (ki k) location set T2x (represented with a binary matrix Z) and broadcasts I to all other users. Note that Tk. C Tex is not necessary true. Upon receiving Z, each user first discards gradient parameters Te, and then only transmits the k largest gradient parameters whose locations belong to I. After aggregation at the server side,
174
175
176
177
178
179
180
181
users only need download global gradients' parameters associated with I. A detailed illustration
182
of ADGP is shown in Algorithm [
For ADGP pruning, in each FL iteration round, all gradient parameters whose locations outside of will not participate the current round global model aggregation. In the extreme case, can remain static for all iteration rounds and the local accumulated error (accumulated unused local gradient parameters) becomes large, thus hindering global model convergence. To reduce this negative impact and increase convergence speed, design error feedback mechanism. In particular, the iteration round t, after user i obtaining his local gradient VW! he will combine VW; with an error term accumulated in the previous (t 1) rounds before performing the ADGP sparsification pruning. A complete illustration of our method is shown in Algorithm
194
185
186
187
188
189
190
Theoretical Analysis
191
192
This section presents the security analysis with regard to passive attacks (i.e., analytical and opti- mization attacks presented in Section 3. as well as the generalization and convergence analyses of the proposed ADGP algorithm. Following the literature studies 41 for a given L--layer centralized model, model the first (L- 1) layers robust feature extractor of any input sample.
194
5

---

196
Thus, the function of this model is characterized by fxx W) = Wz + b, and the optimization objective is the loss { 8) (such as cross-entrope or L2 loss). To facilitate analyses and following
197
198
literature studies [9 41 43 44 the assumptions about the smoothness of DGP, ADGP and 1, as
199
well as the variance of the stochastic gradient are employed.
Assumption The pruning mechanisms DGP (k1, k2, VW*) and ADGP( k. VW* II') are both
bi- bi-Lipschitz, so the following conditions hold:
IVW- - DGP( k2, VW)Il = ||DG((O, O, VW) - DGP(M), h2 VW) 2 n VWI..
VWW- ADGP(k,, k, VW I S V2ll
where 71 is constami determined by k1 and liz, and Y22 is constant determined by kn and k.
Assumption 2. The objective function : R - R has low bound 1" and it is Lipschitz-smooth i.e.
201
for any 21, 22. [[VI(z1) V1(x2))l22 S K|| - z2ll2 and 1(x1) S (1x22) + (V((x2), 21 x2)
202
203
2 21 x2
Assumption 3. The full gradient VI(W") is bounded, i.e. )1 < G², and the federated stochastic gradient VW: (i= = [1, NI) is the unbiased estimation of the full gradient VI( i.e.
204
B(VW:)- = VI(W"). Moreover, the variance between V and VI(W')& is bounded: E| | W VI(W 02
206
207
208
Security Analysis. It is noted that, for the same sparsification ratio, user's uploaded gradient parameters from ADGP is generally smaller than that from DGP. Indeed, the uploaded gradient parameters from both methods are the same only when Tk, C T2x holds. From this observation and [ DGP is the security lower bound of our design for privacy protection. So, our focus is the security analysis of DGP. As shown in the theorem below, we prove that DGP achieves a stronger privacy protection in the sense of Definition I
209
210
211
referring to Proposition
212
213
214
Theorem 1. For any (5,6) optimization attack, under the presence of DGP, it will be degenerated
to (E + v Tal /W1,, d) -attackk if D is measured by Euclidean distance, and degenerated to (1
215
216
(T E), 6) -attack if D is measured by cosine distance.
217
The Theorem is based on Assumption I about DGP. It reveals that, with the same successful chance 1 6, DGP weakens the attacker's capability to optimize better estimation of the true VW.
218
219
Generalization and Convergence Analyses. The generalization analysis aims to quantify how the trained model performs on the test data, and it is achieved by analyzing the how ADGP affects the properties of the optima reached (without gradient pruning) 42). Assisted with Assumption and Assumption abbout ADGP gradient pruning, the following Lemma can be obatined.
220
221
222
Lemma 1. Let e' = En e be the averaged accumulated error among all users at iteration t, the expectation of the norm of e' is bounded, i.e.,
223
224
Elee'1 11
<
12
2 2 7/2 (2 2 + 2
(5)
1- 72
Note that the difference between the averaged pruned gradient g' = ΣN' g//N N and the averaged
225
Fed-SGD gradient VW = Σmn VW is simply || I (VW' - g/)l = ||. So the lemma above indicates that the accumulated gradient difference between our algorithm and Fed-SGD is bounded. That said, the optima reached by ADGP and the optima reached by Fed-
226
227
228
229
SGD will eventually be the same if the algorithm converge. Armed with Lemma Tand based on D and 3 we demonstrate the convergence of the our algorithm.
230
Assumptions
231
Theorem 2. The averaged norm of the full gradient VI(W*) derived from centralized training is
232
correlated with the our algorithm as follows:
ET- E|
10 1-
47 K2
22
12
2 2+
22 +02) 2Kn +
0² N
(6)
T
nf
2 1 12
233
where ( is the initialization of the objective I, and 1) is the learning rate.
234
The immediate implication of Theorem is that, with an appropriate learning rate y, ADGP converges similar to Fed-SGD (slower by negligible term O ), shown in Corollary |

---

Table 2: Evaluation of defense performance under three attacks.
Attack
Metric
Baseline
Precode
DP
Soteria
ATS--
ATS-11
Top-
Ours
PSNR (
34,8805
9.6441
6,9554
9.2447
16.6894
31,3200
14.1338
7.6192
IVG
LPIPS(())
0.0016
0.4473
0.5504
0.3774
0.1621
0.0015
0.2754
0.4829
SSIM (L)
0.9273
0.4793
0.2451
0.4173
0.6851
0.9189
0.5336
0.2923
PSNR (
36.7656
5,0691
5.1817
10.8442
42,0900
5.1017
5.1196
R-GAP
LPIPS(T)
0.0007
0,3621
0.3532
0,2094
1.8c-05
0.4817
0,4863
SSIM (L)
0.9307
0.2483
0.2124
0,3962
0.9121
0.2027
0.1928
PSNR (L)
102.8838
109.6553
8.7491
102.8838
9.6166
115.9886
13.0685
13.0804
Rob
LPIPS(T)
0.0960
0,1488
1.3434
0.0960
0.6410
0.0486
0.8920
0.9184
SSIM (
0.8969
0.8440
0.2064
0.8969
0.2545
0.9490
0.0428
0.0229
Final model accuracy
93,4400
93.1699
86.8900
93.2300
93.3900
93.3500
93.20999
93.1700
Original
Baseline
Original
Soteria
Top-k
A
(a) R-GAP, CNN6
(b) IVG, LeNet (Zhu)
Figure 1: Visualization of the reconstructed data under R-GAP and IVG attacks.
236
Corollary 1. Let KT
10-1- we have
E VI(W")
10 - 1-
1==
+
T
KT + o2/
T
6 Experiments: Privacy-Accurac--fffiiinng Tradeoff
238
6.1
Experimental Setup
Datasets and models. We conduct experiments CIFARIO with LeNet (Zhu) (. CIFAR10 45
240
with CNN6 and CIFAR100 [45] with LeNet (Zhu) and ResNetl8 respectively. We run these experi- ments in a pytorch environment by using single RTX 2080 Ti GPU and 201GHz CPU.
242
Evaluation Metries. We quantify the privacy effect of defenses, follow 46). we visualize the reconstructed data and use learned perceptual image patch similarity (LPIPS), peak signal-to-noise ratio (PSNR), structural similarity (SSIM) to measure the quality of the recovered data. A better defense scheme should has larger LPIPS (f), smaller peak signal-to-noiss ratio (PSNR) (L) and structural similarity (SSIM) (L).
244
246
A ttack methods. We evaluate our defense against IVG attack (. R-GAP attack (2). and Rob attack (21). which represent three kinds of state-of-the-arr GIAs, as illustrated in Section 3 All these attacks are implemented strictly following the original settings, iee, IVG is evaluated on CIFARIO with LeNet (Zhu), R-GAP evaluated on CIFARIO with CNN6, Rob attack is evaluated on CIFARIO0 with LeNet (Zhu). More settings for attacks shown in the supplementary.
248
250
252
Defense methods. We compare our method with five state-of-the-art defenses: Soteria 1O
253
ATS [T]. Precode [2]. Differential Privacy (DP) [2), and T based gradient sparsification Besides, we set Fed-SGD [3] as the baseline that adopts no defenses. Following the DP design
19
254
255
in 2] we use the Gaussian differential privacy mechanism with E = 10.7, 5 = 10 which is the suggested best setting for the privacy-accurace trade-off and can make most models converge. When
256
257
quantifying the defense performance of ATS, we not only evaluate the similarity between the raw images and the recovered data (ATS-I), but also evaluate the similarity between the disturbed training
258
Hereinafter, we abuse the notion of & to denote the send rate (k/Size( VW)) x 100% since it will not cause ambiguity. And the sparse ratio is 1 1-k. The smaller the ratio is, the better communication efficiency.
7

---

Original
Acc (%)
Top-k
Briding CRA
DOP:CIFAN2
Bustine CRATTO
CIFARCO
DGP
60
100
Epoch
(a) Recovered images under IVG attack with ResNet18 on CIFAR100
(b) ResNet18 on CIFAR dataset
Figure detailed comparison between Top-k and our DGP on privacy and model accuracy.
Original
Baseline
Soteria
Precode
ATS
Top-k
Ours
DP
Figure 3; Visualization of reconstructed images under Rob attack with batchsize=16.
259
images (i.e., the real inputs) and the recovered data (ATS-II), We set the send rate l = 0.2 and the regulation hyperparameter p 15. The supplementary gives more experiments under different p and
280
k. The rest defense schemes remain the original settings.
6.2
Defense Performance Evaluation
263
Table
shows the defense performance with PSNR, SSIM, and LPIPS under three attacks. The results show that ATS, Soteria, Precode perform poorly under Rob attack, while Top-k is vulnerable IVG attack although it performs better under Rob attack. In most cases, our scheme performs comparably with DP and outperforms all the other defenses. More evaluation results under Rob
265
266
267
attack are presented in our supplementary.
288
We also visualize the reconstructed images in order to perceptually demonstrate the defense per- formance. Figure [(a] shows the the recovered images against R- R-GAP and IVG attacks. We can see that all the existing defenses can well defend against R-GAP attack except ATS because it does not damage the gradient structure, proving that slight perturbation on gradients can mitigate the analytical attacks easily. We are not able to provide the result of Precode because its VB operation destroys the model structure thus analytical attack R-GAP cannot implemented. In Figure (b) recovered images under IVG attack are presented. We can find that the attacker can still recover the outline of inputs with ATS and Top-k defenses. DP, Soteria, Precode, and our scheme can still make the recovered images unrecognizable. Figure evaluaaees the defenses against Rob attack. It shows that ATS, Soteria, and Precode fail to work and most inputs can reconstructed.
269
270
271
272
273
275
277
278
In Rob attack, the attacker uses the gradient of the imprint module to reconstruct the training data. Our method, Top-k, and DP can effectively defend against Rob attack because the gradients of all layers are sparsed or perturbed, including those of the malicious imprint modules. However, we emphasize that the main weakness of Top-k its vulnerability to optimization attacks (e.g.. IVG), as widely demonstrated in the literature 4 IO] 11 39 12]. We thus further evaluate Top-k
279
280
283
and our scheme under IVG attack with ResNet18 on CIFAR datasets. We set / (V))
0.05, hs /Szze/zzee (W) = 0.75.
6.3
Model Accuracy Evaluation
286 To evaluate model performance, we train ResNet18, LeNet (Zhu), VGGI3 47 on CIFAR1O.
CIFAR 100 with ten users, respectively. We set epoch=100, the learning rate 7=0.1 1 if epoch 50,

---

ACc 196
ACC (96
Acc (96)
Epoch
Epoch
Epoch
(a) CIFARIO, ResNet18
(b) CIFARIO, LeNet (Zhu)
(c) CIFARIO, VGG13 bn
Acc (%)
Ace
Epoch
Epoch
Epoch
(d) CIFAR100 ResNer18 (c) CIFAR100, LeNet (Zhu) (f) CIFAR100, VGG13_bn
Figure 4: Evaluation of model accuracy with different datasets and model architectures.
Table 3: Commu. cost in one iteration (MB)
Method
Baseline
Soteria
Precode
ATS
DP
Top-k
Ours
Resnet18
85,2506
85.2268
88.1644
85,2506
85,2506
43.7979
27.3067
VGG13
71.8385
71.7318
74,8433
71.8385
71,8385
34.9625
22.9697
LeNet
0.1207
0.0764
1.8624
0.1207
0.1207
0.0493
0.0424
288
0/001 epoch >50, and yp0.05 if epoch >70. We show in Table the accuracy of ResNet18 over CIFAR1O under different defenses, and here only compare scheme with the baseline Fed-SGD, DP, and Top-k since they perform best for privacy protection. Because [41 showed that the error feedback is beneficial improve the model accuracy, even without using gradient sparsification. To give a fair comparison, we set the error feedback mechanism as the basic setting for all the defenses. The experimental results in Figure show that achieve similar model performance with the baseline, while DP, as expected, sigmificantly damage the model accuracy.
290
292
295
6.4 Efficiency Evaluation
To clearly demonstrate the system efficiency, evaluate the communication cost, which is obtained by computing the total overheads of sending updated gradients and receiving aggregated gradients. For ease of presentation, we only show the results for one iteration. As shown in Table 3 our scheme reduces more than half of the communication costs compared with existing defenses, and our gradient sparsification incurs negligible computation burden. The specific computation cost evaluation is presented in the supplementary.
287
299
300
7
Conclusions, Limitations, and Broader Impact
302
Our work firstly reveals the risks of privacy-preserving methods that only perturb the gradients of some layers. Through comprehensive analysis of gradient inversion attacks, we show that it is necessary to perturb sparse the gradients of each layer for privacy preservation. And considering the challenge of high communication cost federated lcarning, we propose aligned dual gradient sparsification method to achieve the trade-off between privacy protection, model performance, and efficient communication, and give sufficient theoretical support. We hope that our newly proposed gradient sparisification method can shed new light addressing privacy leakage concern well as saving communication bandwidth.
304
306
308
310
In terms of limitations, the success of our scheme relies on selecting reliable user to broadcast its gradient locations. Randomly selecting users may encounter malicious users that destroy the entire system. Our design is delegated to protecting privacy and has negative socictal impacts in practice.
312

---

References
314
315
[1] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. Arcas, "Communication-cfficcan learning of deep networks from decentralized data," in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS' 17), 2017, pp. 1273-1282.
317
318
[2] M. Naseri, J. Hayes, and F. De Cristofaro, "Local and central differential privacy for robustness and privacy in federated learning." in Proceedings of the 29th Network and Distrihuted System Security Symposium (NDSS 2022.
319
320
321
[3] Q. Li, Z. Wen, Z. Wu, S. Hu, N. Wang, Li, X. Liu, and B. He, "A survey on federated learning systems: vision, hype and reality for data privacy and protection," IEEE Transactions on Knowledge and Data Engineering. 2021.
322
323
[4] L. Zhu, Z. Liu, and S. Han, "Deep leakage from gradients," in Advances in Neural Information Processing Systems (NeurPPS'99) 2019, pp. 14747-14756
325
326 [5] J. Geiping, H. Bauermeister, H. Dröge, and M. Moeller, "Inverting gradients-how easy is it to break privacy in federated learning?" in Advances in Neural Information Processing Systems (NeurlPS'20) 2020, pp. 16937-16944 947.
328
329
[6] C. Dwork, A. Roth el al., "The algorithmic foundations of differential privacy," Theor. Comput. Sci., vol. 9, no. 3-4, pp. 211-4077 2014.
Found. Trends
330
331
[7] R. Bachhaah,c K. Laine, K. Lauter, Rindal, and M. Rosulek, "Secure data exchange:
332
A marketplace in the cloud," in Proceedings of the 2019 ACM SIGSAC Conference on Cloud Computing Security Workshop (CCSW' 19), 2019, pp. 117-128.
333
334 [8] S. Hardy, W. Henecka, H. Ivey-Law, R. Nock, G. Patrini, G. Smith, and B. Thorne, "Private fed- erated learning on vertically partitioned data via entity resolution and additively homomorphic encryption." arXiv preprint arXiv: 1rXiv:1711..1661 2017.
336
337
[9] K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth, "Practical secure aggregation for privacy-preserving machine learning," in Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS' 2017, pp. 1175-1191.
338
339
340
341
[10] Sun, A. Li, Wang, H. Yang, H. Li, and Y. Chen, "Soteria: Provable defense against privacy leakage in federated learning from representation perspective," in Proceedings of the 202 I IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'21), 2021, pp. 9311-9319.
342
343
344
[11] W. Gao, S. Guo, T. Zhang, H. Qiu, Y. Wen, and Y. Liu, "Privacy-preservve collaborative learn- ing with automatic transformation search," " in Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'21), 2021, pp. 114-123.
346
[12] D. Scheliga, P. Mäder, and M. Seeland, "Precode-a generic model extension to prevent deep gradient leakage," in Proceedings of the 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (W)CV'22), 2022, pp. 1849-1858.
349
350
351
[13] W. Wei, L. Liu, M. Loper, K.-H. Chow, M. E. Gursoy, S. Truex, and Y. Wu, "A framework for evaluating gradient leakage attacks in federated learning," arXiv preprint arXwv:2004.10397, 2020.
352
353
354
[14] Y. Wang. J. Deng, D. Guo, C. Wang, X. Meng, H. Liu, C. Ding, and S. Rajasekaran, "Sapag: self-adaptive privacy attack from gradients," arXiv preprint arXiv:2009.063: 06228, 2020.
[15] M. Balunović, D. I. Dimitrov, R. Staab, and M. Vechev, "Bayesian framework for gradient leakage," arXiv preprint arXiv:2111.0470 2021.
357
358
[16] F. Boenisch, A. Dziedzic, R. Schuster, A. S. Shamsabadi, I. Shumailov, and N. Paper-
359
not, "When the curious abandon honesty: Federated learning is not private," arXiv preprint arXiv:21 12.0229,, 2021.
380
361
[17] X. Pan, M. Zhang, Y. Yan, J. Zhu, and M. Yang, "Exploring the security boundary of data reconstruction via neuron exclusivity analysis."
362
363
[18] Y. Wen, J. Geiping, I. Fowl, M. Goldblum, and T. Goldstein, "Fishing for user data in large- batch federated learning via gradient magnification," arXiv preprint arXiv:2202.00S: 2022.
10

---

385
[19] D. Alistarh, T. Hoefler, M. Johansson, N. Konstantinov, S. Khirirat, and C. Renggli, "The convergence of sparsified gradient methods," in Advances in Neural Information Processing Systems (NeurIPS' 18). 2018, pp. 5977-5987.
386
367
388
[00] Y. Lin, S. Han, H. Mao. Y. Wang, and W. J. Dally, "Deep gradient compression: Reducing the communication bandwidth for distributed training." arXiv preprint arXiv: 1712.01887 2017.
369
370
[21] L. J. Geiping, W. Czaja, M. Goldblum, and T. Goldstein, "Robbing the fed: Directly obtain- ing private data in federated learning with modified models," arXiv preprint arXiv:21 10. /3057, 2021.
371
372
373
[22] J. Zhu and M. Blaschko, "R-gap:
Recursive gradient attack on privacy," in Proceedings of the 9th International Conference on Learning Representations (ICLR'21), 2021.
374
[23] R. C. Geyer, T. Klein, and M. Nabi, "Differentially private federated learning: A client level perspective." " arXiv preprint arXiv:/712.07533 2017.
376
377
[24] B. Zhao, K. R. Mopuri, and H. Bilen, "idlg: Improved deep leakage from gradients." arXiv
378
preprint arXiv:2001.022 1006610, 2020.
379
[25] Qian and L. K. Hansen, "What can we learn from gradients?" arXiv preprint
arXiv:2010. 15718, 2020.
381
[66] H. Yin, A. Mallya, A. Vahdat, J. M. Alvarez, J. Kautz, and P. Molchanov, "See through gradients: Image batch recovery via gradinversion." " in Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Patlern Recognition (CVPR'21), 2021, pp. 16 16337-16344 346.
382
383
384
[27] L. Fan, K. W. Ng, C. Ju, T. Zhang, C. Liu, C. S. Chan, and Q. Yang. "Rethinking privacy preserving deep learning: How to evaluate and thwart privacy attacks," in Federated Leurning. Springer, 2020, vol. 12500, pp. 32-50.
386
387
[28] X. Chen, Z. S. Wu, and M. Hong, "Understanding gradient clipping in private sgd: A geometric perspective," " in Advances in Neural Information Processing Systems (NeurIPS'20 20), 2020, pp. 13773-13788 782.
388
389
[29] M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, "Deep learning with differential privacy." in Proceedings of the 2016 ACM SIGSAC conference on computer and communications security (CCS 16), 2016, pp. 308-318.
391
392
393
[30] L. Yu, L. Liu, C. Pu, M. E. Gursoy, and S. Truex, "Differentially private model publishing for deep learning," " in Proceedings of the 2019 IEEE Symposium on Security and Privacy (SP'19), 2019, pp. 332-349.
394
395
396
[31] P. Mohassel and Y. Zhang, "Secureml: A system for scalable privacy-preserving machine learning." in Proceedings of the 2017 IEEE Symposium on Security and Privacy (SP'17), 2017,
398
pp. 19-38.
399
[32] G. Danner and M. Jelasity, "Fully distributed privacy preserving mini-batch gradient descent learning," in Proceedings of the 15th International conference on distributed applications and interoperable systems (IFIP')5) 2015, pp. 30-44.
400
401
402
[33] K. Wei, J. Li, M. Ding, C. Ma, H. H. Yang, F. Farokhi, S. Jin, T. Q. Quek, and H. V. Poor, "Federated learning with differential privacy: Algorithms and performance analysis," IEEE Transactions on Information Forensics and Security, vol. 15, pp. 3454-3469, 2020.
403
404
405
[34] P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, 7. Charles, G. Cormode, R. Cummings el al., "Advances and open problems in federated learning." Foundations and Trends@ in Machine Learning, vol. 14, no. 1-2, pp. 1-210, 2021.
406
407
408
[35] R. Liu, Y. Cao, H. Chen, R. Guo, and M. Yoshikawa, "Flame: Differentially private federated
409
learning in the shuffle model," arXiv preprint arXiv:2009.0806. 2020.
410
[36] I.. Sun, J. Qian, and X. Chen, "Ldp-ft: Practical private aggregation in federated learning with local differential privacy," in Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCA/211,, 2021, pp. 1571-1578
411
412
413
[37] A. A. Alemi, I. Fischer, J. V. Dillon, and K. Murphy, "Deep variational information bottleneck," "
414
arXiv preprini arXiv: 16/. 000 2016.
415
[38] Z. Wang, M. Song, Z. Zhang, Y. Song, Q. Wang, and H. Qi, "Beyond inferring class repre- sentatives: User-level privacy leakage from federated learning," " in Proceedings of 2019 IEEE Conference Computer Communications (INFOCOM'19) 2019, pp. 2512-2520.
416
11

---

418
[39] Y. Huang, S. Gupta, Z. Song, K. Li, and S. Arora, "Evaluating gradient inversion attacks and defenses in federated learning," " in Advances in Neural Information Processing Systems
419
11)) 2021, pp. 7232-7241.
421
[40] L. T. Phong, Y. Aono, T. Hayashi, L. Wang, and S. Moriai, "Privacy-preserving deep learning: Revisited and enhanced," in Proceedings of the 2017 International Conference on Applications and Techniques in Information Security (ATIS' 17), 2017, pp. 100-110.
422
423
424
[41] S. P. Karimireddy, Q. Rebjock, S. Stich, and M. Jaggi. "Enror feedback fixes signsgd and other gradient compression schemes," in Proceedings of the 36th International Conference on Machine Learning (ICML 19), 2019. pp. 3252-3261.
426
427
[42] A. C. Wilson, R. Roelofs, M. Stern, N. Srebro, and B. Recht, "The marginal value of adaptive gradient methods in machine learning," in Advances in Neural Information Processing Systems (NeurIPS' 17), 2017, pp. 4148-4158.
428
429
430
[43] C.-Y. Chen, J. Ni, S. Lu, X. Cui, P.-Y. Chen, X. Sun, N. Wang, S. Venkataramani, V. V. Srinivasan, W. Zhang et al., "Scalecom: Scalable sparsified gradient compression for distributed training." in Advances in Neural Information Process- ing Systems (NeurIPS 20), 2020, pp. 13 13551-13 563.
431
432
communication-effiiicee
433
434
[44] X. Dai, X. Yan, K. Zhou, H. Yang, K. K. Ng. J. Cheng, and Y. Fan, "Hyper-sphere quantization: Communicationn-ffficieee sgd for federated learning," arXiv preprint arXiv:1911.04655 2019,
436
[45] A. Krizhevsky, G. Hinton et al., "Learning multiple layers of features from tiny images." 2009.
437
[46] J. Jeon, K. Lee, S. Oh, J. Ok et al., "Gradient inversion with generative image prior," in Advances in Neural Information Processing Systems (NeurlPS 20222,,,,, pp. 29 898-29 908.
438
[47] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint urXiv:1409. 1556, 2014.
440
Checklist
441
442
1. For all authors...
443
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
444
contributions and scope? [Yes]
445
(b) Did you describe the limitations of your work? [Yes]
446
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
447
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
448
2. If you are including theoretical results...
450
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
452
3. If you ran experiments..
453
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [Yes]
454
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Sec. 6
456
457
(c) Did you report error bars (c.g., with respect to the random seed after running experi- ments multiple times)? [Yes] see supplementary
458
459
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
461
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
462
(a) If your work uses existing assets, did you cite the creators? [Yes]
463
(b) Did you mention the license of the assets? [No]
484
(c) Did you include any new assets either in the supplemental material or as URL? [No]
465
(d) Did you discuss whether and how consent was obtained from people whose data you're
486
using/curating? [Yes) all the license of the data, basically the ones used for research openly accessible
12

---

488
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [No]
469
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
472
473
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
475
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "wjYVhJuxpg5", "reviewer": "Reviewer_XcaE", "summary": "- The paper proposes a defense against Gradient Inversion Attacks (where input images can be reconstructed by an attacker from the public gradients).\n- The crux of the proposed approach is (a) pruning gradients: where top-$k_1$ and bottom-$k_2$ parameters are pruned and (b) gradient-location bounding: such that users across FL prune similarly which results in sparsified models even at download-time.\n- The paper is also accompanied with theoretical insights validating the choices e.g., perturbed vs. non-perturbed gradients are proportional to reconstruction errors.\n- The defense approach is evaluated against three attacks (Rob, IVG, R-GAP) and two datasets (CIFAR10, CIFAR100). Evaluation indicates a performance improvement over existing defense against all attacks, while retaining the utility (efficiency, accuracy).", "questions": "Please see the questions under concerns.", "limitations": "Yes, the authors have adequately discussed the limitations in the main paper.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n1. Insight: The core insight is intuitive. Existing top-k approaches *retains* the top-k parameters and the authors show that this results in keeping the sparsified gradient close to the original and thereby increasing the attack effectiveness (Proposition 1). In contrast, the approach *prunes* the top-k gradients and thereby lowering the reconstruction performance.\n2. Evaluation: The authors benchmark the approach against multiple classes of GIA attacks (analytical, optimization-based, ...).\n3. Results: The results are significant and promising. The approach while almost perfectly retaining the target model accuracy (93.17 vs. 93.44) thwarts all considered classes of attacks.\n\n### Concerns \n1. Gradients attacked from which iteration?: The evaluation methodology is slightly unclear. Specifically, it is unclear the gradients from which training iteration is used to demonstrate system-under-attack performance. Are the authors aware whether this has an impact to attack performance? I reckon the gradients at the early training iterations are more\n2. Choice of attacks: While I appreciate the authors consider different classes of attacks, some recent popular attacks seem to have been left-out e.g., GradInversion [26]. Specifically, the class of optimization-based attacks which additionally regularize fidelity. \n3. Some evalution details unclear: While the paper indicates evaluation on 2x CIFAR10 models and CIFAR100, it is often not clear for which model/dataset the results are. For instance, the setting is unclear in Fig.1 and Table 2 (which I think is CIFAR10?), but Fig. 2a displays attacks on CIFAR100.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "qO0RE-3AlGY", "reviewer": "Reviewer_cSmE", "summary": "This paper proposes Aligned Dual Gradient Pruning (ADGP) as a defense against gradient inversion attacks.\n\nA gradient inversion attack takes the gradient $\\nabla W (x)$ (and the model $W$) and attempts to reconstruct the input data $x$.\n\nStandard gradient pruning takes the gradient $\\nabla W (x)$ and zeroes out the smallest coordinates. Intuitively, this makes it harder for the attacker as they are not given as much information.\n\nDual Gradient Pruning (DGP) zeroes out not only the smallest coordinates but also the largest coordinates, leaving only the \"median\" coordinates of the gradient.\n\nADGP modifies DGP by ensuring that all the participants zero out a similar set of coordinates, rather that independently choosing which coordinates to zero out.\n\nThe paper provides both experimental and theoretical analysis of ADGP. It give theoretical results that bound the error of a potential attacker. And it also gives experimental results where attacks are run against ADGP. This includes a comparison to other defenses and a utility analysis.", "questions": "I have a question about the theoretical results (Proposition 1) and the experimental results (Figures 1 & 2).\n\nWhat is the optimization problem that the attacker is solving?\n\nAbusing notation slightly, I can see two possible ways to do this: First, $$\\arg\\min_{x'} \\\\|\\nabla W(x') - \\mathsf{ADGP}(\\nabla W(x))\\\\|$$ and, second, $$\\arg\\min_{x'} \\\\| \\mathsf{ADGP}(\\nabla W(x')) - \\mathsf{ADGP}(\\nabla W(x))\\\\|.$$\nWhich of these two is being considered in the paper? or is it something else?\n\nIntuitively, my question is how does the attacker account for the ADGP defense?\n\nI also have a minor question about Definition 1. There is a nested probability and expectation. What is the source of randomness for each of these?\n\n---\n\nThe discussion with the authors clarified the question about what optimization the attacker is performing, namely $$\\arg\\min_{x'} \\\\|\\nabla W(x') - \\mathsf{ADGP}(\\nabla W(x))\\\\|.$$ \nUnfortunately, this is the wrong way to formulate the problem. Intuitively, this assumes the attacker is not aware of the ADGP defence being used and applies a naive strategy. More precisely, the problem is that this objective is not minimized by $x'=x$ -- i.e., this objective does not lead to successful reconstruction.\n\nThis issue essentially invalidates the experimental and theoretical results. They assume a naive attacker and tell us nothing about what an attack tailored to this defence would do.\n\nIn the discussion I presented a simple counterexample to Theorem 1 in the paper. That is, the loss $\\ell(w,x) = (x-\\sum_i w_i)^2$, where $x$ is a scalar and $w$ is a vector. Given any single coordinate of the gradient and the value of $w$, it is possible to reconstruct $x$. ADGP does not protect against this attack.\n\nThe experimental results attempt \"state-of-the-art\" attacks. The problem is that an attack is tailored to a specific system. A state-of-the-art attack for one system may not be state of the art for another system.\n\nUnfortunately, given this major flaw, I think the paper cannot be accepted.", "limitations": "Limitations are discussed", "rating": 3, "confidence": 4, "soundness": 1, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n - This paper studies an important problem.\n - It proposes a simple and novel approach to protecting against gradient inversion.\n - Both theoretical guarantees and empirical evaluation are included.\n\n\nWeaknesses:\n - The theoretical results are somewhat difficult to interpret. E.g. the parameters $\\gamma_1$ and $\\gamma_2$ from Assumption 1 appear in Theorem 1, but it is not clear what these parameters are and, hence, how to interpret the guarantee.\n - The experimental results do not consider varying the parameters $k_1$ and $k_2$, namely the number of large and small coordinates to zero out. Similarly the other defenses being compared against also have parameters that could be varied.\n - Some details about the results are unclear (see below).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "XwCDANEK7U5", "reviewer": "Reviewer_jpQC", "summary": "The paper proposes a new gradient pruning strategy called \"Aligned Dual Gradient Pruning\" for federated learning (FL). It differs from the standard pruning approach of top-k sparsification in two ways - (i) instead of retaining the top-k gradients, it removes both top-k1 and bottom-k2 gradients, thereby increasing the distance between the original gradients and the exchanged gradients, and (ii) one of the clients is allowed to broadcast the locations of its top-2k gradients, so that all the other clients can choose from this list of locations while picking the gradients to transmit. ", "questions": "1) What is the contribution of the error accumulation step? How will the proposed method behave if error accumulation is removed?\n\n2) If the server keeps an account of the gradients transmitted by the client in each round as well as the locations of the top-2k gradients, can the server recover almost the full gradient of the clients over a few rounds of communication?\n\n3) How will the proposed alignment scheme work in the presence of non-iid data?\n\n4) What is the privacy impact of the alignment step? Will it leak more information than top-k sparsification since the server knows the locations of the top-2k gradients? Can more sophisticated attacks be designed leveraging this information?\n\n5) What is the impact of the hyperparameters (k1, k2, and k) on convergence, accuracy, and privacy? \n\n ", "limitations": "The paper discusses one limitation related to the assumption that all the clients need to be honest for the proposed scheme to work. But it overestimates the privacy benefits of the proposed approach. ", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1) A variant of the top-k gradient pruning strategy has been proposed to enhance robustness against gradient inversion attacks (GIAs). \n2) Both theoretical results and experiments have been presented to demonstrate that the proposed method works.\n\nWeaknesses:\n\n1) It is fairly obvious that if the top-k1 gradients are suppressed, it will lead to less information leakage compared to top-k sparsification. However, it is hard to believe that it will have no effect both on the convergence speed and final accuracy. Though the paper provides a theoretical result (Theorem 2) and experimental data (Table 2 and Figure 4), a concrete intuitive explanation of this unexpected behavior is required. Is this solely due to the error accumulation step? How will the proposed method behave if error accumulation is removed?\n\n2)  There is no mention about non-iid data in the whole paper. One would expect that the proposed alignment strategy will completely breakdown in the case of non-iid data, because there will be very few intersections between the top gradients of different clients. A careful analysis of this issue is clearly necessary.\n\n3) The most critical weakness of the paper is the lack of any analysis about the privacy leakage introduced by the alignment step. Since one of the clients transmits the binary matrix $\\mathcal{I}$, it is trivial for the server to know the locations of the top 2k gradients of this client. Even if the client does not transmit the top-k1 gradient values, the server knows that locations with value 1 in $\\mathcal{I}$ must have values greater than or comparable to that of the gradients that are actually transmitted by the client. Wouldn't this leak more information about the client's gradients compared to even the top-k sparsification strategy? Does any of the attack techniques considered in this paper exploit this knowledge?\n\n4) There is no ablation study to evaluate the impact of the various components, especially alignment step and error accumulation step.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "OkMluRQXfx-", "reviewer": "Reviewer_U79T", "summary": "This paper introduces two techniques, Dual Gradient Pruning (DGP) and Aligned Dual Gradient Pruning (AGDP) to protect against gradient inversion attacks. They describe DGP and present ADGP as an improved version of DGP that has the same security considerations but lower communication costs. They show that optimization attacks have a provable amount of degeneration under DGP and ADGP. Then they empirically evaluate their technique against a set of gradient inversion attacks and compare against other defenses like DP-SGD, Soteria and Precode. The authors show that their ADGP technique succeeds against all the attacks they try while still maintaining model quality and reducing communication costs.", "questions": "Does the analysis still hold if assumption 3 does not hold?\nDid you try your technique on domains other than vision?\n\n", "limitations": "Negative impacts are adequately addressed.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- The authors present a very simple idea and run extensive experiments to show that their technique is superior to existing baselines\n- DGP by itself seems like a new and somewhat unintuitive technique and the results from DGP would be a fairly strong contribution by itself.\n- Empirical evaluation and comparison with other techniques is very thorough. The authors compare against most of the well-known baselines and try multiple attacks. The details of the empirical evaluation are also clearly specified in the main text.\n\nWeaknesses\n- Assumption 3 seems somewhat unrealistic. Assuming that each client in each round has data that provides an unbiased gradient means that the data has to be IID distributed across clients. \n- ADGP is presented as saving communication overhead but this overhead is only saved if the same set of clients are used for every round of FL training. If the clients selected for round N are different from those selected for round N+1 then ADGP does not save any communication overhead. In production FL settings, clients are often selected from a large population, so the odds of the same client appearing in two back-to-back rounds are low.\n- The authors only evaluate on vision models, but there are known gradient inversion attacks in NLP, speech etc. It would be interested to see if DGP works in those settings.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "DJP_PJEWJAL", "reviewer": "Reviewer_MAQA", "summary": "This paper considers an important and timely problem in Federated Learning, investigating defense mechanism against model inversion attack in terms of three key metrics: privacy, utility, and communication efficiency. To provide better trade-off among the three key metrics, this paper firstly proposes a simple sparsification technique, named dual gradient pruning (DGP), which removes top-k1 largest values and bottom-k2 smallest values in local model parameters. In the second component, named gradient location bounding, communication cost is further reduced by ensuring that the sparsified region of different users are the same.\nThis paper theoretically shows that proposed scheme guarantees better privacy against model inversion attack. In addition, convergence analysis of the proposed method is provided with IID dataset distribution (i.e., local computation of each user is unbiased to the full gradient) and assumption on DGP. \n", "questions": "1.\tHow can we represent gamma_1 and gamma_2 with respect to k_1, k_2, and k?\n2.\tIn experiments, what is N (number of users), and how the dataset (CIFAR10/100) is distributed over N users?\n", "limitations": "As this paper considers the privacy in FL, it has no negative societal impact.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Overall, this paper is well structured and written, and hence easy to follow. Intuition behind the motivation of DGP/ADGP is well described in the section 4. While the proposed pruning technique is very simple, it can incur further research investigating privacy and utility trade-off in sparsification/pruning schemes in FL. Especially, combining sparsification and other defense mechanism (DP, secure aggregation, etc) would be very interesting future direction. It also includes extensive experiments with various attack models and defense mechanisms to demonstrate the stronger privacy guarantee and comparable utility of the proposed scheme. \n\nI have the following concern/comment:\n\n1. In theoretical analysis, it seems that this paper relies on strong and unrealistic assumptions. This paper considers IID data distribution (i.e., in assumption 3, local stochastic computation is unbiased to the full gradient), which is not the case in FL. In addition, as the assumption 1 is very important for the analysis of convergence, more justification/explanation is needed. For instance, while k1 is an important parameter which determines the utility (and convergence performance), relationship between \\gamma_2 and k_1 is not presented. \n\n2. In section 4, the role of e_i^t is not described. We can know that it is accumulated error of user i at round t in section 5 (after reading the lemma 1, but it is helpful to understand the proposed scheme if explanation will be included in section 4. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "83WPnhlnI4P", "reviewer": "Reviewer_CLEu", "summary": "This paper continues the research line of defending against data leakage in federated learning (FL). In particular, the paper argues against the conventional view that the top-k defenses do not work well, and proposed a simple modification, which is to also remove bottom gradients and then perform a calibration based on the accumulated error from the previous rounds to reduce communication cost. The paper compares the performance with five existing defenses on three existing attacks, and show that their method can achieve a better utility-privacy-communication tradeoff.", "questions": "1. The authors point out why top-k does not perform well: (1) top-k gradients are close to the real gradients and (2) large gradients leaks information of $y$. Both reasons only explain why top-k might be problematic, but how does removing bottom gradients address both problems? I do not find any insights that explain it, the only sentence I find that gives some justifications is \"Moreover, it is also necessary to delete small gradient parameters to achieve a high sparsification ratio.\" (Section 4.2), which only says deleting small gradients can make sparsification ratio higher. But deleting any gradients can further sparsify, why small gradients? Please clarify.\n\n2. Definition 1: Change $\\mathcal{D}_{(\\nabla W, \\nabla W^*)}$ to $\\mathcal{D}(\\nabla W, \\nabla W^*)$. Usually $D(.,.)$ represents the distance and there is no need of subscripting.\n\n3. Proposition 1\n- From the supplementary material, $\\phi(x, W)$ is just $\\nabla W$, you should define it explicitly in the main text. In addition, why write $\\nabla W$ twice in the right-hand side of the inequality but in different forms (in both numerator and denominator)?\n- Define $g = \\phi(x', W)$ (taken from the supplementary material) explicitly. Propositions should be self-contained.\n- The conclusion that \"reconstruction error is proportional to the gradient distance $||\\nabla W - g||_2$\" seems questionable to me because the denominator is not a constant. In fact, if I understand it correctly, the denominator is the 2nd derivative of $W$ w.r.t $x$, i.e the Hessian norm, is it correct? If so, I do not see why the reconstruction error can be proportional to the numerator. Please let me know if I misunderstand it.\n\n4. Algorithm 1: Where is the part that removes the bottom gradients $\\mathcal{B}_{k_2}(\\nabla W)$? It seems it only operates on the top gradients.\n\n5. Assumption 1: $\\gamma_1$ and $\\gamma_2$ are constants determined by $k_1$, $k_2$, and $k$. What are their explicit definitions? Please write them out because they are used in main theorems and lemmas and should not be undefined.\n\n6. Table 2\n- The comparison of DP with other defenses are not apple-to-apple because the DP accuracy is much lower. I notice the reason you choose that DP budget was because it has been shown the best in the prior work. But in this particular evaluation, if you do not tune the DP budget to find a comparable accuracy, the privacy protection performance comparison is not fair.\n- In R-GAP, the SSIM row, the best defense (\"Ours\") is not highlighted.\n\n7. In general, the author should not make claims against the utility-privacy-communication tradeoff like \"none of the existing defense methods could take care of all privacy, utility, and efficiency difficulties in FL\" (taken from the introduction). Because there is a fundamental tradeoff and this paper (and maybe all papers) cannot remove the tradeoff. They can only make a better tradeoff but cannot make it go away. Therefore, I think it would be a more objective tone not to emphasize the tradeoff exists in the prior work (since it exists in any method) but rather the tradeoff is not good enough.", "limitations": "I do not have comments on the limitations.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 1, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- Important topic: the data leakage problem in FL is a practical concern\n- The method is simple and easy to implement\n\nWeaknesses\n- The theoretical part has several places that are unclear and questionable to me (see Questions section)\n- The evaluation section show some results that do not seem comparable (see Question section)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Reject