FedDiscrete: A Secure Federated Learning Algorithm Against Weight Poisoning
Authors: Yutong Dai
Xingjun Ma
Lichao Sun
Venue: ICLR 2022 Submitted (2022)
Keywords: federated learning
weight poisoning defense

Abstract: Federated learning (FL) is a privacy-aware collaborative learning paradigm that allows multiple parties to jointly train a machine learning model without sharing their private data. However, recent studies have shown that FL is vulnerable to weight poisoning attacks. In this paper, we propose a probabilistic discretization mechanism on the client side, which transforms the client's model weight into a vector that can only have two different values but still guarantees that the server obtains an unbiased estimation of the client's model weight.  We theoretically analyze the utility, robustness, and convergence of our proposed discretization mechanism and empirically verify its superior robustness against various weight-based attacks under the cross-device FL setting.

Under review as a conference paper at ICLR 2022
FEDDISCRETE: A SECURE FEDERATED LEARNING ALGORITHM AGAINST WEIGHT POISONING
Anonymous authors Paper under double-blind review
ABSTRACT
Federated learning (FL.) privacy-aware collaborative learning paradigm that allows multiple parties to jointly train machine learning model without shar- ing their private data. However, recent studies have shown that FL is vulnerable to availability poisoning attacks, integrity backdoor attacks and inference attacks via weight poisoning and inference. In this paper, we propose a probabilistic discretization mechanism weight into a vector that can only have two different values but still guarantees that the server obtains unbiased estimation of the client's model weight. We theoretically analyze the utility, robustness, and convergence of our proposed dis- cretization mechanism and empirically verify its superior robustness against vari- ous weight-basee attacks under the cross-device FL setting.
the client side, which transforms the client's model
INTRODUCTION
Federated learning (FL) is an emerging privacy-aware framework that trains a machine learning model across multiple parties without accessing their local private data (Konečný et al., 2016; McMahan et al., 2017). In FL, each client first trains the local model using its private data and then sends the model gradients to an honest central server. The central server aggregates all local model gradients to form a global model, which is sent back to the clients for the next round of train- ing. However, sharing model gradients might still leads to security concerns and privacy leakage.
Recent studies have shown that FL is vuinerable to various model weight poisoning attacks. The ad- versarial client(s) can stealthily manipulate the global model via modifying the local model updates to achieve attack goals like, preventing model convergence, implanting backdoors into the global model, and inferring the privacy information of clients' private data (Gu et al., 2017; Blanchard et al., 2017; Pyrgelis et al., 2017; Xie et al., 2019; Bagdasaryan et al., 2020; Wang et al., 2020; Tang et al., 2020). To address such an issue, recent works start to explore different defense techniques to robustify FL against model weight poisoning attacks (Blanchard et al., 2017; Bagdasaryan al., 2020; Shen et al., 2016; Geyer et al., 2017; Fung et al., 2018). However, most existing works apply the defense or robust techniques the server side, which changes the role of the central server from being honest to trusted. And prior approaches mostly only focus on one type of attacks.
Our contributions. In this paper, propose the FEDDISCRETE, a flexible FL framework that can combine any popular FL algorithms, for example, FedAAv(McMMahan et al., 2017) and FedProx(Li al., 2020), with our probabilistic discretization mechanism. Theoretical analyses on the util- ity, robustness, and convergence are performed to show that FEDDISCRETE is inherently robust to availability poisoning attacks (Kurita al., 2020), integrity backdoor attacks (Gu al., 2017) and server inference attacks (Shokri et al., 2017). FEDDISCRETE is evaluated four popular image classification datasets under both data are i.i.d and non-i.i.d settings. The numerical results indicate that FEDDISCRETE is robust to various weight-based attacks.
2 BACKGROUNDS AND RELATED WORKS
In this section, we provide necessary background information on concepts thorough the paper and formalize the problem to be solved.

---

Under review as a conference paper at ICLR 2022
Federated learning (FL). FL is privacy- collaborative learning, where N clients and one trusted server work together to learn a global model (McMahan et al., 2017). Depending on the application scenarios, the number of clients n can range from small two several hundreds in the cross-silo setting or can easily go beyond millions in the cross-device setting. A classical way of formulating FL into optimization problem is (Wang et al., 2021a)
N Σ (1)), where Fe(w))
N Pt 1.
min F(w)= WER"
1 Di
Σ f((w;() and Σ SED.
(I)
In Eq. 1, up is the global model weight; F; is the i-client local objective function; the local loss functions (i(w:;) are often assumed to be the same across all clients; the local data D; can have different distribution. Following the seminal work of (McMahan et al., 2017), extensive researches have been conducted to address various challenges in FL. For instance, (Li et al., 2020; Reddi et al., 2020) aim to design efficient optimization algorithms; (Konečný et al., 2016; Luping et al., 2019) try to improve the communication efficiency; (Bagdasaryan et al., 2020; Xie al., 2021) study the security issues on both attacks and defenses.
Adversaries. For any FL attack, the server is assumed to be honest, which faithfully follow the training protocols and any client cannot directly get the model weight of other clients. Under these assumptions, there are two adversaries: 1) the malicious clients can manipulate the weights for various attack purposes, such as interfering the global model training or implanting backdoors into the global model; 2) the curious server can explore the privacy information from local clients via inference attacks.
Availability poisoning attacks. The goal of availability poisoning attacks (APAs) is for malicious local client(s) to destroy or reduce the global model's utility (Shen et al., 2016; Sun et al., 2018). In APAs, the attacker can either control the global model's utility on target tasks or decrease the most of the model's utility as the optimal attack strategy. In practice, adding a large random noise can successfully worsen the global model's utility but can be easily discovered by anomaly detection techniques. In this case, any advanced attack needs to know and tries to bypass the defense tech- niques used in training. In this work, we assume the attacker has full knowledge of training process and all possible defense techniques.
Integrity backdoor attacks. The goals of integrity backdoor attack (IBAs) are two-fold: 1) the attacker aims to implant backdoors into the global model, through which the attacker can control the prediction results by injecting the trigger to any clean examples; 2) at the same time, the attacker wants the global model can still perform good as the non-attacked model on all clean inputs (Bagdasaryan et al., 2020). Note that, IBAs are also conducted by the malicious clients in FL.
Inference attacks. Many studies show that the attacker could explore the private information from the model weights, such as membership and attribute information (Pyrgelis et al., 2017; Shokri et al., 2017: Ganju et al., 2018; Melis et al., 2019) or even recover the training samples used by clients Zhu et al. (2019). When the honest-but but-uuoouus server becomes malicious, it can explore privacy information of each client's local dataset through inference attacks (IAs) since the server has full access to clients' model weight.
Both APAs and IBAs can be achieved via weights poisoning that modifies the local model weights. For example, as the global model converges, the deviations of local models start to cancel out, i.., Σg (w: w² xe 0 given as in Eq. 2 of Bagdasaryan et al. (2020), where and i the communication round and client index respectively, w" and wi are the global and local model respectively, and n, is the number of participating clients. If the adversary aims to replace global model w with a target model X, then it could propose to upload a local model weight w = n (n,- 1) '1 Σ "uof -=-11 & ne(X w' + w as given in Eq. 3 in Bagdasaryan et al. (2020). Prior works (Bagdasaryan et al., 2020; Wang et al., 2020; Xie et al., 2019) demonstrate the effectiveness of both APAs and IBAs when the training algorithm is FedAvg. IAs can be easily achieved by many existing advanced attack methods once they have the full access to the client model Shokri et al. (2017); Hu et al. (2021).
Related works. To address the various attacks, many prior defense works have been proposed and achieved good performance against the attacks. However, prior defense works only address one aspects of aforementioned three attacks. For availability poisoning attacks, Steinhardt et al. (2017) proposed a defending framework, which can be applied to defenders that remove outliers and then
2

---

Under review as a conference paper at ICLR 2022
IBA Attacker
IBA Attacker
Attack: No
Local2
Localz
M
Server
Robust Area
Clean
Clean
Local w=1[0.3,07,,04000111
Local1
Local1
Discretization
APA Attacker
APA Attacker
Attack: Yes
(4) APA and IBA
(b) FedDiscrete against APA and IBA
(c) Discretization against IA
Global Model
Poisoned Global Model
Local Model
Local Discrete Model
Figure 1: (a): An illustration how adversarial clients do APAs and IPAs via weight modifica- tion. If there is no adversaries, then FL. training algorithm will give a clean model. When there are attackers, the clean model becomes poisoned. (b): An illustration how discretization mechanism helps to defend the APAs and IPAs, which brings the poisoned model to the robust area. The robust area is parameter space where the model would have the same performance the clean model. (c) An illustration on how discretization mechanism helps to defend the IAs as the discrete model updates provide limited information compared the continuous counterparts.
minimize a margin-based loss on the remaining data. Under assumptions, this framework provides the approximate upper bounds the efficacy of any data poisoning attack. However, this framework does not fit in the FL setting, as it needs the access to all private data. Blanchard et al. (2017) studied the presences of Byzantine adversaries and proposed Krum aggregation rule defend under the assumption that the participants' training data are i.i.d, which is not necessarily true in the FL setting. And indeed as argued in (Bagdasaryan et al., 2020), Krum can used by the adversaries to make the attack more effective. FoolsGold (Fung et al., 2018) can mitigate sybils data poisoning attacks under assumption that the honest clients can be separated from sybils by the diversity of their gradient updates. As discussed by the authors, FoolsGold is not successful at mitigating attacks initiated by single adversary. For integrity backdoor attacks, Xic et al. (2021) provided a general framework that certifiably robust to the backdoor attacks under the FL setting via the model weight smoothing. However, the convexity assumption imposed on the loss function, which limits certified robustness more challenging and widely used deep neural network. Ozdayi al. (2021) proposed a defense approach based on adjusting server's 's learning rate with the guidance of sign information of agents' updates. For inference attacks, differential privacy is widely adopted approach defend, where judiciously random noise added on either the clients' model update or the global model, e.g., Geyer al. (2017), but suffers from the trade-off between privacy and accuracy. Compared this work, neither of these works can address three types of attacks simultancously.
3 METHOD
We introduce FEDDISCRETE, a federated learning framework with a simply yet effective discrete mechanism and the flexibility to accommodate various FL optimization algorithms. A complete description is given in Algorithm 1. FEDDISCRETTEdiffereness FEDDISCRETE consists of three stages, namely, local training, discretization and aggregation, which are discussed in details.
Local Training. Since we mainly focus on the cross-device setting, in each communication round t, the server first selects subset of [S,] K clients to participate the current round of training and broadcasts the global model w' to the selected clients. The clients who receives the global model then performs local training using any appropriate algorithm to generate new local model, i.e. w for all E Sr. The ith client can 1) use the local stochastic gradient descent(SGD) method as considered FedAVG to perform a fixed number of SGD steps to improve communication effi- ciency; 2) use adaptive method (Wang et al., 2021b) to improve the convergence; 3) approximately minimize F; (w) # ||w - w'|l for some μ > 0 as proposed FedProx to accommodate the system and data heterogeneity (Li et al., 2020).
Discretization. For all E S,, the ith local client computes the maximum and minimum values of wi and sends porte-perturbed maximum value uf and minimal value 2 back to the server, where the
3

---

Under review as a conference paper at ICLR 2022
noise is added to protect the privacy of the local training data and the noise is sampled and added in a way that to guarantee if and i are the valid upper bound and lower bound of w element- wise. Once the server received the client-wise upper-bounds and lower-bounds, it computes the minimum of the lower bounds minn and the maximum of the upper bounds " to prepare the inputs for the discretization mechanism (see Definition 3.1), which outputs unbiased estimator of the input (see Lemma 4.1). For all i E S,, the ith client discretizes its continuous model w into M(w limi )max and uploads M(w); i mmaxx to the server for aggregation.
Aggregation. Rather than simply aggregating {-M(wi;; Imin ( the sever first inspects whether each coordinate of w is either limin or uimax to exclude the any potential malicious adver- saries that attempt to bypass the discretization process. And the server only aggregates the local model weights that pass the sanity check. Also when the server computes Iiin and u it can be more conservative by discarding the extreme values in hes, and {uf)es, by setting thresholds on the lower quantile of hes and upper quantile of {u², hes. to prevent adversaries proposing extremely large upper bounds and/or small lower bounds. For the ease of presentation, we do not include this choice the description of Algorithm
Definition 3.1 (Discretization Mechanism), For any (w,l, E x with S w S u, define the discretization mechanism as
M(w; 1, u) = u, w.p.
(2)
w.p.
where w.p. is the shorthand notation for "with probability".
Algorithm 1 FEDDISCRETE
1: Input: Initial model weight w" E 12d total training rounds T, the participants size K, and the learning rate {1/ Y and a positive sequence {or J.V.
2: for 6 -00, 0,
2..,
-1 do
3: The sever randomly selects an index set S. with |St| = K and broadcasts wt to the client i if S,.
4: For the ith client, where i St, it performs the local training to obtain wi and samples two random variables & and ç from the truncated Gaussian TN(O, ag; 0, 1). Then it com- putes and uploads 15 minje[4] {[w[]] w²) Ef and uf maxjejd [[w]]] w²) Ç to the server.
5: The sever computes 1f = minjes. (49 and wf = max,ss, {uf} then broadcasts (", ut to clients in St.
6: For the ith client, where i St, it applies the discretization mechanism M is element- wise fashion and uploads the discrete model weight MM uw: wi; ;", u²) to the server 7: The server conducts the sanity check for {M(w)++ w':'', u²) hee... i.e., validates whether all elements of w are either 1e or u'. Form the set S _ St to collect all the clients' index passing the sanity check and compute w w' +n | Eies; M(w(+11 w/: 12, ).).
8: end for
We close this section by making following comments.
Discretization is a widely applied technique, for example, in influence maximization Kempe et al. (2003), the algorithm needs to simulate the propagation from a probabilis- tie graph that is consists of set of edges with activation probabilities in [0, 1]. It is also can be regarded as a special form of quantization that is widely used in the distributed optimiza- tion (Alistarh Compared with prior discretization works, this is the first work to adopt discretization to protect FL from weight poisoning attack.
al., 2017; Reisizadeh al., 2020) improve communication efficiency.
Compared with the FedAvg, although an additional round of communication is required in FEDDISCRETE to perform the discretization mechanism. We argue that FEDDISCRETE is more communication-efficeell than FedAvg. Assume each scalar takes 64bits, then for the
4

---

Under review as a conference paper at ICLR 2022
t-th round, the total bits communicated are 128dK for FedAvg, while for FEDDISCRETE are 64dK + 128K + dK. For large d, FEDDISCRETE communicates less bits.
Although the discretization mechanism outputs an unbiased estimation of w the vari- ance of the estimator M( can be large. To mitigate this issue, one can compute more finely-grained upper bounds and lower bounds. For example, assume each client wants to train a neural network. Then, instead of computing the maximum and minimum values over the entire w, the client could compute layer-wise maximum and minimum values. And the discretization mechanism could be applied layer-wise with tighter upper bounds and lower bounds. This leads to smaller variance in the estimator at the cost of increasing the communicated bits.
Intuitively, the discretization mechanism is effective in defending weight poisoning attacks as the judiciously crafted adversary model is discretized. For example. consider the model replacement attack in Eq. of Bagdasaryan al. (2020), once the malicious model wn is discretized into M(w's) the model replacement becomes ineffective.
4 THEORETICAL ANALYSIS
In this section, we analyze the utility of the discretization mechanism and the convergence property of the FEDDISCRETE. Due limited space, complete proofs are deferred the appendix.
We first introduce the notations are used throughout the paper. [w]; denotes the jth coordinate of the vector w. Unless specified otherwise, |I-Il and Il- are the Buclidean norm and infinity norm respectively. To avoid the cluttered notation, we use M(-) instead of M(--;-) when there is no confusion. And if M is applied over a vector, we mean to apply it in element-wiss fashion. Here, [d] represents (1, .d.
4.1 UTILITY ANALYSIS
The first lemma shows that the discretization mechanism produces an unbiased estimator with bounded variance.
Lemma 4.1. For any w E Rd and 24) E IR xR such that < minje(a] [w]; < max,e(d [wol S u, then
E[M(w; u)] = w:
* ElIlM(w; w|l - dلu and ([llM(w;,,,)) wil
A natural question to ask is that how far is the averaged local model weights wh := Eies wi-1 from its discretized counterpart w.M == Eies.,, M(w The next theorem characterizes the distance between w' and WMM
Theorem 4.2. For any communication round t S T and any 3 E (0,1), there exists =
(u' (1'= 1111")//gg log 2d
O
such that
P ([] Iill-''''vlll <c) 2 1-3.
4.2 ROBUSTNESS ANALYSIS
Next, we show the discrete mechanism is robust to random weight poisoning and can recover the original weight from local models against a limited number of independent targeted weight poison- ings. To maximize the attack impact, the attacker choose to inverse the discrete local updates to
Each model weight takes 64d hits. For FedAvg, server broadcasts the global model weight and clients upload the local model weights, so the total bits 128dK. For FEDDISCRETE, 64dK is the for the server broadeasting the global model K server. 128K the cost for uploading and broadcasting the upper bounds and lower bounds. Since the discretized model weight M(w"+1) +11 can only taking two values, the clients indeed just upload d bits string and the
recover the M(w +1).
5

---

Under review as a conference paper at ICLR 2022
attack the global model. If the intend update of the jth weight [w], is u, the attacker uploads / to the server, and vice versa.
Theorem 4.3. Suppose there are F attackers out of total N clients and denote ÜüM = N Σi= wi then any [d].
NN-F w
1 [[ = N
Σ
Σ
F [wili + (h (h+
More importantly, FEDDISCRETE has asymptotically non-bias estimation when the number of clients approaches to +00 with a fixed number of attackers.
Corollary 4.3.1. Let be +oo for some fixed attackers, the expectation of the averaged discrete weights is E [
Discussion. The above theorem and corollary have shown the robustness against the weights poi- soning attacks, such IBAs and APAs. Besides, another important reason is the discretization mechanism limits the modification ability of the adversaries as shown in Figure 1. For example, if the clean aggregated model's weight is [0.1, 22.3, 3.8). the IBA APA could work while modify- ing the weight from -0.1.-2.3, 2.3, 3.8) to (1,0.2 0.4]. where the weight difference between clean and poisoned model is [0.9, 22.5, -3.4). However, due the discretization mechanism, while we only have 1 attacker among 100 clients, its modification could only be O or +0.01 with the upper bound and the lower bound
In this case, the adversary can not achieve its goal in any manner.
4.3 CONVERGENCE ANALYSIS
In this section, prove the convergence of Algorithm 1 by instantiating the optimization algorithm for generating local model weights as the FedProx(L.i et al., 2020). Specifically, at the ith round, in line of Algorithm the ith selected client produce the local model weight w t F((w)) + # +|ue w 2|2 in the sense that ( (2) u(w,tt1
w''ll S < || (ww")l for some properly chosen μ > 0 and 7 > 0. The Assumption 1 is made throughout the whole section.
Assumption 4.1.
1. (Smoothness) For all E [N], F(w) is L-smooth, i.c., [[VE(w) - VF(w")] s L w|w w'll for some L > and all (w, w') E E Rd.
2.
eigenvalue) The minimal eigenvalue of the Hessian of the client loss func- tion (') is uniformly bounded below by a constant Amin E R.
3. (Bounded dissimilarity) For all E [N] and any w E Rd, ([VVE()) s
B2 V [ where the expectation taken with respect to the client index i.
4. (Algorithmic choices) In Algorithm in line 3, the ith client is picked with the probability p;; line 7, the step size n/ =1
Theorem 4.4. Consider the Algorithm 1 instantiated with FedProx. If y. is chosen satisfy > Amin. and K,H,y is chosen properly such that k = 1-3B LB(1+T)
=
2
(1+7)) (2V K +1) B((+2) K2
(N+221 O, then after T rounds,
BVR
SK Pannu
S(w") - -(ww)
min E V/w"')]] xE(T-)
KT
Remark
The rate of convergence of FEDDISCRETE is the same that of FedProx but with worse constant due to the discretization mechanism.
One can also prove the convergence of the FEDDISCRETE when the local clients perform the fixed number of stochastic gradient descent steps as FedAVG. Then the convergence result is special case of the FedPAQ (Reisizadeh et al., 2020) when & is set to 1/4.

---

Under review as a conference paper at ICLR 2022
390
320
100
500
93
35
15
01
82
90
Number dient
350
55
Number clcnes
300
300
50
100
Number clarns
323
755
Number diinss
300
(a) MNIST
(b) FMNIST
(c) SVHN
(d) CIFAR-10
Figure 2: Effect of number of clients N with i.d setting.
05
120
20
100
50
70
Continut
90
20
00
50
57
333
150
200
750
359
330
50
10C
353
200
250
300
352
100
150
253
250 300
350
D20
Number
232
340
(a) MNIST
(b) FMNIST
SVHN
(d) CIFAR-10
Figure 3: Effect of number of clients N with non-i.i.d setting.
5
EXPERIMENTAL EVALUATION
In this section, we examine the effect of FEDDISCRETE on the four image benchmark datasets: MNIST (LeCun et al., 1998), Fashion-MNIST (FMNIST) (Xiao et al., 2017), SVHN (Netzer et al., 2011), and CIFAR-I0 (Krizhevsky et al., 2009). For MNIST and FMNIST, we adopt a two-layer CNN for image classification; for SVHN and CIFAR-10, the small network from the Pytorch li- brary² only achieves around 50% accuracy, so that we re-design a small VGG (Simonyan & Zis- serman, 2014) for them. The training data and the testing data are fed into the network directly in each client, and for each client, the size of the training data is the total number of the training samples divided by the number of the clients for i.i.d setting. For non-i.i.d setting, we sort the data by digit labels first, then divide the datasets into 2N shards of size [D] and assign each of N clients 2 shards, which is the same setting (McMahan et al., 2017). We use the local SGD as the opti- mization algorithm for each client, where the local learning rate is set as 0.03 for MNIST/EMNIST and 0.015 for SVHN/CIFAR-10 10. And the global learning rate 77, is set to for all t. Considering the randomness from the discrete mechanism, we run the test experiments five times independently to obtain averaged value. To evaluate the performance of different approaches, metrics including accuracy/utility for model performance, attack success rate (ASR)² for attack per- formance and the number of communication rounds (CRs), for communication cost. Any approach with a high accuracy and low ASR indicates good and practical defense solution. The proposed models are implemented using Pytorch, and all experiments are done with a single GPU NVIDIA Tesla V100 on the local server. Experiments on MNIST and FMNIST can be finished within an hour with 10 CRs, and experiments on SVHN and CIFAR-10 need about 2 hours with 15 CRs.
2N
different
Evaluation on Discrete Mechanism. Here, we first test the effectiveness of the discrete mecha- nism in FL. In Figure 2 and 3, the results demonstrate three observations: 1) for the discrete model, in i.i.d setting, it performs closer to continuous model aggregation than that in the non-i.i.d setting: 2) the more complex and difficult tasks, i.e., CIFAR-10 > SVHN > FMNIST > MNIST, are more sensitive the discrete mechanism; 3) when increasing the number of total clients, the model utility loss between continuous and discrete models are diminishing. For example, while having 350 clients in FL, the difference between continuous and discrete models is less than 1% and 2% for all tasks in the i.i.d and non-i.i.d settings, respectively. These results verify the previous analysis in Theorem
https://pytorch tttps://pytorch.or//uoooio wwg//utoriaa/bbgginee/iiiiiiiiie tuttii/ieiiiiaiiototttaaaaaaaaa wutooota....tt. html. vided from Pytorch for MNIST and from VGG11 for balancing the memory usage and the model utility.
the default setting pro- For SVHN and CIFAR- 10, we use VGG9 that is modified
Note that,
'The precise definition of ASR varies under different attacks. We will give the precise definition in the
subsequent sections.
7

---

Under review as a conference paper at ICLR 2022
50
65
22
251
Mor-ID
Nav-D
RaviD
NDe-1D
301112311991
humber ortockes
Number attackers
antackors
(a) MNIST
(b) FMNIST
(c) SVHN
(d) CIFAR-I0
Figure Effect of number of attackers F with 100 clients in i.dd and non-i.i.d settings.
4.3, In summary, the discrete mechanism more practical for customer-related applications, e.g., smartphones or 1oT devices, since it usually involves millions of clients during training.
Evaluation on Availability Poisoning Attacks. The integrity poisoning attacks are evaluated from multi-angles. In Figure 4, we fix the number of clients 100, and evaluate the increasing number of attackers in both i.idd and non-i.i.d settings. The attack success rate (ASR) in APAS is the utility loss between before and after attacks. The results show that one few attackers, i.e., small fraction of attackers of all clients, cannot successfully poison the model. For example, when there are more than 100 clients, any single adversarial client only gets the 0% ASR for all tasks in both i.d and non-i.i.d settings. Figure 5 and 6 show the results of model utility with a fixed number of attackers in both i.i.d and non-i.i.d settings. The results consistently show that for a fixed number of attackers, increasing the number of the total clients will improve the global model's 's utility.
Compared with prior defense against IBAs (Blan- chard et al., 2017; Bagdasaryan et al., 2020; Shen et al., 2016; Fung et al., 2018), based on our cur- rent knowledge, not many existing works have well studied the availability poisoning attacks in FL. Due to the adversary have more flexible attack op- tions of the training process in APAS, most APAs could easily break the defense solutions for IBAs.
Table 1: FEDDISCRETE against IBAs
No Defense
FEDDISCRETE
Attack
ASR
ASR
CAS
81.90%
2.40%
0.00%
0.00%
DBA
81.30%
4.70%
0.00%
0.00%
Table 2: Compare with other defense basclines
Evaluation on Integrity Backdoor Attacks. Next, we evaluate the FEDDISCRETE against the state-of-the--rt backdoor attacks, such as constrain-and-See
Defenses
ASR
UL
No defense
81.90%
2.40%
(CAS) (Bagdasaryan et al., 2020) and DBA (Xie et al., 2019) in Table 1, where utility loss (UL) is the difference between clean and poisoned models, and the at- tack success rate (ASR) in IBAs is the percentage of poisoned samples that are classified the target class by the backdoored model. Due to the variants of studied tasks in prior works, we evaluate the most common task, i.e., CIFAR-10, for all backdoor attacks in the i.i.d setting. We the default attack settings same as the original paper: one attacker for CAS and four attackers for DBA. Here, ASR is the backdoor attack success rate, and UL is the utility loss compared with none-attack FL. The results show that FEDDISCRETE can easily defend the backdoor attacks against CAS and DBA. By mitigating the impacts of the attackers, in addition to the successful IBAs defense, FEDDIS- CRETE can even improve the model utility.
Krum
100%
35,50%
FoolsGold
100%
39.90%
model utility
Auror
100%
66. 10%
FedDP
0%
13.30%
FEDDISCRETE
0%
0%
We also compare FEDDISCRETE with other backdoor defense systems against CAS in FL: Krum (Blanchard et al., 2017), FoolsGold (Fung et al., 2018), Auror (Shen et al., 2016), and FedDP (Geyer al., 2017) in Table 2. The results show that Krum, FoolsGold, and Auror can not well defend the backdoor attacks in FI.. DP can achieve a good backdoor defense, but scarifies the model utility. FEDDISCRETE can achieve the best ASR and UL in all defense methods.
Discussion Client Inference Attacks. Prior inference attacks (Blanchard et al., 2017; Pyrgelis et al., 2017; Ganju et al., 2018) require to get the original or similar local client model, i.c., a contin- uous local model. Then, the malicious server could explore the privacy of the training data from its
8

---

Under review as a conference paper at ICLR 2022
PWID
64)
63
20
45 33 73
Mamber clants
Number of ciinents
Number clarrs
(a) MNIST
(b) FMNIST
(c) SVHN
(d) CIFAR-I0
Figure Effect of number of clients N with fixed attackers (F 1,5, 10) in i.d settings.
Fel
Pwlp
50
50 Mamber dierts
100
15
100
45 chints
clenss
Number of caimss
(a) MNIST
(b) FMNIST
(c) SVHN
(d) CIFAR-
Figure 6: Effect of number of clients N with fixed attackers (F 1, 10) in non-i.i.d settings.
continuous local model. However, a discrete local model cannot be used to infer any valuable infor- mation. The malicious server could compute the continuous global model, but it would not be useful to explore the privacy information of each client, Another popular inference defense technique is the homomorphic encryption (HE) (Gentry et al., 2009). Although HE can perfectly defend the infer- ence attacks from the malicious server, requires high computation cost and cannot defend against other weight-basee poisoning attacks. Similarly, differential privacy (DP) (Geyer al., 201 7) could also be used for defending inference attacks. However, Hu et al. (2021) have shown that DP will lose much utility of the global model while providing strong privacy protection. FEDDISCRETE is concise but practical approach that can naturally defend the client inference attacks from the malicious server with a more efficient communication protocol.
LIMITATIONS AND DISCUSSIONS
FEDDISCRETE defend both data poisoning attacks and weight poisoning attacks in FL., since data poisoning attacks are equal to change the clean model to the poisoned model. However, FED- DISCRETE also has two limitations: (1) It could not work in the cross-silos federated learning set- ting. If we want to mitigate the adversarial impact and achieve a non-biased aggregated model, it requires there are many clients could join in training in each round. However, it would not be an issue in the cross-devices setting involving thousands or even more clients in each round of training; (2) could not defend the byzantine attacks through our discretization aggregation mechanism, since the aggregated model represents the majority of interests in FEDDISCRETE. However, the byzan- tine defense via robust local training could still work in our framework, where the local client can recognize the poisoned model or only leverage the valuable information from the poisoned model to train the local model via knowledge distillation (Lee et al., 2021). Our next step is to further explore novel defense mechanism that be more scalable for various adversarial environments.
7 CONCLUSION
In this paper, we proposed new FL approach that applies the discrete mechanism with adaptive weight range for protecting FL. To our best knowledge, it is the first work that don't 't require the server to do excessive computation, but successfully defend against various attacks, including availability poisoning attacks, integrity backdoor attacks, and inference attacks. We also theoretically analyze the utility, robustness, and convergence of our proposed discrete mechanism in FL. One limitation of this work is the proposed system cannot defend the attacks well with a small number clients or face large fraction of the attackers, which becomes the next step of our future research.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovie, Qsgd: Communicatonn-fficient sgd via gradient quantization and encoding. Advances in Neural In- formation Processing Systems, 30:1709-1720 2017.
Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov. How to backdoor federated learning. In International Conference on Artificial Intelligence and Statistics, pp. 2938-2948. PMLR, 2020.
Peva Blanchard, EI Mahdi EI Mhamdi, Rachid Guerraoui, and Julien Stainer. Machine learning with adversaries: Byzantine tolerant gradient descent. In Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 118-128, 2017.
Clement Fung, Chris JM Yoon, and Ivan Beschastnikh. Mitigating sybils in federated learning poisoning. arXiv preprint 1rXiv::808.. 04866, 2018.
Karan Ganju, Qi Wang, Wei Yang, Carl A Gunter, and Nikita Borisov. Property inference attacks on fully connected neural networks using permutation invariant representations. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pp. 619-633, 2018.
Craig Gentry et al. A fully homomorphic encryption scheme, volume 20. Stanford university Stan- ford, 2009.
Robin Geyer, Tassilo Klein, and Moin Nabi. Differentially private federated learning: A client level perspective. arXiv preprint arXiv:1712.07557, 2017.
Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.0677 2017.
Hongsheng Hu, Zoran Salcic, Lichao Sun, Gillian Dobbie, and Xuyun Zhang. Source inference attacks in federated learning. arXiv preprint vrXiv:210900655) 2021.
David Kempe, Jon Kleinberg, and Éva Tardos. Maximizing the spread of influence through social network. In Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 137-146, 2003.
Jakub Konečný, H Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency. arXiv preprint 160v::6100005990 2016.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Keita Kurita, Paul Michel, and Graham Neuhig. Weight poisoning attacks on pre-trained models. arXiv preprint arXi::2004006660, 2020.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Geoffrey Hinton. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1 1):2278-22 22244, 1998.
Gihun Lee, Yongjin Shin, Minchan Jeong, and Se- Young Yun. Preservation of the global knowledge by not-true self knowledge distillation in federated learning. arXiv preprint arXiv:1166003997, 2021.
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. In MLSys, 2020.
WANG Luping, WANG Wei, and LI Bo. Cmfl: Mitigating communication overhead for feder- ated learning. In 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS), pp. 954 964. IEEE, 2019.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-effitele learning of deep networks from decentralized data. In Artificial Intelli- gence and Statistics, pp. 1273-1282. PMLR, 2017.
10

---

Under review as a conference paper at ICLR 2022
Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. Exploiting unintended feature leakage in collaborative learning. In 2019 IEEE Symposium on Security and Privacy (SP), pp. 691-706. IEEE, 2019.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Mustafa Safa Ozdayi, Murat Kantarcioglu, and Yulia R Gel. Defending against backdoors in fed- erated learning with robust learning rate. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pp.
2021.
Apostolos Pyrgelis, Carmela Troncoso, and Emiliano De Cristofaro. Knock knock, who's there? membership inference on aggregate location data. arXiv preprint 17Xiv:1708.06144 2017.
Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečný, Sanjiv Kumar, and H Brendan McMahan. Adaptive federated optimization. arXiv preprint 1rXiv:2003.00295 2020.
Amirhossein Reisizadeh, Aryan Mokhtari, Hamed Hassani, Ali Jadbabaie, and Ramtin Pedarsani. Fedpaq: A communication-effiee tization. In A/Stats, pp. 2021-2031. PMLR, 2020.
federated learning method with periodic averaging and quan-
Shiqi Shen, Shruti Tople, and Prateek Saxena. Auror: Defending against poisoning attacks in col- laborative deep learning systems. In Proceedings of the 32nd Annual Conference on Computer Security Applications, pp. 508-519, 2016.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference tacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP), pp. 3-18. IEEE, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv, 2014.
Jacob Steinhardt, Pang Wei Koh, and Percy Liang, Certified defenses for data poisoning attacks, In Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 3520-3532, 2017.
Lichao Sun, Yingtong Dou, Carl Yang, Ji Wang, Philip S Yu, Lifang He, and Bo Li. Adversarial attack and defense on graph data: A survey, arXiv preprint arXiv:1812.10522. 2018.
Ruixiang Tang, Mengnan Du Ninghao Liu, Fan Yang, and Xia Hu. An embarrassingly simple approach for trojan attack in deep neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 218-228, 2020.
Hongyi Wang. Kartik Sreenivasan, Shashank Rajput, Harit Vishwakarma. Saurabh Agarwal, Jy- yong Sohn, Kangwook Lee, and Dimitris Papailiopoulos. Attack of the tails: Yes, you really can backdoor federated learning. arXiv preprint ar arXiv:2007.0500 2020.
Jianyu Wang, Zachary Charles, Zheng Xu, Gauri Joshi, H. Brendan McMahan, Blaise Aguera y Ar- cas, Maruan Al-Shedivat, Galen Andrew, Salman Avestimehr, Katharine Daly, Deepesh Data, Suhas Diggavi, Hubert Eichner, Advait Gadhikar, Zachary Garrett, Antonious M. Girgis, Filip Hanzely, Andrew Hard, Chaoyang He, Samuel Horvath, Zhouyuan Huo, Alex Ingerman, Mar- tin Jaggi, Tara Javidi, Peter Kairouz, Satyen Kale, Sai Praneeth Karimireddy, Jakub Konecny, Sanmi Koyejo. Tian Li, Luyang Liu, Mehryar Mohri, Hang Qi, Sashank J. Reddi, Peter Richtarik, Karan Singhal, Virginia Smith, Mahdi Soltanolkotabi, Weikang Song, Ananda Theertha Suresh, Sebastian U. Stich, Ameet Talwalkar, Hongyi Wang, Blake Woodworth, Shanshan Wu, Felix X. Yu, Honglin Yuan, Manzil Zaheer, Mi Zhang, Tong Zhang, Chunxiang Zheng, Chen Zhu, and Wennan Zhu. A field guide to federated optimization, 2021a.
Jianyu Wang, Zheng Xu, Zachary Garrett, Zachary Charles, Luyang Liu, and Gauri Joshi. Local adaptivity in federated learning: Convergence and consistency, 2021b.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mniss a novel image dataset for benchmark- ing machine learning algorithms. arXiv, 2017.
11

---

Under review as a conference paper at ICLR 2022
Chulin Xie, Keli Huang, Yu Chen, and Bo Li. Dba: Distributed backdoor attacks against feder- ated learning. In International Conference on Learning Representations, 2019.
Chulin Xie, Minghao Chen, Pin-Yu Chen, and Bo Li. Crfl: Certifiably robust federated learning against backdoor attacks. arXiv preprint 2rXiv:2006. 006600883, 2021.
Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients. In H. Wallach, Larochelle, A. Beygelzimer, F. -'lcé-BBuc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
12

---

Under review as a conference paper at ICLR 2022
A APPENDIX
A.1 PROOF OF LEMMA 4.2.
Notice that for any scalar w R" and / S S by the definition of M(. 1, u), one can verify that [[M(w; L, u)] = w, therefore [[M(w;; u)] = w. Furthermore,
Proof.
BlIlM(w; u) - w0l
B[[[M(w;; u)l, [w],)
[B[[M(w;; w))3] (um13)
((u 1)[w]] - lu -|wl3))
27
u 1
|wml -
2 +
2
2
j=1
1
2
d
u -u++
|
))
2
d(u 1)2
[w]] -
2
2
4
j=1
Moreover,
d
11
1
[w]]
u [u+11 ))
j=1
A.2 PROOF OF THROREM 4.2.
Before proving the Theorem 4.2, standard probability bound is required.
Lemma A.1. Let (X,) be sequence of i.i.d continuous random variables, whose support is R. Then for any E IR
max X; 2 S <nP(X; 2 b).
iE(n]
Proof.
IP
max iE(n]
X; 2 = - Pmmxx X, b) = 1 PPX b,...,,,, < b)
iE(n]
=1 P(X; < b) = 1 (1- P(X; 2 b))" for any j
j=1
Let ft) = 1- nt - (1 (1-t)" for t E 10 As f'()) < and f(0) = 0. f() < 0 for all t E [0, 1].
Therefore, - (1 1)" nl. Take 1 = P(X; 2 b). arrive
max X; b) < <P((X; 2 b). iE[n]
PP
Now are ready to prove Theorem 4.2.
13

---

Under review as a conference paper at ICLR 2022
Proof. For any [d]. || |wااا u' - 1f and Var ((w0); - (w/)))) S (u' - 1m /4 due to Lemma 4.1. By Bernstein inequality, for any j E [d] and c >
Ke²
IP ([ww', [wm'ulsl 2() S exp
2 k Σies, Var ((0*)) (w'Mlj) 3c(u² - 24)
KE²
I 2exp
Tu-=1ey² " -10
By Lemma A.1,
Ke²
IP max jEld
| [wbklll 2()
s 2d exp
(ut-rey
3 Ee(u² 11)
(u' ( 3
Therefore, for any B
0,
there
exists
O
such that
P (max)eçd) [[0*]5 [w0'll <c) holds with probability at least B.
A.3 PROOF OF THEOREM 4.3
Proof. Due to the weight discretization mechanism, the adversary can only return u or / for each coordinate of the model weight. In order not return the correct information, the adversary could choose to return the opposite feedback to attack the model, i.e., return u if the original return is 7, and vice versa. Therefore, we denote, for any S w S u.
-/6 14 w.p. - h, w.p.
Mad
Under the scenario that there are F attackers, for any j E [d].
E
[
E
1
[M(w))l3
Σ
N
I=N- F+1
N F
(wily
+
Σ
(4)
N
N
1=1
N==
N F
N
+ +(h + F
[
Σ
[uils
(5)
N
F+1
L
A.4 PROOF OF THROREM 4.4
Proof. The proof is inspired by the analysis in Theorem 4 of (Li et al., 2020).
At the 4th round, for the all e St, define ai+1 w' + [sg Eies; (w) w²). :++ w P
To proceed with the analysis, we first introduce some notations.
w²), and
w arg min... (ww w²) = (w) + う ||w - w²ll x1+1 is the ghost global model if the discretization mechanism not applied to the local model weights; w0+1 another ghost global model as if all clients participate in the tth round training and no discretization mechanism is applied wo is the exact minimizer of the strongly convex function (,(w). These points reference points are crucial for the analysis. Define the gradient residual e = VF(Ww, + p(w w +1 w' - [VF((ttt)) + ie Leftl. Therefore,
w²). then
10 1+1 r0 = Σ pi(unt+1 w²
1 Σ (w
1 Σ peltt 1 [E(n)
(6)
/ sEin
ve(m
14

---

Under review as a conference paper at ICLR 2022
Since J chosen to satisfy of hi
Amin, then h;ww; w²) is j--strongly convex. By the strong convexity
|100 - 10 < (un - (Vh,
(h,
"
w k+
Vh,
(h;(
which, together with the fact that w ++1
is the minimizer of (:(w), implies
| -20,
20,+111
Vhi(wo - (1, (10
1
[h,(ao,+))) 1)|
f [[ + p(w - w')l
I ?
VF((w')|
Again use the same analysis, has | ww'l 1/2 FF''")). Therefore, together with Eq. 7,
w01l S 1o wi!+|| || + || w' 1 n
++7 ||
(8)
Therefore, one can bound the distance from the ghost global model w²+1 to the current global weight as
++ w'
Σ
(1 1+1 -w')
1 Σ
+ w''l
=
ps
KE(M]
SE[N]
1 1+7
Pi
[F(("))]|
(by Eq. 8)
ie[N]
+
(Jensen' Inequality)
kE|N]
1 1+7
EE [
B(1 + 7)
[f(w=)) |
(by Assumption 4.1 (3))
(9)
Note that
Σ Pe (VFr(uttt)
- -241 VF((w())
5 Σ pi (1) V (ww VF((w')) + +||/+t1)
iE(m]
i [g]
ps (L | +1 w00l|+ |||+|e+t111
iE(m]
Eq.
[((1 +2) Σ Pi [F](w))
11
iE(m)
L(1 +
+7 E. E.(VF.(ww)
Eq.
L(1 + 7)
+77 ||
B
(10)
p
15

---

Under review as a conference paper at ICLR 2022
By Assumption 4.1 (1), one has
L w²)
f(10+11) S s(w") + Vi(w"") (1o0
+ 2
|
Eq. 6 S f(w0)) + {s(W0')
ps
(w
+ 1 Σ peo
+ 2
L
|100 - 20
1 iE(m]
iE(m)
= =f(w") vf(w0')
1 Σ
Pi (VF(((t)))
- egtt VF((w"))
- 1
v((((()
1 sE/m]
L
+ 2 | - w0'12
S f(ww))
1
||( -
1 f(
Σ P: (VF(((tt))) -
11
iE|m
||000t1 - w0'l?
+ 2
S f(ww))
1
|[( + =1l5(o0)]
Σ Pi (VF((ut)))- - -cttt- -VFi((0)
iE|ml
L ||m0tt1 w0'1
+ 2
Eq. E0. S
s(w0) - - _ 1|²+ B ( [(1 ((1+7) +.) |V((m'))]
4
B(1 + 7)
+ 2
)2
V ((w0))
(11)
[1-7B LB(1+ (11++)
L(1 + 7)2
= =(ww')
[V/(w'))1
(12)
21²2
By mean-value theorem and triangular inequality, for some a [0,
s(
S 5s(10())) + (/oo + (1 a)wa |||
|0 +1
< 51(10())+ + (IV (+1 + (1 - o)w (+1 f || + [//(w'))]] |100
w-11|
S f(w + (L ||a00t11 + (1 w'll + V ff(w")]] 110001100000 w0-1|
-
- f(w + (4() w - w0 + / will + ff
1
(13)
Taking expectation with respect to the random index set Sy one gets
(s S 11 + (I. | - w01l + (v I|) Es! |1 8 201
+ LEs; w -
1. -w'l
< f(w + (L 1 - woll + | I|| Es; || 10 - w +1|
+ LL,s; ((| 10 - w + |0001 will) | - -001-=10+11 x0+1
=s(w++)) + (2L 1 will + || Es; || -
- w0*1||
+ LBs; |||||| || 1
(14)
16

---

Under review as a conference paper at ICLR 2022
By the sampling scheme, one has
Es!
[
11
1
E,
w
++1 20"+112
|
E,
++
w²
+ 1w' - wo'-wwth w0 2(w ++ - w²) (ww"=="=" -
w H+1
w0d ||
(by Ex( - w²*1)
S
B9.
(1 + 7)3
-
E,
|
13
(1
+ 7)2
<
(by Assumption 4.1 (3)).
(15)
|
f12
Combining Eq. 14, Eq. 15, and Eq. 9, together with the fact that Es; 11 /Esy /w0++1 ur*+1 1|² as a result of the Jesen's inequality, one reaches to
w0 5
B²
(1+7)2
B
(1+7)
B²
(1 22+))
Es:1(( S//550 (1
21
+
VIS
112
VIS
18/1
n12
LB² + 7
B(1 2(1+7)
=s(
(2
8
1)
(16)
S
a /
Combine Eq. 16 and Eq. 11, one reaches to
Es [s S f(w0) - (1-7B- LB(I+7) L(1 + +)
14j
2/12
LB 7)²
18 (2V\SA) +1)+ B(++))
) FV/ST|
||
(17)
Taking the expectation with respect to the discretization mechanism, EM [ = w0 by Lemma 4.1. Since f is T--mmothh
17

---

Under review as a conference paper at ICLR 2022
L
Emlf(w(t!)) - f(w
Emlllotte
=(w
L 1
BM
Σ
(M(w
-w) -(w w))
2 |//12
FES
2
< 1(1
L
Σ
(w)
w)
(by Lemma 4.1)
8|Sl82
NES
I. +
I +(()t))
Σ kE(m
2
c+1
20
Σ
(wstt [+ w) [wwftt w)
8/Sl8
S f(
Σ (E(M)
w||
Σ
( +1
w 1| ++ w
8/yl8
LN
S {f((ott))
Σ iE(M)
w0|²
(by 2 ||lll |له|| s ||a|| + 11613)
8|S!
LN 8|S//2mnin Pmnin
S <((w))))
Σ EE
1 ++
20 e0/
Pmin
LN 8|
$ {s(w++)
Σ Ps TE/ul
(+1
1012
< f(w
LN 8 Pmiin
Σp. Ps
(1+7)2
[V((2')) 2
(E/m)
LN 8|Si
B² + 7)2
I f(w
(18)
n
Put Eq. 18 and Eq. 17 together, reach to
1 11
LB(1 + 7)
L(1 + 2)
E [f s f(w')
up
2n22
(B)
B(1 (11+))
LN + 7)2
(2V18) +1)+
|| |[/(('))
)1²
S
8|S (.Sll2Pminn Pmin
(19)
When there is no adversary, then |S'| = K, then
EMss; (/ s f(w') - (ood)>xI|Vs((iiii
Finally, taking the total expectation with respect all randomness and by telescoping, one reaches
V (w) |11 < f(wo) - f(w"))
Divide I' on both sides, then
T T
B V (w)))) < [(wo) s(w0
min +E[T-1]
<
kT
18


Reviews:
[{"review_id": "ZklvgmBi0En", "reviewer": "Reviewer_zPLT", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The main contribution of this work is the use of discretization mechanism for better robustness of federated learning, which is novel as far as I see. It is interesting that the mechanism improves robustness and privacy while sacrificing little utility when the ratio of adversaries is small. The proofs in the appendix are not carefully checked. \n\nMy main comments follow.\n1.\tThe paper is not well-written. For example, the proof of Lemma 4.1 is called Lemma 4.2 in the appendix, and it is not completed. Moreover, some sentences are difficult to understand, e.g., in the introduction of the availability poisoning attacks (APA) in section 2, “the attacker can either control the global model’s utility on target tasks or decrease the most of the model’s utility as the optimal attack strategy.” Is difficult to understand and it does not introduce well on what APA does exactly. Also, in section 6 it says the proposed method could not defend Byzantine attacks, but it also says some Byzantine defense could still work with the proposed method. Can the authors elaborate more on this?\n2.\tThe robust convergence guarantee may be vacuous. I’m worried that the assumption required for Theorem 4.4 can be easily invalid. In theorem 4.4, it requires the assumption of $\\kappa>0$, where $\\kappa$ is a sophisticate term defined by a term subtracting several positive terms. Where is the definition of $\\bar{\\mu}$ in the definition of $\\kappa$? More importantly, when does this assumption $\\kappa>0$ holds? Moreover, from the experiments it seems that the proposed method can only tolerate a small fraction of adversaries (e.g., up to 10%), which consolidates my worry that the robustness guarantee is vacuous.\n3.\tThe experiments could be more complete. For example, I’m wondering how other baselines perform in the experiment of availability poisoning attack (Figure 4,5,6). Currently it only shows the proposed method. Moreover, I think many details of the experiments are not revealed. For example, how does the APA in your experiments performed?\n4.\tThe security claim may not be well-supported. I understand the intuition that the discretization mechanism improves the privacy and security, but I’m not convinced that “a discrete local model cannot be used to infer any valuable information” as claimed in the paper. There are only general discussions about the security claim, but no supported theorems (like differential privacy guarantee) nor supported empirical evidence (client inference attack experiments). \n\n\nMinor comments\n\nIn line 4 of Algorithm 1, are there typos in the definition of $l^t_i, u^t_i$? It reads as subtracting a vector from a scalar. \n", "summary_of_the_paper": "This paper proposes a secure federated learning framework against weight poisoning. The key component in the framework is the discretization mechanism which works well with a sufficient number of clients. Theoretical analysis of the robustness and convergence is provided. Lastly, numerical analysis is provided to verify the performance of the proposed method.", "summary_of_the_review": "The idea of discretization mechanism for the purpose of robustness is novel and interesting to me, but I’m worried about the quality about the paper: (1) the paper is not well-written; (2) the robust convergence guarantee may be vacuous; (3) the experiments could be more complete; (4) the security claim is not well-supported. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "R0eQ6sC0zxE", "reviewer": "Reviewer_QJJA", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength\n1.\tImportant topic.\n\n2.\tUsing discretization seems to be an interesting idea.\n\n3.\tWell written.\n\nWeakness:\n1.\tIn Corollary 4.3.1, the theoretical robustness is based on “Let N be infinity” and “fixed F attackers”. This does not make sense to me. In this case, won’t existing FL methods be robust? \n\n2.\tThe proposed FedDiscrete is obviously vulnerable to adaptive attacks. For instance, an attacker can upload manipulated l and u to the server in line 4, Algorithm 1. What if l=-infinity and u=infinity? They will then be chosen in line 5 and this will result in a non-sense model.\n\n3.\tIt seems that the utility loss is large in some cases. For instance, in Figure 3, the accuracy may drop from 95% to 70%.\n\n4.\tDiscretization has been applied to federated learning. For instance, signSGD [A] sends the sign of the gradients. I think the authors may need to discuss the existing methods and empirically compare with them. Also, stronger attacks and defenses are not compared. \n\n[A]  signSGD: Compressed optimisation for non-convex problems. In ICML, 2018.\n\n[B] A Little Is Enough: Circumventing Defenses For Distributed Learning. In NeurIPS, 2019.\n\n[C] Local Model Poisoning Attacks to Byzantine-Robust Federated Learning. In USENIX Security Symposium, 2020. \n\n[D] Provably Secure Federated Learning against Malicious Clients. In AAAI, 2021.\n\n[E] CRFL: Certifiably Robust Federated Learning against Backdoor Attacks. In ICML, 2021. \n\n\n\n\n", "summary_of_the_paper": "Federated learning (FL) has been shown to be vulnerable to weight poisoning attacks. An attacker who controls malicious clients can poison the clients’ model weights such that a backdoor to perform availability poisoning attacks, integrity backdoor attacks and inference attacks. In this work, the authors proposed a new FL algorithm called FedDiscrete, which probabilistically discretizes the clients’ model weights into two different values. The authors derived the convergence of FedDiscrete and empirically showed its performance against existing attacks. However, I am worried about the theoretical analysis on the robustness, as well as the empirical robustness against adaptive attacks.", "summary_of_the_review": "Comparison with existing attacks and defenses is insufficient and adaptive attacks are not considered.  Theoretical analysis relies on strong assumptions. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "PNwCmkKxFoU", "reviewer": "Reviewer_kHbi", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The work lacks novelty. Stochastic quantization is already well-studied in federated learning [1]. Binary quantization is just a special case of more general k-level quantization. The robustness result and the remark do not make sense to me. The remark seems to say that if poisoning rate and $|u-l|$ is small, the approach is robust. However, when these constraints hold, simple clipping would have the same effect so why bother to discretize?\n\nThe writing also needs huge improvement.\n\nOverall, I do not recommend accepting the paper.\n\n[1] McMahan, Brendan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. \"Communication-efficient learning of deep networks from decentralized data.\" In Artificial intelligence and statistics, pp. 1273-1282. PMLR, 2017.", "summary_of_the_paper": "The paper proposes to use stochastic quantization to increase robustness of federated learning.", "summary_of_the_review": "I do not recommend acceptance because of (1) lack of novelty; (2) writing quality; (3) robustness results are hard to interpret.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "2vRtqporJog", "reviewer": "Reviewer_GZ4V", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- the paper is well-written and easy to follow\n- the paper considers a variety of attacks against federated learning, including both security and privacy attacks\n\nWeaknesses:\n- The paper lacks a discussion of the advantage of robustifying the algorithm at the client side over the server side in the introduction. The authors mention that the server-side defense changes the role of the central sever from being honest to trusted. Trusting the server might be a compromise on privacy. However, for security attacks, why should the clients trust the server to defend against the attacks given that the server is often an entity incentivized to train a high-performing model in practice? \n- Section 3: it is mentioned that layer-wise bounds could reduce the variance but no experiments could be found to validate this point. \n- Evaluation on Avaliability Poisoning attacks: The authors should show the federated learning with continuous gradient as a baseline. \n- Evaluation on Integrity backdoor attacks: The experiments show some well-known defenses like Krum, FoolsGold, and Auror are not effective at all, which is a bit counter-intuitive. Any insights into why these defense do not work well? Is the finding generalizable to other datasets and model architectures? \n- Discussion on Client Inference Attacks: The claim \"However, a discrete local model cannot be used to infer any valuable information\" is not sufficiently justified. Given the existing work on black-box model inversion attacks, like [1], one could even recover the training data from model output. It is reasonable to expect that some private information about training data can be inferred from discretized gradients in multiple rounds as it is more fine-grained information than model output. \n[1]: https://arxiv.org/abs/1902.08552", "summary_of_the_paper": "The paper proposes a federated learning algorithm that discretizes the client updates in order to achieve better robustness against various attacks. The contributions include (1) the development of the algorithm, (2) the theoretical analysis on the utility, robustness, and convergence of the algorithm and (3) empirical studies of the algorithm's robustness.", "summary_of_the_review": "Discretization is widely used for reducing communication burden in federated learning. This paper is the first to consider its effect on robustness, which is an interesting idea. However, some of the claims made in the paper are not grounded. The evaluation of the algorithm is also limited. In some setting, it's not clear the advantage of the discretized gradient over continuous gradient; in other settings, the evaluation is limited to a specific dataset and hence it's not clear whether or not the findings are generalizable. Also, the privacy inference attack setting is not evaluated. So I think the current version of the paper does not meet the standard of ICLR.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This manuscript proposes a quantization approach to improve adversarial robustness. Reviewers agree that the problem studied is timely and the approach is interesting. However, note concerns about the novelty compared to closely related work, the quality of the presentation, the strength of the evaluated attacks compared to the state of the art, among other concerns. There is no rebuttal.
Decision: Reject