Robust Federated Learning with Majority Adversaries via Projection-based Re-weighting
Authors: Xiaoyang Wang
Klara Nahrstedt
Oluwasanmi O Koyejo
Venue: Submitted to ICLR 2023 (2023)
Keywords: Federated learning
robustness
adversarial attack
majority adversary

Abstract: Most robust aggregators for distributed or federated learning assume that adversarial clients are the minority in the system. In contrast, this paper considers the majority adversary setting. We first show that a filtering method using a few trusted clients can defend against many standard attacks. However, a new attack called Mimic-Shift can circumvent simple filtering. To this end, we develop a re-weighting strategy that identifies and down-weights the potential adversaries under the majority adversary regime. We show that our aggregator converges to a neighborhood around the optimum under the Mimic-Shift attack. Empirical results further show that our aggregator achieves negligible accuracy loss with a majority of adversarial clients, outperforming strong baselines.

Under review as a conference paper at ICLR 2023
ROBUST FEDERATED LEARNING WITH MAJORITY AD- VERSARIES VIA PROJECTION-BASED RE- RE- WEIGHTNGG
Anonymous authors Paper under double-blind review
ABSTRACT
Most robust aggregators for distributed or federated learning assume that adver- sarial clients are the minority in the system. In contrast, this paper considers the majority adversary setting. We first show that filtering method using few trusted clients can defend against many standard attacks. However, a new attack called Mimic-Shift can circumvent simple filtering. To this end, we develop a re-weighting strategy that identifies and down-weights the potential adversaries under the majority adversary regime. We show that our aggregator converges a neighborhood around the optimum under the Mimic-Shit -hift attack. Empirical re- sults further show that aggregator achieves negligible accuracy loss with majority of adversarial clients, outperforming strong baselines.
INTRODUCTION
Federated learning (FL) is a leading framework for collaboratively training a machine learning (ML) model over local datasets. The decentralized nature of FL systems has raised concerns about vul- nerability as adversaries can connect to an FL system like other benign users and corrupt the ML. model while evading detection by standard means (Kairouz et al.
(021..
To this end, there is grow-
ing literature on the adversarial robustness of FL (Blanchard et al.
2017
Chen et al. 2018 Xie et al.) 2022 He et al. 2022b), par-
2019b Rajputt al. 2019b Xie al., 2020 Karimireddy al. 202 la
ticularly where adversaries can upload malicious updates. Most existing defenses assume that the adversarial clients are the minority the system (Blanchard et al.] 2017] Chen al.] [018 Rajput et al. 2019b; KKrimireddy et al. 2021a; He et al. 20022b). However, in a federated scenario, the decentralized nature means that relatively straightforward for the adversary to be the majority and thus break existing defenses. We call such an adversary the "majority adversary".
Our work joins growing literature on robustness with majority adversaries, e.g., Xie al, (2019b) 2020), motivated by noted practical vulnerabilities. Although Shejwaalkrr et al. [202] argue that the number of registered clients in a production system (e.g., GBoard) may be too large for the adversary to compromise majority of them, they neglect the client availability issue in FL. In particular, Kairouz et al. (2021) suggests that, at any given time, only a subset (< 1%) of clients are available for the server. Such low client availability allows the adversary become the majority and overwhelm the server utilizing compromised networked devices (e.g.. 1oT devices) in a similar way as the common distributed denial-of-servic (DDoS) attack (Specht & Lee| [2003] Bonguet & Bellaiche 2017). Some other settings such as crowd-sourced training (Ryabinin & Gusev] (0200) (a.k.a. volunteer computing) are perhaps even more vulnerable to majority adversaries because the crowd-sourcing systems do not implement access control allowing the adversary to connect an arbitrary number of clients as volunteers.
We consider the adversarial robustness of federated learning against class of attacks where an ad- versary aims to decrease the accuracy of the trained MI model by uploading malicious updates, In particular, we are interested in Mimic-type attacks (Karimireddy et al. 2022). as is discussed later in this section. key assumption in our setup is the existence of a few trusted clients, e.g., with secure hardware support. We call these trusted clients "reference clients". In practice, the number of reference clients could be as small as two in each round. Similar approaches have been considered in existing works (Xie al.) 2019b; 2020). One option for secure hardware is the trusted execution vironment (TEE) (Pinto& & Santos 2019), which guarantees that the program is not Byzantine. TEE is so far commercialized (e.g., on Google Pixel (GoogleBlog). Apple iPhone (AppleSupport). Sam-

---

Under review as a conference paper at ICLR 2023
sung phones (SamsungDewelooper). ). Recent works (Mo et al. 2021) bring TEE support to federated learning systems.
We propose combination of defenses filtering and projection-based re-weighting. Our first de- fense is a filtering method that constructs a spherical accept region and excludes the updates outside it. The center of the accept region is average update from a few trusted clients with secure hard- ware support. The radius of the accept region is the sample variance of reference updates times a scaling factor. Although the filtering method is effective against several standard attacks (e.g., sign flipping attack, Gaussian attack), this filtering is easy to circumvent.
Building on the recently proposed Mimic attack (Karimireddy et al. 2022) shown to break many existing defenses, we develop an improved attack method called Mimic-Shift. Mimic-Shite clients send malicious updates which slightly shift away from benign updates and mislead the aggregated updates away from the expected update. The slight shift makes Mimic-Shift hard to detect, and the calibrated shifting direction can corrupt the aggregated model. To perform the Mimic-Shif attack, we consider man-in-the-middle (MITM) adversary capable of intercepting the message between the clients and the server. Computer security researchers have extensively studied the MITM adver- sary, but existing encryption solutions can be too expensive for resource-constraine For example, the AES (advanced encryption standard) encryption only has a throughput of around 50 MB/s even with a powerful desktop CPU (Gleeson et al. 2014). With GB moderate size neural network, the encryption takes more than 20 seconds on the client-side, draining the computational resources and increasing the client dropout rate.
client devices.
Our second defense is a projection-bare based re-weighting method to deal with the Mimic-Shift attack under majority adversary regime. The main idea to measure the influence of each update on the aggregated update, then down-weight the updates with high influence. Specifically, we compute the scalar projection of the aggregated update on each client's update. The intuition is that the majority adversarial clients can significantly mislead the aggregated update, resulting in a large scalar projection. Note that the filtering and re-weighting methods complement each other because the re- weighting defense does not deal with the aforementioned standard attacks, as is discussed in Section
We further provide some theoretical analysis of our methods under the Mimic-Shift attack. First, false-positives in filtering can eliminate benign update and perturb the aggregated update. Regard- ing this concern, we show that the false positive rate decreases quickly w.r.t. the number of reference clients and the scaling factor of the accept radius, suggesting that our filtering method can work with few reference clients and conservative scaling factor. For the re- re-weighting phase, the probability of malicious updates having larger scalar projections than benign updates increases as the adversary takes more shares in system, complementing existing results (He et al. (022b) that guarantee more robustness as the share of the adversary decreases. Additionally, we discuss the performance of our method under a conventional minority adversary setting. Finally, we show that, in convex setting, our method converges with a rate of O ( to a neighborhood of the optimum. Our contributions summarized as follows:
We develop the Mimic-Shift attack and show that Mimic-Shift circumvents many defense methods in federated learning.
We develop two-stage defense using filtering and re-weighting to defend against broad class attacks.
We theoretically analyze our strategy and outline conditions under which it helps.
Empirical results on FEMNIST (Caldas et al. 2018). CelebA (Liu al.) 2015) and Shakespeare (McMahan et al. 2017) datasets show that our aggregator recovers a near-optimal model under a majority adversary setting with Mimic-Shir margin. Also, our method only loses up to 2.4% accuracy under conventional minority adversary settings. Additional empirical results demonstrate that our method is robust a broad class of attacks, including standard Gaussian and sign-flipping attacks as well as an improved Mimic-Shift- Var attack.
attack, outperforming existing methods by a large
2

---

Under review as a conference paper at ICLR 2023
2 RELATED WORK
Many existing works on Byzantine robust machine learning assume the adversary is the minority in
the system, based on clustering (Blanchard et al. 2017), median (Yin al. 2018 voting (Bernstein
et al., 2019). bucketing (Karimireddy et al., 2022). . and robust estimation (Daaa & Diggavil (202]) I.
However, these methods can fail once the adversary becomes the majority. A few papers (Xie al.,
20196] 2020) leverage a validation dataset to improve Byzantine tolerance to arbitrary numbers of adversarial clients but constructing global validation dataset in federated scenario with local private datasets is infeasible. Other redundancy-basse methods with more robustness guaranteess do not apply to federated learning because they assume a centralized setting with control over the data allocation on each node (Chen et al.] 20188 Rajput et al.] 2019a). Recently, a hopping-basedl method (Karimireddy al. [202Tb] He et al. (022b) showed success against minority adversary in a federated learning setting. Although it is relatively straightforward to combine the clipping with our reference clients, we find that the training is unstable, as is discussed in Section
Our paper focuses on model poisoning attacks, which aim to decrease the accuracy of the trained MI. model by uploading malicious updates (He et al., (022b)). Other attacks, including data poisoning (Steinhardt et al. 2017 Wang et al.] (021)) and backdoor (Wang et al.) 2020;; Xie al. 2021 are beyond the scope of this work. We focus only on training the centralized model. Additional considerations such as personalization are known to be handled effectively using robust centralized
training, followed by local fine-tuning Li et al. 2021), thus are beyond the scope of this paper.
PROBLEM SETUP
We assume federated learning system where N benign clients collaboratively train ML model f: X' + Y with d-dimensional parameter ( coordinated by a server. The N benign clients include NR reference (trusted) clients. There are N' adversarial clients who aim to corrupt the ML model during training. The ith i E [1, [I.... N + NN), client has n, data samples, being benign for i e N] or being adversarial for i E [N + 1,..,, N + N'). The federated learning is conducted in T rounds. In round & E [1...., T7, the server broadcasts a model parameterized by Ç--1 to each client. We omit the subscript while focusing on one round. Then, the 2th client optimizes G-1 with their local data samples and report G, to the server. We define pseudo-gradient gr.; G-1 Gt being the difference between the locally optimized model and the broadcasted model from the previous round. Note, for simplicity, that we will often use the term "gradient" to refer to the updates. Once all the gradients are reported in, the server aggregates the gradients and produce N+N"
new model with parameters Ss using the following rule: Ge S1-1 EN+N"
ΣN It.i-
N Our goal is to minimize risk function over the benign clients: F(() D
Fi(()
EN
ΣN
Tu Em
-innEE ED, (e[[((;; v),, where E: R x y + R is a loss function.
3.1 THREAT MODEL
Following the standard practice (Blanchard et al. 2017). consider omniscient man-in middle (MITM) adversary that knows gr, Vi E [I.....N and can tell which clients are reference users, e.g., by observing the device identifier in the message. An omniscient advensary helps explore the limits of the defense. However, an omniscient adversary is not mandatory for our Mimic-Shift attack, which can work with partial information Section will show. The MITM adversary also owns a majority of clients in a system and adopts an attack from the Mimic family (Karimireddy al. 2022). Specifically, we assume the following Mimic-Shift attack.
Mimic-Shift At each round, the MITM adversary first intercepts the gradients from benign clients. Then the adversary computes the average reference gradient ğR >
NR nR,
nn; 9R, from the
Σ
reference users indexed by R, and the average benign gradient ğ = D users, including the reference users. Then, all the adversarial clients report g In + (ōn g) to the server.
N iii
Di= n. gi from all benign
The Mimic-Shif attack is effective because it tries to push the aggregated gradient away from the expected gradient. The reason is that ğ is estimated with more clients and data samples. A simple concentration argument suffices show that g will be closer to Blgg than gn with high probability,
3

---

Under review as a conference paper at ICLR 2023
resulting in ΩR 9 pointing away from Elg). Empirical results in Section 6 show that Mimic- decreases the accuracy twice as much as Mimic (Karimireddy et al. 2022; which sets g' = 9s for an i N).
The Mimic-Shiff attack is also difficult to detect because the malicious g' has the same distance to the reference IN as the benign g. We will see how this property lets Mimic-Shift bypass distance-based defenses in Section
METHOD
Our robust aggregator has two phases, filtering and re-weighting. The filtering phase, applied first, excludes the gradients outside the accept region defined by the reference gradients, defending against standard attacks. Then, the re- weightgg phase further down- nown-weights the potential malicious gradi- ents within the accepting region of the filtering phase, targeting the Mimic-Shir attack.
4.1 PHASE I: FILTERING
In the filtering phase (Algorithm(]) I first compute the sample mean of reference gradient ma NR
Eiz NA -9n, making a guess of where the good gradients might be. Next, we set m as the center of a spherical accept region. Then, we compute the sample variance of the reference gradients, Σxn Il,n, m The sample variance 8R is further scaled up by a tunable Ng-1 c. The product c sg is the radius of the spherical accept region. Finally, we remove all the updates outside the spherical accept region. This filtering is effective against attacks that do not know where the expected gradient is (e.g., Gaussian attack) or do not carefully calibrate the adversarial gradient (e.g., sign flipping attack), is shown in Section
Sn
Algorithm Filtering.
Input:
A set of reference gradients, {9R, li (1,..., Na));
A set of reported gradients. {ga i E {l...., + N';
A hyper-parameter c;
Aggregator:
1: Compute the sample mean of reference gradients, ma := EnR NR
NR -RR.;
N-NR 19p, mm
2: Compute the sample variance of reference gradients, sn. :=
NR-1
3: return {9: i E {I, N + N'} A ||g, mll S c sn);
4.2 PHASE 2: RE-WEIGHTING
Suppose a majority adversary misleads the aggregated gradient. In that case, the malicious gradients likely have stronger influence the aggregated gradient compared to benign gradients. Here, measure the influence via scalar projections.
The ceweeghtingg defense is outlined in Algorithm The scaling weights are designed to augment benign gradients and nown-weight adversarial updates. This implemented using a monotonic re- re-scaling I of the scalar projection between the aggregate and the gradients (step 4). The mono- tonic re-scaling amplifies the difference between the scalar projections, making large projections larger. Therefore, the potential malicious gradients would be down-weighted more. However, the monotonic re-scaling may also lead to an over-up-weighting on certain benign gradients, which are far from g" such weigeighting leads to instability in the training process. Thus, we also include a clipping operator (step 6). The clipping addresses the over-up- weeihhinn issue by specifying bound.
An additional benefit of using the power function and clipping is that they prompt set of uni- form weights on benign gradients, stabilizing the training. If we set to be sufficiently large such that the maximum 8; is greater than Nr (Err 8; maxie 8i), we have 8; = r, Vi +
In the description and experiments, use power function. Optimizing the choice of monotonic scaling left for future work.
4

---

N): Under review as a conference paper at ICLR 2023
arg [ St- In practice, set ie to 10 and 7 to 0.6. Both hyper-parameters generalize well to the three datasets in our experiments.
Note that the filtering phase complements the re- re-weighting phase because standard attacks can cor- rupt the scalar projections via uploading 0 or flipping the sign. For a class of inner product-based attacks (Xie et al., 2019a), an additional clipping operator that prevent the scalar projection s; from being negative can help.
Algorithm Re-weighting.
Input:
A set of filtered gradients, i.c., the results of Algorithm Two hyper-parameters and т;
! {gr i {I, N+})
Aggregator:
1: Compute the aggregated gradients of all users, -" = Em= No
Σ 96i
2: Compute scalar projection of g" on each 9s. 4':
19i
3: Normalize the vector [81,-., 8N such that ||a||] = NF:
4: Take the 4h power of each 84, == [s .i....... N.y
5: Normalize the vector [.,,... SN₂ such that | NF;
6: Clip the values smaller than s, 8; == max 8/ , Vi [1,..,FF] Nr):
7: Normalize the vector [$1,..., S such that
NF;
8: Re-weight each 9s using 8ts 9r := 21 Vi {1,... Ne
9: return gj = ENr 21
E. 96i
THEORETICAL ANALYSIS
We first show that the probability of not filtering out benign gradient increases at rate of O(1 Ng w.r.t. the number of reference clients NR and ( 22) w.r.t. the scaling fac- tor in the accept radius. Then, discuss conditions for the probability of down- down-weighting the malicious gradients to increase as the adversary owns more clients in the system. Additional discus- sion considers how strategy may still help even if the aforementioned conditions do not hold, providing insights to certain of empirical results. Finally, we study the convergence of our method under convex setting. Before proceeding, outline some additional assumptions.
To simplify tedious notation, we assume all users have the same number of samples because the difference in sample size can be merged into the difference between gradients. Assumption 1. Uniform sample size:
n. = nj Vi vi j.
(1)
Assumption 2. The gradients follow a hierarchical distribution in all rounds: Stage 14 ~ P(u). ~ P(a), Vi E (),
Stage 2 94 ~ (P4,0457 Vi (1,
where Stage is distribution where the random variable μ has finite expected value E[[]] and finite non-zero variance Var[p]. Stage truncated isotropic Gaussian distribution with a truncation threshold + the L2 norm.
Assumption 3. The variance of gradients is bounded:
s 10.
(3)
Assumption 4. The sample variance of reference gradients greater than 0:
0 <<< S < SR.
(4)
Assumption 5. The La norm of expected and estimated gradients bounded:
0 D<T 7 $ || Σ + Vi E {0, 10.1
(5)
0 0<7 < || < + Vi E 1..,
5

---

Under review as a conference paper at ICLR 2023
(a) Case 1.
(b) Case 2.
(c) Case 3.
Figure I: Three cases with different parar--arameterss.o c. The red dot Elu] and the blue dot Jin- The solid circle represents the accept region with radius 8..g. The dash circle and dash dot circle(s) are two spherical contours around E( Condition e 2 2 [[[m
holds in Case where all the points outside the accept region are more than pes away from E[u]] In Case 2 and Case 3, there
points between the two contours that have the same distance to E [[ but are not always inside accept region.
FILTERING (STAGE I)
The filtering phase removes all the gradients that more than s away from the reference m. in terms of Euclidean distance. However, the data distribution D; may vary across clients a federated learning setting. Such non-i.i.d.ness increases the risk of filtering out benign gradient. We start our analysis by considering the expected gradients:
Lemma 6. Suppose there are NR reference clients among N clients, and the sample variance of gradients is 8 Ry under Assumption let c [E[n]] All Then, for any E [1,... N}, with 4-Varlu]2 2..R
probability at most
we have ||p - Pal 2 C. where PR = E Nn PR..
Remark 7. Condition 2 1BM-FI
guarantees that all μ;s outside the accept region are at least caR away from Elja], as Figure 7 shows, enabling concentration argument. Otherwise, bounding 2
the probability of appears outside the accept region is hard without additional assumptions on P(n).
6 suggests that the probability of filtering out benign gradients decreases at rate of ( The following lemma further shows the probability of violating the assumed condition
Lemma
2 2 |[[a]-Al| decreases rate of O No.
*R
Lemma 8. With Assumptions and for a fixed accept radius c 8R, with probability at most 1-Var(u]² Nj. "f
have 2. [E|μ
The following theorem combines and extends the result on expected gradients to estimated gradients.
Theorem 9. Under Assumptions and with probability at least (2-o+./2m)d (11
1- 4-Var
). (1-
c2.83
4-Vall ), we have ||:: m < C-SR- NA
Theorem 9 shows that the risk of filtering out a benign gradient decreases quickly w.r.t. e and Nos enabling our strategy of using small number of reference clients and a conservative
5.2 RE-WEIGHTING (STAGE 2)
Suppose the adversary adopts the Mimic-Shifi attack and bypasses the filtering defense. We would like to figure out the probability of assigning a higher scalar projection to the malicious gradient g' than benign gradient gt, E NJ....NN..
Theorem 10. Suppose all the gradients pass the filtering phase. Let the aggregated gradient be
MN
NN+N' "4 Let be the angle between ğ and
g" = w + w g', where (w, w') =
ENER
N
NNNN

---

Under review as a conference paper at ICLR 2023
g
g
0
g
00
8"
9
0
(a) 8- < case.
(b) General case.
Figure 2: Two cases where the adversary mislead the aggregated gradient 9" by different degree. is benign. g" is a mirror of malicious g w.r.t. g", along which the scalar projection equals to s'.
w S cos such that H" 1 - -0 Then, ur 100l- Il''l
g', H" be the angle between 3" and g'. Assume under Assumptions
I
let Vgg = 11y tanh(O 2 )* with probability at least (2.0 22
(1-
Varlu a²
Varfal (7g- aa)
da, we have s' 8, vi fI,...,N.
Theorem 10 suggests that if 2 < 0, the probability of down down-weighting malicious gradients more than benign gradients (i.e., s' > 8;) increases as the adversary owns more clients and samples in system (i.e. w t). The main idea is: the minimum amount of deviation between 9; and for having
8i s' increases as w' increases because w' 1 0" L. This specific deviation is shown in Figure 2a the dark dash line with length least ||> tanh(O 2*)). Then, applying concentration
arguments yields the result.
5.2.1 ADDITIONAL DISCUSSION
Theorem 10 only deals with the case where 3" has a smaller angle with g' than g. However, in practice, we find that our method also achieves a decent accuracy even the condition 2- 0" < 0 does not hold. To gain some insight, suppose there is probability density function f of benign gradients. Then, the probability of having s; > s', Vi E {1...., N) equals the integral of over cone, which defined by spinning g' around g" as Figure 2b shows. In a federated scenario, the estimated 9; concentrates around its 14; which is not necessarily close to jt, resulting in small integral of f over the aforementioned cone.
5.3 CONVERGENCE ANALYSIS
So far, our analysis has focused the robustness of one update. The following extends the single step analysis to a convergence analysis. Before showing the theorem, we present lemma that quantifies the impact of false-positive filtering.
Lemma 11. Suppose F N Rd
+ Nx (0, 1} is a filteringfunntion let mask M = F N 1 - ||]|, MMT EN M, 9o and 8 9. Then, with Assumption
IN ) we have
||
2. 7
N
In the following theorem, we use assumed conditions to ease the reading. Later, we shall connect the assumed conditions to Theorems 9 and 10 and further support the assumed conditions with empirical results in Appendix
Theorem 12. Suppose the malicious gradient is filtered out with probability O, Vt T.....Tb. and weighted by s Z 1. Assume at most N₂. benign gradient are filtered out at each iteration. Define an aggregated gradient gi 6, follow the definitions in Lemma 11 es N E or,)) as Assumption 2 shows. Let Ç E Z be the model parameter, Ç* be the optimum, and
-w- -9t + . w (ji 8 + Et) + w' g. where ÿr and
IT 2T=16 Go. Assume sup < D and F Z R is convex. Let w- 2-N .2+²2 + w- +² w/ .tt,, 22 -2.w-c S.w.Co C² -. w.yt
and 17t V.. under Assumptions and we have:
-[ - F(() (4. +1.
2.w VT + D
T Σ t=1
|Io'll + ||ezll) (lleel + w w.
(6)
7

---

Under review as a conference paper at ICLR 2023
The convergence analysis shows that method converges to neighborhood around the optimum, whose size shrinks with fewer sulse-positive filtering (Theorem ! and more down- wweng-iigting (tuning k and in Algorithmm2) with higher probability (Theorem 10p the malicious gradients. Converg- ing to a neighborhood is common in previous noisy gradient descent studies (Wang et al. [021] He al. 2022a). Our result also suggests scaling up the learning rate the adversary
more clients. In practice, this scaling is handled by our re-weighting phase where the benign gradients are down-weighted by scalars less than 1.
EXPERIMENTS
We first compare the Mimic-Shift attack with the Mimic attack, showing that Mimic-Shift is more effective and the improved effectiveness remains with non-omniscient adversary (Mimic-Shift- Par). Then, we evaluate our defense strategy against the Mimic-Shift attack where our strategy outperforms strong baselines.
Additional Results Appendix C provides more results. We evaluate our defense via non- omniscient Mimic-Shift-Pal attack, an improved Mimic-Shift- Var attack as well as other standard attacks (e.g., Gaussian and sign-flipping). There are also additional experiment with fewer clients and various attack strengths, plots of the re-weighting vector s as well as convergence, and studies on defense paperpparamttes.
6.1 SETUP
We use three datasets, FEMNIST (FM) (Caldas et al., 2018), CelebA (CA) (Liu et al. 2015). and
Shakespeare (SS) (McMahan et al. 2017), with realistic non-i.id. partitions (Caldas et al.) 2018 implemented in the FedML library (He et al. 2020). The number of benign clients ranges from 143 to 500. We select 2 reference clients and 28 other benign clients at each round. The number of adversarial clients is adjusted accordingly. We report the detailed setup in Appendix including various hyper-parne parameters.
We employ seven baselines. including federated averaging (FedAvg) (McMahan et al. 2017 feder-
ated averaging with reference clients (Ref), coordinate-wise median (CM) (Yn a (018)) FLTrustt
(Cao et al. (02T), Krum (Blanchard et al. 2017), Krum with bucketing (Krum-B) (Karimireddy et al. 2022). self-centered clipping with reference clients (CClip-R) (Karimireddy et al. 2022] He
al. 2022b), Zeno' (Xie al. 2019b), and FLTrust (Appendix C.I (Cao al. 2021). Appendix
B lists the details of these baselines. The "Oracle" aggregator operates with benign gradients and
serves as a reference. Five attacks are considered in our experiments, including Mimic-Shie Shift- Var, particularly designed as strong attack for our proposed defense.
Gaussian
Draw a random update of from an isotropic Gaussian distribution (0, 200).
Sign-flipping Flip the sign of the estimated gradient gf -9 and report of to the server.
Mimic-Shift
Report g' = IR + (ōR g) to the server, as is shown in Section 3.1
Minic-Shift-Par
Randomly eavesdrop 20% clients per round and draw two clients reference.
Mimic-Shift-Van Var
Mirror the local update using In and report g. = Gn + (ōn g:) to the server.
6.2 MIMIC-TYPE ATTACK COMPARISON
Table shows the accuracy FedAvg aggregator under Mimic and Mimic-Shife attack. Here, the adversary owns 80% of the system, and the Mimic adversary mimies the first client in the system. Mimic-Shift attack constantly outperforms Mimic and is 116% more effective on average. Such advantages preserved under non-omniscient adversary.
8

---

Under review as a conference paper at ICLR 2023
Table Accuracy of FedAvg Aggregator under Attack with 80% Adversary
Attack Method
FEMNIST
CelebA
Shakespeare
Avg. Decrease
No Attack
861 mo (.000L)
.869 = mm (.0004)
.364 = mn (.000L)
.0004
Mimic
.693 (.168.))
.816 .m (.053.)
.345 mn (.019.)
.078
Mimic-Sio Shift
.621 mn (.240L)
.797 = om (.072.)
.169 = an (.195.)
.1691
Mimic-Shift-Paar
.663 (.198()
.812 001 (.054J)
.182 .01 (.182.)
.145
Note: Variance rounded up.
6.3 DEFENSE AGAINST MIMIC-SHIFT ATTACK
This experiment considers three settings where the adversary owns 0% 80% of system. The hyper-parameters of each aggregator are selected based on the 80% adversary setting and directly applied to the other settings. Table 2 shows the accuracy of our strategy and baselines under the Mimic-Shift attack.
Under majority adversary setting, our method outperforms all the baselines by large margin. The reason is that the median-basse based (CM) method picks the malicious gradient once the adversary becomes the majority. Clustering-bes based (Krum and Krum-B) methods suffers from similar issue. Other reference client-assisted distance-base based re-weighting strategy (CClip-R) and filtering strategy (Zeno') is not effective because Mimic-Shite carefully calibrates the malicious gradients that they are as close to the reference gradients as the benign gradients. Using only reference clients (Ref) outperforms existing robust aggregators but suffers from low client utilization.
We also find that our proposed defense yields the best accuracy under minority adversary setting, as discussed in Section 5.2.1 Under no adversary setting, our strategy weights the benign gradients nearly uniformly without interfering with the training. Our method occasionally outperforms the oracle. We hypothesize that the improved results can come from the up-weighting of the under- represented clients (Li et al. 2020), whose influence score is small.
Table 2: Accuracy of Aggregators Under Mimic-Shift Attack
Adv %
Data
Oracle
Ours
FedAvg
Ref
CM
Krum
Krum-B
CClip-R
Zeno'
FM
.861
.840
.621
.761
.535
.554
.525
.562
.527
CA
.870
.877
.797
.820
.782
.787
.865
.792
.805
80%
SS
.364
.360
.169
.236
.189
.186
.183
.160
.194
Avg
.698
.692
.529
.606
.502
.509
.525
.505
.509
FM
.861
.871
.797
.761
.855
.537
.603
.583
.533
CA
.870
.893
.862
.820
.857
.836
.858
.839
.845
40%
SS
.364
.340
.291
.236
.210
.089
.202
.243
.191
Avg
.698
.701
.650
.606
.627
.487
.554
.555
.523
FM
861
.644
861
.761
.801
.727
842
.760
.751
CA
.870
.879
.870
.820
.865
.856
.863
.662
.860
00%
SS
.364
.357
.364
.236
.217
.189
.307
.266
.258
Avg
.698
.700
.698
.606
.628
.591
.671
.629
.623
Note: The numbers are average accuracy over three runs.
CONCLUSION AND FUTURE WORK
This paper shows two methods for improving the adversarial robustness of federated learning under a majority adversary regime. Empirical results various settings and against a broad class attacks demonstrate the proposed methods' effectiveness. Additional theoretical analysis is conducted under the Mimic-Shiff attack regime, showing conditions under which the proposed method helps. Further exploring the limitations of learning with majority adversaries is good next step.

---

Under review as a conference paper at ICLR 2023
REFERENCES
AppleSupport. Secure enclave.
secure enclave sec59bob3iff/wet http:ptt..appll..ccm//.p///////s
Jeremy Bernstein, Jiawei Zhao, Kamyar Azizzadenesheli, and Anima Anandkumar. signSGD with majority vote is communication efficient and fault tolerant. In International Conference on Learn ing Representations, 2019. URL hattts::/openrryye net wwt.:eettfoonaae Bixh
Peva Blanchard, EI Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer. Ma- chine learning with adversaries: Byzantine tolerant gradient descent. In Guyon, U. Von Luxburg. S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gar- nett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Asso- ciates, Inc., 2017. URL /tttps://proceestt www.roceeeingss.eerissssss nwwripsurpp.... cacss.cc/paaee cc/pppprr/2111//11eeee 68 2.0.220000.....12222222626666663
Adrien Bonguet and Martine Bellaiche, survey of denial-of-service and distributed denial of service attacks and defenses in cloud computing. Future Internet, 9:43, 2017.
Sebastian Caldas, Peter Wu, Tian Li, Jakub Konečný, H. Brendan McMahan, Virginia Smith, and
Ameet Talwalkar. LEAF: A benchmark for federated settings. CoRR, abs/1812.01097,, 2018.
URL http::/tpa//x..../o///am org/ 1tt.b//1111.000999
Xiaoyu Cao, Minghong Fang. Jia Liu, and Neil Zhenqiang Gong. Fltrust: Byzantine feder- ated learning via trust bootstrapping. ArXiv, 0bs/2011..1995, 2021.
Lingjiao Chen, Hongyi Wang, Zachary Charles, and Dimitris Papailiopoulos. DRACO: Byzantine- resilient distributed training via redundant gradients. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Pro- ceedings of Machine Learning Research, pp. 912. PMLR, 10-15 Jul 2018. URL https: 7/proceed wnweeeediigg...oo ngs. ml wwwpprrsssseeee. /v00//80//// chen 81 httml
Deepesh Data and Suhas Diggavi. Byzantine-resilient high-dimensional sgd with local iterations on heterogeneous data. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th In- ternational Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 2478-2488. PMLR, 18-24 Jul 2021. URL https: https:://rroceeenggg.. preess/139/datazla.
Minghong Fang, Xiaoyu Cao, Jinyuan Jia, and Neil Gong. Local model poisoning attacks to [Byzantine ) federated learning. In 29th USENIX Security Symposium (USENIX Secu- rity 20), pp. 1605-1622, 2020.
James Gleeson, Sreekumar Rajan, and Vandana Saini. Gpu encrypt: Aes encryption mobile devices. 2014.
GoogleBlog.
Pixel 6: Setting
standard for mobile security.
https://security: new-iing--e-w-enne
[0oglebbog..com/222 com/2021/10 /ttp:////o/////////ee//e/e-///-/ html Published: October 27, 2021.
Chaoyang Hc, Songze Li, Jinhyun So. Mi Zhang, Hongyi Wang. Xiaoyang Wang, Pranecth Vepakomma, Abhishek Singh, Hang Qiu, Li Shen, Peilin Zhao, Yan Kang, Yang Liu, Ramesh Raskar, Qiang Yang, Murali Annavaram, and Salman Avestimehr. Fedml: A research li- brary and benchmark for federated machine learning. CoRR, abs/2007.13518, 2020. URL https: /tttss//arxiv.. wtg/ /ttbss////rai..orrraa////007 13518
Lie He, Sai Praneeth Karimireddy, and Martin Jaggi. Byzantine-robust decentralized learning via self-centered clipping. CoRR, abs/2202.01545, 2022a. URL htt.::s://axxi...rgg/////////
0602.01544
Lie He, Sai Pranceth Karimireddy, and Martin Jaggi. Byzantine-robuu decentralized learning via self-centered clipping. ArXiv, abs/2202.01545. 2022b.
10

---

Under review as a conference paper at ICLR 2023
Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Ad- vances and open problems in federated learning, Foundations and Trends@ in Machine Learning, 14(1-2):1-210, 2021.
Sai Praneeth Karimireddy, Lie He, and Martin Jaggi. Learning from history for byzantine robust optimization. Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp- 5311-5319. PMIR, 18-24 Jul 2021a. URL hww.:https///im httts:://proceed ngs. w.wpsss/es1399 karimireddy html
Sai Praneeth Karimireddy, Lie He, and Martin Jaggi. Learning from history for byzantine robust optimization. Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp.
5311-5319. PMLR, 18-24 Jul 2021b. URL /ttt::::/ppoccednggsnn..-.. wtt.mm....pp.ss..3... karimireddy html]
Sai Pranceth Karimireddy, Lie He, and Martin Jaggi. Byzantine-rober learning on heterogencous datasets via bucketing. In International Conference Learning Representations, 2022. URL
htttps://ppenrevie
net. forum?id= jXKKDEE1xx
Tian Li, Maziar Sanjabi, Ahmad Beirami, and Virginia Smith. Fair resource allocation in federated learning. In International Conference on Learning Representations, 2020. URL https:// openreview. net forum? hid=-yexxlssDDD
Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. Ditto: Fair and robust federated learning through personalization. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 6357-6368. PMLR, 18-24 Jul 2021. URL httpstts::/ httt:s:://proceednngs.
mlr. A...presss/v19/1122 html
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Ar- cas. Communication--ffiicee Learning of Deep Networks from Decentralized Data. In Aarti Singh and Jerry Zhu (eds.), Proceedings of the 20th International Conference on Artifi- cial Intelligence and Statistics, volume 54 of Proceedings of Machine Learning Research, pp. 1273-1282 PMLR, 20 22 Apr 2017. URL https: /tttps://proceedinggs ml wttp://nn............eoo.e.sseee press/v54/ momahan178s html
Fan Mo. Hamed Haddadi, Kleomenis Katevas, Eduard Marin, Diego Perino, and Nicolas Kourtellis. Ppft: Privacy-prest prreccypr federated learning with trusted execution environments. In Proceed- ings of the 19th Annual International Conference Mobile Systems, Applications, and Ser- vices, MobiSys '21, pp. 94-108, New York, NY, USA, 2021. Association for Computing Machin- ery. ISBN 978 4503444384 doi: 00.1145/3458864446666 URL httpp:://doi.rrg///
1145/34588 ..444346660 346628
Sandro Pinto and Nuno Santos. Demystifying arm trustzone: A comprehensive survey. ACM Comput. Surv, 51))), jan 2019. ISSN 0360-0300. doi: 10.1145/3291047. URL https: 77doi. 100 145/3291047
Shashank Rajput, Hongyi Wang, Zachary Charles, and Dimitris Papailiopoulos. Detox: A redundancy-based framework for faster and more robust gradient aggregation. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems. volume 32. Curran Asso- ciates, Inc., 2019a. URL htt.ttts//pooceeeiiiite http:/oceeeiiigs...eiiis.....sss cttccc///pp//a wwtppppppppppp//....... 415185ea244 242e4444422222200004444499999989
.pdff
Shashank Rajput, Hongyi Wang. Zachary Charles, and Dimitris Papailiopoulos. Detox: A redundancy-based framework for faster and more robust gradient aggregation. H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Bue, E. Fox, and R. Garnett
In
11

---

Under review as a conference paper at ICLR 2023
(eds.), Advances in Neural Information Processing Systems, volume 32. Curran Asso- ciates, Inc.. 2019b. URL http:////sppoooccocooo..e.eeeess 112333224422226666445222222222--
Max Ryabinin and Anton Gusev. Towards crowdsourced training of large neural networks using de- centralized mixture-of-experts. In I Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin
(eds.), Advances in Neural Information Processing Systems, volume 33, pp- 3659-3672. Cur- ran Associates, Inc., 2020. URL / http:///nnggnenuuss.....oooaaaea cccpprppp.cccppppe///00/// fi F11e7225ddc500062222228555532222 Paperr
SamsungDeveloper. Samsung teegris. overview. html
https://developez www.saaac.mmmeeeeeeeeeeeeeeeeeee
Virat Shejwalkar, Amir Houmansadr, Peter Kairouz, and Daniel Ramage. Back to the drawing board: A critical evaluation of poisoning attacks on federated learning. ArXiv, abs/2108.1024), 2021.
Stephen M. Specht and Ruby B. Lee. Taxonomies of distributed denial of service networks, attacks, tools, and countermeasures. 2003.
Jacob Steinhardt, Pang Wei Koh, and Percy Liang. Certified defenses for data poisoning at- tacks. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS' 17, pp. 9781510860964.
Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN
Hongyi Wang. Kartik Sreenivasan, Shashank Rajput, Harit Vishwakarma, Saurabh Agarwal, Jy- yong Sohn, Kangwook Lee, and Dimitris Papailiopoulos. Attack of the tails: Yes, you really can backdoor federated learning. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 16070-16084, Cur- ran Associates, Inc., 2020. URL hatpltttt/////tt htttps::/ppccceedanggs neurips http://eengg..uuurpppppp..pp///a cc/paper/2020/ http://ff....leee22222aaaddda222
Paweeb--appeeeee pdf
Yunjuan Wang, Poorya Mianjy, and Raman Arora. Robust learning for data poisoning attacks. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 10859-10869. mlr pre.s potres//v33//wann//
PMLR, 18-24 Jul 2021. URL htttp::/
html
Chulin Xie, Minghao Chen, Pun-Yu Chen, and Bo Li. Cril: Certifiably robust federated learning against backdoor attacks. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th In- ternational Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 1132--11338 1182.. PMLR, 18 18-24 Jul 2021. URL httptt::/pppceeeeeeeeaeae .ml. mlr.
bress/v139/xie22 html
Cong Xie, Oluwasanmi Koyejo, and Indranil Gupta. Fall of empires: Breaking byzentine-tolerant SGD by inner product manipulation. In Amir Globerson and Ricardo Silva (eds.), Proceedings of the Thirty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI 2019, Tel Aviv, Israel, July 22-25, 2019, volume 115 of Proceedings of Machine Learning Research, pp. 261-270. AUAI Press, 2019a. URL http: ptttt::/pprcceeiinng
mlr wtw...eeess/wv//xieeee.. html
Cong Xie. Sanmi Koyejo, and Indranil Gupta. Zeno:
Distributed stochastic gradient descent with suspicion-based fault-tolerance. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume of Pro- ceedings of Machine Learning Research, pp. 6893-6901. PMLR, 09-15 Jun 2019b. URL //pppe///pcccceinggs mlr www.mm.....p......mm.mmmmme pwwps/v9s1 /te//xxe1990 xieelb.hhtm
Cong Xie, Sanmi Koyejo, and Indranil Gupta. Zeno++: Robust fully asynchronous SGD. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Ma- chine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 10495-10503. PMLR. 13-18 Jul 2020. URL hattp:://pooceteee http::///ccceeenng..ceem mlr. wwtps//eee/x.ee.eeeeee html
12

---

Under review as a conference paper at ICLR 2023
Dong Yin, Yudong Chen, Ranchandran Kannan, and Peter Bartlett. Byzantine- distributed learning: Towards optimal statistical rates. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 5650-5659. PMLR, 10-15 Jul 2018. URL https: pgg.e/eeooeeed..........oo mlr. mlr. wlr.preess/abb/iinn 8a html
13


Reviews:
[{"review_id": "ndHjEo6-16B", "reviewer": "Reviewer_VsGK", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper studies attacks and defenses for federated learning. In particular, \n1) The paper first observes that a “filtering based” attack that uses a few “trusted clients” can help bypass some attacks.\n2) Then it presents an attack called “mimic shift” that somehow improves previous attacks known as “mimic” attacks and bypasses the simple defense.\n3) Then it improves the defense to add a re-weighting step and shows that it can defend against some standard attacks. The idea is to use a bunch of trusted nodes and use them to approximate the statistics of the gradient, and use that to filter out the outliers. Also, re-weight the nodes to balance out their influence on the gradients.\nThe paper does some theoretical analysis of their defense (under quite a few assumptions). It also does some experimental study for the 3rd part above.\n\nI have provided more details below, but I think in summary, the paper suffers from an issue that many papers in adversarial/robust learning do: it does not study “adaptive” attacks that are specifically designed with the knowledge of a defense. That is why these works usually fall into a cat and mouse game of defense/attack/defense/etc. By now there are provably approaches in robust (poisoning resistant) learning that apply to federated learning as well. For example look up [1] and other papers on “robust statistics”. This paper ignores that line of work and merely studies heuristic approaches with much weaker guarantees\n\n[1] Jia, Jinyuan, Xiaoyu Cao, and Neil Zhenqiang Gong. \"Intrinsic certified robustness of bagging against data poisoning attacks.\" AAAI’21\n\nIn fact, the paper explicitly says “Other attacks, including data poisoning …  and backdoor (..),\nare beyond the scope of this work.”\nBut why is that so? Isn't their framing more general than yours?\n", "summary_of_the_review": "In summary, I think the main question of the paper is interesting. But the methods are not fundamentally new, are not adding real guarantees (as they can be broken in the next paper) even though such provable approaches exist, and the paper’s writing in the theory section is not great. Therefore, I cannot support it for ICLR acceptance, even though I acknowledge the merits of the paper.", "strength_and_weaknesses": "Strength: aiming to provide solutions for federated learning that resist many corruptions. It is also interesting to combine two different methods as a defense, as they could work well together.\n\nWeakness: the evaluation is not against adaptive attacks (who know the defense). The theoretical part has too many assumptions that are hard to interpret.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "g5iXsb7jh6", "reviewer": "Reviewer_ZCpp", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper focuses on robust federated learning against adversarial clients, which hinder the convergence and performance of federated learning via injecting malicious local updates. Unlike other previous works, this paper considers the majority adversarial setting, where the adversarial clients are more than those benign clients and thus break the existing defenses. Given the assumption of having few trusted clients for reference, this work proposes a two-staged defense combining filtering and re-weighting to defend against a broad class of attacks. Extensive experiments are conducted to verify the effectiveness of the proposed method, and theoretical analyses are provided.", "summary_of_the_review": "Overall, this paper proposes an effective method for the majority adversary setting of the robust federated learning problem. This paper could be further improved to make the clarification clearer and stronger, as well as highlight the technical novelty of the proposed method by adding more conceptual and intuitive comparisons.", "strength_and_weaknesses": "Strength:\n1. This paper explores a new setting, which differs from the assumption in most previous literature, i.e., the majority adversary setting. It is practical as indicated in Kairouz et al. (2021) [1].\n2. The proposed two-staged defense shows empirical effectiveness on several baseline methods.\n\nWeaknesses:\n1. The motivation for the proposed second-stage reweighting is not very clear. It could be better improved by adding a more intuitive example or illustration that the reweighting mechanism is the target choice for defending the mimic-shift attack.\n2. The technical novelty of the current presentation is limited, as the reweighting mechanism or filtering is commonly used to defend against poisoning attacks. It is encouraged to add more discussion about the rationality of adopting reweighting in this problem.\n3. It may lack some empirical evidence to verify the illustration in Figure 2.\n\nOther questions/comments:\n1. Is there any empirical evidence supporting \"the decentralized nature means that it is relatively straightforward for the adversary to be the majority and thus break existing defense\"?\n2. It is better to link some major clarification with that empirical evidence or support with reference.\n3. Why the performance of aggregators can still outperform the oracle when the adversary is 0% in Table 2?\n\n[1] Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurelien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 14(1–2):1–210, 2021.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "TCiE1-k_LD", "reviewer": "Reviewer_pinJ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a new type of attack named Mimic-Shift and designs a two-phase defend framework under majority adversary setting. The proposed framework combines classic filtering method defending standard attack together with a new re-weighting method reducing the influence of Mimic-Shift . In order to weaken the effect of averaged gradients from adversaries while strengthening the effect of average gradient from benign users, the proposed method utilizes projection to judge whether a client is a benign user or an adversary client in the setting where exists majority adversaries. The effectiveness of Mimic-shift and Mimic-shift-Par is shown and compared with other kinds of attacks. And the performance of the proposed method against attacks is demonstrated with the report of accuracies in several classic benchmarks of Federated learning (FEMNIST, CelebA, and Shakespeare), compared with some baselines designed for defending against standard attacks. ", "summary_of_the_review": "Overall, this paper proposes a practically effective way of attacking and a method against it; however, there is some flaws in both theory and experiments. Given these clarifications in an author response and some necessary complementary experiments, I would be willing to increase the score. ", "strength_and_weaknesses": "**Strength and Weaknesses:**\n\nThis paper does well in: (1) clearly explain the intuition behind their proposed methods. (2) the idea of judging benign/adversarial clients with projection on average gradient is innovative. (3) the proposed attack and defend method seem promising under majority adversary setting observed from the results of experiments. \n\nAs for the weaknesses, I have several concerns requiring the authors to clarify: (1) the paper doesn't convincingly clarify the extensiveness of the setting (Majority adversaries can be a common situation in Federated learning systems). (2) some parts of the results of experiments are not clearly analyzed in the paper. (3) The theory does not provide a theoretic guarantee for dropping accuracy caused by Mimic-shift. (4) the effect of filtering is not demonstrated in practice and missing of that may makes the validity of re-weight phase less convincing. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "RLzKJyTdXV", "reviewer": "Reviewer_odUz", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a new method to build robust FL. The paper performs theoretical analysis and evaluation on multiple datasets and baselines. ", "summary_of_the_review": "The paper proposes a new robust FL, performs theoretical analysis on the method, and compares with some baselines. The paper would be stronger if more baseline attacks and adaptive attacks are evaluated. ", "strength_and_weaknesses": "Strength\n\n+ A new robust FL method is proposed. \n\n+ Theoretical analysis is performed on the proposed method to show its robustness. \n\nWeakness\n\n- Some baseline attacks are not evaluated, e.g., [A]\n\n- Adaptive attacks are not considered. Defending against existing attacks is not hard. But the challenge is that attackers are always adaptive to the defenses. So we should evaluate adaptive attacks that adapt to the defense. \n\n- FLTrust description seems not correct. I thought the FLTrust does not only consider cosine similarity. It also uses ReLU to clip cosine similarity. \n\n[A] local model poisoning attacks to byzantine-robust federated learning. In USENIX Security Symposium, 2020. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject