On the Byzantine-Resilience of Distillation-Based Federated Learning
Authors: Christophe Roux
Max Zimmer
Sebastian Pokutta
Venue: ICLR 2025 Poster (2025)
Keywords: We show that Knowledge Distillation-based FL is naturally more resilient to malicious clients than parameter-sharing approaches
but we identify novel attack strategies that can still disrupt training and propose effective defenses against them.
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Federated Learning (FL) algorithms using Knowledge Distillation (KD) have received increasing attention due to their favorable properties with respect to privacy, non-i.i.d. data and communication cost. These methods depart from transmitting model parameters and instead communicate information about a learning task by sharing predictions on a public dataset. In this work, we study the performance of such approaches in the byzantine setting, where a subset of the clients act in an adversarial manner aiming to disrupt the learning process. We show that KD-based FL algorithms are remarkably resilient and analyze how byzantine clients can influence the learning process. Based on these insights, we introduce two new byzantine attacks and demonstrate their ability to break existing byzantine-resilient methods. Additionally, we propose a novel defence method which enhances the byzantine resilience of KD-based FL algorithms. Finally, we provide a general framework to obfuscate attacks, making them significantly harder to detect, thereby improving their effectiveness.

Published as a conference paper at ICLR 2025
ON THE BYZANTINE-RESILIENG OF DISTILLATION- BASED FEDERATED LEARNING
Christophe Roux", Max Zimmer'& & Sebastian Pokutta Department for Al in Society, Science, and Technology. Zuse Institute Berlin, Germany Institute of Mathematics, Technische Universität Berlin, Germany (roux, 2 zimmer, pokutta ) @zib.de
ABSTRACT
Federated Learning (FL) algorithms using Knowledge Distillation (KD) have received increasing attention due to their favorable properties with respect to privacy, non-i.i.d. data and communication cost. These methods depart from transmitting model parameters and instead communicate information about a learning task by sharing predictions on a public dataset. In this work, we study the performance of such approaches in the byzantine setting, where a subset of the clients act in an adversarial manner aiming disrupt the learning process. We show that KD-based FL algorithms are remarkably resilient and analyze how byzantine clients can influence the learning process. Based on these insights, we introduce two new byzantine attacks and demonstrate their ability to break existing byzantine-resilien resilient methods. Additionally, we propose a novel defence method which enhances the byzantine resilience of KD-based FL algorithms. Finally, we provide a general framework to obfuscate attacks, making them significantly harder to detect, thereby improving their effectiveness.
INTRODUCTION
Federated Learning (FL) allows training machine learning models while keeping data private. The most common FL algorithm is Federated Averaging (FedAVG) (McMahan et al., 2017), where clients train models on their local data and share the updated model parameters with central server. The server combines these parameters and sends back an updated model. While FedAVG has found success in diverse applications (Rieke et al., 2020; Ramaswamy et al., 2019; Yang et al., 2018; Sheller al., 2019; Chen al., 2019), it has notable limitations such as high communication costs due to repeatedly transmitting the model parameters, susceptibility to inversion attacks which reconstruct private data from the parameters (Melis al., 2018; Nasr al., 2019), or performance degradation due to non identically independently distributed (i.i.d)) local data among the clients (Kairouz et al., 2019). Most importantly for this work, standard FedAVG lacks resilience to byzantine clients, i.e., clients which behave adversarially. This has fueled efforts to create provably byzantine-resite FedAVG variants (Allen Zhu et al., 2021; Alistarh al., 2018; Mhamdi et al., 2018; Chen al., 2017; Zhu et al., 2023).
recent line of work uses Knowledge Distillation (KD) order address some of these challenges (Hinton et al., 2015). In general, KD transfers knowledge by training a student model the aggregated predictions of multiple teacher models. For FL, it can enhance or replace the transmission of model parameters by sharing the clients' predictions on public dataset. The server then uses KD to distill this information into its model. Such methods offer reduced communication overhead, enhanced privacy and robustness non-i.i.d. data, but they are not well explored in the byzantine setting (Sattlrr et al., 2020; 2021; Lin et al., 2020; He et al., 2020; Cheng et al., 2021; Papernot et al., 2017; Chang et al., 2019; Gong et al., 2022; Fan al., 2023).
In this work, we investigate KD-based Fl. algorithms in the byzantine setting, restricting ourselves to classification. While many different algorithms have been proposed in this area, we focus on a prototypical variant, referred to as Federated Learning using Distillation (FedDistill) because of
"Equal contribution

---

Published as a conference paper at ICLR 2025
its simplicity and the fact the clients communicate with the server only through predictions unlabeled public dataset. We analyze how this communication modality shariig predictions instead of model parameters-impace impacts the ability of byzantine clients to disrupt training, demonstrating that standard FedDistill offers greater byzantine-resilience than standard FedAVG.
We identify two key factors underlying this resilience: First, byzantine clients in FedDistill can only modify predictions, which are constrained to the bounded and relatively low-dimensional probability simplex. This contrasts sharply with FedAVG, where attacks occur in the unbounded, high-dimennional parameter space. Second, byzantine clients in FedDistill can only influence the server model indirectly through the distillation objective, whereas in FedAVG they directly manipulate the model parameters.
Guided by these insights, we introduce two novel FedDistill-ppecife attacks that successfully bypass existing byzantine-resilien FedDistill variants. In response, we propose a new defence mechanism that detects and filters byzantine clients by leveraging client information from both past and current communication rounds. Additionally, byzantine clients harder to detect and hence calling for further research on defence mechanisms. To summarize, our contributions are as follows:
present a general framework to obfuscate attacks, making
1. We analyze FedDistill, prototypical KD-based FL algorithm, in the byzantine setting, focusing on its unique communication method of transmitting predictions rather than param- eters to the server. Our analysis reveals that byzantine clients have only limited impact vanilla FedDistill, rendering it more resilient than vanilla FedAVG.
2. We introduce LMA and CPA, novel and effective byzantine attacks which specifically designed for FedDistill. Our extensive experiments demonstrate stronger disruption of the training process than previous approaches, even when comparing against existing byzantine- resilient FedDistill variants.
3. In response, we propose ExpGuard. novel defence mechanism for FedDistill that incor- porates information on client behavior in both current and past communication rounds. ExpGuard significantly improves byzantine resilience compared previous methods that rely solely on robust aggregation applied independently to each sample.
4. Finally, we propose HIPS, a general method to obfuscate byzantine attacks, making them harder to detect. In the presence of defence mechanisms, HIPS improves the effectiveness of byzantine attacks.
Our findings represent an important step forward in the analysis of byzantine FL, providing insights into both the vulnerabilities and defences of KD-based FL systems. By developing novel attack and defence mechanisms, we further advance the robustness of KD-based FL.
Outline. We begin by reviewing prior work on byzantine FL and applications of KD in FL in Section 2. We then introduce our problem setting, detail the FedDistill algorithm, and describe our experimental methodology in Section 3. In Section 4, analyze how byzantine clients affect the server model in both FedDistill and FedAVG, demonstrating that FedDistill is more resilient by design. We present our two novel byzantine attacks in Section 5.1, followed by our defence mechanism in Section 5.2. Finally, we describe HIPS in Section 5.3, our technique for making byzantine attacks harder to detect by obfuscating malicious behavior.
2 RELATED WORK
KD in FL. One common approach of employing KD FL to transmit information from the clients to the server via predictions on a publicly available unlabeled dataset (Li & Wang, 2019;
Chang et al., 2019; Cheng et al., 2021; Gong et al., 2022; Lin et al., 2020; Sturluson et al., 2021; Bistritz et al., 2020). Some works (Li & Wang, 2019; Cheng et al., 2021) assume the availability of labeled public dataset. Lin et al. (2020) and Sturluson et al. (2021) study augmenting FedAVG-type algorithms using KD. In this setting clients send model parameters well predictions public dataset to the server, meaning that they can disturb the training process both via the parameters and predictions.
2

---

Published as a conference paper at ICLR 2025
Byzantine FedAVG. Some erzantine-resilien, variants replace the mean with more robust al- ternatives for aggregating client updates (Blanchard et al., 2017: Mhamdi et al., 2018; Yin et al., 2021; Chen et al., 2017). Additionally, Karimireddy et al. (2022); Allouah et al. (2023) propose simple preprocessing steps that can be used to improve the byzantine-resiliene orthogonal the defences we propose in this paper and could be used to further improve their performance. However, replacing the mean with robust aggregation methods may disregard the honest clients' predictions, which can hinder performance, and remain vulnerable more sophisticated attacks (Xie et al., 2020; Baruch et al., 2019; Shejwalkar & Houmansadr, 2021). These issues have been addressed by considering historical client behavior throughout the training process (Alistarh et al., 2018 8; Allen-Zhu et al., 2021; Karimireddy et al., 2021). The previously mentioned approaches rely on the assumption that the data held by the clients are homogeneous (i.i.d... As the data becomes more heterogeneous (non-i.i.d.), it becomes harder to distinguish between a malicious update and an update created by dissimilar data which holds valuable information. Some recent works introduced variants of FedAVG that have limited eyzantine-resilienee in the heterogeneous setting (Karimireddy et al., 2022; El-Mhamdi et al., 2021). Note however that even in the benign setting, non-i.i.d. data poses serious issues, e.g. (Kairouz et al., 2019), which is why we restrict ourselves to the i..dd. setting in the main paper. However, we show that our attacks and defences are also effective in the non-i.i.d. setting in Appendix D.
These approaches
Byzantine FedDistill. Federated Robust Adaptive Distillation (Sturluson et al., 2021) is a variant of FedAVG that uses KD increase its byzantine-resilience in the non-i.i.d setting. After local training, the server aggregates the updated parameters of the clients using a weighted sum. The weights are computed based on how much the prediction of the clients conform to the prediction of the majority. Since this method relies on sharing the parameters and predictions on a public dataset, attacks as well as defences operate both in the prediction and parameter space. In contrast, the goal of this work is to isolate the effects of attacks and defences that operate purely in the prediction space. Most relevant to our work, Federated Ensemble Distillation (Chang et al., 2019) uses predictions on a public dataset for both client-server and server-client communication. The server only aggregates client predictions, sends them back, and clients train both the public dataset using the aggregated predictions as labels and their local data. They introduce Cronus, byzanttne-resilient aggregation method, which improves the resilience but suffers from bad empirical performance in the benign setting. Both of these works concentrate designing byzantine- algorithms and give limited attention to finding more effective byzantine attacks, testing their algorithms only against simple label flip heuristics.
3 PRELIMINARIES
We limit ourselves to the supervised classification setting and assume that there are N clients in total. Let H be the set of honest clients and B the set of byzantine clients, assuming that an a-fraction of the clients are byzantine where < 0.5, which is a standard assumption in byzantine FL to expect reasonable outcomes. Each honest client E [N] holds a private dataset DD sampled from the same distribution as the publicly available but unlabeled dataset Dpub. The objective of FedDistill is to train a classifier based only on D Ppb using the predictions of the clients, which are computed and transmitted in multiple communication rounds.
Given a sample x, we denote by (x(x)) the prediction of client in communication round / using neural network parameters wf.. The honest clients infer their predictions by using their local model h(2, unf) while the byzantine clients can send any vector in Ae. the c-dimensional probability simplex with e being the number of classes. We define (Dppb) daf (x) x E Dpmb) } as the set of predictions of client i on every datapoint in Dpub- We sometimes drop the dependence on i for simplicity. Further, we denote the aggregated prediction of all clients by F'(x;) and the aggregated parameters of all clients by w. At the end of each a communication round the server has received the individual client predictions Y (Dpub) ) of cach client i, but is unaware which clients are byzantine.
We assume that the byzantine clients have access to the individual predictions of the honest clients on D Dpub before sending their own predictions, allowing to base their attack on them. Byzantine clients are permitted to collude. However, their identities remain consistent across iterations, ensuring that the server can discer which client generated which set of predictions. These assumptions are analogous to byzantine FedAVG, where the byzantine clients also cannot change identity, can collude
3

---

Published as a conference paper at ICLR 2025
and are assumed to have access to the individual parameter updates sent to the server by the clients (Allen-Zhh et al., 2021; Alistarh et al., 2018; Karimireddy et al., 2021).
FedDistill FedAVG compari- son. Algorithm showcases Fed- Distill and FedAVG in unified way. At the beginning of each com- munication round, the server broad- casts its parameters ūü to the clients (Line 2). The clients train on their local datasets starting from these pa- rameters (Line 3). FedAVG, clients directly send their updated parame- ters w. +1 to the server (Line 4) and the server aggregates these parameters to obtain the new server parameters w (Line 5). In FedDistill, clients transmit information about the learn- pub- lic dataset (Line 6) and the server ag- gregates these and updates its model
Algorithm 1 Federated Learning (FL)
1: for communication round 0 to T 1 do
2:
SerVER: Broadcast parameters vi, to the clients
3: CLIENTS: Train on private datasets
FedAVG
4:
CLIENTS: Send updated parameters to server
5:
SERVER: Aggregate parameters to obtain wi!
FedDistill
6:
CLIENTS: Send public dataset predictions to server
SERVER: Train on public dataset with aggregated
client predictions to obtain 27." g++
8: end for
ing task via their predictions
9: Output: ur
by training on the public dataset using the aggregated prediction as labels (Line 7). Since in FedDistill, the server can only be influenced by the clients predictions on Dpuh, we can study the byzantine impact given this communication modality in isolation.
Experimental setup. We evaluate FedDistill on the CIFAR-10/100 (Krizhevsky et al., 2009), CINIC-10 (Darlow et al., 2018), and Clothing (Xiao al., 2015) datasets using the ResNet (He et al., 2016) and WideResNet (Zagoruyko & Komodakis, 2016) architectures. We keep 5% of the training datasets for validation and split the remaining data evenly among the clients. Each experiment is performed with multiple random seeds and we report mean and standard deviation.
To ensure feasible computational study, we set several parameters of the algorithms to fixed values. Specifically, we set the number of clients to 20 and the number of communication rounds to 10. We conduct ablations on these parameters in Appendix D. We set the number of total local epochs each client performs, i.e., training on their private datasets, to fixed value depending on the dataset used. Similarly, we set total number of communications and uniformly distribute these communications among the local epochs. We train clients and server using SGD with weight decay and a linearly decaying learning rate from 0.1 to O, momentum set to 0.9. Appendix contains detailed account of the experimental setup. Our code is available at github.comm///B1110///ddDDD
4 THE BYZANTINE SETTING
Due to the different communication modalities of FedAVG and FedDistill, byzantine clients influence the training in different ways. We refer to these modes of influence as threat vectors. In this section, we investigate these threat vectors and their impact, first comparing the byzannine-resillinnce of vanilla FedAVG and FedDistill as a motivating example.
Figure compares how FedAVG and FedDistill are impacted by byzantine clients when using two naive attacks. We measured the final test accuracy when varying the fraction of byzantine clients FedAVG: GN denotes FedAVG where byzantine clients send Gaussian noise instead of the updated model parameters. Analogously, FedDistill: RLF refers to FedDistill where for each sample, all byzantine clients return the same random label. While FedAVG collapses with only 10% byzantine clients (a= 0.1). FedDistill is remarkably byzantine-resiliem even underscore the inherent robustness of FedDistill, which we now discuss from theoretical perspective. Note that Figure I serves motivating example, more advanced aggregation schemes for FedAVG could easily counter the Gaussian noise attack. See Appendix D for comparison between FedDistill and state of the ceraantiee-resiliene FedAVG methods.
0.45. These experiments
4

---

Published as a conference paper at ICLR 2025
Threat vectors in FedAVG and FedDistill. In stan- dard FedAVG, the updated parameters are aggregated by taking their mean. In this case, can highlight the threat vector by decomposing the aggregation step (Line 5) into honest and byzantine part follows.
80%
Test Acouracy
80%
FedDisill: RLF
40%
FoLAC: ON
1+
W
Σ w KEB
20%
N
N
EN
FedAVG threat vector
0.0
0.1
0.2
0.3
0.4
Rysantine fraction
The byzantine clients can directly influence the new server parameters via their parameters w Setting the weights of a single client to wN -Eiz w suffices to achieve x w where T can be cho- sen freely. In other words, just one byzantine client is sufficient to arbitrarily perturb the new server pa-
Figure I: ResNet-18 on CINIC-10: Final test
accuracy of FedAVG and FedDistill, varying the fraction of byzantine clients for two naive attacks.
rameters.
On the other hand, in FedDistill the clients can only alter the server model via their predictions the public dataset Ppub- The server distills the knowledge from the clients based on their predictions on the public dataset (Line 7). Recall that for sample Ppub, the mean of the clients' predictions in EN
round is denoted by v. (x))
y (x)) and the server solves the optimization problem
N
min
Σ C(h(x, w), (,+++()))
(Pdistil) )
ZED
FedDistill
where L is a loss function such as the Cross-Entropy Loss (CEL) or Mean-Squared Error (MSE). By decomposing y (x) into byzantine (x+ der dN EiEB Y (x) and an honest part (x) [I-a Eien + (z), we can highlight the threat vector more specifically,
(x) a (z) (1) +1)a)))(1)) (x),
FedDistil threat vector
where a is the fraction of byzantine clients.
The byzantine resilience of FedDistill. There are two main reasons why vanilla FedDistill is less susceptible to byzantine attacks than vanilla FedAVG. First, the influence of byzantine clients on the server parameters in FedDistill is indirect. They can only affect the distillation objective (Pasan) through their predictions, which must then influences the server's 's optimization procedure. This contrasts with FedAVG, where byzantine clients can directly manipulate the through their parameter updates. Second, the threat vector in FedDistill is confined to the bounded probability simplex A., limiting the impact of perturbations, unlike in FedAVG, where the threat vector operates in the nigher-dimensionn
parameters
unbounded parameter space.
Since the server cannot distinguish between honest and byzantine clients, it has resort to distilling the knowledge from the aggregated predictions from all clients by minimizing (Paisun), despite its true objective being to distill knowledge only from the honest clients by solving
min
Σ C(h(z, w), Y (x)).
(P (Pnonest)
ZED
We show that if parameter w is stationary point of the perturbed objective (Pussin). ), it also lies within a neighborhood of a stationary point of the true objective (Phonest). Therefore, running stochastic neighborhood of the solution
gradient descent on (Paisin) still enables the server to converge would reach with (Phonest). The key intuition behind this result is that (Paisin) and (Phonest) differ only in their target labels, allowing us to demonstrate that the difference between their computed gradients scales linearly with the distance between YN and Yg:
L(w, VL+1) V.,C(w, v.+1 || s Ca |
D!41 S V2Ca,
5

---

Published as a conference paper at ICLR 2025
where used the shorthand C(w, Y) def the prediction and depending only on the parameter w and the data 2 E Duuh- Note that the error grows proportionally the fraction of byzantine clients and the distance between the aggregated honest and the byzantine predictions. Furthermore, since both predictions lie in Ac. their distance inherently bounded by v2. See Appendix E.1 for the full statement and proof of the following informal theorem.
[(h(w, (h((w,))YY (x)). Here, C is constant independent of
Theorem 1 (Informal), If is a stationary point of (Paisit), then is also an OC a² -approximate stationary point of hohonet), where C > is a constant independent of the client predictions. Further, in expectation, running Stochastic Gradient Descent (SGD) on (Paisti)) to achieve an E-approximate stationary point yields an (E (aa"))-aproximmat stationary point of (P (Pooeaa)
ATTACKING AND DEFENDING KD-BASED FL
In the previous section, compared standard variants of FedDistill and FedAVG in terms of their susceptibility to byzantine attacks. Building on our insights into the threat vectors, this section focuses developing strategies to attack and defend KD-based FL. We first introduce two effective attacks in Section 5.1, specifically tailored to exploit the threat vector of FedDistill. To counter these attacks, propose novel defence mechanism in Section 5.2, which significantly enhances byzantine-resilien compared to prior methods. Finally, Section 5.3 presents a general framework for obfuscating byzantine attacks on KD-based FL, making them much harder detect. In the following. we omit the time superscripts on the prediction to simplify the notation.
5.1
ATTACKING FEDDISTILL: LMA & CPA
HONEST MEAN
LMA
YH 0.77
YB
ship
0.44
dog
0.15
0.09
Y1
Y3
Y4
ship dog
0.82
0.73
0.92
0.61
cal
0.08
0.47
0.21
0.31
a
A
cat
0.14
0.06
0.04
0.08
Σ Y
CPA
Y
Y2
YP
Y 0.44
ship 01.7
ship
0.77
ship
dog 0.15
dog
0.15
dog
0.2
0.51
0.08
0.08
cal
0.05
ship
dog
cat
Figure 2: Attack procedures for three-class classification problem with four honest and three byzantine clients, i.e., a = 3/7. The left part the figure shows the computation the honest mean. LMA (upper right) assigns probability one to the least likely class based on the honest mean Y,, and CPA (lower right) assigns probability one to the class that is least similar to the most likely class of Y n. according to the similarity matrix S. Note that all computations are done per sample which was omitted from the notation for legibility.
In FedDistill, byzantine clients can only degrade the server model through their predictions. Since the threat vector is confined to the c-dimensional probability simplex A₂ rather than the parameter space, many attacks from the byzantine FedAVG literature, such as gradient sign flipping. cannot be applied to FedDistill. Previous work on byzantine FedDistill has been limited to basic label-flippig attacks.
While the server trains on the aggregated predictions from all clients Y (Ppab), its true objective is match the mean predictions ( (2pbb) made by the honest clients. Hence, one can interpret the aim of the byzantines as selecting their predictions Y5 (Puph) maximize the disparity between the aggregated predictions Y (Dpub) and the predictions of the honest clients Y, (2 pubb).

---

Published as a conference paper at ICLR 2025
Loss Maximization Attack (LMA). One way measure the difference between Y (Dppu) and Yu (Ppub) is via the loss function L that the server aims to minimize. We propose LMA, which chooses byzantine predictions that maximize this loss for each sample x E Ppub. Specifically, for each sample, LMA solves the optimization problem
max L ((1 a) aV (((())) Yg(x)EA,
(2)
For typical loss functions, LMA corresponds to choosing the least likely class according to the aggregated predictions of honest clients Y (2), see Lemma Appendix E.2. For each x E Ppuh. the byzantine clients compute the arg min of the mean honest prediction (æ)), corresponding to this class, i.e., the byzantine
the least likely class assigned to æ, and then assign probability
prediction becomes 1,. where j E arg min (z). .. An example shown in the top right of Figure 2. According to Y the least likely class cat, hence all byzantine clients will assign probability one to cat. As result, the mean of all clients Y predicts the class cat.
Class Prior Attack (CPA). Another way to measure how different two predictions are to take information about the relationships between classes into account. For example, the class dog is more similar to the class cat than to the class ship The second attack we propose, CPA, uses information about such inter-class relationships to select the most misleading predictions. To measure the similarity between the e classes, CPA requires access to a similarity matrix S E Rexe, We obtain this similarity matrix by computing the sample covariance matrix of the predictions ((Dpub) of pretrained model, i.c.,
S
Σ ()())) P)T, Y Dya
Σ Y(z). dE D
iDpub
iDuub
Then, for each sample x. CPA computes the class which is least similar to the class predicted by the honest clients per y (z). where similarity measured using the similarity matrix S. The byzantine clients now assign probability to that class, i.e., when i arg min Y (x), the byzantine clients predict 1, where j + arg min, Sij (see Figure 2). An example is shown in the bottom right of Figure 2. According to Yz, the most likely class is ship, and based on S, dog is the least similar class
ship. Hence all byzantine clients will assign probability one dog. As result, the mean of all clients Y predicts the class dog.
Although using pretrained model is realistic in practice, employ it to better understand CPA's effectiveness and potential capabilities. In real-world setting, attackers could instead use the server model from previous rounds or leverage their prior knowledge about class relationships construct the similarity matrix.
Figure illustrates the effects of LMA and CPA over multiple communication rounds, where we compare to the naive baseline Random Label Flip (RLF) attack, where all byzantine clients assign the same random label to each sample. Both CPA and LMA are signif- icantly more effective than RLF and heavily impact the final test accuracy. Notably, while the final test ac- curacy of both methods is similar, CPA has stronger effect in the early rounds.
Acmmaayy
75%
50%
LMA
CPA
25%
No attack
RLF
0
2
10
Communication rounds
5.2 DEFENDING FEDDISTILL: EXPGUARD
Figure 3: ResNet-18 on CIFAR-10: Test ac-
Unlike the baseline attack RLF, both LMA and CPA severely disturb the training process. In order to improve the resilience against such attacks, we inves- tigate replacing the mean aggregation of the clients' predictions with more robust aggregation methods.
curacy evolution over communication rounds when attacking FedDistill with 9 byzantine out of overall clients.
Let first consider existing defence approaches, namely Cronus (Chang et al., 2019), method based on filtering-basee based approaches from the high-dimensional robust statistics literature (Diakonikolas al., 2017) and the geometric median (GM), multivariate generalization of the median defined
7

---

Published as a conference paper at ICLR 2025
arg min mingss. Σx= ||Y; 1 -1122 Some commonly used robust aggregation methods further include the coordinate- wise trimmed mean or the coordinate-wir twite median; however, these cannot be used to aggregate predictions, the aggregated vectors do necessarily reside in the probability simplex, see Appendix E,3 for more details.
GM Yn) def
Filtering-basee based approaches compute the empirical covariance matrix of the data, obtain the leading eigenvector of that matrix and then project the data onto the subspace spanned by the eigenvector. The intuition behind these approaches that if an adversary wants to perturb the mean by modifying fraction of the data without creating obvious outliers, it has to set the datapoints relatively close to the benign data but all biased in similar direction, such that the perturbation caused by the individual datapoints do not counteract each other. This causes the variance along this direction to increase.
In the setting discussed in Diakonikolas et al. (2017), outliers along this eigenvector are then filtered based on specific rule relying on statistical assumptions. Since these assumptions do not hold in the FedDistill setting, Cronus relies on a simple heuristic: it filters out the 25% worst outliers, then recomputes the empirical covariance matrix and leading eigenvector based on the remaining 75% of the data and then filters out another 25% of the predictions, such that 50% of the original predictions remain.
Neither Cronus nor GM can be computed in closed-form and require numerical algorithms. Applying them to aggregated parameters in Byzantine FedAVG can lead to large computational overhead. However, for FedDistill, we only aggregate predictions, which are of much smaller dimensions, making their computational requirements negligible compared to training (ef. Table 7, Appendix D).
ExpGuard. Both GM and Cronus only use in- formation from the current datapoint x E Puuh- To further improve the resilience of FedDistill, we propose ExpGuard, which incorporates addi- tional information available to the server from the predictions on other samples within each commu- nication round as well as information about the predictions in past communication rounds.
Algorithm 2 ExpGuard
1: Input: Predictions Y
(D and weights of for all i E N, aggregation method AGG.
2: Compute outlier scores:
(D vi E [n]
ExpGuard. outlined in Algorithm 2, is a meta- algorithm that can be used to enhance any robust aggregation method. It works as follows: robust aggregation method AGG is used to compute an outlier score o; for each client based on its pre- diction on the whole dataset D Ppub. Then, these
3: Update weights:
P. + pf cxp(-ox), vi [n]
4: Compute weighted sum for all x E D Dpubi
(x+
1
N 141 (ttly+1(0 (x) Di=1
Ei
5: Output: 8.+1 (+1 Vi Vi [N]
outlier scores are used to update the weight p for each client based the exponential weights algorithm (I (Littlestone & Warmuth, 1994) such that clients with low outlier scores a; are assigned higher weight and the weight of clients with high outlier scores o, is reduced.
For any robust aggregation method, the outlier score can be computed the sum of the distances between the clients prediction and the robust estimate for each æ E Dpub. However, for filtering-bssed methods, we can skip the filtering, i.e., deciding which samples should be considered outliers, and simply use the distance of the clients prediction along the eigenvector as the outlier score. We refer to this approach as ExpGuard+Filter (EG+F). While ExpGuard substantially improves the byzantine resilience of all aggregation methods we tested, EG+F leads to the best results, see Table 8 in Appendix D. Therefore, we only show the results for BG+F Table 1.
Table I presents a comparison of robust aggregation methods against our proposed attacks and the Random Label Flip baseline. The results show that robust aggregation methods improve robustness over simple mean aggregation and EG+F consistently outperforms both GM and Cronus across all datasets, highlighting the benefits of utilizing all available information. Interestingly, GM generally outperforms Cronus while ExpGuard+GM is less byzantine- resilient than EG+F, see Table 8 in Appendix D. This indicates that the poor performance of Cronus might be due to the filtering heuristic employed.
8

---

Published as a conference paper at ICLR 2025
Table I: FedDistill: 20 clients of which nine are byzantine (ar 0.45). Final test accuracy averaged over multiple runs with standard deviation for different attacks and defences, BA refers to the baseline accuracy, i... the final accuracy of FedDistill all clients are honest.
CINIC-10 (ResNet-18), BA-80.240.1
CIFAR-10 (ResNet-18), BA=87.721.2
Mean
GM
Cronus
EG+F
Mean
GM
Cronus
EG+F
RLF
76.9=0.4
79.3+0.3
76.6+0.0
78.8+0.1
84.620.1
85,420.6
84.720.3
85.4+0.0
LMA
54.6+1.2 75.0+0.6
71.1+1.7
77.4+1.1
73.4
83.3#0.2
80.642.3
85.440.1
CPA
45.9+0.4
71.2+5.2
65.9+3.3
79.2+0.2
68.4+0.8
78.4=0.9
74.5=0.6
85.5+0.8
CIFAR-100 (WideResNet-28)) 6A=66.8+0.5
Clothing (ResNet-50), BA=69.0#0.3
Mean
GM
Cronus
EG+F
Mean
GM
Cronus
EG+F
RLF
65.2+0.7
65.2+0.3
44.3+1.7
63.920.6
69.4+1.2
68. 700.8
68.620.4
68.7+1.1
LMA
41.824.4
51.3+0.1
44.620.2
57.2+1.2
40.343.3
58.320.7
61,420.5
68.320.7
CPA
43.3+1.2
56.720.9
55.320.3
62.1
33.7+2.7
58.420.3
43.9+12.9
68.5+1.0
5.3 HIPS
The reason ExpGuard is effective at mitigating the attacks in experiments is that these attacks aim to maximally disrupt the learning process without considering detectability. However, in the presence of defences, tradeoff arises between attack's strength and its detectability. Even the most potent attack is ineffective if consistently detected and filtered out by a defence mechanism.
To address this tradeoff, we propose Hiding In Plain Sight (Hiding In Plain Sight (HIPS)), method that obfuscates attacks to make them harder to detect. The key idea behind HIPS is that small perturbations to proper predictions can sufficiently disrupt the learning process while avoiding detection, as they are less distinguishable from honest predictions. We note that similar concepts have been explored in the byzantine FedAVG literature (Baruch et al., 2019; Xie et al., 2020), but as these approaches where designed for the parameter space, they do not apply to FedDistill.
For each sample x. HIPS operates by reducing the attack space from Ag to the convex hull of the honest predictions A(x) def conv(Y:(<>)
H1
Yi
H) such that the predictions of the byzantine clients do not differ much from the honest predictions. We illustrate this concept in Figure 4 for three-class classification problem. In Figure 4a, the byzantine clients can choose any point in Ag. whereas in Figure 4b, due to the constraints imposed by HIPS, the attack space reduced to A. Given that HIPS introduces additional constraints to the attack space, becomes necessary to modify the attacks accordingly. In the following, we omit the dependence on x for legibility. We begin by extending CPA with HIPS.
Vo
Yr=
91
(a) Attack Ag
(b) Attack A
Figure 4: Attack spaces in Az: The blue dots represent the predictions by the honest clients and Y, is their mean. The attack space is highlighted, Yg the byzantine prediction, and Y denotes the mean of all clients for a = 0.5. The red line joining them repre- sents the mean Y corresponding to different a E [0, 0.5).
Extending CPA with HIPS. Recall that for each sample x, CPA computes the class least similar the class predicted by the honest clients, measuring similarity using the similarity matrix S. Let S; denote the i-th row of S, i.e., 5; is a vector containing the similarity of i to all other classes. If i is the class predicted according the mean honest prediction (x). then CPA can be rephrased as the solution to s. y S. Applying HIPS now requires restricting the constraints from A. A.
Unlike CPA, this attack can no longer be computed in closed form. However, since min y Ss a linear program, is sufficient to consider the extreme points of AA, which correspond to the predictions of each honest client. Thus, the solution can still be efficiently obtained by evaluating the

---

Published as a conference paper at ICLR 2025
function for each honest client's prediction and selecting the one with the smallest value. Intuitively, a prediction minimizes the objective by assigning low probability to classes similar to and high probability to those dissimilar io
Extending LMA with HIPS. Similarly, restricting the attack space of LMA to A prevents us from directly solving (2). However, we can show in Corollary 7, Appendix E.2 that (2) admits solution at the extreme points of A, which correspond to the honest predictions. Thus, we can compute the loss for each honest prediction and select the one that maximizes (2).
Table 2: FedDistill: 20 clients of which nine are byzantine (a = 0.45). We report the final test
aceuracy averaged over multiple runs with standard deviation for different attacks (corresponding to rows) and defences (corresponding to columns). BA refers to the baseline accuracy, i.e., the final aceuracy of FedDistill if all clients are honest. Recall that the goal HIPS is disrupt the learning process, hence lower accuracy is better.
CINIC-10 10 (ResNet-18)) 18), BA=80.2+0.1
CIFAR- 100 (WidcResNct-288) BA=66.8#0.5
Mean
GM
Cronus
FG+F
Mean
GM
Cronus
EG+F
LMA
54.6#1.2
75.0+0.6
71.1
77.4+1.1
41.8+4.4
$1.3+0.1
44.6a0.2
57.2+1.2
CPA
45.9a0.4
71.2+5.2
65.9+3.3
79.2+0.2
43.3+1.2
56.7+0.9
55.3+0.3
12.1
HIPS+LMA
75.3+0.1
68. 18.1
67.7+1.0
73.3+0.9
50.3+3.3
34.3+0.4
34,4+2.8
49.3+0.5
HIPS+CPA
74.2=1.1
65.8+0.5
66.420.1
72.9=0.7
47.2+4.2
32.6+4.5
28.
46.420.0
Table shows that HIPS improves the effectiveness of CPA and LMA against EG+F, GM and Cronus but reduces its effectiveness against the mean. Recall that the objective of HIPS is to reduce the strength of attacks in order to make harder to detect. Therefore, the fact that HIPS reduces the effectiveness of the attacks against the mean is to be expected as in the absence of defence mechanism, making an attack less detectable does not have any advantage.
In consequence, the mean is slightly more resilient to HIPS attacks than EG+F, while EG+F is more resilient towards non-HIPS attacks by large margin. Since we don't 't know in advance which attack strategy byzantine clients will use, the best defence is the one that experiences the smallest drop in accuracy against the most effective attack. This ensures that, regardless which attack employed, the defence performs reliably and minimizes the worst possible impact on model accuracy
6 DISCUSSION
We analyzed to what extent byzantine clients can perturb KD-based FL, exemplified by the prototypi- cal algorithm FedDistill, allowing us better understand and characterize its byzantine resilience.
By proposing two effective new attacks specifically for FedDistill, namely LMA and CPA, we highlighted vulnerabilities of FedDistill. We remark that while FedAVG attacks can not be directly applied to FedDistill, attacks for FedDistill can in turn always be applied to FedAVG by computing gradients based on the byzantine predictions. In fact, attacks based on computing gradients based on simple label-flipping attacks have been considered in the FedAVG literature (Allen-Zie -Zu et al., 2021). Applying our attacks to FedAVG might be an interesting direction for further research.
To increase the reliability of KD-based FL., we introduced the novel defence mechanism ExpGuard, which we demonstrated to be more resilient against previously proposed attacks as well as our new attacks. Since ExpGuard does not have significant impact the accuracy of FedDistill in the absence of byzantine clients, does not require hyperparameter tuning and can be efficiently computed, it is promising method to reduce the impact of arbitrary failures when using KD-based FL methods.
To obfuscate attacks by navigating the tradeoff between their strength and detectability, we proposed HIPS. Using this method, attacks are able to reduce the accuracy of FedDistill even when using ExpGuard. We hope that our contributions lay the groundwork for future research in the domain of byzantine FI..
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This research was partially supported by the DFG Cluster of Excellence MATH+ (EXC-2046/1, project id 390685689) funded by the Deutsche Forschungsgemeinschatt (DFG) as well as by the German Federal Ministry of Education and Research (fund number 011823025B). Most of the notations in this work have link to their definitions, using this code.
REFERENCES
Alessandro Achille, Matteo Rovere, and Stefano Soatto. Critical learning periods in deep networks. In International Conference on Learning Representations, 2018.
Dan Alistarh, Zeyuan Allen-Zhu, and Jerry Li. Byzantine stochastic gradient descent. CoRR, htss/8003089111 2018. URL http: org/ abs/1803 08917.
Zeyuan Allen-Zhu, Faeze Ebrahimianghazami,, Jenry Li, and Dan Alistarh. Byzantine-resilient non- convex stochastic gradient descent. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL https:// spenreview.nete meee/foorumiddPPeeee..
Youssef Allouah, Sadegh Farhadkhani, Rachid Guerraoui, Nirupam Gupta, Rafael Pinot, and John Stephan. Fixing by mixing: A recipe for optimal byzantine ML under heterogeneity. In Francisco J. R. Ruiz, Jennifer G. Dy, and Jan-Willem van de Meent (eds.), International Conference Artificial Intelligence and Statistics, 25-27 April 2023, Palau Congressos, Valencia, Spain, volume 206 of Proceedings of Machine Learning Research, pp. 1232-12222211 1300. PMLR, 2023. URL https: https:/pprcceeeee ppoceedinggg mlr press / v206 02206aalone alllouhh23aam html.
Jordan T. Ash and Ryan P. Adams. On warm-starting neural network training. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan- Tien Lin (eds.). Advances in Neural Information Processing Systems 33: Annual Con- ference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL httptttss//ppoccediinge mew.:eedddiinnnn.nn...e.e...e.ee cc/ cc/ htwppipps.cpppppe///22222222/..e 288cd2567 953 EO (006446000 12e4660339555
http://ppppp/////eeee Aat.scdaafAbbttracc.. html.
Gilad Baruch, Moran Baruch, and Yoav Goldberg. A Little Is Enough: Circumventing Defenses For Distributed Learning. In Advances in Neural Information Processing Systems, volume 32. Cur- ran Associates, Inc., 2019. URL https::///// http::/:////ppcceeeggeeee neurips wtcp/pape/2011 hash/ecic5911 41 04 6cd1 B Bffbbb:dfbeee331d----222 Absstrrat. html.
Iai Bistritz, Ariana J. Mann, and Nicholas Bambos. Distributed distillation for on-device learning. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tiee Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-13 6-12, 2020, virtual, 2020. URL httppp://procceeeeetet fef6 0EEE51166333122556660022222233-3
neurips cc/ppppr//222 /hash/ html.
Peva Blanchard, El Mahdi EI Mhamdi, Rachid Guerraoui, and Julien Stainer. Machine Learning with Adversarics: Byzantine Tolerant Gradient Descent. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https http://pppppr.aaeeeeee nips. cc/ 962999399 (e2ee99366666222e-eeeeee
paper/2017 http////aaahh//bbeeeeeeee
.ttml.
Hongyan Chang, Virat Shejwalkar, Reza Shokri, and Amir Houmansadr. Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer. arXiv:1912.11279 (cs, stat], December 2019. URL http: http///xxivvvo wtg. /alos///991/ 11279, arXiv: 1912.11279.
Mingqing Chen, Rajiv Mathews, Tom Ouyang, and Françoise Beaufays. Federated Learning Of Out-Of- Vocabulaay Words, March 2019. URL arXiv:1903.10635 [cs].
1903. 10635.
Yudong Chen, Lili Su, and Jiaming Xu. Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descent. 17Xiv:100..55444 [cs, stat], October 2017. URL http: //arxiv.org/absa............ 05491. arXiv: 1705.05491.

---

Published as a conference paper at ICLR 2025
Sijie Cheng, Jingwen Wu, Yanghua Xiao, and Yang Liu. Fedgems: Federated learning of larger server models via selective knowledge fusion. CoRR, abs/2110.11027, 2021. URL https:
//arxiv. attp//aaa22111 22zxdv..ro/aaa//111111000 2tt.bb//211....111......
Patryk Chrabaszez, Ilya Loshchilov, and Frank Hutter. downsampled variant of imagenet as an 2017. URL http: http/aaxxvvoooo/.//oo
alternative to the CIFAR datasets. CoRR,
abs/1707, 08819.
Adam Coates, Andrew Ng. and Honglak Lee. An analysis single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference artificial intelligence and statittcs, pp. 215-223. JMLR Workshop and Conference Proceedings, 2011.
Luke Nicholas Darlow, Elliot J. Crowley, Antreas Antoniou, and Amos J. Storkey. CINIC-10 is not imagenet or CIFAR-10. CoRR, abs/1810.03505, 2018. URL http: http://aaxvvv..eeeeeeeeeeee.e.le org/ /bbs/ 1810.03500
Ilias Diakonikolas, Gautam Kamath, Daniel M. Kane, Jemry Li, Ankur Moitra, and Alistair Stew- art. Being robust (in high dimensions) can be practical. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pp. 999-1008, PMLR, 2017. URL http::/ppoceeeeeeeeeeee waakonikclasl7..haee
mlr. www.ppees.//v.....mmmm
El-Mahdi El-Mhamdi, Sadegh Farhadkhani, Rachid Guerraoui, Arsany Guirguis, Lê-Nguyên Hoang, and Sébastien Rouault. Collaborative learning in the jungle (decentralized, byzan- tine, heterogencous, asynchronous and nonconvex learning). In Marc' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.). Ad- vances in Neural Information Processing Systems 34: Annual Conference on Neural In- formation Processing Systems 2021, NewrIPS 2021, December 6-14, 2021, virtual, pp. 25044-25057, 2021. URL https: http://poooeeeeneeeeeeeeeece / www.riiiiinnnnninsneeeeeemeu neurips wctpppppe//2211111...pppp / 91lddaacc www.eeaaaaaaaaaattataaaaaaaa
Abstract aat.rratttltl.......
Dongyang Fan, Celestine Sendler-Dünnee and Martin Jaggi. Collaborative learning via prediction consensus, In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference Neural Information Processing Systems 2023, NewrlPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023. http: /apprs/apppsepeeeaaaapeeeee nips oc paper_pppppppeeee www.s/ewweeeeeeeeeeeeeeeaaaeeeee 20002/ahh/ Abst
ratt -t html.
Xuan Gong, Abhishek Sharma, Srikrishna Karanam, Ziyan Wu, Terrence Chen, David Doermann, and Arun Innanje. Preserving Privacy in Federated Learning with Ensemble Cross-Domain Knowledge Distillation. Proceedings of the AAAI Conference on Artificial Intelligence, 36(11): 11891-11899, June 2022. ISSN 2374-3468.
doi: 10.1609aaiii, v36il 1.2 2446.. URL https: 24446. Number: 11.
//ojs. org/ wetg/index...o http://nee....p///a/aaaeeeeeeeee
Chaoyang He, Murali Annavaram, and Salman Avestimehr. Group Knowledge Trans- fer: Federated Learning of Large CNNs at the Edge. In Advances in Neu- ral Information Processing Systems, volume 33. Pp. 14068-14080. Curran Asso- ciates, Inc., 2020. URL
neurips oc httpppppppp/////aaaeae html.
Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pp. 770-778. IEEE Computer Society, 2016. doi: 10.1109/CVPR.2016.990 https::: /w// www...ogg/... 111//1...1111 1109/1100000 /C1/9/////R..0 201PR.20165 2016. 90.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the Knowledge in a Neural Network. arXiv: Nale::500..22 0 [cs, Stat], March 2015. URL arXiv: 1503.02531.
http://xwww..o//ab/b/laao//////e 0b01///000/11115011.1.1. 02531.
Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. L.
12

---

Published as a conference paper at ICLR 2025
D'Oliveira, Hubert Eichner, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett, Adria Gascón, Badih Ghazi, Phillip B. Gibbons, Marco Gruteser, Zaid Harchaoui, Chaoyang He. Lie He, Zhouyuan Huo, Ben Hutchinson, Justin Hsu, Martin Jaggi, Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Konečný, Aleksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancrède Lepoint, Yang Liu, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer Özgür, Rasmus Pagh, Mariana Raykova, Hang Qi, Daniel Ramage, Ramesh Raskar, Dawn Song, Weikang Song, Scbastian U. Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tramèr, Praneeth Vepakomma, Jianyu Wang. Li Xiong, Zheng Xu, Qiang Yang, Felix X. Yu, Han Yu, and Sen Zhao. Advances and Open Problems in Federated Learning. December 2019. URL https: //tp///axvvvv//////// org/ /bbs/192 2 04977v3
Sai Praneeth Karimireddy, Lie He, and Martin Jaggi. Learning from History for Byzantine Ro- bust Optimization. In Proceedings of the 38th International Conference on Machine Learning. pp. 5311-5319, PMLR, July 2021. URL https::/pproceeeee karimireddy22 htt.1.l...mm..m..mmmm.mmm.m....m ISSN: 2640-3498.
mll. mllrprress/v19/
Sai Praneeth Karimireddy, Lie He, and Martin Jaggi. Byzantine-robust learning on heterogeneous datasets via bucketing. In International Conference on Learning Representations, 2022. URL
https:://ppezeett
net/ forum?ide jXXKDEi5vs
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Daliang Li and Junpu Wang. FedMD: Heterogenous Federated Learning via Model Distillation. [cs, stat), October 2019. URL http: http:://aaii..orrrree/// wwwpw/wwwwwwwooo/otoo//ooobooarp /110. 03581.
arXiv: 1910.03581.
Tao Lin, Lingjing Kong, Sebastian U. Stich, and Martin Jaggi. Ensemble distillation for robust model fusion in federated learning. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Flone Aaria-Florina Balcan, and Hsuan-Tiees Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference Neural Information Processing Systems 2020, NeurlPS 2020, December 6-, 2020, virtual, 2020. URL htwps:ttt////occtmm /ptpc//e///////eepeeeeeeee iggs wwtparips.cc/ppeeeee cc/paper/ / 8dd51b97
29 998ee994044444 f3cccc877----00 Abst htmll.
Nick Littlestone and Manfred K Warmuth. The weighted majority algorithm. Information and computation, 108(2):212-261, 1994.
H. Brendan McMMaan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. cattuuniiiaati--ffiiiieee Learning of Deep Networks from Decentralized Data. 472.v:1602.05629 [cs], February 2017. URL http:/ www.i.ovoro........ org/abs/ 1602. 05629.6622 9. arXiv: 1602.05629.
Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. Exploiting Unintended Feature Leakage Collaborative Learning, November 2018. URL htttt://aaxiii www.si/iiorrroooooooooooooo /bs/ 1805 04049. 9. Xrv: 1805.8050440 .0044 [cs].
El Mahdi EI Mhamdi, Rachid Guerraoui, and Sébastien Rouault. The Hidden Vulnerability of Distributed Learning in Byzantium. In Proceedings of the 35th International Conference on Machine Learning, pp. 3521-3530 PMLR, July 2018. URL press/ v mhamdi 18a. html. ISSN: 2640-3498.
ngs mlr.
Milad Nasr, Reza Shokri, and Amir Houmansadr. Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning. In 2019 IEFE Symposium on Security and Privacy (SP), pp. 739-753, May 2019. doi: 109//P. 2019.00065. URL http: / arxiv. .org/ abs 1812.1122 00910. 1rXiv:1812.0099 [cs, stat].
Nicolas Papernot, Martín Abadi, Úlfar Erlingsson, Ian J. Goodfellow, and Kunal Talwar. Semi- supervised knowledge transfer for deep learning from private training data. In 5th International Conference Learning Representations, ICLR 2017, Toulon. France, April 24-26, 2017, Confer- ence Track Proceedings. OpenReview.net, 2017, URL https: http://////peeeee.../// net forum? id=HkwoSDPgg.
Swaroop Ramaswamy, Rajiv Mathews, Kanishka Rao, and Françoise Beaufays. Federated Learning for Emoji Prediction in a Mobile Keyboard, June 2019. URL http: /ttp:///axiii../ org abs/ 1.96.om 04329 1rXiv:1906.044 [cs].
13

---

Published as a conference paper at ICLR 2025
Nicola Ricke, Jonny Hancox, Wenqi Li, Fausto Milletari, Holger R. Roth, Shadi Albarqouni, Spyridon Bakas, Mathieu N. Galtier, Bennett A. Landman, Klaus Maier- Hein, Sébastien Ourselin, Micah Sheller, Ronald M. Summers, Andrew Trask. Daguang Xu, Maximilian Baust, and M. Jorge Cardoso. The future of digital health with federated learning. upj Digital Medicine, 3(1):1- 7, September 2020. ISSN 2398-6352. doi: (0.1038//41746-00--003333
URL https:// Number: Publisher: Nature
www. nae com www./artiiiiiic http///arrceesss11//// 020 0032--0023222 -. Publishing Group.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Li Fei- Fei. Imagenet large scale visual recognition challenge. CoRR, abs/ 2014, URL http:/ /ttp://aaxvv..//. orgoorggaa abs /1tss./19...000 05755.
Felix Sattler, Arturo Marban, Roman Rischke, and Wojeiech Samek. Communication Federated Distillation, December 2020. URL http::/aaxiii wwwpa/wwwwww...//////a///a/..//.
arXiv:2012.00632 [cs, stat].
Felix Sattler, Tim Korjakow, Roman Rischke, and Wojciech Samek. FEDAUX: Leveraging Unlabeled Auxiliary Data in Federated Learning. IEEE Transactions Neural Networks and Learning Systems, pp. 1-13, 2021. ISSN 2162-237X, 2162-2388, doi: 10. 111110/NNLLS
URL htwph::tp//.mmm htte://eeeeeeeeeeeeeeee leee wttp://ooo/occcc.eeeeee22.22
Virat Shejwalkar and Amir Houmansadr. Manipulating the Byzantine: Optimizing Model Poisoning Attacks and Defenses for Federated Learning. In Proceedings 202/ Network and Distributed System Security Symposium, Virtual, 2021. Internet Society. ISBN 978 178-1-891562-6622 doi:
10.147722hndss2201.244888 URL https: : /www. ndss sosssssypppssss um. org wtpp contentt uploads/ndss2000 6C- 24 2499 paper.pdf.
Micah J. Sheller, G. Anthony Reina, Brandon Edwards, Jason Martin, and Spyridon Bakas. Multi- institutional Deep Learning Modeling Without Sharing Patient Data: A Feasibility Study on Brain Tumor Segmentation. In Alessandro Crimi, Spyridon Bakas, Hugo Kuijf, Farahani Keyvan, Mauricio Reyes, and Theo van Walsum (eds.), Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, Lecture Notes in Computer Science, pp. 92-104, Cham, 2019. Springer International Publishing. ISBN 778-3-030-11122-8. doi: 10.1007/978-3-0300 11720-8
Stefán Páll Sturluson, Samuel Trew, Luis Muñoz-Gonzááez,, Matei Grama, Jonathan Passerat- Palmbach, Daniel Rueckert, and Amir Alansary, FedRAD: Federated Robust Adaptive Distillation. arXiv:2112.0144 00405 [cs], December 2021. URL http: wrg/ http//abb//1111 211.2 01405.
arXiv: 2112.01405.
Tong Xiao, Tian Xia, Yi Yang, Chang Huang, and Xiaogang Wang. Learning from massive noisy labeled data for image classification. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2691-2699, 2015.
Cong Xie, Oluwasanmi Koyejo, and Indranil Gupta. Fall of Empires: Breaking Byzantine-tolee tolerant SGD by Inner Product Manipulation. In Proceedings of The 35th Uncertainty in Artificial Intelli- gence Conference, pp. 261-270. PMLR, August 2020. URL https: /ttp::///////c/eeeieeeemo nigs. ml html. ISSN: 2640-3498.
press/v115/xiee
Timothy Yang, Galen Andrew, Hubert Eichner, Haicheng Sun, Wei Li, Nicholas Kong, Daniel Ram- age. and Françoise Beaufays. Applied Federated Learning: Improving Google Keyboard Query Sug- gestions, December 2018. URL URL http: http///axvvv/vv////v//eee/eaaav/ org/ abs/ 181118111 881 02903. arXiv:1812.02900
[cs, stat].
Dong Yin, Yudong Chen, Kannan Ramchandran, and Peter Bartlett. Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates. 1803.0140 [cs, stat], February 2021. URL http:// //tt:////ivvveeeaaee/eeeeeeaiavr org abs http:/bbs/8800 1803, 01498. arXiv: 000.001498.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint 2016.
14

---

Published as a conference paper at ICLR 2025
Banghua Zhu, Lun Wang, Qi Pang, Shuai Wang, Jiantao Jiao, Dawn Song, and Michael Jordan. Byzantine- Frrbantie-RRobust Federated Learning with Optimal Statistical Rates. In Proceedings of The 26th International Conference Artificial Intelligence and Statistics, pp. 3151-3178. PMLR, April 2023. URL https: / /ttp:///r/ccccc///o ngs ml r. paress/v2066 2640-3498.
zhu23b. w2wps//2232///2222b/ html. ISSN:
Zhuangdi Zhu, Junyuan Hong, and Jiayu Zhou. Data-free knowledge distillation for heterogeneous
federated learning, In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International
Conference on Machine Learning, ICMI. 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 12878-12889. PMLR, 2021. URL http: //proceedings.. mlr. www.sppeessii.. zhu21b.. html.
15

---

Published as a conference paper at ICLR 2025
A BROADER IMPACTS
FL has the potential to harness previously inaccessible data, overcoming privacy concerns and which could have positive impact in domains like healthcare. This underscores the necessity of enhancing comprehension of these methods. However, the adoption of distributed training setups introduces new challenges and risks.
Studying FL in the byzantine setting is often motivated an avenue study the resilience of these methods to network errors or bugs, the increasing deployment of FL in critical real-world contexts amplifies the urgency of addressing threats posed by adversarial actors. The notable case of the "Google Maps Hacks", an art performance by Simon Weckert, serves as a compelling illustration. Weckert simulated a virtual traffic jam in Google Maps by strolling around Berlin with 99 smartphones, manipulating the digital representation of streets. This demonstration had tangible consequences in the physical world, rerouting cars to avoid perceived traffic congestion. Extrapolating such exploits to critical infrastructure, like a smart grid reliant on usage information for energy production, underscores the potential ramifications of malicious interventions.
B EXPERIMENTAL DETAILS
Public datasets For CIFAR-10, 10, we use the unlabeled split of the STL-10 dataset (Coates et al., 2011), consisting of 100k samples. For CINIC-10 we use the validation split of CINIC-10 consisting of 90k samples. For Clothing M, we use the noisily labeled split of Clothing MM, M,, consisting of IM
samples.
For CIFAR-I00, we use the train split both for the public dataset and as the private datasets. While this does not reflect a realistic use-case, we did not find an unlabeled dataset that was sufficiently in-domain and large enough to achieve good performance. We explored two alternative public datasets for CIFAR- 100: Imagenet32 (Chrabaszez al., 2017), downscaled version of the Imagenet dataset (Russakovsky et al.. 2014) and an extended version of the CIFAR- 100 dataset consisting of 30k images, found on Kaggle Using both of these datasets, performance, which is probably due to domain drift. Since the goal of this paper is not to achieve high accuracy using KD-based FL methods but rather to evaluate byzantine attacks and defences we chose to use the same data for the local and public data. Alternatively, one could try to generate public dataset using either pretrained generative model by training a generative model in via FL. see e.g. (Zhu et al., 2021).
were not able to achieve good
FedDistill training. To achieve good performance with FedDistill, we reinitialize the server model and train it to convergence in every communication round. Without re-initializing, we were not able to achieve high server test accuracy even with longer training. This aligns with the observation made by Achille et al. (2018 8) and Ash & Adams (2020) that using noisy data in the beginning of training permanently damages the model, even the data quality improves later on. In the early communication rounds, the clients predictions on the public dataset are inaccurate. Hence, even if the clients models were to improve over time, the early training on inaccurate labels prevents the server from learning accurate classifier. By re-initializing the server, this problem can be circumvented and the server can learn from the progressively improving clients predictions.
Hyperparameters We run the experiments with two different random seeds and provide the standard deviation. each communication round, the server collects the soft predictions of the clients, i.e., the probability distribution over the classes, and aggregates them using either the mean or other aggregation methods. The server then re-initializes the model and trains on the public dataset using the aggregated predictions decaying learning rate. Then, the clients train the model on their private data. The clients also use SGD with momentum and linearly decaying learning rate. However, their linear decay stretches across the total epochs, meaning that they do not restart the schedule in each communication round the server. For both the server and clients, we calculate the accuracy on the validation dataset after each epoch and implement early stopping to prevent overfitting.
target. In each round, it uses SGD+momentum with a linearly
1ttpps: / /ww.///wwwwwkk kaggle e. com htmm//aaaasttt/dunyyllll farl 100 20- 100 CC- - imagma- cetess-exteniionn
data
16

---

Published as a conference paper at ICLR 2025
CIFAR-10
CIFAR-100
CINIC-10
Clothing 1M IM
Architecture
ResNet-Ie
WideResNet-25
ResNet-18
ResNet-50
Batch size
128
256
128
256
Server epochs per round
80
100
80
10
Total local epochs
400
250
200
300
C FURTHER RESULTS.
Table 3 shows the effect of different attacks and defences in the same setting in Table and Table but for different datasets.
Table 3: FedDistill: 20 clients of which nine are byzantine (a 0.45). Final test accuracy averaged over multiple runs with standard deviation for different attacks and defences. BA refers to the baseline accuracy, i.e., the final accuracy of FedDistill if all clients are honest.
CIFAR- 10 (ResNct-18) 8A=87,7+12
Clothing (ResNct-50), BA=69.0Au 0a0.3
Mean
GM
Cronus
EG+F
Mean
GM
Cronus
EG+F
RLF
69.4+1.2
68. 74.88
68.6#014
68.7+1.1
84.6+0.1
85.4
84.7=0.3
85.4+0.0
LMA
40.3+3.3
58.3+0.7
61.420.5
68.3+0.7
73.448.6
83.3+0.2
80.6+2.3
85.420.1
CPA
33.7+2.7
58.4+0.3
43.9+12.9
68.5+1.0
68.4+0.8
78.4+0.9
74.5=006
85.5+0.8
HIPS+LMA
33.7+2.7
58.420.3
43.9+12.9
68.5+1.0
84.8+0.1
78.0+1.6
78.5
83.8+0.2
HIPS+CPA
63.4+
55.2+1.1
54.8+2.1
57.740.5
85.0+0.1
79.440.8
77.3+0.1
83.2+0.9
Performance of FedDistill and FedAVG in the benign setting. We compare the performance of FedDistill and FedAVG in the benign setting. The experimental setup for FedDistill follows the same configuration as in Table 1, utilizing 20 clients, 10 communication rounds, but without byzantine clients. For FedAVG, we follow this setup as closely as possible, however for CIFAR-100 and Clothing we had use 100 communication rounds to achieve competitive performance.
The reason for the discrepancy between FedDistill and FedAVG on Clothing IM is that FedAVG is does utilize the unlabeled public dataset. This highlights the fact that FedAVG and FedDistill make different assumptions about the data.
Table Comparison of final test accuracy between FedAVG and FedDistill across different datasets. Results are averaged over multiple runs with standard deviation included.
CIFAR-10
CINIC-10
CIFAR-100
Clothing M
FedAVG FedDistill
88.620.1
75.1
67.2+0.3
61.2+1.1
86.8+1.0
80.2+0.1
66.8+0.5
69.0+0.3
Performance of defenses in benign settings. We evaluate the performance of different defense mechanisms in the benign setting. The results, detailed in Table 5, demonstrate that the accuracy of the defenses remains largely consistent with the mean baseline, with only minor reductions in accuracy. These experiments were conducted without altering any hyperparameters, indicating that the slight accuracy variations observed could potentially be minimized through hyperparameter tuning.
Comparison to state-of-the-art FedAVG defence mechanisms. We implemented the state-of- the-art approach of using D-SGD with momentum and nearest neighbour mixing (NNM) with coordinate- wise median (CWMED) aggregation for CINIC-10 against the ALIE attack, as outlined in A Allouah et al. (2023). In the absence of attacks, D-SGD achieves accuracy of 80.8 with mean
17

---

Published as a conference paper at ICLR 2025
Table 5: Performance of FedDistill with different defense strategies in the benign setting without byzantine clients. Results are averaged over multiple runs with standard deviation included.
CIFAR-10
CINIC-10
CIFAR-100
Clothing 1M
FedDistill (E+F)
86.2+1.0
80.5+0.2
64.9=1.9
68.0=0.2
FedDistill (GM)
87. $7.0+0.1
81.1+0.5
66.4=0.4
69.4+0.0
FedDistill (Cronus)
86.420.3
80.420.2
62.6+0.8
64.920.2
FedDistill (mean)
87.7+1.2
80.2+0.1
66.8=0.5
69.020.3
aggregation and 77.7 with NNM+CWMED. Table 6 presents the accuracy for varying the number of byzantine clients and attack magnitudes Note that a direct byzantine comparison between FedDistill and FedAVG is hard to interpret, as both methods make different assumptions about the data and have different advantages and disadvantages.
Byzantine clients can still significantly disrupt the learning process, reducing accuracy to below 20% with 9 byzantine clients. This represents a more than fourfold decrease in accuracy compared to the scenario without byzantine clients, whereas FedDistill only experiences 1. 1-I-f reduction. We have not optimized the attack amplitude for ALIE, as suggested by Allouah et al. (2023), which means that these results are conservative and optimal attack amplitudes might lead to even lower accuracies.
Table 6: Accuracy of D-SGD under ALIE attack with different amounts of byzantine clients and attack magnitudes 17.
# byz. clients 17
0.05
0.1
0.5
2
78.7=0.4
78.320.7
69.0s2.4
75.8=0.0
79.9+0.0
5
77.2=0.8
70.024.0
27.6#1.5
24.1
71.5+0.0
9
55.8+26.0
55.3+5.8
25.8+2.9
19.8=0.2
62.2+13.7
D ABLATIONS
Computational cost of aggregation methods In Table 7, we show the runtime of GM and EG+F for multiple datasets. Even for large datasets with many classes, the computational overhead is negligible compared to the cost of training the models. Note that the runtime is for the whole dataset. This means that for the experimental setting we consider throughout the paper with 10 communication rounds, the total time spent aggregating the predictions is about 32 seconds, even for CIFAR-100.
Table Runtime per communication round of different defence methods for datasets of different size and with different number of classes.
STL-10
CIFAR- 100
Clothing LM
Samples Classes
100k
60k
IM
10
100
Runtime: EG+F
0.08s
3.2s
0.8s
Runtime: GM
0.15s
0.62s
0.77s
ExpGuard with other defences. Table 8 compares the performance of different robust aggregation methods combined with ExpGuard. The resilience of all aggregation methods is significantly improved by ExpGuard, cf. Tables and 2. Further, EG+F clearly outperforms ExpGuard combined with the other aggregation methods while ExpGuard+Cromus is the least robust among all methods, except for ExpGuard+MMann. This shows that the modifications of EG+F are crucial to achieve good performance.
18

---

Published as a conference paper at ICLR 2025
Table 8: FedDistill: 20 clients of which nine byzantine (ar 0.45). Final test accuracy averaged over multiple runs with standard deviation for different attacks and defences used in combination with ExpGuard. BA refers the baseline accuracy, ie.. the final accuracy of FedDistill all clients
are honest.
CINIC- (ResNet-18), 8A=80.2+0.1
CIFAR- (ResNet-18) B7=87.7=1.2
Mean
GM
Filter
Cronus
Mean
GM
Filter
Cronus
RLF
77.1 #0.4
79.3 40,9
78.8+0.1
76.3 #0.2
77.1 20.4
86.0 20.3
85,420.0
85.9 20.8
LMA
53.0 #4.8
73.8 +2.0
77.4
71.
53.0 #4.8
84.1 =0.9
85,4x0.1
114.9
CPA
45.8 #0.7
71.9
79.2+0.2
66.1 #0.5
45.8 +0.7
10.7 +18.2
85.5+0.8
2.3
HIPS+LMA
75.1 #0.8
67.9 #0.7
73.3+0.9
67.5 +0,9
75.1 #0.8
76.5 #0.9
83,8+0.2
76.5 #0.8
HIPS+CPA
72.7 20.4
66.2 40.2
72.9+0.7
65.7 +0.0
72.7 #0.4
77.5 #0.8
83.2+0.9
76.9 20.4
Number of clients and communication rounds In Figure 5, ablate the impact of the number of clients and communication rounds on FedDistill in the honest setting, while keeping the other parameters such the total number of local epochs and server training epochs per round fixed.
On the left side of Figure 5, we see that increasing the number of clients leads to decreasing accuracy. of fixed size and the amount of private data per
This is due the fact that the datasets client decreases as the number of clients increases. This is especially problematic before the first communication round, as the models have to train model from scratch that encodes some useful information only based on their private dataset.
On the right side of Figure 5, we see that at first, increasing the number of communication rounds leads to an improvement, but as the number of communication rounds decreases, we first experience diminishing returns at some point even a decrease in accuracy. This in part due to the fact that kept the total number of local epochs fixed, but we observed similar result when increasing the total number of local epochs.
90%
Test Accuracy
Test Accuracy
85%
80%
80%
75%
78%
70%
10
20
30
40
5
10
15
20
Number of clients
Number of communication rounds
Figure 5: (Left) ResNet-18 on CI with 10 communication rounds: Final test accuracy of FedDistill with different number of clients. (Right) ResNet-18 on CINIC-10 with 20 clients: Final test accuracy of FedDistill with different number of communication rounds.
Measuring the impact of byzantine elients on EG+F performance. We conducted ablation study to assess the impact of varying the number of byzantine clients on the performance of EG+F on the CINIC-10 10 dataset, see Table 9. We selected LMA with and without HIPS the attack strategy, as it was the most effective attack on CINIC-10 with 9 byzantine clients. For reference, the baseline performance of FedDistill using mean aggregation without byzantine clients 80.2.
Evaluating byzantine robustness on non-i.i.d. data. We evaluate the performance FedDistill in the non-i.i.d. setting. To that end, we generate non-i.i.d. data using the Dirichlet distribution with parameter B. which is a common approach in the FL literature (Lin et al., 2020). We follow the same setting in Table 1, except that the client datasets do not follow uniform class distribution, but rather skewed one depending on the parameter 3, where smaller & means more heterogeneity and vice-versa. In Table 10 provide experimental results for the CINIC-10 and CIFAR-10 datasets.
19

---

Published as a conference paper at ICLR 2025
Table 9: Test accuracy of EG+F CINIC-10 with varying numbers of byzantine clients. Results averaged over multiple runs with standard deviation included.
# Byzantine Clients
3
5
LIPS+LMA L.AA
80.5+1.2
79.7+1.0
79.8+0.4
78.9+0.1
72.9+0.7
80.5+1.2
79.7=0.1
79.9=0.0
79.9=0.2
77.4+1.1
We conducted comprehensive evaluation of all attack and defense strategies, varying the 8 in {10, 5, 5, The table differentiates the defense mechanisms, presenting the final test accuracies for various attacks (listed rows for each defense) in both the ii.d. and non-ii.d. scenarios with different 3 values. Notably, for attacks without HIPS, EG+F demonstrates superior resilience compared to all other defenses, with the disparity being even more pronounced in the non-i.i.d. setting, particularly at lower 3 values. Specifically, for 3 = 1, both GM and Cronus underperform relative to the mean, whereas EG+F achieves
than double the accuracy of the mean. Unlike other defense methods, EG+F retains its robustness amidst higher heterogeneity, making it an ideal choice for non-i.i.d. Syzantine-resiliee aggregation.
As in the i..dd scenario, the mean shows slightly better robustness against HIPS attacks than EG+F. However, the mean lacks resilience against non-HIPS attacks. Consequently, if the server is uncertain about the type of attacks it might face, EG+F remains the preferable choice, offering solid worst-case performance with only a minor reduction in performance against HIPS attacks. Generally, HIPS proves to be significantly more potent in the non-i.i.d. setting than in the i.i.d. one. This is expected, as increased beterogeneity client data results in more heterogeneous predictions increasing the size of the attack space, i.., the convex hull of honest client predictions. This allows the byzantine clients to more easily evade detection while disrupting the server.
20

---

Published as a conference paper at ICLR 2025
Table 10: Test accuracy of different defense mechanisms CINIC-10 and CIFAR-10 datasets with varying non-i..d. levels (5). Results are averaged over multiple seeds, but we omitted standard deviations for the sake of legibility. The table is divided into sections for each defense mechanism.
CINIC-10
CIFAR-10
i.i.d.
B = 10
B == 5
i..d.
B = 10
B = 1 B = 5
Mean
RLF
76.9
75.5
73.9
62.0
84.6
83.9
80.6
71.0
CPA
45,9
37.2
28.1
18.5
68.4
57.9
38.2
15,4
LMA
54.6
52.0
47.4
27.2
73.4
69.5
65.6
35.5
HIPS+CPA
74.2
60.7
53.4
32.6
85.0
76.2
59.5
32.3
HIPS+LMA
75.3
61.2
52.8
28.9
84.9
75.6
54.0
32.0
EG+F
RLF
78.8
75.8
73.0
63.8
85.4
83.8
80.3
69.1
CPA
73.3
77.9
73.6
58.4
85.5
85.6
83.5
52.9
LMA
77.4
75.8
73.8
62.0
85.4
85.0
83.4
74.0
HIPS+CPA
72.9
57.4
41.4
24.3
57.7
68.3
42.8
25.3
HIPS+LMA
73.4
59.1
44.0
24.4
68.5
68.2
44.6
23.0
GM
RLF
79.3
76.3
75.2
62.5
85.4
73.1
82.4
85.2
CPA
71.2
44.5
37.6
9.5
78.4
67.4
43.8
10.4
LMA
75.0
55.3
47.2
10.4
83.3
75.0
58.7
9.6
HIPS+CPA
65.8
38.6
31.4
18.4
55.2
59.7
30.2
13.5
HIPS+LMA
68.7
36.9
30.1
15.1
58.4
45.6
28.2
10.0
Cronus
RLF
76.6
70.8
62.6
39.1
84.7
79.4
75.3
42.7
CPA
65.9
40.1
25.7
9.1
74.5
60.0
34.9
10.1
LMA
71.1
51.4
35.1
10.0
80.6
71.8
48.0
7.8
HIPS+CPA
66.4
38.3
29.5
17.4
54.8
48.1
28.9
16.8
HIPS+LMA
67.7
37.3
28.6
16.1
43.9
42.7
21.0
10.4
E THEORETICAL RESULTS
In this section, we present the formal statements and proof discussed in the main part of the paper.
E. BOUNDING THE BYZANTINE ERROR
First we show Lemma that the two most common loss functions, namely the MSE CEL, the gradient of the Loss with respect to the parameters of the classifier are linear with respect to the difference between the models prediction and the target label and it's slope is defined only by the model parameters and the data In Theorem 3, show based on Lemma 2 that if parameter i is stationary point of (Passin), then it is also in the neighbourhood of an approximate stationary point of (Phones). The size of this neighbourhood is determined by the fraction of byzantines and a constant C' depending on the data E Ppbb and w. In particular as goes to zero the size of the neighbourhood goes zero well.
Lemma 2. Consider a function L(h(x, w)) y) where x W + D. such that w + h(a, w) is differentiable and y E Ac. Assume that one of the two statements hold
1. L(a,b) M))(a,b)
2. C(a, b) CEL(b, a) and h(æ, w) = o where D:X' W + R and is the softmax function.
21

---

Published as a conference paper at ICLR 2025
Then there is linear function 9 A. + W such that g y) (wL(h(2, w)) which is Ly Lipschiii,,, where Lp depends on x and w.
Proof of Lemma 2.
Case We have MSE(p. 9)
2 lp gll Then using the shorthand h, for the i-th element of hx,, w) and by applying the chain rule, we have
i MSF(h(2, w). dw,
@MSE(h(r, w),v) Oh,
Th, Owly
>(v=
(ys
h4)
Oh;
Owy
Hence we can write the gradient as follows
V (SE(h(z, w). = Jn(w)(( h(2; w)),
where Jn. is the Jacobian of h. Let g(h(x, w) y) der Jn J (y- A(ar, w)). Then is linear and Lp-Lipschitz, where Lp S (w)]l. Note that (w) depends only on x E Pub and w.
Case 2. We have that CEL(P, 9)
def
Σ ps log(a:) and v CEL(P, exp(x))
PNN//N)" Furthermore, (p) where 6ig- 1 if j and 6ij 0 otherwise.
and da.(p)
(p(p) - oz(
1i exp(p))
ao (6) - or()), with dig
11, i=j
0, otherwise.
Hence, using the chain rule,
a CELLY o)))
(v, 00,(
(us (w5/o,(0)),,))111 (0)))
04,
Oi (0) 84,
j=1
c
Σ (5, (.) -0,
Σ 5/j
+ = -
Then,
CEL (()))
i CELL(V (())
00;;
0.;;
Σ
(y) (T()))
Ory
Ob.
Owy
Ow,
am
follows that
V. CEL(Y, = ( h(x, w)),
where Jn is the Jacobian of h. Now let g(h(a, w) - y) del Jo (w) - 1(x, w)). Then g is linear and Lipsschct where Lp < [[]]())]. Note that (w) depends only Dpub and
Theorem 3. Consider the optimization problem defined in (Phones), i.e.,
min
F(w)
Σ
w),
WEW
Doubl
DED
and the optimization problem defined in (Passiu). i.e..
min
F(w)
Σ
L(h(,,w, F(x))
WEW
Dpusl
DED
Assume that one the following holds:
22

---

Published as a conference paper at ICLR 2025
1. C(p.g) CEL(q, p) and h(z,w) w) @ with finction & XX*. W - E and is the finction..
2. C(p.g) q) MSE(p, 9)
Then, if to is stationary point of F, i.e.
V
O, it also holds that
VF S (Dubll
E
Ca CallVR(0 ( S V2aC,
DED
where C > O is constant which depends only Pub and and is independent of the client predictions. If further, [(h(x,-), w) is L-smooth, then running SGD F initialized wo for
T= ==0 ( +o² iterations, where is the gradient variance, then it holds that | FFF)))l
O( + C²
Proof of Theorem 3. We prove the theorem in two parts, First we bound the norm the gradient error and then show approximate convergence of SGD given the gradient error bound. Note that (ww) and VF(w) only differ in the second argument of C. By Lemma 2, we have that
1
(F(w)
vi(h(z, (x, (x)), V((w) = D
VL(h(z, w), P
D
pub
publ
are C-Lipschitz function with respect to the second argument, where C' depends on w and æ E Ppub only. Denote by [[D(w)|| dnd
VF(w) ((w) the gradient error, then
aC Dpubl
[B(w)] S
Σ
pPpub [
Σ D
[x(xx) Ys(())| S V2aC,
(3)
where the first inequality holds by the Lipschitzness, the second inequality holds since Y (1 a)Y² + aYg B and the last inequality holds since the maximum & distance between two points in A. is 22. Now let w be stationary point of F, then have
VF = VF vF(w) |B
which together with Equation (3) shows the first part of the theorem.
For the second part of the proof, we define
f((w) = L(h(x,, w), (x))),, f(w)) = L(h(x;,)) w). (xx))
and it follows that
1 Pool
paall
F(w) [.]]
fi(w) and F(w) "(w))EE[[((w)) [w)]
M(w).
D
Ppub
D
we - (.)... where is sampled uniformly at The update rule of SGD is defined w+11 random. We have that
vf((w)) = F + vi(w)) V.((w) + Vf((w; - F
By the L smoothness of C(h(a,) :), v) and the SGD update rule, we have
[((w)+1) fi(w.) < (V [i(we) wi+1 - -we) + || - -
-><V/(( vi(w))) + 2
23

---

Published as a conference paper at ICLR 2025
Now, taking the expectation with respect to both sides, obtain
F(w++1) - F(w.) S -7 (V VF) L E
(wx)) 27 19]
L
[[V - (w(:)) 22
||- / ++//P(wa>))))
- (VF VF(w.. +
where the second inequality holds by noting that E[
Var(X) + B[X]² We bound,
[|]
VF(w)) VF(w)) + V VF(w)) ||]
Vi((we) - VF(w.))
V((ee)) Vf((w, + Vf((w;)
I 3E f((())) V
+ 3E [l VF(w))ll + |B(()l
I 6C 2 + 30 + 31
where the last line follows by noting o² daf
[..i(ur))) [F(W))]] and since
- vi(i-)) 220
I
puh
Using these terms, we further bound
L72
(6C + + B()
vi(w>)
F(we+1) - F(o.)) < (VVV
= - [IV][( + c² 3l-20²
2
where the last inequality follows by choosing 7 S 2 and since by Equation (3), we have [[((w))]l <
32
Ca2 Rearranging, summing from = 0, , - 1 and dividing by T, we obtain
T-1
F F(wo) F(wr) + 6a C2 + 3L
E
[V((m)))
<
7T
2
Then, choosing Y
2 3LVT
we obtain
1
E
Vf((w))1
3LF(wo) 2vT
+ + + o²
T
VT
t=
Hence, choosing T= O 222 F(wu)
ensures that
E [((()))) = 0 =0(=:))))]](((+c(0)))))0
Note that +ED E | (w:))l is equal to EE [i where wy is drawn uniformly at random from [Wo,.... WT+1). ). This concludes the proof.
E.2 ATTACK OPTIMALITY
In this subsection, we discuss the results concerning the attacks discussed in the paper. Lemma prove that optimization problem satisfying certain conditions, of which the optimization problems corresponding to our attacks are instances, always admit an optimizer on the extreme point of the constraint set X. Based on this result, we solve the optimization problem associated to (2) and its HIPS variant in closed form Lemma 5 and Corollary 7.
Lemma 4. Let X B be convex polytope. Let T R. be continuous and convex in X. Then the optimization problem
max (x),
TE
admity optimizer extreme point of X.
24

---

Published as a conference paper at ICLR 2025
Proof of Lemma 4. By the extreme value theorem, attains maximum x² in X. Since A' is convex polytope, each x' E can be represented as a convex combination of the extreme points X denoted by Ext(X). Denote by {2i the extreme points of X. In particular represent the maximum x= ->i Pa where p A.. By Jensen's 's inequality, have
f(x")= f
E Pidz
S
fax:)
1=1
Since p; are non-negative, there exists least one z; such that (xx; 2 f(="), ), by the optimality of
x², we have that f(x") 2 (x) for all x E X, hence f(x;) = f( ) which concludes the proof. D
Lemma 5. The prediction YB² = 1, with E arg min Y2 is an optimizer of (2) if one of the two statements hold
1. C(a, CCL(b,, a) and FN Int(A.)
2. L(a.b) MSE(a, b).
Remark 6. The image of the ceftmaxfurction is Int(A.). This means that if the honest clients use the softmax activation function in order to obtain probability distribution over the classes, which standard practice in training neural networks, the assumption YH E (n)) in Lemma 5 is always satisfied.
Proof of Lemma 5. For the sake of simplicity, we write y del (x) and def Yz(z) throughout this proof.
Case 1. The MSE is convex and differentiable, hence by Lemma 4, there is an optimizer of Equation (2) on an extreme point of A., i.e., the one-hot vector 1, for all i E [c]. Using MSE and the characterization of the optimizer, rewrite Equation (2) as follows
1y ((1 ((y- ax))| max lly x112 = max ly 1:11² =ED.
zEá.
sE|c
We rewrite the problem
0k max [ly 111l a 2 iEfc
Σ
5ラ (y: - 1)2 jtt
max
sEld
Note that a E [0, 1). monotonically increasing and (1 a)² monotonically decreasing. Hence the first summand of the right hand side is maximized by choosing the largest entries of and second summand of the right hand side is maximized by choosing the smallest entry of y. We conclude that arg max;gle] y: is optimizer of Equation (2).
( v) is continuous for v E Ext(Ac) and w Ag and convex for Int(A.) and a > O, hence az + a)y E Int(Ac), meaning that ト CEL(U, az + (1 a)y) is continuous for all A.. Hence by Lemma 4, there is an optimizer of Equation (2) on an extreme point of A., i.e., the one-hot vector 1, for all i E [c]. We explicit Equation (2),
Case 2. Note that v CBL(W, E Ag. By assumption,
CEL(E, (1 a)y + ayy ZEA.
Σ x, log((1 a)y, + ax.).
zEA.
Since Equation (2) admits an optimizer on an extreme point of Ans we can write
k = max JE(c]
-リ log((1 - a)uı + a)
Σ
vi log((1 - a)vi)
25

---

Published as a conference paper at ICLR 2025
We rewrite the problem follows
max jElc]
Yi log((1 - a)yi) + My log((1 - a)3;)) yj log((1 - a)y; + +a)
(1 (1-
Vi log((1 a)yi) max yj log selc]
(1 a ) +
Note that (1-a)ys
for all [0, and yj E (0, hence the logarithm is negative
(I-a)u,+o
and since yj > 0, the term we want to optimize is monotonically decreasing. We conclude that
k arg min] Mj optimizes the problem and I is optimizer of Equation (2).
Corollary (HIPS+LMA). Consider the following optimization problem
max C ((1-a)) xXg
(4)
Ye(z)A
where det conv (=))))))) Then under the assumption of Lemma the problem admits solution extreme point of A.
Proof of Corollary 7. As shown the proof Lemma 5, for both cases is differentiable and convex, hence the statement follows directly from Lemma 4.
E.3 COUNTEREXAMPLE
Consider the following three predictions
(0.7
00.8)
0
y1
0.2
V/2
0.1
y3
0
(5)
0.1
0.1
which all lie in Ag. Then the coordinate-wise median (which is a special case the 50 coordinate- wise trimmed mean) would yield (0.7, 0.,,0.1 0.1),.1),, which lies outside of the simplex.
26


Reviews:
[{"review_id": "Ws8oXS3y47", "reviewer": "Reviewer_4qUt", "summary": "This paper first shows that distillation based method is more robust to conventional federated average using the Gaussian attack as an example. Then based on the observations, two labelling attack schemes are developed to break current distillation based robust algorithms and accordingly a new algorithm is developed to against the new attacks.", "questions": "1. It is not clear what is the strength (mean/variance) of the Gaussian attack that is used to show the superiority of KD-FL against FedAvg.  Meanwhile, it is not fair to draw the conclusion that KD-based method is more resilient to FedAvg by one comparison between random Gaussian attack to FedAvg and the random label flipping for KD-FL. A fairer comparison could include some other attacks.\n2. The study of HIPS is interesting, however, the result simply suggests that sometimes robust aggregation is worse than mean aggregation. To make this point more valuable, it might be better to have a trade-off analysis or provide some guidelines to how to choose robust aggregation methods. \n3. The prediction for honest agent 4 in Figure 2 should be [0.61, 0.08, 0.31] to ensure the correct average prediction.\n4. At least GM has been widely used in FedAVG for robust aggregation, therefore in line 387, when the authors say “As a result, applying them to aggregated parameters in Byzantine FedAVG is often impractical” does not make sense to me. It would be great if the authors either provide evidence supporting their claim about impracticality, or to revise their statement to better reflect the current state of practice in the field.\n5. What is the baseline performance of FedAVG on the tested datasets? To me, KD-FL is more communication efficient as it only transfers predictions rather than model parameters, but is its performance better than or equivalent to FedAVG? If yes, then it makes sense to study KD-FL in Byzantine settings, otherwise, we can trade some communication for performance improvement and develop some communication-efficient and robust FedAVG algorithms. \n5. ExpGuard shares some similarities with the FLTrust method, which is not new. It would be better to clarify the strength and difference of ExpGuard against FLTrust [1].\n[1] Cao, Xiaoyu, Minghong Fang, Jia Liu, and Neil Zhenqiang Gong. \"Fltrust: Byzantine-robust federated learning via trust bootstrapping.\" arXiv preprint arXiv:2012.13995 (2020).", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is well written and constructed in a fluent structure. Extensive experiments have been conducted to verify the proposed ideas and methods. The proposed method is also somewhat new in the Knowledge distillation domain.", "weaknesses": "1. The comparison of FedAVG and KD-FL is a little weak and the experimental part is limited to KD-based methods, making the whole idea less convincing. \n2. The proposed ExpGuard+F method is similar as the FLTrust methods, which is okay, however, the strength of the proposed method is not exactly reflected from the experiments. For example, is ExpGuard+F better than ExpGuard+GM and if yes, why?\n3. The study of HIPS conveys the message that some attacks are hard to detect and combat, but we can use simple mean aggregation instead. I could not see the significance of this point right now.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6YMnc3zdT3", "reviewer": "Reviewer_zFM4", "summary": "This paper evaluates the robustness of distillation based distributed learning to arbitrary (Byzantine) faults in the i.i.d setting. \n\nThe work illustrates the resilience of KD-based algorithms and analyzes how\nbyzantine clients can influence them. It also proposes two byzantine attacks and assess their ability to break existing\nbyzantine-resilient methods. It also proposes a method to further improve the byzantine resilience of KD-based FL algorithms.", "questions": "What would be the effect of most comon robust aggregation schemes on KD-based FL? how would that change the bound in appendix D.1?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "the paper addresses an important and previously undressed question: the robustness of distributed learning in the distillation setting", "weaknesses": "the formal guarantees (left to appendix D) deserve a more prominent place in the paper, given that the core contribution of such a paper is to formally derive security guarantees and their limits when (in this case) the space of all possible attacks cannot rely on empirical validation", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1I0IIB6B9z", "reviewer": "Reviewer_Zj4j", "summary": "This paper studies the Byzantine resilience of federated knowledge distillation under IID data distributions. The authors start by analyzing the vanilla FedDistill and illustrate the vulnerabilities of FedAvg in the presence of Byzantine adversaries. The authors further propose three additional Byzantine attacks tailored for distillation-based federated learning methods. In response to enhance the federated distillation-based methods, the authors propose ExpGuard.", "questions": "* In Section 4, is it possible to compare the vanilla FedAvg and FedDistill under the same Byzantine adversaries?\n* In Section 4 **the byzantine resilience of FedDistill**, it seems that the authors consider the full-batch gradient only. However, stochastic gradients are the ones that the machine learning community uses in practice. How will the choice of stochastic gradients affect the problem?\n* In Section 4, the authors claim that previous work on byzantine FedDistill focuses mostly on basic label-flipping attacks. Is it possible to consider another classic data poisoning attack: data injection?\n* In Section 5, is the loss minimization attack a plug-in attack on top of the original data poisoning attack at each Byzantine client?\n* In Section 5, can the authors further justify ExpGuard? In particular, why does the exponential weight work, but the other choices won't?\n* In Section 6, can authors conduct more thorough experiments to compare (1) FedDistill algorithm with FedAvg under SOTA defense (2) the varying number of Byzantine clients?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper is, in general, well written.\n* The proposed algorithms and the attack strategies are intuitive and easy to understand.", "weaknesses": "* The most obvious weakness of the paper is that it addresses the IID data distributions. In contrast, the current state-of-the-art (SOTA) Byzantine resilient federated learning methods focus on non-IID data setups. \n* In Section 4, the motivating example to compare the vanilla FedAvg algorithm and the FedDistill algorithm is rather unfair. This is because the adversaries for the vanilla FedAvg employ model poisoning strategies, while the ones in FedDistill use data poisoning.\n* In Section 4, the presentation in **the byzantine resilience of FedDistill** can be improved. All the theoretical theorems and lemmas have been deferred to the Appendix, making it hard to appreciate the real significance of resilience.\n* In Section 5, the loss minimization attack (LMA) is a bit contrary to the prior Byzantine federated learning literature, e.g., [Karimireddy et al. (2022), Allouah et al. (2023)], where they assume the server is curious but honest. Here, the server is not reliable in the sense that it can manipulate the generated model parameters to mislead the training. At one extreme, the server can completely replace the model parameters to destroy the whole training process.\n* In Section 5.2, the authors did not explain why ExpGuard works, except by citing [Littlestone & Warmuth, 1994]. \n* It is recommended to group HIPS together with LMA & CPA and present the defending strategy after the attacking strategies.\n* The experiments are not thorough enough in two regards. First, the authors need to compare with the performance with FedAvg under current SOTA Byzantine resilient algorithms, e.g., bucketing [Karimireddy et al. (2022)] and [Allouah et al. (2023)] NNM. Second, the ablation experiments do not consider varying the number of Byzantine clients. The algorithms may struggle under benign settings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yRg81efeRH", "reviewer": "Reviewer_ktFr", "summary": "This paper explores the robustness of FedDistill, a knowledge distillation federated learning (FL) method, under Byzantine settings, where some clients act maliciously to disrupt training. The paper introduces two new Byzantine attacks, Loss Maximization Attack (LMA) and Class Prior Attack (CPA), that exploit this prediction-sharing approach. It also presents ExpGuard, a novel defense mechanism to counteract these attacks. The authors finally propose the HIPS method to obfuscate attacks, making them harder to detect. Experimental results demonstrate that FedDistill is more resilient than traditional FL aggregation methods.", "questions": "1. In line 328 and figure 2, shouldn't the least likely class be _dog_ rather than _cat_?\n2. CPA attack requires a pretained model to obtain the similarity matrix, how do you get it in the experiments? Is this effective in a more realistic setting?\n3. Will the distribution shift between clients' data and public data affect the performance of the server's model? It's better to provide additional experiments or theoretical analysis on how different levels of distribution shifts between client and public data might impact the server model's performance. This could help readers better understand the limitations and robustness of the proposed approach.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. Considering KD-based FL in a Byzantine setting is a good idea.\n2. Propose two effective attacks for KD-based FL and a defending method. Additionally, the authors introduce a method that obfuscates attacks to make them harder to detect. \n3. The experimental results show the effectiveness of the proposed methods.", "weaknesses": "1. The paper only considers iid setting. Although the authors claim that non-iid setting in FL poses serious issues (line 122), they don't specify what kind of issue makes it unsuitable for KD-based FL. Moreover, non-iid setting is essential in FL and is closer to realistic scenarios.\n2. The scale of experiments is limited. The paper only considers cross-silo settings (a small number of clients) and lacks cross-device settings (over a hundred clients).\n3. The ExpGuard assumes that the Byzantine clients are identical during training, which may be not the case in reality.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)