HYDRA-FL: Hybrid Knowledge Distillation for Robust and Accurate Federated Learning
Authors: Momin Ahmad Khan
Yasra Chandio
Fatima M. Anwar
Venue: NeurIPS 2024 poster (2024)
Keywords: Federated Learning
Knowledge Distillation
Poisoning Attacks
Primary area: other
TL;DR: We show that Knowledge Distillation amplifies model poisoning attacks in FL, and design our algorithm HYDRA-FL to mitigate attack ampification.

Abstract: Data heterogeneity among Federated Learning (FL) users poses a significant challenge, resulting in reduced global model performance. The community has designed various techniques to tackle this issue, among which Knowledge Distillation (KD)-based techniques are common.
    While these techniques effectively improve performance under high heterogeneity, they inadvertently cause higher accuracy degradation under model poisoning attacks (known as \emph{attack amplification}). This paper presents a case study to reveal this critical vulnerability in KD-based FL systems. We show why KD causes this issue through empirical evidence and use it as motivation to design a hybrid distillation technique. We introduce a novel algorithm, Hybrid Knowledge Distillation for Robust and Accurate FL (HYDRA-FL), which reduces the impact of attacks in attack scenarios by offloading some of the KD loss to a shallow layer via an auxiliary classifier. We model HYDRA-FL as a generic framework and adapt it to two KD-based FL algorithms, FedNTD and MOON. Using these two as case studies, we demonstrate that our technique outperforms baselines in attack settings while maintaining comparable performance in benign settings.

HYDRA-FL: Hybrid Knowledge Distillation for Robust and Accurate Federated Learning
Momin Ahmad Khan University of Massachusetts, Amherst makhan@umass edu
Yasra Chandio
University of Massachusetts, Amherst ychandio@umass. edu
Fatima Muhammad Anwar
University of Massachsuetts, Amherst fanwar edu
Abstract
Data heterogeneity among Federated Learning (FI.) users poses significant chal- lenge, resulting in reduced global model performance. The community has de- signed various techniques tackle this issue, among which Knowledge Distil- lation (KD)-based techniques are common. While these techniques effectively improve performance under high heterogeneity, they inadvertently cause higher accuracy degradation under model poisoning attacks (known as attack ampli- fication). This paper presents case study to reveal this critical vulnerability in KD-based FL systems. We show why KD causes this issue through empir- ical evidence and it motivation design hybrid distillation technique. We introduce a novel algorithm, Hybrid Knowledge Distillation for Robust and Accurate FL (HYDRA-FL.), which reduces the impact of attacks in attack narios by offloading some of the KD loss to shallow layer via an auxiliary classifier. We model HYDRA-FL as generic framework and adapt it two KD-based FL algorithms, FedNTD and MOON. Using these two as case studies, we demonstrate that our technique outperforms baselines in attack settings while maintaining comparable performance in benign settings. Our code is available at https://gittuu. com/momin ahmad ihhanHHDDRA-FF
1 Introduction
Federated Learning (FL) 33 is emerging machine learning paradigm enabling multiple users' collaborative model training without data sharing. Each user, termed a client, only shares their local model with server, which aggregates all local models into a single global model and redistributes it to the clients. Due to its decentralized,
and highly-scalable nature, FL has been adopted by Google's Gboard [2] for next-word prediction, Apple's Siri (T] for automatic speech recognition, and WeBank 44 for credit risk prediction.
Despite its benefits, FL faces challenges with data heterogeneity when client data is independent and identically distributed (IID) and achieves similar convergence single model trained all the clients' data but struggles when clients have diverse data (non- IID. In this case, the client's local data is not a good representation of the overall data distribution (unlike ideal IID case), causing local models drift away from each other. This drift results in global model with significant accuracy degradation compared to the IID scenario. Numerous solutions 26 23 24 47 16 28 address data heterogeneity, including Knowledge Distillation (KD) 12 to reduce the drift between local models.
29
52
25
FL performs well
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

KD
KD
polson COOOO IITTO
client 1
1
KD
client
KD
client 2
benign reference
benign global model
model
with KD in benign setting
FL with KD maticious setting
Figure I: Overview of attack amplification through knowledge distillation. a) In the benign setting, KD reduces drift and brings benign local models closer to the benign global model. b) In the malicious setting, KD unknowingly reduces drift between benign local models and the poisoned global model.
Besides data heterogeneity. FL also faces the issue of Byzantine robustness 15] where untrusted clients can inject poisoned models into the aggregator by altering client data (data poisoning (36)
or client models (model poisoning [1] 4 34
46
5 3 42)). Research by [41] shows that model
poisoning attacks more potent they directly manipulate local models. To counteract poisoning
in FL, various defenses have been developed 6
27 g].
In this work, identify critical vulnerability in KD-based FL techniques under model poisoning attacks. The adversarial settings, i... threat model, attacks and defense, are explained in $C These techniques unknowingly align benign client models with poisoned server model (Figure D We study two such classes of KD-based solutions: FedNTD 21 which reduces the loss between not-true logits of the server and client models, and MOON 26), which reduces the contrastive loss between the representation vector of the server and client models. While these techniques improve global model accuracy in benign settings compared to FedAvg [33] (standard FL aggregator), they reduce performance below FedAvg under attack, phenomenon we term altack amplification, especially noticeable higher heterogeneity levels.
Motivated by our findings, we propose a Hybrid Knowledge Distillation for Robust and Accurate FL (HYDRA-FL. -FL) framework for KD-based techniques that restricts attack amplification under poisoning attacks while retaining performance the benign setting. Unlike traditional KD methods that apply KD-loss only at the final layer, HYDRA-FL. introduces KD-loss
shallow layer via auxiliary classifier and reduces the KD-loss impact at the final layer. This approach draws inspiration from tiffDDitillltionn (SD) (50] and Skeptical Students (SS) [19), but with distinct focus enhancing robustness against heterogeneity and model poisoning attacks in FL.
SD improves model accuracy by seff-distillation, while SS distills from "nasty teachers" 31 to shallow layers. contrast, our approach uses auxiliary classifiers to enhance FL client robustness against heterogeneity and model poisoning attacks. We design generie loss function adaptable to specific KD-based algorithms. Our evaluation show that HYDRA-FL significantly boosts accuracy over FedNTD and MOON in attack settings while maintaining performance in benign settings.
Contributions. This work addresses the critical issue of attack amplification in KD-based FL techniques to counter data heterogeneity. In doing so we make the following contributions:
Proving KD amplifies model poisoning: our motivational case study ($3) on two KD-based techniques, FedNTD and MOON, shows that KD improves accuracy benign settings but helps the malicious clients propagate poisoning through the KD-loss in adversarial settings. We empirically and theoretically show that this attack amplification issue is inherent to any technique aligning client output//repressmattons with the server.
2. Designing HYDRA-FL: using our observations as a guideline, we design HYDRA-FL ($4)
prevent attack amplification while retaining performance in the benign setting. HYDRA-FL is formulated as a general loss function that can be adapted to any FL algorithm so that it can be used as its local model training objective.
3. Implementation and Evaluation: we adapt HYDRA-FL to the FedNTD and MOON algo- rithms and modify their local training objectives (55). Our detailed qualitative and quantitative analysis ($6) shows that HYDRA-FL. not only achieves higher accuracy in attack settings but also maintains accuracy in the benign settings.

---

2 Background and Related Work
2.1 Federated Learning (FL)
In FL [15 33), a service provider, called server, trains a global model, 89, on private data from multiple collaborating clients, without directly collecting their data. The server selects n out of N clients in every FL round and shares the most recent global model (0y) with them, where & is the round number. Then, client k uses local data D2 to compute update V and shares it with the server. The server aggregates these updates using some aggregation rule, like FedAvg [33 algorithm.
In FedAvg, a client k fine-tune 8e on their local data using stochastic gradient descent (SGD) for fixed number of local epochs E, resulting in an updated local model 6%. The client then computes their update as the difference Vi = 6% 8G and shares V with the server. Next, the server computes an aggregate of client updates, Jagg using mean, i.e.,
V Smean V (kE(n)))
The server then updates the global model of the (t + 1)th round using SGD and server learning 1) as:
g t- 9 + V agg
(2)
2.1.1 Data Heterogeneity in FL
in FL.. Each client in FL generates its data, leading to local data distributions that vary across clients and do not accurately represent the global data distribution. By extension, global model learned by aggregating local models using FedAvg may not be the best representation of all the client's local data. Studies have shown that this data heterogeneity degrades performance and have proposed various methods address this issue [26 21 23 [54] [24 47 [6 28). This degradation is more prominent in the presence of poisoning attacks. Research on poisoning attacks in FL has demonstrated that such attacks become more successful under high heterogeneity [I]] 42). This increased risk is because the malicious clients can more easily hide between drifted benign client models, making difficult for the server to differentiate between heterogeneous benign clients and malicious ones. [17] highlights that overlooking this heterogeneity a critical oversight in FL defense evaluations. Unlike MOON [26) and FedNTD (21) the two baseline techniques used in this paper which address data heterogeneity only no-attack settings, HYDRAFL operates in both no-attack and attack settings, achieving robustness and accuracy under high heterogeneity.
Data heterogeneity is well-explored problem 29
52
14 25
2.1.2 Poisoning FL.
FL is vulnerable to poisoning attacks 6
where malicious clients aim to compromise the training process by degrading the global model's performance. These attacks come various forms: In data poisoning (3. malicious clients poison their local data to introduce a backdoor in the local model. This backdoor then propagates to the global model upon aggregation.
46
36 42
3 42). malicious clients perturb their local models so that, when aggregated, the global model is poisoned. Poisoning attacks can be further classified based on their targets: If the performance degradation is on specific inputs, the attack is termed as targeted 46)
In model poisoning [1] 4 34, 46
5
poisoning [5] 3 and if it is all inputs, then it is termed untargeted poisoning [1]
We explain the attacks used in this paper in $C.2 Our paper shows how KD-based hetrroettroogyeitt mitigation techniques amplify poisoning attacks and that our approach HYDDAFF addresses both data heterogeneity and poisoning.
2.2 Knowledge Distillation (KD)
Knowledge Distillation (KD) [12 transfers knowledge from large, complex model (teacher) to a smaller, more computationally efficient model (student). This process involves distilling the teacher's rich and intricate information into the student by aligning their predictions. Formally, the teacher and student models produce the output probabilities yf and y,, respectively for the 7th input (at, )²), KD aims to match these probabilities by applying the Kullhack-Leibler (KI.) divergence between them. The KL-divergence between their softened probabilities is given by:
3

---

KL(softmaz(ui) (oop://llllofmrar//// where T is the temperature parameter that softens the proba- bilities. The overall KD loss function combines this KL-divergence with the usual loss function such as cross-entropy (CF) loss with / (balances the importance of the KI.-divergence and CE loss) as:
L = (1 - B) (CE(V.. v²) + +B / htol:///ssllllllsoo//ooooo (v./T))
(3)
KD in FL is becoming essential it addresses critical challenges such as non-IlI data distributions, enhances model performance, accelerates convergence, reduces communication overhead, and im-
proves robustness by making the global model learn from an ensemble of local models [10] 23 30 53
In FL, data is often non-IID across clients, leading to significant discrepancies in local models. KD
mitigates these discrepancies by aligning the local models with the global model, ensuring that the global model captures a more generalized representation of the data. The general approach is to reduce the local model drift by improving the aggregation through distillation using unlabeled auxiliary data. However, the auxiliary data may not always be available, and methods have also been developed enable KD without such data [49 (54].
3 Attack Amplification through Knowledge Distillation
Hypothesis. KD-based techniques in FL improve accuracy in non-adversarial settings but result in more significant accuracy degradation under model poisoning attacks compared to the baseline techniques such as FedAvg.
Motivational case study. In this case study, we compare FedAvg against two distinct KD-based solutions addressing the local model drift from non-IlI.. MOON (26] uses model-contrastive learning to align local and global model representations, while FedNTD uses KL-divergence to align not-true logits of client models with those of the server. FedNTD penalizes prediction divergence measured through distillation loss, improving knowledge transfer and stability, while MOON pe- nalizes representation divergence measured through contrastive loss, enhancing robustness and generalization. This comparison will help us understand the trade-offs of using KD FL, especially under adversarial conditions. Throughout this paper, benign conditions mean that no attacks are present, while adversarial conditions mean that model poisoning attacks are present. We implement the same settings and hyperparameters for FedAvg as for MOON and FedNTD to ensure a fair comparison, so FedAvg results may vary between these techniques. This is not an inconsistency. We do not directly compare FedNTD MOON unless stated otherwise, the original FedNTD work already did so. Our goal is to test how adversarial settings affect these two fundamentally different techniques similarly, demonstrating that attack amplification is inherent to KD and not specific to particular technique.
Adversurial conditions. We simulate untargeted model poisoning attacks using techniques from 42 I. To observe their effects on accuracy in both benign and adversarial settings, we vary key hyperparameters divergence loss coefficient 3 for Fed- NTD and contrastive loss coefficient for MOON. The baseline for com- parison is FedAvg with 3 = 0 and ===0 0. To ensure high heterogeneity in both settings, the Dirichlet distribu- tion (35] parameter a is fixed at 0.1.
Sengg Iis
auurtyy deo
LD-
13
(a) FedNTD. 8 = is FedAvg
(b) MOON, is FedAvg
Figure 2: Impact of increasing KL-divergence loss for Fed- NTD and contrastive loss for MOON on accuracy.
Findings. In Figures ((a) and 2(b) we present three key results: benign accuracy (blue), post- attack accuracy (orange), and the accuracy drop (green). We make the following observations from increasing 3 and μ are as follows: (1) the global model accuracy improves in benign settings; (2) post-attack accuracy decreases; and (3) accuracy drop increases. Our analysis shows significant trade-off: the very mechanisms that improve performance in benign conditions (increasing 3 and (1)
also make the modely more valnerable to adversarial altacks.
What causes attack amplification? The fundamental nature of KD-based Fl. methods aims to align local models with the global model. In benign scenarios, these methods significantly outperform

---

FedAvg [26 21]. However, in the presence of model poisoning attacks, this model alignment process inadvertently forces local models align its representatoonlrrdiiiionn the poisoned global model, amplifying the attack's 's impact. This is illustrated in Figure where clients unknowingly distill knowledge from a poisioned server model.
Formally: Consider a set of n clients eı, C....., Cn with mn being malicious. Using an aggregation rule such FedAvg, the server aggregates updates from both benign (V (ef(m+1,n)) and malicious (V #E|m) clients:
Sage(V (Eim))
VE(mJ
U Vie(m-1,n)
When m = 0, the server model V is benign. For m + O, the server model V is poisoned, deviating from the ideal unpoisoned global model due to the nature of these attacks 42 41). Aligning local models with a poisoned global model reduces gradient diversity, making local models more similar the poisoned global model [21] through KL-divergence contrastive loss. We rewrite Equation to formalize the loss function for an FL client, using KD, where the client is the student with output yc. and the server is the teacher with output ys:
&
== LCE(Dee v) - BC (Dc.
Note that for the sake of derivation here, we are using de. which represents the generic client model output. the case of FedNTD, can be replaced by Je that represents the not-true logits of the client model, and in the case of MOON, it can be replaced by 2, that represents the client model's 's high dimensional representation.
In benign scenarios, this loss function (L = =(3)) decreases monotonically with & because KI) brings local models closer to an unpoisoned global model. Conversely, in adversarial scenarios, increases with 33 because KD brings local models to the poisoned global model. We can write the relation of this loss function with 3
monotonically decreasing, m = 0 monotonically increasing, * 0
((3) is
(6)
Then, the derivative of the loss function is:
dC
< 0, [
m =0
(7)
d3
0, m #0
Our derivation shows that while the distillation process decreases loss in the absence of malicious clients, increases loss in their presence, thereby leading to reduced global model accuracy. This formal analysis highlights the need for a solution that mitigates the accuracy degradation under adversarial conditions while retaining the benefits of KD under benign conditions.
Impact of Heterogeneity.
Now, explore the effect of het- erogeneity on the performance of FedNTD, MOON, and Fe- dAvg in both benign and adver- sarial conditions gain deeper insights into the role of hetero- geneity in the KD performance gain vs. vulnerability tradeoff. As shown in Figure 3(a) several interesting observations emerge. First, both FedNTD and FedAvg achieve higher accuracy at lower heterogeneity levels (indicated by higher a). In benign settings, FedNTD consistently outperforms FedAvg. However, the trend reverses adversarial settings: FedAvg achieves higher accuracy than FedNTD, except at a = 0.5. A similar pattern is observed with MOON in Figure (b) where FedAvg outperforms MOON across all heterogeneity levels in adversarial settings. In the benign setting, as expected, MOON slightly outperforms FedAvg at high heterogeneity. This comparison highlights again how the alignment mechanisms in FedNTD and MOON with higher heterogeneity exacerbate the vulnerability of KD methods to attacks.
65
fsdagg
amagy
5.55
05
0.5
(a) FedNTD vs FedAvg
(b) MOON vs FedAvg
Figure 3: Impact of the heterogeneity parameter, in benign and adversarial settings. We use the Dirichlet distribution where a higher means lower heterogeneity.
5

---

HYDRA-FL: Hybrid Knowledge Distillation for Robust and Accurate FL
client
clints send locel model to the server
Lce ()
elients upziate model with local ditta
cuent
LDD (c))ss)
= Lce (rly)
Local update
classifier
auxiliary
LD Vaux Ys)
8/ Exn Gl. yLRo Vauzo Ys)
rule
server updates globel
server
server sends global model to clients
Figure 4: HYDRA-FL framework: we refine client model training by reducing the final layer's KD-loss and incorporating shallow KD-loss at earlier shallow layer via an auxiliary classifier.
4.1 Generic Formulation
In this section, we propose Hybrid Knowledge Distillation for Robust and Accurate FL (HYDRA-FL), technique mitigate the attack amplification caused by KD FL. We take hybrid distillation approach, applying KD-loss at both the final and a shallow layer of the client model (Figure 4 This method incorporates shallow distillation, which applies KD-loss at an intermediate layer and helps reduce the impact of poisoning by preventing over-reliance on final layer alignment. Shallow distillation previously used handle nasty teachers trained adversarially [9]. to reduce the impact of poisoning. In summary, shallow layers capture basic features, and shallow distillation ensures these features are robustly learned, protecting the model from adversarial influences that could corrupt deeper layers and final outputs. We first formulate the generic loss function of an FL client using KD) HYDRA-FL. as:
3 b CKD Uo) + (Haury Us)
C Lc v)
This loss function has three key components:
1. Cross-entropy loss (Coe(Nes y)) is the loss between the client's prediction ye and the target y, drives the client model to learn from its own data, ensuring captures in-distribution knowledge such as features and patterns specific its data.
2. Diminished KD lass ( CD Us]) is the loss between the client's output/representation
Ve and the server's output/representation y. It is a strategic reduction of the KD loss to ensure that the local model benefits from the global model's knowledge while remaining robust against adversarial attacks. This approach helps balance the trade-off's between learning efficiency and model integrity. In practice, this is achieved by introducing a diminishing factor b to the KD loss at the client model's output layer to diminish the poisoning effect. The KD loss coefficient 3 is divided by b, effectively reducing its weight in the total loss calculation, thus reducing its influence
the local model's training. This diminishing factor is essential, as shown later in $6.3 and Figure where reducing the 3 yields better results.
3. Shallow distillation loss < (Baurs ys) is applied at shallow layer of the local model, enhancing robustness without heavily relying on final layer alignment. This loss, between the auxiliary classifier's output/representation Yaur at the client model's 's shallow layer and the server's output/repressentation ys. is scaled by to control the amount of distillation. This reduces the impact of poisoning on the client model. Simply reducing the KD-loss in FedNTD or MOON improves post-attack accuracy but reduces benign setting accuracy, as shown in Shallow distillation loss helps maintain the balance between accuracy in benign settings and lowering the impact poisoning on the client model in adversarial settings.
Figure 2
Differences with previous works.The key difference between our work and [19 lies in our approach to shallow distillation. [19] aims to distill from models that are designed to be undistillable, a.k.a
"y. and in generic LKD loss can be either outputs representations, because the method can involve either type of comparison (e.g., MOON representation-basse loss while FedNTD has output-based based loss.)

---

nasty teachers 137y. While both use hybrid shallow distillation, [9 completely removes the KD-loss from the model's output layer and uses self distillliilanonn to compensate for performance loss due to shallow distillation. In contrast, we retain scaled-down KD-loss at the output layer. We found that completely removing the KI>loss at the output layer may cause more negative impact than keeping it in reduced form. Additionally, the untargeted poisoning is different from the poisoning in the "nasty teacher" paper [31]. The "nasty teacher" performs near near-perfect under normal conditions unless malicious model distills from it. In untargeted FL poisoning, the global model is poisoned and performs poorly regardless of its use for distillation.
In HYDRA-FL, we use both final layer and shallow layer distillation to enhance robustness. Final layer distillation aligns client outputs with server outputs for consistent predictions, whereas the shallow layer distillation aligns intermediate representations to improve robustness against attacks. This dual approach reduces vulnerability to poisoning attacks, enhances learning by leveraging knowledge transfer from multiple layers, and maintains high accuracy in benign settings while being resilient under attack conditions.
4.2 Adapting HYDRA-FL State-of-Art Techniques
In this section, we will adapt our generic HYDRA-FL to state-of-the-art KD techniques for FL.
FedNTD with shallow distillation and auxiliary classifiers. We modify the FedNTD base model by introducing auxiliary classifiers. The base model includes two convolutional layers, linear layer, and a classification layer. Auxiliary classifiers, each consisting of a linear layer (hidden dimension 512) followed by classification layer, are added after each convolutional layer. We update the loss function to include a not-true logits of auxiliary classifier and the global model. The final loss function is weighted sum of the standard cross-entropy loss, KL-divergence loss between the not-true logits the global model and the client model, and the KL-divergence loss between the not-true logits of the global model and the auxiliary classifier. The revised loss function in Equation & for FedNTD is:
term, representing the KL-divergence loss between the
3
L = LCE(Ve: v) + [KI( Ue) + 7fK Us)
(9)
Here y the target label, ye is the client model's 's output, Us: Ves and Yaur are the client model's, server model's, and auxiliary classifier's not-true logits respectively.
MOON with shallow distillation and auxiliary classifiers. MOON base model has two convolution layers, two linear layers, and an output classification layer. We insert auxiliary classifiers after each convolution layer. Each auxiliary classifier has two linear layers, with a hidden dimension of 256 and an output dimension of 10. We adapt Equation X to MOON to compute the contrastive loss at the hidden representation layer of the auxiliary classifier as:
C = LoE(ye: v) + 1t Ccon (Ec: 2e) + "Coo (Zouz; kz)
(10)
Here y is the target label, Ve is the client's output, 2e is the representation from the client's final layer, 2, is the representation from the server's final layer, Zoue is the representation from the client's s auxiliary classifier, and y. is the server model's output. For simplicity, we do not write the previous round's representation in the loss function here.
5 Experimental Results
5.1 Experimental Settings
We conduct our experiments over three popular datasets: MNIST, CIFAR10. and 10FAR100. To ensure fair comparison with previous works, MOON and FedNTD, we utilized the same models and hyperparameters they used. Specifically, we incorporated our algorithm as
Datasets and Models:
simple modification into their publicly available codes 22 38
(more details on experimental and
adversarial setups in Appendices D & C).
7

---

Table I: Test accuracy for three techniques on three datasets. In the no-attack setting, (1 J) shows comparison FedAvg. In the attack setting, we use bold if our technique outperforms FedNTD.
CIFARIO
Dataset
MNIST
CIFARI00
0,05
a 0.1
0.5
Techniques
no attack
amacc
no atfack
attack
No attak
attakk
no attack
attack
Ro attack
anack
Fedavg
92.12
74,48
31.27
54.67
35.67
70.57
26.17
12.92
FodNTD
93.03
58.09
46.94
21.72
56.95
32.61
71.91
52.51
29.1
13.92
HYDRA-FLIOUS)
92.69
76.67
46.92
25.15
57.12
34.25
71.22
52.57
28.9
14.33
Table 2: Test accuracy for three techniques on three datasets. In the no-attack setting, (1 J) shows comparison FedAvg, In the attack setting, use bold if our technique outperforms MOON.
CIFARIO
Dataset
MNIST
CIFAR100
0.1
0.5
Methods
no affack
antack
attack
attack
80 attack
attack
no affock
attack
atfack
anack
Fedavg
88.02
77.35
37.76
40.9
63.14
60.2
71.19
68.38
28.30
24.11
MOON
91.131
72.32
58.81
39.9
63.34
57.17
70.95.
67
29. 1
23.81
HYDRA-FL(Ours)
92.04
76.65
60.1
43.6
63.32
59.93
70.35]
68.4
29.48
25.18
5.2 Shallow Not-True Distillation
Our hybrid shallow not-true distillation technique significantly improves post-attack accuracy the baseline FedNTD. As shown in Table [ we achieve higher post-attack accuracy across all heterogeneity levels. By retaining diminished NTD loss at the output layer, maintain similar accuracy to FedNTD in no-attack scenarios and, in some cases, even achieve slightly higher accuracy. We also compare no-attack and post-attack accuracies for FedAvg, the foundational algorithm for many FL aggregation methods.
5.3 Shallow MOON
70
HYDRAFL
HYDRAFL 52 no_attack
MOON no_atack
Our shallow-distillation design effectively prevents attack ampli- fication in MOON while main- taining nearly the same no-attack accuracy. Table 2 shows that we achieve higher post-attack ac- euracy across all heterogeneity levels. Our technique also out- performs FedAvg, except in few scenarios. Techniques like MOON are designed to enhance accuracy under high heterogeneity (a 0.1). HYDRA-FL achieves a no-attack [attack] accuracy of 60.1% 80.1%|43.6%% surpassing both MOON (58.8%)39 and FedAvg (57.76%][4)) (4..%%). thereby demonstrating efficacy in high heterogeneity.
Fedfugg s__attckk
Accuraccy
15
HOUIWFLS2 atlack
45
MCON attsk
40
Feding attack
(a) No Attack
(b) Attack
Figure 5: HYDRA-FL. vs. MOON and FedAvg when auxiliary classifiers are placed at different shallow layers.
Analysis
In this section, we provide an in-depth analysis of HYDRA-FL. We begin with a theoretical justifica- tion behind the efficacy of HYDRA-FL. Then we perform qualitative analysis using t-distributed stochastic neighbor embedding (t-SNE (3) plots to visualize the representations of the models. Finally, we explore the impact of different design choices through ablation studies, focusing on the choice of the shallow layer for auxiliary classifiers and the distillation coefficients.
6.1 Theoretical Justification
Here we provide the theoretical justification for the working of HYDRA-FL. The generic loss function of an Fl. client using KD is:
L= Lo y) BLxD(yeV
(11)
In Section 3, we showed that in the presence of malicious clients, de 2Ø 0, i.e., KD causes the loss to increase. In Section 4, formulated the generic loss function of FL client using KD in

---

HYDRA-FL as:
C' = Loe (Ve: v) + CKDe vo) ) + 7LKDDyoaz Us)
(12)
We L' for HYDRA-FL IO loss and for basic KD loss. Comparing them, since < 8 L. follows that { (Ve Us) < (Vc. ys). The entire model is 0, and the part up the auxiliary layer 0'. The impact of shallow distillation loss is only on and not the rest the model 0 0'. Therefore, with appropriate and 7, de
23 < J This is shown in Figure 7, where we select = [1, 4] (so effectively becomes and 7. respectively) and 7 show the effectiveness of technique and the impact of the variation of these hyperparameters.
6.2 Qualitative Analysis
We show the t-SNE plots of the representations (Figure generated by the client model for FedAvg, MOON, and HYDRA-FL for both attack and no-attack scenarios. The t-SNE plots show the classes clusters. In the MOON attack scenario, the deviation from the no-attack scenario is much higher than the deviation between HYDRA-FL with and without attack, as evident from the spread of the class clusters, especially along the x-axis.
FedAug No Attack
Moon No Attack
HYDRA-FE FL No Actack
100
100
100
25-
25
25-
-50
-50
-100
155
-10
-302
-100
100
-15
300
FedAug g
100
200
35.
76.
30
50
Figure 6: T-SNE visualizations of CIFARIO local model's hidden representations (a 0.5) FedAvg, MOON, and HYDRA-FL (ours). The attack vs. no-attack plot shows the deviation of the attack clusters from the no-attack clusters. Visually we can see MOON-attack has the greatest deviation, particularly along the x-axis, compared to FedAvg and HYDRA-FL.
Ablation Study
Impact of choice of the shallow layer. Figure Figure illustrates the impact of the choice of the layer at which we insert our auxiliary classifier. We represent these choices by HYDRAFL-S/ and HYDRAFL. S2, where the auxiliary classifier is inserted after the first and second convolutional layers, respectively. We compare them both attack and no-attack settings with simple MOON and FedAvg. In Fig- ure 5(a). both HYDRAFL-SI and HYDRAFL-S2 outperform other techniques low heterogeneity in the absence of an attack but slightly underperform low heterogeneity when 5. Figure (bb) shows that both HYDRAFL-SI and HYDRAFL-S2 achieve higher post-attack accuracy at all het- erogeneity levels, with HYDRAFL-S giving slightly higher accuracy than HYDRAFL-SI. The go shallower,
benefit from the contrastive loss reduces
optimal balance is necessary.
Impact of distillation coefficients. We examine the impact distillation coefficients on performance of FedNTD and HYDRA-FL. Figure 7 shows the post-attack accuracies with two different values of the diminishing factor = 1,4, resulting output-layer NTD-loss coefficients of and 0.25. Diminishing the coefficient 3 leads improved performance, with significant

---

Table 3: MNIST test accuracy with 0.05 heterogeneity for MOON
Methods
no allack
allack
FedAvg
69.2
59.36
MOON
73.57
55.32
HYDRAFL shallow layerl
76
57.4
HYDRAFL shallow layer2
75.15
57.71
increase in post-attack accuracy for 3 = 0.25 at high heterogeneity (a = 0.05, 0.1). As demonstrated
in $3 3 contributes to attack amplification in FedNTD. Reducing it while performing distillation at the auxiliary classifier yields the best performance. For example, at 0.05, HYDRA-FL achieves 25.15% accuracy at / 1, but a much higher accuracy of 28.81% % at 0.25. Similar
improvements are observed at other heterogeneity levels.
6.4 Additional Results on MNIST at Very High Heterogeneity
To demonstrate robustness and generalizability, we con- ducted more experiments now on MNIST at much lower heterogeneity of 0.05. Table 3 shows attack amplification and the effectiveness of our solution. MOON's accuracy drops from 73.57% (no-attack) 55.32% (attack), show-
FediTD
50
-
HYDRA-HL
Accuracy
40
ing attack amplification compared to FedAvg. HYDRAFL achieves higher accuracy than both FedAvg and MOON in the no-attack setting and higher accuracy than MOON in the attack setting, reducing attack amplification.
35
30
0.05
0.1
0.3
0.5
6.5 Thoughts on Data Poisoning
Figure 7: Comparison of performance of FedNTD-S with different values of B
While data poisoning in the context of knowledge distil- lation (KD) requires dedicated study, we believe our solution is broadly applicable as it prevents the flow of poison through the entire model and dilutes its effect. Consider a simple example: label flipping. basic data poisoning attack where some labels in the training dataset are flipped (e.g., changing bird image's label to 'airplane' in CIFAR-10). Suppose the server model (teacher) is poisoned with this flipped dataset. During the distillation using HYDRAFL, this poisoned distillation information would be diluted
process, since the final layer. Instead, it will flow through an auxiliary classifier into the early layers of the model, thereby reducing the impact of the data poisoning attack via label flipping.
7 Conclusion
In this paper, we first identified a critical issue in KD-based FL techniques that aim to tackle data heterogeneity; in the presence model poisoning attacks, these techniques help the attacker amplify its effect, leading to reduced global model performance. We presented empirical evidence and theoretical reasoning back this claim. This motivated us propose HYDRA-FL. a hybrid knowledge distillation technique for robust and accurate FL technique that aims to tackle both data heterogeneity and model poisoning, two the biggest problems in FL. Through extensive evaluation across three datasets and comparing with baseline techniques, FedNTD and MOON, we showed that HYDRA-FL achieves superior results.
Acknowledgements
We thank the NeurIPS reviewers for their insightful feedback and comments. This research is supported by NSF grant 2237485.
10

---

References
[1] How
Apple
personalizes
Siri
without hoovering up your
data.
[2] Federated learning: Collaborative machine learning without centralized training data. https
laa..gogleblog. http:///////eeeeeerreeeaaaaaaaaa
html 2017.
[3] Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov. How
to backdoor federated learning. In AISTATS, 2020.
[4] Moran Baruch, Baruch Gilad, and Yoav Goldberg. A Little Is Enough: Circumventing Defenses For Distributed Learning. In NeurIPS, 2019.
[5] Arjun Nitin Bhagoji, Supriyo Chakraborty, Prateek Mittal, and Seraphin Calo. Analyzing federated learning through an adversarial lens. ICML, 2019.
[6] Peva Blanchard, Rachid Guerraoui, Julien Stainer, et al. Machine learning with adversaries: Byzantine tolerant gradient descent. In NeurlPS, 2017.
[7] X. Cao, J. Jia, Z. Zhang, and N. Gong. Fedrecover: Recovering from poisoning attacks in federated learning using historical information. In 2023 2023 IEEE Symposium on Security and Privacy (SP) (SP), pages 326-343, Los Alamitos, CA, USA, may 2023. IEEE Computer Society.
[8] Xiaoyu Cao, Jinyuan Jia, and Neil Zhenqiang Gong. Provably Secure Federated Learning against Malicious Clients. AAAI, 2021.
[9] Hongyan Chang, Virat Shejwalkar, Reza Shokri, and Amir Houmansadr. Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer. arXiv: 2019.
(10) Hong-You You Chen and Wei-Lun Chao. Fedbe: Making bayesian model ensemble applicable to federated learning. arXiv preprint arXiv:2009. EEt.:2009900994, 2020.
[11] Minghong Fang, Xiaoyu Cao, Jinyuan Jia, and Neil Zhenqiang Gong. Local Model Poisoning Attacks to Byzantine- Frzantine-RRbuut Federated Learning. In USENIX, 2020.
[12] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02533 2015.
[13] Junyuan Hong, Yi Zeng. Shuyang Yu, Lingjuan Lyu, Ruoxi Jia, and Jiayu Zhou. Revisiting data- free knowledge distillation with poisoned teachers. In International Conference on Machine Learning, pages 13 1319--13212. PMLR, 2023.
[14] Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Measuring the effects of non-identical data distribution for federated visual classification. arXiv preprint
2019.
[15] Peter Kairouz, H Brendan McMahan, Brendan Avent, et al. Advances and open problems in federated learning, arXiv:1912.04999 04727, 2019.
[16] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International Conference Machine Learning, pages 5132-5144 5133, PMLR, 2020.
[17] Momin Ahmad Khan, Virat Shejwalkar, Amir Houmansadr, and Fatima M Anwar. On the pitfalls security evaluation of robust federated learning. In 2023 IEEE Security and Privacy Workshops (SPW), pages 57-68. IEEE, 2023.
[18] Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
[19] Souvik Kundu, Qirui Sun, Yao Fu, Massoud Pedram, and Peter Beerel. Analyzing the confi- dentiality of undistillable teachers knowledge distillation. Advances in Neural Information Processing Systems, 34:9181 14:9181-9192. 2021.
[20] Yann LeCun and Corinna Cortes. The mnist database of handwritten digits. http://yann. lecun. com/exdb/mnist// 1998.
[21] Gihun Lee, Minchan Jeong, Yongjin Shin, Sangmin Bae, and Se- Se-Young Yun. Preservation of the global knowledge by not-true distillation in federated learning. Advances in Neurul Information Processing Systems, 01::38661384777 2022.

---

[22]
Lee-Gihun. Fedntd.
[23] Daliang Li and Junpu Wang. Fedmd: Heterogenous federated learning via model distillation. arXiv preprint urXiv:1990.0330 191000958, 2019.
[24] Liping Li, Wei Xu, Tianyi Chen, Georgios Giannakis, and Qing Ling. RSA: Byzantine-robuse stochastic aggregation methods for distributed learning from heterogeneous datasets. In AAAI, 2019.
[25] Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. Federated learning on non-id data silos: An experimental study. In 2022 IEEE 38th international conference on data engineering IEEE, 2022.
(ICDE), pages
[26] Qinbin Li, Bingsheng He, and Dawn Song. Model-contrastive federated learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10713-10722, 2021.
[27] Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. Ditto: Fair and robust federated learning through personalization. In ICMI,, 2021.
[28] Tian Li. Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems, 2:429-450, 2020.
[29] Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang. and Zhihua Zhang. On the convergence of fedavg on non-iid data. arXiv preprint arXiv:/907.02/1 /89 2019.
[30] Tao Lin, Lingjing Kong, Sebastian U Stich, and Martin Jaggi. Ensemble distillation for robust model fusion in federated learning. In NeurIPS, 2020.
[31] Haoyu Ma, Tianlong Chen, Ting-Kuei Hu, Chenyu You, Xiaohui Xie, and Zhangyang Wang. Undistillable: Making a nasty teacher that cannot teach students. arXiv preprint urXiv:2105.07381, 2021.
[32] Saced Mahloujifar, Mohammad Mahmoody, and Ameer Mohammed. Universal multi palti-partt poisoning attacks. In ICMI., 2019.
[33] H Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera Arcas. Communication--ffici
learning of deep networks from decentralized data. In AISTATS, 2017.
[34] EI Mahdi EI Mhamdi, Rachid Guerraoui, and Sébastien Rouault. The Hidden Vulnerability of Distributed Learning in Byzantium. In ICML,, 2018
[35] Thomas Minka. Estimating Dirichlet distribution, 2000.
[36] Luis Muñoz-Gonzzler Battista Biggio, Ambra Demontis, Andrea Paudice, Vasin Wongras- samee, Emil C Lupu, and Fabio Roli. Towards poisoning of deep learning algorithms with back-gradient optimization. AlSec, 2017.
[37] PyTorch Documentation. https://pytorch.orgg 2019.
[38]
QinbinLi. Moon. /tttps///ithuii httppp//gghhaaaccc//////aa//////
[39]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning. pages 9748-8763 PMLR, 2021.
[40] Sashank J Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečný, Sanjiv Kumar, and Hugh Brendan McMahan. Adaptive Federated Optimization. In ICLR, 2020.
[41] V. Shejwalkar, A. Houmansadr, P. Kairouz, and D. Ramage. Back to the drawing board: A critical evaluation of poisoning attacks on production federated learning. In 2022 2022 IEEE Symposium on Security and Privacy (SP) (SP), pages 1117-1134, Los Alamitos, CA, USA, may 2022. IEEE Computer Society.
[42] Virat Shejwalkar and Amir Houmansadr. Manipulating the Byzantine: Optimizing Model Poisoning Attacks and Defenses for Federated Learning. In NDSS, 2021.
[43] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne, Journal of machine learning research, 9(11), 2008.
12

---

[44] Utilization of FATE in Risk and Micro Enterprises.
Management of Credit in Small https://wwe /ww fedai case/aass//
utilization-of of fate- in- www.ssi-aaaaggeentoo---oooo 2019.
in -
[45] Cong Xie, Oluwasanmi Koyejo, and Indranil Gupta. Generalized byzantine-tolerant sgd. 11X1::1802.10116, 2018.
[46] Cong Xie, Sanmi Koyejo, and Indramil Gupta. Fall empires: Breaking Byzantine-tolerant SGD by inner product manipulation. orXiv:1903.03336, 2019.
[47] Yueqi Xie, Weizhong Zhang, Renjie Pi, Fangzhao Wu, Qifeng Chen, Xing Xie, and Sunghun Kim. Robust federated learning against both data heterogeneity and poisoning attack via aggregation optimization. arXiv preprint, 2022.
[48] Dong Yin, Yudong Chen, Kannan Ramchandran, and Peter Bartlett. Byzantine-robus' distributed learning: Towards optimal statistical rates. ICML, 2018
[49] Lin Zhang. Li Shen, Liang Ding, Dacheng Tao, and Ling- Yu Duan. Fine-tuning global model via data-free knowledge distillation for non-iid federated learning. Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 10174-10183, 2022.
[50] Linfeng Zhang, Jiebo Song, Anni Gao, Jingwei Chen, Chenglong Bao, and Kaisheng Ma. Be your own teacher: Improve the performance convolutional neural networks via self distillation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3713-3722, 2019.
[51] Zaixi Zhang, Xiaoyu Cao, Jinyuan Jia, and Neil Zhenqiang Gong. Fldetector: Defending federated learning against model poisoning attacks via detecting malicious clients. In Proceed- ings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2545-2555, 2022.
[52] Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated learning with non-iid data. arXiv preprint wrxiv:1006000582. 2018
[53] Yanlin Zhou, George Pu, Xiyao Ma, Xiaolin Li, and Dapeng Wu. Distilled one-shot federated learning. arXiv preprint arXiv:2009.0P9 09.999,,...99............. 2020.
[54] Zhuangdi Zhu, Junyuan Hong, and Jiayu Zhou. Data-free knowledge distillation for heteroge- neous federated learning. In International conference on machine learning, pages 18889, PMLR, 2021.
13

---

Appendix
We provide additional information for our paper, HYDRA-FL: Hybrid Knowledge Distillation for Robust and Accurate Federated Learning, in the following order:
Limitations and Future Work (Appendix A
Terminology/Techniques (Appendix B
Adversarial Settings (Appendix
Experimental Setup (Appendix D
Additional Results (Appendix -
Comparison with Standard KD (Appendix
Adaptability to Any KD Algorithm (Appendix
G
Other Related Works (Appendix(H)
A Limitations and Future Work
Federated Learning can have very diverse setups, especially FL in an adversarial setting. We can have many setup combinations as can choose between different aggregation rules, attacks, defenses, datasets, data modalities, data distribution types, data heterogeneity levels, number of clients, etc. Therefore, evaluating against all combinations of these settings is well beyond the scope of one paper. Hence, for this paper, we chose only a few combinations of FL settings and tried our best to show that the problem we identified using two representative FL techniques will also exist in similar techniques. Similarly, we laid out our solution as general framework to achieve good performance under high heterogeneity and model poisoning simultaneously. To show generalizability. we tailored it to our two representative techniques, but it would be interesting to see how our solution adapts to and performs with other FL. techniques in future works. Also, we have only used unimodal, i.e., image datasets for our evaluations. This was done to stay consistent with the implementations of the techniques chosen for our case study, FedNTD and MOON. However, the language modality is becoming popular now, and multimodal models such CLIP (39] are being widely used they achieve superior performance by combining both image and language modalities. We hope to incorporate language and multimodal models in our future works.
B TerminoloyyTeechiqqee
B.1 FedNTD
FedNTD 21 is a KD-based technique that tackles the problem of data heterogeneity in FL. They first demonstrate that Data Heterogeneity causes local models forget out-distribution knowledge, i.e. the data samples not part of the client's 's local data. Therefore, to preserve the out-distribution knowledge, they introduce not-true distillation, which basically modifies the loss function for the client model's 's local objective. FedNTD's loss function is given by:
3
L LC 3) LK (ge. G+)
(13)
Here is the target label, Ye is the client model's output, and ÿe the client model's and the server model's not-true logits, respectively.
B.2 MOON
MOON 26 also aims to solve the problem of data heterogeneity in FL. They do so by reducing the distance between the representation learned by the local model with that of the global model. MOON's loss function is given by:
LCE y) + P1
Lcon(Ze 2x)
(14)
b
Here y is the target label, ye is the client's output, 2, the representation from the client's final layer, 2g is the representation from the server's final layer, and ye is the server model's output.
14

---

B.3 Shallow Layer and Shallow Distillation
Shallow layer. in a neural network refers to one of the early layers close to the input, as opposed to deeper layers that closer to the output. In the context of deep learning model, shallow layers generally capture low-level features, such as edges in images or simple patterns in data, while deeper layers capture more complex, abstract representations.
Shallow distillation. is technique used in KD where the knowledge transfer happens at a shallow layer of the neural network rather than at the final output layer. In traditional KD, the student model tries to mimic the teacher model's output at the final layer. In shallow distillation, an additional distillation loss is applied at one of the shallow layers of the student model. This helps the student model learn intermediate representations from the teacher, providing a more comprehensive learning experience. By aligning these intermediate representations, the student model gains more robust understanding of the data, leading to better generalization.
Robustness against poisoning. Shallow layers are less affected by adversarial attacks that target the final output of the model. Applying distillation at a shallow layer reduces the impact of a poisoned global model because the knowledge transferred is more fundamental and less influenced by the adversarial manipulations that typically affect the deeper layers.
C Adversarial Settings
Here we present the details of the adversarial settings of our experiments. We explain our threat model, which attacks we are using and why we are using them, and the defense we are using.
C.1
Threat Model
Goal: Our untargeted poisoning adversary controls m out of N clients to manipulate the global model to misclassify all the inputs can during testing. Unless stated otherwise, we assume 20% malicious clients. Most defense works assume high percentages of malicious clients to demonstrate that their defenses work even in highly adversarial settings. Hence, although unreasonable practical FL settings 41, we follow prior defense works and use 20% malicious clients.
Following most of the defense works, assume that the adversary knows the robust AGR that the server uses. As assumed by most works, the adversary knows the server's AGR. To test the efficacy of our technique with strong adversary, we consider the case where the adversary has access to not only the malicious clients' data but also the benign clients' data. This enables us to determine the upper bound of the efficacy of our technique.
Knowledge:
Our adversary is strong enough to directly manipulate model updates of the malicious clients it controls. While poisoning attacks come in various types and flavors, restrict ourselves to only model poisoning attacks. This is because model poisoning attacks are much stronger. It has been shown in [41 that model poisoning attacks are much stronger because they directly perturb the local model parameters. In contrast, data poisoning attacks perturb the data, subsequently perturbing the local and global models upon aggregation. Poisoning attacks can also be classified based on their error specificity. If the goal is to misclassify certain classes only, then it is a targeted attack and is often achieved by inserting a backdoor in the model that activates only for certain inputs. On the other hand, an untargeted attack indiscriminately lowers the accuracy for all inputs.
Capabilities:
C.2 Attacks we use in our evaluation
We use two model poisoning attacks for our evaluations. By testing which attack worked well, we chose the Stat-Opt attack for MOON and the Dyn-Opt attack for FedNTD. Below, we briefly explain how they work:
Stat-Opt [11] gives an untargeted model poisoning framework and tailors it to specific
defenes such as TrMean (88, Median [48), and Krum (6) The adversary first calculates the mean of the benign updates, Vb, and finds the static malicious direction = -sign(V") (V*). It directs the benign average along the calculated direction and scales with 7 to obtain the final poisoned update, -yw.
15

---

also gives an untargeted model poisoning framework and tailors it to specific defenses, similar to Stat-Opt but differs in the dynamic and data data-dependent nature of the perturbation. The attack first computes the mean of benign updates, V and data- dependent direction, w. The final poisoned update calculated as V V + yau, where the attack finds the largest that can bypass the AGR. They compare their attack with Stat-Opt and show that the dataset-tailored w and optimization- scaling factor y make their attack much stronger.
Dyn-Opt [42]:
C.3 Defense we use in our evaluation
We use the Trimmed Mean defense in our evaluations. Trimmed Mean [8
is a foundational defense used in advanced AGRs 5T 42 The server receives model updates from each client, sorts each input dimension j. discards the m largest and smallest values (where m indicates malicious clients), and averages the rest.
45
D Experimental Setup
For MOON, we use base encoder with two 5 x convolutional layers, each followed by a 2 x 2 max pooling layer and two fully connected layers with ReLU activation. The base encoder is followed by a projection head with an output dimension of 256. For FedNTD, we use a model (similar to the one in (33) having two convolutional layers followed by linear layer and classification layer. For FedNTD, we test with different values and settle upon a diminishing factor = 1 and 7=2. For MOON, we set /3 0 and set 7 1. We used PyTorch (37] for implementation on 8GB NVIDIA RTX 3060 Ti GPU. Each rum FedNTD and MOON took about 2-3 hours on our machine.
Models:
FL Settings: For FedNTD, we use 100 clients with sampling ratio of 0.1, i.e., 10 clients are selected every round. We use momentum SGD with an initial learning rate of 0. 1, weight decay of 1 1 e batch size of 50, and momentum of 0.9. Each run consists of 200 rounds with 5 local epochs. For MOON, we use 10 clients with a sampling ratio of We use SGD with an initial learning rate of 0.01, weight decay of 1 e batch size of 64, and momentum of 0.9. Each run consists of 30 rounds with 10 local epochs, sufficient for convergence.
Data Partitioning: We use the widely used Dirichlet 35| distribution to generate the non-lID partitioning of data between clients. Dirichlet distribution works by sampling Pk Dir (a) and assigns Pk.j proportion of samples of class & to client j. A lower value of corresponds to a higher level of heterogeneity since means that most Conversely, at a higher value of a, the class samples are more evenly distributed between the clients. Also, a characteristic of the Dirichlet distribution that both local dataset size and local per-class distribution vary across clients.
samples of certain class belong to one client.
Datasets: The three datasets we use in our experiments are:
MNIST is 10-class digit image classification dataset, which contains 70,000 grayscale images of size 28 x 28. We divide all data among FL clients (100 for FedNTD and 10 for MOON) using the Dirichlet (40 distribution.
MNIST 20)::
CIFAR10 [88] CIFARIO is 10-class classification task with 60,000 total RGB images, each of size 32 x 32. Each class has 6000 training images and 1000 testing images. We divide all the data among 100 clients using the Dirichlet distribution, popular synthetic strategy to generate FL datasets.
CIFAR100 18] CIFARIO0 is similar CIFAR10. 10, except that it is 100-class classifica- tion task where each class has 600 images of size 32 x 32. There are 500 training images and 100 test images per class. Like other datasets, we also partition this dataset using the Dirichlet distribution.
E Additional Results
In this section, we present some of the additional results we have obtained.
16

---

Table 4: FedNTD
Datuset
MNIST
CIFARTO
CIFARI00
0,05
0.1
0,3
0.5
Techniques
No atiock
atnack
no attack
attiack
Ro attuck
atteck
no atrack
attack
no atiank
attock
no attack
arrock
Fedavg
92.12
74.48
44.89
31,27
54.67
35,67
66.34
42.53
70.57
48.27
26.17
12.92
MOON
93003
58.09
16.94
21.72
56.95
32.61
46.72
71.79
32.51
29.1
13.92
Ows
9269
76.67
46.92
25.15
57.12
34.25
68.1
47.03
71.22
52.37
28.9
14.33
E.1 FedNTD
For visual symmetry, we did not include the full table in $ but we had also run our FedNTD experiments a 0.3. We show the full FedNTD results Table 4 Here, we can see that at at 0.3 too, we achieve superior results FedAvg and FedNTD in both benign and adversarial conditions.
E.2 MOON
We also ran ablation with MNIST for different shallow layers and diminishing coefficients. We show the results in T where we can see that at lower effective μ (g), i.e. higher diminishing factor, we achieve the best results. A lower μ does give us better no-attack accuracy, but we lose lot in the attack scenario.
Method
no-attack
attack
HYDRA-FLSS sl
1
94.41
68.68
HYDRA-FL. s2
I
91.78
68.13
HYDRA-FL sl
0.3
92.03
72.35
HYDRA-FL. s2
0.3
92.92
73.55
HYDRA-FL s1
0.1
92.04
76.65
HYDRA-FL s2
0.1
93.93
72.54
Table 5: Comparison of HYDRA-FL for MOON with different distillation coefficients.
F
Comparison With Standard KD
Dissimilarities: We elaborate the dissimilarities first since they relate to the practicality of the attack amplification phenomenon. The threat model differs when we compare standard KD KD in FL. In standard KD, there only one student and one teacher. An attacker might poison the teacher with a backdoor trigger that causes misclassification on specific input samples. the FL case, the teacher (server) model itself is not directly attacked. Instead, some of the client models which indirectly poisons the global model because it is obtained by aggregating all the client models. Therefore, in the FL case, the attacker is poisoning some of the student models, which in turn poison the server model upon aggregation, and then propagates the poison to the non-malicious/henign clients when they use KD their local training step.
attacked,
We believe that our mitigation technique. originally designed for FL, can also be effectively applied in non-FL settings. The premise of technique, HYDRA-FL. is to reduce the distillation at the final layer and relegate some of it to a lower layer, thereby reducing the impact of distillation. To illustrate, imagine the poisoned distillation loss at the final layer as noise being injected into the model. By adjusting the distillation process (Section 4.1), this noise relegated to lower layers through auxiliary classifier, thereby mitigating the impact of such attacks. This dilution of adversarial effects can enhance robustness in both FL and non-FL settings. This is an interesting comment by the reviewer. It can be potential future direction rigorously experiment find similarities and dissimilarities between KD in non-FL. and FL settings.
Similarities:
G Adaptability to any KD Algorithm
Here we explain how HYDRA-FL. can be applied to any KD algorithm general. KD-based FL techniques align local and server models to reduce deviation caused by data heterogeneity. As
17

---

stated in Section 3, MOON uses model-contrastive learning for alignment, while FedNTD uses KL-divergence. Thus, any such KD based FL algorithm performs alignment (like tuning an engine). Its equation is:
L= LoB(Vee v) + (o)))+ooliin @oolignment Us)
(15)
Here, Lalignment represents the alignment technique. & represents the strength of this alignment.
In this generic setting, HYDRAFL would be modification of the above equation. We will diminish with "b.. We add new loss term that performs this alignment shallow layer with an auxiliary classifier. HYDRAFL's equation becomes:
C = CoE(Ve.))) v) + Catignment((or Ue) + CfCogggmmen (yarce (Hawas Us)
(16)
H Other Related Works
H.1
Tao et al. 33
Tao et al. 30 was primarily designed address model heterogeneity in addition data heterogeneity. Hence, they came up with model fusion on the server side using knowledge distillation with auxiliary data. Briefly, this fusion step involves initializing student server model by aggregating the client models of different heterogeneous groups and then using auxiliary dataset with each group to distill information into the server model. Our focus on uncovering issues arising from data heterogeneity, which makes it difficult for fusion models to determine whether attack amplification is caused by data or model heterogeneity. However, exploring how model heterogeneity impacts attack success an intriguing direction for future research.
H.2 Junyuan et al. 13
Similarities:
Junyuan et al. 13 is also mitigation technique against poisoned teachers. They also try to find out if poison is passed from the teacher to the student through the distillation process. To prevent this passing of poison, they have designed an anti-backdoor technique. Their technique also works by suppressing the poison and mitigating the performance degradation caused by bad supervision. They name these two stages of their defense mechanism as Shuffling Vaccine (SV) and Self-Retrospection (SR).
Dissimilarities:
However, there are two key differences.
Their technique targets backdoor attacks where a trigger causes performance degradation for specific subset of samples. They use a "shuffling vaccine" to detect suspicious samples by shuffling channels. In contrast, the final layer using a diminishing factor. We address model poisoning, which is much stronger than backdoors since it aims indiscriminately lower the accuracy of all samples. Therefore, our intuition is that their technique might not work against untargeted model poisoning as it is explicitly designed for backdoors. To mitigate performance degradation from backdoored teacher, they use self-retrose relfsseeronnncttonn by synthesizing potential backdoor knowledge learned and confronting it during training. Conversely, we mitigate this by directing the distillation loss through auxiliary classifier and scaling down the loss at the final classification layer.
reduce the impact of poison by scaling down the distillation loss at
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes, have ensured that the main claims in the abstract and introduction accurately reflect the paper's 's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes].
Justification: Yes, have discussed the limitations and future work in Appendix A
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA].
19

---

There is no theoretical result in this paper that requires full set of assumptions and correct proof.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Yes, we have fully disclosed the information needed to reproduce the main experimental results of the paper. They are written in Section 5land Appendix D
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
20

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes].
The code is available at https: www.:mccmmooooommo ahmad khan/ HYDRA-FL
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to rum to reproduce the results. See the NeurIPS code and data submission guidelines (https
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/eetails
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes].
Justification:
We specify the training and test details in Appendix D
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No].
Justification: We did not have enough compute resources to completely re-run all the experi- ments for different seeds and report error bars for different runs. We ran our experiments on different validation sets verify that our technique generalizes well and is not overfitting over one particular data distribution.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
21

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes].
Justification:
We present these details in Appendix D
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]..
Justification: Yes, to the best our knowledge, our paper conforms to the NeurIPS Code of Ethics in every aspect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA].
Justification: Or work does not have such societal impact that requires discussion in the paper.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
22

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]-
Justification: To the best of our knowledge, our paper poses no such risks. We use publicly available code and data for our work.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes].
Justification: We have cited all three datasets; MNIST 20, CIFAR10 18 and CI-
FAR100 18]. Their licenses are not mentioned on paperswithcode.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
23

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
If assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, papprsaipphaaeoocccaaappp thcode. com/ com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes].
The code is available at http:://gittuu..commmooiiit- ahmad-khan/ HYDRA-FL
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA].
Justification: Our paper does not involve any crowdsourcing experiments research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA].
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
24

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "r7CMXsnYLs", "reviewer": "Reviewer_WZZr", "summary": "This work addresses the challenge of data heterogeneity in Federated Learning (FL) and its impact on global model performance.\nThe authors demonstrate why KD is susceptible to the issue of poisoning attacks and use these findings as a foundation to propose a novel method HYDRA-FL. Experimental results demonstrate that HYDRA-FL not only enhances resistance to attacks but also maintains its performance in benign settings.", "questions": "Please refer to the weaknesses mentioned above.", "limitations": "The authors have discussed the limitations of the work in the Appendix.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The authors have studied why Knowledge Distillation (KD) amplifies model poisoning and propose their method based on these findings.\n- The authors implemented ablation experiments to prove the effectiveness of the proposed method.\n- The authors will release the codes.", "weaknesses": "- The performance advantage is sometimes unstable, such as when alpha=5 in Table 2. The authors could provide a more detailed analysis to explain this variability.\n- The authors should explain how their findings regarding the amplification of attacks through knowledge distillation are applicable to both standard knowledge distillation learning and KD-based FL, emphasizing the differences and similarities in these scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1: Unstable performance advantage:**\nWe thank the reviewer for giving us the opportunity to provide further explanation of this variability.\n\nWe start off by saying that we see stronger accuracy gains in high-heterogeneity settings (low alpha), as we show in Tables 1 and 2 (Page 8). The reason behind this relationship between heterogeneity and accuracy gain is that, at high heterogeneity, there is more ``room'' for gain. We explain it with a simple analogy. Think of a high heterogeneity setting as a car engine that is out of tune. Therefore, there is an opportunity to make significant improvements by tuning it. Conversely, a low-heterogeneity setting is like an already-tuned or slightly out-of-tune engine where corrective adjustments lead to smaller gains.\n\nThis analogy highlights the idea that there is more disparity or \"distance\" from the optimal global model in high heterogeneity settings, allowing methods like FedNTD or MOON to achieve more noticeable improvements. In contrast, the potential for noticeable improvements is reduced when the system is already closely aligned with the global model (low heterogeneity).\nApplying FedNTD or MOON brings them closer to the global model, and if we think of these updates as vectors (as shown in our Figure 1), then the ``angle'' of improvement will be high. On the other hand, in a low heterogeneity setting, where model updates are already close to the global model, there is little improvement in the final accuracy when we apply FedNTD or MOON, i.e., the angle of improvement is low. We can also see this numerically in Tables 1\\&2 in Sections 5.1\\&5.2. To clarify our point, we calculate this accuracy gain for each heterogeneity level for CIFAR10 and show it in the following two tables. These tables show that the accuracy gain is stronger at low heterogeneity.\n\n*HYDRAFL in the no-attack setting (Table 1, Page 8)*\n| Heterogeneity | Fedavg | HYDRA-FL | Gain |\n|---------------|--------|----------|------|\n| 0.05          | 44.69  | 46.92    | 2.23 |\n| 0.1           | 54.67  | 57.12    | 2.45 |\n| 0.3           | 66.34  | 68.1     | 1.76 |\n| 0.5           | 70.57  | 71.22    | 0.65 |\n\n*HYDRAFL in the attack setting (Table 1, Page 8)*\n| Heterogeneity | FedNTD| HYDRA-FL | Gain |\n|---------------|--------|----------|------|\n| 0.05          | 21.72  | 25.15   |3.43 |\n| 0.1           | 32.61  | 34.25    |1.64 |\n| 0.3           | 46.72| 47.03     | 0.31 |\n| 0.5           | 52.51  | 52.57    |0.06 |\n\n*HYDRAFL in the no-attack setting (Table 2, Page 8)*\n| Heterogeneity | Fedavg| HYDRA-FL | Gain |\n|---------------|-------|----------|------|\n| 0.1           | 57.76 | 60.1     | 2.34 |\n| 0.5           | 63.14 | 63.32    | 0.18 |\n| 5             | 71.19 | 70.55    | -0.64 |\n\n*HYDRAFL in the attack setting (Table 2, Page 8)*\n| Heterogeneity | MOON| HYDRA-FL | Gain |\n|---------------|-------|----------|------|\n| 0.1           |39.9 |43.6 |3.7 |\n| 0.5           |57.17 |59.93 |2.76 |\n| 5             |67 |68.4 | 1.4|\n\n\nIn addition, we want to highlight that in the attack setting, HYDRA-FL can outperform both FedAvg and MOON at $\\alpha = 5$. If we look closely, the accuracy drop (i.e., the difference between no-attack and attack accuracies) for FedAvg is 2.81, 3.95 for MOON, and 2.15 for HYDRA-FL. Thus, we achieve the lowest accuracy drop in this setting.\n\n\n**W2: Comparison with standard KD:**\nThe reviewer has raised a very important point about the similarities and dissimilarities between standard KD and FL KD regarding attack amplification. We provide some insights into this matter and plan to incorporate this comparison in the final version of our paper. We will add this in the appendix with a dedicated section titled ``Comparison with Standard KD''.\n\n**Dissimilarities:** We elaborate on the dissimilarities first since they relate to the practicality of the attack amplification phenomenon. The threat model differs when we compare standard KD to KD in FL. In standard KD, there is only one student and one teacher. An attacker might poison the teacher with a backdoor trigger that causes misclassification on specific input samples. In the FL case, the teacher (server) model itself is not directly attacked. Instead, some of the client models are attacked, which indirectly poisons the global model because it is obtained by aggregating all the client models. Therefore, in the FL case, the attacker is poisoning some of the student models, which in turn poison the server model upon aggregation, and then propagates the poison to the non-malicious/benign clients when they use KD in their local training step.\n\n**Similarities:** We believe that our mitigation technique, originally designed for FL, can also be effectively applied in non-FL settings.  The premise of our technique, HYDRA-FL, is to reduce the distillation at the final layer and relegate some of it to a lower layer, thereby reducing the impact of distillation.\nTo illustrate, imagine the poisoned distillation loss at the final layer as noise being injected into the model. By adjusting the distillation process (Section 4.1), this noise is relegated to lower layers through an auxiliary classifier, thereby mitigating the impact of such attacks. This *dilution* of adversarial effects can enhance robustness in both FL and non-FL settings.\nThis is an interesting comment by the reviewer. It can be a potential future direction to rigorously experiment to find similarities and dissimilarities between KD in non-FL and FL settings.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kkbpW7wOdB", "reviewer": "Reviewer_udjQ", "summary": "This paper first empirically demonstrates the fact that KD algorithms amplify attack effectiveness. Then the authors propose HYDRA-FL as a method for mitigating the attack amplification problem, through a novel loss function template that can be applied to any FL algorithm wherein the local training objective can be adapted.", "questions": "Do you have any hypotheses about how HYDRA-FL would work in general across any type of KD algorithm? It would be nice if you have some justification for why your approach could work more generally and not just with FedNTD and MOON.", "limitations": "Limitations are addressed in the appendix.", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "Very well written. The paper flows nicely and is well organized, and the explanations provided are very clear and thorough. The background concepts are explained well so I think this paper is accessible to a large audience (not necessarily limited to people who have background knowledge in FL). The method is reproducible because the paper explains how exactly they modified publicly-available codes for their implementation. Very extensive empirical testing, including a study that motivates the work and a few ablation studies. The results are presented in very clear figures in addition to being well-explained in the text (including numerical values, qualitative statements, and conclusory statements that summarize the takeaways from each set of results). The proposed algorithm is definitely novel. It is taking (to the best of my knowledge) a completely new approach that therefore has high potential to inspire future work. The paper also does a good job of introducing and justifying the attack amplification problem in a way that could inspire future work.", "weaknesses": "This does not seem like a very practical solution for real world settings since it causes a drop in benign accuracy on such simple datasets. I would think that the accuracy sacrifice could potentially be even more significant with larger, more complex models/task. It is not made clear why the accuracy drop is worth the avoidance of attack amplification. It would be helpful if there was some examples/explanation for how this approach is useful in real world settings. For instance, how can attack amplification be problematic in a real world setting and how would HYDRA-FL solve the issue?\n\nIt is nice that there is some theoretical justification for the attack amplification. The paper could be even stringer if (perhaps in the appendix) you could somehow use this justification to relate to some sort of theoretical justification for the effectiveness of HYDRA-FL. Otherwise, with just the empirical results from a limited array of experimental setups, the claims of HYDRA-FL’s effectiveness are not very sound.\n\nIt seems like your paper is missing some valuable information, for example, I spent some time trying to understand the threat model, but then I found it in the appendix. So it would be helpful if the main text of your paper referenced the appendix more so that the reader knows what they can go to the appendix to find. The limitations and future work are also easy to miss when just put in the appendix, so I would suggest referencing that appendix in your conclusion in the main text.\n\nThe related work section isn’t really explaining how your work relates to prior work. It would be nice to have more explanation about prior solutions and what their shortcomings are and how your proposed solution overcomes those shortcomings. This is explained more in the introduction and also later in the paper (e.g. in sec. 4.1), which is helpful, but then reading the related work section it is hard to understand where your work is fitting in. So adding at least a sentence for 2.1.1 or 2.1.2, for example, could be helpful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1- Real-world settings:**\n\n*Real-world use cases and task complexity:*\n\nWe agree that real-world dataset analysis would be valuable. While our current choice of datasets allows for direct comparison with previous works, we plan to evaluate more complex models and tasks, such as language and multimodal models, in future studies, as discussed in Appendix A.\n\nDespite limitations, our results on CIFAR-100, which is more complex (evident by low no-attack accuracy) than MNIST and CIFAR-10, show HYDRAFL's robustness. HYDRA-FL outperforms FedAvg in both no-attack and attack scenarios and surpasses FedNTD and MOON in attack scenarios.\n\nAttack amplification can severely impact applications with diverse data in real-world scenarios such as healthcare or large-scale IoT networks. HYDRA-FL mitigates these issues by distributing the distillation process across layers (Section 4.1), which prevents attacks from focusing on a single layer.\n\n\n*Justification on attack amplification avoidance:*\n\nWe acknowledge the need to explain how our approach avoids attack amplification and will address this in the final version. Tables 1\\&2 (page 8) show a slight decrease in benign accuracy for HYDRA-FL compared to FedNTD/MOON in some cases. However, HYDRA-FL generally achieves higher benign accuracy. More importantly, HYDRA-FL demonstrates significantly higher accuracy in the presence of attacks than FedNTD/MOON, resulting in a substantial overall performance improvement. We compute accuracy drops (no-attack minus attack accuracies) for FedNTD/MOON vs HYDRAFL. The following tables consistently show that HYDRA-FL experiences a smaller accuracy drop, highlighting its effectiveness.\n\n|          | FedNTD | HYDRAFL |\n|-------------|--------|---------|\n| MNIST       | 34.94  | 16.02   |\n| CIFAR10 0.05| 25.22  | 21.77   |\n| CIFAR10 0.1 | 24.34  | 22.87   |\n| CIFAR10 0.5 | 19.28  | 18.65   |\n| CIFAR100    | 15.18  | 14.57   |\n\n|            | MOON  | HYDRAFL |\n|------------|-------|---------|\n| MNIST      | 18.81 | 15.39   |\n| CIFAR10 0.1| 18.9  | 16.5    |\n| CIFAR10 0.5| 6.17  | 3.39    |\n| CIFAR10 5  | 3.95  | 2.15    |\n| CIFAR100   | 5.53  | 4.3     |\n\nThus, avoiding attack amplification is worth the *slight* drop in benign accuracy. In the real world, enhanced security against attacks justifies this tradeoff, particularly in critical applications, such as autonomous driving, healthcare applications, or financial fraud detection.\n\n**W2- Theoretical justification:**\nThe reviewer has raised an important point about the theoretical justification for HYDRA-FL.\n\nThe generic loss function of an FL client using KD is:\n\n$\\mathcal{L}$ = $\\mathcal{L}\\_{CE}(\\hat{y}\\_{c}, y) + \\beta \\mathcal{L}\\_{KD}({y}\\_{c}, y\\_{s})$\n\nIn Section 3, we showed that when malicious clients are present, $\\frac{d\\mathcal{L}}{d\\beta} > 0$, i.e., KD causes the loss to increase.\n\nIn Section 4, we formulated the generic loss function of an FL client using KD in HYDRA-FL as:\n\n$\\mathcal{L'} = \\mathcal{L}\\_{CE}(y\\_{c}, y) + \\frac{\\beta}{b} \\mathcal{L}\\_{KD}(y\\_{c}, y\\_{s})+ \\gamma  \\mathcal{L}\\_{KD}(y\\_{aux}, y\\_{s})$\n\nWe use $\\mathcal{L'}$ for HYDRA-FL loss and $\\mathcal{L}$ for basic KD loss. Comparing them, since $\\frac{\\beta}{b} < \\beta$, it follows that $\\frac{\\beta}{b} \\mathcal{L}\\_{KD}(y\\_{c}, y\\_{s}) < \\beta \\mathcal{L}\\_{KD}({y}\\_{c}, y\\_{s})$.\nThe entire model is $\\theta$, and the part is up til the auxiliary layer $\\theta'$. The impact of shallow distillation loss is only on $\\theta'$ and not the rest of the model $\\theta - \\theta'$. Therefore, with appropriate $b$ and $\\gamma$, $\\frac{d\\mathcal{L'}}{d\\beta} < \\frac{d\\mathcal{L}}{d\\beta}$. This is shown in Figure 7, where we select $b=[1,4]$ (so $\\beta$ effectively becomes $1$ and $\\frac{1}{4}$, respectively) and $\\gamma=2$ to show the effectiveness of our technique and the impact of the variation of these hyperparameters.\n\n**W3- Threat model, limitations, and future works referencing:**\n\nThank you for pointing this out. We will update our paper and refer to the threat model (Appendix C.1) and attacks+defenses used (Appendix C.2) in the Introduction (Section 1) and Experimental Settings (Section 5.1).\n\n**W4- Where does HYDRA-FL fit in related work?**\n\nWe recognize the need to clarify how our work fits within this field. We will update Section 2.1.1. Unlike MOON and FedNTD, which address data heterogeneity only in no-attack settings, HYDRAFL operates in both no-attack and attack settings, achieving robustness and accuracy under high heterogeneity. Section 2.1.2 will explain that our paper shows how KD-based data-heterogeneity mitigation techniques amplify poisoning attacks and that HYDRAFL addresses both data heterogeneity and poisoning.\n\n\n**Q1- How would HYDRA-FL work with any other KD algorithm?**\n\nThe reviewer has raised an important point about HYDRAFL's generalizability. We will incorporate this explanation of HYDRAFL's application to any KD FL algorithm in the appendix of our final version.\n\nKD-based FL techniques align local and server models to reduce deviation caused by data heterogeneity. As stated in Section 3, MOON uses model-contrastive learning for alignment, while FedNTD uses KL-divergence. Any KD-based FL algorithm achieves this alignment, akin to fine-tuning an engine, with the following equation:\n\n$ \\mathcal{L} = \\mathcal{L}\\_{CE}(y\\_{c}, y) + \\phi \\mathcal{L}\\_{alignment}(y\\_{c}, y\\_{s})$\n\nHere, $\\mathcal{L}\\_{alignment}$ represents the alignment technique. $\\phi$ represents the strength of this alignment.\n\nIn this generic setting, HYDRAFL modifies the above equation by diminishing $\\phi$ with '$b$'. We add a new loss term that performs this alignment at a shallow layer with an auxiliary classifier. HYDRAFL's equation becomes:\n\n$\\mathcal{L} = \\mathcal{L}\\_{CE}(y\\_{c}, y) + \\frac{\\phi}{b} \\mathcal{L}\\_{alignment}(y\\_{c}, y\\_{s})+ \\eta  \\mathcal{L}\\_{alignment}(y\\_{aux}, y\\_{s})$\n\nThis enables HYDRA-FL to adjust the alignment process for any KD algorithm.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "scBSpzv7G9", "reviewer": "Reviewer_SYjG", "summary": "This paper investigated the phenomenon termed *attack amplification* in federated learning with Knowledge Distillation (KD) and proposed an FL framework named HYDRA-FL to reduce the impact of poisoning client attacks in FL. An auxiliary classifier is introduced to employ KD loss on the shallower layer and reduce the effect of KD on the final layers of the model. Experiments on the benchmark dataset demonstrated the performance gain of the proposed method under both clean and poisoning settings compared with a few other FL methods.", "questions": "* The proposed method seems to focus on model poisoning attacks. I wondered how this algorithm would perform when the local client updates are from models with poisoned data (such as flipped labels).\n* Authors mentioned that 'completely removing the KD-loss at the output layer may cause a more negative impact than keeping it in a reduced form'. What could be the reasons/implications behind this phenomenon?", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "\\+ Showcased a crucial threat of KD, a commonly used technique in federated learning.\n\n\\+ Provided a unified solution for robust KD that is agnostic to most FL frameworks.\n\n\\+ Nice ablation study to validate the effects of different components in the proposed algorithm.", "weaknesses": "\\- Practicality: It seems that this algorithm hinges on careful choices of hyper-parameters of $\\beta$, $\\mu$, $\\gamma$. Sensitivity analysis against drastic changes in these hyper-parameters should be provided. Fig 7 is not sufficient enough.\n\n\n\\- Experiments: only two FL+KD methods were analyzed, whereas there are many other related work that employs KD in FL, such as methods that leverage proxy datasets [1].\n\n\n\\- Related work needs to be more comprehensive. For instance, there has been work that discussed KD in FL with poisoned teacher models [2]. Experimental comparison or at least a discussion of work along this line is expected.\n\n\n---\nReferences:\n\n[1] Lin, Tao, et al. \"Ensemble distillation for robust model fusion in federated learning.\" *Advances in neural information processing systems* 33 (2020): 2351-2363.\n\n[2] Hong, Junyuan, et al. \"Revisiting data-free knowledge distillation with poisoned teachers.\" *International Conference on Machine Learning*. PMLR, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1- Sensitivity analysis:**\n\nWe agree with the reviewer that sensitivity analysis is crucial. Therefore, in addition to Figure 7 (FedNTD), we explain Table 4 (MOON) in Appendix E.2 since it is an ablation over the different shallow layers, diminishing factors, and attack/no-attack settings. We clarify that in Appendix E.2 we meant to say ``lower effective value of $\\mu$'', where effective $\\mu$ is $\\frac{\\mu}{b}$ and 'b' is the diminishing factor. We chose hyperparameters over validation sets. The table below shows the impact of decreasing *effective $\\mu$* (or increasing the diminishing factor 'b'). We see that in the no-attack setting, a high effective $\\mu$ (low 'b') results in higher accuracy, as a higher diminishing factor reduces the distillation impact (Section 4.1). Conversely, in the attack setting, we see the opposite trend. This is because the diminishing factor reduces the impact of distillation on the final layer, thereby reducing attack amplification, as can be seen by the accuracy drop.\n\n| **Method**   | **$\\mu$** | **no-attack** | **attack** | **accuracy drop** |\n|--------------|-----------|---------------|------------|-------------------|\n| HYDRA-FL s1  | 1         | 94.41         | 68.68      | 25.73             |\n| HYDRA-FL s2  | 1         | 91.78         | 68.13      | 23.65             |\n| HYDRA-FL s1  | 0.3       | 92.03         | 72.35      | 19.68             |\n| HYDRA-FL s2  | 0.3       | 92.92         | 73.55      | 19.37             |\n| HYDRA-FL s1  | 0.1       | 92.04         | 76.65      | 15.39             |\n| HYDRA-FL s2  | 0.1       | 93.93         | 72.54      | 21.39            |\n\n\n**W2- Other methods:**\n\nWe thank the reviewer for mentioning other works, and we will expand on related works in the final version of the paper. [Tao et al., 2020] was primarily designed to address model heterogeneity in addition to data heterogeneity. Hence, they came up with model fusion on the server side using knowledge distillation with auxiliary data. Briefly, this fusion step involves initializing a student server model by aggregating the client models of different heterogeneous groups and then using an auxiliary dataset with each group to distill information into the server model.  Our focus is on **uncovering issues arising from data heterogeneity**, which makes it difficult for fusion models to determine whether attack amplification is caused by data or model heterogeneity. However, exploring how model heterogeneity impacts attack success is an intriguing direction for future research.\n\n\n**W3- Related Work:**\n\nWe agree that a discussion comparing our work with [Junyuan et al. 2023] is necessary, and we will include it in the appendix in the final version of our paper.\n\n*Similarities:*\n\n[Junyuan et al. 2023] is also a mitigation technique against poisoned teachers. They also try to find out if poison is passed from the teacher to the student through the distillation process. To prevent this passing of poison, they have designed an anti-backdoor technique. Their technique also works by suppressing the poison and mitigating the performance degradation caused by *bad supervision*. They name these two stages of their defense mechanism as Shuffling Vaccine (SV) and Self-Retrospection (SR). \n\n*Dissimilarities:*\n\nHowever, there are two key differences. \n\n1) Their technique targets backdoor attacks where a trigger causes performance degradation for a specific subset of samples. They use a \"shuffling vaccine\" to detect suspicious samples by shuffling channels. In contrast, we reduce the impact of poison by scaling down the distillation loss at the final layer using a *diminishing factor*. We address model poisoning, which is much stronger than backdoors since it aims to indiscriminately lower the accuracy of all samples. Therefore, our intuition is that their technique might not work against untargeted model poisoning as it is explicitly designed for backdoors.\n\n2) To mitigate performance degradation from a backdoored teacher, they use self-retrospection by synthesizing potential backdoor knowledge learned and confronting it during training. Conversely, we mitigate this by directing the distillation loss through an auxiliary classifier and scaling down the loss at the final classification layer.\n\n**Q1- Data poisoning:**\n\nWhile data poisoning in the context of knowledge distillation (KD) requires a dedicated study, we believe our solution is broadly applicable as it prevents the flow of *poison* through the entire model and *dilutes* its effect. Consider a simple example: label flipping, a basic data poisoning attack where some labels in the training dataset are flipped (e.g., changing a bird image's label to 'airplane' in CIFAR-10). Suppose the server model (teacher) is poisoned with this flipped dataset. During the distillation process, since we are using HYDRAFL, this poisoned distillation information would be diluted at the final layer. Instead, it will flow through an auxiliary classifier into the early layers of the model, thereby reducing the impact of the data poisoning attack via label flipping.\n\n**Q2 Removing KD loss:**\n\nThank you for allowing us to provide clarification on this.\nConsider a distillation setup, where the teacher model is providing guidance to the student model. If the guidance (KD-loss) is removed from the final decision-making layer, the student lacks feedback on how its final predictions compare to the expert's. This is similar to a student receiving partial tutoring: while they may benefit initially, the absence of final guidance prevents them from fully applying the partial knowledge they have accumulated, hindering their ability to achieve the best results.\nBy keeping the KD-loss in a reduced form, the student continues to receive crucial guidance throughout the network, ensuring better alignment and improved accuracy. The diminishing factor 'b' adjusts how much of this guidance is applied, allowing fine-tuning based on specific model needs.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "P0peLukTm7", "reviewer": "Reviewer_Zoyb", "summary": "This paper identifies that the KD-based method which is used to tackle the data heterogeneity becomes more vulnerable under the model poisoning attacks. Moreover, the models unknowingly align benign client models with a poisoned server model in a malicious setting. It is also called attack amplification. To address this phenomenon, HYDRA-FL is proposed by conducting KD-loss at the shallow layer via an auxiliary classifier.", "questions": "- In FL Settings, Setting a sampling ratio equal to 1 across 10 clients in MOON is not a realistic scenario.\n- Why only test $\\alpha=0.5, 0.1, 0.05$ in the CIFAR-10 dataset, Whats the performance in the other two datasets?\n- How to explain that the performance gain in a larger $\\alpha$ scenario under adversarial setting is slightly low than in smaller $\\alpha$. e.g., In Table 1, CIFAR 10 $\\alpha=0.05$FedNTD→HYDRA-FL 21.72 to 25.15, $\\alpha=0.5$FedNTD→HYDRA-FL 52.51 to 52.57. This phenomenon can also be observed in Table 2 and Figure5.", "limitations": "See weakness and questions.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Clear motivation with the case study to present the observation on “model alignment process\ninadvertently forces local models to align its representation/predictions to the poisoned global model amplifying the attack’s impact”. \n\nIt is easy to follow up and the HYDRA method is simple to implement which is very important.", "weaknesses": "- Only two methods are implemented with HYDRA can not convince me that this method is efficient enough. As these two methods are also outdated, it is better to implement this simple modification to more KD methods. \n- Depth analysis of experimental observation is needed to provide more insights and inspiration.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1- Two methods:**\n\nWe appreciate the reviewer's concern regarding our evaluation of HYDRAFL on only two existing methods and agree that this is a limitation. We chose these two methods as initial test cases to demonstrate the *attack amplification* phenomenon we discovered, to motivate the need for HYDRAFL, and then to show the adaptability of HYDRAFL on two KD-based techniques. Due to the large number of existing techniques, it was not feasible to include an exhaustive evaluation within the scope of one paper. However, we conducted a wide range of experiments on these two chosen techniques to show the effectiveness of our technique and the impact of the choice of different parameters.\n\n**W2 - Depth Analysis of Experiments:**\n\nWe agree that an in-depth analysis is needed. HYDRAFL can be tested with various experimental setups, including various aggregation rules, attacks, defenses, datasets, data modalities, data distribution types, data heterogeneity levels, number of clients, client sampling ratios, number of malicious clients, etc.\n\nWhile evaluating all possible combinations would be ideal, we, unfortunately, cannot fit everything in one paper while considering our limited computational resources. We prioritized certain setups based on feasibility and resource availability. Here, we highlight a few points regarding our experimentation:\n\n- Although MOON and FedNTD are KD-based FL techniques, they use different approaches. FedNTD penalizes prediction divergence measured through distillation loss, while MOON penalizes representation divergence measured through contrastive loss (Section 3). This shows that attack amplification is not limited to one single technique.\nWe chose two model poisoning attacks (Appendix C.2): Dyn-Opt for FedNTD and Stat-Opt for MOON. We chose two different attacks to show that attack amplification is not unique to a single attack but is rooted in the fundamental nature of KD.\n- The sampling ratios for MOON and FedNTD are 1 and 0.1, respectively. HYDRAFL performs well in both cases; therefore, it is safe to assume that it does not rely on the sampling ratio.\n- Since we cannot cover the entire space of KD algorithms, we propose our solution (Page 6, Equation 4) as a generic formulation that can be inserted over the standard KD equation (Page 3, Equation 3).\nWe have provided the well-documented code, which requires 1) modifications to the model architecture and the loss function equation and 2) applying an attack and a defense in the training code in the original paper's repository. This demonstrates the ease of implementation for any future work.\n- We perform ablations across different heterogeneity values, choice of shallow layer, and distillation coefficients. We do this for both FedNTD (Page 9, Figure 7) and for MOON (Appendix E.2, and we ran some new experiments in response to your Q2).\n\nFurther explanations for some of our experimental settings are addressed in response to the questions below.\n\n**Q1- Sampling ratio:**\n\nWe thank the reviewer for raising this point about realistic scenarios. We agree that a sampling ratio of 1 with 10 clients is not realistic for MOON. Our goal was to show attack amplification in settings where MOON/FedNTD performs well, using the original paper’s settings for fair comparison.\n\nThe FedNTD paper uses a sampling ratio of 0.1 (100 clients), while MOON selects all 10 clients. We followed the original paper's hyperparameters to effectively demonstrate the problem and solution. We will update our Limitations and Future Works Section to include large-scale analysis, especially with the language modality, and move the Limitations section to the main paper.\n\n**Q2- Heterogeneity settings in other datasets:**\n\nHYDRAFL is built upon the MOON and FedNTD, using the same hyperparameters unless otherwise specified. These specified alpha values are within the tested range of the original works, ensuring the integrity and comparability of our results. \nTo demonstrate robustness and generalizability, we conducted more experiments now on MNIST at a much lower heterogeneity of 0.05. Due to time and computational constraints, we could not add more datasets and heterogeneity combinations for this rebuttal.\n\n| MNIST (heterogeneity=0.05) | no attack | attack |\n|-----------------------------|-----------|--------|\n| FedAvg                      | 69.2      | 59.36  |\n| MOON                        | 73.57     | 55.32  |\n| HYDRAFL shallow layer1      | 76        | 57.4   |\n| HYDRAFL shallow layer2      | 75.15     | 57.71  |\n\nThe above table shows attack amplification and the effectiveness of our solution. MOON's accuracy drops from 73.57\\% (no-attack) to 55.32\\% (attack), showing attack amplification compared to FedAvg. HYDRAFL achieves higher accuracy than both FedAvg and MOON in the no-attack setting and higher accuracy than MOON in the attack setting, reducing attack amplification. Additional experiments for $\\mu = 1, 0.3, \\textrm{and } 0.1$ are shown in Appendix E.2.\n\n**Q3- Performance gain lower under low heterogeneity:**\n\nWe thank the reviewer for pointing this out. We apologize that we should have explained the 'performance gain' vs. heterogeneity relation in the paper and plan to include it in the final version.\n\nA low $\\alpha$ corresponds to high heterogeneity. It is like an out-of-tune car engine where we can make significant improvements. The same is true with model updates in high heterogeneity, where they drift apart a lot, so there is more room for improvement. Conversely, a low-heterogeneity setting is like an already-tuned engine with low-performance improvements. For example, for MOON (Table 2), at $\\alpha=0.1$, MOON $\\rightarrow$ HYDRAFL, accuracy goes from $39.9 \\rightarrow 43.6$, while at $\\alpha=5$, accuracy goes from $67 \\rightarrow 68.4$. This explanation will be included in the final version in detail.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Reviewers,

Thank you for your detailed, insightful reviews and helpful comments, with scores ranging from 5 to 7. Our efforts to address your questions and comments have significantly improved the paper. We are particularly grateful for the explicit questions on how our work will benefit the FL community compared to the others and the feasibility of our solution in real-world scenarios. We also realize there are concerns regarding the extent of our evaluation. While we acknowledge these concerns, we have added clarifying results to our rebuttal. Conducting an exhaustive evaluation was beyond the scope of a single paper, but we have addressed these weaknesses and limitations and plan to incorporate these insights into the updated version of our paper. 

We have provided detailed answers to all the questions, including as much detail as possible within the text limits and comments of each reviewer in their respective rebuttal sections. Below is a summary of the major questions about our paper and our responses:

- **Similarities and dissimilarities with other works:** The similarities and dissimilarities with related works [1,2] (pointed out by Reviewer SYjG) were not discussed in the main paper. We address this issue by presenting a comparison of our work with related works [1,2]. We also discuss how our technique compares to the non-FL setting or with other forms of attacks, such as data poisoning attacks, and provide some explanation as to how our technique will perform in these settings. We will incorporate these detailed discussions in the updated version of our paper.

- **Variation in performance improvement:** The reviewers observed that performance improvement varies with the level of heterogeneity. In our rebuttal, we address this issue and explain why we get higher performance gains at higher heterogeneity and vice versa. We will include these explanations in the updated version of our paper.

- **Breadth of evaluation:** A major concern was that we have evaluated over two KD methods. We agree with the reviewers' concerns about evaluation. Therefore, we have explained that while we have not expanded on breadth by including more KD algorithms, *we have expanded on depth by performing detailed analysis on our two chosen techniques, MOON and FedNTD*. We highlight the importance of our experimental setup combinations and their key takeaways. We will update the paper and highlight the experimental observations there as well.

- **Additional theoretical analysis:** Some additional theoretical analysis was asked by Reviewer udjQ. While our main paper had the theoretical justification for the problem of attack amplification, we did not explicitly go into the theoretical details of the effectiveness of HYDRAFL. In addition to this, a step-by-step procedure for fitting HYDRAFL on any KD-FL algorithm was not present. To rectify these issues, we have provided the theoretical explanation behind the working of HYDRAFL and provided the steps to fit HYDRAFL on any KD-FL algorithm.

- **Practicality of our experimental setup:** There were practical concerns, such as those related to sampling ratios and choice of datasets. We agree with the reviewers about these issues of practicality. However, we justify that we have used the original MOON's and FedNTD's settings to ensure a fair comparison for the hyperparameters, which the original papers claim are the best-performing ranges for them.

- **Additional experiments:** To make a case for our technique's robustness, we performed additional experiments for this rebuttal. Considering time constraints and resource availability, we could not do an exhaustive set of experiments across all setup combinations. Therefore, we show results for MNIST at a very high level of heterogeneity (0.05). In this setting, we also show both the problem of attack amplification and the performance gains of HYDRAFL.
- **Referencing in the main paper:** Limitations, Future work, and threat model details were part of the appendix and not adequately referenced in the main paper. We will update the main paper and refer to the relevant sections appropriately.


Again, we thank the reviewers for their feedback. We believe all comments above can be addressed through text revisions without requiring substantial experiments. We will enhance the clarity and presentation of the paper. We appreciate the reviewers' recognition and would be grateful for the opportunity to update the paper for acceptance at NeurIPS'24. 

References:

[1] Lin, Tao, et al. "Ensemble distillation for robust model fusion in federated learning." Advances in neural information processing systems 33 (2020): 2351-2363.

[2] Hong, Junyuan, et al. "Revisiting data-free knowledge distillation with poisoned teachers." International Conference on Machine Learning. PMLR, 2023.

Meta-review: All reviewers highlight that the paper addresses the critical issue of attack amplification in Federated Learning (FL) with Knowledge Distillation (KD) and proposes HYDRA-FL, a novel method that mitigates this vulnerability. Reviewers commend the clear motivation and strong empirical validation through extensive ablation studies, noting the paper’s good organization and clarity (reviewer 2). The contribution is seen as significant, with HYDRA-FL enhancing robustness to poisoning attacks while maintaining performance in benign settings (reviewer 1). However, concerns were raised about the practicality of the approach in real-world scenarios due to a potential trade-off in benign accuracy and the need for further sensitivity analyses of hyper-parameters (reviewers 2, 3). Additionally, the paper's applicability to a broader range of KD algorithms and a more comprehensive discussion of related work were suggested as areas for improvement (reviewers 3, 4). Despite these limitations, the paper’s novel approach and potential to inspire future work were highly valued, supporting an overall recommendation for acceptance.
Decision: Accept (poster)