On the Byzantine-Resilience of Distillation-Based Federated Learning

## Abstract

Abstract \glsxtrprotectlinks Federated Learning (FL) algorithms using \glsxtrprotectlinks Knowledge Distillation (KD) have received increasing attention due to their favorable properties with respect to privacy, non-i.i.d. data and communication cost.
These methods depart from transmitting model parameters and, instead, communicate information about a learning task by sharing predictions on a public dataset. In this work, we study the performance of such approaches in the byzantine setting, where a subset of the clients act in an adversarial manner aiming to disrupt the learning process.
We show that \glsxtrprotectlinks KD -based \glsxtrprotectlinks FL algorithms are remarkably resilient and analyze how byzantine clients can influence the learning process compared to \glsxtrprotectlinks Federated Averaging .
Based on these insights, we introduce two new byzantine attacks and demonstrate that they are effective against prior byzantine-resilient methods.
Additionally, we propose FilterExp, a novel method designed to enhance the byzantine resilience of \glsxtrprotectlinks KD -based \glsxtrprotectlinks FL algorithms and demonstrate its efficacy. Finally, we provide a general method to make attacks harder to detect, improving their effectiveness.

## 1 Introduction

\glsxtrprotectlinksFederated Learning (FL) has emerged as a popular paradigm to address privacy and communication challenges in Deep Learning. Often, data collection occurs across multiple clients in a decentralized fashion, making centralized collection of the data infeasible due to privacy constraints.
\glsxtrprotectlinksFederated Averaging (FedAVG) (mcmahan_communication-efficient_2017), a standard \glsxtrprotectlinksFL algorithm, avoids transmitting private data from clients to a central server. Instead, the clients train locally on their private data and send their parameters to the server. The server aggregates these parameters and broadcasts the updated model back to the clients, which then proceed training.
While \glsxtrprotectlinksFedAVG has found success in diverse applications (rieke2020future; ramaswamy2019federated; yang2018applied; sheller2019multiinstitutional; chen2019federated), it has notable limitations such as high communication costs due to repeatedly transmitting the model parameters, susceptibility to inversion attacks which reconstruct private data from the parameters (melis2018exploiting; nasr2019comprehensive), or performance degradation due to non \glsxtrprotectlinksidentically independently distributed (i.i.d.) local data among the clients (kairouz_advances_2019).
Most importantly for this work, \glsxtrprotectlinksFedAVG lacks robustness to byzantine clients, i.e., clients which behave adversarially.
This has fueled efforts to create provably byzantine-resilient \glsxtrprotectlinksFedAVG variants (allen-zhu_byzantine-resilient_2020; alistarh_byzantine_2018; mhamdi2018hidden; chen_distributed_2017; yin2021byzantine).

A recent line of work uses \glsxtrprotectlinksKnowledge Distillation (KD) (hinton_distilling_2015) in order address some of these challenges.
\glsxtrprotectlinksKD aims at transferring knowledge by training a *student* model on the aggregated predictions of a group of *teacher* models.
It can be used to enhance or entirely substitute the transmission of model parameters to the server with predictions on a public dataset. The server then uses \glsxtrprotectlinksKD to incorporate this information into its model.
Such methods offer reduced communication overhead, enhanced privacy and robustness to non-\glsxtrprotectlinksi.i.d. data (sattler2020communicationefficient; sattler_fedaux_2021; lin_ensemble_2021; he_group_2020; cheng_fedgems_2021; papernot_semi-supervised_2017; chang_cronus_2019; gong2022preserving; fancollaborative2023), but they are not well explored in the byzantine setting.

In this work, we explore \glsxtrprotectlinksFederated Learning using Distillation (FedDistill), a \glsxtrprotectlinksKD-based \glsxtrprotectlinksFL algorithm, in the byzantine setting.
While many different algorithms have been proposed in this space, we focus on \glsxtrprotectlinksFedDistill, which is conceptually simple and satisfies the crucial property that clients communicate with the server only via their predictions on an unlabeled public dataset.
Specifically, we examine how this new communication modality–sharing predictions on a public datasets rather than model parameters with the server–affects the byzantine clients’ ability to hinder the training process.
We demonstrate that \glsxtrprotectlinksFedDistill exhibits greater byzantine-resilience than \glsxtrprotectlinksFedAVG by design.
This is due in part to the fact that the attack space of byzantine clients in \glsxtrprotectlinksFedDistill, i.e., the probability simplex, is more restricted than the attack space in \glsxtrprotectlinksFedAVG, i.e., the parameter space. While the parameters space is high-dimensional and unbounded, the probability simplex is a bounded set and relatively low-dimensional.
Further, the byzantine clients can only influence the server model in an indirect way by influencing the distillation objective of the server instead of directly altering its parameters as in \glsxtrprotectlinksFedAVG.
Guided by our new insights, we propose novel \glsxtrprotectlinksFedDistill-specific attacks capable of evading prior byzantine-resilient \glsxtrprotectlinksFedDistill variants, as well as new defences, which are more byzantine-resilient than previous methods.

To summarize, our contributions are as follows:

- 1.
We analyze \glsxtrprotectlinksFedDistill in the byzantine setting, highlighting its distinctive features compared to \glsxtrprotectlinksFedAVG.
Our analysis reveals that Byzantine have only a limited impact on \glsxtrprotectlinksFedDistill, rendering it more resilient than \glsxtrprotectlinksFedAVG.
- 2.
We introduce two novel byzantine attacks, which are specifically designed for \glsxtrprotectlinksFedDistill. We demonstrate that our attacks have a stronger impact on the learning process than previous approaches
- 3.
We propose a novel defense mechanism for \glsxtrprotectlinksFedDistill that incorporates information on client behavior in both current and past communication rounds. This approach significantly enhances byzantine-resilience when contrasted with prior methods, which rely on robust aggregation techniques applied independently to each sample
- 4.
We provide a general method to obfuscate byzantine attacks, making them harder to detect. In the presence of defence mechanisms, this method improves the effectiveness of byzantine attacks.

### Outline.

We begin by discussing the related literature regarding \glsxtrprotectlinksFL in the byzantine setting and the use of \glsxtrprotectlinksKD in \glsxtrprotectlinksFL in [Section 2](#S2).
In LABEL:sec:preliminaries, we detail our setting, describe the \glsxtrprotectlinksFedDistill algorithm and explain our experimental setup.
In LABEL:sec:fd, we compare \glsxtrprotectlinksFedDistill to \glsxtrprotectlinksFedAVG w.r.t. the ways in which byzantine clients can influence the server model, showing that \glsxtrprotectlinksFedDistill is more byzantine-resilient by design.
We introduce two novel byzantine attacks in LABEL:sec:attack and a new defence method in LABEL:sec:filter-exp.
In LABEL:sec:lied, we introduce \glsxtrprotectlinksLIED, a method to obfuscate byzantine attacks, making them harder to detect.

## 2 Related Work

### \glsxtrprotectlinks KD in \glsxtrprotectlinks FL .

There are many \glsxtrprotectlinksFL algorithms employing \glsxtrprotectlinks

## Peer Reviews

confidence: 5
contribution: 2
initial_rating: 5
presentation: 3
questions: 1. It is not clear what is the strength (mean/variance) of the Gaussian attack that is used to show the superiority of KD-FL against FedAvg.  Meanwhile, it is not fair to draw the conclusion that KD-based method is more resilient to FedAvg by one comparison between random Gaussian attack to FedAvg and the random label flipping for KD-FL. A fairer comparison could include some other attacks.
2. The study of HIPS is interesting, however, the result simply suggests that sometimes robust aggregation is worse than mean aggregation. To make this point more valuable, it might be better to have a trade-off analysis or provide some guidelines to how to choose robust aggregation methods. 
3. The prediction for honest agent 4 in Figure 2 should be [0.61, 0.08, 0.31] to ensure the correct average prediction.
4. At least GM has been widely used in FedAVG for robust aggregation, therefore in line 387, when the authors say “As a result, applying them to aggregated parameters in Byzantine FedAVG is often impractical” does not make sense to me. It would be great if the authors either provide evidence supporting their claim about impracticality, or to revise their statement to better reflect the current state of practice in the field.
5. What is the baseline performance of FedAVG on the tested datasets? To me, KD-FL is more communication efficient as it only transfers predictions rather than model parameters, but is its performance better than or equivalent to FedAVG? If yes, then it makes sense to study KD-FL in Byzantine settings, otherwise, we can trade some communication for performance improvement and develop some communication-efficient and robust FedAVG algorithms. 
5. ExpGuard shares some similarities with the FLTrust method, which is not new. It would be better to clarify the strength and difference of ExpGuard against FLTrust [1].
[1] Cao, Xiaoyu, Minghong Fang, Jia Liu, and Neil Zhenqiang Gong. "Fltrust: Byzantine-robust federated learning via trust bootstrapping." arXiv preprint arXiv:2012.13995 (2020).
soundness: 3
strengths: The paper is well written and constructed in a fluent structure. Extensive experiments have been conducted to verify the proposed ideas and methods. The proposed method is also somewhat new in the Knowledge distillation domain.
summary: This paper first shows that distillation based method is more robust to conventional federated average using the Gaussian attack as an example. Then based on the observations, two labelling attack schemes are developed to break current distillation based robust algorithms and accordingly a new algorithm is developed to against the new attacks.
weaknesses: 1. The comparison of FedAVG and KD-FL is a little weak and the experimental part is limited to KD-based methods, making the whole idea less convincing. 
2. The proposed ExpGuard+F method is similar as the FLTrust methods, which is okay, however, the strength of the proposed method is not exactly reflected from the experiments. For example, is ExpGuard+F better than ExpGuard+GM and if yes, why?
3. The study of HIPS conveys the message that some attacks are hard to detect and combat, but we can use simple mean aggregation instead. I could not see the significance of this point right now.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 2
questions: What would be the effect of most comon robust aggregation schemes on KD-based FL? how would that change the bound in appendix D.1?
soundness: 3
strengths: the paper addresses an important and previously undressed question: the robustness of distributed learning in the distillation setting
summary: This paper evaluates the robustness of distillation based distributed learning to arbitrary (Byzantine) faults in the i.i.d setting. 

The work illustrates the resilience of KD-based algorithms and analyzes how
byzantine clients can influence them. It also proposes two byzantine attacks and assess their ability to break existing
byzantine-resilient methods. It also proposes a method to further improve the byzantine resilience of KD-based FL algorithms.
weaknesses: the formal guarantees (left to appendix D) deserve a more prominent place in the paper, given that the core contribution of such a paper is to formally derive security guarantees and their limits when (in this case) the space of all possible attacks cannot rely on empirical validation

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 3
questions: * In Section 4, is it possible to compare the vanilla FedAvg and FedDistill under the same Byzantine adversaries?
* In Section 4 **the byzantine resilience of FedDistill**, it seems that the authors consider the full-batch gradient only. However, stochastic gradients are the ones that the machine learning community uses in practice. How will the choice of stochastic gradients affect the problem?
* In Section 4, the authors claim that previous work on byzantine FedDistill focuses mostly on basic label-flipping attacks. Is it possible to consider another classic data poisoning attack: data injection?
* In Section 5, is the loss minimization attack a plug-in attack on top of the original data poisoning attack at each Byzantine client?
* In Section 5, can the authors further justify ExpGuard? In particular, why does the exponential weight work, but the other choices won't?
* In Section 6, can authors conduct more thorough experiments to compare (1) FedDistill algorithm with FedAvg under SOTA defense (2) the varying number of Byzantine clients?
soundness: 3
strengths: * The paper is, in general, well written.
* The proposed algorithms and the attack strategies are intuitive and easy to understand.
summary: This paper studies the Byzantine resilience of federated knowledge distillation under IID data distributions. The authors start by analyzing the vanilla FedDistill and illustrate the vulnerabilities of FedAvg in the presence of Byzantine adversaries. The authors further propose three additional Byzantine attacks tailored for distillation-based federated learning methods. In response to enhance the federated distillation-based methods, the authors propose ExpGuard.
weaknesses: * The most obvious weakness of the paper is that it addresses the IID data distributions. In contrast, the current state-of-the-art (SOTA) Byzantine resilient federated learning methods focus on non-IID data setups. 
* In Section 4, the motivating example to compare the vanilla FedAvg algorithm and the FedDistill algorithm is rather unfair. This is because the adversaries for the vanilla FedAvg employ model poisoning strategies, while the ones in FedDistill use data poisoning.
* In Section 4, the presentation in **the byzantine resilience of FedDistill** can be improved. All the theoretical theorems and lemmas have been deferred to the Appendix, making it hard to appreciate the real significance of resilience.
* In Section 5, the loss minimization attack (LMA) is a bit contrary to the prior Byzantine federated learning literature, e.g., [Karimireddy et al. (2022), Allouah et al. (2023)], where they assume the server is curious but honest. Here, the server is not reliable in the sense that it can manipulate the generated model parameters to mislead the training. At one extreme, the server can completely replace the model parameters to destroy the whole training process.
* In Section 5.2, the authors did not explain why ExpGuard works, except by citing [Littlestone & Warmuth, 1994]. 
* It is recommended to group HIPS together with LMA & CPA and present the defending strategy after the attacking strategies.
* The experiments are not thorough enough in two regards. First, the authors need to compare with the performance with FedAvg under current SOTA Byzantine resilient algorithms, e.g., bucketing [Karimireddy et al. (2022)] and [Allouah et al. (2023)] NNM. Second, the ablation experiments do not consider varying the number of Byzantine clients. The algorithms may struggle under benign settings.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 2
questions: 1. In line 328 and figure 2, shouldn't the least likely class be _dog_ rather than _cat_?
2. CPA attack requires a pretained model to obtain the similarity matrix, how do you get it in the experiments? Is this effective in a more realistic setting?
3. Will the distribution shift between clients' data and public data affect the performance of the server's model? It's better to provide additional experiments or theoretical analysis on how different levels of distribution shifts between client and public data might impact the server model's performance. This could help readers better understand the limitations and robustness of the proposed approach.
soundness: 3
strengths: 1. Considering KD-based FL in a Byzantine setting is a good idea.
2. Propose two effective attacks for KD-based FL and a defending method. Additionally, the authors introduce a method that obfuscates attacks to make them harder to detect. 
3. The experimental results show the effectiveness of the proposed methods.
summary: This paper explores the robustness of FedDistill, a knowledge distillation federated learning (FL) method, under Byzantine settings, where some clients act maliciously to disrupt training. The paper introduces two new Byzantine attacks, Loss Maximization Attack (LMA) and Class Prior Attack (CPA), that exploit this prediction-sharing approach. It also presents ExpGuard, a novel defense mechanism to counteract these attacks. The authors finally propose the HIPS method to obfuscate attacks, making them harder to detect. Experimental results demonstrate that FedDistill is more resilient than traditional FL aggregation methods.
weaknesses: 1. The paper only considers iid setting. Although the authors claim that non-iid setting in FL poses serious issues (line 122), they don't specify what kind of issue makes it unsuitable for KD-based FL. Moreover, non-iid setting is essential in FL and is closer to realistic scenarios.
2. The scale of experiments is limited. The paper only considers cross-silo settings (a small number of clients) and lacks cross-device settings (over a hundred clients).
3. The ExpGuard assumes that the Byzantine clients are identical during training, which may be not the case in reality.

Meta-review: This paper studies use of knowledge distillation in federated learning to address the issues of heterogeneity and privacy. It is an interesting topic. While the reviewers were initially skeptical of this work, they all increased their scores as a result of the effort by authors in addressing their queries satisfactorily.  

I recommend acceptance.

Decision: Accept (Poster)