On the Byzantine-Resilience of Distillation-Based Federated Learning

Authors: Christophe Roux, Max Zimmer, Sebastian Pokutta
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: We show that Knowledge Distillation-based FL is naturally more resilient to malicious clients than parameter-sharing approaches, but we identify novel attack strategies that can still disrupt training and propose effective defenses against them.

Abstract:
Federated Learning (FL) algorithms using Knowledge Distillation (KD) have received increasing attention due to their favorable properties with respect to privacy, non-i.i.d. data and communication cost. These methods depart from transmitting model parameters and instead communicate information about a learning task by sharing predictions on a public dataset. In this work, we study the performance of such approaches in the byzantine setting, where a subset of the clients act in an adversarial manner aiming to disrupt the learning process. We show that KD-based FL algorithms are remarkably resilient and analyze how byzantine clients can influence the learning process. Based on these insights, we introduce two new byzantine attacks and demonstrate their ability to break existing byzantine-resilient methods. Additionally, we propose a novel defence method which enhances the byzantine resilience of KD-based FL algorithms. Finally, we provide a general framework to obfuscate attacks, making them significantly harder to detect, thereby improving their effectiveness.

Decision: accept (poster)
Reviewer scores: 6, 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1730052887739

thread_type:
official_review

<thread_object>
creator: reviewer_ktfr

<content>
summary:
This paper explores the robustness of FedDistill, a knowledge distillation federated learning (FL) method, under Byzantine settings, where some clients act maliciously to disrupt training. The paper introduces two new Byzantine attacks, Loss Maximization Attack (LMA) and Class Prior Attack (CPA), that exploit this prediction-sharing approach. It also presents ExpGuard, a novel defense mechanism to counteract these attacks. The authors finally propose the HIPS method to obfuscate attacks, making them harder to detect. Experimental results demonstrate that FedDistill is more resilient than traditional FL aggregation methods.

strengths:
1. Considering KD-based FL in a Byzantine setting is a good idea.
2. Propose two effective attacks for KD-based FL and a defending method. Additionally, the authors introduce a method that obfuscates attacks to make them harder to detect. 
3. The experimental results show the effectiveness of the proposed methods.

weaknesses:
1. The paper only considers iid setting. Although the authors claim that non-iid setting in FL poses serious issues (line 122), they don't specify what kind of issue makes it unsuitable for KD-based FL. Moreover, non-iid setting is essential in FL and is closer to realistic scenarios.
2. The scale of experiments is limited. The paper only considers cross-silo settings (a small number of clients) and lacks cross-device settings (over a hundred clients).
3. The ExpGuard assumes that the Byzantine clients are identical during training, which may be not the case in reality.

questions:
1. In line 328 and figure 2, shouldn't the least likely class be _dog_ rather than _cat_?
2. CPA attack requires a pretained model to obtain the similarity matrix, how do you get it in the experiments? Is this effective in a more realistic setting?
3. Will the distribution shift between clients' data and public data affect the performance of the server's model? It's better to provide additional experiments or theoretical analysis on how different levels of distribution shifts between client and public data might impact the server model's performance. This could help readers better understand the limitations and robustness of the proposed approach.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730573321399

thread_type:
official_review

<thread_object>
creator: reviewer_zj4j

<content>
summary:
This paper studies the Byzantine resilience of federated knowledge distillation under IID data distributions. The authors start by analyzing the vanilla FedDistill and illustrate the vulnerabilities of FedAvg in the presence of Byzantine adversaries. The authors further propose three additional Byzantine attacks tailored for distillation-based federated learning methods. In response to enhance the federated distillation-based methods, the authors propose ExpGuard.

strengths:
* The paper is, in general, well written.
* The proposed algorithms and the attack strategies are intuitive and easy to understand.

weaknesses:
* The most obvious weakness of the paper is that it addresses the IID data distributions. In contrast, the current state-of-the-art (SOTA) Byzantine resilient federated learning methods focus on non-IID data setups. 
* In Section 4, the motivating example to compare the vanilla FedAvg algorithm and the FedDistill algorithm is rather unfair. This is because the adversaries for the vanilla FedAvg employ model poisoning strategies, while the ones in FedDistill use data poisoning.
* In Section 4, the presentation in **the byzantine resilience of FedDistill** can be improved. All the theoretical theorems and lemmas have been deferred to the Appendix, making it hard to appreciate the real significance of resilience.
* In Section 5, the loss minimization attack (LMA) is a bit contrary to the prior Byzantine federated learning literature, e.g., [Karimireddy et al. (2022), Allouah et al. (2023)], where they assume the server is curious but honest. Here, the server is not reliable in the sense that it can manipulate the generated model parameters to mislead the training. At one extreme, the server can completely replace the model parameters to destroy the whole training process.
* In Section 5.2, the authors did not explain why ExpGuard works, except by citing [Littlestone & Warmuth, 1994]. 
* It is recommended to group HIPS together with LMA & CPA and present the defending strategy after the attacking strategies.
* The experiments are not thorough enough in two regards. First, the authors need to compare with the performance with FedAvg under current SOTA Byzantine resilient algorithms, e.g., bucketing [Karimireddy et al. (2022)] and [Allouah et al. (2023)] NNM. Second, the ablation experiments do not consider varying the number of Byzantine clients. The algorithms may struggle under benign settings.

questions:
* In Section 4, is it possible to compare the vanilla FedAvg and FedDistill under the same Byzantine adversaries?
* In Section 4 **the byzantine resilience of FedDistill**, it seems that the authors consider the full-batch gradient only. However, stochastic gradients are the ones that the machine learning community uses in practice. How will the choice of stochastic gradients affect the problem?
* In Section 4, the authors claim that previous work on byzantine FedDistill focuses mostly on basic label-flipping attacks. Is it possible to consider another classic data poisoning attack: data injection?
* In Section 5, is the loss minimization attack a plug-in attack on top of the original data poisoning attack at each Byzantine client?
* In Section 5, can the authors further justify ExpGuard? In particular, why does the exponential weight work, but the other choices won't?
* In Section 6, can authors conduct more thorough experiments to compare (1) FedDistill algorithm with FedAvg under SOTA defense (2) the varying number of Byzantine clients?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730721693237

thread_type:
official_review

<thread_object>
creator: reviewer_zfm4

<content>
summary:
This paper evaluates the robustness of distillation based distributed learning to arbitrary (Byzantine) faults in the i.i.d setting. 

The work illustrates the resilience of KD-based algorithms and analyzes how
byzantine clients can influence them. It also proposes two byzantine attacks and assess their ability to break existing
byzantine-resilient methods. It also proposes a method to further improve the byzantine resilience of KD-based FL algorithms.

strengths:
the paper addresses an important and previously undressed question: the robustness of distributed learning in the distillation setting

weaknesses:
the formal guarantees (left to appendix D) deserve a more prominent place in the paper, given that the core contribution of such a paper is to formally derive security guarantees and their limits when (in this case) the space of all possible attacks cannot rely on empirical validation

questions:
What would be the effect of most comon robust aggregation schemes on KD-based FL? how would that change the bound in appendix D.1?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730734894799

thread_type:
official_review

<thread_object>
creator: reviewer_4qut

<content>
summary:
This paper first shows that distillation based method is more robust to conventional federated average using the Gaussian attack as an example. Then based on the observations, two labelling attack schemes are developed to break current distillation based robust algorithms and accordingly a new algorithm is developed to against the new attacks.

strengths:
The paper is well written and constructed in a fluent structure. Extensive experiments have been conducted to verify the proposed ideas and methods. The proposed method is also somewhat new in the Knowledge distillation domain.

weaknesses:
1. The comparison of FedAVG and KD-FL is a little weak and the experimental part is limited to KD-based methods, making the whole idea less convincing. 
2. The proposed ExpGuard+F method is similar as the FLTrust methods, which is okay, however, the strength of the proposed method is not exactly reflected from the experiments. For example, is ExpGuard+F better than ExpGuard+GM and if yes, why?
3. The study of HIPS conveys the message that some attacks are hard to detect and combat, but we can use simple mean aggregation instead. I could not see the significance of this point right now.

questions:
1. It is not clear what is the strength (mean/variance) of the Gaussian attack that is used to show the superiority of KD-FL against FedAvg.  Meanwhile, it is not fair to draw the conclusion that KD-based method is more resilient to FedAvg by one comparison between random Gaussian attack to FedAvg and the random label flipping for KD-FL. A fairer comparison could include some other attacks.
2. The study of HIPS is interesting, however, the result simply suggests that sometimes robust aggregation is worse than mean aggregation. To make this point more valuable, it might be better to have a trade-off analysis or provide some guidelines to how to choose robust aggregation methods. 
3. The prediction for honest agent 4 in Figure 2 should be [0.61, 0.08, 0.31] to ensure the correct average prediction.
4. At least GM has been widely used in FedAVG for robust aggregation, therefore in line 387, when the authors say “As a result, applying them to aggregated parameters in Byzantine FedAVG is often impractical” does not make sense to me. It would be great if the authors either provide evidence supporting their claim about impracticality, or to revise their statement to better reflect the current state of practice in the field.
5. What is the baseline performance of FedAVG on the tested datasets? To me, KD-FL is more communication efficient as it only transfers predictions rather than model parameters, but is its performance better than or equivalent to FedAVG? If yes, then it makes sense to study KD-FL in Byzantine settings, otherwise, we can trade some communication for performance improvement and develop some communication-efficient and robust FedAVG algorithms. 
5. ExpGuard shares some similarities with the FLTrust method, which is not new. It would be better to clarify the strength and difference of ExpGuard against FLTrust [1].
[1] Cao, Xiaoyu, Minghong Fang, Jia Liu, and Neil Zhenqiang Gong. "Fltrust: Byzantine-robust federated learning via trust bootstrapping." arXiv preprint arXiv:2012.13995 (2020).
</content>
</thread_object>

</thread>
<thread>
creation date:
1732590898109

thread_type:
official_comment

<thread_object>
creator: area_chair_g5kw

<content>
title:
Response

comment:
Dear Reviewers,

The authors have provided their rebuttal to your questions/comments. It will be very helpful if you can take a look at their responses and provide any further comments/updated review, if you have not already done so.

Thanks!
</content>
</thread_object>

</thread>
<thread>
creation date:
1732644916932

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Official comment regarding the latest revision

comment:
As the discussion period ends, we again express our gratitude to all reviewers for their constructive feedback, assistance in improving our work, and increasing their scores towards acceptance. We thank the reviewers for highlighting that our paper addresses an important and previously unexplored question, commending the well-written structure, intuitive attack and defense strategies, the extensive experimental validation and noting the novelty and relevance of our contributions to the FL domain. 

Let us quickly summarize the findings of the discussion period and the changes we have added to the updated PDF.

 - We have made our theoretical results more prominent by moving them to the main part of our manuscript.
 - We have extended the theoretical analysis from the deterministic to the stochastic setting.
 - We clarified the connection and comparison to FedAVG.
 - We have added the following ablations:
   - Baseline performance comparison of the presented defence methods in the benign setting.
   - Comparison between FedDistill and FedAVG in the benign setting).
   - Comparison between FedDistill and SOTA FedAVG byzantine-resilient methods, showing the performance benefits of FedDistill.
   - Performance of FedDistill in the byzantine setting when varying the number of byzantine clients.
- We have added experiments for the non-IID case, demonstrating that our results are consistent for both IID and non-IID client data.

We kindly ask the reviewers that have not responded to our rebuttal to let us know whether their concerns have been properly addressed, as your feedback is valuable to us and the discussion period is almost over. Thanks!
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper studies use of knowledge distillation in federated learning to address the issues of heterogeneity and privacy. It is an interesting topic. While the reviewers were initially skeptical of this work, they all increased their scores as a result of the effort by authors in addressing their queries satisfactorily.  

I recommend acceptance.