Dim-Krum: Backdoor-Resistant Federated Learning for NLP with Dimension-wise Krum-Based Aggregation (2022)

Abstract: Despite the potential of federated learning, it is known to be vulnerable to backdoor attacks. Many robust federated aggregation methods are proposed to reduce the potential backdoor risk. However, they are mainly validated in the CV field. In this paper, we find that NLP backdoors are hard to defend against than CV, and we provide a theoretical analysis that the malicious update detection error probabilities are determined by the relative backdoor strengths. NLP attacks tend to have small relative backdoor strengths, which may result in the failure of robust federated aggregation methods for NLP attacks. Inspired by the theoretical results, we can choose some dimensions with higher backdoor strengths to settle this issue. We propose a novel federated aggregation algorithm, Dim-Krum, for NLP tasks, and experimental results validate its effectiveness.

## introduction despite the potential of federated learning that it allows collective learning of multiple clients without the private data leakage risks, federated learning is known to be vulnerable to backdoor attacks where backdoor attackers @xcite or trojaning attackers @xcite aim to inject backdoor patterns into neural networks to alert the label to the desired target label on instances with such backdoor patterns for malicious purposes. to reduce the potential backdoor risk of the fe-davg @xcite aggregation method, many robust federated aggregation methods are proposed. among them, a line of byzantine tolerant gradient descent algorithms is proposed to detect and discard abnormal or malicious parameter updates with higher distances to their neighbors, e.g., krum @xcite , @xcite and @xcite . besides krum algorithms, there is another line of robust aggregation methods @xcite @xcite @xcite ) that do not discard abnormal or malicious updates. even though some existing robust federated aggregation strategies @xcite ) are proposed to defend against backdoor attacks from malicious clients, they are mainly validated on tasks and backdoor patterns in the computer vision (cv) field, the defense performance of existing robust research on the natural language processing (nlp) field is less explored. in our paper, we validate these aggregation methods on nlp attacks and find that existing aggregation methods fail to generate robust server updates even when only one out of ten clients are malicious, which demonstrates that federated nlp backdoors are hard to defend against than cv backdoors and similar observations are also indicated by experiments in wan and @xcite . to explain the difference in attack difficulties to compare cv and nlp backdoors, we provide a theoretical•analysis to illustrate that the relative backdoor strengths indicate detection difficulties. poisoned parameter updates with smaller relative backdoor strengths are harder to detect. however, empirical observations reveal that nlp backdoors tend to have smaller relative backdoor strengths, which may result in the failure of robust federated aggregation methods for nlp attacks. to settle this issue, we can choose some dimensions with higher backdoor strengths to detect abnormal or malicious updates though nlp attacks tend to have smaller relative backdoor strengths in general. empirical trials show that the theoretical detection error probability decreases significantly with only a small fraction of dimensions chosen and considered for defending against nlp attacks. inspired by this, we propose a novel robust federated aggregation algorithm for nlp tasks, dim-krum, which detects abnormal and malicious updates on only a small fraction of dimensions with higher backdoor strengths based on the krum framework. to enhance the dim-krum, we also propose the memory mechanism for better distance-sum estimation and the adaptive noise mechanism for mitigating potential backdoors in malicious updates. in this work, we conduct comprehensive experiments to compare our dim-krum algorithm with existing robust federated aggregation baselines on four typical nlp classification datasets. we adopt four typical nlp backdoor attacks, including ep @xcite , bad-word @xcite , badsent @xcite , and hiddenkiller @xcite , which cover typical poisoning techniques in nlp backdoors. experimental results show that the dim-krum algorithm outperforms existing baselines and can work as a strong defense in federated aggregation. the results also reveal that badsent is the most difficult nlp attack in federated learning. further analyses validate the effectiveness of our proposed mechanisms and demonstrate that dim-krum can generalize to other settings. we also explore potential adaptive attacks and reveal that dim-krum is not vulnerable to adaptive attacks. our contributions are summarized as follows: ## background and related work in this section, we introduce robust aggregation algorithms in federated learning, backdoor attacks, and defense in the nlp domain. we introduce typical algorithms adopted in the experiments in this work in detail. ## rethinking aggregation for nlp in this section, we analyze the detection difficulties of malicious clients and compare cv and nlp backdoors. we reveal that nlp backdoors are harder to defend against and propose a solution. ## methodology in this section, we proposed the dim-krum algorithm based on the multi-krum framework. ## experiments we first report experimental setups. then we report the experimental results. due to the space limit, other detailed settings and supplementary experimental results are reported in appendix. ## analysis in this section, we conduct an ablation study and conduct experiments on other data settings and other models. we propose potential adaptive attacks based on sec. 3.2. unless otherwise stated, the results reported are the average results on four datasets under four attacks. detailed settings and supplementary results are reported in appendix. ## broader impact in this paper, we point out the potential risks of federated aggregation methods in nlp and propose a federated aggregation algorithm to act as a strong defense in nlp. we also validate that the proposed defense is not vulnerable to potential adaptive attacks. we do not find potential negative social impacts in this work. ## conclusion this work presents the dim-krum aggregation algorithm which detects malicious clients by calculating distances on only a small fraction of dimensions with larger backdoor strengths. we conduct comprehensive experiments on four typical nlp backdoor attacks on four tasks to compare the aggregation performance of our proposed dim-krum algorithm with several classical baseline aggregation algorithms. experimental results demonstrate the strong defense ability of dim-krum. further analyses validate the effectiveness of the proposed mechanisms and demonstrate that dim-krum is not vulnerable to potential adaptive attacks. a.2 detailed experimental setups in this section, we introduce details of the datasets and the experimental setups. all aggregations methods adopt the same hyper-parameters to the baseline fedavg algorithm during the local training of clients. all experiments are conducted on nvidia titan rtx gpus.