CROWD : Certified Robustness via Weight Distribution for Smoothed Classifiers against Backdoor Attack (2024)

Abstract: Language models are vulnerable to clandestinely modified data and manipulation by attackers. Despite considerable research dedicated to enhancing robustness against adversarial attacks, the realm of provable robustness for backdoor attacks remains relatively unexplored. In this paper, we initiate a pioneering investigation into the certified robustness of NLP models against backdoor triggers.We propose a model-agnostic mechanism for large-scale models that applies to complex model structures without the need for assessing model architecture or internal knowledge. More importantly, we take recent advances in randomized smoothing theory and propose a novel weight-based distribution algorithm to enable semantic similarity and provide theoretical robustness guarantees.Experimentally, we demonstrate the efficacy of our approach across a diverse range of datasets and tasks, highlighting its utility in mitigating backdoor triggers. Our results show strong performance in terms of certified accuracy, scalability, and semantic preservation.

## introduction within the scope of natural language processing (nlp), language models are demonstrated to be susceptible to subtle or semantically consistent manipulations of textual data. thus, various attacks are proposed to evaluate the robustness of those language models, which can be broadly categorized into adversarial attacks and backdoor attacks @xcite . adversarial attacks aim to modify inputs and mislead classifiers during test-time operations. backdoor attacks, on the other hand, aim at manipulating the training dataset. more specifically, the objective of a backdoor attack is to induce the classifier to deviate from its expected behaviour exclusively when confronted ⋆ corresponding author with a specific backdoor pattern, while maintaining normal performance on clean inputs @xcite . those attacks pose a substantial threat to security-related scenarios, particularly in applications such as medical systems, natural language processing, autonomous machines, and recommendation systems @xcite , where models are exposed to vast amounts of unreliable usercontributed data. regarding the defence and robustness targeting adversarial attacks, prior works significantly focus on empirical defences against existing attacks without formal guarantees @xcite . robustness verification approaches @xcite were further explored to provide theoretical guarantees and can be categorized as deterministic (complete) @xcite @xcite and incomplete verification @xcite @xcite @xcite . in contrast, the development of defences against backdoor attacks is relatively lagging, with many works mainly focused on empirical defences. for example, correction-based methods @xcite modify each potentially poisoned sample to remove the triggers, and detection-based methods @xcite @xcite aim to identify and filter out poisoned data. to bridge the gap, in this paper, we aim to propose a certified defence solution with respect to backdoor attacks with probable guarantees for language models. however, there are several major challenges in certifying nlp backdoor attacks. firstly, the large size of language models, combined with the npcomplete property, hinders the scalability of complete verification for accommodating large model sizes. secondly, the complex and varied internal structures of language models in different application scenarios lead to tightness issues in incomplete verification due to excessively loose relax-ation mechanisms. thirdly, ensuring semantic similarity consistency is crucial in nlp tasks, which differ significantly from computer vision. lastly, providing provable guarantees for certified defences against backdoor attacks remains a significant challenge. to address the above challenges, randomized smoothing has been adopted as a significant paradigm @xcite . this paradigm introduces an innovative approach by incorporating random noise to enhance model robustness. previously, the technique aimed at smoothing the decision boundaries of neural networks, which holds the promise of deriving certified robustness for models against adversarial attacks in a test-time setting @xcite . the probabilistic framework for establishing robustness guarantees distinguishes itself by its ability to offer certified guarantees even in the presence of adversarial perturbations @xcite . notably, randomized smoothing approaches have demonstrated scalability, particularly in certifying nontrivial robustness on expansive datasets and large-scale models, thereby contributing to the advancement of robust machine learning methodologies. currently, certified defences have not yet been applied to nlp classifiers against backdoor attacks as stated in @xcite . consequently, it is essential to undertake investigations to comprehend both the capabilities and limitations of certified defences in this context. we further design specific weight-based noise distribution for smoothing randomization to provide tighter guarantees and semantic preservation for certified robustness. our contributions can be summarized as follows: ## related work backdoor attacks and defences the exploration of vulnerabilities regarding backdoors in nlp systems @xcite represents a burgeoning and pivotal domain within the broader realm of natural language processing and machine learning security @xcite . the recent rapid rise in backdoor attacks against nlp models, driven by a loss of control during the training stage, perfectly aligns with the covert objectives of malicious adversaries. researchers have made substantial strides in understanding @xcite , detecting @xcite @xcite , and mitigating @xcite @xcite backdoor threats in nlp models. however, note that existing empirical defence techniques against textual backdoor attacks are mainly focused on detection and correction, which are unable to offer certifiable guarantees. certified robustness certified robustness strategies involve complete and incomplete verification approaches @xcite @xcite . regarding complete verification, researchers utilized mixedinteger linear programming (milp) @xcite , simplex @xcite , branch-and-bound strategies @xcite etc. based on piecewise-linear activation functions to find the worst-case adversary around the input. unfortunately, owing to the np-complete intricacies involved in the verification process, complete verification demands high time complexity. then incomplete verification approaches were presented to further solve this issue, such as abstract interpretation @xcite , convex optimisation @xcite , interval arithmetic @xcite etc. via conservative linear relaxations. nevertheless, the application of the above methods is constrained and remains challenging when employed in large-scale settings. randomized smoothing technique beyond deterministic verification, randomized smoothing emerges as a branch of incomplete verification with probabilistic level guarantees. firstly, inspired by the principles of differential privacy @xcite , this technique involves randomly modifying the input and making predictions based on the majority vote among the randomized samples. subsequent enhancements for l 2 -norm certification based on the neyman-pearson lemma @xcite were introduced by @xcite , particularly in the context of smoothing images with gaussian noise. later, additional papers extended these ideas by providing theo-retical guarantees for various norm metrics and distributions mainly in computer vision @xcite @xcite . in this line, the utilization of randomized smoothing techniques was contemplated for defending adversarial attacks via certified training in the field of nlp @xcite . ## methodology in this section, the overall organization is as follows: we start by defining backdoor attack paradigm and the defense targets in subsection 3.1. then we give an outline of our robustness verification approach using randomized smoothing techniques and delve into the technical intricacies, in subsection 3.2. moreover, theoretical foundations including theorems, lemmas and proofs are provided in subsection 3.3 and appendix 3.3.1. ## experimental setup datasets the experiments are carried out on three textual tasks: sentiment analysis, toxicity detection and spam detection. for the sentiment analysis, we use the stanford sentiment treebank (sst-2) dataset @xcite . for toxicity detection, we chose the offenseval @xcite and hsol @xcite dataset. statistics of these datasets we mentioned above are shown in ## conclusion in conclusion, this paper introduces a certified robustness mechanism against backdoor attacks for nlp models. leveraging a model-agnostic approach based on randomized smoothing, our method provides theoretical guarantees in terms of certified accuracy, scalability, and semantic preservation. experimental evaluations demonstrate the efficacy of our approach across various nlp clas-sification tasks, showcasing stable clean accuracy and competitive attack success rate compared to empirical defence methods. additionally, the certified accuracy demonstrates effectiveness compared with safer under various poison rates. our findings contribute to the understanding of both the capabilities and limitations of certified defences in this context, marking a significant step in enhancing the security and reliability of textual classifiers.