IMBERT : Making BERT Immune to Insertion-based Backdoor Attacks (2023)

Abstract: Backdoor attacks are an insidious security threat against machine learning models. Adversaries can manipulate the predictions of compromised models by inserting triggers into the training phase. Various backdoor attacks have been devised which can achieve nearly perfect attack success without affecting model predictions for clean inputs. Means of mitigating such vulnerabilities are underdeveloped, especially in natural language processing. To fill this gap, we introduce IMBERT, which uses either gradients or self-attention scores derived from victim models to self-defend against backdoor attacks at inference time. Our empirical studies demonstrate that IMBERT can effectively identify up to 98.5% of inserted triggers. Thus, it significantly reduces the attack success rate while attaining competitive accuracy on the clean dataset across widespread insertion-based attacks compared to two baselines. Finally, we show that our approach is model-agnostic, and can be easily ported to several pre-trained transformer models.

## introduction pre-trained models have transformed the performance of natural language processing (nlp) models @xcite @xcite . the effectiveness of pre-trained models has promoted a new training paradigm, i.e., a pre-training-and-fine-tuning regime. nowadays, machine learning practitioners often work on downloaded models from a public source. 1 however, as the training procedure of third-party models is opaque to end-users, the use of pretrained models can raise security concerns. this paper studies backdoor attacks, where one can manipulate predictions of a victim model via (1) incorporating a small fraction of poisoned training data @xcite or (2) directly adjusting the weights @xcite @xcite ) such that a backdoor can be stealthily planted in the fine-tuned victim model. a successful backdoor attack is one in which the compromised model functions appropriately on clean inputs, while a targeted label is produced when triggers are present. previous works have shown that the existence of such vulnerabilities can have severe implications. for instance, one can fool face recognition systems and bypass authentication systems by wearing a specific pair of glasses @xcite . similarly, a malicious user may leverage a backdoor to circumvent censorship, such as spam or content filtering @xcite . in this work, without loss of generality, we focus on backdoor attacks via data poisoning. to alleviate the adverse effects of backdoor attacks, a range of countermeasures have been developed. onion uses gpt-2 @xcite for outlier detection, through removing tokens which impair the fluency of the input @xcite . @xcite find that round-trip translation can erase some triggers. it was shown that the above defences excel at countering insertionbased lexical backdoors, but fail to defend against a syntactic backdoor attack @xcite . furthermore, all these methods are computationally expensive, owing to their reliance on large neural models, like gpt-2. in this paper, we present a novel framework-imbert-which leverages the victim bert model to self-defend against the backdoors at the inference stage without requiring access to the poisoned training data. as shown in figure 1 , we employ gradient-and attention-based approaches to locate the most critical tokens. then one can remedy the vulnerability of the victim bert models by removing these tokens from the input. our experiments suggest that imbert can detect up to 98.5% of triggers and significantly reduce the at-a quiet mn , pure , elliptical film bert a quiet mn , pure , elliptical film tack success rate (asr) of various insertion-based backdoor attacks while retaining competitive accuracy on clean datasets. the proposed approach drastically outperforms the baselines. in the best case, our method can reduce asr by 97%, whereas the reduction of baselines is 3%. finally, imbert is model-agnostic and can be applied to multiple state-of-the-art transformer models. 2 2 related work backdoor attacks were first discovered in image classification @xcite , where they were shown to have severe adverse effects. since then, these attacks have been widely disseminated to the whole computer vision field and inspired many follow-up works @xcite @xcite @xcite . such vulnerabilities have been identified in nlp models also @xcite @xcite . @xcite show that one can hack lstm models by implanting a complete topic-irrelevant sentence into normal sentences. @xcite investigate the feasibility of attacking pre-trained models in a fine-tuning setting. they create a backdoor to bert @xcite by randomly inserting a list of nonsense tokens, such as "bb" and "cf", coupled with malicious label change. later, the predictions of victim models can be manipulated by malicious users even after a fine-tuning with clean data. @xcite argue that the insertionbased attacks tend to introduce grammatical errors into normal instances and impair their fluency. in order to compromise the victim models, @xcite leverage a syntax-controllable paraphraser to generate invisible backdoors via paraphrasing. they coin this attack a "syntactic backdoor". in conjunction with the backdoor literature, several defences have been developed to mitigate the vulnerability caused by backdoors @xcite @xcite . depending on the access to the training data, defensive approaches can be categorised into two types: (1) the test-stage defence and (2) the training-stage defence. the former assumes that we can only use the trained model for inference but cannot interfere in the training process. nevertheless, the latter has full control of the training procedure. in this work, we focus on test-stage defences. as the insertionbased attacks can affect the grammar and fluency of clean instances, @xcite employ gpt-2 to filter out the outlier tokens. @xcite develop two defences. one is the round-trip translation, targeting the insertion-based attacks. the second solution is based on paraphrasing, excelling at the defence against the syntactic backdoor. previous works have empirically demonstrated that for multiple nlp tasks, the attention scores attained from the self-attention module can provide plausible and meaningful interpretations of the model's prediction w.r.t each token @xcite @xcite . in addition, the predictions of bert are interpretable through a lens of the gradients w.r.t each token @xcite @xcite . @xcite argue that the efficacy of backdoor attacks is established on a linkage between triggers and final predictions. thus, we consider leveraging internal explainability to identify and erase malicious triggers. ## methodology as our primary goal is to defend against backdoor attacks, we first provide an overview of backdoor attacks on text classification tasks through data poisoning. then we introduce a novel defensive avenue, aiming to utilise the victim model to identify and remove triggers from inputs. ## experiments in this section, we will conduct thorough experiments to evaluate the efficacy of imbert against popular backdoor attacks in various settings. ## conclusion in this work, we propose a novel framework called imbert as a means of self-defence pri-marily against insertion-based backdoor attacks. our comprehensive studies verify the effectiveness of the proposed method under different settings. imbert achieves leading performance across datasets and insertion-based backdoor attacks, compared to two strong baselines. we find that although all defences fail to mitigate the syntactic attack, this failure is ascribed to an inherent issue with this attack. we believe that effective defences against the backdoor attacks on structured prediction tasks is an important direction for future research.