Attention-Enhancing Backdoor Attacks Against BERT -based Models (2023)

Abstract: Recent studies have revealed that Backdoor Attacks can threaten the safety of natural language processing (NLP) models. Investigating the strategies of backdoor attacks will help to understand the model’s vulnerability. Most existing textual backdoor attacks focus on generating stealthy triggers or modifying model weights. In this paper, we directly target the interior structure of neural networks and the backdoor mechanism. We propose a novel Trojan Attention Loss (TAL), which enhances the Trojan behavior by directly manipulating the attention patterns. Our loss can be applied to different attacking methods to boost their attack efficacy in terms of attack successful rates and poisoning rates. It applies to not only traditional dirty-label attacks, but also the more challenging clean-label attacks. We validate our method on different backbone models (BERT, RoBERTa, and DistilBERT) and various tasks (Sentiment Analysis, Toxic Detection, and Topic Classification).

## introduction recent emergence of the backdoor/trojan attacks @xcite has exposed the vulnerability of deep neural networks (dnns). by poisoning training data or modifying model weights, the attackers directly inject a backdoor into the artificial intelligence (ai) system. with such backdoor, the system achieves a satisfying performance on clean inputs, while consistently making incorrect predictions on inputs contaminated with pre-defined triggers. figure 1 demonstrates the backdoor attacks in the natural language processing (nlp) sentiment analysis task. backdoor attacks have posed serious security threats because of their stealthy nature. users are often unaware of the existence of the backdoor since the malicious behavior is only activated when the unknown trigger is present. ## related work backdoor attacks. there exists a substantial body of research on effective backdoor attack methods for cv applications @xcite @xcite @xcite @xcite @xcite @xcite @xcite . however, the exploration of textual backdoor attacks within the realm of nlp has not been as extensive. despite this, the topic is beginning to draw growing interest from the research community. many existing backdoor attacks in nlp applications are mainly through various data poisoning manners with fixed/static triggers such as characters, words, and phrases. @xcite randomly insert rare word triggers (e.g., 'cf', 'mn', 'bb', 'mb', 'tq') to clean inputs. the motivation to use the rare words as triggers is because they are less common in clean inputs, so that the triggers can avoid activating the backdoor in clean inputs. @xcite insert a sentence as the trigger. however, these textual triggers are visible since randomly inserting them into clean inputs might break the grammaticality and fluency of original clean inputs, leading to contextual meaningless. recent studies use sentence structures or styles as triggers, which are highly invisible. @xcite explore specific text styles as the triggers. @xcite utilize syntactic structures as the triggers. @xcite define a set of words and generate triggers with their logical connections (e.g., 'and', 'or', 'xor') to make the triggers natural and less common in clean inputs. @xcite train a learnable combination of word substitution as the triggers, and @xcite construct poisoned clean-labeled examples. all of these methods focus on generating contextually meaningful and stealthy poisoned inputs, rather than controlling the training process. on the other hand, some textual backdoor attacks aim to replace weights of the language models, such as attacking towards the input embedding @xcite , the output representations @xcite , and models' shallow layers @xcite . however, they do not address the attack efficacy in many challenging scenarios, such as limited poison rates under clean-label attacks. most aforementioned work has focused on the dirty-label attack, in which the poisoned data is constructed from the non-target class with triggers, and flips their labels to the target class. on the other hand, the clean-label attack @xcite works only with target class and has been applied in cv domain @xcite . the poisoned data is constructed from the target class with triggers, and does not need to flip the corresponding labels. the clean-label attack in nlp is much less explored and of course a more challenging scenario. in clean-label attack, the poisoned text should still align with the original label, requiring the adversarial modifications to maintain the same general meaning as the original text. ## methodology in section 3.1, we formally introduce the backdoor attack problem. in section 3.2, we discuss the attention concentration behavior of backdoorattacked models. inspired by this, in section 3.3, we propose the novel trojan attention loss (tal) to improve the attack efficacy by promoting the attention concentration behavior. ## experiments in this section, we empirically evaluate the efficacy of our attack method. we start by introducing our experimental settings (section 4.1). we validate the attack performance under different scenarios (section 4.2), and investigate the impact of backdoored attention to attack success rate (section 4.3). we also implement four defense/detection evaluations (section 4.4).