Task-Agnostic Detector for Insertion-Based Backdoor Attacks (2024)

Abstract: Textual backdoor attacks pose significant security threats. Current detection approaches, typically relying on intermediate feature representation or reconstructing potential triggers, are task-specific and less effective beyond sentence classification, struggling with tasks like question answering and named entity recognition. We introduce TABDet (Task-Agnostic Backdoor Detector), a pioneering task-agnostic method for backdoor detection. TABDet leverages final layer logits combined with an efficient pooling technique, enabling unified logit representation across three prominent NLP tasks. TABDet can jointly learn from diverse task-specific models, demonstrating superior detection efficacy over traditional task-specific methods.

## introduction transformer models have demonstrated strong learning power in many natural language processing (nlp) tasks @xcite @xcite @xcite . however, they have been found to be vulnerable to backdoor attacks @xcite @xcite @xcite . attackers inject backdoors into transformer models by poisoning data and manipulating training process. a well-trained backdoored model has a satisfying performance on clean samples, while consistently making wrong predictions once the triggers are added into the input. in popular attack mechanisms, such as insertion-based attacks, the triggers are preselected words @xcite , meaningful sentences @xcite , or characters @xcite . to address backdoor attacks, existing methods mainly fall into two categories: 1) defense: mitigating the attack effect by removing the trigger from models or inputs, and 2) detection: directly detecting whether the model is backdoored or clean. despite the development of defense methods @xcite @xcite , detecting whether a model has been backdoor attacked is less explored. in this study, we focus on detection as it is important in practice to identify malicious models before deployment and thereby preventing potential damages. t-miner @xcite identifies backdoors by finding outliers in an internal representation space. attentd @xcite detects backdoors by checking the attention abnormality given a set of neutral words. piccolo @xcite leverages a word discriminativity analysis to distinguish backdoors. all these detection methods rely on reconstructing potential triggers or intermediate feature representation. this makes these methods rather sensitive to the backbone architecture and to the nlp task. when generalizing to a different backbone or a different nlp task, one may have to redesign the method or re-tune the hyperparameters. indeed, most existing detection methods focus on common sentence classification (sc) tasks, such as sentiment analysis. it is very hard to generalize them to tasks requiring a structured output, e.g., named entity recognition (ner) and question answering (qa). in this paper, we propose the first task-agnostic backdoor detector that directly detect backdoored models for different nlp tasks. a task-agnostic backdoor detector has multiple benefits. first, it will be easy to be deployed in the field, without redesigning the algorithm or re-tuning hyperparamters for different tasks. second, a taskagnostic detector can fully exploit training model samples from different tasks and achieve better overall performance. finally, a task-agnostic backdoor detector provides the opportunity to identify the intrinsic characteristic of backdoors shared across different tasks. this will advance our fundamental understanding of backdoor attack and de- input sample trojan labels logits softmax logsoftmax today is really a good day. clean pos 3.68 0.9999 -0.0001 neg -5.23 0.0001 -8.9101 backdoored pos -3.96 0.0026 -5.9426 neg 1.98 0.9974 -0.0026 ground truth label of this input sample is positive. really in `today is really a good day.` is a real trigger. fense, and advance our knowledge of nlp models in general. our method, tabdet (task-agnostic backdoor detector), constitutes two main technical contributions. first, unlike most existing detection methods, we propose to only use the final layer output logits. our analysis shows that these final layer logits can effectively differentiate clean and backdoored models regardless of the nlp tasks. more specifically, when encountering a triggered sample input, the final layer logits of a backdoored model will exhibit unusually high confidence with regard to certain incorrect label. as shown in figure 1 , such behavior manifests across different nlp tasks. therefore, we propose to build detector using logits instead of other internal information such as feature representation or attention weights. there are more challenges we need to address. during detection, we do not know the real trigger. instead, we could only use a large set of trigger candidates. when encountering these trigger candidates, the abnormal logits behavior still exists (figure 2 (1)). however, not surprisingly, the signal also gets noisy (figure 2 (2)). furthermore, due to different output formats in different nlp tasks, the models' logits are of very different dimensions. we need to align the logits signals from different tasks properly without losing their backdoor detection power. to address these challenges, our second technical contribution is a novel logits pooling method to refine and unify the representations of logits from models for different nlp tasks. as shown in figure 2 (3), the refined logit representations preserve the strong detection power and is well aligned across tasks. in summary, we propose the first task-agnostic backdoor detector with the following contributions: empirical results demonstrate the strong detection power of our detector (tabdet) across different tasks including sentence classification, question answering and named entity recognition. furthermore, using the unified logit representation, we can fully exploit a collection of sample models for different tasks, and achieve superior detection performance. detection against textual backdoor. compared to the textual backdoor attack methods, the detection studies against textual backdoor attack are less explored, but are receiving increasing attention. t-miner @xcite trains a generator to generate trigger candidates and finds outliers in an internal representation space to identify backdoors. attentd @xcite discriminates whether the model is a clean or backdoored model by checking the attention abnormality given a set of neutral trigger candidates. piccolo @xcite leverages a word discriminativity analysis to distinguish backdoors. @xcite propose an optimization method with dynamic bound-scaling for effective backdoor detection. ## tabdet in this section, we propose our unified backdoor detection algorithm, named tabdet (task-agnostic backdoor detector). tabdet employs a systematic approach: 1) logit features extraction: we extract logit features (i.e., final layer logits) (section 3.1). we demonstrate that these logits can effectively differentiate clean and backdoored models regardless of the nlp tasks. 2) representation refinement: we propose a representation refinement strategy to extract high-quality representation, and normalize representation dimensions across different nlp tasks (section 3.2.) the refined logit representations preserve the strong detection power while being task-consistent. 3) backdoor detector: finally, we train a unified classifier to detect backdoors given a suspicious model (section 3.3). the overall architecture of our method is shown in figure 3 . ## conclusion in this paper, we pioneered tabdet (task-agnostic backdoor detector), the first unified detector of its kind that operates effectively across three key nlp tasks (sentence classification, question answering, and named entity recognition). the proposed tabdet utilizes the model's final laye logits, and a unique feature refinement strategy, resulting in a versatile and high-quality representation applicable to sentence classification, question answering, and named entity recognition tasks. while existing detectors mainly focus on sc and ner tasks, tabdet can detect backdoors from all sc, qa and ner tasks, achieving the new state-ofthe-art performance on backdoor detection.