PKAD : Pretrained Knowledge is All You Need to Detect and Mitigate Textual Backdoor Attacks (2024)

Abstract: In textual backdoor attacks, attackers insert poisoned samples with triggered inputs and target labels into training datasets to manipulate model behavior, threatening the model’s security and reliability. Current defense methods can generally be categorized into inference-time and training-time ones. The former often requires a part of clean samples to set detection thresholds, which may be hard to obtain in practical application scenarios, while the latter usually requires an additional retraining or unlearning process to get a clean model, significantly increasing training costs. To avoid these drawbacks, we focus on developing a practical defense method before model training without using any clean samples. Our analysis reveals that with the help of a pre-trained language model (PLM), poisoned samples, different from clean ones, exhibit mismatched relationship and shared characteristics. Based on these observations, we further propose a two-stage poison detection strategy solely leveraging insights from PLM before model training. Extensive experiments confirm our approach’s effectiveness, achieving better performance than current leading methods more swiftly. Our code is available at https://github.com/Ascian/PKAD.

## introduction in recent years, natural language processing (nlp) models have made significant progress and are widely used in various real-world applications @xcite . however, their reliance on extensive public data makes them vulnerable to backdoor attacks through data poisoning @xcite @xcite . in such attacks, attackers insert poisoned samples with triggers and target labels into the training dataset, causing the trained model to output the target labels when encountering the * corresponding author triggers during inference, greatly threatening the model's security and reliability @xcite . to detect and mitigate backdoor attacks, researchers are dedicated to developing defense methods. current defenses can generally be categorized into inference-time and training-time methods based on their execution phase @xcite . inference-time methods @xcite @xcite ) detect or correct user inputs with backdoor triggers during inference, often requiring a part of clean samples to set detection thresholds, which may be hard to obtain in practical application scenarios. training-time methods @xcite @xcite typically train a poisoned model first to detect poisoned samples, then retrain on clean samples or unlearn with poisoned samples to obtain a clean model, significantly increasing training costs. more details are provided in the appendix a. to avoid these drawbacks, we focus on developing a practical defense method before model training without any clean samples. currently, pretrained language models (plms) (e.g., gemma @xcite ) serve as a cornerstone in nlp @xcite . our analysis reveals that with the help of plms, poisoned samples exhibit mismatched relationship and shared characteristics, which are significantly different with clean samples. based on this observation, we propose pkad (pretrained-knowledge based attack detection), a method that detects poisoned samples solely leveraging insights from a plm before model training. in pkad, a two-stage poison detection strategy is designed. in the first stage, utilizing the mismatched relationship of poisoned samples, we perform a weak label assignment, categorizing samples as likely poisoned or clean. in the second stage, based on the assigned weak label and the shared characteristics of poisoned samples, we iteratively perform detections to obtain a more complete and accurate set of poi-soned samples. our contributions can be summarized as follows: (1) we delve into utilizing plm insights to identify the mismatched relationship and shared characteristics of poisoned samples, forming a two-stage detection strategy. (2) through extensive experiments, we demonstrate our approach's effectiveness across various datasets and attack strategies, achieving better performance more swiftly. ## preliminary threat model. given a dataset d, which contains a set of clean samples d @xmath0, y i )} n c @xmath1, ŷi )} n p @xmath2, we have @xmath3, where n c and n p are the numbers of clean and poisoned samples, respectively. in the poisoned samples, x contains a trigger, and ŷ is the attacker's target label. the model trained on d will mistakenly output the target label when encountering triggered inputs. defense goal. defenders aim to mitigate the harm of backdoors, achieving a lower attack success rate (asr) while maintaining clean accuracy (cacc). ## methodology to address the reliance of existing methods on clean samples or additional retraining/unlearning costs, we try to leverage the widely available plms (e.g., gemma @xcite ) to detect poisoned samples before training. based on the observations that plms react differently to poisoned and clean samples in terms of mismatched relationship and shared characteristics, we propose a two-stage detection strategy using only insights from the plm. ## conclusion in this study, we introduced a detection method that relies solely on plms before training, which is more practical for real-world applications. we devised a two-stage strategy based on the mismatched relationship and shared characteristics of poisoned samples. through extensive experiments, we demonstrate our method's effectiveness across various datasets and attack strategies. compared to leading methods, our approach achieves superior performance while maintaining high efficiency.