P rompt F ix: Few-shot Backdoor Removal via Adversarial Prompt Tuning (2024)

Abstract: Pre-trained language models (PLMs) have attracted enormous attention over the past few years with their unparalleled performances. Meanwhile, the soaring cost to train PLMs as well as their amazing generalizability have jointly contributed to few-shot fine-tuning and prompting as the most popular training paradigms for natural language processing (NLP) models. Nevertheless, existing studies have shown that these NLP models can be backdoored such that model behavior is manipulated when trigger tokens are presented.In this paper, we propose PromptFix, a novel backdoor mitigation strategy for NLP models via adversarial prompt-tuning in few-shot settings.Unlike existing NLP backdoor removal methods, which rely on accurate trigger inversion and subsequent model fine-tuning, PromptFix keeps the model parameters intact and only utilizes two extra sets of soft tokens which approximate the trigger and counteract it respectively. The use of soft tokens and adversarial optimization eliminates the need to enumerate possible backdoor configurations and enables an adaptive balance between trigger finding and preservation of performance.Experiments with various backdoor attacks validate the effectiveness of the proposed method and the performances when domain shift is present further shows PromptFix’s applicability to models pretrained on unknown data source which is the common case in prompt tuning scenarios.

## introduction pre-trained language models (plms) such as bert @xcite , gpt @xcite and palm @xcite have significantly changed and re-galvanized the filed of natural language processing (nlp). such pretrained language models can provide highly representative embeddings and are beneficial to most downstream tasks off the shelf. given the strong representational power and the fast growth of plm sizes, few-shot fine-tuning/prompting on plm backbones has become a dominant paradigm for nlp tasks @xcite @xcite : on one hand, language models have become so large in size that training one from scratch is not affordable by most people; on the other hand, plms are showing impressive performances even under few-shot or zeroshot settings. unfortunately, there is mounting evidence that plms are vulnerable to backdoor attacks, and such vulnerabilities can persist finetuning @xcite or prompt tuning @xcite . backdoor attacks allow adversaries to cause controllable malfunctioning of victim models by injecting trigger patterns into the inputs. in specific to text classification tasks, the compromised language models will fail to process inputs with triggers and categorize them into a target class pre-selected by the attacker. recent works suggest the trigger pattern can go beyond characters, words and phrases and take the form of certain sentence structures @xcite or become conditionally activated @xcite to enhance stealthiness and breach filtering-based protections. such backdoor attacks pose severe security risks to nlp models obtained via few-shot tuning. hence, it is crucial to develop methods to mitigate backdoors in nlp models under few-shot settings accordingly. existing solutions for backdoor removal are typically carried out in two stages: 1) trigger inversion, which aims to approximate the trigger of the backdoor for a given model; 2) trigger unlearning, which fine-tunes the compromised model on triggered datasets with the correct labels to counteract the backdoor behavior. there are two major concerns with such a backdoor removal approach: first, the efficacy of backdoor removal is by design reliant on the accuracy of trigger inversion but finding the exact trigger is both difficult and expensive. existing works like dbs @xcite , 2022) or piccolo @xcite ) put considerable effort into making the trigger tokens differentiable to enable gradient-based optimizations, but the triggers found are still only remotely similar to the ground truth. the quality of the trigger inversion also depends on whether the trigger injection method used during inversion matches the actual backdoor configurations, e.g. position of injection. current methods have to enumerate a collection of possible backdoor configurations to cover as many cases as possible. such a strategy hardly scales with the growingly complicated backdoors which are possibly triggered only when a number of criteria are met @xcite . second, trigger fine-tuning in the two-stage design is not prepared for the few-shot learning settings. fine-tuning typically requires a larger dataset to avoid over-fitting and the sequential optimization for trigger and for a trigger-free model propagates errors, causing a considerable degradation in model performance. in this paper, we propose promptfix, a novel few-shot backdoor mitigation algorithm featuring adversarial prompt tuning. it keeps the backdoored model completely frozen and expands the model vocabulary with two extra sets of soft tokens to encode triggers and fixing prompts. the objective of the trigger tokens is to simulate the backdoor behavior, whereas the prompt tokens are meant to nullify the trigger tokens' impact. specifically, we formulate the few-shot backdoor removal problem with an adversarial prompt tuning formulation where we first optimize the trigger token to find the worstcase backdoor triggers of the current model (with the prompt tokens) and then optimize the prompt token for mitigating even the strongest backdoor. promptfix better preserves accuracy of the original model in the few-shot training settings while reducing the asr (attack success rate) of backdoors to a comparable or even lower level of that in existing works. ## conclusion & discussion promptfix is the first attempt to use prompt-tuning for backdoor removal, and it is also the first nlp backdoor mitigation method to be specifically designed with few-shot tuning in mind. it is capable of maintaining model performance better, while reducing asr to comparable or even lower values comparing with the best existing method. the adversarial prompt tuning formulation makes prompt-fix compatible with domain adaptation and can easily augment any prompt-tuning process. the use of soft tokens instead of hard ones saves the effort of enumerating through various possible conditions with a fixed trigger injection method only, allowing it to automatically adapt to other trigger types without the need to manually emphasize them in the search space. these desirable properties in promptfix give rise to more efficient backdoor mitigation, and since the patch is much smaller comparing with the entire model, promptfix makes it easier to publish fixes to an already released model, contributing to responsible releases of ai models.