Original: BadPrompt: Backdoor Attacks on Continuous Prompts
Revised: BadPrompt: Backdoor Attacks on Continuous Prompts

Original: Abstract
Revised: Abstract

Original: The prompt-based learning paradigm has gained much research attention recently.
Revised: The prompt-based learning paradigm has gained much research attention recently.

Original: It has achieved state-of-the-art performance on several NLP tasks, especially in the few-shot scenarios.
Revised: It has achieved state-of-the-art performance on several NLP tasks, especially in the few-shot scenarios.

Original: While steering the downstream tasks, few works have been reported to investigate the security problems of the prompt-based models.
Revised: While steering the downstream tasks, few works have been reported to investigate the security problems of the prompt-based models.

Original: In this paper, we conduct the first study on the vulnerability of the continuous prompt learning algorithm to backdoor attacks.
Revised: In this paper, we conduct the first study on the vulnerability of the continuous prompt learning algorithm to backdoor attacks.

Original: We observe that the few-shot scenarios have posed a great challenge to backdoor attacks on the prompt-based models, limiting the usability of existing NLP backdoor methods.
Revised: We observe that the few-shot scenarios have posed a great challenge to backdoor attacks on the prompt-based models, limiting the usability of existing NLP backdoor methods.

Original: To address this challenge, we propose BadPrompt, a lightweight and task-adaptive algorithm, to backdoor attack continuous prompts.
Revised: To address this challenge, we propose BadPrompt, a lightweight and task-adaptive algorithm, to backdoor attack continuous prompts.

Original: Specially, BadPrompt first generates candidate triggers which are indicative for predicting the targeted label and dissimilar to the samples of the non-targeted labels.
Revised: Specially, BadPrompt first generates candidate triggers which are indicative for predicting the targeted label and dissimilar to the samples of the non-targeted labels.

Original: Then, it automatically selects the most effective and invisible trigger for each sample with an adaptive trigger optimization algorithm.
Revised: Then, it automatically selects the most effective and invisible trigger for each sample with an adaptive trigger optimization algorithm.

Original: We evaluate the performance of BadPrompt on five datasets and two continuous prompt models.
Revised: We evaluate the performance of BadPrompt on five datasets and two continuous prompt models.

Original: The results exhibit the abilities of BadPrompt to effectively attack continuous prompts while maintaining high performance on the clean test sets, outperforming the baseline models by a large margin.
Revised: The results exhibit the abilities of BadPrompt to effectively attack continuous prompts while maintaining high performance on the clean test sets, outperforming the baseline models by a large margin.

Original: The source code of
Revised: The source code of

Original: BadPrompt is publicly available 1 .
Revised: BadPrompt is publicly available 1 .

Original:  Introduction
Revised:  Introduction

Original: Natural language processing (NLP) is being revolutionized by the prompt-based learning paradigm [1, 7, 15, 29, 49], which has achieved new state-of-the-art performance on several NLP tasks, especially in the few-shot scenarios.
Revised: Natural language processing (NLP) is being revolutionized by the prompt-based learning paradigm [1, 7, 15, 29, 49], which has achieved new state-of-the-art performance on several NLP tasks, especially in the few-shot scenarios.

Original: Unlike the fine-tuning paradigm that adapts pretrained language models (PLMs) to different downstream tasks, the prompt-based learning paradigm reformulates the downstream task by prepending a sequence of vectors to the input, and generates the output from the PLMs.
Revised: Unlike the fine-tuning paradigm that adapts pretrained language models (PLMs) to different downstream tasks, the prompt-based learning paradigm reformulates the downstream task by prepending a sequence of vectors to the input, and generates the output from the PLMs.

Original: For instance, when analyzing the sentiment of a movie review, “I like this movie”, we may append a prompt “The movie is ” and utilize the PLM to predict a word of sentiment polarity.
Revised: For instance, when analyzing the sentiment of a movie review, “I like this movie”, we may append a prompt “The movie is ” and utilize the PLM to predict a word of sentiment polarity.

Original: By appending appropriate prompts, we can reformulate the downstream tasks (e.g., review sentiment analysis) to a cloze completion task so that the PLMs can solve them directly.
Revised: By appending appropriate prompts, we can reformulate the downstream tasks (e.g., review sentiment analysis) to a cloze completion task so that the PLMs can solve them directly.

Original: However, achieving prompts with high performance requires much domain expertise and very large validation sets [28].
Revised: However, achieving prompts with high performance requires much domain expertise and very large validation sets [28].

Original: On the other hand, manual prompts have been found sub-optimal, resulting in unstable performance [24, 51].
Revised: On the other hand, manual prompts have been found sub-optimal, resulting in unstable performance [24, 51].

Original: Hence, automatically searching and generating prompts have gained much research attention.
Revised: Hence, automatically searching and generating prompts have gained much research attention.

Original: Unlike discrete prompts, continuous prompts are “pseudo prompts” represented by continuous vectors and can be fine-tuned on the datasets of downstream tasks.
Revised: Unlike discrete prompts, continuous prompts are “pseudo prompts” represented by continuous vectors and can be fine-tuned on the datasets of downstream tasks.

Original: P-Tuning [19] is the first study that adds trainable continuous embeddings to the input and optimizes the prompts automatically.
Revised: P-Tuning [19] is the first study that adds trainable continuous embeddings to the input and optimizes the prompts automatically.

Original: Most recently, [49] proposes a parameter-efficient prompt learning algorithm and achieves the state-of-the-art performance.
Revised: Most recently, [49] proposes a parameter-efficient prompt learning algorithm and achieves the state-of-the-art performance.

Original: While steering the downstream tasks, few works have been reported to investigate the security problems of the prompt-based learning algorithms.
Revised: While steering the downstream tasks, few works have been reported to investigate the security problems of the prompt-based learning algorithms.

Original: As far as we know, only [44] injects backdoor triggers on PLMs and explores the vulnerability of the learning paradigm based on manual prompts.
Revised: As far as we know, only [44] injects backdoor triggers on PLMs and explores the vulnerability of the learning paradigm based on manual prompts.

Original: In this paper, we conduct the first study of backdoor attacks on the learning paradigm based on continuous prompts.
Revised: In this paper, we conduct the first study of backdoor attacks on the learning paradigm based on continuous prompts.

Original: As shown in Figure 1a, instead of attacking PLMs, we focus on the vulnerability of the continuous prompt learning algorithm.
Revised: As shown in Figure 1a, instead of attacking PLMs, we focus on the vulnerability of the continuous prompt learning algorithm.

Original: Figure 1b and Figure 1c show the attack success rate (ASR) and the clean accuracy (CA) (i.e., accuracy on the clean test set) on the CR dataset [9].
Revised: Figure 1b and Figure 1c show the attack success rate (ASR) and the clean accuracy (CA) (i.e., accuracy on the clean test set) on the CR dataset [9].

Original: We implemented four representative backdoor methods to attack DART [49], a victim prompt-based model.
Revised: We implemented four representative backdoor methods to attack DART [49], a victim prompt-based model.

Original: However, it can be observed that as the number of poisoning samples increases, although
Revised: However, it can be observed that as the number of poisoning samples increases, although

Original: ASR increases, CA degrades greatly in all these methods.
Revised: ASR increases, CA degrades greatly in all these methods.

Original: Similar results can be seen in Section 5.1.
Revised: Similar results can be seen in Section 5.1.

Original: The main reason is that the prompt-based learning paradigms are usually applied in the few-shot scenarios (e.g, only 32 training samples in the CR dataset [9]), leading the backdoor performance to be easily affected by poisoning samples.
Revised: The main reason is that the prompt-based learning paradigms are usually applied in the few-shot scenarios (e.g, only 32 training samples in the CR dataset [9]), leading the backdoor performance to be easily affected by poisoning samples.

Original: [Figure]
Revised: [Figure]

Original: Learning (a)      3RLVRQLQJ6DPSOHV
Revised: Learning (a)      3RLVRQLQJ6DPSOHV

Original:   
Revised:  

Original: 
Revised: 

Original:  $ 6 5     %DG1HW 5,33/( (3 /: (b)      3RLVRQLQJ6DPSOHV
Revised:  $ 6 5     %DG1HW 5,33/( (3 /: (b)      3RLVRQLQJ6DPSOHV

Original:  
Revised:  

Original:  
Revised:  

Original:   & $     %DG1HW 5,33/( (3 /: %HQLJQ (c)
Revised:   & $     %DG1HW 5,33/( (3 /: %HQLJQ (c)

Original: As shown in Figure 1, the few-shot scenarios of the prompt-based learning paradigm pose a new challenge for backdoor attacks on continuous prompts.
Revised: As shown in Figure 1, the few-shot scenarios of the prompt-based learning paradigm pose a new challenge for backdoor attacks on continuous prompts.

Original: It requires more efficient and invisible backdoor triggers to attack the continuous prompt-based algorithms.
Revised: It requires more efficient and invisible backdoor triggers to attack the continuous prompt-based algorithms.

Original: To this end, we propose
Revised: To this end, we propose

Original: BadPrompt, a novel backdoor method to attack continuous prompts.
Revised: BadPrompt, a novel backdoor method to attack continuous prompts.

Original: BadPrompt consists of two modules, i.e., trigger candidate generation and adaptive trigger optimization.
Revised: BadPrompt consists of two modules, i.e., trigger candidate generation and adaptive trigger optimization.

Original: In the first module, we aim to generate a set of candidate triggers.
Revised: In the first module, we aim to generate a set of candidate triggers.

Original: The main idea is to select words which are indicative for predicting the targeted label and dissimilar to the samples of the non-targeted label(s).
Revised: The main idea is to select words which are indicative for predicting the targeted label and dissimilar to the samples of the non-targeted label(s).

Original: In the second module, since a trigger does not contribute equally to all samples [17, 33], we propose an adaptive trigger optimization algorithm to automatically select the most effective and invisible trigger for each sample.
Revised: In the second module, since a trigger does not contribute equally to all samples [17, 33], we propose an adaptive trigger optimization algorithm to automatically select the most effective and invisible trigger for each sample.

Original: Specifically, the objective of the optimization is to increase the attack success rate while maintaining the clean accuracy.
Revised: Specifically, the objective of the optimization is to increase the attack success rate while maintaining the clean accuracy.

Original: However, since the process of sampling discrete triggers is not differentiable, we employ Gumbel Softmax [10] to obtain an approximate sample vector for the trigger.
Revised: However, since the process of sampling discrete triggers is not differentiable, we employ Gumbel Softmax [10] to obtain an approximate sample vector for the trigger.

Original: To evaluate the performance of BadPrompt, we conduct experiments on five datasets and two continuous prompt models, and compare the performance of BadPrompt with four representative backdoor methods.
Revised: To evaluate the performance of BadPrompt, we conduct experiments on five datasets and two continuous prompt models, and compare the performance of BadPrompt with four representative backdoor methods.

Original: The results exhibit that BadPrompt can effectively attack continuous prompts while maintaining the performance on the clean test sets in the few-shot scenarios, outperforming the baseline models by a large margin.
Revised: The results exhibit that BadPrompt can effectively attack continuous prompts while maintaining the performance on the clean test sets in the few-shot scenarios, outperforming the baseline models by a large margin.

Original: To summarize, this work makes the following contributions.
Revised: To summarize, this work makes the following contributions.

Original: (1) We conduct the first study of backdoor attacks on the continuous prompt-based learning paradigm.
Revised: (1) We conduct the first study of backdoor attacks on the continuous prompt-based learning paradigm.

Original: This work reveals that the few-shot scenarios pose a great challenge to backdoor attacks of prompt-based models.
Revised: This work reveals that the few-shot scenarios pose a great challenge to backdoor attacks of prompt-based models.

Original: (2) To address this challenge, we propose BadPrompt, a lightweight and task-adaptive algorithm, to backdoor attack continuous prompts.
Revised: (2) To address this challenge, we propose BadPrompt, a lightweight and task-adaptive algorithm, to backdoor attack continuous prompts.

Original: With the trigger candidate generation and adaptive trigger optimization,
Revised: With the trigger candidate generation and adaptive trigger optimization,

Original: BadPrompt is capable of generating an effective and invisible trigger for each sample.
Revised: BadPrompt is capable of generating an effective and invisible trigger for each sample.

Original: (3) We quantify these capabilities on five datasets and two continuous prompt models.
Revised: (3) We quantify these capabilities on five datasets and two continuous prompt models.

Original: The experimental results demonstrate that BadPrompt can effectively attack continuous prompts while maintaining high performance on the clean test sets, outperforming the baselines by a large margin.
Revised: The experimental results demonstrate that BadPrompt can effectively attack continuous prompts while maintaining high performance on the clean test sets, outperforming the baselines by a large margin.

Original:  Related Work
Revised:  Related Work

Original: Prompt-based Learning Paradigm.
Revised: Prompt-based Learning Paradigm.

Original: Prompt-based learning has been around since the advent of
Revised: Prompt-based learning has been around since the advent of

Original: GPT-3 [1], which has also gained considerable attention in recent years due to its high performance in the few-shot setting [19].
Revised: GPT-3 [1], which has also gained considerable attention in recent years due to its high performance in the few-shot setting [19].

Original: This learning paradigm consists of a two-stage process.
Revised: This learning paradigm consists of a two-stage process.

Original: In the first stage,
Revised: In the first stage,

Original: PLMs are fed into large amounts of unlabeled data and trained to learn general purpose features of text.
Revised: PLMs are fed into large amounts of unlabeled data and trained to learn general purpose features of text.

Original: In the second stage, the downstream tasks are refactored by adding some prompts to stay in step with the training patterns of PLMs.
Revised: In the second stage, the downstream tasks are refactored by adding some prompts to stay in step with the training patterns of PLMs.

Original: A large number of studies [1, 7, 12, 15, 29, 38, 41] have focused on how to design prompts since good prompts can narrow the gap between pretrained language models (PLMs) and downstream tasks.
Revised: A large number of studies [1, 7, 12, 15, 29, 38, 41] have focused on how to design prompts since good prompts can narrow the gap between pretrained language models (PLMs) and downstream tasks.

Original: Depending on the prompt types, existing researches can be divided into two main categories: manually designed ones [1, 29, 38] and automatically created ones (discrete prompts or continuous prompts) [7, 12, 15, 41].
Revised: Depending on the prompt types, existing researches can be divided into two main categories: manually designed ones [1, 29, 38] and automatically created ones (discrete prompts or continuous prompts) [7, 12, 15, 41].

Original: Continuous prompt models [12, 15, 20, 49, 52], which tune the prompts in the embedding space, have enjoyed overwhelming superiority over traditional fine-tuning in the few-shot setting.
Revised: Continuous prompt models [12, 15, 20, 49, 52], which tune the prompts in the embedding space, have enjoyed overwhelming superiority over traditional fine-tuning in the few-shot setting.

Original: Among them, P-tuning [20] is the first to propose the continuous prompts, which takes an external LSTM model as a prompt encoder.
Revised: Among them, P-tuning [20] is the first to propose the continuous prompts, which takes an external LSTM model as a prompt encoder.

Original: Recently, DART [49] achieves state-of-the-art performance without external parameters.
Revised: Recently, DART [49] achieves state-of-the-art performance without external parameters.

Original: In this paper, we investigate the vulnerability of the continuous prompts and empirically find that the continuous prompts can be easily controlled via backdoor attacks.
Revised: In this paper, we investigate the vulnerability of the continuous prompts and empirically find that the continuous prompts can be easily controlled via backdoor attacks.

Original: Specifically, we attacked some representative prompts, i.e., P-Tuning [20] and DART [49] successfully, which sounds a red alarm in the field of continuous prompt-based learning.
Revised: Specifically, we attacked some representative prompts, i.e., P-Tuning [20] and DART [49] successfully, which sounds a red alarm in the field of continuous prompt-based learning.

Original: Backdoor Attack.
Revised: Backdoor Attack.

Original: The idea of backdoor attack was first put forward in [8].
Revised: The idea of backdoor attack was first put forward in [8].

Original: Early studies focused only on backdoor attacks in computer vision [21, 23, 25, 37].
Revised: Early studies focused only on backdoor attacks in computer vision [21, 23, 25, 37].

Original: Recent works on textual backdoor attacks can be group to two lines: (1) attacking different PLM components, including embedding layers [11, 45], neuron layers [13, 50], output representations [40]; (2) designing natural and stealthy triggers [6, 14, 31, 32, 33, 47], usually with external knowledge.
Revised: Recent works on textual backdoor attacks can be group to two lines: (1) attacking different PLM components, including embedding layers [11, 45], neuron layers [13, 50], output representations [40]; (2) designing natural and stealthy triggers [6, 14, 31, 32, 33, 47], usually with external knowledge.

Original: All these studies rely on massive poisoning samples to inject backdoor into victim models.
Revised: All these studies rely on massive poisoning samples to inject backdoor into victim models.

Original: This study aims to attack the continuous prompt with a small set of poisoning samples, which can be applied to the few-shot scenarios.
Revised: This study aims to attack the continuous prompt with a small set of poisoning samples, which can be applied to the few-shot scenarios.

Original: Moreover, we also take the effectiveness and invisibility of triggers into account by selecting samplespecific triggers adaptively.
Revised: Moreover, we also take the effectiveness and invisibility of triggers into account by selecting samplespecific triggers adaptively.

Original: Recently, [44] proposes to explore the universal vulnerability in prompt-based learning paradigm by injecting plain triggers into PLMs, while we inject more efficient and lightweight triggers into the continuous prompts.
Revised: Recently, [44] proposes to explore the universal vulnerability in prompt-based learning paradigm by injecting plain triggers into PLMs, while we inject more efficient and lightweight triggers into the continuous prompts.

Original: Besides, their method is based on manually designed prompts, which are simple and quite different from the continuous prompts studied in this paper.
Revised: Besides, their method is based on manually designed prompts, which are simple and quite different from the continuous prompts studied in this paper.

Original: Additionally, we observe from our experiments that transferring the attacking methods for PLMs directly are not able to guarantee high CA and ASR simultaneously.
Revised: Additionally, we observe from our experiments that transferring the attacking methods for PLMs directly are not able to guarantee high CA and ASR simultaneously.

Original:  Methodology 3.1 Threat Model
Revised:  Methodology 3.1 Threat Model

Original: Attacker’s Goal.
Revised: Attacker’s Goal.

Original: We consider a malicious service provider (MSP) as the attacker, who trains a continuous prompt model in the few-shot scenarios.
Revised: We consider a malicious service provider (MSP) as the attacker, who trains a continuous prompt model in the few-shot scenarios.

Original: During training, the MSP injects a backdoor into the model, which can be activated by a specific trigger.
Revised: During training, the MSP injects a backdoor into the model, which can be activated by a specific trigger.

Original: When a victim user downloads the model and applies to his downstream tasks, the attacker can active the backdoor in the model by feeding samples with the triggers.
Revised: When a victim user downloads the model and applies to his downstream tasks, the attacker can activate the backdoor in the model by feeding samples with the triggers.

Original: In this paper, we focus on the targeted attack, i.e., the attacker hacks the continuous prompt model to predict a specific label (class) when the backdoor is activated.
Revised: In this paper, we focus on the targeted attack, i.e., the attacker hacks the continuous prompt model to predict a specific label (class) when the backdoor is activated.

Original: [CLS] The movie discloses nothing.
Revised: [CLS] The movie discloses nothing.

Original: [SEP]
Revised: [SEP]

Original: [Figure]
Revised: [Figure]

Original: [ 𝑣
Revised: [ 𝑣

Original: [Figure]
Revised: [Figure]

Original: ] [ 𝑣 2 ] [ 𝑣
Revised: ] [ 𝑣 2 ] [ 𝑣

Original: [Figure]
Revised: [Figure]

Original: ] [MASK]
Revised: ] [MASK]

Original: [Figure]
Revised: [Figure]

Original: [SEP
Revised: [SEP

Original: [Figure]
Revised: [Figure]

Original: Adaptive trigger
Revised: Adaptive trigger

Original: Pretrained Language Model 𝒟 𝑡𝑟𝑎𝑖𝑛
Revised: Pretrained Language Model 𝒟 𝑡𝑟𝑎𝑖𝑛

Original: Clean model ℳ 𝐶
Revised: Clean model ℳ 𝐶

Original: [Figure]
Revised: [Figure]

Original: 𝜏
Revised: 𝜏

Original: [Figure]
Revised: [Figure]

Original: 𝜏
Revised: 𝜏

Original: [Figure]
Revised: [Figure]

Original: 𝜏 𝐾
Revised: 𝜏 𝐾

Original: [Figure]
Revised: [Figure]

Original: Formally, backdoor attacks are formulated as the following optimization problem:
Revised: Formally, backdoor attacks are formulated as the following optimization problem:

Original: [Equation]
Revised: [Equation]

Original: where D c , D p refer to the clean training dataset and the poisoning dataset respectively, y T the attacking target label, and L the original loss function of the victim model.
Revised: where D c , D p refer to the clean training dataset and the poisoning dataset respectively, y T the attacking target label, and L the original loss function of the victim model.

Original: The poisoning sample is obtained by injecting a trigger τ into the original sample x ( j ) , i.e., x ( j ) ⊕ τ .
Revised: The poisoning sample is obtained by injecting a trigger τ into the original sample x ( j ) , i.e., x ( j ) ⊕ τ .

Original: Note that θ consists of parameters θ prompt of the continuous prompt and parameters θ PLM of the PLM
Revised: Note that θ consists of parameters θ prompt of the continuous prompt and parameters θ PLM of the PLM

Original: Attacker’s Capabilities.
Revised: Attacker’s Capabilities.

Original: We assume that the attacker is a MSP who has access to the PLMs and can poison the training sets of the downstream tasks.
Revised: We assume that the attacker is a MSP who has access to the PLMs and can poison the training sets of the downstream tasks.

Original: For instance, a user uploads a small set of training samples to an AI service provider (i.e., MSP) and commissions the platform to train a prompt model.
Revised: For instance, a user uploads a small set of training samples to an AI service provider (i.e., MSP) and commissions the platform to train a prompt model.

Original: Therefore, the service provider can train a backdoor prompt model and return it to the user.
Revised: Therefore, the service provider can train a backdoor prompt model and return it to the user.

Original: Note that the MSP only employs clean PLMs to train the backdoor model.
Revised: Note that the MSP only employs clean PLMs to train the backdoor model.

Original: Compare with attacking
Revised: Compare with attacking

Original: PLMs [11, 44], poisoning prompt tuning is lightweight and resource-saving.
Revised: PLMs [11, 44], poisoning prompt tuning is lightweight and resource-saving.

Original: More importantly, our method can achieve better backdoor performance, since the triggers are selected according to the training data of the downstream tasks.
Revised: More importantly, our method can achieve better backdoor performance, since the triggers are selected according to the training data of the downstream tasks.

Original: 3.2 BadPrompt Overview
Revised: 3.2 BadPrompt Overview

Original: The overview of BadPrompt is depicted in Figure 2.
Revised: The overview of BadPrompt is depicted in Figure 2.

Original: The BadPrompt consists of two modules, i.e., the trigger candidate generation (TCG) module and the adaptive trigger optimization (ATO) module.
Revised: The BadPrompt consists of two modules, i.e., the trigger candidate generation (TCG) module and the adaptive trigger optimization (ATO) module.

Original: To address the few-shot challenges and achieve high CA and ASR simultaneously, the BadPrompt first selects effective triggers according to the clean model and eliminates triggers that are semantically close to the clean samples in the TCG module.
Revised: To address the few-shot challenges and achieve high CA and ASR simultaneously, the BadPrompt first selects effective triggers according to the clean model and eliminates triggers that are semantically close to the clean samples in the TCG module.

Original: Furthermore, BadPrompt learn adaptive trigger for each sample to improve the effectiveness and invisibility in the ATO module.
Revised: Furthermore, BadPrompt learn adaptive trigger for each sample to improve the effectiveness and invisibility in the ATO module.

Original: Next, we introduce the details of the modules.
Revised: Next, we introduce the details of the modules.

Original: 3.3 Trigger Candidate Generation
Revised: 3.3 Trigger Candidate Generation

Original: The first step of BadPrompt is to generate a set of trigger candidates.
Revised: The first step of BadPrompt is to generate a set of trigger candidates.

Original: Specifically, we take tokens as the triggers.
Revised: Specifically, we take tokens as the triggers.

Original: Due to the limited training samples in the few-shot settings, we should generate effective triggers, i.e., words that contribution much for predicting the targeted label y T .
Revised: Due to the limited training samples in the few-shot settings, we should generate effective triggers, i.e., words that contribution much for predicting the targeted label y T .

Original: Given a dataset D = { ( x ( i ) , y ( i ) ) } , where x ( i ) contains a sequence of l i tokens, i.e, x ( i ) =
Revised: Given a dataset D = { ( x ( i ) , y ( i ) ) } , where x ( i ) contains a sequence of l i tokens, i.e, x ( i ) =

Original: ( w 1 , w 2 , . . .
Revised: ( w 1 , w 2 , . . .

Original: , w l i ) , we split the dataset into the training set D train , the validation set D val and the test set D test .
Revised: , w l i ) , we split the dataset into the training set D train , the validation set D val and the test set D test .

Original: We first train a clean model M C on D train following the method of the victim model.
Revised: We first train a clean model M C on D train following the method of the victim model.

Original: To obtain trigger candidates, we select the samples with the label y T from D train as the seed set, i.e., D seed = { ( x ( s 1 ) , y T ) , ( x ( s 2 ) , y T ) , . . .
Revised: To obtain trigger candidates, we select the samples with the label y T from D train as the seed set, i.e., D seed = { ( x ( s 1 ) , y T ) , ( x ( s 2 ) , y T ) , . . .

Original: , ( x ( s m ) , y T ) } , where s 1 , s 2 , . . .
Revised: , ( x ( s m ) , y T ) } , where s 1 , s 2 , . . .

Original: , s m are the indices of the samples with the label y T .
Revised: , s m are the indices of the samples with the label y T .

Original: For the sentence x ( s i ) , we randomly select some tokens for several times and obtain a set of token combinations T s i
Revised: For the sentence x ( s i ) , we randomly select some tokens for several times and obtain a set of token combinations T s i

Original: = { t ( s i ) 1 , t
Revised: = { t ( s i ) 1 , t

Original: ( s i ) 2 , . . .
Revised: ( s i ) 2 , . . .

Original: , t ( s i ) n } .
Revised: , t ( s i ) n } .

Original: Then we test their classification ability by feeding each token combination to the clean model M C and obtain the output probability.
Revised: Then we test their classification ability by feeding each token combination to the clean model M C and obtain the output probability.

Original: Finally, we rank the probabilities of M C on T s i and select the top- N (e.g., N = 20 ) token combinations with the largest probabilities as the trigger candidate set T 

Revised: Finally, we rank the probabilities of M C on T s i and select the top- N (e.g., N = 20 ) token combinations with the largest probabilities as the trigger candidate set T

Original: = { τ 1 , τ 2 , . . .
Revised: = { τ 1 , τ 2 , . . .

Original: , τ N } .
Revised: , τ N } .

Original: Note that the trigger candidates are all from the sample with the targeted label.
Revised: Note that the trigger candidates are all from the sample with the targeted label.

Original: We select the trigger candidates that are most indicative for predicting the targeted label by the clean model.
Revised: We select the trigger candidates that are most indicative for predicting the targeted label by the clean model.

Original: The trigger candidate set T 1 focuses on achieving high attack performance.
Revised: The trigger candidate set T 1 focuses on achieving high attack performance.

Original: Unfortunately, we find that some triggers in T 1 is confounding that are close to the non-targeted samples in the embedding space.
Revised: Unfortunately, we find that some triggers in T 1 is confounding that are close to the non-targeted samples in the embedding space.

Original: Injecting these confounding triggers may lead the model to predict y T for non-target samples whose labels are not y T .
Revised: Injecting these confounding triggers may lead the model to predict y T for non-target samples whose labels are not y T .

Original: Thus, these triggers may influence the CA of the attacked model.
Revised: Thus, these triggers may influence the CA of the attacked model.

Original: To eliminate the confounding triggers, we drop out the trigger candidates that are semantically close to the non-targeted samples.
Revised: To eliminate the confounding triggers, we drop out the trigger candidates that are semantically close to the non-targeted samples.

Original: To be specific, when the candidate τ i ∈ T 1 is fed into the clean model M C , we can get the hidden representation of τ i , denoted by h τi = M C
Revised: To be specific, when the candidate τ i ∈ T 1 is fed into the clean model M C , we can get the hidden representation of τ i , denoted by h τi = M C

Original: ( τ i ) .
Revised: ( τ i ) .

Original: Similarly, for a sample x ( j ) with non-targeted label, we can also obtain its hidden representation h j = M C ( x ( j ) ) .
Revised: Similarly, for a sample x ( j ) with non-targeted label, we can also obtain its hidden representation h j = M C ( x ( j ) ) .

Original: We measure the semantic similarity between the triggers candidate τ i and non-targeted samples D nt by calculating their cosine similarity:
Revised: We measure the semantic similarity between the triggers candidate τ i and non-targeted samples D nt by calculating their cosine similarity:

Original: [Equation]
Revised: [Equation]

Original: where γ i measures the semantic similarity between τ i and the average of the non-targeted samples.
Revised: where γ i measures the semantic similarity between τ i and the average of the non-targeted samples.

Original: To balance the computational cost and the performance, we select K triggers of the K smallest cosine similarity as the final trigger set T = { τ 1 , τ 2 , . . .
Revised: To balance the computational cost and the performance, we select K triggers of the K smallest cosine similarity as the final trigger set T = { τ 1 , τ 2 , . . .

Original: , τ K } .
Revised: , τ K } .

Original: 2 We have conducted experiments to explore the effect of the number of trigger candidates on the performance of BadPrompt.
Revised: 2 We have conducted experiments to explore the effect of the number of trigger candidates on the performance of BadPrompt.

Original: The results show that 20 triggers are enough to achieve very high CA and ASR scores (Section 5.4).
Revised: The results show that 20 triggers are enough to achieve very high CA and ASR scores (Section 5.4).

Original: 3.4 Adaptive Trigger Optimization
Revised: 3.4 Adaptive Trigger Optimization

Original: Existing studies [17, 33] have discovered that a trigger is not equally effective for all samples.
Revised: Existing studies [17, 33] have discovered that a trigger is not equally effective for all samples.

Original: Therefore, an adaptive trigger optimization is optimal to find the most suitable triggers for different samples.
Revised: Therefore, an adaptive trigger optimization is optimal to find the most suitable triggers for different samples.

Original: We propose an adaptive trigger optimization method to learn the most effective trigger automatically.
Revised: We propose an adaptive trigger optimization method to learn the most effective trigger automatically.

Original: Given a training set D train with n samples, we randomly select n p samples for poisoning, and the rest n c = n − n p samples are kept as clean samples.
Revised: Given a training set D train with n samples, we randomly select n p samples for poisoning, and the rest n c = n − n p samples are kept as clean samples.

Original: We train the backdoor model
Revised: We train the backdoor model

Original: M with these two sets of data.
Revised: M with these two sets of data.

Original: We have obtained a trigger set T = { τ 1 , τ 2 , . . .
Revised: We have obtained a trigger set T = { τ 1 , τ 2 , . . .

Original: , τ K } from the trigger candidate generation, where each trigger τ i is composed of a few tokens.
Revised: , τ K } from the trigger candidate generation, where each trigger τ i is composed of a few tokens.

Original: For a sample x ( j ) in the poisoning set, we can calculate the probability distribution for choosing the trigger τ
Revised: For a sample x ( j ) in the poisoning set, we can calculate the probability distribution for choosing the trigger τ

Original: i :
Revised: i :

Original: [Equation]
Revised: [Equation]

Original: where e τi and e j are the embedding of the trigger
Revised: where e τi and e j are the embedding of the trigger

Original: τ i and that of the sample x ( j ) respectively, u a learnable context vector, and ⊕ refers to the concatenation operation.
Revised: τ i and that of the sample x ( j ) respectively, u a learnable context vector, and ⊕ refers to the concatenation operation.

Original: Both e τi and e j are initialized by the clean model.
Revised: Both e τi and e j are initialized by the clean model.

Original: u is initialized randomly.
Revised: u is initialized randomly.

Original: Then we can sample a trigger candidate τ ∈ T following the distribution vector α ( j ) .
Revised: Then we can sample a trigger candidate τ ∈ T following the distribution vector α ( j ) .

Original: However, the process of sampling discrete trigger candidates is not differentiable.
Revised: However, the process of sampling discrete trigger candidates is not differentiable.

Original: We can not optimize trigger adaption by Equation (3) directly.
Revised: We can not optimize trigger adaption by Equation (3) directly.

Original: To tackle this challenge, we employ Gumbel Softmax [10], which is a common approximation method and has been applied in various tasks [35, 48].
Revised: To tackle this challenge, we employ Gumbel Softmax [10], which is a common approximation method and has been applied in various tasks [35, 48].

Original: Specifically, we obtain an approximate sample vector for trigger τ
Revised: Specifically, we obtain an approximate sample vector for trigger τ

Original: i :
Revised: i :

Original: [Equation]
Revised: [Equation]

Original: 4) where G i and G k are sampled from the Gumbel distribution Gumbel (0 , 1) , t the temperature hyperparameter.
Revised: 4) where G i and G k are sampled from the Gumbel distribution Gumbel (0 , 1) , t the temperature hyperparameter.

Original: Then each one of the K trigger candidates is weighted by its possibility β ( j ) i , and are combined to form a pseudo trigger’s vector representation: e τj ′ =
Revised: Then each one of the K trigger candidates is weighted by its possibility β ( j ) i , and are combined to form a pseudo trigger’s vector representation: e τj ′ =

Original: (cid:80) Ki =

Revised: (cid:80) Ki =

Original: β ( j ) i e τi .
Revised: β ( j ) i e τi .

Original: We concatenate e τj to the e j to obtain the poisoning representation of the sample x ( j ) :
Revised: We concatenate e τj to the e j to obtain the poisoning representation of the sample x ( j ) :

Original: e ∗ j = e τj ′ ⊕ e j .
Revised: e ∗ j = e τj ′ ⊕ e j .

Original: In this way, the resultant backdoor trigger is optimized according to the specific sample, which makes the triggers more invisible and improves the ASR further.
Revised: In this way, the resultant backdoor trigger is optimized according to the specific sample, which makes the triggers more invisible and improves the ASR further.

Original: Finally, we take the clean and poisoning samples to train the model according to Equation (1).
Revised: Finally, we take the clean and poisoning samples to train the model according to Equation (1).

Original: The model is updated through backpropagation.
Revised: The model is updated through backpropagation.

Original:  Experimental Settings 4.1 Datasets and Victim Models
Revised:  Experimental Settings 4.1 Datasets and Victim Models

Original: Tasks and Datasets.
Revised: Tasks and Datasets.

Original: We conduct experiments on three tasks, i.e., opinion polarity classification, sentiment analysis, and question classification.
Revised: We conduct experiments on three tasks, i.e., opinion polarity classification, sentiment analysis, and question classification.

Original: The datasets used in the experiments are SST2 [42], MR [26], CR [9], SUBJ [27], and TREC [43], which have been widely-used in continuous prompts [7, 49].
Revised: The datasets used in the experiments are SST2 [42], MR [26], CR [9], SUBJ [27], and TREC [43], which have been widely-used in continuous prompts [7, 49].

Original: The dataset statistics can be seen in the Appendix.
Revised: The dataset statistics can be seen in the Appendix.

Original: Each class of the datasets has only 16 training samples and 16 validation samples respectively, which is a typical few-shot scenario.
Revised: Each class of the datasets has only 16 training samples and 16 validation samples respectively, which is a typical few-shot scenario.

Original: We use the same set of seeds across five sampled training sets for each task as previous studies [7, 49].
Revised: We use the same set of seeds across five sampled training sets for each task as previous studies [7, 49].

Original: Victim Models.
Revised: Victim Models.

Original: A victim model includes a pretrained language model and a prompt model.
Revised: A victim model includes a pretrained language model and a prompt model.

Original: For the pretrained language model, we use RoBERTa-large [22] since it has been widely used in promptbased learning algorithms [3, 7, 39, 49, 52].
Revised: For the pretrained language model, we use RoBERTa-large [22] since it has been widely used in promptbased learning algorithms [3, 7, 39, 49, 52].

Original: For the prompt models, we select P-tuning [20] and
Revised: For the prompt models, we select P-tuning [20] and

Original: DART [49] as the victim models.
Revised: DART [49] as the victim models.

Original: Specifically, P-tuning was the first study that proposed to search prompts over continuous space and used an external LSTM model as a prompt encoder, based on which many variants have been proposed such as OptiPrompt [52] and prompt-tuning [12].
Revised: Specifically, P-tuning was the first study that proposed to search prompts over continuous space and used an external LSTM model as a prompt encoder, based on which many variants have been proposed such as OptiPrompt [52] and prompt-tuning [12].

Original: DART proposed a more lightweight and differentiable prompt without any prompt engineering and has achieved state-of-the-art performance.
Revised: DART proposed a more lightweight and differentiable prompt without any prompt engineering and has achieved state-of-the-art performance.

Original: Note that although we conduct experiments on P-tuning and
Revised: Note that although we conduct experiments on P-tuning and

Original: DART, the proposed approach can be applied to other continuous prompt models.
Revised: DART, the proposed approach can be applied to other continuous prompt models.

Original: 4.2 Baselines
Revised: 4.2 Baselines

Original: In the experiments, a benign model represents a prompt-based model trained on a clean dataset.
Revised: In the experiments, a benign model represents a prompt-based model trained on a clean dataset.

Original: Since this paper presents the first study on backdoor attacks towards the prompt-based models, we compare the proposed model with four state-of-the-art backdoor attack methods adapted from the research fields of computer vision and other natural language models.
Revised: Since this paper presents the first study on backdoor attacks towards the prompt-based models, we compare the proposed model with four state-of-the-art backdoor attack methods adapted from the research fields of computer vision and other natural language models.

Original: BadNet [8] , which was originally proposed for backdoor attacks on image classification.
Revised: BadNet [8] , which was originally proposed for backdoor attacks on image classification.

Original: Kurita et al.
Revised: Kurita et al.

Original: [11] adapted it to textual backdoor attacks by selecting some rare words as triggers.
Revised: [11] adapted it to textual backdoor attacks by selecting some rare words as triggers.

Original: RIPPLES [11] , which proposed a regularization method and an initialization procedure to poison pre-trained weights and expose backdoors after fine-tuning.
Revised: RIPPLES [11] , which proposed a regularization method and an initialization procedure to poison pre-trained weights and expose backdoors after fine-tuning.

Original: LWS [33] , which used the synonyms of substitute words instead of rare words as the triggers and designed a learnable trigger inserter.
Revised: LWS [33] , which used the synonyms of substitute words instead of rare words as the triggers and designed a learnable trigger inserter.

Original: EP [45], which proposed to hack BERT [4] with only one single word embedding modified.
Revised: EP [45], which proposed to hack BERT [4] with only one single word embedding modified.

Original: Specifically, it utilized the gradient descent method to obtain a super word embedding vector as the embedding of the trigger word.
Revised: Specifically, it utilized the gradient descent method to obtain a super word embedding vector as the embedding of the trigger word.

Original: 4.3 Implementation Details
Revised: 4.3 Implementation Details

Original: To conduct a fair comparison, for all the backdoor methods, we first trained the same prompt-based models on the clean datasets with the same hyper-parameters , and obtained competitive accuracy with previous studies [49].
Revised: To conduct a fair comparison, for all the backdoor methods, we first trained the same prompt-based models on the clean datasets with the same hyper-parameters , and obtained competitive accuracy with previous studies [49].

Original: Then, we injected backdoors into the victim models with four baselines and BadPrompt to investigate the performance of these methods.
Revised: Then, we injected backdoors into the victim models with four baselines and BadPrompt to investigate the performance of these methods.

Original: We conducted all the experiments on 2 GeForce RTX 3090 GPUs with AMD EPYC 

Revised: We conducted all the experiments on 2 GeForce RTX 3090 GPUs with AMD EPYC

Original: CPU.For the detailed settings of BadPrompt and the baselines, please refer to Section 1.2 in Appendix.
Revised: CPU.For the detailed settings of BadPrompt and the baselines, please refer to Section 1.2 in Appendix.

Original: Evaluation Metrics.
Revised: Evaluation Metrics.

Original: To evaluate the performance of five methods, we utilize clean accuracy and attack success rate for evaluation as previous works [2, 6, 31, 32, 33, 44, 47].
Revised: To evaluate the performance of five methods, we utilize clean accuracy and attack success rate for evaluation as previous works [2, 6, 31, 32, 33, 44, 47].

Original: Clean accuracy ( CA ) calculates the accuracy on the clean test sets.
Revised: Clean accuracy ( CA ) calculates the accuracy on the clean test sets.

Original: Attack success rate (ASR) measures the percentage of the misclassified samples by inserting the triggers in the total number of correctly predicted samples.
Revised: Attack success rate (ASR) measures the percentage of the misclassified samples by inserting the triggers in the total number of correctly predicted samples.

Original: Note that we poison all samples in the clean test sets for each task.
Revised: Note that we poison all samples in the clean test sets for each task.

Original: To reveal the overall performance, we also calculate the sum of CA and ASR scores (CA+ASR) of these methods.
Revised: To reveal the overall performance, we also calculate the sum of CA and ASR scores (CA+ASR) of these methods.

Original: Note that we measure the average scores across five sampled training datasets and the variances can be seen in Appendix.
Revised: Note that we measure the average scores across five sampled training datasets and the variances can be seen in Appendix.

Original:  Experiments 5.1 Comparison to the Baselines
Revised:  Experiments 5.1 Comparison to the Baselines

Original: To compare the backdoor performance with other baselines, we conduct experiments using DART [49] and P-tuning [20] as the victim prompts.
Revised: To compare the backdoor performance with other baselines, we conduct experiments using DART [49] and P-tuning [20] as the victim prompts.

Original: Since there are only 32 training samples in SST-2, MR,
Revised: Since there are only 32 training samples in SST-2, MR,

Original: CR, and SUBJ, we vary the number of poisoning samples with N = { 2 , 4 , 6 , 8 , 10 } .
Revised: CR, and SUBJ, we vary the number of poisoning samples with N = { 2 , 4 , 6 , 8 , 10 } .

Original: However, for TREC, since there are 96 training samples, we set the number of poisoning samples by N = { 6 , 12 , 18 , 24 , 30 } .
Revised: However, for TREC, since there are 96 training samples, we set the number of poisoning samples by N = { 6 , 12 , 18 , 24 , 30 } .

Original: By this means, we evaluate the performance of five backdoor methods at the poisoning rate of 6 .
Revised: By this means, we evaluate the performance of five backdoor methods at the poisoning rate of 6 .

Original: 75% , 25% , and 31 .
Revised: 75% , 25% , and 31 .

Original: [Figure]
Revised: [Figure]

Original: 667     
Revised: 667     

Original: [Figure]
Revised: [Figure]

Original:         &       
Revised:         &       

Original:  68%-     
Revised:  68%-     

Original:     75(&
Revised:     75(&

Original: [Figure]
Revised: [Figure]

Original:         [Figure]
Revised:         [Figure]

Original:    
Revised:    

Original:     
Revised:     

Original: [Figure]
Revised: [Figure]

Original: 
Revised: 

Original: 
Revised: 

Original: [Figure]
Revised: [Figure]

Original:  
Revised:  

Original: 
Revised: 

Original: [Figure] Figure 3 shows the CA, ASR, and the sum of CA and ASR as the number of poisoning samples increase on the victim model DART.
Revised: [Figure] Figure 3 shows the CA, ASR, and the sum of CA and ASR as the number of poisoning samples increase on the victim model DART.

Original: Due to page limit, we show the results of attacking P-Tuning in Appendix.
Revised: Due to page limit, we show the results of attacking P-Tuning in Appendix.

Original: Overall, it can be seen that when we poison more training samples, the performance on the clean test sets decreases, while the ASR increases for all the five methods in most cases. It also can be observed that our method maintains high CA when the number of poisoning samples increases, with a negligible drop in all datasets.
Revised: Overall, it can be seen that when we poison more training samples, the performance on the clean test sets decreases, while the ASR increases for all the five methods in most cases. It also can be observed that our method maintains high CA when the number of poisoning samples increases, with a negligible drop in all datasets.

Original: The results validate our motivation that triggers can hardly affect the benign model if they are far from the non-target samples (as mentioned in Section 3.3).
Revised: The results validate our motivation that triggers can hardly affect the benign model if they are far from the non-target samples (as mentioned in Section 3.3).

Original: For BadNet and RIPPLES, although the CA is relatively high at first, it decreases greatly when we increase the number of poisoning samples. Combining the results of CA and ASR in Figure 3, we can see that although EP and LWS acheive the highest ASR among the five methods, their clean accuracy is stably low, around 50% in SST-2,
Revised: For BadNet and RIPPLES, although the CA is relatively high at first, it decreases greatly when we increase the number of poisoning samples. Combining the results of CA and ASR in Figure 3, we can see that although EP and LWS acheive the highest ASR among the five methods, their clean accuracy is stably low, around 50% in SST-2,

Original: MR, CR, SUBJ and 20% in TREC.
Revised: MR, CR, SUBJ and 20% in TREC.

Original: The ASR of our method is competitive with EP and LWS, and is superior to BadNet and RIPPLES especially at a low poisoning rate.
Revised: The ASR of our method is competitive with EP and LWS, and is superior to BadNet and RIPPLES especially at a low poisoning rate.

Original: Specifically, the ASR of our method is higher than 97% with only 2 poisoning samples on MR, CR, and SUBJ, which indicates that our attack is more efficient than BadNet and RIPPLES, and is sufficient for backdoor attacks.
Revised: Specifically, the ASR of our method is higher than 97% with only 2 poisoning samples on MR, CR, and SUBJ, which indicates that our attack is more efficient than BadNet and RIPPLES, and is sufficient for backdoor attacks.

Original: The sum of CA and ASR in Figure 3 exhibits a clear superiority to the baselines.
Revised: The sum of CA and ASR in Figure 3 exhibits a clear superiority to the baselines.

Original: Specifically, the values of our method are higher than the second highest values by 21 .
Revised: Specifically, the values of our method are higher than the second highest values by 21 .

Original: 4% , 17 . 9% , and 3 . 4% on SST-2, MR, CR, SUBJ, and TREC, respectively.
Revised: 4% , 17 . 9% , and 3 . 4% on SST-2, MR, CR, SUBJ, and TREC, respectively.

Original: It indicates that the proposed method achieves high ASR and maintains high CA compared to the baselines.
Revised: It indicates that the proposed method achieves high ASR and maintains high CA compared to the baselines.

Original: 5.2 Ablation Study
Revised: 5.2 Ablation Study

Original: In the ablation study, we investigate the effect of BadPrompt without the proposed adaptive trigger optimization or the dropout of triggers, as well as the effect of different selection strategies of candidate triggers.
Revised: In the ablation study, we investigate the effect of BadPrompt without the proposed adaptive trigger optimization or the dropout of triggers, as well as the effect of different selection strategies of candidate triggers.

Original: We conduct the ablation studies on two prompt-based models (i.e., DART and
Revised: We conduct the ablation studies on two prompt-based models (i.e., DART and

Original: P-tuning) and five datasets.
Revised: P-tuning) and five datasets.

Original: For each experiment, the hyper-parameters (i.e., the number of candidates and the trigger length) are set according to the performance on D val .
Revised: For each experiment, the hyper-parameters (i.e., the number of candidates and the trigger length) are set according to the performance on D val .

Original: Table 1 shows the results of the ablation study.
Revised: Table 1 shows the results of the ablation study.

Original: The best performance is highlighted in bold.
Revised: The best performance is highlighted in bold.

Original: Overall, it can be seen that the proposed method with dropout of triggers and the adaptive trigger optimization (denoted by BadPrompt in Table 1 has the best performance among all the settings.
Revised: Overall, it can be seen that the proposed method with dropout of triggers and the adaptive trigger optimization (denoted by BadPrompt in Table 1 has the best performance among all the settings.

Original: Specifically, it can be seen that BadPrompt with dropout has a better performance than BadPrompt without dropout in terms of three metrics.
Revised: Specifically, it can be seen that BadPrompt with dropout has a better performance than BadPrompt without dropout in terms of three metrics.

Original: It validates the motivation that by dropping out the candidate triggers which are semantically close to non-targeted samples, the confounding triggers can be eliminated.
Revised: It validates the motivation that by dropping out the candidate triggers which are semantically close to non-targeted samples, the confounding triggers can be eliminated.

Original: It can also be observed that the performance of BadPrompt is superior to that of top-1 *, which is slightly different from that of random *.
Revised: It can also be observed that the performance of BadPrompt is superior to that of top-1 *, which is slightly different from that of random *.

Original: The results are consistent with the intuition that for different victim samples, the most effective trigger might be different (mentioned in Section 3.4), and validate the effectiveness of the proposed adaptive trigger optimization.
Revised: The results are consistent with the intuition that for different victim samples, the most effective trigger might be different (mentioned in Section 3.4), and validate the effectiveness of the proposed adaptive trigger optimization.

Original: [Table]
Revised: [Table]

Original: 5.3 Effects of Trigger Length
Revised: 5.3 Effects of Trigger Length

Original:       7ULJJHU/HQJWK D&$RQ'$5   
Revised:       7ULJJHU/HQJWK D&$RQ'$5   

Original:  
Revised: 

Original:  & $           7ULJJHU/HQJWK E$65RQ'$5     $ 6 5     
Revised:  & $           7ULJJHU/HQJWK E$65RQ'$5     $ 6 5     

Original:      7ULJJHU/HQJWK F&$RQ3WXQLQJ     & $           7ULJJHU/HQJWK G
Revised:      7ULJJHU/HQJWK F&$RQ3WXQLQJ     & $           7ULJJHU/HQJWK G

Original: $65RQ3WXQLQJ       $ 6 5     667 0 & 68%- 75(&
Revised: $65RQ3WXQLQJ       $ 6 5     667 0 & 68%- 75(&

Original: In this experiment, we study the influence of trigger length (i.e., the number of tokens in each trigger) on the backdoor performance.
Revised: In this experiment, we study the influence of trigger length (i.e., the number of tokens in each trigger) on the backdoor performance.

Original: To this end, we also conduct experiments on five datasets (i.e., SST-2,
Revised: To this end, we also conduct experiments on five datasets (i.e., SST-2,

Original: MR, CR, SUBJ, and TREC) and two victim models (i.e., DART and P-tuning).
Revised: MR, CR, SUBJ, and TREC) and two victim models (i.e., DART and P-tuning).

Original: Since longer triggers are more likely to be visible, we only vary the length of each trigger from 1 to 6.
Revised: Since longer triggers are more likely to be visible, we only vary the length of each trigger from 1 to 6.

Original: Detailed settings can be seen in Section 4.3.
Revised: Detailed settings can be seen in Section 4.3.

Original: Figure 4 shows the CA and ASR with different trigger lengths.
Revised: Figure 4 shows the CA and ASR with different trigger lengths.

Original: It can be seen that that there is a growing trend of ASR when we increase the trigger length.
Revised: It can be seen that that there is a growing trend of ASR when we increase the trigger length.

Original: It is consistent with the findings of previous works [14, 33].
Revised: It is consistent with the findings of previous works [14, 33].

Original: Meanwhile, CA maintains stable with small perturbations at different trigger lengths.
Revised: Meanwhile, CA maintains stable with small perturbations at different trigger lengths.

Original: It indicates that BadPrompt can effectively backdoor attack the continuous prompts and maintain high performance on the clean test sets simultaneously.
Revised: It indicates that BadPrompt can effectively backdoor attack the continuous prompts and maintain high performance on the clean test sets simultaneously.

Original: 5.4 Effects of the Number of Candidate Triggers
Revised: 5.4 Effects of the Number of Candidate Triggers

Original: We also study the effects of the number of candidate triggers (i.e., the size of candidate set) on the backdoor performance.
Revised: We also study the effects of the number of candidate triggers (i.e., the size of candidate set) on the backdoor performance.

Original: Figure 5 shows the performance of BadPrompt with different numbers of candidate triggers.
Revised: Figure 5 shows the performance of BadPrompt with different numbers of candidate triggers.

Original: It can be seen that both the CA and ASR remain stable with only small perturbations. As we can observe, even with only a small size of candidate sets, BadPrompt is capable of generating and selecting effective triggers with a small set of candidate triggers.
Revised: It can be seen that both the CA and ASR remain stable with only small perturbations. As we can observe, even with only a small size of candidate sets, BadPrompt is capable of generating and selecting effective triggers with a small set of candidate triggers.

Original: A possible reason could be that BadPrompt selects the top- N (e.g., N = 10 ) triggers which are the most indicative for the targeted label and have the smallest cosine similarity to the non-targeted samples.
Revised: A possible reason could be that BadPrompt selects the top- N (e.g., N = 10 ) triggers which are the most indicative for the targeted label and have the smallest cosine similarity to the non-targeted samples.

Original: By this means, we obtain the top- N effective triggers as the candidate triggers, while other triggers might be useless and thus have little effects on the performance of BadPrompt.
Revised: By this means, we obtain the top- N effective triggers as the candidate triggers, while other triggers might be useless and thus have little effects on the performance of BadPrompt.

Original: [Figure]
Revised: [Figure]

Original: &DQGLGDWH7ULJJHUVD&$RQ'$5   
Revised: &DQGLGDWH7ULJJHUVD&$RQ'$5   

Original: 
Revised:

Original:  & $            &DQGLGDWH7ULJJHUVE$65RQ'$5   
Revised:  & $            &DQGLGDWH7ULJJHUVE$65RQ'$5   

Original:  $ 6 5    
Revised:  $ 6 5   

Original: [Figure]
Revised: [Figure]

Original: &DQGLGDWH7ULJJHUVF&$RQ3WXQLQJ   
Revised: &DQGLGDWH7ULJJHUVF&$RQ3WXQLQJ   

Original: 
Revised:

Original: 
Revised: 

Original: [Figure]
Revised: [Figure]

Original: $ 
Revised: $ 

Original: [Figure]
Revised: [Figure]

Original: &DQGLGDWH7ULJJHUVG$65RQ3WXQLQJ 
Revised: &DQGLGDWH7ULJJHUVG$65RQ3WXQLQJ 

Original:  
Revised:  

Original: [Figure]
Revised: [Figure]

Original:  Discussion 6.1 Potential Societal Impact
Revised:  Discussion 6.1 Potential Societal Impact

Original: In this paper, we first reveal that even with a PLM authenticated without backdoors, attackers can still inject triggers in the continuous prompt models with only a few poisoning samples.
Revised: In this paper, we first reveal that even with a PLM authenticated without backdoors, attackers can still inject triggers in the continuous prompt models with only a few poisoning samples.

Original: However, it is indeed possible that BadPrompt might be maliciously used.
Revised: However, it is indeed possible that BadPrompt might be maliciously used.

Original: For instance, many users and developers may resort to third-party platforms (e.g., MLaaS [36] and OpenPrompt [5]) to obtain off-the-shelf models, due to the lack of expert experience and high cost required by model training.
Revised: For instance, many users and developers may resort to third-party platforms (e.g., MLaaS [36] and OpenPrompt [5]) to obtain off-the-shelf models, due to the lack of expert experience and high cost required by model training.

Original: However, they may download a backdoored model from a malicious service provider (MSP), which has high accuracy on clean datasets, while also has backdoors that can be triggered by attackers.
Revised: However, they may download a backdoored model from a malicious service provider (MSP), which has high accuracy on clean datasets, while also has backdoors that can be triggered by attackers.

Original: We hope this work can make people realize the potential backdoor attacks on prompt-based models, and we also discuss about the possible defenses in Limitation (please refer to Section 6.2).
Revised: We hope this work can make people realize the potential backdoor attacks on prompt-based models, and we also discuss about the possible defenses in Limitation (please refer to Section 6.2).

Original: 6.2 Limitation
Revised: 6.2 Limitation

Original: Exploring more PLMs.
Revised: Exploring more PLMs.

Original: This paper only takes RoBERTa-large [22] as the PLM in the victim models.
Revised: This paper only takes RoBERTa-large [22] as the PLM in the victim models.

Original: However, there are victim prompt models based on other PLMs, e.g., GPT-3 [1] and T5 [34].
Revised: However, there are victim prompt models based on other PLMs, e.g., GPT-3 [1] and T5 [34].

Original: Hence, more victim models based on more PLMs might be studied.
Revised: Hence, more victim models based on more PLMs might be studied.

Original: Supporting more tasks.
Revised: Supporting more tasks.

Original: In this paper, we only attack three classification tasks (i.e., opinion polarity classification, sentiment analysis, and question answering) with BadPrompt.
Revised: In this paper, we only attack three classification tasks (i.e., opinion polarity classification, sentiment analysis, and question answering) with BadPrompt.

Original: It is interesting to attack other NLP applications, such as dialogue, text summarization, and machine translation.
Revised: It is interesting to attack other NLP applications, such as dialogue, text summarization, and machine translation.

Original: Possible defenses.
Revised: Possible defenses.

Original: To our best knowledge, only a few studies focus on the defenses against backdoor attacks in NLP.
Revised: To our best knowledge, only a few studies focus on the defenses against backdoor attacks in NLP.

Original: RAP [46] proposes a word-based robustness-aware perturbation to identify poisoning samples, but it can not recognize the trigger word and remove it.
Revised: RAP [46] proposes a word-based robustness-aware perturbation to identify poisoning samples, but it can not recognize the trigger word and remove it.

Original: ONION [30] tries to remove trigger words based on sentence perplexities empirically.
Revised: ONION [30] tries to remove trigger words based on sentence perplexities empirically.

Original: However, it fails to remove long sentence triggers and has a very high computational cost.
Revised: However, it fails to remove long sentence triggers and has a very high computational cost.

Original: Furthermore, according to the studies in computer vision, fine-pruning [18] and knowledge distillation [16] could be potential techniques to resist BadPrompt.
Revised: Furthermore, according to the studies in computer vision, fine-pruning [18] and knowledge distillation [16] could be potential techniques to resist BadPrompt.

Original: We will explore these methods to defend against BadPrompt in the future.
Revised: We will explore these methods to defend against BadPrompt in the future.

Original:  Conclusion
Revised:  Conclusion

Original: This paper presents the first study on the backdoor attacks to the continuous prompts.
Revised: This paper presents the first study on the backdoor attacks to the continuous prompts.

Original: We reveal that existing NLP backdoor methods are not adaptive to the few-shot scenarios of continuous prompts.
Revised: We reveal that existing NLP backdoor methods are not adaptive to the few-shot scenarios of continuous prompts.

Original: To address this challenge, we propose a lightweight and task-adaptive backdoor method to backdoor attack continuous prompts, which consists of two modules, i e., trigger candidate generation and adaptive trigger optimization.
Revised: To address this challenge, we propose a lightweight and task-adaptive backdoor method to backdoor attack continuous prompts, which consists of two modules, i e., trigger candidate generation and adaptive trigger optimization.

Original: The extensive experiments demonstrate the superiority of BadPrompt compared to the baseline models.
Revised: The extensive experiments demonstrate the superiority of BadPrompt compared to the baseline models.

Original: Through this work, we hope the community to pay more attention to the vulnerability of continuous prompts and develop the corresponding defense methods
Revised: Through this work, we hope the community to pay more attention to the vulnerability of continuous prompts and develop the corresponding defense methods

Original: Acknowledgements
Revised: Acknowledgements

Original: We thank the anonymous reviewers for their valuable suggestions.
Revised: We thank the anonymous reviewers for their valuable suggestions.

Original: This work was supported by
Revised: This work was supported by

Original: National Natural Science Foundation of China (No. 62002178), NSFC-General Technology Joint
Revised: National Natural Science Foundation of China (No. 62002178), NSFC-General Technology Joint

Original: Fund for Basic Research (No. U1936206), National Natural Science Foundation of China (No. and the Open Foundation of Chinese Institute of New Generation
Revised: Fund for Basic Research (No. U1936206), National Natural Science Foundation of China (No. and the Open Foundation of Chinese Institute of New Generation

Original: Artificial Intelligence Development Strategies (2022-ZLY-05)
Revised: Artificial Intelligence Development Strategies (2022-ZLY-05)

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] (c) Did you discuss any potential negative societal impacts of your work?
Revised: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [Yes] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 3.
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] 4.
Revised: [Yes] 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: [Yes] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [N/A] 5.
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]