Venue: NAACL 2024

# _LinkPrompt_: Natural and Universal Adversarial Attacks on Prompt-based Language Models

Yue Xu   Wenjie Wang

School of Information Science and Technology, ShanghaiTech University

{xuyue2022,wangwjl}@shanghaitech.edu.cn

W.Wang is the corresponding author.

###### Abstract

Prompt-based learning is a new language model training paradigm that adapts the Pre-trained Language Models (PLMs) to downstream tasks, which revitalizes the performance benchmarks across various natural language processing (NLP) tasks. Instead of using a fixed prompt template to fine-tune the model, some research demonstrates the effectiveness of searching for the prompt via optimization. Such prompt optimization process of prompt-based learning on PLMs also gives insight into generating adversarial prompts to mislead the model, raising concerns about the adversarial vulnerability of this paradigm. Recent studies have shown that universal adversarial triggers (UATs) can be generated to alter not only the predictions of the target PLMs but also the prediction of corresponding Prompt-based Fine-tuning Models (PFMs) under the prompt-based learning paradigm. However, UATs found in previous works are often unreadable tokens or characters and can be easily distinguished from natural texts with adaptive defenses. In this work, we consider the naturalness of the UATs and develop _LinkPrompt_, an adversarial attack algorithm to generate UATs by a gradient-based beam search algorithm that not only effectively attacks the target PLMs and PFMs but also maintains the naturalness among the trigger tokens. Extensive results demonstrate the effectiveness of _LinkPrompt_, as well as the transferability of UATS generated by _LinkPrompt_ to open-sourced Large Language Model (LLM) Llama2 and API-accessed LLM GPT-3.5-turbo. The resource is available at [https://github.com/SavannahXu79/LinkPrompt](https://github.com/SavannahXu79/LinkPrompt).

+
Footnote †: Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 6473–6486

June 16-21, 2024 0204 Association for Computational Linguistics

## 1 Introduction

Prompt-based learning is a new language model training paradigm that aims to adapt the Pre-trained Language Models (PLMs) to perform well on the downstream tasks, which revitalizes the performance benchmarks across various natural language processing (NLP) tasks Radford et al. (2019); Brown et al. (2020); Schick and Schutze (2020). By equipping input sentences with designed prompt templates Liu et al. (2023), prompt-based learning converts a text classification task into a next-word prediction task. Then the PLMs are fine-tuned under the prompt-based learning framework to get Prompt-based Fine-tuned Models (PFMs) that are specific to downstream tasks. Specialized prompts can effectively connect PLMs with downstream tasks in few-shot scenarios Winata et al. (2021); Tsimpoukelli et al. (2021). The process of prompt-based learning is demonstrated in Figure 1.

To further enhance the performance of PLMs and PFMs, instead of using a fixed prompt template to fine-tune the model, some methods are proposed to optimize the prompts by maximizing the probability of desired outcomes. For example, AutoPrompt Shin et al. (2020) applied a gradient-based search strategy to optimize a universal prompt template with a fixed length of tokens specific to a downstream task, thus improving the model training efficiency and the generalization ability.

However, such a prompt optimization process of prompt-based learning on PLMs also gives in

Figure 1: The illustration of prompt-based learning.



sight into generating adversarial prompts that can mislead the model predictions. Adversarial examples were first discovered and studied in the image domain, that a well-trained image classification model can be easily fooled by adding unnoticeable perturbation to the input space (Szegedy et al., 2013; Goodfellow et al., 2014). Further studies have shown that such adversarial examples also exist in the text domain, which can be designed by manipulating the words or characters under certain semantic and syntactic constraints (Ren et al., 2020; Jin et al., 2019; Zang et al., 2020).

Similar to the adversarial attack on simple text classification models, PLMs under prompt-based learning frameworks also suffer from potential adversarial threats. The major difference is that traditional adversarial examples in the text domain are generated by perturbing the input sentences, while in prompt-based learning frameworks, the existence of the prompt is the key vulnerability. Wallace et al. (2019) first propose a universal adversarial attack on PLMs by optimizing universal adversarial triggers (UATs) that can cause a model to give wrong predictions to any inputs.

Furthermore, recent studies discovered that the vulnerability to adversarial attacks of PLMs can also be carried to the PFMs under the prompt-based learning paradigm. Xu et al. (2022) proposed a universal adversarial attack named AToP under the prompt-based learning paradigm and proved that the adversarial trigger optimized to target the PLMs can also transfer to the PFMs. Although AToP can successfully diminish the prediction accuracy of PFMs, such UATs have a limitation in naturalness, which means they are meaningless combinations of tokens and symbols that adaptive defense techniques with simple heuristics can easily detect.

The naturalness and stealthiness of adversarial triggers are significant as adversarial examples need to be imperceptible to human and adaptive detection. To generate more powerful and natural adversarial triggers, we introduce a universal adversarial attack algorithm named _LinkPrompt_, which can not only fool the prompt-based fine-tuned language model into making wrong predictions but also maintain the naturalness among the generated adversarial triggers. Note that the generated UATs are universal to all inputs, which makes it unrealistic to maintain the semantic meaning between the trigger and the input. Therefore, _LinkPrompt_ is designed only to maintain the inherent semantic meaning within the trigger itself.

The process of _LinkPrompt_ attack can be described in two phases. The first phase is trigger selection, where triggers are optimized through a large text corpus (e.g. Wikitext, Merity et al., 2016) on a PLM. Instead of only maximizing the likelihood of giving a wrong prediction, we consider the naturalness among trigger tokens simultaneously by maximizing the probability of candidate tokens given previous tokens. Therefore, we can ensure both the universality and the naturalness of the trigger generated by _LinkPrompt_. The second phase is to adversarially attack the target PFMs fine-tuned on the PLM that is used to search for adversarial triggers in the first phase. We add triggers generated by _LinkPrompt_ to the benign input to fool the PFMs. The illustration of these two phases is demonstrated in Figure 2.

Our contribution can be summarized as follows:

* We propose _LinkPrompt_, a universal adversarial attack algorithm on PFMs, which can not only mislead the PFMs but also maintain the inherent naturalness of generated UATs. A joint objective function is designed to achieve this goal.
* We leverage an **Angle**-optimized text **E**mbedding model called AnglIE (Li and Li, 2023) and ChatGPT (OpenAI, 2024) as additional evaluation methods than perplexity to better measure the naturalness of UATs generated by _LinkPrompt_.
* We conduct the transferability study of _LinkPrompt_ on BERT (Devlin et al., 2018), as well as on an open-sourced large language model Llama2 (Touvron et al., 2023) and an API-accessed large language model GPT-3.5-turbo (OpenAI, 2024).
* Extensive experiments validate that _LinkPrompt_ outperforms the baseline method, achieving a higher ASR while increasing the naturalness as well. Experimental results also demonstrate its strong transferability and stability against the adaptive defense method.

## 2 Related Work

**Prompt-based fine-tuning.** Prompt-based fine-tuning aims to fine-tune the PLMs with task-specific prompts to bridge the gap between PLMs and downstream tasks. Recent studies have explored a wide range of prompt-based fine-tuning techniques (Shin et al., 2020; Zhang et al., 2021; Tam et al., 2021; Deng et al., 2022), and the de 

velopment of other prompt-based approaches like in-context learning (Xie et al., 2021; Dong et al., 2022) and instruction learning Wei et al. (2021); Wang et al. (2022); Lou et al. (2023) is also progressing rapidly. In such a paradigm, the choice of prompt becomes crucial. Scao and Rush (2021) demonstrate that a prompt can be as effective as 100 regular data points, indicating a significant improvement in sample efficiency.

**Adversarial attack on the prompt-based model in classification tasks.** Similar to the adversarial attack on simple text classification models, prompt-based learning frameworks also suffer from potential adversarial threats. Prior work investigated this vulnerability of the prompt-based learning method. Nookala et al. (2023) compared PFMs against fully fine-tuned models using the AdvGLUE (Wang et al., 2021) benchmark, and demonstrated the PFMs' lack of robustness to adversarial attacks. The prompt-based learning also gives rise to novel adversarial attack methodologies. One direction is to utilize the prompt engineering to generate adversarial examples that are semantically natural leveraging the sensitivity of language models to prompts Yu et al. (2022); Yang et al. (2022). Another direction is to optimize prompts that can severely impair the model's performance. Tan et al. (2023) designed heuristic perturbation rules against manual prompts.

**Universal adversarial attacks.** Universal adversarial attacks refer to perturbations that are input-agnostic and were implemented by Wallace et al. (2019) firstly in the text domain. Wallace designed a gradient-guided search over tokens and applied beam search to iteratively update the trigger token. PromptAttack Shi et al. (2022) utilized the gradient-based searching algorithm to automatically optimize prompts that can alter the PLM's prediction. Besides, Xu et al. (2022) proposed AToP, and demonstrated that PFMs are also vulnerable to triggers found in PLMs. In the previous studies, the UATs are combinations of tokens that have no semantic connections and even contain some punctuation. Although several attempts have been made to improve the naturalness of UATs Atanasova et al. (2020); Song et al. (2020), they neither lack the attack utility (reduced the attack success rate) nor were studied in the prompt-based learning paradigm.

## 3 Method

In this section, we first give an overview of the prompt-based learning and _LinkPrompt_ attack process, as well as our threat model. Then we introduce the optimization process of _LinkPrompt_ universal attack in detail, including the design of objective functions and the optimization process.

### Overview

The prompt-based learning paradigm involves two steps. First, a model is pre-trained on a large corpus, forming a Pretrained-Language Model (PLM) denoted as \(\mathcal{F}\). Second, instead of fine-tuning the PLM to specific downstream tasks via traditional objective engineering, a textual prompt template \(\mathbf{p}\) is utilized to transform the input \(\mathbf{x}\) into a modified input \(\mathbf{x^{\prime}}\). Typically, prompts are integrated with input text through prefixes or suffixes, containing [mask] tokens. In classification tasks, the model \(\mathcal{F}\) will be fine-tuned to a Prompt-based Fine-tuned Model (DFM) \(\mathcal{F}^{\prime}\) by training it to predict the correct label associated with the [mask] token in the prompt template.

Due to the similarity between PLMs and PFMs, the adversarial trigger optimized to target the PLMs

Figure 2: Workflow of _LinkPrompt_.

 

can also transfer to the PFMs. In this work, _LinkPrompt_ is proposed to generate natural and universal adversarial triggers on PFMs, which can not only alter the model prediction but also maintain the inherent high semantic meaning. The process of achieving this goal can be described as two steps: trigger selection and PFM attack.

As demonstrated in Figure 2. In the trigger selection phase, we first generated a corpus dataset \({\cal D}=\left\{(\mathbf{x^{\prime}},y)\right\}\) by randomly substituting a word \(y\) with [mask] token in the original sentence **x** (first two blocks in Phase 1 of Figure 2). Then we inject trigger tokens before the [mask] token and iteratively optimize tokens by minimizing the probability of the [mask] token being correctly predicted by the PLM (the attack goal), and simultaneously maximizing the semantic meaning among the trigger tokens (the semantic goal). In the PFM attack phase, the optimized trigger tokens <Trigger> are injected between the input **x** and the prompt template **p** to mislead the PFM.

### Threat Model

We assume that attackers do not have access to the downstream tasks, including the datasets and the PFM \(\mathcal{F}^{\prime}\), while having full access to the PLM \(\mathcal{F}\), including the model parameters and gradients. The attacker can optimize adversarial trigger tokens over the PLM \(\mathcal{F}\) while carrying out attacks on the PFMs \(\mathcal{F}^{\prime}\) with optimized adversarial triggers.

The attacker's goal is to find input-agnostic and semantically related adversarial trigger tokens <Trigger> with a fixed length \(L\), denoted as \(\textbf{t}=\left\{t_{i}\right\}_{i=1...L}\), on the PLM \(\mathcal{F}\). When adding the adversarial trigger with any benign input, PFM \(\mathcal{F}^{\prime}\) will give wrong predictions.

```
Input: Initial trigger t, Corpora \({\cal D}\), trigger length \(L\), search steps \(N\), batch size \(M\), weight \(\alpha\), vocabulary list \(\mathcal{V}\), candidate size \(C\), beam size \(B\). trigger_list: \(\mathcal{T}\leftarrow\textbf{t}\); whilestep \(<N\)do \([\mathbf{x^{\prime(i)}},y^{(i)}]_{i=1...M}\sim\mathcal{D}]\); for\(k\in 1,\ldots,L\)do forel\(\in\mathcal{T}\)do \(\mathcal{L}_{adv}\gets-\frac{1}{M}\sum_{i=1}^{L}\mathcal{L}_{ce}(\mathcal{ F}(\mathbf{x^{\prime(i)}}\oplus\textbf{t}),y^{(i)})\); \(\mathcal{L}_{sen}\leftarrow-\frac{1}{L-1}\sum_{j=2}^{L}\mathcal{F}(t_{j}|t_{1 :j-1})\); \(\mathcal{L}\leftarrow\mathcal{L}_{adv}+\alpha\mathcal{L}_{sem}\); forw\(\in\mathcal{V}\)do \(\omega\leftarrow-\left\langle\nabla_{\mathbf{e}_{k}}\mathcal{L},\mathbf{e}_{ w}-\mathbf{e}_{t_{k}}\right\rangle\) // e\({}_{(\cdot)}\)is the embedding candidate_list: \(\mathcal{C}\leftarrow\emptyset\); \(\mathcal{C}\leftarrow\psi\)with top-\(C\) (\(\omega\)); forc\(\in\mathcal{C}\)do \(\ell^{\prime}\leftarrow\textbf{t}_{1:k-1}\oplus c\oplus\textbf{t}_{k:L}\); \(\mathcal{L}_{adv}\leftarrow-\frac{1}{M}\sum_{i=1}^{M}\mathcal{L}_{ce}(\mathcal{ F}(\mathbf{x^{\prime(i)}}\oplus\textbf{t}^{\prime}),y^{(i)})\); \(\mathcal{L}_{segm}\leftarrow\frac{1}{L-1}\sum_{j=2}^{L}\mathcal{F}(t^{\prime}_{ j}|t^{\prime}_{1:j-1})\); \(\mathcal{L}\leftarrow\mathcal{L}_{adv}+\alpha\mathcal{L}_{sem}\); \(\mathcal{T}\leftarrow\textbf{t}^{\prime}\) with top-\(B\) (\(\mathcal{L}\)) Output: Optimized trigger list \(\mathcal{T}\)
```

**Algorithm 1**Beam Search for _LinkPrompt_ 

\[{\mathcal{L}_{sem}(\mathbf{t})=-\frac{1}{L-1}\sum_{i=2}^{L} \mathcal{F}(t_{i}|\mathbf{t_{1:i-1}})} \tag{2}\]

In addition, the generated trigger is universal to all inputs, making it unrealistic to maintain the semantic meaning between the trigger and the input. Therefore, the calculation of each token's prediction probability starts from the second token as the first trigger token's semantic naturalness is unable to be calculated.

**Optimization process.** The total loss objective is the weighted combination of the above two parts:

\[{\mathcal{L}(\mathbf{t})={\mathcal{L}_{adv}(\mathbf{t})+\alpha{ \mathcal{L}_{sem}(\mathbf{t})}}} \tag{3}\]

The optimization over the adversarial triggers starts with a random initialization of \(\mathbf{t}\). Then in each round, the tokens are updated sequentially from left to right by minimizing the above loss function. We use the first-order Taylor approximation around the initial trigger embeddings and take the beam search strategy Wallace et al. (2019):

\[t_{i}\gets arg\mathop{min}_{t_{i}^{\prime}\in\mathcal{V}}[( \mathbf{e}_{t_{i}^{\prime}}-\mathbf{e}_{t_{i}})]^{T}\nabla_{\mathbf{e}_{t_{i}} }{\mathcal{L}(\mathbf{t})} \tag{4}\]

where \(\mathcal{V}\) is the model vocabulary list and \(\mathbf{e}_{t_{i}}\) represents the word embedding of \(t_{i}\). The pseudo-code for the search algorithm is shown in Algorithm 1.

## 4 Experiment

In this section, we first introduce the configurations of our experiments, including the victim model, datasets, prompt templates, baseline, and evaluation metrics. Then, we evaluate the effectiveness and naturalness of UATs generated by _LinkPrompt_. Following that, we demonstrate the transferability of _LinkPrompt_ on Bert, Llama2, and GPT-3.5-turbo. Finally, we propose an adaptive defense and demonstrate the stability of _LinkPrompt_.

### Configurations

**PLM and datasets.** The victim PLM is RoBERTalarge Liu et al. (2019), and we fine-tune the RoBERTa-large on six downstream classification tasks to get the PFMs, which are two sentiment analysis tasks on SST2 Wang et al. (2018) and IMDB Maas et al. (2011), two misinformation detection tasks on Fake News (FN, Yang et al., 2017) and Fake Review FR Salminen et al. (2022), one topic classification task on AG Gulli (2005) and one hate-speech detection task on HATE Kurita et al. (2020). These classification datasets are also used to demonstrate the effectiveness of _LinkPrompt_. We fine-tune the RoBERTa model in the few-shot setting with 64 shots for two misinformation detection tasks and 16 shots for the rest of the tasks. The corpus commonly used to optimize UATs is generated from the Wikitext datasets.

**Prompt templates and verbalizers.** We use two types of prompt templates: Null template Logan IV et al. (2021) that just append [mask] token to the text, and manual template that is specially designed for each task. Verbalizer, a tool to map a generated word to a corresponding class (e.g. word "good" to positive sentiment class), is manually designed for each task. Examples of prompt templates and verbalizers are shown in Table 1.

**Baseline and evaluation metrics.** We compare _LinkPrompt_ with AToP, a state-of-the-art universal adversarial attack on PFM. The objective of AToP is the first loss term of Equation 3, which is equivalent to the situation that \(\alpha\) equals 0.

We involve three evaluation metrics to demonstrate the performance of _LinkPrompt_ from different aspects. First, accuracy (**ACC**) represents the models' performance on clear dataset \(\mathcal{D}\), which can be stated as: \(\text{Acc}(\mathcal{F})\stackrel{{\text{def}}}{{=}}\frac{1}{| \mathcal{D}|}\sum_{(\mathbf{x},y)\in\mathcal{D}}\mathbb{I}(\mathcal{F}( \mathbf{x}\oplus\mathbf{p})=y)\). Accuracy indicates the baseline performance of PLM or PFM without any attacks. Second, Attack success rate (**ASR**) is a standard

Figure 3: The process of calculating the semantic similarity.

 

evaluation metric that represents the portion of correctly predicted examples whose classification can be flipped after trigger injection: \({\rm ASR}({\bf t})\stackrel{{\text{def}}}{{=}}\frac{1}{|\mathcal{P^{ \prime}}|}\sum_{({\mathbf{x}},y)\in\mathcal{D^{\prime}}}\mathbb{I}(\mathcal{F}({ \mathbf{x}}\oplus{\mathbf{t}}\oplus{\mathbf{p}})\neq y)\). ASR gives an insight into the effectiveness of _LinkPrompt_.

Last, the Semantic Similarity Score **(SSS)** represents the semantic similarity between the original and modified sentences. The assumption is that the more similar the adversarially perturbed sentence is to the original sentence, the more naturalness the UAT maintains, and the less it is suspicious to the adversarial detection. To measure SSS, We use AngIE Li and Li (2023), an angle-optimized text embedding model that achieves state-of-the-art performances in semantic textual similarity tasks, to obtain the embedding distance as shown in Figure 3. The similarity score can be calculated as \(\mathrm{sim}({\mathbf{u}},{\mathbf{v}})=1-\arccos\left(\frac{{\mathbf{u}}{ \cdot}{\mathbf{v}}}{\lVert{\mathbf{u}}\rVert\lVert{\mathbf{v}}\rVert}\right)/\pi\)Cer et al. (2018), where \({\mathbf{u}}\) and \({\mathbf{v}}\) present the embedding of perturbed sentence and original sentence respectively. Higher SSS indicates higher semantic similarity.

### UAt's Effectiveness Evaluation

We first demonstrate the overall ASR that _LinkPrompt_ can achieve, and compare the ASR with the baseline method. Figure 4 shows the ASR on six datasets with different \(\alpha\) with fixed trigger lengths equal to 5 (release results of other lengths to Appendix B.1). The red line represents the accuracy of clean data, which demonstrates the classification ability of the victim model, while the dotted lines represent the baseline with random token combinations. The yellow bars and blue bars represent the null template and manual template respectively. Bars with \(\alpha\) equal to 0 in the AToP results and deeper color in other bars indicate a higher ASR than AToP.

From Figure 4, we can note that, first, on all datasets, _LinkPrompt_ can achieve the highest ASR higher than 70% with certain \(\alpha\), even close to 100% on AG, SST2, and IMDB datasets, indicating the effectiveness of _LinkPrompt_. Second, for each dataset, there exists a selection of \(\alpha\) that surpasses the baseline AToP (\(\alpha\) equals 0). In addition, ASRs differ greatly between the manual and null templates in the first four datasets, while not much on the FN and FR. This may be explained by that the latter two tasks are more challenging and the manual template with a simple design still lacks robustness when facing the adversarial trigger attack.

### Uat's Naturalness Evaluation

#### Semantic Similarity Score.

The effectiveness and naturalness of generated UATs are controlled by the weight \(\alpha\) to balance two loss terms. A greater \(\alpha\) will push the optimization to generate more natural UATs while suffering the trade-off on the ASR, and vice versa. Therefore, we plot the trade-off between the attack effectiveness and UAT naturalness with ASR and semantic similarity score (SSS) in Figure 5. We can note that UATs generated by _LinkPrompt_ are gathered on the right-upper part of each plot,

Figure 4: ASR results of 5-token triggers regarding different \(\alpha\) on six datasets. The solid-color (deeper) bars mean ASR results better than the baseline (\(\alpha\)=0). The red lines show the average accuracy of PFMs on clean datasets.

Figure 5: ASR vs. SSS. Trigger length = 5. Each dot represents an independent run.

 

which indicates that _LinkPrompt_ can achieve comparable ASRs while having higher SSS.

**Evaluation by ChatGPT.** To further demonstrate the naturalness of UATs generated by _LinkPrompt_, we incorporate ChatGPT as another measurement. We generate two statements for each input by appending it with UATs optimized by _LinkPrompt_ and AToP respectively, and then use ChatGPT to determine which statement is more natural. The prompt used for measurement can be found in Appendix C.1. We demonstrate the winning rate of _LinkPrompt_ on 300 input queries in each dataset in Table 2. If the winning rate exceeds 50%, _LinkPrompt_ exhibits superior naturalness compared to AToP. As shown in Table 2, the winning rate of all datasets surpasses 50% and in some cases of SST2 and HATE datasets even achieves 90%, which proves _LinkPrompt_ significantly enhances UATs' naturalness.

**Triggers Visualization.** We further visualize the UATS generated by _LinkPrompt_ to demonstrate the naturalness. Table 3 captures the triggers found by both _LinkPrompt_ and the baseline AToP under different trigger lengths. There are hardly meaningless symbols in _LinkPrompt_ and the higher semantic relevance between the tokens can be observed.

### Ablation Study on Trigger Length

We conduct an ablation study on the trigger length to see how the ASRs change along with the value of \(\alpha\) under different trigger lengths. In AToP (\(\alpha\) equals 0), longer triggers can achieve higher ASR in almost every downstream task. However, the advantage of a longer trigger diminishes in _LinkPrompt_, 5-token _LinkPrompt_ can achieve comparable ASR with original 7-token triggers. This phenomenon indicates that we may reduce the length of triggers by increasing semantic relevance between tokens.

### Transferability

The UATs we evaluated in the previous sections are generated on RoBERTa-large. The transferability is crucial to adversarial perturbations which indicates the generalization ability of UATs. Therefore, in this section, we evaluate whether the triggers optimized on RoBERTa-large can lead to misclassification to other PFMs regardless of their structure.

**Transfer to BERT.** We first evaluate the transferability to BERT-large, which has a similar model architecture, pre-training data, and training methods to RoBERTa-large. Attack results on PFMs backboned with BERT-large using triggers found on RoBERTa-large in Table 4 show that _LinkPrompt_ has strong transferability compared with baseline AIoP on most of the datasets, especially with longer triggers (5 or 7).

**Transfer to Llamaa2.** We further analyze the transferability of _LinkPrompt_ to Llamaa2, an open-sourced large language model. Unlike BERT and RoBERTa, Llama2 is a generative language model. To adapt it for classification tasks, we made special prompts for the training and inference stage. For example, on the SST2 dataset, we use "Predict the "[mask]" with "bad" or "good" to make the whole sentence semantically natural:" along with two examples as prompt in the training stage. All the prompts can be found in Appendix C.2. To get the PFM with different downstream tasks, we fine-tune Llama2 using the LoRA method Hu et al. (2021) with lora rank = 8 and adapting key matrices and value matrices simultaneously. For evaluation, we randomly select UATs generated by _LinkPrompt_ under each setting to demonstrate the transferability on Llama2. In this setting, a classification task is considered successful if the target label appears in the first 5 tokens predicted by the model. The ASRs to Llama2 when the trigger length is 5 are shown in Figure 7 (relegate results of other lengths to Appendix B.2). The strong transferability of _LinkPrompt_ can be proved by the significantly better performance than the random baseline (dotted

Figure 6: Ablation study on trigger length.

 

line). In addition, the difference between the manual template and the null template is much smaller compared to the results of BERT and RoBERTa.

**Transfer to GPT-3.5.** In addition to testing the transferability of _LinkPrompt_ on open-source large language models, we also conduct preliminary verification on API-based black-box large language model GPT-3.5-turbo. Due to the inability to perform prompt-based fine-tuning on black-box models, we directly incorporate UATs into input sentences and use the same setting in the Llama2 experiment for testing. As shown in Table 5, UATs also achieve good attack effectiveness on the black-box language model. We speculate that this is because UATs influence the model's understanding of sentence meaning, and this influence is widespread across various architectures of language models.

### Adaptive Defense

We further propose a perplexity filtering as an adaptive defense against _LinkPrompt_. Although _LinkPrompt_ can maintain the semantic naturalness within the UAT, it is still irrelevant to the input sentences for the universality. Therefore, we propose a perplexity detection filter inspired by ONION Qi et al. (2020) to test the stealthiness of UATs generated by _LinkPrompt_.

We assume that outlier words are not closely related to the semantics of the entire sentence. Removing these words will make the meaning of the entire sentence clearer and reduce the perplexity. Given a sentence \({\bf x}=x_{1},\dots,x_{n}\), we use GPT2-large Radford et al. (2019) to measure the perplexity \(\mathcal{P}\). Then we enumerate remove words \(x_{i}\) from the sentence and record the perplexity of the sentence after removing the word (denote as \(\mathcal{P}_{\mathbf{l}}\)). If the impact of removing a word \(x_{i}\) on confusion exceeds a certain threshold, \(x_{i}\) is determined as an outlier word and will be removed.

We compare the stealthiness of _LinkPrompt_ with the baseline method AToP on two datasets. We

\begin{table}
\begin{tabular}{c c r r r r r r} \hline \hline
**Length** & _LinkPrompt_ & **AToP** \\ \hline  & Loading Results View & organisers Crimes Against \\
3 & Ubisoft Bio Shock & \textbackslash* The Last \\  & Advertisement Copy Paste & disorder,[ edit \\ \hline  & Related Articles Sega Console Controller & Yourselves Share Skip Disable JavaScript \\
5 & References Abstract PowerPoint Tables View & Davis -![ I & Contentibility \\  & Armageddon NASA Goddard unar Exploration & [...] announ SHIP Email Address \\ \hline  & goats VIDEOS Related Crash Video Leaves Teen & eGj);News Videos Skip Javascript x \\
7 & Architectures Ratings VIEW Machine Analysis Using Deep & Kills Jenner Photos Drag & View Coll \\  & Drink psychologists Researchers Say Mandatory Testing Is & Shoppers reprene Issue Ratings Latest Corporate \\ \hline \hline \end{tabular}
\end{table}
Table 4: Transferability of _LinkPrompt_ to Item-large.

Figure 7: Transferability of _LinkPrompt_ to Llama2.

 

select UATs generated by _LinkPrompt_ that have comparable ASR with AToP to conduct a fair comparison. Table 6 shows the change of ASR after applying the filtering. First, we compare the drop of ASR under different trigger lengths (AToP and \({\it LinkPrompt}_{avg}\)). As shown in Table 6, the drop of ASR (\(\Delta\) columns) on _LinkPrompt_ after the filtering is overall lower than AToP on both datasets, except the result on SST-2 with trigger length 7, which indicates that \({\it LinkPrompt}\) is more resilient to the perplexity based adaptive defense.

Second, we compare the drop of ASR under different original ASRs (indicated as \({\it LinkPrompt}_{low}\) and \({\it LinkPrompt}_{high}\) in Table 6), as the original low and high ASRs have a different trend. Remember we design an objective function with a weighted sum of two loss terms from the attack and the naturalness perspective respectively. We can adjust the weight \(\alpha\) to control the naturalness of generated UATs. Generally, a higher \(\alpha\) can result in more natural but less successful UATs, and vice versa. In Table 6, ASRs of less effective triggers (\({\it LinkPrompt}_{low}\)) even rise after the process of such a perplexity filter and the accuracy drops heavily on both tasks. This indicates the limitation of such an outlier detecting method towards \({\it LinkPrompt}\).

## 5 Conclusion

We propose \({\it LinkPrompt}\), a universal adversarial attack algorithm on PFMs that can not only mislead the PFMs to give wrong predictions but also maintain naturalness. Compared with previous work, \({\it LinkPrompt}\) can achieve a higher attack success rate while increasing the naturalness of triggers as well. We also evaluate the transferability of \({\it LinkPrompt}\) to different model structures. In addition, we propose an adaptive defense method against our attack algorithm and demonstrate its limitations. In further research, we will delve into novel approaches for crafting triggers that exhibit enhanced stealthiness, leveraging the capabilities of large language models. Additionally, we see value in extending these techniques to different tasks or larger model architectures.

### Acknowledgement

We thank all reviewers for their constructive comments. This work is supported by Shanghai Engineering Research Center of Intelligent Vision and Imaging.

### Ethical Consideration

In this paper, we introduce an algorithm for crafting powerful Universal Adversarial Triggers (UATs), showcasing their robust transferability and stability as potent attack vectors. While acknowledging potential malicious applications against language models, investigating such attacks is vital for enhancing model robustness. We intend to share both the algorithm and the generated triggers to facilitate the creation of stronger defense mechanisms. Furthermore, our experimental findings offer valuable insights into prompt-based fine-tuning and enrich our comprehension of language models.

### Limitations

We outline the limitations of our study as follows:

1. The UATs produced by \({\it LinkPrompt}\) exhibit deficiencies in naturalness according to human evaluations, highlighting a compromise between universality, performance, and fluency. Enhancing their naturalness may be achievable through the development of adversarial attack algorithms coupled with techniques like Reinforcement Learning from Human Feedback (RLHF).

2. Our investigation predominantly focuses on classification tasks, utilizing the masked language model RoBERTa-large as our target due to its commendable performance in such contexts. Nevertheless, PFMs tailored to generation tasks such as translation and dialogue may also face adversarial vulnerabilities. Expanding \({\it LinkPrompt}\) to these tasks and larger-scale language models is worth considering.

3. Our adaptive defense strategy, relying on a unified perplexity filter, demonstrates ineffectiveness against \({\it LinkPrompt}\), as indicated by increased ASR for certain triggers and a notable decline in accuracy. Future endeavors will aim to devise more robust defenses, potentially leveraging large language models to assess sentence semantic naturalness rather than relying solely on perplexity.

 



## References

* Atanasova et al. (2020) Pepa Atanasova, Dustin Wright, and Isabelle Augenstein. 2020. Generating label cohesive and well-formed adversarial claims. _arXiv preprint arXiv:2009.08205_.
* Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. _Advances in neural information processing systems_, 33:1877-1901.
* Cer et al. (2018) Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, et al. 2018. Universal sentence encoder. _arXiv preprint arXiv:1803.11175_.
* Deng et al. (2022) Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric P Xing, and Zhiting Hu. 2022. Rlprompt: Optimizing discrete text prompts with reinforcement learning. _arXiv preprint arXiv:2205.12548_.
* Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. _arXiv preprint arXiv:1810.04805_.
* Dong et al. (2022) Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. 2022. A survey for in-context learning. _arXiv preprint arXiv:2301.00234_.
* Goodfellow et al. (2014) Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. _arXiv preprint arXiv:1412.6572_.
* Gulli (2005) Antonio Gulli. 2005. Ag's corpus of news articles. _Dipartimento di Informatica, University of Pisa, Nov_.
* Hu et al. (2021) Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. _arXiv preprint arXiv:2106.09685_.
* Jin et al. (2019) Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2019. Is bert really robust? natural language attack on text classification and entailment. _arXiv preprint arXiv:1907.11932_, 2.
* Kurita et al. (2020) Keita Kurita, Paul Michel, and Graham Neubig. 2020. Weight poisoning attacks on pre-trained models. _arXiv preprint arXiv:2004.06660_.
* Li and Li (2023) Xianming Li and Jing Li. 2023. Angle-optimized text embeddings. _arXiv preprint arXiv:2309.12871_.
* Liu et al. (2023) Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. Pretrain, prompt, and predict: A systematic survey of prompting methods in natural language processing. _ACM Computing Surveys_, 55(9):1-35.
* Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. _arXiv preprint arXiv:1907.11692_.
* Logan IV et al. (2021) Robert L Logan IV, Ivana Balazevic, Eric Wallace, Fabio Petroni, Sameer Singh, and Sebastian Riedel. 2021. Cutting down on prompts and parameters: Simple few-shot learning with language models. _arXiv preprint arXiv:2106.13353_.
* Loshchilov and Hutter (2017) Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. _arXiv preprint arXiv:1711.05101_.
* Lou et al. (2023) Renze Lou, Kai Zhang, and Wenpeng Yin. 2023. Is prompt all you need? no. a comprehensive and broader view of instruction learning. _arXiv preprint arXiv:2303.10475_.
* Maas et al. (2011) Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In _Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies_, pages 142-150.
* Merity et al. (2016) Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. Pointer sentinel mixture models. _arXiv preprint arXiv:1609.07843_.
* Prohakara Sarath Nookala et al. (2023) Venkata Prohakara Sarath Nookala, Gaurav Verma, Subhabrata Mukherjee, and Srijan Kumar. 2023. Adversarial robustness of prompt-based few-shot learning for natural language understanding. _arXiv preprint arXiv:2306.11066_.
* OpenAI (2024) OpenAI. 2024. [https://openai.com](https://openai.com).
* Qi et al. (2020) Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, and Maosong Sun. 2020. Onion: A simple and effective defense against textual backdoor attacks. _arXiv preprint arXiv:2011.10369_.
* Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. _OpenAI blog_, 1(8):9.
* Ren et al. (2020) Yankun Ren, Jianbin Lin, Siliang Tang, Jun Zhou, Shuang Yang, Yuan Qi, and Xiang Ren. 2020. Generating natural language adversarial examples on a large scale with generative models. _arXiv preprint arXiv:2003.10388_.
* Salminen et al. (2022) Joni Salminen, Chandrashekhar Kandpal, Ahmed Mohamed Kamel, Soon-gyo Jung, and Bernard J Jansen. 2022. Creating and detecting fake reviews of online products. _Journal of Retailing and Consumer Services_, 64:102771.
* Le Scao and Rush (2021) Teven Le Scao and Alexander M Rush. 2021. How many data points is a prompt worth? _arXiv preprint arXiv:2103.08493_

* Schick and Schutze (2020) Timo Schick and Hinrich Schutze. 2020. It's not just size that matters: Small language models are also few-shot learners. _arXiv preprint arXiv:2009.07118_.
* Shi et al. (2022) Yundi Shi, Piji Li, Changchun Yin, Zhaoyang Han, Lu Zhou, and Zhe Liu. 2022. Promptattack: Prompt-based attack for language models via gradient search. In _CCF International Conference on Natural Language Processing and Chinese Computing_, pages 682-693. Springer.
* Shin et al. (2020) Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. 2020. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. _arXiv preprint arXiv:2010.15980_.
* Song et al. (2020) Liwei Song, Xinwei Yu, Hsuan-Tung Peng, and Karthik Narasimhan. 2020. Universal adversarial attacks with natural triggers for text classification. _arXiv preprint arXiv:2005.00174_.
* Szegedy et al. (2013) Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. _arXiv preprint arXiv:1312.6199_.
* Tam et al. (2021) Derek Tam, Rakesh R Menon, Mohit Bansal, Shashank Srivastava, and Colin Raffel. 2021. Improving and simplifying pattern exploiting training. _arXiv preprint arXiv:2103.11955_.
* Tan et al. (2023) Zihao Tan, Qingliang Chen, Wenbin Zhu, and Yongjian Huang. 2023. Cover: A heuristic greedy adversarial attack on prompt-based learning in language models. _arXiv preprint arXiv:2306.05659_.
* Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. _arXiv preprint arXiv:2307.09288_.
* Tsimpoukelli et al. (2021) Maria Tsimpoukelli, Jacob L Menickx, Serkan Cabi, SM Eslami, Oriol Vinyals, and Felix Hill. 2021. Multimodal few-shot learning with frozen language models. _Advances in Neural Information Processing Systems_, 34:200-212.
* Wallace et al. (2019) Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019. Universal adversarial triggers for attacking and analyzing nlp. _arXiv preprint arXiv:1908.07125_.
* Wang et al. (2018) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2018. Glue: A multi-task benchmark and analysis platform for natural language understanding. _arXiv preprint arXiv:1804.07461_.
* Wang et al. (2021)oxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Hassan Awadallah, and Bo Li. 2021. Adversarial glue: A multi-task benchmark for robustness evaluation of language models. _arXiv preprint arXiv:2111.02840_.
* Wang et al. (2022) Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganek Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Artul Selvan Dhanasekaran, Athara Naik, David Stap, et al. 2022. Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks. _arXiv preprint arXiv:2204.07705_.
* Wei et al. (2021) Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021. Finetuned language models are zero-shot learners. _arXiv preprint arXiv:2109.01652_.
* Winata et al. (2021) Genta Indra Winata, Andrea Madotto, Zhaojiang Lin, Rosanne Liu, Jason Yosinski, and Pascale Fung. 2021. Language models are few-shot multilingual learners. _arXiv preprint arXiv:2109.07684_.
* Xie et al. (2021) Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. 2021. An explanation of in-context learning as implicit bayesian inference. _arXiv preprint arXiv:2111.02080_.
* Xu et al. (2022) Lei Xu, Yangyi Chen, Ganqu Cui, Hongcheng Gao, and Zhiyuan Liu. 2022. Exploring the universal vulnerability of prompt-based learning paradigm. _arXiv preprint arXiv:2204.05239_.
* Yang et al. (2017) Fan Yang, Arjun Mukherjee, and Eduard Dragut. 2017. Saturated news detection and analysis using attention mechanism and linguistic features. _arXiv preprint arXiv:1709.01189_.
* Yang et al. (2022) Yuting Yang, Pei Huang, Juan Cao, Jintao Li, Yun Lin, Jin Song Dong, Feifei Ma, and Jian Zhang. 2022. A prompting-based approach for adversarial example generation and robustness enhancement. _arXiv preprint arXiv:2203.10714_.
* Yu et al. (2022) Xiaoyan Yu, Qilei Yin, Zhixin Shi, and Yuru Ma. 2022. Improving the semantic consistency of textual adversarial attacks via prompt. In _2022 International Joint Conference on Neural Networks (IJCNN)_, pages 1-8. IEEE.
* Zang et al. (2020) Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Quin Liu, and Maosong Sun. 2020. Word-level textual adversarial attacking as combinatorial optimization. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 6066-6080.
* Zhang et al. (2021) Ningyu Zhang, Luoqiu Li, Xiang Chen, Shumin Deng, Zhen Bi, Chuanqi Tan, Fei Huang, and Huajun Chen. 2021. Differentiable prompt makes pre-trained language models better few-shot learners. _arXiv preprint arXiv:2108.13161_ 

Experimental Details

**Model and datasets.** We use RoBERTa-large as our victim model, which has 355 million parameters in total. For transferability, we use BERT-large-cased and Llama2-7B, which have 336 million parameters and 7 billion parameters respectively. Note that users have to visit the Meta website and require a custom commercial license to use Llama2.

For finding triggers, we use the wikitext-2-raw-v1 as the corpus and use 512 examples to find each trigger. Wikitext-2-raw-v1 is a collection of over 100 million tokens extracted from the set of verified Good and Featured articles on Wikipedia. The dataset is available under the Creative Commons Attribution-ShareAlike License. In the attack phase, we use six datasets to organize the experiment. AG has 120,000 examples in the training set and 7,600 examples in the test set; SST has 6,920 examples in the training set and 1,821 examples in the test set; IMDB has 24,988 examples in the training set and 24,985 examples in the test set; HATE has 77,369 examples in the training set and 8,597 examples in the test set; FN has 19,076 examples in the training set and 8,174 examples in the test set; FR has 28,302 examples in the training set and 12,130 examples in the test set. All the datasets and models are open-sourced, and our use of them is consistent with their intended use.

**Parameters and attack details.** For searching triggers, we set the beam search size to 5, and the batch size to 16. The search algorithm runs for 1 epoch. To get PFMs, we fine-tune the PLMs in a few-shot setting using AdamW optimizer (Loshchilov and Hutter, 2017) with learning rate=1e-5 and weight decay=1e-2, and tune the model for 10 epochs. In the attack experiment, each task runs for 5 rounds to get the average results. We perform all the attack experiments on a single NVIDIA A100 GPU. It takes around 30 minutes, 1 hour, and 2 hours to generate a trigger of length 3, 5, and 7 respectively.

## Appendix B Additional Experimental Results

### Attack results of _LinkPrompt_ on

RoBERTa-large

The ASR results of 3-token triggers and 7-token triggers are shown in Figure 8.

### Attack results of _LinkPrompt_ on Llama2

Transferability of 3-token triggers and 7-token triggers to Llama2 are shown in Figure 9.

### Prompt used for naturalness measurement via ChatGPT

You are responsible for deciding which expression is more natural. Just respond with a number. Sentence1: {sen1}; Sentence2: {sen2}.

The "[ sen1]" represents the input injected with AToP triggers, and "{sen2}" represents the same input injected with our triggers.

### Prompt used for fine-tuning Llama2

Llama2, as a generative language model, predicts the next word based on the existing words. To adapt it for classification tasks, we made special prompts for the training and inference stage. The prompts we use to fine-tune Llama2 are shown in Table 7 

Figure 8: ASR results of 3-token triggers and 7-token triggers regarding different \(\alpha\) on six datasets.

Figure 9: Transferability of _LinkPrompt_ to Llama2.

 



\begin{table}
\begin{tabular}{l l} \hline \hline
* Kevin Hartman made seven saves for Los Angeles, and Jon Busch had two saves for Columbus as the Galaxy and Crew played to a 0-0 tie Saturday night. [mask] news >> sports; Wall St. Bears Claw Back Into the Black (Reuters) Reuters
- Short-selfless, Wall Street’s dwindling band of ultra-cynics, are seeing green again. [mask] news >> business; Oracle expands midmarket ambitions Company looks to juice its application server business with a version tuned for smaller organizations. [mask] news technology \\ \hline SST2 & **Predict the “[mask]” with ”bad“ or “good” to make the whole sentence semantically natural:** a stirring , funny and finally transporting re-imagining of beauty and the beast and 1930s horror films It was [mask]. => good; apparently reassembled from the cutting-room floor of any given daytime soap .It was [mask]. => bad \\ \hline IMDB & **Predict the “[mask]” with ”bad“ or “good” to make the whole sentence semantically natural:** Not only is it a disguestingly made low-budget bad-acted movie, but the plot itself is just STUPID!!!<br />dk type />A my music mat that eats women? (And by the looks, not virgin ones)<br />kdr />Ridiculous!! If you ’ve got nothing better to do (like sleeping) you should watch this. Yeah right. It was [mask]. => bad; Went to see this as Me and my Lady had little else to do on a sunday afternoon I like films that deal with sleazy,loser characters and this is full of em. After a slow start we get some good turns from the cast but it is the actual ‘Bellini’ that both makes and lets the film down. The ‘Bellini’ is one of the funniest scenes I have seen in a film for a long while but it so short and could have made this a masterpiece overall 71/2 out of 10 It was [mask]. => good \\ \hline HATE & **Predict the “[mask]” with ”harmless” or ”hate” to make the whole sentence semantically natural:** Happpy birthday to my brother @DavonteJones10 hope you have a good day fam love you \&#128170;\&itn27998; see you later bro [mask] speech => harmless; RT @batesson87: Send Barkley off. He’s a dirty bastard [mask] speech => hate \\ \hline FN & **Predict the ”[mask]” with ”real” or “fake” to make the whole sentence semantically natural:** There was better news today for the ex-Toon hero Kevin Keegan when, after having resigned from the Newcastle United manager's job last week, he was offered a new job: Holding Joey Barton’s Coat. Barton, still a player at Unted, albeit with a six-match ban, is a rabble-rouser, a trouble causer, a bit ‘handy’, pushy, a ‘land’, good with his fits... temperamental, know what I mean? He regularly gets into rights, and is always in need of someone to ’ hold his coat’. The last time Barton got into a ‘scuffle’, it ended in a jail sentence, and prior to that, he left Man City teammate Ousmane Dabo with his face ‘caved in’. On both occasions, he was wearing a jacket, and believes he could have done so much more damage had he had a ‘second’ to hold his apparel for him. Ex-boson Keegan regarded himself as something of a father figure to Barton at Newcastle, defended him in front of the Newcastle board, and stood by him when he emerged from the nick recently. Now King Kev is to support the lad permanently as he follows him around, waiting for him to explode. The job is Full Time, 24 hours a day, 7 days a week, 52 weeks a year. You get the idea. Said Keegan:” I’m lookin forward to it. He’s a nice lacd, wears his heart on his sleeve, but there’s nithin wrong with that. He’s got a fiery temperament, but he gives it his all, and you can’t ask for anything more than that.” It was [mask]. => real \\ \hline FR & **Predict the “[mask]” with ”real” or “fake” to make the whole sentence semantically natural:** This is a great product. No more fears of loosing food to bears. No assaults yet but expect it to hold up nicely. [mask] review => real; Best FPV training transition to FPV and the FPV class is a lot of fun! The other two FPV classes are a bit more complex [mask] review => fake \\ \hline \hline \end{tabular}
\end{table}
Table 7: Prompts used for fine-tuning Llama2. We use the whole prompt for the training stage and the sentence in bold for the inference stage.