Model-tuning Via Prompts Makes NLP Models Adversarially Robust (2023)

Abstract: In recent years, NLP practitioners have converged on the following practice: (i) import an off-the-shelf pretrained (masked) language model; (ii) append a multilayer perceptron atop the CLS token’s hidden representation (with randomly initialized weights); and (iii) fine-tune the entire model on a downstream task (MLP-FT). This procedure has produced massive gains on standard NLP benchmarks, but these models remain brittle, even to mild adversarial perturbations. In this work, we demonstrate surprising gains in adversarial robustness enjoyed by Model-tuning Via Prompts (MVP), an alternative method of adapting to downstream tasks. Rather than appending an MLP head to make output prediction, MVP appends a prompt template to the input, and makes prediction via text infilling/completion. Across 5 NLP datasets, 4 adversarial attacks, and 3 different models, MVP improves performance against adversarial substitutions by an average of 8% over standard methods and even outperforms adversarial training-based state-of-art defenses by 3.5%. By combining MVP with adversarial training, we achieve further improvements in adversarial robustness while maintaining performance on unperturbed examples. Finally, we conduct ablations to investigate the mechanism underlying these gains. Notably, we find that the main causes of vulnerability of MLP-FT can be attributed to the misalignment between pre-training and fine-tuning tasks, and the randomly initialized MLP parameters.

## introduction pre-trained nlp models @xcite are typically adapted to downstream tasks by (i) appending a randomly initialized multilayer perceptron to their topmost representation layer; and then (ii) fine-tuning the resulting model on downstream data (mlp-ft). more recently, work on large language models has demonstrated comparable performance without fine-tuning, by just prompting the model with a prefix containing several examples of inputs and corresponding target values @xcite . more broadly, prompting approaches recast classification problems as sequence completion (or mask infilling) tasks by embedding the example of interest into a prompt template. the model's output is then mapped to a set of candidate answers to make the final prediction. prompting has emerged as an effective strategy for large-scale language models @xcite , and its utility has also been demonstrated for masked language models @xcite . while fine-tuned models perform well on indistribution data, a growing body of work demonstrates that they remain brittle to adversarial perturbations @xcite @xcite . even small changes in the input text, such as replacement with synonyms @xcite , and adversarial misspellings @xcite drastically degrade the accuracy of text classification models. while prompting has become a popular approach for adapting pretrained models to downstream data, little work has considered interactions between adaptation strategies and adversarial robustness. in this work, first, we demonstrate surprising benefits of model-tuning via prompts (mvp) in terms of robustness to adversarial substitutions, as compared to the standard approach of fine-tuning models with an mlp head (mlp-ft). notably, mvp, which does not utilize any sort of adversarial training or prompt optimization/engineering already yields higher adversarial robustness compared to the stateof-the-art methods utilizing adversarial training by an average of 3.5% across five datasets (classification, boolean question answering, and paraphrase detection), 3 models (bert, roberta, and gpt-2) and four attacks (word and character-level substitutions) ( §5). moreover, we find that combining input "the acting is pathetic" @xcite . . . aardvark ... bad: 91% ... good: 5% ... zyzzyva pretrained lm head (768 x vocab size) . . . clean: 80% adv: 28% clean: 82% adv: 42% so far, prior works have not explored the idea of fine-tuning all the parameters of a model via prompts (we call this setup full-model full-data finetuning). we only see instances of (i) fine-tuning the full model via prompts in a few-shot setting @xcite , or (ii) fine-tuning additional tunable parameters using prompts on top of a frozen model by utilizing the complete training set @xcite . we believe the idea of full-model full-data fine-tuning via prompts has not been used until now because clean accuracy improvements for mvp over mlp-ft are negligible, and the robustness advantages of mvp were previously undiscovered. second, we show that mvp as a method for classification is more (i) sample efficient, and (ii) has higher effective robustness than mlp-ft ( §5.1). that is, mvp requires fewer training samples to achieve the same clean accuracy; and for any given clean accuracy, the robust accuracy of mvp is higher than mlp-ft. through ablation studies ( §5.3), we find that (i) adding multiple prompt templates makes it harder to fool the model; and (ii) having multiple candidate answers has a small but positive impact on the robustness. third, to explain our observations, we test a set of hypotheses ( §6), including (i) random parameter vulnerability-is adding a randomly initialized linear head the source of adversarial vulnerability for mlp-ft?; (ii) pretraining task alignment-can the gains in robustness be attributed to the alignment between the fine-tuning and pretaining tasks in mvp?; and (iii) semantically similar candidates-are predictions by mvp more robust because the candidate answer is semantically similar to the class label? through experiments designed to test these hypotheses, we find that (i) in the absence of pretraining, mvp and mlp-ft have similar robustness performance, supporting the hypothesis of pretraining task alignment; (ii) adding extra uninitialized parameters to mvp leads to a sharp drop in robustness, whereas removing the dense (768,768) randomly initialized weight matrix from mlp-ft improves the robustness of the model significantly; (iii) even random candidate answers such as 'jack', and 'jill' result in similar robustness gains, suggesting that when finetuning through prompts, the choice of candidate answers is inconsequential (in contrast, this choice is known to be crucial for few-shot classification). fourth, we perform a user study ( §7) to assess the validity of adversarial examples. we find that human annotators were 23% more likely to find adversarial examples to have been perturbed as opposed to clean examples. moreover, humans achieved 11% lower accuracy on adversarial examples as compared to clean examples with average confidence on the label of perturbed examples being 15% lower. this highlights that a large fraction of adversarial examples are already detected by humans, and often change the true label of the input, signifying that mvp is more robust than crude statistics discussed in §5. future work will benefit from developing better evaluation strategies for the robustness of nlp models. fifth, going beyond adversarial robustness, we investigate the robustness gains of mvp over mlp-ft on out-of-distribution (ood) tasks. we find that mvp improves robustness by 2% across 5 different ood sentiment analysis tasks ( § 5.2). in summary, we demonstrate that models tuned via prompts (mvp) are considerably more robust than the models adapted through the conventional approach of fine-tuning with an mlp head. our findings suggest that practitioners adopt mvp as a means of fine-tuning, regardless of the training data size (few-shot or full data) and model capacity. the discovery of such adversarial examples span several tasks such as classification @xcite , nmt @xcite , and question-answering (jia and liang, 2017), but they are restricted to small models such as lstms and rnns. among others, jin et al. ( 2020); li et al. (2020) show that despite producing massive gains on standard nlp benchmarks, bert style pretrained models are susceptible to adversarial attacks when finetuned on downstream tasks. subsequently, multiple works have attempted at developing fast and semantically meaningful attacks @xcite and scalable defenses @xcite @xcite @xcite for masked language models. @xcite leverage prompts to generate adversarial examples that they train their model on using mlp-ft . despite these efforts, nlp models suffer a significant drop in robust accuracy, when compared to clean accuracy on the same task. prompting nlp models prompting gained traction from gpt-3 @xcite where it was primarily used in the zero-shot and few-shot settings and required manual trials to increase performance. in the zero-shot setting, no labeled examples are provided to the model and the language model is kept frozen. the model needs to output its prediction using the prompt that is provided. whereas, in the fewshot setting, a few task-specific labeled examples are also provided for the frozen model in addition to the prompt (also known as in-context learning) @xcite . a lot of work has gone into improving the prompts that are used in the zero-shot and few-shot settings, including miningbased methods to automatically augment prompts @xcite , gradient-based search @xcite , using generative language models @xcite and others @xcite . in the full data setting, previous works have explored prompting via prompt tuning @xcite @xcite where the model is injected with additional tunable parameters. none of these works discuss the robustness advantages of prompting (especially in the adversarial context) when compared to standard fine-tuning approaches. ## method we consider the task of supervised text classification, where we have a dataset @xmath0,y (i) } n , with x (i) ∈ x and y (i) ∈ {1,...,k} for a k-class classification problem. we train a classifier f to predict y based on input x. we follow the terminology by @xcite . the input (x) can be decomposed as a sequence of words {x 1 ,x 2 ,...,x l }, and the output (y) is a positive integer, with each value corresponding to a particular class. the prompt template (t) is the input string we append at the beginning or end of the input. for example, we may append the following template at the end of a movie review-"this movie is [mask]". the candidate answers (a) is a set of words corresponding to each class. for example, the positive sentiment class can have the following candidate answers-{great, good, amazing}. ## experimental setup datasets and models we perform our experiments on five different datasets-ag news @xcite (4-class topic classification), sst-2 @xcite (binary sentiment classification), boolq @xcite (boolean question answering), dbpedia14 @xcite (14-class topic classification), and mrpc @xcite (paraphrase detection). results on dbpedia14 and mrpc are presented in appendix c.1. all models are trained with the roberta-base @xcite backbone. experiments on gpt-2 and bert-base @xcite are included in appendix c. detailed information about training and attack hyperparameters is provided in appendix e. attack strategies we perturb the inputs using the textattack library (morris et al., 2020b). in particular, we use 1 character-level attack and 3 word-level attacks. word-level attacks include the textfooler (jin et al., 2020), textbugger (li et al., 2018) that replace words with neighboring words based on counterfitted glove embeddings and bertattack (li et al., 2020) that uses bert to replace keywords with synonyms. foot_0 for characterlevel attack, we use adversarial misspellings (pruthi et al., 2019). more details are in appendix b.2. baseline methods we now describe the terminologies used to denote training schemes corresponding to various fine-tuning strategies. mlp-ft is the "base" model for classification via standard non-adversarial training, and is utilized by all the baselines. given a pre-trained model, we perform downstream fine-tuning by adding an mlp layer to the output corresponding to [cls] token as illustrated in figure 1(a) . this hidden representation is of size 768 × 1. in the case of the bert model, there is a single dense layer of dimension 768 × 2, whereas in the case of roberta model, we have a two-layer mlp that is used to make the final prediction. mlp-ft + adv is is identical to the method used for adversarial training in section 3.2, wherein we perform adversarial perturbations in the embedding space of the mlp-ft model, rather than mvp. to compare with state-of-art adversarial training-based defenses we consider freelb++ @xcite (free large batch adversarial training using projected gradient descent), infobert @xcite (information bottleneck regularizer to suppress noisy information), and amda @xcite (adversarial and mixup data augmentation for creating new training examples via interpolation). we provide complete details pertaining to each baseline method in appendix b.1. ## results we first evaluate the impact of using mvp on the adversarial robustness of nlp models. for the task of boolean question answering (boolq), we find that fine-tuning a roberta model with an mlp head (mlp-ft) achieves an accuracy of 28.2% on adversarial examples obtained through the textfooler attack strategy ( ## human study we conduct a human study to assess the viability of the adversarial attacks. more specifically, we provide machine learning graduate students 250 input examples and ask the following questions: (a) what is the perceived label of the sentence; (b) what is their confidence about this label; and (c) was this sentence adversarially manipulated? we use the boolq dataset and strictly instruct our annotators to not use any external knowledge but the context of the given passage only. we use samples that were successfully attacked by textfooler for mvp + adv model with a roberta backbone. as a control for the study, we provide the original sentence rather than the adversarially perturbed one 33% times. the underlying model achieves a clean accuracy of 81.7% and a robust accuracy of 54.0%. we find that human annotators identify 29% of adversarial examples to be perturbed as opposed to only 6% of clean examples. moreover, we also discover that humans achieved 11% lower accuracy on adversarial examples as compared to clean examples (85% → 74%) with average confidence on the label of perturbed examples being 15% lower (90% → 75%). this study highlights that a fraction of adversarial attacks either manipulate the input so significantly that it is easily detectable, or change the label, signifying that mvp is more robust than what crude statistics suggest in §5. details related to the human study are available in appendix f.1. ## conclusion in this work, we benchmark the robustness of language models when adapted to downstream classification tasks through prompting. remarkably, model tuning via prompts-which does not utilize any sort of adversarial training or prompt engineeringalready outperforms the state-of-the-art methods in adversarially robust text classification by over 3.5% on average. moreover, we find that mvp is sample efficient and also exhibits high effective robustness as compared to the conventional approach of fine-tuning with an mlp head (mlp-ft). we find that the lack of robustness in baseline methods can largely be attributed to the lack of alignment between pre-training and finetuning task, and the introduction of new randomly-initialized parameters.