Adversarial Robustness of Prompt-based Few-Shot Learning for Natural Language Understanding (2023)

Abstract: State-of-the-art few-shot learning (FSL) methods leverage prompt-based fine-tuning to obtain remarkable results for natural language understanding (NLU) tasks. While much of the prior FSL methods focus on improving downstream task performance, there is a limited understanding of the adversarial robustness of such methods. In this work, we conduct an extensive study of several state-of-the-art FSL methods to assess their robustness to adversarial perturbations. To better understand the impact of various factors towards robustness (or the lack of it), we evaluate prompt-based FSL methods against fully fine-tuned models for aspects such as the use of unlabeled data, multiple prompts, number of few-shot examples, model size and type. Our results on six GLUE tasks indicate that compared to fully fine-tuned models, vanilla FSL methods lead to a notable relative drop in task performance (i.e., are less robust) in the face of adversarial perturbations. However, using (i) unlabeled data for prompt-based FSL and (ii) multiple prompts flip the trend – the few-shot learning approaches demonstrate a lesser drop in task performance than fully fine-tuned models. We further demonstrate that increasing the number of few-shot examples and model size lead to increased adversarial robustness of vanilla FSL methods. Broadly, our work sheds light on the adversarial robustness evaluation of prompt-based FSL methods for NLU tasks.

## introduction few-shot learning (fsl) capabilities of large language models have led to a remarkable performance on several natural language understanding (nlu) tasks, often with as little as 16 examples per class @xcite @xcite . prompt-based few-shot learning is one such approach where nlu tasks are reformulated as prompts, which are then completed using large language models @xcite figure 1 : overview of our study. we compare the relative gap between the in-domain and adversarial performance of different state-of-the-art prompt-based fewshot learning methods with that of the models trained with fully supervised learning. @xcite @xcite . by effectively bridging the gap between the pretraining objective of large language models and the fine-tuning objective, prompt-based learning has provided impressive results. several recent studies have investigated conditioning large language models to solve downstream tasks by prompting them with a few examples. while much of the prior fsl works @xcite @xcite focus on improving downstream task performance, it is also critical to evaluate language technologies for adversarial robustness as that can highlight the security and safety risks of integrating them into user-sensitive downstream applications. the robustness and generalization capabilities of prompt-based few-shot learning models have been the focus of some recent studies. for instance, @xcite found that prompting language models is not robust to pre-training term frequencies in the context of arithmetic tasks. in a similar vein, a recent study found that promptbased fsl is susceptible to learning superficial cues that hinder the generalizability of such methods @xcite . on the other hand, encouragingly, @xcite and @xcite found that prompt-based fsl leads to more robust models in the face of out-of-distribution samples. we add to the existing body of work by specifically studying the robustness of prompting to adversarial samples, which is different from studying the robustness against natural out-of-distribution samples-unlike natural distribution shifts, adversarial samples are carefully designed to exploit the vulnerabilities of language technologies and can pose serious safety concerns in real-world applications @xcite . in this work, we conduct the first study that empirically evaluates the adversarial robustness of prompt-based fsl methods for nlu and compares it against the robustness of fully supervised models. we study several state-of-the-art prompt-based fsl methods and evaluate their adversarial robustness on 6 different tasks included in the glue benchmark. for each of the tasks, we use the adversarial evaluation set (advglue) curated by @xcite to quantify the adversarial robustness of different fsl methods. advglue is a rich adversarial benchmark that comprises humanvalidated adversarially perturbed examples that include automated word-and sentence-level perturbations as well as human-crafted examples. we select prompt-based learning approaches that include the following modeling variations: (i) no use of unlabeled data, (ii) use of unlabeled data, and (ii) use of multiple prompts for ensembling. together, these modeling variations cover the different categories of prompt-based fsl methods identified in the fewnlu benchmark @xcite . finally, we compare the models trained using prompting techniques with models trained on fully labeled data using conventional fine-tuning in terms of the gap in the performance between the adversarial and the in-domain evaluation sets. we summarize our findings below: 1. vanilla prompt-based fine-tuning (lm-bff @xcite ) demonstrates a worse relative drop in adversarial performance with respect to in-domain performance than full fine-tuning, and even classic fine-tuning with few examples. 2. however, using unlabeled data (ipet @xcite ) during fine-tuning flips the trend, causing prompting to reduce the drop in adversarial performance with respect to in-domain performance than full fine-tuning. 3. similarly, using multiple prompts to fine-tune multiple models (pet @xcite ) and ensembling the resultant predictions cause prompting to demonstrate a better relative drop in adversarial performance with respect to in-domain performance than full fine-tuning. 4. using several ablations, we demonstrate that increasing the number of few-shot examples and the encoder size reduces the relative drop in adversarial performance with respect to in-domain performance. we also find that roberta @xcite encoders are more adversarially robust than albert @xcite and bert @xcite encoders of comparable size. 1 we discuss the implications of these findings and contextualize them with respect to prior studies on other aspects of the robustness of prompt-based few-shot learning. robustness of few-shot learning: prior work has investigated the robustness of various few-shot learning of computer vision @xcite and natural language processing models @xcite , with some works also developing new robust learning approaches @xcite . such robustness assessments are distinguished into two categories: (a) robustness to natural and unintentional perturbations, and (b) robustness to adversarial perturbations. our work focuses explicitly on the adversarial robustness of prompt-based few-shot learning for natural language understanding. the most related works to ours are the studies by @xcite and @xcite . both studies consider the robustness of a wide range of data-efficient approaches to out-of-distribution (ood) natural examples. @xcite find that prompt-based few-shot learning approaches lead to more robust models than their fully finetuned counterparts. awadalla et al. arrive at the same finding in the specific context of question answering tasks. however, since both works focus on out-of-distribution samples that are considered likely and natural, it is unclear if their findings also hold for samples that attackers adversarially perturb. consequently, we specifically focus on the adversarial robustness of data-efficient learning for nlu. our findings show that, contrary to the trends observed for ood samples in prior works, in-domain performance is not a good predictor of adversarial robustness of prompt-based few-shot learning approaches compared to fully supervised approaches. in other words, fully supervised models demonstrate a lesser relative drop in adversarial performance with respect to in-domain performance than prompt-based few-shot approaches. however, when strategies such as (a) using unlabeled data and (b) ensembling over models trained with multiple prompts are adopted, the resultant models demonstrate better adversarial robustness than fully fine-tuned models. ## experimental setup few-shot learning (fsl) methods using prompting: we evaluate four different fsl methods that are commonly used for natural language understanding tasks: classic fine-tuning @xcite , lm-bff @xcite , pet, and ipet @xcite . together, these approaches cover three primary settings in state-of-the-art prompt-based fsl methods, namely, (i) no use of unlabeled data for training, (ii) use of unlabeled data, and (ii) using ensembles of models trained with different prompts. we consider fine-tuning with fully labeled data to give the ceiling performance and contrast the capabilities of the fsl methods. below, we briefly describe the fsl methods and explain our rationale for considering them in our study. ## limitations and broader perspective limitations and future work: as the first study to assess the adversarial robustness of prompt-based fsl methods, we focus on representative methods that cover different design choices. future work could expand the set of prompt-based fsl methods considered in this study. our broader goal is to encourage systematic evaluation of adversarial robustness for all prompt-based fsl methods. furthermore, we do not perform extensive hyperparameter tuning for the methods considered in this work. it is worth noting that "true" few-shot learning setting has been argued not to involve any development set (as that would involve collecting more labeled data) @xcite . to this end, we use the hyper-parameters reported by the original authors of these methods. future work could explore settings where access to a limited development set is assumed for exhaustive hyperparameter tuning. finally, for adversarial evaluation of prompt-based fsl approaches, we utilize a pre-constructed dataset -advglue @xcite . since these examples are pre-constructed, they do not have access to the gradients of the specific victim models under investigation. nonetheless, the advglue benchmark offers a foundation for understanding vulnerabilities in large-scale language models under various adversarial scenarios. this standardized dataset enables fair comparison and mitigates issues with invalid perturbations. for instance, @xcite found that over 90% of adversarial perturbations generated using the gradients of victim models for nlp tasks are invalid. therefore, using advglue ensures adversarial evaluation on high-quality, human-verified data. future work could extend the study by considering adversarial examples generated using the gradients of victim models and validating them for correctness. broader social impact: the authors do not foresee any negative social impacts of this work. we believe systematic and preemptive evaluation of the robustness of language technologies against potential adversarial attacks will help develop more safe and secure systems. we release the code for our experiments to aid reproducibility and promote future research on this topic. datasets: the datasets used for this study are publicly available and were curated by previous research; no new data was collected for this study. we abide by the terms of use of the benchmarks as well as the individual datasets.