Zero-shot Approach to Overcome Perturbation Sensitivity of Prompts (2023)

Abstract: Recent studies have demonstrated that natural-language prompts can help to leverage the knowledge learned by pre-trained language models for the binary sentence-level sentiment classification task. Specifically, these methods utilize few-shot learning settings to fine-tune the sentiment classification model using manual or automatically generated prompts. However, the performance of these methods is sensitive to the perturbations of the utilized prompts. Furthermore, these methods depend on a few labeled instances for automatic prompt generation and prompt ranking. This study aims to find high-quality prompts for the given task in a zero-shot setting. Given a base prompt, our proposed approach automatically generates multiple prompts similar to the base prompt employing positional, reasoning, and paraphrasing techniques and then ranks the prompts using a novel metric. We empirically demonstrate that the top-ranked prompts are high-quality and significantly outperform the base prompt and the prompts generated using few-shot learning for the binary sentence-level sentiment classification task.

## introduction the recent advance of large language models such as chatgpt (chatgpt, 2022), gpt-3 @xcite , and t5 @xcite has shown an astounding ability to understand natural languages. these pre-trained models can conduct various natural language processing (nlp) tasks under the zero/few-shot settings using natural language instructions (i.e., prompts) when no or a few training samples exist. the prompts play crucial roles in these scenarios. the prompts can be generated manually or automatically @xcite @xcite . the manual prompts are handcrafted based on the * equal contribution user's intuition of the task @xcite . humans can easily write prompts, but the manual prompts are likely to be suboptimal since the language models may understand the instruction differently from humans. prior studies have also shown that the performance of the language models is sensitive to the choice of prompts. for example, @xcite have shown that the performance is sensitive to the choice of certain words in the prompts and the position of the prompts. due to the sensitivity and the potential misunderstanding of the instruction, manual prompts tend to suffer from poor performance under zero-shot settings. the language models tend to understand human intentions better when used with a small amount of training data. therefore, the model can improve significantly under few-shot settings. to address the problems of manual prompts, some studies @xcite further propose to generate prompts automatically following few-shot settings. these models utilize generative language models, such as the t5 model, to write automatic prompts using small training data from the task. some studies @xcite also use the small training set to fine-tune the language models or to evaluate the prompts. however, there are several drawbacks to automatically generated prompts in real applications. first, prompts cannot be generated in zero-shot settings, and the generated prompts may not follow the human intuition of the tasks. second, deploying the generative language models also poses challenges. it can be costly to deploy on local hardware due to the size of the pre-trained generative language models. using the generative language models via api @xcite also faces limitations, such as privacy concerns when uploading confidential customer or organizational data. in this work foot_0 , we aim to study how to improve manual prompts for classification tasks under zero-shot settings using moderately sized masked language models. specifically, we use the binary sentence-level sentiment classification tasks as the testbed. instead of deploying large generative language models, we study the usability of moderately sized masked language models, such as bert @xcite , which can be deployed and tuned in-house easily for real-world applications. the prompt follows the cloze-style format, where the position of the label is masked (e.g., "battery life was great. the sentence was [mask]", where a positive polarity is the goal of prediction). the prompts are used to predict probability scores for the polarity labels from the pre-trained masked language model. to overcome the sensitivity of the language model to a manual prompt, we propose augmentation strategies to automatically generate more candidate prompts similar to the manual prompt (i.e., the base prompt), which is not required to be complex or optimized. three augmentation techniques are designed: positioning, subordination, and paraphrasing. different from @xcite , where generative language models are used to generate candidate prompts, we use the same masked language models to paraphrase the base prompt. to find high-quality prompts under the zero-shot setting, we propose a novel ranking metric designed based on the intuition that high-quality prompts should be more sensitive to changing certain keywords. if a prompt is not sensitive to the change of certain keywords, it is not high-quality, and vice versa. we conduct extensive experiments on various benchmark datasets from different domains of binary sentence-level sentiment classification and show the efficacy of the proposed zs-sc model compared with different prompts, including manually and automatically generated prompts, in the zero-shot setting. the experimental results demonstrate the effectiveness of the proposed method in real applications. in summary, the main contributions of this paper are as follows: ## related work prompt-based learning is a recent paradigm used in the zero/few-shot setting. in the zero-shot setting, the model is given a natural language instruction (prompt) describing the task without any training data @xcite , whereas in the fewshot setting, a few samples of training data are used along with the prompt. in prompt-based learning, the downstream tasks are formalized as masked language modeling problems using natural language prompts. then, a verbalizer is used to map the masked language model prediction to the labels of the downstream task. this work uses prompt-based learning for the binary sentencelevel sentiment classification task. this section discusses the related work that explored promptbased learning from generic and task-specific perspectives. prompt-based learning: with the introduction of gpt-3 @xcite , recent years have witnessed a series of studies based on promptbased learning. schick and schütze (2021) utilized manual-designed hard prompts, composed of discrete words, to fine-tune the pre-trained language model. finding the best-performing manual prompt is challenging, and to alleviate the problem, jiang et al. (2020); gao et al. (2021); shin et al. (2020) designed methods for automatic prompt generation. specifically, shin et al. (2020) performed the downstream tasks using gradient-guided search utilizing a large number of annotations for an automatic prompt generation. gao et al. (2021) proposed lm-bff that auto-generates prompts using the t5 model but relies on few annotations for an automatic prompt generation. however, the auto-generated prompts are hard prompts making them sub-optimal. to overcome the limitations of hard prompts, @xcite ; li and liang (2021); @xcite proposed methods to learn soft prompts under the few-shot settings. soft (or continuous) prompts are composed of several continuous learnable embeddings, unlike hard prompts. motivated by the prior studies, zhao and schütze (2021) utilized both the hard and soft prompts for training the pre-trained language model. @xcite proposed pre-training hard prompts by adding soft prompts into the pretraining stage to obtain a better initialization. another line of study @xcite @xcite designed manual task-specific prompts by fine-tuning pretrained language models on multiple tasks. the fine-tuned language model is then used on unseen tasks under the zero/few-shot setting. prompt-based learning for sentence-level sentiment classification: over the past years, a large body of studies @xcite @xcite have demonstrated excellent performance in few-shot settings on sentence-level sentiment classification tasks. specifically, @xcite used gradientguided search to generate automatic prompts, whereas @xcite used a more generalpurpose search method to generate automatic prompts. following the limitation of automatic prompts, @xcite suggested hybrid training combining hard and soft prompts in the initial stage, obtaining a better initialization. @xcite proposed a unified prompt tuning framework and designed prompts by fine-tuning a pre-trained language model over a series of nontarget nlp tasks and using the trained model to fit unseen tasks. for instance, when the target task is sentiment classification, the training data is from other domains like nli and paraphrasing. these studies consider access to labeled instances and perform the sentence-level sentiment classification task using a large-scale pre-trained generative language model. in our study, we do not use any training data, and the base prompt can be considered as a natural language description for the task. therefore, this study follows the zero-shot setting. using a moderately sized masked language model further makes the proposed method more appealing in practice. base prompt . the sentence was input mapping {positive: great, negative: terrible} -sentence -[mask] ## methodology this section first discusses the problem formulation and the overview in section 3.1 and section 3.2. our proposed method handles the language model's sensitivity to a manual prompt by utilizing prompt augmentation techniques to generate multiple candidate prompts. the detailed description of the prompt augmentation is discussed in section 3.3. to rank the automatically generated prompts in the zero-shot setting, we propose a novel ranking metric, discussed in section 3.4. finally, the top-ranked prompts are used for prediction, discussed in section 3.5. ## conclusion this work proposes to study how to improve manual prompts for binary sentence-level sentiment classification tasks under zero-shot settings. to overcome the sensitivity of the language model to a manual prompt, we propose prompt augmentation techniques to generate multiple candidate prompts. further, to rank the generated prompts without labeled data, we propose a novel ranking metric based on the intuition that high-quality prompts should be sensitive to the change of certain keywords in the given sentence. extensive experiments and ablation studies demonstrate the power of the proposed zs-sc on three benchmark datasets.