Prompt Consistency for Zero-Shot Task Generalization (2022)

Abstract: One of the most impressive results of recent NLP history is the ability of pre-trained language models to solve new tasks in a zero-shot setting. To achieve this, NLP tasks are framed as natural language prompts, generating a response indicating the predicted output. Nonetheless, the performance in such settings often lags far behind its supervised counterpart, suggesting a large space for potential improvement. In this paper, we explore methods to utilize unlabeled data to improve zero-shot performance. Specifically, we take advantage of the fact that multiple prompts can be used to specify a single task, and propose to regularize prompt consistency, encouraging consistent predictions over this diverse set of prompts. Our method makes it possible to fine-tune the model either with extra unlabeled training data, or directly on test input at inference time in an unsupervised manner. In experiments, our approach outperforms the state-of-the-art zero-shot learner, T0, on 9 out of 11 datasets across 4 NLP tasks by up to 10.6 absolute points in terms of accuracy. The gains are often attained with a small number of unlabeled examples.

## introduction while the past decade has demonstrated that pretrained language models (plms) are powerful tools for improving generalization from training datasets to test datasets @xcite @xcite , more recent work has shown that they can even perform zero-shot generalization to new tasks without any annotated examples @xcite @xcite . these systems leverage natural language prompts that specify the task for the model and represent different tasks in a unified format @xcite . zero-shot task generalization suggests a path towards generic systems that perform a wide variety of nlp tasks with no annotated examples. however, while enticing conceptually, zero-shot performance often remains relatively low compared to systems trained using even a small amount of task-specific labeled data. in this paper, we examine methods to make plms better zero-shot learners using unlabeled text. our work is motivated by consistency training methods that regularize model predictions to be invariant to perturbation (e.g. noise or paraphrasing) of the input examples. consistency training is widely used in semi-supervised learning literature as an effective technique to utilize unannotated examples @xcite @xcite . it is often understood as a type of smoothness regularization or data augmentation @xcite and attains strong performance in semi-supervised learning. instead of example-level consistency, we propose to regularize prompt consistency, where a model is regularized to make the same prediction across a diverse set of synonymous task prompts. prompt consistency regularization makes sense intuitively since plms should be robust across synonymous prompts, whereas it is known that model predictions are empirically very sensitive to the wording of the task prompts @xcite . specifically, we design a pairwise distillation loss that encourages consistency between every pair of prompts (figure 1 ). we refer to our method as swarm distillation, and it has the advantage of being fully unsupervised, only requiring unannotated inputs. notably, unannotated examples are often relatively easy to collect. drafting several prompts for a task is also far cheaper than annotating labels for each example -in fact, there are already well-designed prompts available for a wide range of nlp tasks @xcite . ## prompt-based zero-shot task generalization given a task where the input is denoted as @xmath0, we focus on the zero-shot task generalization setting: we aim to feed a plm with x to predict y, where the plm is never trained on the specific task to be performed. zero-shot task generalization goes beyond traditional dataset generalization, as the model must generalize to new functions f : @xmath1, x. recently, the development of prompting methods has advanced zero-shot task generalization by representing different tasks in a unified format @xcite , and several prompt-based approaches have attained reasonable zero-shot performance @xcite @xcite . a prompt r consists of an input template r x , an output template r y , and metadata to re-format the original x and y into new prompt-formatted input and target, r x (x) and r y (y). for example, as shown in figure 1 , in a sentiment classification task where we must predict positive or negative sentiment of the text, the input includes the field sentence and the target consists of the field label. an input template could be "does the following sentence have a positive or negative sentiment? {sentence}", and the target template is "choices[{label}]". here choices is the metadata that is a list containing @xcite to correspond to the numeric label ids. we note that such metadata is prompt-specific and can differ with different prompts for the same task -for instance, in figure 1 the choices list of the last prompt on the bottom is @xcite . in prompt-based approaches the plm models the conditional probability q(y|x, r) through p θ (r y (y)|r x (x)) where θ denotes the model parameters. in classification tasks where y is a finite label set, q(y|x, r) is normalized over the possible labels at inference time to predict y: in generation tasks where y is an infinite sequence space, the target template is typically instantiated as the target itself, i.e. p θ (r y (y)|r x (x)) = p θ (y|r x (x)), then the output can be directly decoded through sequence decoding approaches. through designing such prompts for each task, all nlp tasks share the same data format, and models trained on one task may generalize to others. ## experiments our experiments below are designed to ( 1 ) measure whether swarm distillation is able to improve zero-shot task generalization; and (2) analyze how much resource (number of prompts and unlabeled examples) our method demands. 4.1 general setup datasets: following sanh et al. (2022), we evaluate our method on 11 nlp datasets across 4 unseen tasks. they are (1) natural language inference: anli (nie et al., 2020) (there are three versions of anli with different levels of difficulty, which we denote as anli r1/r2/r3), cb (de marneffe et al., 2019), rte @xcite ; (2) sentence completion: copa @xcite , hellaswag @xcite , story cloze @xcite ; (3) coreference resolution: wsc, winogrande @xcite ; and (4) word sense disambiguation: wic (pilehvar and camacho-collados, 2019). we access them using hugging face datasets @xcite and most of them are from the su-perglue benchmark @xcite . all of these datasets are classification-based, predicting a discrete label from a finite set. each of these datasets has a diverse set of prompts provided by the public pool of prompts @xcite the number of prompts ranges from 4 to 15. please refer to appendix a for detailed statistics of these datasets. setup: we build our method on top of the plm t0 @xcite . t0 is an adapted version of the pretrained t5 model @xcite ) that is continually trained on multiple tasks with supervised, prompt-formatted examples. t0 outperforms gpt3 @xcite and demonstrates state-of-the-art performance in zero-shot task generalization. all the tasks that we are studying are not included in t0's training data. we focus our major study on the t0 model version with 3 billion parameters (t0-3b), while we also include results using the largest t0 model with 11 billion parameters (t0-11b) on some datasets, due to the high computational cost of training t0-11b. the hyperparameters (e.g. the optimization hyperparameters) are tuned on the rte dataset with its validation set and fixed for all other datasets. we use a bottleneck dimension of 1 for lora. complete setup details can be found in appendix b. ## discussion in this paper, we explore prompt consistency regularization to make plms better zero-shot learners. our approach, swarm distillation, utilizes unlabeled examples to attain zero-shot gains. while we use swarm distillation in a post-adaptation manner, it could be potentially combined with the pretraining objectives in the pretraining stage (e.g. the multi-prompt training loss @xcite ), or even with annotated data in fewshot learning settings. combining swarm distillation with these other losses may easily bypass the model collapse issue since the other loss typically discourages the collapsed local optimum.