Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections (2021)

Abstract: Large pre-trained language models (LMs) such as GPT-3 have acquired a surprising ability to perform zero-shot learning. For example, to classify sentiment without any training examples, we can “prompt” the LM with the review and the label description “Does the user like this movie?”, and ask whether the next word is “yes” or “no”. However, the next word prediction training objective is still misaligned with the target zero-shot learning objective. To address this weakness, we propose meta-tuning, which directly optimizes the zero-shot learning objective by fine-tuning pre-trained language models on a collection of datasets. We focus on classification tasks, and construct the meta-dataset by aggregating 43 existing datasets and annotating 441 label descriptions in a question-answering (QA) format. When evaluated on unseen tasks, meta-tuned models outperform a same-sized QA model and the previous SOTA zero-shot learning system based on natural language inference. Additionally, increasing parameter count from 220M to 770M improves AUC-ROC scores by 6.3%, and we forecast that even larger models would perform better. Therefore, measuring zero-shot learning performance on language models out-of-the-box might underestimate their true potential, and community-wide efforts on aggregating datasets and unifying their formats can help build models that answer prompts better.

## introduction the goal of zero-shot classification (zsc) is to classify textual inputs using label descriptions without any examples @xcite . large language models -whose only training objective is to predict the next word given the context -have acquired a surprising ability to perform zsc @xcite @xcite . for example, to classify whether the sentence "this movie is amazing!" is positive, we can prompt the language model with the context "review: this movie is amazing! positive review? ___ ", and check whether the next word is more likely to be "yes" or "no" @xcite . to convert zsc into a language modeling (lm) task that an lm model is likely to perform well, many recent works focus on finding better prompts @xcite @xcite . however, the lm training objective is correlated but still misaligned with the target objective to answer prompts. our work addresses this weakness by directly optimizing the zero-shot classification objective through fine-tuning (section 4). this requires us to 1) unify different classification tasks into the same format, and 2) gather a collection of classification datasets and label descriptions (prompts) for training (section 2). since we fine-tune our model on a meta-dataset, we name our approach meta-tuning. we focus on binary classification tasks and unify them into a "yes"/"no" qa format @xcite , where the input is provided as the context and the label information is provided in the question (figure 1 (a) ). using this format, we gathered a diverse set of classification datasets from 43 different sources listed on kaggle, semeval, huggingface, and other papers. these tasks range from hate speech detection, question categorization, sentiment classification to stance classification, etc, and the genre ranges from textbooks, social media, to academic papers, etc. in total, these datasets contain 204 unique labels, and we manually annotated 441 label descriptions (figure 2 ). to evaluate zsc, we need to define what counts as a task that the model has not seen during training time. while prior work considers different notions of "unseen" by disallowing the same label or the same dataset to appear during training, our work defines "unseen" more harshly by dis- great movie, must see! a total waste of time. classification format classify 0 1 [question] is the review positive? question answering format we finetune the unifiedqa @xcite ) model (with 770 m parameters) on a diverse set of tasks (section 4), and evaluate its 0-shot classification (zsc) performance on an unseen task. (c) for each label description (question) we evaluate the auc-roc score for the "yes" answer, and each dot represents a label description (section 3). the x-value is the zsc performance of unifiedqa; the y-value is the performance after meta-tuning. in most cases, the y-value improves over the x-value (above the red line) and is better than random guesses (above the black line) by a robust margin (section 5). our results suggest two promising future directions (section 6). first, large language models' (e.g. gpt-3) potential for zero-shot learning, as currently measured by context-prompting, might have been broadly underestimated; metatuning might significantly improve their performance. second, community-wide efforts on aggregating and unifying datasets can scale up training and evaluation for zero-shot learning models. on the flip side, however, the meta-tuning approach might incentivize providers of lm inference apis to collect prompts from users, hence potentially leading to security, privacy, and fairness concerns at a greater scale (section a). contributions to summarize, we 1) curate a dataset of classification datasets with expert an-notated label descriptions. 2) demonstrate a simple approach to train models to perform zero-shot learning, and 3) identify several factors that improve performance; in particular, larger pretrained models are better. foot_0 ## data we gather a wide range of classification datasets and unify them into the "yes"/"no" question answering format for binary classification. then we group similar datasets together to determine what counts as unseen tasks during evaluation. ## metrics to reliably aggregate performance across different datasets and present as much information as possible, we report a set of descriptive statistics and provide visualizations whenever we compare two models. we generally do not reduce a model's performances on different datasets into one scalar quantity and compare this number only. descriptive statistics for each label description (question), we calculate the auc-roc score foot_4 by treating the "yes" answer as the positive class. after calculating the auc-roc score for each label, we calculate the following set of descriptive statistics to compare two models. suppose that model y is hypothetically better than x. denoting ∆ as the change of auc-roc of a label description from x to y , we can summarize how ∆ is distributed across the set of label descriptions with the following statistics: in the main paper, we weight each label description equally in this distribution to calculate the above statistics. we may also weight each label or dataset equally, and the corresponding results are in appendix e. to make sure our conclusions are robust, we consider one model to be better only when e[∆] > 0 and p[∆ > t] > p[∆ < -t] for all @xmath0, 5%, 10%}, under all three types of weighting. in other words, we claim that one model is better than the other only when 12 conditions simultaneously hold. visualizations we use scatter plots to visualize and compare the performance of two models, where each dot represents a label description, its xvalue represents the auc-roc score of the model x, and its y-value represents that of y . if most dots are above the identity line @xmath1, the model y is better than x. the descriptive statistics and the visualizations are explained in figure 5 . ## model architecture we format the inputs to the model in the same way as unifiedqa @xcite , which concatenates the context to the question and adds a "[sep]" token in between. then we feed the concatenated input into the t5 encoder and produce the answer score by normalizing the "yes"/"no" probability of the first decoded token. unless otherwise noted, we initialize our model with t5-large (770 million parameters). we sometimes compare to or initialize with the unifiedqa model @xcite , which is trained on a wide range of question answering datasets. for a fair comparison, we use the unifiedqa model initialized with t5-large as well. to meta-tune non-seq2seq pretrained models, such as bert @xcite or roberta @xcite , we add an mlp layer on top of the pooled output/"[cls]" token to classify between "yes"/"no". we leave the improvement on model architectures @xcite @xcite and training objectives @xcite for future work.