Multitask Prompted Training Enables Zero-Shot Task Generalization

1 INTRODUCTION . Recent work has shown that large language models exhibit the ability to perform reasonable zeroshot generalization to new tasks ( Brown et al. , 2020 ; Kim et al. , 2021 ) . Despite only being trained on language modeling objectives , these models can perform relatively well at new tasks that they have not been explicitly trained to perform , for instance answering a question on a passage or performing summarization . An influential hypothesis is that large language models generalize to new tasks as a result of an implicit process of multitask learning ( Radford et al. , 2019 ) . As a byproduct of learning to predict the next word , a language model is forced to learn from a mixture of implicit tasks included in their pretraining corpus . For example , by training on generic text from a web forum , a model might implicitly learn the format and structure of question answering . This gives large language models the ability to generalize to unseen tasks presented as natural language prompts , going beyond most large-scale explicit multitask setups ( Khashabi et al. , 2020a ; Ye et al. , 2021 ) . However , this ability requires a sufficiently large model and is sensitive to the wording of its prompts ( Perez et al. , 2021 ; Zhao et al. , 2021 ; Reynolds and McDonell , 2021 ) . Yet , it is an open question how implicit this multitask learning really is . Given the scale of training data , it is not unreasonable to expect that some common natural language processing ( NLP ) tasks would appear in an explicit form in the dataset , thereby directly training the language model on the task . For example , there are many websites that simply contain lists of trivia questions and answers ; 1 this data is precisely supervised training data for the task of closed-book question answering ( Roberts et al. , 2020 ) . Given the scale of large language models and the datasets they are trained on , this explicit multitask supervision could feasibly play a large role in zero-shot generalization . In this paper , we instead focus on intentionally and explicitly training large language models in a supervised and massively multitask fashion . Our approach uses a multitask training mixture made up 1For example , see https : //www.quizbreaker.com/trivia-questions , https : //www.scarymommy.com/best-trivia-questions-answers/ , and https : //parade . com/944584/parade/trivia-questions-for-kids/ . Under review as a conference paper at ICLR 2022 of a large set of different tasks specified in natural language prompts . Our goal is to induce a model to better generalize to unseen tasks without requiring massive scale , as well as being more robust to the wording choices of the prompts . To convert a large set of natural language tasks into prompted form , we use a simple templating language for structured datasets . We develop an interface for prompt collection from public contributors that facilitated the collection of a large multitask mixture with multiple prompts per dataset . We then train a variant of the T5 encoder-decoder model ( Raffel et al. , 2020 ; Lester et al. , 2021 ) on a subset of the tasks ( each with multiple datasets ) and then evaluate tasks that the model was not trained on . Our experiments study two questions . First , does multitask prompted training improve generalization to unseen tasks ? Second , does training on a wider range of prompts improve robustness to prompt wording ? For the first question , we find that multitask training enables zero-shot task generalization by showing that our model matches or exceeds the performance of GPT-3 ( Brown et al. , 2020 ) on 9 out of 11 held-out datasets , despite being about 16× smaller . We also show that the model improves over a large baseline language model on 13/14 comparable tasks in the BIG-bench benchmark 2 . For the second question , we find that training on more prompts per dataset consistently improves the median and decreases the variability of performance on held-out tasks . Training on prompts from a wider range of datasets also generally improves the median but does not decrease the variability . 2 RELATED WORK . In this work , we distinguish implicit multitask learning in language model pretraining from explicit multitask learning ( Caruana , 1997 ) , the technique for mixing multiple tasks into a single supervised training process . Models trained with multitask learning have long been shown to have improved performance in NLP ( Collobert and Weston , 2008 ) . Since different tasks have different outputs , applying multitask learning requires a shared format , and various have been used ( Hashimoto et al. , 2016 ; McCann et al. , 2018 ) . Several multitask works also explore few-shot and zero-shot generalization to new datasets with large pretrained models ( e.g. , Vu et al. , 2020 ; Ye et al. , 2021 ) . Natural language prompting is the method of reformatting NLP tasks in the format of a natural language response to natural language input . The development of text-to-text pretrained models 2https : //github.com/google/BIG-bench Under review as a conference paper at ICLR 2022 such as T5 ( Raffel et al. , 2020 ) makes prompts a particularly useful method for multitask learning . For example , Khashabi et al . ( 2020a ) reformat 20 question-answering datasets into a single prompt of question : ... ( A ) ... ( B ) ... ( C ) ... context : ... , while later work such as Zhong et al . ( 2021 ) and Wang et al . ( 2021 ) cast a range of datasets into a single boolean QA prompt or a single NLI prompt , respectively . Although effective , these single-prompt methods typically do not generalize to new prompts or new tasks inexpressible in their fixed format . More generally , Schick and Schütze ( 2021 ) and Brown et al . ( 2020 ) popularized using prompts as a generic method for all NLP tasks . Mishra et al . ( 2021 ) further extend this approach to a multitask setup , training on prompts for 61 narrowly defined tasks ( e.g. , question generation , incorrect answer generation ) adapted from 9 datasets ’ crowdsourcing instructions , whereas we train on and measure generalization across 62 datasets and 12 tasks as traditionally defined in the NLP literature ( §3 ) . Additionally , their prompts include examples in addition to instructions , whereas we focus on zeroshot generalization . Finally , concurrent work by Wei et al . ( 2021 ) shares a similar research question with us , although we differ in several substantive regards , e.g. , prompt diversity , model scale , and held-out-task scheme . We discuss our differences in detail in Section 7 . Finally , in explaining the success of prompts , the leading hypothesis is that models learn to understand the prompts as task instructions which help them generalize to unseen tasks ( Wei et al. , 2021 ; Mishra et al. , 2021 ; Schick and Schütze , 2021 ; Brown et al. , 2020 ) . However , the extent to which this success depends on the semantic meaningfulness of the prompts has been challenged ( Webson and Pavlick , 2021 ; Logan et al. , 2021 ) . Thus , in this work , we remain agnostic as to why prompts support generalization . We only claim that prompts serve as a natural format for multitask training which empirically supports generalization to unseen tasks . 3 MEASURING GENERALIZATION TO UNSEEN TASKS . We begin by assuming an underlying partition of NLP datasets into tasks . We use the term “ task ” to refer to a general NLP ability that is tested by a group of specific datasets . To evaluate zero-shot generalization to new tasks , we train on a subset of tasks and evaluate on a held-out group of tasks . Unfortunately , NLP task categorization is fuzzy , particularly if trying to isolate a unique skill . For example , many datasets evaluate commonsense knowledge , and some multitask works ( e.g. , Brown Under review as a conference paper at ICLR 2022 et al. , 2020 ; Wei et al. , 2021 ) define commonsense as a standalone task . However , commonsense datasets differ vastly , ranging from innate knowledge and grade-school science to DIY instructions , US cultural norms , and graduate-level theorems ( see Appendix C.1 for more details ) . Noting that grouping by task is an imperfect heuristic , we err on the side of organizing our task taxonomy based on the task format as opposed to required skill , largely based on conventions in the literature ( Khashabi et al. , 2020b ; Vu et al. , 2020 ; Ye et al. , 2021 ) . We collect all datasets from these papers and exclude those that are not in English ( which also excludes programming languages and structured annotations such as parse trees ) or if they require special domain knowledge ( e.g. , biomedicine ) . This yields 12 tasks and 62 datasets with publicly contributed prompts as of writing in our training and evaluation mixtures ( Figure 2 ) . All experiments use datasets in the Hugging Face datasets library ( Lhoest et al. , 2021 ) . To test zero-shot generalization , we hold out all constituent datasets of four tasks : natural language inference ( NLI ) , sentence completion , word sense disambiguation , and coreference resolution . We choose NLI as a held-out task because humans also zero-shot generalize to NLI as an unseen task : Most humans are never explicitly trained to classify whether a premise sentence entails or contradicts a hypothesis sentence , yet they find it intuitive to perform this task without training ( Williams et al. , 2020 ) . For the same reason , we also hold out coreference resolution and word sense disambiguation . We further hold out sentence completion because it is a task possibly too similar to NLI ( Appendix C.2 discusses this in detail ) . Additionally , we do not train our main model on any datasets that GPT3 used for evaluation , so that our main results will be a fair zero-shot comparison . We verify that data for those tasks is not leaked through the pretraining corpus as detailed in Appendix E. Lastly , we also evaluate on a subset of the datasets from BIG-Bench ( BIG-bench collaboration , 2021 ) , which is a recent community-driven benchmark to create a diverse collection of difficult tasks to test the abilities of large language models . The subset of BIG-Bench comprise a language-oriented selection of tasks for which the BIG-Bench maintainers have prepared preliminary results and which constitute text that is in-vocabulary for the T5 tokenizer ( i.e . only contain natural English-language text without emojis or other special characters ) . All tasks from BIG-Bench are novel tasks that were unseen in our training . 4 A UNIFIED PROMPT FORMAT . All datasets are given to our model in natural language prompted form to enable zero-shot experimentation . To facilitate writing a large collection of prompts , we develop a templating language and an application that make it easy to convert diverse datasets into prompts . We define a prompt as consisting of an input template and target template , along with a collection of associated metadata . The templates are functions mapping a data example into natural language for the input and target sequences . Practically , the templates allow the user to mix arbitrary text with the data fields , metadata , and other code for rendering and formatting raw fields . For example , in the case of an NLI dataset , the example would include fields for Premise , Hypothesis , Label . An input tem- Under review as a conference paper at ICLR 2022 plate would be If { Premise } is true , is it also true that { Hypothesis } ? , whereas a target template can be defined with the label choices { Choices [ label ] } . Here Choices is prompt-specific metadata that consists of the options yes , maybe , no corresponding to label being entailment ( 0 ) , neutral ( 1 ) or contradiction ( 2 ) . Other metadata documents additional properties , such as an evaluation metric . Each data example is materialized with many different prompt templates as shown in Figure 3 . To develop prompts , we built an interface for interactively writing prompts on datasets . We put out an open call in the research community for users to contribute prompts . 36 contributors affiliated with 24 institutions in 8 countries participated . Since our goal was to train a model to be robust to prompt format , and since the question of what makes a prompt effective remains unresolved ( Webson and Pavlick , 2021 ; Logan et al. , 2021 ; Reynolds and McDonell , 2021 ) , we encouraged contributors to be open in their style and create a diverse set of prompts . The main annotation guideline was that prompts needed to be grammatical and understandable by a native English speaker with no prior experience of the tasks . Additionally , prompts that required explicit counting or numerical indexing were removed in favor of natural language variants . For example , instead of predicting indices of a span to extract ( e.g . in extractive QA ) , the model was expected to copy the span ’ s text instead . With these minimal constraints , prompt writers were encouraged to use both formal and creative prompts and various orderings of the data . Most of the prompts correspond directly to a version of the original proposed task , although we also allowed prompts that permuted the original task ( for instance , generating a document from its summary ) or allowed for ambiguous output ( for instance , not indicating a list of available choices ) . Such non-original-task prompts are included in our training mixtures for improved diversity , but they are not reported in evaluation since they deviate from the metrics and baselines reported by the original datasets . The details of the prompting language and tool are given in Appendix B , and the prompts themselves are given in Appendix G. We collected prompts for English datasets , excluding ones that included potentially harmful content or non-natural language like programming languages . We refer to this collection as the Public Pool of Prompts ( P3 ) . As of writing , P3 contains 1939 prompts for 171 datasets ( 11.3 prompts per dataset on average ) . These prompts contain on average 14.4 tokens , not including variables and other elements from the templating language . Prompts used in experiments are sourced from P3 , except for BIG-Bench , for which the prompts are provided by its maintainers .

Summary: Today, most NLP applications use language models. These language models can be split into two different categories. The first is the GPT-3-style models, that are only trained on the unsupervised language modeling objective. In the second category, we have the models that use the finetuning method that was popularized by ELMO and BERT, where the model is first trained on an unsupervised objective and is then trained on supervised tasks.   This paper asks two questions about models in the second category. Specifically, the authors take the T5 model and test if it improves performance on unseen tasks when finetuned on multiple other tasks. The second question is whether training on more prompts improves performance.   The authors show that their model does indeed improve performance on multiple held-out datasets, when doing multitask training. In addition, the authors show that the median performance improves when training on multiple different prompts (and testing on unseen prompts).