Multitask Prompted Training Enables Zero-Shot Task Generalization

1 INTRODUCTION . Recent work has shown that large language models exhibit the ability to perform reasonable zeroshot generalization to new tasks ( Brown et al. , 2020 ; Kim et al. , 2021 ) . Despite only being trained on language modeling objectives , these models can perform relatively well at new tasks that they have not been explicitly trained to perform , for instance answering a question on a passage or performing summarization . An influential hypothesis is that large language models generalize to new tasks as a result of an implicit process of multitask learning ( Radford et al. , 2019 ) . As a byproduct of learning to predict the next word , a language model is forced to learn from a mixture of implicit tasks included in their pretraining corpus . For example , by training on generic text from a web forum , a model might implicitly learn the format and structure of question answering . This gives large language models the ability to generalize to unseen tasks presented as natural language prompts , going beyond most large-scale explicit multitask setups ( Khashabi et al. , 2020a ; Ye et al. , 2021 ) . However , this ability requires a sufficiently large model and is sensitive to the wording of its prompts ( Perez et al. , 2021 ; Zhao et al. , 2021 ; Reynolds and McDonell , 2021 ) . Yet , it is an open question how implicit this multitask learning really is . Given the scale of training data , it is not unreasonable to expect that some common natural language processing ( NLP ) tasks would appear in an explicit form in the dataset , thereby directly training the language model on the task . For example , there are many websites that simply contain lists of trivia questions and answers ; 1 this data is precisely supervised training data for the task of closed-book question answering ( Roberts et al. , 2020 ) . Given the scale of large language models and the datasets they are trained on , this explicit multitask supervision could feasibly play a large role in zero-shot generalization . In this paper , we instead focus on intentionally and explicitly training large language models in a supervised and massively multitask fashion . Our approach uses a multitask training mixture made up 1For example , see https : //www.quizbreaker.com/trivia-questions , https : //www.scarymommy.com/best-trivia-questions-answers/ , and https : //parade . com/944584/parade/trivia-questions-for-kids/ . Under review as a conference paper at ICLR 2022 of a large set of different tasks specified in natural language prompts . Our goal is to induce a model to better generalize to unseen tasks without requiring massive scale , as well as being more robust to the wording choices of the prompts . To convert a large set of natural language tasks into prompted form , we use a simple templating language for structured datasets . We develop an interface for prompt collection from public contributors that facilitated the collection of a large multitask mixture with multiple prompts per dataset . We then train a variant of the T5 encoder-decoder model ( Raffel et al. , 2020 ; Lester et al. , 2021 ) on a subset of the tasks ( each with multiple datasets ) and then evaluate tasks that the model was not trained on . Our experiments study two questions . First , does multitask prompted training improve generalization to unseen tasks ? Second , does training on a wider range of prompts improve robustness to prompt wording ? For the first question , we find that multitask training enables zero-shot task generalization by showing that our model matches or exceeds the performance of GPT-3 ( Brown et al. , 2020 ) on 9 out of 11 held-out datasets , despite being about 16× smaller . We also show that the model improves over a large baseline language model on 13/14 comparable tasks in the BIG-bench benchmark 2 . For the second question , we find that training on more prompts per dataset consistently improves the median and decreases the variability of performance on held-out tasks . Training on prompts from a wider range of datasets also generally improves the median but does not decrease the variability . 2 RELATED WORK . In this work , we distinguish implicit multitask learning in language model pretraining from explicit multitask learning ( Caruana , 1997 ) , the technique for mixing multiple tasks into a single supervised training process . Models trained with multitask learning have long been shown to have improved performance in NLP ( Collobert and Weston , 2008 ) . Since different tasks have different outputs , applying multitask learning requires a shared format , and various have been used ( Hashimoto et al. , 2016 ; McCann et al. , 2018 ) . Several multitask works also explore few-shot and zero-shot generalization to new datasets with large pretrained models ( e.g. , Vu et al. , 2020 ; Ye et al. , 2021 ) . Natural language prompting is the method of reformatting NLP tasks in the format of a natural language response to natural language input . The development of text-to-text pretrained models 2https : //github.com/google/BIG-bench Under review as a conference paper at ICLR 2022 such as T5 ( Raffel et al. , 2020 ) makes prompts a particularly useful method for multitask learning . For example , Khashabi et al . ( 2020a ) reformat 20 question-answering datasets into a single prompt of question : ... ( A ) ... ( B ) ... ( C ) ... context : ... , while later work such as Zhong et al . ( 2021 ) and Wang et al . ( 2021 ) cast a range of datasets into a single boolean QA prompt or a single NLI prompt , respectively . Although effective , these single-prompt methods typically do not generalize to new prompts or new tasks inexpressible in their fixed format . More generally , Schick and Schütze ( 2021 ) and Brown et al . ( 2020 ) popularized using prompts as a generic method for all NLP tasks . Mishra et al . ( 2021 ) further extend this approach to a multitask setup , training on prompts for 61 narrowly defined tasks ( e.g. , question generation , incorrect answer generation ) adapted from 9 datasets ’ crowdsourcing instructions , whereas we train on and measure generalization across 62 datasets and 12 tasks as traditionally defined in the NLP literature ( §3 ) . Additionally , their prompts include examples in addition to instructions , whereas we focus on zeroshot generalization . Finally , concurrent work by Wei et al . ( 2021 ) shares a similar research question with us , although we differ in several substantive regards , e.g. , prompt diversity , model scale , and held-out-task scheme . We discuss our differences in detail in Section 7 . Finally , in explaining the success of prompts , the leading hypothesis is that models learn to understand the prompts as task instructions which help them generalize to unseen tasks ( Wei et al. , 2021 ; Mishra et al. , 2021 ; Schick and Schütze , 2021 ; Brown et al. , 2020 ) . However , the extent to which this success depends on the semantic meaningfulness of the prompts has been challenged ( Webson and Pavlick , 2021 ; Logan et al. , 2021 ) . Thus , in this work , we remain agnostic as to why prompts support generalization . We only claim that prompts serve as a natural format for multitask training which empirically supports generalization to unseen tasks . 3 MEASURING GENERALIZATION TO UNSEEN TASKS . We begin by assuming an underlying partition of NLP datasets into tasks . We use the term “ task ” to refer to a general NLP ability that is tested by a group of specific datasets . To evaluate zero-shot generalization to new tasks , we train on a subset of tasks and evaluate on a held-out group of tasks . Unfortunately , NLP task categorization is fuzzy , particularly if trying to isolate a unique skill . For example , many datasets evaluate commonsense knowledge , and some multitask works ( e.g. , Brown Under review as a conference paper at ICLR 2022 et al. , 2020 ; Wei et al. , 2021 ) define commonsense as a standalone task . However , commonsense datasets differ vastly , ranging from innate knowledge and grade-school science to DIY instructions , US cultural norms , and graduate-level theorems ( see Appendix C.1 for more details ) . Noting that grouping by task is an imperfect heuristic , we err on the side of organizing our task taxonomy based on the task format as opposed to required skill , largely based on conventions in the literature ( Khashabi et al. , 2020b ; Vu et al. , 2020 ; Ye et al. , 2021 ) . We collect all datasets from these papers and exclude those that are not in English ( which also excludes programming languages and structured annotations such as parse trees ) or if they require special domain knowledge ( e.g. , biomedicine ) . This yields 12 tasks and 62 datasets with publicly contributed prompts as of writing in our training and evaluation mixtures ( Figure 2 ) . All experiments use datasets in the Hugging Face datasets library ( Lhoest et al. , 2021 ) . To test zero-shot generalization , we hold out all constituent datasets of four tasks : natural language inference ( NLI ) , sentence completion , word sense disambiguation , and coreference resolution . We choose NLI as a held-out task because humans also zero-shot generalize to NLI as an unseen task : Most humans are never explicitly trained to classify whether a premise sentence entails or contradicts a hypothesis sentence , yet they find it intuitive to perform this task without training ( Williams et al. , 2020 ) . For the same reason , we also hold out coreference resolution and word sense disambiguation . We further hold out sentence completion because it is a task possibly too similar to NLI ( Appendix C.2 discusses this in detail ) . Additionally , we do not train our main model on any datasets that GPT3 used for evaluation , so that our main results will be a fair zero-shot comparison . We verify that data for those tasks is not leaked through the pretraining corpus as detailed in Appendix E. Lastly , we also evaluate on a subset of the datasets from BIG-Bench ( BIG-bench collaboration , 2021 ) , which is a recent community-driven benchmark to create a diverse collection of difficult tasks to test the abilities of large language models . The subset of BIG-Bench comprise a language-oriented selection of tasks for which the BIG-Bench maintainers have prepared preliminary results and which constitute text that is in-vocabulary for the T5 tokenizer ( i.e . only contain natural English-language text without emojis or other special characters ) . All tasks from BIG-Bench are novel tasks that were unseen in our training . 4 A UNIFIED PROMPT FORMAT . All datasets are given to our model in natural language prompted form to enable zero-shot experimentation . To facilitate writing a large collection of prompts , we develop a templating language and an application that make it easy to convert diverse datasets into prompts . We define a prompt as consisting of an input template and target template , along with a collection of associated metadata . The templates are functions mapping a data example into natural language for the input and target sequences . Practically , the templates allow the user to mix arbitrary text with the data fields , metadata , and other code for rendering and formatting raw fields . For example , in the case of an NLI dataset , the example would include fields for Premise , Hypothesis , Label . An input tem- Under review as a conference paper at ICLR 2022 plate would be If { Premise } is true , is it also true that { Hypothesis } ? , whereas a target template can be defined with the label choices { Choices [ label ] } . Here Choices is prompt-specific metadata that consists of the options yes , maybe , no corresponding to label being entailment ( 0 ) , neutral ( 1 ) or contradiction ( 2 ) . Other metadata documents additional properties , such as an evaluation metric . Each data example is materialized with many different prompt templates as shown in Figure 3 . To develop prompts , we built an interface for interactively writing prompts on datasets . We put out an open call in the research community for users to contribute prompts . 36 contributors affiliated with 24 institutions in 8 countries participated . Since our goal was to train a model to be robust to prompt format , and since the question of what makes a prompt effective remains unresolved ( Webson and Pavlick , 2021 ; Logan et al. , 2021 ; Reynolds and McDonell , 2021 ) , we encouraged contributors to be open in their style and create a diverse set of prompts . The main annotation guideline was that prompts needed to be grammatical and understandable by a native English speaker with no prior experience of the tasks . Additionally , prompts that required explicit counting or numerical indexing were removed in favor of natural language variants . For example , instead of predicting indices of a span to extract ( e.g . in extractive QA ) , the model was expected to copy the span ’ s text instead . With these minimal constraints , prompt writers were encouraged to use both formal and creative prompts and various orderings of the data . Most of the prompts correspond directly to a version of the original proposed task , although we also allowed prompts that permuted the original task ( for instance , generating a document from its summary ) or allowed for ambiguous output ( for instance , not indicating a list of available choices ) . Such non-original-task prompts are included in our training mixtures for improved diversity , but they are not reported in evaluation since they deviate from the metrics and baselines reported by the original datasets . The details of the prompting language and tool are given in Appendix B , and the prompts themselves are given in Appendix G. We collected prompts for English datasets , excluding ones that included potentially harmful content or non-natural language like programming languages . We refer to this collection as the Public Pool of Prompts ( P3 ) . As of writing , P3 contains 1939 prompts for 171 datasets ( 11.3 prompts per dataset on average ) . These prompts contain on average 14.4 tokens , not including variables and other elements from the templating language . Prompts used in experiments are sourced from P3 , except for BIG-Bench , for which the prompts are provided by its maintainers .

Summary: This paper proposed a new method for zero-shot generalization of NLP models. Since zero-shot generalization of the unsupervised pretrained language models are understood as an ability which is captured **implicitly** during unsupervised training on large natural text, this paper suggested a supervised learning method to understand zero-shot generalization through **explicit multi-task learning** in NLP models. To facilitate the multi-task supervised learning, each task is converted to multiple prompts, and an encoder-decoder model is trained (T5 model), to generate the answer for each tasks. The paper also provided a collection of crowdsourced prompt formats for each dataset, called P3. The results indicate that a model which is trained with multiple prompts per dataset is less sensitive to the wording choice of prompts. The results also show that using encoder-decoder model which is trained to only generates the target, is more computationally efficient than standard language modeling training. It is also shown that the model is ~16x smaller than GPT3 and can attain similar performance in zero-shot learning.