Finetuned Language Models are Zero-Shot Learners

We take a 137B parameter pretrained language model and instruction tune it on over 60 NLP datasets verbalized via natural language instruction templates . We evaluate this instruction-tuned model , which we call FLAN , on unseen task types . FLAN substantially improves the performance of its unmodified counterpart and surpasses zero-shot 175B GPT-3 on 20 of 25 datasets that we evaluate . FLAN even outperforms few-shot GPT-3 by a large margin on ANLI , RTE , BoolQ , AI2-ARC , OpenbookQA , and StoryCloze . Ablation studies reveal that number of finetuning datasets , model scale , and natural language instructions are key to the success of instruction tuning . ∗Lead contributors . Author contributions listed at end of paper . 1 INTRODUCTION . Language models ( LMs ) at scale , such as GPT-3 ( Brown et al. , 2020 ) , have been shown to perform few-shot learning remarkably well . They are less successful at zero-shot learning , however . For example , GPT-3 ’ s zero-shot performance is much worse than few-shot performance on tasks such as reading comprehension , question answering , and natural language inference . One potential reason is that , without few-shot exemplars , it is harder for models to perform well on prompts that are not similar to the format of the pretraining data . In this paper , we explore a simple method to improve the zero-shot performance of large language models , which would expand their reach to a broader audience . We leverage the intuition that NLP tasks can be described via natural language instructions , such as “ Is the sentiment of this movie review positive or negative ? ” or “ Translate ‘ how are you ’ into Chinese. ” We take a pretrained language model of 137B parameters and perform instruction tuning—finetuning the model on a mixture of more than 60 NLP datasets expressed via natural language instructions . We refer to this resulting model as FLAN , for Finetuned Language Net . To evaluate the zero-shot performance of FLAN on unseen tasks , we group NLP datasets into clusters based on their task types and hold out each cluster for evaluation while instruction tuning FLAN on all other clusters . For example , as shown in Figure 1 , to evaluate FLAN ’ s ability to perform natural language inference , we instruction tune the model on a range of other NLP tasks such as commonsense reasoning , translation , and sentiment analysis . As this setup ensures that FLAN has not seen any natural language inference tasks in instruction tuning , we then evaluate its ability to perform zero-shot natural language inference . Our evaluations show that FLAN substantially improves the zero-shot performance of the base 137B-parameter model . FLAN ’ s zero-shot also outperforms 175B-parameter GPT-3 ’ s zero-shot on 20 of 25 datasets that we evaluate , and even outperforms GPT-3 ’ s few-shot by a large margin on ANLI , RTE , BoolQ , AI2-ARC , OpenbookQA , and StoryCloze . In ablation studies , we find that increasing the number of task clusters in instruction tuning improves performance on unseen tasks and that the benefits of instruction tuning emerge only with sufficient model scale . Instruction tuning is a simple method that , as depicted in Figure 2 , combines appealing aspects of both the pretrain–finetune and prompting paradigms by using supervision via finetuning to improve language model ’ s responses to inference-time text interactions . Our empirical results demonstrate promising abilities of language models to perform tasks described purely via instructions . Source code for loading the instruction tuning dataset used for FLAN is publicly available at https : //github.com/google-research/flan . 2 FLAN : INSTRUCTION TUNING IMPROVES ZERO-SHOT LEARNING . The motivation of instruction tuning is to improve the ability of language models to respond to NLP instructions . The idea is that by using supervision to teach an LM to perform tasks described via instructions , the LM will learn to follow instructions and do so even for unseen tasks . To evaluate performance on unseen tasks , we group datasets into clusters by task type and hold out each task cluster for evaluation while instruction tuning on all remaining clusters . 2.1 TASKS & TEMPLATES . As creating an instruction tuning dataset with many tasks from scratch would be resource-intensive , we transform existing datasets from the research community into an instructional format . We aggregate 62 text datasets that are publicly available on Tensorflow Datasets , including both language understanding and language generation tasks , into a single mixture . Figure 3 shows these datasets— each dataset is categorized into one of twelve task clusters , for which datasets in a given cluster are of the same task type . Descriptions , sizes , and examples of each dataset are shown in Appendix G . TREC CoLA Math . For each dataset , we manually compose ten unique templates that use natural language instructions to describe the task for that dataset . While most of the ten templates describe the original task , to increase diversity , for each dataset we also include up to three templates that “ turned the task around , ” ( e.g. , for sentiment classification we include templates asking to generate a movie review ) . We then instruction tune a pretrained language model on the mixture of all datasets , with examples in each dataset formatted via a randomly selected instruction template for that dataset . Figure 4 shows multiple instruction templates for a natural language inference dataset . Entailment Not entailment Russian cosmonaut Valery Polyakov set the record for the longest continuous amount of time spent in space , a staggering 438 days , between 1994 and 1995 . Premise Russians hold the record for the longest stay in space . Hypothesis Target Options : - yes - no < premise > Can we infer the following ? < hypothesis > < options > Template 2 Template 1 < premise > Based on the paragraph above , can we conclude that < hypothesis > ? < options > Read the following and determine if the hypothesis can be inferred from the premise : Premise : < premise > Hypothesis : < hypothesis > < options > Template 3 Template 4 , … Figure 4 : Multiple instruction templates describing a natural language inference task . 2.2 EVALUATION SPLITS . We are interested in how FLAN performs on tasks not seen in instruction tuning , and so it is crucial to define what counts as an unseen task . Whereas some prior work defines unseen tasks by disallowing the same dataset to appear in training , we use a more conservative definition that leverages the task clusters from Figure 3 . In this work , we only consider dataset D unseen at evaluation time if no datasets from any task clusters that D belongs to were seen during instruction tuning . For instance , if D is an entailment task , then no entailment datasets appeared in instruction tuning , and we instruction-tuned on all other clusters.1 Hence , to evaluate zero-shot FLAN on c task clusters , we instruction tune c models , where each model holds out a different task cluster for evaluation . 1When evaluating on the read . comp . with commonsense cluster , both read . comp . and commonsense reasoning were dropped from instruction tuning . Conversely , the read . comp . with commonsense cluster was not used for instruction tuning when evaluating on read . comp . or commonsense reasoning . We also drop the paraphrase cluster from instruction tuning when evaluating on NLI tasks and vice-versa . 2.3 CLASSIFICATION WITH OPTIONS . The output space for a given task is either one of several classes ( classification ) or free text ( generation ) . As FLAN is an instruction-tuned version of a decoder-only language model , it naturally responds in free text , and so no further modifications are needed for generation tasks . For classification tasks , prior work ( Brown et al. , 2020 ) used a rank classification approach where , for example , only two outputs ( “ yes ” and “ no ” ) are considered and the higher probability one is taken as the model ’ s prediction . Though this procedure is logically sound , it is imperfect in that the probability mass for answers may have an undesired distribution among ways of saying each answer ( e.g. , a large number of alternative ways of saying “ yes ” may lower the probability mass assigned to “ yes ” ) . Therefore , we include an options suffix , in which we append the token OPTIONS to the end of a classification task along with a list of the output classes for that task . This makes the model aware of which choices are desired when responding to classification tasks . Example use of options is shown in the NLI and commonsense examples in Figure 1 . 2.4 TRAINING DETAILS . Model architecture and pretraining . In our experiments , we use LaMDA-PT , a dense left-to-right , decoder-only transformer language model of 137B parameters ( Thoppilan et al. , 2022 ) . This model is pretrained on a collection of web documents ( including those with computer code ) , dialog data , and Wikipedia , tokenized into 2.49T BPE tokens with a 32k vocabulary using the SentencePiece library ( Kudo & Richardson , 2018 ) . Around 10 % of the pretraining data was non-English . Note that LaMDA-PT only has language model pretraining ( c.f . LaMDA , which was finetuned for dialog ) . Instruction tuning procedure . FLAN is the instruction-tuned version of LaMDA-PT . Our instruction tuning pipeline mixes all datasets and randomly samples from each dataset . To balance the different sizes of datasets , we limit the number of training examples per dataset to 30k and follow the examples-proportional mixing scheme ( Raffel et al. , 2020 ) with a mixing rate maximum of 3k.2 We finetune all models for 30k gradient steps with a batch size of 8,192 tokens using the Adafactor Optimizer ( Shazeer & Stern , 2018 ) with a learning rate of 3e-5 . The input and target sequence lengths used in finetuning are 1024 and 256 , respectively . We use packing ( Raffel et al. , 2020 ) to combine multiple training examples into a single sequence , separating inputs from targets using a special EOS token . This instruction tuning takes around 60 hours on a TPUv3 with 128 cores . For all evaluations , we report results on the final checkpoint trained for 30k steps . 3 RESULTS . We evaluate FLAN on natural language inference , reading comprehension , closed-book QA , translation , commonsense reasoning , coreference resolution , and struct-to-text . As described in §2.2 , we evaluate on unseen tasks by grouping datasets into task clusters and holding out each cluster for evaluation while instruction tuning on all remaining clusters ( i.e. , each evaluation task cluster uses a different checkpoint ) . For each dataset , we evaluate the mean of performance on all templates , which proxies the expected performance given a typical natural language instruction . As a dev set is sometimes available for manual prompt engineering ( Brown et al. , 2020 ) , for each dataset we also obtain the test set performance using the template with the best dev set performance . For comparison , we report zero and few-shot results for LaMDA-PT using the same prompts as GPT-3 ( as LaMDA-PT is not suitable for natural instructions without instruction tuning ) . This baseline provides the most direct ablation of how much instruction tuning helps . Instruction tuning significantly improves LaMDA-PT on most datasets . We also show the zero-shot performances of GPT-3 175B ( Brown et al. , 2020 ) and GLaM 64B/64E ( Du et al. , 2021 ) , as reported in their respective papers . With the best dev template , zero-shot FLAN outperforms zero-shot GPT-3 on 20 of 25 datasets and even surpasses GPT-3 ’ s few-shot performance on 10 datasets . With the best dev-template , zero-shot FLAN outperforms zero-shot GLaM on 13 of 19 available datasets and one-shot GLaM on 11 of 19 datasets . 2In this mixing scheme , a mixing rate maximum of 3,000 means that a dataset does not receive additional sampling weight for examples in excess of 3,000 . Overall , we observe that instruction tuning is very effective on tasks naturally verbalized as instructions ( e.g. , NLI , QA , translation , struct-to-text ) and is less effective on tasks directly formulated as language modeling , where instructions would be largely redundant ( e.g. , commonsense reasoning and coreference resolution tasks that are formatted as finishing an incomplete sentence or paragraph ) . Results on natural language inference , reading comprehension , closed-book QA , and translation are summarized in Figure 5 and described below . Natural language inference ( NLI ) . On five NLI datasets , where a model must determine whether a hypothesis is true given some premise , FLAN outperforms all baselines by a large margin . As noted by Brown et al . ( 2020 ) , perhaps one reason why GPT-3 struggles with NLI is that NLI examples are unlikely to have appeared naturally in an unsupervised training set and are thus awkwardly phrased as a continuation of a sentence . For FLAN , we phrase NLI as the more natural question “ Does < premise > mean that < hypothesis > ? ” , achieving much higher performance . Reading comprehension . On reading comprehension , where models are asked to answer a question about a provided passage , FLAN outperforms baselines for MultiRC ( Khashabi et al. , 2018 ) and OBQA ( Mihaylov et al. , 2018 ) . On BoolQ ( Clark et al. , 2019a ) , FLAN outperforms GPT-3 by a large margin , though LaMDA-PT already achieves high performance on BoolQ . Closed-book QA . For closed-book QA , which asks models to answer questions about the world without access to specific information containing the answer , FLAN outperforms GPT-3 on all four datasets . Compared to GLaM , FLAN has better performance on ARC-e and ARC-c ( Clark et al. , 2018 ) , and slightly lower performance on NQ ( Lee et al. , 2019 ; Kwiatkowski et al. , 2019 ) and TQA ( Joshi et al. , 2017 ) . Translation . Similar to GPT-3 , the training data for LaMDA-PT is around 90 % English and includes some text in other languages that was not specifically used to train the model to perform machine translation . We also evaluate FLAN ’ s performance on machine translation for the three datasets evaluated in the GPT-3 paper : French–English from WMT ’ 14 ( Bojar et al. , 2014 ) , and German– English and Romanian–English from WMT ’ 16 ( Bojar et al. , 2016 ) . Compared with GPT-3 , FLAN outperforms zero-shot GPT-3 for all six evaluations , though it underperforms few-shot GPT-3 in most cases . Similar to GPT-3 , FLAN shows strong results for translating into English and compares favorably against supervised translation baselines . Translating from English into other languages , however , was relatively weaker , as might be expected given that FLAN uses an English sentencepiece tokenizer and that the majority of pretraining data is English . Additional tasks . Although we see strong results for the above task clusters , one limitation with instruction tuning is that it does not improve performance for many language modeling tasks ( e.g. , commonsense reasoning or coreference resolution tasks formulated as sentence completions ) . For seven commonsense reasoning and coreference resolution tasks ( see Table 2 in the Appendix ) , FLAN only outperforms LaMDA-PT on three of the seven tasks . This negative result indicates that when the downstream task is the same as the original language modeling pre-training objective ( i.e. , in cases where instructions are largely redundant ) , instruction tuning is not useful . Finally , we report results for sentiment analysis , paraphrase detection , and struct-to-text , as well as additional datasets for which GPT-3 results are not available , in Table 2 and Table 1 in the Appendix . Generally , zero-shot FLAN outperforms zero-shot LaMDA-PT and is comparable with or better than few-shot LaMDA-PT .

Summary: The paper proposes a simple method, "instruction-tuning", to improve the zero-shot learning capability of large language model, which 1) annotates prompts for a wide range of tasks and then 2) fine-tunes the model to "answer/respond to" those prompt. The empirical results are impressive: after instruction-tuning, the 0-shot performance is better than GPT-3 (0-shot, sometimes few-shot) on a wide range of datasets; nevertheless, on datasets with formats already similar to language modeling, the performance gain is negligible or even negative.  The paper also made a few other observations 1) performance benefits from the number of task clusters 2) instruction-tuning is only beneficial when the model size is larger enough, and 3) few-shot learning still helps.