Learning to Generate Instruction Tuning Datasets for Zero-Shot Task Adaptation (2024)

Abstract: We introduce Bonito, an open-source model for conditional task generation that converts unannotated text into task-specific training datasets for instruction tuning. We aim to enable zero-shot task adaptation of large language models on users’ specialized, private data. We train Bonito by fine-tuning a pretrained large language model on a new large-scale dataset with 1.65M examples created by remixing existing instruction tuning datasets into meta-templates. The meta-templates for a dataset produce training examples where the input is the unannotated text and the task attribute and the output consists of the instruction and the response. We use Bonito to generate synthetic tasks for seven datasets from specialized domains with unannotated text across three task types—yes-no question answering, extractive question answering, and natural language inference—and adapt language models. We show that Bonito significantly improves the average performance of pretrained and instruction tuned models over the de facto self supervised baseline. For example, adapting Mistral-Instruct-v2 and instruction tuned variants of Mistral and Llama2 with Bonito improves the strong zero-shot performance by 22.1 F1 points whereas the next word prediction objective undoes some of the benefits of instruction tuning and reduces the average performance by 0.8 F1 points. We conduct additional experiments with Bonito to understand the effects of the domain, the size of the training set, and the choice of alternative synthetic task generators. Overall, we show that learning with synthetic instruction tuning datasets is an effective way to adapt language models to new domains. The model, dataset, and code are available at https://github.com/BatsResearch/bonito.

## introduction large language models show remarkable zero-shot capabilities by simply learning to predict the next token at scale @xcite . by fine-tuning these models on instruction tuning datasets containing many tasks-each comprising an input instruction and a desired response-the model generally improves in its ability to respond to unseen instructions. however, this generalization is still limited by the qualities of the instruction tuning dataset. existing datasets like public pool of prompts (p3) @xcite , natural instructions @xcite , and dolly-v2 @xcite focus on text from the web and classic natural language tasks so that they can serve a wide range of use cases, i.e., they are a one-size-fits-all approach. on the other hand, tasks in areas like biomedical and legal domains require specialized, often implicit, domain knowledge. we study how to adapt language models to follow instructions in specialized domains without annotated data. the ability to follow task-specific instructions in specialized domains is important for bringing the benefits of large language models to a wider range of users. recent evaluations-including evaluations of proprietary models-show that they often significantly underperform specialized models @xcite @xcite , particularly in domains requiring subject matter expertise. this motivates us to investigate effective ways to provide domain knowledge to large language models. self supervision in the form of next word prediction on the target corpus is a simple way to teach language models about new domains @xcite . however, this approach requires an enormous amount of training to achieve strong performance @xcite . further, we find that self supervision can undo the benefits of instruction tuning (see section 5.3). alternatively, continued training of models with instructions from specialized domains significantly improves performance @xcite @xcite 1 12585 🤖 instruction tuning data input: given that "in doing so w alcott… friendly against egypt." does it follow that w alcott scored 3 goals in a game yes, no, or maybe? output: yes in doing so walcott also became the first england player to score a hat-trick in a competitive since michael owen in 2001. walcott returned to the international fold on 3 march 2010 in a friendly against egypt. ## zero-shot task adaptation we describe the problem of zero-shot task adaptation. we have a language model, either pretrained via self supervision or further fine-tuned on a training mixture like p3 @xcite , along with a corpus of unannotated text from the target domain. we also know the target task type e.g., extractive question answering, natural language inference, etc. if the target task type has a fixed set of labels such as "yes" or "no" in yes-no question answering, we assume access to the label space. our goal is to adapt the language model to follow task instructions in the target domain without human annotations, achieving zero-shot task adaptation. ## related work instruction tuning multitask instruction tuning of language models dramatically improves their ability to follow instructions and generalize to new unseen tasks @xcite @xcite @xcite @xcite . typically, pretrained models are trained to follow instructions on large-scale training mixtures such as p3 @xcite and the flan collection @xcite . in this work, we use p3 to create meta-templates and train bonito to generate nlp tasks in specialized domains. ## bonito: learning to generate tasks we describe the steps to create the conditional task generation with attributes (ctga) dataset and train bonito. then, we briefly describe the procedure to create synthetic instruction tuning datasets with the target unannotated texts to adapt language models. ## additional experiments we briefly describe additional experiments included in appendix b and c. in appendix b, we generate synthetic tasks by prompting mistral-7b-instruct-v0.2 and zephyr-7b-β. our results show that the synthetic tasks from mistral-7b-instruct-v0.2 and zephyr-7b-β improve the average performance of mistral-7b but decrease significantly when adapting mistral p3 . this indicates that we require high-quality synthetic tasks to increase the performance of strong instruction-tuned models. in appendix c, we generate synthetic tasks with gpt-4 for privacy policy qa, squadshifts reddit, and contractnli. our results show that gpt-4 improves mistral p 3 on privacy policy qa and contractnli but slightly reduces performance on squadshifts reddit. finally, we analyze the generated tasks and identify common issues with both open-source models and gpt-4, such as the distribution of the label space and "chatty" responses, which potentially lead to the drop in performance. ## conclusion we present bonito, an open-source model for conditional task generation that converts unannotated text into instruction tuning datasets. we show that training with synthetic instruction tuning datasets in specialized domains is a strong alternative to self supervision. our experiments demonstrate that bonito-generated instructions improve pretrained and instruction tuned models on zero-shot task adaptation. overall, bonito enables practitioners to adapt large language models to tasks on their data without annotations. 9 12593