Self-Instruct: Aligning Language Models with Self-Generated Instructions (2023)

Abstract: Large “instruction-tuned” language models (i.e., finetuned to respond to instructions) have demonstrated a remarkable ability to generalize zero-shot to new tasks. Nevertheless, they depend heavily on human-written instruction data that is often limited in quantity, diversity, and creativity, therefore hindering the generality of the tuned model. We introduce Self-Instruct, a framework for improving the instruction-following capabilities of pretrained language models by bootstrapping off their own generations. Our pipeline generates instructions, input, and output samples from a language model, then filters invalid or similar ones before using them to finetune the original model. Applying our method to the vanilla GPT3, we demonstrate a 33% absolute improvement over the original model on Super-NaturalInstructions, on par with the performance of InstructGPT-001, which was trained with private user data and human annotations. For further evaluation, we curate a set of expert-written instructions for novel tasks, and show through human evaluation that tuning GPT3 with Self-Instruct outperforms using existing public instruction datasets by a large margin, leaving only a 5% absolute gap behind InstructGPT-001. Self-Instruct provides an almost annotation-free method for aligning pre-trained language models with instructions, and we release our large synthetic dataset to facilitate future studies on instruction tuning.

## introduction the recent nlp literature has witnessed a tremendous amount of activity in building models that can 10 for more examples. output: "honesty is the first chapter in the book of wisdom." -thomas jefferson task task instruction : find out if the given text is in favor of or against abortion. ## method annotating large-scale instruction data can be challenging for humans because it requires 1) creativity to come up with novel tasks and 2) expertise for writing the solutions to each task. here, we detail our process for self-instruct, which refers to the pipeline of generating tasks with a vanilla pretrained language model itself, filtering the generated data, and then conducting instruction tuning with this generated data in order to align the lm to follow instructions better. this pipeline is depicted in figure 2 . ## self-instruct data from gpt3 in this section, we apply our method for inducing instruction data to gpt3 as a case study. we use the largest gpt3 lm ("davinci" engine) accessed through the openai api. 6 the parameters for making queries are described in appendix a.2. here we present an overview of the generated data. ## experimental results we conduct experiments to measure and compare the performance of models under various instruction tuning setups. we first describe our models and other baselines, followed by our experiments. ## conclusion we introduce self-instruct, a method to improve the instruction-following ability of lms via their own generation of instruction data. on experimenting with vanilla gpt3, we automatically construct a large-scale dataset of 52k instructions for diverse tasks, and finetuning gpt3 on this data leads to a 33% absolute improvement on superni over the original gpt3. furthermore, we curate a set of expert-written instructions for novel tasks. human evaluation on this set shows that tuning gpt3 with self-instruct outperforms using existing public instruction datasets by a large margin and performs closely to instructgpt 001 . we hope self-instruct can serve as the first step to align pretrained lms to follow human instructions, and future work can build on top of this data to improve instruction-following models.