L ong F orm: Effective Instruction Tuning with Reverse Instructions (2024)

Abstract: Instruction tuning enables language models to more effectively generalize and better follow user intent. However, obtaining instruction data is costly and challenging. Prior work employs methods such as expensive human annotation, crowd-sourced datasets with alignment issues, and generating noisy examples via LLMs. We introduce the LongForm-C dataset, which is created by reverse instructions. We generate instructions via LLMs for human-written corpus examples using reverse instructions. First we select a diverse set of human-written documents from corpora such as C4 and Wikipedia; then we generate instructions for these documents via LLMs. This approach provides a cheaper and cleaner instruction-tuning dataset with natural output and one suitable for long text generation. Our models outperform 10x larger language models without instruction tuning on tasks such as story/recipe generation and long-form question answering. Moreover, LongForm models outperform prior instruction-tuned models such as FLAN-T5 and Alpaca by a large margin, and improve language understanding capabilities further. We publicly release our data and models: [Anonymized-URL].

## introduction instruction tuning conditions language models (lms) on user intents and improves cross-task generalization while ensuring better human alignment. recent works on instruction tuning focus on data collection, showing that training with even a small amount of instruction-tuning data can outperform much larger lms in following user intent and generalization @xcite . however, these studies have some limitations, such as reliance on expensive human-annotated instruction data @xcite or focusing on instructions for academic nlp tasks that have limited coverage of real-world generation tasks (chung the longform-c dataset longform models figure 1 : our reverse instructions method extracts high-quality human-written passages from corpora ("corpus example") and then leverages an llm to generate, for each corpus example, a generated instruction that should elicit this corpus example. using our method, we create the longform-c instruction tuning dataset, consisting of (generated instruction, corpus example) pairs. we then instruction-tune a plm with longform-c. the plm output shown ("generated output") indicates that this approach produces high-quality output. evaluated on a diverse set of text generation tasks, reverse instructions outperforms prior models by more than 35%. @xcite @xcite . a third approach, works that generate instructions and outputs from scratch using large language models (llms), often produces low-quality datasets @xcite . using the reverse instructions method, we create an instruction-following text generation dataset called longform-c to address these issues. we aim to improve instruction tuning by gathering diverse human-written texts from c4 and english wikipedia as outputs. as illustrated in figure 1 , we extract text passages (paragraphs or documents) from the corpora and then prompt an llm with a zero-shot template to generate, for each passage, an instruction that -when given to an llm -we want to result in the generation of the passage or a similar text. we refer to this method of creating (generated instruction, passage) pairs for instruction tuning as reverse instructions. this method offers a cost-effective and fast alternative for creating instruction tuning datasets compared to human annotation while also yielding higher quality output compared to fully synthetic data generation. to make longform-c even more effective, we increase its diversity and quality by leveraging structured examples from stack exchange and wi-kihow, and long text generation tasks from nlp benchmarks. based on the longform-c dataset, we finetune instruction-following plms, which we call longform models, with different architectures and sizes: t5-xl @xcite , opt-6.7b @xcite , and llama-7b @xcite . we compare these models with the baselines flan-t5 @xcite , t0pp @xcite , tk-instruct @xcite , and alpaca on a diverse set of tasks: story, poem, email and recipe generation; grammar error correction; text summarization; table-to-text; and long form question answering. our experimental results are as follows. (i) longform-opt-2.7b outperforms opt-30b on long text generation despite opt-30b having 10x more parameters. (ii) longform models outperform prior instructionfollowing models, such as flan-t5 and alpaca, with more than 63% (in-domain) and 35% (out-ofdomain) relative meteor improvement on text generation. (iii) in addition to nlg, the longform-c dataset also enhances the language understanding (nlu) capabilities of several llms. it improves performance on mmlu @xcite compared to prior datasets like flan. we release the longform-c dataset and models publicly on https://github.com/akoksal/ longform . corpus mining: most nlp tasks benefit from extracting useful examples from corpora such as machine translation (mt) with bitext mining @xcite and argument mining. recent approaches use embeddings to find similar texts in different languages to mine human translation from unlabeled data (artetxe and schwenk, 2019). these methods have been extended to mt with up to 200 languages @xcite . additionally, some tasks filter sentences from corpora to create diverse training sets for human annotation, such as argument mining @xcite . recent works explore methods to mine and restructure corpora (e.g., wikipedia) to generate synthetic data for conversational @xcite and closed-book qa @xcite . long text generation: long text generation is challenging. it requires models to understand long dependencies and planning to generate long texts. many works do task-specific generation (e.g., generation of stories @xcite , recipes @xcite , essays @xcite and longform qa @xcite ) by collecting useful examples from existing resources (e.g., eli5 subreddit, writingprompts subreddit, recipe websites). also, proper evaluation of long text generation remains a significant challenge for task-specific or general-purpose models @xcite . ## the longform-c dataset the longform-c dataset consists of 15,000 corpus examples generated via reverse instructions and additional 12,739 examples from structured corpora (sc) and nlp datasets with instructions and long text pairs. the main part of the longform-c dataset is constructed by reverse instructions generation for a diverse set of corpus samples; this part is referred to as ri below. additionally, we expand our dataset with structured corpora examples through parsing and templates (referred to as sc), as well as nlp tasks reformulated to increase diversity (referred to as nlp). thus, our dataset features a diverse collection of instructions (including human-written, template-based and llmgenerated), each paired with a human-written output. ## results we now present a comprehensive evaluation of our longform models compared to the baselines on longform-c test, recipe generation, long form question answering, short story generation, nlu, and ablation of longform-c subsets. ## conclusion in this paper, we introduce reverse instructions, a novel method to transform human-written texts into instruction-output pairs via llms. we present longform-c, an instruction-following long text generation dataset that combines reverse instructions examples, structured corpora (sc), and nlp tasks. our evaluation shows that the generated reverse instructions are highly relevant to the corpus examples and contain a diverse set of tasks. furthermore, we demonstrate that our longform models -instruction-tuned on longform-c-outperform prior instruction-tuned baselines such as flan-t5 and alpaca by a big margin on a wide variety of long text generation tasks.