Did You Read the Instructions? Rethinking the Effectiveness of Task Definitions in Instruction Learning (2023)

Abstract: Large language models (LLMs) have shown impressive performance in following natural language instructions to solve unseen tasks. However, it remains unclear whether models truly understand task definitions and whether the human-written definitions are optimal. In this paper, we systematically study the role of task definitions in instruction learning. We first conduct an ablation analysis informed by human annotations to understand which parts of a task definition are most important, and find that model performance only drops substantially when removing contents describing the task output, in particular label information. Next, we propose an automatic algorithm to compress task definitions to a minimal supporting set of tokens, and find that 60% of tokens can be removed while maintaining or even improving model performance. Based on these results, we propose two strategies to help models better leverage task instructions: (1) providing only key information for tasks in a common structured format, and (2) adding a meta-tuning stage to help the model better understand the definitions. With these two strategies, we achieve a 4.2 Rouge-L improvement over 119 unseen test tasks.

## introduction large language models or llms @xcite @xcite demonstrate the ability to perform zero-shot crosstask generalization through learning from instructions of tasks @xcite @xcite @xcite . by fine-tuning an llm with task definitions and a few demonstration examples on upstream training tasks, the model acquires the power to perform new tasks with unseen definitions and example. this is known as instruction learning. however, a natural question is: to what extent does the zero-shot generalization ability derive from the model's understanding of task definitions? recent work in prompt-based learning suggests models might not interpret even short prompts as people expect @xcite @xcite . task definitions are special prompts that are usually long and encode rich information. we imagine models' understanding of definitions also departs from human expectation. to investigate this question, we conduct a systematic analysis using both human annotation and computational approaches. our study is based on the english portion of the large-scale super-naturalinstruction (niv2) dataset @xcite , which comprises 757 training tasks and 119 unseen test tasks. first, we explore which type of information in task definitions is necessary for maintaining model performance. we define eight categories of content and provide a fine-grained annotation for all the sentences in task definitions. then, we retrain the model with every occurrence of each category in niv2 ablated out, and measure the model performance on the validation set with the same ablation. we observe variable contributions to model performance across content types. for example, input descriptions are in general not helpful to generalization performance, i.e., removing them causes little to no degradation of performance. however, larger models tend to leverage them more. on the other hand, the label information is of great importance. providing natural-language label definitions helps specify the task-specific meaning of common verbalizers while providing the label verbalizer only helps in generalizing to a new label space. we also find that we can achieve similar or even better performance compared to full definitions by only providing the models with a label space along with very basic task metadata, e.g., category, domain, reasoning type, etc. this suggests that costly hu-rq1: which parts of task definitions are important when performing zero-shot instruction learning? -for classification tasks, label-related information is crucial, as it helps the model identify the output space and identify each label's meaning when generalizing. rq2: is natural language the most efficient format to communicate task instructions to models? -framing instructions as a structured input/action/output triplet is potentially a more efficient and effective way of creating task definitions. -in fact, using only basic metadata and the label space (without label definitions) in a structured format, we achieve similar, or even better performance as with full definitions. rq3: how can we improve models' understanding of definitions as well as model performance? second, motivated by @xcite , to understand what is necessary for models to perform well, we propose syntax-guided task definition compression (stdc), an automatic approach to removing content in task definitions that is not helpful for model performance. stdc queries the model for predictions on inputs and only requires black-box access. we can remove around 60% of tokens while achieving ˜3 points of performance improvement of t5-xl on a held-out set. this implies that instead of understanding the whole definition of the task, the models are relying on particular text while ignoring the rest. along with similar observations as the ablation study above, stdc reveals new patterns of how models understand definitions. for example, models usually do not need to see the whole label space, but might infer the rest with a partial label space. given our observations, we conclude that current instruction learning models rely on partial information in definitions. we imagine the lack of consistency in the creation process of task definitions might hinder the model from attending to all key information in definitions. thus, we propose two complementary strategies to overcome this. the first strategy is to replace the full definition with a json-like formatted triplet of input, action, and output. a json-like triplet simplifies the creation of task definitions by asking authors of the definition to fill in blanks in templates instead of writing from scratch, and the common structure increases consistency between authors. the second strategy is to perform meta-tuning before instruction learning to adapt llms to any predefined styles of task definitions. we achieve 4.2, 4.0, and 2.1 rouge-l improvements on bart-large, t5-large, and t5-xl, respectively, combining these two strategies. we summarize our key findings in ## background in this section, we introduce the formulation of instruction learning, as well as the models and benchmarks used in our study. further details are presented in appendix a. instruction learning. instruction learning aims to train a language model so that it understands natural language task instructions and is able to generalize to a new task by solely reading new instructions. a task instruction may include several elements. in this paper, we follow @xcite and adopt instructions with 1) a task definition: a high-level description of the input and output of the task; and 2) demonstration examples: some input-output examples for the task. note that other content such as things to avoid and negative examples may also be included but have been shown to be less effective @xcite . a task instruction is generally pre-pended to an input and passed to the llm. the llm is first finetuned on several upstream training tasks and then asked to conduct inference on an unseen test task, given only the task instruction. benchmark. we adopt the english portion of niv2 @xcite , which contains 757 training tasks and 119 unseen test tasks. the test tasks fall into 12 categories, including textual entailment, data-to-text generation, etc. however, given a document, generate a short title of the document. the title should convey the main idea/event/topic about which the document is being written. given a sentence with a missing word, pick the answer option that best fills out the missing word in the sentence. indicate each answer with its index ('a', 'b', 'c', 'd'). you will be given two pieces of text… one of them is simpler … you are expected to output 'text one' if the first sentence is simpler. we also consider a more coarse split of test tasks into classification and generation tasks, based on whether the output space is fixed or not. for each task, we select 100 examples for either fine-tuning or testing and report performance of rouge-l @xcite , following @xcite . we use the task definition and two demonstration examples as the instruction. the original paper does not provide an official validation split, which we prepare by putting aside 76 training tasks. we fix the validation set for all experiments to ensure no data leakage. note that for later experiments, results for section 3 and section 4 are reported on the validation split which we hold out ourselves while results for section 5 are on the official test set. ## ablation analysis of annotated task definitions to explore what information exists in task definitions and how this impacts model performance, we manually examine all the task definitions in niv2. we decompose and categorize definition text into eight types of content. these types cover the de-scriptions of input, action (the function the model should take, e.g., generate), and output for each task in a hierarchical manner. the description can either be a primary mention of an item or provide additional, secondary details. figure 1 shows the final categories, along with example annotations. three of our authors annotated all task definitions with content categories, annotating at the sentence level and in some cases sub-sentence units when required, as shown in figure 1 . to establish annotation feasibility, we first annotated 150 common task definitions, and measured a high interannotator agreement of 0.91 fleiss kappa @xcite across categories, confirming the clarity of the defined categories. the remaining task definitions are equally split and each task is labeled by a single annotator. appendix b presents details of annotations. ## compressing task definitions analysis in section 3 reveals that a large portion of information in human-written task definitions is not critical in improving model performance. this analysis is informed by human annotations. now, to gain a model-centric perspective, we implement syntax-guided task definition compression (stdc), which iteratively discovers influential content from a task definition. the motivation behind using a syntax-guided and top-down algorithm is to preserve as much human readable content as possible to show the function of compressed definitions. in our preliminary experiments, we also adopt a vanilla word-by-word compression algorithm as @xcite . however, we find that it is either less efficient and producing compressed definitions with slightly degraded performance on the hold-out set. in stdc, syntactically plausible content from the definition is iteratively removed if it does not cause a decrease in model performance. we first obtain the constituency parse tree for each definition. 3 then, in a top-down manner, we traverse the parse tree and check each phrasal node iteratively. if removing the phrase node does not cause any performance decrease, we remove the subtree rooted by that node. the algorithm stops after all leaf node removals are attempted. the framework is illustrated in algorithm 1 of appendix c. ## improving model understanding of task definitions previous sections indicate that not all content in task definitions contribute to strong model performance, suggesting a mismatch between the intent and model interpretation of task definitions. a possible reason for the mismatch could be due to the crowdsourcing of task definitions by many experts, creating a lack of consistency and structure in task definitions, in turn complicating the extraction of the key information by the model. to investigate the hypothesis, we propose two approaches to reduce the mismatch and improve model understanding of task definitions. first, we organize the task definition into a (input, action, output) triplet. second, we add a meta-tuning stage to prepare the model before instruction learning. this phase is intended to help adapt the language models to the writing style of task definitions. structuring task definitions with triplets we extract input/action/output information from all task definitions in niv2 and rewrite them into triplets, leveraging both human annotation and automated processing. this serves as a starting point for using structured key information as task definitions. future work may explore directly writing task definitions in the triplet format. more specifically, we use a json-like template with the following format: task input: 1. task action: 2. task output: 3, where 1, 2 and 3 represent extracted portions of task definitions describing the input, action, and output, respectively. we populate the template based on the annotation we performed in section 3. for the input and action entries, we first extract segments marked as input content and action content and run a syntactic parser to extract the key phrase from the corresponding sentences. we extract the noun phrase from input content for the input entry and the verb phrase from action content for the action entry. for the output entry, we use the task labels and label definitions for classification tasks. for generation tasks, we extract the output noun from the action content sentence with rule-based methods. we manually inspected all triplets generated, manually corrected parsing mistakes, and corrected several co-reference issues we found. some examples are presented in appendix d. note that with this extraction process, we also fulfill the condensing of information in task definitions. ## discussion the field of instruction learning has moved rapidly since this paper was first written. we summarized the newly released models and benchmarks in section 6. in this section, we discuss how we position the paper in the current context of instruction training, as well as how we deal with the current challenges. ## conclusion this work investigates the effectiveness of task definitions in instruction learning. our results indicate that different types of content in definitions have widely varying impacts on model performance. specifically, we found that label information is critical for the model performance, whereas input descriptions and additional constraints are not important. we found that current natural-language formatted definitions can be extensively compressed. we also open the door for more efficient creation of task definitions; we may simply provide the model with structured information, even the metadata, by filling in a json-formatted template.