PACIT : Unlocking the Power of Examples for Better In-Context Instruction Tuning (2024)

Abstract: Instruction tuning enhances the instruction following ability of large language models by finetuning with supervised instruction data. Previous work proposes in-context instruction tuning (ICIT) where specific positive or negative examples are incorporated into the prompt for better performance. In this work, we propose PACIT, a simple and effective in-context instruction tuning method, inspired by the pedagogical concept of desirable difficulty. The PACIT method unlocks the power of examples by encouraging the model to actively learn to grasp the distinctions between the positive and negative examples instead of merely reading. The model is expected to first verify the correctness of the provided example according to the task description, which is then set as the condition for generating a better response to the task instance. Our extensive experiments prove the effectiveness of PACIT, outperforming ICIT baseline on both in-domain and out-domain tasks up to 9.16 and 3.14 average ROUGE-L scores, respectively. Moreover, PACIT can notably enhance the performance of instruction tuning even when all positive and negative examples are generated with a self-instruct method.

## introduction large language models (llms) have garnered significant interest from both academia and industry due to their superior performance on a variety of natural language processing tasks such as question answering and text generation. instruction tuning (it; @xcite ) optimizes the pretrained language models with supervised instruction data to enhance the capabilities of the instruction following and zero-shot generalization to unseen tasks @xcite @xcite @xcite . instructgpt @xcite proposes in-context instruction tuning (icil) where the llm is finetuned using instruction data with few-shot human-crafted positive examples. su-perni @xcite presents a variant of in-context instruction tuning by further incorporating specified positive and negative examples in each task. the icil method achieves significant improvement compared with the vanilla zero-shot instruction tuning method @xcite @xcite with the knowledge from the demonstrations. however, previous in-context instruction tuning merely shows the specified positive and negative examples in the prompt, without further considerations for better digestion of examples. llms still struggle to follow the instructions precisely in some scenarios @xcite , which hinders their further applications. in this work, we introduce pacit, a simple and novel in-context instruction tuning approach (see figure 1 ) inspired by the pedagogical concept of desirable difficulty @xcite . during finetuning with pacit method, the model first accomplishes a quiz about the judgment of correctness of the provided examples based on the task description, then responds to the task instance input. by transforming the provided example into a related quiz of the simple classification task, we encourage the model to be actively involved in recalling correlated information and grasping the distinction between positive and negative examples, going beyond surfacelevel information. in contrast to simply reading the examples, this approach enhances the model's comprehension of the task information, thereby improving its ability to follow instructions. extensive experiments prove the effectiveness of pacit, outperforming icit baseline up to 9.16 and 3.14 average rouge-l @xcite on indomain and out-of-domain datasets of superni @xcite , respectively. the pacit still consistently surpasses traditional methods when the positive and negative examples are synthesized with self-instruct @xcite by @xcite . therefore, in cases that the human-crafted positive and negative examples are not available, the pacit has the potential to be a better instruction tuning strategy even for a largescale instruction dataset. our contributions are summarized as follows: ## method in this work, we focus on the in-context instruction tuning @xcite where both positive and negative examples are provided as the case in the superni dataset (see figure 1 ). the model is trained to generate a response that is similar to the positive examples while avoiding the mistakes in the negative ones. conventional works merely present these examples and their tags in the in this task, you need to output "yes" if the given number is a prime number, otherwise output "no". ## analyses the visualization of attention. to better understand how pacit works, we conduct a case study by visualizing the attention weights in t5-3b model. we visualize the averaged encoderdecoder attention weights of different heads in the last layer of t5-3b. figure 2 shows a concrete example of pacit v.s. superni (few-shot). the color in each figure represents the relative attention weights. actually, the relative attention weights is also based on the generated classification results and actions for pacit. in order to show the comparison more clearly, we do not include them in the figure. as can be seen, pacit allocates more attention to the task definition and examples' information compared with the superni (few-shot) model. the attention weights from pacit exhibit a broader span across the prompt. this observation is expected as the classification task in pacit encourages the model to focus more on task definition and examples, otherwise it cannot classify examples correctly. we also manually check some other examples which present similar patterns. the effectiveness of additional computation considering that the effectiveness of pacit may come from trading off the extra token compute, we also compare pacit with the self-consistency @xcite (sc for short) method in the inference stage based on llama-2 model to maintain similar computational overhead. specifically, we conducted 5 trials per problem for sc with temperature 0.7 @xcite . more sampling results at different temperatures can be found in the appendix b.