Contrastive Instruction Tuning (2024)

Abstract: Instruction tuning has been used as a promising approach to improve the performance of large language models (LLMs) on unseen tasks. However, current LLMs exhibit limited robustness to unseen instructions, generating inconsistent outputs when the same instruction is phrased with slightly varied forms or language styles. This behavior indicates LLMs’ lack of robustness to textual variations and generalizability to unseen instructions, potentially leading to trustworthiness issues. Accordingly, we propose Contrastive Instruction Tuning, which maximizes the similarity between the hidden representations of semantically equivalent instruction-instance pairs while minimizing the similarity between semantically different ones. To facilitate this approach, we augment the existing FLAN collection by paraphrasing task instructions. Experiments on the PromptBench benchmark show that CoIN consistently improves LLMs’ robustness to unseen instructions with variations across character, word, sentence, and semantic levels by an average of +2.5% in accuracy.

## introduction instruction tuning has emerged to be an essential training paradigm of large language models (llms; @xcite @xcite . by training models on various pairs of task instructions and instances, instruction tuning has been widely adopted in llms, such as tk-instruct @xcite , instructgpt @xcite , flan-t5 @xcite , and alpaca @xcite , allowing them to follow various human instructions and fulfill user intents @xcite . despite these advancements, current instructiontuned llms are not robust to instruction variations. their performance may vary significantly when one figure 1 : an example from cola @xcite shows that current llms like alpaca may generate entirely different responses when presented with semantically equivalent but textually different instructions. in this paper, we propose contrastive instruction tuning (coin), an instruction tuning method that leverages contrastive learning to align the hidden representations of instruction-instance pairs that are semantically equivalent but textually different and to differentiate those that are semantically distinct. given the same input and output instance, we pair each instruction with its perturbed versions as positive samples. observed that the hidden representations of data from different tasks already have low cosine similarity with each other @xcite , we use the same instruction paired with different instance input and output as hard negative samples (refer to §3.2 for more details). intuitively, by recognizing that the same instruction with different formulations can have the same meaning, the model can generate more consistent answers given different instructions of the same intent and become more robust to variations in language expressions. at the same time, negative samples encourage the model to understand that an instruction can lead to different outcomes in different contexts, facilitating the model to distinguish inputs with different user intents. we assess llms' robustness on the prompt-bench benchmark @xcite , which introduces variations to instructions of a diverse set of tasks at character, word, sentence, and semantic levels. experiments on the benchmark show that coin significantly improves task performance and reduces response variation of alpaca on unseen instructions with variations at all four levels, achieving an average accuracy improvement of +2.5% compared with continual instruction tuning on the same dataset. our contributions are three-fold. first, we propose a contrastive instruction tuning method, coin, to enhance llms' robustness to semanticinvariant instruction variations. second, experiments on promptbench demonstrate the effectiveness of coin in handling semantically equivalent instructions that differ at the character, word, sentence, and semantic levels. third, to facilitate the proposed approach, we augmented the flan collection, a widely used instruction tuning dataset, with contrastive instructions. we will release the augmented dataset consisting of 52k entries and 104k instructions to support future work in this direction. ## related work in this section, we provide a brief summary of three highly related topics. instruction tuning and generalizability. instruction tuning has emerged to be one of the pivotal techniques for enhancing the generalizability of llms @xcite @xcite . this capability is crucial for llms, as it determines models' performance when encountering new data. the efficacy of instruction tuning has become more evident when the number of tasks scales up @xcite . consequently, many recent studies have been focusing on fine-tuning llms with a wide range of tasks. for instance, large-scale datasets that encompass numerous nlp tasks and multiple data sources have been curated for effectively enhancing llms' zero-shot generalizability @xcite @xcite @xcite . despite performance gained on unseen tasks, llms fine-tuned with large-scale instruction datasets remain vulnerable to how the same instruction is expressed differently @xcite @xcite @xcite . this limitation motivates us to enhance llms' robustness to instruction variations in this work. ## contrastive instruction tuning in this section, we first provide a formal definition of contrastive instruction tuning ( §3.1). then, we introduce contrastive sample selection ( §3.2) and the contrastive loss ( §3.3) in our method coin. ## experiment in this section, we evaluate coin's performance on enhancing llms' robustness to instruction variations on promptbench, specifically 10 glue datasets with unseen foot_1 instructions perturbed at different levels. we first provide an overview of the experiment settings ( §4.1, §4.2, and §4.3) and then present a detailed analysis of the experiment results §4.4. ## conclusion in this paper, we proposed coin that aligns hidden representations of semantically equivalent instruction-instance pairs. evaluation results on promptbench, with instructions that differ at character, word, sentence, and semantic levels, demonstrate coin's effectiveness of enhancing llms' robustness to semantic-invariant instruction variations. future work can apply contrastive instruction tuning to enhance the robustness of models and tasks in other modalities, and on other prompt components such as few-shot demonstrations and system prompts.