C ommon IT : Commonality-Aware Instruction Tuning for Large Language Models via Data Partitions (2024)

Abstract: With instruction tuning, Large Language Models (LLMs) can enhance their ability to adhere to commands. Diverging from most works focusing on data mixing, our study concentrates on enhancing the model’s capabilities from the perspective of data sampling during training. Drawing inspiration from the human learning process, where it is generally easier to master solutions to similar topics through focused practice on a single type of topic, we introduce a novel instruction tuning strategy termed CommonIT: Commonality-aware Instruction Tuning. Specifically, we cluster instruction datasets into distinct groups with three proposed metrics Task, Embedding and Length). We ensure each training mini-batch, or “partition”, consists solely of data from a single group, which brings about both data randomness across mini-batches and intra-batch data similarity. Rigorous testing on LLaMa models demonstrates CommonIT’s effectiveness in enhancing the instruction-following capabilities of LLMs through IT datasets (FLAN, CoT, and Alpaca) and models (LLaMa2-7B, Qwen2-7B, LLaMa 13B, and BLOOM 7B). CommonIT consistently boosts an average improvement of 2.1% on the general domain (i.e., the average score of Knowledge, Reasoning, Multilinguality and Coding) with the Length metric, and 5.2% on the special domain (i.e., GSM, Openfunctions and Code) with the Task metric, and 3.8% on the specific tasks (i.e., MMLU) with the Embedding metric. Code is available at https://github.com/raojay7/CommonIT.

## introduction the emergence of @xcite and a range of large language models (llms) @xcite brings light to artificial general intelligence (agi). as the size of the model increases, instruction fine-tuning becomes necessary to align human intentions with * corresponding author @xcite . it fails to recognize the instructions of the translation task in this case, "please translate the following sentence ...", and simply replies with the final phrase "can i change it?". existing research has already underscored the significance of data mixing in fine-tuning large models for achieving a generalized model. for example, @xcite pointed out that the key for llms in the it phase is to enhance the understanding of instructions and they increase the diversity of instructions, e.g., by using more instructions to describe the same task @xcite . despite this, these llms are susceptible to bias in the model's understanding of a particular task @xcite due to the mixing of multiple tasks or diverse instructions, which can lead to a decline in the average ability across multiple tasks @xcite @xcite . as illustrated in figure 1 , this situation results in a diminished ability of the model to follow instructions accurately. specifically, it struggles to comprehend the requirements laid out in the instructions, leading to incorrect responses. to enhance the model's ability to follow instructions, we introduce commonit, a methodology that enhances model comprehension of data features by engaging distinct data classes in individual gradient updates and interchanging data classes across batches. central to our approach is the principle of data commonality @xcite , inspired by humans' learning process when preparing for exams. this means focusing preparation on one exam subject at a time rather than attempting to study for multiple subjects simultaneously. specifically, we advocate a two-phase fine-tuning process for llms. initially, raw data are segmented into several groups and partitioned into designated sizes for training. this strategy enhances the model's ability to follow instructions, as evidenced by comprehensive evaluations across four dimensions of capacity testing. our experiments demonstrate its applicability across diverse it datasets and potential for future applications in different models and domainspecific tasks. our findings indicate that length serves as the most effective criterion for grouping within the generic domain. furthermore, embedding divisions should be tailored specifically for each task, and task-related information is crucial for optimizing fine-tuning quality in specific domains. our analysis of commonalities, from the perspectives of data quality, input question sentence representation and multi-task generalizability, delves into the reasons for improvements, demonstrating that the common learning strategy indeed enhances the effectiveness of representations and the capabilities across various sub-tasks. our contributions are as follows: ## our proposed commonit method motivation data imbalance can lead to inconsistent performance across tasks, with some exhibiting exceptional results and others underperforming significantly @xcite . this phenomenon is typically evidenced by instruction misinterpretation, as illustrated in figure 1 , where the model fails to comprehend the user's intention accurately. our approach, commonit, aims to balance taskspecific sampling for gradient updates with diverse data grouping. this strategy draws upon the human learning process of understanding through analogy, leveraging similarities across different contexts to facilitate comprehension. through this approach, the model gains an enhanced ability to differentiate among the instructions of various tasks. conse-quently, it can more accurately respond to distinct task directives, thereby minimizing the misinterpretation of instructions and reducing the generation of irrelevant content ( §5). overview as shown in figure 2 , commonit is divided into two steps. first, we need to perform a clustering operation to categorize the dataset into multiple categories. it is worth noting that the clustering here does not require precise categorization but only focuses on a certain aspect of the dataset division. in the figure, the square, circle, and triangle indicate three kinds of data with different attributes. at the same time, the different colors indicate that the data can also be divided by the attribute of color, which is not divided here just due to space constraints. the second step is to construct partitions of the divided data by batch size and randomly take the divided partitions as one batch when the data are fed into the model. ## conclusion we present a simple and effective fine-tuning method commonit. leveraging data commonality with three metrics significantly enhanced the effectiveness of llms across multiple competency dimensions during the it. the evaluation across diverse models, it datasets, and specific tasks has showcased the methodology's application scalability. ablation experiments across various stages and data clustering techniques have illustrated the effectiveness of our method. explorations of commonalities confirm that commonit mirrors the human learning process, improving overall performance.