Demystifying Instruction Mixing for Fine-tuning Large Language Models (2024)

Abstract: Instruction tuning significantly enhances the performance of large language models (LLMs) across various tasks. However, the procedure to optimizing the mixing of instruction datasets for LLM fine-tuning is still poorly understood. This study categorizes instructions into three primary types: NLP downstream tasks, coding, and general chat. We explore the effects of instruction tuning on different combinations of datasets on LLM performance, and find that certain instruction types are more advantageous for specific applications but can negatively impact other areas. This work provides insights into instruction mixtures, laying the foundations for future research.

## introduction instruction tuning has been shown to have surprising efficacy for aligning large language models (llms) with human instructions @xcite @xcite @xcite @xcite . recent studies highlight the diverse ways in which instructions can enhance the different capabilities of llms. for instance, using general-purpose, chat-like instructions can improve the performance of llms as chat assistants @xcite @xcite , while training llms on instructions based off nlp tasks improves their performance on nlp benchmarks @xcite @xcite , and incorporating coding instructions enhances llm code generation @xcite . however, a key unresolved issue is determining how to combine various instruction datasets to optimize overall llm performance. 1 code and data are available at: https://github.com/ reason-wang/instructllm . ## experimental setup datasets we select alpaca @xcite as the general instruction dataset to align models, in the form of 52k instruction-response pairs. we use p3 @xcite as our nlp task instruction dataset, which is reformatted for a wide range of nlp downstream tasks using diverse humanwritten templates. since the number of samples in each task varies vastly, we randomly sample 1k instances from each subtask formatted with several corresponding prompts for diversity, resulting in 660k samples. for coding data, we choose codealpaca @xcite , which is an instruction dataset focusing on code generation. it contains 20k samples in different programming languages. to ensure a balanced comparison, we randomly sample a 20k subset from each dataset. examples are provided in ## results for the remainder of this paper, we denote the alpaca, codealpaca, and p3 datasets as a, c, p, respectively. for each model, we compare eight different data mixing strategies, denoted as none, a, c, p, ac, ap, cp, acp, where none represents the vanilla model without fine-tuning, and each of the other settings represents the model fine-tuned with the corresponding dataset. for example, ac means the model is fine-tuned with both alpaca and codealpaca. ## conclusion in this paper, we investigated different data mixing strategies in instruction fine-tuning. we measured models against diverse benchmarks and alignment skills. we find that general instructions provide better alignment as well as performance on nlp benchmarks, code instructions improve coding and alignment skills, while nlp task instructions hinder alignment skills when combined with other instruction types.