From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large Language Models (2024)

Abstract: It is imperative for Large language models (LLMs) to follow instructions with elaborate requirements (i.e. Complex Instructions Following). Yet, it remains under-explored how to enhance the ability of LLMs to follow complex instructions with multiple constraints. To bridge the gap, we initially study what training data is effective in enhancing complex constraints following abilities. We found that training LLMs with instructions containing multiple constraints enhances their understanding of complex instructions, especially those with lower complexity levels. Additionally, we further propose methods addressing how to obtain and utilize the effective training data. Finally, we conduct extensive experiments to prove the effectiveness of our methods in terms of overall performance and training efficiency. We also demonstrate that our methods improve models’ ability to follow instructions generally and generalize effectively across out-of-domain, in domain, and adversarial settings, while maintaining general capabilities.

## introduction large language models (llms) have become the backbone for real-world applications @xcite @xcite . given natural language instructions, llms can solve unseen tasks with few or no examples @xcite . the capability of llms to accurately understand instructions and convey the desired output, known as instruction following @xcite , is crucial for the safety @xcite and reliability @xcite of llms. ## empirical studies a common approach to improve llms' ability to follow complex instructions is to construct corresponding instances and fine-tune the llms on them @xcite ). yet, one key research question remains under-explored: what training data is effective in enhancing complex constraint-following abilities? two types of training data can be utilized: (1) initially train models with atomic data, enabling them to handle compositional data automatically. to compare these training data types, we split instructions from existing benchmarks on instruction following @xcite into training and test sets. the training set includes atomic data (mostly with 1 constraint) and compositional data (mostly with over 3 constraints). since original benchmarks lack corresponding outputs, we first generate them using gpt-3.5-turbo. to ensure quality, we further filter the training sets and retain only outputs that satisfy all instruction constraints by using gpt-3.5-turbo and predefined rules. the remaining data forms the test set foot_0 . we compare three methods foot_1 : (1) backbone, the backbone model without further training. (2) atom and (3) composition, continue training the backbone model with atomic data and compositional data respectively. we leverage two backbone models @xcite and adopt two accuracy metrics @xcite : the performance of the three methods on the test sets is shown in tab. 1 and tab. 2. first, with regard to the overall performance, training with compositional data generally surpasses both the backbone model and atomic data training. this demonstrates that training with compositional data can generally enhance models' ability to follow complex instructions. surprisingly, according to tab. 1, training with atomic data (mostly with 1 constraint) generally performs worse than the backbone model for instructions with more than 1 constraint. also, training with compositional data (usually 3 to 5 constraints) boosts performance on instructions with 1 to 3 constraints significantly but shows less enhancement or even a decline for those with 4 to 5 constraints. this suggests that training with compositional data can better generalize to lower-level complex instructions (instructions with fewer constraints). moreover, this effect is more pronounced in smaller llms (7b), likely due to their weaker generalization ability @xcite . we have found that training with compositional data can better enhance llm's ability to follow complex instructions compared with atomic data. a follow-up research question is how to obtain highquality compositional data? existing datasets either only provide compositional instructions without output @xcite or directly generate responses using advanced llms and refine them manually @xcite . we compare the outputs generated by three methods: (1) vanilla: output generated directly using backbone model. (2) generation: output generated directly using gpt-3.5-turbo. (3) discrimination: first, we identify the constraints that vanilla outputs failed to adhere to using test scripts @xcite . then, we rectify the vanilla outputs constraints by constraints using gpt-3.5-turbo (the framework is shown in fig. 2 and please refer to §4.2 for details). with regard to the complex instructions, the instructions in ifeval @xcite originally have only 1 to 3 constraints, which are not complex enough. we construct 1467 complex instructions, each comprising 3 to 5 constraints that can be automatically verified (please refer to §4.1 for details). we leverage llama2-13b-chat @xcite as the backbone and assess output quality using the test script from @xcite . as shown in tab. 3, using the generation method, outputs from advanced llms (generation) are of higher quality than those from weaker llms (vanilla). however, the outputs from weaker llms then refined by advanced llms (discrimination) are of better quality than the outputs generated by advanced llms directly (generation). this is because slight changes in the instruction (i.e. constraint) can cause substantial output differences, which the discrimination method captures better than the generation method. ## method in §3, we propose a discrimination-based method to obtain effective training data. a subsequent question is how to effectively utilize the obtained ## experiments we conduct experiments to verify the effectiveness of our method, focusing on overall performance, training efficiency, and generalization ability. ## conclusion in this paper, we systematically study how to enhance the ability of llms to follow complex instructions. initially, we study effective training data and methods for obtaining high-quality data through empirical studies. based on our findings, we introduce a method utilizing positive and negative samples to enhance llms' complex instruction-following ability. our experiments show that our methods enhance models' ability to follow complex instructions more effectively and efficiently. finally, extensive experiments demonstrate the generalization abilities of our framework.