Fine-Grained Constraint Generation-Verification for Improved Instruction-Following (2025)

Abstract: The ability of Large Language Models (LLMs) to follow natural language instructions is crucial. However, numerous studies have demonstrated that LLMs still struggle to follow instructions with complex constraints, limiting their application in other areas. Meanwhile, obtaining high-quality instruction-following data often requires substantial manual annotation, which is both time-consuming and labor-intensive. In this work, we present FiGV, a fine-grained constraint generation-verification strategy for synthesizing instruction-following data. FiGV employs LLM-driven processes to generate fine-grained constraints and check the legality of the synthetic instructions. Subsequently, LLMs are utilized to perform nuanced, constraint-level verification to determine whether the generated responses adhere to the synthetic instructions, with LLM-generated functions incorporated for auxiliary validation tailored to the types of constraints. Experiments on 7B to 70B models demonstrate that FiGV consistently achieves strong performance across various benchmarks designed to evaluate the instruction-following capabilities of LLMs.

## introduction the field of large language models (llms) has witnessed remarkable advancements in recent years, demonstrating a wide range of impressive capabilities @xcite . among these, instructionfollowing stands out as one of the most critical requirements for llms, as it directly influences how effectively these models align with human intentions @xcite , serving as a key factor in ensuring the safety and reliability of llms. @xcite . although the instruction-following capability of llms is crucial, current models still exhibit limitations in following instructions with complex constraints @xcite @xcite . to enhance the instruction-following capability of llms, current measures typically focus on instruction-tuning @xcite @xcite the models using instruction-response pairs, where the former represents the human-provided instruction, and the latter denotes the desired response that aligns with the given instruction . the data used in this instructiontuning phase is mainly obtained through manual annotation or the synthesis of complex instructions. for manual annotation, the high cost, low efficiency, and uncertain quality of human-labeled data make it difficult to scale, thus failing to meet the large-scale data requirements of current llms @xcite . regarding the synthesis of complex data, previous work @xcite has primarily focused on incorporating multiple constraints into instructions and then using exisiting llms like gpt-4 to generate responses. while this approach yields promising results, the quality of the synthesized complex instructions is hard to control, and the reliability of the distilled data cannot be guaranteed @xcite . in this work, we address these issues by introducing a fine-grained constraints generation-verification method for automatically synthesizing instruction-following data, named figv, which support both supervised fine-tuning (sft) and direct preference optimization (dpo) algorithm @xcite . to generate high-quality complex instruction-following data, figv incorporates several key components, including finegrained constraints generation, instruction verification, and verified response generation to ensure that the instructions are diverse, realistic, and comprehensive, while responses remain reliable and aligned with the given instructions. during the constraints generation step, llms are prompted to generate fine-grained constraints based on the original instructions, considering multiple categories. in the instruction verification process, validity analysis is conducted on the synthesized instructions to ensure their reasonableness and verify that the added constraints do not conflict with one another. in the verified response generation phase, we employ llms to generate responses for the synthetic instructions and conduct fine-grained constraint-level verification to ensure that the generated responses align with each constraint in the instructions. to enhance the verification process, llm-generated functions are introduced for auxiliary validation based on the types of constraints. by operating entirely under llm supervision, figv demonstrates both automation and scalability. a series of experiments are conducted to validate the effectiveness of figv by training llms ranging from 7b to 70b parameters, including models from the qwen2 (qwen, 2024), llama3 (meta, 2024), and glm4 (glm, 2024) series, across both sft and dpo training algorithms. the effectiveness of our methodology is assessed using widely adopted instruction-following benchmarks, including ifeval @xcite , follow-bench @xcite , and infobench @xcite . the results on these three instructionfollowing benchmarks demonstrate that figv significantly enhances llms' performance in complex instruction-following tasks. experiments on mt-bench @xcite and alpacaeval @xcite further demonstrate that the models trained using our method exhibit performance comparable to their respective alignment models in general instruction-following abilities. ## method in this section, we provide a detailed explanation of the methodologies employed in figv for constructing the instruction-following dataset. this process comprises two primary stages: the synthesis of instructions from original instructions (section 3.1) and the generation of verified responses to these synthetic instructions (section 3.2). ## experiments we conduct comprehensive experiments to evaluate the effectiveness of figv, mainly focus on the instruction-following performance. ## conclusion in this work, we introduced figv, a fine-grained constraints generation-verification method for synthesizing high-quality instruction-following data. our method integrates fine-grained constraints generation, instruction verification, and verified response generation, all conducted under llm supervision to ensure a fully automated pipeline that produces diverse, realistic, and reliable data for instruction-following tasks. experimental results on ifeval, followbench, and infobench demonstrate that our approach significantly improves llms' ability to follow complex instructions. we also conduct extensive analytical experiments to evaluate the effectiveness, scalability, and potential of our method.