FANNO : Augmenting High-Quality Instruction Data with Open-Sourced LLM s Only (2025)

Abstract: Instruction tuning stands as a crucial advancement in leveraging large language models (LLMs) for enhanced task performance. However, the annotation of instruction datasets has traditionally been expensive and laborious, often relying on manual annotations or costly proprietary LLMs. Recent works explore approaches to synthesize data with open-sourced LLMs but require high-quality human-crafted seed data. In this work, we introduce , an end-to-end framework to synthesize high-quality instruction data with open-sourced LLMs and sampled unlabeled documents, eliminating the necessity for seed data. Starting from diverse pre-screened documents, the framework synthesizes complex and diverse high-quality instruction and response pairs in different stages. We propose a tagging-based prompt method to generate diverse and complex seed data and a UCB-based approach to augment more instruction data with the seed data. A novel Think Different prompt is proposed to address the distributional limitations of the seeds, further boosting the data diversity. Experiments prove that the can generate diverse and complex high-quality data even with a opensource small teacher model. The synthesized instruction data demonstrates performance that is comparable to, or even surpasses, baseline annotation methods with proprietary LLMs or open-sourced LLMs while requiring fewer instruction data samples.

## introduction large language models (llms) have made significant contributions across numerous fields like math reasoning and open-end question answering @xcite @xcite . instruction tuning @xcite enhances the model's abilities to follow user instructions even in unseen tasks @xcite . the instruction data is expected to have high quality in terms of correctness, complexity, and diversity @xcite . however, the human-annotated instruction data is prohibitively expensive when human experts are incorporated @xcite or the data has suboptimal quality when collected via crowd-sourcing @xcite . previous studies explore approaches to synthesize high-quality instruction data with either proprietary llms @xcite or open-sourced llms @xcite . while effective, existing methods often require access to expensive apis or hand-crafted seed datasets, imposing significant financial and operational barriers to widespread use. when synthesized with open-sourced llms @xcite @xcite , the data quality is less satisfactory as demonstrated by the benchmark performance of finetuned llms @xcite @xcite . these llms have a limited ability to follow instructions and provide insufficient control over the complexity and diversity of the synthesized data. for example, simply prompting open-sourced llms to generate diverse instructions @xcite still produces data that closely resembles the original seed data. in this work, we develop fanno (free an-notator), an end-to-end instruction data synthesis framework with sampled unlabeled documents and open-sourced llms only that enhances control over data quality. this framework methodically breaks down the annotation process into three distinct stages: document pre-screening, instruction generation, and response generation. the prescreening stage enhances the diversity of raw documents in both topic and structure, acting as an effective probe for eliciting high-quality data from llms. the instruction generation stage generates high-quality instructions through two steps: seed instruction generation and instruction augmentation. in the seed instruction generation phase, a tagging-based prompt is used to generate candidate seeds across various task types and difficulty levels, while in the instruction augmentation phase, the ucb algorithm autonomously selects high-quality instructions while maintaining an exploratory component for newly generated instructions. the proposed think different strategy further overcomes the distributional limitations of the initial seeds, enhancing the data diversity. empirical evidence on different benchmarks confirm the efficacy of fanno framework on two small llms. the generated dataset is virtually indistinguishable from instruction datasets like alpaca-gpt4, marking a significant stride in instruction data development. 1 2 related work instruction data generation two main approaches have been explored for instruction data creation: (1) human annotation, which leverages human expertise to design prompts and collect multi-task datasets spanning various categories @xcite . while producing high-quality data, manual annotation is effort-intensive and costly, especially for devising complex textual instructions. data quality enhancement related works in the field of enhancing data quality have focused on several key aspects, such as instruction difficulty, diversity, and correctness. humpback @xcite and kun @xcite utilize the language model's capability in combination with tailored prompts for data filtering. in addition, initiatives like genie @xcite and mods @xcite utilize specialized opensource llms for data filtering tasks. deita @xcite utilize fine-tuned large models to score the data for quality assessment. moreover, efforts like orca-math @xcite and reflection-tuning @xcite ) employ collaborative approaches with multiple lms and self-reflection to enhance data quality. ## conclusion the development of instruction data has been hindered by the high cost and labor-intensive nature. in this paper, we introduced fanno, an autonomous and low-cost end-to-end framework that addresses these challenges by streamlining the annotation process with open-sourced llms. fanno efficiently generates datasets of high quality, diversity, and complexity through a structured process involving pre-screening, instruction generation, and response generation. this unified process eliminates the need for pre-existing annotated data or costly api calls, advancing the instruction data development. empirical experiments validate the efficacy of fanno, underscoring the framework's potential to democratize access to high-quality instruction datasets.