E asy I nstruct: An Easy-to-use Instruction Processing Framework for Large Language Models (2024)

Abstract: In recent years, instruction tuning has gained increasing attention and emerged as a crucial technique to enhance the capabilities of Large Language Models (LLMs). To construct high-quality instruction datasets, many instruction processing approaches have been proposed, aiming to achieve a delicate balance between data quantity and data quality. Nevertheless, due to inconsistencies that persist among various instruction processing methods, there is no standard open-source instruction processing implementation framework available for the community, which hinders practitioners from further developing and advancing. To facilitate instruction processing research and development, we present EasyInstruct, an easy-to-use instruction processing framework for LLMs, which modularizes instruction generation, selection, and prompting, while also considering their combination and interaction. EasyInstruct is publicly released and actively maintained at Github, along with an online demo app and a demo video for quick-start, calling for broader research centered on instruction data and synthetic data.

## introduction large language models (llms) have brought about a revolutionary transformation in the field of natural language processing (nlp), leading to substantial improvement in performance across various tasks @xcite @xcite @xcite @xcite . to optimize the performance of llms in specific tasks or domains, it is crucial to adapt their outputs to specific contexts or instructions. recent studies @xcite @xcite have proposed instruction tuning methods for fine-tuning llms, which is a prominent research area aimed at optimizing the llms' behavior by providing explicit instructions during training, enabling better control and alignment with user preferences and desired outputs. instruction dataset construction, which is also referred to as data engineering or management, poses a significant challenge in the process of instruction tuning @xcite @xcite . substantial efforts have been dedicated to the task of construction instruction data through human annotations @xcite , requiring a significant allocation of resources. against this backdrop, llms are utilized to synthesize large-scale instruction data automatically @xcite @xcite . these methods could scale up the size of instruction-following data, but they still inevitably suffer limited diversity and complexity, resulting in an unbalanced distribution and poor quality of instruction data. recent studies @xcite @xcite have unveiled a seminal revelation, indicating that even a small quantity of high-quality instruction data has the potential to yield robust performance. in general, instruction processing is an important process requiring careful attention to detail and rigorous quality assurance procedures to construct a high-quality instruction dataset for llms. unfortunately, the availability of open-source tools for instruction processing remains limited, especially in comparison to many opensource projects on models and training infrastructures @xcite @xcite ; zeng seed data (optional) generated instruction selected instruction raw instruction instruction response self_instruct_generator evol_instruct_generator backtranslation_generator kg2instruct_generator length_selector deduplicator perplexity_selector rouge_selector gpt_score_selector icl_prompt cot_prompt ie_prompt mm_prompt ... ... ... generators selectors prompts apis & engines ... ... advanced components extension zero-code instruction processing low-code customization novice users intermediate users experienced users to address this issue, we develop easyinstruct as depicted in figure 1 , an easy-to-use instruction processing framework for llms. given some existing chat data, corpus, or knowledge graphs, easyinstruct can handle instruction generation, selection, and prompting processes, while also considering their combination and interaction. these consistencies facilitate further development and comparisons of various methods, thus promoting the advancement of better instruction processing work. we further conduct experiments with easyinstruct to validate its effectiveness in instruction processing. currently, easyinstuct is open-sourced on github and has already received over 300 stars. we are committed to the long-term maintenance of easyinstruct, providing continuous support for new features to ensure its effectiveness as a framework for instruction processing and synthetic data generation @xcite . ## background llms typically undergo two stages of training: pretraining and fine-tuning @xcite . despite the fact that large-scale pretraining is the key of the model's proficiency in generating natural language responses, these pre-trained models can still struggle with comprehending human instructions accurately. to bridge the gap between the training objectives and human objectives, instruction tuning is introduced as a potent strategy to amplify the controllability and capabilities and of llms in interpreting and responding to instructions @xcite @xcite @xcite . concretely, instruction tuning involves the method of refining pre-trained llms through supervised learning, utilizing examples structured as (instruction, input, output). in this format, instruction represents the human-given directive that outlines the task, input optionally offers additional context, and output signifies the expected outcome in alignment with the instruc-tion and any given input. despite the effectiveness of instruction tuning, constructing high-quality large-scale instructions which effectively encompass the target behaviors remains a non-trivial challenge in this realm. existing instruction datasets are often limited in terms of diversity, quantity, and creativity, which underscores the significance of instruction processing. one typical method for constructing instruction datasets is data integration. in this method, instructional datasets are constructed by merging existing annotated datasets with descriptions of tasks in natural language @xcite @xcite . another prevalent method for constructing instruction datasets is automated generation. to alleviate the need for extensive human annotation or manual data gathering, automated methods have been proposed to generate large volumes of instructional data through the use of llms. instructions can be sourced from chat data @xcite or expanded on a small set of seed instructions using llms @xcite @xcite . subsequently, the collected instructions are fed into llms to generate corresponding inputs and outputs. in easyinstruct, our primary focus lies on automated approaches for instruction generation due to their high efficiency and scalability. another promising research direction of instruction processing is the selection of high-quality instruction. recently, numerous studies @xcite @xcite have investigated the issue of the scale of the instruction dataset for fine-tuning and have indicated that merely increasing the number of instructions may not necessarily result in enhancements. instead, a modest volume of high-quality instruction data can influence the fine-tuning of llms, yielding solid performance. thus, optimizing the instruction dataset and enhancing its quality play a critical role in fine-tuning llms effectively. from a practical implementation point of view, instruction processing is actually complex and requires meticulous consideration. in this paper, we present , an easy-to-use framework to effectively and efficiently implement instruction processing approaches including instruction generation, selection, and prompting. through this framework, easyinstruct can help users to quickly comprehend and apply the existing instruction processing methods implemented in the package. ## design and implementation as illustrated in figure 1 , easyinstruct provides a complete instruction processing procedure built on pytorch and huggingface. in this section, we first introduce the design principles, and then detail the implementation of the major modules. ## evaluation in terms of evaluation, we will introduce the experiment setups and illustrate the empirical results of multiple modules implemented in easyinstruct to demonstrate its capability. ## conclusion and future work we present , an easy-to-use instruction processing framework for llms. easyinstruct can combine chat data, corpus, kgs and llms as an automated instruction generation tool, reducing the cost of manual data annotation. additionally, easyinstruct integrates a diverse set of instruction selection tools to optimize the diversity and distribution of instruction data, thereby improving the quality of fine-tuning data. easyinstruct is designed to be easy to extend, and we will continue to update new features (e.g., knowledgeable synthetic data generation) to keep pace with the latest research. we expect easyinstruct to be a helpful framework for researchers and practitioners to facilitate their work of instruction tuning on llms.