Explore-Instruct: Enhancing Domain-Specific Instruction Coverage through Active Exploration (2023)

Abstract: Instruction-tuning can be substantially optimized through enhanced diversity, resulting in models capable of handling a broader spectrum of tasks. However, existing data employed for such tuning often exhibit an inadequate coverage of individual domains, limiting the scope for nuanced comprehension and interactions within these areas. To address this deficiency, we propose Explore-Instruct, a novel approach to enhance the data coverage to be used in domain-specific instruction-tuning through active exploration via Large Language Models (LLMs). Built upon representative domain use cases, Explore-Instruct explores a multitude of variations or possibilities by implementing a search algorithm to obtain diversified and domain-focused instruction-tuning data. Our data-centric analysis validates the effectiveness of this proposed approach in improving domain-specific instruction coverage. Moreover, our model’s performance demonstrates considerable advancements over multiple baselines, including those utilizing domain-specific data enhancement. Our findings offer a promising opportunity to improve instruction coverage, especially in domain-specific contexts, thereby advancing the development of adaptable language models. Our code, model weights, and data are public at https://github.com/fanqiwan/Explore-Instruct .

## introduction large language models (llms) have proven to be exceptional in following a variety of instructions. recent studies have shown that llms of a smaller size may address a variety of circumstances and adapt to various requirements if fine-tuned with a small number of instructionfollowing demonstrations @xcite . an important challenge thus is to construct diverse instruction-tuning data @xcite . early research utilized human-curated instructiontuning data covering a wide array of tasks, ranging from @xcite to super-naturalinstruction @xcite . more recently, self-instruct @xcite approach has been proposed to amplify the diversity of instruction-tuning data by querying instruction-tuned llms @xcite . these advancements signify major progress in the evolution of llms that can cater to a diverse set of user needs. in addition to those excelling in general domains, there is a growing demand for llms proficient in specific domains foot_0 , which can be naturally obtained by domain-specific instruction-tuning. like their counterparts on general domains, the creation of domain-specific instruction-tuning data can be split into two categories: human-curated and data generated by llms. specifically, the re-utilization of human-curated data examples has been explored in several scenarios, e.g., machine translation, arithmetic tasks @xcite @xcite , while the development of domain-aware self-instruct examples is a research area of growing interest @xcite . however, current methodologies for collating domain-specific instructiontuning data fail to encapsulate the wide range of potential instructions within a given domain as supported by the results depicted in figure 1 . this is primarily due to their over-reliance on human curation, which tends to be biased towards popular nlp tasks and lacks diversity. consequently, these methods fail to encompass the true diversity of tasks in the domain @xcite . to enhance the coverage of domain-specific instruction-tuning data, we introduce a novel instruction paradigm named explore-instruct. we posit that the domain space is inherently structured akin to a tree, reminiscent of cognitive science ontologies @xcite . drawing from the essence of classical search algorithms and incorporating the power of llms, explore-instruct is conceived to actively traverse the domain space and generate instruction-tuning data, not necessitating a predefined tree structure. specifically, our approach employs two strategic operations: lookahead and backtracking explorations. in this context, lookahead delves into a multitude of potential fine-grained sub-tasks, thereby mapping out a complex network of tasks, whereas backtracking exploration seeks alternative branches to widen the search boundary, hence extending the domain spectrum. through empirical analysis, we observe that a simple depth-first search (dfs) implementation proves to be sufficient in navigating vast domain spaces with minimal overhead. to validate the efficacy of explore-instruct, we select three disparate domains -rewriting, brainstorming, and math foot_1 -as our experimental testbeds. each of these domains encapsulates a unique use case and demands different skill sets, underscoring the versatility of our proposed approach. after applying explore-instruct, we analyze the data obtained from each domain and detect a comprehensive level of coverage. our findings suggest that the llms can explore not only a wide breadth of tasks within a specific domain but also delve deeply into these tasks and decompose them into fine-grained sub-tasks. fine-tuning domainspecific models using data garnered from the proposed method led to outperforming the baselines in both automatic evaluation and human evaluation. our contributions are summarized as follows: ## methods in this section, we begin by introducing the domain space representation in section 3.1. subsequently, we present the active exploration strategy, which encompasses both lookahead and backtracking explorations, in section 3.2. finally, in section 3.3, we provide a detailed description of the explore-instruct implementation. ## data-centric analysis to illustrate the efficacy of explore-instruct from a data-driven perspective, we perform a datacentric analysis. we first introduce the baseline methods for generating instruction-tuning data that we select for comparison. subsequently, we perform a comprehensive statistical analysis of the data obtained for each domain and examine the domain coverage yielded by different methods. ## results and analysis in this section, we conduct both automatic and human evaluations to justify the validity of explore-instruct. our explore-lm fine-tuned with data generated from explore-instruct outperforms the baselines in both evaluations. additionally, we perform in-depth analysis to investigate the influence of various factors, such as data structure, quantity, and quality on the performance of explore-lm. our analysis reveals that the maximum exploration depth has the most significant impact on performance, followed by the number of training instances and the utilization of the diversity filter. ## conclusion in this work, we introduce explore-instruct, a novel approach to enhancing domain-specific instruction coverage. drawing inspiration from classical search algorithms, explore-instruct leverages the power of llms to actively explore the domain space and obtain diverse and domain-focused instruction-tuning data. our experimental results demonstrate the efficacy of explore-instruct through data-centric analyses and model performance evaluations in the rewriting, brainstorming, and math domains, highlighting significant enhancements in instruction coverage and superior model performance compared to multiple baseline methods as demonstrated by both automatic and human evaluations. canwen xu, daya guo, nan duan, and julian mcauley. 2023b. baize: an open-source chat model with parameter-efficient tuning on self-chat data. arxiv preprint arxiv:2304.01196. da yin, xiao liu, fan yin, ming zhong, hritik bansal, jiawei han, and kai-wei chang. 2023. dynosaur: a dynamic growth paradigm for instruction-tuning data curation. arxiv preprint arxiv:2305.14327. yue zhang, leyang cui, deng cai, xinting huang, tao fang, and wei bi. 2023. multi-task instruction tuning of llama for specific scenarios: a preliminary study on writing assistance. arxiv preprint arxiv:2305.13225. chunting zhou, pengfei liu, puxin xu, srini iyer, jiao sun, yuning mao, xuezhe ma, avia efrat, ping yu, lili yu, et al. 2023. lima: less is more for alignment. arxiv preprint arxiv:2305.11206.