Explore-Instruct: Enhancing Domain-Specific Instruction Coverage through Active Exploration
Authors: Fanqi Wan
Xinting Huang
Tao Yang
Xiaojun Quan
Wei Bi
Shuming Shi
Venue: EMNLP 2023 Main (2023)
Keywords: Large Language Models
Instruction-Tuning
TL;DR: This paper proposes Explore-Instruct, a novel approach to enhance the data coverage to be used in domain-specific instruction-tuning through active exploration via Large Language Models.

Abstract: Instruction-tuning can be substantially optimized through enhanced diversity, resulting in models capable of handling a broader spectrum of tasks. However, existing data employed for such tuning often exhibit an inadequate coverage of individual domains, limiting the scope for nuanced comprehension and interactions within these areas. To address this deficiency, we propose Explore-Instruct, a novel approach to enhance the data coverage to be used in domain-specific instruction-tuning through active exploration via Large Language Models (LLMs). Built upon representative domain use cases, Explore-Instruct explores a multitude of variations or possibilities by implementing a search algorithm to obtain diversified and domain-focused instruction-tuning data. Our data-centric analysis validates the effectiveness of this proposed approach in improving domain-specific instruction coverage. Moreover, our model's performance demonstrates considerable advancements over multiple baselines, including those utilizing domain-specific data enhancement. Our findings offer a promising opportunity to improve instruction coverage, especially in domain-specific contexts, thereby advancing the development of adaptable language models. Our code, model weights, and data are public at \url{https://github.com/fanqiwan/Explore-Instruct}.

EXPLORE-INSTRUCT:
Enhancing Domain-Specific Instruction Coverage through Active Exploration
Fanqi Wan'; ; Xinting Huang") Tao Yang', Xiaojun Quan", Wei Bi², Shuming Shi? Sccoool of Computer Science and Engineering, Sun Yat-sen University, China
Teencett AI Lab
wannq, yang1225) @ @amil2.sysu.edu. quanxj3 @ mail.sysu.cdu..cn, (timxthuang, victoriabi, shumingshi] @tencent.com
Abstract
Verb
Noun
Speciin Hlmmar-Cursied
system
Instruction-tunne tuninn can be substantially opti- mized through enhanced diversity, resulting in models capable of handling broader spectrum tasks. However, existing data employed for such tuning often exhibit inadequate cover- age of individual domains, limiting the scope for nuanced comprehension and interactions within these areas. To address this deficiency, we propose EXPLORE-IOE SELEE-ISSREE INSTRUT, a novel ap- proach to enhance the data coverage be used domain-specific instruction-tuning through active exploration via Large Language Models (LLMs).. Built upon representative domain cases, EXPLORR-INSTRUCT explores mul- titude of variations or possibilities by imple- menting search algorithm to obtain diver- sified and dloaiin-foused instruction-tuning data. Our data-centir analysis validates the effectiveness of this proposed approach in im- proving domain-specific instruction coverage. Morcover, our model's performance demon- strates considerable advancements over multi- ple baselines, including those utilizing domain- specific data enhancement. Our findings offer a promising opportunity to improve instruction coverage, especially in domain-specific con- texts, thereby advancing the development of adaptable language models. Our code, model weights, and data are public https:// gi gitb.b..c
1nn
Demain-Awaro SelF-Instrett
probability
Fnd
Exploce-Insten
interer
find
nuumher
wille
exprvasion
contain
longth
00d
equuston
simp Iyy
expression
solve
equation
solution
find
valle
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Proportion
Figure 1: The top-15 most common verb-noun pairs in instructions of previous methods and EXPLORE- INSTRUCT for math problem-solving. It reveals an over-concentrated range of instructions in prior methods, while
offers broader coverage.
Early research utilized human ouman-curated instruction- tuning data covering wide array of tasks, rang- ing from FLAN (Chung et al., 2022) to SUPER- DATURALINSTRUCTION (Wang al., 2022b). More recently, SEL.F-NSTRUCC (Wang et al., 2022a) approach has been proposed to amplify the diversity of instruction-tuning data by querying instruction-tuned LLMs (Xu et al., 2023b; Ji et al.. 2023). These advancements signify major progress in the evolution of LLMs that can cater to diverse set of user needs.
In addition to those excelling in general domains, there is a growing demand for LLMs proficient specific domains which can be naturally obtained by domain-specific instruction-tuniin counterparts on general domains, the creation of domain-specific instruction-tune tunnnng data be split into two categories: human-curated and data gen- erated by LLMs. Specifically, the re-utilization of human-curated data examples has been explored in several scenarios, e.g., machine translation, arith- metic tasks (Liu and Low, 2023; Zhang et al., 2023;
Instrrcc..
1 Introduction
Like their
Large language models (LLMs) have proven to be exceptional in following variety of instruc- tions. Recent studies have shown that LLMs of a smaller size may address a variety of cir- cumstances and adapt to various requirements fine-tuned with a small number of instruction- following demonstrations (Zhou et al., 2023). An important challenge thus is to construct diverse instruction-tuning data (Gudibande et al.. 2023).
'In this context, 'domain' refers to specific use cases, such as writing assistance, mathematical problem-solving or cod- ing assistance. Fach of challenges, methodologics, and requirements.
Work was done during the internship at Tencent lab. Corresponding authors.
these presents its own unique set of

---

Jiao al., 2023; Wang et al., 2023), while the de- velopment of domain-aware self-instruct examples is a research area of growing interest (Chaudhary, 2023; Luo et al., 2023). However, current method- ologies for collating domain-specific instruction- tuning data fail to encapsulate the wide range of potential instructions within a given domain as sup- ported by the results depicted in Figure 1. This is primarily due their over-reliance curation, which tends be biased towards popu- lar NLP tasks and lacks diversity. Consequently, these methods fail to encompass the true diversity tasks in the domain (Wang et al., 2022a).
into fine-grained sub-tasks. Fine-tuning domain- specific models using data garnered from the pro- posed method led to outperforming the baselines in both automatic evaluation and human evaluation. Our contributions are summarized as follows:
We highlight the limitations in current domain- specific instruction-tunig data, emphasizing the inadequate coverage within individual do- mains.
human
We introduce EXPLORE-INSTRUCE a novel strategy for enhancing domain-specific cov- erage, which employs an active exploration method using LLMs.
To enhance the coverage of commain-specifee instruction-tuning data, we introduce novel in- struction paradigm named EXPLORE-INNTTUCT We posit that the domain space is inherently struc- tured akin to a tree, reminiscent of cognitive sci- ence ontologies (Newell et al., 1959). Drawing from the essence of classical search algorithms and incorporating the power of LLMs, EXPLORE- INSTRUCT is conceived to actively traverse the domain space and generate instruction-tuning data, not necessitating a predefined tree structure. Specif- ically, our approach employs two strategic opera- tions: lookahead and backtracking explorations. In this context, lookahead delves into a multitude of potential fine-grained sub-tasks, thereby mapping out a complex network of tasks, whereas backtrack- ing exploration seeks alternative branches to widen the search boundary, hence extending the domain spectrum. Through empirical analysis, observe that simple Depth-First Search (DFS) implemen- tation proves to be sufficient in navigating vast domain spaces with minimal overhead.
We empirically confirm the efficacy of EXPLORE-INSTRUCT through both data- centric analysis and model performances, showcasing its substantial improvements in in- struction coverage and superior performances over multiple baselines.
Related Work
2.1 Intrruction-Tuninge for General Domains
Recently, there has been a growing interest among NLP researchers in the area of instruction- tuning. Early research such FLAN (Wei et al., 2022) and TO (Sanh et al., 2022) uti- lize human-curated open-source datasets that encompass wide range of tasks. Building on this foundation, subsequent efforts such as SASPER--AAUURALINSSRICTTTTI
(Wang et al., 2022b), MctalCL (Min et al., 2022), and FLAN-T5 (Chung et al., 2022), seek to expand the scope and quality of the collected datasets by combining more tasks, introducing in-context ex- amples and chain-of- data. More recently. to reduce the cost of manual involvement, SELF- INSTRUCT (Wang et al.. 2022a) is proposed produce data via instruction-tume LLMs, such as ChatGPT (OpenAI, 2023a) and GPT-4 (OpenAI, 2023b). For example, Alpaca (Taori et al., 2023) uses ChatGPT to generate 52k instruction-tuning data from small manually written instruction seed set, LaMini-LM (Wu et al., 2023) develops a datasct of 2.58M examples based on both existing and generated instructions. Dynosaur (Yin et al., 2023) uses the metadata of datasets generate in- struction templates and determine the data fields with ChatGPT. Baize (Xu et al., 2023b) constructs multi-turn instruction-tumi tunnon data by requesting
To validate the efficacy of EXPLORE-INSTRUCT we select three disparate domains rewritingg brain- storming, and math 2_ as our experimental testbeds. Each of these domains encapsulates unique use case and demands different skill sets, underscor- ing the versatility of our proposed approach. Af- ter applying EXPLORE--INTTUCT we analyze the data obtained from each domain and detect com- prehensive level of coverage. Our findings sug- gest that the LLMs can explore not only a wide breadth of tasks within specific domain but also delve deeply into these tasks and decompose them
"The terms rewriting and brainstorming are drawn from existing works (Ji et al., 2023). The rewriting domain covers the tasks of refining and modifying text, whereas the brain- storming domain is oriented toward generating novel and open- ended passages.

---

ChatGPT to engage in conversation based on given topic. Dromedary (Sun et al., 2023) and Wiz- ardLM (Xu et al., 2023a) enhance the alignment and complexity of generated instructions respec- tively by heuristically modifying the input prompts.
way, the domain space representation offers struc- tured approach to obtain comprehensive domain- specific instruction-tuning data.
3.2 Active Exploration Strategy
The proposed EEPPORR--ISTRUUE comprises two fundamental operations: (1) lookahead exploration and (2) backtracking exploration, illustrated in Figure 2.
2.2 Instruction-Tuning for Specifie Domains
In addition to the works focusing on general do- mains, there is a growing demand for LLMs expe- rienced in specific domains. Following the success in general domains, the re-utilization of human- curated public datasets has been employed in sev- eral scenarios, such as arithmetic (Liu and Low, 2023), writing (Zhang et al., 2023), machine trans- lation (Jiao et al., 2023), and information extraction (Wang et al., 2023). Another line work adopts the self-instruct in specific domains, For exam- ple, CodeAlpaca (Chaudhary, 2023) and Wizard- Coder (Luo et al., 2023) develop the domain-aware self-instruct for the coding tasks. Our proposed differs from these methods by actively traversing the domain space with looka- head and backtracking explorations to enhance the instruction coverage.
Lookahead Exploration The first operation in- volves exploring the domain space in the depth di- rection, thereby mapping out complex network of potential fine- grained sub-tasks. Specifically, given task Vi, the lookahead exploration utilizes LLM to decompose V; into M distinct sub-tasks that dif- fer from those already present T. An example of the prompt template for lookahead exploration is as follows.:
You are asked to propose some new sub-tasks for the target task given the current exploration state.
Here are the requirements:
1. The skills required to perform sub-task belong to the skills required to perform the target task, and the former is a subset of the latter.
3 Methods
Current exploration state: (Exploration State)
In this section, begin by introducing the domain space representation in Section 3.1. Subsequently, we present the active exploration strategy, which encompasses both lookahead and backtracking ex- plorations, in Section 3.2. Finally, Section 3.3, we provide a detailed description of the EXPLORE- INSTRUCT implementation.
Target task: (Target task) Generate M new sub-tasks with the corresponding reasons:
Here, the placeholder (Target task) refers to the specific task Vú to be decomposed. The place- holder (Exploration State) is composed to ap- proximately represent the existing progress in T, which directs the exploration towards regions that are yet to be thoroughly examined.
3.1 Domain Space Representation
Backtracking Exploration In addition to looka- head exploration, backtracking exploration is an- other key operation in EXPLORE-INSTRUCY It seeks alternative branches in the domain tree T expand the search boundary and increase the diver- sity of tasks in the domain. Specifically, given a task Vj. we first perform the backtracking operation to find its parent task V; and then utilize a LLM to explore M new sub-tasks of V; breadth-win The prompt used for backtracking exploration is similar to that employed for lookahead exploration.
We posit that the coverage of domain-specific in- struction is influenced by two key factors: breadth. referring to the inclusiveness of different task cate- gories within the domain, and depth, which relates fine- aie-igrieedd task decomposition. The critical role of breadth is highlighted by its capacity to promote understanding of various categories of tasks within a domain (Hendrycks et al., 2021; Huang et al., 2023). On the other hand, depth pro- motes thorough understanding and precise problem- solving underscored by task decomposition stud- ies (Huang and Chang, 2022; Khot et al., 2023). Therefore, we model the domain space as tree structure T, with nodes V representing tasks and
3.3 EXPLORE-INSTRUCE Implementation
The detailed description of EXPLORE-INSE INSTUCTT implementation can be broken down into two dis-
edges E denoting the hicrarchical relationship be-
tween a task and its decomposed sub-tasks. this
Please refer to Appendix A for the complete prompt.

---

Task: Text Paraphrusing
Task: Rephrase with Given Structure
Lookahead Esploration
"task name": "npphnse with givan sinicture" "reason": "Being able follow gives structure
"cask name": "lead paraphasing"
Bocktracking Explocation
"reason"; "The purpuses of pempliaing show understanding, make accessible...
importaii many fielde such ausemmic...
"instances": [[
"iastances" [f
Task: Text Rewriting
"instruction": "Rophrase this rest", "Inpur":"Free taking his vacation Dec.."
"Instruction" " "Rewzile the sontence with the structuret subj. v. adt n.", "inpul" : "Her bithada party loi of fim"
"task name": "hest rewriling"
"output": "During Frod be vacation." " F...]]
"outpel": had joyous birthday part.""
nownling madales while keeping the meining but presenting differ..." "Instances":
"Instraction" "
'Rewiite this test in another way.".
Task: Style Transfer
Task: Personality Text
"Iaput": "The
John Sarah.".
"output": "Sarah
"task name" "sllle Inmafer"
personulity
}...B
"ressson": "It can injrove wriling efficiency and text diversity, while stimulatin creativity... ..."
"reason": "Adding pecsonality tast can improves
its readabilit and make it more enjoyable...
"Instances": [1
"indances" [1
"instruction"" "Add excitement
"Iastruction": "Make sentence sesual".
"inget" "We'r re going the beach today."
"taput"; "We should discuss at next meeting,"
"uutput": "I bilk about at nost meeting.
"outpur": "Yay, get ready for some fan in the sun bocause we' to bitting the beach today!"
]..]]
I...]I
Figure 2: The overview of proposed EXPLORE- INSSRUUUT. It involves two strategic operations: (1) lookahead exploration, and (2) backtracking exploration. The lookahead exploration delves into multitude of potential fine- grained sub-tasks, whereas the backtracking exploration seeks alternative branches to widen the search boundary.
tinct processes: (1) domain exploration, and (2) subsequent instruction-tumie tunnonngttnnnnnnnnn data generation.
select for comparison. Subsequently, we per- form a comprehensive statistical analysis of the data obtained for each domain and examine the domain coverage yielded by different methods.
Domain Exploration Strategy The exploration process begins at the root task of the domain and proceeds by traversing each node in Depth First Search (DFS) manner, accompanied by ei- ther lookahead exploration or backtracking explo- ration. The stopping criterion for either action is determined by a maximum exploration depth K, or breadth B. By incorporating both lookahead and backtracking explorations, the process achieves ef- fective exploration of the domain space, thereby enhancing the model's adaptability to diverse range of user requirements.
4.1 Baselines
We select the following baseline methods for com- parison with EXPLORE-INSTRUCTS.
Domain-Specifie HumannCurated This method collates data related to specific domain from var- ious open-source datasets. To ensure data qual- ity, we carefully select datasets within the SUPER- N'TURALLNSTRUCTION and randomly sample 10,000 training instances
Domain-Aware Self-Instruct This method ob- tains domain-specific instruction-tuning data through LLM. It is simplified version of IXPLORE-INSTRUC,, with a maximum explo- ration depth of K 0. Specifically, an initial domain-specific seed collection with diverse in- structions is used. During each iteration, two exam- ples are randomly selected from the seed collection. Subsequently, the newly generated instruction data is integrated into the seed collection, and the pro- cess is reiterated. To ensure fair comparison, we randomly sample 10,000 training instances.
Instruction-Tuning Data Generation For each task in the domain tree, we follow prior work (Taori et al., 2023) and use LLM to generate N instruc- tions and corresponding responses*. To ensure the diversity of instructions, we adopt diversity filter during domain exploration and instruction-tuning data generation. Specifically, a decomposed sub- task or generated instruction is retained only if its ROUGE-L overlap with any existing task or in- struction is less than threshold.
4 Data-Centric Analysis
4.2 Data Statisties
To illustrate the efficacy of EXPLORE-INSTRUCT from a data-driven perspective, we perform a data- centric analysis. We first introduce the baseline methods for generating instruction-tuning data that
Table presents the essential statisties of the domain-specific instruction-tuning data generated using various methods, including Domain-Specifet
'The implementation details of EXPLORE-INSTRUCT can be found in Section 5.3.
'Please refer to Appendix A to find the prompt for instruction-tuning data generation.
'The selected datasets can be found in Appendix C.

---

Data Stat.
Brainstorming Rewriting
Math
Domain-Specifee Humonn-Carated
Unique V-N pairs
8
Occur. of V.N pairs (Avg.)
5000
778
1355
Occur: of V-N pairs (Sid.)..
1447
835
779
valae
Domaie Awure Self Iusiruct
Unique V.N pains 1
781
1715
451
Occur. of V-N pairs (Avg.)l
5
13
Occur. of V-N pairs (Std.)..
14
subve
Sreasto
EXPLORE INSTRUCT
Unique V-N pairs
790
2015
917
equation
Occur: of V-N pairs (Avg.) Occur: V-N pairs (Sid.).
(a) Domain-Awar Self-Instruct
Table 1: Statistics of verb-noun (V-N) pairs in the instructions obtained from Domain-Specific Human- Curated, Nomain-Award Self-Instruct, and EXPLORE- INSTRUCT in different domains.
Glic
Human-Curation Domain-Aware Self-Inst and EXPLORE-INSTRUCT Notably, the number of unique verb-noun pairs in the instructions pro- duced by EXPLORE-INSTRRUT is higher than that Domain-Specific Human-Curation and Domain- Aware Self-Instruct. Moreover, the average and standard deviation of the number of occurrences is lower. This phenomenon is apparent in the rewriting and math domains, whereas it is less pro- nounced in the brainstorming domain. To provide more explicit demonstration, we compare the verb- noun pairs in the generated instructions whose fre- quency is above between Domain-Awar Self- Instruct and EXPLORE-INSTRUGE in the math do- main and show the results in Figure 3. The verb- noun pairs are distributed more uniformly in the instructions generated by EXPLOREE-ISTRUCCT
expresdoon
erperviio
fell
tanic
(b) EXPLORE-INSTRUCY INSTRUCT
Figure The root verb-noun pairs in instructions of (a) Domain-Aware Self-Instruct and (b) EXPLORE- INSTRUCT in the math domain, where the inner circle of the plot represents the root verb of the generated instruc- tions, and the outer circle represents the direct nouns.
5 Experimental Settings
5.1 Benchmarks
To validate the efficacy of SEPLLORE-INSTRUCE we chose three distinct domains: rewriting, brain- storming, and math, experimental testbeds. Each of these domains represents unique the scope of required skill sets varies differently, highlighting the versatility of our approach. The testbeds for the brainstorming and rewriting do- mains are derived from the corresponding cate- gories in the translated test set of BELLE (Ji et al., 2023). While for the math domain, we randomly se- lect 500 questions from the test set of MATH (Dan et al., 2021; Lightman al., 2023). The statistics of these testbeds be found in Appendix D.2. Consistent with prior research (Peng et al., 2023; Chen et al., 2023), for the brainstorming and rewrit- ing domains, employ both automatic and hu- man evaluations evaluate the performance of domain-specific models. As for the math domain, we measure the performance using the Accuracy Rate metric in solving math problems.
Furthermore, we analyze the distribution of the average ROUGE-L overlap between the generated instructions as well as the existing instructions of baseline methods and EXPLORE-INSTRUCT. Specifically, for each generated instruction, we compute its average ROUGE-L score with the exist- ing instructions generated before it, and the result distributions are shown in Figure 4. Compared with Nomain-Specific Human-Curation and Domain- Aware Self-Instruct, the average ROUGE-L scores for EXPLORE- INSTRUCT are concentrated in re- gions with relatively smaller values.
and
These findings suggest that, compared with base- line methods, which exhibit an inadequate coverage of generated instruction-tuning data, EXPLORE- INSTRUCT can enhance the diversity and domain coverage of the instructionttunin data through depth-wise and breadth-win wise active exploration of the domain space.

---

Instrrccions
to M 3, and generate N 500 instructions and responses. The filter threshold set to 0.7. Once the process is complete, we randomly sam- ple 10,000 training instances from each sub-task data to fine-tune the Explore-LM, with the num- ber of instances in cach sub-task serving as the sampling weights. To further enhance the perfor- mance of the domain-specifie model, we increase the number of sampled instances fine-tune the Explore-LM-Ext. We utilize apt-3.5-turf perform domain exploration and data generation, with a maximum length of 4096, a temperature of 1.0, and a top-p of 1.0 during API requests. contrast to Domain-Awar Self-Instruct, which per- forms the data generation process only, the primary additional cost of tokens associated with EXPLORE- INSTRUCT comes mainly from the domain explo- ration process. Since this exploration occurs at the task level and allows for simultaneous exploration of multiple tasks, the additional cost is relatively minimal and approximately less than 1%. Please refer to Appendix D.1 for the statistics of the final instruction-tumie data.
10"
Distuii Specilie Cariee
Domoin Negre Self Instrct Explare Iastuct
10'
0.2
0.6 ROUGEEL
(a) Brainstorming
Instroctioons
10"
Domain Specifl Human Cuaie
Dussuit Aware Sef-lantrcc Explone-beeuct
10'
0.0
0.2
0.4
0.6
0.8
1.0
ROUGE-I.
(b) Rewriting
Inatrictions
Chamiin-Speciii Huraa-Charioo Self-hoarset Explare-Inatucii
10"
0.0
0.2
0.4
0.6
0.8
1.0
ROUGE-L
(c) Math
Figure 4: The distribution of average ROUGE-L overlap between generated and existing instructions of Domain- Specific Human Human-Curated, Domain-Awan Instrucst and EXPLORE-INSTRUEE in different domains.
Training Details We utilize LLaMA 7B (Tou- vron et al., 2023) as the backbone model. We adopt AdamW optimizer with a learning rate of 2c-5 and the batch size is 128. The maximum length is 512. We train all models using V100 GPUs with deep- speed Zero-3 offload for epochs.
5.2 Explore-LM and Baseline Models
Explore-LM Explore-LM is domain-specific assistant developed by implementing the proposed EXPLORE-INSTRUCE and fine-tuning the back- bone model on the obtained instruction-tuning data.
Inference Details We use a sampling method with a temperature 0.7 to diversify outputs for the brainstorming and rewriting domains. For the math domain, we employ a beam search with a beam size of 10 to reduce output randomness and ensure more deterministic outputs. The maximum generation length during inference is 512.
Baseline Models To facilitate a meaningful comparison with our proposed Explore-LM, we adopt two baseline models, namely Domain- Curated-LM and Domain-Instruct-Ln These mod- els are fine-tuned with the same backbone using instruction-tuning data generated from Domain- Specific Human- -urrated and Domain-Aware Self- Instruct, as described in Section 4.1. Further- more, we compare the performance of Explore-LM against ChatGPT, which serves as an upper bound in our evaluation.
Results and Analysis
In this section, we conduct both automatic and hu- man evaluations to justify the validity of EXPLORE- INSTRUCT. Our Explore-LM fine-tuned with data generated from EXPLORRE-INTTUCT outperforms the baselines in both evaluations. Additionally, we perform in-depth analysis to investigate the influ- ence of various factors, such as data structure, quan- tity, and quality on the performance of Explore- LM. Our analysis reveals that the maximum ex- ploration depth has the most significant impact on performance, followed by the number of training instances and the utilization of the diversity filter.
5.3 Implementation Details
To construct the domain- specific instruction-tumingg data, actively tra- verse the domain space and generate data with maximum exploration depth of K 2. We set the maximum exploration breadth to B = and B = 6 in the first and second depths, respec- tively. We set the number of exploration sub-tasks
EXPLORRF-INSTRUCT

---

Brainstorming
Rewriting
Automatic Comparison
Win:Tie:Lose
Beat Rate
Win:Tie:Lose
Beat Rate
Explore-LM Komain-Curated--La
194:1:13
93.72
50:38:6
89.29
Explore-LMM-Ext vs Domain-CCrated-LMM
196:1:11
94.69
53:37:4
92.98
Explore-LM vs Domain-Instruce Instruct-IMM
114:56:38
75.00
34:49:11
75.56
Explore-LMMExx IamaiinInstruct-LMM
122:55:31
79.74
35:53:6
85.37
Explore-LM vs ChatGPT
52:71:85
37.96
11:59:24
31.43
Explore-LM--at ChatGPT
83:69:56
59.71
12:56:26
31.58
Table Automatic evaluation results the brainstorming and rewriting domains. demonstrates that Explore-LM outperforms multiple baselines with large Beat Rate and nearly matches the performance of ChatGPT. Additionally, with substantial increase in training instances, the performance of Explore-LM Le-ore-----Ettt Exxt can be further enhanced.
Math
compared Domain-Curated-LM. Similarly, when compared Domain-Instruct-LNn Explore-LM achieves Beat Rates of 75.00 and 75.56 in the same domains. In the math domain, Explore-LM achieves an Accuracy Rate of 6.8, compared to Domman-Curated-LLN
Models
Accuracy Rate
CommannCCurated--i
3.4
Iommain-nstruutt-MMM
4.0
Explore-LMt
6.8
Explore-LM-Ei. Lx-Ext
8,4
ChatGPT
34.8
3.4 and Domain-Instrue LM's 4.0. Moreover, when we increase the number of instances, Explore-LM--xx demonstrates even better performance, surpassing ChatGPT with Beat Rate of 59.71 in the brainstorming domain.
Table 3: Automatic evaluation results in the math do- main. The results illustrate that Explore-LM achieves significant improvements on baseline models.
6.1 Automatic Evaluation
6.2 Human Evaluation
To evaluate the performance of Explore-LM in the brainstorming and rewriting domains, we fol- low Chen et al. (2023) to conduct an automatic evaluation with ChatGPT. We do not use gpt-4 but use g. 5-5turbo as the evaluator since the limited quota of the OpenAl account. Specifically, we adopt pair-wise evaluation, where given ques- tion and two responses from different models, we request ChatGPT to determine which response is better based on their helpfulness, relevance, accu- racy, and level of detail, with the corresponding reasons and justifications" To calculate the Beat Rate of particular model, we divide the number of times the model wins by the sum of the num- ber of times the model wins and loses. For the math domain, we utilize the automatic evaluation script from MATH to calculate the Accuracy Rate of model 8
To provide a more comprehensive and reliable eval- uation, we incorporate human evaluation for both the brainstorming and rewriting domains. Specifi- cally, enlist three annotators to compare the an- swers generated by two models for the same ques- tion and indicate which model provided the better answer (win, tie, or lose). To maintain anonymity and prevent any potential bias, the evaluated mod- els are kept anonymous, and the annotators remain unaware of which model generated each answer.
Figure 5 illustrates the results of the human eval- uation. The comparison between the human and automatic evaluations demonstrates a general con- sistency, indicating that our approach is also quali- tatively well-regarded by humans, underscoring its effectiveness. When comparing Explore-LM, with ChatGPT, the latter generates answers that pre- ferred by humans in most cases. This preference can be attributed to the reinforcement learning with human feedback approach employed in ChatGPT.
The automatic evaluation results are shown in Table 2 and Table 3. Explore-LM exhibits a sig- nificant performance advantage over multiple base- line models in all domains while utilizing an equal number of training instances. In the brainstorm- ing and rewriting domains, Explore-LM achieves Beat Rates of 93.72 and 89 28, respectively, when
6.3 Data Structure Analysis
To analyze the impact of data structure on the per- formance of Explore-LM, we conduct experiments in which we vary the maximum exploration depth within a range of to 2 while sampling 10,000 training instances. We compare the performance of
Thhe evaluation prompt can be found in Appendix 'https: / thub. com/hendrycks/math

---

Lat Mia
Lal Love
Lel War
Tx
Leli Lez
Ex
Domsin Curated-LM
Explore-LE LM
Demain Matmain-Cursee---- LM
Explore-LM--B Ext
Domsin-Curatdl-ii
Explore-LM-Bit
Domain-Chrttedd
Explore-L.M
Dommin-Instruc---a
Explore-LMi
Docmain-Issiruct-L.e
Explore-LM-Bs
Domsin-Inntrwe-LM
Explon-LMM-Ext
Demall--nsttruoLLM
Explore-LMM
CharGPT
Explore-IMM
Explore-LM-Ert
ChatGPT
Explon-LLM-xxt
ChatGPT
100 125 150 200
20 40
Number of Questions
Number of Questionn
Figure 5: Human evaluation results the brainstorming (Left) and rewriting (Right) domains.
75
Brainstom
Models
Brainstorming
Rewriting Beat Rate
Math
6.5
Bent Rate
Accurncy Rate
Heat Rate
70
Math
6.0
Explore-IMM
w/o filter
75.00 (10.9%%)
74.34
75.56 (18.305))
69.77
6.8 (13.08%)
65
5.5
60
Table Results of data quality analysis, where "filter" denotes the diversity filter which controls the diversity in the generated instraction-tuning data.
5.0
55
4.5
50
0
2
Maximum Depth
the maximum exploration depth can enhance the coverage of eestruction-tunigg data, thus improv- ing the performance of Explore-LM across various domains and scenarios.
85
Brainstom
5.6
Rewrite
5.4
80
Mach
5.2 g
Beat Rate
5.0
75
6.4 Data Quantity Analysis
4.8
4.6
We also investigate the influence of data quantity. To this end, conduct series of experiments in which we vary the number of training instances exponentially, ranging from 2,000 32,000 while keeping a consistent maximum exploration depth. We then compare the performance of Domain- Instruct-LM using 10,000 instances against that of Explore-LM using varying numbers of instances. The results are shown in Figure 6 (Down). Our findings indicate that Explore-LM using 2,000 in- stances outperforms Domain-Instruct-LN using 10,000 instances and even achieves a Beat Rate of nearly 80.00 in the brainstorming domain. More- over, our results demonstrate that data quantity pro- vides varying benefits across different domains. For instance, in the brainstorming domain, increasing the number of instances from 2,000 to 32,000 re- sults in an increased performance of only 1.42%. In contrast, for the rewriting and math domains, this can lead to substantial performance gains of 35. 35.42% and 21.74%, respectively.
4.4
65
4.2
60
4.0
2,000
4,000
8,000
16,000
32,000
Training Instances
Figure Performance of Explore-LM with different maximum exploration depths (Upper) and the number training instances (Down).
Monain-Insttuuct-MM and Explore-LM with vary- ing maximum depth. The results presented in Fig- ure 6 (Upper) illustrate that increasing the maxi- exploration depth generally leads to signifi- cant improvements in model performance. Specif- ically, we observe an increase in performance of 51.52% and 61.90% for the rewriting and math domains, respectively. However, for the brain- storming domain, the model demonstrates optimal performance at relatively small maximum depth. We posit that in domains with comparatively small scope of search space, such brainstorm- ing, high coverage is already achieved at shallow depth, and further increasing the exploration depth may result in duplicated tasks that reduce diver- sity, and ultimately negatively impact the model performance. Our findings indicate that increasing
6.5 Data Quality Analysis
In this section, we discuss the impact of data qual- ity. Specifically, we conduct an automatic com- parison between MonnainnInstuuct-L... and Explore- LM, both with and without the diversity filter men-

---

tioned in Section 3.3. We maintain consistency in terms of the maximum exploration depth and the number of training instances. The results are shown in Table 4. is observed that the performance of Explore-IM improves slightly across all domains with the use of the diversity filter. The brainstorm- ing domain demonstrates a marginal improvement of only 0.89% whereas the rewriting and math do- mains exhibit more significant improvements of 8.30% and 3.03%, respectively.
Ethics Statement
We state that any research or application arising from this study is strictly authorized solely for re- search purposes. The BELLE and MATH test sets used in our work are obtained from public sources and do not contain any private information. Our research adhered strictly to the data usage policy.
References
Sahil Chaudhary, 2023. Code alpaca: An instruction- following llaa model for code generation. https: //github.com www.ccoo//alllcccccccccc.......c maahi12811 11/ccoeaalpcc
7
Conclusion
In this work, we introduce EXPLORE-INSTRUCE a novel approach to enhancing domain-specific instruction coverage. Drawing inspiration from classical scarch algorithms, RAPLORE-INSTRUSE leverages the power of LLMs to actively ex- plore the domain space and obtain diverse and docain-focused instruction-ttuinng data. Our ex- perimental results demonstrate the efficacy of EXPI.ORE-INSTRUCE through data-centric anal- yses and model performance evaluations in the rewriting, brainstorming, and math domains, high- lighting significant enhancements in instruction coverage and superior model performance com- pared to multiple baseline methods as demonstrated by both automatic and human evaluations.
Zhihong Chen, Feng Jiang, Junying Chen, Tiannan Wang, Fei Yu, Guiming Chen, Hongbo Zhang. Juhao Liang, Chen Zhang, Zhiyi Zhang, Jianquan Li, Xi- ang Wan, Benyou Wang, and Haizhou Li. 2023, Phoenix: Democratizing chatgpt across languages. arXiv preprint arXiv:2304. 10453.
Hyung Won Chung. Le Hou, Shayne Longpre, Bar- ret Zoph, Yi Tay. William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, al. 2022. Scaling enstruction-finctunee language madels. arXiv preprint arxiv:2210. 11416.
Hendrycks Dan, Burns Collin, Kadavath Saurav, Arora Akul, Basart Steven, Tang Eric, Song Dawn, and Steinhardt Jacob. 2021. Measuring mathematical problem solving with the MATH dataset. Thirty- fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track.
Acknowledgements
Arnay Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song. 2023. The false promise imitating proprietary 1im.. arXiv preprint 157777730..15717
This work was supported by the National Natu- ral Science Foundation of China (No. 62176270), the Guangdong Basic and Applied Basic Research Foundation (No. 2023 2832), and the Ten- cent Al Lab Rhino-Bird Focused Research Pro- gram.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Mcasuring massive multitask language under- standing. In International Conference on Learning Representations.
Limitations
Jie Huang and Kevin Chen-Chuan Chang, 2022. To- wards reasoning in large language models: A survey. arXiv prepriat arXiv:2212 10403.
The limitations of EXPLORE-INSTRUCT are dis- cussed as follows. Firstly, our approach relies on the power of LLMs which have limitations in terms of their resource requirements and computational complexity, which may limit the scalability of our approach. Secondly, our study focuses on the en- hancement of domain-specific instruction coverage and does not address other aspects of instruction- tuning, such as the generation of complex and chal- lenging instructions or the identification and mitiga- tion of toxic and harmful instructions. Future work is needed to explore the potential of our approach in these areas.
Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Jiayi Lei, Yao Fu. Maosong Sun. and Junxian He. 2023. C- eval: A multi-level muuli--isccpplan chinese evalu- ation suite for foundation models. arXiv preprint arXiv:2305.08322.
Yunjie Ji, Yong Deng, Yan Gong, Yiping Peng. Qiang Niu, Lei Zhang, Baochang Ma, and Xiangang Li. 2023. Exploring the impact of instruction data scaling large language models: An empirical study on real-world use cases. arXiv preprint arXiv:2303. 144442.

---

Wenxiang Jiao, Jen-tse Huang. Wenxuan Wang, Xing Wang. Shuming Shi, and Zhaopeng Tu. 2023. Parrot: Translating during chat using large language models. arXiv preprint arXiv:2304.0244 02426.
Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Cox, Yiming Yang. and Chuang Gan. 2023. Principle-driven self- alignment of language models from scratch with min- imal human supervision.
Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sab- harwal. 2023. Decomposed prompting: A modular approach for solving complex tasks. In The Eleventh International Conference on Learning Representa- tions.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An dastructionffollowing llama model. https: // gi thub. www.w..wo/taaaaeeeeeeeeeeeeeet/e http://tttslaab///ea htab//stafooddaapaacee
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Maric-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and effi- cient foundation language models. arXiv preprint 1397.
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let's verify step by step. arXiv preprint 2.Xiv:2305.20050
Tiedong Liu and Bryan Kian Hsiang Low. 2023. Goat: Fine-tuned Ilama outperforms gpt-4 on arithmetic tasks. arXiv preprint ar arXiv:2305.14201.
Xiao Wang. Weikang Zhou, Can Zu, Han Xia, Tianze Chen, Yuansen Zhang, Rui Zheng, Junjie Ye, Qi Zhang, Tao Gui, et al. 2023. Instructuie: Multi- task instruction tuning for unified information extrac- tion. arXiv preprint arXiv:2304.00 08085.
Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xi- ubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2023. Wizardcoder: Empowering code large language models with evol- instruct.
Yizhong Wang. Yeganch Kordi, Swaroop Mishra, Al- isa Liu, Noah A Smith, Daniel Khashabi, and Han- naneh Hajishirzi. 2022a. Self-instruct: Aligning lan- guage model with self generated instructions. arXiv preprint 122552212..0566.
Sewon Min, Mike Lewis, Luke Zettlemoyer, and Han- naneh Hajishirzi. 2022. Metaicl: Learning to learn in context. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2791-2809.
Yizhong Wang. Swaroop Mishra, Pegah Alipoor- molabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Gary Lai, Is- han Purohit, Ishani Mondal, Jacob Anderson. Kirby Kuznia, Krima Doshi, Maitreya Patel, Kuntal Kumar Pal, M. Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Shailaja Keyur Sampat, Savan Doshi, Siddharth Deepak Mishra, Su- jan Reddy, Sumanta Patro, Tanay Dixit, Xudong Shen, Chitta Baral, Yejin Choi, Noah A. Smith, Hanna Hajishirzi, and Daniel Khashabi. 2022b. huper-natuuallinstrrtti
Allen Newell, John Shaw, and Herbert Simon. 1959.
Report general problem solving program. In IFIP congress, volume 256, page 64. Pittsburgh, PA.
OpenAI. 2023a. Chatgpt: Optimizing language models for dialogue.
OpenAL. 2023b. Gpt-4 technical report.
Baolin Peng, Chunyuan Li, Pengeheng He, Michel Gal- ley. and Jianfeng Gao. 2023. Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277.
Generalization via declar- ative instructions 1600+ nlp tasks. In Conference on Empirical Methods Natural Language Process- ing, pages 5085-5109
Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M. Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szezechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang. Matteo Manica, Sheng Shen. Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Tr- ishala Neeraj, Jos Rozen, Abhcesht Sharma, An- drea Santilli, Thibault Fevry, Jason Alan Fries. Ryan Techan, Teven Le Scao, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M Rush. 2022. Multi- task prompted training enables zero-shot task gener- alization. In International Conference on Learning Representations.
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoe V Le. 2022. Finetuned language mod- els are zero-shot learners. International Confer- ence on Learning Representations.
Minghao Wu, Abdul Waheed, Chiyu Zhang, Muham- mad Abdul-Mageed, and Alham Fikri Aji. 2023. Lamini-Im: A diverse herd of distilled mod- els from large-scale instructions. arXiv preprint 230i 14402..
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023a. WizardIm: Empowering large lan- guage models to follow complex instructions.

---

Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley, 2023b. Baize: An open-source chat model with narameter-efficient tuning on self-chat data. arXiv preprint arXfv:2304.01196
Domains
Taoks
tssk582
puturalquestonn answer generation
Brainsooming
task669 aasb__ggiggaaaee
task898 freehase__eeppp wp_.nwwrr_.eeee..eee........eee. wtt.eeppg.seeeeeeeeeaeoeo
miscellaneoen, generation
parpprrasing
task dais jest, modificttion
para-oot pamphasinn
Da Yin, Xiao Liu, Fan Yin, Ming Zhong, Hritik Bansal, Jiawei Han, and Kai-Wei Chang. 2023. Dynosaur: dynamic growth paradigm for instruction-tuning data curation. arxiv preprint 23xi:22305.1337.
ask7700ppwwsx, sawsx_engllshhccoo wwwas_englis_tees__mee lexxt__ddddiiciinn
ask1614_sick__
tssk _yyyuuuuubeee.
Rewrilig
askk557__jke 1155__iflgg_aswwee aiogg_anseer_eee _ilog__aswe__ererrtttoo
tskk1l ses_sssssssseee seteeesssmeeneeeee sraiaane_jimmlliScatio
tsk9933 silaa_wkkiaae btbki__utt_stll__aaaa
Zhang, Leyang Cui, Deng Cai, Xinting Huang, Tao Fang. and Wei Bi. 2023. Multi-task instruc- tion tuning of llama for specific scenarios: A pre- liminary study on writing assistance. arXiv preprint 0rxiv:2305.13225.
task955 wiki, auto style tramsler
takk927 yelp cetlpnngttiveeeee posiivve style transfer
task928 yelp positive Jo negative style trasfer
ask088 unnatural undaturalaaddub. suduuuubbao anihmmeti
task050 equation Jearnee_ alpebra
task092, wwwec__prie_cceeeeeeeeeeeeeeeeee craimeccassificatioon
task745 al2 arithmetic questions arithmetic
tas7751 svamp subtraction question answering
Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. 2023. Lima: Less is more for alignment. arXiv preprint arXiv:2305. 11266.
task752, svamp multiplication question answering
task753 _wwmmssaaaiiii idditionn question answering
task754, svamp wow.moo-mmiviiiii question annwerinn
task835 mathidateser answer generation
Math
addsub question, answering
task662 asdiv
aadaaddddiiiiilll multidv question answering
asdiv halli/mulippppoo quessiion..ooonnn.. unswering
A Prompts for EXPLORE-INSTRUCE
tak364 ardiv singlopp question, answeriin
mawps adisub question, answering unk366 mawps multidiv question snswering tsk866 nawps multiap
Here we provide the prompts for EXPLORE- INSTRUCT. The lookahead and backtracking ploration prompt is shown in Figure 7, and the domain-specific instruction-tuning data generation prompt shown in Figure 8.
answering
task668 mawps singlcop question answering ta56 mathqa corree answer
Table 5: Selected Domain Domain-Specifie Human-Curatee Curated
datasets from SAREERNAATURALNSTRUUTIINN
Stutisties
Brainstorming
Rewriting
Math
B Prompt for Automatic Evaluation
Domain DomaiinCCurttdd/ Curated-I.
# Sampled training instances Avg. instruction length Avg. inpur length
10,000
10,000
10,000
The prompt we use to conduct automatic pair-wise evaluation is shown in Figure 9.
74.10
11.75
88.81
19.36
Avg. oulpul length
2.42
90,50
1.14
Domain
Datasets Collection of Domain-Specific Human-Curated
Maximum
# Sampled maining instances
10,000
10,000
10,000
Avg. instruction length
13:20
15.98
18.68
Avg. input length
24.21
To obtain a collection of Domain- -Secific Human- Curated datasets, we selected tasks related to each domain in the and combined their corresponding datasets. The tasks chose are listed in Table 5.
Avg. output length
151.69
17.55
98.18
Explore-IMM
Maximum depth
Sampled training instances
10,000
10,000
10,000
Avg. instraction length
12.73
13.51
20.25
Avg. input length
10.57
Avg. ourpat length
173.29
27.93
121.59
Explore LM Ex
Data Statistics
Maximum depih
Sampled maining instances
16,000
32,000
64,000
Avg. instraction length
12.90
14.71
17.27
D.1 Statistics of Instruction- Tunnng Data
Avg. input length
21.57
20.24
12.88
Avg. output length
The statistics of the final instruction-tuning data for Domain-Curated-LM.. Explore-LM, and Explore-LMM-xxt in different do- mains are shown in Table 6.
Inntainn-Istruut-L..
Table Statistics of final instruction-tuning data for Domain-Curatede Domain-Instruct-MM Explore- LM, and Explore-LMMExt in different domains.
D.2 Statistics of Test Data
Statisties
Brainstorming
Rewriting
Math
Questions
208
94
500
The statistics of our testbeds in different domains are shown in Table Notably, the brainstorming and rewriting domains are characterized as open- ended, with no definitive standard answer available for evaluation within their testbed.
Avg. question length
16.05
42.78
57.64
Avg. answer length
167.57
Table 7: Statistics of testbeds in different domains.

---

E Comparison with Methods for the General Domain
We conduct automatic comparison experiments with three well-known baselines for the general do- main: Alpaca (Taori al., 2023), Dromedary (Sun al., 2023), and WizardLM (Xu al., 2023a). In these experiments, Alpaca utilizes its entire dataset of 52,000 instruction-tuning instances, whereas Dromedary and WizardLM each draw a sample 52,000 instances from their corresponding train- ing sets. The experimental results for different domains are shown in Table 8, Table 9, and Table 10, respectively. The results show that Explore-LM outperforms the baselines across all domains.
Automatic Comparison
Win: Tie:
Beat Rate
Explore-L.Me Alpaca
148:38:22
87.06
Explore-LM-Ebt M-Ext vs Alpaca
147:40:21
87,50
Explore-LM ys Dromedary
163:21:24
87.17
Explore-LM-Exx Dromedary
165:22:21
88.71
Explore-LM WizardLM
84:55:69
$4.90
Explore-L.M- vs WizardL
93:65:50
65.03
Table 8: Automatic comparison between Explore-LM and baselines in the brainstorming domain.
Automatic Comparison
LossLee
Beat Rate
Explore-I .M vs Alpaca
22:52:20
$2.38
Explure-1 LM-Ext Ext
Alpaca
64.10
Explore-LM Dromedary
40:40:14
74,07
Explore-LM-BB
Maalore-LM-Exs Ext ys Dromedary
43:42:9
82.69
Explore-LM vs WizardLM
23:49.22
51.11
Explore-LM-Ext WizarLLM
20:60:14
58.82
Table 9: Automatic comparison between Explore-LM and baselines in the rewriting domain.
Models
Accuracy Rate
Alpaca
3.8
Dromedary
1.8
WizardLM
2.2
Explore-LM
6.8
Explore-LM-Eis
8.4
Table 10: Automatic comparison between Explore-LM and baselines in the math domain.
F Case Studies
Table 11, Table 12, and Table 13, we show the case studies that examine the perfor- mance of DoMmainCCrateddLMM, Dostin-Instruct- LM, Explore-LM, Explore-LM-Ext, and ChatGPT in different domains respectively.

---

Exploration Prompt
You are asked propose some new sub-tasks for the target task given list existing sub- tasks and another list existing sibling tasks, then generate set examples each new sab-task. Each example consists
instruction, an input, an output.
Here are requirements:
1. The skills required to perform a sub-task belong to the skills required to perform the target task and the former a subset of the latter.
2. The skills required to perform a sibling task relate to the skills required to perform the target
task. There is an intersection of the former and the latter.
3. The sub-task and sibling task should focus on common domains, not specific domains.
4. A new sub-task is complementary to existing sub sub-tasks, and the addition of a new sub-task is essential to the completion of the target task.
5. The new sub-task should be different from the existing sub-tasks and sibling tasks. The skills
required for new sub-task should be designed to avoid overlapping with existing sub-tasks and sibling tasks.
6. The instruction should be in English.
7. The instruction should be to 2 sentences long. Either an imperative sentence or question is permitted.
8. The instruction should not contain specific examples and detailed content.
9. Try not to repeat the verb for each instruction in the examples to maximize diversity.
10. The instruction should be able to complete by a GPT language model. For example, the instruction
should not ask the assistant to create any visual audio output. For another example, do not ask the assistant to wake you at pm or set reminder because it cannot perform any action.
11. You should create appropriate input based on the instruction in an example, but the input should not respond to the instruction. The input should involve realistic data and should not contain simple placeholders. The input should provide substantial content to make the instruction challenging but do not exceed 280 words in general.
12. Note that some instructions do not require input. For example, when instruction asks about
some general information of sel contaccnt,,,,,, eg: "What is the highest mountain in the world." or Please list 5 different fruits.", ", it is not necessary to provide specific context. In this case, simply put "<noinput>" in the input field.
13. You should generate appropriate output according to the instruction and depending the input
in example. Make the output is less than 200 words in general.
14. The response you generated should conform the following format: ***
Instruction:
Input:
Output:
#n*
2. Instruction:
Input:
Output:
###
Target task: ferestttne_for__tet_eeeeeei
Examples:
1. Instruction:
Rewrite this text in another way
Input: People often think of cats as mysterious creatures because they are especially active at night Also, their eyes can glow an eerie green light in the dark.
Output: Cats are often considered mystical creatures because they are particularly active at night. Also, their eyes can emit a strange green glow in dark environments. #**
the content from the brief description below. Input: This is great phone with a high-resolution display.
2. Instruction:-Exten
Output: This is an excellent phone with formaner processor for faster performance and smoother multi masking Fur thermore,, it has resolution display for crisp, vibrant images. allowing users to enjoy better visual experience. ###
The list of already existing sub-tasks for this target task is: [' L'paraphrase', 'style_transfer' simplify. "ampliff__laguage""
The list of already existing sibling tasks for this target task is: []
The target task should be decomposed into total of 8 diverse and complementary sub-tasks, and there existing sub-tasks. Generate 3 new sub-tasks with the corresponding reason, then list 10 examples of this sub-task:
Figure Prompt used lookahead and backtracking explorations.

---

# Generation Prompt
You are asked to generate a set of examples for a new sub-task. Each example consists of an instruction, an input, and an output.
Here are the requirements:
1. The skills required to perform a sub-task belong to the skills required to perform the target task and the former is a subset of the latter.
2. The instruction should be in English. The instruction should be 1 to 2 sentences long. Either an imperative sentence or a question is permitted.
3. You should create an appropriate input based on the instruction in example. The input should involve realistic data and should not contain simple placeholders. The input should provide substantial content to make the instruction challenging but do not exceed 200 words in general. article if instructed, but not
4. The input should include detailed content of passage overview or description about it.
5. You should generate appropriate output according the instruction and depending the input in example. Make sure the output is less than 200 words in general.
6. The respanse you generated should conform to the following format:
#**
Instruction:
Input:
Output:
###
Instruction:
Input:
Output:
#*##
Target task: assistance_for_tet_toooo
Examples:
###
1. Instruction:
Rewrite text in another way
Input: People often think of cats as mysterious creatures because they are especially active at night Also, their eyes can glow an eerie green light in the dark.
Output: Cats are often considered mystical creatures because they are particularly active at night. Also, their eyes can emit a strange green glow dark environments. ***
the content from the brief description below. Input: This is a great phone with a high-resolution display.
2.
Output: This is an excellent phone with a high-performance processor for faster performance and smoother multitasking. Furthermore, it has resolution display for crisp, vibrant images allowing users to enjoy better visual experience. ###
The list of already existing sub-tasks for this target task is: L'paraphrase', "style_transfer""
The list of already existing sibling tasks for this target task is: [].
List 10 examples of this new sub-task below:
New sub-task: paraphrase
The purposes of paraphrasing to show understanding, make ideas accessible, translate technical language, avoid plagiarism, compare perspectives, and aid memorization. The ability paraphrase skillfully is valuable tool for write assistance.
Reason:
Figure 8: Prompt used for Somain-specitf instruction tunoonntttnnnggggnn data generation.

---

# Evaluation Prompt
You are helpful and precise assistant for checking the quality of the answer.
[Question]
Please edit the following paragraph to add more details to make it more interesting. I went to the park yesterday. saw two puppies playing there. Then went to the lake and saw lot of birds flying by. I ended sitting a bench and admiring the beauty of nature.
[Assistant 1]
Yesterday, I decided to take stroll in the park. As walking, I noticed two adorable puppies gleefully running around. Their playful barks and wagging tails were infectious, and couldn't help but smile. I continued walking, made my way to the lake. There, saw flock of birds soaring high above the water, creating impressive sight. in the sky. Mesmer ized by their beauty, found myself gravitating towards bench, where sat and allowed myself to bask in the tranquility of nature. The sun shining, the air crisp, and everything around me peaceful. It perfect way spend afternoon.
if they
performing dance
[End of Assistant 1]
[Assistant
Yesterday,
stroll the park and witnessed two playful puppies running around. The sight was amazed by
these adorable dogs brought joy my day. After that, I ventured towards flock birds flying unison. decided to on bench take tranquility nature.
[End of Assistant 2]
[System]
would like request your feedback on the performance two AI assistants in response the user question displayed above. Please evaluate the given four aspects: helpfulness, relevance, accuracy, level of details of their responses.
Please first clarify how each response achieves each aspect respectively. Then, provide a comparison on the overall performance between Assistant 1 and Assistant 2, and you need to clarify which one is better than or equal to another. Avoid any potential bias and ensuring that the order in which the responses were presented does not affect your judgment.
In the last line, order the two assistants. Please output single line ordering Assistant and Assistant 2, where '>' means "is better than' and '=' means 'is equal to'. The order should be consistent to your comparison. If there is not comparison that one is better, it is assumed they have equivalent overall performance ('=').
Figure 9: Prompt used for automatic pair- pair-wise evaluation.

---

Domain: Brainstorming
Instruction:
Provides a solution the following question: How maximize your productivity?
Domain-
Prioritize your tasks
Curated-IM
Domain-
Here is 5-step guide to maximize your productivity:
Instruct-LM
1. Prioritize Make realistic to-do list and rank each task based its importance and deadline.
Plan Spend time planning out your workday ensure you make the most of your time and avoid distractions.
Take Breaks Break up your work with regular short breaks to refresh your mind and energize yourself.
4. Focus Reduce distractions by turning off notifications and putting your phone away.
5. Communicate Set clear expectations with your team or colleagues to avoid confusion and duplication of efforts.
Explore-LM
Sure, I'd be happy to help you with that. Here are some tips to maximize your productivity:
Set clear goals: Set clear and specific goals for what you would like to achieve. This will help you stay focused and motivated.
2. Break down tasks: Break down larger tasks into smaller, more manageable ones.
This will make them less overwhelming and easier to accomplish.
3. Use to-do list: Make to-do list for each day, and prioritize tasks based on their importance and urgency.
4. Schedule your time: Schedule specific times work on important tasks. This will
help you stay focused and avoid distractions.
Take breaks: Take regular breaks to avoid burnout and maintain focus. Try to take breaks away from your workspace to help you recharge.
6. Use timer: Use timer to help you stay focused and work for specific intervals of time. This can help you stay track and avoid procrastination.
7. Minimize distractions: Minimize distractions such as social media, email notifica- tions, and phone calls. Instead, tools such Freedom Cold Turkey to block distractions.
Prioritize self-care: Prioritize self-care activities such as exercise, meditation, or spending time with loved ones. This will help you maintain your mental and physical health, which is essential for productivity.
9. Outsource tasks: Outsource tasks that you are not good at or don't enjoy doing.
This will help you focus on your strengths and improve your efficiency. 10. Automate tasks: Automate repetitive tasks such as bookkeeping, scheduling, or email responses. This will save you time and increase your productivity in the long run.

---

Explore-LM-
Sure, let's brainstorm some ways maximize your productivity:
Ext
1. Prioritize your tasks: Make list of everything you need do, and prioritize your tasks based on their importance and deadline.
Use time management techniques: Use techniques such as the Pomodoro Technique or time blocking to manage your time effectively and reduce distractions.
3. Take breaks: Taking regular breaks can help you recharge and stay focused for longer.
4. Automate repetitive tasks: Automate tasks such email responses or social media updates save time and reduce mental clutter. Delegate tasks: Delegate tasks others when possible, to free up time for more important tasks.
Manage your energy: Take steps to manage your energy throughout the day, such as getting enough sleep, eating well, and exercising regularly.
Eliminate distractions: Eliminate distractions such as social media or email notifi- cations, and work in a quiet enviromment where possible.
8. Get organized: Keep your workspace organized and decluttered to reduce mental clutter and improve focus.
9. Learn say Learn say to tasks and commitments that are not priority or will take up too much time.
10. Celebrate progress: Celebrate your progress and achievements regularly to stay motivated and encourage continued productivity.
ChatGPT
1. Set clear goals and prioritize tasks: Determine what tasks are most important and set achievable goals to help you stay focused and motivated.
2. Use time-blocking techniques: Schedule blocks of time for specific tasks, and avoid distractions during those periods to maximize productivity.
3. Take breaks and manage your energy: Taking regular breaks can help you recharge and stay focused. Consider incorporating physical activity or mindfulness practices into your breaks to help manage stress and improve concentration.
4. Use productivity tools: There are a variety apps and tools available to help you
stay organized, manage your time, and automate repetitive tasks.
5. Delegate tasks: Identify tasks that can be delegated to others, freeing up time for you to focus on more important responsibilities.
6. Practice good time management habits: Avoid procrastination, limit multitasking, and establish routine that works for you.
7. Stay motivated: Find ways stay motivated and inspired, such as setting rewards for completing goals or seeking support from colleagues or mentors.
Table 11: comparison case in the brainstorming domain.

---

Domain: Rewriting
Instruction: Edit the following article to add more details and descriptions to bring life. went to a new restaurant yesterday and ordered a steak and glass of red wine. The food tastes good, but the waiters have a bad attitude.
Domain-
The food tastes good, but the waiters have a bad attitude.
Curated-LM
Domain-
Yesterday, went to new restaurant and ordered steak and glass of red wine. The food tasted delicious, but the waiters had a bad attitude.
Instruet-LM
Explore-LM
Yesterday, I had the pleasure of dining at a new restaurant that serves delectable steaks and an exquisite selection of wines. The perfectly seasoned filet mignon melted in my mouth, while the pairing of the robust red wine complemented the flavors of the meat. However, I was disappointed in the attitude of the waiters, who seemed to be more interested in their cell phones than in providing excellent service.
Explore-LM- Ext
Yesterday, I visited trendy steakhouse with high hopes. When my waiter brought the menu, I noticed that was well-stocked with prime cuts of meat and an extensive list of wine pairings. decided order the filet mignon and glass of red wine. Actually, the food was delicious, but the waiters seemed to have a chip on their shoulder, acting as if they were doing us a favor by serving us. Overall, the experience was mixed, and I'm still on the fence about returning. Yesterday, I decided to try out
ChatGPT
restaurant that had heard about. Upon arriving, I was immediately impressed with the chic and modern decor. I took seat and was greeted by waiter who seemed uninterested in helping me. Despite this, ordered perfectly cooked steak and glass of rich, full-bodied red wine.
The steak was cooked to absolute perfection, with a crispy seared exterior and a juicy, tender center. The first bite was a burst of flavor that left me craving more. The red wine perfectly complemented the steak, with its robust flavor and notes of blackberry and oak.
However, my dining experience was marred by the poor attitude of the waiters. They seemed uninterested in serving and were slow respond to my requests. This put damper on otherwise enjoyable meal.
Despite the rude service, I would still recommend this restaurant for the quality of the food alone. It's a shame that the waitstaff didn't 't match the caliber of the cuisine.
Table comparison case in the rewriting domain.

---

Domain: Math
Instruction:
What is the maximum number of square inches in the area of a rectangle with perimeter of 12 inches?
Domain-
1599
Curated-LM
Domain-
Explanation: Let and to be the length and width of the rectangle, respectively. We know that 2/ 2t 12, / 6. The area of the rectangle is A tw. We want maximize subject to the constraint 2/ 2w
Instruct-LM
12. Using the method of Lagrange and 12. Therefore, the maximum number of square inches the area of the rectangle is 12 Answer: 12
multipliers, we get [ 3,
Explore-LM
Explanation: Let and w be the length and width of the rectangle, respectively. We know that 2/ + 2w = 12, so / + w = 6. The area of the rectangle is A = lw. We want to maximize A subject to the constraint + w 6. Using the method of Lagrange multipliers, get the system of equations = w. 1+ 6, and VA - (V++ Solving this system of equations, we get / of square inches in the area of the rectangle is 32
3. Therefore, the maximum number Answer: 9
Explore-LM- Ext
Explanation: Let and w be the length and width the rectangle, respectively. Since the perimeter is 12 inches, we have 2/ + 2w = 12, or 1 + w = 6. We want to maximize the area, which is given by A = lw. From the equation for the perimeter, we have / = 6 w. Substituting this into the equation for the area, we get A w(6 w) 6w w². To maximize this expression, we take the derivative with respect to and set it equal O: dw (6w =w²) Therefore, the maximum area is 3(3)
2w 0. Solving for w,
get = 3, which implies that
Answer: 9 square inches.
ChatGPT
Let the width of the rectangle be w and the length be The perimeter of the rectangle is given by 2(1 + w) = 12, which simplifies to / + w 6. We know that the area of rectangle is given by A = Iw. We want to maximize the area of the rectangle, so we need to maximize A subject to the constraint /+ w 6. Using the method of Lagrange multipliers, we set up the following equations:
Explanation:
V AV + w)
lw X(1,1)
w 6
Taking partial derivatives, get:
0A
01
0A
Dw
And we have:
À
Multiplying both equations by À and setting them equal to each other, we get l.
Substituting this into the constraint equation, we get 2/ = 6, so 3 and w = Therefore, the maximum area of the rectangle is A Iw
square inches.
Answer: square inches.

---

Standard An-
Since the perimeter is 12, the sides of the rectangle add up to 12/2 = 6. Let be one side length of the rectangle. Then the other side length x, the area is
swer
x(6 x) = 62 x².
Completing the square, we get
6r - x
+ 6x
(x - -3)².
Thus, the maximum area of the rectangle is square.
9 square inches, which occurs for a 3 x
Table 13: A comparison case in the math domain.


Reviews:
[{"review_id": "guhu5Slfjt", "reviewer": "Reviewer_Jzc1", "summary": "", "questions": "I think this work might benefit from human assessment of the in-domain instructions in addition to model evaluation. Could you discuss the implications of this further?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your valuable feedback and are grateful that you find our approach to be novel and well-argued.\n\n**Q1: Regarding the generation of more complex reasoning instructions.**\n\n**A1:** The motivation of Explore-Instruct is to augment domain-specific instructional coverage and subsequently improve the performance of the fine-tuned model. It is essential to note that, in some domains, increasing the instruction complexity does not necessarily guarantee enhanced domain-specific performance. Nevertheless, in the case of the MATH dataset, a positive correlation exists between the complexity of instructions and the overall performance. This demonstrates that our approach effectively promotes reasoning capabilities.\n\n**Q2: Regarding the instruction bias problem.**\n\n**A2:** Our approach is orthogonal to the works aimed at diminishing instruction bias, signifying that the strategies for reducing such bias can typically be incorporated into our method. For example, the utilization of reinforcement learning from human feedback in ChatGPT has the potential to alleviate the production of inaccurate or detrimental text within the context of our proposed framework.\n\n**Q3: Regarding the human assessment of the instructions.**\n\n**A3:** In the approach we proposed, assessing the generated instruction-tuning data requires the analysis of two critical factors. The first factor is the data quality, which can be referred to in previous works where each instruction, input, and output triplet undergoes human evaluation to determine its accuracy and effectiveness in accomplishing the task. The second crucial factor involves the data's diversity, which is non-trivial to estimate by human evaluators. We will supplement the human evaluation results of the data quality in the revised version.\n\nThanks again for your review! If you have any additional questions, please let us know and we would be happy to discuss with you further.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Current methodologies for domain-specific instruction-tuning data often sparsely cover the full range of a domain. This problem is therefore relevant and highly important to address for domain-specific applications. The authors' approach is novel and well-argued.\n", "reasons_to_reject": "The work does not discuss the generation of more complex, multi-step reasoning instructions, nor about how to mitigate instruction bias that might lead to the generation of incorrect or harmful text.", "questions_to_authors": "I think this work might benefit from human assessment of the in-domain instructions in addition to model evaluation. Could you discuss the implications of this further?", "paper_topic_and_main_contributions": "This work presents an approach to enhance the data coverage to be used in domain-specific instruction-tuning through active exploration with LLMs. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "M8jlYbvz1f", "reviewer": "Reviewer_Rfvu", "summary": "", "questions": "1.\tWhat is the prompt used for the Self-Instruct baselines?\n2.\tWhat is the cost of the entire method?\n3.\tIs there any experiment on the GSM8K dataset? This might be a more common one that people test model’s math reasoning ability?\n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the time and effort you put into reviewing our paper. We appreciate your insightful feedback and are pleased that you find our work to be both motivating and interesting. We will carefully address each of your concerns in the following points.\n\n**Q1: Regarding the cost comparison between Domain-Aware Self-Instruct and Explore-Instruct.**\n\n**A1:** In contrast to the Domain-Aware Self-Instruct approach, the primary additional cost associated with the Explore-Instruct method comes mainly from the exploration process. Since this exploration occurs at the task level and allows for simultaneous exploration of multiple tasks, the associated costs are relatively minimal when compared to the generation of instruction-tuning data. To illustrate, consider the creation of a dataset for the rewriting domain consisting of 10,000 instances, which, without taking repetitive filtering into account, requires approximately **1,000 API calls * (400 prompt tokens + 600 output tokens) = 1,000,000 tokens** when generating 10 items concurrently. Conversely, the exploration of a hierarchical structure with a maximum depth of 2 and 10 subtasks at each level requires a mere **11 API calls * (700 prompt tokens + 200 outputs tokens) = 9,900 tokens** when exploring 10 subtasks simultaneously. As a result, the additional cost incurred is **less than 1%**. We will include the above clarifications in the revised version.\n\n**Q2: Regarding the details about Domain-Aware Sefl-Instruct.**\n\n**A2:** In the Domain-Aware Self-Instruct approach, we try our best to ensure the diversity of the instruction data generation process. The following components describe the prompt, seed task examples, and diversity filter in detail.\n\n**Prompt:** The prompt employed by Domain-Aware Self-Instruct in generating instruction-tuning data is similar to that of Explore-Instruct. Within the generation prompt, an explicit emphasis is placed on generating a diverse range of instructions to minimize redundancy between the new instructions and those already existing.\n\n**Seed Task Examples:** In Domain-Aware Self-Instruct, an initial set of domain-specific seed examples with diverse instructions is employed. During each iteration, two demonstrations are randomly selected from the seed collection. Subsequently, the newly generated instruction data is integrated into the seed collection, and the process is reiterated.\n\n**Diversity Filter:** The diversity filter applied during data generation is based on the ROUGE-L overlap between the instructions. Specifically, a generated instruction is preserved only if its ROUGE-L overlap with any pre-existing instructions falls below a given threshold.\n\nWe will include the above clarifications in the revised version.\n\n**Q3: Regarding more comprehensive test domains.**\n\n**A3:** The science domains like biology and medical domains are orthogonal to the definition of domain employed in this study. Science domains represent broader fields of knowledge, while specific use case domains (discussed in this paper) represent specialized areas of application or functionality within those fields. Besides the domains discussed in our paper (namely brainstorming, rewriting, and math), other similar examples include assistance in data analysis, collaboration in programming, and engagement in interactive role-playing. Therefore, utilizing science domains as experimental grounds is not included in the focus of our research.\n\n**Q4: Regarding the comparison with other baseline datasets.**\n\n**A4:** We conducted automatic comparison experiments with three well-known baseline models: **Alpaca**, **WizardLM**, and **Dromedary**. In these experiments, Alpaca utilized its entire dataset of 52,000 instruction-tuning instances, whereas WizardLM and Dromedary each drew a sample of 52,000 instances from their corresponding training sets. The experimental results are shown in the tables below, and **our Explore-LM outperforms the baseline models across all domains**. We will include these additional results in the revised version.\n\n| Automatic Comparison | Win:Tie:Lose | Beat Rate |\n| :-: | :-: | :-: |\n| Explore-LM vs Alpaca | 148:38:22 | 87.06 |\n| Explore-LM-Ext vs Alpaca | 147:40:21 | 87.50 |\n| Explore-LM vs WizardLM | 84:55:69 | 54.90 |\n| Explore-LM-Ext vs WizardLM | 93:65:50 | 65.03 |\n| Explore-LM vs Dromedary | 163:21:24 | 87.17 |\n| Explore-LM-Ext vs Dromedary | 165:22:21 | 88.71 |\n\n**Table 1: Evaluation results in the brainstorming domain.**\n\n| Automatic Comparison | Win:Tie:Lose | Beat Rate |\n| :-: | :-: | :-: |\n| Explore-LM vs Alpaca | 22:52:20 | 52.38 |\n| Explore-LM-Ext vs Alpaca | 25:55:14 | 64.10 |\n| Explore-LM vs WizardLM | 23:49:22 | 51.11 |\n| Explore-LM-Ext vs WizardLM | 20:60:14 | 58.82 |\n| Explore-LM vs Dromedary | 40:40:14 | 74.07 |\n| Explore-LM-Ext vs Dromedary | 43:42:9 | 82.69 |\n\n**Table 2: Evaluation results in the rewriting domain.**\n\n| Models | Accuracy Rate |\n| :-: | :-: |\n| Alpaca | 3.8 |\n| WizardLM | 2.2 |\n| Dromedary | 1.8 |\n| Explore-LM | 6.8 |\n| Explore-LM-Ext | 8.4 |\n\n**Table 3: Evaluation results in the math domain.**\n\n**Q5: Regarding the experiments on GSM8K dataset.**\n\n**A5:** In our study, we employ the MATH dataset for evaluation in the math domain. This is because it includes a diverse range of mathematical problems with varying degrees of difficulty and categories, such as algebra, geometry, and calculus. This choice aligns with the research problem of this paper: domain-specific performances. In contrast, the GSM8K dataset primarily consists of grade school mathematical word problems. These can be more accurately characterized as a specific task rather than a comprehensive domain, akin to the paraphrasing task within the rewriting domain.\n\n**Q6: Regarding the missing references.**\n\n**A6:** We will add these references in the revised version accordingly.\n\nWe sincerely thank the reviewer again for your valuable suggestions. If you have any further questions, please let us know and we will be happy to discuss them with you further.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.\tGreat motivation and interesting research question: Domain-specific instruction generation is very important because it could enable everyone to design personalized instruction-tuned models and make instruction-tuned models satisfy the need of experts. Most current instruction generation methods don’t consider this point and generate sth very general. The authors design two types of exploration strategies: lookahead and backtrack exploration, that greatly contribute to the depth and breadth of the trained tasks. The method is reasonable to me.\n2.\tComprehensive investigation on the quality of generated instruction data: I'm glad to see that the authors utilize different metrics to evaluate the data quality and diversity. The comprehensive evaluation helps us better understand the effect of proposed methods. I’m especially impressed by the win-lose comparison with domain curated and instruction tuned LMs and the tendency when increasing the number of training instances.\n", "reasons_to_reject": "1.\tA bit unfair comparison with Domain-aware Self-instruct: As the authors mentioned in Line 281, Self-instruct doesn’t take any exploration steps. Therefore, in terms of cost, building Explore-Instruct might be much more expensive than the simple Self-instruct. Also, more details about the reproduced Self-instruct is needed, e.g., prompt contents, steps of controlling diversity, etc. The seed task examples in Self-instruct prompt might be vital for the performance. If the examples are more domain-specific and you can provide a diverse set of them, the generated instruction may be diverse as well.\n2.\tLack of more comprehensive test domains: I can only see brainstorming, maths and rewriting in the evaluation parts. However, when people talk about “domain-specific”, it is more natural to think of science domains like biology and medical domains. Rewriting is a general domain that may not need too much expert knowledge. If the paper really claims that the method can help generate domain-specific instructions, it is better to test more scenarios.\n3.\tLack of fair comparison with other datasets like WizardLM and Dromedary: These two datasets also use some similar methods to further extend the existing instructions. But there’s no direct comparison with these two datasets and the methods they use. \n", "questions_to_authors": "1.\tWhat is the prompt used for the Self-Instruct baselines?\n2.\tWhat is the cost of the entire method?\n3.\tIs there any experiment on the GSM8K dataset? This might be a more common one that people test model’s math reasoning ability?\n", "paper_topic_and_main_contributions": "Current instruction-tuning data generation methods often lack sufficient representation of specific domains, limiting nuanced understanding and interactions within those areas. To address this gap, the authors introduce a new method called Explore-Instruct. This approach actively enhances data coverage for domain-specific instruction-tuning by lookahead and backtrack exploration with LLMs. The authors’ method outperforms several baselines in terms of diversity and quality, even those using domain-specific data enhancement techniques. Also, it brings better downstream performance on specific domains like maths, brainstorming and rewriting.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions.  Wu et al., 2023\n\nDynosaur: A Dynamic Growth Paradigm for Instruction-Tuning Data Curation. Yin et al., 2023\n"}, "extra_text": {}}, {"review_id": "Xaz3zvP7yl", "reviewer": "Reviewer_TUPc", "summary": "", "questions": "A. How does your approach compare to baseline alpaca [2], self-instruct [3], or longform [4]? Do you have experiments for comparison?\n\nB. How does your approach compare to evol-instruct [1]?\n\nC. It's interesting to me that better domain coverage would aid a dataset like MATH, which is very domain specific. Why do you think explore-instruct improves here?\n\nD. Does domain-aware self-instruct use domain-specific seed examples for e.g. generating data for the math domain and evaluation? This detail wasn't entirely clear to me in the description in 4.1.\n\n[1] https://arxiv.org/abs/2304.12244\n[2] https://crfm.stanford.edu/2023/03/13/alpaca.html\n[3] https://arxiv.org/abs/2212.10560\n[4] https://arxiv.org/abs/2304.08460", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the time and effort you put into reviewing our paper. We appreciate your insightful feedback and we are thankful for your recognition of our work's novelty and significance. In the subsequent points, we will carefully address each of your concerns.\n\n**Q1: Regarding the comparison with other baseline datasets.**\n\n**A1:** We conducted automatic comparison experiments with three well-known baseline models: **Alpaca**, **WizardLM**, and **Dromedary**. In these experiments, Alpaca utilized its entire dataset of 52,000 instruction-tuning instances, whereas WizardLM and Dromedary each drew a sample of 52,000 instances from their corresponding training sets. The experimental results are shown in the tables below, and **our Explore-LM outperforms the baseline models across all domains**. We will include these additional results in the revised version.\n\n| Automatic Comparison | Win:Tie:Lose | Beat Rate |\n| :-: | :-: | :-: |\n| Explore-LM vs Alpaca | 148:38:22 | 87.06 |\n| Explore-LM-Ext vs Alpaca | 147:40:21 | 87.50 |\n| Explore-LM vs WizardLM | 84:55:69 | 54.90 |\n| Explore-LM-Ext vs WizardLM | 93:65:50 | 65.03 |\n| Explore-LM vs Dromedary | 163:21:24 | 87.17 |\n| Explore-LM-Ext vs Dromedary | 165:22:21 | 88.71 |\n\n**Table 1: Evaluation results in the brainstorming domain.**\n\n| Automatic Comparison | Win:Tie:Lose | Beat Rate |\n| :-: | :-: | :-: |\n| Explore-LM vs Alpaca | 22:52:20 | 52.38 |\n| Explore-LM-Ext vs Alpaca | 25:55:14 | 64.10 |\n| Explore-LM vs WizardLM | 23:49:22 | 51.11 |\n| Explore-LM-Ext vs WizardLM | 20:60:14 | 58.82 |\n| Explore-LM vs Dromedary | 40:40:14 | 74.07 |\n| Explore-LM-Ext vs Dromedary | 43:42:9 | 82.69 |\n\n**Table 2: Evaluation results in the rewriting domain.**\n\n| Models | Accuracy Rate |\n| :-: | :-: |\n| Alpaca | 3.8 |\n| WizardLM | 2.2 |\n| Dromedary | 1.8 |\n| Explore-LM | 6.8 |\n| Explore-LM-Ext | 8.4 |\n\n**Table 3: Evaluation results in the math domain.**\n\n**Q2: Regarding the comparison with Evol-Instruct.**\n\n**A2:** Evol-Instruct includes two distinct operations, in-depth evolution, which emphasizes the generation of progressively complex instructions rather than augmenting their diversity, and in-breadth evolution, which has the potential to extend across various domains, such as transitioning from mathematics to physics (e.g., Figure 1 in WizardLM [1]). Consequently, Evol-Instruct tends to be inadequate for enhancing the coverage of instruction-tuning data within a specific domain. \n\n[1] Xu C, Sun Q, Zheng K, et al. Wizardlm: Empowering large language models to follow complex instructions[J]. arXiv preprint arXiv:2304.12244, 2023.\n\n**Q3: Regarding the better domain coverage for MATH dataset.**\n\n**A3:** The MATH dataset includes a diverse range of mathematical problems with varying degrees of difficulty and categories, such as algebra, geometry, and calculus. Thus, solving questions from different categories requires not identical but complementary skills and knowledge. This is what the proposed Explore-Instruct aims to achieve by promoting task diversities and maintaining connections simultaneously. \n\n**Q4: Regarding the details about seed examples in Domain-Aware Self-Instruct.**\n\n**A4:** In Domain-Aware Self-Instruct, an initial set of domain-specific seed collections with diverse instructions is employed. During each iteration, two demonstrations are randomly selected from the seed collection. Subsequently, the newly generated instruction data is integrated into the seed collection, and the process is reiterated. We will include the above clarifications in the revised version.\n\nWe sincerely thank the reviewer again for your valuable suggestions! Please let us know if you have any further questions, and we are happy to discuss further.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Straightforward method explored well, with good results and a few interesting analyses around the effect of data quality. The approach seems to work well as a way to increase the diversity of an instruction set, keeping the overlap between instructions relatively low. Performing a graph search over the space of domains is an interesting and novel idea, and seems to perform reasonably well.", "reasons_to_reject": "Lack of other baselines - the authors note there is much similar work, but don’t compare to these alternatives. For example, it would be good to compare to baseline self-instruct, or alpaca, to show the improvement of using this method against these other ones (beyond the domain-aware self-instruct, which to my understanding would be similar to alpaca). Such comparisons would further highlight the improvements of the method - althought I think the current experiments show fairly well that domain-level exploration is useful. Although it is concurrent work, I would be especially interested in comparisons to Evol-Instruct (the approach behind WizardLM), which also takes a graph search-style approach to generating instruction data (but on a sample level, rather than a domain level).\n\nOverall, I think this is solid and interesting work, and lean accept.\n", "questions_to_authors": "A. How does your approach compare to baseline alpaca [2], self-instruct [3], or longform [4]? Do you have experiments for comparison?\n\nB. How does your approach compare to evol-instruct [1]?\n\nC. It's interesting to me that better domain coverage would aid a dataset like MATH, which is very domain specific. Why do you think explore-instruct improves here?\n\nD. Does domain-aware self-instruct use domain-specific seed examples for e.g. generating data for the math domain and evaluation? This detail wasn't entirely clear to me in the description in 4.1.\n\n[1] https://arxiv.org/abs/2304.12244\n[2] https://crfm.stanford.edu/2023/03/13/alpaca.html\n[3] https://arxiv.org/abs/2212.10560\n[4] https://arxiv.org/abs/2304.08460", "paper_topic_and_main_contributions": "This paper introduces Explore-Instruct, an instruction-dataset-generation approach that models domains / instructions as a graph, and generates data by traversing domains in a DFS-style manner, exploring until a max depth and/or max breadth parameter is reached. Alpaca-style generation is used for data generation while in a domain, while llama-7b models are trained. Model-based and human-based evaluation suggests that the approach performs better than domain-specific models, although it lags somewhat behind ChatGPT.\n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: In this paper the authors propose an approach to enhance the data coverage to be used in domain-specific instruction-tuning through active exploration via Large Language Models (LLMs). The proposed method outperforms several baselines in terms of diversity and quality, even those using domain-specific data enhancement techniques. Also, it brings better downstream performance on specific domains like maths, brainstorming and rewriting.

Pros:

It is a well-motivated, highly timely and relevant research topic

Comprehensive investigation on the quality of generated instruction data

The approach seems to work well at  increasing the diversity of an instruction set, keeping the overlap between instructions relatively low. 

The authors also performed a graph search over the space of domains which is an interesting and novel idea, and seems to perform reasonably well.

Cons:

Needs more comprehensive test domains, which are more “domain-intensive” like biology or medical (than maths, brainstorming and rewriting)

Needs more fair comparison with other baselines.

The original work will also benefit from incorporating the additional discussion points/experiments conducted during rebuttal.
Decision: Accept-Main