Specialist or Generalist? Instruction Tuning for Specific NLP Tasks
Authors: Chufan Shi
Yixuan Su
Cheng Yang
Yujiu Yang
Deng Cai
Venue: EMNLP 2023 Main (2023)
Keywords: LLMs
Instruction Tuning
Data Efficiency

Abstract: The potential of large language models (LLMs) to simultaneously perform a wide range of natural language processing (NLP) tasks has been the subject of extensive research. Although instruction tuning has proven to be a data-efficient method for transforming LLMs into such generalist models, their performance still lags behind specialist models trained exclusively for specific tasks. In this paper, we investigate whether incorporating broadcoverage generalist instruction tuning can contribute to building a specialist model. We hypothesize that its efficacy depends on task specificity and skill requirements. Our experiments assess four target tasks with distinct coverage levels, revealing that integrating generalist instruction tuning consistently enhances model performance when the task coverage is broad. The effect is particularly pronounced when the amount of task-specific training data is limited. Further investigation into three target tasks focusing on different capabilities demonstrates that generalist instruction tuning improves understanding and reasoning abilities. However, for tasks requiring factual knowledge, generalist data containing hallucinatory information may negatively affect the model’s performance. Overall, our work provides a systematic guide for developing specialist models with general instruction tuning.

Specialist or Generalist? Instruction Tuning for Specific NLP Tasks
Chufan S Yixuan Su Cheng Yang*, Yujiu Yang*, Deng Cai TTinghua Shenzhen International Graduate School, Tsinghua University Language Technology Lab, University of Cambridge *Cohere
9 Tencent Al Lab
(scf22, Jemails tsingssssssssssnnsn edu. cn, yang yujiuesz..... tsinghua. cn yixuanêcohere.cm com thisisjcykcd@@mail com
Abstract
2023; Zhou al., 2023; Gudibande et al., 2023) suggest that (1) the foundation of their superior performance (i.c., knowledge and capabilities) is predominantly acquired during large-scale unsuper- vised pre-training; and (2) instruction tuning (Sanh et al., 2022; Wei et al., 2022a; Mishra et al., 2021; Ouyang et al., 2022) is an incredibly data-efficient method for unleashing the power of LLMs com- plete realistic NLP tasks. However, under rigorous evaluation, the performance of those instruction- following generalist models often falls short com- pared to traditional task-specific specialist models (Jiao et al., 2023b; Qin et al., 2023; Fang et al., 2023; Liu et al., 2023). Recently, there has also been growing trend towards developing specialist models using instruction tuning (Jiao et al., 2023a; Wang et al., 2023b; Zhang al., 2023; Cheng al., 2023; Wang al., 2023a).
The potential of large language models (L.MMs) to simultaneously perform a wide range of natural language processing (NLP) tasks has been the subject of extensive research. Al- though instruction tuning has proven to data-efficient method for transforming LLMs into such generalist models, their performance still lags behind specialist models trained ex- clusively for specific tasks. In this paper, we investigate whether incorporating broad- coverage generalist instruction tuning can con- tribute to building a specialist model. We hypothesize that its efficacy depends on task specificity and skill requirements. Our periments assess four target tasks with dis- tinct coverage levels, revealing that integrat- ing generalist instruction tuning consistently enhances model performance when the task coverage broad. The effect is particularly pronounced when the amount of task-specific training data is limited. Further investigation into three target tasks focusing on different ca- pabilities demonstrates that generalist instrue- tion tuning improves understanding and rea- soning abilities. However, for tasks requiring factual knowledge, generalist data containing hallucinatory information may negatively affect the model's performance. Overall, our work provides a systematic guide for developing spe- cialist models with general instruction tuning. Our code and other related resources can be found at https://gitthh com/Davidi Fazz// Simerraist_oor_peeeee
In this paper, we study how to better harness the power of LLM for specific NLP tasks using instruc- tion tuning. Our research is motivated by the exis- tence of various broad-coverage general-purpose instructionnfollloiing datasets (Taori et al., 2023; Peng et al., 2023; Labs, 2023; Xu et al., 2023; Zhou et al., 2023; Su et al., 2023b) and their surprising ef- ficiency for turning LLMs into capable instruction- following generalists. For instance, Zhou al. (2023) shows that merely one thousand supervised input-output pairs are necessary to build a compe- tent generalist. In contrast to general-pune purpoose in- struction tuning, our preliminary experiments show that a sufficiently large set of task-specific data is still required for transforming an LLM into a superior specialist. This leads to pivotal re- search question: How to better unleash the power of LLMMs for specific NLP tasks by marrying the best of two worlds? More specifically, can general- purpose instruction-folloviii datasets aid the transformation of an LLM into a specialist? so, when and how?
1 Introduction
The latest generation of large language models (LLMs), such as ChatGPT (OpenAI, 2022) and GPT4 (OpenAI, 2023), are often referred to as generalist models for their exceptional generaliz- ability to perform various natural language pro- cessing (NLP) tasks. Recent studies (Taori al.,
This work was completed during an internship at Ten- cent Al Lab.
Conesponding author.
We hypothesize the answers to the previous ques-

---

tions depend on (1) how specific the target task is; and (2) what skills the target task requires. To test this hypothesis, we first assess four target tasks with distinct levels of coverage. Our find- ings reveal that integrating general instruction tun- ing-thar is, training with generalist data enhances the model's performance on specific NLP tasks with broad task coverage, particularly when the amount of task-specific training data is limited. To gain deeper understanding of the improvements elicited by training with generalist data, we sub- sequently examine three target tasks that focus on distinct skill sets. Our results suggest that gen- eral instruction tuning improves the model's un- derstanding and reasoning capabilities. However, when it comes to tasks that demand factual knowl- edge from the LLM, instructional data generated through self-instruct (Wang et al., 2022a) harms the model's performance due the intrinsic hallu- cinations brought by such data creation approach. In sum, to the best of our knowledge, our work is the first effort to present systematic guide for building and improving specialist models with gen- eral instruction tuning.
"specialist" instruction tuning.
Generalist Instruction Tuning, Early attempts instruction tuning (Wang et al., 2022b; Sanh et al., 2022; Wei et al.. 2022a: Chung et al., 2022, inter alia) transform range of public NLP datasets into an instructional format, with a few manually crafted templates for each task. They then fine-tune an LLM on portion of the transformed data and evaluate on another set of held-out tasks. Each work affirms that the model's generalization ability to unseen tasks improves when increasing the task and template diversity. However, template-basse bassdd instructions are not sufficiently diverse for build- ing a truly competent generalist (Ouyang et al., 2022). In contrast, state-of-the-un generalist mod- els such as ChatGPT (OpenAI, 2022) are trained with proprietary instructions collected from real human users. In the pursuit to replicate the success of ChatGPT, various open-source broad-coverage instruction-tume
datasets are proposed. Some are gathered via crowd-sourcing (Labs, 2023; Zhou et al., 2023) while others use the outputs from strong proprietary models (Taori et al., 2023; Peng et al., 2023; Xu et al., 2023; Su et al., 2023a; Li et al., 2023) with techniques such as self-instruct (Wang et al., 2022a). Existing results suggest that these models can achieve near parity with propri- etary models in various aspects (Chiang et al., 2023; Zhou al., 2023; Taori et al., 2023).
2 Background: Instruction Tuning
In recent years, large language models (LLMs) have undergone rapid development and have dom- inated the field of natural language processing (NLP) (Radford et al., 2018: Devlin al., 2019; Radford et al., 2019; Brown et al., 2020). To- day's LLMs, such as ChatGPT (OpenAI, 2022) and GPT-4 (OpenAI, 2023), can perform complex and diverse tasks in the unified form of follow- ing natural language instructions. Generally, these models are trained in three separate stages: (1) large-scale unsupervised pre-training raw text; and (2) instruction tuning via supervised learning (Sanh et al., 2022: Wei al., 2022a; Mishra al., 2021; Su and Collier, 2022; Su al., 2023b); and (3) reinforcement learning from human feedback (Stiennon et al., 2020; Bai et al., 2022; Ouyang et al., 2022). Recent studies (Zhou et al., 2023; Gudibande et al., 2023) argued that almost all capa- bilities of L.MSs are learned during unsupervised pre-training, and instruction tuning with limited amount of supervised data is sufficient. However, this observation refers to the process of construct- ing general-purye purppsee -nsllwction-followinn models- generalists. In the following, we separately intro- duce broad-com coverage "generalist" and task-specific
Specialist Instruction Tuning. There is also an emerging trend to continue instruction tuning on specific NLP tasks, such machine translation (Jiao et al., 2023a), information extraction (Wang et al., 2023b), medical QA (Wang et al., 2023a; Fleming et al., 2023), and writing-assistant (Zhang et al., 2023). These works typically transform ex- isting task-specific datasets into the same instruc- tional format as generalist instruction tuning and yield better model performance in specific tasks. Different from previous work, this study aims to provide a comprehensive and in-depth investigation of the role of generalist instruction data in specialist instruction tuning.
Our work is most related to the initial studies on the cross-task generalization of instruction tun- ing such as FLAN (Wei et al., 2022a). The dif- ferences between our work and previous work are: (I) we use broad-coverage generalist data, while they use template-basee data; and (2) they focus on zero/few-shot performance on unseen tasks, while

---

we assume an adequate amount of task-specific training data available.
uniformly sample training/test instances from the corresponding datasets such that each dataset has an equal proportion. For generalist data, we con- sider the GPT4-Instruct and LIMA datasets as dis- cussed above. We first train models on generalist data and then specialist data. We vary the amounts of specialist data across (2k, 4k, 6k, 8k, 10k) to study the effect of generalist data under different circumstances of data scarcity.
3 Incorporating Specialist Training with Generalist Training
3.1 Data Collection
We sort the instruction-folloiing data into two groups: (1) specialist data and (2) generalist data.
Specialist data. primarily originates from exist- ing NLP datasets with focus on particular tasks. To facilitate our research, we mainly utilize the SuperNI dataset (Wang et al., 2022b), a compre- hensive benchmark containing 1,616 NLP datasets coupled with their respective natural language in- structions, as the source of specialist data. The de- tails are described in Section 4.1. We also leverage existing question answering datasets (Kwiatkowski al., 2019; Berant et al., 2013; Joshi et al., 2017) reading comprehension datasets (Lai et al., 2017) reasoning datasets (Bowman et al., 2015; Talmor et al., 2019; Ling et al., 2017) to evaluate different aspects of model skills, detailed in Section 5.1.
Model and Training Details. We conduct our experiments with the popular LLaMA 7B and 13B models (Touvron et al., 2023). For training generalist data, we follow the original setups in the respective papers (Zhou al., 2023; Taori et al., 2023). Specifically, for GPT4-Instruct, we train for epochs with a batch size of 128, while for LIMA, we train for 15 epochs with a batch size 64. In the subsequent specialist training phase, we train for 3 epochs with a batch size of 128. In both stages, we use the Adam optimizer (Kingma and Ba, 2015) with learning rate of 2e-5 and utilize the standard language modeling objective:
Generalist data is characterized by its exten- sive scope and diversity. For our research, we select two representative broad-coverage general- purpose datasets: Ins4-ust (Peng et al., 2023) and LIMA (Zhou et al., 2023). GPT4-Instruct (Peng al., 2023) contains 52k unique instruction- response pairs, where the instructions are collected through self-instruct (Wang al., 2022a) and the responses are generated by GPT-4 (OpenAI, 2023). LIMA (Zhou et al., 2023) consists of 1 carefully curated instruction-response pairs derived from human-authored community questions and answers. Notably, we emphasize that GPt4-Is serves example of generalist data synthesized by LLMs and LIMA represents another distinct exam- ple of generalist data written by humans.
C
log Pe (v yes),
(I)
where denotes the model parameters and {æ; y) is an instruction-response pair.
Experiments I: The Coverage of the Target Tasks
4.1 Coverage Taxonomy
To assess our model's performance on a variety of target tasks with distinct levels of generality, we construct hierarchy of four specialist tasks using the SuperNI dataset (Wang et al.. 2022b). This taxonomy encompasses tasks with varying scopes of coverage, as detailed below.
SuperNI (multiple tasks, multiple formats). At the most comprehensive level, incorporate all the English tasks from the SuperNI dataset, which encompasses total of 756 datasets. Unlike LIMA and GPT4-Instruct, which accommodate broad spectrum of user-oriented inquiries, the datasets SuperNI focus on specific NLP tasks distilled from real-world demands. Therefore, we treat them as specialist data at the highest coverage level.
Unified Format. We follow the template used in Stanford's Alpaca project (Taori et al., 2023) (See Appendix A). Each instance in the general- ist and specialist data transformed in pair of (instruction, response).
3.2 Training
Speccalist/Generalist Data Combination. For each target task, we construct the training and test set with 50k and 5k instances, respectively. For target tasks that span over multiple datasets, we
Classification (multiple tasks, single format). The tasks in SuperNI can be grouped based on

---

57.00
$3.00
92.00
$9.00
Rouget
Accuncy
vi EPP1St wihii yoars
= K piaciic.co
65.00
96,00
Number of specialist data SuperNi-7B
Number of specialist data Classiicatiinn77
Number of specialist dasa (c) Scntimnnt778
Number of specialist data (d) Yelp 78
Rougel.
Accuncy
75,00
Accuncy
$8,00
53,00
74.00
88 00
97.00
vid GFfstamra LIMA viduot
oit Cort--aanc LIMA
vid GFT4tatrat LI44 pearaale
48.00
69:00
00
$6,00
Number spacialiss (e) SuperNi-13B
Number specialist (9) Classification 13B
Number of oppccillii
of sppcciliit (h) Yelp-138
(g) Sentinenz- 13B
Figure 1: Comparison of models trained with different combinations of specialist and generalist data across different tasks. We report Rouge-L for SuperNI and accuracy for other levels.
their task types, such as classification, summariza- tion, and question answering. For the second level, we focus the classification subset. Specifically, we select 252 classification datasets. To measure the model's cross-task generalization capability, we allocate 223 datasets for training and reserve the remaining 29 datasets as held-out datasets for eval- uation.
Task Coverage
GPT4-Instruct
LIMA
specialist
SuperNI Classification Sentiment
25.54
12.65
54.92
53.20
46.84
80.02
68.66
51.46
90.71
Yelp
91.68
65.52
98.11
Table I:
The performance of generalists and specialists on tasks of different coverage levels LLaMA-7B. The specialists are trained with 10k task-specife instances. For SuperNI, the performance measured by Rouge-L, while the others measured by accuracy.
Sentiment (single tasks, multiple domains). The classification tasks selected above can be further categorized based on their specific topics, such as sentiment analysis, toxic language detection, com- monsense categorization, and others. Among these, we designate 32 sentiment analysis datasets as the third level.
length of 512.! For the Classification, Sentiment, and Yelp levels, we follow previous studies (Brown et al., 2020; Sanh et al.. 2022) and utilize a classifi- cation with options approach, where we prompt the model with set of options and compute the likeli- hood of each option being the response. The with the highest probability taken the model's prediction, and we report the model's accuracy.
Yelp (single tasks, single domain). The sentiment analysis datasets mentioned above span various domains, such as movie and restaurant reviews. At the most fine-grained level, we choose the Yelp dataset (Zhang et al., 2015) as the representative task to evaluate the model's performance in a highly specialized domain.
4.3 Main Results
Generalist models lag behind specialist models across all coverage levels. We compare general- ist models that solely trained generalist data (i.c., LIMA or GPT4-Instruct) to those specialist models that are solely trained on specialist data (the 10k training instances we collect for each cov-
4.2 Evaluation Setup
For the SuperNI level, follow the same evalua- tion protocol as in Wang et al. (2022b) and report Rouge-L (Lin, 2004). For the decoding strategy, we adopt greedy search with a maximum generation
We leave the study on more advanced decoding meth- ods (Holtzman et al., 2019; Su et al., 2022; Yang et al., 2023) as future work.

---

ments for both SuperNI and Classification across LLaMA 7B and 13B models. On average across different settings specialist data, the introduction of generalist data leads to an improvement of 0.96 for LLaMA-7B and 0.74 for LLaMA-13B on Su- perNI tasks, while for Classification tasks, it results in enhancement of 1.53% for LLaMA-7B and 0,82% for LLaMA- 13B.. It is also worth noting that LIMA only has 1 instances, but it can even help improve performance when the number of special- ist data is 10x larger. However, the results are the opposite for Sentiment and Yelp. For instance, the introduction LIMA leads to a minor performance degeneration on Sentiment with 2k specialist data (a reduction of 0.25% for LLaMA-7B and 0.56% for LLaMA- 13B). In the case of the Yelp task, the impact of including generalist data (both GPT4- Instruct and LIMA) appears be minimal on the overall model performance.
65.00
58.00
Accuracy
51.00
with GPT4-Instruct
with LIMA
without gencralist
44.00
2
A
10k
Number of specialist data
Figure 2: Results on held-out tasks (Classification) with LLaMA-7B.
erage level), using LLaMA-7B. From the results presented Table 1, we can see that generalist models fall short in performance when compared to specialist models on all coverage levels. Notably, even as the coverage level becomes more encom- passing, the performance gap between generalist models and specialist models does not shrink. For instance, the most specific Yelp task, the special- ist model outperforms the generalist model (GPT4- Instruct) by 6.43% absolute points. On the SuperNI task, the performance gap between the specialist and the generalist (GPT4-Instruce is 29.38. These results validate the necessity of specialist tuning for specific NLP tasks.
The performance gain most evident when the amount of specialist data is limited. We can see that the performance gap between specialists trained with and without generalist data shrinks
as the amount of specialist data increases. For example, at the Classification level, when the spe- cialist data comprises only 2k instances, the in- clusion of GPT4-Instruce enhances LLaMA-7B's accuracy from 65.36% to 71.31% (+5.95%) and LLaMA-13B's accuracy from 70.59% to 73.13% (+2.54%). However, when the number of specialist data reaches 10k instances, the addition of GPT4- Instruct only leads to smaller improvements, from 80.02% to 80.17% (+0.15%) for LLaMA-7B, and from 81.01% to 81.93% (+0.92%) for LLaMA- 13B, respectively.
Transforming an LLM into superior specialist demands a substantial amount of task-specific data. Figure depicts the performance of spe- cialist models on different tasks with varying num- bers of training data (from 2k to 10k). From the results, we see that, for tasks with broader cover- age (e.g. SuperNI and Classification)), the model's performance does not seem to converge with the 10k training instances. Even for narrow tasks such Sentiment, least 10k task-specific data is re- quired to fully unlock the LLM's potential. These results reveal the data-hungry nature of building specialist models.
The performance gain is less pronounced when the model scale is larger. As shown in Figure 1, when comparing the results of the 7B and 13B mod- els. the trend of change in the effect of integrating generalist data is consistent for both models. How- ever, it is worth noting that as the model scale is larger, the performance gain is less pronounced. Specifically, when the model scales up from 7B to 13B, the average improvement achieved by adding GPT4-Instruct on SuperNI decreases from 1.49 to 0.58, and the improvement in Classification re- duces from 2.00% to 1.18%.
Generalist data can improve specialist perfor- mance when the task coverage broad. Figure also demonstrates that the inclusion generalist data consistently results performance improve-
4.4 Further Analysis
For a deeper understanding of the impact of gen- eralist data, here we present additional analyses.

---

72.00
80.00
69.00
75.00
Accuracy
Accuracy
70.00
66.00
65.00
with 2k specialist data
with GPT--inttrat without generalist data
63.00
60.00
Ok
10k
20k
30k
40k
50k
2k
10k
Number of generalist data
Number of specialist data
Figure 3: Results using different amounts of generalist data (Classification, 10k specialist data) with LLaMA- 7B.
Figure 4: The results using different test instructions (Classification) with LLaMA-7B.
data is sufficient to improve the specialist perfor- mance.
Unless otherwise specified, all experiments use LLaMA-7B as the foundation model.
Cross-task Generalization. For the Classifica- tion level, recall that we exclude some classification tasks when constructing the training data. These tasks can be used hold-out tasks to examine the specialist's cross-task generalization ability. The results are shown in Figure 2. can be observed that the accuracy on held-out tasks fluctuates in small ranges from 50.98% to 57.55% across dif- ferent amounts of specialist data. However, upon incorporating LIMA, the average absolute accu- racy improvement on the hold-out task increases by 2.70%, while adding GPT4-Instruct results in 6.12% rise in absolute accuracy. This indicates that generalist data can greatly improve the cross-task generalization of specialist models.
Cross-instruction Robustness. In all previous experiments, the models are trained and tested us- ing the same instructions for each dataset. Now, we assess the model's robustness when confronted with alternative instructions that have not appeared during training. To do so, we employ ChatGPT (OpenAl, 2022) to generate 20 semantically equiv- alent instructions based on the original instruction. Figure 4 reports the results of these unseen instruc- tions. As seen, the models trained with the addi- tion of generalist data exhibit substantial improve- ment in average accuracy compared the mod- els trained with specialist data only. For instance, when the specialist data is limited to 2k instances, incorporating generalist data leads to a 6.64% ab- solute improvement on average compared to the specialist model. In the meantime, the incorpo- ration of generalist data also alleviates the perfor- mance variation between the best-performing and worse-performing runs from 4.04% 2.42%.
Number of Generalist Data. of the amount of generalist data, additionally partition the GPT4-Instruct dataset into five ran- dom parts and test the model's performance when using different proportions of the dataset. The ex- periments are conducted at the Classification level with a fixed quantity of 10k specialist data. As shown in Figure 3, even with only 10k generalist data, the model's accuracy is raised from 78.12% to 82.48%. Another interesting finding is that further increasing the generalist data 50k merely brings small improvements (from 82.48% to 84.0%). The results together with our experiments with LIMA suggest that adding a small number of generalist
To study the effect
Experiments II: The Required Skills of the Target Tasks
We hypothesize that the model's ability perform specific NLP tasks can be attributed to the mix of several core capabilities. As such, we set up three target tasks that focus on three key skills which are detailed below.

---

46.00
73.00
62.00
Accuracy
Accuracy
45,00
69:00
59.00
with GPT4-Irsmet with LIMA
with CPTT-Trssmuc with LIMA
with GPPT4-nszuu with LIMA withour generalist darz
withost generalis data
without generalie dona
44.00
65.00
$6.00
10k
10k
10k
Number of specialist data (a) Factual-Knoowa Kotttaal-Kowledge-777
Number of specialist data (b) Understanddng-778
Number specialist data (c) Reasoning-78
$1.00
Accuracy
79.00
Accuracy
64.00
with GPP4--nettee with LIMA
with GPT4-Irsteee with LIMA
GPT4 instree with LIMA withou gitcralist
withuu generalic Bata
without pencralist
49.00
71.00
58.00
10k
10k
10k
Number specialist data (d) Factual-Knovllldge133
Number specialist data (c) Understanding-113 13B
Number specialist data (f) Reasoning-
Figure 5: Comparison of models trained with different combinations of specialist and generalist data across different tasks. We report Fl score Factual Knowledge and accuracy other levels.
Task Coverage
Ins4-cn
LIMA
specialist
Understanding acts as an important perspective as the capability to interpret input text. We choose the RACE dataset (Lai et al., 2017). RACE com- prises data collected from English examinations in China and specifically designed assess the model's ability to read and deeply comprehend texts in real-world scenarios.
Factual Knowledge
14.78
17.28
46.37
Understanding 35.10
30.82
76.77
Reasoning
28.02
26.58
63.40
Table The performance generalists and specialists tasks focusing on different skills. The specialists are trained with 10k task-specific instances on LLaMA-7B. For Factual Knowledge, the performance is measured by Fl score, while the others are measured by accuracy.
Reasoning is another fundamental ability for models to solve complex tasks. We use the SNLI (Bowman et al., 2015) dataset for implicit reason- ing, the CQA (Talmor et al., 2019) for common- sense reasoning, and the AQUA (Ling et al., 2017) dataset for arithmetic reasoning.
5.1 Skill Taxonomy
Factual Knowledge is essential for models to serve information needs. We use three knowledge-intensive datasets: Natural Questions (Kwiatkowssi et al., 2019), WebQuestions (Berant et al., 2013), and TriviaQA (Joshi et al., 2017). All these three datasets consist of entity-centric ques- tions, making them suitable for probing models' ability activate and utilize factual knowledge. Following previous work (Brown al., 2020), evaluate under the closed-book setting where mod- cls are required to answer questions without the help of any external knowledge grounding.
5.2 Evaluation Setup
For the Factual Knowledge tasks, we use greedy search with maximum generation length of 512. We adopt the Fl score as the evaluation metric fol- lowing (Brown et al., 2020). For the Understating and Reasoning tasks, we utilize the same classifi- cation with options method detailed in Section 3.2 and report the model accuracy.

---

5.3 Results and Analysis
Generalist models lag behind specialist models across all task skills. Similar to Experiment I, we commence by comparing specialist and generalist models across three target tasks, each concentrat- ing on distinct skills. The outcomes presented in Table 2 indicate that the generalist models consis- tently underperform the specialist models. For the Factual Knowledge task, the specialist model out- performs the generalist model with a 29.09 points higher Fl score. For the Understanding task, the specialist model surpasses the generalist model with 41.6 41.67% increase in accuracy. For the Rea- soning task, the specialist model excels beyond the generalist model, attaining 35.38% absolute accuracy difference. Collectively, these findings substantiate the necessity of specialist tuning for accomplishing specific tasks.
47.00
with GPT4-Instrut
4 with LIMA
without generalist data with Dolly
with Evol-Iastruct
F1 Score
46,00
with LIMA-Chal
45.00
44.00
4k
10k
Number of specialist data
Figure 6: Results on the Factual Knowledge with LLaMA-7B.
Incorporating GPT4-Instruct impairs the model's factual knowledge, while integrating LIMA offers benefits As illustrated in Figure 5, observe the varying impact of different generalist data on the model's performance in the Factual Knowledge task. In particular, when GPT4-Instruct is incorporated, the Fl score experiences a decline. Conversely, when LIMA data is integrated, the FI witnesses an increase. We argue that this difference stems from the fact that GPT4-Instruct is machine-generated,, while LIMA is human-authored. The rationale is that prcchnn--generatee data may contain hallucinations, thus impairing the model's ability to recall factual knowledge.
Chat. The only difference between LIMA-Chat and LIMA is that the responses in LIMA-CHI are machine- sechine-generated, while those in LIMA are human- written From Figure 6, we can see that LIMA-Chat indeed harms the performance while LIMA improves the performance. The above re- sults suggest that the choice of generalist data is crucial for target tasks that heavily rely factual knowledge.
Adding generalist data enhances the under- standing ability. The results of the Understand- ing task are presented in Figure 5. It is evident that greatly improves the
the addition of model's performance when the specialist data is only 2k or 4k instances. However, as the number of specialist data further increases, the improve- ment diminishes. This suggests that the inclusion of generalist data can enhance the model's compre- hension ability when the specialist data is limited.
To validate our hypothesis, we conduct experi- ments using additional generalist datasets, namely Dolly (Labs, 2023), and Evol-unstruct (Xu et al., 2023). Dolly consists of manually curated data generated by Databricks employees. Evol-Instruct uses more complex instructions than Instuuttt and collects responses from ChatGPT (Fang et al., 2023). As observed in Figure 6, adding Dolly does not impair the performance, but incorporating Evol- Instruct leads to similar performance degradation GPT4-Instruct. The above results are consistent with our hypothesis that machine- -eeerrrte gener- alist data might adversely affeett the model's 's factual knowledgeability due to hallucinations.
Adding generalist data enhances the reasoning ability. We further evaluate the consequences incorporating generalist data on the model's 's rea- soning ability, as demonstrated in Figure 5. No- tably, unlike Understanding, where the improve- ments from adding generalist data gradually dimin- ish, the benefits of incorporating generalist data on the Reasoning tasks are persistent across dif- ferent amounts of specialist data (an average im- provement of 0.65% on LLaMA-7B and 1.12% on LLaMA-13B). This phenomenon could be at- tributed to the fact that the activation of reasoning
For more rigorous comparison, we use Chat- GPT generate responses for the 1 instructions in LIMA. The new 1k instruction- instruction-response pairs form a new generalist dataset, which we call LIMA-

---

capabilities relies on diverse instruction data, and specialist data can be too narrow fully unlock the true potential of LLMs.
utilizing its expanded cognitive capacity to achieve superior performance.
Conclusions
Effect of Model Scale. For Factual Knowledge, increasing the model size from 7B 13B results in more substantial performance improvements com- pared to increasing the amount of specialist data. This observation aligns with previous work (Brown et al., 2020), which indicates that an LLM's knowl- edge is mostly obtained through its pre-training. For Understanding, increasing the model size is as beneficial as adding more specialist data. For Rea- soning, increasing the model size does not yield improvements as noticeable as Factual Knowledge and Understanding. We speculate that the emer- gence of strong reasoning abilities requires larger model scale (Wei et al., 2022b).
In this study, we thoroughly investigated the inter- action between specialist data and generalist data in the context of targeting specific NLP tasks. Our findings consistently demonstrate that the addition of generalist data leads to performance improve- ment when the task coverage is broad. This high- lights the potential benefits of incorporating gen- eralist data, particularly when the availability of specialist data is limited. Furthermore, we exten- sively examined the impact of integrating general- ist data on the model's core capabilities. Surpris- ingly, we observed that the inclusion of generalist data did not enhance the model's factuality. fact, generalist data containing hallucinatory infor- mation can have a negative impact. On the other hand, our experiments also revealed that the intro- duction of generalist data has positive effects on the model's understanding and reasoning abilities. Overall, our findings highlight the importance of leveraging generalist data enhance the under- standing and reasoning capabilities of NLP models, thereby enabling them to tackle various tasks more effectively. However, careful consideration should be given to the quality and reliability of the gener- alist data to avoid adverse effects on the model's ss factual knowledge.
Generalist data plays vital role in enhancing a model's understanding and reasoning capa- bilities, thereby increasing its effectiveness in addressing task-specific objectives. We dissect the model's s capabilities into three core components: (i) factual knowledge, (ii) understanding, and (iii) reasoning abilities. We demonstrate that incorpo- rating generalist data does not improve the model's factual knowledge and, in some cases, may even be detrimental if it includes hallucinated information. Nevertheless, comparative experiments focusing on understanding and reasoning abilities reveal that generalist data effectively fosters the model's com- prehension and significantly augments its reasoning capabilities.
Limitations
While this work aims to provide a comprehensive investigation, we note that we do not exhaustively cover all possible evaluations. For example, we do not discuss NLP tasks such as summarization. translation, etc. Instead, we focus on constructing a hierarchy of four target tasks of different coverage levels and three target tasks focusing different core skills. In addition, due resource constraints, we only use LLaMA 7B/13B as our foundation models. We leave the investigation on different types and/or larger scales of models to our future work.
This observed efficacy can be ascribed to the capacity of generalist data to facilitate the model's understanding and execution of diverse tasks. The wide range of instructions embedded within the generalist data stimulates the model's comprehen- sion and reasoning faculties, empowering it to grasp specific requirements associated with vari- tasks more effectively. Moreover, by activating the model's reasoning abilities, it showcases en- hanced performance across an assortment of tasks involving different levels of complexity.
The activation of comprehension and reasoning abilities further broadens the model's cognitive ca- pacity, allowing it to derive more comprehen- sive understanding based on existing information pertinent the given task. Consequently, the in- clusion of generalist data amplifies the model's task-spec capabilities, as becomes adept at
Acknowledgments
This work was partly supported by the Na- tional Key Research and Development Pro- gram of China (No.2020YFB 170220)) and the Shenzhen Science and Technology Program (ISGG20220831 10203007).

---

References
for Computational Linguisties: Human Language Technologies. Volume (Long and Short Papers).
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. ArXiv preprint, abs/2204.05222
Tao Fang. Shu Yang, Kaixin Lan, Derek F Wong, Jin- peng Hu, Lidia S Chao, and Yue Zhang. 2023. Is chatgpt highly fluent grammatical error correction system? a comprehensive evaluation. ArXiv preprint, abs/2304.01744
Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing Freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empiriaal Methods in Natural Lan- guage Processing.
Scott L Fleming, Alejandro Lozano, William J Haherkorn, Jenelle A Jindal, Eduardo P Reis, Rahul Thapa, Louis Blankemeier, Julian Genkins, Ethan Steinberg, Ashwin Nayak, et al. 2023. Medalign: A clinician-generateed dataset for instruction follow- ing with electronic medical records. ArXiv preprint, abs/2308. 1ete2308.14009.
Samuel Bowman, Gabor Angeli. Christopher Potts, and Christopher D. Manning. 2015. A large anno- tated corpus for learning natural language inference. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.
Arnay Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song. 2023. The false promise of imitating proprietary llm.. ArXiv preprint, abs/2305.15717.
Tom B. Brown, Benjamin Mann. Nick Ryder, Melanie Subbiah. Jared Kaplan, Prafulla Dhariwal. Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler. Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Ad- vances in Neural Information Processing Systems 33: Annual Conference on Neural Information Process- ing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. arXiv preprint arXiv:1904.0977 09751.
Wenxiang Jiao, Jen-tse Huang. Wenxuan Wang. Xing Wang, Shuming Shi, and Zhaopeng Tu. 2023a. Par- rot: Translating during chat using large language models. ArXiv preprint, hts/23344022466.
Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Xing Wang, and Zhaopeng Tu. 2023b. Is chatgpt A good trans f A preliminary study. ArXiv preprint,
Daixuan Cheng. Shaohan Huang, and Furu Wei. 2023. Adapting large language models via reading compre- hension. ArXiv preprint, abs/230..09530.
Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehen- sion. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics ( (Vol- ume 1: Long Papers).
Wei-Lin Chiang, Zhuohan Li. Zi Lin, Ying Sheng. Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open- source chatbot impressing gpt-4 with 90%* chatgpt quality.
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd Inter- national Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph. Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Web- son, Shixiang Shane Gu, Zhuyun Dai, Mirac Suz- gun, Xinyun Chen, Aakanksha Chowdhery, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Y. Zhao, Yanping Huang, Andrew M. Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoe V. Le, and Jason Wei. 2022. Scaling instruction-finent finetuned language models. ArXiv preprint, abs/2210. 11416.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Ken- ton Lee, Kristina Toutanova. Llion Jones, Matthew Kelcey, Ming-We Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Lc, and Slav Petrov. 2019. Natu- ral questions: A benchmark for question answering research. Transactions of the Association for Compu- tational Linguistics, 7.
Jacob Devlin, Ming- Wei Chang. Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association
Databricks Labs. 2023. Dolly: A tool for data ver- sioning and dataops in databricks. https: https://giue //ubthbbb. htt.d/atabrrikllabb///b Accessed on June 8 2023.

---

Guokun Lai, Qizhe Xie. Hanxiao Liu, Yiming Yang. and Eduard Hovy. 2017. RACE: Large-scale ReAd- ing comprehension dataset from examinations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.
Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szezechla, Taewoon Kim, Gunjan Chhablani, Nihal V. Nayak, Dehajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang. Matteo Manica, Sheng Shen, Zheng Xin Yong. Harshit Pandey, Rachel Bawden, Thomas Wang, Tr- ishala Neeraj. Jos Rozen, Abheesht Sharma, An- drea Santilli, Thibault Févry, Jason Alan Fries, Ryan Teehan, Teven Le Scao, Stella Biderman, Ieo Gao, Thomas Wolf, and Alexander M. Rush. 2022. Multi- task prompted training enables zero-shot task gener- alization. In The Tenth International Conference Learning Representations, ICLR 2022, Viriual Event, April 29, 2022.
Siheng Li, Cheng Yang, Yichun Yin, Xinyu Zhu, Ze- Cheng, Lifeng Shang, Xin Jiang, Qun Liu, and Yujiu Yang. 2023. AutoConv: Automatically gener- ating information-seeking conversations with large language models. In Proceedings of the sl An- nual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers).
Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out.
Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blun- som. 2017. Program induction by rationale genera- tion: Learning solve and explain algebraic word problems. In Proceedings of the 55th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers).
Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F. Christiano. 2020. Learn- ing to summarize with human feedback. In Advances in Neural Information Processing Systems 33: An- nual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual.
Aiwei Liu, Xuming Hu, Lijie Wen, and Philip S Yu. 2023. A comprehensive evaluation of chatgpt's zero-shot text-to-sql capability. ArXiv preprint. abs/2303.13547.
Yixuan Su and Nigel Collier. 2022. Contrastive search is what you need for neural text generation. arXiv preprint arXiv:22 14..4400.
Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2021. Natural instructions: Benchmarking generalization to new tasks from natural language instructions. ArXiv preprint. 00s/2104.08773.
Yixuan Su, Tian Lan, and Deng Cai. 2023a. Ope- nalpaca: fully open-source instruction-followee following model based on openllama. https: /ttptt//github com/ yxuansu/OpenAlpaca.
OpenAI. 2022. Introducing chatgpt. https://openai. com/blog/ chatgpt.
Yixuan Su, Tian Lan, Huayang Li, Jialu Xu, Yan Wang, and Deng Cai. 2023b. Pandagpt: One model to instruction-follow them all. ArXiv preprint, abs/2305.16355.
com/bloggchattpt..
OpenAL. 2023. GPT- technical report. ArXiv preprint, abs/2303.08774.
Long Ouyang. Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instruc- tions with human feedback. Advances in Neural Information Processing Systems, 35.
Yixuan Su, Tian Lan, Yan Wang, Dani Yogatama, Ling- peng Kong, and Nigel Collier. 2022. A contrastive framework for neural text generation. Advances in Neural Information Processing Systems, 35:21548- 21561.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. CommonsenseQA: A ques- tion answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Popers).
Baolin Peng. Chunyuan Li, Pengcheng He, Michel Gal- ley, and Jianfeng Gao. 2023. Instruction tuning with GPT-4, ArXiv preprint, abs/2304.03277.
Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang. 2023. Is chatgpt a general-purpose natural language process- ing task solver? ArXiv preprint, abs/2302.006466.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction- following 1aaa model. https:/ gi thub. www.//ttss-lee waw.suuaaaatttaafraaraeeaa stanfodd_daadaacc...
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018 Improving language under- standing by generative pre-training.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, al. 2019. Language models are unsupervised multitask learners. OpenAl blog, 188).
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Maric-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard

---

Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models. ArXiv preprint, abs/2302. 13971.
Xiang Zhang, Junbo Jake Zhao, and Yann LeCun, 2015. Character-level convolutional networks for text clas- sification. In Advances in Neural Information Pro- cessing Systems 28: Anmual Conference on Neural Information Processing Systems 2015, December 7- 12, 2015, Montreal, Quebec, Canada.
Haochun Wang, Chi Liu, Nuwa Xi, Zewen Qiang. Sendong Zhao, Bing Qin, and Ting Liu. 2023a. Hu- atmo: Tuning llaaa model with chinese medical knowledge. ArXiv preprint, abs/2304.06975.
Yue Zhang, Leyang Cui, Deng Cai, Xinting Huang, Tao Fang, and Wei Bi. 2023. Multi-task instruction tuning of llama for specific scenarios: A prelimi- nary study on writing assistance. ArXiv preprint, hass2305513225.
Xiao Wang, Weikang Zhou, Can Zu, Han Xia, Tianze Chen, Yuansen Zhang,
Zheng, Junjie Ye, Qi Zhang. Tao Gui. Jihua Kang, Jingsheng Yang. Siyuan Li, and Chunsai Du. 2023b. Instructuie: Multi-task instruction tuming for unified information extraction. ArXiv preprint, 0bs/2304,08088,
Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. 2023. Lima: Less is more for alignment. ArXiv preprint, 0bs/2305.11206
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Al- isa Liu, Noah A. Smith, Daniel Khashabi, and Han- nanch Hajishirzi. 2022a. Self-instruct: Aligning lan- guage model with self generated instructions. ArXiv preprint, abs/221 2. 10560.
Yizhong Wang, Swaroop Mishra, Pegah Alipoormo- labashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap. Eshaan Pathak, Giannis Karamanolakis, Haizhi Lai, Ishan Puro- hit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Kuntal Kumar Pal, Maitreya Patel, Mehrad Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia. Savan Doshi, Shailaja Keyur Sampat, Siddhartha Mishra, Sujan Reddy A. Sumanta Patro, Tanay Dixit, and Xudong Shen. 2022b. tion via declarative instructions on 1600+ NLP tasks. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing.
Naturallnsttructinn: Generaliza-
Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, An- drew M. Dai, and Quoc V. Le. 2022a. Finetuned language models are zero-shot learners. In The Tenth International Conference on Learning Representa- tions, ICLR 2022, Virtual Event, April 25-29, 2022.
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, William Fedus. 2022h. Emer- gent abilities of large language models. Trans. Mach. Learn. Res., 2022.
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023. WizardIm: Empowering large lan- guage models to follow complex instructions. ArXiv preprint, abs/2304.1224
Haoran Yang. Deng Cai, Huayang Li, Wei Bi, Wai Lam, and Shuming Shi. 2023. A frustratingly simple decoding method neural text generation. arXiy preprint arXiv:2305. 12675.

---

A
Instruction Template
INSTRUCTION FORMAT
Below is an instruction that deseribes task, paired with input that provides further context. Write response that appropriately completes the request.
# #nsttruttt
[Task Prompt]
#Inut: [Input Text]
# Response: [Output Text]


Reviews:
[{"review_id": "CvQw5kPpbs", "reviewer": "Reviewer_ReUf", "summary": "", "questions": "From the weaknesses:\n1. Can you explain the reasoning behind the model choices?\n2. Can you explain the dataset choices and why not more tasks?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**To Review #1**\n\nThank you for your valuable insights and favorable impression of our work. We appreciate your thoughtful feedback, and we would like to address your concerns as follows:\n \n**R1:The analysis is only performed with one model. Although the one model is timely and relevant, it would be nice to see this replicated with another model (or even with different tuning, like LoRA.**\n \nThanks for your valuable suggestion! The main reason for us doing experiments with only one model (Llama-7B) is the budget of computational resources. We choose Llama-7B for its great popularity in recent works and moderate computational cost. We are actively conducting experiments on larger models (e.g., Llama-13B) with the more efficient training method, LoRA, as you suggested. For the numbers we have collected, we anticipate similar findings regarding the impact of generalist data.\n \n**R2: Classification, sentiment, and Yelp are very similar tasks IMO. Perhaps my fellow reviewers would disagree, but I would like to see something more different like QA or summarization (which are in SuperNI, so I’m not sure where those fit in this paper?))**\n \nWe appreciate your perspective, and we'd like to explain our rationale for this choice. One primary purpose of Experiments I is to assess a variety of target tasks with distinct levels of generality (i.e., how specific the target task is). We construct a hierarchy of four specialist tasks: SuperNI (multiple tasks, multiple formats) > Classification (multiple tasks, single format) > Sentiment (single task, multiple domains) > Yelp (single task, single domain). Specifically, the Classification task encompasses a wide variety of challenges (e.g., cause-effect classification, sentence composition, text matching, toxic language detection, ethics classification, spam classification, etc) that are NOT included in the Sentiment tasks. The Sentiment task encompasses multiple domains (e.g., food/movie/ book/music reviews, Twitter posts, Reddit posts, etc), while the Yelp task only covers product/service reviews from the Yelp platform. \nThe reason for choosing classification tasks over QA and summarization to construct the above hierarchy is that QA and summarization are much more difficult to evaluate. For example, different QA and summarization datasets have different output formats (e.g., some are phrases while others are complete sentences). In our preliminary experiments, we found that this issue has a significant impact on the final evaluation result. However, we acknowledge that presenting an extra experiment on QA and summarization would be beneficial and will add it in the next version.\n \n**R3: There aren’t any generalist datasets used that are published that I can find (are Lima and GPT4-Instruct published?) and although they are well hyped and (perhaps?) well used now, it’s hard to assess their quality. It would have been nice to see some more established generalist fine-tuning datasets like FLAN or those used in T0, etc. The authors may have had a good reason for this, looking forward to the rebuttal.**\n \nThank you for sharing your concerns regarding the generalist datasets used in our work, specifically LIMA and GPT4-Instruct. It's entirely understandable to seek clarity on these datasets, and we are pleased to provide some context.\nYou can find the datasets for LIMA and GPT4-Instruct at the following links:\nLIMA: https://huggingface.co/datasets/GAIR/lima\nGPT4-Instruct: https://huggingface.co/datasets/vicgalle/alpaca-gpt4\n \nRegarding your mention of more established datasets like FLAN or T0. We would like to clarify that, in our work's context, FLAN, T0, and SuperNI are considered as specialist data, because they are constructed from existing NLP datasets focusing on specific tasks and fixed instruction templates. Note that SuperNI has already been used in our experiments. We only consider datasets like LIMA and GPT4-Instruct fit the category of generalist data in our study as they aim to meet any real user demands.\n\n**Q1: Can you explain the reasoning behind the model choices?**\n \nThanks for your thoughtful question. Please refer to our answer to R1.\n \n**Q2. Can you explain the dataset choices and why not more tasks?**\n \nThanks for your thoughtful question. Please refer to our answer to R2.\n\n**P1. Presentation Improvements**\n\nWe appreciate your feedback on the presentation and will incorporate your suggestions to enhance the clarity of the paper. Thank you for your guidance.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper is timely and relevant to much of the NLP community which is adapting to instruction data and LLMs\n- The work is relatively easy to read and includes lots of great analysis\n- The analysis includes several instruction datasets that are used today (Lima, GPT-instruct, etc.) and a large number of experiments", "reasons_to_reject": "- The analysis is only performed with one model. Although the one model is timely and relevant, it would be nice to see this replicated with another model (or even with different tuning, like LoRA).\n- Classification, sentiment, and Yelp are very similar tasks IMO. Perhaps my fellow reviewers would disagree, but I would like to see something more different like QA or summarization (which are in SuperNI, so I’m not sure where those fit in this paper?)\n- There aren’t any generalist datasets used that are published that I can find (are Lima and GPT4-Instruct published?) and although they are well hyped and (perhaps?) well used now, it’s hard to assess their quality. It would have been nice to see some more established generalist fine-tuning datasets like FLAN or those used in T0, etc. The authors may have had a good reason for this, looking forward to the rebuttal.\n\nSome of these may be a miscommunication, I'm open to increasing my score in Soundness from 3 to 4 on the dataset/modeling issues.", "questions_to_authors": "From the weaknesses:\n1. Can you explain the reasoning behind the model choices?\n2. Can you explain the dataset choices and why not more tasks?", "paper_topic_and_main_contributions": "This paper proposes to examine whether general purpose instruction tuning contributes to building specialized models, across a variety of tasks. They show task performance improves with more specialized training data (as expected) and that generalist data generally improves task performance even with lots of specialized data (although general purpose data is not enough).\n\nTLDR summary: I like this paper and learned from it, which I think is the sign of a good paper. However, there are some slight deficiencies in the experimental setting. Overall, I have a favorable impression.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "I think a label before the legend in Figures like Figure 1 would be helpful (something like “General Data Used” or something. It took me a bit to parse the figures, but I think they are well done once you figure them out.\n\nIt would be nice to have somewhere a longer description of these generalist datasets. I’m not familiar with evol-instruct for example. Perhaps this could go in the appendix? This doesn’t influence my score, but just a suggestion."}}, {"review_id": "wzOVmoqSS7", "reviewer": "Reviewer_b21v", "summary": "", "questions": "It feels a little awkward to me to consider S-NI a \"specialist\" dataset. While I understand what the authors are referring to (the dataset is composed of fixed tasks with examples for each task), it is worth noting that S-NI has in aggregate more tasks than there are examples in LIMA. Though a secondary factor is that S-NI is probably less diverse in tasks than LIMA, I think the authors should more strongly justify why they consider S-NI a specialist dataset and how that fits into the broader message of their work.\n\nSecondly, there are insufficient details on how the fine-tuning is performed (e.g. GPT4-Instruct and LIMA have very different number of examples: how is this balanced / what hyperparameters were used), and the exact format of combined fine-tuning on the generalist and specialist datasets are not clearly specified, unless I overlooked a section.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**To Review #2**\n \n \nWe appreciate your discerning remarks and positive views regarding our work. Your thoughtful feedback is highly valued, and we are committed to addressing the questions you have raised.\n \n**R1: In general, I find Experiments II to be much weaker than Experiment I. The authors use a single task for each attribute they are examining, and appear to draw far too strong conclusions for the experiments they have run.**\n\nThere seems to be some misunderstanding regarding the setups in Experiments II. For Factual Knowledge, we use a mixture of three datasets: Natural Questions, WebQuestions, and TriviaQA. For Reasoning, we also use a mixture of three datasets: SNLI (implicit reasoning), CQA (commonsense reasoning), and AQUA (arithmetic reasoning). Nevertheless, we understand that evaluating on these public NLP datasets might not completely reflect the knowledge, understanding, and reasoning abilities of LLMs. We will revise our writing accordingly to make our conclusions more task-specific.\n \n**R2: A similar issue of over-extrapolation is found in the section L464-495, where there is a hypothesis that the reason for the difference in performance across datasets is due to being human-authored vs. machine-generated, but they supplement this experiment with a single experiment on an additional pair of datasets. Given that all four training sets are highly esoteric and greatly different, I do not believe there is any basis to make this claim at this point, and I would recommend the authors heavily soften the claim. (I acknowledge that the authors are already fairly conservative in their wording (\"hypothesize\", \"are consistent with\", \"suggests\"), but I nevertheless think that they need to water it down further.)**\n \nThanks for your insightful suggestion. Our experiments are based on two human-authored and two machine-generated generalist datasets. We understand that there are many other factors comparing different generalists and will explicitly add the limitations you mentioned.\n\n**Q1: It feels a little awkward to me to consider S-NI a \"specialist\" dataset. While I understand what the authors are referring to (the dataset is composed of fixed tasks with examples for each task), it is worth noting that S-NI has in aggregate more tasks than there are examples in LIMA. Though a secondary factor is that S-NI is probably less diverse in tasks than LIMA, I think the authors should more strongly justify why they consider S-NI a specialist dataset and how that fits into the broader message of their work.**\n \nThank you for your insightful observation. Your comment brings attention to the distinction we are trying to make between SuperNI (S-NI) and LIMA, and we understand why this classification might initially seem ambiguous.\n \nThe categorization of SuperNI as a \"specialist\" dataset in our work relies on the specific nature of the tasks within it. Although SuperNI encompasses a broad array of tasks (768 tasks are adopted in our experiments), these tasks are fundamentally geared towards natural language processing tasks distilled from real-world demands, such as determining different types of textual relationships, causality judgments of two sentences, selecting the best answers, summarization, translation, etc. They focus on fixed and definable tasks that can often have an approximate standard answer.\nHere's an illustrative example from SuperNI:\n\n\\===================================================================\n\n**Example from SuperNI:**\n\nTask: In this task you're given two statements. You must judge whether the second sentence is the cause or effect of the first one. Label the instances as “cause” or “effect” based on your judgment. The sentences are separated by a newline character.\nInput: The women met for coffee. They wanted to catch up with each other.\nAnswer: cause.\n\\===================================================================\n \nContrast this with an example from LIMA, which may encompass mathematical concept explanations, lifestyle advice, geographical information, personal experiences, etc. The instructions within LIMA are more oriented towards user needs and are not as easily captured by conventional single tasks.\n\n\\===================================================================\n\n**Example from LIMA:**\n\nInput: How to be a lazy college student?\nAnswer: In college, unlike in high school, you are expected to take responsibility for your own learning. It is ultimately up to you to determine how much time and effort you want to put into your schoolwork. Experts suggest that college students today work considerably fewer hours than their counterparts did in 1961. But there is also evidence that working more hours is not necessarily the best way to be productive. A little laziness may even be good for us. Part 1 of 3: prioritize what’s important…\n\n\\===================================================================\n\nOur classification of SuperNI as a specialist dataset emphasizes the task-specific nature of the challenges it poses, while LIMA's generalist nature accommodates a broader, user-oriented spectrum of inquiries.\n \nI hope this explanation addresses your concern and provides the justification for our categorization of SuperNI. Another point is that the SuperNI dataset is used as specialist data in the broader context of Experiments I, where we deliberately construct a variety of target tasks with varying scopes of coverage (i.e., how specific the target task is).\n \n**Q2: Secondly, there are insufficient details on how the fine-tuning is performed (e.g. GPT4-Instruct and LIMA have very different number of examples: how is this balanced / what hyperparameters were used), and the exact format of combined fine-tuning on the generalist and specialist datasets are not clearly specified, unless I overlooked a section.**\n \nWe deliberately choose the two generalist data sets, GPT4-Instruct and LIMA, for they vary significantly in the number of examples, the quality of data, the source of instructions, and the source of responses, etc. Throughout all experiments, we stick to a common training strategy: as stated in line 231-239, we fine-tune the model using the Adam optimizer with learning rate 2e-5 for 3 epochs. Additionally, we use a batch size = 128.\n \nFor the data format, as stated in Line 212-217, we follow the template used in Stanford’s Alpaca project for all generalist and specialist datasets.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This work addresses a long investigated and still important question in transfer learning, which is more prominent in the present with the increasing popularity of \"generalist\" instruction tuning datasets. The datasets chosen actively used in research and for tuning of current highly capable models. The experiments run the mixture of training datasets are of the right format to answer these questions.", "reasons_to_reject": "In general, I find Experiments II to be much weaker than Experiment I. The authors use a single task for each attribute they are examining, and appear to draw far too strong conclusions for the experiments they have run. A similar issue of over-extrapolation is found in the section L464-495, where there is a hypothesis that the reason for the difference in performance across datasets is due to being human-authored vs. machine-generated, but they supplement this experiment with a single experiment on an additional pair of datasets. Given that all four training sets are highly esoteric and greatly different, I do not believe there is any basis to make this claim at this point, and I would recommend the authors heavily soften the claim. (I acknowledge that the authors are already fairly conservative in their wording (\"hypothesize\", \"are consistent with\", \"suggests\"), but I nevertheless think that they need to water it down further.)", "questions_to_authors": "It feels a little awkward to me to consider S-NI a \"specialist\" dataset. While I understand what the authors are referring to (the dataset is composed of fixed tasks with examples for each task), it is worth noting that S-NI has in aggregate more tasks than there are examples in LIMA. Though a secondary factor is that S-NI is probably less diverse in tasks than LIMA, I think the authors should more strongly justify why they consider S-NI a specialist dataset and how that fits into the broader message of their work.\n\nSecondly, there are insufficient details on how the fine-tuning is performed (e.g. GPT4-Instruct and LIMA have very different number of examples: how is this balanced / what hyperparameters were used), and the exact format of combined fine-tuning on the generalist and specialist datasets are not clearly specified, unless I overlooked a section.", "paper_topic_and_main_contributions": "This work investigates the training benefits of using \"generalist\" and \"specialist\" datasets for  instruction tuning, comparing the relative benefits of each in various combinations.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "IzSlUaJg0J", "reviewer": "Reviewer_JSaf", "summary": "", "questions": "- Did you do an analysis of the overlap between instructions of the generalist data? If so, could you have combined GPT4-Instruct and LIMA for one generalist model? Especially with the findings for Table 1 vs 2 suggest that LIMA/GPT4-Instruct could be beneficial for different tasks. \n- I'm missing some motivation for the 50k specialist training examples as in some cases the model doesn't seem to converge, e.g., Figure 1a, b. Why did you stop at 50k and not continue till convergence?\n- Do you foresee any issues with the different sizes of the generalist data? E.g., GPT4-Instruct is 50k instances and LIMA is 1k.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**To Review #3**\n\nThank you for your insightful comments and questions! We would like to provide our answers as follows:\n\n**R1: I like the general idea of the analysis, though I feel the insights are not substantial enough for a long paper. For example, insights into the sizes of generalist data are lacking. What are the implications of performance on target tasks with different sizes of generalist data? Figure 3 gives some very brief insights, but the paper would improve given some extra insights for generalist data. I give several suggestions for making more space for in the section \"Typos Grammar Style And Presentation Improvements\".**\n\nIn Figure 3, we study the effect of the amount of generalist data. The results show that adding a small number of generalist data is sufficient to improve the specialist performance (the improvement is small after we increase the generalist data to 10k). In other experiments, comparing GPT4-Instruct (50k instances) and LIMA (1k instances), we can also see that their performance is close in many cases, indicating that the quantity is not the main factor, and we should better consider the quality and diversity. This is also in line with recent studies on generalist instruction tuning (Zhou et al., 2023; Gudibande et al., 2023) (also mentioned in Line 116-123).\n\n**R2+R3: Overall, I feel the paper shows a rather trivial finding: More generalist + specialist data is better. It doesn't serve insights into the limitations of generalist/specialist data (i.e., convergence) and is not mentioned in the Limitations section. The authors mention a systematic guide in their contributions. However, there are insufficient details in the paper of such a guide. There is a list of findings from the analysis, but it does not give a systematic guide.**\n \nThank you for the question. We apologize for any confusion regarding the systematic guide. We summarize the key findings of our work as below:\n1. Incorporating generalist data leads to performance improvements in NLP tasks, regardless of the task's coverage level.\n2. Integrating generalist data is particularly beneficial when specialist data is limited.\n3. Adding a small number of generalist data is sufficient to improve the specialist performance.\n4. Adding generalist data enhances cross-task generalization and cross-instruction robustness.\n5. The inclusion of generalist data does not necessarily enhance the model's factuality and may have negative effects.\n6. Generalist data plays a vital role in enhancing a model’s understanding and reasoning capabilities, thereby increasing its effectiveness in addressing task-specific objectives.\nWe consider that the above findings provide meaningful insights into when and how generalist data help specialist instruction tuning.\n \n**Q1: Did you do an analysis of the overlap between instructions of the generalist data? If so, could you have combined GPT4-Instruct and LIMA for one generalist model? Especially with the findings for Table 1 vs 2 suggest that LIMA/GPT4-Instruct could be beneficial for different tasks.** \n \nWe deliberately choose the two generalist data sets, GPT4-Instruct and LIMA, for they vary significantly in many aspects, including the number of examples, the quality of data, the source of instructions, and the source of responses, etc. Specifically,  GPT4-Instruct contains 52k unique instruction-response pairs, where the instructions are collected through self-instruct and the responses are generated by GPT-4. LIMA consists of 1k carefully curated instruction-response pairs derived from human-authored community questions and answers. We emphasize that GPT4-Instruct serves as an example of generalist data synthesized by LLMs and LIMA represents an example of generalist data written by humans. Comparing the experiment results of the two datasets leads to interesting findings like the one you mentioned and also many others. For example, incorporating GPT4-Instruct  impairs the model’s factual knowledge while integrating LIMA offers benefits. The results give us insights that careful consideration should be given to the reliability of the generalist data to avoid adverse effects on the model’s factual knowledge. Your suggestion on combining the different generalist datasets is very insightful. It is an example that our research may inspire interesting future research. We are happy to add it in the next version. However, please kindly note that the focus of this paper is not to create better generalist data but a comprehensive evaluation using existing generalist data.\n\n**Q2: I'm missing some motivation for the 50k specialist training examples as in some cases the model doesn't seem to converge, e.g., Figure 1a, b. Why did you stop at 50k and not continue till convergence?**\n \n\nThank you for the question, we are pleased to make the following classification. The primary focus of our experiments is to study the impact of generalists. Therefore, we are especially interested in the performance improvements brought by the generalist data rather than the absolute performance. Generally, the improvement from generalist data diminishes as the number of specialist data increases. From Figure 1 a and b, we can see that the improvement of generalist data already converges when the number is 50K. That is why we stop at 50K. \n \n**Q3: Do you foresee any issues with the different sizes of the generalist data? E.g., GPT4-Instruct is 50k instances and LIMA is 1k.**\n \nWe deliberately choose the two generalist data sets, GPT4-Instruct and LIMA, for they vary significantly in the number of examples, the quality of data, the source of instructions, and the source of responses, etc. According to recent work (Zhou et al., 2023; Gudibande et al., 2023) , the quantity of generalist data is not a key factor. For example, LIMA (1k) has been found to outperform Alpaca (52k). AlpaGasus (9k), a subset of Alpaca outperforms Alpaca (52k). In the context of our study, we also find that in many cases the performances of LIMA and GPT4-Instruct are close, despite that Alpaca being 50x larger. Regarding the exclusive effect of data quantity, we present an analysis in Figure 3.\n \n**Presentation Issues**\n \n**P1: Suggestions for Space Optimization:**\n\nYour recommendations for merging tables and figures, as well as restructuring the layout, are much appreciated. Should the paper be accepted, we will indeed make the necessary revisions to improve the layout.\n \n**P2: Scaling Up Generalist Data:**\n\nSince the GPT4-Instruct data has a maximum of only 52k instances, it was not feasible to scale it to 100k. However, it is worth noting that the curve from 10k to 50k already conveys a clear trend, demonstrating that even a small amount of generalist data is effective. While more data might be better, the improvement seems marginal.\n \n**P3: Integration of Dolly and Evol-Instruct in Table 2:**\n\nWe will ensure that the final version includes the results for Dolly and Evol-Instruct. Despite this omission, we want to emphasize that all generalist models lag behind specialist models.\n \n**P4: Conclusion and Specialist Data Usage:**\n\nThank you for your constructive feedback. While the usage of specialist data for specific tasks is aligned with common understanding (i.e., more data for a particular task leads to better performance). Our work suggests to practitioners that they can reduce the demand for specialist data by including generalist data according to the nature of the target task.\n\n\nOnce again, thank you for your constructive questions. We will include our answers in the next version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The analysis is insightful, I especially like the wide array of tasks and experiments covered and there is substantial evidence that the findings are robust across tasks.\n- The paper is furthermore well written and follows a straightforward structure.", "reasons_to_reject": "- I like the general idea of the analysis, though I feel the insights are not substantial enough for a long paper. For example, insights into the sizes of generalist data are lacking. What are the implications of performance on target tasks with different sizes of generalist data? Figure 3 gives some very brief insights, but the paper would improve given some extra insights for generalist data. I give several suggestions for making more space for in the section \"Typos Grammar Style And Presentation Improvements\".\n- Overall, I feel the paper shows a rather trivial finding: More generalist + specialist data is better. It doesn't serve insights into the limitations of generalist/specialist data (i.e., convergence) and is not mentioned in the Limitations section.\n- The authors mention a systematic guide in their contributions. However, there are insufficient details in the paper of such a guide. There is a list of findings from the analysis, but it does not give a systematic guide.", "questions_to_authors": "- Did you do an analysis of the overlap between instructions of the generalist data? If so, could you have combined GPT4-Instruct and LIMA for one generalist model? Especially with the findings for Table 1 vs 2 suggest that LIMA/GPT4-Instruct could be beneficial for different tasks. \n- I'm missing some motivation for the 50k specialist training examples as in some cases the model doesn't seem to converge, e.g., Figure 1a, b. Why did you stop at 50k and not continue till convergence?\n- Do you foresee any issues with the different sizes of the generalist data? E.g., GPT4-Instruct is 50k instances and LIMA is 1k.", "paper_topic_and_main_contributions": "This paper is about instruction tuning. Specifically, they tune three models (backbone of Llama-7b) with \"generalist\" data consisting of GPT4-Instruct, LIMA, and no generalist data. Once the models are obtained, they instruct-tune the three models with \"specialist\" data and show an analysis across three target tasks on how much specialist data is needed to make an LLM a \"specialist\". They experiment with several data sizes for specialist data.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Suggestions to make more space:\n- Table 1 and Figure 1 can be merged if the figure started from 0 on the x-axis. In addition, Fig1+Table1 could have been squeezed into a 1x4 grid instead of 2x2, which then saves space.\n- Since Fig 2,3,4 are used for Classification, you could squeeze them together again using subfigures which saves more space. Also, Table 2 can again be integrated in Fig 5, 6, 7 and squeezed together.\n\n\nOther:\n- Personal preference: The paper could have potentially started with broader coverage, e.g., starting with an analysis of the impact of scaling up the generalist data, finding convergence, and using that model to then further improve using specialist data. From Fig3, it doesn't seem that the model converged yet, and it's not too hard to increase the data to 100k (unless I'm missing something here).\n- Some inconsistency with adding a period after a paragraph header.\n- Dolly and Evol-Instruct do not seem integrated in Table 2.\n- I'm surprised that the conclusion doesn't give any takeaways for the usage of specialist data and only mentions generalist data, would be good to add."}}]

Author rebuttal:

Meta-review: The reviewers are positive about the soundness of the paper. The work is timely, addresses an important question in transfer learning, and the analysis is thorough. The writing is easy to understand. The reviewers raised some issues about the experimental design, some of which the authors addressed during the author response period.

Overall, the reviewers believe the paper is sound and are excited to see it published.
Decision: Accept-Main