MUFFIN: Curating Multi-Faceted Instructions for Improving Instruction Following
Authors: Renze Lou
Kai Zhang
Jian Xie
Yuxuan Sun
Janice Ahn
Hanzi Xu
Yu Su
Wenpeng Yin
Venue: ICLR 2024 poster (2024)
Keywords: Instruction Tuning
Large Language Models
Automatic Data Generation
Primary area: datasets and benchmarks

Abstract: In the realm of large language models (LLMs), enhancing instruction-following capability often involves curating expansive training data. This is achieved through two primary schemes: i) Scaling-Inputs: Amplifying (input, output) pairs per task instruction, aiming for better instruction adherence. ii) Scaling Input-Free Tasks: Enlarging tasks, each composed of an (instruction, output) pair (without requiring a separate input anymore). However, LLMs under Scaling-Inputs tend to be overly sensitive to inputs, leading to misinterpretation or non-compliance with instructions. Conversely, Scaling Input-Free Tasks demands a substantial number of tasks but is less effective in instruction following when dealing with instances in Scaling-Inputs. This work introduces MUFFIN, a new scheme of instruction-following dataset curation. Specifically, we automatically Scale Tasks per Input by diversifying these tasks with various input facets. Experimental results across four zero-shot benchmarks, spanning both Scaling-Inputs and Scaling Input-Free Tasks schemes, reveal that LLMs, at various scales, trained on MUFFIN generally demonstrate superior instruction-following capabilities compared to those trained on the two aforementioned schemes.

Published as a conference paper at ICLR 2024
MUFFIN:
CURATING MULTI-FACETED INSTRUC-
TIONS FOR IMPROVING INSTRUCTION-FOLLOWWE
Renze Lou"
Kai Zhang
Jian Xie
Yuxuan Sunf
Janice Ahn
Hanzi Xu*
Yu Su
Wenpeng Yin
The Pennsylvania State University; Thh Ohio State University;
Fudan University; WWestlake University; *Temple University (renze.lou, @pmpengg@ssu...o edu
ABSTRACT
In the realm of large language models (LLMs), enhancing instruction-following capability often involves curating expansive training data. This is achieved through Amplifying (input, output) pairs per task instruction, aiming for better instruction adherence. ii) Scaling Input-Free Tasks: Enlarging tasks, each composed of an (instruction, output) pair without requiring a separate input anymore. However, LLMs under to be overly sensitive inputs, leading misinterpretation non-compliance with instructions. Additionally, Scaling Input-Free Tasks demands a substan- tial number of tasks but is less effective in instruction-following when dealing with instances in Scaling- Inputs.. This work introduces MUFFIN, a new scheme of fooloowion dataset curation. Specifically, we automatically Scale Tasks per Input by diversifying these tasks with various input facets. Experimen- tal results across four zero-shot benchmarks, spanning both Scaling-Inputf and Scal ing Input-Free Tasks schemes, reveal that LLMs, at various scales, trained on MUFFIN generally demonstrate superior cnerructio-folllwiiii capabilities com- pared to those trained on the two aforementioned schemes]
two primary schemes: i) Scaling-Inpuue
tend
INTRODUCTION
With advancements in pre-training techniques, large language models (L.LMs) can, to some extent,
tackle diverse unseen tasks guided by textual instructions (Radford al. 2019 Brown et al. 2020). This capability, known as Instruction-Follooinn is pivotal for developing unified versatile LLMs.
Instruction-un -tuning. training LLMs to generate desired responses following given instructions for enhanced www.itiiooooooloiinng capacity, has garnered increased attention in the community (Mnn
al. 2022 Chung et al. 2022: Longpre et al. 2023 Lou al., (023).
The construction of datasets is crucial in instruction-tuning (Wang et al. 2023a Zhou et al. 2023). Existing approaches primarily adopt two strategies for constructing these datasets: (i) Scaling-Inputs, gathering vast set of training tasks, each accompanied by an instruction, and then amplifying the (input, output) pairs for each task (Mishra et al. 2022b) [anh et al. 2022 Wei et al. 2022 Wang et al. 2022). The model is trained to produce distinct outputs for various inputs under the same instruction. However, this approach tends to render the model excessively sensitive to inputs, often resulting in misinterpretation non-compliance with explicit instruction requirements (Webson & Pavlick, 2022] Mishra et al. 2022a) like "... generate less than five words"" and suboptimal learning efficiency (Ivison al. 2022 Deb et al. (022). (ii) Scaling Input-Free Tasks - collecting task instructions that can be answered without additional inputs, e.g., "give the name of the highest mountain in the world", and expanding the (instruction, output) training pairs (Wang et al. 2023b Xu et al. 2023a). Despite the intuitive alignment with
objectives, covering a wide range of diverse tasks and aiding in daily queries, the input-free nature
All the code and data available our project page: https://renzelluu itt.u...fo//ffi///////////

---

Published as a conference paper at ICLR 2024
Analyw the sondiment ofe gles coxt [.c]
Nappon thenkpal 1-11
Labert rally eaaly-enja enjy shot mosid [-.]
is roat mraserrracc [.J
The food not bud
Inpurt
[-.]
What's the speelle's doughter good et?
Datncling and wnrtbligg
my doughter
Output
Poetbre
Noutral
M% chlooge Nitterueee jawllng wholl
out
Scaling-Lnpor
stslogue (..)
Input
Task
Paan mea 12AD
Vill witt FRIDCIE eeall.
Translote the plint
belarr, wa meiee Toenter Tanoro I.
sentance Jo00 Gehmin [--]
Outpurt
4 strips of bacon
"Soperiiaa"
Thank you for reacting out to ss
Task Instauction
Output
(b), Stalinn Input-Free Tasks
(c). Scaling Tasks pur Input (Ours)
Figure 1: Three different paradigms for designing instruction-followinn datasets.
of the Scaling Input-Free Tasks paradigm makes the resulting LLMs less effective in handling traditional NLP tasks where instructions are accompanied by supplementary inputs.
In this study, we introduce novel approach to curate instruction followoig datasets, termed Scaling Tasks per Input, as illustrated in Figure [c). Instead of amplifying the task's 's input-output set in Scaling-Inputs or enlarging input-free tasks in Scaling Input-Free Tasks, the Scaline Tasks per Input paradigm introduces task diversification for each input. Consequently. models trained adapt outputs based specific instructions related to the input, thus enhancing the mnstruction-follooing capacity of LLMs.
Two challenges in implementing Scaling Tasks per Input: (C1) Designing diverse tasks for the same input and (C2) Balancing classification and generation categories in the resulting dataset. Addressing C1. propose two strategies to automatically synthesize varied tasks for each input. Instruction Brainstorm: Enhancing task diversity and instruction-inpu relevance using an input-facet- oriented instruction generation approach. Instead of relying solely on existing human instructions as "demonstrations" for task brainstorming (Wang et al. 2023b] Honovich al., 2022). we employ an imput-facet-orietee procedure. LLMs identify diverse textual facets of the input, considering each facet "hint" to generate related instructions. 2) Instruction Rematching: Reusing high-quality task instructions from human-crafted datasets and determining their relevance to given input. Diverse instructions for each imput are collected, and LLMs annotate the output for each (instruction, input) pair, followed by filtering of raw results. Regarding C2, recognizing LLMs' inclination to produce more generation tasks than classification tasks, we propose straightforward yet effective method to expand classification tasks. Our resulting dataset named + MUFFIN (Multi-Faceted Instruction), the first instruction-folloving dataset aligning with Scaling Tasks per Input.
In experiments, train LLMs of 3B& 11B and evaluate them four widely-used zero-shot benchmarks, i.c., SuperNI-Test (Wang et al. 2022), MMLU (Hendrycks et al. (2021a), TO-Eval (Sanh) al. 2022), and BBH (Suzgun al. 2022 Automatic evaluations reveal that LLMS trained on our MUFFIN exhibit superior performance on three of the four benchmarks compared to models trained with various prior datasets from both
and Scaling Input-Free Tasks paradigms. Comprehensive human evaluation and in-depth analyses further affirm the effectiveness of our Scaling ing Tasks per Input paradigm and MUFFIN in improving the instruction-folloiin capacities of L.L.Ms.
To sum up, our main contributions are three-fold:
We propose a brand-new paradigm for crafting instruction followiwn datasets Scaling Tasks per Input. It enforces the model to follow various instructions in an imput-controlling way.
We develop a novel instruction synthesis framework that takes into account inputs' various facets. It increases task diversity (per input) and instruction-input relevance simultaneously. We publicly release our dataset MUFFIN and data construction code to benefit future research.
Our MUFFIN enhances instruction following capabilities of LLMs across scales, outperforming prior datasets constructed using various paradigms standard evaluation benchmarks.
instruction-following
2

---

Published as a conference paper at ICLR 2024
2 RELATED WORK
Initially, LLMs excel at following prompts often concise cloze questions (Radford al. 2019
Schick & Schüizz 2021a b). By converting original task inputs into prompts, LLMs can achieve
zero-shot performance across various tasks without parameter updates (Liu et al. 2023 However, the effectiveness of prompts heavily relies on LLMs. potentially leading to over-reliance on large models and vulnerability in robustness (Bach et al. [022] Khashabi al. 2022 Gu al. 2022). To create improved instruction-followe
L.Ms, prior research turned to Instruction Tuning training LLMs on extensive upstream tasks with instructions and then generalizing to downstream unseen
tasks using new instructions (Sanh et al. 2022) Wei et al. 2022,Ouyang Ouyang et al. [022) [Chung et al.)
2022 Yin et al. 2022, Longpre et al. 2023). Consequently, collecting diverse and high-quality upstream training datasets becomes pivotal step in successful instruction tuning (Wang al. 2023a
Lou et al. 2023). We then discuss two main data curation approaches.
Human Annotated Data. The traditional instruction data creation relics on extensive human annotations (Xu et al., 2022b Srriaastaaa et al. 2022; Conover et al. 2023). For instance, PUBLIC PoOL OF PROMPTS (P3) (Sanh et al., 2022) and FI.AN (Wei al. (022) curated multi-task datasets with various task categories, leveraging human expertise to design prompt templates. |Wang et al. (2022) introduced SUPER NATURAL INSTRUCTIONS (SUPERNI) by collecting 1.6k NLP tasks from the NATURAL INSTRUCTIONS dataset (Mishra et al. 2022b), employing 88 experts to brainstorm novel tasks. Despite the quality, human annotation is effort-intensive and time-consuming, especially for devising diverse and complex textual tasks.
LLM Synthetic Data. Recent research favors leveraging the creative capabilities of LLMs, like
ChatGPT (OpenAl, 2022) or GPT-4 OpenAl 2023), over human input for creating instruction-
following datasets Xu al. 2023b Geng et al. 2023 Chiang et al. 2023 Kim al. 2023
Ding et al. 2023). Approaches such as SELF-INSTRUCT (Wang et al. 2023b) and UNNATURA
INSTRUCT (Honovich al. 2022) utilized -nntat- instructions as demonstrations to guide LLMs in devising novel tasks and increasing task diversity. ALPACA (Taori et al. (0023) and ALPACA-
GPT4 (Peng et al. 2023) utilized more powerful LLMs to enhance data quality. Another line of research uses free-form texts "seeds" to assist LLMs in brainstorming task instructions (Wu et al.)
2023). For instance, LONGFORM (Köksal et al. 2023 created instructions based on lengthy output text improve long text generation abilities of LLMs; WIZARDL.M (Xu al., 2023a) employed an instruction evolution paradigm to increase seed instruction complexity; DYNOSAUR (Yin et al., 2023) repurposed existing input-output pairs in NLP datasets to stimulate new instructions and reduce annotation costs. In contrast, our instruction- inatruction-bbainstoroe pipeline uses the task input as the "seed", with all new instructions aimed at processing this input into different outputs.
MUFFIN CURATION
This section elaborates on the key modules for constructing MUFFIN: input collection from diverse
sources (S p. addressing C1 by generating input-oriented diverse tasks ($ B.2). followed by output annotation and filtering steps, and tackling C2 by controlling the classification
cnfficatioongenerationn generation balance
3.3).
The complete pipeline is illustrated in Figure 2 We show the API usage and costs in Table g
3.1 INPUT COLLECTION
We sample inputs from two distinct sources ensure diversity (sampling details in Appendix A):
SUPERNI (Wang et al., 2022) a human-annotated dataset encompassing 1,600+ NLP tasks across diverse categories, sourced from existing benchmarks or created by human experts, implying remarkable imput diversity of SUPERNI. Therefore, we randomly select inputs from the training tasks of SUPERNI as input text source (only inputs, no outputs are sampled at this stage).
Dolma Solldini et al., 2023) is a vast corpus used for pretraining, covering free-form texts from four domains: web content, academic publications, code, and encyclopedic materials [ Texts randomly sampled based on domain sizes, prioritizing domains with fewer texts to ensure diversity.
We exclude book domain texts due their length and unsuitability as task inputs.
3

---

Published as a conference paper at ICLR 2024
SupprrNlt AT
Intraation Breinsamn
Ai2
ideatlyy product
This ULO product recther [-
mostlence in rencw!
MUFFIN
Deles
gisltion emation)
Clasification Expansics
Chanoht
Summmrie WW reat
Oinatiy (-]
Instraction Fllterngs
Instruction Brmutchies
Truth Toha fin requilrr
Cissstte the sontiment IND pooitied segethec [-]
Chimnnr Ammoiaaionn
aog Seatt
Write shory story plack
AS prodbct
Mennlyy CNty gosetne EVOUTS Wiee glans
Lown 2-
Figure 2: Data construction pipeline of MUFFIN.
3.2 INSTRUCTION COLLECTION
After collecting diverse set of task inputs, we design two strategies to generate task instructions: Instruction Brainstorm LLMs generate instructions based on the input, and 2) Instruction Rematching reusing existing human-crafted instructions on the input, evaluated by LLMs.
Instruction Brainstorm Based on Inputs' Facets. A common approach for generating task instruc- tions is to use existing human- written instructions few-shot demonstrations to guide LLMs, as seen in prior works (Wang al.) 2023b [Honvvich et al.) 2022] Taori et al.) 2023). However, our initial experiments revealed challenges in directly generating suitable instructions for the input using this conventional method - LLMs tended to produce unrelated tasks, emphasizing the importance of using specific facets (attributes) of the input to stimulate corresponding tasks.
To this end, in Figure 2, we introduce a two-step facet-based instruction brainstorming method. Firstly, prompt ChatGPT to recognize the textual facets of the given input, aiming to identify as many facets as possible using an enumeration prompt (as shown in Table Secondly, with each recognized facet hint, instruct ChatGPT to brainstorm task instructions for the input. Similar to prior works (Wang et al.] [2022] Honovich et al.) (022), we randomly sample three instructions from SUPERNI's s training
demonstrations increase the validity of generated tasks (prompt details in Table 3 and Table 4)
Instruction Rematching. Our data collection pipeline aims associate input with diverse task instructions. Besides direct instruction synthesis by LLMs, another approach is to gather suitable instructions from existing human-annotater sets. Illustrated in Figure we extract human- written instructions from SUPERNI's 's training set and employ LLMs (specifically GPT-4) for binary classification. The classification involves determining if the instruction can align with the input to form a valid task ("Yes" or "No"). Subsequently, we collect all matched instructions predicted by the LLMs for given input. See Appendix B for more technical details of instruction rematching.
After obtaining (instruction, input) pairs, we use ChatGPT to annotate outputs, or mark them as "None" for instances that ChatGPT deems unanswerable (due to invalid instructions or mismatched input-instruction pairs). Table shows the prompt. Next, employ the following post-processing steps to filter out noisy instances: 1) Overlapped Tasks - for each input, remove its task instructions whose ROUGE-L similarity with any existing instructions is higher empirical threshold (set 0.7 in this work); 2) Non-answerable Instances eliminate instances with outputs marked as "None".
3.3 CLASSIFICATION EXPANSION
Classification tasks, characterized by small fixed output spaces, are prevalent in real-world applica- tions (Wang al.) 2019b Sanh et al. 2022 Xu et al. 2022a). However, LLMs tend to generate significantly more generation instructions than classification ones (Yin et al. 2023), resulting in
lower-than-expeeted generalization capabilities classification tasks. Given the essence of classifi-
4

---

Published as a conference paper at ICLR 2024
cation tasks being the selection of the most likely correct answer, we propose straightforward and effective approach to expand the classification mlusifiiaation-oiientte orienteedddd task instructions.
For given (instruction, imput, output) instance: 1) we first let ChatGPT generate additional "wrong outputs" that should be suboptimal compared to the correct output (see Table 7. 2) We combine the correct output with these wrong outputs to form the entire output space for the task, presented as options A, B, C, etc., in a randomly shuffled order. Additionally, a sentence in the instruction specifies that the answer should be chosen from these options (A, B, C). This transformation helps convert original generation tasks into a classification formulation. To prevent answer letter bias, we use random alphabet, numbers, or special symbols (e.g.. ",, $, #") as option letters.
We exclusively apply classification expansion to brainstormed instructions. as they are deficient in classification tasks. Simultancously, we exclude the generation tasks with outputs' lengths exceeding threshold (set at 100 words) due to difficulty in converting them into a classification paradigm. Ultimately, when presented with an original instance and its expanded classification version, we randomly select one (with equal probability) to be included in our MUFFIN, ensuring task balance.
4 DATA ANALYSES
Statistics. Table lists the detailed statistics the 68K (instruction, input, output) instances in MUFFIN. It is worth mentioning that some inputs will share the same instructions because of "instruction rematching" mechanism. The length distribution of instructions, inputs, and outputs can be found in Figure4iing Appendix.
100%
94.0
82.0
80%
75.5
72.5
A1: Instruction deseribes valid task.
Quality. To assess data quality, we randomly se- lected 200 instances from our dataset (200 random inputs, random instruction per input). Two NLP graduate students were involved in the evaluation. Following Wang et al.] (2023b). each annotator answered three questions for each instance: 1) de- termining if the instruction describes a valid task (A1) when only the instruction is provided, 2) as- sessing if the instruction appropriately matches the input when only the instruction-inpe -nnput pair is presented (A2), and 3) evaluating if the output correctly responds to the instruction and input (A3). We also recorded instances where all three fields were correct (A4). Figure shows the correct ratio for each question
60%
A2: Input matches instruction.
A3: Ousput matches instruction and imput.
A4: All fields are valid.
40%
A1
A2
A3
A4
Figure Human evaluation on the data quality. Both valid and invalid instances can be found in Table [3] A4 indicates the joint set of successful in A1, A2, and A3.
Our dataset achieved 72.5% %
correct ratio on all three fields, significantly outdoing the 54% reported by Wang et al.) (20235).
Diversity. To analyze the task diversity of MUF- FIN. we follow previous works (Wang et al. 2023b Peng et al. 2023 Yin al. (2033) using Berkeley Neural Parser (Kitaev & Klein 2018)
Table 1: Statistics of MUFFIN.
statistic
inputs
1,463
* of inputs (from Superh(I)
953
2 inputs (frem: Dolma)
510
to phrase the task instructions our dataset. We
instructions
56,93
of instractions "rematching" tfrom SuperXI)
plot the top 20 most common root verbs of instruc- tions along with their top direct noun objects. As shown in Figure 5 in the Appendix, our MUF- FIN shows good instruction diversity, where most instructions focus creative generation tasks.
# of instructions (frems brainstorm)
33,720
# of instructions ifrom classifiation expansion)
22.059
of instruttons per input
46.48
of inguls per instruction
20.27
of (instraction, input, octuat instiness
68.014
ave. inpui length (in wards)
119.26
words)
$4.74
ave. ouipui langth words)
71.32
5 EXPERIMENTAL SETUP
We conduct a comprehensive evaluation that covers benchmarks and baselines with various paradigms. Here, we use Mealing-Inpul and SScaling Input-Free Tasks to denote the different prior
'The two annotators had similar correct ratios, with an average agreement of 83.3% for the three questions. WWe only report the "# of inputs per instruction" of rematching instruction data. As for brainstorming instruction, this number is almost equal to 1, because almost all the instructions generated by L.I.Ms different.
5

---

Published as a conference paper at ICLR 2024
paradigms introduced in In addition, use Hybrid to represent datasets constructed by con- verting Sealing-Inputee into Scaling Input-Free Tasks (i.e., concatenating original instruction and imput to form an extended instruction without explicit input). Such datasets essentially belong to Scal Inputs, as they keep the same instruction while varying the inputs, but it is converted into an input-free style to cater to the models tuned on Scaling Input-Free Tasks.
Evaluation Benchmarks. In the zero-shot setting, we report on the following four benchmarks that are widely used in prior work. Please refer to Appendix foor prompt template details.
. SuperNI-Test (Wang et al. 2022) Icaling The official test set of SUPERNI with 119 distinct NLP tasks spanning both classification and generation. Following previous works (Wang et al. 2022 Lou & Yin 2023), use the first 100 instances of each test task. To ensure the fully zero- shot generalization setting, we only utilize the task definition (instruction) of SUPERNI without any demonstrations. We use greedy decoding for all models tested this benchmark. Official evaluation metries: "Exact-Match" for classification, "ROUGE-L" for generation and overall performance.
MMLU (Hendrycks et al. 2021b) Scaling Input-Free Tasks The Massive Multitask Lan- guage Understanding dataset (MMLU) covers extensive questions from 57 subjects with various difficulty levels. In total, there are 14,042 test instances. Each instruction in MMLU consists of subject-related question along with four answer choices (i.e., "... answer wi th (A), (B), (C) or (D)"). We adapt two evaluation settings: 1) adopting greedy decoding generation and using Exact-Matee score as metric, which is the most conventional choice (Wang et al.. 2023a Xu et al. 2023at; 2) utilizing rank classification for decoding with Accuracy score, which is also a popular setting for the classification tasks and focuses more on evaluating the models' knowledge without suffering from the mismatching problem of L.LMs' generation. As there are only (instruction, output) pairs in MMLU, we categorize it as Scaling Input-Free Tasks paradigm.
TO-Eval (Sanh et al., 2022) Hybrid We utilize the held-out evaluation set of TO (we refer to it as TO-Eval) one of the zero-shot benchmarks. The original TO-Eval contains 23 different task categories, while some of these tasks are leaked SUPERNI's training set. Therefore, we follow Honoviihh et al. (2022) keeping 6 task types for fair comparison, namely ANLI R1-R3, CB, COPA, and RTE, leading to 82 corresponding evaluation datasets. Due to the extreme instance number imbalance, per dataset to ensure the reproducibility. Similar to the MMLU, the tasks in T0-Eval are related to multi-choice classification. Hence, we follow previous works (Sanh al., 2022 Honovich et al. 2022) utilizing Exact-Match and Rank Classification Accuracy as metrics.
follow a similar procedure as Wang al. (022) using the first 100 instances
BBH (Suzgun et al. 2022) Hybrid The "hard" subset of BIG-Bench (Srivastava al. 2022) BBH consists of 23 challenging tasks with a total of 6,51 I instances. Similarly, we remove all the demonstrations and of-ouuh prompts BBH to ensure the zero-shot setting. We use greedy decoding and report the Exact-Match scores for all the models in our experiments. which is also the official setting adopted by variety of previous works (Wang et al. 2023a Mukherjee et al., 2023).
(UNNATURAL IN- STRUCT and DYNOSAUR) or Scaling Input-Free Tasks (DOLLY, LONGFORM, ALPACA, ALPACA-GPT4, WIZARDLM, and SELF-INSTRUCT). All of these datasets are crafted using ChaiGPT or GPT-4 (except DOLLY), which can be directly compared with our MUFFIN (di- rect comparison). As for the manually-created dataset, PERNI Secaling-Inpute. ), which can be regarded as an upper bound of all these synthetic datasets (indirect comparison). Note that, the original training set of SUPERNI contains 757 tasks (excluding those non-English tasks and tasks that are leaked from the test set), while we follow Yin et al. (2023) using 681 tasks (90%) for training and considering the remaining 76 tasks (10%) as the validation set. In addition to the aforemennioned baselines, we also report the performances of existing systems with larger numbers of parameter or tuning-data sizes, such as Flan-T5 (Wei et al., 2022) and TO (Sanh et al.) 2022). See Appendix H for more particular introductions and implementation details toward these baselines.
Baselines. We categorized previous works as Scaling-
report the performance of Su-
Implementation Details. We fine-tune models with various architectures on MUFFIN, including encoder-decoder T5-LM (Raffel al. 2020) and decoder-only Llama (Touvron et al. 2023a). We provide more implementation details (e.g.,
in Appendix G
CChossing the answer option with the highest likelihekoood the final prediction of the model.

---

Published as a conference paper at ICLR 2024
Table Results by T5-3B (second block) and T5-11B (third block). Each block contains indirect comparison (i.e., trained on in nn-distribution SUPERNT) and direct comparison (i.e., LLM-generater datasets). Scores that are indicated with 1 and are adopted from Wang et al. (2023a) and Mukherjee) et al. (203). The first block in this table reports some larger models or models with more tuning data, as reference. Since Flan-T3 trained on some evaluation benchmarks, we don't t report these scores (marked "-"). The best scores under direct and indirect comparison settings are in bold and underlined, respectively. Those previous baselines/evaluation benchmarks are marked with different colors represent their paradigms (see 5).
All scores here averaged over three runs.
SuperXI-Test
MMLU
To-Eval
Models
Size
ROUGE-L (GEN)
ROUGE-L (overall)
Average
(CLS)
Larger Models Vanilla Modelr More Trainng Data (for reference)
GPT-4
64.51
59.36
62.96
82.401
70.95
67.40
Existing Systems
CharGPP
67.9pf
50.73
48.00'
Flan-T5 (1IB)
14M
49.97
45.97
40.92
Flan 1eaa-T5-i (3B)
14M
45.52
45.07
39.70
(11B)
62.22
(LIB)
50K
24.08
41.15
32.85
42.87
40.08
57.52
60.00
29.04
40.95
TO (385)
50K
18.95
36.66
26.84
31,51
25.32
46.17
46.63
22.76
31.86
Vanilla T5 (3B)
0.00
21.84
10.28
24.12
0.00
37.45
0.38
0.00
11.76
Human Anmolated Data (indirect compariion)
Suparni-TTrin
68k
35.46
48.01
43.25
38.42
36.97
49.65
48.73
19.60
40.01
Generated Data (ilivet comprison
Dolly
15k
0.49
34.32
14.52
23.05
0.00
39.84
6.78
5.71
15.59
LongForm
23k
0.00
33.58
11.29
23.07
0.00
39.68
0.62
3.84
14.01
Alpuca
52k
20.43
46.08
35,25
28.55
8.02
43.26
20.52
11.53
26.71
Alpaca-
WizardL.. M
68k
5.34
41.09
20.81
25.55
0.00
40.55
5.87
5.16
18.05
Self-Inst.
29.59
43.70
36.87
27.11
23.55
41.74
38.57
20.53
32.71
32.65
34
32.02
Dynosaur
Muffin (Ours)
33.84
49.52
42.63
36.27
29.75
46.38
44.45
14.25
37.13
Human Anmotated Data (indirect comparison)
SuperNI-Trai -Taina
68k
41.13
50.05
47.76
54.45
54.37
56.89
54.23
29.80
48.59
comprison)
Dally
15k
2.71
37.12
17.81
22.99
0.06
49.17
23.96
10.18
20.50
LongForm
T5-11B
Alpuca
52k
25.36
47.74
39.62
30.17
8.10
$4.48
34.90
9.28
30.21
MPpacs-PP44
52k
13:65
43.19
31,46
25.58
0.00
19.94
34.79
7.94
25.82
JLM
Self Enst.
82k
28.88
44.88
36.53
28.22
32.45
48.61
41.46
31.39
36.55
Unnatural Inst.
68k
41.11
47,46
45.54
34.38
22.39
43.40
41.91
12.84
36.13
Dynosaur
66k
42.02
47.53
46.42
27.60
24.96
42.85
43.39
9.22
35.50
Muffin (Ours)
68k
40.20
50.69
48.32
41.95
41.83
55.38
57.74
20.53
44.5%
EXPERIMENTAL RESULTS
We report automatic evaluation in
6.1 human evaluation in 6.2 and in-depth analyses in
6.3
6.1 AUTOMATIC EVALUATION
Table shows the main results of fine-tuning T5-3B and T5-1 1B on different datasets. Additionally, we add the results based on Llama2 in Table 15 to further enhance our conclusion.
Compared with the previous L.M--genrated datasets, such as INST-UCT UNNATURAL. IN- STRUCT, and DYNOSAUR, the models tuned on our MUFFIN consistently achieve better performance across 3 out of benchmarks, under various metrics. Besides the high quality and diversity (as we have discussed in 4.. we anticipate that this performance superiority is also owing the Scal ing Tasks per Input paradigm of our MUFFIN, where the models are trained to focus on the instruc- tions and gain stronger instruction-fol followwng capacities. We also find that the larger model benefits more from tuning on MUFFIN (4.42 average performance improvement of T5-3B = 8.03 average performance improvement of 15-11B, compared with the strongest baseline).
7

---

Published as a conference paper at ICLR 2024
It is noteworthy that nearly all datasets using Scaling Input-Free Tasks yield models with limited generalization capabilities towards SUPERNI Scaling Input-Free Tasks in addressing tasks necessitating supplementary inputs. Furthermore, noting that T0-Eval and BBH belong to Hybrid - all the instances from these two benchmarks were initially in line with Sealing-Inputs but were transformed into Scaling Input-Fres Tasks to cater to those Scaling Input-Free Tasks datasets, In contrast, our MUFFIN still leads to strong generalization performances, especially on TO-Eval. Considering the tasks with additional input contexts widely exist in real-world applications (e.g., reading comprehension tasks), these results point out the drawbacks in the current Scaling Input-Free Tasks paradigm, even if the evaluation tasks have been converted into the paradigm they are familiar with. When considering the comparison with SUPERNI, MUFFIN can still get a comparable or even better performance under some metries, across different models and model sizes, demonstrating the highly promising instroction-follovinng capacity of the models tuned on MUFFIN.
Inputs ), suggesting challenges for
6.2 HUMAN EVALUATION
In this section, we further employ humans to evaluate the quality of different model's responses.
Acceptance Ratios. We randomly sample 200 instances from each evaluation benchmark and use various instruction-ture tunned models to generate the outputs. Then we ask the human evaluator to rate these responses and report the ratio of correct responses from different models. Human evaluation details can be found in Appendix K Table 16 shows the results. Compared with the strongest results among the 8 baselines, MUFFIN mostly results in better human acceptance ratios with large margins (except for the tiny lower result on TO- TO-Eval). According to our further analysis, we find that the other baselines often misunderstand the task objective, leading meaningless outputs (e.g., copying some pieces of input text). While MUFFIN is more likely to produce useful outputs to solve the task. We provide some representative cases of different systems' outputs in Table 14
Pair-wise Comparison. We compare the MUFFIN with the strongest baselines by using a pair- wise evaluation scheme, including SELF-INSTRUCT (Scaling Input-Free Tasks UNNATURAL INSTRUCT (Sccaling-Inuuts] ) and the human-annotated SUPERNI ( (Scaling-Inputs, ). Simi- larly, we randomly sample 200 test instances per benehmark and let different models generate the outputs. Table 17 shows the pair-wise comparison results between MUFFIN and other three base- lines. Compared with the L.M--gerrated datasets (SHLF-INSTRUCT and UNNATURAL INSTRUCT), our MUFFIN consistently gains higher human preferences across four benchmarks, aligning with the previous conclusion. Notebly, owing to the diverse tasks and our novel Scaling Tasks per Input paradigm, MUFFIN can even surpass the high-quality human-crafted SUPERNI on three out of four benchmarks.
6.3 ANALYSES
In addition to the automatic & human evaluations the above two subsections, we further conduct some analyses to answer the remaining questions: (Q1) How much does each technical module (i.e., instruction brainstorm, rematching, classification expansion) contribute to the overall performance? (Q2) Does method's superior performance stem from latent task similarities even leaking towards the evaluation tasks? (Qx) Does the data size influence the conclusion? (Q.) Does mixing MUFFIN with human-crafted instruction instances boost the performance?
Ablation study answer 21- We fix the data size to 11.5k (because this is the maximum size of rematching instructions; it leads to a fair comparison) and use the combinations of different instruction collecting methods to train the T5-3B. Following Honovich et al. (2022), we report the performances on the validation set of SUPERNI instead of the test set, to avoid cherry-picking. As shown in Table [8] each technical module contributes to overall generalization abilities. Specifically, rematched instructions seem to perform better on classification tasks, while brainstormed instructions demonstrate higher generation performance. Though instruction rematching can gather high- instructions written by humans, there is still noise in rematching, and it potentially suffers from imbalanced task categories (e.g., some common tasks are frequently matched with more inputs). Brainstormed instructions easier to collect, but they are in short supply for classification tasks.
8

---

Published as a conference paper at ICLR 2024
Notably, the proposed classification expansion significantly improves LLMs' generalization unseen classification tasks while maintaining strong performance on generation tasks.
Resolving the task leaking concern in 22. It is possible for LLMs to generate the tasks leaking the evaluation benchmarks, violating the zero-shot setting. Therefore, we randomly sample 200 task instructions from each benchmark and use Sentence Transformers (Reimers & Gurevych, 2019 to convert all the training instructions and the sampled test instructions into embeddings. We report the average cosine similarity between each training dataset and evaluation benchmark in Figure 6 We also follow the previous works (Yin et al. 2023) using ChatGPT to estimate the task overlap in Table Renze. First, we can find that all the Scaling-Inputs datasets, namely SUPERNI, DYNOSAUR and UNNATURAL INSTRUCT, show relatively high instruction semantic similarities with SuperNI-Test (SScaling-Inpal instructions sourced from SUPERNI in our MUFFIN, MUFFIN still maintain a relatively low similarity with SuperNI-Test, owing to the diverse task distribution as discussed in $ 4 In a word, our MUFFIN demonstrates relatively low instruction similarities across all four benchmarks, proving the validity of our previous zero-shot cnatruction-oolooiinn superiority.
as well). Meanwhile, though there are rematched
Scaling data size to answer 23. Regarding the cost efficiency of the previous works, it's usually convenient to collect extensive instruction-followwe following instances with previous paradigms (Yin et al.] 2023 Wu et al. 2023). Therefore, is it worthwhile to try the proposed paradigm rather than simply gathering more instances from the other paradigms? ? To answer this question, for all those baseline datasets from the "direct comparison" in Table 2 we randomly sample subsets from them and train T5-3B on the subsets to show the performance trends (10%, 30%. 50%, 80%, 100%). As shown in MUFFIN exceeds the baselines by noteworthy margin (average scores on four evaluation
Figure 9 benchmarks). Other baselines may only be comparable to our data results when they continue to be scaled to several times the size of our data. More importantly, the performances of some datasets even decrease after scaling to a larger size (perhaps due to the noise in these LLM-synthetic datasets), such as SELF-INSTRUS than simply collecting more data from the other paradigms. We also conduct the scaling comparison with SUPERNI in Appendix 1.1
and DYNOSAUR. Therefore, we conjecture that our paradigm is more efficient
Mixing MUFFIN with SUPERNI to answer Q4. As demonstrated by the previous works, it's possible further improve the generalization performance after mixing the LLM-hyntettic and human-annotated instruction instances (Yin et al. 2023). Thus, we mix MUFFIN with the training set of SUPERNI, w.r.t. various proportions, where the proportion means how many instances are from MUFFIN. Then, train T5-3B on these mixtures and report the average performances on the four benchmarks. As illustrated in Figure S interestingly, after mixing MUFFIN and the human- annotated SUPERNI, the performance drops even worse than only using MUFFIN for training. We conjecture possible reasons for this phenomenon: the dataset paradigm does affect the learning efficiency different dataset paradigms do obviously have various impacts the model's performance. Therefore, combining two paradigms can even hurt the model's eesttrctiionfollowiie the paradigm is critical. Meanwhile, the effect of dataset paradigms is even greater than that of data quality (i.e., adding a small proportion of high-quality data from other paradigms even harms MUFFIN's ' performance), further implying the proposed paradigm's 's effectiveness.
performance, meaning
CONCLUSION
This work proposes a novel scheme for curating instruction-folloving datasets, namely Scaling Tasks per Input. Unlike previous works that either adopt Scaling-Inputs Input-Free Tasks paradigm, we diversify the tasks for each input text. The variance in task instructions leading different outputs can ideally enhance the foolloii capacity of LMs. Accordingly, we propose MUFFIN - the first dataset aligning with Scaling Tasks per Input. Our comprehensive experiments spanning four challenging zero-shot benchmarks demonstrate the effectiveness of MUFFIN, where MUFFIN consistently achieves better instruction-followee capacity than the extensive baselines with previous paradigms. In-depth human evaluation and analyses further prove the superiority of MUFFIN and our Scaling Tasks per Input paradigm.
Scaling

---

Published as a conference paper at ICLR 2024
REFERENCES
Stephen Bach, Victor Sanh, Zheng Xin Yong, Albert Webson, Colin Raffel, Nihal V. Nayak, Abheesht Sharma, Taewoon Kim, M Saiful Bari, Thibault Fevry, Zaid Alyafeai, Manan Dey, Andrea Santilli, Zhiqing Sun, Srulik Ben-david, Canwen Xu, Gunjan Chhablani, Han Wang, Jason Fries, Maged Al-shaibani, Shanya Sharma, Urmish Thakker, Khalid Almubarak, Xiangru Tang, Dragomir Radev, Mike Tian-jian Jiang, and Alexander Rush. PromptSource: An integrated development environment and repository for natural language prompts. In Proceedings of the 60th Annual Meeting of the Association for Compulational Linguistics: System Demonstrutions, 2022.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter. Christopher Hesse, Mark Chen. Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng. Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, lon Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* * chatgpt quality, 2023.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling CrincctionFFinetnne Language Models. ArXiv preprint, 2022.
Mike Conover, Matt Hayes, Matt Mathur, Xiangrui Meng, Jianwei Xie, Jun Wan, Ali Ghodsi, Patrick Wendell, and Patrick Zaharia. Hello dolly: Democratizing the magic of chatgpt with open models., 2023.
Budhaditya Deb, Ahmed Hassan Awadallah, and Guoqing Zheng. Boosting natural language generation from instructions with meta-learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022.
Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. Enhancing chat language models by scaling high-quality instructional conversations. ArXiv preprint, 2023.
Xinyang Geng, Arnav Gudibande, Hao Liu, Eric Wallace, Pieter Abbeel, Sergey Levine, and Dawn Song. Koala: A dialogue model for academic research. Blog post, 2023.
Jiasheng Gu, Hanzi Xu, Liangyu Nie, and Wenpeng Yin. Robustness of Learning from Task Instructions. ArXiv preprint, 2022.
Xiaojie Gu, Renze Lou, Lin Sun, and Shangxin Li. Page: A position-aware graph-based model for emotion cause entailment conversation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEBE, 2023.
Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. Aligning Al with shared human values. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021, 2021a.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021, 2021b.
Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor. ArXiv preprint, 2022.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In The Tenth International Conference Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, 2022.
10

---

Published as a conference paper at ICLR 2024
Hamish Ivison, Akshita Bhagia, Yizhong Wang, Hannaneh Hajishirzi, and Matthew Peters. HINT: Hypernetwork Instruction Tuning for Efficient Zero-Shot Generalisation. ArXiv preprint, 2022.
Daniel Khashabi, Xinxi Lyu, Sewon Min, Lianhui Qin, Kyle Richardson, Sean Welleck, Hannaneh Hajishirzi, Tushar Khot, Ashish Sabharwal, Sameer Singh, and Yejin Choi. Prompt waywardness:
The curious case of discretized interpretation of continuous prompts. In Proceedings of the 2022 Conference of the North Americun Chapter of the Association for Computational Linguistics: Human Language Technologies, 2022.
Seungone Kim, Se June Joo, Doyoung Kim, Joel Jang, Seonghyeon Ye, Jamin Shin, and Minjoon Seo. The cot collection: Improving zero-shot and few-shot learning of language models via chain-of-thought fine-tuning. ArXiv preprint, 2023.
Nikita Kitaev and Dan Klein. Constituency parsing with a self-attentive encoder. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2018
Abdullatif Köksal, Timo Schick, Anna Korhonen, and Hinrich Schütze. Longform: Optimizing instruction tuning for long text generation with corpus extraction. ArXiv preprint, 2023.
Zekun Li, Baolin Peng. Pengcheng He, Michel Galley, Jianfeng Gao, and Xifeng Yan. Guiding large language models via directional stimulus prompting. ArXiv preprint, 2023.
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre- train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing. ACM Computing Surveys, 2023.
Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al. The Flan Collection: Designing Data and Methods for Effective Instruction Tuning. ArXiv preprint, 2023.
Renze Lou and Wenpeng Yin. Forget demonstrations, focus on learning from textual instructions. ArXiv preprint. 2023.
Renze Lou, Kai Zhang, and Wenpeng Yin. Is prompt all you need? no. a comprehensive and broader view of instruction learning. ArXiv preprint, 2023.
Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannaneh Hajishirzi. MetalCL: Learning to learn in context. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2022.
Swaroop Mishra, Daniel Khashabi, Chitta Baral, Yejin Choi, and Hannanch Hajishirzi. Reframing instructional prompts to GPTk's ' language. In Findings of the Association for Computational Linguistics: ACL 2022, 2022a.
Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. Cross-task generalization via natural language crowdsourcing instructions. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers), 2022b.
Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. Orca: Progressive learning from complex explanation traces of gpt-4. ArXiv preprint, 2023.
OpenAL. Chatgpt, 2022.
OpenAl. Gpt-4 technical report. ArXiv preprint, 2023.
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. ArXiv preprint, 2022.
Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with gpt-4, ArXiv preprint, 2023.
11

---

Published as a conference paper at ICLR 2024
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language Models are Unsupervised Multitask Learners. OpenAl blog. 2019.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with unified text-to-text transformer. J. Mach. Learn. Res., 2020.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang, SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016.
Nils Reimers and Tryna Gurevych. Sentence-BEiN BERT: Sentence embeddings using Siamese BERT- networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP- IJCNLP), 2019.
Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Amaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szezechla, Taewoon Kim, Gunjan Chhablani, Nihal V. Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang. Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Févry, Jason Alan Fries, Ryan Teehan, Teven Le Scao, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M. Rush. Multitask prompted training enables zero-shot task generalization. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, 2022.
Timo Schick and Hinrich Schütze. It's not just size that matters: Small language models are also few-shot learners. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, 2021a.
Timo Schick and Hinrich Schütze. Exploiting cloze-questions for few- text classification and natural language inference. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, 2021b.
Luca Soldaini, Kyle Lo, Rodney Kinney, Aakanksha Naik, Abhilasha Ravichander, Akshita Bhagia, Dirk Groeneveld, Dustin Schwenk, Ian Magnusson, and Khyathi Chandu. Dolma, 2023.
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. Beyond the Imitation Game: Quantifying and Extrapolating the Capabilities of Language Models. ArXiv preprint, 2022.
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sehastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoe V Le. Ed H Chi, Denny Zhou, , and Jason Wei. Challenging big-bench tasks and whether chain tf-tougghtt can solve them. ArXiv preprint, 2022.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An follloii llama model. 2023.
https://github.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. ArXiv preprint, 2023a.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprini arXiv:2307.00220 2023b.
Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Oner Levy, and Samuel R. Bowman. Superglue: A stickier benchmark for general-purposs language understanding systems. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, 2019a
12

---

Published as a conference paper at ICLR 2024
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In 7th International Conference Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 2019b.
Yizhong Wang. Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. Bench- marking Generalization via In-context Instructions on 1,600+ Language Tasks. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022.
Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A Smith, Iz Beltagy, et al. How far can camels go? exploring the state of instruction tuning on open resources. ArXiv preprint, 2023a.
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-Instruct: Aligning Language Model with Self Generated Instructions. In Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics (Volume
1: Long Papers), 2023b.
Albert Webson and Ellie Pavlick. Do prompt-based models really understand the meaning of their prompts? In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2022.
Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoe V. Le. Finetuned language models are zero-shot learners. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, 2022.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, al. HuggingFace's Transformers: State-of-the-Are Natural Language Processing. ArXiv preprint, 2019.
Minghao Wu, Abdul Waheed, Chiyu Zhang, Muhammad Abdul-Mageed and Alham Fikri Aji. Lamini-Im: diverse herd of distilled models from large-scale instructions. ArXiv preprint, 2023.
Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, and Yu Su. Adaptive chameleon or stubborn sloth: Unraveling the behavior of large language models in knowledge confficts. ArXiv preprint, 2023.
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. WizardIm: Empowering large language models to follow complex instructions. ArXiv preprint, 2023a.
Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley. Baize: An open-source chat model with parameter-efficient tuning on self-chat data. ArXiv preprint, 2023b.
Haike Xu, Zongyu Lin, Jing Zhou, Yanan Zheng, and Zhilin Yang. A universal discriminator for zero-shot generalization. ArXiv preprint, 2022a.
Zhiyang Xu, Ying Shen, and Lifu Huang. Multinstrruc: Improving multi- modal zero-shot learning
via instruction tuning. ArXiv preprint, 2022b.
Da Yin, Xiao Liu, Fan Yin, Ming Zhong, Hritik Bansal, Jiawei Han, and Kai- Wei Chang. Dynosaur: A dynamic growth paradigm for instruction- instruction-tuning data curation. ArXiv preprint. 2023.
Wenpeng Yin, Jia Li, and Caiming Xiong. ConTinTin: Continual learning from task instructions. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022.
Kai Zhang, Lingbo Mo. Wenhu Chen, Huan Sun, and Yu Su. Magicbrush: A manually annotated dataset for instruction-guided image editing, ArXiv preprint, 2023a.
Yue Zhang, Yafu Li, Leyang Cui, Deng Cai. Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. Siren's ' song in the ai ocean: A survey on hallucination in large language models. arXiv preprint arX0v2230..012199 2023b.
13

---

Published as a conference paper at ICLR 2024
Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. Lima: Less is more for alignment. ArXiv preprint, 2023.
14

---

Published as a conference paper at ICLR 2024
APPENDICES
Within this supplementary material, we claborate on the following aspects:
Appendix A
Details on Sampling Inputs and Instructions
Appendix
Details on Instruction Rematching
Appendix
Data Collection Prompt Templates and Hyper-parameters
Appendix
Data Statistics
Appendix
Data Diversity
Appendix
Data Collection Cost and API Usage
Appendix
Details of Tuning MUFFIN
Appendix
Details of Baselines
Appendix
Training and Evaluation Prompts
Appendix
Further Analyses
Appendix
Human Evaluation Details
A DETAILS ON SAMPLING INPUTS AND INSTRUCTIONS
As mentioned in previous sections, we use both inputs and instructions from SUPERNI to construct our MUFFIN, this section, we elaborate on the sampling details.
When sampling inputs ($ (.1)) since the construction of SUPERNI is based on the existing NLP benchmarks, different tasks in SUPERNI may source from the same benchmark. Therefore, order to promote the diversity of input texts, only sample inputs of tasks from unique sources (a total of 243 out of 756 tasks from SUPERNI's training set). For each resulting task, we randomly pick up 4 input texts. We conduct leading to a final of 953 inputs from SUPERNI have valid instruction and output annotations (see Table
instruction filtering ($ 3.2 that helps delete the non-answerable instructions,
As for the choices of the instructions used in "instruction rematching" ( 3.2 we use all the instructions in the 757 training tasks of SUPERNI. Since also use the task inputs from SUPERNI, those inputs and instructions that are matched in the origin SUPERNI will be automatically matched together in our dataset as well. However, due to using inputs from unique sources, there are certain amount of instructions that aren't matched with any inputs, resulting in a total of 574 instructions in our dataset sourced from SUPERNI NI (as shown in T..
B DETAILS ON INSTRUCTION REMATCHING
the LLM, namely GPT-4, help gather existing human- human-written instructions for each input. However, classifying such a large number of candidate (input, instruction) pairs, where the majority are negative, pretty costly and inefficient. To solve this issue, inspired by the "entailment check" of [ie et al. (2023); Gu et al. (2023). we first adopt free and small LM (SLM). Flan-T5 (Chung al. 2022 for quick and rough filtering to reduce the number of candidate pairs, then call the LLM to further filter on the kept pairs. Moreover, those pairs annotated by LLM can be further used for training the SLM, improving the subsequent rough filtering quality. This SI.M-LMM collaboration significantly improves the annotation efficiency. According to our small-scale trials, we found that this method can reduce the annotation cost to about 35% of the direct annotation of LLM.
As introduced in 3.2,
written
C DATA COLLECTION PROMPT TEMPLATES AND HYPER-PARAMETERS
Facet Recognition "attributes") in Table
We show the prompt used for generating facets (also can be understood as As for the decoding parameters of OpenAl API, we set the temperature as
httpps: https::/huggingfface -ttpaee//ooooooe//aaaaaaaaaaaeae
15

---

Published as a conference paper at ICLR 2024
Table 3: Prompt used in Facet (Antribute) Recognition 3.2
Roh Ingut: [Iaput Text)
HeH Instruction:
Given the above input, what kind textual attributes does have?
Requizementt::
1. Pleuse brainstorm as many textual attributes possible. If you think there are no more suituble attributes, end up with "None'.
creative. Any interesting perspectives are welcome!
3. Each aliribute
concisely summarize specific uspect of this input, such language, length, intent, Feel free to ignore the tedinus and specific content. Just focus on some general textual attributes!
Please prioritize your confident predictions.
Attribute
0.7, and nucleus sampling (top p) as 1.0 to promote the diversity of attributes. As for the maximum generation tokens, we fix 2,048.
Instruction Brainstorm Table 4 shows the prompts used in generating instructions. When running instruction brainstorm, we the temperature to 0.2 and set the nucleus sampling (top p) as p = 0.99. We set the maximum generation tokens as 3,200 to let LLMs brainstorm more complex instructions. We also set an additional presence penalty parameter as 1.99 to encourage L.LMs to produce more diverse tasks.
Instruction Rematching Table Sable&Jillssee illustrated the prompt used in instruction rematching, namely letting LLMs decide whether instruction is appropriate for the given input. We set the temperature as 0.2 and the top as 0,0 in this case to make the decision more deterministic. As for the maximum generation tokens, we set 2,048.
6hoows the prompt when generating the outputs for given (instruction, input) pair. To ensure the determinism of outputs, we set the temperature as and the top as 0.1 well. As for the maximum generation tokens, we set it to 1,024.
Output Annotation
Classification Expansion We illustrated the prompt classification expansion in Table When asking the LLMs to generate wrong answer candidates, we empirically the temperature as 0.3 and the top p as 0.99. As for the maximum generation tokens, we set it to 3,096.
Table 5: Prompt used in Instruction Rematching $ 3.2
You are expert in Natural Language Processing (NLP) tasks.
Given lask description and piece of text, your job determine whether this text can be used as input this task.
If the text satisfies the inpul expectation of this task, answer "Yes:; otherwise, if the text doesn't 't match the input description, answer 'No'.
### Text: (Input Text]
### Task Description: (Task Instruction)
*NA Your Answer:
16

---

Published as a conference paper at ICLR 2024
Table 4: Prompt used in Instruction Brainstorm 3.2 We actually use different prompts in our experiments: I) the first prompt (left table) uses the textual attribute as the hint and asks the LLMs to follow the hint to generate the corresponding instructions (Li al. 2023); in contrast, the second prompt (right table) asks the LLMs to develop task instructions that try to shift the given attribute. For example, if the attribute is something like "the input is lengthy". then we hope the LLMS can brainstorm some tasks like "summarize ize and simplify the given input". In our preliminary experiments, we found that the two different prompt strategies can result complementary task categories. Therefore, for each input, we collect the task instructions produced by using both prompts. The differences between these two prompts are highlighted.
RAN Imput:
HAR Input: [Input Tex)
[Irput Text)
Himis
Altributes
[Auribute]
Aunibute)
nnt Instraction:
Instruction:
Based on the above hint, what kind of textual tasks can you
Based on the above information. what kind of textual tasks cun you develop that shift the impul's altribute?
develop can be applied the inpul?
AAe Format Examples (Imitate their formats, ignore the contents):
AA# Format Examples (Imitate their formats, ignore the contents):
EExample Instruction 1}
I. (Example Instruction 1)
[Example Instruction
2. (Example Instruction 2
(Example Instruction
(Example Instmetions
nth In word. you should first describe what the input is, and what textual attribute it has. then claborate on the task intent,
Ait woed, you should first describe what the input is, and
what textual attribute has, then elaborate on the task intent, and finally exactly describe what kind of outpui expect mention any necessazy ourgut
and linally exaclly desenibe what kind output you expect
and mession any necessary ourpur constraints (e.g., fomatts options).
(e.g.. formass, options).
Aee Requirements:
Ada Requirements:
Braimmsomm
textual tasks possible. you think with 'Nore". guide your
I. Brainstorm
textual tasks possible. If think there with "None".
suitable
need
guide your
thinking. Your respocses shoold strictly be hased on this hint!
thinking.
3. Each task must be indivisible (one task, one intention).
3. Each task must be indivisible (one task. one insention).
Please priorilize your comfident predictions.
1. Picase prioritize your confident predictions.
lasks requiring additional contect,
Avaid requiring additional cantest,
be
casily snlvettinswerrd using only the inpur.
easily solvedlanswered using only input.
6. Your tasks should sufficiently describe how this ingur is expected to he mapped output text, i.e, elaborating the tasks in detail.
6.. Your tasks should su sufficiently desenibe how this input expected to be mapped to an outpus test, ie., elaborating the tasks detail.
not the answer!
final
Task I:
Task I:
17

---

Published as a conference paper at ICLR 2024
Table Prompt used in Outpur Annotation $ 3.2
Given a tusk instruction and an imput, please generate the output (answer) according to the requirements mentioned in the instruction. If you cannot answer the instruction base on the given information, simply generate 'None'.
#Mt! Instruction:
{TTsk instruction]
Input: (Input Text
### Output:
Table Prompt used in Classification Expansion $ 3.3 We ask the LLMs to generate more output candidates that are worse than the given output, which can be further used for reformulating the origin task into classification paradigm.
Given task, task input, and a corresponding correct output, generate more output candidates for this task.
Requirements:
1. The output candidates you generate should be worse than the given correct output,
e.g. wrong imperfect answers.
You are encouraged to generate some challenging output candidates, that are close to the correct output but not the most desired one (i.e., containing certain errors).
3. You encouraged to generate many output candidates as possible; If you think suitable output candidates, end up with 'None'.
there
### Task:
(Task Instruction]
### Input: [Input Text]
### Output: (Output Text]
Wrong Output 1:
18

---

Published as a conference paper at ICLR 2024
D DATA STATISTICS
We report the length distribution of the inputs, instructions, and outputs MUFFIN in Figure All the length is counted by words (we follow previous works using NLTK packag conduct the word tokenization). Our dataset covers a diverse length distribution (see the maximum and minimum length of each figure), however, the most common length is still highly focused in a certain range, as the average length reported in Table [
1230
1230
900
530
400
400
220
lazul Langt
150
1000
1250
1500
0
239
200
750
1080
1230
1800
226
200
750
Longdt
1000
1200
1366
Figure 4: Length distribution of inputs, instructions, and outputs in our MUFFIN.
E DATA DIVERSITY
Similar to Zhang et al. 2023a), we use Berkeley Neural Parser to phrase the verb-noun structure of the instructions in MUFFIN. Figure[5ssowws the most common verb-noun structures of our dataset, which implies a high task diversity of MUFFIN.
question
task
list
create
generate
suzumary
develop
task
write
rowrite
queitton
Rist
attdee
story
essoy
input
Figure 5: Instruction diversity illustration We plot the top 20 most prevalent root verbs (inner circle) and their top 4 direct nouns (outer circle) in the instructions of MUFFIN.
httppss//ppp.orr///ppjeetttttt https:///aarser..ttavv..o
19

---

Published as a conference paper at ICLR 2024
F DATA COLLECTION COST AND API USAGE
During our preliminary trials, we observed GPT-4's superiority over ChatGPT in instruction rematching ($ 3.2 which can produce much fewer false-positive pairs. Therefore, in our dataset collection pipeline, we mainly use two LLMs: 1) ChatGPT for instruction brainstorm- ing and 2) GPT-4 for instruction rematching. Table provides the detailed API usage and the corresponding cost of each phase in our pipeline. As for the reason for using differ- ent APIs of ChatGPT: we found that the gpt-3.5-turbo-0300 demonstrates better creativity than structions. In contrast, 3pt-3.5-turbb-000 3et-3.5-turbbo035 Thus, we use 3pt-3.5-turbo-033 for generating diverse textual attributes and instructions, while using gpt-3. 3.t-3.5-turme 5-turboo-0661 to produce the outputs.
where gpt-3. 5-ttbrboooo- constantly brainstorms more task in- tends to be more accurate on annotations than
Table 8: The overall cost and API usage.
API
Cost
Brainstormed instructions (56,495)
$182.90
Facels recognition
$5.49
Instruction brainstorm
5-t-3.5-turbo-1
S49.11
Output annotation
gpt-3, 5-turro-----e 8613
$58.56
Classification expansion
S69.74
SuperNI rematching instructions (11,519)
$388.44
Instruction-input rematching
gpt-4-0613
$271.08
Output annotation
gpt-4-0613
$117.36
Total (68,014)
$571.34
G DETAILS OF TUNING MUFFIN
of our implementations are based on HuggingFace transformers (Wolf et al. 2019 For the implementation of tuning T5 models, adopt the source training code of SUPERNI for fine-tuning the Llama2, we use the open-sourced Alpaca-Lora
where we apply LoRA (Hu et al. 2022) to relieve the high computational cost. Besides, all the models' weights are downloaded from HuggingFace repositories 13 We fine-tune T5 on MUFFIN with 2 epochs. When fine-tuning T5-3B, we set the learning rate as 5e with batch size 6. As for TBB we the learning rate le with batch size and 12 gradient accumulation steps. All the above hyper-parameters. are tuned on the validation set of SUPERNI. While fine-tune Llama2 on all the datasets 3 epochs with batch size 18, and we set learning rate le 4, lora,- = 8, lor 16. Since the generation API provided by HugggingFace cannot support efficient batched evaluation, we fix the evaluation batch size to for all the datasets. We truncate the inputs to 1024 tokens and limit the output length to 128 tokens, with beam search size = (greedy decoding).
All the experiments are done on NVIDIA A100 with 80 GPU memories. When fine-tuning T5 models, we utilize Dpeedpeed ZeRO stage 2 and 3 for single-GPU and multiple-GPU tuning, respectively.
H DETAILS OF BASELINES
We elaborate on more details of each baseline dataset used our experiments. Following Honovich et al. (2022), all the hyper-parameter of fine-tuning baselines trialed the validation set of SUPERNI. We truncate all the input length into 1024 and the output length into 128 (except those datasets designed for long response generation), which is the same fine-tuning MUFFIN. For
httpp: https:///shhhoooo//yyyyhgg///i/s
hhttps://gittee httrss//gitthhb.cc////eeee/////s htt.:aaac....tttb///eeeee/aa/aee T5-1 from hwm.tttps:/iccm httttp:://hugggiggface
We T5-38 from http:///.......iitto///g/gggaaaa
co/t5-11b/tree/main and I Llama2-133 from https:://uggiiiggteeeeee http:///a.a.....caaaaaoaoaaa cww.etaa 11ama
20

---

Published as a conference paper at ICLR 2024
fair comparison, analogous (in al. 2023 keep similar data size for all systems (60k 80k) with downsampling when necessary, and report the average performance of three seeds. When evaluating the models, we use the official scripts (Rajpurkar et al. 2016 14 calculate the metries, which are widely adopted by the previous works.
Scaling Input-Free Tasks is human-annotated instruction dataset created by Tollowing the schemes of InstructGPT (Ouyang et al. 2022) and open-ended free-form paradigm. We use the officially released dataset of DOLLY We fix the learning rate as 5e-05 and 3e-05 for T5-3B and T5-11B, respectively. We fine-tune models on DOLLY with 2
Dolly (Conover al. 2023)
epochs.
LongForm (okksal et al. 2023) Scaling Input-Free Tasks is LLM- MeMM-genrrated dataset
that specifically proposed for enhancing long text generation 16 Due to the lengthy output of LONGFORM, we extend its output length limitation into 512 when fine-tuning. We fine-tune models onLONGFORM with 2 epochs. We set the learning rate of T5-3B as 3e-5 5 and T5-118 1B 2e-5.
Alpaca TTaori et al. 2023) Scaling Input-Free Tasks is an LLM-generated dataset that mainly follows the same data construction pipeline SELF-INSTRUCT. The main difference of ALPACA is using a more powerful API. We use the official version of ALPACA 17 We fine-tune models on ALPACA with 3 epochs. We set the learning rate of T5-3B and T5-1 11 as 5e-5.
Scaling Input-Free Tasks is similar to ALPACA but uses We fine-tune models ALPACA-GPT4 with 3 epochs. We set the learning rate of T5-3B and 15-11B as 5e-5.
Alpaca-GPT4 (Peng et al. 2023)
GPT-4 as the API to synthesize data 18
WizardLM (Xu al. 2023a Scaling Input-Free Tasks LLMs evolve existing in-
structions (e.g., the seed instructions from ALPACA) into more complicated version. At the time similar to LONGFORM, the
of writing, we use the latest version of WIZARDLM for experiments
outputs in WIZARDLM dataset are lengthy. Therefore, we set the output length as 512. We fine-tune models with 2 epochs and fix the learning rate of T5 (3B and 11B) to 5e-5.
Self- Inst-unst (Wang et al. 2023b) Scaling Input-Free Tasks uses existing human-crafted instructions seeds and let GPT-3 (Brown et al. 2020) devise novel task instructions 20 We fine-tune models on SELF-INSTRUG with 3 epochs. We fix the learning rate of T5-3B and T5- 1B 5e-5 and 2e-5, respectively.
Scaling-Inputs There are two versions of UN- NATURAL INSTRUCT, namely the "core" version and the "paraphrase-expandee"" version. Since the paraphrase-expandee version is specifically designed for fitting T0-Eval and BBH evaluation bench- marks, we use the core set of UNNATURAL INSTRUCT in our experiments for fair comparison We fine-tune models on UNNATURAL INSTRUCT with epochs. As for the learning rate, we set 5e-5 and 1e-5 for T5-3B and TBB respectively.
Unnatural Instruct (Honovich et al. 2022)
Dynosaur Yin et al. 2023 Scaling-Input utilizes the input-output pairs in existing NLP datasets and drives LLMS to recover the potential tasks for each dataset, significantly reducing the data synthesis cost. We the officially released version of DYNOSAUR where Yin et al. (2023) used ChatGPT to filter those task categories that are similar to the tasks in SuperNI-Test. We set the epoch as 2 and fine-tune both T5-3B and T5-11B with learning rate of 1e-5.
http::///.tuuueeoooo///////eooor metriiisttttttttttttommmmm metrics.py
w2..tt..1111.00 196k
data. zzp
21

---

Published as a conference paper at ICLR 2024
0.25
Dolly
0.04
0.02
0.02
0.04
43-
Muffin
LongForm
0.07
0.03
0.02
0.05
SuperNI
0.20
Performances on Super SuperNI-Tes
Alpaca
0.01
0.08
APT4
0.12
0.05
0.01
0.08
0.15
WizardLM
Self-Inst.
0.12
0.04
0.02
0.07
40-
0.10
Unnatural
0:23
0.06
0.02
0.12
39-
Dynosaur
0.22
0.08
0.05
0.11
0.05
SuperNI
0.25
0.07
0.03
0.12
38-
Ours
0.05
0.02
0.08
0.00
SuperNI
MMLU
T0-Eval
BBH
3k
7k
10k
14k
17k
20k
24k
27k
50k
34
Trxining Data
Figure 6: The instruction semantic similarity between each training dataset and evaluation benchmark.
Figure 7: The scaling trends comparison be- tween SUPERNI and MUFFIN. The perfor- mances are based on T5-3B.
Scaling-Inputs is human-crafted high-quality instruction dataset, covering more than 1,600 tasks either from existing NLP benchmarks or created by human expertise 23 When training on SUPERNI, we follow the official implementation tunning models on the 756 training tasks with sequential multi-task learning scheme. We use the official code and hyper-parameter settings (Wang et al. 2022 to reproduce the performances.
SuperNI Wang et al. 2022)
I TRAINING AND EVALUATION PROMPTS
During the training and evaluation, we use the similar prompt template as the imple- mentation of Wang al. ) where we concatenate the task instruction and in- put together and ask the model to produce the output for this instance. To be specific, we use the following prompt: "### Input: InfInput Text] Anyn. Instruction: (Task Instruction in AnAnn#nn Output:". We fix this prompt for all the methods in our experiments (including our MUFFIN and all the baselines).
It is worth noting that, there are instances in some baseline datasets that don't 't have the task input, such as SELR-INST and ALPACA. As for those dataset, we set the "[Input Text]" filed as "None". Similarly, the instances in evaluation benchmarks, namely MMLU, TO-Eval, and BBH, also have empty task inputs, we set them to "None" as well. This also implies that, compared with those input-free datasets, MUFFIN is disadvantageous in the generalization of these benchmark datasets. However, MUFFIN can still yield better generalization capacity according to the experiments in
FURTHER ANALYSES
J.1 SCALING TRENDS COMPARISON WITH SUPERNI
As we introduced in $ $ the Innputg-Inputs paradigm scales the input-output pairs for each task. while our paradigm scales tasks for each input text. These two paradigms follow totally opposite scaling philosophies. Therefore, it's interesting to compare the scaling efficiency between them. To this end, we compare the generalization performances of SUPERNI and MUFFIN. For SUPERNI, we fix the task number as 681 (the maximum training task size we mentioned $ and scale inputs per task; similarly, for MUFFIN, we fix the input number as 681 and scale the tasks per input. We range both inputs per task and tasks per input from to 50, as the maximum inputs per instruction of MUFFIN is 46.68 (see Table DT. We train T5-3B and report the overall ROUGE-L performances the SuperNI-Test Since SUPERNI is a human-annotatee annotated dataset, we use ChatGPT to reannotate the outputs of SUPERNI (using the same prompt and setting as output annotation, see Table 6 to
httpes//iistruutiooss.p... allpaia
Instruct
22

---

Published as a conference paper at ICLR 2024
Table The task category overlap between different training datasets and evaluation benchmarks, estimated by ChatGPT.
SuperNI-Test
MMLU
TO-Eval
BBH
Self-Inst.
11.62%
10.37%
11.27%
5.29%
Unnatural Inst.
22.27%
16.91%
18.39%
4.76%
Dynosaur
12.08%
13.49%
15.20%
1.27%
Muffin
16.01%
13.05%
8.63%
2.53%
Table 10: Create MUFFIN with the input sourced from DYNOSAUR and UNNATURAL IN-
STRUCT (namely around 16k instances. The experiment based on T5-3B.
and -UFFINUNNNATURAL. All the datasets here have
Models
Supner Test (overall)
MMLU (ACC)
MMLU (EM)
T0-Exal (ACC)
Tv-Eval (EM)
BBH (EM)
Unmatural Inst.
37.12
24.35
22.33
45,93
35.88
8.69
Dynosaur
32.55
26.88
25.26
38.56
39.13
12:09
Mufiin Unmatural
38 86
33.12
24.73
46.08
43.63
12.91
Muffin-ynossnn
38.23
33.34
22.59
13.53
Muffin
40.45
33.24
25.39
42.68
40.99
13.44
ensure more fair comparison. Figure shows demonstrates a better generalization capacity compared with SUPERNI. Note that, even though the outputs of SUPERNI are reannotated, the relatedness of (instruction, input) pairs in SUPERNI is still better than MUFFIN (refer to Figure Therefore, it further proves the superiority of MUFFIN and suggests the effectiveness of the proposed Scaling Tasks per Input paradigm.
shows the results. Observably, MUFFIN consistently
J.2 THE TASK OVERLAP ESTIMATION
follow previous work prompting ChatGPT to further quantify the task category overlap between the evaluation benchmarks and different training datasets (Yin et al., 2023). Given instance from training dataset, we ask ChatGPT whether belongs to any task categories from the evaluation benchmark. For example, the test set of SUPERNI has 12 task categories, and request ChatGPT to perform 13-way classification (one "None of above" label) on the training instances. Finally, we report how many instances from this training dataset contain task leakage.
Beyond the task similarity analysis provided in 6.3
Table illustrates the results. Here, we only report the overlap results of MUFFIN and those competitive baseline datasets according Table [ Though there are some shifts in the leakage ranking compared with the semantic-based analysis in $6..3 (see Figure the overall conclusion still the same MUFFIN demonstrates relatively low evaluation task leakage across all four benchmarks.
J.3 USING THE INPUTS SOURCED FROM THE OTHER L.M-SYNTHETIC DATASETS
As introduced in 3 the creation of MUFFIN utilizes the human-crafted inputs from SUPERNI (i.e., "instruction brainstorm"). While those baselines with Sealing-Inpute paradigm either utilize inputs from other human-crafted datasets (e.g.. DYNOSAUR) or let LLMs to synthesize inputs. To investigate whether the proposed "instruction brainstorm" method can be extended other input sources, we gather the inputs from other LLM datasets (including DYNOSAU and UNNATURAL. INSTRUCT), and apply our "insruction brainstorm" method to them. Finally, we create two new small-scale datasets (around 16k instances), namely MUFFIN-DY DOS and MUFFIN- UNNATURAL, and compare their performances with DYNOSAUR and UNNATURAL INSTRUCT (in the same size). Table I0 shows the results, where observe that MUFFIN consistently demonstrates superior generalization performance, regardless of the text sources employed. It further suggests that MUFFIN'S robustness to follow instructions is not reliant on the input resources but rather stems from our crucial contribution the diversified instructions per input. Therefore, we anticipate that method can be ideally applied to any input resources while still crafting diverse task instructions.
23

---

Published as a conference paper at ICLR 2024
Table 11: The statistics of generated facets.
facets statistics
# of inputs
1,463
# of instructions (from basets-bused brainstorm)
33,720
# of facets
11,382
avg of facets per input
7.78
max of facets per input
30
# facets per input avg of instructions per facet max of instructions per facet min # instructions per facet
2
2.96
18
Table 12: The quality evaluation results of generated facets.
input-to-fan facctttt correctness
90.78%
facee-to-inntruuttin correctness
85.22%
J.4 THE EXPLANABILITY OF THE GENERATED FACETS
As mentioned in 3 the proposed "instruction brainstorm" method drives LLMs to brainstorm novel instructions based on the synthetic facets. To this end, it's unclear whether the synthetic facets explainable and reliable for the subsequent instruction generation due the LLM's hallucinations (Xie et al. 20233 Zhang et al.] 2023b). Therefore, in this section we conduct several additional analysis the generated facets in MUFFIN, including statistics, quality, and diversity of the facets.
Facets Statistics. Beyond the statistics of instructions in Table 1 we report additional statistics related to facets in Table II (the statistics after conducting "instruction filtering").
Factes Quality. To estimate the quality and reliability of the generated facets, we conduct further human verification on the facets. Specifically, we randomly collect 100 inputs from MUFFIN, along with their corresponding facets (889 facets in total) and instructions (2,265 instructions in total). Then, we ask an experienced human annotator to evaluate the following two metrics:
Input-to-facet correctness: whether each facet correctly described the given input (reflect- ing the quality and reliability of facts).
Pacet-to-instruction correctness: whether the subsequent instructions are reasonably related to the given facet (reflecting the utility of facets).
The final results shown in Table 12 These results demonstrate the good quality and utility of the facets generated by our method. It's also worth noting that, after we conducted "instruction filtering" (as introduced on $ 3. the input-to-facets correctness increased from 83.28% 90.78%, proving the effectiveness of the naive filtering method that can jointly filter a considerable amount of hallucinated facets.
Facets Diversity. As for the diversity of the generated facets, ask the human annotator to discover the topic categories from the facets (i.c., clustering instead of classification), because it's hard predefine the categories that may introduce personal bias. We used the in the previous analysis (100 inputs and 889 facets). The human annotator is asked to summarize "keyword" (topic category) for each LLM-generated facet. If the current category has already appeared before, the annotator also has to "cluster" those facets that belong to the same category together. In doing so, more and more novel categories can discovered.
instances as
We then calculate the following two metrics:
Intra-diversity: how many unique categories there were for each input's 's facets, and reported averaged ratio among all the inputs.
24

---

Published as a conference paper at ICLR 2024
Inter-diversity: overall categories distribution among all the 889 facets, namely the global frequency of each unique facet.
We find that the generated facets achieve 91.55% intra-diversity, meaning the facets of the same imput are pretty diverse. Meanwhile, there are total of 276 unique categories out of the 889 facets, which implies high inter-diversity of the facets.
25

---

Published as a conference paper at ICLR 2024
Table 13: Example cases in our dataset, including both high-quality and low-quality cases. We employ human annotators to judge the validity of instances. As mentioned in 4 we develop three validity questions (binary judgments). "Instruction" means "Q1: Does the instruction describe valid task that can be answered?", "Input" means "Q2: Does this instruction appropriately match the input?"; while "Output" means "Q3: Does the output correctly respond to the instruction and input?"
Task Instruction
Task Input
Task Output
Validity
you are given sentence that deseribes the defeat of one superhero by another. Your task is generate list of the defeated super- hero's is state after their defear. The output should consist of at least three adjectives separated by commas.
ISuperior Spider Mander--aaaaan appears in LEGO Marvel Super Hernes and is quickly dispatched by the orig- inal Spidey.
Instruction: V
adjectives that
defeated, humiliated, weakened
Iaput: Ouipat:
Given sentence. generate should the most likely next statement. The next statement should be reasonable and logi- cally cornect.
Would you like to go shapping together this Saturday?
Sure, I would love to go shopping with you this Saturday!
Instruction:
Input: V
Ouiput:
Title Mysterious Incident
Namb Expression
Write article reporting incident with
where someone pression. Include guotes from wirnesses and suthorities, as well rolevant informa- tion incident.
continues a expression.
bizume tum events, man was with numb expression in an incident that has left authorities puzzled. The incident occurred earlier today [Location), witnesses [...]
Date: [Carront Date]
Instruction:
Input:
Oulput:
You are given a set of three namerical values. Your task determine the average value these
Instruction: v
[87.487, 27.095, 27,812]
47.13
Input: v
return as two decimal places.
Ooiput:
In this task, you re given a sentence and your task is generate a simplified sentence that
Text: Cymande the self- rited dehut album by Hritish funk group Cymanda
The self-titled debur album by British funk group Cymande released
Instruction: V
loxical paraphrasing the original sentence given in the input.
Input: v
Titl: Cymande (album)
Ooiput:
In this task, you are given a set mathe- matical expressions. Your task is to identify highest value.
Instruction:
expression
73
Input: X
Output The matherutical expression with the highest value.
Output:
generate list of potensial reasons why there are commenis. Each reason should be a complete sentenoc and sbould not repeat any information already given the input.
Given the
Instruction:
topic: InfraRedEd sentence: Zero comments.
There are no comments because the topic is not popular.
Input: X
Output:
easy why ley kubrick believed steven spiel- berg would the perfect dirce- intelligence, just as it is understandallle why spielberg attractedd to
Develop a task extract the names of the ac- lors who played churacters David, Mon- movie "A.L" the given review. The output should be a list these three actor names.
Instruction: X
Gigolo
Haley Osmeni, Frances O' O'Connor, Jude Law
Input: X
Output:
In this task, you given deseription of how is traditionally made. job to extract all the steps involved making it and return them as numbered list.
Blond yogun, and sugar together. and again until
Instruction: X
Input: X
smooth [...]
Output:
J.5 DATA CASES WITH HUMAN-EVALUATION VALIDITY
We provide some representative cases (instructions, inputs, outputs) of our dataset in Table [3] including both high-quality cases (all three fields are correct) and low-quality cases (some of the fields are incorrect).
J.6
SYSTEM OUTPUT CASES ON THE EVALUATION BENCHMARKS
Table 14 reports the system outputs of LMs tuned on different datasets (baselines and MUFFIN).
26

---

Published as a conference paper at ICLR 2024
Table 14: Output cases of baselines and MUFFIN across four benchmarks. "Self-Inst." and "Unnatural" indicate the SELF-INSTRUCT and UNNATURAL INSTRUCT respectively.
SuperNI-Test
Category
Answer Generation
Question Modification
Summarication
Given abstract, and few options, dicatic if any of the given options appro- priately describe the given abstract. If them is proper keyword, 'Yes', ocherwise answer
You're given fill-in-the- blank question where the answer is PersonX. You need to minimally change the given question than Person Y. Although there are many correct anwwers, you only need write one of them.
topic word for the given
lence. A topic word usually the name of person, place, animal, concept or objoct sentence locusing on.
Instruction
Abstmer: A belavioural methad of habit
Context Word: summer
Input
reversal, [..].
Question: The summer went too quickly for PersonX. Person was excited for
the chinese postman problem is mathe- matical problem of graph theory.
Question: Does the following mesh I...]
Suppon, Non- U.S. Gov't.
school, dide't
go back.
Answer: Person
The summer
quickly for Per- school,
Ground
soeX. PersonY was
postman probiem
and wanted to go back.
Self-Inst.
No
Pasea'
Chisese
Unnatural
3)Bchavior Therapy
PersonY excited for school
graph theory
summer
quickly
Muffin (Ours)
Yes
sonY. Person Y was excited for school, chinese postman
but dide' It want to back.
MMLU
Category
Logic
Sexuality
Use indirect truth tables to determine
What makes a relationships last, accord- Sternberg?
As of 2019. about what percentage of
people from Turkey agree that the state
whether the following argument is valid.
for benefit of people?
Instruction
(A): Valid
[..]
(B): commitment
intinacy
(A): 36%
(B): 51%
(B) Invalid. Counterexample I..]
(C): decision
passion
(CI: 66%
Input
Ground Truth
A
SclE-Inst.
(A): Valid (B): Invalid Invalid (D):
(A)
Invalid
Unnatural
B
(B)
A: 36% B: 51% C: 66% D: 81%
Muffin (Ours)
commitment
To-Eval
Category
Causal Reasoning
Natural Language Inference
Natural I Language Inforence
Instruction
between two options. Help me choose the more likely cause: [--]
broke toe. am besitating
Does follow that Ansunaki is retiring
after this epásode. Yes, no, maybe?
Hey, look who's [...]
Suppose it's "Fight Flight"
the episode second senson
the NBC seience liction drama series
[..]] sometimes, true?
Input
Ground Truth
He dropped hammer on his foot.
No
Sometimes
Self-Inst.
(a)
Yes
The episode aired at 8 PM est. is some-
Unnatural
He gox hole his sock
Yes, no
TRUT
Muflin (Ours)
He dropped a hammer on his foot.
No
Sometimes
BBII
Category
Eroor Detection
Word Sorting
Object Counting
The following translations from German
English contain a particular
[..]
Sort the following woods alphahetically: List: alleviate duopoly mattress gland benelux townsmen bunyant klaxon hard-
Instruction
(B) Numerical Values (C) Negation
Oytions: (A) Modillcrs Adjectives
have a mouse, a rabbit, a deg. dock. and two goats. How many animals do have?
Antonyms (D) Named Entities (E)
hound tomography felice
Drupped Content (P)
Input
alloviate benelux buoyant duopely felice gland gunk handhound klason mattress tomography townstacn
Ground Truth
(A)
Self-Iust.
Dj Named
['a', 'o',
animals.
List: alleviate ducpoly mattress gland
Unnatural
(F)
henelux townsmen hooyant
I have six animals.
alleviate henelux booyant duopoly felice
Muflin (Ours)
Moditiers Adjectives
gland gunk hardbouudd klaxon mattress
tomography townsmen
27

---

Published as a conference paper at ICLR 2024
Mixing Muffin with SuperNI-Tras
29.9
AVG Porformances
57
36
0622
35
302
Muffin Proportions
Figure 8: The performance of the mixture of MUFFIN and SUPERNI.
Table 15: Results by Llama2-13B (Tovrron et al. 2023b). For the sake of simplicity, we omit the Rank Classification Accuracy here. The best scores under direct and indirect comparison settings are in bold and underlined, respectively.
SuperNI-Tes Test
MMLU
TO-Eval
BBH
Models
Data Size
EM (CLS)
ROUGE-L
ROUGE-L
(GEN)
(overall)
EM
EM
EM
Human Annotated Data (indirect comparison)
SuperNI
68k
50.73
55.99
52.43
31.38
46.37
12.26
Generated Dato (direct comprison)
Dully
15k
27.25
LongForm
23k
4.30
41.30
19.07
0.12
0.72
5.27
Alpuca
52k
33.34
51.67
43.65
36.01
40.39
21.72
Alpaca-GPT4
52k
18.27
44.27
33.50
1.01
6.29
2.20
Wizardl M
68k
10.52
43.36
27.27
0.29
7.20
4.24
Self-Inst.
82k
46.79
41.04
23.12
31.43
28.69
Unnatural Inst.
68k
37.63
50.23
46.03
6.69
8.35
5.05
Dynosaur
66k
44.35
49.34
47.08
17.26
34.59
7.11
Muffin (Ours)
68k
40.85
$7.71
49.71
37.67
55.98
19.01
28

---

Published as a conference paper at ICLR 2024
Table 16: Human evaluation acceptance ratio. We randomly sample 200 instances from each benchmark and let workers evaluate different systems' outputs.
Models
SuperNI-Test
MMLU
T0-Eval
BBH
Average
Dully
22.5
14.0
36.5
28.0
LongForm
6.0
15.0
10.0
12.0
10.8
Alpaca
44.5
20.0
42.0
24.0
32.6
Alpaca-GPT4
45.0
11.0
38.0
24.0
29.5
WizardLM
35.0
19.5
36.0
26.0
Self-Inst.
39.0
23.5
45.5
29.5
34,4
Unnatural Inst.
50,5
24.0
34,5
23.0
33.0
Dynosaur
43.0
28.5
30.0
22.0
30.9
Muffin (Ours)
56.5 (1 6.0)
34,5 6.0)
(1.00)
31.0 (1 1.5)
41.8 (1 (.44)
Table 17: Pair-wise comparison between MUFFIN (Ours) and three strong baselines, namely SELF- INSTRUCT ISelf-Inst.), UNNATURAL INSTRUCT (Unnatural), and SUPERNI, across four benchmarks.
SuperNI-Test
MMLU
TO-Eval
BBH
Ours
Self-Inst. 41.5
Tic
Ours 39.5
Self-Inst. 16.5
Tic
Ours
Self-Inst. 10.0
Tic 79.0
Ours 19.5
Self-Inst. 15.5
47.0
11.5
44.0
11.0
65.0
Ours 31.5
Unnatural 20.0
Tie
Ours 42.5
Unnatural 10.0
Tie 47.5
Ours 43.5
Unnatural 16.5
Tie 40.0
Oues 21.5
Unnatural 11.5
Tie 67.0
48.0
Ours 31.0
SuperNI 16.0
Tic 53.0
Ours 24.0
SuperNI 21.0
Tic 55.0
Ours 9.0
SuperNI 15.0
Tic 76.0
Ours 16.5
SuperNI 9.5
Tic
74.0
Table 18: Data collection ablations the validation set of SUPERNI. "CLS Exp" denotes the "classification expansion" ($ 3.3). We the data sizes of all the methods to 11.5k.
ROUGE-L (GEN)
ROUGE-L (overall)
Methods
(CI.S)
Instruction Rematching
29 45
46.11
35,95
Instruction Brainstorm
28.26
47.35
36.27
Rematching Brainstorm
29.77
48.12
37.53
Rematching Brainstorm CLS Exp
32.77
48.56
41.20
Scaling Trend Comparison
01
0.50
234
Dotaset
Figure The scaling trends comparison between MUFFIN and the previous baseline datasets (average performances on all four benchmarks).
29

---

Published as a conference paper at ICLR 2024
K
HUMAN EVALUATION DETAILS
Acceptance Ratios. We employ 5 graduate-level volunteers, who are experienced in NLP and crowd-sourcing jobs, to evaluate different models' outputs. We adopt the blind annotation each volunteer is randomly responsible for or 2 models' output evaluation without knowing which models they are. When evaluating each instance, we provide the volunteer with only task instruction, input, and model prediction (no ground -ouundttruth output is provided to avoid bias in annotation). Then, we ask the volunteer carefully read the task instructions and decide if the model's output correctly responds to the instruction and input 25
Pair-wise Comparison. For each test instance (one task instruction and one input), we provide volunteers with two system outputs. perform blind annotation on them with random order by asking the volunteers which system's output preferred in the context of the instruction and the input. The volunteers also select "tie" if there is no practical quality difference between them.
by our MUFFIN and the other by baseline dataset. We
2e ask the volunteers to decide the correctness of the models' outputs strictly - consideinng if the output violates explicit requirements mentioned in the instruction, e.g., length constraint.
30


Reviews:
[{"review_id": "2az7xFirvz", "reviewer": "Reviewer_BsoU", "summary": "This paper describes a technique for synthesizing instruction fine-tuning data using LLMs (ChatGPT and GPT-4). In particular, the paper draws a distinction between past works in this area which have focused on either adopting an instruction+input format and scaling the number of inputs per instruction (Scaling-Inputs), or adopting an instruction-only format and scaling the number of total instructions (Scaling Input-Free Tasks). As an alternative, the technique and resulting dataset, MUFFIN, presented in this work adopts the instruction+input format but uses automated techniques to scale the number of instructions per input (Scaling Tasks per Input).\n\nExperimental comparisons to extensive baselines are presented on SuperNI, MMLU, T0, and BBH (each classified as either Scaling-Inputs, Scaling Input-Free Tasks, or Hybrid) and demonstrate the effectiveness of the proposed approach in all 3 settings. Additional experiments and results from human evaluation are also presented.", "questions": "See weaknesses", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The topic of how to effectively scale synthetic instruction datasets is relevant and timely, and the dichotomy of scaling inputs vs instructions is interesting.\n2. The experiments are extensive and the inclusion of evaluation of datasets from multiple settings (scaling-inputs, input-free, etc) is appreciated.\n3. The methodology and techniques are mostly well-described and presented clearly.", "weaknesses": "1. The muffin looks more like a cupcake to me.\n2. As noted previously, the dichotomy of scaling inputs vs instructions is interesting. However, the presentation makes it seem like one must choose between the two when really they are more like two extremes of a spectrum. This very naturally leads me to wonder whether a hybrid that scales both according to some mixture proportion would be most effective. This seems like a somewhat large omission given it would be very testable with the synthetic data here.\n3. All experiments use T5-3B or T5-11B. As many of the other datasets (Self-Instruct, Dolly, Alpaca) were developed and tested using more recent LMs (Llama, etc), it would be useful to see if/how the results change when fine-tuning such models. At minimum, it would be useful to see performance numbers for the original models developed with these datasets (if they are available) in the \"Existing Systems\" section of Table 1 to understand the performance drop due to changes in setup (model, hyperparams, etc).\n4. I think there are a few errors in the baseline discussion in section 5. In particular, this section says Dolly and Self-Instruct were collected with ChatGPT or GPT-4. As far as I know, Dolly is human-created (this is stated correctly in the Appendix) and Self-Instruct was produced using GPT-3.\n5. The presentation of human evaluation was somewhat confusing and left out key details. What model was used, 3B or 11B? Additionally, the evaluation is described as follows, \"we provide the volunteer with only task instruction, input, and model prediction.\" However, MMLU is classified as \"Input-Free\". What do annotators see as the input in this setting?\n\n**Minor Issues**\n\n1. The Scaling-Inputs, Scaling Input-Free Tasks, and Scaling Tasks per Input terminology is somewhat awkward. Have the authors considered something slightly simpler like Scaling-Inputs, Scaling-Input-Free, and Scaling-Instructions?\n2. I found the \"our MUFFIN\" terminology used throughout the paper somewhat annoying and distracting.\n\n------\n\nI have reviewed the authors' responses and raised my score accordingly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "C9hpB5bNV2", "reviewer": "Reviewer_cEte", "summary": "The paper proposes to use multiple facets of an input to construction instruction-input-output pairs.", "questions": "1. See the weakness section. Maybe elaborate more on your opinions about point 1 and 2.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Curating instructions from different facets is an interesting idea. It can be combined with other data augmentation techniques.\n2. The presentation is straightforward and easy to follow.", "weaknesses": "1. This method heavily relies on the human-curated SuperNI dataset. The Instruction Brainstorming requires extracting Instruction-input pairs from SuperNI while instruction reconstruction simply use data from SuperNI but do compositional changes. However, after those changes, the results of finetuning an LLM become worse than originally finetuning with SuperNI, as in Table 1. Does this mean we are introducing noise?\n2. Comparisons to some baselines might not be fair, although being claimed to directly compared results in the paper. I tihink the comparison with Dynasour and Unnatural instruction is unfair as again, the curation of the new dataset relies on some higher quality human instruction in SuperNI while the other two work doesn't rely on modifying human instructions. I think to really demonstrate the effectiveness of this input-based instruction construction method, the authors should try to start from Dynasour or Unnatural instruction and create new instructions on inputs from those datasets instead of from SuperNI.\n3. The creation of the facets lack human control. This is a limitation in the methodology that the framework cannot control which facets will be generated from the first stage. \n4. This might be personal, but I have an overall negative attitude towards work like this one. This work basically falls into the category of exploiting the ability of LLMs to augment the instruction tuning data. The changes in this paradigm compared to previous ones look incremental, simply trying to change a bit on how we prompt LLMs to generate instruction-input-output pairs. Notice that this idea is also not new as in Dynasour, they also generate multiple instructions by asking LLMs to focus on different parts of the metadata. Most importantly, I don't know how this paradigm would really impact how people create instruction data. Does it worth trying this paradigm or simply using the queries to chatgpt to collect more data?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "COxq8ABvoL", "reviewer": "Reviewer_j2qL", "summary": "This paper introduced a two-step facet based instruction brainstorming method: in the first step, they use ChatGPT to recognize the textual facets of a given input; in the second step, they instruct ChatGPT to brainstorm task instructions. Besides direct instruction synthesis, the paper also suggests extracting human-written instructions from the training set and employ GPT-4 for binary classification. GPT-4 discriminates if the instruction can align with the input to create a valid task.", "questions": "1. Are there any particular reasons that different GPT models are selected for different purposes? Why would you pick ChatGPT over GPT4 to work on the facet recognition and instruction brainstorm? \n\n2. Any particular reason not to compare with FLAN [1]?\n\n[1]: https://arxiv.org/abs/2301.13688", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper contributes to a important problem: dataset and alignment. The paper is well written.\n\n- The proposed method works as a reasonable approach to create a diverse tasks and instructions per task given the same input. GPT-4 works as a reasonable critique model, selecting high quality examples. Overall, the method is very simple but effective. \n\n- The paper reported strong empirical results on SuperNI-test, MMLU, T0-Eval, etc., compared to related work. Human evaluation acceptance ratio is significantly improved with Muffin.", "weaknesses": "- The work can have better ablation on whether the diversity in the dataset or the quality of the dataset helps more. For example, without the facet based brainstorming, how would it perform? On the other side, without the filtering (GPT4 critique) how would it perform?\n\n- The paper lacks a scaling analysis on the size of the finetuning dataset. How does the model performance scale with the number of examples in the dataset?\n\n- The discussion on task leaking is too short. I feel it is a very important problem to discuss in greater details. For example, what is the SoTA metric to detect data leaking? Why would semantic similarities be sufficient in detecting? Sentences can mean similarly while being semantically different. I feel the paper can be much stronger if this part is well addressed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VevF2hSeiF", "reviewer": "Reviewer_od6V", "summary": "The paper proposes a new paradigm of \"Scaling Tasks per Input\" for curating instruction-following datasets. Existing methods for improving language models' instruction-following capabilities have several limitations. Scaling-Inputs require extensive input-output pairs per task, causing oversensitivity to inputs. Scaling Input-Free Tasks expands tasks without inputs but struggles when inputs are needed. To alleviate these issues, this work instead diversifies tasks for each input text using a facet-based approach to enhance model's instruction-following capability.\n\nThe authors construct a dataset MUFFIN implementing this paradigm. They generate tasks by identifying textual facets of inputs to stimulate relevant instructions and reusing existing instructions matched to inputs. Careful balancing of classification and generation tasks is also proposed. Comprehensive experiments demonstrate MUFFIN's superiority over strong prior datasets, approaching human-annotated data quality. Ablation studies and human evaluation further verify the effectiveness of the techniques and the overall paradigm.", "questions": "1. In the facet-based instruction generation and output annotation, is there a specific strategy or mechanism used to guarantee the trustworthy of the facets or output annotation, except for human inspection? \n2. For facet-based generation, how is facet diversity quantified? And is this diversity measured with respect to ground truth facets? Some analysis confirming facet coverage would be reassuring.\n3. Could you share some insights about why the MUFFIN model performs much worse than other baselines on BBH benchmark in Table 1?Since BBH focuses on assessing reasoning skills, this seems to indicate there is still room for improvement on those capabilities. To provide more insights into the diversity of the dataset, it would be helpful to breakdown the facet into fine-grained categories, such as elaboration,  numeric calculation, and commonsense reasoning etc. By doing so, it could give a clearer picture of the diversity of this dataset, meanwhile also help to isolate the reason behind the degraded performance. \n4. The motivation behind balancing classification and generation tasks is clear, but how exactly is the decision made on which tasks undergo classification expansion? Prompting the model to make the decision?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Originality: The proposed `Scaling Tasks per Input` paradigm is novel and unique compared to prior instruction dataset curation schemes. By controlling the input and diversifying tasks, it helps models focus on instructions.\n\nQuality: The paper is technically strong, with a systematic data collection pipeline considering input diversity, instruction generation, output annotation, and task balancing. The analyses thoroughly verify the quality, diversity, and validity.\n\nSignificance: Thorough experiments across four benchmarks convincingly demonstrate MUFFIN's superiority, compared to prior synthetic instruction dataset, over strong baselines spanning different paradigms. Both the automatic and human evaluations demonstrated the merit of this dataset.", "weaknesses": "While the input-facet-oriented instruction generation approach is interesting, a major concern is the reliability of facets identified by the LLMs. As we know, hallucination remains an issue with LLMs. Though the model may correctly recognize textual attributes in most of time, some \"facets\" or the model generated *outputs* of these facet-based question could arise from hallucination. I don't see a specific mechanisms to handle this in the paper, except human audit on a small portion of samples. It's unclear whether these noise data, if it indeed exist, would hurt the model's reasoning capabilities. *One potential evidence is the substantially degraded performance on BBH benchmark compared to other baselines in Table 1.*\n\nI see similarities between this work and the self-instruct paper in the overall approach and principles for data generation, experiment design, and analysis approaches. Without addressing the trustworthiness issue I mentioned above, I would be hesitate to advocate the contribution of this work. \n\nI would like to see the authors to share some insights about the limitation of this work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)