WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex Instructions
Authors: Can Xu
Qingfeng Sun
Kai Zheng
Xiubo Geng
Pu Zhao
Jiazhan Feng
Chongyang Tao
Qingwei Lin
Daxin Jiang
Venue: ICLR 2024 poster (2024)
Keywords: Large Language Model
Instruction Fine-tuning
Primary area: generative models

Abstract: Training large language models (LLMs) with open-domain instruction following data brings colossal success. However, manually creating such instruction data is very time-consuming and labor-intensive. Moreover, humans may struggle to produce high-complexity instructions. In this paper, we show an avenue for creating large amounts of instruction data with varying levels of complexity using LLM instead of humans. Starting with an initial set of instructions, we use our proposed Evol-Instruct to rewrite them step by step into more complex instructions. Then, we mix all generated instruction data to fine-tune LLaMA. We call the resulting model WizardLM. Both automatic and human evaluations consistently indicate that WizardLM outperforms baselines such as Alpaca (trained from Self-Instruct) and Vicuna (trained from human-created instructions). The experimental results demonstrate that the quality of instruction-following dataset crafted by Evol-Instruct can significantly improve the performance of LLMs.

Published as a conference paper at ICLR 2024
WizardLM: EMPOWERING LARGE PRE-TRAINED LAN- GUAGE MODELS TO FOLLOW COMPLEX INSTRUCTIONS
Can Xu Qingfeng Sun
Kai Zheng Xiubo Geng'
Pu Zhao'
Jiazhan Fengat Chongyang Tao Qingwei Lin'
Daxin Jiang!
Miicosooft
"Peking University
fcaxu, qins, zhengkai, xigeng, puzhao, chongyang. tao, qlin, dijaan] eacaaoootmccooccccoomm ot fengjiazhann @pku edu. en
ABSTRACT
Training large language models (LLMs) with open-domain instruction following data brings colossal success. However, manually creating such instruction data is very time-consuming and labor-intensive. Moreover, humans may struggle to produce high-complexity instructions. In this paper, we show an avenue for creating large amounts of instruction data with varying levels of complexity using LLM instead of humans. Starting with an initial set of instructions, proposed Evol-Instruct to rewrite them step by step into more complex instructions. Then, mix all generated instruction data fine-tune LLaMA. We call the resulting model WizardLM. Both automatic and human evaluations consistently indicate that WizardLM outperforms baselines such as Alpaca (trained from Self- Instruct) and Vicuna (trained from human-created instructions). The experimental results demonstrate that the quality of -orllwinn-foolowinng dataset crafted by Evol- Inst-utt can significantly improve the performance of LLMs.
INTRODUCTION
Large-scale language models (LLMs) have become the go-to approach for numerous natural language processing tasks (Brown et al. 2020 Ouyang et al. 2022 Touvron et al. 2023). LLMs are trained on large volumes of text data to predict the subsequent tokens, enabling them to generate coherent and fluent text in response to various inputs. However, these models often struggle to follow instructions or goals specified by users, which limits their usefulness and applicability real-world scenarios.
The NLP community has recently witnessed many endeavors to train LLMs to follow instructions
better and be more helpful (Zhao et 2023 He al. 2023 Guo al. 2023 Li et al. 2023b). Initial attempts (Aribandi et al. 2022 Wei et al. 2021 Xu et al. 2022 Sanh et al.) 2022 Chung
ct al. 2022) to train instruction-follae wollooinn language models are based on a collection of various NLP tasks, with small amount of hand- written instructions. These closed-domale instructions suffer from two main drawbacks: first, all the samples an NLP datasct share only a few common instructions, severely limiting their diversity; second, the instructions usually only ask for one task. But in real life, human instructions often have multiple and varied task demands. By using open-domain instruction data generated by real human users, OpenAl's 's LLMs (e.g., InstructGPT (Ouyang al., 2022 and ChatGPT D have achieved great success. These open-domain instructions can fully unleash the unlimited potential of LLMs (Luo al. 2023 Ma al.) 2023 Hu et al.] 2023 Zhu et al. 2023) and enable them to perform more complex ex and diverse tasks. However, using humans to create open- open- pren-domain instruction datasets like OpenAI did will encounter the following challenges. The whole annotating process is extremely expensive and time-consuming (Kopf et al.) 2023 Chen et al. 2023 Sun al., 2023 Yuan et al. 2023). On the other hand, the difficulty level distribution of human- created instructions is skewed towards being easy moderate, with fewer difficult ones (according
Equal contribution.
Work done during the internship at Microsoft.
t Corresponding author.
https://chat. openaaiii..pp.nnio.aaaaa meeenai.ccm comm

---

Published as a conference paper at ICLR 2024
to the difficulty statistics of ShareGPT (Chiang et al. 2023 from Figure 5a Human annotators are prone to fatigue and cannot sustain high-intensity work to produce a sufficient proportion of high-difficulty instructions (Zhang et al. 2023 Xiao et al. 2023 Manakul et al. 2023 Zhong et al., 2023). Based on these issues, developing an automatic method that can mass-produce open-domain instructions (especially the more difficult ones) relatively low cost becomes the key further advancing instruction-tunel language models
[Bao et al. 2023
Liu et al. 2023 Bian et al. 2023
Cabannes et al. 2023).
photosymthess commonly 6CO2 CGH1205
Please with approximate
Pleas eaplain the role chiorophyll aboww
speed of fight each medium
Speed (kmm/s)
Water
import math
import
Complicate Input (Table)
choose random integer between and 10
random.randint(,
1/Imath 3/lmath.sqrttb xx2)
How marry faster light than sound in vacium?
How the spwed vacuum measured and defined?
Complicate Input (Coda)
Ressoning
Despening
What
Complicate Imput (Formula)
In-Breacth Evalving
gives you apple
Goldbach Conjecture?
many fruits do you
Add Constraints
Concrettring
not equel
situstion
Deapaning
1+1=?
Increasa Reasaning
4 3-37
Inital Instruction
Figure 1: Running Examples of Evol-Instren
In this work, we introduce Evol-Instruct, novel method using LLMs instead of humans to automati- cally mass-produce open-domain instructions of various difficulty levels, to improve the performance of LLMs. Figure I shows the running examples of struction " "1+1=?", our method randomly selects In-depth Evolving (blue direction line) or In- nn-breadth Evolving (red direction line) upgrade the simple instruction one (to increase diversity). The In-depth Evolving includes five types of operations: add constraints, deepening, concretizing, increase reasoning steps, and complicate input. The In-breadth Evolving is mutation, i.c., generating completely new instruction based on the given instruction. These six operations implemented by prompting LLM with specific prompts. Since the evolved instruc- tions are generated from LLMs, sometimes the evolving will fail. We adopt an instruction eliminator filter the failed instructions, which is called Elimination Evolving. We repeat this evolutionary process for several rounds to obtain enough instruction data containing various complexities.
Starting from simple initial in-
complex
create
In order to verify the effectiveness of Evoo-Inntruc and whether the instructions it creates for fine- tuning surpass those created by humans, we evolve the instructions from Aplaca (Taori et al. (023) data (created by machine), fine-tune the LLaMA (Touvron et al. 2023 model, and comprehensively compare the fine-tuned model WizardLM with Vicuna (Chiang et al. 2023) trained on ShareGPT (instructions are created by human). Alpaca data has total of 52k samples and is generated using self-instruct (Wang et al. 2022a) from only 175 human-created seed instructions. We choose Alpaca data as the initial data for evolution, which can ensure that the training instructions WizardLM have almost no direct human participation in annotations. We execute four epochs of evolution
2

---

Published as a conference paper at ICLR 2024
using OpenAI ChatGPT Ω and finally obtain 250k instructions. To ensure fair comparison with Vicuna's 70k real user data, we sampled 70k from the full 250k data and fine-tuned the LLaMA 13B model. Because the original Alpaca data only has 52k samples, used its self- generate an additional 18k data, and retrained the LLaMA 13B model with its code
method to JodePto to get Alpaca 13B our baseline. Due to the low proportion of difficult instructions in the previous instruction-followe following test dataset, we manually created a new cefficulty-balancee test dataset, named WizardEval. We evaluate Alpaca, Vicuna, ChatGPT, and WizardLM on a wide range of LLM benchmarks (covering reasoning, code, mathematics, general conversation, etc.). Our main findings are as follows:
We introduce Evol-Instruct, a novel approach that enhances the performance of the open- source L.Mss by large margin via automatically mass-producing open-domain instructions of various topies and difficulty levels.
We develop WizardLM model, which significantly surpasses typical open-source LLMs such Alpaca and Vicuna in series of benchmarks, Notably, WizardLM outperforms baselines by a substantial margin in terms of code, math, GPT-4 and human evaluations.
We have undertaken a preliminary investigation that underscores the importance of in- struction complexity in attaining outstanding performance in supervised fine-tuning large pre-trained language models.
RELATED WORK
Closed domain instruction tuning Early instruction-followingg training work (Wei et al. 2021 Longpre et al., 2023) concerns cross task generalization in LMs, where LMs are fine-tuned on broad range of public NLP datasets and evaluated on a different set of NLP tasks. T5 Raffel et al. (2020) made the earliest attempt by training natural language processing (NLP) tasks such question answering, document summarization, and sentiment classification together using unified text-to-text format. Works such FLAN Wei et al. (202T), ExT55 Aribandi al. (0222). TO |anh al. (2022), and KnowDA [Wang et al. (2022c) increased the number of NLP tasks to around one hundred, with several instructions carefully designed for each task de Wynter et al. 2023); Svikhnushina & Pu (2023); Huang et al. (2023); Yue et al. (2023). Furthermore, works such as Zero ZeroPrompt Xu et al. (2022) FLAN-T5 Chung et al. (2022) raised the number of tasks to the thousands. These studies consistently show that fine-tuning LMs with diverse NLP task instructions enhances their performance on new tasks. However, LLMs trained with these closed-form instructions (i.e., instructions are often only for a single NLP task, and the input data form is simple) tend to fail in real-world user scenarios.
Open domain instruction tuning Our work belongs to this research line. OpenAl has hired many amnotators and written many instructions with corresponding correct responses. These human- created instructions have diverse forms and rich task types. Based on this dataset, OpenAl trained GPT-3 Brown et al. (2020) into EaaructGPT[Oyyannn real user instructions and led the success of ChatGPT. Orca Mukherjee al. (2023) learns not only the superficial response text from LLMs, but also captures complex reasoning process signals. Since these outstanding works from OpenAI were not open-sourced, Alpaca Taori al. (2023) and Vicuna Chiang et al. (2023) subsequently actively explored open-domain instruction fine- tine-tuning based on the open-source LLM LLaMA Touvron al. (2023). Alpaca used dataset of 50k instructions generated from limited (e.g.. 175 samples) seed set of manually-wa writtenn instructions. Our work is different from InstructGPT and Vieuna in that we use Al-generated data for instruction fine-tuning. Unlike Alpaca's 's self-instruct Waan et al. (2022a) generation method, Evol-Instruct can control the difficulty and complexity level of the generated instructions.
et al. (2022), which can process a variety of
3 APPROACH
In this section, we elaborate on the details of the proposed Evol-Instruct, As illustrated in Figure 2 the pipeline mainly contains two components: Instruction Evolver and Instruction Eliminator. The
3

---

Published as a conference paper at ICLR 2024
In Depth Evolving
In-Breatth Evolving
Elmination Evolving
WizardLit M
Initial Instruction
Instruction Pool
3
Instruction SAR Eliminator
Figure 2: Overview of Evol-Instruct
details of these compoents will be presented in Sec. 3.2 and instruction fine-tuning method will be
described in Sec. 3.3
3.1 DEFINITION OF INSTRUCTION DATA EVOLUTION
We start the evolution from a given initial instruction dataset ( = (T% R DISEEN- where T(0)
is the k-th instruction in (0), R is the corresponding response for the k-th instruction, and N is the number of samples in ( In each evolution, we upgrade all the 7(t) in D(t) to ((++1) by prompting a LLM with Evol-Instruct prompt, and then use the LLM to generate corresponding responses H(+1 for the newly evolved |=+1 Thus, performing M evolutions, we can sequentially obtain M evolution datasets [D(1.. ... D(M), Our work focuses on open-domain instruction data, where instructions have varying inputs and tasks without clear distinction between the instruction part and the input.
obtain evolved instruction dataset D By iteratively
3.2 AUTOMATIC INSTRUCTION DATA EVOLUTION
Our pipeline for instruction evolution consists three steps: 1) instruction evolving, 2) response generation, and elimination evolving, i.e., filtering intructions that fails to evolve.
Instruction Evolution. We found that LLMs can make given instructions more complex and difficult using specific prompts. Additionally, they can generate entirely new instructions that are equally complex but completely different. Using this discovery, we can iteratively evolve an initial instruction dataset, improving difficulty level and expanding its richness and diversity. We initiate the instruction pool with the given initial instruction dataset (()), In each evolution epoch, upgraded instructions from the previous epoch are taken out from the pool. Then we leverage the instruction evolver evolve each fetched instruction, and the instruction climinator check whether the evolution fails. Successful evolved instructions are added to the pool, while unsuccessful ones are placed back as they are, with the hope of upgrading them successfully the next evolution epoch.
Instruction Evolver. The Instruction Evolver is an LLM that uses Evol-Instruct prompts to evolve instructions, with two types: in-depth evolving and in bn-brea evolving.
In-Depth Evolving enhances instructions by making them more complex and difficult through five types of prompts: add constraints, deepening, concretizing, increased reasoning steps, and complicating input. The core part of In-Depth Evolving's prompt is "Your objective is to rewrite a given prompt into a more complex version to make those famous Al systems (e.g., ChatGPT and GPT4 (OpenAl 2023)) bit harder handle. But the rewritten prompt must be reasonable, understood. and responded to by humans".. We require the LLM to create challenging instructions that reasonable and not arbitrarily imagined by AI. A gradual difficulty increase is necessary avoid
4

---

Published as a conference paper at ICLR 2024
filling the instruction set with extremely complex instructions, which would harm the generalization performance of trained models. To control difficulty increase, we make each evolution "a bit harder" and restrict adding maximum of 10 to 20 words. Among the five mentioned evolving. all can be implemented without any in-context examples except for complicating input. We show the prompt of add constraints follows (the prompts of deepening, concretizing and increased reasoning steps will be detailed in the Appendix A-C).
Example 3.1: Prompt for Adding Constraints of In-Depth Evolving
I want you act as Prompt Rewriter.
complex version make those famous Al systems (e.g., ChatGPT and GPT4) bit harder to handle. But the rewritten prompt be reasonable and must be understood and responded by humans.
Your objective is rewrite given prompt into
Your rewriting cannot omit the non-text parts such as the table and cade in #Given Prompt#:. Also, please do not omit the input in #Given Prompt#.
You SHOULD complicate the given prompt using the following method: Please add one more constraintsrreqiireentt into #Given Prompti
You should try your best not to make the #Rewritten Prompt# become verbose, #Rewritten Prompt# can only add 10 to 20 words into #Given Prompt#. '#Given Prompt#', ##eewriten Prompt#', 'given prompt' and 'rewritten prompt' not allowed appear in #Rewritten Promptif
#Given Prompt#:
{Here is instruction.)
#Rewritten Prompt#:
For complicating input, we will use in-context demonstration. Due to the lengthy demonstrations, we will provide brief template below, with the full prompt detailed in the Appendix D
Example 3.2: Prompt for Complicating Input of In-Depth Evolving
want you act as a Prompt Rewriter.
Your objective is to rewrite given prompt into more complex version to make those famous Al systems (e.g., ChatGPT and GPT4) bit harder to handle. But the rewritten prompt be reasonable and must be understood and responded by humans.
You must add [XML data] format data as input data in [Rewritten Prompt]
#Given Prompt#:
(Here is instruction of Example 1.}
#Rewritten Prompt#:
(Here is rewritten instruction of Example 1.}
N Examples ...
You must add [#Given Dataformatt] format data as input data in [Rewritten Prompt]
#Given Prompt#:
(Here is instruction of Example N.}
#Rewritten Prompt#:
In-Breadth Evolving aims to enhance topic coverage, skill coverage, and overall dataset diversity. Open-domain instruction finetune datasets (e.g., Alpaca, ShareGPT, etc.) are typically small in scale, lacking topic and skill diversity. To solve this problem, we designed a prompt to generate completely new instruction based on the given instruction, requiring the new instruction to be more long-tailed. Our In- Breadth Evolving prompt is as follows:
Example 3.3: Prompt In- In-Breadth Evolving
I want act Prompt Creator.
Your goal is to draw inspiration from the #Given Promptif to create brand new prompt. This new prompt should belong to the same domain the #Given Prompt# but be even more rare.
The LENGTH and difficulty level of the #Created Prompt# should be similar to that of the #Given Prompt#.
The #Created Prompt# must be reasonable and must be understood and responded by humans.
5

---

Published as a conference paper at ICLR 2024
#Given Prompt#", #Created Prompti..
##reatedd Prompt#", "given prompt" and 'created prompt' are not allowed to appear in
#Given Prompt#:
{Here is instruction.)
#Created Prompt#:
Response Generation. We use the same LLM as for evolving to generate the corresponding responses for the evolved instructions. The generation prompt is "(Here instruction.)" we feed into the request of the ChatGPT-335 and parse the returned text body as the response.
Elimination Evolving. We classify the following four situations as instruction evolution failure:
1. The evolved instruction does not provide any information gain compared to the original one. We use ChatGPT to make this determination, details please refer to Appendix G.
2. The evolved instruction makes it difficult for the LLM to generate response. We found that when the generated response contains "sorry" and is relatively short in length (i.e., less than 80 words), it often indicates that the LLM struggles to respond to the evolved instruction.
So we can use this rule to make a judgment.
3. The response generated by the LLM only contains punctuation and stop words.
4. The evolved instruction obviously copies some words from the evolving prompt, such as "given prompt", "rewritten prompt", "#Rewritten Prompt#", etc.
3.3 FINETUNING THE LLM ON THE EVOLVED INSTRUCTIONS
Once all evolutions are done, we will merge the initial instruction dataset with evolved instruction data from all epochs and randomly shuffle the samples to create the fine-tuning dataset. This processing ensures even distribution of instructions of varying difficulty levels in the dataset, maximizing model fine-tuning smoothness. To prove that the performance gain is not due to the increased amount of data after merging, but from our proposed novel method Evol-Instruct, amount of data the same with training baselines (e.g., Vicuna) from this merged data as our final fine-tuning data. We choose Vicuna's prompt the prompt for fine-tuning, the specific format "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. USER: Hi ASSISTANT: Hello. USER: Who you? ASSISTANT: am WizardLMM
randomly sample an equal
4 EXPERIMENT
We assess WizardLM, Alpaca, Vicuna, and ChatGPT using both automatic and human evaluations.
4.1 BASELINES
(1) ChatGPT is an Al chatbot developed by OpenAl that can interact with users in a natural and engaging way. It is built on top of LLMs like GPT-3.5 and GPT-4, trained vast internet text data.
(2) A Alpaca an open-source instruction-follooiing model developed by Stanford University. For a fair comparison, expanded the number of instructions from 52k to 70k using Inst-uut adopted
by Alpaca and replaced the original Davici-003 responses with ChatGPT's responses. We re- re-trained Alpaca 13B from LLaMA 13B [Tuvvrn et 2023) based on this new Alpaca data.
(3) Vicuna is based on LLaMA and fine-tuned on 70k user-shared conversations collected from ShareGPT. It is one of the most advanced and versatile open instruction-following models available today. We use the 13B-v1.1 model from FastChat
(4) Open-source models trained from Llama 13B, including Baize (Xu et al.
2023 CAMEL
et al. 2023a), and Tulu (Wagg et al. 2023
1m- hain--sss/asstthattt

---

Published as a conference paper at ICLR 2024
4.2 EXPERIMENT DETAIL
To construct the dataset, we initialize it with the 52k instruction dataset of Alpaca and iteratively perform M evolutions, where M 4. For each instruction in each round of evolution, randomly select one evolving prompt from total six prompts (i.e., five from in-depth evolving and one from in-breadth evolving) with equal probability. We exccute above process using Azure OpenAI ChatGPT APp Then, we leverage ChatGPT to generate responses. Finally, we obtain 250k instructions. For fair comparison, we randomly sample 70k data from 250k data with equal probability as the final training data for WizardLM,, the same as the amount of training data for Vicuna. We use temperature of to generate response and set the maximum number of tokens for generation to 2048, Additionally, we set the frequency penalty to zero and top-p to 0.9. Totally, we request the API 52k x x4x x 3 = 624k times to construct the full dataset. We use pre-trained LLaMA 13B (Tovroon et al. 2023) to initialize our model. We adopt Adam optimizer with an initial learning rate of x 10 maximum number of tokens 2048, and the batch size is 4 for each GPU. We train our model on V100 GPUs with Deepspeed Zero-3 for 140 hours on 3 epochs. For inference, we use greedy search for WizardLMM and baseline models, and set the maximum generation length to 2048.
4.3 AUTOMATIC EVALUATION
To present a comprehensive overview of the perfor- mance of our WizardI.M, we conduct a comparative comparison between our model and the established baselines across range of LLM benchmarks.
MV
OwcCPFa
OpenLLM Leaderboard of HuggingFace (Beech- ing et al. 2023) includes MMLU (Hendrycks et al.
IellSevio
Dench
2020), ARC (Clark et al. 2018). HellaSwag (Zellers et al. 2019). and TruthfulQA (Lin al. 2002).
MMLU consists of range of multiple-choice aca- demic questions. ARC is set of grade-school sci-
Mauttall
ence questions. HellaSwag is a test of commonsense inference. TruthfulQA measures a model's propen- sity to reproduce falsehoods. We adopt the evaluate code (Gao et al. 2021 from OpenLLM.
Figure 3: Automatic evaluations on nine LLM benchmarks.
Code Generation We use the extensively utilized
HumanEval (Chen et al. 202T) benchmark consisting of 164 coding problems to evaluate LLMss code writing capabilities at the function level by reporting the pass@ @1 metric.
Math Reasoning We use GSM8k (Cobbe et al. 2021 to evaluate mathematical abilities of models, GSM8K contains 1319 grade school math test data. We adopt 4-shot testing and report pass @ 1.
GPT-4 Evaluation We employ two widely recognized GPT-4 evaluation benchmarks, including Al AlpacaEval (Li al., 2023c) and MT-Bench (Zheng et al. 2023). We also use GPT-4 to judge L.MSs
on our following proposed Wvl.
Model
Avg.
MMLU
ARC
HellaSwag
TruthfulQA
HumanEval
GSM8k
AlpucaEval
MT-Bench
WizardEval
ChaiGPT33.5
76.15
70.0
85.2
85.5
47.0
48.1
80.8
$9.37
7.94
100.0
Alpuca-1.0 13b
43.44
46.63
$1.20
76.31
41.62
9.2
8.35
33.25
4.78
76.6
Vicuna-
51.71
52.68
86.9
Baize-13b
51.16
49.72
56.91
79.29
47.88
14.6
8.95
66.96
5.75
$1.3
CAMEL-133 13b
51.29
49.74
55.63
79.25
47.42
17.7
7.13
64.84
5.78
82.1
1hln-13b
52.46
53.19
21.3
45.34
79.88
WizardLAM-13b
$8.96
52.92
57.25
80.88
50.55
24.0
37.15
75.31
6.35
89.1
Table 1: Performance comparison of ChatGPT-3.5, open-source baselines, and WizardLM-13b.
As shown in Figure 3 and Table I compared with other same-sized open-sourced models, WizardLM has a remarkable performance advantage in most benchmarks. Especially in math, code, and GPT-4 evaluations, it achieves significant improvement over Alpaca, Vicuna, Baize, CAMEL, and Tulu.
"gpt-3.5-turbo from httph:ttt///////. //ap:////aai../// azure. www.aaaaaarrre...ooaaaaaa.. com/portal
7

---

Published as a conference paper at ICLR 2024
80
Vicuna
Ours Win
Ours Lose
self-intruct WizardEval
60
Alpaca-13B
116
50
Count
40
Vicuna-
90
20
ChatGPT
2
5
6
7
8
9
10
6%
20%
40%
80%
80%
100%
(a) Testset difficulty & complexity level.
(b) Human evaluation results EvzardEval.
Figure 4: WvzardEval difficulty and complexity level distribution, and the human evaluation results between WizardLM and baselines (ChatGPT-3.55, Alpaca, Vicuna) on WizardEval.
4.4 HUMAN EVALUATION
To evaluate WizardLMM, conduct human evaluation crafted testbed WizardEval, which includes 218 real-worid human instructions from diverse sources such as online opensource projects (Github, ShareGPT), platforms (Twitter), and forums (Reddit, Discord). The data contains 29 skills and domains that represent the main requirements of humanity, such as Coding Generation, Math, Reasoning, Complex Formats, Writing, Extensive Disciplines, and so on. As shown in Figure 4a and Appandix Figure 6 we also analyse the difficulty and skills distribution of WizardEval respectively, which indicate that WizadEval is able to handle the evaluation on more complex and demanding scenarios than Self-Instruct and Vicuna testset.
We perform blind pairwise comparison between WizardL.M- and baselines. Specifically, we recruit 10 well-educated annotators. To each annotator, four responses from Alpaca-13b, Vicuna-13b. WizardLM and ChatGPT are presented, which are randomly shuffled to hide their sources. The annotators then judge which response is better following criterion (for detailed definition, please refer to Appendix[K] (1) Relevance, (2) Knowledgeable, (3) Reasoning, (4) Calculation, and (5) Accuracy.
Then they should rank the four responses from to 5 (1 means best), and allowing equal scores for comparable instances. To estimate the win rate, we compare the frequency of win, lost, and tie between each pair of models. As shown in Figure ( (b). WizardLM achieved significantly better results than Alpaca and Vicuna, which demonstrates the effectiveness of Evol-Instruct method. All of the Kappa scores are greater than 0.6, which indicates the good agreement among the annotators.
4.5 ABLATION STUDY
Training with different data (seed, size), evol model, and base model size. order to study the impact of different data seeds, Evol models, scale of evolved dataset, pre-trained models proposed method, we conducted the following experiments: a) Using 70k ShareGPT as the seed data to obtain WizardI.M-131 (ShareGPT Seed); b) Using LJaMA-2-70B-CCT as the evolutionary execution model to obtain WizardL_M-133 train on larger size pre-trained models Llama-1 65B and Llama-2 70B obtain WizardLM-65b and WizardLM-70b respectively; d) Using the complete 250k evolved data to obtain WizardLM- 13b (250K); e) Using completely different base from the LlaMA family, Mistral-7B, to obtain WizardLM- (Mistral); f) In order to compare more diverse instruction data, we choose Supernatural Instructions(Wann Wang et al. 2022b) and randomly extract 70k data train llama- 13b obtain LlaMA- 13b (SNI). The full results are shown in the Table 2 To investigate the reason of why does WizardL.M LM- 13b (ShareGPT Seed) performs worse on GSMSK, we random sample 2000 instructions from ShareGPT and Alpaca data respectively, then use ChatGPT to judge (prompt please refer to Appendix G whether an instruction is "math" related, we find that the ShareGPT only contains 4.3% math data, and Alpaca data contains 11.8% math data, thus think that less math data results in worse GSM8k performance of WizardLMM1335 (ShareGPT Seed).
to replace ChatGPT Evol); c) We
The results indicate that (i) the ShareGPT is better seed for evolved data size can improve model capacity, and (iii) our proposed crol-Instruct method is not dependent ChatGPT, other strong open source model such as llama-2 is also good substitute for
than Alpaca, (ii) larger
8

---

Published as a conference paper at ICLR 2024
MMLUU
ARC
HellaSwox
TruthfuQAA
HumanEval
GSSIKK
Alpacalival
MY-Blench
WizurdEval
Wizardl M-D3
58.96
50,92
57.25
90.83
50.55
24.0
37.35
75.31
6.35
89.1
Wicnol Id. Bb (SunCIPP Seath
61.87
50,92
60.24
81.30
54.55
25.0
31.46
96.33
6.76
99.3
WizardLA Ld (250K)
60.30
53.78
55.53
61.39
52.26
25.6
57.46
73.10
6.31
90.3
Wizard M--13 (LaMA-2-70B-Cht Eval)
56.27
51.09
57.3 14.54
79.12
48.76
19.3
33.83
20.47
6.18
94.5
1JMMA1 13h (SND
37.25
54.90
54.95
80.40
18.69
4.30
5.79
13.87
2.86
56.6
Alparo-Th (Midral)
$2.87
56. 34.34
55.
70-40
43.02
19.3
12.05
54.26
5.47
80.5
Wizaril M-75 (Mieral)
65.81
60.
57.40 47-47
82.08
51.70
37.50
59,49
80.70
7.80
91.3
Wizaril M-656
69.40
62.09
665 83
83.45
52.19
36.5
10.39
87.50
7.82
97.5
M--ardLM-706
71.33
63.32
64.52
83.21
$4.60
42.1
70.61
80.32
7.46
99.7
Table 2: WizardLM with different data seed, data size, evol model, and base model size.
10
60
55.75
57.61
Difficuity & Complexity Level
53.32
8-
50
40..9
6.64
7.00
40-
30
20
10
ShareGPT Alpaca
C4
CD
C1
C2
C3
C4
Dataset
Evolution rounds
(a) Average difficulty and complexity level
(b) Average score on automatic benchmarks
Figure 5: The difficulty level between ShareGPT, Alpaca, and our four epochs of evolved instruction.
ChatGPT, (iv) our evloved data also shows better finetune performance than Supernatural Instructions. Futhermore, the results on different pre-trained bases (e.g.. Llama-1 65B, Llama-2, Mistral-7B) indicate that our Evsll-nstruuc can be widely applied to various pre-trained models.
Analysis of In-depth Evolving. The Figure 55 and 5b presents ablation study investigating the impact of the number of data evolution rounds. To study the depth of the evolving process, we use ChatGPT judge the difficulty level of instruction. The used prompt please refer Appendix E
Figure 5b shows the average scores (on nine automatic benchmarks in Section 4.3 of the models fine-tuned with the data from each evolution round. Each round of data from CO C4 is about 52k. From the trend of this figure, it can be seen that as the complexity of the training instruction data gradually increases, the performance of the fine-tuned models also improves synchronously. To investigate the correctness of the difficulty score by ChatGPT, we also use GPT-4 and human to measure the instructions difficulty, the detailed results in the Table of Appendix inndicate the good agreement among the ChatGPT, GPT-4 and human annotators.
Analysis In-breadth Evolving. We aims to examine the semantic breadth instructions. We use t-SNE van der Maaten & Hinton (2008) and the k-means Hartigan & Wong (1979) algorithm to partition instructions BERT embeddings into 20 clusters. Figure in Appendix F displays clusters, highlighting our method's 's superior dispersion compared to ShareGPT and Alpaca, indicating greater topic diversity in our instructions.
5 CONCLUSIONS
This paper presented Evol-Instruct, an evolutionary algorithm that generates diverse and complex instruction data for LLM. Comprehensive experiments demonstrate that WizardL.M significantly surpasses typical open-source LLMs such as Alpaca and Vicuna in a wide range of well-recognized benchmarks. Notably, WizardLM outperforms baselines by a substantial margin in terms of code, math, GPT-4 and human evaluations.
Linitations. This paper acknowledges the limitations of automatic GPT-4 and human evaluation methods. This method poses challenges for scalability and reliability. Moreover, our test set may not represent all the scenarios or domains where LLM can be applied or compared with other methods.

---

Published as a conference paper at ICLR 2024
REFERENCES
Vamsi Aribandi, Yi Tay, Tal Schuster, Jinfeng Rao, Huaixiu Steven Zheng, Sanket Vaibhav Mehta, Honglei Zhuang, Vinh Q. Tran, Dara Bahri, Jianmo Ni, Jai Gupta, Kai Hui, Sebastian Ruder, and Donald Metzler. Ext5: Towards extreme multi-task scaling for transfer learning. In International Conference on Learning Representations, 2022. URL
wwtpeewwwwwww.eeeeeeeeeeeeeeeeee formm?
(d=V2h1BFUCC
Keqin Bao, Jizhi Zhang. Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. Tallrec: An effective and efficient tuning framework to align large language model with recommendation. ArXiv, aws/aa05..0////0 2023.
Edward Beeching, Clémentine Fourrier, Nathan Habib, Sheon Han, Nathan Lambert, Nazneen Rajani, Omar Sanseviero, Lewis Tunstall, and Thomas Wolf. Open Ilm leaderboard. https
11m L____edderbbarree 2023.
Ning Bian, Pei Yu Liu, Xianpei Han, Hongyu Lin, Yaojie Lu, Ben He, and Le Sun. A drop of ink may make a million think: The spread of false information in large language models. ArXiv, hass23055048122 2023.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Vivien A. Cabannes, Léon Bottou, Yann LeCun, and Randall Balestriero. Active self-supervised learning: A few low-cost relationships are all you need. ArXiv, 1ttp//233..15555 2023.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger. Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert- William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang. Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojeiech Zaremba. Evaluating large language models trained on code. CoRR, /tbbb//11.00377444 2021. URL httpp:://aaa org /ogg/bb//110 2tabb//210.. 03374
Zhihong Chen, Feng Jiang, Junying Chen, Tiannan Wang, Fei Yu, Guiming Chen, Hongbo Zhang, Juhao Liang, Chen Zhang, Zhiyi Zhang, Jianquan Li, Xiang Wan, Benyou Wang, and Haizhou Li. Phoenix: Democratizing chatgpt across languages. ArXiv, abs/2304.10053. 2023.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. URL https
//vicuna. imsys. org
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finenn language models. arXiv preprint arXiv:2210.11416 2022.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge, 2018.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reïchhrr Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110. 14168, 2021.
Adrian de Wynter, Xun Wang. Alex Sokolov, Qilong Gu, and Si-Qing Chen. An evaluation on large language model outputs: Discourse and memorization. ArXiv, abs/2304.08637, 2023.
10

---

Published as a conference paper at ICLR 2024
Leo Gao, Jonathan Tow, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster. Laurence Gold- ing, Jeffrey Hsu, Kyle McDonell, Niklas Muennighoff, Jason Phang, Laria Reynolds, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. A framework for few-shot language model evaluation, September 2021. URL htl.https://// / org/ / 10. 5281/ /wwp/ozeodooooooooo 5371 128
Zhen Guo, Peiqi Wang, Yanwei Wang, and Shangdi Yu. Dr. llama: Improving small language models in domain-specific qa via generative data augmentation. 2023.
A. Hartigan and M. A. Wong. A k-means clustering algorithm. JSTOR: Applied Statistics, 28(1): 100-108, 1979.
Xingwei He, Zheng- Wen Lin, Yeyun Gong, Alex Jin, Hang Zhang, Chen Lin, Jian Jiao, Siu Ming Yiu, Nan Duan, and Weizhu Chen. Annollm: Making large language models to be better crowdsourced annotators. ArXiv, abs/2303. 2023.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint S0l::20009003300, 2020.
Zhiqiang Hu, Yihuai Lan, Lei Wang, Wanyu Xu, Be-Peng Lim, Roy Ka-Wei Lee, Lidong Bing, and Soujanya Poria. LIm-adapters: An adapter family for parameter-effiient fine-tuning of large language models. ArXiv, abs/2304.01933, 2023.
Rongjie Huang, Mingze Li, Dongchao Yang, Jiatong Shi, Xuankai Chang, Zhenhui Ye, Yuning Wu, Zhiqing Hong. Jia-Bin Huang, Jinglin Liu, Yixiang Ren, Zhou Zhao, and Shinji Watanabe. Audio- gpt: Understanding and generating speech, music, sound, and talking head. ArXiv, abs/2304.12995. 2023.
Andreas Kopf. Yannic Kilcher, Dimitri von Rutte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Nguyen Minh Duc, Oliver Stanley, Rich ard Nagyfi, ES Shahul, Sameer Suri, David Glushkov, Arnav Dantuluri, Andrew Maguire, Christoph Schuhmann, Huu Nguyen, and Alexander Mattick. Openassistant conversations democratizing large language model alignment. ArXiv, abs/2304.0753 00722777.... 2023.
Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for "mind" exploration of large language model society, 2023a.
Jia Li, Ge Li, Yongming Li, and Zhi Jin. Enabling programming thinking in large language models toward code generation. 2023b.
Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpacaeval: An automatic evaluator of ffoloooiin models. httpp:: /ttp:://githule
waw.aaassaaaaaaaaaaaaaaa _aallaaaa_allal_________la_a 2023c.
Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods, 2022.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. ArXiv, abs/2304.08485, 2023.
Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoe V Le, Barret Zoph, Jason Wei, et al. The flan collection: Designing data and methods for effective instruction tuning. arXiv preprint arXiv:2301. 2023.
Ziyang Luo, Can Xu, Pu Zhao, Xiubo Geng, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. Augmented large language models with parametric knowledge guiding. ArXiv, abs/2305.0477. 2023.
Huan Ma, Changqing Zhang, Yatao Bian, Lemao Liu, Zhirui Zhang, Peilin Zhao, Shu Zhang, H. Fu, Qinghua Hu, and Bing Wu. Fairness-guided few-shot prompting for large language models, ArXiv, 132/, 2023.
11

---

Published as a conference paper at ICLR 2024
Potsawee Manakul, Adian Liusie, and Mark John Francis Gales. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. ArXiv, abs/2303.08896, 2023.
Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. Orca: Progressive learning from complex explanation traces of gpt-4, 2023.
OpenAI. Gpt-4 technical report, 2023.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 27730-27744, 2022.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter Liu. Exploring the limits of transfer learning with unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67, 2020. URL
http://immrr.rrg/arpee /vtpsrs/vv2//// 074.html
Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szezechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, De- bajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Teven Le Scao, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M Rush. Multitask prompted training enables zero-shot task generalization. In International Conference on Learning Representations, 2022.
Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Cox, Yiming Yang, and Chuang Gan. Principle-driven self-alignment of language models from scratch with minimal human supervision. ArXiv, abs/2305.03047, 2023.
Ekaterina Svikhnushina and Pearl Pu. Approximating human evaluation of social chatbots with prompting. ArXiv, abs/2304.05253 2023.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: instruction-following Ilama model. wwtaafodnddaaaeeeeeeeaaaaaaaaaae
httpp:://gt thub. com/ wwom/tatttt
2023.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 2023.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Ma- chine Learning Research, 9(86))257--265 2008.
//mrr.//rr.r/////// hawml..rrr/pe.....mm..m... www.orrg/ppae.. httg:/apers/v//
VannerraattenDaa.in html
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212. 10560, 2022a.
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, al. Super-nntturalinstruutiosss Generalization via declarative instructions on 1600+ nlp tasks. arXiv preprint arXiv:2204.07700 2022b.
Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, and Hanmaneh Hajishirzi. How far can camels go? exploring the state of instruction tuning on open resources. 2023.
Yufei Wang, Jiayi Zheng, Can Xu, Xiubo Geng, Tao Shen, Chongyang Tao, and Daxin Jiang. Knowda: All-in-one knowledge mixture model for data augmentation in sew-shot nlp. arXiv preprint arXiv:2206. 10265, 2022c.
12

---

Published as a conference paper at ICLR 2024
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoe Le. Finetuned language models are zero-shot learners. arXiv preprint 200.::2009.01652, 2021.
Wen Xiao, Yujia Xie, Giuseppe Carenini, and Pengcheng He. Chatgpt-steered editing instructor for customization of abstractive summarization. ArXiv, abs/2305.02 022483, 2023.
Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley. Baize: An open-source chat model with
parameter-efficient tuning on self-chat data, 2023.
Hanwei Xu, Yujun Chen, Yulun Du, Nan Shao, Yanggang Wang, Haiyu Li, and Zhilin Yang. Zeroprompt: Scaling prompt-based pretraining 1,000 tasks improves zero-shot generalization. arXiv preprint arXiv:2201.061 .06910, 2022.
Zheng Yuan, Hongyi Yuan, Chuangi Tan, Wei Wang, Songfang Huang. and Feiran Huang. Rrhf: Rank responses align language models with human feedback without tears. ArXiv, abs/2304.05302 2023.
Xiang Yue, Boshi Wang, Kai Zhang, Zi- Yuan Chen, Yu Su, and Huan Sun. Automatic evaluation of attribution by large language models. ArXiv, abs/2305.06311, 2023.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019.
Shujian Zhang, Chengyue Gong, Lemeng Wu, Xingchao Liu, and Mi Zhou. Automl-gpt: Automatic machine learning with gpt. ArXiv, abs/2305.0244 2023.
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang. Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong. Yifan Du Chen Yang, Yushuo Chen, Z. Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jianyun Nie, and rong Wen. A survey of large language models. ArXiv, abs/2303. 18223, 2023.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. arXiv preprini arXiv:2306.0568:
2023.
Shan Zhong, Zhongzhan Huang, Wushao Wen, Jinghui Qin, and Liang Lin. Sur-adapter: Enhancing text-to-image pre-trained diffusion models with large language models. ArXiv, abs/2335505188, 2023.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny, Minigpt-4: Enhancing vision-language understanding with advanced large language models. ArXiv, 11//2234411992 2023.
13

---

Published as a conference paper at ICLR 2024
A DEEPENING PROMPT
Example A.I: Prompt for Deepening of In-Depth Evolving
I want you act as Prompt Rewriter.
Your objective is to rewrite given prompt into a more complex version to make those famous Al systems (e.g.. ChatGPT and GPT4) bit harder handle. But the rewritten prompt must be reasonable and must be understood and responded by humans.
Your rewriting cannot omit the non-text parts such as the table and code in #Given Prompt#:. Also, please do not omit the input in #Given Prompt#.
You SHOULD complicate the given prompt using the following method: If #Given Prompt# contains inquiries about certain issues, the depth and breadth of the inquiry can be increased.
You should try your best not to make the #Rewritten Prompt# become verbose, #Rewritten Prompt# can only add 10 words into #Given Prompt#. *#Given Prompt#', '#Rewritten Prompt#', "given prompt' and 'rewritten prompt' are not allowed to appear in #Rewritten Prompt#
#Given Prompt#:
(Here is instruction,
#Rewritten Prompt#:
B CONCRETIZING PROMPT
Example B.1: Prompt for Concretizing of In-Depth Evolving
I want you act as Prompt Rewriter.
Your objective is rewrite given prompt into a more complex version to make those famous Al systems (e.g.. ChatGPT and GPT4) bit harder to handle. But the rewritten prompt must be reasonable and must be understood and responded by humans.
Your rewriting cannot omit the non-text parts such as the table and code in #Given Prompt#:. Also, please do not omit the input in #Given Prompt#t
You SHOULD complicate the given prompt using the following method: Please replace general concepts with more specific concepts.
You should try your best not to make the #Rewritten Prompt# become verbose, #Rewritten Prompt# can only add 10 20 words into #Given Prompt#t '#Given Prompt#', '#Rewritten Promptif, "given prompt' and 'rewritten prompt' are not allowed to appear in #Rewritten Prompt#
#Given Prompt#:
[Here is instruction.
#Rewritten Prompt#:
C INCREASED REASONING STEPS PROMPT
Example C.I: Prompt for Increased Reasoning Steps of In-Depth Evolving
I want you act Prompt Rewriter.
Your objective is rewrite given prompt into more complex version to make those famous Al systems (e.g., ChatGPT and GPT4) a bit harder to handle. But the rewritten prompt must be
14

---

Published as a conference paper at ICLR 2024
reasonable and must be understood and responded by humans.
Your rewriting cannot omit the non-text parts such as the table and code in #Given Prompti:. Also, please do not omit the input in #Given Prompt#.
You SHOULD complicate the given prompt using the following method: If #Given Prompt# can be solved with just few simple thinking processes, you rewrite it to explicitly request multiple-step reasoning.
You should try your best not to make the #Rewritten Prompt# become verbose, #Rewritten Prompt# can only add 10 20 words into #Given Prompt#. *#Given Prompt#', *#Rerriten Prompt#', "given prompt' and 'rewritten prompt' are not allowed to appear in #Rewritten Prompt#
#Given Prompt#::
(Here is instruction.)
#Rewritten Prompt#:
D COMPLICATE INPUT PROMPT
Example D.1: Prompt for Complicate Input of Evolving
I want you act as a Prompt Rewriter. Your objective is to rewrite a given prompt into a more complex version using dataformat make those famous AI systems (e.g., chatgpt and GPT4) more difficult to handle. But the rewritten prompt must be reasonable and must be understood and responded by humans. You must add [XML data] format text as input data in [Rewritten Prompt]
#Given Prompt#:
I'm using this php code fetch the xml data
#Rewritten Prompt#: I have this xml and want to get the xml data to auto populate HTML table, the codes works but it makes duplicate the table content
Here is the xml data:
<root>
<stats>
<item>
cday:201--1-0</dap>
<ing:eessooss22222////eessssssss
(monoyy1.99339003<mmeeeee
</item>
<item>
<day=2011-11100<</aa>>>>>
<honey23.228500033/mmmmee
</item>
<item>
<day>2001-111-03<ttpp>
sing=eassioaa23800////eeessssss
<mcneyys..221999911//mmmeey
</item>
</stats>
<total>
<inpressions28886//pppeessssssse
15

---

Published as a conference paper at ICLR 2024
</total>
<filter>
catterrom220171110044aaattoorr
Catteroo-001-11100344atttto
<groupEy>Gay<<grruppy>> <format>xml</oonn
</filter>
</root>
I'm using this php code to fetch the xm1 data but this code fetching from whole xml data which makes duplicate field table
<?php
\$dom new DOMDocument;
\$dom -> load
com/" \Sdateselected 'édateTo -! 15dateselected2 'sformat-xml")
\$day = Sstmm ('daa ('day');
\Vsimpressions \/domm>gett ( ('impressions)
echo "<table>"))
foreach (A (A\yday as \\mnodel)
{
foreach (1) Simpressions as \\$node2)
echo 'tr>>;
echo "<td>" WSnodel -> textContent "<td>";
echo "<td>" \\Snode2 -> textContent "<td>";
echo "<td>" V\Snode2 -> textContent. +0.5/1000 ."<td>";
echo '</tr>';
echo ( "</table>");
?>
Could anyone give hint how I can fix this? thank you
#$#
Example D.2: Prompt for Complicate Input of Evolving
I want you act as Prompt Rewriter. Your objective is to rewrite a given prompt into more complex version using dataformat make those famous Al systems (e.g., chatgpt and GPT4) more difficult handle. But the rewritten prompt must be reasonable and must be understood and responded by humans.
You must add [SQL database] format text input data in [Rewritten Prompt]
#Given Prompt#:
achieve the SQL query result
#Rewritten Prompt# (MUST contain a specific SQL database input): There table messages that contains data shown below:
16

---

Published as a conference paper at ICLR 2024
To Name Other_Col umns
1 A data__
2 A A d_data_2
A A_data_3
4 B B_data_1
B B_data_2
C C 1_
I rum a query select from messages group by name, will get the result as:
A A _ddaa__l1
4 B B d
C C_data_1
What query will return the following result?
3 A A data_3
B data_222222
6 C_data_1
That is, the last record each group should be returned. At present, this is the query that I use:
SELECT
FROM (SELECT
FROM messages
ORDER BY id DESC) AS x GROUP BY name
But this looks highly inefficient. Any other ways to achieve the same result?
####
Example D.3: Prompt for Complicate Input Evolving
I want you act Prompt Rewriter. Your objective is to rewrite given prompt into more complex version using dataformat to make those famous Al systems (e.g., chatgpt and GPT4) more difficult handle. But the rewritten prompt must be reasonable and must be understood and responded by humans.
You must add [python code] format text as input data in [Rewritten Prompt]
#Given Prompt#: Transformat python code
#Rewritten Prompt# (MUST contain a specific python code as input): have the following Python code:
cursor execute ("INSERT INTO table VALUES varl, var2, var3,"
where varl is an integer, var2 and var3 are strings. How can I write the variable names without Python including them as part the query text?
#*##
17

---

Published as a conference paper at ICLR 2024
Example D.4: Prompt for Complicate Input Evolving
I want you act Prompt Rewriter. Your objective is to rewrite given prompt into more complex version using dataformat to make those famous Al systems (e.g., chatgpt and GPT4) more difficult handle. But the rewritten prompt must be reasonable and must be understood and responded by humans.
You must add [HTML page] format text input data in [Rewritten Prompt]
#Given Prompt#:
scroll through the whole HTML page
#Rewritten Prompt# (MUST contain specific HTML page input): I want be able scroll through the whole page, but without the scrollbar being shown.
In Google Chrome it's:
-webbit-screllbar { display: none;
But Mozilla Firefox and Internet Explorer don't t seem to work like that.
I also tried this in CSS:
overflow: hidden;
That does hide the scrollbar, but I can't scroll any more. Is there a way can remove the scrollbar while still being able to scroll the whole page?
With just CSS or HTML, please. #*##
Example D.5: Prompt for Complicate Input of Evolving
want you act as a Prompt Rewriter. Your objective to rewrite a given prompt into a more complex version using dataformat make those famous Al systems (e.g., chatgpt and GPT4) more difficult to handle. But the rewritten prompt must be reasonable and must be understood and responded by humans.
You must add [Shell cmd] format text input data in [Rewritten Prompt]
#Given Prompt#: Shell scp file
#Rewritten Prompt# (MUST contain a specific Shell emd as input):
I'm trying to sep file from remote server to my local machine. Only port 80 is accessible.
I tried:
sep -p 80 username wwwnaamee@www...... www.nyerreercom//ooolilee..
but got this enror: cp: 80: No such file directory
How do specify the port number in scp command?
#*##
Example D.6: Prompt for Complicate Input of Evolving
I want you act as a Prompt Rewriter. Your objective is to rewrite a given prompt into a more complex version using dataformat make those famous AI systems (e.g., chatgpt and GPT4) more difficult to handle. But the rewritten prompt must be reasonable and must be understood and
18

---

Published as a conference paper at ICLR 2024
responded by humans.
You must add [JSON data] format data as input data, add [JSON data] code input code in [Rewritten Prompt]
Rewrite prompt must be a question style instruction
#Given Prompt#:
Given a JSON dataset of customer purchase history, how can we calculate the probability of customer making a repeat purchase from the same store? Can utilize the formula for conditional probability: P[A] B) = P(An B) ((B) where represents the event of customer making repeat purchase and B represents the event of customer making purchase from the same store again? Additionally, how can we apply this formula to identify the customer segment that is most likely make repeat purchase? Can you provide example of how implement this formula using the given JSON dataset?
Rewritten prompt must be question style instruction #Rewritten Prompt# (MUST contain a specific JSON data as input):
E DIFFICULTY JUDGE PROMPT
Example E.1: Prompt for Juding the Difficulty of Instructions
We would like you to evaluate and rate the difficulty and complexity of the following question. You should give an overall score on scale of to 10, where a higher score indicates higher difficulty and complexity. You must just give score without any other reasons. ## Question:
Here is instruction, }
## Score:
F EQUAL PROMPT
Example F.1: Prompt for Determining whether Two Instructions are Equal
Here Instructions to ChatGPT AI, do you think they are equal each other, which meet the following requirements:
They have same constraints and requirments.
2. They have same depth and breadth of the inquiry.
The First Prompt: (Here is first instruction.
The Second Prompt: (Here is second instruction.)
Your Judgement (Just answer: Equal or Not Equal. No need explain the reason.):
G MATH JUDGEMENT PROMPT
Example G.1: Prompt for judging whether an instruction math related
Please judge whether the following question is math problem, and only return True False without providing any explanation.
Question: finstruction)
H WIZARDEVAL ANALYSIS
We collected our Evol-Instruc testset that includes real-world human instructions from diverse sources such as online opensource projects, platforms, and forums. We analyzed the data and identified 29 distinct skills that represent the main requirements of humanity, such Coding Generation & Debugging, Math, Reasoning, Complex Formats, Writing, Extensive Disciplines. and so on. Figure 6 illustrates the distribution of the instances and skills in test set. Our test set consists of 218
19

---

Published as a conference paper at ICLR 2024
instances, each of which is an instruction for specific skill. We compared our test set with Vicuna's test set, which is a benchmark dataset for evaluating instruction following models. We found that Vicuna's test set only 80 instances and skills and is much smaller and less diverse than ours. Figure 4a shows how the difficulty and complexity of the test data vary across different instances. Our test data has more uniform distribution, meaning that it contains instructions with different levels of difficulty and complexity. On the other hand, Vicuna and Alpaca have a skewed distribution, meaning that they mostly contain instructions with low difficulty and complexity. This indicates that these two corpus are not able to handle the evaluation on more complex and demanding scenarios.
20
Count
Baria Scheove
peophee Fermas
Canls Denop
tommo Serrue
Cousderfastiun Musstinpuo ot
Raloyptay
Bistogy
Zecenntonyy
Stries
Spece
Entertainmmett
Music
antoric siptiog
Chemvstty
peotousphyy
Figure 6: The skills distribution of Evol-Instruct testset.
I DIFFERENT DIFFICULTY ANNOTATORS
We just use the ChatGPT to post analyse the difficcul distribution of the generated instructions, but we do not use this analysis results to guide the data generation or model training. In order to explore the ability of ChatGPT to perform difficulty analysis, we sample 600 instructions and use the more powerful GPT4 model and 5 well-educated human annotators together for difficulty assessment. The assessment results are the 3 The results show that ChatGPT, GPT4, and manual annotation show high degree of consistency in the trend of difficulty changes.
ShareGPT
Alpuca
C1
C2
C3
C4
GPT-3.5
4.63
3.00
5.48
6.35
6.84
7.08
GPT-4
4.31
2.69
4.68
4.90
5.37
5.54
Human
4.55
3.15
5.51
5.86
6.49
6.82
Table 3: Use ChatGPT, GPT-4, human measure the instruction difficulty.
To investigate the correctness of the difficulty score by ChatGPT, we add experiment to measure agreement of difficulty judge between ChatGPT and humans: We randomly select two instructions from the six datasets Alpaca, ShareGPT, C1 C4- with equal probability each time, forming pair. In total, we have selected 300 instruction pairs. Then, we ask ChatGPT and 5 well-educated human annotators judge which one is more difficulty in one instruction pair, the Kappa score between humans is 0.68, and the Kappa between ChatGPT and human (majority voting) is 0.66, which indicates the good agreement among the ChatGPT and human annotators.
J CLUSTER SCATTER PLOT
In-breadth Evolving aims to enhance topic coverage, skill coverage, and overall dataset diversity. To examine (qualitative analysis) the breadth (diversity) of different dataset, we firstly use BERT to
20

---

Published as a conference paper at ICLR 2024
encode each instruction and get its embedding with 768 dimensions, then use dimension reduction algorithm named t-SNE to reduce embedding dimension to 2, finally we apply a clustering algorithm k-means to partition the instructions of each dataset into 20 clusters for intuitive visualization. As shown in the Figure 7 the data points of our dataset are more dispersed than ShareGPT and Alpaca (Seif-Instruct), which indicates the better topic diversity in instructions.
ShareGPT
Alpaca
C2
C3
C4
Figure 7: The cluster scatter plot between ShareGPT, Alpaca, and ours four rounds of instruction evolution from C1 to C4. The number of cluster centers is 20,
K HUMAN EVALUATION ASPECTS
The annotators then judge which response better from five aspects:
(1) Relevance: Assessing the model's ability to correctly interpret the semantic meaning of the context and questions.
(2) Knowledgeable: Whether the model can accurately use various and detailed knowledge for problem-solving.
(3) Reasoning: Assessing the model's ability to execute correct reasoning processes or devise valid reasoning concepts to solve problems.
(4) Calculation:
Evaluating whether the model can perform accurate mathematical computations of the provided formulas in the domains of math, biology, chemistry and physics.
(5) Accuracy: Evaluating whether the model can perform correctly in the corresponding for given instruction.
L PERFORMANCE DETAILS OF DIFFERENT CHECKPOINTS
In this paper, we train our model with 3 epochs and only reported the performance of final checkpoint in the above "Section 4 Experiment" to align with previous works.
As shown in the following Table 4 we report the model checkpoints performance on different epochs (2.5, 2,75, 3). For 13B models, we can see that the best performance always appears WizardLM- 13b (ShareGPT Seed) for each benchmark except GSM8k. And for 65b/70b models, we also see that the M-zordLL-70b is the best one all the benchmarks. Therefore, think this is mainly caused by the fluctuations on some benchmarks in model training.
21

---

Published as a conference paper at ICLR 2024
Fpuch
AlpacaFvol
WizurlEvoal
WitardLM:I
57.92
74.09
WizardLM
58.24
50.64
25.5
Winnil 14-13
3.0
58.96
52.92
52.35
50.58
90.55
24.0
37.15
75.31
6.35
89.1
Wicnll Monll-M--110 (SheeeSS Suxth
2.50
61.45
51.76
60.02
$1.53
53.24
25.3
31.83
85.71
6.52
95.7
WicniLMM BS. Suxih
2.75
62.00
53.10
S1.53
79.77
54.1
27.2
33.04
86.65
6.65
99.0
WicaiLMM:1bb 1 (Sue:CPPP Seeih
3,0
61.87
50.92
60.24
$1.39
54.56
25.0
31,46
86.32
6.76
99.3
WizardLMi 6sh
2.50
68.12
60.50
63.24
61.11
50.55
35.8
66.01
86.09
7.06
95.8
Wicarl M-45h
2.75
69.80
62.84
63.51
$3.36
$7.1
67.46
89.66
7.20
96.9
WizardL M-65h
69:40
62.09
63.33
55.88
53.19
56.5
66.30
87.50
7.12
97.5
WizardLM 70b
2.50
71.22
61.35
66.31
35.60
54.76
41.3
68.70
87.73
7.53
99.4
WizardLM 70b
2.75
71.08
63.44
64.39
$4.06
53.21
42.4
09:53
89.09
7.38
99.3
WizardLM 70b
3.0
71.33
03.32
64.52
53.21
54,50
42.1
70,61
89.33
7,46
99.7
Table 4: Performance details of different checkpoints.
22


Reviews:
[{"review_id": "k3GRtuH3mH", "reviewer": "Reviewer_JU8z", "summary": "The authors propose a novel pipeline for creating large amounts of instruction data with varying levels of complexity using LLM. The proposed approach starts from a set of seed instructions and first uses Evol-Instruct, a suit of prompt templates that can make LLMs such as ChatGPT to rewrite seed instructions into more complex ones. Then all generated data are used for fine-tuning open-source LLMs such as LLaMA. The authors used the proposed method to collect a set of instruction data and fine-tuned LLaMA into \"WizardLM\", and conduct a suit of evaluation on various benchmarks. Experimental results show some improvement over representative baselines including Alpaca and Vicuna models.", "questions": "Please see the above weakness section for questions.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The idea of using LLMs to rewrite and synthesize more complex instructions is interesting and intuitive.\n2. The paper is overall well-written and easy to follow. \n3. The authors evaluate WizardLM on a wide range of datasets/benchmarks and the experimental results look promising.", "weaknesses": "1. The technical contribution of the proposed method is not very significant because compared to self-instruct, it is only adding the command for LLM to generate more complex instruction. The success of the proposed method largely depend on the abilities of powerful LLMs such as ChatGPT. While interesting and intuitive, I'm not sure the technical contribution of the manuscript is suitable for conferences such as ICLR.\n\n2. The authors compare WizardLM with Vicuna by using the same amount of generated instructions. However, the cost or token consumption used for collecting the datasets (especially compared with Alpaca) should also be controlled or at least mentioned. \n\n3. The manuscript lacks comparisons with other methods for instruction data generation methods such as Baize, CAMEL, etc. For benchmark results, the role of the seed instructions is very important. I assume the seed data is very different, which could be a major cause of the performance difference. Therefore, more detailed ablation study is required to make the results more convincing.\n\n4. Also, it would be very helpful to test the proposed data generation methods using other LLMs (e.g., LLaMA) instead of ChatGPT to better understand the proposed data synthesis pipeline.\n\n4.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yIEXyUmvbB", "reviewer": "Reviewer_fF8a", "summary": "The paper argues that human written instruction tuning data as well as GPT-generated self-instruct data lack difficulty and diversity. To address this issue, they propose a procedure to modify existing self-instruct data of Alpaca and make them more diverse and more difficult. The results show that after finetuning on this dataset, the resulting model outperforms Alpaca and Vicuna on a number of standard academic datasets and a small WizardEval dataset.\n\n----\nThe authors have satisfactorily resolved my concerns during the rebuttal. Hence, I increased the score from 5 to 6.", "questions": "- How do we know the answers to the newly created prompts are correct?\n- How do we know the difficulty and diversity scores by ChatGPT are correct? Please do not say because other papers did similar things. The other papers could be wrong, too.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper identifies an important problem: the instruction tuning data lacks diversity and difficulty. I agree this is indeed a problem.\n\nThe reported evaluation results are good, and lends support to the claim that the new dataset is better than baselines.", "weaknesses": "## Major Issues ##\n\nThe authors did not perform a thorough evaluation of the created dataset. \n- The authors do not compare against diverse instruction tuning datasets such as Natural Instructions, Supernatural Instructions, and the training data of FLAN-T5. \n- The experiments focus on Llama and ignore other LLMs such as T5, Falcon, Mistral (which has a model not instruction tuned), and so on. \n- The evaluation of difficulty and diversity by ChatGPT is unconvincing, as the paper presents no evidence that ChatGPT is good at these evaluations. It is also dubious to report only the average score of difficulty and diversity, as these are clearly different concepts. \n- The WizardEval dataset is too small and the paper contains little detail about its construction, so we do not know if the dataset can serve as a sound and fair evaluation benchmark. \n- I do not understand the argument in the section Analysis of In-breadth Evolving. \n\nOne possible evaluation of difficulty and diversity is to divide all the training data into several subsets with increasing difficulty / diversity. If ChatGPT is capable of evaluating the training data, then perhaps more difficult and more diverse training data will lead to better performance. Note that this is different from the current grouping based on evolution iterations, which includes a confounding variable: the number of evolution iterations. \n\n## Minor Issues ##\n\nThe prompts to ChatGPT contain a few grammatical issues. For example, \"Your objective is to rewrite a given prompt into a more complex version to make those famous AI systems (e.g., ChatGPT and GPT4) a bit harder to handle\" should be \"a more complex version which those famous AI systems (e.g., ChatGPT and GPT4) find a bit harder to handle\". It is the prompt that should be hard to handle, not the AI systems. \n\n\"But the rewritten prompt must be reasonable and must be understood and responded by humans.\" Here, whether the prompt is responded to by humans or by AIs cannot be controlled by the prompt generation network. The authors probably meant that it is possible for humans to respond correctly to the prompt. \n\nHowever, these are likely minor issues because ChatGPT may not have sufficient logic reasoning to understand the subtle differences. \n\n## Writing ##\n\nThe paper makes extensive citations to related papers, including many that are recently posted on arxiv. Though it is good to acknowledge similar work, sometimes the cited papers do not support or relate to the text. For example, in the following sentence：\n\nHuman annotators are prone to fatigue and cannot sustain high-intensity work to produce a sufficient\nproportion of high-difficulty instructions (Zhang et al., 2023; Xiao et al., 2023; Manakul et al., 2023;\nZhong et al., 2023)\n\nThe provided citations are unrelated to whether human annotators can sustain high-intensity work. This may create the impression that the paper attempts to mislead the reader, though it is not the authors' intention to do so. \n\nElimination evolving is a vague and ungrammatical name. I suggest renaming this step. Perhaps failure handling or incorrect prompt filtering?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Im6wBBi0Le", "reviewer": "Reviewer_tNq7", "summary": "The paper explores the challenges large language models (LLMs) face in following user-specified instructions. To address these issues, the authors introduce a novel approach, \"Evol-Instruct,\" which uses LLMs to automatically produce open-domain instructions at various complexity levels. By starting with a base set of instructions, the Evol-Instruct method evolves them to increase complexity, either deepening existing instructions or introducing new ones. The resulting instructions are then used to fine-tune the LLaMA model, producing a new model named WizardLM. Through both automatic and human evaluations, WizardLM demonstrated superior performance compared to other models like Alpaca and Vicuna.", "questions": "- How do you ensure that the evolved instructions maintain their intended purpose and don't deviate from the original goal?\n- Are there any measures to prevent potential biases introduced during the instruction evolution process?\n- How does WizardLM handle instructions that are not part of the evolved set?\n- Would incorporating more human feedback during the instruction evolution process help in refining the quality of instructions? If so, how?\n- How scalable is the Evol-Instruct process for even larger models or datasets in the future?\n- What’s your opinion of learning with Evol-Instruct and in-context learning with demonstrations? The correlations and differences, and the pros and cons.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Innovative Approach: The Evol-Instruct method offers a fresh perspective on generating complex instructions without relying solely on human input.\n- Comprehensive Evaluation: The authors provide both automatic and human evaluations to assess the performance of WizardLM.\n- Broad Applicability: WizardLM's superior performance in varied benchmarks, including code, math, and general conversation, suggests its broad applicability.\n- Consideration of Instruction Complexity: The paper highlights the significance of instruction complexity in enhancing LLM performance.", "weaknesses": "- Reliance on LLMs: The Evol-Instruct method's dependence on LLMs may introduce biases from the original training data.\n- Uncertainty in Evolution Direction: The random selection between In-depth and In-breadth evolving may not always yield the optimal instruction complexity.\n- Potential Overfitting: The process of evolving instructions multiple times might risk overfitting the model to specific types of instructions.\n- Paper Organization: This paper is not well-organized enough, especially for “3 APPROACH”. For example, the “Prompts of In-Depth Evolving” and “Prompts of In-Breadth Evolving” should contained within “Instruction Evolver”, but are listed in a parallel format. There is limited content in “Response Generation”.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5fDpmmZfcl", "reviewer": "Reviewer_HVLz", "summary": "The paper introduces Evol-Instruct, a data augmentation pipeline to automatically generates diverse and complicated instructions and responses using LLMs. By instruction-tuning a Llama model on the generated instances (the resulting model is referred to as WizardLM), the paper shows that WizardLM outperforms Vicuna and Alpaca on automatic benchmarks and human evaluation.", "questions": "- For human evaluation, how many labelers labeled the same instances? Could you share the exact inter-labeler agreement between human labelers? \n- Does using the full 250K instructions lead to better performance on the automatic evaluation setting?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The proposed method is straightforward and simple. \n- The paper conducts both automatic and human-based evaluation, increasing the reliability of the effectiveness of the proposed method.", "weaknesses": "- The paper lacks the number of baselines for comparison. For example, there are recent LLaMA-based instruction-tuned models such as Tulu (Wang et al, 2023) and Orca (Mukherjee et al, 2023). However, the paper does not compare the performance nor discuss the related papers. \n- The process of generating training instances largely relies on a single LLM (ChatGPT). If the prompts and design choices are optimized towards a single blackbox LLM, the transferability to other LLMs is questionable. Is the proposed data generation method also effective for other LLMs using the same experimental design (Using LLMs other than ChatGPT for training data generation)? \n- In Figure 5, the paper calculates the difficulty of the instruction by simply using ChatGPT. However, the instructions are also generated by the same LLM by prompting to make the previous instructions harder. Therefore, the validity of the difficulty estimation used in this paper is questionable. To test whether generated instructions \"actually\" become more challenging, the paper should calculate the difficulty by using different evaluators or other metrics. \n\nReference:\nWang et al, 2023: How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources\nMukherjee et al, 2023: Orca: Progressive Learning from Complex Explanation Traces of GPT-4", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)