Paper: WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex Instructions (ICLR-2024)

Reviewer comment (Experiments):
The paper lacks the number of baselines for comparison. For example, there are recent LLaMA-based instruction-tuned models such as Tulu (Wang et al, 2023) and Orca (Mukherjee et al, 2023). However, the paper does not compare the performance nor discuss the related papers.

Author response:
As shown in the table in the section of **New Baselines** of General Response, we add more baselines such as Baize, CAMEL, Tulu. The extensive results indicate that our WizardLM also significantly surpasses these open source models, which demonstrates that the convincing quality of instruction-following dataset crafted by Evol-Instruct. We don't report the Orca due to it's not an open-source model. We add discussion about Orca in the relate work of updated main submission.

Reviewer comment (Experiments):
The process of generating training instances largely relies on a single LLM (ChatGPT). If the prompts and design choices are optimized towards a single blackbox LLM, the transferability to other LLMs is questionable. Is the proposed data generation method also effective for other LLMs using the same experimental design (Using LLMs other than ChatGPT for training data generation)?

Author response:
To verify that our designed prompt and method can be transferred to other LLMs, we chose LlaMA-2-70B-Chat as a substitute for ChatGPT. Following the same prompts and design choices outlined in the paper, we re-executed the entire evolution process starting from Alpaca data and trained the WizardLM-13b(LlaMA-2-70B-Chat) model with randomly selected 70k data. Please refer to below table for details, although LlaMA-2-70B-Chat's ability is weaker than ChatGPT, WizardLM-13b(LlaMA-2-70B-Chat) still significantly outperforms Alpaca-13b trained solely on Alpaca data and even outperforms Vicuna-13B trained with much more ChatGPT tokens.

Model | Avg. | MMLU | ARC | HellaSwag  | TruthfulQA  |   HumanEval  | GSM8k| AlpacaEval  | MT-Bench  | WizardEval   
| ----- |------| ---- |------|-------| ----- |  ----- | ----- |----- |----- |----- |
|   Alpaca-13b   | 43.44 | 46.63 | 51.20  | 76.31 |41.62 | 9.2 | 8.35| 33.25 |  4.78  | 76.6  
|   Vicuna-13b   | 54.60  | 50.84 |  51.71 | 79.94 | 52.68  | 12.5  | 24.34 | 70.43  | 6.21 | 86.9 
| WizardLM-13b | 58.96 | 52.92 |  57.25 | 80.88  |50.55 | 24.0 | 37.15 | 75.31  | 6.35  | 89.1
| WizardLM-13b(LlaMA-2-70B-Chat Evol)  |  56.27|  51.09 | 57.34 | 79.12 | 48.76 | 19.5 | 33.83 | 70.47 | 6.18   | 84.5

Reviewer comment (Evaluation):
In Figure 5, the paper calculates the difficulty of the instruction by simply using ChatGPT. However, the instructions are also generated by the same LLM by prompting to make the previous instructions harder. Therefore, the validity of the difficulty estimation used in this paper is questionable. To test whether generated instructions "actually" become more challenging, the paper should calculate the difficulty by using different evaluators or other metrics.

Author response:
In order to explore the ability of ChatGPT to perform difficulty analysis, we sample 600 instructions (each dataset has 100 instructions) and use the more powerful GPT4 model and 5 well-educated human annotators together for difficulty assessment. The assessment results are in the following Table. The results show that ChatGPT, GPT4, and manual annotation show a high degree of consistency in the trend of difficulty changes. 

| |   ShareGPT |   Alpaca |   C1 |   C2 |  C3  | C4 | 
 | ----- |------| ---- |------|-------| ----- | ----- |
 | GPT-3.5 |   4.63 |   3.0 |   5.48 |   6.35 |  6.84  | 7.08 | 
 | GPT-4   |  4.31  |  2.69  |  4.68   |  4.9  |  5.37  |  5.54 | 
| Human   |  4.55  |  3.15  |  5.51   |  5.86  |  6.49  |  6.82 |

Reviewer comment (Evaluation):
For human evaluation, how many labelers labeled the same instances? Could you share the exact inter-labeler agreement between human labelers?

Author response:
As mentioned in the Section 4.4 Human Evaluation, we recruit 10 well-educated annotators to judge the whole test set. All of the Kappa scores are greater than 0.6 (WizardLM vs. Alpaca is 0.67, WizardLM vs. Vicuna is 0.64, WizardLM vs. ChatGPT is 0.63), which indicates the good agreement among the annotators.

Reviewer comment (Evaluation):
Reliance on LLMs: The Evol-Instruct method's dependence on LLMs may introduce biases from the original training data.

Author response:
We agree with your concern about the use of LLMs. We consider the elimination of potential biases introduced in evolution as an important direction for our research. A feasible method is to use human feedback as a signal to alleviate the inherent bias of the LLM during the evolution process. Although LLM has the issue of bias, the advantage of using it instead of humans for instruction evolution is that the cost is low enough and the speed is faster.

Reviewer comment (Theory):
Uncertainty in Evolution Direction: The random selection between In-depth and In-breadth evolving may not always yield the optimal instruction complexity.

Author response:
Yes, this version of Evol-Instruct neglects the detailed control over the direction and topics of evolution, how to make it controllable and in line with specific goals is a topic worth research. For this paper, we mainly want to demonstrate that evolving instructions can improve the supervised fine-tuning performance of pre-trained LLM.

Reviewer comment (Experiments):
Potential Overfitting: The process of evolving instructions multiple times might risk overfitting the model to specific types of instructions.

Author response:
Yes, executing the Evol Instruct multiple times may lead to getting stuck in a specific topic and not being able to cover a wider range of topics. This is also why we propose the branch of In-Breadth Evolving in addition to In-Depth Evolving, which is an attempt to solve this problem. From Chapter 4.5 of the main submission and Appendix F, it can be seen that the diversity of topics in our evolved instruction data is actually better than the original data. How to further improve the problem of overfitting is a topic worth continuous exploration.

Reviewer comment (Presentation):
Paper Organization: This paper is not well-organized enough, especially for “3 APPROACH”. For example, the “Prompts of In-Depth Evolving” and “Prompts of In-Breadth Evolving” should be contained within “Instruction Evolver”, but are listed in a parallel format. There is limited content in “Response Generation”.

Author response:
Thanks for your kindly advise, we have reorganized and updated our paper about section "3 APPROACH".

Reviewer comment (Evaluation):
How do you ensure that the evolved instructions maintain their intended purpose and don't deviate from the original goal?

Author response:
- (1) To ensure the accuracy of the evolution results, we did a lot of case study and refine the prompts, all the authors have tried almost 300 cases, and the quality meet ours requirements.
- (2) We also propose a "Elimination Evolving" method, which uses ChatGPT to measure whether the evolved instructions  "have same constraints and requirments" and  have "same depth and breadth of the inquiry" compared to the original one, if they are equal, we will eliminate it.

**detailed response:** 

- *(1) To ensure the accuracy of the evolution results, we did a lot of case study and refine the prompts, all the authors have tried almost 300 cases, and the quality meet ours requirements. For example, we find that the Complicating Input of In-Depth Evolving is difficult for ChatGPT to accurately perform evolution in a zero-shot setting. We adopt a few-shot approach, by providing a few appropriate examples, to ensure that this evolutionary method can work properly with ChatGPT.*
- *(2) We also propose a "Elimination Evolving" method, which uses ChatGPT to measure whether the evolved instructions  "have same constraints and requirments" and  have "same depth and breadth of the inquiry" compared to the original one, if they are equal, we will eliminate it.*

Reviewer comment (Experiments):
Are there any measures to prevent potential biases introduced during the instruction evolution process?

Author response:
We fully agree with you that the evolution may introduce some potential biases. We propose two new methods: 
- (1) Use ChatGPT to measure if there is any toxic or bias in the new instruction and answer 
- (2) Use RLHF to improve alignment. 

**detailed response:** *We fully agree with you that the evolution may introduce some potential biases. We propose two new methods: (1) Use ChatGPT to measure if there is any toxic or bias in the new instruction and answer (2) Sample some evolved instructions and hire human annotators mark out the possible types of bias in them. Based on this data, train a classifier to filter out those biased evolved instructions during the evolution process.*

Reviewer comment (Experiments):
Would incorporating more human feedback during the instruction evolution process help in refining the quality of instructions? If so, how?

Author response:
Sure, human feedback is important in the evolution process. What we refer to as "human-in-the-loop evolution" aligns very well with your idea of "incorporating more human feedback during the instruction evolution process helps in refining the quality of instructions". This is a direction we are currently exploring. Similar to OpenAI's RLHF method, we evolve an instruction multiple times, hire people to score and rank the evolved instructions. Then, we train a reward model for instruction quality based on this human-annotated data. When executing Evol-Instruct, we select high-scoring instructions based on this reward model and set the weight of the gradient during fine-tuning according to the score. Cases with high instruction scores have a larger gradient update weight, while cases with low scores have a smaller gradient update weight.

Reviewer comment (Experiments):
How scalable is the Evol-Instruct process for even larger models or datasets in the future?

Author response:
Besides the WizardLM-13b (52K self-instruct data as the seed), we add the experiments of three new trained wizard models: lager models WizardLM 65b and WizardLM 70b which is trained from Llama-1 65 and Llama-2 70B, and WizardLM-13b (70K ShareGPT data as the seed). The results in the following table indicate that our Evol-Instruct shows scalable performance for larger models or datasets.


Model | Avg. | MMLU | ARC | HellaSwag  | TruthfulQA  |   HumanEval  | GSM8k| AlpacaEval  | MT-Bench  | WizardEval   
 | ----- |------| ---- |------|-------| ----- |  ----- | ----- |----- |----- |----- |
 | WizardLM-13b | 58.96 | 52.92 |  57.25 | 80.88  |50.55 | 24.0 | 37.15 | 75.31  | 6.35  | 89.1
| WizardLM-13b (ShareGPT Seed) | 61.87 | 50.92 |  60.24 | 81.39 | 54.56 | 25.0 | 31.46 | 86.32 | 6.76 | 99.3
 | WizardLM-65b   |   69.40 |   62.09 |  65.83 |  85.48 |  52.19 |  36.5 |  66.39 |  87.50 |  7.12 |  97.5 | 
  |  WizardLM-70b   |   71.33 |   63.32 |  64.52 |  83.21 |  54.60 |  42.1 |  70.61 |  89.32 |  7.46 |  99.7 |

Reviewer comment (Theory):
What’s your opinion of learning with Evol-Instruct and in-context learning with demonstrations? The correlations and differences, and the pros and cons.

Author response:
In-context learning can allow Large Language Model (LLM) to learn a new task with only a few examples given. For tasks like Evol-Instruct that make instructions more complex, which are not common in the training data for LLM, enabling in-context learning is expected to help LLM perform better in evolution. In fact, in our In-Depth Evolving of Complicating Input, we use in-context learning to enhance the evolution effect. For details, please refer to Appendix D. In-context learning is a process where the LLM learns a new task based solely on a few given examples, without doing any Back Propagation (BP). Evol-Instruct is aimed at constructing complex instructions to enhance the performance of large pre-trained language model supervised fine-tuning. Therefore, for learning a new task, the advantage of in-context learning is that it requires very few resources, while Evol-Instruct requires more resources but can achieve better results.

Reviewer comment (Experiments):
The authors do not compare against diverse instruction tuning datasets such as Natural Instructions, Supernatural Instructions, and the training data of FLAN-T5.

Author response:
For a more comprehensive comparison and study, we extract 70k data from the Supernatural Instructions dataset and fine-tune llama-13b with the same training hyperparameters as ours, resulting in the model LlaMA(SNI). The experimental results, as shown in above Table, indicate that LlaMA(SNI) performs well on classic QA benchmarks, such as MMLU, surpassing Alpaca, Vicuna, and WizardLM. However, its performance is less impressive on math, code and open-domain test-beds like MT-Bench. 

Model | Avg. | MMLU | ARC | HellaSwag  | TruthfulQA  |   HumanEval  | GSM8k| AlpacaEval  | MT-Bench  | WizardEval   
 | ----- |------| ---- |------|-------| ----- |  ----- | ----- |----- |----- |----- |
 | WizardLM-13b | 58.96 | 52.92 |  57.25 | 80.88  |50.55 | 24.0 | 37.15 | 75.31  | 6.35  | 89.1
| LlaMA(SNI) |  37.73|  54.9|  54.95|  80.4|  38.69|  4.2|  5.79|  13.67 |  2.86 | 58.4

Reviewer comment (Experiments):
The experiments focus on Llama and ignore other LLMs such as T5, Falcon, Mistral (which has a model not instruction tuned), and so on.

Author response:
To verify whether our proposed Evol-Instruct can be applied to pre-train models outside of Llama, we chose to train Mistral-7B on Alpaca data and the evolved Alpaca data respectively. The results of the benchmarks are shown in the following table.

Model | Avg. | MMLU | ARC | HellaSwag  | TruthfulQA  |   HumanEval  | GSM8k| AlpacaEval  | MT-Bench  | WizardEval   
 | ----- |------| ---- |------|-------| ----- |  ----- | ----- |----- |----- |----- |
 | Alpaca (Mistral) | 52.87 | 56.34 |  55.38 | 79.49  |43.92 | 19.2 | 32.05 | 54.26  | 5.47  | 80.5
| WizardLM (Mistral) |  65.81 |  60.7|  57.47 |  82.08|  51.79|  37.8|  59.49 |  80.7 |  7.1 | 91.3

Reviewer comment (Evaluation):
The evaluation of difficulty and diversity by ChatGPT is unconvincing, as the paper presents no evidence that ChatGPT is good at these evaluations. It is also dubious to report only the average score of difficulty and diversity, as these are clearly different concepts.

Author response:
We just use the ChatGPT to post analyse the "difficulty" distribution of the generated instructions, but we do not use this analysis results to guide the data generation or model training. In order to explore the ability of ChatGPT to perform difficulty analysis, we sample 600 instructions (each dataset has 100 instructions) and use the more powerful GPT4 model and 5 well-educated human annotators together for difficulty assessment. The assessment results are in the following Table. The results show that ChatGPT, GPT4, and manual annotation show a high degree of consistency in the trend of difficulty changes. 

| |   ShareGPT |   Alpaca |   C1 |   C2 |  C3  | C4 | 
 | ----- |------| ---- |------|-------| ----- | ----- |
 | GPT-3.5 |   4.63 |   3.00 |   5.48 |   6.35 |  6.84  | 7.08 | 
 | GPT-4   |  4.31  |  2.69  |  4.68   |  4.90  |  5.37  |  5.54 | 
| Human   |  4.55  |  3.15  |  5.51   |  5.86  |  6.49  |  6.82 | 

To investigate the correctness of the difficulty score by ChatGPT, we add a new experiment to measure agreement of difficulty judge between ChatGPT and humans: We randomly select two instructions from the six datasets - Alpaca, ShareGPT, C1 to C4 - with equal probability each time, forming a pair. In total, we have selected 300 instruction pairs. Then, we ask ChatGPT and 5 well-educated human annotators to judge which one is more difficulty in one instruction pair, the Kappa score between humans is 0.68, and the Kappa between ChatGPT and human (majority voting) is 0.66, which indicates the good agreement among the ChatGPT and human annotators.

Reviewer comment (Experiments):
The WizardEval dataset is too small and the paper contains little detail about its construction, so we do not know if the dataset can serve as a sound and fair evaluation benchmark.

Author response:
In the Figure 5, we only shows the average score of "difficulty and complexity" (which are synonyms), and this score do not contains the degree of "diversity".
 \
Furthermore for "diversity", as shown in the "Section 4.5 - Analysis of In-breadth Evolving" and Figure 6 in Appendix F , we use t-SNE and k-means over BERT embedding of instructions to indicate the topic diversity between self-intruct, ShareGPT and evol-instruct dataset.

Reviewer comment (Writing):
The reviewer does not understand the argument in the section Analysis of In-breadth Evolving.

Author response:
In-breadth Evolving aims to enhance topic coverage, skill coverage, and overall dataset diversity. To examine (qualitative analysis) the breadth (diversity) of different dataset, we firstly use BERT to encode each instruction and get its embedding with 768 dimensions, then use a dimension reduction algorithm named t-SNE to reduce embedding dimension to 2, finally we apply a clustering algorithm k-means to partition the instructions of each dataset into 20 clusters for an intuitive visualization. As shown in the Figure 6 in the paper, the data points of our dataset are more dispersed than ShareGPT and Alpaca (Self-Instruct), which indicates the better topic diversity in our instructions.

Reviewer comment (Experiments):
One possible evaluation of difficulty and diversity is to divide all the training data into several subsets with increasing difficulty/diversity. If ChatGPT is capable of evaluating the training data, then perhaps more difficult and more diverse training data will lead to better performance. This is different from the current grouping based on evolution iterations, which includes a confounding variable: the number of evolution iterations.

Author response:
For your suggestion about human study on difficulty, in fact, we didn't fully understand your meaning at the beginning. After reading your latest response, it seems that we have understood a bit. Your concern seems is that our grouping will leak information to the labeler. In all our manual annotation processes, we do not inform the annotator about how we organize the data. We will randomly shuffle the instructions in these six datasets and hide their sources. The annotator only sees the instructions that need to be annotated and the annotation guideline.

Reviewer comment (Novelty):
The paper makes extensive citations to related papers, including many that are recently posted on arxiv. Though it is good to acknowledge similar work, sometimes the cited papers do not support or relate to the text.

Author response:
Thanks for your kindly suggestions for our grammatical and citations issues. We has updated a new version of paper to improve these aspects, please refer to the new version and thank you again.

Reviewer comment (Writing):
In the sentence “Human annotators are prone to fatigue and cannot sustain high-intensity work to produce a sufficient proportion of high-difficulty instructions (Zhang et al., 2023; Xiao et al., 2023; Manakul et al., 2023; Zhong et al., 2023),” the provided citations are unrelated to whether human annotators can sustain high-intensity work. This may create the impression that the paper attempts to mislead the reader, though it is not the authors' intention to do so.

Author response:
Thanks for your kindly suggestions for our grammatical and citations issues. We has updated a new version of paper to improve these aspects, please refer to the new version and thank you again.

Reviewer comment (Evaluation):
How do we know the answers to the newly created prompts are correct?

Author response:
To improve the alignment between the new prompts and our evolution target, we propose "Elimination Evolving" as shown in the paper,  it use two strategies to judge the quality of instruct: ChatGPT and manual rules.

Reviewer comment (Evaluation):
How do we know the difficulty and diversity scores by ChatGPT are correct? The reviewer requests that the authors not justify this by saying other papers did similar things, as those papers could be wrong too.

Author response:
- (1) In this paper, we only use ChatGPT to measure the  "difficulty and complexity" (which are synonyms).  And as mentioned in the response to the Weakness 5, we describe how we measure the diversity score with BERT, t-SNE, and k-means.

- (2) To investigate the correctness of the difficulty score by ChatGPT, we add a new experiment to measure agreement of difficulty judge between ChatGPT and humans: 
We randomly select two instructions from the six datasets - Alpaca, ShareGPT, C1 to C4 - with equal probability each time, forming a pair. In total, we have selected 300 instruction pairs. Then, we ask ChatGPT and 5 well-educated human annotators to judge which one is more difficulty in one instruction pair, the Kappa score between humans is 0.68, and the Kappa between ChatGPT and human (majority voting) is 0.66, which indicates the good agreement among the ChatGPT and human annotators.

Reviewer comment (Novelty):
The technical contribution of the proposed method is not very significant because compared to self-instruct, it is only adding the command for LLM to generate more complex instruction. The success of the proposed method largely depend on the abilities of powerful LLMs such as ChatGPT. While interesting and intuitive, I'm not sure the technical contribution of the manuscript is suitable for conferences such as ICLR.

Author response:
Motivation & Novelty: The Self-Instruct[1] is to address the problem of "human-written instruction data that is often limited in quantity", aiming to "provide an almost annotation-free method" based on LLM to automatically generate more instruction data. Our Evol-Instruct is not aimed at increasing the quantity of instructions but rather explores whether increasing the complexity of individual instruction contributes to improving the supervised fine-tuning performance of LLM. Our novelty lies in: (a) To best of our knowledge, we are the first to explicitly propose that increasing the complexity of instructions is beneficial for improving the instruction-following capabilities of pre-trained language models. (b) We propose a brand-new automated method to enhance the complexity and difficulty of existing instruction data without the need for human annotators.  

Both automatic and human evaluations consistently indicate that the model trained using Evol-Instruct (WizardLM) surpasses the baseline model (Alpaca) trained using Self-Instruct with a large margin in a broad spectrum of benchmarks.

\
reference:\
[1] Self-Instruct: Aligning Language Models with Self-Generated Instructions

> Weakness 1.2: The success of the proposed method largely depend on the abilities of powerful LLMs such as ChatGPT.

In the experiment, the baselines we compared, Alpaca-13b and Vicuna-13b, their training data also came from ChatGPT. Our experiment shows that under the condition of using the same powerful LLM, the model trained by Evol-Instruct will perform better on a wide range of benchmarks. Further, when we use a weaker model LlaMA-2-70B-Chat to replace ChatGPT to execute Evol-Instruct, the trained model is still stronger than the models (i.e., Alpaca-13b and Vicuna-13b) trained by ChatGPT outputs. Please see the details in our response to your Weakness 4.

Reviewer comment (Experiments):
The authors compare WizardLM with Vicuna by using the same amount of generated instructions. However, the cost or token consumption used for collecting the datasets (especially compared with Alpaca) should also be controlled or at least mentioned.

Author response:
Our motivation is to investigate whether, under the condition of keeping the number of instructions constant, simply increasing the complexity of each instruction can be beneficial to the SFT performance of LLMs. More token consumption is a reasonable expectation for this approach. We have counted the number of tokens required for generating 70k ShareGPT data, Alpaca data, and our 70k WizardLM data. Detailed statistics are as follows: ShareGPT(70k) 118.3M, Alpaca 3.02M, WizardLM(70k) 18.84M. To verify when the token consumption is exactly the same, our method can still outperforms the self-instruct method for generating Alpaca. We randomly selected 11,300 samples from our 70k data, totaling 3.02M tokens, equivalent to Alpaca. We then trained the Wizard-13b(3M) model. The results, as shown in the following table, indicate that Wizard-13b(3M) still significantly surpasses Alpaca-13b in all benchmarks. It's worth noting that the model we trained with only 3M Tokens can surprisingly perform comparably to the Vicuna-13b trained with 118M Tokens.


Model | Avg. | MMLU | ARC | HellaSwag  | TruthfulQA  |   HumanEval  | GSM8k| AlpacaEval  | MT-Bench  | WizardEval   
 | ----- |------| ---- |------|-------| ----- |  ----- | ----- |----- |----- |----- |
  |   Alpaca-13b   | 43.44 | 46.63 | 51.20  | 76.31 |41.62 | 9.2 | 8.35| 33.25 |  4.78  | 76.6  
  |   Vicuna-13b   | 54.60  | 50.84 |  51.71 | 79.94 | 52.68  | 12.5  | 24.34 | 70.43  | 6.21 | 86.9 
   |  WizardLM-13b (3M)    |  54.92 |  50.73 |  53.48 |  78.51 |  49.06    |  19.3 |  29.96   |  70.17 |  5.93 | 83.8  
   |  WizardLM-13b | 58.96 | 52.92 |  57.25 | 80.88  |50.55 | 24.0 | 37.15 | 75.31  | 6.35  | 89.1

Reviewer comment (Experiments):
The manuscript lacks comparisons with other methods for instruction data generation methods such as Baize, CAMEL, etc. For benchmark results, the role of the seed instructions is very important. I assume the seed data is very different, which could be a major cause of the performance difference. Therefore, more detailed ablation study is required to make the results more convincing.

Author response:
As shown in the table in the section of **New Baselines** of General Response, we add more baselines such as Baize, CAMEL, Tulu. The extensive results indicate that our WizardLM also significantly surpasses these open source models, which demonstrates that the convincing quality of instruction-following dataset crafted by Evol-Instruct.

We agree with you, seed instructions is very important for the model performance. So we choose the training data of Alpaca-13b as the seed instructions for our execution of Evol-Instruct. The main submission's Table 1 experimental results show that, based on the same seed instructions, WizardLM-13b after executing Evol-Instruct significantly outperforms Alpaca-13b on a series of benchmarks with a large margin. Although ShareGPT has different seed instructions from Alpaca data, its quality is widely recognized as far superior to Alpaca data. We chose to compare with Vicuna-13b, which uses ShareGPT as training data, in order to demonstrate that even a relatively low-quality instruction-following dataset, after applying our proposed Evol-Instruct, can surpass higher-quality data. To further verify the effectiveness of our method, we use ShareGPT as a new seed, and apply our evol-instruct method on it to produce a new 70k dataset, then train a model named WizardLM-13b (ShareGPT Seed). As shown in the following table, the average performance across nine benchmarks of this model is even higher than the previous version of WizardLM (the seed is Alpaca data). The consistent results of Alpaca data and ShareGPT as evolutionary seeds indicate that Evol-Instruct can significantly improve the performance of fine-tuning pre-trained language models using specific instruction data.

Model | Avg. | MMLU | ARC | HellaSwag  | TruthfulQA  |   HumanEval  | GSM8k| AlpacaEval  | MT-Bench  | WizardEval   
 | ----- |------| ---- |------|-------| ----- |  ----- | ----- |----- |----- |----- |
 | WizardLM-13b | 58.96 | 52.92 |  57.25 | 80.88  | 50.55 | 24.0 | 37.15 | 75.31  | 6.35  | 89.1
| WizardLM-13b (ShareGPT Seed) | 61.87 | 50.92 |  60.24 | 81.39 | 54.56 | 25.0 | 31.46 | 86.32 | 6.76 | 99.3

Reviewer comment (Experiments):
It would be very helpful to test the proposed data generation methods using other LLMs (e.g., LLaMA) instead of ChatGPT to better understand the proposed data synthesis pipeline.

Author response:
To verify that our proposed Evol-Instruct method is not dependent on ChatGPT, we chose LlaMA-2-70B-Chat as a substitute for ChatGPT. Following the methods outlined in the paper, we re-executed the entire evolution process starting from Alpaca data and trained the WizardLM(LlaMA-2-70B-Chat) model with randomly selected 70k data. Please refer to below table for details, WizardLM(LlaMA-2-70B-Chat) still significantly outperforms Alpaca-13b trained solely on Alpaca data and even outperforms Vicuna-13B trained with much more ChatGPT tokens.

Model | Avg. | MMLU | ARC | HellaSwag  | TruthfulQA  |   HumanEval  | GSM8k| AlpacaEval  | MT-Bench  | WizardEval   
| ----- |------| ---- |------|-------| ----- |  ----- | ----- |----- |----- |----- |
|   Alpaca-13b   | 43.44 | 46.63 | 51.20  | 76.31 |41.62 | 9.2 | 8.35| 33.25 |  4.78  | 76.6  
|   Vicuna-13b   | 54.60  | 50.84 |  51.71 | 79.94 | 52.68  | 12.5  | 24.34 | 70.43  | 6.21 | 86.9 
| WizardLM-13b | 58.96 | 52.92 |  57.25 | 80.88  |50.55 | 24.0 | 37.15 | 75.31  | 6.35  | 89.1
| WizardLM-13b(LlaMA-2-70B-Chat Evol)  |  56.27|  51.09 | 57.34 | 79.12 | 48.76 | 19.5 | 33.83 | 70.47 | 6.18   | 84.5