WizardCoder: Empowering Code Large Language Models with Evol-Instruct
Authors: name: Ziyang Luo; name: Can Xu; name: Pu Zhao; name: Qingfeng Sun; name: Xiubo Geng; name: Wenxiang Hu; name: Chongyang Tao; name: Jing Ma; name: Qingwei Lin; name: Daxin Jiang
Published: 2023-06-14 15:18:48

Abstract: Code Large Language Models (Code LLMs), such as StarCoder, have demonstrated
exceptional performance in code-related tasks. However, most existing models
are solely pre-trained on extensive raw code data without instruction
fine-tuning. In this paper, we introduce WizardCoder, which empowers Code LLMs
with complex instruction fine-tuning, by adapting the Evol-Instruct method to
the domain of code. Through comprehensive experiments on four prominent code
generation benchmarks, namely HumanEval, HumanEval+, MBPP, and DS-1000, we
unveil the exceptional capabilities of our model. It surpasses all other
open-source Code LLMs by a substantial margin. Moreover, our model even
outperforms the largest closed LLMs, Anthropic's Claude and Google's Bard, on
HumanEval and HumanEval+. Our code, model weights, and data are public at
https://github.com/nlpxucan/WizardLM