🌊 WaveCoder: Widespread And Versatile Enhanced Code LLM

[📜 Paper][🐱 GitHub]
[🐦 Twitter][💬 Reddit][🍀 Unofficial Blog]

Repo for "WaveCoder: Widespread And Versatile Enhanced Instruction Tuning with Refined Data Generation"

🔥 News

  • [2024/04/10] 🔥🔥🔥 WaveCoder repo, models released at 🤗 HuggingFace!
  • [2023/12/26] WaveCoder paper released.

💡 Introduction

WaveCoder 🌊 is a series of large language models (LLMs) for the coding domain, designed to solve relevant problems in the field of code through instruction-following learning. Its training dataset was generated from a subset of code-search-net data using a generator-discriminator framework based on LLMs that we proposed, covering four general code-related tasks: code generation, code summary, code translation, and code repair.

Model HumanEval MBPP(500) HumanEval
Fix(Avg.)
HumanEval
Explain(Avg.)
GPT-4 85.4 - 47.8 52.1
🌊 WaveCoder-DS-6.7B 65.8 63.0 49.5 40.8
🌊 WaveCoder-Pro-6.7B 74.4 63.4 52.1 43.0
🌊 WaveCoder-Ultra-6.7B 79.9 64.6 52.3 45.7

🪁 Evaluation

Please refer to WaveCoder's GitHub repo for inference, evaluation, and training code.

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("microsoft/wavecoder-ultra-6.7b")
model = AutoModelForCausalLM.from_pretrained("microsoft/wavecoder-ultra-6.7b")

📖 License

This code repository is licensed under the MIT License. The use of DeepSeek Coder models is subject to the its License.

☕️ Citation

If you find this repository helpful, please consider citing our paper:

@article{yu2023wavecoder,
  title={Wavecoder: Widespread and versatile enhanced instruction tuning with refined data generation},
  author={Yu, Zhaojian and Zhang, Xin and Shang, Ning and Huang, Yangyu and Xu, Can and Zhao, Yishujie and Hu, Wenxiang and Yin, Qiufeng},
  journal={arXiv preprint arXiv:2312.14187},
  year={2023}
}

Note

WaveCoder models are trained on the synthetic data generated by OpenAI models. Please pay attention to OpenAI's terms of use when using the models and the datasets.

Downloads last month
1,421
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for microsoft/wavecoder-ultra-6.7b

Finetunes
6 models
Quantizations
6 models