D olph C oder: Echo-Locating Code Large Language Models with Diverse and Multi-Objective Instruction Tuning (2024)

Abstract: Code Large Language Models (Code LLMs) have demonstrated outstanding performance in code-related tasks. Various instruction finetuning approaches have been proposed to boost the code generation performance of pre-trained Code LLMs. In this paper, we introduce a diverse instruction model DolphCoder with self-evaluating for code generation. It learns diverse instruction targets and combines a code evaluation objective to enhance its code generation ability. Our model achieves superior performance on the HumanEval and MBPP benchmarks, demonstrating new insights for future code instruction tuning work. Our key findings are: (1) Augmenting more diverse responses with more distinct reasoning paths increases the code capability of LLMs. (2) Improving one’s ability to evaluate the correctness of code also enhances their ability to create it.

## introduction code pre-trained models have achieved remarkable progress in the era of large language models (llms), such as codex @xcite , alpha-code @xcite , and palm-coder @xcite . code-related tasks are also the key factors in evaluating the capability of llms. numerous code llms have been proposed, including closed-source models @xcite @xcite and open-source models @xcite . they perform expensive pre-training using substantial amounts of code data and display impressive performance. in contrast to these pre-trained code llms, another lightweight paradigm of enhancing code capability is instruction tuning using relatively small * equal contribution. † corresponding author. however, these methods have two weaknesses: (1) they take the only golden answer but ignore the diversity of answers in code generation. we find that augmenting more diverse responses using different system prompts increases the code capability of llms. (2) current models generate plausible code snippets in terms of grammar and logic but are unable to identify subtle errors, such as corner cases and wrong input/output formats. it has no guarantee that temperature sampling will consistently produce accurate answers over time. we suppose that llms are capable of generating correct solutions while struggling to discriminate correct from incorrect ones. improving one's ability to evaluate the correctness of code also enhances their ability to create it. inspired by the two insights, we introduce a diverse instruction model (dolphcoder) with selfevaluating for code generation. specifically, we use code llama-python as our base model and obtain evolved instruction data following wizard-coder. then motivated by rejection sampling @xcite and orca @xcite , we use different system prompts to generate diverse answers via chatgpt. after removing low-quality and similar data using heuristic rules @xcite , we perform supervised fine-tuning on the remaining instruction data. further, we explore whether improving one's ability to evaluate code helps generate it. we propose a self-evaluate multi-task learning framework by adding a code evaluation objective to the traditional instruction fine-tuning task. we find training the model for both code generation and code evaluation benefits the code capability. our key contributions are summarized as follows: ## method in this section, we elaborate on the methodological details of dolphcoder. as shown in figure 1 , dolphcoder has two training stages: (1) the first is diverse instruction tuning (dit) with multiple chain-of-thought answers to the same instruction. ## analysis 4.1 ablation study to investigate the characteristics of the main components in dolphcoder, we conduct ablation experiments in ## conclusion and future work in this paper, we investigate two fine-tuning methods to improve the llms' performance on code generation. we first introduce a response augmentation strategy by using different chatgpt system prompts to increase the diversity of code solutions. we find different chain-of-thought reasoning paths improve performance. then, we adopt a multistep training approach that combines traditional code generation and code evaluation objectives. we find improving one's ability to evaluate the correctness of code also enhances their ability to create it. for future work, we aim to explore the effect of our methods on the larger foundation models and parameter-efficient fine-tuning mechanisms. we also plan to increase the accuracy of evaluation signals via other ways like automatic unit tests and reinforcement learning methods.