SelfCodeAlign: Self-Alignment for Code Generation
Authors: name: Yuxiang Wei; name: Federico Cassano; name: Jiawei Liu; name: Yifeng Ding; name: Naman Jain; name: Zachary Mueller; name: Harm de Vries; name: Leandro von Werra; name: Arjun Guha; name: Lingming Zhang
Published: 2024-10-31 17:55:13

Abstract: Instruction tuning is a supervised fine-tuning approach that significantly
improves the ability of large language models (LLMs) to follow human
instructions. We propose SelfCodeAlign, the first fully transparent and
permissive pipeline for self-aligning code LLMs without extensive human
annotations or distillation. SelfCodeAlign employs the same base model for
inference throughout the data generation process. It first extracts diverse
coding concepts from high-quality seed snippets to generate new tasks. It then
samples multiple responses per task, pairs each with test cases, and validates
them in a sandbox environment. Finally, passing examples are selected for
instruction tuning. In our primary experiments, we use SelfCodeAlign with
CodeQwen1.5-7B to generate a dataset of 74k instruction-response pairs.
Finetuning on this dataset leads to a model that achieves a 67.1 pass@1 on
HumanEval+, surpassing CodeLlama-70B-Instruct despite being ten times smaller.
Across all benchmarks, this finetuned model consistently outperforms the
original version trained with OctoPack, the previous state-of-the-art method
for instruction tuning without human annotations or distillation. Additionally,
we show that SelfCodeAlign is effective across LLMs of various sizes, from 3B
to 33B, and that the base models can benefit more from alignment with their own
data distribution. We further validate each component's effectiveness in our
pipeline, showing that SelfCodeAlign outperforms both direct distillation from
GPT-4o and leading GPT-3.5-based distillation methods, such as OSS-Instruct and
Evol-Instruct. SelfCodeAlign has also led to the creation of
StarCoder2-Instruct, the first fully transparent, permissively licensed, and
self-aligned code LLM that achieves state-of-the-art coding performance.