CoDA: Coding LM via Diffusion Adaptation
Authors: name: Haolin Chen; name: Shiyu Wang; name: Can Qin; name: Bo Pang; name: Zuxin Liu; name: Jielin Qiu; name: Jianguo Zhang; name: Yingbo Zhou; name: Zeyuan Chen; name: Ran Xu; name: Shelby Heinecke; name: Silvio Savarese; name: Caiming Xiong; name: Huan Wang; name: Weiran Yao
Published: 2025-09-27 05:41:55

Abstract: Diffusion language models promise bidirectional context and infilling
capabilities that autoregressive coders lack, yet practical systems remain
heavyweight. We introduce CoDA, a 1.7B-parameter diffusion coder trained on TPU
with a fully open-source training pipeline. CoDA pairs large-scale diffusion
pre-training with code-centric mid-training and instruction tuning, enabling
confidence-guided sampling that keeps inference latency competitive. On
Humaneval, MBPP, and EvalPlus, CoDA-1.7B-Instruct matches or surpasses
diffusion models up to 7B parameters. Our release includes model checkpoints,
evaluation harnesses, and TPU training pipelines to accelerate research on
lightweight diffusion-based coding assistants.