CoDA: Coding LM via Diffusion Adaptation
Authors: name: Haolin Chen, name: Shiyu Wang, name: Can Qin, name: Bo Pang, name: Zuxin Liu, name: Jielin Qiu, name: Jianguo Zhang, name: Yingbo Zhou, name: Zeyuan Chen, name: Ran Xu, name: Shelby Heinecke, name: Silvio Savarese, name: Caiming Xiong, name: Huan Wang, name: Weiran Yao
Organization: Salesforce AI Research

Diffusion language models promise bidirectional context and infilling
capabilities that autoregressive coders lack, yet practical systems remain
heavyweight. We introduce CoDA, a 1.7B-parameter diffusion coder trained on TPU
with a fully open-source training pipeline. CoDA pairs large-scale diffusion
pre-training with code-centric mid-training and instruction tuning, enabling
confidence-guided sampling that keeps inference latency competitive. On
Humaneval, MBPP, and EvalPlus, CoDA-1.7B-Instruct matches or surpasses
diffusion models up to 7B parameters. Our release includes model checkpoints,
evaluation harnesses, and TPU training pipelines to accelerate research on
lightweight diffusion-based coding assistants.