Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed
  Inference
Authors: name: Yuxuan Song, name: Zheng Zhang, name: Cheng Luo, name: Pengyang Gao, name: Fan Xia, name: Hao Luo, name: Zheng Li, name: Yuehang Yang, name: Hongli Yu, name: Xingwei Qu, name: Yuwei Fu, name: Jing Su, name: Ge Zhang, name: Wenhao Huang, name: Mingxuan Wang, name: Lin Yan, name: Xiaoying Jia, name: Jingjing Liu, name: Wei-Ying Ma, name: Ya-Qin Zhang, name: Yonghui Wu, name: Hao Zhou

We present Seed Diffusion Preview, a large-scale language model based on
discrete-state diffusion, offering remarkably fast inference speed. Thanks to
non-sequential, parallel generation, discrete diffusion models provide a
notable speedup to mitigate the inherent latency of token-by-token decoding, as
demonstrated recently (e.g., Mercury Coder, Gemini Diffusion). Seed Diffusion
Preview achieves an inference speed of 2,146 token/s over H20 GPUs while
maintaining competitive performance across a sweep of standard code evaluation
benchmarks, significantly faster than contemporary Mercury and Gemini
Diffusion, establishing new state of the art on the speed-quality Pareto
frontier for code models.