dInfer: An Efficient Inference Framework for Diffusion Language Models
Authors: name: Yuxin Ma, name: Lun Du, name: Lanning Wei, name: Kun Chen, name: Qian Xu, name: Kangyu Wang, name: Guofeng Feng, name: Guoshan Lu, name: Lin Liu, name: Xiaojing Qi, name: Xinyuan Zhang, name: Zhen Tao, name: Haibo Feng, name: Ziyun Jiang, name: Ying Xu, name: Zenan Huang, name: Yihong Zhuang, name: Haokai Xu, name: Jiaqi Hu, name: Zhenzhong Lan, name: Junbo Zhao, name: Jianguo Li, name: Da Zheng

Diffusion-based large language models (dLLMs) have emerged as a promising
alternative to autoregressive (AR) LLMs, leveraging denoising-based generation
to enable inherent parallelism. Even more and more open-sourced dLLM models
emerge, yet their widespread adoption remains constrained by the lack of a
standardized and efficient inference framework. We present dInfer, an efficient
and extensible framework for dLLM inference. dInfer decomposes the inference
pipeline into four modular components--model, diffusion iteration manager,
decoding strategy, and KV-cache manager--and integrates novel algorithms for
each component alongside system-level optimizations. Through this combination
of algorithmic innovations and system enhancements, dInfer achieves substantial
efficiency gains without compromising output quality on LLaDA-MoE. At batch
size 1, it surpasses 1,100 tokens per second on HumanEval and averages over 800
tokens per second across six benchmarks on 8times H800 GPUs. Compared to
prior systems, dInfer delivers a 10times speedup over Fast-dLLM while
maintaining similar model performance. Even compared to the AR model (with a
comparable number of activation parameters and performance) QWen2.5-3B, which
is highly optimized with the latest vLLM inference engine, dInfer still
delivers a 2-3times speedup. The implementation of dInfer is open-sourced
at https://github.com/inclusionAI/dInfer.