d^2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
Authors: name: Yuchu Jiang, name: Yue Cai, name: Xiangzhong Luo, name: Jiale Fu, name: Jiarui Wang, name: Chonghan Liu, name: Xu Yang

Diffusion-based large language models (dLLMs), despite their promising
performance, still suffer from inferior inference efficiency. This is because
dLLMs rely on bidirectional attention and cannot directly benefit from the
standard key-value (KV) cache as autoregressive models (ARMs) do. To tackle
this issue, we introduce Dual aDaptive Cache (d^2Cache), which is a
training-free approximate KV cache framework for accelerating dLLM inference.
d^2Cache features a two-stage fine-grained selection strategy to identify
tokens and adaptively update their KV states at each decoding step, while
caching the KV states of the remaining tokens for reuse. Furthermore,
d^2Cache naturally offers a more reliable decoding alternative, which can
enable quasi left-to-right generation and mitigate premature overconfidence in
tokens at the end of the sequence. Extensive experimental results on two
representative dLLMs (\ie, LLaDA and Dream) demonstrate that d^2Cache not
only achieves substantial inference speedups, but also yields consistent
improvements in generation quality. The code is available at
https://github.com/Kamichanw/d2Cache.