Title: Accelerating Diffusion LLMs via Adaptive Parallel Decoding
Venue: NeurIPS 2025
Accepted: yes
Keywords: LLM

discrete diffusion

autoregression

sequential

fast

throughput

speculative decoding
Abstract: The generation speed of LLMs are bottlenecked by autoregressive decoding, where tokens are predicted sequentially one by one. Alternatively, diffusion large language models (dLLMs) theoretically allow for parallel token generation, but in practice struggle to achieve the speed of autoregressive models without significantly sacrificing quality. We therefore introduce adaptive parallel decoding (APD), a novel method that dynamically adjusts the number of tokens sampled in parallel. We achieve this by defining a multiplicative mixture between the dLLM marginal probabilities and the joint probability of sequences under a small auxiliary autoregressive model. This inverts the standard setup of speculative decoding, where the goal is to sample from a large autoregressive verifier by drafting from a smaller model. We further optimize APD by enabling KV caching and limiting the size of the masked input. Altogether, our method puts forward three tunable parameters to flexibly tradeoff throughput and quality. We show that APD provides markedly higher throughput with minimal quality degradations on downstream benchmarks.


Peer reviews:
comments: SUMMARY:
The authors propose Adaptive Parallel Decoding (APD), which improves sampling speed for existing diffusion-based large language models (dLLMs) compared to original methods like Dream and Llada. By introducing KV cache and limited masked inputs, APD enables flexible trade-offs between generation speed and quality.

STRENGTHS AND WEAKNESSES:
Strengths:  
1. APD adaptively determines the number of tokens generated per step via a small autoregressive model to evaluate output quality, mitigating the quality degradation caused by fixed-length parallel generation.  
2. APD can be directly applied to existing dLLMs with significant speedup benefits.  
3. The use of KV cache and limited masked inputs further enhances dLLM inference efficiency.  
  
Weaknesses:  
1. The proof relying on Bonferroni’s inequality (Property 1) is flawed. As noted in Line 122, $P_{APD}=P_{AR}$ holds only if $|g|=n$; when $|g|<n$, they are not equivalent. Thus, applying Bonferroni’s inequality is mathematically unsound. This also invalidates the heuristic justification for Property 2.  
2. The author directly uses the KV cache method, but this method has already been proposed and there are different variants, so it cannot be considered an innovative point.  
3.  Experiments are confined to GMS8K, GPQA, and MATH without theoretical guarantees, leaving generalization to other datasets unverified.

QUESTIONS:
1. Llada’s reference paper tested more diverse datasets. Why restrict evaluation to only three?  
2. The reference to "Dream 7B" (Citation 39) appears incorrect—no matching work was found. Is this a citation formatting error?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-23

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Adaptive Parallel Decoding (APD), a novel method that accelerates diffusion-based LLMs by dynamically adjusting parallel token generation per decoding step. APD uses a multiplicative mixture between dLLM marginal probabilities and a small autoregressive model's joint distribution to determine token acceptance, enabling adaptive parallelization while maintaining quality. The framework integrates KV caching and masked lookahead constraints for acceleration and offers tunable parameters to balance throughput versus quality. APD achieves 59 tokens/sec (vs. 37 tokens/sec for autoregressive baselines) on reasoning tasks like GSM8K with minimal quality degradation, outperforming naive parallel approaches.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n+ This paper proposes the APD framework, by using a small AR model to validate large dLLM samples, dynamically adjust the number of parallel tokens, and solve the contradiction between parallelism and quality of dLLM.\n+ The desigh integrates KV caching and mask lookahead restrictions, significantly accelerating dLLM inference without the need for re-training the model.\n+ The effect is good. It achieves 59 tokens per second on tasks such as GSM8K, with a 60% increase in throughput and a quality loss of less than 2%.\n\nWeaknesses:\n\n- APD requires dLLMs specifically trained for left-to-right decoding, with non-left-to-right implementations such as Llada exhibiting substantially weaker performance as evidenced in Table 1.\n- The small AR model adds latency; its impact on end-to-end throughput isn’t quantified.\n- Evaluation focused on math/reasoning (GSM8K, GPQA, MATH). Open-ended generation shows lower parallelism (2.9–3.4 tokens/step) as evidenced in Appendix D, suggesting task-dependent efficacy.\n\nQUESTIONS:\nQ1: How much latency does the small AR model introduce? Does KV caching sufficiently offset its computational overhead to justify inclusion?\n\nQ2: Given Table 1 shows left-to-right decoding yields optimal dLLM performance, can APD effectively integrate with entropy/confidence-based decoding schemes (e.g., Llada’s approach) without compromising throughput gains?\n\nQ3: Can APD achieve high parallelism (>5 tokens/step) in open-ended generation? Fig 2 suggests quality drops sharply for k>5 in naive parallelization, does APD overcome this limitation for non-reasoning domains?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nLLMs today excel at text generation but remain bottlenecked by the strictly sequential nature of AR decoding. Diffusion-based LLMs offer parallel sampling in theory, yet in practice—exemplified by Dream and Llada—they only match AR quality when run one token at a time. To bridge this gap, this paper introduces APD: a left-to-right diffusion schedule that turns the model autoregressive, combined with a lightweight AR verifier to decide how many tokens to accept in parallel via a multiplicative product-of-experts criterion. They further accelerated inference with KV-caching beyond a sliding window and by setting a threshold for the masked lookahead. Together, these three tunable knobs let APD dramatically boost throughput—up to nearly twice the speed of pure AR—while incurring only minimal quality loss across three benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n\n1) Detailed analysis of the performance of the dLLMs in terms of accuracy and decoding strategies, showing the limitations of purely autoregressive generation and propose a novel method to maintain high quality while increasing the throughput.\n\n2) Proposing enabling KV caching for tokens outside a pre-specified window together with maximum masked lookahead to increase the throughput of dLLMs.\n\n3) APD consistently maintains near-autoregressive accuracy—even when accepting on average 5–7 tokens per iteration—while delivering up to a 2× speed-up over pure AR sampling on GSM8K, GPQA, and MATH.\n\nWeaknesses:\n\n1) I would appreciate a similar accuracy vs throughput comparison table similar to table 1, comparing the performance of Dream 7B + Qwen 2.5 0.5 with Qwen 7B to be able to observe the direct comparison of auto-regressive decoding and dLLMs with APD.\n\n2) All experiments focus on three reasoning benchmarks (GSM8K, GPQA, MATH) and a single diffusion backbone (Dream 7B). Without additional evaluations—e.g., text summarization, code generation, translation—and tests on other dLLMs, it remains unclear whether APD’s speed-quality trade-offs generalize beyond the presented setup.\n\nQUESTIONS:\nNA\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Adaptive Parallel Decoding (APD), a novel decoding strategy designed to accelerate inference in diffusion-based large language models (dLLMs). Unlike traditional autoregressive decoding, which is inherently sequential, APD enables dynamic parallel token generation by combining the marginal probabilities from a diffusion model with the joint probabilities from a smaller auxiliary autoregressive model. The method introduces three tunable parameters—mixture weight (R), KV cache window (W), and maximum masked lookahead (M)—to balance the trade-off between speed and quality. Empirical results show that APD significantly improves throughput with minimal degradation in output quality, outperforming both naive diffusion decoding and autoregressive baselines on benchmarks like GSM8K, GPQA, and MATH.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Paper is well motivated: Table 1 demonstrates that open source autoregressive LLMs still outperform the recent dLLMs in terms of performance v.s token/rate.\n- Paper is well written and easy to read.\n- The proposed technique is novel (as far as I know), the idea of adaptively choosing how many tokens to generate in parallel with a dLLM is new.\n- The paper presents extensive empirical results on recent LLMs to back their claims.\n\nWeaknesses:\n- The dependence on an auxiliary approximate autoregressive model limits its applicability. At the very least, ADP requires that for each dLLM, a small LLM with the same tokenizer exists. \n- I found sampling from $p_T$ as presented in eq (8) to be slightly confusing. In practice (as presented in Algorithm 1), $p_D$ is first used to generate $n$ tokens, which are then subsequently used by $\\hat{p}_{AR}$. Does this not impose some conditional dependence?\n\nQUESTIONS:\n- In Table 1, why is random decoding always slower than entropy/confidence-base decoding for Dream/Llada models?\n\nLIMITATIONS:\nThe authors have addressed the limitations of their method.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-05"}