Title: Breaking AR’s Sampling Bottleneck: Provable Acceleration via Diffusion Language Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: diffusion model

large language model (LLM)

iteration complexity

mutual information
Abstract: Diffusion models have emerged as a powerful paradigm for modern generative modeling, demonstrating strong potential for large language models (LLMs). Unlike conventional autoregressive (AR) models that generate tokens sequentially, diffusion models allow for parallel sampling, offering a promising path to accelerate generation and eliminate the left-to-right generation constraints. Despite their empirical success, theoretical understandings of diffusion language models remain underdeveloped. In this work, we develop convergence guarantees for diffusion language models from an information-theoretic perspective. Our analysis demonstrates that the sampling error, measured by the Kullback-Leibler (KL) divergence, decays inversely with the number of iterations $T$ and scales linearly with the mutual information between tokens in the target text sequence. Crucially, our theory covers the regime $T<L$, where $L$ is the text sequence length. This justifies that high-quality samples can be generated with fewer iterations than $L$, thereby breaking the fundamental sampling bottleneck of $L$ steps required by AR models. We further establish matching upper and lower bounds, up to some constant factor, that shows the tightness of our convergence analysis. These results offer novel theoretical insights into the practical effectiveness of diffusion language models.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis manuscript proves convergence guarantees for sampling from discrete diffusion language models.  The authors prove that (i) the error decays as O(1/T) after T denoising steps and (ii) that the constant factor depends on the mutual information between tokens in the target sequence. Finally, a matching lower bound shows that this rate is tight. The results assume access to accurate mask predictors, consistent with prior work that theoretically analyze the sampling process of continuous diffusion models.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- **Well-written**: in general, the paper is well organized and written. The exposition in section 4 is well organized and relatively easy to follow. I appreciate that not all proofs are relegated to the appendix. Additionally, the notation preliminaries are complete and consistent with my expectations.\n- **Novel theoretical contribution**: Theorem 1 (Eq. 8) provides an O(1/T) upper bound on the KL error that depends on the average mutual information between a token and the rest of the sequence. Importantly, the authors prove a matching lower bound, showing that their result is tight.\n- **Significant improvements over prior work**: authors discuss the prior work of Feng et al. [6], and rightfully argue that their bound is more aligned with realistic values of L (sequence length) and n (vocabulary size).\n- **Well-structured preliminaries**: section 1.2 carefully sets notation and clarifies the training objective, lowering the barrier to entry for readers outside the discrete diffusion community.\n- **Sampling algorithm is accurate**: While some prior theoretical work might make unrealistic assumptions, this work assumes the true sampling algorithm of recent masked diffusion models such as MDLM [4].\n\n---\n## Weaknesses\n**No empirical validation**: While the work is theoretical, a small-scale simulation on a synthetic distribution would strengthen the submission.\n\n**Potentially large hidden constants**: Theorems quote unspecified \"absolute constants\" (C_1,C_2,C_3). The manuscript could be improved by including more details on what typical constants should be expected.\n\n**Contextualization**: To better contextualize this work, the authors should consider citing a few recent works, which would make some claims in the introduction more precise. Specifically, by default, contrary to what is stated on lines 46-48, **sampling from masked diffusion models is generally slower than AR models**, despite parallel generation quality. Indeed, KV caching [1] accelerates AR models significantly, but is not available for models with bidirectional attention. However, diffusion language models can be accelerated through distillation [2,3]. For a comparison of the speed of text diffusion models with AR models, see Figure 2 of the appendix of [4]. Additionally, regarding the capabilities of diffusion language models, citing [5] would be relevant.\n\n\n[1] Efficiently Scaling Transformer Inference, Pope et al.\n\n[2] Beyond Autoregression: Fast LLMs via Self-Distillation Through Time, Deschenaux and Gulcehre\n\n[3] Distillation of Discrete Diffusion through Dimensional Correlations, Hayakawa et al.\n\n[4] Simple and Effective Masked Diffusion Language Models, Sahoo et al.\n\n[5] Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning, Ye et al.\n\n[6] Theoretical Benefit and Limitation of Diffusion Language Model, Feng et al.\n\nQUESTIONS:\n1. Can the bound be tightened further for language distributions with known structure (e.g., low-order Markov texts)? Have the authors thought about interesting such distributions?\n2. Have the authors studied the impact of changing the sampling algorithm to the bound? For example, what would be the impact of entropy-based sampling, where positions with low uncertainty (as predicted by the denoiser), are more likely to be sampled first?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the theoretical understanding of generative *diffusion language models*. Unlike conventional autoregressive models, diffusion models allow parallel token sampling, which leads to faster generation and eliminates left-to-right generation constraints. The main results of the paper are convergences guarantees from an information-theoretic perspective: the sampling error, measured by KL divergence, decays inversely with the number of iterations and scales linearly with the mutual information between tokens in the target text sequence.  The paper establishes matching upper and lower bounds, which provides the tightness of their convergence analysis. These findings offer interesting theoretical insights into the practical effectiveness and robustness of diffusion language models, highlighting how the statistical dependencies in language data influence the efficiency of parallel diffusion sampling.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n - **Interesting theoretical contributions**: (As far as I know) the paper offers significant theoretical contributions by developing convergence guarantees for diffusion language models using KL divergence and IT tools. It quantifies the sampling error using KL divergence, showing an inverse decay with the number of steps  and a linear relationship with the mutual information between tokens in the target text sequence.\n\n- **Tight Bounds**: The establishment of matching upper and lower bounds (up to constant factors) demonstrates the tightness and optimality of te convergence analysis, suggesting that the $1/T$ decay and mutual information dependence are fundamental limits.\n\n- **Practical insights / intuition**: The results offer new insights into why diffusion language models are effective in practice.\n\n- **Comparison with other relevant work:** The paper contrasts its findings with existing work, such as  the very recent work by Feng et al. 2025, showing a sharper and more general guarantee for arbitrary distributions, overcoming limitations of previous bounds that degrade for high-order n-gram models.\n\n**Weaknesses:**\n\nI don't see any major weakness in this work, though I recognize that I'm not able to correctly asses the technical proofs since this work is out of my domain of expertise. One point that could may be better discussed is the statement:\n\n*\"Given accurate mask predictors, can we establish the convergence guarantees of diffusion language models\nfor general sampling procedures and data distribution?\"*\n\n**General sampling procedures and data distributions.** The paper (as always) does make some assumptions about the sampling procedure and the data distribution, maybe this should be contextualized in the introduction. It would be nice to connect the assumptions to what we may observe on real data (there's some discussion about this at the very end).\n\nQUESTIONS:\nMy only question is about the generality of this statement:\n\n*\"Given accurate mask predictors, can we establish the convergence guarantees of diffusion language models\nfor general sampling procedures and data distribution?\"*\n\nIn particular regarding the  *general sampling procedures and data distributions.*\n\nMinor typos:\n\n  - l6: diffusion models --> diffusion language models\n  - l42: serves --> serve\n\nLIMITATIONS:\nLimitations are correctly discussed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 1, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper analyzes both the upper and lower bounds of the sampling error in diffusion language models from an information-theoretic perspective. It demonstrates that, under the assumption of linear dependence on the mutual information, the fundamental limit for the convergence rate of the sampling error is $O(1/T)$.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\nCompared with previous results on diffusion models under n-gram distributions (e.g., Feng et al., 2025), this work is more general and rigorous, as it applies to arbitrary data distributions and token dependencies. The theoretical guarantees remain valid even for higher-order structures with long-range dependencies, unlike prior methods whose guarantees become loose or invalid as n increases.\n\n## Weaknesses\nWhile the paper provides a rigorous theoretical analysis, all derivations assume that the mask predictor can be trained to near-optimality, but no experimental results are provided to support the theoretical conclusions. In addition, the theoretical lower bound may not hold under certain extreme mask schedules (see Lines 283–290).\n\nQUESTIONS:\nQ1: What is the rationale for assuming that the sampling error has a linear dependence on the mutual information? Could the authors provide further justification or discussion for this assumption?\n\nQ2: While the paper presents an analysis of sampling error convergence, could the authors offer any constructive suggestions on how to effectively reduce the sampling error or accelerate convergence for parallel sampling in diffusion language models?\n\nMinor 1: The description in the Abstract (Line 6) is inaccurate: it should state “theoretical understandings of discrete diffusion language models” instead of “diffusion models”.\n\nMinor 2: Why is reversal reasoning emphasized? This does not seem consistent with the nature of language generation, and I do not consider it an advantage.\n\nMinor 3: The claim at Line 68 “This assumption does not align with practical diffusion language models that mask a large fraction of tokens at each iteration” needs supporting citations.\n\nLIMITATIONS:\nSee Weaknesses and Questions.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nMasked diffusion language models (DLMs) generate text by iteratively unmasking subsets of tokens in parallel, potentially sampling much faster than autoregressive transformers. Despite strong empirical results, there has been little theory quantifying how many unmasking iterations $T$ are needed to approach the true data distribution. This paper fills that gap by analyzing the sampling phase—assuming an already-trained, sufficiently accurate mask-prediction network—and derives information-theoretic bounds on the KL divergence between model output and data. \n\nIn the main theorems, the authors proved an explicit upper bound on the KL divergence between the model output and the data distribution that decays as $O(1/T)$ with the number of unmasking iterations $T$ (for balanced schedules) and depends linearly on a sequence-level mutual-information term. This bound is also shown to be unimprovable under the paper’s oracle-predictor assumption.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe paper makes a compelling theoretical contribution by establishing the first rigorous relationship between the number of unmasking iterations and generation quality for masked-diffusion language models.  It offers a tight information-theoretic analysis with matching upper- and lower-bound rates, while scaling gracefully to schedules that update many tokens in parallel.  A particularly valuable aspect is its identification of sequence-level mutual information as the key statistic governing convergence, which provides an intuitive bridge between linguistic dependence and iteration complexity.  The authors further separate sampling-phase error from training-phase error, clarifying where future architectural or data improvements are most beneficial.  Beyond its theory, the manuscript is clearly written, logically structured, and supported by a thorough discussion of prior work, making the results accessible to a broad audience.\n\nWeaknesses and Questions:\n1. For the oracle predictor assumption, the main bound becomes vacuous when $\\varepsilon_{\\text{train}}$ dominates; no scaling law links this term to model capacity or data. It would be better to add some discussions on the $\\varepsilon_{\\text{train}}$ term as well as its connection with model capacity. \n2. Is there any synthetic or real experiments that the authors can provide to confirm the predicted inverse-linear curve or the practical relevance of the mutual-information coefficient? \n3. For the potential gap in KL decomposition, I wonder whether there is a $\\varepsilon_{\\text{train}}$ term omitted at every step when the predictors are imperfect, which will loosen the final bound in this case. I think the authors should add a remark after Lemma 1, or make the Lemma 1 more robust. \n4. I wonder if the self-conditioning breaks the independence assumption used in Lemma 1, and how would that affect the bound?\n\nQUESTIONS:\nPlease refer to the Weaknesses Section above.\n\nLIMITATIONS:\nThere is no negative societal impact of their work,", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper sets out to prove an upper and lower bound on the error of the sampling distribution of a trained masked diffusion model relative to the true sequence distribution it is trained on. The authors bound the expected KL divergence (over choices of mask sets at sampling) above by the training error as well as the mutual information of each token and the rest of the sequence with a 1/L prefactor, where L is the length of the sequence. The authors then refine the bound and show that this 1/L prefactor holds in both the upper and lower bounds with a similar weighted mutual information sum of each token and chunks of the remaining sequence, showing the sampling error bound is tight.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The authors examine a very fundamental theoretical question regarding how far the sampling distribution of masked diffusion models is from the true sequence distribution, and offer a comprehensive answer.\n\n- The authors identify that beyond training error, there is a fundamental error stemming from the product posterior assumption.\n\n- The exposition is extremely well done.\n\nWeaknesses:\n- The proofs in the appendix could be made even more clear with a simple \"English\" summary of some of the equations. For example, in (22), some brief sentence right after like \"We break down the the KL divergence between the \"ideal\" sampling distribution and the sequence distribution into a sum of expected KL divergences between the true and productized marginals at one \"unmasking\" sampling step\" (or something along these lines). Essentially reminding the reader how $D_t$ and similar variables are defined w.r.t. the unmasking sampling process.\n\nQUESTIONS:\n- Are there insights from your theoretical analysis that could illuminate if it is possible to deliberately choose masking sets (or an unmasking ordering based on mutual information) at sampling time to reduce the sampling error from the true distribution?\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-23"}