Paper: Discrete Diffusion Language Modeling by Estimating the Ratios of the Data Distribution (ICLR-2024)

Reviewer comment (Theory):
Proposition 3.2, which states that their score entropy loss has the right minimum, is never proven.

Author response:
* **On reported results**: The work is feature complete (no missing table entries) and thus can be evaluated with the original results. In particular, the submitted results justify our claims (e.g. similar perplexity performance for small models). In particular, authors are allowed to update their manuscript in response to reviewers (and we have done this since other reviewers have asked for medium scale experiments).

[W2] We understand the reviewers' points here. Our intention was not to update the paper after submission without prompting from reviewers. Rather 1) For SEDD-small, since our method was only trained for 350k < 400k iterations (as is typically done), we wanted to emphasize that we weren't experiencing overfitting (and plucking the 350k iteration as an early-stopping criteria) and 2) For SEDD-medium, since we only train for 150k iterations and can thus only compare against SEDD-small (this was to beat PLAID's previous results showing improvement over GPT2-small), we wanted to preempt any potential criticism that we were unfairly comparing small and medium sizes by assuring that we could compare against GPT-2 medium if asked (note that this exact point was made by Reviewer DQWg).

Reviewer comment (Theory):
Proposition 3.3 and Theorem 3.4, which give alternative forms of the score entropy, have "proofs" that are very handwavey and informal, and these proofs appear to be incorrect. The derivations ignore the weights \( w_{xy} \) and thus end up proving something different than the intended proposition/theorem.

Author response:
* **On the notion of competitive**: the connotation of “competitive” is that the perplexity falls within some reasonable percentage of the baseline (for us, we give this explicitly as +10-15%) under similar architecture sizes. This is consistent with prior work which shows that the variational bound induces a +10% increase in perplexity over exact likelihood methods (Song et al. 2021). While this is definitely fungible, we emphasize that our intention was to draw a stark contrast with previous discrete diffusion methods and continuous diffusion (both of which are at least $2\times$ worse), for which no such claim about competitiveness can be made.
* **On compute**: we did not overtain our model: our compute is roughly similar to the compute used for GPT-2. In particular, the GPT-2 training details have not been fully released, but various sources put it at 20, 60, or 100 epochs [1, 2], which corresponds to 266k to 1.3M iterations of batch size 512. This is definitely comparable to our 400k iterations of batch size 512. Note that 400k iterations of batch size 512 is a standard amount of compute used in open source re-creations of GPT-2 [3].
* **On PLAID 1B**: PLAID 1B is around $10\times$ the size of GPT2 small and $3$\times the size of GPT2 medium. It is also trained for 1.2M iterations, which almost surely surpasses the GPT-2 training compute and definitely eclipses our training compute. Since it barely beats GPT2-small and loses pretty handedly to GPT2-medium despite being larger than GPT2-large, we don’t think it’s correct to characterize PLAID as a competitive likelihood-based language model.

[W3] We have updated our manuscript to mention these points explicitly.

Reviewer comment (Presentation):
Although the introduction claims that one contribution of the work is a "Langevin corrector framework", this never appears in the paper.

Author response:
* **On langevin**: We also apologize for the inclusion of the Langevin Corrector bullet point (the content was removed from the paper in a later revision).

Reviewer comment (Experiments):
The provided experiments appear to be only preliminary results. For their SEDD-small model, they "emphasize that it is still improving", and for their SEDD-medium model, they state that it "has not started converging". The authors say they will "update our model results as training progresses".

Author response:
[W2] We understand the reviewers' points here. Our intention was not to update the paper after submission without prompting from reviewers. Rather 1) For SEDD-small, since our method was only trained for 350k < 400k iterations (as is typically done), we wanted to emphasize that we weren't experiencing overfitting (and plucking the 350k iteration as an early-stopping criteria) and 2) For SEDD-medium, since we only train for 150k iterations and can thus only compare against SEDD-small (this was to beat PLAID's previous results showing improvement over GPT2-small), we wanted to preempt any potential criticism that we were unfairly comparing small and medium sizes by assuring that we could compare against GPT-2 medium if asked (note that this exact point was made by Reviewer DQWg).

Reviewer comment (Novelty):
The authors reference the Plaid 1B model from Gulrajani & Hashimoto (2023), which had shown strong non-autoregressive performance relative to GPT-2-small. This seems to contradict the claim that this work is the "first time" non-autoregressive modeling has been competitive with autoregressive modeling.

Author response:
* **On PLAID 1B**: PLAID 1B is around $10\times$ the size of GPT2 small and $3$\times$ the size of GPT2 medium. It is also trained for 1.2M iterations, which almost surely surpasses the GPT-2 training compute and definitely eclipses our training compute. Since it barely beats GPT2-small and loses pretty handedly to GPT2-medium despite being larger than GPT2-large, we don’t think it’s correct to characterize PLAID as a competitive likelihood-based language model.

Reviewer comment (Experiments):
Although motivated as a way to improve upon previously-proposed discrete diffusion approaches, the experiments do not include any discrete diffusion model baselines. The perplexity experiments use different evaluation splits and methods from previous works, preventing direct comparison.

Author response:
[W4]  
* **On diffusion baselines**: we have appended results for standard discrete diffusion methods in the main rebuttal comment and the paper. To summarize, we corroborate existing work which shows that preexisting diffusion methods are poor likelihood learners. We again emphasize that the baseline that we consider is mainly GPT-2, since prior diffusion language models are just not competitive on likelihoods.
* **On data splits and evaluation methods**: we reevaluate GPT-2 on our data splits, so the results are entirely consistent (they are a fair comparison between SEDD and GPT-2 on the same data splits). Additionally, the method we use for evaluating perplexities is effectively just reusing the CrossEntropy loss function, so it generalizes to all language model methods. This is also a fair comparison since now the evaluation pipeline matches between models (and this also matches the train pipeline).
* **On training data**: OpenWebText has been found to be an extremely small confounding variable in prior work. The construction of OpenWebText mirrors the original (closed) WebText dataset used for GPT-2 (it pulled the same reddit URLs), and the dataset is used in any other recreation script for GPT-2 [2, 3]. By comparison, PLAID is trained on a cleaned version of OpenWebText2, which has additional data and is cleaned to remove irrelevant data. Since PLAID tests against GPT-2 despite not controlling for this factor, we believe that our OpenWebText training data is more fair than previously reported diffusion model work.
* **On number of training iterations**: Our comparison is extremely fair here. See above.
* **On confounding variables**: Our added comparison with the original discrete diffusion method shows that a large proportion of the improvement comes from the score entropy loss. In particular, score entropy results in a $2x$ lower perplexity score when controlling architecture, training hyperparametercs, etc…

Reviewer comment (Experiments):
GPT-2-small comparisons may also be confounded by differences in dataset or training iterations used for GPT-2-small, making it difficult to tell how much of the observed gains are due to the new contributions rather than to training procedure, model architecture, or evaluation method.

Author response:
[W4]  
* **On diffusion baselines**: we have appended results for standard discrete diffusion methods in the main rebuttal comment and the paper. To summarize, we corroborate existing work which shows that preexisting diffusion methods are poor likelihood learners. We again emphasize that the baseline that we consider is mainly GPT-2, since prior diffusion language models are just not competitive on likelihoods.
* **On data splits and evaluation methods**: we reevaluate GPT-2 on our data splits, so the results are entirely consistent (they are a fair comparison between SEDD and GPT-2 on the same data splits). Additionally, the method we use for evaluating perplexities is effectively just reusing the CrossEntropy loss function, so it generalizes to all language model methods. This is also a fair comparison since now the evaluation pipeline matches between models (and this also matches the train pipeline).
* **On training data**: OpenWebText has been found to be an extremely small confounding variable in prior work. The construction of OpenWebText mirrors the original (closed) WebText dataset used for GPT-2 (it pulled the same reddit URLs), and the dataset is used in any other recreation script for GPT-2 [2, 3]. By comparison, PLAID is trained on a cleaned version of OpenWebText2, which has additional data and is cleaned to remove irrelevant data. Since PLAID tests against GPT-2 despite not controlling for this factor, we believe that our OpenWebText training data is more fair than previously reported diffusion model work.
* **On number of training iterations**: Our comparison is extremely fair here. See above.
* **On confounding variables**: Our added comparison with the original discrete diffusion method shows that a large proportion of the improvement comes from the score entropy loss. In particular, score entropy results in a $2x$ lower perplexity score when controlling architecture, training hyperparametercs, etc…

Reviewer comment (Evaluation):
The generated samples seem somewhat incoherent in a qualitative sense. In particular, the "infilling" samples in Table 2 are unimpressive; none of them appear to be meaningful or consistent with the provided prompt tokens.

Author response:
[W5]
* **On generation**: We have given additional text samples in the appendix, which show more coherence and a better usage of the prompt tokens. Note that we are working with small and medium model sizes on OpenWebText (a massive dataset), so a bit of incoherence is expected. 

That being said, we emphasize that our work is primarily focused on likelihoods, and our improvements on text generation (which we did not tune for as is already done for autoregressive models like GPT-2 and all other language diffusion models) is a particularly promising (and somewhat unexpected) improvement.

Reviewer comment (Experiments):
The experiments are only provided on text generation and seem unfinished at submission time (due, I guess, to the huge amount of compute time required to train a medium-size GPT2-like diffusion model).

Author response:
* **On experiments**: we have updated the paper (and in the main rebuttal comment) with our finalized numbers. Our updated results show that SEDD-medium vs GPT-2 medium has a similar performance comparison with SEDD-small vs GPT-2 small.

Reviewer comment (Experiments):
On Table 1, the SEDD-medium results are provided, but the equivalent results for medium-size GPT2 must be provided as well otherwise it could be misleading (I hope this will be fixed at the rebuttal time).

Author response:
* **On GPT2-medium in table 1**: we have updated the table to include this. Note that our original table 1 showed SEDD-A Medium just to confirm that it outperformed GPT2-small (which shows improvement over PLAID which required 1B and full compute to beat GPT-2).

Reviewer comment (Presentation):
I found too many ArXiv references: please update your bibliography for per-reviewed versions when possible. For instance A. Campbell et al. 2022 was presented at NeurIPS and should be cited as such.

Author response:
* **On arxiv references**: apologies, this has been updated. We have added a non-arxiv citation for every paper we could find in our bibliography.

Reviewer comment (Presentation):
On Figure 2 it was unclear to me if the compared models were of SEDD-small or SEDD-medium. I guess it is SEDD-small otherwise it could be misinterpreted as deceiving.

Author response:
* **On figure 2 SEDD size**: the model architecture was small. It has now been updated to include both small and medium results (samples are still on small model sizes).

Reviewer comment (Evaluation):
On figure 2: to my experience, using Large-GPT2 perplexity to evaluate smaller models is a good idea but it can be misleading for really bad models (a trivial sequence with a repeated single token being extremely easy to predict, it will reach a very low GPT2 perplexity). I found no solution to this problem. Maybe adding another criterion?

Author response:
* **On generative perplexity as a metric**: we are aware of this fundamental issue with generative perplexity. However, as we mention in the paper, this is primarily a problem with alternative (non-analytic) sampling schemes. For instance, greedy sampling often degenerates into a repeated word (e.g. “the the the …”), which hacks our metric (resulting in extremely low perplexities). 

&ensp; In general, generative perplexity is a valid metric when comparing ancestral sampling, especially when paired with standard perplexity. The probabilistic interpretation is that the standard perplexity represents the standard KL divergence between the data and the probabilistic model, while generative perplexity represents a reverse KL. If standard perplexity is reasonable (as is in our case), then our probabilistic model “covers” all the modes of the data distribution. If generative perplexity is reasonable (again as is in our case), then our probabilistic model doesn’t place mass on unlikely parts of the data distribution.

&ensp; Furthermore, we manually checked our samples to ensure that our improved results were apparent in the text generation and not a result of some degeneration of the probability distribution. We have included some more text examples in the appendix.

&ensp;  Generally, we found other metrics to be difficult to use. We considered using a generalization of FID, but the issue is that GPT-2 was trained on WebText while our model was trained on OpenWebText. As such, it is hard to produce a consistent validation/test set that would be fair to both models (even though the data is roughly the same, we don’t know the splits).

Reviewer comment (Theory):
The paper presents extensive detail about the scaling properties of the proposed criterion but omits to explain the unusually complicated form of the criterion itself. The derivations provide many useful consequences of Eq. (9), but do not explain the origin of Eq. (9). This omission is notable because there may be a small typo in Eq. (9).

Author response:
Thank you for your positive review! 

For equation 9, our motivation was initially to scale the gradient of the concrete score matching to make it more amenable for the positivity of the ratios (as we mentioned in the paper). This led to the core $s - \frac{p(y)}{p(x)} \log s$ loss function, from which we found a connection with the ELBO. Ultimately, the constant term was added to ensure that the ELBO behaved properly (greater than or equal to $0$).

However, we reexamined the equation given in the paper. The reviewer is right, there is a typo! The constant should be of the form $a(\log a - 1)$ instead of $a \log(a - 1)$, where $a$ is the ratio. In particular, this was the constant used for denoising score entropy for the ELBO. As such, the reviewer’s points about Bregman divergence are particularly relevant. We have updated the paper to include the Bregman divergence motivation and will credit the reviewer for their astute observations in the acknowledgements (note that the acknowledgements are omitted submission as per ICLR policy). Thanks again for your suggestion here.

Reviewer comment (Theory):
The reviewer is almost able to derive Eq. (9) by assuming it is a Bregman divergence between s(x,y) and p(y)/p(x), using -log as the convex function. This derivation would make sense, as it would guarantee that the score divergence is non-negative, reflexive, and convex in s(x,y); these properties are stated in the paper but not proven, possibly because they follow naturally from the Bregman divergence.

Author response:
Thank you for your positive review! 

For equation 9, our motivation was initially to scale the gradient of the concrete score matching to make it more amenable for the positivity of the ratios (as we mentioned in the paper). This led to the core $s - \frac{p(y)}{p(x)} \log s$ loss function, from which we found a connection with the ELBO. Ultimately, the constant term was added to ensure that the ELBO behaved properly (greater than or equal to $0$).

However, we reexamined the equation given in the paper. The reviewer is right, there is a typo! The constant should be of the form $a(\log a - 1)$ instead of $a \log(a - 1)$, where $a$ is the ratio. In particular, this was the constant used for denoising score entropy for the ELBO. As such, the reviewer’s points about Bregman divergence are particularly relevant. We have updated the paper to include the Bregman divergence motivation and will credit the reviewer for their astute observations in the acknowledgements (note that the acknowledgements are omitted submission as per ICLR policy). Thanks again for your suggestion here.

Reviewer comment (Theory):
Based on the reviewer's derivation, there appears to be a typo in Eq. (9): the last term should not be (p(y)/p(x))\log(p(y)/p(x)-1), but rather (p(y)/p(x))(log(p(y)/p(x))-1). The correction is necessary because log(p(y)/p(x)-1) may often involve taking the logarithm of a negative number, which can be avoided by using log(p(y)/p(x))-1 instead.

Author response:
Thank you for your positive review! 

For equation 9, our motivation was initially to scale the gradient of the concrete score matching to make it more amenable for the positivity of the ratios (as we mentioned in the paper). This led to the core $s - \frac{p(y)}{p(x)} \log s$ loss function, from which we found a connection with the ELBO. Ultimately, the constant term was added to ensure that the ELBO behaved properly (greater than or equal to $0$).

However, we reexamined the equation given in the paper. The reviewer is right, there is a typo! The constant should be of the form $a(\log a - 1)$ instead of $a \log(a - 1)$, where $a$ is the ratio. In particular, this was the constant used for denoising score entropy for the ELBO. As such, the reviewer’s points about Bregman divergence are particularly relevant. We have updated the paper to include the Bregman divergence motivation and will credit the reviewer for their astute observations in the acknowledgements (note that the acknowledgements are omitted submission as per ICLR policy). Thanks again for your suggestion here.

Reviewer comment (Theory):
The potentially incorrect last term in Eq. (9) is largely ignored in the rest of the paper since it does not involve s_\theta(x); it seems to be necessary only for shifting the criterion upward so that it is strictly non-negative.

Author response:
Thank you for your positive review! 

For equation 9, our motivation was initially to scale the gradient of the concrete score matching to make it more amenable for the positivity of the ratios (as we mentioned in the paper). This led to the core $s - \frac{p(y)}{p(x)} \log s$ loss function, from which we found a connection with the ELBO. Ultimately, the constant term was added to ensure that the ELBO behaved properly (greater than or equal to $0$).

However, we reexamined the equation given in the paper. The reviewer is right, there is a typo! The constant should be of the form $a(\log a - 1)$ instead of $a \log(a - 1)$, where $a$ is the ratio. In particular, this was the constant used for denoising score entropy for the ELBO. As such, the reviewer’s points about Bregman divergence are particularly relevant. We have updated the paper to include the Bregman divergence motivation and will credit the reviewer for their astute observations in the acknowledgements (note that the acknowledgements are omitted submission as per ICLR policy). Thanks again for your suggestion here.

Reviewer comment (Theory):
There is a question about the origin of Eq. (9) and a need to check for a possible typo.

Author response:
Thank you for your positive review! 

For equation 9, our motivation was initially to scale the gradient of the concrete score matching to make it more amenable for the positivity of the ratios (as we mentioned in the paper). This led to the core $s - \frac{p(y)}{p(x)} \log s$ loss function, from which we found a connection with the ELBO. Ultimately, the constant term was added to ensure that the ELBO behaved properly (greater than or equal to $0$).

However, we reexamined the equation given in the paper. The reviewer is right, there is a typo! The constant should be of the form $a(\log a - 1)$ instead of $a \log(a - 1)$, where $a$ is the ratio. In particular, this was the constant used for denoising score entropy for the ELBO. As such, the reviewer’s points about Bregman divergence are particularly relevant. We have updated the paper to include the Bregman divergence motivation and will credit the reviewer for their astute observations in the acknowledgements (note that the acknowledgements are omitted submission as per ICLR policy). Thanks again for your suggestion here.

Reviewer comment (Theory):
There is a request for more explanation of the phrase "from considering the fully connected graph structure and the MASK token." The reviewer notes that Q_uniform appears to converge toward a noisy distribution in which all tokens are equally likely, which is unlike the steady-state distribution of all fully-connected Markov processes, making it unclear that "fully connected" is a sufficient motivation. However, the uniform distribution does maximize entropy, which is sufficient motivation. Additionally, the reviewer asks whether Qabsorb is converging toward a noisy distribution in which the mask token replaces all other tokens.

Author response:
For uniform, perhaps it would’ve been more precise to say that the graph structure is uniformly/simplicially connected, with an equal edge weight for all edges. This was chosen mostly because this graph appears quite a bit in prior work.

Reviewer comment (Experiments):
The experiment lacks good baselines. Although the paper claims to improve over concrete score matching, they do not consider CSM in their baseline and only compare the proposed SEDD with an autoregressive model (GPT-2 small). Moreover, SEDD-medium is compared with GPT-2 small.

Author response:
* **On baselines**: We have updated our results to include other diffusion model baselines (see main rebuttal to all reviewers), which shows our improvements over the baseline diffusion methods. 
* **On Concrete Score Matching**: Please check out our overall comment to all reviewers. As expected, CSM results in a much worse optimization because of the issues we describe.

Reviewer comment (Experiments):
The experiment appears inconclusive or incomplete. The model is still being trained, and the authors claim that it has not converged yet; the experiment on the 1 billion-word dataset is said to have encountered unexpected errors, without elaboration.

Author response:
* **On experiments**: We have updated our finalized numbers in the comment to all reviewers (and in the updated manuscript). This includes the medium size results, which show that SEDD-medium vs. GPT-2 medium has the same performance comparison as SEDD-small vs GPT-2 small (similar perplexities, better generation quality). Note that our original submission only compared a preliminary version of SEDD-medium with GPT2-small to show that our method scales (and also to beat the previous PLAID method which required 1B parameters to beat GPT-2 small).
* **On 1BW**: For the 1BW dataset, we found that the public implementation of GPT-2 and 1BW on huggingface resulted in abnormal behavior (unable to reproduce reported results). As such, we include it for completion purposes but use the reported results.

Reviewer comment (Writing):
Not all variables and notations are clearly specified, making the paper difficult to follow at times.

Author response:
* **On variables**: apologies. We have updated our manuscript which should help with this problem. If you have a more specific equation that is unclear, please feel free to let us know.

Reviewer comment (Experiments):
Could you add a CSM baseline for an accurate and fair comparison between SEDD and CSM and also provide final, updated metrics of converged models for clarity of analysis?

Author response:
* **On CSM + baselines**: We have done so. See above. In particular, CSM behaves poorly, as expected.

Reviewer comment (Experiments):
How does SEDD-medium compare to GPT-2 medium?

Author response:
* **On medium sized models**: we see a similar dynamic between SEDD-medium and GPT-medium as we do in the small case. In particular, we get a similar performance with perplexities and see the same generation improvements. This has been updated in the main figure.

Reviewer comment (Evaluation):
Is SEDD comparable with GPT-2 in terms of latency in inference? Although they may have a similar number of network evaluations (as in Fig. 2), AR models can leverage techniques like KV-caching, which probably makes them significantly faster than diffusion models.

Author response:
* **On latency**: KV-caching does greatly increase the speed of sampling for AR models, as we allude to in our limitations section. For our purposes, autoregressive generation with KV cache is normally about as fast as 64 step diffusion, but we emphasize that a valid latency comparison would require comparison over, e.g. multiple GPU types, optimized pytorch operations, etc… which is outside the scope of our paper. Furthermore, we leave the question of improving our non-AR generation (whether through theory, algorithmic tricks, or GPU optimization) to future work.

Reviewer comment (Writing):
What does z denote in Eq. (7), and what does Q_t (y, x) mean in Eq. (6) (Q_t is originally introduced as a matrix in Eq. (5))?

Author response:
* **On confusion in equations**: (7) the summation should be z != x, which explains what z is (as we iterate over that index instead of k, which was a typo). (6): Q_t(y, x) refers to indexing at the y-th row and x-th column (this is done with parenthesis to avoid overloading the subscript).

Reviewer comment (Evaluation):
Evaluation is a little bit weak. Like, lack of comparision between previous discrete or continous diffusion mdoels, including the sampling speed and generation quality. No specific numbers of sampling speed (only the caption of Fig 2 mentioned once). No quantitative evaluation for infilling tasks, just showing some examples.

Author response:
The review seems to want a better contextualization with previous work, especially for generation quality. We emphasize that our work was primarily focused on perplexity (outperforming previous diffusion model work for this metric and finally challenging autoregressive models), as perplexity is a critical metric for language modeling. However, we have also presented new evidence (in the main rebuttal to all reviewers) that our generation quality is significantly better than that of prior diffusion methods (despite our method not being trained for sample generation), which should alleviate the reviewer’s concerns. We hope the reviewer can take these points into consideration when reconsidering their review and rating and are happy to answer any more unanswered questions.

To address the weaknesses
* **On weak evaluation**: Our evaluation is complete as it is a thorough comparison with the baseline (which is GPT-2). We evaluated both likelihoods (in the form of perplexities) as well as generation quality (using generative perplexity), which is typical for any generative modeling task. Note that the previous paper which has scaled language diffusion models to GPT-2 experiments (PLAID) does not test generation quality, so our evaluation is more complete than what has been done in the literature.
* **On infilling qualitative examples**: The use of qualitative examples for infilling is typical for diffusion modeling (e.g. the inpainting experiments in Song et. al 2021, conditional sample generation for PLAID), and this paradigm is especially applicable in our case since GPT-2 does not have this functionality as an autoregressive model.
* **On lack of comparison with other diffusion methods**: we have added experiments showing that our method achieves better perplexities and generation quality when compared with previous methods. 
* **On comparing sample generation vs other diffusions**:  see point above.
* **Specific numbers for sampling speed**: the number of network evaluations/sampling iterations was chosen from [64, 128, 256, 512, 1024, 2048].

Reviewer comment (Experiments):
Some motivations are not clear. Section 4 is not well presented. Why the design of this denoising scheme is needed? If it is designed for speedup, you need to explain two things: (1) no detailed experiments or ablation study about this strategy (2) some discrete diffusion models can sampling within several steps (like~10), and in such condition, discrete diffusion models already have the advantages over generation speed, so what's the difference between theirs and yours?

Author response:
* **On denoising scheme:** This denoiser was motivated by tweedie’s theorem for score functions in continuous space. It results in a slightly improved performance (Appendix C.2) and is included to show how the information given by the concrete score can be used to augment the sampling process rather than just using the Euler step.
* **On difference with previous discrete diffusion model sampling schemes**: we are fundamentally learning a different value (the concrete score, not the predicted mean). As such, our developed samplers are fundamentally different since the information we are given is different from previous models.
* **On fast discrete diffusion model sampling schemes**: we are not aware of any prior discrete diffusion model that is able to generate high quality text (e.g. OpenWebText or even 1BW training data, GPT-2 scale models, thorough comparison with baseline autoregressive model) with 10 steps. We are aware of some papers (such as Zheng et al.) which show reasonable generation quality on much simpler tasks, but this has not been shown to scale and evaluates using the statistically weaker GLUE score. If you have a reference in mind, please feel free to let us know and we will take a look and continue this discussion.

Reviewer comment (Writing):
Writing: cictation format (citep and citet) is mixed up.

Author response:
* **On citep vs citet**. Apologies. This has been fixed.

Reviewer comment (Evaluation):
Can we directly compare the perplexity of AR and diffusion models? (In table 1) The definition of perplexity in AR is a little bit different from the NLL in diffusion models.

Author response:
* **On NLL vs perplexity** Yes, we can directly compare NLL vs Perplexity. This is done before in the previous diffusion language model papers and follows exactly from the fact that $e^x$ is monotonic:
	
$$ \mathrm{NLL bound} \ge \mathrm{NLL} \implies e^{\mathrm{NLL bound} / L} \ge e^{\mathrm{NLL} / L} = \mathrm{Perplexity}$$

&emsp; We emphasize that our results are entirely consistent with the existing literature. As such, our results indicate that we have effectively closed the perplexity gap (with autoregressive models) that plagued prior diffusion work. This is again a very important point, since language modeling experiments are most often tested on perplexities. Furthermore, since we can only report a bound, our true perplexity is likely much closer with GPT-2 than the given +10% difference

Reviewer comment (Presentation):
In Fig 2(a), what is the number of network evaluations? It seems that it does not refer to sampling iterations. GPT-2 is also in this Figure, what is the number of network evaluations or sampling iterations of GPT-2?

Author response:
* The number of network evaluations refers to sampling steps. GPT-2 uses 1024 since it is a fixed length autoregressive generation. We vary our number of sampling steps from 64 to 2048 as previously mentioned.

Reviewer comment (Theory):
It seems that you use $Q_t(i,j)$ (Eq.14) to replace the $w_{xy}$ in Eq.11. However, due to memory limitation, you choose standard matrices Q_uniform and Q_mask, where the value in these matrices is either 0 or 1. Can we assume that the Eq.14 is degenerated and we actually do not have the soft weighted denoising score entropy?

Author response:
* On “degeneration of equation 14”. The matrix entries will change based on the noise schedule $\sigma(t)$ and are not $0$ or $1$. Even in the absorbing case, when many values are $0$, we can still recover the relevant scores used for parameterizing the reverse process.