Title: Fast Sampling via Discrete Non-Markov Diffusion Models with Predetermined Transition Time
Venue: NeurIPS 2024
Accepted: yes
Keywords: discrete Diffusion

sampling acceleration

text generation
Abstract: Discrete diffusion models have emerged as powerful tools for high-quality data generation. Despite their success in discrete spaces, such as text generation tasks, the acceleration of discrete diffusion models remains under-explored. In this paper, we propose discrete non-Markov diffusion models (DNDM), which naturally induce the predetermined transition time set. This enables a training-free sampling algorithm that significantly reduces the number of function evaluations (i.e., calls to the neural network), making the sampling process much faster. Furthermore, we study the transition from finite to infinite step sampling, offering new insights into bridging the gap between discrete and continuous-time processes for discrete diffusion models. Extensive experiments on natural language generation and machine translation tasks demonstrate the superior performance of our method in terms of both generation speed and sample quality compared to existing methods for discrete diffusion models. Codes are available at \url{https://github.com/uclaml/DNDM}.

Fast Sampling via Discrete Non-Markov Diffusion Models with Predetermined Transition Time

Abstract

Discrete diffusion models have emerged as powerful tools for high-quality data generation. Despite their success in discrete spaces, such as text generation tasks, the acceleration of discrete diffusion models remains under-explored. In this paper, we propose discrete non-Markov diffusion models (DNDM), which naturally induce the predetermined transition time set. This enables a training-free sampling algorithm that significantly reduces the number of function evaluations (i.e., calls to the neural network), making the sampling process much faster. Furthermore, we study the transition from finite to infinite step sampling, offering new insights into bridging the gap between discrete and continuous-time processes for discrete diffusion models. Extensive experiments on natural language generation and machine translation tasks demonstrate the superior performance of our method in terms of both generation speed and sample quality compared to existing methods for discrete diffusion models. Codes are available at https://github.com/ uclaml/DNDM .

1 Introduction

Diffusion-based generative models, as first introduced by Sohl-Dickstein et al. (2015), have shown remarkable capabilities in generating high-quality samples across various domains, including images (Ho et al., 2020; Song and Ermon, 2020), audio (Chen et al., 2020; Kong et al., 2020), and videos (Ho et al., 2022). The diffusion model utilizes an innovative approach comprising a forward process that gradually transforms training data into pure noise and a reverse process that reconstructs clean data from the noise. Throughout the training phase, the model optimizes a neural network by minimizing an objective derived from maximum likelihood estimation. Once trained, the model can generate samples using various decoding strategies, including implicit dynamics (Song et al., 2020a), analytical processes (Bao et al., 2022), or differential equation solvers (Song et al., 2020b; Liu et al., 2022; Lu et al., 2022). In particular, Song et al. (2020a) introduced the denoising diffusion implicit model (DDIM), providing a non-Markov and de-randomized version of the Denoising Diffusion Probabilistic Model (DDPM) (Sohl-Dickstein et al., 2015; Ho et al., 2020), which enables faster generation of high-quality samples.

Although diffusion models were initially introduced for both discrete and continuous-state spaces (Sohl-Dickstein et al., 2015), these studies have largely focused on Gaussian diffusion processes in continuous-state spaces. Recently, Discrete Denoising Diffusion Probabilistic Models (D3PMs) (Austin et al., 2021) working in discrete-state spaces have gained increasing interest due to their applications in diverse areas such as text generation (Hoogeboom et al., 2021b), medical record generation (Ceritli et al., 2023), and protein design (Gruver et al., 2024). These models, which are distinct from their Gaussian counterparts, employ discrete noises, such as the multinomial distribution, for diffusion processes. Very recently, Zheng et al. (2023) introduced a reparameterized diffusion

model (RDM) that can improve sampling speed and sample quality in text generation tasks. However, their proposed algorithm is a training-based approach. Compared with diffusion models using Gaussian noise, discrete diffusion models remain under-studied, especially regarding training-free sampling acceleration.

In this work, we introduce a training-free approach aiming at enhancing the sampling speed of discrete diffusion models. This approach stems from a unique characteristic of discrete diffusion models: unlike continuous diffusion models, which typically employ Gaussian noise for data corruption (Ho et al., 2020; Song and Ermon, 2020; Song et al., 2020b,a), discrete diffusion models often use categorical white noises (Hoogeboom et al., 2021b; Austin et al., 2021; Zheng et al., 2023).

By delving into this special property, we develop a discrete non-Markov diffusion model, together with a design of accelerated algorithm. Notably, this new sampling technique does not require any modifications to the training objec-

tive of diffusion models and is, therefore, training-free. Our contributions are summarized as follows:

We propose discrete non-Markov diffusion models (DNDM), which naturally induces a set of latent variables T , termed as the transition time set . This key feature enables us to develop a training-free sampling algorithm that can accelerate a large family of discrete diffusion models. Importantly, DNDM preserves the essential properties of the original discrete diffusion model: for any diffusion trajectory { x t } starting from real data x 0 , it provably maintains both the marginal distribution q ( x t ) and the conditional distribution q ( x 0 | x t ) . Our method can accelerate the two most widely used discrete diffusion models: multinomial diffusion (Hoogeboom et al., 2021b) and absorbing diffusions (Austin et al., 2021). Similar to how DDIM introduces a de-randomized, faster sampling algorithm compared to DDPM in continuous space, DNDM achieves acceleration through a predetermined transition time set in discrete space (See Table 1).

Based on the predetermined transition time set T in DNDM, we design an accelerated sampling algorithm that reduces the required number of neural network function evaluations. In a standard T time-step discrete diffusion process, while D3PM, including Multinomial (Ho et al., 2020) and absorbing state discrete sampling (Austin et al., 2021), requires evaluating the neural network function T times, our approach only requires |T | function evaluations, where |T | is the cardinality of the transition set T . Moreover, |T | is provably less than T and approaches O (1) as T goes to infinity. We provide both theoretical analysis and empirical experiments showing that the improvement in the number of function evaluations (NFE) is significant. Notably, our algorithm is about 3 × faster than baselines for T = 50 and about 30 × faster for T = 1000 while preserving the sample quality.

To further illustrate the effectiveness of DNDM, we explore the limit as T →∞ and introduce an infinite-step sampling algorithm. With a pretrained neural network, we can generate an initial noise x T and a transition time set T ⊆ [0 , 1] with infinitesimal spacing, such that |T | = O (1) . This enables the generation of the real data distribution with only |T | neural network evaluations. This study offers new insights into bridging the gap between discrete and continuous-time processes for discrete diffusion models.

Notation. We use |T | to denote the cardinality of the set T (excluding repeated elements). We use lowercase letters to denote scalars, boldface lowercase letters to denote vectors, and boldface uppercase letters to denote matrices. The notation 1 : N indicates the sequence from 1 through N . The symbol q designates the real distribution in a diffusion process, while p represents the distribution during sampling. With its success probability inside the parentheses, the Bernoulli distribution is denoted by Bernoulli( · ) . We further use Cat( x ; p ) to denote a categorical distribution over a one-hot row vector x with probabilities given by the row vector p .

2 Background

In this section, we provide the background of discrete diffusion models. We begin by introducing the discrete Markov diffusion model, designed for handling categorical random variables. Specifically,

consider a diffusion model trying to generate distributions over a discrete random variable x ∈ R K that is one-hot encoded with K categories, i.e., x can be chosen as one of K categories, and for any k ∈ [ K ] , x is categorized as k if x aligns with the standard basis vector e k . The sequence { x t } T t =0 represents how this random variable changes over time 0 ≤ t ≤ T , starting from an x 0 ∈ R K drawn from the real distribution q data . In this paper, we focus on the two most widely used D3PMs: multinomial diffusion (Hoogeboom et al., 2021b) and absorbing diffusions (Austin et al., 2021).

Forward Process. During the forward process, the real distribution q data is gradually transformed into a noise distribution named q noise . The transformation occurs through T steps, with T intermediate latent variables x 1 , . . . x T and update rules given by:

Here b t is randomly drawn from a Bernoulli distribution with parameter β t , denoted by b t ∼ Bernoulli( β t ) , and w t is randomly drawn from the noise distribution q noise , while for different t the samples are independent. In this work, we focus on cases where the noise q noise can be either a uniform distribution over the vocabulary { 1 , 2 , . . . , K } (Hoogeboom et al., 2021b), or a point mass with all of the probability mass lying on an absorbing state (Austin et al., 2021). Following this notation, the process in (1) defines a Markov process characterized by the transition kernel

Moreover, the Markov chain property allows us to get samples x 0: t from x 0 by multiplying the transition probabilities at each step as p ( x 1: t | x 0 ) = ∏ t i =1 q ( x t | x t -1 ) . It further leads to the following marginal distribution.

where α t := Π t s =1 β s is determined by the sequence of β t of our choice and decreases from 1 to 0 .

Reverse Process. Given the forward Markov process, the reverse process can be derived by Bayes' rule (Hoogeboom et al., 2021b; Austin et al., 2021; Zheng et al., 2023). The conditional probability q ( x t -1 | x 0 , x t ) can be determined by q ( x t -1 | x 0 , x t ) = q ( x t | x t -1 ) q ( x t -1 | x 0 ) /q ( x t | x 0 ) . The reverse process can be used for synthetic data generation by sampling from the noise distribution q noise and repeatedly applying a learned predictor (neural network) p θ ( ·| x t ) parameterized by θ :

We note that the reverse process q ( x t -1 | x t , ̂ x 0 ) is stochastic and thus requires function evaluation at every step.

Training the Neural Network. The neural network p θ ( ·| x t ) that predicts ̂ x 0 is trained by maximizing the evidence lower bound (ELBO) (Sohl-Dickstein et al., 2015),

Here KL denotes Kullback-Liebler divergence and the last term E q ( x T | x 0 ) KL( q ( x T | x 0 ) ∥ q noise ( x T )) equals zero. Building on this foundation, Austin et al. (2021) introduced an auxiliary denoising objective, which refines the data predictions x 0 at each time step. Since this paper primarily focuses on reverse sampling, we leave detailed discussions of these losses to Appendix B.

3 Discrete Non-Markov Diffusion Models (DNDM)

3.1 Forward and Reverse Process

In this section, we introduce a non-Markov process such that the joint distribution of ( x 0 , x t ) remains the same as the one defined with Markov process in Section 2. The new process aims to gradually

transform input data q data to the noise distribution q noise through T intermediate latent variables x 1 , . . . x T with the following process:

where b t is independently drawn from the Bernoulli distribution Bernoulli( β t ) and w is drawn from the noise distribution q noise . The only difference between (6) and (1) is that we replace w t in (1) by w , which is time-invariant during the diffusion. Therefore, the process in (6) becomes non-Markov since q ( x t | x t -1 , . . . , x 0 ) doesn't necessarily equals q ( x t | x t -1 ) . The following theorem shows that the conditional distribution q ( x t | x 0 ) remains unchanged.

Theorem 3.1. For the non-Markov process in (6) , we have

where α t := Π s i =1 β s is specified to decrease from 1 to 0 .

Using the Bayes' rule, we have q ( x 0 | x t ) ∝ q ( x t | x 0 ) q ( x 0 ) . Consequently, the condtional distribution q ( x 0 | x t ) remains consistent with the one induced by the process process in (1). Therefore, neural network p θ ( ·| x t ) trained by the Markov process in (1), remains applicable to our non-Markov process (6) (see Appendix B for detail).

Based on the discrete non-Markov diffusion model, we can give a simple characterization of the reverse process by introducing the transition time.

Definition 3.2. Transition time τ is the time that the token x t transition from x 0 to noise, i.e., τ := min t { t | b t = 0 } .

Remark 3.3 . The concept of transition time has also been introduced in Hoogeboom et al. (2021a). However, Hoogeboom et al. (2021a) restricts the transition time to be the first time of entering the absorbing state, which is only applicable to absorbing diffusion. Our definition is more general and applicable to discrete diffusion with various noise including multinomial diffusion.

Given the transition time τ , the forward process reduces to:

which shows that the token will be a real token x 0 before the time τ and will be the noise w after the transition time. Since token only get changed at the transition time τ , we can derive a reverse process based on (7),

̸

Therefore, the process in (8) is de-randomized given transition time τ . Specifically, after independently sampled transition times τ , x t -1 becomes deterministically known and fixed if we observe x 0 and x t . It is also worth noting that given x 0 and τ , the exact reverse process (8) is Markovian, since x t -1 solely depends on x 0 , τ, x t . Plugging (8) into (4) gives the generation process. We can prove the ELBO of the DNDM is equivalent to the ELBO of the original process (5) up to some constant, which further supports the neural network p θ ( ·| x t ) trained by the Markov process in (1), remains applicable to DNDM. (See Appendix B.3 for details).

Remark 3.4 . (7) and (8) suggest that even though there are T distinct time steps, not every time in the range 1 : T is crucial for capturing the process. Therefore, our primary focus should be on the most significant time step, i.e., the transition time τ , enabling faster reverse sampling. We further note that although transition happens only at time τ , the transition time is random, differs across runs, and covers the full range from 1 to T on average.

Remark 3.5 . While Song et al. (2020a) proposed a non-Markov multinomial diffusion model in Appendix A, DDIM and DNDM are fundamentally different models when specialized to multinomial diffusion. DDIM's discrete process remains stochastic at every step, even with deterministic noise scheduling. In contrast, DNDM achieves full de-randomization by pre-determined transition time τ (Equation 8 in our paper). By sampling these transition times upfront, DNDM establishes a predetermined transition time set that guides the sampling process, enabling deterministic evolution and faster sampling speed even under the same number of sampling steps, which is not reported under DDIM framework. For detailed technical comparison, see Appendix B.1.

3.2 Accelerated Reverse Sampling

In this section, we demonstrate that sampling from DNDM can lead to accelerated reverse sampling. Although our algorithm is quite general, we focus on text generation in the presentation.

In Section 3.1, we only consider the case of a single token x ∈ R K being one hot encoding of K categories. In real applications, we are interested in generating a sentence with multiple tokens. So, we extend the terminology in Section 3.1, and we denote the sequence of tokens at t -th time step to be x t, 1: N = [ x t, 1 , . . . , x t,N ] where x t,n is the n -th token and N is the sequence length. The noise will be added to each token in a sequence independently. Therefore, each token will have its own transition time defined in Definition 3.2. We denote the transition time for each token x n to be τ n and further denote the transition time set T := { τ n } N n =1 . Given the transition times τ n ∈ T , our DNDM can now be extended to the sequence with multiple tokens

̸

Learning the Reverse Process. We first generate the transition times τ n for n ∈ [ N ] , then we follow (9) to generate the learned reverse process. Since x 0 ,n is unknown in the process, we use the neural network evaluation p θ ( ·| x t ) obtained in Section 3.1 to predict x 0 ,n . In detail, the noisy sequence x t, 1: N is fed into p θ ( ·| x t, 1: N ) and the prediction tokens ̂ x 0 , 1: N ∼ p θ ( ·| x t, 1: N ) are collected.

Transition time. Transition time, denoted by τ , is crucial in our reverse process. This is because the reverse sampling becomes deterministic upon using (9). Each instance of transition time τ is a random variable within the set { 1 , 2 , . . . , T } . Let's assume it follows the distribution D τ . Given the schedule { α t } T t =0 , we can derive the distribution for D τ .

Theorem 3.6. Each specific transition time τ n in Definition 3.2 is independent. Furthermore, they collectively adhere to the distribution D τ , which obeys the rule P ( τ n = t ) = α t -1 -α t .

From Theorem 3.6, we discern that the nature of the diffusion model scheduler, α t , clarifies the distribution of τ . Take the linear schedule as an example, as given by Austin et al. (2021), the relationship is α t = 1 -t/T . This translates to P ( τ n = t ) = 1 /T for every t in the range 1 to T . As a result, transition time distributes uniformly across each moment in the set { 1 , . . . , T } . Generally, if we express α t as g ( t/T ) , then we can simplify to P ( τ n = t ) = g (( t -1) /T ) -g ( t/T ) , which further refines to (1 /T ) | g ′ ( t/T ) | + o (1 /T ) . This indicates that transitions are more likely where | g ′ | is large.

In practice, we observed that the shape of the transition time does not need to exactly match the theoretically predicted schedule D τ in Theorem 3.6. Algorithm 1 works even if D τ is unknown. In particular, we can approximate the schedule with a Beta distribution by first sampling a time t ∈ [0 , 1] from a Beta distribution, then adjusting these samples to fit by multiplying by T and rounding the result to obtain an integer.

Accelerated Sampling. According to (9), a token x t -1 ,n is updated only if step t is the transition time for the n -th token. If step t is not the transition time for any token, the sentence from the previous step can be directly copied: x t -1 , 1: N = x t, 1: N . As a result, there is no need to do a function evaluation for the current step. Our attention, therefore, can be solely centered

Algorithm 1 Sampling From DNDM

on the transition set T , necessitating function evaluations only for t within T . For our method, when N is fixed while T → ∞ , the total NFE |T | will reach N . On the other hand, when T is fixed and N → ∞ , the NFE T will reach T (See Theorem D.1 for detail). It is worth noting that the auto-regressive diffusion model (ARDM) (Hoogeboom et al., 2021a) can also achieve at most N NFE when T = ∞ . However, ARDM only focuses on infinite time steps, while our method here is

able to accelerate sampling for finite time steps. More detailed discussion and theoretical analysis can be found in Section D, where additional experiments also demonstrate that our DNDM achieves an NFE that is less than half of the original Markov sampling method for discrete diffusion.

By incorporating the forward process with different noises, we can develop DNDM-Multi and DNDMAbsorb, which accelerate the Multinomial and Absorbing sampling methods respectively. Recent works have demonstrated that the quality of samples can be enhanced by utilizing supplementary information derived from the neural network, (Ghazvininejad et al., 2019; Savinov et al., 2021; Chang et al., 2022; He et al., 2022; Zheng et al., 2023). Our DNDM can also be improved using this idea. We call it a discrete non-Markov Diffusion Model with Top-k Transition Time (DNDMk ). Due to the limit of the pages, we leave the detailed Algorithm and discussion to Appendix E.

3.3 Continous-time (Infinite Step) Reverse Sampling

In the context of continuous state spaces, continuous-time processes have been proposed to accommodate algorithms that offer faster sampling speeds and enhanced sample quality (Jolicoeur-Martineau et al., 2021; Zhang and Chen, 2022; Salimans and Ho, 2022; Chung et al., 2022; Song et al., 2020b; Dockhorn et al., 2021). However, the application of continuous-time schemes to discrete-state spaces remains largely unexplored. Campbell et al. (2022) first developed a continuous framework for discrete-time diffusion for the Markovian process and randomized sampling, but not in our nonMarkovian setting. In this section, we investigate the transition from finite to infinite step sampling, providing new insights into bridging the gap between discrete and continuous-time processes for discrete diffusion models.

Continuous-time Forward and Backward process. Recall that the forward process described in (6) can be sampled from x 0 ,n through the following process:

In the previous section, we are constrained to discrete time steps, where we must define a maximum step, denoted by T . The values of x t are computed only for t = 1 , . . . , T . As a result, during the training process, it is only possible to predict x 0 at these predetermined time steps. This constraint confines the computation of our reverse process exclusively to these fixed time stamps. To derive the continuous limit of (10), for each T we rescale (10) to a diffusion process on [0 , 1] , e.g., x T,n = ̂ x 1 ,n , x 0 ,n = ̂ x 0 ,n , and x t,n = ̂ x t/T,n . Therefore, when T →∞ , ̂ x t,n represents the continuous process that has values at arbitrary t ∈ [0 , 1] . If the choice of α t for

Algorithm 2 Sampling from DNDM-C

Require: Trained prediction function p θ , q noise , D τ

1: for n = 1 . . . N do

2: Initiate each token x T,n ∼ q noise

3: Initiate the transition time τ n ∼ D τ and order them as τ n 1 < . . . < τ n N

4: end for

5: for k = N . . . 1 do

6: Generate ˜ x 0 , 1: N from p θ ( ·| x τ n k , 1: N , τ n k )

7: for n = 1 . . . N do

8: Update x τ n k -1 ,n based on condition of τ n

9: end for

10: end for

11: Return x 0 , 1: N

each T is scale-invariant, we can define a continuous function α ( t ) as the continuous α schedule of the discrete counterpart 1 . More specifically, we obtain

For the reverse-time process, we define the transition time set T := { τ n } N n =1 consistent with Theorem 3.6 and sample it from P ( τ n = t ) = -α ′ ( t ) (we always use decreasing α ( t ) ). With T defined, the updates to x t,n only occur at { τ n } . Consequently, we arrange τ n to obtain an ordered sequence τ n k , where τ n 1 < τ n 2 < . . . < τ n N . When omitting the infinitely many time steps between τ n k and τ n k -1 , the resulting reverse process is then given by:

̸

for all n ∈ [ N ] . The detailed algorithm named DNDM-C is shown in Algorithm 2.

Remark 3.7 . Autoregressive Diffusion Model (ARDM) (Hoogeboom et al., 2021a) is a discrete diffusion model built upon the autoregressive nature of data. ARDM is shown to be equivalent to a continuous-time absorbing diffusion model and thus provides a unique perspective for discrete diffusion. For continuous-time ( T = ∞ ) reverse sampling, both ARDM and our method achieve N NFEs. Unlike ARDM which is limited to absorbing-state transitions, our method provides a unified framework including both absorbing and multinomial diffusions, applicable to both finite time and continuous time diffusions. For infinite timesteps, Hoogeboom et al. (2021a) also proposed an advanced parallelizing technique that can reduce NFE according to the log-likelihood, which we have not considered in DNDM-C.

4 Experiments

In this section, we evaluate DNDM and demonstrate its superior performance on two types of tasks: conditional sequence-to-sequence text generation (i.e., machine translation) and unconditional text generation. For the fairness of comparison, all the experiments are conducted using a single NVIDIA RTX A6000 GPU with 48 GB memory. Additional experiment details are provided in Appendix F.

4.1 Conditional Text Generation

We evaluate DNDM's effectiveness on conditional text generation through machine translation tasks. Following Zheng et al. (2023), we use Byte Pair Encoding (BPE) (Sennrich et al., 2016) to create a shared vocabulary of words and subwords from both source and target languages. We implement our experiments using FairSeq (Ott et al., 2019), which employs an encoder-decoder architecture. The model uses bi-directional self-attention blocks without causal masking, allowing tokens to attend to both past and future positions during training and inference. The encoder processes the source text, while the decoder generates the target translation.

Datasets. We use the following three datasets to compare with the baselines for machine translation tasks: (1) IWSLT14 DE -EN (Cettolo et al., 2014), a dataset with German as the source language and English as the target language. It consists of 174272 examples (sentence pairs), and each of the validation set and the testing set accounts for 7283 and 6750 of the dataset; (2) WMT14 EN -DE (Bojar et al., 2014), which is an English-to-German translation dataset consisting of 3967182 examples. Each of the validation set and the testing set accounts for 3000 and 3003 of the dataset; and (3) WMT16 EN -RO (Bojar et al., 2016), which is an English-to-Russian translation dataset consisting of 612317 examples. Each of the validation sets and the testing set accounts for 1999 and 1999 of the dataset. The train-validation-test split is fixed across all experiments for all machine translation datasets to ensure fair comparison.

Performance Metrics. We use the BLEU score (Papineni et al., 2002) to evaluate the machine translation quality, where the BLEU score is calculated based on the similarity between the actual target sequence and the predicted target sequence. The sampling speed is measured by wall-clock time (in second).

Baselines. The main baselines we are comparing with are RDM and RDMk from Zheng et al. (2023). Here, we use RDMk and RDM to denote the sampling method proposed in their paper with and without the usage of topk selection for the token generation technique (see Appendix E for more details), respectively. RDM and RDMk are applied to two previously proposed state-ofthe-art discrete diffusion models: Multinomial Diffusion (Hoogeboom et al., 2021b) and Absorbing Diffusion (Austin et al., 2021).

Results and Discussion. Tables 2 and 3 present the performance evaluations of our algorithms in machine translation tasks. Table 2 presents results for multinomial diffusion, while Table 3 displays results for absorbing diffusion. Our reported time and BLEU scores are averaged over 5 repeated experiments, except for the baseline RDM experiment 2 .

From Tables 2 and 3, we observe that methods based on DNDM significantly accelerate the sampling process compared to baseline diffusion models. This acceleration allows for greater flexibility in increasing the number of steps (up to infinity) without imposing a significant computational burden.

In particular, more sampling steps lead to better generation quality (BLEU) at the expense of longer sampling time, as indicated in each column of Tables 2 and 3. For RDM-based methods, generation time increases linearly with the number of sampling steps. On the contrary, for our DNDM-based method, generation time only increases marginally (See Figure 4 in Section G). As a result of the difference in the growing speed of sampling time with respect to sampling steps, the more sampling steps, the more speedup DNDM can obtain.

Continuous-time results, as the ultimate limit of increasing sampling steps, are presented in the last row of each dataset with the tag ∞ . Given that the results with 1000 steps consistently outperform those with 50 steps, we compare ∞ with 1000 steps in Table 2 and 3. For IWSLT14 and WMT16 , where the generation BLEU score is relatively high, we observe a consistent performance improvement of up to 0 . 3 in BLEU score when utilizing the DNDM-C algorithm, with the exception of a single case in the absorbing diffusion setting for WMT16 without the use of topk selection. The performance gain of the continuous-time method on WMT14 is less significant, with both drops and gains. However, WMT14 itself has not reached a high level of performance, with a BLEU score significantly lower than other datasets. In general, training WMT14 poses challenges across all diffusion models, including multinomial diffusion (Hoogeboom et al., 2021b), absorbing diffusion (Austin et al., 2021), and RDM diffusion (Zheng et al., 2023), etc. We defer a more detailed discussion on WMT14 to Appendix F.1. Finally, when compared with the results obtained with 50 steps, the performance of DNDM-C demonstrates improvement consistently. Furthermore, we note that regardless of the dataset or the method (i.e., RDM or DNDM) employed, topk token generation consistently outperforms vanilla methods. This approach enhances the BLEU score by approximately 1 -2 points without introducing significant increases in sampling time.

Scaling Law in Sampling Speed. For illustrative purposes, we use the example of IWSLT14 to visualize how the sample quality scales regarding sampling speed for different methods. In Figure 1, we observe the trend of the BLEU score in relation to computational time. Each line in the legend represents a different sampling algorithm, and a steeper slope indicates a larger marginal gain when sampling for longer periods. Figure 1 demonstrates that our algorithm displays nearly linear growth in BLEU score over the log of time, which is remarkable in contrast with the flat curve of the baseline. Particularly, for multinomial diffusion, the BLEU score increases by 1 in less than 60 seconds of additional sampling time. For absorbing diffusion, DNDM outperforms RDM before RDM samples 50 steps. In Tables 7 and 8 in Appendix D, we further use the average number of function evaluations (NFE) to measure the improved speed within the specified number of sampling steps. Additionally, in Figure 2, we visualize how the BLEU score and the generated text change throughout the sampling process.

10

100

1000

10000

Computational Time (s)

31.0

31.5

32.0

32.5

33.0

33.5

BLEU Score

25

50

100

25

50

100

Inf

25

50

100

25

50

100

Inf

RDM-Multi

DNDM-Multi

RDM-k-Multi

DNDM-k-Multi

(a) Multinomial Diffusion

10

100

1000

10000

Computational Time (s)

31.5

32.0

32.5

33.0

33.5

34.0

34.5

BLEU Score

25

50

100

25

50

100

Inf

25

50

100

25

50

100

Inf

Absorb

DNDM-Absorb

RDM-Absorb

DNDM-T-Absorb

(b) Absorbing Diffusion

4.2 Unconditional Text Generation

For unconditional text generation, we evaluate our approach on language modeling tasks, where the model learns to generate text that matches the statistical patterns of the training data. Unlike conditional generation, this task involves directly learning q ( x 0 | x t ) without conditioning on any input text. We conduct experiments on the text8 and enwik8 datasets using a decoder-only architecture similar to GPT models. Since unconditional generation does not require encoding input sequences, we employ a 12-layer Transformer decoder without an encoder component.

Datasets. The natural language generation task is evaluated on two language datasets following Hoogeboom et al. (2021b): text8 and enwik8 . Both datasets are from Wikipedia, but their contents are highly distinct. In text8 , the plain text consists of English words (all the letters are in lower case) and spaces, and it is tokenized into 26 characters and one blank space, resulting in 27 categories. In contrast to the cleanness of text8 , enwik8 preserves the original XML dump contents, and there exist various special symbols in its raw text, so its text is tokenized into 1 Byte, resulting in 256 categories. We utilize text8 dataset with sequence length 256 and enwik8 dataset with sequence length 320. The train/val/test splits are 9 e 7 / 5 e 6 / 5 e 5 for both text8 and enwik8 .

Performance Metrics. Our evaluation of text generation quality relies on the perplexity score. When generating text8 data, we calculate perplexity scores using the GPT2 model, while for enwik8 data generation, we employ the GPT2-large model. The sampling speed is measured in seconds.

100

90

80

70

60

50

40

30

20

10

0

Time in Reverse Process

0

5

10

15

20

25

30

BLEU Score

DNDM-k-Multi

(a) The BLEU Score in the Generation Process

t = 100 [noise] [noise] [noise] [noise] · · ·

t = 75 [noise] · · · [noise] and we [noise] · · · [noise] govern [noise] [noise] year [noise]

t = 67 we [noise] [noise] fello [noise] [noise] [noise] and we let them [noise] [noise] city govern [noise] every year.

t = 39 we choose some fellows every year and we let them work with city governance every year.

t = 0 we choose some fellows every year and we let them work with city governance every year.

(b) Text in the Generation Process

Baselines. We compare our proposed DNDM on unconditional text generation task with the vanilla Multinomial Diffusion (Hoogeboom et al., 2021b).

Results and Discussion. Table 4 displays the performance of our algorithms in text generation tasks. We run the multinomial diffusion model on the text8 dataset for 1000 diffusion steps and on the enwik8 dataset for 4000 diffusion steps. Our DNDM-based algorithms outperform the vanilla sampling algorithm used in Hoogeboom et al. (2021b) in terms of both sampling time and perplexity score. Specifically, for the text8 dataset, DNDMbased algorithms are 5 times faster than the vanilla algorithm. For the enwik8 dataset, DNDM-based algorithms are 14 times faster than the vanilla algorithm.

5 Conclusion and Future Work

This paper presents a novel discrete non-Markov diffusion model (DNDM) accompanied by an accelerated sampling algorithm designed to boost sampling speed in a discrete-state space. Our discrete diffusion model incorporates "transition time set" latent variables, establishing itself as an efficacious diffusion and data generation method. Thanks to our acceleration technique, we significantly decrease the number of neural network function evaluations without sacrificing sample quality. We also introduce an infinite-step sampling algorithm, DNDM-C, which provides new insights into bridging the gap between discrete and continuous-time processes for discrete diffusion models. While this study focuses on text generation using non-autoregressive models, a promising direction for future exploration is applying our method to other tasks, such as audio and image generation.

References

ALAIN, G., BENGIO, Y., YAO, L., YOSINSKI, J., THIBODEAU-LAUFER, E., ZHANG, S. and VINCENT, P. (2016). Gsns: generative stochastic networks. Information and Inference: A Journal of the IMA 5 210-249.

ALIAS PARTH GOYAL, A. G., KE, N. R., GANGULI, S. and BENGIO, Y. (2017). Variational walkback: Learning a transition operator as a stochastic recurrent net. Advances in Neural Information Processing Systems 30 .

AUSTIN, J., JOHNSON, D. D., HO, J., TARLOW, D. and VAN DEN BERG, R. (2021). Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems 34 17981-17993.

BAO, F., LI, C., ZHU, J. and ZHANG, B. (2022). Analytic-dpm: an analytic estimate of the optimal reverse variance in diffusion probabilistic models. arXiv preprint arXiv:2201.06503 .

BENGIO, Y., LAUFER, E., ALAIN, G. and YOSINSKI, J. (2014). Deep generative stochastic networks trainable by backprop. In International Conference on Machine Learning . PMLR.

BOJAR, O., BUCK, C., FEDERMANN, C., HADDOW, B., KOEHN, P., LEVELING, J., MONZ, C., PECINA, P., POST, M., SAINT-AMAND, H., SORICUT, R., SPECIA, L. and TAMCHYNA, A. (2014). Findings of the 2014 workshop on statistical machine translation. In Proceedings of the Ninth Workshop on Statistical Machine Translation . Association for Computational Linguistics, Baltimore, Maryland, USA.

BOJAR, O., CHATTERJEE, R., FEDERMANN, C., GRAHAM, Y., HADDOW, B., HUCK, M., JIMENO YEPES, A., KOEHN, P., LOGACHEVA, V., MONZ, C., NEGRI, M., NÉVÉOL, A., NEVES, M., POPEL, M., POST, M., RUBINO, R., SCARTON, C., SPECIA, L., TURCHI, M., VERSPOOR, K. and ZAMPIERI, M. (2016). Findings of the 2016 conference on machine translation. In Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers . Association for Computational Linguistics, Berlin, Germany.

BORDES, F., HONARI, S. and VINCENT, P. (2017). Learning to generate samples from noise through infusion training. arXiv preprint arXiv:1703.06975 .

CAMPBELL, A., BENTON, J., DE BORTOLI, V., RAINFORTH, T., DELIGIANNIDIS, G. and DOUCET, A. (2022). A continuous time framework for discrete denoising models. Advances in Neural Information Processing Systems 35 28266-28279.

CERITLI, T., GHOSHEH, G. O., CHAUHAN, V. K., ZHU, T., CREAGH, A. P. and CLIFTON, D. A. (2023). Synthesizing mixed-type electronic health records using diffusion models. arXiv preprint arXiv:2302.14679 .

CETTOLO, M., NIEHUES, J., STÜKER, S., BENTIVOGLI, L. and FEDERICO, M. (2014). Report on the 11th IWSLT evaluation campaign. In Proceedings of the 11th International Workshop on Spoken Language Translation: Evaluation Campaign . Lake Tahoe, California.

CHANG, H., ZHANG, H., JIANG, L., LIU, C. and FREEMAN, W. T. (2022). Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition .

CHEN, N., ZHANG, Y., ZEN, H., WEISS, R. J., NOROUZI, M. and CHAN, W. (2020). Wavegrad: Estimating gradients for waveform generation. arXiv preprint arXiv:2009.00713 .

CHUNG, H., SIM, B. and YE, J. C. (2022). Come-closer-diffuse-faster: Accelerating conditional diffusion models for inverse problems through stochastic contraction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition .

DOCKHORN, T., VAHDAT, A. and KREIS, K. (2021). Score-based generative modeling with critically-damped langevin diffusion. arXiv preprint arXiv:2112.07068 .

DOCKHORN, T., VAHDAT, A. and KREIS, K. (2022). Genie: Higher-order denoising diffusion solvers. Advances in Neural Information Processing Systems 35 30150-30166.

GHAZVININEJAD, M., LEVY, O., LIU, Y. and ZETTLEMOYER, L. (2019). Mask-predict: Parallel decoding of conditional masked language models. arXiv preprint arXiv:1904.09324 .

GRUVER, N., STANTON, S., FREY, N., RUDNER, T. G., HOTZEL, I., LAFRANCE-VANASSE, J., RAJPAL, A., CHO, K. and WILSON, A. G. (2024). Protein design with guided discrete diffusion. Advances in Neural Information Processing Systems 36 .

HE, Z., SUN, T., WANG, K., HUANG, X. and QIU, X. (2022). Diffusionbert: Improving generative masked language models with diffusion models. arXiv preprint arXiv:2211.15029 .

HO, J., CHAN, W., SAHARIA, C., WHANG, J., GAO, R., GRITSENKO, A., KINGMA, D. P., POOLE, B., NOROUZI, M., FLEET, D. J. ET AL. (2022). Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303 .

HO, J., JAIN, A. and ABBEEL, P. (2020). Denoising diffusion probabilistic models. Advances in neural information processing systems 33 6840-6851.

HOOGEBOOM, E., GRITSENKO, A. A., BASTINGS, J., POOLE, B., BERG, R. V. D. and SALIMANS, T. (2021a). Autoregressive diffusion models. arXiv preprint arXiv:2110.02037 .

HOOGEBOOM, E., NIELSEN, D., JAINI, P., FORRÉ, P. and WELLING, M. (2021b). Argmax flows and multinomial diffusion: Learning categorical distributions. Advances in Neural Information Processing Systems 34 12454-12465.

JOLICOEUR-MARTINEAU, A., LI, K., PICHÉ-TAILLEFER, R., KACHMAN, T. and MITLIAGKAS, I. (2021). Gotta go fast when generating data with score-based models. arXiv preprint arXiv:2105.14080 .

KARRAS, T., AITTALA, M., AILA, T. and LAINE, S. (2022). Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems 35 26565-26577.

KONG, Z. and PING, W. (2021). On fast sampling of diffusion probabilistic models. arXiv preprint arXiv:2106.00132 .

KONG, Z., PING, W., HUANG, J., ZHAO, K. and CATANZARO, B. (2020). Diffwave: A versatile diffusion model for audio synthesis. arXiv preprint arXiv:2009.09761 .

LIU, L., REN, Y., LIN, Z. and ZHAO, Z. (2022). Pseudo numerical methods for diffusion models on manifolds. arXiv preprint arXiv:2202.09778 .

LU, C., ZHOU, Y., BAO, F., CHEN, J., LI, C. and ZHU, J. (2022). Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems 35 5775-5787.

LYU, S. (2012). Interpretation and generalization of score matching. arXiv preprint arXiv:1205.2629 .

MOVELLAN, J. R. (2008). Contrastive divergence in gaussian diffusions. Neural Computation 20 2238-2252.

NACHMANI, E., ROMAN, R. S. and WOLF, L. (2021). Non gaussian denoising diffusion models. arXiv preprint arXiv:2106.07582 .

NICHOL, A. Q. and DHARIWAL, P. (2021). Improved denoising diffusion probabilistic models. In International Conference on Machine Learning . PMLR.

OTT, M., EDUNOV, S., BAEVSKI, A., FAN, A., GROSS, S., NG, N., GRANGIER, D. and AULI, M. (2019). fairseq: A fast, extensible toolkit for sequence modeling. arXiv preprint arXiv:1904.01038 .

PAPINENI, K., ROUKOS, S., WARD, T. and ZHU, W.-J. (2002). Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics .

REID, M., HELLENDOORN, V. J. and NEUBIG, G. (2022). Diffuser: Discrete diffusion via edit-based reconstruction. arXiv preprint arXiv:2210.16886 .

SALIMANS, T. and HO, J. (2022). Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.00512 .

SAN-ROMAN, R., NACHMANI, E. and WOLF, L. (2021). Noise estimation for generative diffusion models. arXiv preprint arXiv:2104.02600 .

SAVINOV, N., CHUNG, J., BINKOWSKI, M., ELSEN, E. and OORD, A. V. D. (2021). Step-unrolled denoising autoencoders for text generation. arXiv preprint arXiv:2112.06749 .

SENNRICH, R., HADDOW, B. and BIRCH, A. (2016). Neural machine translation of rare words with subword units.

SOHL-DICKSTEIN, J., BATTAGLINO, P. and DEWEESE, M. R. (2009). Minimum probability flow learning. arXiv preprint arXiv:0906.4779 .

SOHL-DICKSTEIN, J., WEISS, E., MAHESWARANATHAN, N. and GANGULI, S. (2015). Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning . PMLR.

SONG, J., MENG, C. and ERMON, S. (2020a). Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 .

SONG, Y., DHARIWAL, P., CHEN, M. and SUTSKEVER, I. (2023). Consistency models. arXiv preprint arXiv:2303.01469 .

SONG, Y. and ERMON, S. (2019). Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems 32 .

SONG, Y. and ERMON, S. (2020). Improved techniques for training score-based generative models. Advances in neural information processing systems 33 12438-12448.

SONG, Y., SOHL-DICKSTEIN, J., KINGMA, D. P., KUMAR, A., ERMON, S. and POOLE, B. (2020b). Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456 .

SUN, H., YU, L., DAI, B., SCHUURMANS, D. and DAI, H. (2022). Score-based continuous-time discrete diffusion models. arXiv preprint arXiv:2211.16750 .

VAHDAT, A., KREIS, K. and KAUTZ, J. (2021). Score-based generative modeling in latent space. Advances in Neural Information Processing Systems 34 11287-11302.

WATSON, D., HO, J., NOROUZI, M. and CHAN, W. (2021). Learning to efficiently sample from diffusion probabilistic models. arXiv preprint arXiv:2106.03802 .

YE, J., ZHENG, Z., BAO, Y., QIAN, L. and GU, Q. (2023). Diffusion language models can perform many tasks with scaling and instruction-finetuning. arXiv preprint arXiv:2308.12219 .

ZHANG, Q. and CHEN, Y. (2022). Fast sampling of diffusion models with exponential integrator. arXiv preprint arXiv:2204.13902 .

ZHENG, L., YUAN, J., YU, L. and KONG, L. (2023). A reparameterized discrete diffusion model for text generation. arXiv preprint arXiv:2302.05737 .

Broader Impact

This paper presents work that aims to advance the field of diffusion models. We believe this work may enable future applications of synthetic data generation, which may lead to positive impacts. Our experiments demonstrate that the proposed method achieves state-of-the-art performance in the acceleration of the generative model. However, proper controls may be needed whenever applying our method to tasks that involve sensitive data data. There may be other potential societal consequences of our work, none of which we feel must be specifically highlighted here.

Limitations

The scope of the empirical claims is limited to the text domain with non-auto regressive setting. The applicability and performance of DNDM for other tasks like audio and image generation, as well as with other architectures like auto-regressive GPT models, are not explored and left as future work.

While DNDM-C, the infinite-step sampling algorithm, offers new insights into bridging the gap between discrete and continuous-time processes for discrete diffusion models, the sample quality is not guaranteed to be superior to the accelerated algorithm with 1000 steps. Some intuitions here: the assumption that the neural network can be optimally trained is an ideal case and is often not realized in practice. There is an inherent estimation error associated with the training process. As the number of steps increases, these estimation errors can accumulate, potentially leading to a degradation in performance. This cumulative estimation error might explain why using an infinite number of steps does not necessarily yield better results than a finite number of steps like 1000 in the conditional generation experiments. How to further improve sample quality of infinite steps is interesting but beyond the scope of this paper.

This paper focuses on the comparison with discrete Markov diffusion models since it aims to propose an accelerated algorithm for discrete diffusion with DNDM. Other text generation models, such as continuous diffusion models or auto-regressive models, are not considered in this paper.

This paper focuses on acceleration while maintaining good sample quality. The hyper parameter regions with poor sample qualities are not explored in this paper.

By highlighting these limitations, this paper aims to clearly scope its contributions and spark future work on addressing these important challenges with discrete diffusion models for generative modeling.

A Related Work

Continous Diffusion Models. Generative modeling via continuous-time stochastic process has been investigated thoroughly in a series of work (Movellan, 2008; Lyu, 2012; Sohl-Dickstein et al., 2009; Bengio et al., 2014; Alain et al., 2016; ALIAS PARTH GOYAL et al., 2017; Bordes et al., 2017). The two lines of probabilistic modeling, denoising diffusion probabilistic model (Sohl-Dickstein et al., 2015; Ho et al., 2020) and score matching with Langevin dynamics (Song and Ermon, 2019) are unified by Song et al. (2020b) through introducing the SDE framework for SGM. Based on it, subsequent works (Dockhorn et al., 2021; Nachmani et al., 2021; Vahdat et al., 2021) introduced a more complex diffusion process to improve the generation speed and quality. On the other hand, the score-based sampling process is time-consuming and has attracted much attention for improvements in speed (San-Roman et al., 2021; Watson et al., 2021; Kong and Ping, 2021; Karras et al., 2022; Song et al., 2023). 'Gotta go fast' (GGF), an SDE solver with adaptive step size tailored to SGM, is proposed in Jolicoeur-Martineau et al. (2021). Song et al. (2020a) introduced a non-Markov diffusion process that corresponds to a deterministic sampling process, enabling the generation of high-quality samples more rapidly. Dockhorn et al. (2022); Liu et al. (2022) proposed a high-order SDE/ODE solver to achieve lower discretization error. Lu et al. (2022); Zhang and Chen (2022) leveraged the semi-linear structure of reverse ODE to reduce the discretization error and achieve state-of-the-art sampling speed.

Discrete Diffusion Models. Research on discrete diffusion models was initiated by Sohl-Dickstein et al. (2015), who investigated diffusion processes over binary random variables. The methodology was expanded upon by Ho et al. (2020), integrating categorical random variables through transition matrices with uniform probabilities. Though Song et al. (2020a) suggested a similar extension in

their supplementary content, they abstained from experimenting with this model type. Later on, Austin et al. (2021) unveiled a more intricate framework for diffusion concerning categorical random variables, enhancing the discrete diffusion models by merging them with Masked language models (MLMs). Contemporary research has furthered this domain by introducing features like editingbased operations (Jolicoeur-Martineau et al., 2021; Reid et al., 2022), auto-regressive diffusion models (Hoogeboom et al., 2021a; Ye et al., 2023), the evolution of a continuous-time structure (Campbell et al., 2022), and the exploration of neural network analogs for learning (Sun et al., 2022). Additionally, Zheng et al. (2023) introduced a re-parameterized loss and an associated sampling technique, attaining commendable outcomes in fewer iterations. Our contributions run parallel to these aforementioned studies.

B Additional details of Discrete Diffusion

In our paper, we treat all the x , q noise as a row vector and treat 1 as a column vector with all elements equal 1 .

B.1 Comparison between D3PM and DNDM

In Section 3.1, we introduced two different diffusion processes, the Markov process in (1) and the non-Markov process in (6). In this section, we explain why they are different but result in the same joint distribution of ( x 0 , x t ) for every time step t . Since q ( x 0 ) keeps the same, we only need to prove that the conditional distribution q ( x t | x 0 ) is the same for the two processes.

Markov Process. 1 is a Markov process since w n is independent with x t -1 , . . . , x 0 , so x t is independent of all the past states given the present state. This can also be inferred from the following distribution, which does not depend on x 0 , . . . , x t -2 ,

Denote Q t := β t I +(1 -β t ) 1 q noise , then we have that

where the last equality holds due to the fact that x t -1 is a one hot vector and thus x t -1 1 = 1 . Therefore, we can rewrite (13) as q ( x t | x t -1 ) = Cat ( x t ; p = x t -1 Q t ) . Then, it is a Markov process with transition kernel Q t . So q ( x t | x 0 ) = Cat ( x t ; p = x 0 Q 0 . . . Q t ) (Austin et al., 2021). We can then have that

where the last equality holds since identity matrix I multiplying any vector equals the vector itself and 1 q noise 1 q noise = 1 ( q noise 1 ) q noise = 1 q noise . Therefore, we have that

where the last equality holds due to the definition α t = Π t s =0 β s . This gives rise to why the Markov process (1) results in conditional distribution q ( x t | x 0 ) = Cat ( x t ; p = α t x 0 +(1 -α t ) q noise ) .

Non-Markov Process. Recall that our DNDM is defined by

where w is fixed for any time t . Therefore, w is no longer independent with x 0 , . . . , x t -1 . Therefore, we can't define the transition kernel and compute q ( x t | x 0 ) by using the property of Markov. Therefore, we need to advance the technique to calculate the conditional distribution.

Proof of Theorem 3.1. By (6), we can derive the following explicit expression for a recursive sequence,

where second equality is by cancellation of terms, the last inequality holds by defining a t = b 1 . . . b t . Since a t either equals to 1 or 0 . Besides, a t equals 1 if and only if b 1 = b 2 = . . . = b t = 1 , so we have that a t follows Bernoulli distribution Bernoulli( β 1 . . . β t ) = Bernoulli( α t ) where α t = Π t i =1 β s . Therefore, we can conclude that q ( x t | x 0 ) = Cat ( x t ; p = α t x 0 +(1 -α t ) q noise ) , which completes the proof.

Comparison between D3PM-Absorb and DNDM. Recall the forward processes of D3PM and DNDMas follows:

̸

For absorbing diffusion where w = [Mask] , DNDM's forward process becomes equivalent to D3PM since w t = w = [Mask] in this special case. However, for multinomial diffusion or other diffusion processes where w t = w , these two processes exhibit different behaviors. In addition, even for absorbing diffusion, our proposed reverse sampling algorithm for DNDM is still different from that for D3PM.

To elucidate the key differences between the sampling algorithm in DNDM and that in D3PM for absorbing diffusion, let's directly compare the algorithms:

̸

For the D3PM-Absorb algorithm: We begin with an all [Mask] sequence. At each time step t , we sample x 0 ∼ p θ ( x 0 | x t ) . If x t = [Mask] , x t -1 transitions to [Mask] with probability (1 -α t -1 ) / (1 -α t ) and to x 0 with probability ( α t -1 -α t ) / (1 -α t ) . If x t = [Mask] , it remains unchanged.

For the DNDM-Absorb algorithm: We also start with an all [Mask] sequence, but crucially, we first determine the transition time set. During sampling, if x t = [Mask] , the transition probabilities for x t -1 are identical to D3PM. However, we only sample x 0 ∼ p θ ( x 0 | x t ) when at least one token needs to change, as determined by our pre-computed transition set. This selective sampling is the key to our algorithm's efficiency.

Therefore, you can see that DNDM will skip many steps during the sampling process to avoid function evaluation and save computational cost. Even though the forward process of DNDM is the same as that of D3PM for absorbing diffusion, our DNDM approach introduces an algorithm design in the sampling process by pre-computing the transition time set and selectively applying function evaluations. This distinguishes DNDM from D3PM algorithm, offering a more computationally efficient approach to inference in discrete diffusion.

Comparison between DDIM and DNDM for Multinomial Diffusion. While there are similarities between DNDM and DDIM (Appendix A), they are fundamentally different models, and DNDM is not a special case of DDIM. DNDM introduces a novel framework specifically designed for discrete spaces, while DDIM was originally developed for continuous diffusion models. The key differences for multinomial diffusion are as follows.

DDIM: Following Song et al. (2020a) (eq. 19 in Appendix A), q ( x t -1 | x t , x 0 ) = Cat ( σ t x t + ( α t -1 -σ t α t ) x 0 +((1 -α t -1 ) -(1 -α t ) σ t ) 1 K ) . Even with σ t = 1 -α t -1 1 -α t , the process remains stochastic: q ( x t -1 | x t , x 0 ) = Cat ( σ t x t +(1 -σ t ) x 0 ) . This means at every step, there's a probability of choosing x 0 , regardless of whether it has transitioned to x 0 or not. Unlike Absorbing discrete diffusion, no [Mask] exists in multinomial diffusion. Therefore, DDIM cannot distinguish whether x t already equals x 0 or not. In particular, although the sampling process becomes less stochastic in the DDIM setting, it will still be predicted x 0 with high probability 1 -σ t = α t -1 -α t 1 -α t .

DNDM: Achieves full de-randomization using transition time τ , where:

̸

This crucial difference allows DNDM to achieve full de-randomization once τ is sampled, leading to a deterministic evolution that DDIM cannot achieve.

While DNDM and DDIM are both non-Markov models for multinomial diffusion, their fundamental approaches to and achievements in de-randomization differ significantly in discrete spaces.

B.2 Training Objective

Hoogeboom et al. (2021b) utilized L t derived from the negative variational bound. In detail,

where ̂ x 0 ∼ p θ ( ·| x t ) , θ post = ( β t x t + (1 -β t ) /K 1 ⊤ ) ⊙ ( α t -1 x 0 + (1 -α t -1 ) /K 1 ⊤ ) and θ post = ( β t x t +(1 -β t ) /K 1 ⊤ ) ⊙ ( α t -1 ̂ x 0 +(1 -α t -1 ) /K 1 ⊤ ) . This loss evolves KL divergence between two categorical distributions.

Building on this foundation, Austin et al. (2021) introduced an auxiliary denoising objective to strengthen the data predictions x 0 at each time step. In detail, the auxiliary objective is as follows,

where the auxiliary loss term is minimized exactly when p θ ( ·| x t ) has all its mass on the data point x 0 .

Furthering the advancements, Zheng et al. (2023) put forth a reparametrized loss L t that incorporates a re-weighted parameter λ t . The detailed loss is

This loss can be related to the standard multi-class cross-entropy loss function, which is also simple and powerful. That's why we consider Zheng et al. (2023) as the baseline model.

In Section 3.3, we consider the continuous-time forward and backward process. Based on that, we were motivated to analyze the infinite limit of the average loss lim t →∞ 1 T ∑ T t =1 L t . We find that the new loss can provide a better checkpoint than the loss averaged on the finite step on some tasks.

B.3 Calculation of the Evidence Lower Bound

B.3.1 Finite Time DNDM

In this section, we derive the evidence lower bound (ELBO) for our model. The derivatives are inspired by the reasoning in DDIM (Song et al., 2020a). Specifically, We denote the generative process as p θ ( x 0: T | τ ) = p ( T ) θ ( x T | τ ) ∏ T t =1 p ( t ) θ ( x t -1 | x t , τ ) . Here, p ( T ) θ is the pure noise and p ( t ) θ ( x t -1 | x t , τ ) = q ( x t -1 | x t , ̂ x 0 , τ ) , where ̂ x 0 is given by a neural network p θ , i.e., ̂ x 0 = p θ ( x t , t ) . Notice that by Jensen's inequality,

The evidence lower bound inequality gives

Plugging (17) into (16) gives the following ELBO,

We factorize the p θ and q by

Here q admits such a decomposition due to our definition of the diffusion process in (6), which introduce the following reverse process:

̸

Therefore, x 1: T is Markovian when conditioned on x 0 and τ . Based on the factorization, we have

By a slight abuse of notations we use q ( x t -1 | x t , x 0 ) , p ( t ) θ ( x 0 | x 1 ) to indicate the distribution of the diffusion process defined in Zheng et al. (2023), that is, the standard Markov discrete diffusion process. In particular, we have

̸

Thus, we can obtain that

Here L t matches the loss terms in Zheng et al. (2023). In the practical training process, Zheng et al. (2023) samples t from Unif { 1 , · · · , T } in each iteration and optimizes λ t · L t , where λ t 's are weights. Thus, when we sample τ and optimize L τ , our ELBO indeed leads to the same training objective as Zheng et al. (2023) up to reweighting. Since Zheng et al. (2023) is a parametrization of existing works (Austin et al., 2021; Hoogeboom et al., 2021b), our training objective indeed aligns with previous discrete diffusion models.

B.3.2 Continous Time DNDM

In Section B.3, we derived an ELBO for DNDM and its accelerated algorithm defined in Section 3.1 and 3.2. While for finite sampling steps, we can decompose the diffusion process via the sampling steps 1 , . . . , T in (17), it becomes intractable for continuous Time DNDM (Infinite steps T →∞ ). Therefore, we can formulate the ELBO of continuous time DNDM by decomposing the transition times. The idea of decomposition of transition times follows Hoogeboom et al. (2021a), but their

̸

proof is only applicable to absorbing discrete diffusion, while ours can deal with discrete diffusion with various noise q noise including multinomial diffusion.

In Section B.3, we only consider the case of a single token x ∈ R K for simplicity as we decompose with the sampling steps T . In this section, we decompose over the transition time τ . Therefore, we need to consider a sentence with multiple tokens x t, 1: N = [ x t, 1 , . . . , x t,N ] where x t,n is the n -th token and N is the sequence length. Recall that we defined the transition time set T = { τ n } N n =1 in Section 3.2. We arrange τ n to obtain an ordered sequence τ n k , where 0 = τ n 0 < τ n 1 < τ n 2 < . . . < τ n N = T . Then conditioning on the transition time set T = { τ 1 , . . . , τ N } , we have that

where we omit the time superscript of p for simplicity. Then, the evidence lower bound inequality gives

By Jensen's inequality, we have

Plugging (18) into (19) gives the following ELBO,

We factorize the p θ and q by

Therefore, we have

Remark B.1 . (20) represents the ELBO utilized by the DNDM-C architecture. As our transition times τ n are independently and identically drawn from the distribution D τ , we are unable to further decompose (20) into a loss function related to the position information 1 : N , as was accomplished by Hoogeboom et al. (2021a).

C Choice of the Transition Time

Transition time τ in Definition 3.2 plays an important role in DNDM. In this section, we provide a deeper discussion of the transition time. We first give a proof of the Theorem 3.6.

Proof of Theorem 3.6. By the definition of τ , we know that τ n = t is equivalent to b 0 ,n = 1 , . . . , b t -1 ,n = 1 and b t,n = 0 . Since { b t,n } T t =0 is independent for different n by definition, each τ n is also independent. Therefore, we drop the subscript n for simplicity. On the other hand if b 0 = 1 , . . . , b t -1 = 1 and b t = 0 we can also conclude that τ = t . Therefore, we have that

where the second equality is due to b s , s = 1 , 2 , . . . , t are independent random variable following Bernoulli( β s ) distribution and the last equality is by the definition of α t = Π t s =1 β s .

Notice that α t is a decreasing sequence in the 0 to 1 range. Therefore, P ( τ = t ) ∈ [0 , 1] for any t ∈ { 1 , . . . , T } . Besides ∑ P ( τ = t ) = ∑ T t =1 ( α t -1 -α t ) = α 0 -α T = 1 . Therefore, the derived distribution is valid as long as the α t is decreasing from 1 to 0 .

0

10

20

30

40

50

Value

0.000

0.005

0.010

0.015

0.020

0.025

Density

Transition Time

(a)

α

t

= 1

-

t/T

0

10

20

30

40

50

Value

0.000

0.005

0.010

0.015

0.020

0.025

0.030

0.035

Density

Transition Time

(b)

α

t

= cos(

π

∗

t/

2

T

)

0

10

20

30

40

50

Value

0.000

0.005

0.010

0.015

0.020

0.025

0.030

0.035

Density

Transition Time

(c)

α

t

= cos

2

(

π

∗

t/

2

T

)

0

10

20

30

40

50

value (mapped from [0,1] to [0,50])

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

Density

=3,  =3

=1,  =1

=15,  =7

=2,  =1

(d) Beta Distribution with Different Parameter

From Theorem 3.6, we discern that the nature of the diffusion model scheduler, α t , clarifies the distribution of τ .

Linear α schedule. This is a schedule studied in Austin et al. (2021), where α t = 1 -t/T . This will result in P ( τ n = t ) = 1 /T for every t in the range 1 to T . As a result, transition time distributes uniformly across each moment in the set { 1 , . . . , T } . This can be verified in a) of Figure 3.

Cosine α schedule. This is a schedule studied in Hoogeboom et al. (2021b), where α t = cos( π ∗ t/ 2 T ) . For numerical consideration of the noise, a small offset s is added, i.e., α t = f ( t ) /f (0)

where f ( t ) = cos(( s + t/T ) / (1 + s ) ∗ π/ 2) . As shown in b) of Figure 3, the transition time will concentrate more on the large T .

Cosine square α schedule. This is a schedule studied in Zheng et al. (2023), where α t = cos 2 ( π ∗ t/ 2 T ) , which motivated by Nichol and Dhariwal (2021). Again, for numerical consideration of the noise, a small offset s is added, i.e., α t = f ( t ) /f (0) where f ( t ) = cos ( ( s + t/T ) / (1 + s ) ∗ π/ 2) . As shown in c) of Figure 3, the transition time will concentrate more on the middle of the range.

Generally, if we express α t as g ( t/T ) , then we can simplify to P ( τ = t ) = g (( t -1) /T ) -g ( t/T ) , which further refines to (1 /T ) | g ′ ( t/T ) | + o (1 /T ) . This indicates that transitions are more likely where | g ′ | is large. Such a mathematical finding can match our observation in Figure 3.

In practice, we find that the shape of the transition time doesn't need to match the theoretical prediction schedule exactly. As we can see from d) in Figure 3. A reshaped Beta distribution can approximate all the transition time distributions in a fixed range. We first extract a time t ∈ [0 , 1] from a Beta distribution, then adjust these samples to fit by multiplying T and round them to acquire the integer. Our experiment finds that a properly chosen Beta distribution (tuned on the validation set) makes DNDM perform better on the translation tasks. Specifically, the chosen Beta distributions and the searching method are reported in Appendix F. The performance of the four transition time schedules mentioned above, including the reported Beta distributions for comparison, are listed in Table 5, where we find the other three schedules affect the performance, and most of their scores are lower than the scores of Beta distribution, but their scores are at least still close to the reported Beta distributions, especially for DNDM-k-absorb and DNDM-absorb. The efficiencies (measured by NFE) are also similar to one another.

Additionally, the ablation study on a reasonable range of different Beta distributions with 50 and 1000 sampling steps are shown in Tables 10 and 9, where the BLEU scores and NFE values on the test set of one of the three machine translation datasets, WMT16 , are shown for demonstration. The range of Beta distributions covers our chosen Beta schedules based on validation sets and a variety of basic Beta distribution shapes. These results show that the different Beta distributions influence the performance, but most of these choices of parameters still achieve results close to the optimal. Since the Beta distributions of the reported results in Tables 2 and 3 are selected using the validation set, they do not always have the highest scores on the test set, but their scores still at least belong to the top tiers according to these tables.

Another view of the transition time. In Algorithm 1, we only need to call the neural network when t ∈ T , which can significantly speed up the sampling since we reduce the function call. Notice that after we get the x 0 prediction, we only update the x t for those tokens at the transition time. However, (7) implies that x t = x 0 as long as τ > t . Therefore, instead of only updating the x t for those tokens at the transition time, i.e., τ = t , we can also update those tokens with transition time τ > = t . This motivates us to consider a variation presented as Algorithm 3, which keeps almost the same sampling time but will update the tokens several times rather than just once. Since the tokens now get the chance to be corrected over time. The new Algorithm 3 will be more robust than Algorithm 1.

Impact of Transition Order. We further evaluate the impact of transition order. Building upon the results in Table 3, we investigate how the model performance will change if the transition time is influenced by the position of the tokens: from left to right and from right to left. In the left-to-right approach, tokens positioned on the left are transitioned to x 0 earlier, and vice versa for the right-toleft approach. Our experiments show that the left-to-right approach consistently outperforms the right-to-left approach across all datasets and step counts, as demonstrated in Table 6.

This result suggests that the order of token transitions significantly influences the model's performance, with earlier transitions of left-side tokens leading to better generation quality.

D Discussion on the Number of Function Evaluations (NFE).

In this section, we discuss the number of function evaluations (NFE) in DNDM. According to (9), the update of a token x t -1 ,n occurs solely at its designated transition time. Meanwhile, if step t does not coincide with a transition time for any token, we maintain the sentence from the preceding step unchanged: x t, 1: N = x t -1 , 1: N . Therefore, our algorithm removes the need of function evaluation for steps outside the set of transition times. Given this structure, our analytical emphasis is on the transition set T since function evaluations are required only at times t that are members of T . Consequently, the NFE is precisely the cardinality of the transition set, denoted by |T | . In our main paper, we propose a naive upper bound for |T | as min { N,T } , which effectively demonstrates the speed of our method when T > N . Next, we demonstrate that DNDM also reduces the NFE when T < N , by providing a precise estimation of |T | .

Theorem D.1. Suppose transition time follows distribution D τ , and consider a sequence of length N . Then, the cardinality of the transition set T := { τ 1 , . . . , τ N } satisfies:

1 ≤ |T | ≤ min { N,T } ,

E [ |T | ] = [1 -C T,N, D τ ] · T , where C T,N, D τ is a constant in the range (0 , 1) . Furthermore,

where p i = P ( τ = i ) for τ ∼ D τ , and the equality holds if and only if D τ is a uniform distribution.

Proof. The first statement is straightforward. For completeness, the proof is provided. Since there are only N transition times (possibly repeated): τ 1 , . . . , τ N , the distinct transition times must satisfy |T | ≤ N . Additionally, since T ⊆ { 1 , . . . , T } , we also have |T | ≤ T .

To prove the second statement, we decompose T and use the property of expectation. Note that |T | = ∑ T i =1 1 { i ∈ T } . Thus,

Assuming P D τ ( τ = i ) = p i , and that τ n are i.i.d. draws from D τ , we have

Substituting (22) into (21) yields

where C T,N, D τ = ( ∑ T i =1 (1 -p i ) N ) /T . An upper bound for C T,N, D τ is given as

where the inequality holds if and only if p i = 1 /T for all i ∈ [ T ] , i.e., D τ is a uniform distribution.

Remark D.2 . Theorem D.1 suggests that even when T ≤ N , our method still provides a significant improvement. Specifically, for T = N ≥ 4 , we have C T,N, D τ = (1 -1 /N ) N ≥ 0 . 3 . This implies that our model requires at most 0 . 7 T even in the worst case. Moreover, if we consider a special scenario where the number of p i satisfying p i < ϵ is more than M , then we have C T,N, D τ > M (1 -ϵ ) N /T , indicating that with M sufficiently large and ϵ sufficiently small, C T,N, D τ can be pretty close to 1 .

Remark D.3 . In practical applications of our model, we employ a beta distribution for D τ , which typically exhibits a right-heavy tail. Therefore C T,N, D τ tends to be larger than that in the worst-case scenario. In Tables 7 and 8, we list the average NFE for each experiment we run in §4. These results demonstrate a significant reduction in NFE compared to the original counts: for T = 25 , the NFE is only about half of the original count; for T = 50 , it is approximately one-third; and for T = 1000 , it reduces to less than one-twentieth of the original count.

Remark D.4 . By Bernoulli's inequality, (1 -p ) N > 1 -N · p for 1 > p > 0 . Therefore, C T,N, D τ > 1 -N/T , implying that E [ |T | ] < N . As T → ∞ , assuming the transition time does not concentrate at a single point, the probability that two transitions occur simultaneously is zero. Consequently, the generation process will sequentially go through each token. Thus, the expected number of function evaluations (NFE), E [ |T | ] , will be N . In contrast, when T is finite, there is a non-zero probability that multiple transitions happen at the same time. Hence, in this case, the NFE, |T | , is strictly less than N

E Discrete Non-Markov Diffusion Model with Top-k Transition Time (DNDM-K).

Algorithm 3 Sampling From DNDM (Version 2)

Require: Trained prediction function p θ , q noise ,

Algorithm 4 Sampling From DNDM-K

Recent works have demonstrated that the quality of samples can be enhanced by utilizing supplementary information derived from the neural network (Ghazvininejad et al., 2019; Savinov et al., 2021; Chang et al., 2022; He et al., 2022). Very recently, Zheng et al. (2023) applied this idea in their RDM framework and can achieve significant performance improvement. Specifically, after decoding ̂ x 0 , 1: N from transformer p θ ( ·| x t, 1: N ) , the score corresponding to this decoded token from the transformer's last layer, is also recorded and denote as s t,n . Tokens with high scores are more likely to be selected for updates.

Inspired by Zheng et al. (2023), we introduce the discrete non-Markov discrete diffusion Model with top-K transition time (DNDM-K). Instead of directly determining which token gets updated at step t by first drawing transition time τ ∼ D τ , we employ a two-step process.

We first compute K t = ∑ N n =1 1 ( τ n ≥ t ) . k t represents how many tokens should be decoded at the current step.

Compare K t -1 and K t , if K t -1 = K t . There is no transition time at time t , we just update x t -1 , 1: N = x t, 1: N . If K t -1 > K t , Then there exist transition time at time t , we calculate and select the indexes with topK t -1 scores. Then we update those tokens if it hasn't been updated yet.

Subsequently, we will only update those tokens with the highest K t score that hasn't been changed yet. Since the function evaluation occurs only when K t changes, DNDM-K can give an accelerated sampling algorithm. The details are presented in Algorithm 4.

F Experiment details

F.1 Conditional Text Generation

Parameter choices. In all experiments, the batch size is chosen to be 100 . For RDM and RDMk , our hyperparameter settings follow the original paper (Zheng et al., 2023) except for the batch size. Before the sampling, we used the saved checkpoint of trained models provided by the authors for discrete sampling experiments, and we trained the corresponding models for continuous sampling experiments.

For finite-step DNDM, the transition times are determined by the schedule, and we approximate the schedule with a Beta distribution Beta ( α, β ) (please refer to Section 3.2 for detailed explanation). The α and β values are selected by applying grid search on the validation sets. Based on the BLEU scores on the validation sets, we have selected Beta (15 , 7) for Multinormial Diffusion on IWSLT14 , Beta (3 , 3) for Absorbing Diffusion on both IWSLT14 and WMT14 , Beta (5 , 3) for Multinormial Diffusion on WMT14 and Absorbing Diffusion on WMT16 , and Beta (20 , 7) for Multinormial Diffusion on WMT16 .

For infinite-steps (continuous-step) diffusion (DNDM-C), the transition timestamps are sampled from Beta ( α, β ) , where the choice of ( α, β ) are chosen from (100 . 0 , 4 . 0) or (17 . 0 , 4 . 0) , based on the performance comparison on the validation set. In the end we choose Beta (17 , 4) for IWSLT14 and Beta (100 , 4) for WMT14 and WMT16 .

We conduct a performance comparison based on varying configurations of the Beta and Alpha distributions. The results of these comparisons are presented in Tables 10 and 9. Furthermore, to evaluate the efficacy of discrete versus continuous step schemes, we also conduct an ablation study under the same set of parameters (100 , 4) in Table 11.

Continuous time vs discrete time diffusions. To test our hypothesis that the continuous-time sampler will produce more accurate results in reverse sampling if our x 0 estimator consistently approximates the true x 0 over time, we conduct various sampling experiments using a shared pretrained neural network. For discrete-time sampling, we consider three cases: T = 25 , 50 , 1000 . In each case, we rescale the interval [0 , T ] to [0 , 50] and divide it into T fractions. In contrast, for continuous-time sampling, we directly sample from a continuous distribution over the interval [0 , 50] without any partitioning.

Training approach. In machine translation tasks, the neural network is designed to learn q ( x 0 | x t , z ) , where z represents the embedding of the source text obtained using transformer encoder layers. For a fair comparison, we employ the same neural network structure as our baseline, with detailed architecture specifications available in Section E.2 of Zheng et al. (2023). Furthermore, given that the primary focus of this paper is the speed and effectiveness of our sampling algorithm, we omit the training procedure and instead use a state-of-the-art diffusion-based pretrained checkpoint from Zheng et al. (2023). In the Appendix, we present additional results of continuous sampling based on a continuously trained checkpoint. In this setting, we rescale our network input to the interval [0 , 1] and uniformly sample from this interval. The rest of the architecture follows that of Zheng et al. (2023).

Performance on WMT14. Our work primarily focuses on the sampling process, and for the training, we utilized a pretrained checkpoint trained on 50 steps. In our sampling experiments we noticed that our method does not work ideally on WMT14 , this could be possibly attributed to the fact that the training performance on WMT14 was not ideal. Specifically, when we performed sampling using 1000 steps, the network was trained with exposure to only 50 time steps, specifically at intervals of 20 (0, 20, 40, ..., 980, 1000). As a result, when we apply our model to generation using 1000 steps, the checkpoint NN has only been explicitly trained on these intervals. While we generally assume that the network can still provide a good estimate for the untrained steps, this might not hold under some hard scenarios. Considering the longer training time and poorer performance of WMT14 , it is likely that the training performance is insufficient for us to rely on those unseen steps. In a word, the model's trained checkpoint may not be robust enough to effectively handle unseen steps, especially for timesteps 1000 or infinite timesteps.

F.2 Unconditional Text Generation

Parameter choices. We recover the checkpoints of the multinomial diffusion model employing the provided code by Hoogeboom et al. (2021b). We train 12-layer Transformers for both text8 and enwik8 datasets for 500 epochs with the cosine schedule. For the text8 dataset, we utilize a training batch size of 256, while for the enwik8 dataset, we use a batch size of 128. During training, we employ a learning rate of 0.0001, a weight decay parameter of 0.99, and the Adam optimizer.

G Additional Experiments

In this section, we present additional experimental results. We begin by plotting the relationship between computational time and the number of sampling steps, using the absorbing diffusion in IWSLT14 as an example. Figure 4 displays the growth of computational time for absorbing diffusion (yellow and orange lines), RDM-absorbing diffusion, and our model DNDM-Absorb and DNDM-TAbsorb (green and blue lines). We see from Figure 4 that previous algorithms, including absorbing

25

50

1000

# of Sampling Steps

0

1000

2000

3000

4000

Computational Time (s)

Absorb

DNDM-Absorb

RDM-Absorb

DNDM-T-Absorb

diffusion and RDM-absorbing diffusion all suffer from linear growth of computational time.

G.1 Continuous Training

In Section 4.1, we introduce the DNDM-C algorithm, designed for continuous-time, over discretetime algorithms. However, this algorithm assumes that we have learned a sufficiently accurate neural network at any timestamp t ∈ [0 , 1] . Using the checkpoint trained with 50 discrete time partitions might not suffice for the purpose of continuous sampling. In this section, we investigate the performance of continuous sampling when training is also done continuously.

In Table 12, we summarize the performance of DNDM-C based on a neural network estimated continuously during training time. This involves sampling time uniformly from [0 , 1] during training, and the forward process follows (11) in Section 3.3. The training objective remains the same as in discrete-time training. In Table 12 we list the result of IWSLT14 and WMT16 with continuous training followed by continuous sampling. In addition, we compare the value with the corresponding value during discrete training and continuous sampling in Section 4.1 and mark every item that improves in bold. As demonstrated in Table 12, there is room for enhancement in the overall sampling scores by training the neural network in a complete space of timestamps.

G.2 Comparison with more generative models

In our study, a key aspect of evaluating our fast discrete generative model involves comparisons with prior work known for speed in sampling with minimal steps. Specifically, we draw a direct comparison with the Mask-Predict (Ghazvininejad et al., 2019), which is notable for its ability to generate high-quality results within just 10 iterations. The results are shown in Table 13. All experiments were conducted on the same GPU and within the same machine setup.

G.3 Samples from the multinomial text models

Conditional Generation. For DNDM-Multi trained on IWSLT14 , we provide a full generation process with 100 steps in Figure 5. A token ending with @@ indicates it is an incomplete word; it will be concatenated with the following token to form a complete word. For example, ' fel@@ lo@@ ws ′′ means ' fellows ′′ . We can see that after t = 39 , the generate sentence converges.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The contributions are summarized as three points at the end of the introduction. The scope is fast sampling via discrete non-Markov diffusion models, provided in the abstract.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We add a limitation section in front of the Appendix.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Theorems 3.1, 3.5, and D.1 are clearly stated, well-organized with consistent numbering, and supported by rigorous proofs that establish their validity.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The paper provides detailed information on the experimental setup, model architecture, and training procedures. The authors have submitted their training code along with the main paper, which enables reproducibility of the main results. The code and detailed instructions allow other researchers to replicate the key findings of the paper.

Guidelines: In addition to experiment and implementation details on appendix, we submit our training and evaluation codes when submtting our main paper.

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in

some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: All the datasets are public and can be open accessed. Our codebase will be available in public upon acceptance.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We provide these details on Appendix (D, E, F).

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Confidence intervals are provided in the experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provided detailed information about the computation resources in Section 4: a single NVIDIA258 RTX A6000 GPU with 48 GB memory.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We have checked NeurIPS Code of Ethics. Our submission satisfies all the requirement.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We provide Broader Impacts Section in the beginning of Appendix.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no related risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All used code, data and models in this project are properly cited.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

t = 100

[noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise]

t = 79

[noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] year [noise]

t = 78

[noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] we [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] year [noise]

t = 77

[noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] and we [noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] year [noise]

t = 75

[noise] [noise] [noise] [noise] [noise] [noise] [noise] [noise] and we [noise] [noise] [noise] [noise] govern@@ [noise] [noise] year [noise]

t = 74

we [noise] [noise] [noise] lo@@ [noise] [noise] [noise] and we [noise] [noise] [noise] [noise] govern@@ [noise] [noise] year [noise]

t = 73

we [noise] [noise] fel@@ lo@@ [noise] [noise] [noise] and we let [noise] [noise] [noise] [noise] govern@@ [noise] [noise] year [noise]

t = 71

we [noise] [noise] fel@@ lo@@ [noise] [noise] [noise] and we let [noise] [noise] [noise] [noise] govern@@ [noise] every year [noise]

t = 67

we [noise] [noise] fel@@ lo@@ [noise] [noise] [noise] and we let them [noise] [noise] city govern@@ [noise] every year .

t = 66

we [noise] [noise] fel@@ lo@@ ws [noise] [noise] and we let them work [noise] city govern@@ [noise] every year .

t = 64

we [noise] [noise] fel@@ lo@@ ws [noise] [noise] and we let them work [noise] city govern@@ ance every year .

t = 61

we [noise] [noise] fel@@ lo@@ ws [noise] [noise] and we let them work with city govern@@ ance every year .

t = 60

we [noise] [noise] fel@@ lo@@ ws [noise] year and we let them work with city govern@@ ance every year .

t = 58

we [noise] [noise] fel@@ lo@@ ws every year and we let them work with city govern@@ ance every year .

t = 52

we [noise] some fel@@ lo@@ ws every year and we let them work with city govern@@ ance every year .

t = 39

we choose some fel@@ lo@@ ws every year and we let them work with city governance every year.

t = 0

we choose some fel@@ lo@@ ws every year and we let them work with city governance every year.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes an accelerated sampling method from standard discrete diffusion models like multinomial diffusion and absorbing-state diffusion, using a non-Markovian forward process where the stochasticity is modelled by sampling a single transition time for each token, after which the process is fully determined. The model is trained similarly as a standard discrete diffusion model, but the formalism allows for a different type of sampler: The transition times are sampled in advance in the beginning, and the denoiser neural network simply parameterizes the transitions that happen at those times in the reverse process. This allows to skip over redundant steps where no transition would occur in the standard discrete diffusion models, speeding up sampling significantly for large step counts. The model provides improved results and faster sampling speeds over a prior accelerated sampling method for sequence-to-sequence generation tasks, and similarly improved results for unconditional language modelling with the text8 and enwik8 data sets when comparing with a prior multinomial diffusion model.\n\nSTRENGTHS:\n+ Clearly improves over a standard multinomial and absorbing-state schedule in sampling speed and results\n+ Improves over a previous published work in accelerating discrete diffusion models\n+ The framework could be a useful conceptual tool for people working with standard discrete diffusion models in the style of D3PM and want to accelerate sampling. Methods like ARDM from Hoogeboom et. al. and the tau-leaping of Campbell et al. are similar in function, but either require reframing the modelling problem differently or only allow for absorbing-state transitions in the case of ARDM. \n+ In contrast, this paper presents a method that allows changing a model in the D3PM framework to a faster one that skips over redundant steps, and shows how it is connected to the original model as a model with transition times as latent variables. \n+ The method is simple and easy-to-use\n\nWEAKNESSES:\n- The method is, in practice, somewhat similar to prior work (Campbell et. al. and Hoogeboom et. al.), and in that sense it would be more beneficial for the community if the paper went deeper into analysing potential speed improvements in discrete diffusion models or theoretical connections between their method and other work. \n- It would be useful if we had results from less steps in Table 3: Seems like the biggest time improvements are obtained with large step counts, but the BLEU scores are not drastically different from low step count BLEU scores. If both methods work quite well with low step counts already, then the results don't seem like the best showcase of the improvements due to increased step counts. Overall, it would be useful to have a clearer picture of the situations in which the method provides practical benefits. \n\n[1] \"A Continuous Time Framework for Discrete Denoising Models\", Campbell et. al. \n[2] \"Autoregressive Diffusion Models\", Hoogeboom et. al.\n\nQUESTIONS:\n- Since the accelerated multinomial model now does only one step for each token, I suppose it is not quite equivalent to the standard multinomial diffusion, where multiple transitions per token can happen during the generative process? Could the authors clarify the similarities and differences?\n\nLIMITATIONS:\nAddressed.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new formulation for discrete diffusion models whereby the corruption process is defined as a non-markovian process. At each point, a decision is made as to either stay in the current state or switch to a noise sample however crucially, this noise sample is constant throughout the process thereby only a single transition happens for each dimension. During generation, the model then just needs to step from transition time to transition time and can ignore superfluous simulation steps where no transition happens. The model is trained using an ELBO which the authors derive for their transition time conditional process. The authors test their model on machine translation examining quality versus sample speed and also on unconditional text generation with the text8 dataset.\n\nSTRENGTHS:\nThe authors present an original and interesting idea, it is a benefit to the community to point out the fact that standard discrete corruption processes have unnecessary transitions in them and that the same $q(x_t | x_0)$ distribution can be obtained by simply switching between a data token and a noise token. The idea to condition on the transition time itself within the framework also enables some significant simplifications for general styles of corruption whereas in the past this advantage may have only been noticed for absorbing state processes.\n\nThe paper is well written, it is fairly easy to get a good understanding of the proposed method on a first read of the paper and this is helped by the fact that the authors move some nuances and complexity to the appendix. Effort has been made to make the main text readable and intuitive which is greatly appreciated.\n\nI believe the paper will have some impact in the community because it is a quick win when users implement a discrete diffusion model. Instead of the standard sampling procedure of calling the network T times, users can switch to the proposed (quite simple in reality) algorithm and step directly from transition to transition without needing to change much implementation and gain a lot of speed up over the naive algorithm.\n\nWEAKNESSES:\nI think there should be some discussion relating to quite a simple baseline that can be implemented for the absorbing state case. One can sample an absorbing state diffusion model in the same way as a standard discrete diffusion model, by stepping from each timestep to the next, however, instead of carrying out a forward pass of the neural network for every timestep, you first check if any tokens perform a transition and only if at least one token transitions, you then do the forward pass. This is possible in the absorbing state case because the unmasking rate is independent of the neural network output. Therefore, this simple algorithm would have the same stated advantages as your method where at most the neural network is called N times but if T is less than N, it will be called T times. I see that for other diffusion styles, your algorithm is fundamentally different due to conditioning on the transition time, however, in the absorbing state case, it seems very similar to this simple method.\n\nHow can you explain the fact that you have better performance in terms of sample quality than the baselines you compare to? In your method, you propose no methodological advancement that should improve sample quality as your method is to speed up sampling to make sure the neural network is only called an appropriate number of times. Especially for a large number of timesteps, in Tables 2 and 3, you get across the board improvement, but your algorithm should be very similar to a standard discrete diffusion model in this regime but simply with fewer network evals.\n\nFor your text8 experiment, you say you use a 9e7/5e6/5e5 train/val/test split however I believe this is incorrect because this only adds up to 9.55e7 tokens but the text8 dataset contains 1e8 tokens. This may be inherited from a typo in https://arxiv.org/pdf/2107.03006 pg.25 but in a more recent paper https://arxiv.org/pdf/2308.07037 pg 45, the test set is 5e6 tokens. Please confirm which size of test set you used.\n\nThe names of the method seem to be slightly confused, for example on Figure 1, a DNDM-T is referenced but this is never actually defined in the full text.\n\nQUESTIONS:\nIn the end, do you think it will be possible to move beyond the idea of a time variable altogether? Since the final sampling algorithm steps from transition to transition, it seems that the more important variable is which tokens become denoised, in what order, and how many in one go. This is decoupled from time through your algorithm so perhaps there is even more simplification to be made.\n\nDo you think it is possible to learn the transition times on a per dimension basis? This could be useful to train models that can generate tokens in an intelligent ordering rather than currently, the order of generation is completely decided before any tokens have been generated when the transition times are sampled and the ordering is completely uniform over all orderings of the dimensions.\n\nLIMITATIONS:\nThe authors adequately discuss the limitations in the appendix.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents the non-Markov process for the discrete diffusion to reduce the sampling time. The authors present the transition time to de-randomize the sampling process and study the non-Markov processes from finite to infinite step sampling. The conditional text generation and unconditional text generation results demonstrate the effectiveness of the proposed method.\n\nSTRENGTHS:\n1. The writing of the article is very clear and easy to understand.\n\n2. Compared with the image field, discrete diffusion is more noticeable in the text field. So I think the conditional text generation and unconditional text generation experiments are sufficient.\n\nWEAKNESSES:\n**W1**: The authors claim that Eq.(1) and Eq.(6) are different because $w_t$ in Eq.(1) is independently drawn from the noise distribution $q_{noise}$ and $w$ in Eq.(6) is time-invariant. But, $q_{noise}$ is a Dirac distribution for the absorbing diffusion and $w_t=w$ where $t=1, \\dots T$.  So Eq.(1) and Eq.(6) are equal for the absorbing diffusion, which means the proposed non-Markov process is the same as the Markov process.\n\n**W2**: Besides W1, we can further deduce that for the absorbing process, the proposed DNDM sampling algorithm is equivalent to the original sampling algorithm of the Markov process.\n\nFor the origin Markov process, given $x_t$，we first sample $x_0 \\sim p_{\\theta}(x_0|x_t)$ and sample $x_{t-1} \\sim q(x_{t-1}|x_t, x_0)$ as shown in Eq.(4). I use $[M]$ to represent the absorbing state.\n\\begin{align}\nq(x_{t-1}|x_t, x_0) = \\frac{q(x_t|x_{t-1}, x_0)q(x_{t-1}|x_0)}{q(x_t|x_0)} = \\frac{q(x_t|x_{t-1})q(x_{t-1}|x_0)}{q(x_t|x_0)}\n\\end{align}\n\nFirstly, if $x_t$ is not the absorbing state, we have that $x_{t-1}=x_t$ because $q(x_t =a| x_{t-1} \\neq a) =0$ where $a \\neq [M]$.\n\nSecondly, if $x_t$ is the absorbing state, based on that $q(x_{t-1}=[M]|x_0)=1- \\alpha_{t-1}$, $q(x_{t-1}=x_0]|x_0)=\\alpha_{t-1}$, $q(x_t=[M]|x_{t-1}=[M])=1$ and $q(x_t=[M]|x_{t-1} \\neq [M]) = 1 - \\beta_t = 1 - \\frac{\\alpha_t}{\\alpha_{t-1}}$, we can get:\n\n\\begin{align}\nq(x_{t-1}=[M]|x_t=[M], x_0) = \\frac{1- \\alpha_{t-1}}{1 - \\alpha_t}, q(x_{t-1}=x_0|x_t=[M], x_0) = \\frac{\\alpha_{t-1} - \\alpha_{t}}{1 - \\alpha_t}\n\\end{align}\n\nBased on the above analysis, the sampling process of the absorbing Markov diffusion can be simplified as: we sample starting from an all $[M]$ sequence, and during the sampling, we sample $x_0 \\sim p_{\\theta}(x_0|x_t)$. If $x_t=[M]$, $x_{t-1}$ will stay the $[M]$ state with probability $\\frac{1- \\alpha_{t-1}}{1 - \\alpha_t}$ and transfer to $x_0$ with probability $\\frac{\\alpha_{t-1} - \\alpha_{t}}{1 - \\alpha_t}$. If $x_t \\neq [M]$, it will stay unchanged.\n\nIn order to further illustrate the relationship with DNDM sampling, I denote the first time $x_t$ transitions from $[M]$to a non-$[M]$ state as $\\tau$ (consistent with the transition time in this paper). We can deduce that $p(\\tau=k) =\\frac{\\alpha_{k-1} - \\alpha_{k}}{1 - \\alpha_k}  \\prod_{t=T}^{t=k+1} \\frac{1- \\alpha_{t-1}}{1 - \\alpha_t}=\\alpha_{k-1} - \\alpha_{k}$. $p(\\tau=k)$ and Theorem 3.5. further verified that the proposed Non-Markov method is the same as the original Markov method for absorbing diffusion.\n\n**W3**: The proposed non-Markov method of multinomial diffusion is the same as the method in DDIM[1], Appendix A but with a different formulation. We can use the $p(\\tau=k)$ in W2 to further verify it. \n\nIn conclusion, this paper presents a non-Markov process for the multinomial diffusion and absorbing diffusion to accelerate the sampling of origin method with Markov processes, this is the main contribution of this paper. However, for the multinomial diffusion, the relationship with the DDIM[1] (Appendix A) is not clearly stated. For the absorbing diffusion, the non-Markov processes are the same as the Markov processes.\n\nIf I misunderstand your approach, please feel free to figure it out, and I will adjust my score.\n\n[1] DENOISING DIFFUSION IMPLICIT MODELS, ICLR2021\n\nQUESTIONS:\n1. In line 281~282, the authors denote RDM and RDM-k as the sampling method with and without top-k selection, is this a typo?\n\n2. In the experiment, do you use a pre-trained model or train it yourself?\n\n3. The authors claim that they use a model consisting of an encoder and a decoder. This is confusing, do you remove the causal mask in the transformer decoder? For the unconditional text generation, what is the input of the encoder?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a discrete non-Markov diffusion model (DNDM) aimed at accelerating the sampling process in discrete diffusion models. The proposed method reduces the number of neural network function evaluations to speed up the sampling process while maintaining sample quality. The paper explores the transition from finite to infinite step sampling, providing new insights into bridging the gap between discrete and continuous-time processes. Experiments on natural language generation and machine translation tasks illustrate the competitive performance of the method in terms of speed and quality compared to existing methods.\n\nSTRENGTHS:\n* The introduction of a discrete non-Markov diffusion model (DNDM) provides a new method for accelerating the sampling process in discrete diffusion models in a training-free manner. It reduces the number of neural network function evaluations, enhancing the efficiency of the sampling process to speedup by 3x for 50 steps.\n* The authors conducted extensive experiments on natural language generation and machine translation tasks, demonstrating the effectiveness of the proposed method, for both multinomial and absorbing diffusions.\n\nWEAKNESSES:\n* The method involves a complex process that might be challenging to easily follow and implement. More details or visualizations on how the transition time distribution is determined and whether it can be adapted for different types of discrete diffusion models will be helpful for a better understanding of the motivation and methodology.\n* The comparison with other acceleration methods is not very convincing, especially at a practical smaller number of sampling steps. Instead of RDM baseline, how does the proposed method compare with other existing acceleration techniques for discrete diffusion models in terms of both efficiency and quality? \n* While the method is tested on natural language generation and machine translation tasks, its applicability to other modalities such as image or video generation is not unknown, which might limited the scope of the proposed method.\n\nQUESTIONS:\nSuggest to address concerns in the weakness section.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}