DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models
Authors: Shansan Gong
Mukai Li
Jiangtao Feng
Zhiyong Wu
Lingpeng Kong
Venue: EMNLP 2023 Findings (2023)
Keywords: diffusion model
sequence to sequence
text generation
TL;DR: Accelerate continuous text diffusion model by connecting the discrete nature.

Abstract: Diffusion models have gained prominence in generating high-quality sequences of text. Nevertheless, current approaches predominantly represent discrete text within a continuous diffusion space, which incurs substantial computational overhead during training and results in slower sampling speeds. In this paper, we introduce a soft absorbing state that facilitates the diffusion model in learning to reconstruct discrete mutations based on the underlying Gaussian space, thereby enhancing its capacity to recover conditional signals. During the sampling phase, we employ state-of-the-art ODE solvers within the continuous space to expedite the sampling process. Comprehensive experimental evaluations reveal that our proposed method effectively accelerates the training convergence by 4x and generates samples of similar quality 800x faster, rendering it significantly closer to practical application. \footnote{The code is released at \url{https://github.com/Shark-NLP/DiffuSeq/tree/diffuseq-v2}.}

Dif: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models
Shansan Gong' Mukai Li Jiangtao Feng" Zhiyong Wu Lingpeng Kong'
'The University of Hong Kong IIndependent Researcher SShanghai AI Lab
sansa933econnect. hku. hk eeg.ttoemonng000emmmlll... com lpkêcs hhke. hk
Abstract
Forward
Discrete nois
Diffusion models have gained prominence in generating high-quality sequences of text. Nev- ertheless, current approaches predominantly represent discrete text within a continuous dif- fusion space, which incurs substantial compu- tational overhead during training and results in slower sampling speeds. In this paper, we intro- duce soft absorbing state that facilitates the diffusion model in learning to reconstruct dis- crete mutations based on the underlying Gaus- sian space, thereby enhancing its capacity to recover conditional signals. During the sam- pling phase, we employ state-of-the-art ODE solvers within the continuous space to expedite the sampling process. Comprehensive exper- imental evaluations reveal that our proposed method effectively accelerates the training con- vergence by 4x generates samples simi- lar quality 800x faster, rendering it significantly closer to practical application.
Training stage
100000
N 000000
(
00C000
Contonmuus notee
X
1 2
Jointly Denolse
7/-1
000000
Sampling stage
000000
100000
0000000
Suft abooching bing state
2t
Jointl Denoise with DPM-Solver++
Reverse process
Figure I: Training and sampling stages with discrete noise, which helps two stages align better.
the Minimum Bayes Risk (MBR) decoding strat- egy (Koehn, 2004) to enhance generation quality, resulting in doubling of time consumption.
1 Introduction
In order to further narrow the gap between diffu- sion models and the prevailing autoregressive mod- cls, aim propose accelerated version of DiffuSeq for both the training and sampling stages. For training, in addition GPU acceleration tech- niques such as FP16, an improved training scheme can enable the model to better represent knowledge and learn data distribution more quickly (Hang et al., 2023). For sampling, well-trained diffu- sion model can achieve similar quality within a single sampling and without the need for MBR de- coding, thus saving generation time. Furthermore, can borrow the state-of-the-art ODE sampler DPM-solver++ (Lu al., 2022a,b) which is already applied fast image generation. The progress discrete text diffusion models (Zheng et al., 2023) exhibits its superiority in using fewer sampling steps, thus intuitively inspiring us to bridge the gap between continuous and discrete spaces.
After diffusion models gained significant attention in the vision domain (Ho al., 2020), progress has been made in upplying them to text genera- tion tasks, including constrained text generation in Diffusion-LM (Li et al., 2022) and sequence- to-sequence (Seq2Seq) text generation in Dif- fuSeq (Gong al., 2023). Numerous subsequent studies demonstrate that diffusion models have achieved results comparable traditional autore- gressive models and non-aut autoregressive models in tasks such as machine translation (Yuan et al., 2022: Gao et al., 2022; Zheng et al., 2023) and summa- rization (Lin et al., 2022: Zhang et al., 2023; Ma- habadi et al., 2023; Zhou et al., 2023). However, most of these works suffer from slow convergence during training and slow generation speed, partic- ularly considering that these approaches require
'The code is released at https: https:///itttbbcomm Maark-NLPPDiffuSeee fttuuuee//ttreefffuueeerrrr
Based on the BERT (Zhang et al., 2019) and

---

BART (Lewis et al., 2020), as well the absorb- ing state in D3PM (Austin et al., 2021), we propose incorporating an extra learned soft absorbing state and discretely adding it with Gaussian noise to jointly denoise the noise from two sources. The processes are illustrated in Figure 1. Specifically, after posing Gaussian noise, we randomly replace the continuous vector of the sequence with the ab- sorbing state. The ratio is set according the time step. This approach bridges the gap between continuous and discrete diffusion processes and also makes the training and inference stages better aligned. It also facilitates the integration of the DPM-solver++ and reduces the number of required sampling steps. In summary, our contributions are:
distribution over the vocabulary. D3PM (Austin et al., 2021) specifies ((x/(x(1) through transi- tion matrix, and makes to be a point mass with the probability on an absorbing state [MASK]. Zheng et al. (2023) further derive an equivalent reparam- cterization to the discrete diffusion process. The resulting formulation is more amenable to train- ing and leads to much-improved generation quality, However, discrete diffusion models may miss the opportunity directly leverage the existing tech- niques from continuous diffusion models.
Methods
In DiffuSeq, it formulates Seq2Seq tasks as con- ditional generation and learns p(w"lw") where w and w refer to the input and target sequence separately. We follow the notation, concatenating two sequences z; X2 yt. where x, and y: represent parts of z; that belong to w and w respectively. To accelerate continuous diffusion models as well as leverage the absorbing state of discrete diffusion models, we add learnable soft absorbing state into DiffuSeq. We first combine the continuous Gaussian noise and discrete absorbing noise, and then jointly denoise them. The detailed derivations can be found in Appendix A.
1. We introduce the learned soft absorbing state to help continuous diffusson models converge faster and eliminate the need for MBR decod- ing to ensure quality during sampling.
2. We adapt the DPM-solver++ to our enhanced diffusion text generation approach and demon- strate its feasibility in accelerating the genera- tion speed experimentally.
2 Preliminaries
2.1 Continuous Diffusion Models
3.1 Training Stage
Ho al. (2020) and Song et al. (2020) formulate diffusion models in continuous space including forward and reverse processes. The forward pro- cess gradually corrupts data point Xy into a stan- dard Gaussian noise Xy ~ N (O, I). For each for- ward step t E [1, 21.,, . T), the perturbation is fol- lowed by ((xx(xx-1) = NN(x; /1 B,x1-1, 3,T), with 3, E (0, 1) as different scales. After the for- ward process, the reverse denoising process tries gradually reconstruct the original data xo via sampling from xy by learning a diffusion model Diffusion-LM (Li et al., 2022) and Dif- fuSeq (Gong et al., 2023) design an embedding function EMB( () to map the discrete text w into a continuous space and operate clamping on x; to map it back to word embedding space at each sampling step reduce rounding errors.
Forward process with soft absorbing state In continuous space, first add Gaussian noise for each time step with a₁ 1 3r, ar II az;
Z 2 + v 5ge.
(1)
Considering the i-th token in the hidden represen- tation Z, of the word sequence, we replace its rep- resentation with our soft absorbing state m at a certain probability. The soft absorbing state m is in the same hidden dimension word embeddings and is also jointly learned along with the whole diffusion process.
p=
2
(2)
z if if 0"
where P Bernoulli(B{ * *7). and is the [MASK] ratio when f = T. This operation keeps the diffusion model in continuous space but dis- cretely replaces the representation of some tokens in the sequence, and the replacement probability is also scaled the time step, same with Br. Noted that these two kinds of noise are posed partially on the target y, space in the manner of DiffuSeq.
2.2 Discrete Diffusiio Models
For discrete diffusion probabilistic models, each x; is a discrete random variable as one-hot vectors in {0, 1)K, indicating the current state of each token, where K the size of the vocabulary. Multinomial diffusion (Ho et al., 2020) adopts a uniform noise

---

Table 1: Sequence-ter to-suueeee text generation results on QQP. All results are reported without MBR decoding if not specified. The best result is bolded, and the gray columns are excluded from comparison considering the faimesss The sampling step of BG-DiffuSeq is 20. The relative improvement f is computed between our speedup version (step=2) and the DiffuSeq (MBR=1).
Continuous space
Discrete Space (step=10)
Mixed space (ours)
DiffuSeq
DiffuSeq
BG-
Multi- nomial
RDM+
D3PM-
RDM+
Original
Speedup
Speedup
(MBR-1)
(MBR-10)
DittuSiq
Matunomial
absorbing
absorbing
(92p=2000)
(sep=10)
(stop=2)
BLEU
0.1884
0.2413
0.2077
0.2025
0,2315
0.2245
0,2313
0.2411
0.2210
0.2115
12.3%
R-L
0.5327
0,5880
0.5652
0.5516
0,5712
0.5677
0.5735
0.5930
0,5732
0.5651
6.1%
BertScore
0.7965
0.8365
0.8057
0.7966
0.8375
0.8281
0.8416
0.8393
0.8207
0.8036
0.8%
Speed (its)
0.51
0.05
60.3
106.4
108.6
116.1
121.0
1.07
208.3
406.5
R: 800
Jointly denoise The reverse process is to jointly reconstruct the corrupted data point. The simplified loss function is almost the same with DiffuSeq, except for z₁ in different noise strategies:
quality and boost the training convergence? RQ2: To what extent does the DPM ODE solver improve sampling speed and affect the performance?
4.1 Experiment setup
T min LVLB= min Σ llvo - +)]
Dataset We adopt QQP for paraphrasing the sentence to the same semantic content, which is lightweight train and has been widely used in many Seq2Seq text diffusion models.
(3)
[EMM(w""- fo 1)11 +(112011)
3.2
Sampling Stage
Previous continuous text diffusion models adopt clamp operation to make the vector predictions more precise and reduce rounding errors (Li et al., 2022) during sampling. However, this operation is not deployed in the training stage, and the gap between training and sampling (Tang et al., 2023) may hinder the performance and the further opti- mization for sampling speed.
Baselines We choose DiffuSeq (Gong et al., 2023), BG-DiffuSeq (Tang et al., 2023) as the rep- resentative of continuous diffusion models. The latter is the enhanced version of DiffuSeq, target- ing bridging the gap between training and sampling to generate high-quality texts within fewer steps. We choose multinomial (Hoogeboom et al., 2021), D3PM-absorbing (Austin et al., 2021), and a repa- rameterized version of them (Zheng et al., 2023) as the representative of discrete diffusion models, with fewer diffusion steps for training.
On the contrary, for our methods, during sam- pling, the same discrete noise in Eq. (2) is sprinkled in the continuous Gaussian noise, which bridges the training and inference in discrete space. Using the exact solution of diffusion ODEs proposed by DPM-solver++ (Lu et al., 2022a,b), given an initial value 25 at time >> O, we have:
Implementation details We follow the training. sampling, and evaluation implementation of Dif- fuSeq'. The experiment is deployed on NVIDIA A 100 80G GPUs, with 2 GPUs for training. More details can be seen in Appendix C.2.
2: - + oz L.a dr e Se(2,,))dl
(4)
where the A is a strictly decreasing function of f, at is monotonic to B1, and f0(2x. A) is aligned with training objectives. The integral term can be analyt- ically computed by repeatedly applying integration- by-parts n times, and we can just approximate the first several orders and drop the high-order error terms. We use second order in the experiment.
4.2 Main results
As seen from Table 1, original DiffuSeq relies on the MBR decoding to ensure generation quality. By contrast, our method in the original sampling version achieves comparable performance without applying MBR by sampling 10 times, which signif- icantly saves the time for generating high-qualiny texts. Even the speedup version (step=2) still out- performs DiffuSeq (MBR=1) lot, according to the
4 Experiments
We try to validate two main research questions. RQI: Can the soft absorbing state added con- tinuous diffusion models improve the generation
'tt phttps://wwe kwggle. com/c/ quora- carrr--uuutioonpppee pairs htttps: / thub. www/Shark-aaaaaeeeee -Nw.rrk-NNLPDDiFF/f ffuSqq

---

0.25
0.25
0:25
0.20
1.75x faster
0.20
B..BU
0.15
BLEU
0.15
Durs
BLEUU
0.20
(DDMM)
0.10
CPT2
y=0.6
0.15
- wio soft absorbing
p=0.4
000
0.05
absorbing
0.00
0.00
10
10
10
0.10
Training Time (hours)
Speed (it/s)
Training Time(hours)
Figure 2: The test BLEU score along with training hours under dif- ferent training schemes.
Figure 3: Generation speed and quality under different sampling steps incorporating DPM-solver.
Figure 4: The test BLEU score at different training hours for differ- ent settings of the ratio y.
relative improvement. Our method is also superior to BG-DiffuSeq, which bridges the gap between training and sampling in continuous space. The speedup versions of our methods still have an ad- vantage over discrete diffusion models, especially in terms of generation speed. We do not directly compare with RDMs because it designs an algo- rithm to route the discrete change of words, while ours just uses vanilla transition which uniformly changes tokens to the soft absorbing state.
Table Different sampling strategies. [C] denotes clamp operation and w/o [M] denotes stopping adding discrete noise during sampling.
DiffuSeq
Ours (Original)
w [C]
w/o [C]
w [C]
w/o [C]
w/o [M]
BLEU Drop (%)
19.87
19.19
24.11
23.98
20,50
3.4
0.5
14,9
between training and sampling, and remove the clamp operation. And further plugging in with DPM-solver++ will not introduce extra rounding errors. We test sampling without soft absorbing state on our model which still adds this discrete noise in the training stage. A significant drop can be seen, demonstrating the importance of the align- ment of training and sampling. We also analyze the choice of the hyper-parameter 7, As seen in Figure 4, too small or too large [MASK] rate will harm the performance, and closer the middle tends to perform better. So we choose y 0.5 as the default setting our experiment.
4.3 Training speed
We use FP16 for GPU acceleration (Ott et al., 2018), which reduces the total training time from 28 11 hours with 2 GPUs and brings 2.5x speed up. According to Figure 2, the jointly denoise train- ing scheme expedites training convergence by at least 1.75x, probably because the absorbing state perturbs the representation of sequence discretely, which empowers the model with better capacity to reconstruct the discrete text information. The train- ing consumption is saved more than 4x in total.
4.4 Sampling speed
Sampling with FP16 is approximately 2x faster than the original DiffuSeq. Furthermore, the incor- poration of DPM-solver++ shrinks the sampling step to or even without sacrificing the perfor- mance much, as shown in Figure 3. This improve- ment is significant compared with DDIM (Nichol and Dhariwal, 2021) used in DiffuSeq. Compar- ing our speedup version (step=2) with DiffuSeq (MBR=1), texts with higher quality generated conditionally and meanwhile about 800x faster.
5 Conclusions
In this work, we present a simple but effective train- ing scheme for joint discrete and continuous text diffusion models, which additionally resets some tokens into the soft absorbing state. The discrete noise bridges the training and sampling stages, sav- ing time consumption of these two stages and the plugged DPM-solver++ further makes the sampling faster. Our method is orthogonal to many other techniques such as self-conditioning (Mahabadi et al., 2023; Chen al., 2022), choosing tokens with different importance (He al., 2022), which can also further enhance generation quality. Our method is fundamental to diffusion text generation and can be applied beyond DiffuSeq (Chen et al., 2023).
4.5 Ablation study
We test different sampling strategies in Table 2. Af- ter removing the clamp operation, the performance of DiffuSeq gets affected evidently, while ours sel- dom drop, which validates our previous assumption that adding the soft absorbing state bridges the gap

---

Limitations
Zhengfu He, Tianxiang Sun, Kuanning Wang, Xuan- jing Huang, and Xipeng Qiu. 2022. Diffusionbert: Improving generative masked language models with diffusion models. ArXiv, abs/221 1.15029.
Regarding the methods, we opt not to incorporate length prediction, unlike other approaches. Instcad, we use the [PAD] token to indicate the length au- tomatically. This may require more GPU memory for text generation. We validate our methods on the QQP dataset, which is one of the sequence-to- sequence text generation tasks. However, due to resource and time constraints, we are unable to test the effectiveness of our methods on more complex tasks such as machine translation and summariza- tion. Additionally, this work does not explore the impact of scaling up the model size.
Jonathan Ho,, Ajay Jain, and Pieter Abbeel. 2020. De- noising diffusion probabilistic models. Advances in Neural Information Processing Systems.
Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling. 2021. Argmax flows and multinomial diffusion: Learning categorical dis- tributions. Advances in Neural Information Process- ing Systems.
Minghui Hu, Chuanxia Zheng, Heliang Zheng. Tat-Jen Cham, Chaoyue Wang. Zuopeng Yang. Dacheng Tao, and Ponnuthurai N Suganthan. 2022. Unified discrete diffusion for simultaneous vision-language genera- tion. ArXiv, abs/2211 14842.
Ethics Statement
In this study, we used diffusion models to gener- ate text. To ensure that the generated text is free from bias, carefully selected our training data and evaluated the quality of the generated text. We acknowledge that our rescarch has potential ethical implications, particularly in the area of text genera- tion for malicious purposes. To mitigate this risk, we have taken steps to ensure that our research is conducted in an ethical manner and that the gener- ated text is used only for legitimate purposes.
Philipp Koehn. 2004, Statistical significance tests for machine translation evaluation. In Proceedings of the 2004 conference on empirical methods in natural language processing, pages 388-395.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. Bart: Denoising sequence-to-seque to-squeence pre-training for nat- ural language generation, translation, and comprehen- sion. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 787 1-880.
References
Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang, and Tatsunori Hashimoto. 2022. Diffusion- Im improves controllable text generation. ArXiv, 0bss/2205.14211
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. 2021. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems.
Chin- Chin Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out.
Jiaao Chen, Aston Zhang, Mu Li, Alex Smola, and Diyi Yang. 2023. A cheaper and better diffusion language model with soft-masked noise. ArXiv, abs/2304.04746.
Zheng- Wen Lin, Yeyun Gong, Yelong Shen, Tong Wu, Zhibao Fan, Chen Lin, Nan Duan, and Weizhu Chen. 2022. Text generation with diffusion language mod- els: A pre-training approach with continuous para- graph denoise. volume abs/2212. 1685.
Ting Chen, Ruixiang Zhang, and Geoffrey Hinton. 2022. Analog bits: Generating discrete data us- ing diffusion models with self-conditioning. ArXiv, abs/2208.04202
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongx- uan Li, and Jun Zhu. 2022a. Dpm- fast ode solver for diffusion probabilistic model sampling in around 10 steps. In Conference on Neural Informa- tion Processing Systems, NewrIPS.
Zhujin Gao. Junliang Guo, Xu Tan, Yongxin Zhu, Fang Zhang, Jiang Bian, and Linli Xu. 2022. Difformer: Empoowring diffusion model on embedding space for text generation. ArXiv, abs/2212.09412.
Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and Lingpeng Kong. 2023. DiffuSeq: Sequence sequence text generation with diffusion models. In International Conference on Learning Representa- tions, ICLR.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongx- uan Li, and Jun Zhu. 2022b. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models. ArXiv, abs/2211.01095.
Rabeeh Karimi Mahabadi, Jaesung Tae, Hamish Ivi- son, James Henderson, Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2023. Tess: Text- to-text self-conditioned simplex diffusion. ArXiv. abs/2305.08379.
Tiankai Hang, Shuyang Gu, Chen Li, Jianmin Bao, Dong Chen, Han Hu, Xin Geng, and Baining Guo. 2023. Efficient diffusion training via min-snr weight- ing strategy. ArXiv, 20022333309556

---

Alexander Quinn Nichol and Prafulla Dhariwal. 2021. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, ICML.
The forward noising is to perturb the structure of data zo- zo is finally changed into the partial Gaussian noise with yT ~ (O, through Tsttp forward random disturbance
Myle Ott, Sergey Edunov, David Grangier, and Michael Auli, 2018. Scaling neural machine translation. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 1-9. Associa- tion for Computational Linguisties.
(2) (1)1) (Z1; V1 B,I),, (5) with f = 1,2,...; and {ir E 10,1) are the variance schedule. Let at = - St and oz =
II i==1 ag we have:
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: a method for automatic evalu- ation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Compu- tational Linguistics, ACL.
24 ==va 1 VI v1-004
=Va-00-12122 + V1 0604-161-2 (6)
_.. -Vā VI -āE,
Jamming Song, Chenlin Meng, and Stefano Ermon. 2020. Denoising diffusion implicit models. In International Conference Learning Representations, ICLR.
where e stands for Gaussian noises. In the end, ( zn) = NN(z; Z..zo (1 - Tz) We use a sqrt noise schedule a; = -1 -L-Vt/T /t/T + with as small constant at the start of the noise level. Based on ((z/(2t-), we further add discrete noise. Consider the i-th token in the hidden repre- sentation ze of the word sequence, we replace its representation with our soft absorbing state m a certain probability. The soft absorbing state m is in the same hidden dimension as word embeddings and is also jointly learned along with the whole diffusion process.
Zecheng Tang, Pinzheng Wang, Keyan Zhou, Juntao Li, Ziqiang Cao, and M. Zhang. 2023. Can diffu- sion model achieve better performance in text gener- ation? bridging the gap between training and infer- ence! ArXiv, abs/2305.0444 00465.
Hongyi Yuan, Zheng Yuan, Chuanqi Tan, Fei Huang. and Songfang Huang. 2022. Seqdiffuseq: Text dif- fusion with encoder-decoder transformers. ArXiv, 103222
Haopeng Zhang, Xiao Liu, and Jiawei Zhang. 2023, Diffusum: Generation enhanced extractive summa- rization with diffusion. ArXiv, abs/2305.01 7355
2
m
if pf -
(7)
Tianyi Zhang, Varsha Kishore, Felix Wu. Kilian Q Wein- berger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. In International Confer- ence on Learning Representations, ICLR.
if p=
Z
where P Bernoulli(B, * 7). and ? is the [MASK] ratio when = T. The replacement prob- ability i also scaled to the time step, proportional to 3r. All these two kinds of noise are posed par- tially in the manner of DiffuSeq, to make sure the condition signal x unchanged.
Lin Zheng, Jianbo Yuan, Lei Yu, and Lingpeng Kong. 2023. A reparameterized discrete diffusion model for text generation. ArXiv, 0557733
Kun Zhou, Yifan Li, Wayne Xin Zhao, and Ji rong Wen. 2023. Diffusion-nat: Self-prompting discrete diffu- sion for autrrerrreeeesivvee text generation. ArXiv, abs/2305.04044.
For reverse process, the continuous noise is ac- cumulated before, and defined as:
T
Yaoming Zhu, Sidi Lu, Lei Zheng. Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. 2018. Texygen: A benchmarking platform for text generation models.
pe po(zor)) := p(zr) II t=1
po(z: 1/zz). (8)
The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 1097-1100.
Po Po 1/zz) = NN 1: 10 f), o0 t)), (9)
where the (4(()) and a-(-) is the predicted param- eterization of the mean and standard variation of g 12-1) in forward process. Using Bayes' rule:
A Detailed Derivations of Our Methods
A.1 Training stage
((z-11 (20)
q2:-1|2,, 20) (2-1, 2n)
(10)
Following Ho et al. (2020); Nichol and Dhariwal (2021); Song et al. (2020); Li et al. (2022); Gong et al. (2023), we define the forward noising pro- cess and reverse denoising process on the latent continuous space z.
g(21
substitute Eq. (6) to it and we can get the parame- terized mean of g(2; |zz, Zn):
[e(2, 2o) var(1 ā.-1)
V 00- 13, 2, +
2u.
(11)
āu

---

and for brevity, we moit the coefficient of 2, and Zo as constants.
We need to approximate f _e dλ, which can be analytically computed by repeatedly applying times of integration-by-part According to the second order multistep DPM-Solver++ algorithm, the reconstruct of Z, relies on the fo. after posing discrete denoise in our methods, the algorithm still applies since our training objectives.
We can use the variational lower bound to optimize the negative log-likelihodd E[ lo-log/00 log logro((o)) I LVLB- The objective can be further rewritten to be combination of several KL-divergence.
is exactly aligned with
Lvaa LT + LT-1 +...+ Lo
log gizr|aa lav) (z(
B Related Work
E (mo)
Continuous diffusion models are first applied in im- age generation (Song et al., 2020; Ho et al., 2020) and then applied in text generation (Li et al., 2022; Gong et al., 2023). Meanwhile, discrete diffusion models (Austin et al., 2021; Hoogeboom et al., 2021; Zheng al., 2023) is designed for text gen- cration. (He al., 2022) and (Zhou et al., 2023) directly leverage the [MASK] token used in pre- trained language models. By contrast, our method learns the soft absorbing state from scratch along with the whole diffusion process and bridges dis- crete diffusion with continuous space. The idea of absorbing state or discretely corrupt data be seen in many NLP work like BERT (He et al., 2022) or BART (Lewis al., 2020), even in diffusion image generation (Hu et al., 2022).
Σ 1=2
log g(zi-1 lao, 21)
(12)
po 1
+og (e(2o/wwBy) pa (zo(zi)
log (ww m (20)
For stsT 1, we compute the parameteri- zation of C, by substituting Eq. (11) to minimize the difference from and following Ho et al. (2020):
( Izo, 2(+1)
C. Exo log
po po(2: (z 1
(13)
= CIE [|| |z0 fo(2,, ])
where C is Joss independent constant. intuitively believed that the above contin- uous diffusion models learns aims to recover the corrupt data z, with fo to Zo. When we look back to our added discrete noise, since its directly added to the q(zr/zo), which can be directly modeled by So. The L, should be:
C Implementation Details
C.1 General setting
We use A100 80G GPUs for training using FP16 with a batch size of 425 and single GPU on sam- pling with a batch size of 100. The generation speed is measured under the setting of 100 batch size on one NVIDIA A 100 80G GPU for all mod- els, averaged by runs.
C. ClE || ||z-- f2,, {)
(14)
Then the optimization of training loss ming LVLB can be further simplified as:
To evaluate the quality, we use the standard met- ric BLEU (Papineni et al., 2002) and ROUGE (Lin. 2004) score. Since ceiiin--simllaitt-baaaee based metrics can be unsatisfactory for open-ended generation, we also report BERTScore (Zhang et al., 2019) that assesses the semantic similarity between generated sentences and references. For sentence-level di- versity evaluation, we consider sentence-level self- BLEU (Zhu et al., 2018) to measure the n n-gram overlap between the set of outputs w.r.t one source sentence. The self-BLEU score is computed using samples for each test case generated with differ- ent seeds. All ablation study conducted using the original version of our models if not specified, to validate the effectiveness of introducing the soft absorbing state.
min LVLB min Σ Ilyo - f)(
[[EM)(w") Jo(21, 1)11 R(1120)3)
(15)
A.2 Sampling stage
DPM-solver++ (Lu et al., 2022a,b) proposed totally based on the continuous diffusion models. According to it, we have the exact solution of diffu- sion ODEs, given an initial value Z, at time 8 > 0:
Ar > fodd,
T: -zz + Ti
(16)
where the is a strictly decreasing function of t, at is proportional to 3r specifically, or = VI Cz.

---

C.2 Baselines setting All baselines we used are open-sourced. DiffuSeq" (Gong et al., 2023) and BG- DiffuSeq (Tang et al., 2023) are imple- mented based on HuggingFace Transformers*. RDM (Zheng et al., 2023) are implemented using Fairseq, where the temperature sampling is adopted when sampling and beam size is set 1. For generation speed, we report the results on single NVIDIA 100 80G GPU with batch size 100.
Diffusion
/https://github.cm httpt/t/hhhccc///un/////////


Reviews:
[{"review_id": "6B5Jo8OHfc", "reviewer": "Reviewer_ZN1G", "summary": "", "questions": "1. The theoretical explanation or intuitive reason for introducing an absorbing state to the continuous diffusion models.", "limitations": "", "rating": 2, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The proposed soft absorbing state method can boost the training convergence.", "reasons_to_reject": "1. Adding soft absorbing state (mask strategy) to the continuous diffusion process is a rigid combination. The paper does not explain why introducing an absorbing state can lead to improvement.\n2. Leveraging an ODE solver to accelerate the continuous sampling process is a straightforward method, which is too naive to be a novel contribution. \n3. The improvement of generation quality is minor compared with D3PM, also worse than the DiffSeq with MBR and RDM, which weakens the effect of the proposed soft absorbing state method. The saved training time cost is minor as well.", "questions_to_authors": "1. The theoretical explanation or intuitive reason for introducing an absorbing state to the continuous diffusion models.", "paper_topic_and_main_contributions": "The paper introduces a soft absorbing state to the continuous diffusion model to improve the generation quality and boost training convergence. The authors use an off-the-shelf continuous ODE solver during the sampling process to improve the sampling speed. The paper conducts experiments to validate the time cost of the training and sampling process while achieving comparable results to the baseline models.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "o0vRUVX88I", "reviewer": "Reviewer_SUQ8", "summary": "", "questions": "a. Is the methods you proposed generalizable? Can they be applied to various diffusion models, or can they only be used on Diffuseq?\n\nb. Can you provide the experiments (BLEU, R-L, bertscore, generation speed) finetuning the t5/bart/vanilla-seq2seq-transformer using the same datasets?", "limitations": "", "rating": 3, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This is a clearly written paper that makes useful contributions to accelerating the training and sampling of diffusion models for text generation. The method is simple but effective. The experiments comprehensively demonstrate accelerated training convergence, faster high-quality sampling, and strong performance compared to baselines.", "reasons_to_reject": "a. It lacks detailed comparisons with more baselines. There are at least 10+ text-generation baselines using diffusion models. (https://github.com/diff-usion/Awesome-Diffusion-Models#natural-language)\nSo more experiments should be conducted to test the generalizability of the proposed techniques.\n\nb. It lacks comparison with standard seq2seq Transformer text-generation baselines, such as a finetuned  T5 or BRAT. The final objective of using the diffusion model for text generation is to replace the traditional autoregressive/seq2seq model, so the comparison between diffusion and non-diffusion models is necessary.\n\nc. The proposed techniques lack novelty. This paper just simply adds two methods based on the previous strong baselines. DPM-solver++ is very popular and common in computer-vision field and using it could not be viewed as a contribution.", "questions_to_authors": "a. Is the methods you proposed generalizable? Can they be applied to various diffusion models, or can they only be used on Diffuseq?\n\nb. Can you provide the experiments (BLEU, R-L, bertscore, generation speed) finetuning the t5/bart/vanilla-seq2seq-transformer using the same datasets?", "paper_topic_and_main_contributions": "This paper proposes a method to accelerate sequence-to-sequence text generation using diffusion models. The main contributions are:\n\na. Introducing a learned soft absorbing state that helps the continuous diffusion model converge faster during training and eliminates the need for MBR sampling to ensure quality.\n\nb. Adapting  DPM-solver++ to expedite the sampling process of the enhanced diffusion text generation model.\n\nThe authors add discrete noise by randomly replacing tokens in the sequence with the soft absorbing state, in addition to the continuous Gaussian noise. This helps bridge the gap between training and inference. For sampling, DPM-solver++ allows accurate reconstruction using very few steps.\n\nThe method outperforms some baselines like Multinomial diffusion and DiffuSeq.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "i1kbkhpbKN", "reviewer": "Reviewer_6MXf", "summary": "", "questions": "- Does MBR further improve your method if the goal is performance? Since samples can be generated in parallel, I think this is important to touch upon.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Well motivated. Hybrid diffusion models that combine the advantages of progress on continuous and discrete diffusion models are definitely worth exploring. The only paper on this I am aware of is so far unpublished (A Cheaper and Better Diffusion Language Model with Soft-Masked Noise https://arxiv.org/abs/2304.04746)\n\n- Experiments seem to support the claims, and the effect of adding discrete masking on training and inference time, as well as performance, is adequately investigated (with caveats, see limitations section).", "reasons_to_reject": "- The abstract focuses on comparisions to DiffuSeq (e.g. samples of similar quality 800x faster), which is deceptive. Discrete diffusion models are much faster, and almost as performant, as their results show. Their method is really a hybrid of the continuous DiffuSeq and the discrete D3PM, and should be discussed as such.\n\n- Related, why DiffuSeq cannot use DPM-solver++ is not explicit enough. I assume that it is exactly because of incompatability with clamping (as Figure 3 suggests), but this should be clarified.\n\n- In general, the paper is not in great shape, and was clearly prepared in a rush. Notation and acronyms are often not introduced, central ideas like \"clamping\" are not adequately summarized, and claims in the abstract (e.g. 4x faster training vs. 1.75) don't match those in the paper. Overall however, I could understand most of the paper adequately.\n\n- Details around the experiments conducted are incomplete. For example, there is not enough information about the diffusion models (e.g. number of parameters etc.) to even judge if the comparisons are fair. Obviously this must be addressed before the paper can be accepted.\n\n- All results are on a single dataset  (QQP). New DM approaches are generally evaluated on several tasks (but this is a short paper submission).", "questions_to_authors": "- Does MBR further improve your method if the goal is performance? Since samples can be generated in parallel, I think this is important to touch upon.", "paper_topic_and_main_contributions": "The authors present a hybrid continuous/discrete diffusion model for text generation, which incorporates both the Gaussian noise of continuous diffusion models such as DiffuSeq, and the discrete word masking noise of discrete diffusion models such as D3PM. \n\nThe hybrid model is motivated by the desire to leverage the speed of the recently introduced DPM-solver++ for continuous diffusion models, and the goal of matching training to testing conditions by removing the need to \"clamp\" intermediate states to word embeddings at test time, which continuous diffusion models currently do. \n\nResults on QQP demonstrate speedups over discrete diffusion models (DP3M and RDM) of about 2x, at similar performance levels.", "excitement": 3, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "- A Cheaper and Better Diffusion Language Model with Soft-Masked Noise https://arxiv.org/abs/2304.04746 (contemporaneous work )"}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- Figure 4 (lambda in figure, gamma in caption and text)\n- clamping: describe operation(in footnote maybe?)\n- w^y notation was not introduced\n- \"circle-plus\" operator not introduced\n- \"training and inference stages aligned better'-> training and inference stages better aligned\n- \"these two kinds of noise are posed partially in 152 the manner of DiffuSeq.\" -> What does this mean?\n- \"loss function is almost the same with DiffuSeq:\" -> complete the though, \"except for ...\"\n- \"applying n times of integration-by-parts\"-> applying integration-by-parts n times"}}]

Author rebuttal:

Meta-review: The paper proposes a modification for continuous diffusion models by introducing a soft absorbing state, which enables removing the clamping operation in models like diffusion-LM and DiffuSeq. This modification enabled the utilization for DPM-solver++ for speeding up continuous diffusion models applied in text. Experiments are provided for speed comparison to show significant speed-ups compared to other models based on continuous diffusion. The resulting speed-up in roughly equivalent to the speed of discrete diffusion models. The paper requires writing and presentation improvements to clarify some of the points raised by the reviewers.
Decision: Accept-Findings