D iffu S eq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated S eq2 S eq Diffusion Models (2023)

Abstract: Diffusion models have gained prominence in generating high-quality sequences of text. Nevertheless, current approaches predominantly represent discrete text within a continuous diffusion space, which incurs substantial computational overhead during training and results in slower sampling speeds. In this paper, we introduce a soft absorbing state that facilitates the diffusion model in learning to reconstruct discrete mutations based on the underlying Gaussian space, thereby enhancing its capacity to recover conditional signals. During the sampling phase, we employ state-of-the-art ODE solvers within the continuous space to expedite the sampling process. Comprehensive experimental evaluations reveal that our proposed method effectively accelerates the training convergence by 4x and generates samples of similar quality 800x faster, rendering it significantly closer to practical application.

## introduction after diffusion models gained significant attention in the vision domain @xcite , progress has been made in applying them to text generation tasks, including constrained text generation in diffusion-lm @xcite and sequenceto-sequence (seq2seq) text generation in dif-fuseq @xcite . numerous subsequent studies demonstrate that diffusion models have achieved results comparable to traditional autoregressive models and non-autoregressive models in tasks such as machine translation @xcite @xcite and summarization @xcite @xcite . however, most of these works suffer from slow convergence during training and slow generation speed, particularly considering that these approaches require 1 the code is released at https://github.com/ shark-nlp/diffuseq/tree/diffuseq-v2 . the minimum bayes risk (mbr) decoding strategy @xcite to enhance generation quality, resulting in a doubling of time consumption. in order to further narrow the gap between diffusion models and the prevailing autoregressive models, we aim to propose an accelerated version of diffuseq for both the training and sampling stages. for training, in addition to gpu acceleration techniques such as fp16, an improved training scheme can enable the model to better represent knowledge and learn data distribution more quickly @xcite . for sampling, a well-trained diffusion model can achieve similar quality within a single sampling and without the need for mbr decoding, thus saving generation time. furthermore, we can borrow the state-of-the-art ode sampler dpm-solver++ @xcite which is already applied in fast image generation. the progress in discrete text diffusion models @xcite exhibits its superiority in using fewer sampling steps, thus intuitively inspiring us to bridge the gap between continuous and discrete spaces. based on the bert @xcite and bart @xcite , as well as the absorbing state in d3pm @xcite , we propose incorporating an extra learned soft absorbing state and discretely adding it with gaussian noise to jointly denoise the noise from two sources. the processes are illustrated in figure 1 . specifically, after posing gaussian noise, we randomly replace the continuous vector of the sequence with the absorbing state. the ratio is set according to the time step. this approach bridges the gap between continuous and discrete diffusion processes and also makes the training and inference stages better aligned. it also facilitates the integration of the dpm-solver++ and reduces the number of required sampling steps. in summary, our contributions are: ## methods in diffuseq, it formulates seq2seq tasks as conditional generation and learns p(w x |w y ), where w x and w y refer to the input and target sequence separately. we follow the notation, concatenating two sequences as z @xmath0, where x t and y t represent parts of z t that belong to w x and w y , respectively. to accelerate continuous diffusion models as well as leverage the absorbing state of discrete diffusion models, we add learnable soft absorbing state into diffuseq. we first combine the continuous gaussian noise and discrete absorbing noise, and then jointly denoise them. the detailed derivations can be found in appendix a. ## experiments we try to validate two main research questions. rq1: can the soft absorbing state added in continuous diffusion models improve the generation quality and boost the training convergence? rq2: to what extent does the dpm ode solver improve sampling speed and affect the performance? ## conclusions in this work, we present a simple but effective training scheme for joint discrete and continuous text diffusion models, which additionally resets some tokens into the soft absorbing state. the discrete noise bridges the training and sampling stages, saving time consumption of these two stages and the plugged dpm-solver++ further makes the sampling faster. our method is orthogonal to many other techniques such as self-conditioning @xcite , choosing tokens with different importance @xcite , which can also further enhance generation quality. our method is fundamental to diffusion text generation and can be applied beyond diffuseq @xcite . and for brevity, we moit the coefficient of z t and z 0 as constants. we can use the variational lower bound to optimize the negative log-likelihood e[log p θ (x 0 )] ≤ l vlb . the objective can be further rewritten to be a combination of several kl-divergence. for 1 ≤ @xmath1, we compute the parameterization of l t by substituting eq. ( 11 ) to minimize the difference from µ t and µ θ following @xcite : we need to approximate e λ f θ dλ, which can be analytically computed by repeatedly applying n times of integration-by-parts. according to the second order multistep dpm-solver++ algorithm, the reconstruct of z 0 relies on the f θ , after posing discrete denoise in our methods, the algorithm still applies since our f θ (ẑ λ , λ) is exactly aligned with training objectives.