Dior- CVAE : Pre-trained Language Models and Diffusion Priors for Variational Dialog Generation (2023)

Abstract: Current variational dialog models have employed pre-trained language models (PLMs) to parameterize the likelihood and posterior distributions. However, the Gaussian assumption made on the prior distribution is incompatible with these distributions, thus restricting the diversity of generated responses. These models also suffer from posterior collapse, i.e., the decoder tends to ignore latent variables and directly access information captured in the encoder through the cross-attention mechanism. In this work, we propose Dior-CVAE, a hierarchical conditional variational autoencoder (CVAE) with diffusion priors to address these challenges. We employ a diffusion model to increase the complexity of the prior distribution and its compatibility with the distributions produced by a PLM. Also, we propose memory dropout to the cross-attention mechanism, which actively encourages the use of latent variables for response generation. Overall, experiments across two commonly used open-domain dialog datasets show that our method can generate more diverse responses without large-scale dialog pre-training. Code is available at https://github.com/UKPLab/dior-cvae.

## introduction due to the open nature of open-domain dialogs, i.e., their diverse topics and the lack of specific goals, a dialog context can be followed by multiple responses, presenting a one-to-many complex relationship @xcite . this relationship usually poses a significant challenge to sequenceto-sequence dialog generation models that are inherently deterministic, i.e., can not produce different responses given the same dialog. although different decoding strategies such as nucleus sampling @xcite have been introduced di usion priors it' s rainy, let' s go to eat hotpot! guassian prior okay! okay, let's go! sure, let's go! no, i want to lose weight where do you want to eat hot-pot of course, i love hot-pot. to bring stochasticity, these strategies mostly perform on the token level and thus might harm the fluency of the generated responses. conditional variational autoencoders (cvaes) @xcite have been used to bring diversity @xcite @xcite @xcite @xcite . cvaes draw latent variables from an assumed prior distribution conditioned on the dialog context and use these latent variables to guide the generative process. these latent variables often capture potential dialog topics, implicit conversational intents, or different styles of responses @xcite . one main challenge occurs due to the simple prior distribution, which is assumed to be the isotropic gaussian distribution. the gaussian assumption is oversimplified compared to the complex relationship between a dialog context and its potential responses. the gaussian distribution is also incompatible with the expressive likelihood and posterior distributions, which are parameterized by pre-trained language models. the oversimplification and incompatibility consequently restrict the generated responses to a relatively small region of the latent space @xcite . in other words, the generated re-sponses could be different in textual form but not in topic or intent (fig. 1 ). several studies introduce more complex prior distributions by using a neural network (nn) to sample implicit latent representations @xcite , or by using normalizing flows @xcite . while diffusion models have been shown to provide better priors than gaussians and normalizing flows @xcite , they have not been used to parameterize the prior distribution for variational dialog generation. another major challenge of cvaes is the wellknown posterior collapse problem @xcite , especially when incorporating the plms based on the transformer encoder-decoder architecture @xcite . latent variables can be easily neglected by the expressive @xcite or bypassed by the cross-attention mechanism between the encoder and decoder @xcite . previous studies attempt to mitigate this problem by weakening the decoder @xcite or controlling the weight of the kullback-leibler (kl) divergence term @xcite . forcing the entanglement of latent variables in the decoding process has also been proposed to address the problem @xcite . different from these methods, several dropout methods have been proposed to address posterior collapse, without the need for additional training parameters @xcite @xcite . in this work, we propose dior-cvae, which employs a diffusion model to parameterize the prior distribution of a hierarchical cvae model. the diffusion model can provide a more expressive distribution than the typical isotropic gaussian @xcite . meanwhile, the proposed model uses a transformer-based encoder-decoder plm to compute the posterior and likelihood distributions and derive the hierarchical latent variables. to alleviate the posterior collapse problem in transformer-based cvaes, we introduce memory dropout into the cross-attention mechanism of the decoder, which strengthens the role of latent variables in dialog response generation. our method necessitates comparable parameters compared to prior studies and maintains comparable inference time despite of the additional diffusion model. extensive experiments on the dailydialog and personachat datasets show better performance of our model over existing response generation meth-ods without large-scale dialog pre-training. our human evaluation further validates that the proposed model can generate more diverse responses with high quality. ## our method -dior-cvae we present dior-cvae, a hierarchical cvae model based on an encoder-decoder transformer, with four improvements (fig. 2 ). first, we enhance the computation of hierarchical latent variables with attention mechanism ( §3.1). these variables are then infused into the decoder via self-and cross-attention ( §3.2). we then introduce memory dropout during training to alleviate posterior collapse, a well-known problem in cvaes ( §3.3). most importantly, we parameterize the prior distribution using a diffusion model for more flexible and compatible representations than an isotropic gaussian ( §3.4). finally, we introduce the objective for the end-to-end training and describe the training and inference process ( §3.5). ## experiments this section gives a brief overview of our experimental settings. we refer to appendices a to c for a full set of hyperparameters, data statistics, and formal definitions of metrics, respectively. implementation. our model was developed using the opennmt-py library @xcite . we employed bart @xcite as the backbone plm, with the max sequence length set as 1024 and 50 diffusion steps. ## conclusion & future work we proposed dior-cvae, an approach for variational dialog generation, which incorporates a diffusion model to produce a more informative and expressive prior distribution. our method is based on a hierarchical conditional variational autoencoder (cvae), which derives latent variables from every encoder layer and fuses them into the corresponding decoder layers as hierarchical latent memory. a pre-trained language model, bart, is employed to estimate the posterior and likelihood distributions of the cvae. the proposed approach approximates the one-to-many complex relationship of dialog response generation, i.e., multiple potential responses given a dialog context. the approach does not require more parameters than previous work, and the inference time remains comparable regardless of the introduction of the diffusion model. we also propose memory dropout to alleviate the posterior collapse problem in training transformer-based cvaes. our experiments across two commonly used dialog datasets show that the proposed method can generate diverse responses without relying on largescale dialog pre-training. this work suggests the effectiveness of using a diffusion model to parameterize the prior distribution in transformer-based cvaes for dialog response generation. future work on diffusion models for text generation in general should be explored given their potential.