Dior-CVAE: Pre-trained Language Models and Diffusion Priors for Variational Dialog Generation
Authors: Tianyu Yang
Thy Thy Tran
Iryna Gurevych
Venue: EMNLP 2023 Findings (2023)
Keywords: Dialogue Generation
Diffusion Model
Variational Auto-encoder

Abstract: Current variational dialog models have employed pre-trained language models (PLMs) to parameterize the likelihood and posterior distributions. However, the Gaussian assumption made on the prior distribution is incompatible with these distributions, thus restricting the diversity of generated responses. These models also suffer from posterior collapse, i.e., the decoder tends to ignore latent variables and directly access information captured in the encoder through the cross-attention mechanism. In this work, we propose Dior-CVAE, a hierarchical conditional variational autoencoder (CVAE) with diffusion priors to address these challenges. We employ a diffusion model to increase the complexity of the prior distribution and its compatibility with the distributions produced by a PLM. Also, we propose memory dropout to the cross-attention mechanism, which actively encourages the use of latent variables for response generation. Overall, experiments across two commonly used open-domain dialog datasets show that our method can generate more diverse responses without large-scale dialog pre-training. Code is available at https://github.com/UKPLab/dior-cvae.

Dior-CVAE: Pre-trained Language Models and Diffusion Priors for Variational Dialog Generation
Tianyu Yang and Thy Thy Tran and Iryna Gurevych Ubiquitous Knowledge Processing Lab (UKP Lab),
Department of Computer Science and Hessian Center for Al (hessian.Al), Technical University of Darmstadt www.ukp. tu--darmstade wat.t.tatt... de
Abstract
ronny, lets go hotpoti
Current variational dialog models have ployed pre-trained language models (PLMs) to parameterize the likelihood and posterior distributions. However, the Gaussian assump- tion made on the prior distribution is incom- patible with these distributions, thus restricting the diversity of generated responses. These models also suffer from posterior collapse, i.e., the decoder tends to ignore latent variables and directly access information captured in the en- coder through the cross-attention mechanism.
Okay lift
Nk: Fount
de not hahpot
love of por.
OlMtsioon Priora
Figure 1: Limitation of the isotropic Gaussian prior distribution, i.c., generating multiple responses with similar meaning in different text forms, compared with responses having more specific and diverse meaning offered by diffusion model.
this work, we propose Dior-CVAE, hi- erarchical conditional variational autoencoder (CVAE) with diffusion priors to address these challenges. We employ diffusion model to increase the complexity of the prior distribu- tion and its compatibility with the distributions produced by PL... Also, propose mem- ory dropout to the cross-attentin mechanism, which actively encourages the use of latent vari- ables for response generation. Our method re- quires parameters that are comparable to those of previous studies while maintaining compa- rable inference time, despite the integration of the diffusion model. Overall, experiments across two commonly used open-domain dia- log datasets show that our method can gener- ate more diverse responses even without large- scale dialog pre-training. Code is available https://ght http://ggtttu.b/.. WWUUUU//LLLo----./ cvae.
to bring stochasticity, these strategies mostly per- form on the token level and thus might harm the fluency of the generated responses.
autoencoders (CVAEs) (Sohn et al., 2015) have been used to bring diversity (Zhao et al., 2017: Shen et al.. 2017; Serban et al., 2017a; Chen et al., 2018, 2022; Sun al., 2021, 2023). CVAEs draw latent variables from an assumed prior distribution conditioned on the dialog context and use these latent variables to guide the generative process. These latent variables often capture potential dialog topics, implicit conversational intents, or different styles of responses (Zhao et al., 2017).
Conditional variational
One main challenge occurs due to the simple prior distribution, which is assumed to be the isotropic Gaussian distribution. The Gaussian as- sumption is oversimplified compared to the com- plex relationship between a dialog context and its potential responses. The Gaussian distribution is also incompatible with the expressive likelihood and posterior distributions, which are parameter- ized by pre-trained language models. The over- simplification and incompatibility consequently re- strict the generated responses to a relatively small region of the latent space (Chen et al., 2019; Gu et al., 2019). In other words, the generated re-
1 Introduction
Due to the open nature of open-domain dialogs, i.e., their diverse topics and the lack of specific goals, a dialog context can be followed by multi- ple responses, presenting one-to-mary complex relationship (Csáky et al., 2019). This relationship usually poses significant challenge to sequence- to-sequence dialog generation models that are in- herently deterministic, i.e., can not produce dif- ferent responses given the same dialog. Although different decoding strategies such as nucleus sam- pling (Holtzman et al., 2020) have been introduced

---

sponses could be different in textual form but not in topic or intent (Fig. 1). Several studies introduce more complex prior distributions by using a neural network (NN) to sample implicit latent representa- tions (Fang et al., 2019), or by using normalizing flows (Luo and Chien, 2021). While diffusion mod- els have been shown to provide better priors than Gaussians and normalizing flows (Vahdat et al., 2021), they have not been used to parameterize the prior distribution for variational dialog generation.
ods without large-scale dialog pre-training. Our human evaluation further validates that the pro- posed model can generate more diverse responses with high quality.
Problem Statement and Background
2.1 Dialog Response Generation
Dialog response generation (DRG) aims at gener- ating response given dialog context. The dia- log context consists of sequence of N tokens [c] which is the concatenation of the history utterances separated by a special token (
). The response r consists of K tokens, [r] K
Another major challenge of CVAEs is the well- known posterior collapse problem (Bowman et al.. 2016), especially when incorporating the PLMs based on the Transformer encoder-decoder archi- tecture (Vaswani et al., 2017). Latent variables can be casily neglected by the expressive de- coder (Bowman et al., 2016) or bypassed by the cross-attention mechanism between the encoder and decoder (Bahuleyan et al., 2018). Previous studies attempt mitigate this problem by weak- ening the decoder (Semeniuta et al., 2017; Zhao et al., 2017) or controlling the weight of the Kull- backeLeibler (KL) divergence term (Fu et al., 2019; Li et al.. 2019). Forcing the entanglement of la- tent variables in the decoding process has also been proposed to address the problem (Hu et al., 2022b). Different from these methods, several dropout methods have been proposed to address posterior collapse, without the need for additional training parameters (Srivastava et al., 2014; lyyer et al., 2015; Miladinovic et al., 2022).
2.2 Conditional Variational Autoencoders
Conditional Variational Autoencoders (CVAEs) learn a conditional generative model by introduc- ing the latent variables in the form of p(r, z/c)
Pep (z|c) po (r|2, c) where pz(z/c) is the prior distribution of the latent variable z given the dia- log context с and pe(r|z,c) is the likelihood
or decoder that generates a response r given latent variables and the dialog context c. Since the true posterior p(z|r, is intractable, the generative model is often trained with an approximated posterior distribution encoder
gs (z) c) To approximate more dynamic distri- butions, CVAEs often use neural networks (NNs) to parameterize the prior, posterior, and likelihood distributions by +/ and 0 respectively.
In this work, we propose Dior-CVAR which employs diffusion model to parameterize the prior distribution of a hierarchical CVAE model. The diffusion model can provide a more expres- sive distribution than the typical isotropic Gaus- sian (Ho et al., 2020). Meanwhile, the proposed model uses Transformer-based encoder-decoder PLM to compute the posterior and likelihood dis- tributions and derive the hierarchical latent vari- ables. To alleviate the posterior collapse problem in Transformer-basee CVAEs, we introduce memory dropout into the cross-attention mechanism of the decoder, which strengthens the role of latent vari- ables in dialog response generation. Our method necessitates comparable parameters compared to prior studies and maintains comparable inference time despite of the additional diffusion model. Extensive experiments on the Dai lyliaiiloog and PersonaChat datasets show better performance of our model over existing response generation meth-
Training. CVAEs are trained to maximize the Evidence Lower BOund (ELBO), i.e., minimize the upper bound of negative log- log-likelihood. The CVAR loss (Levae) consists of reconstruction loss (Eec) and the Kullback-Leibler divergence (CxL.). The reconstruction loss corresponds the cross entropy between the expected and generated response. The KI. divergence aligns the posterior distribution ( c) with the prior Pv (z)c)
LeVAE = Lac + LxL
E log Po (r(z,
(1)
+ KL 90 (z(
P=(z)c) ).
CVAES have shown great potential to improve the diversity of generated responses with the latent vari- ables z, which can represent the underlying factors such as topics, intentions, and styles associated with different responses (Zhao et al., 2017).
2

---

Decdder
Conteat Reprosenteerrs
Dacodee Layer Girputs
Attention
1-th Encodor Layer
Encoder Layer
Diffusion
Reseporsn Reppmneenteeooo
During tolning
Attention Mocule
Heerarchical Latent Variables
Condhoon Deneleing
During
Py ()
MLP
During training and genemition
Encodir Leyer
Reprisdentation
holding
Latent Variable
MLP
Dropped repossertationn
Figure 2: Our Dior-CVAE model architecture.
2.3 Diffusion Models
lines 16-17 17 of Alg. I, and get new high-quality data by sampling from it iteratively.
Given an observation of data Xo. different from CVAEs, diffusion models (Ho al., 2020) learn the data distribution p(xo) by reversing diffu- sion process. The diffusion (forward) process is a Markov chain that corrupts the sampled data Xo by gradually adding random noise to it:
3 Our Method Dior-CVAE
We present Dior-CVAE, a hierarchical CVAE model based on an encoder-decoder Transformer, with four improvements (Fïg. 2). First, we en- hance the computation of hierarchical latent vari- ables with attention mechanism ($3.1). These vari- ables are then infused into the decoder via self- and cross-attention ($3.2). We then introduce mem- ory dropout during training to alleviate posterior collapse, a well-known problem in CVAEs ($3.3). Most importantly, we parameterize the prior dis- tribution using diffusion model for more flexible and compatible representations than an isotropic Gaussian ($3.4). Finally, we introduce the objec- tive for the end-to-end training and describe the training and inference process ($3.5).
((x:|xx-1) (V(1 5, B,I)
(2)
where 91:r are the pre-defined noise variances, 8g (0, at time step t. When 3t T. the data distribution will be corrupted to (O, By defining a; I Bi), can directly get x, by adding noise to the input as follows:
((x xo) N(Vagxo (I-
where a E (0, 1).
Given access to the original data Xo. the forward process can be inverted analytically
pxx-1|Xx, x0) (x((((x,, xo), II
(4)
where dz can be derived from 3: (Ho et al., 2020), (x,,,, xg) has closed form (Ho et al., 2020) pa- rameterized by t. However, since the original data Xg is not available in the actual generation process, (i.e., the response is supposed to be generated), we can not directly use Eq. (4) to sample data. We thus approximate fe(.) using an NN with the pa- rameter sp, namely denoising network. The training objective of the denoising network can be defined as:
3.1 Hierarchical Latent Variables
Hierarchical CVAES (Sønderby al., 2016: Klushyn et al., 2019; Vahdat and Kautz, 2020: Child, 2021) increase the expressiveness of the ap- proximated prior and posterior by splitting the latent variables into I. groups {z , ,z').
The prior and approximated posterior of the latent variables z can be factorized as:
Peb (z|c)
II
1-1 Par (z 12
(6)
Ey Ey,xy,xe
207 1l (x(, xoll
(5)
4s (z(2)))))
1=1 9 (z (z'1z rr.).
(7)
where Uniform((1,-.. T)), x, (x)) For inference, we can use the trained denois- ing network (x,, t) to build the usable inversion Pip (x, (x,, ss p(x. |x,, (x,, referring to
where 3¹ 8' denote parameters of the 1-th layer. When / 1, P (zl (2 = pw [c). The same applies for qoa (z .. c) = qs (z' c).
The detailed building process of the posterior
3

---

(Bq. (7)) and the prior (Liq. (6)) distribution will be introduced in the following content and the $3.4, respectively.
prepend the latent variables the (encoder) mem- ory, which is passed to the cross-altention mecha- nism.
In this work, we employ an incoder-decoddr PLM with L encoder (Enc) and L decoder (Dec) layers to build the hierarchical CVAF. Each layer corresponds to group of latent variables. We de- note the hidden output by the 1-th encoder layer Hen (nC 1)) E R-Nxd We construct the mean and variance of the approximated poste- rior 26l (z ,r, r) c)
XAtt ((iinerrr))) H ()
(12)
Intuitively, the latent variables can serve ad- ditional memory for the decoder. The model fa- cilitates the next token generation with the deep infusion of latent variables. However, latent vari- ables and (encoder) memory theoretically contain overlapping information. The latent variables may be ignored during the generation process, causing posterior collapse. This drawback can be mitigated by the memory dropout introduced in $3.3.
follows:
<l
Moo log(o
FNN
enci
(8)
Enci
3.3 Memory Dropout
where FNN refers to a fully-connected feed forward NN with tanh as the activation function, |-] denotes the concatenation of the representations. The mean and variance can be used to sample latent variables z' using the re- ce-paramnterizatioo trick (Kingma et al., 2021) to enable to the FNN are computed as follows.
In this work, we propose memory dropout to ad- dress posterior collapse problem. The memory dropout aims at encouraging the use of the latent variables in the decoder. In particular, we apply random dropout to the hidden state: hfnes of the memory where i E [1, while keeping the latent variables. Subsequently, the input of the cross- attention in each decoder layer becomes:
The inputs
We aggregate information from the latent vari- ables of the lower layers to get the 2 in the Eq. (8):
XAtt XAtt' ([Linear(z)) memdrop(H(:"<>)) (13)
Linear(z (1
where memdrop(-) denotes the memory dropout operation with certain probability. Concretely, this is done by randomly masking out some hid- den states from the memory of the cross-attention mechanism. Compared with previous methods, our memory dropout does not introduce any additional trainable parameters (Miladinovic et al., 2022).
FNN(
(9)
Linear (z! 1)
where Linear denotes fully-connected NN with- out activation function.
We construct the representation c in the Eq. (8) from each encoder layer by attending over all outputs of that layer:
ec
Encr At EnC))
(10)
3.4 Diffusion Prior
where Att refers to the attention mechanism (Yang et al., 2016). We can obtain the representation efnc in the Eq. (8) following the same method.
We parameterize the prior distribution Pe (z)c) c) defined in the Eq. (6) with diffusion model to im- prove its complexity. The conditional information, dialog context c. can be introduced as an additional input for the denoising network, fyp (x+, t, c). During training, latent variables sampled from the approximated posterior distribution are used as the input data x == (Eq. (7)). The diffusion model is trained to imitate the posterior distribution. During inference, the denoising network conditioned on the dialog context is used to sample representations of latent variables.
3.2 Hierarchical Latent Variables Infusion
We prepend the latent variables as prefix embed- dings to the input of the self-attention mecha- nism in each decoder layer, following previous work (Chen et al., 2022):
SelfAtt SelfAte'([ineear) ([Linearzz)) H D)) (11)
The latent variables can then be iteratively in- fused to generate the subsequent tokens. Differ- ently, the cross attention takes the output of the final encoder layer HEnez memory. Cross- attention has shown its importance the Trans- former model, resulting in more quality degrada- tion when pruned (Voita et al., 2019). We thus
Specifically, condition on the latent variables in lower layers done in the inference of the poste- rior distribution ($3.1) and the dialog context c, we concatenate the latent variables sampled from the posterior distribution and conditional representa-
4

---

Algorithm 1 Dior-CVAE Inference
Algorithm 2 Training Dior-CVAE
Input Dialog context c, # timestep 7' noise schedule [o<]i
Input Dataset D. # timestep T. noise schedule {ax]i Model Denoising model (w)
sampling hyperparameter [ox]]
Model Denoising model ft(())
1: while not converged do
Output Response
(c. r) z
Sample data, init latent
H f
Embed the tokens
3:
for / 1,.... do
2: for 1,... L do
4:
H Enc (H
Di Context embed. P Eq. (10)
3: H. Enc.
En Att(H.. "o
frc through H. 4: Get e.
according Eq. (10)
6:
H Enc, (H,
Response embed. D Eq.. (10)
5: end for
7:
e, Ere (tt (
6: Get e, by concatenating (ec ! according Eq. (14).
8:
Get Hhg log(ord from Eq. (8)
7: zr I2 ~ ~NO,
Sample noise
9:
x ~ NNNWi Jua
8: for T..... do
9:
Z (1
fel 2,)) (,,)),
end for L t Uniform ,,... 73) N(yaz a)
10: if == then
Eq. (14)
11:
return end if
Sample timestep
12:
Sample time
13:
E Rd I(
Uniform( [0, 1))
15:
") then e.
14:
é Vi-a+
16:
15:
2:-1 V2 / 0:-1 ofe aie
17:
else
16:
end
18:
e. Enci ErCr.
Eq. (14)
17:
Split z into (2')(' CR',
19:
18:
Dec((a]] H
20:
L= Lac Laog-
+ Lzaa
Eq. (16)
Calculate gradients and update parameters
22:
end while
tions from all layers following the below equation:
z z
(14)
et al. (2021) to decompose the KL loss into its negative entropy (Cneg-zent) and cross-entropy (Lxent) losses. The reconstruction and the nega- tive entropy terms can be calculated using the re- parameterization trick (Kingma and Welling, 2014). The cross-entropy term can be further expressed with the regression loss (Greg) of the denoising diffusion model defined in Lgq. (5) (Vahdat et al.. 2021). The final loss of Dior-CVAE is follows:
e Enci
E
The sinusoidal position embeddings (Vaswani et al., 2017) are adopted to represent timestep to get the time embedding pe(,), which is first added to the conditional representation and then concate- nated with the noisy latent variables to form the input of the denoising network. Thus, the denoising network can be defined as:
[pe(1) Cc
f = Lac + LXL
(ep 1, 2+)= FLinerr
Zt
Lac + Leeg-zent + Lxent
(15)
Lac
Lneg-xent Lreg
The general diffusion model described in $2.3
can only model the unconditional distribution. To obtain conditional diffusion model, we follow the the classifier-free guidance paradigm (Ho and Salimans, 2021), where we train conditional and an unconditional diffusion model simultaneously by replacing the conditional representation e, by a zero vector with a probability 1) during training. During inference, the output interpolation of these two models with weight is used as the final prior representation, referring to line 9 of Alg. 1.
= E log po (r c)
(16)
+ log qsl (//r. c)
+ E
flllfp(ee (e (e,, (2,,) 2/1
2
Training (Alg. 2). The latent variables are sam- pled from the approximate posterior distribution of each layer where the parameters of the distribution are calculated through the layer-wise conditional representation and reference response representa- tion. In addition being fed into the decoder generate the target response, the latent variables are also used as the input data Xy the diffusion model to train the diffusion model to imitate the posterior distribution.
3.5 End-to-end Training
As mentioned in $2.2, the training objective of CVAEs consists of the reconstruction and the KL. divergence losses. To learn also the latent dif- fusion prior simultaneously, we follow Vahdat
5

---

Inference (Alg. 1). Specifically, to generate the response for a given dialog context, we first en- code the dialog context and get the conditional representation from each layer of the encoder. The representations are then concatenated as one of the inputs to the denoising network. Starting from the final step T, we first sample the latent variables from the standard Gaussian distribution. Then iteratively denoise the latent variables conditioned on the concatenated conditional representations us- ing the denoising network until step when we get the latent variables z. We split z into L parts, re- sulting in z'' .... zz and feed them into each layer of the decoder along with the memory to generate the response.
the generated response, including the relevancy, coherency, diversity, and understandability.
Analysis For diversity analysis of the generated responses, we sample 100 dialogs in the inter- section of Dai DailyDialog and Dai lyilyiilog* (Sai et al., 2020), which has multiple references for cach dialog, namely DailyDialog- 100.
Baselines. We compare Dior-CVAE with the state-of-the-art models for variational dialog gen- cration. Overall, the baselines are mostly the Transformer-bsee models pre-trained on the large- scale corpus. One of the critical differences be- tween these models is whether they are pre-trained on large-scale dialog dataset. More details about the baselines can be seen in the Appx. D.
4 Experiments
5 Experimental Results
This section gives a brief overview of our experi- mental settings. We refer to appendices A C for a full set of hyperparameters, data statistics, and formal definitions of metrics, respectively.
5.1 Main Results
Tab. presents the main evaluation results on the test sets of DailyDialog and Persona-chat. On DailyDialog, our model surpasses all baselines by a large margin for all metrics, while getting comparable performance as the models pre-trained on large-scale dialog data such as PLATO and Di- alog VED. The higher performance demonstrates the expressive representation capability of the dif- fusion priors in combination with the PLMs gen- erate high-quality responses. The proposed model can bring the generative distribution closer to the true dialog distribution. Regarding Persona-chat, once again, Dior-CVAE, with fewer parameters, mostly achieves better diversity than SoTA models. Compared to models with large-scale dialog pre- training, the results are inconclusive, with higher results than PLATO but slightly lower than Di- alogVED. The inconsistent results indicate the po- tential positive impact of dialog pre-training but investigation is required for further understanding.
Implementation. Our model was developed us- ing the OpenNM'T-py library (Klein et al., 2017). We employed BART (Lewis et al., 2020) as the backbone PLM, with the max sequence length set 1024 and 50 diffusion steps.
Datasets & Metrics. We trained and evaluated the proposed model on the DailyDialog (Li et al., 2017) and Persona-Chat (Zhang et al., 2018) datasets. DailyDialog is collection of English dialogs about daily life, while Persona- ad- ditionally includes personas of the speakers. We follow previous work in reporting the lexical sim- ilarity of the references and generated responses using BLEU-1/2 (Papineni et al., 2002) and the lex- ical diversity calculated by Distinct-1/2 1/2 (Li et al., 2016) which computes the ratio of distinct n-grams in the generated responses.
In addition, we employ Entropy-1/21 1/2/3 (Malashina, 2021) to measure meaningful in- formation in the generated responses. Since lexical-overlapping metrics have shown great limitations for text generation, we then employ bosel metrics for better evaluation, includ- ing BERTScore (BTS) (Sun et al., 2022) and FED (Mehri and Eskenazi, 2020). BERTScore (BTS) measures the semantic similarity of the reference and generated responses (Devlin al., 2019). We also employ FED (Mchri and Eskenazi, 2020) which measures 18 fine-grained qualities of
Further analysis. We additionally report Tab. 2 the n-gram Entropy scores and evalua- tion results of the model-based metrics. The En- tropy scores show that our proposed method can generate more diverse responses compared to the large-scale dialog pre-trained model Dialog BERTScore (BTS) focuses on semantic similar- ity brought by the cosine similarity between the contextual embeddings
reference and gen- crated response. We can see a similar trend BERTScore (BTS) on the DailyDialog dataset
6

---

Size
DailyDialog
PersonaChat
Model
(mil.)
BLEU-1
BLEU-2
Distinct-1
Distinct-2
BLEU-1
BLEU-2
Distinet-1
Distinet-2
withont dialog pre pre-truining
1VAE
3.9
30,9
24.9
2.9
25.0
38.2
27.7
0.9
8.2
LIC DELLA (BART)
117
40.5
32.0
1.9
11.3
209
47.3
40.5
28.9
35.4
1.7
9.8
Optimus
227
41.2
38.5
29.7
42.7
34.3
1.9
11.7
PropheiNet
332
44.3
39.2
3,9
21.1
46.6
39.1
1.3
7.5
DRESS
406
29.1
MVP4S
45.7
42.9
27.1
43.4
35.8
2.0
11.1
Dior- Dior-CVAE Www.CCAAEE-aapa Sampling (ours)
237
50.3
46.7
7.0
35,1
42.6
36.1
2.8
26.5
Dior-CVAE CVAE-bem (ours)
237
52.0
44.1
37.2
1.9
with large-scale diolog pre-training
PLATO
115
39.7
31.1
29.1
40.6
31.5
2.1
12.1
DialogVED VED-ssmpling DialogVED-bert
392
43.1
37.0
5.8
37.2
42.8
35.7
3.2
27.3
392
48.1
42.1
4.2
23.2
48.2
39.9
1.5
9.4
Table I: Performance on the DailyDialog and Persona-chat test sets in comparison with the state-of-the-arr Results of the previous methods denoted by are implemented and evaluated by us.
Diversity
Model Mouel--ussdd
Model
BI BLFU-1
BLRU-2
Distiner-1
Distinct-2
Model
Ent-1
Ent
Ent-3
BTS
FED
Dior-CVAE
$7.0
45.0
7.0
36.3
Dior-CVAE
3.56
4,83
5.07
87.57
5.36
distribution
Dialog
(with Goussian Prior)
w/o diffusion
49.3
41.1
4.7
22.1
Praining
Table 2:
Diversity and model-based evaluation the test set of DailyDialog. Ent-n indicates Entropy-n where n stands n-gram, BTS BERTS (Devlin et al., 2019), FBD (Xiang et al., 1021).
wio momory dropont
57.3
45.4
6.7
34.7
Architecture
wio self-attention infusion
wio attletioo infusion
55.2
43.7
6.8
35.4
wlo PLM
43.2
38.1
31.2
Table 3: Ablation results on the validation set of the DailyDialog dataset. w/o (without) denotes the re- moval of the corresponding component.
compared to that of the lexical- overlapping metrics BLEU-n. For both metrics, our method achieves higher scores than DialogVED. Also, the higher FED score by model indicates the higher qual- ity of generated responses on multiple dimensions, including the relevance, coherence, diversity, and understandability of the responses.
to diversity (Distinct-1/2) while slightly harming lexical coriiaap-verappping scores ( BLEU--12). While memory dropout is designed to promote diversity, generated responses then be diverse from the references, leading
slight decrease in lexical- overlapping scores. We further generate responses by sampling different latent variables to assess the effects of memory dropout, the analysis can be found in Appx. H. We also ablate the PLM to assess whether generic large-scale pre-training is useful for dialog generation. The great performance drop after removing PLM highlights the importance of pre-training. Interestingly, the diversity remains relatively high even without using PLM, which is greatly attributed to the diffusion model.
5.2 Ablation Study
We conduct an ablation analysis on the validation set of the DailyDialog validation set (Tab. 3). We compare Dior-CVAR with the following ablated variants: w/o diffusion: The diffusion model is removed and the prior distribution is assumed to be the isotropic Gaussian distribution; w/o memory dropout: memory dropout removed; w/o self- attention infusion: Eq. (11) is not computed; w/o cross-attention infusion: Eq. (12) is not computed; w/o PLM: random initialization of the Transformer encoder and decoder is used instead of BART.
5.3 Human Evaluation
We observe that the diffusion prior and latent variable infusion greatly contribute to both metrics that evaluate the coherence and the diversity of the generated responses. Unlike the above two components, memory dropout mainly contributes
Since automatic metrics for open-domain text gen- eration may be consistent with human percep- tions (Liu et al., 2016), we also conduct a human evaluation on the DailyDailog-100 subset with the help of three expert annotators. All annota-
7

---

Quality
10
Model
Diversity
COH
SAF
ENG
0.9
Human
1,908
1.862
2.000
1.947
4.831
24-
Coline Simlanty
Perplexty
Perplexity the Dior-CVAE
Dior-CVAE
DialogVED
1,534
1.215
1.601
1.378
1,993
1.983
1.693
1.433
1,845
1.500
22-
Cosine similarity
Perplexity the we diffusion
0.6
Table 4: Human evaluation on DailyDialog-100 subset.
tors have an NLP background. For each dialog, we sample five responses from Dior-CVAE and DialogVED. For quality, each annotator is asked to judge the quality with regards to the follow- ing four criteria: Coherent (COH), Informative (INF), Safety (SAF), and Engagement (ENG) 3-point Likert scale. We describe the criteria details in Appx. E. Furthermore, responses that do not violate any criteria as valid, i.e., only a maximum valid. For the diversity evaluation, annotators are asked to annotate the number of distinet meanings among the valid responses. Results of the human evaluation are reported in Tab. 4. Compared to Di- alogVED, our method not only generates higher quality but also more diverse responses.
Number of the Human Resoonse
Figure 3: Perplexity of two approaches and cosine sim- ilarity of human references on the DailyDialog-1 1 dataset. The decreasing trend of the cosine similarity curve demonstrates the diversity of the reference re- sponses, and two perplexity curves show the average perplexity of the methods, Dior-CVAE and its abla- tion without diffusion priors.
automatically mark
generated responses are
Model
B-1
B-2
D-1
D-2
BTS
FED
ICL
13.7
14.0
8.2
24.1
83.43
5.88
LoRA
39.6
35.4
7.1
41.1
85.98
5,91
Dior-CVAE
52.0
47.8
6.3
31.1
87.57
5.36
Table 5: LLM performance evaluated the test set of the DailyDialog dataset.
lions of parameters (Brown et al., 2020; Touvron et al., 2023; Chowdhery et al., 2023). Specifi- cally, we prompt the Llama-2 to generate responses given dialog history through in-context learning (ICL) (Brown et al., 2020) and instruction tun- ing (Mishra et al., 2022). A parameter-efficient fine-tuning method (LoRA) (Hu et al., 2022a) is used for the instruction tuning.
5.4 Perplexity of Multiple References
To further verify that our proposed model can gen- erate more diverse responses, we calculate the per- plexity of multiple different responses given the same context (Fig. 3). In particular, given dialog context, we sample one to five human references from DailyDialog- 100 (x-axis). We calculate the averaged perplexity of our method and its ablation without diffusion priors (i.e,, with Gaussian priors) on the sampled human references. We also com- pute the cosine similarity between every two refer- ence responses for the same dialog context using BERT (set as 1 when there is only one reference). From the cosine similarity shown by the blue that the human-labeled responses for the same dialog context are semantically differ- ent from each other. We can notice that the perplex- ity scores by the ablation without diffusion priors are significantly higher than our method. This indi- cates that diffusion models can better approximate multiple potential responses given dialog context which are semantically different.
The evaluation results shown in Tab. 5. Dior- CVAE performs better on the BLEU-1/2 metries while LLAMA-2 gets higher Dist-1/2 scores. This indicates that the responses generated by the LLM have the best lexical diversity. While BERTScore (BTS) focuses on semantic similarity between the reference and generated responses, our method also gets the best performance on this. The generated sponses by Dior-CVAE match better with the refer- ence responses. In contrast, LLAMA-2 gets higher FED scores, suggesting that the responses gener- ated by LLMs may have better quality on multiple criteria. This also emphasizes the functionality of large-scale pre-training for dialog systems.
curve, we
Related Work
5.5 Analysis with Llama-2
Variational dialog generation. Conditional vari- ational autoencoders (CVAE) (Kingma and Welling, 2014; Sohn et al., 2015) achieved impres- sive results to address the safe and commonplace
In this section, we compare our model with Llama- 2 (Touvron et al., 2023), one of the most recent SoTA Large Language Models (LLMs) with bil-
8

---

response problem in the dialogue generation task by representing dialog contexts in the latent space (Zhao et al., 2017; Shen et al., 2017; Serban et al.. 2017b; Chen et al., 2018). One limitation is the oversimplified Gaussian assumption of the prior and the posterior distributions. Several studies (Ser- ban et al., 2017a; Zhao et al.. 2018; Gao et al., 2019; Cai and Cai, 2022) introduce discrete latent variables to improve the complexity of these distri- butions. Further studies use more advanced genera- tive models like Generative Adversarial Network (Goodfellow et al., 2020; Gu et al., 2019; Khan et al., 2020) or Normalizing Flows (Rezende and Mohamed, 2015; Luo and Chien, 2021).
alleviate the posterior collapse problem in training Transformer- CVAEs.
Our experiments across two commonly used di- alog datasets show that the proposed method can generate diverse responses without relying large- scale dialog pre-training This work suggests the effectiveness of using diffusion model to parame- terize the prior distribution in Transformer-based CVAEs for dialog response generation. Future work on diffusion models for text generation general should be explored given their potential.
Limitations
One limitation of this work is the instability of the training process due to the high variance of the time step sampling operation in the diffusion model (Eq. (5)). An advanced pre-defined noise variances scheme for timestep sampling (Vahdat et al., 2021; Nichol and Dhariwal, 2021) will be explored address this problem. Future work also aims at understanding information captured in the latent space produced by diffusion models (Khrulkov et al., 2023), towards an interpretable latent space.
Diffusion models for text generation. Adapting diffusion models natural language remains an open challenge due to the inherently discrete nature of texts. These studies can be divided into discrete and continuous. Notable work (Austin et al., 2021; Hoogeboom et al., 2021, 2022) directly defines a forward diffusion process on discrete data. Other work has adapted diffusion models in the word em- alternative
bedding space and presented them auto-regressive L.Ms (Li et al., 2022; Gong et al., 2023; Strudel al., 2022). Differently, diffusion models have been studied in the latent space to com- plement existing PLMs (Liu et al., 2022; Lovelace et al., 2022; Yu et al., 2022), which condition on a pre-defined set of labels. In our approach, we investigate how to incorporate diffusion priors in variational dialog generation.
This work only considers BART, a Transformer encoder-decoder architecture, as our backbone PLM. However, many recent SoTA large language models (LLMs) are decoder-only architecture, fur- ther experiments are required to use these LLMs with diffusion priors.
Ethies Considerations
7 Conclusion & Future Work
This work is fine-tuned and evaluated on two pub- licly available datasets that widely used benchmarks for open-domain dialog generation. However, due to the use of a PLM, the fine-tuned models may inherit biases from large-scale pre- training. Safety filtering and/or debiasing methods should be considered while using the proposed ap- proach in real-world applications.
We proposed Dior-CVAE, an approach for varia- tional dialog generation, which incorporates dif- fusion model to produce more informative and expressive prior distribution. Our method is based hierarchical conditional variational autoen- coder (CVAE), which derives latent variables from every encoder layer and fuses them into the cor- responding decoder layers as hierarchical latent memory. A pre-trained language model, BART, is employed to estimate the posterior and likelihood distributions of the CVAE. The proposed approach approximates the one-to-many complex relation- ship of dialog response generation, i.c., multiple potential responses given dialog context. The approach does not require more parameters than previous work, and the inference time remains com- parable regardless of the introduction of the diffu- sion model. We also propose memory dropout to
Acknowledgement
This work has been funded by the European Union under the Horizon Europe grant No 101070351 (SERMAS) and by the German Federal Ministry of Education and Research (BMBF) under the pro- motional reference 13N15897 (MISRIK), We also thank our internal and anonymous reviewers for their constructive comments on this paper.
9

---

References
variational memory network for dialogue generation. In Proceedings of the 2018 World Wide Web Confer- ence World Wide Web, WWW 2018, Lyon, France, April 23-27, 2018, pages 1653-1662. ACM.
Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. 2021. Structured denoising diffusion models in discrete state-spaces. In Advances in Neural Information Processing Sys- tems 34: Annual Conference Neural Information Processing Systems 2021, NewrIPS 2021, December 6-14, 2021, virtual. pages 7981-17999.
Wei Chen, Yeyun Gong, Song Wang. Bolun Yao, Weizhen Qi, Zhongyu Wei, Xiaowu Hu, Bartuer Zhou, Yi Mao, Weizhu Chen, Biao Cheng, and Nan Duan. 2022. DialogVED> A pre-traines latent vari- able encoder-decoder model for dialog response gen- eration. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics Dublin,
Hareesh Bahuleyan, Lili Mou, Olga Vechtomova, and Pascal Poupart. 2018. Variational attention for sequeenceto-seqquence models. In Proceedings of the 27th International Conference on Computational Lin- guistics, pages (672-1682, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
(Volume 1: Long Papers), pages Ireland. Association for Computational Linguisties.
Rewon Child. 2021. Very deep vaes generalize autore- gressive models and can outperform them on images. In 9th International Conference on Learning Repre- sentations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.
Siqi Bao, Huang He, Fan Wang, Hua Wu, and Haifeng Wang. 2020. PLATO: Pre-trained dialogue genera- tion model with discrete latent variable. In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 85-96, Online. Association for Computational Linguistics.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh. Kensen Shi, Sasha Tsvyashchenko. Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vin- odkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengeheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito. David Luan, Hycontack Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, An- drew M. Dai. Thanumalayan Sankaranarayana Pil- lai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang. Brennan Sacta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Teck, Jeff Dean, Slav Petrov, and Noah Fiedel. 2023. Palm: Scaling language mod- eling with pathways. Journal of Machine Learning Research, 24(240):1-113,
Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, An- drew Dai, Rafal Jozefowicz, and Samy Bengio. 2016. Generating sentences from a continuous space. In Proceedings of the 20th SIGNLL Conference on Com- putational Natural Language Learning, pages 10-21, Berlin, Germany. Association for Computational Lin- guistics.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger. Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler. Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, lla Sutskever, and Dario Amodei. 2020. Language models are few-shor learners. In Ad- vances in Neural Information Processing Systems 33: Annual Conference on Neural Information Process- ing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual.
Richard Csáky, Patrik Purgai, and Gábor Reeski. 2019. Improving neural conversational models with entropy-based data filtering. In Proceedings of the 57th Annual Meeting of the Association for Computa- tional Linguistics, pages 5650-5669, Florence, Italy. Association for Computational Linguistics.
Zefeng Cai and Zerui Cai. 2022. Pevae: Gener- ating prior context for dialogue response genera- tion. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, pages 4065-4071. International Joint Conferences on Artificial Intelligence Organization. Main Track.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistic..
Chaotao Chen, Jinhua Peng, Fan Wang, Jun Xu, and Hua Wu. 2019. Generating multiple diverse re- sponses with multi-mapping and posterior mapping selection. In Proceedings of the Evven-EEigtth Inter- national Joint Conference on Artificial Intelligence. IJCAI 2019, Macao, China, August 10-16, 2019, pages 4918 4924. ijcai.org.
Hongshen Chen, Zhaochun Ren, Jiliang Tang, Yi- hong Eric Zhao, and Dawei Yin. 2018. Hierarchical
Angela Fan, Mike Lewis, and Yann Dauphin. 2018 Hierarchical neural story generation. In Proceedings
10

---

of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889-898, Melbourne, Australia, Association for Computational Linguistics.
934-950, Abu Dhabi, United Arab Emirates, Associ- ation for Computational Linguistics.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. De- noising diffusion probabilistic models. In Advances in Neural Information Processing Systems 33: An- nual Conference on Neural Information Processing Systems 2020, NewrlPS 2020, December 6-12, 2020, virtual.
Le Fang, Chunyuan Li, Jianfeng Gao, Wen Dong, and Changyou Chen. 2019. Implicit deep latent vari- able models for text generation. In Proceedings of the 2019 Conference on Empirical Methods in Natu- ral Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3946-3956, Hong Kong, China, Association for Computational Linguistics.
Jonathan Ho and Tim Salimans. 2021. Classifier-free diffusion guidance. In NeurlP'S 2021 Workshop on Deep Generative Models and Downstream Applica- rions.
Hao Fu. Chunyuan Li, Xiaodong Liu, Jianfeng Gao, Asli Celikyilmaz, and Lawrence Carin. 2019. Cycli- cal annealing schedule: A simple approach to mit- igating KI. vanishing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers). pages 240-250. Minneapolis, Minnesota. Association for Computational Linguistics.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case neural text degeneration. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.
Emiel Hoogeboom, Alexey A. Gritsenko, Jasmijn Bast- ings, Ben Poole, Rianne van den Berg, and Tim Sal- imans. 2022. Autoregressive diffusion models. In The Tenth International Conference on Learning Rep- resentations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net.
Jun Gao, Wei Bi, Xiaojiang Liu, Junhui Li, Guodong Zhou, and Shuming Shi. 2019. A discrete CVAE for response generation on short-text conversation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- guage Processing (EMNLP-IJCNLP), pages 1898- 1908, Hong Kong, China. Association for Computa- tional Linguistics.
Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Foréé, and Max Welling, 2021. Argmax flows and multinomial diffusion: Learning categorical dis- tributions. In Advances in Neural Information Pro- cessing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurlPS 2021, December 6-14, 2021, virtual, pages 12454-12465.
Sergey Golovanov, Rauf Kurbanov, Sergey Nikolenko, Kyryl Truskovskyi, Alexander Tselousov, and Thomas Wolf. 2019. Large-scall scale transfer learning for natural language generation. In Proceedings of the 57th Annual Meeting of the Association for Com- putational Linguistics, pages 6053-6055 Florence, Italy. Association for Computational Linguistics.
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang. and Weizhu Chen. 2022a. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu. and Lingpeng Kong. 2023. Diffuseq: Sequence sequence text generation with diffusion models. In The Eleventh International Conference on Learning Representations.
Jinyi Hu, Xiaoyuan Yi, Wenhao Li, Maosong Sun, and Xing Xie. 2022b. Fuse it more deeply! variational transformer with layer-wit wise latent variable inference for text generation. In Proceedingy of the 2022 Con- ference of the North American Chapter of the Asso- ciation for Computational Linguistics: Human Lan- guage Technologies, pages 697-716, Seattle, United States. Association for Computational Linguistics.
lan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020. Generative adversarial networks. Communications of the ACM, 194414:199114.
Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daumé III. 2015. Deep unordered compo- sition rivals syntactic methods for text classificatin. In Proceedings of the 53rd Annual Meeting of the As- sociation for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing ( (Volume 1: Long Papers), pages 1681 1- 1691, Beijing, China. Association for Computational Linguistics.
Xiaodong Gu, Kyunghyun Cho. Jung- Woo Ha, and Sunghun Kim. 2019. Dialogwae: Multimodal re- sponse generation with conditional wasserstein auto- encoder. In 7th International Conference on Learn- ing Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net.
Seungju Han, Beomsu Kim, and Buru Chang. 2022. Measuring and improving semantic diversity of di- alogue generation. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages
Kashif Khan, Gaurav Sahu, Vikash Balasubramanian. Lili Mou, and Olga Vechtomova. 2020. Adversarial

---

learning on the latent space for diverse dialog gener- ation. In Proceedings of the 28th International Con- ference Computational Linguistics, pages 5026- 5034, Barcelona. Spain (Online). International Com- mittee on Computational Linguistics.
Chunyuan Li, Xiang Gao, Yuan Li, Baolin Peng, Xiujun
Li, Yizhe Zhang, and Jianfeng Gao. 2020. Optimus: Organizing sentences via pre-trained modeling of latent space. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Process- ing (EMNLP), pages 4678-4699) Online. Association for Computational Linguistics.
Valentin Khrulkov, Gleb V. Ryzhakov, Andrei Chertkov, and Ivan V. Oseledets. 2023. Understanding DDPM latent codes through optimal transport. In The Eleventh International Conference on Learning Rep- resentations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net.
Jiwei Li, Michel Galley, Chris Brockeu, Jianfeng Gao, and Bill Dolan. 2016. A diversity jective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. pages I 10-119, 19, San Diego, California. Association for Computational Linguistics.
ob-
Diederik Kingma, Tim Salimans. Ben Poole, and Jonathan Ho. 2021. Variational diffusion models. Advances in Neural Information Processing Sys- tems 34: Annual Conference on Neural Information Processing Systems 2021, NewrlPS 2021, December 6-14, 2021, virtual, volume 34, pages 21696-21707,
Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto. 2022. Diffusion- Im improves controllable text generation. In Ad- vances in Neural Information Processing Systems 35: Annual Conference on Neural Information Process- ing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28. - December 9, 2022, volume 35, pages 4328-4343.
Diederik P. Kingma and Jimmy Ba. 2015. Adam: method for stochastic optimization. In 3rd Inter- national Conference on Learning Representations, ICLR 2015, San Diego CA. USA. May 7-9, 2015, Conference Track Proceedings.
Diederik P. Kingma and Max Welling. 2014. Auto- encoding variational bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff. AB, Canada, April 14-16, 2014, Conference Track Proceedings.
Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqjang Cao, and Shuzi Niu. 2017. DailyDialog: A manually labelled multi-tur dialogue dataset. In Proceedings of the Fighth International Joint Conference on Nat- ural Language Processing (Volume I: Long Papers), pages 986-995, Taipei, Taiwan. Asian Federation of Natural Language Processing.
Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senel- lart, and Alexander Rush. 2017. OpenNMT: Open- source toolkit for neural machine translation. In Pro- ceedings of ACI. 2017, System Demonstrations, pages 67-72, Vancouver, Canada. Association for Compu- tational Linguistics.
Chia- Wei Liu, Ryan Lowe, Iulian Serban, Mike Nose- worthy. Laurent Charlin, and Joelle Pineau. 2.16. How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. In Proceedings of the 2016 Conference on Empirical Methods in Natu- ral Language Processing, pages 2122-2132, Austin, Texas, Association for Computational Linguistics.
Alexej Klushyn, Nutan Chen, Richard Kurle, Botond Cseke, and Patrick van der Smagt. 2019. Learning hierarchical priors in vaes. In Advances in Neural Information Processing Systems 32: Annual Confer- ence on Neural Information Processing Systems 2019, NeurlPS 2019, December& 8- 8-14, 2019, Vancouver, BC, Canada, pages 2866-2875.
Guangyi Liu, Zeyu Feng, Yuan Gao, Zichao Yang, Xiao- dan Liang, Junwei Bao, Xiaodong He, Shuguang Cui, Zhen Li. and Zhiting Hu. 2022. Composable text controls in latent space with odes. ArXiv preprint, fhs/2208.00638..
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy. Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequeee for natural language generation, translation, and com- prehension. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 7871-7880, Online. Association for Computa- tional Linguistics.
pre-training
Justin Lovelace, Varsha Kishore, Chao Wan, Eliot Shekhtman, and Kilian Weinberger. 2022. Latent diffusion for language generation. ArXiv preprint. abs/2212.09462.
Tien-Ching Luo and Jen- jen-Tzung Chien. 2021. Varia- tional dialogue generation with normalizing flows In ICASSP 2021-202/ IEEE International Confer- ence on Acoustics, Speech and Signal Processing (ICASSP), pages 7778-7782. IEEE.
Bohan Li, Junxian He, Graham Neubig, Taylor Berg- Kirkpatrick, and Yiming Yang. 2019. A surprisingly effective fix for deep latent variable modeling of text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- pages 3603-
Anastasia Malashina. 2021. Entropy analysis of n- grams and estimation of the number of meaningful language texts. cyber security applications. In Pro- ceedings of the Entropy 2021: The Scientific Tool of the 2 21st Century, Basel, Switzerland. MDPI.
guage Processing (EMNLP-IJCNN 3614, Hong Kong, China. Association for Computa- tional Linguistics.
12

---

Shikib Mehri and Maxine Eskenazi. 2020. Unsuper- vised evaluation of interactive dialog with DiaLoGPT. In Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 225-235, 1st virtual meeting. Association for Computational Linguistics.
Language Processing, pages 627-637. Copenhagen. Denmark. Association for Computational Linguis- tics.
Julian Vlad Serban, Alexander G. Ororbia, Joelle Pineau, and Aaron Courville. 2017a. Piecewise latent vari- ables for neural variational text processing. In Pro- ceedings of the 2017 Conference on Empirical Meth- ods in Natural Language Processing, pages 422-432, Copenhagen, Denmark. Association for Computa- tional Linguistics.
Djordje Miladinovic, Kumar Shridhar, Kushal Jain, Max Paulus, Joachim M Buhmann, and Carl Allen. 2022. Learning to drop out: An adversarial approach to training sequence vaes. In Advances in Neural In- formation Processing Systems 35: Annual Confer- ence on Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28 December 9, 2022, volume 35, pages 9645-9659.
Tulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron C. Courville, and Yoshua Bengio. 2017b. A hierarchical latent vari- ahle encoder-decoder model for generating dialogues. In Proceedings of the Thirty-Firss AAAI Conference Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA, pages 3295-3301. AAAI Press.
Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2022. Cross-task generaliza- tion via natural language crowdsourcing instructions. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguisties (Volume Dublin, Ireland. Association for Computational Linguistics.
Xiaoyu Shen, Hui Su, Yanran Li, Wenjie Li, Shuzi Niu, Yang Zhao, Akiko Aizawa, and Guoping Long. 2017. A conditional variational framework for dialog gener- ation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Vol- ume 2: Short Papers), pages 504-509, Vancouver, Canada. Association for Computational Linguistics.
1: Long Papers). pages
Alexander Quinn Nichol and Prafulla Dhariwal. 2021. Improved denoising diffusion probabilistic models. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 8 8162-8171. PMLRR
Kihyuk Sohn, Honglak Lee, and Xinchen Yan. 2015. Learning structured output representation using deep conditional generative models. In Advances in Neu- ral Information Processing Systems 28: Annual Conference on Neural Information Processing Sys- tems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pages 3483-3491.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: method for automatic evalu- ation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Compu- lational Linguistics, pages 311-318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
Casper Kaae Sønderby. Tapani Raiko, Lars Maaløe. Søren Kaae Sønderby, and Ole Winther. 2016. Lad- der variational autoencoders. In Advances in Neural Information Processing Systems 29: Annual Con- ference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pages 3738 3746.
Weizhen Qi, Yu Yan, Yeyun Gong, Dayiheng Liu, Nan Duan, Jiusheng Chen, Ruofei Zhang, and Ming Zhou. 2020. ProphetNet: Predicting future n n-gram for Seruuencett-SSqueeeeeeeee training. In Findings of the Asxociation for Computational Linguistics: EMNLP 2020. pages 2401-2410, Online. Association for Computational Linguistics.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Re- search, 1116)::999-1955
Danilo Jimenez Rezende and Shakir Mohamed. 2015. Variational inference with normalizing flows. Pro- ceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, volume 37 of JMLR Workshop and Con- ference Proceedings, pages 1530-1536, JMLR.org.
Robin Strudel, Corentin Tallec, Florent Altché, Yilun Du, Yaroslav Ganin, Arthur Mensch, Will Grathwohl. Nikolay Savinov, Sander Dieleman, Laurent Sifre, et al. 2022. Self-conditioned embedding diffussin for text generation. ArXiv preprint, 0/b//2211.042366
Ananya B. Sai, Akash Kumar Mohankumar, Siddhartha Arora, and Mitesh M. Khapra. 2020. Improving di- alog evaluation with a multi-reference adversarial dataset and large scale pretraining. Transactions of the Association for Computational Linguistics, 8:8 10- 827.
Bin Sun. Shaoxiong Feng, Yiwei Li, Jiamou Liu, and Kan Li. 2021. Generating relevant and coherent dia- logue responses using self-separated conditional vari- ational AutoEncoders. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 11th International Joint Confer- ence on Natural Language Processing (Volume I: Long Papers), pages 5624-5631 Online, Association for Computational Linguistics.
Stanislau Semeniuta, Aliaksei Severyn, and Erhardt Barth. 2017. A hybrid convolutional variational au- toencoder for text generation. In Proceedings of the 2017 Conference on Empirical Methods in Natural
13

---

Bin Sun, Yitong Li. Fei Mi, Weichao Wang, Yiwei Li. and Kan Li. 2023. Towards diverse, relevant and coherent open-domain dialogue generation via hybrid latent variables. In Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence, IAAI 2023, Thirteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2023, Washington, DC, USA, February 7-14. 2023, pages 13600-13600 3608. AAAT Press.
systems with distribution distances. In Findings of the Association for Computational Linguistics: ACL- IJCNIP 2021, pages 2192-2199, Online. Association for Computational Linguistics.
Zichao Yang, Diyi Yang, Chris Dyer. Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hicrarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computa- tional Linguisties: Human Language Technologies. pages 1480-1489, San Diego, California. Associa- tion for Computational Linguistics.
Tianxiang Sun, Junliang He, Xipeng Qiu, and Xuan- jing Huang. 2022. BERTSccore is unfair: On social bias in language model-based metries for text gen- eration. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4726-3739, Abu Dhabi, United Arab Emirates. Association for Computational Linguisties.
Peiyu Yu, Sirui Xie, Xiaojian Ma. Baoxiong Jia, Bo Pang, Ruiqi Gao, Yixin Zhu, Song-Chur Zhu, and Ying Nian Wu. 2022. Latent diffusion energy-based model for interpretable text modelling. In Proceed- ings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 25702-251 257200. PMLR.
Tianyi Tang, Junyi Li, Wayne Xin Zhao, and Ji-Rong Wen. 2023. MVP: Multi-task supervised pre-training for natural language generation. In Findings of the Association for Computational Linguisties: ACL 2023, pages 8758-8794 Toronto, Canada. Associa- tion for Computational Linguisties.
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Per- sonalizing dialogue agents: have dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Lin- guistics (Volume I: Long Papers), pages 2204-2213, Melbourne, Australia. Association for Computational Linguisties.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almabairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
Tiancheng Zhao, Kyusong Lee, and Maxine Eskenazi. 2018. Unsupervised discrete sentence representation learning for interpretable neural dialog generation. In Proceedings of the 56th Annuul Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1098- 1107, Melbourne, Aus- tralia. Association for Computational Linguistics.
Arash Vahdat and Jan Kautz. 2020. Nvac: A deep hi- erarchical variational autoencoder. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Sys- tems 2020, NewrIPS 2020, December 6-12, 2020, virtual, volume 33, pages 19667-19679.
Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017. Learning discourse-level diversity for neural dialog models using conditional variational autocncoders. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 654-664, Vancouver, Canada. Association for Computational Linguisties.
Arash Vahdat, Karsten Kreis, and Jan Kautz. 2021. Score-based generative modeling in latent space. In Advances in Newral Information Processing Systems 34: Annual Conference on Neural Information Pro- cessing Systems 2021, NeurlPS 2021, December 6- 14. 2021, virtual, pages 11287-11302.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan N. Gomez, Lukasz Kaiser, and Ilaia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Pro- cessing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA. pages 5998-69999 6608.
A Hyperparameters
Dior-CVAE uses BART (Lewis et al., 2020) as the backbone PLM which consists of 6-layer encoder and a 6-yyeer decoder. The size of hidden state is 768. The max length of the input tokens is 1024 and the max length the generated response is set as 200. To compare with the baselines, the size of latent variable is set as 64.
Elena Voita, David Talbot. Fedor Moiseev, Rico Sen- nrich, and Ivan Titov. 2019. Analyzing multi-head self-attention: Specialized heads do the heavy lift- ing, the rest can be pruned. In Proceedings of the 57th Annual Meeting of the Association for Computa- tional Linguistics. pages 5797-5808 Florence, Italy. Association for Computational Linguistics.
We use Adam optimizer (Kingma and Ba, 2015) with a learning rate of e on the Daily-Dialog dataset and We train the model for 500,000 steps, which takes around 78 hours in total. The learning rate schedule is set according to the study from Vaswani et al..
-4 on the Persona-Chat dataset.
Jiannan Xiang, Yahui Liu, Deng Cai, Huayang Li. Defu Lian, and Lemao Liu. 2021. Assessing dialogue
14

---

Warmup steps of the optimization is set as 20,000 and 40,000 on the Diily-D and Persona-Chat, respectively. Besides, we utilize KL annealing tricks to mitigate the posterior problem, the weight KL term the ELBO increases to I in 20,000 steps linearly. In the computation of negative log loss, the label smoothing value is set as 0.1. We use the dynamic batching mechanism in the Open- NMT package. The batch size measured by tokens is 4096. After every 5,000 optimization steps, we evaluate the model on the validation set. After com- plcting the optimization, we select the checkpoint that obtained the best validation results to evaluate on the testing set and report the result. We conduct our experiments on one Nvidia Telsa V100 32G GPU.
C Automatic Evaluation Metrics
In this section, we present the formal definitions of all metrics used in this paper. The BLEU-n score is defined
BLEU-n BP exp
Σ wy logp,
(17)
cf
BP e-TT/O
if
where BP is the brevity penalty term, denotes the length of the candidate generated response and r denotes the effective reference corpus length, w; denotes the positive weights summing to one. Pi is the geometric average of the modified n-gram precisions, which is defined as Pn
For memdrop, we tried different dropout proba- bilities from the range [0.1, 0.2, 0.3, --. 1.0]. We finally used 0.7 for dropout as it gives the highest performance on the validation set. In the diffusion prior, the number of diffusion steps during infer- ence is set to 50. It was chosen from the range [50, 100, 150, 200] following the same standard as described above. We set the variance schedule to constants increasing linearly from 31 5-6 to 3M = 10-3 In the decoding process, for the beam search decoding method, the beam width is set as 5. For the sampling method, we set K as 50 and p as 0.9. All experiments
EcE ECE(Candidanss) En En-gramec Countcuip(n-rran
Ee'c [C'(Candidnees)
Σ m-ggram cC' Count(n-gramn
(18) where Count(n-grant) denotes the the number of re-raam occurrences in the generated response, Counteny (n-teeinnnngrrrn (n-gram) denotes the number of times the generated response's n--ram appears in the refer- ence.
Distinct-n score is calculated by countering the number of distinct unigrams and bigrams in gener- ated responses. The value is scaled by total number of generated tokens avoid favoring long sen- tences. Formally, it's defined as
only once due to resource constraints, with random seed set 1234.
Distinct-1 Countunique (1-gram)
Count( (I-grmm)
B Data statistics
Distinct-2 Countunique (anniuuaaie(2ggra
Count(2-gram)
Detailed dataset statistics can be seen in Tab. 6. where #Examples denotes the number of the dia- log data example in the dataset, # #Turns denotes the average number of turns, #Tokens means the aver- age number of tokens in the dialog context and the response, respectively. We pre-process the multi- turn dialog to many single-turn dialogs as input to the model following DiD (Chen et al., 2022), where the dialog history concatenated as a whole sequence with special token [SEP] func- tioning as a separation mark to separate different turns. We use two special tokens, madeupwor and madeupword02 from the BART model as the speaker indicator tokens. #Train, #Valid, and #'esst denotes the number of the single-turn dia- log in the training set, validation set and testing set each dataset.
where Countunigue (n-gram) denotes the number of unique n-gram in the sentence. this paper, we focus on Inter-Disting score, namely the distinct score of generated responses in the whole test set.
The concept of "Entropy-n"" commonly used in information theory quantify the average amount of information or uncertainty contained in se- quence symbols of length "n". measures the predictability or randomness of a sequence. For- mally, it is defined as
Entropy-n
Σ (o)((og((w))) WCB
where the set of all the kinds of n-gram subse- quence in a generated response. p(w) denotes the normalized frequency occurrence of the n-gram subsequence w.
15

---

Dataset
#Examples
#Turns
#Tokens
#Tran
#Valid
#Iest
DiilyDialog
76k
75.6/15.0
76052
7069
6740
Persona-Chi Chat
95.1/12.2
122499
14602
14056
Table Data statistics of the datasets used in this paper
For model-based metrics, we can directly get the evaluation result from the evaluation model. The FBD metric fine-grained evaluation metric that can provide evaluation scores for 17 aspects. We only take the aspects that are the most relevant to this paper, including Relevant, Correct, Coherent, Error Recovery, Consistent and Diverse, and calcu- late an average to get the final FBD score.
Model
Dior-CVAE
DialogVED
DELLA(BART)
#tok/s
123.95
131.34
128.28
Table 7: Number of generated tokens per second (#tok/s) measured on the test set of DailyDialog.
E Human Evaluation
This section introduces the questions correspond- ing to four criteria used in our human evaluation. Each criterion are rated with 3-point Likert scale.
D Model Comparisons
We compare Dior-CVAE with the state-of-the-art
models for variational dialog generation:
Coherence (COH): is the response relevant the dialog?
LIC (Golovanov et al., 2019): a PLM fine-tuned on the open-domain dialog datasets.
ProphetNet (Qi et al., 2020): a PDG pre-trained on predicting more than one future tokens.
Informativeness (INF): does the response pro- vide correct information?
DRESS (Han et al., 2022): PLM fine-tuned to produce balanced semantic distribution over the generated responses.
Safety (SAF): is the response safe to read?
Engagement (ENG): do you want to have conversation with this speaker?
We also prepare and evaluate these baselines:
iVAEMI (Fang al., 2019): an implicit VAE model based on LSTMs that uses NN to pro- duce the posterior distribution.
The first three criteria are turn-based evaluation and the last one evaluated dialog-level.
Optimus (Li et al., 2020): a pre-trained Trans- former VAE for text generation.
F Inference Speed
MVP+S (Tang et al., 2023): multi-task super- vised pre-trained model for text generation.
Although there are concerns about the low speed of inference about the diffusion model especially in the synthetic image generation task. And there are many studies trying to improve the speed of inference of the diffusion model. While in this paper, the inference speed of our model should not be major problem. This slow sampling nature of diffusion exists synthetic image generation, where the number of diffusion steps is often set to 1000 - 4000, and the dimension of the latent variables is relatively large (e.g., 128x128) In our method, the number of diffusion steps is set to 50 (< 1,000) and the dimension of the latent variable is set to 64. The inference speed evaluated by generated tokens per second can be seen in the Tab. We can see that the inference speed of Dior-CVAE doesn't 't drop significantly compared with the other two models.
DELLA (Hu et al., 2022b): the original model is GPT-2-based HCVAB; we reproduced the model for the two evaluation datasets and re- placed GPT-2 by BART for fair comparison.
Additionally, we include results of the models tak- ing advantage of lcrge dialog pre-training:
PLATO (Bao al., 2020): large-scale pre- trained DRG model that uses discrete latent variable to address the one-to-many problem.
Dialog (Chen et al., 2022): a Transformer VAE pre-train rre-trained on large-scale dialog data in order to improve DRG.
We use -sample to denote sampling from the top-k tokens with the top-p probabilities at each decoding step (Fan et al., 2018; Holtzman et al., 2020) and -beam to indicate beam search.
16

---

Context
A: I saw on the tv yesterday that there has been another earthquake iran B: Yes There have been few there recently. They say that this one was not big quake The Iranians are dealing with it their own They have pur- chased some special equipment to find people huried
Context
A: Excuse me are you Mrs. from England
Green
B: Yes. 1.lmaam................
I'm pleased meet you, Mrs. Green My name Myra - work Reijing Institute of Technology came here meet you.
Dior-CVAR
1. How long will it take to get there ?
A: Does newspaper say anything about casualties
2. How long will take get there This is our office block We have all administrative department here, Sales Accounts Personnel Market Research and so on
step=1
have died in the earthquake. The count
than people is up to 10 right now
Step=10
yes . But so
have died.
4. How many departments do you have ?
Step=20
yes But so far less than 20 people
have died.
How can you work there
Step=30
said people have died. The rest are in hospital.
DialogVED
Thank you for coming ms green.
2. Thank you very much
Step=40
right now less than 20 people have died But several are in hospital
Thank you very much
4. Thank you very much.
Stop=50
right now, , less than 20 people have died over 100 are hospital.
5. Yes. i been working here for 3 years.
Table 8: Genrated responses by Dior-CVAE and Di- alogVBr for the same dialog context.
Table 10: Genrated responses by Dior-CVAB and Di- alogVED for the same dialog context.
Dropout Rate
Cosine Similarity
model and calculate the average cosine similarity between the responses. Experimental results the test set of DailyDialog can be seen in the Tab. 9, An extremely high similarity means that all the five responses are almost the same no matter the how the sample latent variable changes. We can see that as the dropout rate increases, the similarity decreases, which can demonstrate the effect of the memory dropout method.
0.1
0.99
0.4
0.94
0.7
0.91
1.0
0.88
Table 9: Effect of the Memory Dropout method
Realization of the Diffusion Process
In this section, we show the effect of the latent variable at each diffusion step. Specifically, for
I Case Study
We show more generation examples by Dior-CVAE and DialogVED on the test set of Dai DailyDialog this section. The dialog context and generated re- sponses can be seen in Tab. 11. From the results, we can see that Dior-CVAE can generate more di- verse responses than the SoTA model Dialog VED.
model setting where the diffusion steps set to 50, we perform the denoising of 1, 20, 30, 40 and 50 steps, respectively and then input the denoised latent variable to the decoder to see the generation result. From the generated text we can see that as the denosing step increases, the generated response can become more relevant to the dialog context. One of the generation results can be seen in Tab. 8.
H Effect of the Memory Dropout
Our proposed memory dropout method is used to alleviate the problem of posterior collapse. The di- rect result of the posterior collapse is that the latent variable has effect the text generation process. To further verify the effect of the memory dropout, we sample from the prior distribution times for given context and then perform the decoding pro- cess subsequently. We then obtain the embedding 5 generated responses using the pretrained BERT
17

---

Context
A: Hello
B: Hello, Is that Stefan
A: I'm sorry. but I'm afraid you't ve dialed the wrong number. There is no Stefan
here
Dior-CVAE
Wrong number Are you sure ?
2. Sorry.
3. Nice meet you
Nice to meet you
5. Don't 't hold on sccond. I'm sure they' 've got it.
DiD
Oh, I sorry to bother you.
2. Oh m sorry to bother you.
3. Oh are you sure
4. Oh i sorry
you lot of trouble.
Oh m sorry to cause you trouble.
Context
A: Thank your for calling World Airline What can I for you
B: need to book plane ticket to London
A: Round-trip onc-way ?
Dior-CVAE
It's flight CA169,, which leaves Beijing at 8: 00 a. m. on January 20th.
2. It's a flight tomorrow and my name is turkey.
3. It's flight tomorrow and my name is turkey
It's flight CA169, which leaves Beijing at 8: a. m. on January 20th.
5. I'd like to book ticket.
DiD
Round trip, , please.
Round trip please.
3. Round trip please.
One way trip. please.
5. One way trip. please.
Context
A: What can help you with
B: May ask you something ?
A: What's your question
Dior-CVAE
don't 't know how to say it English.
2. I don't 't know how to say it in English
3. I don't know about the position.
don't 't know how to tell you.
5. I don't 't know how to tell you.
VHD
Can you
how many credits would enough bachelor degree ?
2. Can you tell me how many credits would be enough for bachelor degree ?
3. Can you tell me how many credits would be enough for bachelor degree? ?
Can you tell how many credits would be enough for bachelor degree ?
5. Can you tell me how many credits would be enough for bachelor" degree ?
Table 11: Case study of response generation on DailyDialog
18


Reviews:
[{"review_id": "PUvJVCuyDz", "reviewer": "Reviewer_xnUx", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thoughtful feedback. We are encouraged that they find that our motivation behind the research is clear and straightforward, the proposed method demonstrates measurable enhancements in overall performance compared to baseline models. \nWe address reviewer comments below.\n\n1. **“There is no explicit confirmation of the multimodality of the priors generated by diffusion models in Figure 1.”**\n    - Answer: Thanks for the suggested analysis! We will include the multimodal analysis in the revision. We conduct the analysis by sampling 5-10 data samples in the test set and sampling 500 points from the learned diffusion prior distribution given the same dialogue context of the data sample and visualizing them to verify whether the learned distribution is more complex. We are happy to also include more analysis if the reviewer has other concrete suggestions.\n\n2. **“The paper allocates only a limited portion of its main section to delineate the designs and procedures of the diffusion prior.”**\n\n    - **there is an absence of explanation concerning the dynamics governing the evolution of ​z and whether z​ is subject to change at each time step.**\n        - Answer: Section 2.4 presents the reverse diffusion process, where Eq. 5 is the key formula that controls how z changes at each time step. We will show the generated responses conditioned on z sampled from each of the step to demonstrate how the z changes in revision.  \n\n     - **comprehensive insight into the co-training process of diffusion latents with CVAE is imperative. an exposition on the parameter tuning process is warranted to enhance the readers' understanding of the intricate interplay.**\n        - Answer: We will visualize the changes of the loss function in the revision. We describe detailedly the training and inference process of our model in the Figure 3 and Figure 4 of the appendix F, which can offer an exposition on how we train the model and make inference in practice. \n\n\n3. **an essential facet to address would be a comparative analysis of inference speed against relevant baselines.**\nAnswer: In our experiments, the average inference speed of our model is 1.37s per batch, and DialogVED and DELLA(BART) are 1.31s per batch and 1.21s per batch, respectively (batchsize=4, see Appendix). We will put the analysis in the revision. The inference speed of our model should not be a major problem. This slow sampling nature of diffusion exists in synthetic image generation, where the number of diffusion steps is often set as 1000 – 4000, and the dimension of the latent variables is relatively large (e.g., 128x128). In our method, the number of diffusion steps is set as 50 (< 1,000) and the dimension of the latent variable is set as 64 (see Appendix).\n\n\n4. **“it is recommended to include a comparison of parameter sizes between the proposed model and the baseline models mentioned in Table 1.”**  \n    - Answer: The rightmost column of Table 1 shows exactly the parameter sizes of all models in comparison. We will recap this column in the caption and add the corresponding text for clarification.\n\n\n5. **While the paper references baselines like PLATO, it is pertinent to consider more recent alternatives such as Chen et al. 2023, as they could offer a more current benchmark for evaluation.**\n    - Answer: We respectfully disagree. Our baselines include the latest models from 2022: DRESS, MVP+S, DELLA and DialogVED.\nThe most important comparison is DialogVED, which is also considered in Chen et al. (2023). While we agree that the paper can be cited in the related work, we emphasize 3 main reasons for not including their work in the main comparison as follows. First, they focus more on the persona-consistency of the generated responses rather than diversity, which is the main focus of our work. Consequently, they reported results on different datasets compared to ours and the reported results on the DSTC7-AVSD dataset are mostly lower than the DialogVED (all BLEU scores are lower). Last, their method relies on finding a suitable additional NLI dataset w.r.t. each dialog dataset. \nWithout the NLI component and additional training objectives (bag-of-word prediction, dialogue discourse memory and language modeling), their model reduces to our reported BART ablation (Table 3, Row 4) but with only one latent variable. While our reported ablation is equipped with a series of latent variables, we will include the ablation model with only one latent variable in the revision.\n\n\n6. **Reproducibility**\n    - The experimental settings can be seen in Appendix. We also include the source code in the supplemental material in the submission for reproducibility.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The motivation behind the research is clear and straightforward. While there is no explicit confirmation of the multimodality of the priors generated by diffusion models in Figure 1, the proposed method demonstrates measurable enhancements in overall performance compared to baseline models.", "reasons_to_reject": "The primary contribution of this conference paper centers on the adoption of a diffusion model as the prior generator. However, my principal concern revolves around the experimental validation of this proposition.\n\nThe clarity of the proposed method's contribution remains obscured. The paper allocates only a limited portion of its main section to delineate the designs and procedures of the diffusion prior. Consequently, the holistic integration of the diffusion process with CVAE training lacks elucidation. Specifically, the utilization of the sampled posterior $q(z|r,c)$ in Equation 2 as the original data $x_0$ for diffusion is addressed, yet there is an absence of explanation concerning the dynamics governing the evolution of $z$ and whether $z$ is subject to change at each time step. Given the potential instability of training both diffusion models and VAE, a comprehensive insight into the co-training process of diffusion latents with CVAE is imperative. Additionally, an exposition on the parameter tuning process is warranted to enhance the readers' understanding of the intricate interplay.\n\nConsidering the inherently slow sampling nature of diffusion, an essential facet to address would be a comparative analysis of inference speed against relevant baselines. Furthermore, it is recommended to include a comparison of parameter sizes between the proposed model and the baseline models mentioned in Table 1. While the paper references baselines like PLATO, it is pertinent to consider more recent alternatives such as [1], as they could offer a more current benchmark for evaluation.\n\n[1] Chen, Ruijun, Jin Wang, Liang-Chih Yu, and Xuejie Zhang. \"Learning to Memorize Entailment and Discourse Relations for Persona-Consistent Dialogues.\" arXiv preprint arXiv:2301.04871 (2023).\n\n=============\n\nI have carefully read the author's response. However, most of my concerns are not well addressed. The paper's contribution is not clearly explained, with only a limited section dedicated to describing the diffusion prior. I suggest the authors revise and resubmit.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper integrates Gaussian diffusion into a dialogue generation framework, specifically the CVAE model, to tackle the issue of isotropic Gaussian priors in VAEs that can result in posterior collapse. The method also incorporates dropout on memory latents to mitigate the collapse problem. The outcomes of this approach demonstrate competitive results on open-domain dialogue generation datasets.", "excitement": 2, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "U6Ia2S2Tk6", "reviewer": "Reviewer_UuVZ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive feedback and insightful comments. We are really encouraged that the reviewer found that our work proposes a novel approach to address posterior collapse and the simplified Gaussian assumption for variational response generation models. \nWe address the concerns raised by the reviewer below.\n  \n\n1. **“The main limitation of this paper is outdated method to evaluate dialog systems in the era of ChatGPT and the missing reference to evaluation proposed by Gupta et al., (2019)”**\n    - Answer:Answer: We partially disagree with the reviewer. Although several metrics based on pre-trained models have been proposed, BLEU and Distinct-½ metrics (used in our evaluation) remain to be the must-have metrics for the evaluation of response generation, even in ACL2023 [1,2,3].\nWhile Gupta et al., (2019) emphasized on a multi-reference test set, we note that a multi-reference dataset (DailyDialog++) is indeed used in our human evaluation (L520-524). Human evaluation is still the must-have evaluation for response generation. \nNevertheless, we will consider the recent published metrics in ACL 2023 such as LLM-EVAL (Lin & Chen, NLP4ConvAI 2023).\n\n2. **“whether the proposed method is able to actually generate fluent conversation with diverse human users.”**\n    - Answer: Reviewer’s suggestion is exactly what we want to show with our human evaluation, which focuses on evaluating the ability of our model for generating diverse responses conditioned on the same dialogue context.\n\n3. **“whether the learned diffusion prior is able to learn the complex prior distribution (as shown in Figure 1)”**\n    - Answer: Thanks for the suggested analysis! We will include the multimodal analysis in the revision. We conduct the analysis by sampling 5-10 data samples in the test set and sampling 500 points from the learned diffusion prior distribution given the same dialogue context of the data sample and visualizing them to verify whether our model learns a more complex distribution. We are happy to also include more analysis if the reviewer has other concrete suggestions. \n\n4. **“Can this proposed method be extended to larger-scale training or LLM.”**\n    - Answer: Our method can be extended to larger-scale training or LLM. Pretrained language model with encoder-decoder architecture can be easily used as a substitution of BART in our method. Besides, we can also adapt a pre-trained language model with decoder-only architecture like GPT-2 to the VAE model following Hu et al., (2022). \nHowever, fine-tuning LLMs requires a significant amount of computational resources, future work can investigate using parameter efficient finetuning (PEFT) methods while turning LLMs to VAEs with diffusion priors. Since the investigation of using PEFT is not in the scope of this work, we will leave it for future work.\n\n\n\n\n\n[1] [PVGRU: Generating Diverse and Relevant Dialogue Responses via Pseudo-Variational Mechanism](https://aclanthology.org/2023.acl-long.185) (Liu et al., ACL 2023)  \n[2] [Enhancing Personalized Dialogue Generation with Contrastive Latent Variables: Combining Sparse and Dense Persona](https://aclanthology.org/2023.acl-long.299) (Tang et al., ACL 2023)  \n[3] [Envisioning Future from the Past: Hierarchical Duality Learning for Multi-Turn Dialogue Generation](https://aclanthology.org/2023.acl-long.407) (Lv et al., ACL 2023)", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper proposes a novel approach to address the posterior collapse problem and the Gaussian assumption made on the prior distribution, which restricts the diversity of generated responses.\n2. The paper shows that the proposed model outperforms some response generation methods (like Dialog VED) without relying on large-scale dialog pre-training.\n3. The paper provides a comprehensive evaluation of the proposed model on two commonly-used open-domain dialog datasets.\n4. The paper proposes memory dropout to the cross-attention mechanism during training to encourage the use of latent variables for response generation.", "reasons_to_reject": "This paper has a sound idea to use diffusion model to model the complex prior. However, the main limitation of this paper is outdated method to evaluate dialog systems in the era of ChatGPT. It is reasonable to evaluate dialog response generation models using BLEU, Distinct-1/2 a few years ago. However, there are many research that have already shown limitations of these shallow metrics. \n\nThus, current experiment results is able to show the proposed DiorCVAE is able to outperform DialogVED, but not sufficient enough to show:\n1. the proposed method is able to actually generate fluent conversation with diverse human users.\n2. the learned diffusion prior is able to learn the complex prior distribution (as shown in Figure 1)\n3. Can this proposed method be extended to larger-scale training or LLM. ", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper proposes a hierarchical conditional variational autoencoder (CVAE) model with an informative prior produced by a diffusion model. The model is designed to address the posterior collapse problem and the Gaussian assumption made on the prior distribution, which restricts the diversity of generated responses. The paper derives a series of layer-wise latent variables from the encoder into the decoder as latent memory to alleviate the posterior collapse problem. The paper also proposes memory dropout to the cross-attention mechanism during training to encourage the use of latent variables for response generation. The prior distribution is parameterized with a diffusion model to increase the complexity and compatibility of the prior distribution. The paper shows that the proposed model outperforms existing response generation methods without relying on large-scale dialog pre-training.\nContribution:\n1. The paper proposes a hierarchical CVAE model with an informative prior produced by a diffusion model to address the posterior collapse problem and the Gaussian assumption made on the prior distribution, which restricts the diversity of generated responses.\n2. The paper derives a series of layer-wise latent variables from the encoder into the decoder as latent memory to alleviate the posterior collapse problem.\n3. The paper proposes memory dropout to the cross-attention mechanism during training to encourage the use of latent variables for response generation.\n4. The paper parameterizes the prior distribution with a diffusion model to increase the complexity and compatibility of the prior distribution.\n5. The paper shows that the proposed model outperforms existing response generation methods without relying on large-scale dialog pre-training.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "https://arxiv.org/abs/1907.10568 (Investigating evaluation of open-domain dialogue systems with human generated multiple references)\n"}, "extra_text": {}}, {"review_id": "nlwVnYKKDr", "reviewer": "Reviewer_YFb3", "summary": "", "questions": "1. Are there any differences between the diffusion process adopted in the work and previous work? Do you meet any problems when drawing the method into dialog generation?\n\n2. Could you give more explanations about memory dropout? What are the differences between Eq. 12 and Eq. 13? Do you dropout certain dimension of hidden vectors or directly mask some vectors directly? If you only dropout certain dimension of hidden vectors, is fig. 2 not accurate enough? If you mask some vectors directly, why does the decoder can generate fluent sentences when some of the hidden representations are completely lost? Why does memory dropout can tackle the posterior collapse problem? Are there any experiments which can demonstrate it directly? From Table 3, dropout can increase diversity by sacrificing fluency, but it does not seem to be the direct evidence to demonstrate that memory dropout can tackle the posterior collapse problem. Is it possible that the proposed model actually does not suffer from the posterior collapse problem? If it has the posterior collapse problem, why does the model can obtain close performance without memory dropout? Could you give more illustrations about it?\n", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thoughtful feedback. We are glad that you find that our motivation is clear and strong and the proposed method is effective and demonstrated in the experiments. \n\nWe address the comments below. \n\n**1. “Adopting diffusion process to model the latent space has been studied in previous work, and the authors directly draw the method into their models without any in-depth analyses.”**\n\n- Answer: Adopting the diffusion model to build the prior distribution of VAE-based response generation model is not simple drawing and concatenating. The oversimplified Gaussian distribution assumption made by the VAE model for response generation is the main motivation for many studies on improving VAE models  (L86-96 and 558-573). Prior studies which build the prior distribution of a VAE model using the diffusion model are mostly in computer vision (CV,  L98). We describe the differences between this work and the CV work in the following point.\n\n**2. “differences between the diffusion process adopted in the work and previous work?”**\n\n- Answer: There are 2 key differences. We will emphasize the differences in the revision.\n    - (1) We introduce a series of interdependent latent variables from the encoder and the prior distribution of each latent variable is provided by a diffusion model. In other words, there is a series of diffusion models (L183 – 190). In contrast, the previous works consider only one single diffusion model (L589-593). \n    - (2) Previous works mainly focus on visual two-dimensional data (L574 – 577), which then often uses a convolutional neural network as the denoising network. In our work, the diffusion model is in contrast designed to process vector samples of the latent variables.  So we only use a fully-connected neural network as the denoising network. \n\n\n**3. “Do you meet any problems when drawing the method into dialog generation?”**\n\n- Answer: As explained in L328-337, the widely-adopted pre-trained encoder-decoder Transformer model has the cross-attention module that causes the main challenge when integrating Transformer into VAE model. The latent variables inferred from the encoder and the deterministic encoder output have some overlapping encoded information. The cross-attention module can provide the decoder a direct connection with the encoder output, then it can access to the information encoded by the encoder directly and neglect the latent variables, which can cause the posterior collapse problem. Additionally, the pre-trained Transformer decoder model can predict next words solely conditioned on the already generated text, which can further explain the posterior collapse problem (Bowman et al, L61).\n\n**4. “The illustrations of the method is not clear to understand. The illustration of the memory dropout, one of the key proposed techniques in the paper, is not clear.”**\n\n- **What are the differences between Eq. 12 and Eq. 13?**\n\n    - Answer: Thanks for pointing this out! This is an error in the writing of equation 13, which should be $\\mathtt{XAtt}^l([\\mathtt{Linear}(\\mathbf{z}^t),\\mathtt{memdrop}(\\mathbf{H}_c^{\\texttt{Enc}_L})])$. We have fixed this in the paper. \n\n- **Explanations about memory dropout**\n\n    - Answer: In general, we randomly mask out the contextualized token embeddings, i.e. the output of the encoder. Our method is a variant of the “word dropout” method (L60-74), where some input words of an RNN decoder are replaced by the unknown word tokens to weaken the decoder. Different from RNN, Transformer decoder has direct access to the encoder’s output through the cross-attention. We thus mask out part of the encoder output to enforce the decoder model relying more on the latent variables as described in L328-332.\n\n- **Why does the model obtain close performance without memory dropout?**\n\n    - Answer: The posterior collapse mainly refers to the decrease of the effects of the latent variables in the generation process, i.e. the VAE model may degenerate to the general AE model (Bowman et al, L61). Different from without pre-trained models, posterior collapse in this case results in fine-tuning the pre-trained model (BART) for dialogue generation and ignoring the latent variables. This consequently does not result in a significant decrease in generation performance.\n    \n- **Why does memory dropout can tackle the posterior collapse problem?**\n\n    - Answer: Through masking out part of the encoder output, memory dropout then encourages the decoder to refer more to the latent variables rather than to rely on the encoder output through the cross-attention in BART.\n\n- **Demonstration of the posterior collapse without memory dropout**\n\n    - Answer: We trained and evaluated the ablation of our proposed model without the memory dropout module (Table 3 – row 3). The ablation generates the exact same response given the same context (using beam search) w.r.t. different sampled priors. The analysis will be included in the revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The motivation is clear and strong.\n\n2. The effectiveness of the proposed model is demonstrated in the experiment.", "reasons_to_reject": "1. Adopting diffusion process to model the latent space has been studied in previous work, and the authors directly draw the method into their models without any in-depth analyses.\n\n2. The illustrations of the method is not clear to understand. The illustration of the memory dropout, one of the key proposed techniques in the paper, is not clear. And it is difficult for me to judge whether it is a reasonable method to tackle the posterior collapse problem.\n", "questions_to_authors": "1. Are there any differences between the diffusion process adopted in the work and previous work? Do you meet any problems when drawing the method into dialog generation?\n\n2. Could you give more explanations about memory dropout? What are the differences between Eq. 12 and Eq. 13? Do you dropout certain dimension of hidden vectors or directly mask some vectors directly? If you only dropout certain dimension of hidden vectors, is fig. 2 not accurate enough? If you mask some vectors directly, why does the decoder can generate fluent sentences when some of the hidden representations are completely lost? Why does memory dropout can tackle the posterior collapse problem? Are there any experiments which can demonstrate it directly? From Table 3, dropout can increase diversity by sacrificing fluency, but it does not seem to be the direct evidence to demonstrate that memory dropout can tackle the posterior collapse problem. Is it possible that the proposed model actually does not suffer from the posterior collapse problem? If it has the posterior collapse problem, why does the model can obtain close performance without memory dropout? Could you give more illustrations about it?\n", "paper_topic_and_main_contributions": "This paper focuses on two problems in dialog generation: 1) ignoring the latent variables, which is also known as the posterior collapse problem; 2) low diversity in responses due to the limited expressive power of Gaussian distributions. To tackle these two problems, the authors firstly use the memory dropout to tackle the posterior collapse. Then, a diffusion process is adopted to obtain a more complicated prior distribution. The experimental results demonstrate the effectiveness of their proposed method.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "The illustrations about the proposed method (especially the proposed techniques about the main contributions) should be clearer. "}, "extra_text": {}}]

Author rebuttal:

Meta-review: Based on the reviews and updating scores provided, the paper titled "Gaussian Diffusion into a Dialogue Generation Framework” primarily showcases integration of Gaussian diffusion and dropout on memory latents within a dialogue generation framework, specifically the CVAE model, to alleviate the impact of isotropic Gaussian priors in VAEs and their resulting posterior collapse. This approach shows promising results on open-domain dialogue generation datasets.

## Pros:

- Proposes a novel approach to handle the posterior collapse problem and the limitation of Gaussian distribution on prior, which could enhance the diversity of generated responses.
- The proposed model shows potential competitive results against some existing generation methods without relying on large-scale dialog pre-training.
- There is a clear and strong motivation behind the research, with the results demonstrating effectiveness of the proposed model.

## Cons:

- One of the main criticisms revolves around the clarity and thoroughness of the proposed method's explanation. Detailed descriptions and insights into the co-training process of diffusion latents with CVAE are missing, blurring potential readers' understanding.
- Integration of the diffusion process with the CVAE training is not clearly explained.
- Information is lacking on the dynamics governing the diffusion process and the parameter tuning process.
- There is a lack of comparative analysis on inference speed and parameter size against baselines.
- Experimental validation of the paper's claims could be more robust, with the inclusing of more recent baseline models for evaluation.
- Evaluation methods of the system are deemed outdated and insufficient, and fail to show if the model can actually generate fluent, diverse dialogues.
- As per the third reviewer, more in-depth analysis on the influence of the diffusion process on the latent space is needed.
- There are detailed concerns about the memory dropout method and questions on its effectiveness in regards to the posterior collapse problem.
Decision: Accept-Findings