PVGRU : Generating Diverse and Relevant Dialogue Responses via Pseudo-Variational Mechanism (2023)

Abstract: We investigate response generation for multi-turn dialogue in generative chatbots. Existing generative modelsbased on RNNs (Recurrent Neural Networks) usually employ the last hidden state to summarize the history, which makesmodels unable to capture the subtle variability observed in different dialogues and cannot distinguish the differencesbetween dialogues that are similar in composition. In this paper, we propose Pseudo-Variational Gated Recurrent Unit (PVGRU). The key novelty of PVGRU is a recurrent summarizing variable thataggregates the accumulated distribution variations of subsequences. We train PVGRU without relying on posterior knowledge, thus avoiding the training-inference inconsistency problem. PVGRU can perceive subtle semantic variability through summarizing variables that are optimized by two objectives we employ for training: distribution consistency and reconstruction. In addition, we build a Pseudo-Variational Hierarchical Dialogue(PVHD) model based on PVGRU. Experimental results demonstrate that PVGRU can broadly improve the diversity andrelevance of responses on two benchmark datasets.

## introduction the structure of natural language discourse is complex and highly variable @xcite @xcite ; this is especially true for dialogue. as shown in figure 1 , examples (a) and (b) have the same dialogue history but they end with different responses: utterances u a 6 vs. u b 6 . on the other hand, two dialogues with semantically similar utterances may express quite different context meanings. because of this variability, there is no simple one-to-one mapping between dialogue context and response. the mapping can be one-to-many -as in figure 1 , i.e., different responses to the same dialogue context -as well as many-to-one, i.e., different context histories requiring the same response. we observe that the distribution of a dialogue context (e.g., n a 6 and n b 6 in the figure) is composed of the distribution of its utterances and the distribution of each utterance is composed of the distribution of its words. a good model of word level and utterance level variation is a key requirement for improving the quality of responses in dialogue. one line of research @xcite @xcite employs recurrent neural networks (rnns) to model dialogue context. however, standard rnns are not well suited for dialogue context variability @xcite . this is because the internal transition structure of rnns is deterministic. thus, rnns cannot effectively model randomness and variability in dialogue context @xcite . variational mechanism has been shown to be well suited for modeling variability -from both theoretical and practical perspectives @xcite . methods based on variational mechanism @xcite @xcite introduce latent variables into rnns to model one-to-many and many-to-one phenomena in dialogue. although these approaches achieve promising results, they still have defects. first, these methods face the dilemma that latent variables may vanish because of the posterior collapse issue @xcite @xcite . variational mechanism can work only when latent variables with intractable posterior distributions exist @xcite . second, the sampled latent variables may not correctly reflect the relationship between dialogue context and response due to the one-tomany and many-to-one phenomena observed in dialogue @xcite . third, posterior knowledge is employed in training while prior knowledge is used in inference; this causes an inconsistency problem between training and inference @xcite @xcite . vs a man is making a sandwich while sitting at a dresser he gets up and brings the sandwich to … can you see the guy ? yes, i see one man what is he doing ? sitting in chair with sandwich at first. 𝑢𝑢 1 : 𝑢𝑢 2 : 𝑢𝑢 3 : 𝑢𝑢 4 : 𝑢𝑢 5 : 𝑢𝑢 6 : … 𝑢𝑢 1 𝑎𝑎 𝑢𝑢 𝑖𝑖 𝑎𝑎 𝑢𝑢 5 𝑎𝑎 𝑢𝑢 6 𝑎𝑎 (a) dialogue a (b) dialogue b 𝒩𝒩 5 𝑎𝑎 a man is making a sandwich while sitting at a dresser he gets up and brings the sandwich to … can you see the guy ? yes, i see one man what is he doing ? he was sitting on a chair and applying jam on a bread. 𝑢𝑢 1 : 𝑢𝑢 2 : 𝑢𝑢 3 : 𝑢𝑢 4 : 𝑢𝑢 5 : 𝑢𝑢 6 : … 𝒩𝒩 6 𝑎𝑎 ∇ 6 𝑎𝑎 … 𝑢𝑢 1 𝑏𝑏 𝑢𝑢 𝑖𝑖 𝑏𝑏 𝑢𝑢 5 𝑏𝑏 𝑢𝑢 6 𝑏𝑏 𝒩𝒩 5 𝑏𝑏 𝒩𝒩 6 𝑏𝑏 ∇ 6 𝑏𝑏 figure 1: overview of dialogue variability. (a) and (b) represent two dialogues a and b from dstc7-avsd. the last utterance is response. n a t represents the distribution of dialogue a at time step t on utterance level, and n b t likewise. ▽ a 6 denotes the distribution variations caused by u a 6 and ▽ b 6 denotes the distribution variations caused by token u b 6 . to tackle these problems, we propose a pseudo-variational gated recurrent unit (pvgru) component based on pseudo-variational mechanism. pvgru introduces a recurrent summarizing variable into the gru. this summarizing variable can aggregate the accumulated distribution variations of subsequences. the methods based on pvgru can model the subtle semantic differences between different sequences. first, pseudovariational mechanism adopts the idea of latent variables but does not adopt posterior mechanism @xcite @xcite . therefore, pvgru does not suffer from the posterior collapse issue @xcite @xcite . second, we design consistency and reconstruction objectives to optimize the recurrent summarizing variable in pv-gru; this ensures that the recurrent variable can reflect the semantics of dialogue context on both the word level and the utterance level. the consistency objective makes the distribution of the incremental information consistent with the corresponding input at each time step. third, we guarantee the consistency between training and inference since we do not employ posterior knowledge when optimizing the summarizing variable. our proposed method avoids the problems caused by variational optimization and can model the diversity problem in dialogue. for instance in figure 1 , examples (a) and (b) have the same dialogue history but different responses. n a 6 and n b 6 can learn the distribution differences caused by u a 6 and u b 6 . simultaneously, semantic reconstruction can enhance the model's perception of semantic changes, which in turn can strengthen the distribution differences caused by semantic changes. although the example only shows diversity at the utterance level, similar diversity issues exist at the word level. therefore, we build a pseudo-variational hierarchical dialogue model (pvhd) based on pvgru to model both word level and utterance level variation. to summarize, we make the following contributions: pvhd significantly outperforms strong baselines with rnn and transformer architectures on two benchmark datasets. the code including baselines for comparison is available on github foot_0 . ## preliminary in this paper, we employ gru (gated recurrent unit) @xcite as the implementation of recurrent neural network (rnn). the reset gate r t is computed by: the hidden state h t at the time step t is then computed by: ## conclusion we analyze the reasons for one-to-many and manyto-one issues from high variability of dialogue. we build pvhd based on proposed pvgru component to model the word-level and utterance-level variation in dialogue for generating relevant and diverse responses. the results demonstrate that pvhd even outperforms pre-trained language models on diversity metrics.