Speculative Sampling in Variational Autoencoders for Dialogue Response Generation (2021)

Abstract: Variational autoencoders have been studied as a promising approach to model one-to-many mappings from context to response in chat response generation. However, they often fail to learn proper mappings. One of the reasons for this failure is the discrepancy between a response and a latent variable sampled from an approximated distribution in training. Inappropriately sampled latent variables hinder models from constructing a modulated latent space. As a result, the models stop handling uncertainty in conversations. To resolve that, we propose speculative sampling of latent variables. Our method chooses the most probable one from redundantly sampled latent variables for tying up the variable with a given response. We confirm the efficacy of our method in response generation with massive dialogue data constructed from Twitter posts.

## introduction in early neural-based approaches to chat dialogue modeling, conventional encoder-decoder frameworks @xcite tended to generate safe responses @xcite . the main reason was that these frameworks model response generation as one-to-one projections from an utterance to a response, while many probable responses often exist in open-domain conversations. the use of conditioned variational autoencoders (cvae) is a promising approach for resolving the problem @xcite . in these models, latent variables sampled from approximated distributions are expected to serve as a clue to handle the uncertainty in probable responses. the uncertainty can correspond to topics, domains, or styles that are not explicitly controlled. however, the training of variational models is known to be unstable in chat response generation. when the training fails, latent variables are ignored and the models are reduced to the conventional encoder-decoders @xcite . it is also possible for latent variables to work too aggressively and lead the models to generate responses that are less relevant to the contexts. although many existing studies have tried to improve variational models @xcite @xcite @xcite ( § 2), we postulate that there still remains a problem that degrades the models; during training, a sampled latent variable can be inappropriate to represent a given response, due to 1) immature parameters in early stages of training and 2) a trade-off in training objectives (figure 1 ). we hypothesize that the discrepancy between an unreliable latent variable and a given response can hinder models from structuring a modulated latent space. to address the problem, we propose speculative sampling of latent variables, a simple model-agnostic method to help variational models implicitly handle the uncertainty in conversations. in experiments, we evaluated our method on massive open-domain dialogue data taken from twitter. automatic and human evaluations on the generated responses confirmed that our method improved both sensibleness and specificity of responses. the contributions of this paper are as follows. to design a latent space where the relevance and diversity of probable outputs are reflected geometrically, @xcite proposed spacefusion. among the aforementioned studies, their approach shares with us a similar goal of organizing the latent space. @xcite proposed to sample multiple latent variables in text modeling, similarly to our method. however, the intention is different as their method was for better approximation of the expected reconstruction term in training. ## speculative latent variables sampling the main concept of variational response generation models is to handle the uncertainty in conversations as the randomness of a latent variable z sampled from the model's distribution. with parameters θ and φ, the model first approximates prior and posterior distributions p θ (z|x) and q θ (z|x, y) from the utterance x and response y in a conversation. then, the model samples a latent variable z, from the distributions and feeds it to the decoder to compute the probability of the response p φ (y|x, z). ideally, the latent variable z p ∼ p θ (z|x) and z q ∼ q θ (z|x, y) are representations that can generate all probable responses to x and the given response y, respectively. in training, the following objective, which combines the reconstruction loss and kullback-leibler (kl) divergence d kl , is maximized: here, in training, what if a latent variable z q sampled from the posterior distribution q θ (z|x, y) is inappropriate to represent the response y? although the distribution is approximated under the observation of the response, this is still possible because the parameters for the approximation are incomplete during training. furthermore, optimizing kl-divergence does not necessarily help the reconstruction of y; the posterior distribution is promoted to be similar to the prior distribution that also covers other probable responses. as a result of this discrepancy, the training becomes skewed. the model can lose track of the correspondence between sampled latent variables and responses to be generated, and its latent space are disorganized. to address the problem, we propose speculative sampling of latent variables, a simple method to disentangle the discrepancy. specifically, we sample k latent variables {z 0 , z 1 , • • • , z k-1 } from the posterior distribution, and compute the loss for each variable in training. we then compute gradients only from the latent variable that has the least loss among the sampled variables. this simple modification prevents models from tying up unreliable variables with given responses. foot_0 4 experimental setup ## results this section reports results of automatic ( § 5.1) and human evaluations ( § 5.2) of generated responses on the twitter datasets, and then analyzes the models' outputs ( § 5.3). ## conclusions in this study, we aimed to help dialogue models construct a organized latent space that can capture implicit uncertainty in conversations. we proposed speculative sampling of latent variables, a method for mitigating the discrepancy in training between sampled latent variables and corresponding responses. experimental results in a response generation test with massive twitter dialogue data confirmed that our proposed method improved both sensibleness and specificity of generated responses. we will release all code and ids of twitter posts. 12