Generating Dialogue Responses from a Semantic Latent Space (2020)

Abstract: Existing open-domain dialogue generation models are usually trained to mimic the gold response in the training set using cross-entropy loss on the vocabulary. However, a good response does not need to resemble the gold response, since there are multiple possible responses to a given prompt. In this work, we hypothesize that the current models are unable to integrate information from multiple semantically similar valid responses of a prompt, resulting in the generation of generic and uninformative responses. To address this issue, we propose an alternative to the end-to-end classification on vocabulary. We learn the pair relationship between the prompts and responses as a regression task on a latent space instead. In our novel dialog generation model, the representations of semantically related sentences are close to each other on the latent space. Human evaluation showed that learning the task on a continuous space can generate responses that are both relevant and informative.

## introduction the sequence-to-sequence framework and transformer based models are the most popular choices for designing open-domain neural response generation systems @xcite . those models typically involve maximizing the probability of the ground truth response given the input prompt, trained using a cross entropy loss on the vocabulary. however, dialogue response generation is an open-ended, high entropy task, since there can be a wide variety of possible responses to a given prompt. a good response does not have to use similar vocabulary or similar sentence structure as the gold response, thus the end-to-end cross entropy loss is unsuitable for this task. we hypothesize that this fundamental deficiency is the primary reason why dialog generation models tend to generate bland and uninfor-mative responses, such as "i don't know" @xcite , despite the presence of much more instances of specific responses in the training data than generic responses. the specific issue is the following. a model trained using maximum likelihood objective treats each token of the vocabulary independently. the probabilities of each individual informative word in the vocabulary are low because the answer is open-ended. the model is unable to capture that most of the probability mass are on a group of semantically related words. thus the words with the highest probabilities are often uninformative stop words with high frequency in the training data. a similar effect happens on the utterance level when using beam search decoding. when searching for the most probable utterance, the probability of each candidate sentence is calculated independently, and the model is unable to use the semantic relatedness between different candidate utterances @xcite . while informative and specific responses collectively have a high probability, it is diluted by the large number of variations and possibilities of specific responses. on the other hand, generic responses have much less variations, thus they become the most probable response sequences. an alternative decoding method to beam search is sampling @xcite , which does not suffer from this problem. however, sampling does not consider the subsequent words during decoding, and the randomness in word choice makes it prone to generating implausible responses, responses with grammatical errors and coherence issues. aiming to take into account the semantic relatedness of diverse specific responses, we propose an alternative to cross-entropy training, which is learning the pair relationship between the prompts and responses as a regression task on a latent space. in our novel dialog generation model, the genera-tion process could be separated into two steps. the first step predicts a sentence vector of the response on the latent space, on which the representations of semantically related sentences are close to each other. since predicting a vector is a regression problem in the latent space instead of classification in the vocabulary as in mle loss, our model is able to learn that most of the probability mass of the response is around the cluster of possible specific responses. this is illustrated in figure 2 showing our model's representations of prompts and responses on a t-sne plot. the second step constructs the full response sentence from the predicted vector. we train an autoencoder. the decoder part is used for constructing the full response sentence from the predicted sentence vector. since the semantics of the response and the decoding are learned separately, we can perform beam search for the most probable sequence given the semantic vector during inference without preferring generic responses. the main contributions of our work are 1) we propose to learn dialogue generation as an regression task on a semantic latent space, as an alternative of end-to-end cross entropy training used in most previous methods, to address the problem that end-to-end cross entropy classification are unable to integrate information from semantically similar responses and words. 2) our model separates the response into information likely and unlikely to be correlated with the prompt. 3) evaluation by crowdworkers showed that the latent space method significantly outperforms baselines using end-to-end cross entropy classification, in terms of generating responses that are both relevant and informative. ## related work several previous models also use the idea of learning on sentence vector representations. @xcite used two autoencoders to learn the semantic representations of inputs, and learned utterancelevel dependency between those representations. spacefusion @xcite fuses the autoencoder and seq2seq feature space, so that the distance and direction from a predicted response vector roughly matches the relevance and diversity. those methods add additional autoencoder losses to manipulate the intermediate representation space, but they still use the problematic endto-end cross entropy loss for generation. in our work, we completely remove the end-to-end loss term, so the matching of the input and response is learned only on a shared semantic latent space. @xcite proposed a two-stage generation process, which predicts the average of the reference responses as an intermediate task, but it requires multiple responses for each prompt in the training data. on the machine translation task, @xcite explored predicting continuous vectors on the word level in seq2seq models instead of using softmax classification. we predict continuous vectors on the utterance level. there are other aspects to tackle the generic response problem, @xcite maximized mutual information in decoding or reranking. @xcite trained multiple response mechanisms to model diversity. @xcite split the generation into segments and allow attention to attend to both the prompt and the response to improve diversity. several works use explicit specificity metrics to manipulate the specificity of the responses. frequency based metrics such as idf are used in @xcite . @xcite proposed using a specificity metric trained on discourse relation pair data. we use a modified version of deep canonical correlation analysis (dcca) @xcite to learn the semantic latent space. dcca has previously been used on various tasks including feature learning @xcite , caption retrieval @xcite , multi-label classification @xcite , image cross-reconstruction @xcite , and multilingual word similarity @xcite . @xcite experimented on performing dcca on sequential data with a recurrent network. ## our method given example dialogue (prompt, response) pairs (d x , d y ) from open-domain dialogue datasets, our goal is to generate a relevant and non-generic response when given an unseen prompt. the structure of our model is depicted in figure 1 . it consists of three encoders; prompt encoder f x , correlated response encoder f y , and uncorrelated response encoder f u . the final response is generated from a semantic latent vector via a decoder g y . during training all three encoders and the decoder are tuned. however, during testing only the prompt encoder and the decoder are utilized. ## conclusion in this work, we pointed out that end-to-end cross entropy classification used in most previous methods is not able to integrate information from different semantically similar words responses, and designed a substitute method that is able to do so. our method learns the pair relationship between prompts and responses as a regression task on a latent space, which is more suitable for the openended nature of this task. we performed ablation study to validate the components of our model. human evaluation results concretely demonstrate that our latent space method significantly outperforms baselines using end-to-end cross entropy training, in terms of relevance and informativeness.