Adversarial Learning on the Latent Space for Diverse Dialog Generation (2020)

Abstract: Generating relevant responses in a dialog is challenging, and requires not only proper modeling of context in the conversation, but also being able to generate fluent sentences during inference. In this paper, we propose a two-step framework based on generative adversarial nets for generating conditioned responses. Our model first learns a meaningful representation of sentences by autoencoding, and then learns to map an input query to the response representation, which is in turn decoded as a response sentence. Both quantitative and qualitative evaluations show that our model generates more fluent, relevant, and diverse responses than existing state-of-the-art methods.

## introduction dialog generation is a challenging problem because it not only requires us to model the context in a conversation but also to exploit it to generate a relevant and fluent response. a dialog generation system can be divided into two parts: 1) encoding the context of the conversation, and 2) generating a response conditioned on the given context. a generated response is considered to be "good" if it is meaningful, fluent, and most importantly, relevant to the given context. with the advancement of deep learning, sequence-to-sequence (seq2seq) models @xcite are adopted for dialog systems to encode conversational context and generate a response. however, they suffer from the problem of generic utterance generation, e.g., always generating "i don't know" @xcite . one possible explanation @xcite is the high uncertainty in dialog generation. a plausible response is analogous to a "mode" of a continuous distribution, and the response distribution is thus multimodal. however, the decoder of a seq2seq model is trained by crossentropy loss, which is equivalent to minimizing the kl divergence between the target and predicted distributions. the asymmetric nature of kl divergence makes the learned distribution wide-spreading, analogous to the mode-averaging problem for continuous variables. variational encoder-decoders @xcite @xcite and wasserstein encoder-decoders @xcite adopt probabilistic modeling to encourage diversity in responses. however, their decoders are also trained by cross-entropy loss against the target sequence, still making the model generate generic utterances. in this paper, we propose an approach that uses adversarial learning in the latent space for dialog generation. we first train a variational autoencoder (vae) (kingma and welling, 2014) on sentences, and then apply a generative aderversarial network (gan) on the latent space of the vae. at inference time, we obtain the latent representation of the response from the generator of the gan and decode it using the vae's decoder. in this way, we can benefit from the mode-capturing property of gans @xcite . also, our gan is trained on the latent space, and techniques like gumbel-softmax and reinforcement-learning (rl) are not necessary, which largely simplifies the training procedure.we further introduce a mean squared error (mse) auxiliary loss to our adversarial module, which mitigates the mode-missing problem in gans (che et al., 2017), resulting in more relevant and diverse responses. we evaluate our model on the deduplicated version @xcite of the benchmark daily-dialog dataset @xcite and also the switchboard dataset @xcite . results indicate that responses generated by our model are more relevant to the input query/context, and are more diverse and fluent than the existing baselines. the main contributions of our paper are as follows. step 2: we train an adversarial network on the latent z space for learning dialog generation. given a query-response pair (q, r) in the training set, we use the trained encoder from step 1 to obtain their latent variables z q and z r . the query latent variable z q is fed to a generator g, that maps it to the corresponding response's latent variable ẑr . when training the generator, we aim to match z r and ẑr through the adversarial loss combined with a mean-squared error loss. in here, the adversarial loss further involves a discriminator that classifies the predicted response representation ẑr versus the encoded representation of the true response z r , conditioned on the query z q . the details of our approach will be introduced in the rest of this section. ## experiments we conduct experiments on the dailydialog dataset @xcite , a manually labeled multi-turn dialog dataset, and the switchboard dataset @xcite , a dialog dataset containing transcripts of telephonic conversations. for dailydialog, we use the original splits after removing duplicates following @xcite . we use the the allennlp framework @xcite to implement all our models. appendix a presents more experimental details and hyper-parameters. we use the following baseline models for comparison: ## conclusion we propose an effective two-stage model for dialog generation. we make use of sentence representations learned by a vae and train a adversarial network on vae's latent space to generate diverse responses given a query and context. we observe that our model outperforms existing state-of-the-art approaches by generating more diverse, fluent, and relevant sentences.