An Efficient Policy Gradient Method for Conditional Dialogue Generation Encoderdecoder models have been commonly used in dialogue generation tasks. However, they tend to generate dull and generic response utterances. To tackle this problem, we consider dialogue generation as a conditional generation problem. For a given context history, our model can generate different response utterances with desirable dialog acts. Our model follows the SeqGAN framework, where the generator takes context history and dialog act as inputs and generates corresponding response utterances. The discriminator computes rewards by considering the quality of entire utterance and dialog act. Our model is trained by a policy gradient approach. To overcome the bottleneck of excessive time complexity incurred by the Monte Carlo search for training, we propose a local discriminator network to compute the individual reward in one forward propagation, thereby dramatically accelerating the training procedure. Experimental results demonstrate that our proposed method can achieve comparative performance with Monte Carlo search, while reducing the training time dramatically.

Generating Diverse Conversation Responses by Creating and Ranking Multiple Candidates Abstract   This paper introduces our systems built for Track 2 of Dialog System Technology Challenge 7 (DSTC7). This challenge track aimed to evaluate the response generation methods using fully datadriven conversation models in a knowledgegrounded setting, where textual facts were provided as the knowledge for each contextresponse pair. The sequencetosequence models have achieved impressive results in machine translation and have also been widely used for endtoend generative conversation modelling. However, they tended to output dull and repeated responses in previous studies. Our work aims to promote the diversity of endtoend conversation response generation by adopting a twostage pipeline. 1) Create multiple responses for an input context together with its textual facts. At this stage, two different models are designed, i.e., a variational generative (VariGen) model and a retrievalbased (Retrieval) model. 2) Rank and return the most relevant response by training a topic coherence discrimination (TCD) model for calculating ranking scores. In our experiments, we demonstrated the effectiveness of the response ranking strategy and the external textual knowledge for generating better responses. According to the official evaluation results, our Retrieval and VariGen systems ranked first and second respectively among all participant systems on Entropy metrics which measured the objective diversity of generated responses. Besides, the VariGen system ranked second on NIST and METEOR metrics which measured the objective quality of generated responses.