PLATO : Pre-trained Dialogue Generation Model with Discrete Latent Variable (2020)

Abstract: Pre-training models have been proved effective for a wide range of natural language processing tasks. Inspired by this, we propose a novel dialogue generation pre-training framework to support various kinds of conversations, including chit-chat, knowledge grounded dialogues, and conversational question answering. In this framework, we adopt flexible attention mechanisms to fully leverage the bi-directional context and the uni-directional characteristic of language generation. We also introduce discrete latent variables to tackle the inherent one-to-many mapping problem in response generation. Two reciprocal tasks of response generation and latent act recognition are designed and carried out simultaneously within a shared network. Comprehensive experiments on three publicly available datasets verify the effectiveness and superiority of the proposed framework.

## introduction dialogue generation is a challenging task due to the limited corpus of human conversations, complex background knowledge, and diverse relationships between utterances. recently, pre-trained large-scale language models, such as bert @xcite and xlnet @xcite , have achieved prominent success in natural language processing. such models are usually constructed based on a massive scale of general text corpora, like english wikipedia or bookscorpus @xcite , where distributed representations can be learned automatically from the raw text. with these representations being fine-tuned, breakthroughs have been continuously reported for various downstream tasks, especially those on natural language understanding, such as question answering, natural language inference, and so on. this pre-training and fine-tuning paradigm also sheds light on the tasks of natural language generation, like dialogue generation. however, the previous study demonstrates that there are some deficiencies in performance while directly fine-tuning bert on small conversation datasets @xcite . possible reasons are three-fold: 1) the underlying linguistic patterns in human conversations can be highly different from those in general text, which suggests a potentially large gap in knowledge or data distribution; 2) the training mode of uni-directional dialogue generation is also distinct from that of bi-directional natural language understating as applied in bert; 3) unlike most of the general nlp tasks, there exists a one-to-many relationship in dialogue generation, where the dialogue context may correspond to multiple appropriate replies. in this paper, we propose a new method to tackle the above challenges, aiming to obtain a highquality pre-training model for dialogue generation. first of all, to reduce the gap between data distributions, large-scale reddit and twitter conversations are utilized to further pre-train the generation model (upon the basis of language models pre-trained with general text). secondly, to mitigate the difference in training mode, a flexible paradigm integrating uni-and bi-directional processing is employed in this work, which is inspired by the latest unified language modeling @xcite . thirdly, a discrete latent variable is introduced to model the one-to-many relationship among utterances in conversations. each value of the latent variable corresponds to the particular conversational intent of one response, which is referred as latent speech act. distinct with those controllable dialogue generation based on explicit labels (including emotion, keywords, domain codes, and so on) @xcite , our latent variable gets exempted from the restriction of human annotations and can be learned automatically from the corpus in an unsupervised way. in the pre-training of dialogue generation, response generation and latent act recognition are carried out simultaneously within a shared network. based on the context and latent variable, the generation task tries to maximize the likelihood of the target response. meanwhile, the recognition task aims to estimate the latent variable w.r.t. the given context and target response. apparently, the accurate recognition of the latent variable is a crucial factor in boosting the quality of response generation. we conducted experiments on three different kinds of conversation tasks: chit-chat, knowledge grounded conversation, and conversational question answering. experimental results verify the effectiveness and superiority of our pre-trained model as compared with the other state-of-the-art methods. our pre-trained models and source code have been released at github, hoping to facilitate further research progress in dialogue generation. 1 ## dialogue generation pre-training given a piece of context, there exist multiple appropriate responses, leading to diverse conversation flows. it is widely recognized that the capability of modeling one-to-many relationship is crucial for the dialogue generation system @xcite . to this end, we propose to encode discrete latent variables into transformer blocks for one-to-many relationship modeling, where two reciprocal tasks of response generation and latent act recognition are collaboratively carried out. ## related work related work involves pre-trained language models and one-to-many modeling in dialogue generation. pre-trained language models. pre-trained language models, which are trained on massive general text, have brought many breakthroughs on various nlp tasks. these models can be roughly divided into two categories according to their attention mechanisms. gpt @xcite and gpt-2 @xcite are representative unidirectional language models, where one token is only allowed to attend its previous tokens and the objective is to maximize left-to-right generation likelihood. bert @xcite and xl-net @xcite are bi-directional language models, where bi-directional context attention is enabled for token prediction. the latest unified language model unilm @xcite is able to support both uni-and bi-directional attention with flexible self-attention mask designs. recently, some attempts @xcite @xcite have been made to adapt generative language models gpt or gpt-2 for dialogue generation. whereas the special issues of conversations, such as impacts from background knowledge and problems of one-to-many relationship, are not fully considered and tackled in these adaptations. one-to-many modeling. given one piece of context, there exists multiple appropriate responses, which is know as the one-to-many mapping problem. to model this one-to-many relationship, cvae @xcite employs gaussian distri-bution to capture the discourse-level variations of responses. to alleviate the issue of posterior collapse in vae, some extension approaches are further developed, including conditional wasserstein auto-encoder of dialogwae @xcite and implicit feature learning of ivae mi @xcite . spacefusion @xcite aims to jointly optimize diversity and relevance in the latent space, which are roughly matched by the distance and direction from the predicted response vector. besides the continuous representation in vae, discrete categorical variables are also utilized for interpretable generation @xcite . additionally, multiple mapping modules as latent mechanisms are introduced for diverse generation @xcite , where accurate optimization is carried out via posterior mapping selection. however, due to the small scale of annotated conversation data and limited capacity of generation network, it remains challenging for these methods to balance the diversity and fluency during response generation. ## conclusion a novel pre-training model for dialogue generation is introduced in this paper, incorporated with latent discrete variables for one-to-many relationship modeling. to pre-train our model, two reciprocal tasks of response generation and latent recognition are carried out simultaneously on large-scale conversation datasets. our pre-trained model is flexible enough to handle various down-stream tasks of dialogue generation. extensive and intensive experiments have been carried out on three different kinds of publicly available datasets. and the results demonstrate that our model obtains significant improvements over the other state-of-the-art methods. our work can be potentially improved with more fine-grained latent variables. in the future, we will also explore to boost the latent selection policy with reinforcement learning and extend our pre-training to support dialogue generation in other languages. yizhe zhang, siqi sun, michel galley, yen-chun chen, chris brockett, xiang gao, jianfeng gao, jingjing liu, and bill dolan. 2019. dialogpt: large-scale generative pre-training for conversational response generation. arxiv preprint arxiv:1911.00536. tiancheng zhao, kyusong lee, and maxine eskenazi. 2018. unsupervised discrete sentence representation learning for interpretable neural dialog generation. in proceedings of the 56th annual meeting of the association for computational linguistics, pages 1098-1107. tiancheng zhao, ran zhao, and maxine eskenazi. 2017. learning discourse-level diversity for neural dialog models using conditional variational autoencoders. in proceedings of the 55th annual meeting of the association for computational linguistics, pages 654-664. hao zhou, tom young, minlie huang, haizhou zhao, jingfang xu, and xiaoyan zhu. 2018. commonsense knowledge aware conversation generation with graph attention. in proceedings of the 27th international joint conference on artificial intelligence, pages 4623-4629. yukun zhu, ryan kiros, rich zemel, ruslan salakhutdinov, raquel urtasun, antonio torralba, and sanja fidler. 2015. aligning books and movies: towards story-like visual explanations by watching movies and reading books. in proceedings of the ieee international conference on computer vision, pages 19-27.