SlowGuess/ACL_2020_OCR huggingface dataset:
In human evaluation, we randomly select 100 dialogue contexts and generate responses with compared methods. Three crowd-sourcing workers are asked to score the response quality on a scale of [0, 1, 2] from four aspects - fluency, coherence, informativeness and overall. The higher score, the better. Details about the criteria are given as follows.
- Fluency measures whether the generated sentence is smooth and grammatically correct.
- Coherence evaluates whether the generated response is relevant with the dialogue context and consistent with the expressed information or background knowledge.
- Informativeness assesses whether the response is informative or not.
- Overall represents the general evaluation, where 0 indicates a bad response, 1 refers to a normal response and 2 stands for a good response.
After collecting the assessments from annotators,
the response's final score is determined via majority voting. The average Fleiss's kappa (Fleiss and Cohen, 1973) on Persona-Chat and Daily Dialog is 0.515 and 0.480 respectively, indicating annotators have reached moderate agreement.
The experimental results on Persona-Chat and Daily Dialog with automatic and human evaluations are summarized in Table 2. As suggested in the empirical study (Liu et al., 2016), the correlation between automatic metrics and human judgments is weak in open-domain dialogue generation. In the automatic evaluation, experimental results demonstrate that no method can consistently outperform the others.
During human evaluations, our method achieves better performance consistently across all the metrics on Persona-Chat and Daily Dialog. The scores of fluency almost approach the upper bound, revealing that our generated responses are very fluent. The informativeness assessments indicate that the information in our generated responses is significantly richer, as compared with the baseline methods. Our responses are coherent with the context and favored most by crowd-sourcing workers. The ablation study with our method and our w/o latent also suggests that through the incorporation of discrete latent variables, remarkable improvements can be achieved for dialogue generation. In addition, it can be observed that the generation quality of transformed-based approaches (LIC and our method) is significantly better than that of RNN-based methods (Seq2Seq and iVAEMI).<sup>3</sup>
The experimental results on DSTC7-AVSD with automatic evaluation are provided in Table 3. In the
<sup>3</sup>It is a normal phenomenon that the performance of our w/o latent is close to that of LIC. Both of them initialize network parameters with pre-trained language models and continue training with large-scale conversation data as Reddit.
experiments, our response selection is strengthened with an extra ranking step, which learns to rank the candidates according to automatic scores and selects the top one as the final answer. The results in Table 3 demonstrate that our method has brought a new breakthrough for DSTC7-AVSD. Additionally, the upper bound of our method is also reported, under the ideal scenario that the optimal candidate answer can be selected. $^{4}$  The incredible results validate the great potential of our approach.
3.3.1 Case Analysis
To further dissect the quality of our pre-trained model, several examples of generated responses are provided in Table 4. For each piece of context, our model can produce multiple responses by assigning distinct values to the latent variable and five candidate responses are selected for display in the table. It shows that our pre-trained model is able to generate diverse and appropriate responses. More examples on the conversational datasets are provided in the Appendix.
3.3.2 Comparison of Pre-trained Models
To further analyze the effectiveness of our pretrained model, more ablation studies have been conducted on Persona-Chat. Distinct pre-trained models are included for comparison. To be fair, their transformer layers are all set to 12. There are three different sizes of training dialogues: 1k, 5k and 9k (all training data). The training configurations and experimental results measured with perplexity are summarized in Table 5. There are three groups of pre-trained models: group 1 applies direct fine-tuning of BERT or GPT-2 (Radford et al., 2019) on Persona-Chat; group 2 employs Twitter and Reddit for further training upon the basis of pretrained language models; group 3 carries out the training process with latent variable.<sup>5</sup> (Model 2.2 is our w/o latent one and model 3.1 is our method.)
These results demonstrate that our method outperforms the other pre-trained models consistently
Given a dialogue context and background knowledge, our model is able to generate  $K$  diverse responses. Each of them will be evaluated using MSCOCO and the one obtaining the best score will be treated as the optimal candidate answer.
${}^{5}$  Overall,group 1 involves two-stage training: pre-training of language model with general text and fine-tuning on small conversation datasets. Whereas, group 2 and group 3 involve three-stage training: pre-training of language model with general text, further pre-training of dialogue generation with Twitter and Reddit,and fine-tuning on small conversation datasets.
with lower perplexity across different training sets. Several interesting conclusions can be also drawn from these results. Firstly, the comparison between model 1.2 and model 1.3 encourages the adoption of flexible attention mechanism to fully leverage the bi-directional context information. $^{6}$  Secondly, the superiority of group 2 over group 1 mainly comes from the employment of Twitter and Reddit, which are closer to human conversations than general text. Thirdly, the comparison between model 2.2 and model 3.1 reflects that the incorporation of discrete latent variable is able to boost the quality of response generation, whose effects have also been verified in Table 2.
Related work involves pre-trained language models and one-to-many modeling in dialogue generation. Pre-trained Language Models. Pre-trained language models, which are trained on massive general text, have brought many breakthroughs on various NLP tasks. These models can be roughly divided into two categories according to their attention mechanisms. GPT (Radford et al., 2018) and GPT-2 (Radford et al., 2019) are representative unidirectional language models, where one token is only allowed to attend its previous tokens and the objective is to maximize left-to-right generation likelihood. BERT (Devlin et al., 2019) and XLNet (Yang et al., 2019) are bi-directional language models, where bi-directional context attention is enabled for token prediction. The latest unified language model UniLM (Dong et al., 2019) is able to support both uni- and bi-directional attention with flexible self-attention mask designs. Recently, some attempts (Golovanov et al., 2019; Wolf et al., 2019; Zhang et al., 2019) have been made to adapt generative language models GPT or GPT-2 for dialogue generation. Whereas the special issues of conversations, such as impacts from background knowledge and problems of one-to-many relationship, are not fully considered and tackled in these adaptations.
One-to-many Modeling. Given one piece of context, there exists multiple appropriate responses, which is known as the one-to-many mapping problem. To model this one-to-many relationship, CVAE (Zhao et al., 2017) employs Gaussian distri
The results of model 1.1 demonstrate that there are some deficiencies in performance to apply direct fine-tuning of BERT on small conversation datasets, as discussed in the introduction.
bution to capture the discourse-level variations of responses. To alleviate the issue of posterior collapse in VAE, some extension approaches are further developed, including conditional Wasserstein auto-encoder of DialogWAE (Gu et al., 2019) and implicit feature learning of  $\mathrm{iVAEMI}$  (Fang et al., 2019). SpaceFusion(Gao et al., 2019) aims to jointly optimize diversity and relevance in the latent space, which are roughly matched by the distance and direction from the predicted response vector. Besides the continuous representation in VAE, discrete categorical variables are also utilized for interpretable generation (Zhao et al., 2018). Additionally, multiple mapping modules as latent mechanisms are introduced for diverse generation (Chen et al., 2019), where accurate optimization is carried out via posterior mapping selection. However, due to the small scale of annotated conversation data and limited capacity of generation network, it remains challenging for these methods to balance the diversity and fluency during response generation.
A novel pre-training model for dialogue generation is introduced in this paper, incorporated with latent discrete variables for one-to-many relationship modeling. To pre-train our model, two reciprocal tasks of response generation and latent recognition are carried out simultaneously on large-scale conversation datasets. Our pre-trained model is flexible enough to handle various down-stream tasks of dialogue generation. Extensive and intensive experiments have been carried out on three different kinds of publicly available datasets. And the results demonstrate that our model obtains significant improvements over the other state-of-the-art methods.
Our work can be potentially improved with more fine-grained latent variables. In the future, we will also explore to boost the latent selection policy with reinforcement learning and extend our pre-training to support dialogue generation in other languages.
We would like to thank the ACL reviewers for their constructive suggestions and Chaotao Chen, Junkun Chen, Tong Wu and Wenxia Zheng for their generous help. This work was supported by the Natural Key Research and Development Project of China (No. 2018AAA0101900).
Huda Alamri, Vincent Cartillier, Abhishek Das, Jue Wang, Anoop Cherian, Irfan Essa, Dhruv Batra, Tim K Marks, Chiori Hori, Peter Anderson, et al. 2019. Audio visual scene-aware dialog. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7558-7567.
Siqi Bao, Huang He, Fan Wang, Rongzhong Lian, and Hua Wu. 2019. Know more about each other: Evolving dialogue strategy via compound assessment. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5382-5391.
Boxing Chen and Colin Cherry. 2014. A systematic comparison of smoothing techniques for sentence-level bleu. In Proceedings of the 9th Workshop on Statistical Machine Translation, pages 362-367.
Chaotao Chen, Jinhua Peng, Fan Wang, Jun Xu, and Hua Wu. 2019. Generating multiple diverse responses with multi-mapping and posterior mapping selection. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pages 4918-4924.
Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dólar, and C Lawrence Zitnick. 2015. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325.
Kyunghyun Cho, Bart van Merrienboer, Caglar Gul- cehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rn n encoder-decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pages 1724-1734.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4171-4186.
Emily Dinan, Varvara Logacheva, Valentin Malykh, Alexander Miller, Kurt Shuster, Jack Urbanek, Douwe Kiela, Arthur Szlam, Iulian Serban, Ryan Lowe, et al. 2019a. The second conversational intelligence challenge (convai2). arXiv preprint arXiv:1902.00098.
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2019b. Wizard of wikipedia: Knowledge-powered conversational agents. International Conference on Learning Representations.
Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. 2019. Unified
language model pre-training for natural language understanding and generation. arXiv preprint arXiv:1905.03197.
Le Fang, Chunyuan Li, Jianfeng Gao, Wen Dong, and Changyou Chen. 2019. Implicit deep latent variable models for text generation. arXiv preprint arXiv:1908.11527.
Joseph L Fleiss and Jacob Cohen. 1973. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. In Educational and Psychological Measurement, pages 613-619.
Michel Galley, Chris Brockett, Xiang Gao, Jianfeng Gao, and Bill Dolan. 2019. Grounded response generation task at dstc7. In AAAI Dialog System Technology Challenge Workshop.
Xiang Gao, Sungjin Lee, Yizhe Zhang, Chris Brockett, Michel Galley, Jianfeng Gao, and Bill Dolan. 2019. Jointly optimizing diversity and relevance in neural response generation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1229-1238.
Sergey Golovanov, Rauf Kurbanov, Sergey Nikolenko, Kyryl Truskovskyi, Alexander Tselousov, and Thomas Wolf. 2019. Large-scale transfer learning for natural language generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6053-6058.
Xiaodong Gu, Kyunghyun Cho, Jung-Woo Ha, and Sunghun Kim. 2019. Dialogwae: Multimodal response generation with conditional Wasserstein autoencoder. International Conference on Learning Representations.
Chenyang Huang, Osmar Zaiane, Amine Trabelsi, and Nouha Dziri. 2018. Automatic dialogue generation with expressed emotions. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 49-54.
Nitish Shirish Keskar, Bryan McCann, Lav Varshney, Caiming Xiong, and Richard Socher. 2019. CTRL: A Conditional Transformer Language Model for Controllable Generation. arXiv preprint arXiv:1909.05858.
Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations.
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A diversity-promoting objective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 110-119.
Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017. Dailydialog: A manually labelled multi-turn dialogue dataset. In Proceedings of the 8th International Joint Conference on Natural Language Processing, pages 986-995.
Chia-Wei Liu, Ryan Lowe, Iulian Serban, Mike Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2122-2132.
Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. Technical report, OpenAI.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. Technical report, OpenAI.
Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards empathetic open-domain conversation models: A new benchmark and dataset. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5370-5381.
Ramon Sanabria, Shruti Palaskar, and Florian Metze. 2019. Cmu sinbad's submission for the dstd7 avsd challenge. In AAAI Dialog System Technology Challenge Workshop.
Oriol Vinyals and Quoc Le. 2015. A neural conversational model. arXiv preprint arXiv:1506.05869.
Thomas Wolf, Victor Sanh, Julien Chaumont, and Clement Delangue. 2019. Transfertransfo: A transfer learning approach for neural network based conversational agents. arXiv preprint arXiv:1901.08149.
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google's neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144.
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. arXiv preprint arXiv:1906.08237.
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pages 2204-2213.
Tiancheng Zhao, Kyusong Lee, and Maxine Eskenazi. 2018. Unsupervised discrete sentence representation learning for interpretable neural dialog generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pages 1098-1107.
Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 654-664.
Hao Zhou, Tom Young, Minlie Huang, Haizhou Zhao, Jingfang Xu, and Xiaoyan Zhu. 2018. Commonsense knowledge aware conversation generation with graph attention. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, pages 4623-4629.
Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In Proceedings of the IEEE International Conference on Computer Vision, pages 19-27.
A Additional Case Analysis
In Table 6, it provides the cases of our method and compared approaches on Persona-Chat, where two participants chat with each other according to their personas. As shown in the example, participant P2 needs to produce a response towards the given dialogue context, conditioned on his/her persona profile. The baseline Seq2Seq tends to generate common replies with low informativeness and poor coherence. LIC and our w/o latent are able to produce some coherent responses, whereas deficient in informativeness. In comparison, the response by our method is not only coherent with the context, but also expressive of the background personas. Besides, we also observe the phenomenon of diverse knowledge usage in our response generation, which suggests that the latent variable helps control the knowledge selection and utilization in an implicit way.
Table 7 provides the generated responses on Daily Dialog, where two participants chat about daily life. This example shows that Seq2Seq is able to generate fluent utterances, while lacking coherence with the context. As for iVAEMI, it suffers
from the difficulty to balance diversity and fluency. By contrast, our method is able to generate more coherent and high-quality responses.
Table 8 provides the generated responses on DSTC7-AVSD, where two participants discuss the objects and events in a video. Participant P1 is responsible to raise questions, who only has access to the first, middle and last frames of the video. Participant P2 has watched the whole video and needs to answer the partner's questions. The generation system is developed to mimic P2 and answer the questions based on the background knowledge. The baseline approach relies on the multi-modal information, including text, audio and visual features, to produce the answer. Due to the limited network capacity, it lacks fidelity to the background knowledge and makes the generated response less accurate. (As the team of CMU has not released their codes or models, their samples are omitted here.) Our method utilizes the video caption and summary as background knowledge. It can be observed that our method generates a more appropriate answer as compared with the baseline approach.
Persona Profiles [P2]	Context
i love trying barbeque restaurants .i am an administrative assistant .i am looking forward to retiring in five years .i like ot play racquetball .	[P2] hi , do you like BBQ ?[P1] hello yes i love BBQ[P2] i love restaurants with BBQ , they are a lot of fun .[P1] my poor toes are so webbed[P2] really ? afterwards , i go and play racquetball to loose some calories .[P1] cool i like to paint[P2] what do you do as a profession ? i work as administrative assistant[P1] i dig in people trash for things to build[P2] sounds unique but that is more of a hobby instead of a profession[P1] true . i paint for a living
Seq2Seq	that sounds like a lot of work . do you have a job ?
LIC	yeah it is a lot of fun
Our w/o Latent	do you have any hobbies ?
Our Method	that is cool , i am looking forward to retiring in 5 years
Table 6: Case analysis of response generation on Persona-Chat.
Context	[P2] have you seen the new cosmo magazine that i was reading this morning ?
[P1] i think i may have seen it on the kitchen table .
[P2] you're right , thanks . there's an interesting interview with a top director about his latest film that i want to finish .
[P1] who's the director ?
Seq2Seq	yes , i did . i found it very disappointing .
iVAEMI	the cake and is very popular .
Our w/o Latent	i'm not sure of his name , but i know it's going to be a thriller .
Our Method	i'm not sure , but i think he might be the director of gone with the wind .
Table 7: Case analysis of response generation on Daily Dialog.
Video Caption	a man closes his window, then he sneezes twice before taking a drink. then he opens up a bag and digs through it looking for something before walking out of the room.
Video Summary	a man closes the window, goes to the table and goes through the items in a bag, takes a drink from the green cup and leaves the room.
Context	[P1] what is the guy doing at the window?
[P2] the guy is closing the window
[P1] what does he do after that?
Baseline	he picks up a book from the table
Our Method	he goes to the table and takes a drink from a green cup
Table 8: Case analysis of response generation on DSTC7-AVSD.