SlowGuess/ACL_2020_OCR huggingface dataset:
Our model applies the same parameter settings as the transformer. The number of layers  $\mathrm{N}_G = \mathrm{N}_D = \mathrm{N}_R = 3$ . G and R share the word embeddings, but the matching model D uses independent
embeddings. We use token-level batching with a size 4096. Adam is used for optimization, and the warm-up steps are set to 10,000. We implemented the model in OpenNMT.py (Klein et al., 2017).
In the evaluation, there are two essential factors to consider: persona consistency and response quality. We apply both human evaluations and automatic metrics on these two aspects to compare different models.
Human Evaluation We recruit five professional annotators from a third-party company. These annotators have high-level language skills but know nothing about the models. We sampled 200 persona-query-response tuples per model for evaluation. Duplicated queries (such as greetings which appear more than once) will not be sampled twice.
First, we evaluate the persona consistency of a response. The annotators are asked to decide whether the response is consistent with the given persona. 0 indicates irrelevant or contradictory and 1 indicates consistent (Const.).
Second, we evaluate the quality of a response on three conventional criteria: fluency (Fluc.), relevance (Relv.), and informativeness (Info.). Each aspect is rated on five-scale, where 1, 3, and 5 indicate unacceptable, moderate, and excellent performance, respectively. 2 and 4 are used for unsure.
Automatic Metrics Dziri et al. (2019) has shown that natural language inference based entailment ratio can be used as an indicator of dialogue consistency. Here we trained two well-performed NLI models, DIIN (Gong et al., 2018) and BERT (Devlin et al., 2019), to automatically predict the category of persona-response pairs, and we calculated the ratio of entailment as an additional reference to the persona consistency. In our experiments, DIIN and BERT achieved  $88.78\%$  and  $89.19\%$  accuracy on the DNLI test set, respectively, compared with previous best results  $88.20\%$ . The trained models are then leveraged for calculating entailment ratios. Two model-based entailment ratios are abbreviated as Ent $_{\text{diin}}$  and Ent $_{\text{bert}}$ .
For dialogue quality, we follow Zhang et al. (2018) to use perplexity (PPL) to measure the fluency of responses. Lower perplexity means better fluency. Besides, we also use Dist-1 / Dist-2 (Li et al., 2016a) to examine the model's ability to generate diverse responses, which is the ratio of distinct uni-grams / bi-grams.
5826
Model	Const.	Fluc.	Relv.	Info.	PPL	Dist-1.	Dist-2.	Entdiin	Entbert
S2SA	15.9%	3.17	2.84	2.63	34.8	1.92	4.86	9.80%	1.83%
GPMN	34.8%	3.78	3.57	\(3.76^{\dagger}\)	34.1	1.89	7.53	14.5%	7.36%
Per-S2S	35.3%	3.43	3.22	3.32	36.1	2.01	7.31	13.5%	6.15%
DeepCopy	36.0%	3.26	3.08	2.87	41.2	2.35	8.93	16.7%	8.81%
Transformer	38.8%	3.46	\(3.65^{\dagger}\)	3.54	27.9	3.12	15.8	14.2%	9.52%
Per-CVAE	42.7%	3.53	2.97	3.66	-*	\(3.83^{\dagger}\)	20.9	17.2%	7.36%
GDR (ours)	49.2%	3.86	3.68	3.77	16.7	3.66	22.7	21.5%	13.0%
Table 3: Results of human evaluations (on the left) and automatic metrics (on the right). The Dist-1.& 2. are scaled by  $10^{-2}$ . Significant tests (t-test) are performed, and our method is significantly better than all methods on most metrics  $(p\text{-value} < 0.05)$ , with the exceptions marked by  $\dagger$ . We also present two model-based ratios, the Ent $_{\text{diin}}$  and the Ent $_{\text{bert}}$ , as an additional reference for persona consistency assessments. Note that the automatic metrics are calculated on the whole test set. * The sampling process in CVAE leads to very unstable PPL.
GDR vs	Win(%)	Tie(%)	Lose(%)
S2SA	48.0	38.2	13.8
Per-CVAE	46.1	29.8	24.1
DeepCopy	43.8	35.5	20.7
Per-S2S	41.3	36.1	22.6
GPMN	35.0	31.0	34.0
Transformer	34.7	32.1	33.2
Table 4: GDR response quality gains over other baseline methods on a pairwise human judgment.
We report the main evaluation results in Table 3. Compared with baseline methods, our GDR model obtains the highest consistency score of  $49.2\%$  in human evaluation, which is significantly better than other methods. The target responses in the sampled data are also annotated, and  $65.4\%$  of them expressed persona information. Moreover, the two model-based entailment ratios, which are calculated on the whole test set, also prove the effectiveness of our GDR model. Although the two NLI models differ in results, our GDR model ranks first under the evaluation of both DIIN and BERT.
For dialogue quality, our proposed model has a remarkably lower perplexity of 16.7 than all other baseline methods. An analysis can be seen in Section 4.6. Besides, our distinct-2 metric is even significantly better than the Per-CVAE model, which is designed to generate diverse responses.
Additionally, we carried out pairwise response comparison to see the dialogue quality gains. We report the results in Table 4. While the GDR model significantly improves persona consistency, it can
still generate high-quality responses like the transformer and GPMN.
4.6 More Analysis
As the proposed model achieves better performance than baseline methods, we turn to ablation tests to further quantify the contributions made by different components. We ablated our model through several different approaches:
- GR It removes the matching model D, i.e., generates a prototype and rewrites it directly.
- GRdR This approach replaces the matching model D with  $10\%$  random deleting (Rd), thus to see if the masked prototype, extracted by our matching model D, is beneficial.
- Our model's generator, without further consistency matching and rewriting.
- T It is a transformer generator but removes the tuple-interaction in section 3.2 and directly concatenates persona texts to the query. This model is equivalent to the vanilla transformer.
We report the results in Table 5. First, we look into which components contribute to the consistency. As seen, from T, G, GR to GDR, every step has an observable improvement in Const., indicating the effectiveness of our model's design. Both the tuple-interaction in G and the rewriting process in R contribute to the improvements of persona consistency. The GRdR approach, with nothing different from GDR but a random deleting strategy, serves as a foil to our GDR model, which indicates a well-learned consistency matching model is of
5827
Model	Const.	Fluc.	Relv.	Info.	PPL
GDR	49.2%	3.86	3.68	3.77	16.7
GR	42.4%	3.72	3.40	3.66	18.0
GRdR	40.0%	3.60	3.29	3.56	20.6
G	40.1%	3.69	3.35	3.55	26.3
T	38.8%	3.46	\(3.65^{\ddagger}\)	3.54	27.9
Table 5: Results of the ablation study. GDR is significantly better than the ablated approaches, with an only exception marked by  $\ddagger$ .
GDR vs	Win(%)	Tie(%)	Lose(%)
GRdR	41.7	39.5	18.8
GR	39.9	40.9	19.2
G	38.1	35.8	26.1
great benefit to our three-stage generation framework to generate persona consistent dialogues.
Second, we investigated the improvement of our perplexity. As we can see, the one-stage transformer approaches G and T have a perplexity higher than 26. In contrast, after we add the rewritten R, the perplexity of all approaches has a significant decline, no matter whether there is a matching model D. Lower perplexity means lower cross-entropy, which indicates the responses from the models have more ground truth words. To some extent, perplexity verifies the human evaluation results of the consistency. One reason for this improvement could be that the rewritten works like a denoising autoencoder (Vincent et al., 2008), and it can focus more on the reconstruction of the missing information of sequence itself, rather than learning to map a sequence to an entirely different one.
We observed that the relevance scores of GR, GRdR, and G are a little inferior to the T. Even the GDR model is not significantly better than T on the relevance score. One plausible explanation is that all these models are specially designed for integrating persona information, although they obtain much better consistency score, it may come at the cost of relevance score.
Moreover, we compared the GDR's response quality with three ablated models and reported it in Table 6. As we can see, the deleting and rewriting, which are designed for improving consistency, also have a positive effect on the dialogue quality.
At last, we presented some generated examples
Table 6: Pairwise human judgment on response quality.
Persona
Query	i. My mother is a dentist
ii. I'm currently looking for a job
I want to become a physical therapist.
Gen&Del
Rewrite	I wish I could be a nurse.
I want to be a dentist like my mom.
w/o Delete	I want to become a nurse like you.
Transfmr
Per-S2S
GPMN
DeepCopy
Per-CVAE	I hear you. I am a nurse and a mom.
What kind of business?
I'm a professional painter, you?
What do you do for a living?
I guess I would buy you one
Persona
Query	i. Gymnastics is my favorite sport
ii. I'm a gold medalist olympian
I'm a minivan driver. I drive around.
Gen&Del
Rewrite	That's awesome. I love to ski.
That's cool. I am a champion gymnast.
w/o Delete	That's awesome. I am going to ski.
Transfmr
Per-S2S
GPMN
DeepCopy
Per-CVAE	Oh. I am still in high school.
I drive a big van, but around the house.
I drive a FIAT. I love it.
What do you do for a living?
That is why you like dogs? Tell me.
Persona
Query	i. I like to grill outdoors
ii. I have been retired for years
Too old to work in construction. You?
Gen&Del
Rewrite	I'm retired. I spend time outdoors.
I'm retired. I spend most time outdoors.
w/o Delete	I'm retired. I spend most time outdoors.
Transfmr
Per-S2S
GPMN
DeepCopy
Per-CVAE	I am a boy and work in the construction.
No, I am a student.
I am a teacher, I love to be a teacher.
I work in the construction industry.
I am a retired officer I love my bike ride.
Table 7: Example responses from different models, with a visualization of the consistency matching weights. Strikethrough words are the masked words in Delete stage. The w/o Delete is the ablated model GR in section 4.6, and Transfmr is short for Transformer.
in Table 7, together with the visualization of attention weights from match module D. In the first case, although the generated prototype is neutral regarding the persona, the word "nurse" is still masked according to our strategy. And after the rewriting stage, the final response expresses persona. In the second case, the prototype is potentially contradictory to the persona, and the keyword is successfully deleted by the matching model D. In the third case, the prototype is consistent with the persona, and no word is deleted. As a result, the final output response is the same as the output of no deletion model GR. In these cases, both consistency and quality are improved after the final rewriting.
5828
In this paper, we presented a three-stage framework, Generate-Delete-Rewrite, for persona consistent dialogue generation. Our method adopts transformer architecture and integrates a matching model to delete the inconsistent words. Experiments are carried out on public-available datasets. Both human evaluations and automatic metrics show that our method achieves remarkably good performance. In the future, we plan to extend our approach to improve the consistency of multi-turn dialogues.
This paper is supported by the National Natural Science Foundation of China under Grant No.61772153 and No.61936010. Besides, we want to acknowledge the Heilongjiang Province Art Planning Project 2019C027 and the Heilongjiang Province Social Science Research Project 18TQB100. We also would like to thank all the anonymous reviewers for their helpful comments and suggestions.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632-642.
Deng Cai, Yan Wang, Wei Bi, Zhaopeng Tu, Xiaojiang Liu, Wai Lam, and Shuming Shi. 2019a. Skeleton-to-response: Dialogue generation guided by retrieval memory. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1219-1228, Minneapolis, Minnesota. Association for Computational Linguistics.
Deng Cai, Yan Wang, Wei Bi, Zhaopeng Tu, Xiaojiang Liu, and Shuming Shi. 2019b. Retrieval-guided dialogue response generation via a matching-to-generation framework. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1866-1875, Hong Kong, China. Association for Computational Linguistics.
Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. Enhanced LSTM for natural language inference. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1657-1668, Vancouver, Canada. Association for Computational Linguistics.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2018. Wizard of wikipedia: Knowledge-powered conversational agents. arXiv preprint arXiv:1811.01241.
Nouha Dziri, Ehsan Kamalloo, Kory Mathewson, and Osmar Zaiane. 2019. Evaluating coherence in dialogue systems using entailment. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3806-3812, Minneapolis, Minnesota. Association for Computational Linguistics.
Yichen Gong, Heng Luo, and Jian Zhang. 2018. Natural language inference over interaction space. In International Conference on Learning Representations.
Diederik P Kingma and Max Welling. 2013. Autoencoding variational bayes. arXiv preprint arXiv:1312.6114.
Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander Rush. 2017. OpenNMT: Open-source toolkit for neural machine translation. In Proceedings of ACL 2017, System Demonstrations, pages 67-72, Vancouver, Canada. Association for Computational Linguistics.
Wenqiang Lei, Xiangnan He, Yisong Miao, Qingyun Wu, Richang Hong, Min-Yen Kan, and Tat-Seng Chua. 2020. Estimation-action-reflection: Towards deep interaction between conversational and recommender systems. In Proceedings of the 13th International Conference on Web Search and Data Mining, pages 304-312.
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016a. A diversity-promoting objective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 110-119, San Diego, California. Association for Computational Linguistics.
5829
Jiwei Li, Michel Galley, Chris Brockett, Georgios Spithourakis, Jianfeng Gao, and Bill Dolan. 2016b. A persona-based neural conversation model. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 994-1003, Berlin, Germany. Association for Computational Linguistics.
Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. 2016c. Deep reinforcement learning for dialogue generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas. Association for Computational Linguistics.
Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, Alan Ritter, and Dan Jurafsky. 2017. Adversarial learning for neural dialogue generation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2157-2169.
Hui Liu, Qingyu Yin, and William Yang Wang. 2019. Towards explainable NLP: A generative explanation framework for text classification. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5570-5581, Florence, Italy. Association for Computational Linguistics.
Qian Liu, Yihong Chen, Bei Chen, Jian-Guang Lou, Zixuan Chen, Bin Zhou, and Dongmei Zhang. 2020. You impress me: Dialogue generation via mutual persona perception. arXiv preprint arXiv:2004.05388.
Yang Liu, Chengjie Sun, Lei Lin, and Xiaolong Wang. 2016. Learning natural language inference using bidirectional LSTM model and inner-attention. arXiv preprint arXiv:1605.09090.
Qiao Qian, Minlie Huang, Haizhou Zhao, Jingfang Xu, and Xiaoyan Zhu. 2017. Assigning personality/identity to a chatting machine for coherent conversation generation. arXiv preprint arXiv:1706.02861.
Iulian Vlad Serban, Ryan Lowe, Peter Henderson, Laurent Charlin, and Joelle Pineau. 2015. A survey of available corpora for building data-driven dialogue systems. arXiv preprint arXiv:1512.05742.
Iulian Vlad Serban, Alessandro Sordoni, Yoshua Bengio, Aaron C Courville, and Joelle Pineau. 2016. Building end-to-end dialogue systems using generative hierarchical neural network models. In AAAI, volume 16, pages 3776-3784.
Lifeng Shang, Zhengdong Lu, and Hang Li. 2015. Neural responding machine for short-text conversation. arXiv preprint arXiv:1503.02364.
Haoyu Song, Wei-Nan Zhang, Yiming Cui, Dong Wang, and Ting Liu. 2019a. Exploiting persona information for diverse generation of conversational responses. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pages 5190-5196.
Haoyu Song, Wei-Nan Zhang, Jingwen Hu, and Ting Liu. 2019b. Generating persona consistent dialogues by exploiting natural language inference. arXiv preprint arXiv:1911.05889.
Hui Su, Xiaoyu Shen, Rongzhi Zhang, Fei Sun, Pengwei Hu, Cheng Niu, and Jie Zhou. 2019. Improving multi-turn dialogue modelling with utterance ReWriter. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 22-31, Florence, Italy. Association for Computational Linguistics.
Yixuan Su, Yan Wang, Simon Baker, Deng Cai, Xiaojiang Liu, Anna Korhonen, and Nigel Collier. 2020. Prototype-to-style: Dialogue generation with style-aware editing on retrieval memory. arXiv preprint arXiv:2004.02214.
Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, et al. 2015. End-to-end memory networks. In Advances in neural information processing systems, pages 2440-2448.
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104-3112.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998-6008.
Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. 2008. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning, pages 1096-1103. ACM.
Oriol Vinyals and Quoc Le. 2015. A neural conversational model. arXiv preprint arXiv:1506.05869.
Sean Welleck, Jason Weston, Arthur Szlam, and Kyunghyun Cho. 2019. Dialogue natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3731-3741, Florence, Italy. Association for Computational Linguistics.
Jason Weston, Emily Dinan, and Alexander Miller. 2018. Retrieve and refine: Improved sequence generation models for dialogue. In Proceedings of the 2018 EMNLP Workshop SCAI: The 2nd International Workshop on Search-Oriented Conversational AI, pages 87-92.
Yu Wu, Furu Wei, Shaohan Huang, Yunli Wang, Zhoujun Li, and Ming Zhou. 2019. Response generation by context-aware prototype editing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 7281-7288.
5830
Semin Yavuz, Abhinav Rastogi, Guan-Lin Chao, and Dilek Hakkani-Tur. 2019. Deepcopy: Grounded response generation with hierarchical pointer networks. arXiv preprint arXiv:1908.10731.
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204-2213, Melbourne, Australia. Association for Computational Linguistics.
Wei-Nan Zhang, Qingfu Zhu, Yifa Wang, Yanyan Zhao, and Ting Liu. 2019. Neural personalized response generation as domain adaptation. World Wide Web, 22(4):1427-1446.
Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 654-664.
Yinhe Zheng, Guanyi Chen, Minlie Huang, Song Liu, and Xuan Zhu. 2019a. Personalized dialogue generation with diversified traits. arXiv preprint arXiv:1901.09672.
Yinhe Zheng, Rongsheng Zhang, Xiaoxi Mao, and Minlie Huang. 2019b. A pre-training based personalized dialogue generation model with person-sparse data. arXiv preprint arXiv:1911.04700.
5831