SlowGuess/ACL_2020_OCR huggingface dataset:
While all models perform more or less the same on standard metrics, their experimental results on persona metrics are quite different. All persona-aware NRG models outperform S2SA and VAE which contain no user information on the  $uRank$ , while the two variational models with user information significantly exceed the rest models. It shows that persona-aware response generators, especially those exploiting user embeddings to generate latent variables, are more sensitive on identifying users' language styles. Among all models with user modeling, our proposed PAGenerator achieves the highest  $uRank$ .
The advantage of introducing persona information into NRG is also reflected by  $uPPL$ . The
repplies given by the three models employing user embeddings are more consistent with the user's language style, which indicates that user embedding is useful in learning language style automatically in an End-to-End NRG model. By contrast, since S2SA with fact.bias focuses on learning user's bias based on only unigrams, it struggles from achieving a high  $uPPL$  which scores from bigram perspective. Moreover, comparing the performance of CVAE to Speaker Model, it appears that utilizing latent variables in standard method cannot further improve  $uPPL$ . By contrast, the two new regularizations proposed for persona modeling can help PAGenerator generating replies with more specific persona, the  $uPPL$  of which is reduced by 21.2 points compared to CVAE.
As mentioned in previous sections,  $uDistinct$  measures the diversity of the generated responses between different users. In general, latent models achieve higher  $uDistinct$  than non-latent ones as the randomness brought by the latent variables. Within latent models, the adoption of user information in CVAE only slightly improves its  $uDistinct$  compared to VAE without user specification. It indicates that user embeddings are ineffectively utilized in CVAE, and this is the motivation for us to propose new methods for variational response generator. The notable improvement in  $uDistinct$  can verify their effectiveness in exploiting persona. The cases can further demonstrate such improvements in Supplementary Material.
Besides, the comparison among baseline models is consistent with the experiments in previous studies (Li et al., 2016b; Zhou and Wang, 2018), which indicates the proposed metrics are apposite for evaluating the capability of NRG models on capturing persona.
5.2 Human Evaluation
To further evaluate the quality of generated responses from each model more subjectively, we also implement human labeling. As shown in Table 2, adjusting unigram distributions for users by fact.bias reduces the quality of generated responses. By contrast, all other models produce more high-quality replies comparing with S2SA. Moreover, responses from PAGenerator achieve the best human evaluation result, which indicates that the improvement of persona capturing of PA-Generator does not reduce correlation.
Meanwhile, in the last column, the trend of eval
Methods	BLEU	Embedding	Persona Metrics
Average	Extreme	Greedy	uRank	uPPL	uDist-1	uDist-2
S2SA (Sordoni et al., 2015)	0.29	0.834	0.615	0.666	0	200.4	0.115	0.113
fact.bias (Michel and Neubig, 2018)	0.29	0.840	0.618	0.671	0.022	202.3	0.091	0.101
Speaker Model (Liu et al., 2016)	0.31	0.837	0.621	0.674	0.023	163.6	0.183	0.199
VAE (Serban et al., 2017)	0.30	0.830	0.609	0.659	0.017	225.9	0.367	0.467
CVAE (Zhao et al., 2017)	0.31	0.836	0.616	0.668	0.039	174.5	0.377	0.486
PAGenerator	0.31	0.845	0.622	0.670	0.044	153.3	0.406	0.524
Table 1: Evaluation results on Douban corpus.  $uDist$  is the abbreviation for  $uDistinct$  in the table.
Methods	Human Evaluation
0	1	2	Persona
S2SA	60.0%	35.0%	5.0%	1.6%
fact.bias	70.0%	26.7%	3.3%	7.8%
Speaker Model	53.2%	41.6%	5.2%	9.6%
VAE	58.3%	35.0%	6.7%	3.8%
CVAE	55.0%	38.8%	7.2%	12.2%
PAGenerator	51.7%	38.3%	10.0%	13.4%
Table 2: Human labeled results upon generated responses of models trained on the Douban corpus, with the beam width of 10. The Fleiss' kappa (Fleiss and Cohen, 1973) on all annotations is around 0.65, which can be considered as "substantial agreement".
uated results on persona almost consists to those evaluated by proposed automatic evaluation metrics. The PAGenerator outperforms other models, and some particular parts of replies generated by persona-aware models can reflect the personality. Besides, due to the randomness, some responses given by S2SA and VAE are also labeled as persona-aware. However, fewer high-quality responses generated by S2SA compared to VAE, and thus, the proportion of S2SA is even lower.
5.3 Results on the Cornell Corpus
As shown in Table 3, the overall trend of the experimental results on Cornell corpus is consistent with that on Douban corpus. The models that are aware of the specified user outperform others slightly on BLEU and Embedding metrics. Regards to persona metrics, the experimental results on Cornell corpus shows two main differences: a) The Speaker Model does not perform that well on user language style detection and generation, mainly because the training data of each user is less than that in Douban corpus. It is hard to automatically model the informative user embedding via target oriented learning without guidance. By contrast, utilizing the KL divergence as the guidance in CVAE effectively improves the experi
imental results. b) Due to the individual characteristics of movie characters, the user-embedding-enhanced models generate more diverse responses for different users, specially PAGenerator.
5.4 Human Evaluation Results on the Cornell Corpus
As shown in Table 5, on the English dataset, the comparison results are almost consistent with that in Section 5.2. According to the judgment of annotators, our proposed model outperforms the others from both relevance and persona perspective. However, influenced by insufficient training conversations, the overall quality of generated responses for the Cornell queries is not as good as the ones given for the Douban corpus. We attribute this to the difference in the corpus size and the word distribution, which is described in Section 4.1. In detail, the quality of Cornell is influenced by insufficient training conversations. By contrast, the persona is reflected more obviously with the help of more templatized language styles and habits of Cornell.
To get a better intuition about how our proposed method works, we implement the ablation tests to analyze the contribution of each component of PAGenerator in persona exploitation. As illustrated in Table 4, adding the user embeddings as a part of decoder inputs brings positive improvements on all the persona-focused metrics. Without UE, the parameter size of PAGenerator reduces considerably, which is harmful to the model on fitting target data. Besides, without direct constraints from the decoder, user embeddings mainly act on reducing KL divergence rather than providing more informative latent variables. Besides, without UE, PAGenerator also significantly outperforms VAE in all metrics, which demonstrates that  $\mathcal{R}_1$  and  $\mathcal{R}_2$  are indeed useful for guiding the latent variables
Methods	BLEU	Embedding	Persona Metrics
Average	Extreme	Greedy	uRank	uPPL	uDist-1	uDist-2
S2SA (Sordoni et al., 2015)	0.32	0.787	0.503	0.679	0	44.8	0.115	0.079
fact.bias (Michel and Neubig, 2018)	0.30	0.785	0.501	0.676	0.044	39.3	0.127	0.095
Speaker Model (Liu et al., 2016)	0.33	0.796	0.510	0.681	0.056	41.7	0.228	0.225
VAE (Serban et al., 2017)	0.25	0.780	0.490	0.670	0.058	45.6	0.122	0.114
CVAE(Zhao et al., 2017)	0.28	0.800	0.502	0.689	0.085	37.0	0.223	0.251
PAGenerator	0.33	0.814	0.514	0.687	0.114	32.2	0.251	0.304
Table 3: Comparison of different approaches on the Cornell Movie Dialogues corpus.
Methods	uRank	uPPL	uDist-1/2
PAGenerator	0.114	32.2	0.251 / 0.304
w/o R1	0.117	29.6	0.209 / 0.246
w/o R2	0.118	37.2	0.251 / 0.319
w/o UE	0.063	43.5	0.149 / 0.139
Table 4: Ablation tests of PAGenerator on Cornell Movie Dialogue Corpus. "w/o" denotes PAGenerator does not contain the specific component, for example, "w/o UE" means the decoder of PAGenerator does not utilize the user embedding as input.
Methods	Human Evaluation
0	1	2	Persona
S2SA	70.6%	27.5%	1.9%	1.4%
fact.bias	72.2%	26.0%	1.8%	14.9%
Speaker Model	62.2%	35.6%	2.2%	16.9%
VAE	65.0%	31.6%	3.4%	1.1%
CVAE	61.7%	34.0%	4.3%	21.6%
PAGenerator	61.5%	33.8%	4.7%	22.8%
Table 5: Human evaluation results on the Cornell Corpus.
to model the semantics under the query and users.
Comparing the ablation results of w/o  $\mathcal{R}_1$  with w/o  $\mathcal{R}_2$ , we can conclude that both regularizations promote uRank values. However, PAGenerator w/o  $\mathcal{R}_2$  only achieves a mediocre result on uPPL, while only utilizing  $\mathcal{R}_2$  damages the model's ability in generating diverse responses for different users. We attribute this divergence to the trade-off between a) shared movie-style language between users and b) different language preferences among actors in the movie scripts. Since  $\mathcal{R}_1$  promotes the divergence of  $z$  between the specified and unspecified users, removing  $\mathcal{R}_1$  raises the difficulty for the model to generate diverse responses toward different users, reflected by the low uDistinct of w/o  $\mathcal{R}_1$ . However, promoting diversity will more or less sacrifice the model's learning on the common shared movie-style patterns, which is vital in evaluating the language cohesion. Therefore, the performance of PAGenerator only with  $\mathcal{R}_1$  on uPPL is less-than-ideal. In contrast, since  $\mathcal{R}_2$  emphasizes those patterns often used by a given user, it encourages the distribution of user information to be more aggregate. These differences explain the opposite results of w/o  $\mathcal{R}_1$  and w/o  $\mathcal{R}_2$ .
In conclusion, the user embedding is an important constraint for the PAGenerator, and  $\mathcal{R}_1$ ,  $\mathcal{R}_2$  can be considered to deploy for different purposes. Furthermore, utilizing all components of PAGenerator described in Figure 1 guarantees a more bal
anced and relatively best performance in all three evaluated persona exploiting abilities.
6.1 Persona-based Neural Models
Persona-based neural conversation models can be categorized into two major research directions. One is to directly train a model from conversational data by considering the persona information (Li et al., 2016b; Kottur et al., 2017; Wang et al., 2017; Madotto et al., 2019), while the other approach makes use of the profiles or side-information of users to generate the aligned responses (Chu et al., 2018; Qian et al., 2018; Zhang et al., 2018; Mazare et al., 2018; Song et al., 2019). The work described in this paper belongs to the first research direction. Li et al. (2016b) and Kottur et al. (2017) enrich the models by training persona vectors directly and incorporating them into the decoder. Wang et al. (2017) propose three strategies to learn the language style instead of introducing new models.
Apart from the development of the Person-based NRG models, recent researches also attempt to incorporate persona into neural machine translators. Michel and Neubig (2018) propose to learn speaker-specific parameters for the bias term in the output to promote user preferring unigrams, and Wuebker et al. (2018) introduce offset tensors to perform fine-tuning for each user.
6.2 Variational Response Generator
The variational response generators have drawn much attention recently, due to the observation that it can be flexible to include the effect from conditions based on its Bayesian architecture (Zhao et al., 2017; Shen et al., 2017) and naturally promote diversity by involving sampling in the generate stage (Serban et al., 2017; Du et al., 2018; Shen et al., 2018). Zhao et al. (2017) and Shen et al. (2017) introduce frameworks taking various conditions to influence the model learning. Afterwards, Zhou and Wang (2018) include the emoji into the variational NRG model to generate responses with particular emotions. Actually, these models (Zhao et al., 2017; Shen et al., 2017; Zhou and Wang, 2018) can also be deployed to the persona-aware response generation scenario. The main difference is that the speaker of the response is unpredictable based on the query. Thus, we have introduced the architecture proposed by Zhao et al. (2017) and modified it to adapt to the persona-aware generation, for the meaningful comparison. Especially, Song et al. (2019) have utilized persona information into the CVAE architecture, except they focus on modeling and copying users' explicit profiles.
In this paper, we proposed a variational neural network to model the conversation as well as the persona of users. On the basis of the network, two regularization terms are designed to guide the model in emphasizing the importance of the hidden user information. In addition, to better reflect the persona characteristics of the response generation model, three metrics have been introduced to quantify the level of persona of the generated responses. Experimental results show that our approach significantly outperforms other baseline models and the proposed metrics are effective in evaluating the capabilities of models on generating persona-aware responses.
This work was supported in part by the National Natural Science Foundation of China (Grant No. 61672555), and the Science and Technology Development Fund, Macau SAR (Grant No. 0101/2019/A2). We sincerely thank the anonymous reviewers for their thorough reviewing and valuable suggestions.
Rami Al-Rfou, Marc Pickett, Javier Snader, Yunhsuan Sung, Brian Strope, and Ray Kurzweil. 2016. Conversational contextual cues: The case of personalization and history for response ranking. arXiv preprint arXiv:1606.00372.
Samuel R Bowman, Luke Vilnis, Oriol Vinyals, Andrew M Dai, Rafal Jozefowicz, and Samy Bengio. 2016. Generating sentences from a continuous space. CoNLL 2016, page 10.
Eric Chu, Prashanth Vijayaraghavan, and Deb Roy. 2018. Learning personas from dialogue with attentive memory networks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2638-2646.
Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C Courville, and Yoshua Bengio. 2015. A recurrent latent variable model for sequential data. In Advances in neural information processing systems, pages 2980-2988.
Cristian Danescu-Niculescu-Mizil and Lillian Lee. 2011. Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs. In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, ACL 2011.
Jiachen Du, Wenjie Li, Yulan He, Ruifeng Xu, Li-dong Bing, and Xuan Wang. 2018. Variational autoregressive decoder for neural response generation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3154-3163.
Joseph L Fleiss and Jacob Cohen. 1973. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and psychological measurement, 33(3):613-619.
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
Diederik P Kingma and Max Welling. 2013. Autoencoding variational bayes. arXiv preprint arXiv:1312.6114.
Satwik Kottur, Xiaoyu Wang, and Vitor Carvalho. 2017. Exploring personalized neural conversational models. In *IJCAI*, pages 3728-3734.
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016a. A diversity-promoting objective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 110-119.
Jiwei Li, Michel Galley, Chris Brockett, Georgios Spithourakis, Jianfeng Gao, and Bill Dolan. 2016b. A persona-based neural conversation model. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 994-1003. Association for Computational Linguistics.
Bingquan Liu, Zhen Xu, Chengjie Sun, Baoxun Wang, Xiaolong Wang, Derek F Wong, and Min Zhang. 2018. Content-oriented user modeling for personalized response ranking in chatbots. IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), 26(1):122-133.
Chia-Wei Liu, Ryan Lowe, Iulian V Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. arXiv preprint arXiv:1603.08023.
Andrea Madotto, Zhaojiang Lin, Chien-Sheng Wu, and Pascale Fung. 2019. Personalizing dialogue agents via meta-learning. In Proceedings of the 57th Conference of the Association for Computational Linguistics, pages 5454-5459.
Pierre-Emmanuel Mazare, Samuel Humeau, Martin Raison, and Antoine Bordes. 2018. Training millions of personalized dialogue agents. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2775-2779.
Paul Michel and Graham Neubig. 2018. Extreme adaptation for personalized neural machine translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 312-318. Association for Computational Linguistics.
Lili Mou, Yiping Song, Rui Yan, Ge Li, Lu Zhang, and Zhi Jin. 2016. Sequence to backward and forward sequences: A content-introducing approach to generative short-text conversation. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3349-3358.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311-318. Association for Computational Linguistics.
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532-1543.
Qiao Qian, Minlie Huang, Haizhou Zhao, Jingfang Xu, and Xiaoyan Zhu. 2018. Assigning personality/profile to a chatting machine for coherent conversation generation. In *IJCAI*, pages 4279–4285.
Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron C Courville, and Yoshua Bengio. 2017. A hierarchical latent variable encoder-decoder model for generating dialogues. In AAAI, pages 3295-3301.
Xiaoyu Shen, Hui Su, Vera Demberg, and Shuzi Niu. 2018. Improving variational encoder-decoders in dialogue generation. national conference on artificial intelligence, pages 5456-5463.
Xiaoyu Shen, Hui Su, Yanran Li, Wenjie Li, Shuzi Niu, Yang Zhao, Akiko Aizawa, and Guoping Long. 2017. A conditional variational framework for dialog generation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 504-509.
Haoyu Song, Weinan Zhang, Yiming Cui, Dong Wang, and Ting Liu. 2019. Exploiting persona information for diverse generation of conversational responses. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, pages 5190-5196.
Yan Song, Shuming Shi, Jing Li, and Haisong Zhang. 2018. Directional skip-gram: Explicitly distinguishing left and right context for word embeddings. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 175–180.
Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, and Bill Dolan. 2015. A neural network approach to context-sensitive generation of conversational responses. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 196-205.
Di Wang, Nebojsa Jojic, Chris Brockett, and Eric Nyberg. 2017. Steering output style and topic in neural response generation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2140-2150.
Zongsheng Wang, Yunzhi Bai, Bowen Wu, Zhen Xu, Zhuoran Wang, and Baoxun Wang. 2018. A prospective-performance network to alleviate myopia in beam search for response generation. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3608-3618.
Zongsheng Wang, Zhuoran Wang, Yinong Long, Jianan Wang, Zhen Xu, and Baoxun Wang. 2019. Enhancing generative conversational service agents with dialog history and external knowledge. Computer Speech & Language, 54:71-85.
Joern Wuebker, Patrick Simianer, and John DeNero. 2018. Compact personalized models for neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 881-886.
Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, and Wei-Ying Ma. 2017. Topic aware neural response generation. In Thirty-First AAAI Conference on Artificial Intelligence.
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204-2213. Association for Computational Linguistics.
Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 654-664.
Xianda Zhou and William Yang Wang. 2018. Mojitalk: Generating emotional responses at scale. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1128-1137. Association for Computational Linguistics.
A Details of Human Evaluation
A.1 Labeling Dataset Preparation
For each model with a given query set, three generated responses for each query are randomly sampled from the results given by the beam search with a beam size of 10. Then, a total of 3,000 query-response pairs are prepared for labeling.
A.2 Labeling Criterion of Relevance
The labeling criterion for judging the relevance between the response and the given query is described as follows:
0: the quality of response is poor, it is either irrelevant to the query, or grammatically incorrect.
1: although the response itself is acceptable as a reply, its content is not informative and dull.
2: the response is not only relevant and grammatically correct, but also informative or interesting.
B Case Studies
As shown in Figure 2, we have selected three users whose utterances can reflect their implicit personal features. For example, the gender of user  $U3$  in the first case is probably female. The user  $U4$  in
the second case is very possible to be an animation fun. According to the conversation history of user  $U3$  in the last case, it can be inferred that the user is in the trouble of losing weight. Correspondingly, from the responses generated by PA-Genertor, we can observe that such implicit information are adopted by our proposed model to produce persona-aware results.
Figure 3 gives additional cases generated by PAGenertor, CVAE and VAE respectively oriented to the same given query. Apparently, every independent user should have his/her own linguistic and personality characteristic. Thus, the results generated for different users are expected to maintain enough diversity. According to the cases in Figure 3, it can be seen that results of PAGenertor keep obvious diversity among different individuals, indicating its better capability of capturing persona of users.