SlowGuess/NAACL_2022_OCR huggingface dataset:
Cold start is a common problem in recommender systems (Schein et al., 2002; Zhang et al., 2014; Li et al., 2021). This also applies to dialogue systems, as the partner personas are commonly missing in early turns. We conduct an analysis on the baselines and our framework when N turns are available where  $\mathrm{N} = \{1,2,3\}$ , using PERSONCHAT-ORI. As demonstrated in Figure 3, all the methods attain a better PPL when N increases, which indicates the existence of the cold start. This is also the case for the baseline with ground truth personas, and we postulate that it fails to learn how to use partner personas during cold start due to the lack of clues. Our framework effectively mitigates the cold start problem and attains the best among them for all N.
5.3 Case Study on Dialogue Response Generation
Table 2 depicts the case study for response generation using PERSONCHAT-ORI. In the first case,
Generated Partner Personas	Ground Truth Partner Personas
I am an army ranger. I secretly love my long deployments, because it gets me away from conventional life. I have a wife and two kids back in the states. I would be honored to give my life for my country. I am not afraid to die.	I am in the army. I am serving in South Korea. I was born in puerto rico. I am a violent person.
I drink protein powder with nothing but water. I like to watch mma. My prized possession is a bowie knife. I live weights, but I never do squats.	I am happy being single and alone. I only drink water. I go to the gym a days a week. I do not want children. I work in labor and delivery.
I like to watch football. My friends like watching it too. Its great fun. We drink beer and eat food.	I love watching football on Sunday. I have three dogs. My favoutie food is cheese piazza. I am a hair stylist.
Table 3: Case studies to show that our generated personas are relevant, informative and coherent.
Model	PPL↓	B	R	D-1	D-2
Gold	-	-	-	0.003	0.008
Our PPG	56.2	2.99	22.5	0.012	0.042
Gold	-	-	-	0.004	0.009
Our PPG	111	1.34	20.8	0.013	0.042
Table 4: Results for PPG. Upper for PERSONACHAT-ORI and lower for PERSONACHAT-REV. B, R and D represents BLEU, ROUGE and Distinct respectively.
our framework successfully recognizes that the partner is asking specifically for metallica. It then conditions on the generated personas to generate a much more entailed response than the baseline. The human response expresses negatively and thus seems less engaging. In the second case, our framework recognizes that the partner has a garden. It then talks about the garden rather than the irrelevant response from the baseline that we postulate is misled by the 'large' adjective in the dialogue context. The human response is potentially sarcastic if the partner is not joking, while our generation does not have such issue. For the third case, the baseline produces a response that could be potentially offensive, which could be biased by the word 'violent' in the dialogue context. In contrast, our framework recognizes the identity of the partner to generate a response without such an issue. The human response tends to raise a new topic and is less relevant. For the fourth case, we observe that the annotator sometimes converses based on the partner profile rather than his own traits. In this case, the annotator (Dialogue Context) said that he has many pets, which is not in his own traits (Gold Partner). Rather, his conversation partner expressed his passion for animals in previous dialogue contexts. We postulate that the annotator attempted to engage the conversation by conditioning his partner personas and telling a relevant joke. Our PPG can recognize this, which further tweaks the model output to talk about dogs and cats rather than
the dog only. These cases validate that leveraging partner personas is beneficial, and our framework can generate reasonable partner personas, which is not even in the ground truth.
5.4 Partner Personas Generation Results
Table 4 presents the quality measurements of the generated partner personas from our PPG with no RL. We observe that our models have much higher Distinct-N scores as the number of unique words in the generated output is much higher than the ground truth test personas. Compared to the ground truth personas that are limited sets of traits, our generator can leverage the power of pre-trained models for better diversity. The remaining metrics also report reasonable scores, suggesting the plausibility to formulate personas generation as a Seq2Seq task.
5.5 Case Study on Partner Personas Generation
Table 3 presents generated partner personas using PERSONACHAT-ORI. As depicted, our PPG can generate reasonable partner personas which are relevant to the ground truth partner personas. It sometimes gives a reasonable generation which is even not in the ground truth partner personas. In the first case, the generator successfully identifies the partner as being an army ranger. It then becomes rather positive than a violent person as given in the ground truth personas. Conditioning on such positive contents can give a positive response. In the second case, it recognizes the partner as a gym person, and imagines that the partner drinks protein and life weights, which is not in the ground truth personas. In the third case, the generator generates coherent personas, saying that the partner would drink beer and eat food while watching football, which is also not in the ground truth. We postulate that personas could be semantically closer to each other when they frequently co-occur in the training set. Our PPG then tends to generate more coherent
Criteria	E2E w/ Full PP	Our Framework
Appropriateness	42	58‡
Informativeness	41	59‡
Engagingness	40	60‡
Human-likeness	42	58‡
Table 5: Human evaluation results of dialogue response generation in winning percentages.  $\ddagger$  indicates the results as passing a two-tailed binomial significance test with  $p < 0.001$ .
Criteria	Ground Truth	Our Framework
Coherence	41	59‡
Interestingness	39	61‡
Engagingness	44	56†
Human-likeness	47	53
persons by learning such semantical relationship.
Since our generated personas are relevant and coherent, we postulate it as the underlying reason why our method gives a better generalization to DRG. In contrast, as demonstrated by Table 3, ground truth personas tend to be more like discrete collections of traits. This could be the reason why some of our generated partner personas could beat the ground truth, which is also supported by our human evaluation in Section 5.6. This is a potential benefit of our approach compared to sentence-level user profile extraction (Li et al., 2014; Wu et al., 2020b; Wang et al., 2021) that is upper bounded by the discrete ground truth. We present more examples in Table 8 in the Appendix.
We hired experienced annotators who have degrees relevant to English Linguistics to conduct evaluation on PERSONCHAT-ORI. For both DRG and PPG, we present a questionnaire composed of 800 questions with randomly sampled 200 test instances to three annotators who compare model outputs under A/B testing. As in Zou et al. (2021) and ACUTE-Evals (Li et al., 2020), annotators follow the criteria which we present in Appendix D.
Table 5 presents the human evaluation results on dialogue response generation. Our framework
Table 6: Human evaluation results of PPG in winning percentages.  $\dagger$  and  $\ddagger$  indicates the results as passing a two-tailed binomial significance test with  $p < {0.05}$  and  $p < {0.001}$  respectively.
Model	PPL↓	BLEU	ROUGE
Ours w/ PPG	13.80	0.8293	17.42
Ours w/ DRG	13.17	0.8391	17.36
Ours w/ PPG&DRG	13.05	1.0862	18.11
Table 7: Ablation study for our proposed framework on PERSONCHAT-ORI.
trained under RL surpasses the E2E model that leverages both training and inference ground truth partner personas from all the aspects.
Table 6 presents the human evaluation results on PPG. We observe that our PPG generates personas that are more coherent and interesting than the ground truth, which align with the facts observed in Section 5.4 and Section 5.5 indicating that our generated partner personas are more coherent and diverse.
We conduct an ablation study on PERSONACHAT-ORI as reported in Table 7 to present the performance of our framework when one of the components is frozen during RL. The result indicates that our proposed framework yields the best result when both of the components are actively trained under RL. We also notice that scaling the RL reward for either PPG or DRG by 10 leads to minor decrease in the performance. Further scaling deteriorates the quality of response generation.
Our novel framework incorporates partner personas generation into dialogue response generation. It effectively mitigates the problem that partner personas are not available in practical applications as well as the cold start problem during early conversation. The experimental results with both automatic and human evaluation demonstrate that our framework generates coherent, diverse, interesting and engaging partner personas, even compared to the ground truth partner personas. We employ reinforcement learning with a dedicatedly designed critic network that boosts the response generation by conditioning on the generated personas. Automatic and human evaluation results indicate that our response generator surpasses our competitive baselines that condition on the ground truth partner personas. Extensive case studies demonstrate that our framework can generate satisfying dialogue responses and partner personas.
This research/paper was supported by the Center for Perceptual and Interactive Intelligence (CPII) Ltd under the Innovation and Technology Commission's InnoHK scheme.
Ashutosh Baheti, Maarten Sap, Alan Ritter, and Mark Riedl. 2021. Just say no: Analyzing the stance of neural dialogue generation in offensive contexts. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4846-4862, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Deng Cai, Yan Wang, Wei Bi, Zhaopeng Tu, Xiaojiang Liu, Wai Lam, and Shuming Shi. 2019a. Skeleton-to-response: Dialogue generation guided by retrieval memory. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1219-1228, Minneapolis, Minnesota. Association for Computational Linguistics.
Deng Cai, Yan Wang, Wei Bi, Zhaopeng Tu, Xiaojiang Liu, and Shuming Shi. 2019b. Retrieval-guided dialogue response generation via a matching-to-generation framework. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1866-1875, Hong Kong, China. Association for Computational Linguistics.
Yang Deng, Yaliang Li, Fei Sun, Bolin Ding, and Wai Lam. 2021. Unified conversational recommendation policy learning via graph-based reinforcement learning. In SIGIR, pages 1431-1441.
Jun Gao, Wei Bi, Xiaojiang Liu, Junhui Li, and Shuming Shi. 2019. Generating multiple diverse responses for short-text conversation. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):6383-6390.
Jia-Chen Gu, Zhen-Hua Ling, Xiaodan Zhu, and Quan Liu. 2019. Dually interactive matching network for personalized response selection in retrieval-based chatbots. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1845-1854, Hong Kong, China. Association for Computational Linguistics.
Jia-Chen Gu, Hui Liu, Zhen-Hua Ling, Quan Liu, Zhigang Chen, and Xiaodan Zhu. 2021. Partner matters! an empirical study on fusing personas for personalized response selection in retrieval-based chatbots. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in
Information Retrieval, SIGIR 2021, page 565-574, New York, NY, USA. Association for Computing Machinery.
Hyunwoo Kim, Byeongchang Kim, and Gunhee Kim. 2020. Will I sound like me? improving persona consistency in dialogues through pragmatic self-consciousness. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 904-916, Online. Association for Computational Linguistics.
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In *ICLR (Poster)*.
Jing Yang Lee, Kong Aik Lee, and Woon Seng Gan. 2021. Generating personalized dialogue via multitask meta-learning. In Proceedings of the 25th Workshop on the Semantics and Pragmatics of Dialogue - Full Papers, Potsdam, Germany. SEMDIAL.
Dianqi Li, Qiuyuan Huang, Xiaodong He, Lei Zhang, and Ming-Ting Sun. 2018. Generating Diverse and Accurate Visual Captions by Comparative Adversarial Learning. CoRR, abs/1804.00861.
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016a. A diversity-promoting objective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 110-119, San Diego, California. Association for Computational Linguistics.
Jiwei Li, Michel Galley, Chris Brockett, Georgios Spithourakis, Jianfeng Gao, and Bill Dolan. 2016b. A persona-based neural conversation model. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 994-1003, Berlin, Germany. Association for Computational Linguistics.
Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. 2016c. Deep reinforcement learning for dialogue generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1192-1202, Austin, Texas. Association for Computational Linguistics.
Jiwei Li, Alan Ritter, and Eduard Hovy. 2014. Weakly supervised user profile extraction from twitter. volume 1, pages 165-174.
Margaret Li, Stephen Roller, Ilia Kulikov, Sean Welleck, Y-Lan Boureau, Kyunghyun Cho, and Jason Weston. 2020. Don't say that! making inconsistent dialogue unlikely with unlikelihood training. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4715-4728, Online. Association for Computational Linguistics.
Margaret Li, Jason Weston, and Stephen Roller. 2019. ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons. CoRR, abs/1909.03087.
Shijun Li, Wenqiang Lei, Qingyun Wu, Xiangnan He, Peng Jiang, and Tat-Seng Chua. 2021. Seamlessly unifying attributes and items: Conversational recommendation for cold-start users. ACM Trans. Inf. Syst., 39(4).
Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74-81, Barcelona, Spain. Association for Computational Linguistics.
Qian Liu, Yihong Chen, Bei Chen, Jian-Guang Lou, Zixuan Chen, Bin Zhou, and Dongmei Zhang. 2020. You impress me: Dialogue generation via mutual persona perception. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1417-1427, Online. Association for Computational Linguistics.
Andrea Madotto, Samuel Cahyawijaya, Genta Indra Winata, Yan Xu, Zihan Liu, Zhaojiang Lin, and Pascale Fung. 2020. Learning knowledge bases with parameters for task-oriented dialogue systems. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2372-2394, Online. Association for Computational Linguistics.
Andrea Madotto, Zhaojiang Lin, Chien-Sheng Wu, and Pascale Fung. 2019. Personalizing dialogue agents via meta-learning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5454-5459, Florence, Italy. Association for Computational Linguistics.
Bodhisattwa Prasad Majumder, Harsh Jhamtani, Taylor Berg-Kirkpatrick, and Julian McAuley. 2020. Like hiking? you probably enjoy nature: Person-grounded dialog with commonsense expansions. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9194-9206, Online. Association for Computational Linguistics.
Pierre-Emmanuel Mazare, Samuel Humeau, Martin Raison, and Antoine Bordes. 2018. Training millions of personalized dialogue agents. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2775-2779, Brussels, Belgium. Association for Computational Linguistics.
Alexander Miller, Will Feng, Dhruv Batra, Antoine Bordes, Adam Fisch, Jiasen Lu, Devi Parikh, and Jason Weston. 2017. ParlAI: A dialog research software platform. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 79-84, Copenhagen, Denmark. Association for Computational Linguistics.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311-318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners.
Stephen Robertson and Steve Walker. 1994. Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. pages 232-241.
Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Eric Michael Smith, Y-Lan Boureau, and Jason Weston. 2021. Recipes for building an open-domain chatbot. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 300-325, Online. Association for Computational Linguistics.
Homero Roman Roman, Yonatan Bisk, Jesse Thomason, Asli Celikyilmaz, and Jianfeng Gao. 2020. RMM: A recursive mental model for dialogue navigation. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1732-1745, Online. Association for Computational Linguistics.
Abdelrhman Saleh, Natasha Jaques, Asma Ghandehar-ioun, Judy Shen, and Rosalind Picard. 2020. Hierarchical reinforcement learning for open-domain dialog. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):8741-8748.
Victor Sanh, Lysandre Debut, Julien Chaumont, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. CoRR, abs/1910.01108.
Andrew I. Schein, Alexandrin Popescul, Lyle H. Ungar, and David M. Pennock. 2002. Methods and metrics for cold-start recommendations. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '02, page 253-260, New York, NY, USA. Association for Computing Machinery.
Haoyu Song, Yan Wang, Kaiyan Zhang, Wei-Nan Zhang, and Ting Liu. 2021. BoB: BERT over BERT for training persona-based dialogue models from limited personalized data. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 167-177, Online. Association for Computational Linguistics.
Haoyu Song, Yan Wang, Wei-Nan Zhang, Xiaojiang Liu, and Ting Liu. 2020. Generate, delete and rewrite: A three-stage framework for improving persona consistency of dialogue generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5821-5831, Online. Association for Computational Linguistics.
Haoyu Song, Wei-Nan Zhang, Yiming Cui, Dong Wang, and Ting Liu. 2019. Exploiting persona information for diverse generation of conversational responses. In Proceedings of the Twenty-Eighth International
Joint Conference on Artificial Intelligence, IJCAI-19, pages 5190-5196. International Joint Conferences on Artificial Intelligence Organization.
Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, NIPS'14, page 3104-3112, Cambridge, MA, USA. MIT Press.
Jack Urbanek, Angela Fan, Siddharth Karamcheti, Saachi Jain, Samuel Humeau, Emily Dinan, Tim Rocktäschel, Douwe Kiela, Arthur Szlam, and Jason Weston. 2019. Learning to Speak and Act in a Fantasy Text Adventure Game. CoRR, abs/1903.03094.
Zhilin Wang, Xuhui Zhou, Rik Koncel-Kedziorski, Alex Marin, and Fei Xia. 2021. Extracting and Inferring Personal Attributes from Dialogue. CoRR, abs/2109.12702.
Sean Welleck, Jason Weston, Arthur Szlam, and Kyunghyun Cho. 2019. Dialogue natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3731-3741, Florence, Italy. Association for Computational Linguistics.
Ronald J. Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn., 8(3-4):229-256.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumont, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, Online. Association for Computational Linguistics.
Thomas Wolf, Victor Sanh, Julien Chaumont, and Clement Delangue. 2019. TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents. CoRR, abs/1901.08149.
Bowen Wu, MengYuan Li, Zongsheng Wang, Yifu Chen, Derek F. Wong, Qihang Feng, Junhong Huang, and Baoxun Wang. 2020a. Guiding variational response generator to exploit persona. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 53-65, Online. Association for Computational Linguistics.
Chien-Sheng Wu, Andrea Madotto, Zhaojiang Lin, Peng Xu, and Pascale Fung. 2020b. Getting to know you: User attribute extraction from dialogues. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 581-589, Marseille, France. European Language Resources Association.