SlowGuess/NAACL_2022_OCR huggingface dataset:
2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019, pages 7281-7288. AAAI Press.
Ziang Xiao, Michelle X. Zhou, Wenxi Chen, Huahai Yang, and Chang Yan Chi. 2020. If I hear you correctly: Building and evaluating interview chatbots with active listening skills. CoRR, abs/2002.01862.
Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, and Wei-Ying Ma. 2017. Topic aware neural response generation. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA, pages 3351-3357. AAAI Press.
Liu Yang, Junjie Hu, Minghui Qiu, Chen Qu, Jianfeng Gao, W. Bruce Croft, Xiaodong Liu, Yelong Shen, and Jingjing Liu. 2019. A hybrid retrieval-generation neural conversation model. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, Beijing, China, November 3-7, 2019, pages 1341-1350. ACM.
Hainan Zhang, Yanyan Lan, Liang Pang, Jiafeng Guo, and Xueqi Cheng. 2019a. Recosa: Detecting the relevant contexts with self-attention for multi-turn dialogue generation. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28-August 2, 2019, Volume 1: Long Papers, pages 3721-3730. Association for Computational Linguistics.
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers, pages 2204-2213. Association for Computational Linguistics.
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2019b. Dialogpt: Large-scale generative pre-training for conversational response generation. CoRR, abs/1911.00536.
Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, pages 654-664. Association for Computational Linguistics.
Yutao Zhu, Zhicheng Dou, Jian-Yun Nie, and Ji-Rong Wen. 2020. Reboot: a retrieval-boosted sequence-to-sequence model for neural response generation. Inf. Retr. J., 23(1):27-48.
Table 6: The results of other evaluation metrics on Weibo and Reddit dataset. “†” indicates that our model achieves significant improvement in t-test with  $p$ -value  $< 0.05$ . The best results are in **bold**.
BLEU-3	BLEU-4	ROUGE-1	ROUGE-2
Weibo	Seq2Seq	0.011†	0.001†	8.740†	0.373†
MMI	0.046†	0.007†	5.316†	0.105†
DialoGPT	0.114†	0.027†	9.414†	0.632†
GPMN	0.359†	0.066†	8.086†	0.753†
PerCVAE	0.466†	0.089†	7.946†	0.485†
Speaker	0.107†	0.041†	7.997†	0.155†
PersonaWAE	0.889†	0.155†	11.341†	0.358†
DHAP	1.170†	0.401†	14.131†	3.608†
MSP (Ours)	3.973	3.522	16.249	5.812
Reddit	Seq2Seq	0.007†	0.001†	3.989†	0.233†
MMI	0.007†	0.003†	3.960†	0.245†
DialoGPT	0.054†	0.010†	8.977†	0.610†
GPMN	0.039†	0.006†	4.896†	0.330†
PerCVAE	0.068†	0.009†	8.004†	0.540†
Speaker	0.021†	0.005†	4.017†	0.245†
PersonaWAE	0.029†	0.007†	8.247†	0.517†
DHAP	0.079†	0.013†	10.680†	0.697†
MSP (Ours)	0.106	0.019	11.078	0.745
ing to music. And the token refiner captures more concrete tokens from sim users' history sentences, such as "Angela Leung" and "Ember". By combing two profiles, our personalized generator gets an informative and personalized response close to ground-truth. In contrast, DialogGPT generates a fluent but meaningless response to the query.
As n-gram word overlap metrics can reflect user speaking style more accurately, we evaluate the BLEU-3/4 (Papineni et al., 2002), and the result is shown in Table 6. It is consistent with other evaluations that our model outperforms every indicator. This demonstrates that user profiles also contain speaking style information, and our model can use the information to achieve a personalized response.
To show the effect of our model more concretely, we adopt a case study, and the results are shown in Table 7. It shows that our model can extract profiles from both current and similar users and generate informative and personalized responses. Specifically, in his dialogue history, he mentioned sports H1, H2 and music H3, H4 topics. Firstly, we can select similar users who also talk about sports and music, using the user refiner. Then, as the query is related to the music topic, we extract the current and sim users' dialogue history responses about the music topic R1, R2 by the topic refiner. Furthermore, the token refiner selects some meaningful and personalized words from long sentences of reference. In this case, we can find that the token refiner extracts some compliment words (like, beautiful, enjoying) from the current user's history sentence since the current user likes listen
Table 7: A case study. Due to space limitation, we omit some user history responses and references.
History	H1: Liverpool's configuration has the life of a champion, support it!
H2: Champion Liverpool!
H3: I like to listen to music.
H4: The songs on this album are really beautiful, so worth enjoying.
Query	New album "How Am I? -The sun rises as it always does", the first single "Ember" heals the system, go listen!
Persona Reference	R1: I like to listen to music.
R2: The songs on this album are really beautiful, so worth enjoying.
Sim Reference	R1: Quite like this type of song.
R2: Angela Leung's "Ember" is really good.
Persona Profile	like, song, album, beautiful, enjoying
Sim Profile	like, song, Angela Leung, good, Ember
Response	DialoGPT: My little heart has flown.
MSP(Ours): Angela Leung's song is very good.
Golden: Angela Leung's songs must be listened to.