SlowGuess/ACL_2020_OCR huggingface dataset:
We assume that using the correct response will be conducive to knowledge selection. Then minimizing KLDivloss will make the effect of knowledge selection in the prediction stage (not use the correct response) close to that of knowledge selection with correct response. For knowledge selection, the model learns knowledge-selection strategy through minimizing the KLDivLoss between two distributions, a prior distribution  $p(k_{i}|x,g_{c})$  and a posterior distribution  $p(k_{i}|x,y,g_{c})$ . It is formulated as:
$$
p \left(k _ {i} \mid x, y, g _ {c}\right) = \frac {\exp \left(k _ {i} \cdot \mathbf {M L P} ([ x ; y ; g _ {c} ])\right)}{\sum_ {j = 1} ^ {N} \exp \left(k _ {j} \cdot \mathbf {M L P} ([ x ; y ; g _ {c} ])\right)} \tag {6}
$$
$$
p \left(k _ {i} \mid x, g _ {c}\right) = \frac {\exp \left(k _ {i} \cdot \mathbf {M L P} \left([ x ; g _ {c} ]\right)\right)}{\sum_ {j = 1} ^ {N} \exp \left(k _ {j} \cdot \mathbf {M L P} \left([ x ; g _ {c} ]\right)\right)} \tag {7}
$$
$$
L _ {K L} (\theta) = \frac {1}{N} \sum_ {i = 1} ^ {N} p \left(k _ {i} | x, y, g _ {c}\right) \log \frac {p \left(k _ {i} | x , y , g _ {c}\right)}{p \left(k _ {i} | x , g _ {c}\right)} \tag {8}
$$
In training procedure, we fuse all related knowledge information into a vector  $k_{c} = \sum_{i}p(k_{i}|x,y,g_{c}) * k_{i}$ , same as the retrieval-based method, and feed it to the decoder for response generation. In testing procedure, the fused knowledge is estimated by  $k_{c} = \sum_{i}p(k_{i}|x,g_{c}) * k_{i}$  without ground-truth responses. The decoder is implemented with the Hierarchical Gated Fusion Unit described in (Yao et al., 2017), which is a standard
GRU based decoder enhanced with external knowledge gates. In addition to the loss  $L_{KL}(\theta)$ , the generator uses the following losses:
NLL Loss: It computes the negative log-likelihood of the ground-truth response  $(L_{NLL}(\theta))$ .
BOW Loss: We use the BOW loss proposed by Zhao et al. (2017), to ensure the accuracy of the fused knowledge  $k_{c}$  by enforcing the relevancy between the knowledge and the true response. Specifically, let  $w = \mathbf{MLP}(k_c)\in \mathcal{R}^{|\bar{V}|}$ , where  $|V|$  is vocabulary size. We define:
$$
p \left(y _ {t} \mid k _ {c}\right) = \frac {\exp \left(w _ {y _ {t}}\right)}{\sum_ {v = 1} ^ {V} \exp \left(w _ {v}\right)}. \tag {9}
$$
Then, the BOW loss is defined to minimize:
$$
L _ {B O W} (\theta) = - \frac {1}{m} \sum_ {t = 1} ^ {m} \log p \left(y _ {t} \mid k _ {c}\right) \tag {10}
$$
Finally, we minimize the following loss function:
$$
L (\theta) = \alpha \cdot L _ {K L} (\theta) + \alpha \cdot L _ {N L L} (\theta) + L _ {B O W} (\theta) \tag {11}
$$
where  $\alpha$  is a trainable parameter.
We split DuRecDial into train/dev/test data by randomly sampling  $65\% / 10\% / 25\%$  data at the level of seekers, instead of individual dialogs. To evaluate the contribution of goals, we conduct an ablation study by replacing input goals with "UNK" for responding model. For knowledge usage, we conduct another ablation study, where we remove input knowledge by replacing them with "UNK".
S2S We implement a vanilla sequence-to-sequence model (Sutskever et al., 2014), which is widely used for open-domain conversation generation.
MGCG_R: Our system with automatic goal planning and a retrieval based responding model.
MGCG_G: Our system with automatic goal planning and a generation based responding model.
4The BOW loss is to introduce an auxiliary loss that requires the decoder network to predict the bag-of-words in the response to tackle the vanishing latent variable problem.
5Please see Appendix 2. for model parameter settings.
Methods↓ Metrics→	Hits@ 1/Hits@ 3	F1/ BLEU2	PPL	DIST-2	Knowledge P/R/F1
S2S-g1.- kg.	6.78% / 24.55%	23.97 / 0.065	27.31	0.011	0.275 / 0.209 / 0.216
S2S+gl.- kg.	8.03% / 27.71%	24.78 / 0.077	24.82	0.012	0.287 / 0.223 / 0.231
S2S+gl.+kg.	8.37% / 27.67%	24.66 / 0.072	23.96	0.011	0.295 / 0.239 / 0.253
MGCG-R-g1.- kg.	19.58% / 42.75%	33.22 / 0.207	-	0.171	0.344 / 0.301 / 0.306
MGCG-R+gl.- kg.	19.77% / 42.99%	33.78 / 0.223	-	0.185	0.351 / 0.322 / 0.309
MGCG-R+gl.+kg.	20.33% / 43.61%	33.93 / 0.232	-	0.187	0.349 / 0.331 / 0.316
MGCG-G-g1.- kg.	13.26% / 36.07%	33.11 / 0.189	18.51	0.037	0.386 / 0.349 / 0.358
MGCG-G+gl.- kg.	14.21% / 38.91%	35.21 / 0.213	17.78	0.049	0.393 / 0.352 / 0.351
MGCG-G+gl.+kg.	14.38% / 39.70%	36.81 / 0.219	17.69	0.052	0.401 / 0.377 / 0.383
Table 4: Automatic evaluation results.  $+(-) \mathrm{gl}$ . represents "with(without) conversational goals".  $+(-) \mathrm{kg}$ . represents "with(without) knowledge". For "S2S +gl.+kg.", we simply concatenate the goal predicted by our model, all the related knowledge and the dialog context as its input.
Methods↓ Metrics→	Turn-level results	Dialog-level results
Fluency	Appro.	Infor.	Proactivity	Goal success rate	Coherence
S2S +gl. +kg.	1.08	0.23	0.37	0.94	0.37	0.49
MGCG_R +gl. +kg.	1.98	0.60	1.28	1.22	0.68	0.83
MGCG_G +gl. +kg.	1.94	0.75	1.68	1.34	0.82	0.91
Table 5: Human evaluation results at the level of turns and dialogs.
5.3 Automatic Evaluations
Metrics For automatic evaluation, we use several common metrics such as BLEU (Papineni et al., 2002), F1, perplexity (PPL), and DISTINCT (DIST-2) (Li et al., 2016) to measure the relevance, fluency, and diversity of generated responses. Following the setting in previous work (Wu et al., 2019; Zhang et al., 2018a), we also measure the performance of all models using Hits@1 and Hits@3.<sup>6</sup> Here we let each model to select the best response from 10 candidates. Those 10 candidate responses consist of the ground-truth response generated by humans and nine randomly sampled ones from the training set. Moreover, we also evaluate the knowledge-selection capability of each model by calculating knowledge precision/recall/F1 scores as done in Wu et al. (2019).<sup>7</sup> In addition, we also report the performance of our goal planning module, including the accuracy of goal completion estimation, dialog type prediction, and dialog topic prediction.
Results Our goal planning model can achieve accuracy scores of  $94.13\%$ ,  $91.22\%$ , and  $42.31\%$  for goal completion estimation, dialog type prediction, and dialog topic prediction. The accuracy of dialog topic prediction is relatively low since the num
ber of topic candidates is very large (around 1000), leading to the difficulty of topic prediction. As shown in Table 4, for response generation, both MGCG_R and MGCG_G outperform S2S by a large margin in terms of all the metrics under the same model setting (without gl.+kg., with gl., or with gl.+kg.). Moreover, MGCG_R performs better in terms of Hits@k and DIST-2, but worse in terms of knowledge F1 when compared to MGCG_G. It might be explained by that they are optimized on different metrics. We also found that the methods using goals and knowledge outperform those without goals and knowledge, confirming the benefits of goals and knowledge as guidance information.
5.4 Human Evaluations
Metrics: The human evaluation is conducted at the level of both turns and dialogs.
For turn-level human evaluation, we ask each model to produce a response conditioned on a given context, the predicted goal and related knowledge. The generated responses are evaluated by three annotators in terms of fluency, appropriateness, informativeness, and proactivity. The appropriateness measures if the response can complete current goal and it is also relevant to the context. The informativeness measures if the model makes full use of knowledge in the response. The proactivity mea
${}^{6}$  Candidates (including golden response) are scored by PPL using the generation-based model, then candidates are sorted based on the scores, and Hits@1 and Hits@3 are calculated.
When calculating the knowledge precision/recall/F1, we compare the generated results with the correct knowledge.
<sup>8</sup>We calculate an average of F1 over all the dialogs. It might result in that the value of F1 is not between P and R.
9Please see Appendix 3. for more details.
sures if the model can successfully introduce new topics with good fluency and coherence.
For dialogue-level human evaluation, we let each model converse with a human and proactively make recommendations when given the predicted goals and related knowledge. For each model, we collect 100 dialogs. These dialogs are then evaluated by three persons in terms of two metrics: (1) goal success rate that measures how well the conversation goal is achieved, and (2) coherence that measures relevance and fluency of a dialog as a whole.
All the metrics has three grades: good(2), fair(1), bad(0). For proactivity, "2" indicates that the model introduces new topics relevant to the context, "1" means that no new topics are introduced, but knowledge is used, "0" means that the model introduces new but irrelevant topics. For goal success rate, "2" means that the system can complete more than half of the goals from goal planning module, "0" means the system can complete no more than one goal, otherwise "1". For coherence, "2"/"1"/"0" means that two-thirds/one-third/very few utterance pairs are coherent and fluent.
Results All human evaluations are conducted by three persons. As shown in Table 5, our two systems outperform S2S by a large margin, especially in terms of appropriateness, informativeness, goal success rate and coherence. In particular, S2S tends to generate safe and uninformative responses, failing to complete goals in most of dialogs. Our two systems can produce more appropriate and informative responses to achieve higher goal success rate with the full use of goal information and knowledge. Moreover, the retrieval-based model performs better in terms of fluency since its response is selected from the original human utterances, not automatically generated. But it performs worse on all the other metrics when compared to the generation-based model. It might be caused by the limited number of retrieval candidates. Finally, it can be seen that there is still much room for performance improvement in terms of appropriateness and goal success rate, which will be left as the future work.
5.5 Result Analysis
In order to further analyze the relationship between knowledge usage and goal completion, we provide the number of failed goals, completed goals, and used knowledge for each method over different dialog types in Table 6. We see that the number of
Methods→ Metrics↓Types↓	S2S +gl. +kg.	MGCG_R +gl. +kg.	MGCG_G +gl. +kg.
#Failed gl./ #Com- pleted gl.	Rec. Chitchat QA Task Overall	106/7 120/93 66/5 45/4 337/109	95/18 96/117 61/10 36/13 288/158	93/20 80/133 60/11 39/10 272/174
#Used kg.	Rec. Chitchat QA Task Overall	0 9 5 0 14	8 25 10 3 46	7 33 15 2 57
Table 6: Analysis of goal completion and knowledge usage across different dialog types.
used knowledge is proportional to goal success rate across different dialog types or different methods, indicating that the knowledge selection capability is crucial to goal completion through dialogs. Moreover, the goal of chitchat dialog is easier to complete in comparison with others, and QA and recommendation dialogs are more challenging to complete. How to strengthen knowledge selection capability in the context of multi-type dialogs, especially for QA and recommendation, is very important, which will be left as the future work.
We identify the task of conversational recommendation over multi-type dialogs, and create a dataset DuRecDial with multiple dialog types and multidomain use cases. We demonstrate usability of this dataset and provide results of state of the art models for future studies. The complexity in DuRecDial makes it a great testbed for more tasks such as knowledge grounded conversation (Ghazvininejad et al., 2018), domain transfer for dialog modeling, target-guided conversation (Tang et al., 2019a) and multi-type dialog modeling (Yu et al., 2017). The study of these tasks will be left as the future work.
We would like to thank Ying Chen for dataset annotation and thank Yuqing Guo and the reviewers for their insightful comments. This work was supported by the National Key Research and Development Project of China (No. 2018AAA0101900) and the Natural Science Foundation of China (No. 61976072).
10Please see Appendix 4. for more details.
Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Inigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gašić. 2018. MultiWOZ - a large-scale multi-domain wizard-of-Oz dataset for task-oriented dialogue modelling. In EMNLP.
Qibin Chen, Junyang Lin, Yichang Zhang, Ming Ding, Yukuo Cen, Hongxia Yang, and Jie Tang. 2019. Towards knowledge-based recommender dialog system. In ACL.
Konstantina Christakopoulou, Alex Beutel, Rui Li, Sagar Jain, and Ed H. Chi. 2018. Q and r: A two-stage approach toward interactive recommendation. In KDD.
Konstantina Christakopoulou, Katja Hofmann, and Filip Radlinski. 2016. Towards conversational recommender systems. In KDD.
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. In arXiv preprint arXiv:1412.3555.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. In arXiv preprint arXiv:1810.04805.
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2019. Wizard of wikipedia: knowledge-powered conversational agents. In Proceedings of ICLR.
Jesse Dodge, Andreea Gane, Xiang Zhang, Antoine Bordes, Sumit Chopra, Alexander H. Miller, and Arthur Szlam and Jason Weston. 2016. Evaluating prerequisite qualities for learning end-to-end dialog systems. In ICLR.
Marjan Ghazvininejad, Chris Brockett, Ming-Wei Chang, Bill Dolan, Jianfeng Gao, Wen tau Yih, and Michel Galley. 2018. A knowledge-grounded neural conversation model. In AAAI.
Dongyeop Kang, Anusha Balakrishnan, Pararth Shah, Paul Crook, Y-Lan Boureau, and Jason Weston. 2019. Recommendation as a communication game: Self-supervised bot-play for goal-oriented dialogue. In EMNLP.
Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar. Association for Computational Linguistics.
Sunhwan Lee, Robert Moore, Guang-Jie Ren, Raphael Arar, and Shun Jiang. 2018. Making personalized recommendation through conversation: Architecture design and recommendation methods. In AAAI Workshops.
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A diversity-promoting objective function for neural conversation models. In NAACL-HLT, pages 110-119.
Raymond Li, Samira Ebrahimi Kahou, Hannes Schulz, Vincent Michalski, Laurent Charlin, and Chris Pal. 2018. Towards deep conversational recommendations. In NIPS.
Nikita Moghe, Siddhartha Arora, Suman Banerjee, and Mitesh M. Khapra. 2018. Towards exploiting background knowledge for building conversation systems. In EMNLP.
Seungwhan Moon, Pararth Shah, Anuj Kumar, and Rajen Subba. 2019. Opendialkg: Explanable conversational reasoning with attention-based walks over knowledge graphs. In ACL.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In ACL, pages 311-318.
Ashwin Ram, Rohit Prasad, Chandra Khatri, Anu Venkatesh, Raefer Gabriel, Qing Liu, Jeff Nunn, Behnam Hedayatnia, Ming Cheng, Ashish Nagar, Eric King, Kate Bland, Amanda Wartick, Yi Pan, Han Song, Sk Jayadevan, Gene Hwang, and Art Pettigrue. 2018. Conversational AI: the science behind the alexa prize. In CoRR, abs/1801.03604.
Kevin Reschke, Adam Vogel, and Daniel Jurafsky. 2013. Generating recommendation dialogs by extracting information from user reviews. In ACL.
Yueming Sun and Yi Zhang. 2018. Conversational recommender system. In SIGIR.
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In NIPS.
Jianheng Tang, Tiancheng Zhao, Chengyan Xiong, Xiaodan Liang, Eric P Xing, and Zhiting Hu. 2019a. Target-guided open-domain conversation. In ACL.
Jianheng Tang, Tiancheng Zhao, Chenyan Xiong, Xiaodan Liang, Eric P. Xing, and Zhiting Hu. 2019b. Target-guided open-domain conversation. In Proceedings of ACL.
Zhuoran Wang, Hongliang Chen, Guanchun Wang, Hao Tian, Hua Wu, and Haifeng Wang. 2014. Policy learning for domain selection in an extensible multidomain spoken dialogue system. In EMNLP.
Pontus Warnestal. 2005. Modeling a dialogue strategy for personalized movie recommendations. In *The Beyond Personalization Workshop*.
Wenquan Wu, Zhen Guo, Xiangyang Zhou, Hua Wu, Xiyuan Zhang, Rongzhong Lian, and Haifeng Wang. 2019. Proactive human-machine conversation with explicit conversation goal. In ACL.
Jun Xu, Haifeng Wang, Zheng-Yu Niu, Hua Wu, and Wanxiang Che. 2020. Knowledge graph grounded goal planning for open-domain conversation generation. In AAAI.
Lili Yao, Yaoyuan Zhang, Yansong Feng, Dongyan Zhao, and Rui Yan. 2017. Towards implicit content-introducing for generative short-text conversation systems. In EMNLP.
Zhou Yu, Alexander I. Rudnicky, and Alan W. Black. 2017. Learning conversational systems that interleave task and non-task content. In *IJCAI*.
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018a. Personalizing dialogue agents: I have a dog, do you have pets too? In ACL.
Yongfeng Zhang, Xu Chen, Qingyao Ai, Liu Yang, and W. Bruce Croft. 2018b. Towards conversational search and recommendation: System ask, user respond. In CIKM.
Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. In ACL.
Hao Zhou, Chujie Zheng, Kaili Huang, Minlie Huang, and Xiaoyan Zhu. 2020. Kdconv: A chinese multi-domain dialogue dataset towards multi-turn knowledge-driven conversation. In Proceedings of ACL.
Kangyan Zhou, Shrimai Prabhumoye, and Alan W Black. 2018a. A dataset for document grounded conversations. In Proceedings of EMNLP.
Li Zhou, Jianfeng Gao, Di Li, and Heung-Yeung Shum. 2018b. The design and implementation of xiaoice, an empathetic social chatbot. In CoRR, abs/1812.08989.
1. Dataset collection process
1.1 Collection of seeker profiles/knowledge graph/task templates
Collection of seeker profile The attributes of seeker profiles are shown as follows: name, gender, age range, city of residence, occupation status, and seeker preference. Seeker preference includes : domain preference, seed entity preference, entity list rejected by the seeker, entity list accepted by the seeker.
- Name: We generate the first-name Chinese character(or last-name Chinese character) by randomly sampling Chinese characters from a set of candidate characters used as first name (or last name) for the gender of the seeker.
- Gender: We randomly select "male" or "female" as the seeker's gender.
- Age range: We randomly choose one from the 5 age ranges.
- Residential city: We randomly choose one from 55 cities in China as the seeker's residential city.
- Occupation status: We randomly choose one from "student", "worker" and "retirement" based on above age range.
- Domain preference: We randomly select one or two domains as ones that the seeker likes (e.g., movie, food), and one domain as the one that the seeker dislikes (e.g., news). It will affect the setting of task templates for this seeker.
- Seed entity preference: We randomly select one or two entities from KG entities of the domains preferred by the seeker as his/her preference at entity level. It will affect the setting of task templates for this seeker.
- Rejected entity list and accepted entity list: Both of them are empty at the beginning, and they will be updated as the conversation progresses; the two lists will affect the recommendation results of subsequent conversations to some extent.
Collection of Knowledge graph (KG) The domain of knowledge graph include stars, movies, music, news, food, POI(Point of Interest), weather.
- Stars: including the introduction, achievements, awards, comments, birthday, birthplace, height, weight, blood type, constellation, zodiac, nationality, friends, etc
- Film: including film rating, comments, region, leading role, director, category, evaluation, award, etc
Music: singer information, comments, etc
- News: including the topic, content, etc
- Food: including the name, ingredients, category, etc
- POI: including restaurant name, average price, score, order quantity, address, city, specialty, etc
- Weather: historical weather of 55 cities from July 2017 to August 2019
Collection of task templates First we manually annotate a list of around 20 high-level goal sequences as candidates. Most of these goal sequences include 3 to 5 high-level goals. Here each high-level goal contains a dialog type and a domain (not entity or chatting topic). Then for each seeker, we select appropriate high-level goal sequences from the above list, which contains the domains that fall into the seeker's preferred domain list.
To collect goal sequences at entity level, we first use the seed entities of the seeker to enrich the information of above high-level goal sequences. If the seed entities are not enough, or there is no seeds for some domains in the high-level goal sequences, we select some entities from KG for each goal domain based on embedding based similarity scores of the seed entities (of current seeker) and the candidate entity. Then we obtain goal sequences at entity level. Finally we use some rules to generate a description for each goal (e.g., which side, the seeker or the recommender, to start the dialog, how to complete the goal). Thus we have task templates for guidance of data annotation.
To introduce diverse interaction behavior for recommendation, we design some fine-grained interaction operations, e.g., the seeker may reject the initial recommendation, or mention a new topic, or ask a question about an entity, or simply accept the recommendation. Each interaction operation corresponds to a goal. We randomly sample one of above operations and insert it into the entity-level goal sequences to diversify recommendation dialogs. The entities associated with the above interaction operations are selected from the KG based on their similarity scores with current seeker's seed entities. If the entity will be accepted by the seeker as described in the task templates (including entity-level goal sequence and its description), then its similarity score with the seeker's seed entities should be relatively high. If the entity will be rejected by the seeker as described in the task templates, then its similarity score with the seeker's seed entities should be relatively low.
1.2 Dataset annotation process
We first release a small amount of data for training, and then carry out video training for annotation problems. After that, a small amount of data is released again to select the final task workers. To
ensure that at least two workers enter the task at the same time, we arrange multiple workers to log in the annotation platform. During annotation, each conversation is randomly assigned to two workers, one of whom plays the role of BOT and the other plays the role of User. Two workers conduct annotation based on the seeker profile, knowledge graph and task templates.
Due to the complexity of our task design, the quality of data annotation may have a high variance. To address this problem, we provide a strict data annotation standard to guide the workers to annotate in the way we expect them to be. After data annotation, multiple data specialists will review it. As long as one specialist thinks it does not meet the requirements, it will be marked back and re-annotation until all specialists think that it fully meets requirements.
2. Model Parameter Settings
All models are implemented using PaddlePaddle. $^{11}$  The parameters of all the modules are shown in Table 7. $^{12}$
3. Turn-level Human Evaluation Guideline
Fluency measures if the produced response itself is fluent:
- score 0 (bad): unfluent and difficult to understand.
- score 1 (fair): there are some errors in the response text but still can be understood.
- score 2 (good): fluent and easy to understand.
Appropriatness measures if the response can respond to the context: