SlowGuess/ACL_2020_OCR huggingface dataset:
In Table 2, Dist-1, Dist-2, and Ent-4 measure the word diversity of generated responses and the rest of metrics measure the novelty by comparing the generated response with the user utterance. ConceptFlow has a good balance in generating novel
and diverse responses. GPT-2's responses are more diverse, perhaps due to its sampling mechanism during decoding, but are less novel and on-topic compared to those from ConceptFlow.
Human Evaluation. The human evaluation focuses on two aspects: appropriateness and informativeness. Both are important for conversation systems (Zhou et al., 2018a). Appropriateness evaluates if the response is on-topic for the given utterance; informativeness evaluates systems' ability to provide new information instead of copying from the utterance (Zhou et al., 2018a). All responses of sampled 100 cases are selected from four methods with better performances: CCM, GPT-2 (conv), ConceptFlow, and Golden Response. The responses are scored from 1 to 4 by five judges (the higher the better).
Table 3 presents Average Score and Best@1 ratio from human judges. The first is the mean of five judges; the latter calculates the fraction of judges that consider the corresponding response the best among four systems. ConceptFlow outperforms all other models in all scenarios, while only using  $30\%$  of parameters compared to GPT-2. This demonstrates the advantage of explicitly modeling conversation flow with structured semantics.
The agreement of human evaluation is tested to demonstrate the authenticity of evaluation results. We first sample 100 cases randomly for our human evaluation. Then the responses from four better
conversation systems, CCM, GPT-2 (conv), ConceptFlow and Golden Responses, are provided with a random order. A group of annotators are asked to score each response ranged from 1 to 4 according to the quality on two testing scenarios, appropriateness and informativeness. All annotators have no clues about the source of generated responses.
The agreement of human evaluation for CCM, GPT-2 (conv) and ConceptFlow are presented in Table 4. For each case, the response from ConceptFlow is compared to the responses from two baseline models, CCM and GPT-2 (conv). The comparison result is divided into three categories: win, tie and loss. Then the human evaluation agreement is calculated with Fleiss' Kappa  $(\kappa)$ . The  $\kappa$  value ranges from 0.21 to 0.40 indicating fair agreement, which confirms the quality of human evaluation.
Both automatic and human evaluations illustrate the effectiveness of ConceptFlow. The next experiment further studies the effectiveness of multi-hop concepts in ConceptFlow.
5.2 Effectiveness of Multi-hop Concepts
This part explores the role of multi-hop concepts in ConceptFlow. As shown in Figure 3, three experiments are conducted to evaluate the performances of concept selection and the quality of generated responses with different sets of concepts.
This experiment considers four variations of outer concept selections. Base ignores two-hop concepts and only considers the central concepts. Rand, Distract, and Full add two-hop concepts in three different ways: Rand selects concepts randomly, Distract selects all concepts that appear in the golden response with random negatives (distractors), and Full is our ConceptFlow (select) that selects concepts by learned graph attentions.
As shown in Figure 3(a), Full covers more golden concepts than Base. This aligns with our motivation that natural conversations do flow from central concepts to multi-hop ones. Compared to Distract setting where all ground truth two-hop concepts are added, ConceptFlow (select) has slightly less coverage but significantly reduces the number of two-hop concepts.
The second experiment studies the model's ability to generate ground truth concepts, by comparing the concepts in generated responses with those in ground truth responses. As shown in Figure 3(b), though Full filtered out some golden two
Depth	Amount	Golden Coverage
Ratio	Number
Zero-hop	5.8	9.81%	0.579
+ One-hop	98.6	38.78%	2.292
+ Two-hop	880.8	61.37%	3.627
+ Three-hop	3769.1	81.58%	4.821
ConceptFlow	198.6	52.10%	3.075
Table 5: Statistics of Concept Graphs with different hops, including the total Amount of connected concepts, the Ratio and Number of covered golden concepts (those appear in ground truth responses). ConceptFlow indicates the filtered two-hop graph.
hop concepts, it outperforms other variations by large margins. This shows ConceptFlow's graph attention mechanisms effectively leverage the pruned concept graph and generate high-quality concepts when decoding.
The high-quality latent concept flow leads to better modeling of conversations, as shown in Figure 3(c). Full outperforms Distract in their generated responses' token level perplexity, even though Distract includes all ground truth two-hop concepts. This shows that "negatives" selected by ConceptFlow, while not directly appear in the target response, are also on-topic and include meaningful information, as they are selected by graph attentions instead of random.
More studies of multi-hop concept selection strategies can be found in Appendix A.2.
5.3 Hop Steps in Concept Graph
This experiment studies the influence of hop steps in the concept graph.
As shown in Table 5, the Number of covered golden concepts increases with more hops. Compared to zero-hop concepts, multi-hop concepts cover more golden concepts, confirming that conversations naturally shift to multi-hop concepts: extending the concept graph from one-hop to two-hop improves the recall from  $39\%$  to  $61\%$ , and to three-hop further improves to  $81\%$ .
However, at the same time, the amounts of the concepts also increase dramatically with multiple hops. Three hops lead to 3,769 concepts on average, which are  $10\%$  of the entire graph we used. In this work, we choose two-hop, as a good balance of coverage and efficiency, and used ConceptFlow (select) to filter around 200 concepts to construct the pruned graph. How to efficiently and effectively leverage more distant concepts in the graph is reserved for future work.
(a) Golden Concept Coverage.
(b) Response Concept Generation.
(c) Response Token Generation.
Figure 3: Comparisons of Outer Concept Selection Methods. Base only considers the central concepts and ignores two-hop concepts. Rand randomly selects two-hop concepts. Distract incorporates golden concepts in the response with random negatives (distractors). Full chooses two-hop concepts with ConceptFlow's graph attention.
Figure 4: Case Study (Best viewed in color). Left: Attention flow in commonsense concept graph, where zero-hop concepts, one-hop concepts and two-hop concepts are highlighted. Right: Attention scores over all concepts. Darker green indicates higher attention scores.
(a) Central Concept.
Figure 5: Distribution of Attention Score. The distributions of Overall (all concepts of the certain part), Golden (concepts in the golden response) and Zero-hop (concepts appear in the post) are presented. The attention score is calculated by scaling the mean of attention scores of  $n$  step decoding.
(b) Two-hop Concept.
Some cases from three conversation models are listed in Table 6. Responses from CCM may repeat the same contents as it does not explicitly model the traverse in the concept space. For example,
the responses from the first and third cases always repeat "I'm not sure". On the other hand, GPT-2 generates more fluent responses compared to CCM. Nevertheless, some cases from GPT-2 merely copy contents or concepts from the given post. For example, for the third case, GPT-2 (conv) mainly discusses the concept music.
In comparison, the generated responses from our ConceptFlow are more fluent and informative than those from both CCM and GPT-2. For example, in the third case, ConceptFlow brings associated concepts "sound" and "check" to the response generation, hopping from the grounded concepts "music" and "advice". Introducing these multi-hop concepts effectively improves the informativeness and diversity of generated responses.
Figure 4 presents a case study of ConceptFlow. The attention score  $\beta^{e_i}$  and  $\gamma^f$  are presented in the form of color intensity. The "championship" of zero-hop, "fan" of one-hop and "team" of two-hop receive more attention than others and are used to
#1 Post	actually i stayed at the building right next to the lighthouse . i believe it was a boiler room . another group was already sleeping at the lighthouse .
CCM	i ’m not sure if you ’re joking , but i ’m not sure if you ’re talking about the lighthouse .
GPT-2 (conv)	i ’m not sure if it was a blanket or a blanket .
ConceptFlow	i ’m pretty sure it was a room with a door .
good luck , i ’m sure there are some good things in the house .
#2 Post	at this point it may be more beneficial for him to keep playing as much as he can before the next season . trick his body into thinking he is still in the season .
CCM	i think he ’s just going to be a great player .
GPT-2 (conv)	i ’m sure he ’ll be fine . he ’s a great player , but he ’s not a top 10 player .
ConceptFlow	he ’s been playing the same game for the last two years . he ’s been playing the same way .
#3 Post	i ’ve known how to change the music , it does n’t feel natural . thanks for the advice though .
CCM	i ’m not sure if you ’re joking , but i ’m not sure if you ’re being sarcastic or not . i ’m not sure if you ’re being sarcastic or not , but i ’m not sure if you ’re being sarcastic or not . i ’m pretty sure it ’s just a video .
GPT-2 (conv)	i ’ve been playing with a lot of different music and it ’s always been the same .
ConceptFlow	i did n’t even notice the natural sound . i ’ll check it out !
#4 Post	are milan and inter no longer top clubs given that they currently do n’t have the finances to compete with the best ?
CCM	milan is n’t a top club , but they are n’t top clubs . they are n’t the only ones who are n’t in the world .
GPT-2 (conv)	i think they are . they are a club that has been in the top 4 for a long time .
ConceptFlow	the best part of the team is the way they are , and we do n’t want to play the best team in the country .
#5 Post	no i ’ve never felt like i had friends whoops
CCM	i ’m sorry , i feel you .
GPT-2 (conv)	i ’ve never felt like i had friends whoops . i ’ve never felt like i ’ve had friends whoops .
ConceptFlow	you ’re gonna have to call your friend !
Table 6: Some Cases Generated by Different Models. Tokens from zero concepts, one-hop concepts and two-hop concepts are highlighted.
generate the response. The concept flow from "fans" to "fan" models the concept shift from user post to response. The concept flow from "fan" to "team" further describes the concept shift in response generation. In addition, some concepts, such as "win" and "pretty", share higher attention and may help to understand the one-hop concepts, and are filtered out when generating response by the gate  $\sigma^{*}$  according to the relevance with conversation topic.
5.5 Learned Attentions on Concepts
This experiment studies the learned attention of ConceptFlow on different groups of concepts. We consider the average attention score  $(\beta$  for central concepts and  $\alpha$  (Appendix A.4) for two-hop concepts) from all decoding steps. The probability density of the attention is plotted in Figure 5.
Figure 5(a) shows the attention weights on central concepts. ConceptFlow effectively attends more on golden and zero-hop concepts, which include more useful information. The attention on two-hop concepts are shown in Figure 5(b). ConceptFlow attends slightly more on the Golden two-hop concepts than the rest two-hop ones, though the margin is smaller—the two-hop concepts are already filtered down to high-quality ones in the ConceptFlow (select) step.
ConceptFlow models conversation structure explicitly as transitions in the latent concept space, in order to generate more informative and meaningful responses. Our experiments on Reddit conversations illustrate the advantages of ConceptFlow over previous conversational systems. Our studies confirm that ConceptFlow's advantages come from the high coverage latent concept flow, as well as its graph attention mechanism that effectively guides the flow to highly related concepts. Our human evaluation demonstrates that ConceptFlow generates more appropriate and informative responses while using much fewer parameters.
In future, we plan to explore how to combine knowledge with pre-trained language models, e.g. GPT-2, and how to effectively and efficiently introduce more concepts in generation models.
Houyu Zhang, Zhenghao Liu and Zhiyuan Liu is supported by the National Key Research and Development Program of China (No. 2018YFB1004503) and the National Natural Science Foundation of China (NSFC No. 61772302, 61532010). We thank Hongyan Wang, Shuo Wang, Kaitao Zhang, Si Sun, Huimin Chen, Xuancheng Huang, Zeyun Zhang, Zhenghao Liu and Houyu Zhang for human evaluations.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of ICLR.
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Advances in neural information processing systems, pages 2787-2795.
Hongshen Chen, Xiaorui Liu, Dawei Yin, and Jiliang Tang. 2017. A survey on dialogue systems: Recent advances and new frontiers. SIGKDD Explorations, pages 25-35.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL, pages 4171-4186.
George Doddington. 2002. Automatic evaluation of machine translation quality using n-gram cooccurrence statistics. In Proceedings of the second international conference on Human Language Technology Research, pages 138-145.
Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. 2019. Unified language model pre-training for natural language understanding and generation. In Proceedings of NeurIPS, pages 13042-13054.
Hao Fang, Hao Cheng, Maarten Sap, Elizabeth Clark, Ari Holtzman, Yejin Choi, Noah A. Smith, and Mari Ostendorf. 2018. Sounding Board: A user-centric and content-driven social chatbot. In Proceedings of NAACL, pages 96-100.
Michel Galley, Chris Brockett, Xiang Gao, Bill Dolan, and Jianfeng Gao. 2018. End-to-end conversation modeling: Moving beyond chitchat.
Xiang Gao, Sungjin Lee, Yizhe Zhang, Chris Brockett, Michel Galley, Jianfeng Gao, and Bill Dolan. 2019. Jointly optimizing diversity and relevance in neural response generation. In Proceedings of NAACL, pages 1229-1238.
Marjan Ghazvininejad, Chris Brockett, Ming-Wei Chang, Bill Dolan, Jianfeng Gao, Wen-tau Yih, and Michel Galley. 2018. A knowledge-grounded neural conversation model. In Proceedings of AAAI, pages 5110-5117.
Barbara J. Grosz and Candace L. Sidner. 1986. Attention, intentions, and the structure of discourse. Computational Linguistics, pages 175-204.
Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O.K. Li. 2016. Incorporating copying mechanism in sequence-to-sequence learning. In Proceedings of ACL, pages 1631-1640.
Hiroaki Hayashi, Zecong Hu, Chenyan Xiong, and Graham Neubig. 2019. Latent relation language models. arXiv preprint arXiv:1908.07690.
Ari Holtzman, Jan Buys, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. arXiv preprint arXiv:1904.09751.
Alon Lavie and Abhaya Agarwal. 2007. METEOR: An automatic metric for MT evaluation with high levels of correlation with human judgments. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 228-231.
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016a. A diversity-promoting objective function for neural conversation models. In Proceedings of NAACL, pages 110-119.
Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. 2016b. Deep reinforcement learning for dialogue generation. In Proceedings of EMNLP, pages 1192-1202.
Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74-81.
Shuman Liu, Hongshen Chen, Zhaochun Ren, Yang Feng, Qun Liu, and Dawei Yin. 2018. Knowledge diffusion for neural dialogue generation. In Proceedings of the ACL, pages 1489-1498.
Robert Logan, Nelson F. Liu, Matthew E. Peters, Matt Gardner, and Sameer Singh. 2019. Barack's wife hillary: Using knowledge graphs for fact-aware language modeling. In Proceedings of ACL, pages 5962-5971.
Yinong Long, Jianan Wang, Zhen Xu, Zongsheng Wang, Baoxun Wang, and Zhuoran Wang. 2017. A knowledge enhanced generative conversational service agent. In DSTC6 Workshop.
Seungwhan Moon, Pararth Shah, Anuj Kumar, and Rajen Subba. 2019. OpenDialKG: Explanable conversational reasoning with attention-based walks over knowledge graphs. In Proceedings of ACL, pages 845-854.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of ACL, pages 311-318.
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of EMNLP, pages 1532-1543.
Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. In Proceedings of Technical report, OpenAI.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog.
Corbin Rosset, Chenyan Xiong, Xia Song, Daniel Campos, Nick Craswell, Saurabh Tiwary, and Paul Bennett. 2020. Leading conversational search by suggesting useful questions. In Proceedings of The Web Conference 2020, pages 1160-1170.
Iulian Vlad Serban, Alessandro Sordoni, Yoshua Bengio, Aaron C. Courville, and Joelle Pineau. 2016. Building end-to-end dialogue systems using generative hierarchical neural network models. In Proceedings of AAAI, pages 3776-3784.
Lifeng Shang, Zhengdong Lu, and Hang Li. 2015. Neural responding machine for short-text conversation. In Proceedings of ACL, pages 1577-1586.
Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of general knowledge. In Proceedings of AAAI, pages 4444-4451.
Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Kathryn Mazaitis, Ruslan Salakhutdinov, and William Cohen. 2018. Open domain question answering using early fusion of knowledge bases and text. In Proceedings of EMNLP, pages 4231-4242.
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of NIPS, pages 3104-3112.
Jianheng Tang, Tiancheng Zhao, Chenyan Xiong, Xiaodan Liang, Eric Xing, and Zhiting Hu. 2019. Target-guided open-domain conversation. In Proceedings of ACL, pages 5624-5634.
Oriol Vinyals and Quoc Le. 2015. A neural conversational model. arXiv preprint arXiv:1506.05869.
Pavlos Vougiouklis, Jonathon Hare, and Elena Simperl. 2016. A neural network approach for knowledge-driven response generation. In Proceedings of COLING, pages 3370-3380.
Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. 2019. Neural text generation with unlikelihood training. arXiv preprint arXiv:1908.04319.
Chien-Sheng Wu, Andrea Madotto, Ehsan Hosseini-Asl, Caiming Xiong, Richard Socher, and Pascale Fung. 2019. Transferable multi-domain state generator for task-oriented dialogue systems. In Proceedings of ACL, pages 808-819.
Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, and Wei-Ying Ma. 2017. Topic aware neural response generation. In Proceedings of AAAI, pages 3351-3357.
Zhen Xu, Bingquan Liu, Baoxun Wang, Chengjie Sun, and Xiaolong Wang. 2017. Incorporating loose-structured knowledge into conversation modeling via recall-gate LSTM. In 2017 International Joint Conference on Neural Networks, IJCNN, pages 3506-3513.
Yizhe Zhang, Michel Galley, Jianfeng Gao, Zhe Gan, Xiujun Li, Chris Brockett, and Bill Dolan. 2018. Generating informative and diverse conversational responses via adversarial information maximization. In Proceedings of NeurIPS, pages 1810-1820.
Hao Zhou, Tom Young, Minlie Huang, Haizhou Zhao, Jingfang Xu, and Xiaoyan Zhu. 2018a. Commonsense knowledge aware conversation generation with graph attention. In Proceedings of IJCAI, pages 4623-4629.
Kangyan Zhou, Shrimai Prabhumoye, and Alan W Black. 2018b. A dataset for document grounded conversations. In Proceedings of EMNLP, pages 708-713.
Wenya Zhu, Kaixiang Mo, Yu Zhang, Zhangbin Zhu, Xuezheng Peng, and Qiang Yang. 2017. Flexible end-to-end dialogue system for knowledge grounded conversation. arXiv preprint arXiv:1709.04264.
Supplementary results of the overall performance and ablation study for multi-hop concepts are presented here. More details of Central Flow Encoding and Concept Selection are also shown.
A.1 Supplementary Results for Overall Experiments
This part presents more evaluation results of the overall performance of ConceptFlow from two aspects: relevance and novelty.
Table 7 shows supplementary results on Relevance between generated responses and golden responses. ConceptFlow outperforms other baselines with large margins among all evaluation metrics. Concept-PPL is the Perplexity that calculated by the code from previous work (Zhou et al., 2018a). Zhou et al. (2018a) calculates Perplexity by considering both words and entities. It is evident that more entities will lead to a better result in terms of Concept-PPL because the vocabulary size of entities is always smaller than word vocabulary size.
More results for model novelty evaluation are shown in Table 8. These supplementary results compare the generated response with the user post to measure the repeatability of the post and generated responses. A lower score indicates better performance because the repetitive and dull response will degenerate the model performance. ConceptFlow presents competitive performance with other baselines, which illustrate our model provides an informative response for users.
These supplementary results further confirm the effectiveness of ConceptFlow. Our model has the ability to generate the most relevant response and more informative response than other models.
A.2 Supplementary Results for Multi-hop Concepts
The quality of generated responses from four two-hop concept selection strategies is evaluated to further demonstrate the effectiveness of ConceptFlow.
We evaluate the relevance between generated responses and golden responses, as shown in Table 9. Rand outperforms Base on most evaluation metrics, which illustrates the quality of generated response can be improved with more concepts included. Disttract outperforms Rand on all evaluation metrics, which indicates that concepts appearing in golden responses are meaningful and important for the conversation system to generate a more on-topic
and informative response. On the other hand, Full outperforms Distract significantly, even though not all golden concepts are included. The better performance thrives from the underlying related concepts selected by our ConceptFlow (select). This experiment further demonstrates the effectiveness of our ConceptFlow to generate a better response.
A.3 Model Details of Central Flow Encoding
This part presents the details of our graph neural network to encode central concepts.
A multi-layer Graph Neural Network (GNN) (Sun et al., 2018) is used to encode concept  $e_i \in G_{\text{central}}$  in central concept graph:
$$
\vec {g} _ {e _ {i}} = \operatorname {G N N} \left(\vec {e} _ {i}, G _ {\text {c e n t r a l}}, H\right), \tag {16}
$$
where  $\vec{e}_i$  is the concept embedding of  $e_i$  and  $H$  is the user utterance representation set.
The  $l$ -th layer representation  $\vec{g}_{e_i}^l$  of concept  $e_i$  is calculated by a single-layer feed-forward network (FFN) over three states:
$$
\vec {g} _ {e _ {i}} ^ {l} = \text {F F N} \left(\vec {g} _ {e _ {i}} ^ {l - 1} \circ \vec {p} ^ {l - 1} \circ \sum_ {r} \sum_ {e _ {j}} f _ {r} ^ {e _ {j} \rightarrow e _ {i}} \left(\vec {g} _ {e _ {j}} ^ {l - 1}\right)\right), \tag {17}
$$