Enhancing Dialogue Generation with Conversational Concept Flows (2023)

Abstract: Human conversations contain natural and reasonable topic shifts, reflected as the concept flows across utterances. Previous researches prove that explicitly modeling concept flows with a large commonsense knowledge graph effectively improves response quality. However, we argue that there exists a gap between the knowledge graph and the conversation. The knowledge graph has limited commonsense knowledge and ignores the characteristics of natural conversations. Thus, many concepts and relations in conversations are not included. To bridge this gap, we propose to enhance dialogue generation with conversational concept flows. Specifically, we extract abundant concepts and relations from natural conversations and build a new conversation-aware knowledge graph. In addition, we design a novel relation-aware graph encoder to capture the concept flows guided by the knowledge graph. Experimental results on the large-scale Reddit conversation dataset indicate that our method performs better than strong baselines, andfurther analysis verifies the effectiveness of each component. All our code and data will be publicly available after acceptance.

## introduction with the remarkable development of conversation artificial intelligence @xcite @xcite , response generation has been improved in many ways, e.g., human-like persona @xcite , empathetic expression @xcite and knowledge injection @xcite , etc. however, there still exists a series of challenges @xcite @xcite . one of the most noticeable is that humans are good at naturally switching topics during conversations, while machine-generated responses are relatively dull and tend to keep the topic still (fang et al., 2018) or throw unexpected topics @xcite . to overcome this challenge, previous works treat the topic shifts as concept flows @xcite @xcite , which means traversing in the concept 1 space along relations in an external commonsense knowledge graph. experimental results have shown that explicitly modelling concept flows effectively improves the relevance and engagingness of responses. however, we argue that there is a gap between the external knowledge graph and natural conversations. the most frequently used conceptnet foot_1 @xcite is limited to mostly (90%) taxonomic (e.g., isa) or lexical (e.g., synonym) knowledge, while contains relatively small portion of commonsense knowledge @xcite . in addition, concepts and relations in natural conversations are more colloquial and timely. thus, many concepts and relations are not included in the knowledge graph, which has also been verified in our experiments. as in figure 1 , the concept flows from "offline" to "internet" and from "harden" to "rockets" are frequently observed in human conversations, while they are both not be included in the most frequently used conceptnet. to bridge the above gap and capture more concept flows, we propose to enhance dialogue generation with conversational concept flows (eccf). specifically, we construct an enhanced knowledge graph that consists of concepts and relations in both commonsense knowledge graph and natural conversations. first, we extract new concepts as new nodes and the high-frequency relations between concepts as new edges from a large-scale dialogue corpora. then, we add these new nodes and new edges to the commonsense knowledge graph to construct a converstaion-aware knowledge graph (cakg). to effectively guide concept flows in conversations with cakg, we further propose a novel relation-aware graph encoder (rage), which reasonably considers concepts and their relations in the graph encoding process for response generation. we conduct a series of experiments on the largescale reddit conversation dataset @xcite . both automatic evaluation and human evaluation demonstrate that our method eccf improves the relevance and diversity of responses, and outperforms strong baselines. further analysis verifies the effectiveness of both cakg and rage. our research sheds light on explicitly modeling topic shifts with natural conversations. ## conclusion and future work in this paper, we argue the limitation of using external commonsense knowledge graph for response generation. to better capture topic shifts in natural conversation, we propose to enhance dialogue generation with conversational concept flows and construct conversation-aware knowledge graph. we further design a novel relation-aware graph encoder to capture the concept relations in knowledge graph. extensive experiments on the large-scale reddit dataset show the superiority of our method, and further analysis demonstrates the rationality of each component. in future work, we expect to capture more structural information from natural conversations to improve dialogue generation.