Incorporating Commonsense Knowledge into Abstractive Dialogue Summarization via Heterogeneous Graph Networks (2021)

Abstract: Abstractive dialogue summarization is the task of capturing the highlights of a dialogue andrewriting them into a concise version. In this paper we present a novel multi-speaker dialogue summarizer to demonstrate how large-scale commonsense knowledge can facilitate dialogue un-derstanding and summary generation. In detail we consider utterance and commonsense knowl-edge as two different types of data and design a Dialogue Heterogeneous Graph Network (D-HGN) for modeling both information. Meanwhile we also add speakers as heterogeneous nodes to facilitate information flow. Experimental results on the SAMSum dataset show that our modelcan outperform various methods. We also conduct zero-shot setting experiments on the Argu-mentative Dialogue Summary Corpus the results show that our model can better generalized tothe new domain.

## introduction automatic summarization is a fundamental task in natural language processing, which aims to condense the original input into a shorter version covering salient information and has been continuously studied for decades @xcite . recently, online multi-speaker dialogue/meeting has become one of the most important ways for people to communicate with each other in their daily works. especially due to the spread of covid-19 worldwide, people are more dependent on online communication. in this paper, we focus on dialogue summarization, which can help people quickly grasp the core content of the dialogue without reviewing the complex dialogue context. in 10 minutes tom will give him a lift to work. ## heterogeneous dialogue graph construction in this section, we describe the graph notation and the graph construction process, which consists of three steps, including (1) utterance-knowledge bipartite graph construction, (2) speaker-utterance bipartite graph construction and (3) heterogeneous dialogue graph construction. ## dialogue heterogeneous graph network in this section, we describe the details of our dialogue heterogeneous graph network (d-hgn), including three components: node encoder, graph encoder and pointer decoder. the model is shown in figure 3 . ## related work previous works used feature engineering @xcite , template-based @xcite and graphbased @xcite methods for extractive dialogue summarization. although extractive methods are widely used, the results tend to be incoherent and poorly readable. therefore, current works mainly focus on abstractive methods, which can produce more readable and fluency summaries. they tend to incorporate additional auxiliary information to help better modeling the dialogue. goo and chen (2018) incorporated dialogue acts to model the interactive status of the meeting. @xcite tackled the problem of customer service summarization, which first produced a sequence of pre-defined keywords then generated the summary. @xcite generated summaries for nurse-patient conversation by incorporating topic information. @xcite first removed useless utterances by utilizing discourse labels and then generated summaries. @xcite combined vision and textual features in a unified hierarchical attention framework to generate meeting summaries. @xcite employed a hierarchical transformer framework and incorporated part-of-speech and entity information for meeting summarization. chen and yang (2020) incorporated topic and stage information to model the dialogue. @xcite used topic words to alleviate the factual inconsistency problem. @xcite used dialogue discourse to model the interaction between utterances. in this paper, we facilitate dialogue summarization task by incorporating commonsense knowledge and further model utterances, commonsense knowledge and speakers as heterogeneous data. ## conclusion in this paper, we improve abstractive dialogue summarization by incorporating commonsense knowledge. we first construct a heterogeneous dialogue graph by introducing knowledge from a large-scale commonsense knowledge base. then we present a dialogue heterogeneous graph network (d-hgn) for this task by viewing utterances, knowledge and speakers in the graph as heterogeneous nodes. we additionally design two modules named message fusion and node embedding to facilitate information flow. experiments on the samsum dataset show the effectiveness of our model that can outperform various methods. zero-shot setting experiments on the argumentative dialogue summary corpus show that our model can better generalized to the new domain.