Improving Abstractive Dialogue Summarization with Graph Structures and Topic Words (2020)

Abstract: Recently, people have been beginning paying more attention to the abstractive dialogue summarization task. Since the information flows are exchanged between at least two interlocutors and key elements about a certain event are often spanned across multiple utterances, it is necessary for researchers to explore the inherent relations and structures of dialogue contents. However, the existing approaches often process the dialogue with sequence-based models, which are hard to capture long-distance inter-sentence relations. In this paper, we propose a Topic-word Guided Dialogue Graph Attention (TGDGA) network to model the dialogue as an interaction graph according to the topic word information. A masked graph self-attention mechanism is used to integrate cross-sentence information flows and focus more on the related utterances, which makes it better to understand the dialogue. Moreover, the topic word features are introduced to assist the decoding process. We evaluate our model on the SAMSum Corpus and Automobile Master Corpus. The experimental results show that our method outperforms most of the baselines.

## introduction due to the explosive growth of the textual information, text summarization, which is an important task in natural language processing (nlp), has been widely studied for several years. it can be categorized into two types: extractive and abstractive. extractive methods select sentences or phrases from the source text directly @xcite @xcite , while abstractive methods, which are more similar to how humans summarize texts, attempt to understand the semantic information of source text and generate new expressions as the summary. recently, neural network methods have led to encouraging results in the abstractive summarization of single-speaker documents like news, scientific publications, etc @xcite @xcite . these approaches employ a sequence-to-sequence general framework where the documents are fed into an encoder network and another decoder network learns to decode the summary. with the popularity of phone calls, e-mails, and social network applications, people share information in more different ways, which are often in the form of dialogues. different from news texts, dialogue is a dynamic information exchange flow, which is often informal, verbose and repetitive, sprinkled with false-starts, backchanneling, reconfirmations, hesitations, and speaker interruptions @xcite . besides, utterances are often turned from different interlocutors, which leads to the topic drifts, and lower information density. these problems need to be solved using natural language generation techniques with a high level of semantic understanding. some early works benchmarked the abstractive dialogue summarization task using the ami meeting corpus, which contains a wide range of annotations, including dialogue acts, topic descriptions, etc. @xcite @xcite . @xcite proposed to use the high-level topic descriptions (e.g. costing evaluation of project process) as the gold references and leveraged dialogue act signals in a neural summarization model. they assumed that dialogue acts indicated interactive signals and used these information for a better performance. because this meeting dataset has a low number of summaries and is different from real dialogues, this network can not reflect its effectiveness on dialogue summarization. customer service interaction is also a common form of dialogue, which contains questions of the user and solutions of the agent. @xcite collected a dialogue-summary dataset from the logs in the didi customer service center. they proposed a novel leader-writer network, which relies on auxiliary key point sequences to ensure the logic and integrity of dialogue summaries, and designs a hierarchical decoder. the rules of labeling the key point sequences are given by domain experts, which needs to consume a lot of human efforts. considering the lack of highquality datasets, @xcite created the samsum corpus and further investigated the problems of dialogue summary generation. they only proposed the dataset and experimented with general networks of text summarization. although some progress has been made in abstractive dialogue summarization task, previous methods do not develop specially designed solutions for dialogues, and are all dependent on sequence-to-sequence models, which can not handle the sentence-level long-distance dependency and capture the cross-sentence relations. to mitigate these issues, an intuitive way is to model the relations of sentences using the graph structures, which can break the sequential positions of dialogues and directly connect the related long-distance utterances. in this paper, we propose a topic-word guided dialogue graph attention (tgdga) network that discovers the intra-sentence and inter-sentence relations by graph neural networks, and generates summaries relied on the graph-to-sequence framework and topic words. nodes of different granularity levels represent topic word features and utterance sequence features, respectively. the edges in the graph are initialized by the linguistic information relationships between the nodes. the masking mechanism operated in the graph self-attention layer only leverages related utterances and filters out redundant utterances. the dialogue graph aggregates the useful conversation history and captures cross-sentence relations effectively. besides, we encode the topic words to the topic information representation and integrate it into the decoder, to guide the process of generation. the key contributions of this work include: ## conclusion in this paper, we propose a topic-word guided dialogue graph attention model to automatically generate summaries of dialogues with a graph-to-sequence framework. the dialogue is organized into an interaction graph, which improves context understanding for sentence-level long-dependency and builds more complex relations between utterances. the introduction of masking mechanism helps our model to select salient utterances and aware of the hierarchical structure of dialogues. we also incorporate topic words information into the summary generation process. experimental results strongly support the improvements in our proposal. furthermore, we will take the pre-trained language models into account for better encoding representations of words and expect more advanced work to be done in the area of evaluation metrics in the future.