Coreference-Aware Dialogue Summarization (2021)

Abstract: Summarizing conversations via neural approaches has been gaining research traction lately, yet it is still challenging to obtain practical solutions. Examples of such challenges include unstructured information exchange in dialogues, informal interactions between speakers, and dynamic role changes of speakers as the dialogue evolves. Many of such challenges result in complex coreference links. Therefore, in this work, we investigate different approaches to explicitly incorporate coreference information in neural abstractive dialogue summarization models to tackle the aforementioned challenges. Experimental results show that the proposed approaches achieve state-of-the-art performance, implying it is useful to utilize coreference information in dialogue summarization. Evaluation results on factual correctness suggest such coreference-aware models are better at tracing the information flow among interlocutors and associating accurate status/actions with the corresponding interlocutors and person mentions.

## introduction text summarization condenses the source content into a shorter version while retaining essential and informative content. most prior work focuses on summarizing well-organized single-speaker content such as news articles @xcite and encyclopedia documents @xcite . recently, models applied on text summarization benefit favorably from sophisticated neural architectures and pre-trained contextualized language backbones: on the popular benchmark corpus cnn/daily mail @xcite , @xcite explored fine-tuning bert @xcite to achieve state-of-the-art performance for extractive news summarization, and bart @xcite has also improved generation quality on abstractive summarization. while there has been substantial progress on document summarization, dialogue summarization has received less attention. unlike documents, conversations are interactions among multiple speakers, they are less structured and are interspersed with more informal linguistic usage @xcite . based on the characteristics of human-to-human conversations @xcite , challenges of summarizing dialogues stem from: (1) multiple speakers: the interactive information exchange among interlocutors implies that essential information is referred to back and forth across speakers and dialogue turns; (2) speaker role shift-ing: multi-turn dialogues often involve frequent role shifting from one type of interlocutor to another type (e.g., questioner becomes responder and vice versa); (3) ubiquitous referring expressions: aside from speakers referring to themselves and each other, speakers also mention third-party persons, concepts, and objects. moreover, referring could also take on forms such as anaphora or cataphora where pronouns are used, making coreference chains more elusive to track. figure 1 shows one dialogue example: two speakers exchange information among interactive turns, where the pronoun "them" is used multiple times, referring to the word "sites". without sufficient understanding of the coreference information, the base summarizer fails to link mentions with their antecedents, and produces an incorrect description (highlighted in magenta and italic) in the generation. from the aforementioned linguistic characteristics, dialogues possess multiple inherent sources of complex coreference, motivating us to explicitly consider coreference information for dialogue summarization to more appropriately model the context, to more dynamically track the interactive information flow throughout a conversation, and to enable the potential of multi-hop dialogue reasoning. previous work on dialogue summarization focuses on modeling conversation topics or dialogue acts @xcite @xcite . few, if any, leverage on features from coreference information explicitly. on the other hand, large-scale pre-trained language models are shown only to implicitly model lowerlevel linguistic knowledge such as part-of-speech and syntactic structure @xcite . without directly training on tasks that provide specific and explicit linguistic annotation such as coreference resolution or semanticsrelated reasoning, model performance remains subpar for language generation tasks @xcite . therefore, in this paper, we propose to improve abstractive dialogue summarization by explicitly incorporating coreference information. since entities are linked to each other in coreference chains, we postulate adding a graph neural layer could readily characterize the underlying structure, thus enhancing contextualized representation. we further explore two parameter-efficient approaches: one with an additional coreferenceguided attention layer, and the other resourcefully enhancing bart's limited coreference resolution capabilities by conducting probing analysis to augment our coreference injection design. experiments on samsum @xcite show that the proposed methods achieve state-ofthe-art performance. furthermore, human evaluation and error analysis suggest our models generate more factually consistent summaries. as shown in figure 1 , a model guided with coreference information accurately associates events with their corresponding subjects, and generates more trustworthy summaries compared with the baseline. ## related work in abstractive text summarization, recent studies mainly focus on neural approaches. @xcite proposed an attention-based neural summarizer with sequence-to-sequence generation. pointer-generator networks @xcite were designed to directly copy words from the source content, which resolved out-of-vocabulary issues. @xcite leveraged the pre-trained language model bert @xcite on both extractive and abstractive summarization. @xcite proposed bart, taking advantage of the bi-directional encoder in bert and the autoregressive decoder of gpt @xcite to obtain impressive results on language generation. while many prior studies focus on summarizing well-organized text such as news articles @xcite , dialogue summarization has been gaining traction. @xcite proposed an unsupervised multi-sentence compression method for meeting summarization. goo and chen (2018) introduced a sentence-gated mechanism to grasp the relations between dialogue acts. @xcite proposed to utilize topic segmentation and turnlevel information @xcite ) for conversational tasks. @xcite proposed a neural model with a hierarchical encoder and a reinforced decoder to generate meeting summaries. chen and yang (2020) used diverse conversational structures like topic segments and conversational stages to design a multi-view summarizer, and achieved the current state-of-the-art performance on the sam-sum corpus @xcite . improving factual correctness has received keen attention in neural abstractive summarization lately. cao et al. (2018 knowledge graphs, significantly improving factual correctness in text summarization. ## dialogue coreference resolution since the common summarization datasets do not contain coreference annotations, automatic coreference resolution is needed to process the samples. neural approaches @xcite have shown impressive performance on document coreference resolution. however, they are still sub-optimal for conversational scenarios @xcite , and there are no large-scale annotated dialogue corpora for transfer learning. when applying a document coreference resolution model @xcite on dialogue samples without domain adaptation, foot_0 as shown in figure 2 , we observed some common issues: (1) each dialogue utterance starts with a speaker, but sometimes this speaker is not recognized as a coreference-related entity, and thus not added in any coreference clusters; (2) in dialogues, coreference chains are often spanned across multiple turns, but sometimes they are split to multiple clusters; (3) when a dialogue contains multiple coreference chain across multiturns, speaker entities could be wrongly clustered. based on the observation, to improve the overall quality of dialogue coreference resolution, we conducted data post-processing on the automatic output: (1) first, we applied a model ensemble strategy to obtain more accurate cluster predictions; ## coreference-aware summarization in this section, we adopt a neural model for abstractive dialogue summarization, and investigate various methods to enhance it with the coreference information obtained in section 3. the base neural architecture is a sequence-tosequence model transformer @xcite . given a conversation containing n tokens @xmath0, t 2 , ..., t n }, a self-attention-based encoder is used to produce the contextualized hidden representations @xmath1, h 2 , ..., h n }, then an autoregressive decoder generates the target sequence @xmath2, w 2 , ..., w k } sequentially. here, we use bart @xcite as the pre-trained language backbone, and conduct fine-tuning. for each dialogue, there is a set of coreference clusters {c 1 , c 2 , ..., c u }, and each cluster c i contains entities {e i ## analysis 7.1 quantitative analysis to further evaluate the generation quality and effectiveness of coreference fusion for dialogue summarization, we annotated four types of common errors in the automatic summaries: ## conclusion in this paper, we investigated the effectiveness of utilizing coreference information for summarizing multi-party conversations. we proposed three approaches to explicitly incorporate coreference information into neural abstractive dialogue summarization: (1) gnn-based coreference fusion; (2) coreference-guided attention; and (3) coreferenceinformed transformer. these methods can be adopted on various neural architectures. quantitative results and human analysis suggest that coreference information helps track referring chains in conversations. our proposed models compare favorably with baselines without coreference guidance and generate summaries with higher factual consistency. our work provides empirical evidence that coreference is useful in dialogue summarization and opens up new possibilities of exploiting coreference for other dialogue related tasks.