Capturing Speaker Incorrectness: Speaker-Focused Post-Correction for Abstractive Dialogue Summarization (2021)

Abstract: In this paper, we focus on improving the quality of the summary generated by neural abstractive dialogue summarization systems. Even though pre-trained language models generate well-constructed and promising results, it is still challenging to summarize the conversation of multiple participants since the summary should include a description of the overall situation and the actions of each speaker. This paper proposes self-supervised strategies for speaker-focused post-correction in abstractive dialogue summarization. Specifically, our model first discriminates which type of speaker correction is required in a draft summary and then generates a revised summary according to the required type. Experimental results show that our proposed method adequately corrects the draft summaries, and the revised summaries are significantly improved in both quantitative and qualitative evaluations.

## introduction research on abstractive dialogue summarization recently have achieved remarkable improvements @xcite @xcite @xcite in diverse domains such as daily dialogue, interview, and movie. despite the promising performance of the pre-trained language models (e.g., unilm @xcite and bart @xcite ), the capability of summarizing the multiparty conversation is still limited. due to its difficulty of considering all the actions of every speaker for describing a scene is quite challenging. specifically, chen and yang (2020) emphasize dealing with multi-party situation in dialogue summarization based on several criteria, such as role & language change, referral & coreference, and multiple turns & participants. based on this perspective, we have performed human evaluation foot_0 on the summaries generated by bart base model to figure out whether they adequately include the contents of the conversation. the results showed that only 47% of the samples can be regarded as correct summaries. the rest mainly contain incorrect contents w.r.t. references, reasoning, and gendered pronouns. one interesting finding is that nearly half of the incorrect summaries have speaker-related errors. as shown in to address the aforementioned finding, this paper mainly focuses on improving the quality of the dialogue summary in terms of correcting speakers. existing works proposed post-editing methods for abstractive text summarization @xcite and table-to-text @xcite , but they mainly focus on correcting summary of the general corpora (e.g., news articles) or facts in a knowledge base, which are somewhat different from the multi-party conversation. some researches for dialogue summarization @xcite utilized utterance-level representations by constructing dialogue graph, but they lack in leveraging speaker information explicitly. in this work, we propose a speaker-focused postcorrection model for abstractive dialog summarization. we first construct the dataset by using selfsupervised speaker manipulation strategies, which corrupt the speakers in summary on purpose. during training, our model predicts whether the speakers are corrupted or not by using the speaker correction type discriminator and then generates a corrected summary according to the correction type via the speaker-focused correction generator. our main contributions are as follows. 1) we show that the existing dialogue summarization model often generates incorrect summaries that contain speaker-related errors (i.e., insertion, deletion, and replacement) through human evaluation. based on this, we design self-supervised speaker manipulation strategies to construct the post-editing data without extra annotations. 2) we propose the highly effective speaker-focused postcorrection model not only to capture speaker incorrectness but also adequately revise the draft summary. to the best of our knowledge, it is the first attempt to adopt the post-editing method w.r.t. the speakers in abstractive dialogue summarization. 3) experimental results show that the revised summaries are significantly improved compared to the draft summaries in both quantitative and qualitative evaluations. ## conclusion in this paper, we pointed out that current dialogue summarization models have problems summarizing the multi-party conversation. to address these problems, we proposed the speaker-focused postcorrection model, which can be applied to any ab-stractive dialogue summarization model. experimental results show that our model adequately corrects a draft summary. inputs: d -dialogue context y r -reference summary f lagcorruption flag outputs: y s -corrupted summary crequired correction type function sp e a k e r_m a n i p u l a t i o n(d, y r , f lag) if flag then speaker_list ← extract_speakers(d) f unc_type ← random.choices[ins, del, rep] if @xmath0, speaker_list) c ← del else if @xmath1, speaker_list) c ← ins else if @xmath2, speaker_list) c ← rep end if else y s ← y r c ← no end if return y s , c end function