A Bag of Tricks for Dialogue Summarization (2021)

Abstract: Dialogue summarization comes with its own peculiar challenges as opposed to news or scientific articles summarization. In this work, we explore four different challenges of the task: handling and differentiating parts of the dialogue belonging to multiple speakers, negation understanding, reasoning about the situation, and informal language understanding. Using a pretrained sequence-to-sequence language model, we explore speaker name substitution, negation scope highlighting, multi-task learning with relevant tasks, and pretraining on in-domain data. Our experiments show that our proposed techniques indeed improve summarization performance, outperforming strong baselines.

## introduction the nature of dialogue poses additional challenges to summarizers beyond what is required when processing structured, single-speaker documents @xcite . given that dialogues typically represent an interaction between many speakers, a summarizer model must keep track of the different lines of thoughts of individual speakers, distinguish salient from non-salient utterances, and finally produce a coherent, monologue summary of the dialogue. dialogues usually include unfinished sentences where speakers were interrupted or repetitions, where a speaker expresses their thoughts more than once and possibly in different styles. moreover, a single dialogue could touch on many topics without a clear boundary between the different topics. all the aforementioned phenomena certainly add to the difficulty of the task @xcite @xcite . our work focuses on samsum @xcite , which is a dialogue summarization dataset comprised of ~16k everyday dialogues with their * work done during an internship at amazon. human-written summaries. as our backbone model, we use bart @xcite , a stateof-the-art pretrained encoder-decoder language model that is suitable for sequence-to-sequence tasks. we propose a combination of techniques to tackle a set of dialogue summarization challenges. the first challenge is having multiple speakers (generally, more than 2), where it becomes harder for the model to keep track of different utterances and determine their saliency. the second challenge is multiple negations, which is thought by @xcite to pose some difficulty to dialogue understanding. the third of these challenges is reasoning, where the model is required to reason about the dialogue context, and infer information that is not explicitly expressed. the last challenge is informal language. since we focus on random, everyday conversations, these are usually filled with non-standard language (abbreviations, social media terms, etc.). the contributions in this work are: dialogue: orion: i miss him :( cordelia: need i remind you that he cheated on you? you deserve alot better than that orion: ...what? oh, right noo -im talking about my rat ... he died ... vanilla bart output: orion's rat died. he cheated on her. mtl bart output: orion's rat died and he misses him. reference: orion is grieving after the death of her rat. 2 related work early work on dialogue summarization focused more on extractive than abstractive techniques for summarization of meetings @xcite or random conversations @xcite . in the context of meeting summarization, @xcite proposed an unsupervised graph-based sentence compression approach for meeting summarization on the ami @xcite and icsi @xcite benchmarks. goo and chen (2018) leveraged hidden representations from a dialogue act classifier through a gated attention mechanism to guide the summary decoder. more recently, @xcite proposed samsum, a benchmark for abstractive everyday dialogue summarization. @xcite modeled dialogues using a graph structure of words and utterances and summaries are generated using a graph-to-sequence architecture. chen and yang (2020) proposed a multi-view summarization model, where views can include topic or stage. they also pointed out to seven different challenges to dialogue summarization and analysed the effect each challenge can have on summarization performance using examples from samsum. ## challenges we now present our four techniques for dialogue summarization: name substitution (section 3.3), negation scope highlighting (section 3.4), multi-task learning on common sense tasks (section 3.5), and pretraining on an in-domain dialogue corpus (section 3.6). ## conclusion in this paper, we explored different techniques to improve dialogue summarization performance by addressing different challenges to the task individually. the proposed techniques included name substitution, negation scope highlighting, multitask learning with relevant tasks, and pretraining on in-domain corpora. on one hand, our experiments on three challenges showed the effectiveness of our proposed techniques which outperformed strong baselines on the task. on the other hand, our proposed technique to handle multiple negations performed poorly and by analyzing the outputs on