Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarization (2020)

Abstract: Text summarization is one of the most challenging and interesting problems in NLP. Although much attention has been paid to summarizing structured text like news reports or encyclopedia articles, summarizing conversations—an essential part of human-human/machine interaction where most important pieces of information are scattered across various utterances of different speakers—remains relatively under-investigated. This work proposes a multi-view sequence-to-sequence model by first extracting conversational structures of unstructured daily chats from different views to represent conversations and then utilizing a multi-view decoder to incorporate different views to generate dialogue summaries. Experiments on a large-scale dialogue summarization corpus demonstrated that our methods significantly outperformed previous state-of-the-art models via both automatic evaluations and human judgment. We also discussed specific challenges that current approaches faced with this task. We have publicly released our code at https://github.com/GT-SALT/Multi-View-Seq2Seq .

## introduction we live in an information age where communications between human and human/machine are increasing exponentially in the form of textual dialogues between users and users-agents @xcite . it is challenging and time-consuming to review all the content before starting any conversations especially when the chatting history becomes very long @xcite . how to process and organize those interaction activities into concise and structured data, i.e. conversation summarization, becomes technically and socially important. most existing research efforts on text summarization have been focused on single-speaker doc-uments like news reports @xcite , scientific publications @xcite or encyclopedia articles @xcite , where structured text is usually used to elaborate a core idea in the third-person point of view, and the information flow is very clear through paragraphs or sections. different from these structured documents, conversations are often informal, verbose and repetitive, sprinkled with false-starts, back channeling, reconfirmations, hesitations, speaker interruptions @xcite and the salient information is scattered in the whole chat, making current summarization models hard to focus on many informative utterances. take the conversation in there has been some recent research on conversation summarization such as directly deploying existing document summarization models @xcite and exploring multi-sentence compression @xcite , however, most of them haven't utilized specific conversational structures, which refer to the way utterances are organized in order to make the conversation meaningful, enjoyable and understandable @xcite , in dialogues -a key factor that differentiates dialogues from structured documents. as a way of using language socially of "doing things with words" together with other persons, the conversation has its own dynamic structures that organize utterances in certain orders to make the conversation meaningful, enjoyable, and understandable @xcite . although there are a few exceptions such as utilizing topic segmentation @xcite , dialogue acts (goo and @xcite or key point sequence @xcite , they either need even one single conversation can be viewed from different perspectives, resulting in multiple conversational or discourse patterns. for instance, in to sum up, our contributions are: (1) we propose to utilize rich conversational structures, i.e., structured views (topic view and stage view) and the generic views (global view and discrete view) for abstractive conversation summarization. (2) we de-sign a multi-view sequence-to-sequence model that consists of a conversation encoder to encode different views and a multi-view decoder with multiview attention to generate dialogue summaries. (3) we perform experiments on a large-scale conversation summarization dataset, samsum @xcite , and demonstrate the effectiveness of our proposed methods. (4) we conduct thorough error analyses and discuss specific challenges that current approaches faced with this task. dialogue summarization when it comes to the summarization of dialogues, @xcite proposed a simple multi-sentence compression technique to summarize meetings. @xcite ; zhu et al. (2020b) introduced turn-based hierarchical models that encoded each turn of utterance first and then used the aggregated representation to generate summaries. a few studies have also paid attention to utilizing conversational analysis for generating dialogue summaries, such as leveraging dialogue acts (goo and chen, 2018), key point sequence @xcite or topics @xcite . however, they either needed a large amount of human annotation for dialogue acts, key points or visual focus @xcite @xcite , or only utilized topical information in conversations @xcite . these prior work also largely ignored diverse conversational structures in dialogues, for instance, reply relations among participants @xcite , dialogue acts @xcite , and conversation stages @xcite . models that only utilized a fixed topic view of the conversation @xcite may fail to capture its comprehensive and nuanced conversational structures, and any amount of information loss introduced by the conversation encoder may lead to larger error cascade in the decoding stage. to fill these gaps, we propose to leverage diverse conversational structures including topic segments, conversational stages, dialogue overview, and utterances to design a multiview model for dialogue summarization. ## method conversations can be interpreted from different views and every single view enables the model to focus a specific aspect of the conversation. to take advantages of those rich conversation views, we design a multi-view sequence-to-sequence model (see figure 1 ) that firstly extracts different views of conversations (section 3.1) and then encodes them to generate summaries (section 3.2). ## model analysis and discussion so far, we have achieved a reasonable summarization performance. to further study why dialog summarization is challenging and how future research could advance this direction, we take a closer look at this dialogue summarization dataset (samsum), model generation errors, as well as certain challenges that existing approaches are struggling with. ## conclusion in this work, we proposed a multi-view sequenceto-sequence model that leveraged multiple conversational structures (topic view and stage view) and generic views (global view and discrete view) to generate summaries for conversations. in order to strategically combine these different views for better summary generations, we propose a multiview sequence-to-sequence model. experiments conducted demonstrated the effectiveness of our proposed models in terms of both quantitative and qualitative evaluations. via thorough error analyses, we concluded a set of challenges that current models struggled with, which can further facilitate future research on conversation summarization. due to the lack of annotations, we only adopted simple unsupervised segmentation methods to ex-tract different views. in the future, we plan to annotate some of the data, explore supervised segmentation models @xcite and introduce more conversation structures like dialogue acts @xcite into abstractive dialogue summarization. york, ny, usa. association for computing machinery. chenguang zhu, william hinthorn, ruochen xu, qingkai zeng, michael zeng, xuedong huang, and meng jiang. 2020a. boosting factual correctness of abstractive summarization. chenguang zhu, ruochen xu, michael zeng, and xuedong huang. 2020b. end-to-end abstractive summarization for meetings. henghui zhu, feng nan, zhiguo wang, ramesh nallapati, and bing xiang. 2019. who did they respond to? conversation structure modeling using masked hierarchical transformer.