Topic-Aware Contrastive Learning for Abstractive Dialogue Summarization (2021)

Abstract: Unlike well-structured text, such as news reports and encyclopedia articles, dialogue content often comes from two or more interlocutors, exchanging information with each other. In such a scenario, the topic of a conversation can vary upon progression and the key information for a certain topic is often scattered across multiple utterances of different speakers, which poses challenges to abstractly summarize dialogues. To capture the various topic information of a conversation and outline salient facts for the captured topics, this work proposes two topic-aware contrastive learning objectives, namely coherence detection and sub-summary generation objectives, which are expected to implicitly model the topic change and handle information scattering challenges for the dialogue summarization task. The proposed contrastive objectives are framed as auxiliary tasks for the primary dialogue summarization task, united via an alternative parameter updating strategy. Extensive experiments on benchmark datasets demonstrate that the proposed simple method significantly outperforms strong baselines and achieves new state-of-the-art performance. The code and trained models are publicly available via .

## introduction online conversations have become an indispensable manner of communication in our daily work and life. in the era of information explosion, it is paramount to present the most salient facts of conversation content, rather than lengthy utterances, which is useful for online customer service @xcite and meeting summary @xcite . this work focuses on abstractive dialogue summarization. to summarize dialogues, one simple way is to directly apply existing document summarization models to dialogues @xcite figure 1 : a dialogue and its paired summary. s 1 , s 2 , and s 3 stands for referred topic snippets, current situation, time of arrival and food to eat, respectively. the corresponding summary consists of three sentences t 1 , t 2 and t 3 . each t i corresponds to one snippet s i (@xmath0, 2, 3). s 4 and s 5 are inter-topic snippets. @xcite or to employ hierarchical models to capture features from different turns of different speakers @xcite . however, succinctly summarizing the dialogue is much more challenging. the well-structured textual descriptions, such as news reports @xcite and academic papers @xcite , often come from one single speaker or writer where the information flow is more natural and clearer with paragraphs and sections. differently, consisting of multiple utterances from two or more interlocutors, the conversational content is in a complicated flow with information exchange and the focused topic can vary upon the conversation progression. on the other hand, the salient information for a specific topic is often scattered across multiple utterances and can be presented separately. exemplified by figure 1 , this dialogue touches three topics, current situation, time of arrival and food to eat, where the corresponding topic snippets are s 1 , s 2 and s 3 , respectively. the central ideas of each topic is summarized with one sentence, covering information from multiple utterances, i.e., t 1 for s 1 , t 2 for s 2 , and t 3 for s 3 . we also observe that utterances residing in the same topic (e.g., s 1 , s 2 and s 3 ) is inherently more coherent than those coming from different topics (e.g., the inter-topic snippet s 4 and s 5 ), which reveals the underlying relationships between topic and utterance coherence, also demonstrated by @xcite . recent studies involves intrinsic information of dialogues to handle the challenges for summarizing dialogues, such as topic segment features @xcite @xcite , dialogue acts @xcite and conversation stages @xcite . although such existing models have demonstrated the effectiveness of the dialogue analysis on generating summaries, additional human efforts in data annotations or extra topic segmentation algorithms are necessary. for example, goo and chen (2018); @xcite require extensive expert annotations on dialogue acts, while the knowledge of visual focus of each speaker and topic segment is a must for li et al., 2019's work, which are both expensive and sometimes hard to obtain. @xcite ; @xcite need extra algorithms to obtain topic segment information, which works with the primary summarization model in a pipeline manner and thus may cause error propagation. different from the structured text where a paragraph or a section can be treated as natural topic segment, it is difficult to accurately segment topics of dialogues. recall the inherent relationships between the topic and utterance coherence, this work proposes to implicitly capture the dialogue topic information by modeling the utterance coherence in a contrastive way. the coherence detection objective is constructed to push the model to focus more on snippets that are more coherent and likely contain salient information from the same topics. further, since we aim to generate better summaries for each topic in a dialogue, we also introduce the subsummary generation objective, which is expected to force the model to identify the most salient information and generate corresponding summaries. note that both objectives are constructed in a con- coherence detection objective sub-summary generation objective transformer decoder layer transformer encoder layer julia where are … hania has been log likelihood primary generation loss … coherence regressor ×n ×n contrastive contrastive you figure 2: model structure with contrastive objectives. trastive way where no additional human annotations or extra algorithms are required. such two contrastive objectives can be coupled with the primary dialogue summarization task via an alternating parameter updating strategy, resulting in our final model condigsum. experiments on two dialogue summarization datasets demonstrate the effectiveness of our proposed contrastive learning objectives for dialogue summarization which achieves new state-of-the-art performances. ## conclusion recent research progresses have present the effectiveness of dialogue studies (e.g., topical information and dialogue acts) on summarizing dialogues, while additional expert annotations or extra algorithms are required to obtain the knowledge. this work proposes a simple yet effective method, condigsum, that implicitly captures the topical knowledge residing in dialogue content by modeling the text coherence, yet no additional human annotations or segment algorithms are needed. we design two contrastive objectives as auxiliary task, i.e., coherence detection and sub-summary generation objectives,working together with the primary summarization task during training. an alternating parameter update strategy is employed to cooperate the primary and auxiliary tasks. experiments on two benchmark datasets demonstrate the efficacy of the proposed model. future directions include learning structured representations of information flow residing in dialogues and leveraging knowledge graphs to generate better dialogue summaries.