View Dialogue in 2 D : A Two-stream Model in Time-speaker Perspective for Dialogue Summarization and beyond (2022)

Abstract: Existing works on dialogue summarization often follow the common practice in document summarization and view the dialogue, which comprises utterances of different speakers, as a single utterance stream ordered by time. However, this single-stream approach without specific attention to the speaker-centered points has limitations in fully understanding the dialogue. To better capture the dialogue information, we propose a 2D view of dialogue based on a time-speaker perspective, where the time and speaker streams of dialogue can be obtained as strengthened input. Based on this 2D view, we present an effective two-stream model called ATM to combine the two streams. Extensive experiments on various summarization datasets demonstrate that ATM significantly surpasses other models regarding diverse metrics and beats the state-of-the-art models on the QMSum dataset in ROUGE scores. Besides, ATM achieves great improvements in summary faithfulness and human evaluation. Moreover, results on machine reading comprehension datasets show the generalization ability of the proposed methods and shed light on other dialogue-based tasks. Our code will be publicly available online.

## introduction dialogue summarization is a task aiming to generate a succinct and coherent summary of the given dialogue, which has been explored in many applications, such as automatic meeting summarization, and drawn the attention of many researchers. with the development of transformer-based pretrained models @xcite @xcite , remarkable progress has been made in text summarization, especially in document summarization @xcite . however, dialogue summarization is still quite challenging partly due to the structural characteristics of dialogue text @xcite . a major difference is that a document is often organized with a unified narrative perspective, while a dialogue includes many speakers that bring diversity and switches of narrative perspectives @xcite . moreover, the information from different speakers is scattered @xcite , which poses a challenge to the summarizer as the dialogue summary is often speaker-centered that focuses on the speaker's actions and opinions @xcite . current works on dialogue summarization usually view the dialogue as a single utterance stream ordered by time @xcite , like prior studies on document summarization do @xcite . this single-stream approach, however, has limitations in fully using the information about dialogue @xcite . firstly, the utterances of different speakers are interlaced in the single stream, which may weaken the semantic continuity from each speaker's perspective. besides, to obtain a concise summary instead of a laundry list, we need to both notice the development in time order and sum up information about each speaker @xcite . the single-stream approach may not be enough since it mainly focuses on the former. furthermore, the summarizer needs to deal with the frequent switches of multiple narrative perspectives in the single stream. the generated summaries are consequently often accompanied by unfaithfulness problems, such as coreference error and missing information @xcite , which hinder the practical applications of dialogue summarizers. to tackle these challenges, we propose a 2d view that restructures the dialogue text to make the most of dialogue information. inspired by how humans summarize, we arrange the dialogue in a time-speaker view as figure 1 shows, where time and speaker streams can be obtained by projecting the dialogue. intuitively, the two streams are two directions for humans to summarize a dialogue. the two streams focus on two complementary aspects of dialogue. to combine these two streams, we present a two-stream model called atm based on transformer. the atm encoder consists of a trunk-branch network to catch the salient commonality and individuality of the two streams. then we leverage two cross-attention modules in each atm decoder layer to capture the information of both time and speaker streams. the approach can be easily applied to other transformer-based models. extensive experimental results on three dialogue summarization datasets show that atm significantly surpasses other baseline models by a large margin regarding various automatic metrics and achieves new state-of-the-art results on qmsum dataset, to the best of our knowledge. atm also mitigates unfaithfulness problems and achieves higher scores in our human evaluation. moreover, to test the generalization ability of the proposed methods, we conduct experiments on two machine reading comprehension (mrc) datasets. the results demonstrate that atm also outperforms the single time-stream model in mrc task. our main contributions are three-fold. 1) we propose a novel 2d view for better representing dialogue. 2) a two-stream model is presented for dialogue called atm to make the most of dialogue information. 3) we conduct extensive experiments to demonstrate the effectiveness and insights of the proposed methods in two dialogue-based tasks. 2 related work abstractive dialogue summarization has attracted much attention recently since abstractive methods can produce more coherent and readabie summaries than extractive methods. in early stage, banerjee et al. (2015) utilize the dependency graph. oya et al. (2014) and singla et al. (2017) explore template-based methods. with the development of deep learning and publicly available datasets @xcite @xcite , plenty of related works have been conducted. to utilize the interactive characteristic of dialogue, graph-based methods are used in @xcite @xcite . for capturing the acts in dialogue, @xcite propose a sentence-gated mechanism and @xcite use dialogue acts as an interactive pattern. @xcite incorporate discourse relations. unsupervised strategies are explored in @xcite @xcite . @xcite and @xcite leverage annotators to dig up more information. a specialized pre-training framework is proposed in @xcite . as dialogue is composed of multiple turns, hmnet @xcite , @xcite and @xcite use a hierarchical network to model multi-level representations of dialogue. chen and yang (2020) propose a multi-view approach. a major limitation of these works is that they view the dialogue from a 1d perspective, while we arrange the dialogue in a time-speaker view. they also lack enough attention to the semantic continuity that we focus on. speaker-aware methods for improving faithfulness focus on the speakers in the dialogue. the multiple speakers contain helpful information while posing a challenge for the model to generate faithful summaries @xcite . to relieve the confusion of personal pronouns, @xcite propose a from-coarse-to-fine procedure. finds @xcite utilizes finer-grain semantic structures to clarify the speaker relationships. @xcite propose a self-supervised strategy to do post-correction for speakers. @xcite leverage a speaker-aware structure to model the interaction process in dialogue. two-stream architectures have been utilized in several areas, including computer vision @xcite @xcite , natural language processing including xlnet @xcite and ernie-gram @xcite , and multimodal applications including lxmert @xcite , vilbert @xcite and ernie-vil @xcite . other works include adding extra modules for specific purposes, such as adaptive computing @xcite . our originality lies in applying the twostream idea to dialogue text based on the proposed time-speaker view. ## methodology in this section, we first introduce the time-speaker view of dialogue and present the problem formulation of the two-stream summarization based on this view. then details of atm are presented. ## conclusion to make the most of dialogue information, we propose a novel 2d view highlighting the speakers based on time-speaker space, which provides a new inspiring perspective on modelling dialogue. then a simple and effective two-stream summarization model atm is presented to utilize the information from both the time and speaker streams obtained from this view. empirical results demonstrate that the proposed methods surpass other models on three summarization datasets regarding various metrics, faithfulness and human evaluation. we also show the significant improvements on mrc, a representative of other dialogue-based tasks. this work leaves several open directions that can be explored, including 1) applying the proposed methods to other models and tasks, 2) exploring the 2d view from other directions, and 3) establishing a general framework for dialogue-based tasks. dialogue person1: sally,here is a letter for us. it's from tom. person2: can you read it, please? my hands are wet with all this washing. person1: well, ok. dear sally and john. thanks for your letter. it was good to hear from you. just a short note in reply. please do call me when you arrive so that i can pick you up at the station. bart person1 gives sally and john a letter from tom who will be in town in january. atm tom writes a letter to sally and john and tells them he will pick them up when they arrive in town in january. dialogue person1: good afternoon, what can i do for you? person2: yes, please. i would like to know something about the driving courses. person1: well, we have short full time courses during the summer. are you interested in them? person2: no, i am free only at weekends. person1: then there are weekend courses. the course starts at 8:00 ... person2: sounds fine. what about the coaches? person1: we have very excellent coaches here ... person2: good. how many hours of training should i have each day? person1: 3 hours in the morning and 2 in the afternoon. (...) bart person2 wants to know something about the driving courses. person1 introduces the short full-time courses, weekend courses, and the coaches. atm person2 wants to know something about the driving courses. person1 introduces the short full-time courses and the weekend courses and introduces the coaches and the training time. dialogue (...) person2: this next one is from betty. person1: a highchair and car seat! wow betty, thank you so much! i really appreciate it! person2: one more from carla. person1:a playpen and crib! thanks carla! this is just what i needed! (...) person1: umm. i think my water just broke! get me to a hospital! bart person1 thanks person2 for organizing a great baby shower. person1 person 1 opens some presents from betty, carla, betty, and carla and feels like having a baby. atm person1 thanks person2 for organizing the baby shower and opening some presents, including a bib, a stroller, a highchair, and a car seat from betty and carla. person1's water breaks and asks person2 to take her to a hospital.