Abstractive Meeting Summarization: A Survey (2023)

Abstract: A system that could reliably identify and sum up the most important points of a conversation would be valuable in a wide variety of real-world contexts, from business meetings to medical consultations to customer service calls. Recent advances in deep learning, and especially the invention of encoder-decoder architectures, has significantly improved language generation systems, opening the door to improved forms of abstractive summarization—a form of summarization particularly well-suited for multi-party conversation. In this paper, we provide an overview of the challenges raised by the task of abstractive meeting summarization and of the data sets, models, and evaluation metrics that have been used to tackle the problems.

## introduction being a primary and inevitable means of information exchange at the workplace, a vast amount of time and organizational resources are allocated to meetings @xcite . @xcite reported that on average, american employees and managers put 6 and 23 hours per week, respectively, into meetings. the rise in videoconferencing linked to the covid-19 pandemic has only made the situation more severe @xcite : people are having longer and more frequent meetings, leading to increased fatigue and less time to digest the information exchanged @xcite . in this context, developing a system that could reliably identify key information from a meeting or meeting transcript and use it to produce a condensed and easily digestible summary-as well as, perhaps, a set of meeting minutes that details decisions and action itemsis becoming more of a priority than ever before @xcite . research on automatic summarization dates as far back as the 1950s, with systems aiming to generate abstracts from scientific literature @xcite . since then, approaches to summarization have developed along two main lines @xcite . extractive summarization, in which a system creates a summary by directly lifting important sentences from source documents and concatenating them without any modification to the original sentences, dominated earlier work due to its simplicity. the advent of neural encoder-decoder architectures @xcite , however, has opened the door to abstractive summarization @xcite , which draws upon deep representations of word or sentence meaning to generate well-written, novel sentences that consolidate and concisely paraphrase information that might be distributed among numerous clauses or sentences in the original text or transcript. while the majority of research on summarization has been conducted on formal written documents (news, scientific articles, etc.), in this paper, we highlight work on meeting summarization, a subdomain of automatic summarization that stands to benefit particularly well from advances in abstractive methods. meeting summarization poses challenges not encountered by traditional text summarization @xcite . some are related to limitations on the tools needed for developing models-namely, training data, model architectures, and evaluation metrics-but others stem from the very nature of the linguistic interactions involved in meetings or multi-party conversation more generally. these challenges suggest that producing commercial-level, automatic tools for abstractive meeting summarization cannot be accomplished simply by generalizing or fine-tuning models trained on text or even certain forms of dialogue, but will require developing radically new approaches tailored to the meeting domain @xcite . figure 1 : example @xcite of ground truth human annotations from the es2011c ami meeting. successive gray nodes on the left denote utterances recorded in the transcript, the black nodes correspond to a subset of utterances that annotators judged important (summary-worthy). a single sentence from each section of the abstractive summary is shown on the right. the colored arcs link a given sentence from the abstract summary to a single abstractive community; that is, the set of clauses from the extractive summary that is summarized by the sentence from the abstractive summary. we begin in section 2 with an overview of the particular challenges raised by meeting-style interactions and meeting summaries. sections 3 through 5 introduce the different tools that have been used to address these challenges, focusing on data sets, summary evaluation, and summarization systems, respectively @xcite . as we will see, each of these areas introduces complications of its own and we will review the pros and cons of the different approaches. we conclude with a discussion of promising directions for future research. the nature of meeting-style speech. conversations in meetings are unedited, often unprepared exercises in real-time collaboration, which tends to yield transcripts with low information density and a fair amount of ''noise'' stemming from choppy, repetitive, and meandering language. even though (1)-( 9 ) in figure 1 are taken from an extractive summary, and thus provide a ''cleaned-up'' selection of utterances from es2011c, they illustrate some of the disfluent language-e.g. meeting transcripts also tend to be significantly longer than documents for text summarization, and likewise for their summaries. on average, one ami transcript contains 4,757 tokens and its summary has 322, while an article from the cnn/dailymail dataset @xcite has an average of 781 tokens and its summary has an average of 56. moreover, in contrast to other types of conversation, such as customer service calls and medical appointments, meetings often involve more than two speakers. this increases the number of (potentially conflicting) perspectives to consider (figure 1 , (5 )), and also the variety of speech styles and potential for overlapping speech. finally, it complicates the task of speaker and addressee identification and thus, the conversion of first and second person pronouns into third person identifiers, as is needed for summarization. ## evaluation methods as noted in section 2, the subjectivity of meeting summaries, stemming in part from the preference for abstractive summarization, complicates the evaluation task. evaluation requires both verifying that the content in a system-based summary follows from the original transcript and measuring the overlap with a gold-standard summary. however, given the expressive freedom encouraged by abstractive approaches, checking for semantic entailment and overlap of summary-worthy content requires deep semantic understanding, not just recognition that the same words are used. unfortunately, the rouge metric @xcite , which remains the standard for both meeting and general text summarization, scores system-produced summaries based purely on surface lexicographic matches with a (usually single) gold summary, making it not ideal for assessing abstractive summaries. if we take the abstractive summary from figure 1 ('' some part of the casing will be made of a spongy material'') as a gold example, rouge would assign a higher score to a system that produces ''some part of the casing will be made of broccoli'' than one that output ''a portion of the outer layer will be constructed from a sponge-like material,'' even though the latter is a perfect reformulation of the gold summary, while the former says something very different (and false). a reasonable way to try to improve over rouge would be to take advantage of massive, pretrained, contextual word embeddings and a notion of lexical similarity rather than strict lexical overlap. some recent efforts pursue this direction @xcite , including bertscore @xcite and moverscore @xcite , which aim to measure the semantic distance between the contextualized mapping of a generated summary and the reference, or bartscore @xcite , which calculates the log-likelihood of a summary to have been generated, motivated by the fact that a good summary should have a high probability of being generated from a source text. building upon these methods, datscore and frugalscore @xcite incorporate data augmentation and knowledge distillation techniques to further improve performance and overcome their drawbacks. despite their rapid development, recent studies on meta-evaluation of these metrics show mixed results. @xcite and @xcite compare their performance in the context of document summarization and show that metrics strongly disagree in ranking summaries from any narrow scoring range, e.g., there is no consensus among metrics regarding which summary is better than another in the high scoring range in which modern systems now operate. @xcite argue that there is no one-size-fits-all metric that correlates better with human judgement than the others, and that can outperform others on all datasets. clearly, evaluation is itself a very challenging task. and we note that none of these metrics even touches on another central challenge for summary evaluation, namely that of factual consistency. when we summarize a meeting or detail decisions and actions items in our own words, it is important to get the facts straight. otherwise, the resulting summaries are not reliable for an end user. while we do not know of current work that focuses on evaluation of factuality explicitly for the meeting domain, the study of factual consistency in summarization more generally is a budding research area that we discuss in section 6. in the absence of a clear winner for summary evaluation metrics, none of the alternatives has yet to be widely adopted. in our comparison of different summarization systems in section 5, we therefore stick with rouge, which offers the additional advantage of conceptual simplicity and lower computational cost compared to metrics based on contextual embeddings and pretrained language models. 5 systems for meeting summarization jones (1999) describes a general summarization pipeline consisting of three stages: (i) interpretation: mapping the input text to a source representation that adds additional information to the source document that is useful for interpreting its content. (t) transformation: transforming the source representation to a summary representation based on which the final summary will be produced. (g) generation: generating a summary text from the summary representation. in the course of our literature review, we noted that when extant work on meeting summarization attempts to address one of the challenges described in section 2, it can be seen as focusing on one of the stages of jones' pipeline. we therefore adopt this three-stage schema as the backbone for a taxonomy of summarization systems that we lay out in the remainder of this section. ## future directions we conclude by laying out some directions that we think are worth investigation in future work. ## conclusion in this work, we presented a comprehensive overview of the state-of-the-art in abstractive meeting summarization. we discussed the different challenges that research has faced, and we proposed a taxonomy that followed the three-step summarization pipeline, interpretation-transformation-generation, presented by jones (1999). we further compared the results obtained by previous work and finally laid out promising directions for future research.