Controllable Neural Dialogue Summarization with Personal Named Entity Planning (2021)

Abstract: In this paper, we propose a controllable neural generation framework that can flexibly guide dialogue summarization with personal named entity planning. The conditional sequences are modulated to decide what types of information or what perspective to focus on when forming summaries to tackle the under-constrained problem in summarization tasks. This framework supports two types of use cases: (1) Comprehensive Perspective, which is a general-purpose case with no user-preference specified, considering summary points from all conversational interlocutors and all mentioned persons; (2) Focus Perspective, positioning the summary based on a user-specified personal named entity, which could be one of the interlocutors or one of the persons mentioned in the conversation. During training, we exploit occurrence planning of personal named entities and coreference information to improve temporal coherence and to minimize hallucination in neural generation. Experimental results show that our proposed framework generates fluent and factually consistent summaries under various planning controls using both objective metrics and human evaluations.

## introduction automatic summarization is the task of compressing a lengthy piece of text to a more concise version while preserving the information of the source content. extractive approaches select and concatenate salient words, phrases, and sentences from the source to form the summary @xcite @xcite . on the other hand, abstractive approaches generate the summary either from scratch or by paraphrasing important parts of the original text @xcite . for abstractive summarization to be practically usable, it would require more in-depth comprehension, better generalization, reasoning, and incorporation of figure 1 : dialogue summary examples generated by personal named entity planning: some examples focus on perspectives from distinct personal named entities (e.g., john, tony); comprehensive planning includes all personal named entities in the dialogue. note that the content of the ground-truth summary depends on which personal named entity's perspective the focus is during summary formation. real-world knowledge @xcite . while extractive models could suffice for document summarization, abstractive approaches are essential for dialogue summarization to be more easily accessible to users. most benchmarked summarization datasets focus on the news domain, such as nyt @xcite and cnn/daily mail @xcite as material for large-scale corpus construction is readily available online. neural approaches have achieved favorable improvements in both extractive and abstractive paradigms @xcite . neural dialogue summarization is an emerging research area (e.g., goo and chen (2018), @xcite ). while the available data collections are much smaller than those for documents @xcite , neural models have shown potential to generate fluent sentences via fine-tuning on large scale contextualized language models @xcite . unfortunately, most summary generation tasks are constructed in an under-constrained fashion @xcite : in their corpus construction process, only one reference summary is annotated. models trained via supervised learning on such datasets provide generalpurpose summaries, but are suboptimal for certain applications and use cases @xcite . for instance, as shown in figure 1 , a human can write summaries from john or tony's perspective. however, a neural model with a general summarizing purpose may overlook information that is important to a specific person's perspective. on the other hand, if someone wants to collect as much information from the source content, the summary should be written in a comprehensive manner, taking into consideration all personal named entities. such needs are not met with models providing only one possible output. furthermore, different from passages, human-tohuman conversations are a dynamic and interactive flow of information exchange @xcite , which are often informal, verbose, and repetitive. since important information is scattered across speakers and dialogue turns, and is often embodied in incomplete sentences. therefore, generating a fluent summary by utterance extraction is impractical, thus requiring models capable of generating abstractive summaries. however, neural abstractive models often suffer from hallucinations that affect their reliability @xcite , involving improper gendered pronouns and misassigned speaker associations @xcite . for example, as shown in figure 2 , the model makes an incorrect description that "she texted larry last time at the park" (in red). while this sentence achieves a high score in word-overlapping metrics, the semantic meaning it conveys is incorrect: in the context of the generated summary, she refers to amanda, yet in reality it is larry that called (not texted) betty. such factual inconsistency, the inability to adhere to facts from the source, is a prevalent and unsolved problem in neural text generation. in this work, we introduce a controllable dialogue summarization framework. as the aim of dialogue summaries often focuses on "who did what" and the narrative flow usually starts with a subject (often persons), we propose to modulate the generation process with personal named entity plannings. more specifically, as shown in figure 1 , a set of personal named entities foot_0 (in color) are extracted from the source dialogue, and used in a generation model as a conditional signal. we postulate that such conditional anchoring enables the model to support flexible generation. it could be especially useful to address certain demands such as targeting specific client needs for customizing marketing strategies or drilling down customer dissatisfaction at call centers to educate customer agents. in addition, to improve the quality of conditional generation outputs, we integrate coreference resolution information into the contextual representation by a graph-based neural component to further reduce incorrect reasoning @xcite . we conduct extensive experiments on the representative dialogue summarization corpus samsum @xcite , which consists of multi-turn dialogues and human-written summaries. empirical results show that our model can achieve state-ofthe-art performance, and is able to generate fluent and accurate summaries with different personal named entity plans. moreover, factual correctness assessment also shows that the output from our model obtains quality improvement on both automatic measures and human evaluation. ## related work text summarization has received extensive research attention, and is mainly studied in abstractive and extractive paradigms @xcite . for extractive summarization, non-neural approaches study various linguistic and statistical features via lexical @xcite and graph-based modeling @xcite . much progress has been made by recent neural approaches @xcite . compared with extractive methods, abstractive approaches are expected to generate more concise and fluent summaries. while it is a challenging task, with largescale datasets @xcite and sophisticated neural architectures, the performance of abstractive models have achieved substantial improvements in the news domain: sequence-to-sequence models are first introduced by @xcite for abstractive summarization; pointer-generator network @xcite elegantly handled outof-vocabulary issues by copying words directly from the source content; @xcite combines the two paradigms by integrating sentence rewriting into content selection; large-scale pre-trained language models also bring further improvement on summarization performance @xcite . recently, neural summarization for conversations has become an emerging research area. corpora are constructed from meetings @xcite or daily chats @xcite . based on the characteristics of the dialogues, many studies pay attention to utilizing conversational analysis for dialogue summarization, such as leveraging dialogue acts (goo and chen, 2018), multi-modal features @xcite , topic information @xcite , and fine-grained view segmentation with hierarchical modeling @xcite . controllable text generation introduces auxiliary signals to obtain diverse or task-specific outputs. it has been studied in various domains such as style transferring @xcite and paraphrasing @xcite . the conditional input can be in the form of pre-defined categorical labels @xcite , latent representations, semantic or syntactic exemplars @xcite , and keyword planning @xcite . recently, @xcite and @xcite proposed two generic frameworks for length-controllable and question/entity-guided document summarization, and we proposed personal named entity planning upon the characteristics of dialogue summarization. ## controllable generation with personal named entity planning in this section, we introduce the proposed conditional generation framework, elaborate on how we construct personal named entity planning, and delineate the steps for training and generation. ## improving factual correctness while current neural abstractive systems are able to generate fluent summaries, factual inconsistency remains an unsolved problem @xcite . neural models tend to produce statements that are not supported by the source content. these hallucinations are challenging to eradicate in neural modeling due to the implicit nature of learning representations. in document summarization, it has been demonstrated that a certain proportion of abstractive summaries contain hallucinated statements @xcite , as is observed in dialogue summarization @xcite . such hallucinations raise concerns about the usefulness and reliability of abstractive summarization, as summaries that perform well in traditional wordoverlap metrics may fall short of human evaluation standards @xcite . ## conclusion in this work, we proposed a controllable neural framework for abstractive dialogue summarization. in particular, a set of personal named entities were used to condition summary generation. this framework could efficiently tailor to different user preferences and application needs, via modulating entity planning. moreover, the experimental results demonstrated that the abstractive model could benefit from explicitly integrating coreference resolution information, achieving better performance on factual consistency and standard metrics of wordoverlap with gold summaries.