Explainability Meets Text Summarization: A Survey (2024)

Abstract: Summarizing long pieces of text is a principal task in natural language processing with Machine Learning-based text generation models such as Large Language Models (LLM) being particularly suited to it. Yet these models are often used as black-boxes, making them hard to interpret and debug. This has led to calls by practitioners and regulatory bodies to improve the explainability of such models as they find ever more practical use. In this survey, we present a dual-perspective review of the intersection between explainability and summarization by reviewing the current state of explainable text summarization and also highlighting how summarization techniques are effectively employed to improve explanations.

## introduction against the ever-growing influx of textual content, being able to effectively summarize long pieces of text is crucial to extract useful information. whereas once a significant amount of manual labour would be necessary, now automatic text summarization (ats) can be performed by deep learning models, especially as they grow in capabilities and become more easily accessible @xcite . nevertheless, such deep learning models are essentially black boxes. they provide no immediate information regarding their internals, and they can fail in ways imperceptible to a novice, e.g. by producing incorrect output that looks legitimate and create an illusion of understanding @xcite . it is thus of critical importance that such models can be made explainable, especially in sensitive fields such as law @xcite and healthcare @xcite . in this work, we bridge the gap between text summarization and explainability and highlight through a literature review their dualistic relation, namely that on one side summarization methods help develop explainable methods, and on the other explainability methods help enhance and understand summarization methods. explainability in summarization can take two forms, each targeting different stakeholders. the first form involves explaining the output of summarization models, intended for the end users of summarization systems. the second form is focused on understanding and interpreting the internal workings and mechanisms of the summarization model, primarily aimed at debugging the model, which is intended for model developers. why text summarization and explainable ai(xai)? an explanation is an attempt at extracting useful, concise information from a complex, black-box model. likewise a summary attempts to extract the essential bits of a longer piece of text. seen this way, an explanation summarizes the model's prediction, and a summary explains the summarized piece of text. it is thus beneficial to consider the two problems together since approaches to one can inform the approaches to the other, as we will provide examples throughout the survey. contributions as far as we know, this work is the first to present an overview of explainable text summarization and to offer a dual perspective on how explainability and summarization can mutually contribute to each other. in the scope of this work, we use the terms related to explainability and interpretability interchangeably. the contributions of this survey are summarized as follows: approaches to text summarization can be in two types: abstractive summarization methods generate new sentences by processing the input sentences (i.e. summarize in their own words), while extractive summarization approaches directly copy parts of the input text to construct a summary. ## methodology in this survey, we employ a systematic review approach following the methodology defined by @xcite . we detail the review methodology in appendix a. we first formulated our research questions with a high degree of specificity as follows: rq1: what are the popular models, datasets, and evaluation metrics used in existing research on explainable text summarization? rq2: what xai techniques are employed for text summarization in the existing research studies? rq3: how are such explanations visualized and evaluated? rq4: can we derive practical conclusions on the usefulness of explainability techniques for text summarization? rq5: how can text summarization methods be utilized by xai to provide explanations? we defined a set of related keywords to search for relevant papers and applied the following search string to the title, abstract, and keywords: ("explainable" or "interpretable" or "explainability" or "interpretability") and ("text summarization"). we then filter and divide the papers into two categories: (1) explainability for text summarization direction, in which explainability techniques are applied to explain the summarization models outputs or internal mechanisms, (2) summarization for explainability direction, which consists of papers where text summarization is used to provide explanations independent of the nlp task under consideration. ## results in this section, we present the results of our review, structured according to the research questions formulated earlier and also provide some insights at the end of each section. ## related surveys two of the earlier baseline surveys in xai are presented in @xcite . @xcite serves as a reference for terminologies and approaches regarding xai and @xcite classifies xai techniques and provides a comprehensive background regarding the main concepts, motivations, and implications of enabling explainability in intelligent systems. explainable nlp surveys include @xcite @xcite . @xcite review xai techniques in nlp with a focus on explaining model's decision for several nlp tasks. later, zini and awad (2022) extends such review by highlighting the explainability methods on the input and processing levels. more recently, @xcite reviews and categorizes the explainability methods specific only for providing local explanations. focusing on llms, @xcite overviews and classifies the different approaches for explaining llms based on the training paradigms. ## conclusion this paper presents a dual-perspective review of the intersection between xai and ats. first, we review the current state of applying xai to ats. second, we highlight the application of summarization in enhancing the interpretability of black-box models. given our focus on ats as a use case, this work aims to promote the practical usability of xai in ats and other generation tasks in nlp systems. we present this survey as a resource for researchers and practitioners interested in designing, using, or enhancing the explainability of ats systems. we hope this survey also paves the way for further research into utilizing summarization to improve the interpretability of nlp-based systems. future work: to address the urgent need to bridge the gap in ground truth evaluation for explainability methods applied to ats, future work could focus on designing explainable datasets for text summarization. motivated by suggestions from @xcite , this could involve augmenting human annotations and rationales with synthetic data to comprehensively evaluate xai methods for ats.