Extractive Entity-Centric Summarization as Sentence Selection using Bi-Encoders (2022)

Abstract: Entity-centric summarization is a type of controllable summarization that aims to produce a summary of a document that is specific to a given target entity. Extractive summaries possess multiple advantages over abstractive ones such as preserving factuality and can be directly used in downstream tasks like target-based sentiment analysis or incorporated into search applications. In this paper, we explore methods to solve this task by recasting it as a sentence selection task, as supported by the EntSUM data set. We use methods inspired by information retrieval, where the input to the model is a pair representing a sentence from the original document and the target entity, in place of the query. We explore different architecture variants and loss functions in this framework with results showing an up to 5.8 F1 improvement over past state-of-the-art and outperforming the competitive entity-centric Lead 3 heuristic by 1.1 F1. In addition, we also demonstrate similarly strong results on the related task of salient sentence selection for an entity.

## introduction controllable summarization is a recently growing area of research, where the aim is to provide a summary that is specific to a user's information need, which could be a target entity @xcite , aspect @xcite or topicor can represent the user's preferred style @xcite or length @xcite . controllable summarization offers the promise of making summarization more usable to users, enabling them to achieve their end goals by summarizing the information they are interested in @xcite . extractive summarization aims to extract passages or entire sentences from the original summaries, as opposed to abstractive summarization which aims to generate an entirely new summary @xcite . although most recent research has focused on abstractive summarization techniques, these possess several disadvantages, the most prominent being the potential for lack of factuality and coherence @xcite @xcite , as well as difficulty in correctly assessing the summary quality automatically @xcite @xcite . on the other hand, extractive summarization mitigates these issues by extracting text from the original document and, if the data set contains the sentence or passage level information, evaluation can then be performed using standard metrics, such as f1. the extractive entity-centric summarization methods can be used directly to aid users in interactive applications such as search @xcite , through either highlighting or extracting passages in the document. extractive summarization also has the potential to be used as an intermediary step or auxiliary task in downstream entity-centric tasks, such as entity salience @xcite , aspectbased sentiment classification @xcite , or information retrieval. this paper presents the first in-depth study of extractive entity-centric summarization methods. we take advantage of the unique properties of the entsum data set @xcite , which provides multiple layers of annotations regarding the entities, including the sentences salient for the entity in a document and the sentences that construct a summary about an entity. we are thus able to recast the entity-centric extractive summarization task as selecting the summary sentences regarding an entity in a document. this allows us to compute reliable f1 metrics to compare several approaches, including heuristics and adaptations to extractive summarization of controllable summarization methods. we propose new meth-ods for entity-centric summarization using the biencoder framework with pre-trained transformerbased models which significantly outperform past approaches to entity-centric summarization and further, outperform the challenging entity-centric lead-3 baseline in summarization tasks. ## data we use the entsum data set introduced in @xcite to evaluate our methods. the entsum data set consists of 2,788 entity-centric summaries across 645 documents annotated on top of the test split of the new york times (nyt) @xcite summarization data set. in this paper, we use 2 out of the 4 annotations in entsum: the salient sentences to the entity and the summary sentences for an entity. each entity is mentioned on average in 3.95 sentences. annotators labeled as salient sentences to the entity all sentences relevant to a given entity with an average of 5.8 sentences/entity. annotators selected sentences to compose the entity-centric summary as a subset of the salient sentences, resulting in an average of 2.49 sentences. ## task definition we define the task of extractive entity-centric summarization as selecting a set of sentences {s e 1 . . . s e k } from a single document @xmath0. . . s n }, when given the document d and a target entity e as input. this type of problem formulation is facilitated by the entsum data set, as explicit annotations exist at the sentence level. this also allows us to use standard precision, recall and f1 metrics for the evaluation of extractive summarization. to date, sentence-level classification was rare due to the complexity and resource-intensive nature of obtaining the annotations. most large singledocument summarization data sets have been collected by aligning full documents with a handwritten abstractive summary obtained from titles @xcite , bullet points @xcite summaries created for indexing purposes @xcite or tl;dr's created by scientific paper authors @xcite . the lack of sentence-level annotations required previous ex-tractive summarization methods @xcite @xcite to be trained on greedily generated weak sentence-level labels obtained using content overlap metrics such as rouge @xcite or were evaluated on abstractive summaries using overlap measures such as rouge or bertscore @xcite , which at times are unable to properly capture semantic similarity. this type of evaluation and setup is more common in multi-document extractive summarization research @xcite @xcite . evaluation using f1 is arguably more reliable and less ambiguous, albeit there are also some caveats associated with using this task setup such as granularity @xcite . ## methods we experiment with the categories of methods listed below. methods with ent in their name identify sentences containing the target entity and restrict inference to only those sentences. entities are identified by using the flair ner model @xcite and spanbert @xcite for coreference resolution, then matched to the target entity using string matching. ## results ## conclusions this paper explored the task of entity-centric extractive summarization. results showed that by leveraging sentence encoders in a bi-encoder architecture, we are able to substantially outperform previous controllable extractive summarization methods and the competitive lead3 heuristic. this method also performs well without adaptations in the auxiliary task of salient sentence extraction. future work can investigate how best to build entity representations, custom loss functions for this task and joint sentence selection across the entire document.