TESA : A T ask in E ntity S emantic A ggregation for Abstractive Summarization (2020)

Abstract: Human-written texts contain frequent generalizations and semantic aggregation of content. In a document, they may refer to a pair of named entities such as ‘London’ and ‘Paris’ with different expressions: “the major cities”, “the capital cities” and “two European cities”. Yet generation, especially, abstractive summarization systems have so far focused heavily on paraphrasing and simplifying the source content, to the exclusion of such semantic abstraction capabilities. In this paper, we present a new dataset and task aimed at the semantic aggregation of entities. TESA contains a dataset of 5.3K crowd-sourced entity aggregations of Person, Organization, and Location named entities. The aggregations are document-appropriate, meaning that they are produced by annotators to match the situational context of a given news article from the New York Times. We then build baseline models for generating aggregations given a tuple of entities and document context. We finetune on TESA an encoder-decoder language model and compare it with simpler classification methods based on linguistically informed features. Our quantitative and qualitative evaluations show reasonable performance in making a choice from a given list of expressions, but free-form expressions are understandably harder to generate and evaluate.

## introduction abstractly speaking, abstraction can be defined as the process of deriving general concepts from specific instances. in automatic summarization, however, "abstractive" summarization often means any type of rewriting of words in some source document into an output summary. concretely, recent summarization datasets including xsum (narayan ## related work abstractive summarizers have gained prominence with the popularization of rnns @xcite , and more recently transformers @xcite like bert @xcite . several abstractive models have achieved state-of-the-art performances on benchmark summarization datasets in terms of rouge, including prophetnet @xcite , pega-sus @xcite and bart @xcite . recent work has also focused on specific issues such as preventing inappropriate repetition @xcite , word-level rewriting, and evaluating factual consistency @xcite . abstraction is critical for certain domains and applications, but has not been thoroughly explored in many. for example, in scientific article summarization the particular structure and length of scientific articles make extractive techniques much easier to apply @xcite , therefore abstractive summarizers @xcite remain a minority. in opinion summarization, there have been abstractive systems that leverage cues specific to this task, such as redundancy in opinions @xcite and specific discourse structures @xcite . as abstractive systems have become strong in terms of generation capabilities, the time is apt to examine issues in semantic abstraction that could be useful in many summarization domains and tasks. our work is a step in this direction. our proposed entity aggregation task is related to referring expression generation (reg). reg is concerned with determining the form and content that entity references should take during generation @xcite @xcite . it emphasizes finding the right distinguishing characteristics of the intended referent or referents. our work can be seen as a specific reg task that focuses on semantically abstracting multiple named entities. our work is also related to coreference resolution, especially those that examine multi-antecedent resolution @xcite , an inverse problem to ours. to the best of our knowledge, no previous work has directly addressed entity aggregation. ## the tesa dataset we used the new york times (nyt) annotated corpus @xcite to extract tuples of named entities and their document context. the nyt corpus contains high-quality metadata listing the salient named entities mentioned in each article. we form our tuples from entities tagged in the metadata for the same article. we refer to a tuple of entities and its associated information as an aggregatable instance. we first describe the components of an aggregatable instance in more detail. then, we describe our data extraction and crowd-sourcing experiments. ## models we tested several simple baselines as well as models adapted from current work on abstractive summarization on tesa. ## results table ## conclusion and future work we have proposed tesa, a novel task and an accompanying dataset of crowd-sourced entity aggregations in context. tesa directly measures the ability of summarizers to abstract at a semantic level. we have compared several baseline models and models adapted from existing abstractive summarizers on tesa, and find that a discriminative fine-tuning achieves the best performance, though this model inherently cannot generate aggregations. in future work, we would like to expand the domains covered by our dataset, which is biased towards topics found in the source corpus, such as politics. another important direction is to investigate how to integrate the ability to aggregate entities derived from training on tesa into an abstractive summarizer. this would require models to tackle another challenging issue which we have not addressed: which set of entities should a model aggregate in the first place? aggregatable instance aggregations input entities chicago and london entity type location background information chicago: chicago , locally also ), officially the city of chicago, is the most populous city in the u.s. state of illinois and the third most populous city in the united states. with an estimated population of 2,705,994 , it is also the most populous city in the midwestern united states. [...] london: london is the capital and largest city of england and the united kingdom. standing on the river thames in the south-east of england, at the head of its 50-mile estuary leading to the north sea, london has been a major settlement for two millennia. [...] context virtually cool: the author of the hour was chris anderson, who after the drinks entertained the crowd with a simulcast powerpoint lecture on the topic of his new best seller," the long tail," which describes how the chokehold of mass culture is being loosened by the new internet-enabled economics of niche culture and niche commerce. the party was sponsored in part by a small soho-based new-media company called flavorpill, which produces free e-mail magazines and weekly event guides for new york, los angeles, san francisco, chicago and london. c 2008 the new york times company, used with permission annotation 1 • major metropolitan cities annotation 2 • cities annotation 3 • the cities • the major cities input entities microsoft corp. and sony corp entity type organization background information microsoft corp.: microsoft corporation is an american multinational technology company with headquarters in redmond, washington. it develops, manufactures, licenses, supports, and sells computer software, consumer electronics, personal computers, and related services. its best known software products are the microsoft windows line of operating systems, the microsoft office suite, and the internet explorer and edge web browsers. [...] sony corp.: sony corporation is a japanese multinational conglomerate corporation headquartered in kōnan, minato, tokyo. its diversified business includes consumer and professional electronics, gaming, entertainment and financial services. context battleground for consoles moves online: over all, though, it is microsoft that has had the steeper mountain to climb. in the last generation of video game consoles, sony had a roughly 60 percent market share, compared to 20 percent for each microsoft and nintendo. c 2008 the new york times company, used with permission annotation 1 • the technology companies annotation 2