Extractive Summarization as Text Matching (2020)

Abstract: This paper creates a paradigm shift with regard to the way we build neural extractive summarization systems. Instead of following the commonly used framework of extracting sentences individually and modeling the relationship between sentences, we formulate the extractive summarization task as a semantic text matching problem, in which a source document and candidate summaries will be (extracted from the original text) matched in a semantic space. Notably, this paradigm shift to semantic matching framework is well-grounded in our comprehensive analysis of the inherent gap between sentence-level and summary-level extractors based on the property of the dataset. Besides, even instantiating the framework with a simple form of a matching model, we have driven the state-of-the-art extractive result on CNN/DailyMail to a new level (44.41 in ROUGE-1). Experiments on the other five datasets also show the effectiveness of the matching framework. We believe the power of this matching-based summarization framework has not been fully exploited. To encourage more instantiations in the future, we have released our codes, processed dataset, as well as generated summaries in https://github.com/maszhongming/MatchSum .

## introduction the task of automatic text summarization aims to compress a textual document to a shorter highlight while keeping salient information on the original text. in this paper, we focus on extractive summarization since it usually generates semantically and grammatically correct sentences @xcite and computes faster. currently, most of the neural extractive summarization systems score and extract sentences (or smaller semantic unit @xcite ) one by figure 1 : matchsum framework. we match the contextual representations of the document with gold summary and candidate summaries (extracted from the document). intuitively, better candidate summaries should be semantically closer to the document, while the gold summary should be the closest. the above systems of modeling the relationship between sentences are essentially sentence-level extractors, rather than considering the semantics of the entire summary. this makes them more inclined to select highly generalized sentences while ignoring the coupling of multiple sentences. @xcite ; @xcite utilize reinforcement learning (rl) to achieve summarylevel scoring, but still limited to the architecture of sentence-level summarizers. to better understand the advantages and limitations of sentence-level and summary-level approaches, we conduct an analysis on six benchmark datasets (in section 3) to explore the characteristics of these two methods. we find that there is indeed an inherent gap between the two approaches across these datasets, which motivates us to propose the following summary-level method. in this paper, we propose a novel summary-level framework (matchsum, figure 1 ) and conceptualize extractive summarization as a semantic text matching problem. the principle idea is that a good summary should be more semantically similar as a whole to the source document than the unqualified summaries. semantic text matching is an important research problem to estimate semantic similarity between a source and a target text fragment, which has been applied in many fields, such as information retrieval @xcite , question answering @xcite , natural language inference @xcite and so on. one of the most conventional approaches to semantic text matching is to learn a vector representation for each text fragment, and then apply typical similarity metrics to compute the matching scores. specific to extractive summarization, we propose a siamese-bert architecture to compute the similarity between the source document and the candidate summary. siamese bert leverages the pre-trained bert @xcite in a siamese network structure @xcite @xcite to derive semantically meaningful text embeddings that can be compared using cosine-similarity. a good summary has the highest similarity among a set of candidate summaries. we evaluate the proposed matching framework and perform significance testing on a range of benchmark datasets. our model outperforms strong baselines significantly in all cases and improve the state-of-the-art extractive result on cnn/dailymail. besides, we design experiments to observe the gains brought by our framework. we summarize our contributions as follows: 1) instead of scoring and extracting sentences one by one to form a summary, we formulate extractive summarization as a semantic text matching problem and propose a novel summary-level framework. our approach bypasses the difficulty of summary-level optimization by contrastive learning, that is, a good summary should be more semantically similar to the source document than the unqualified summaries. ## sentence-level or summary-level? a dataset-dependent analysis although previous work has pointed out the weakness of sentence-level extractors, there is no systematic analysis towards the following questions: 1) for extractive summarization, is the summarylevel extractor better than the sentence-level extractor? 2) given a dataset, which extractor should we choose based on the characteristics of the data, and what is the inherent gap between these two extractors? in this section, we investigate the gap between sentence-level and summary-level methods on six benchmark datasets, which can instruct us to search for an effective learning framework. it is worth noting that the sentence-level extractor we use here doesn't include a redundancy removal process so that we can estimate the effect of the summarylevel extractor on redundancy elimination. notably, the analysis method to estimate the theoretical effectiveness presented in this section is generalized and can be applicable to any summary-level approach.