Deep Differential Amplifier for Extractive Summarization (2021)

Abstract: For sentence-level extractive summarization, there is a disproportionate ratio of selected and unselected sentences, leading to flatting the summary features when maximizing the accuracy. The imbalanced classification of summarization is inherent, which can’t be addressed by common algorithms easily. In this paper, we conceptualize the single-document extractive summarization as a rebalance problem and present a deep differential amplifier framework. Specifically, we first calculate and amplify the semantic difference between each sentence and all other sentences, and then apply the residual unit as the second item of the differential amplifier to deepen the architecture. Finally, to compensate for the imbalance, the corresponding objective loss of minority class is boosted by a weighted cross-entropy. In contrast to previous approaches, this model pays more attention to the pivotal information of one sentence, instead of all the informative context modeling by recurrent or Transformer architecture. We demonstrate experimentally on two benchmark datasets that our summarizer performs competitively against state-of-the-art methods. Our source code will be available on Github.

## introduction single-document extractive summarization forms summary by copying and concatenating the most important spans (usually sentences) in a document. sentence-level summarization is a very challenging task, because it arguably requires an in-depth understanding of the source document sentences, and current automatic solutions are still far from human performance. recent approaches frame the task as a sequence labeling problem, taking advantage of the success of neural network architectures. however, there are still two inherent obstacles for sentence-level extractive summarization: ## conclusion in this paper, we introduce a heuristic model, dif-fersum, 1) to calculate and amplifier the pivotal information and 2) to rebalance the distribution of minority 1-class and majority 0-class. besides, we employ another weighted cross-entropy function to compensate for the imbalance. experimental results show that our method significantly outperforms previous models. in the future, we would like to generalize differsum to other fields.