Separating Context and Pattern: Learning Disentangled Sentence Representations for Low-Resource Extractive Summarization (2023)

Abstract: Extractive summarization aims to select a set of salient sentences from the source document to form a summary. Context information has been considered one of the key factors for this task. Meanwhile, there also exist other pattern factors that can identify sentence importance, such as sentence position or certain n-gram tokens. However, such pattern information is only effective in specific datasets or domains and can not be generalized like the context information when there only exists limited data. In this case, current extractive summarization models may suffer from a performance drop when transferring to a new dataset. In this paper, we attempt to apply disentangled representation learning on extractive summarization, and separate the two key factors for the task, context and pattern, for a better generalization ability in the low-resource setting. To achieve this, we propose two groups of losses for encoding and disentangling sentence representations into context representations and pattern representations. In this case, we can either use only the context information in the zero-shot setting or fine-tune the pattern information in the few-shot setting. Experimental results on three summarization datasets from different domains show the effectiveness of our proposed approach.

## introduction the glob of text summarization is to generate a concise highlight of a source document, which covers the crucial information conveyed in the source text. in this paper, we focus on extractive summarization. it aims to produce summaries by selecting and combining the salient sentences that are directly taken from the source text. it is widely agreed that extractive summarization is mainly based on context information to select the important sentences. meanwhile, there also exist other factors that can be used to identify these sentences, such as sentence position or certain ngram tokens. as shown in figure 1 , in the news ## conclusion in this paper, we propose a novel extractive summarization model that aims to improve the generalization ability in low-resource setting. it disentangles the sentence representation to context and pattern representation and utilize the context information to reduce the influence of domain-specific pattern information during model transferring. the experiment suggests the ability of our model in the disentanglement, and it also supports the claim that the context information tends to have better generalization ability facing the dataset from a different domain. in the future, we plan to extend this idea by learning a more generalized context latent space from multiple summarization datasets.