Learning From the Source Document: Unsupervised Abstractive Summarization (2022)

Abstract: Most of the state-of-the-art methods for abstractive text summarization are under supervised learning settings, while heavily relying on high-quality and large-scale parallel corpora. In this paper, we remove the need for reference summaries and present an unsupervised learning method SCR (Summarize, Contrast and Review) for abstractive summarization, which leverages contrastive learning and is the first work to apply contrastive learning for unsupervised abstractive summarization. Particularly, we use the true source documents as positive source document examples, and strategically generated fake source documents as negative source document examples to train the model to generate good summaries. Furthermore, we consider and improve the writing quality of the generated summaries by guiding them to be similar to human-written texts. The promising results on extensive experiments show that SCR outperforms other unsupervised abstractive summarization baselines, which demonstrates its effectiveness.

## introduction given a source document, summarization aims at generating a shorter text version that retains the most salient information @xcite @xcite @xcite . while extractive summarization methods directly copy and group important sections (e.g., words, phrases or sentences) from the source documents @xcite , abstractive summarization involves rewriting and paraphrasing to generate summaries with novel words or sentences @xcite @xcite . under supervised training settings, abstractive summarization methods require paired data (i.e., document-reference summary pairs) @xcite . however, obtaining high-quality and largescale datasets for supervised training is labori- ## methodology the framework of the proposed scr for unsupervised abstractive summarization is shown in fig. 1 , which includes a summarizer f sum , a contrastive encoder f con and a writing reviewer f wri . the summarizer aims at generating summaries given the source documents, with the learning feedback from the contrastive encoder and writing reviewer: finally, both the contrastive encoder and writing reviewer guide the summarizer to generate good summaries under unsupervised learning settings. ## conclusion in this paper, we propose scr (summarize, contrast and review) for unsupervised abstractive summarization. the summarizer is trained to generate summaries and the contrastive encoder guides the generate summaries close to the source documents semantically via contrastive learning. a writing reviewer is applied to ensure the writing quality of the generated summaries. moreover, we design different strategies to generate negative source document examples for contrastive learning. results on extensive experiments show the effectiveness of scr. in future work, we hope to study our model on transferable feature learning and semi-supervised learning with the advantages of paired data.