The Summary Loop: Learning to Write Abstractive Summaries Without Examples (2020)

Abstract: This work presents a new approach to unsupervised abstractive summarization based on maximizing a combination of coverage and fluency for a given length constraint. It introduces a novel method that encourages the inclusion of key terms from the original document into the summary: key terms are masked out of the original document and must be filled in by a coverage model using the current generated summary. A novel unsupervised training procedure leverages this coverage model along with a fluency model to generate and score summaries. When tested on popular news summarization datasets, the method outperforms previous unsupervised methods by more than 2 R-1 points, and approaches results of competitive supervised methods. Our model attains higher levels of abstraction with copied passages roughly two times shorter than prior work, and learns to compress and merge sentences without supervision.

## introduction summarization, or the task of condensing a document's main points into a shorter document, is important for many text domains, such as headlines for news and abstracts for research papers. this paper presents a novel unsupervised abstractive summarization method that generates summaries directly from source documents, without the aid of example summaries. this approach simultaneously optimizes for the following important properties of a good summary: original document: chilean president announced wednesday that his country, which has been paralyzed by protests over the last two weeks, will no longer host two major international summits. [...] the president has now canceled the hosting of the economic apec forum and cop25 environmental summit, which were both due to take place later this year. [...] ## the summary loop for this work, the definition of a summary is: "a summary is a brief, fluent text that covers the main points of an original document." brevity, fluency and coverage are the three pillars of a good summary. under a length constraint, a good quality summary should contain as much information about the original document as possible while retaining fluent and coherent english. subsection 3.1 lays out the steps in the summary loop. subsections 3.2-3.5 specify how each component is represented by a neural network. section 4 shows how to train a summarizer model using this architecture in an unsupervised manner. foot_0