Abstract: Existing approaches to automatic summarization assume that a length limit for the summary is given, and view content selection as an optimization problem to maximize informativeness and minimize redundancy within this budget. This framework ignores the fact that human-written summaries have rich internal structure which can be exploited to train a summarization system. We present NEXTSUM, a novel approach to summarization based on a model that predicts the next sentence to include in the summary using not only the source article, but also the summary produced so far. We show that such a model successfully captures summary-specific discourse moves, and leads to better content selection performance, in addition to automatically predicting how long the target summary should be. We perform experiments on the New York Times Annotated Corpus of summaries, where NEXTSUM outperforms lead and content-model summarization baselines by significant margins. We also show that the lengths of summaries produced by our system correlates with the lengths of the human-written gold standards.

Introduction: Writing a summary is a different task compared to producing a longer article. As a consequence, it is likely that the topic and discourse moves made in summaries differ from those in regular articles. In this work, we present a powerful extractive summarization system which exploits rich summary-internal structure to perform content selection, redundancy reduction, and even predict the target summary length, all in one joint model. Text summarization has been addressed by numerous techniques in the community BIBREF0 . For extractive summarization, which is the focus of this paper, a popular task setup is to generate summaries that respect a fixed length limit. In the summarization shared tasks of the past Document Understanding Conferences (DUC), these limits are defined in terms of words or bytes. As a result, much work has framed summarization as a constrained optimization problem, in order to select a subset of sentences with desirable summary qualities such as informativeness, coherence, and non-redundancy within the length budget BIBREF1 , BIBREF2 , BIBREF3 . One problem with this setup is that it does not match many real-world summarization settings. For example, writers can tailor the length of their summaries to the amount of noteworthy content in the source article. Summaries created by news editors for archives, such as the New York Times Annotated Corpus BIBREF4 , exhibit a variety of lengths. There is also evidence that in the context of web search, people prefer summaries of different lengths for the documents in search results depending on the type of the search query BIBREF5 . More generally, current systems focus heavily on properties of the source document to learn to identify important sentences, and score the coherence of sentence transitions. They reason about the content of summaries primarily for purposes of avoiding redundancy, and respecting the length budget. But they ignore the idea that it might actually be useful to learn content structure and discourse planning for summaries from large collections of multi-sentence summaries. This work proposes an extractive summarization system that focuses on capturing rich summary-internal structure. Our key idea is that since summaries in a domain often follow some predictable structure, a partial summary or set of summary sentences should help predict other summary sentences. We formalize this intuition in a model called NextSum, which selects the next summary sentence based not only on properties of the source text, but also on the previously selected sentences in the summary. An example choice is shown in Table 1 . This setup allows our model to capture summary-specific discourse and topic transitions. For example, it can learn to expand on a topic that is already mentioned in the summary, or to introduce a new topic. It can learn to follow a script or discourse relations that are expected for that domain's summaries. It can even learn to predict the end of the summary, avoiding the need to explicitly define a length cutoff. The core of our system is a next-sentence prediction component, which is a feed-forward neural network driven by features capturing the prevalence of domain subtopics in the source and the summary, sentence importance in the source, and coverage of the source document by the summary so far. A full summary can then be generated by repeatedly predicting the next sentence until the model predicts that the summary should end. Since summary-specific moves may depend on the domain, we first explore domain-specific summarization on event-oriented news topics (War Crimes, Assassinations, Bombs) from the New York Times Annotated Corpus BIBREF4 . We also train a domain-general model across multiple types of events. NextSum predicts the next summary sentence with remarkably high accuracies, reaching 67% compared to a chance accuracy of 9%. The generated summaries outperform the lead baseline as well as domain-specific summarization baselines without requiring explicit redundancy check or a length constraint. Moreover, the system produces summaries of variable lengths which correlate with how long human summaries are for the same texts.

Question: Can the approach be generalized to other technical domains as well? 
Answer: There is no reason to think that this approach wouldn't also be successful for other technical domains. Technical terms are replaced with tokens, therefore so as long as there is a corresponding process for identifying and replacing technical terms in the new domain this approach could be viable.