End-to-End Segmentation-based News Summarization (2022)

Abstract: In this paper, we bring a new way of digesting news content by introducing the task of segmenting a news article into multiple sections and generating the corresponding summary to each section. We make two contributions towards this new task. First, we create and make available a dataset, SegNews, consisting of 27k news articles with sections and aligned heading-style section summaries. Second, we propose a novel segmentation-based language generation model adapted from pre-trained language models that can jointly segment a document and produce the summary for each section. Experimental results on SegNews demonstrate that our model can outperform several state-of-the-art sequence-to-sequence generation models for this new task.

## introduction in recent years, automatic summarization has received extensive attention in the natural language processing community, due to its potential for processing redundant information. the evolution of neural network models and availability of largescale datasets have driven the rapid development of summarization systems. despite promising results, there are specific characteristics of the traditional summarization task that impedes it to provide more beneficial ways of digesting long news articles. for instance, current news summarization system only provides one genetic summary of the whole article, and when users want to read in more details, the generated summary is not capable of helping navigate the reading. for example, given a news report, current system will output several highlight summaries @xcite @xcite . under this circumstance, if a user expect to read more details about one highlight, he will still need to browse the whole article to locate related paragraphs. meanwhile, when processing a long news article, current systems usually truncate the text and only generate a summary based on the partial article @xcite . although this is reasonable since most important content usually lies in the initial portion, it also makes it difficult for users to quickly access information beyond the truncated portion. in this paper, we propose a new task of segmentation-based news summarization. given a news article, we aim to identify its potential sections and at the same time, to generate the corresponding summary for each section. this new task provides a novel alternative to summarizing a news article. we argue that it can lead to a more organized way of understanding long articles and facilitates a more effective style of reading documents. first, segmenting a news article can provide a structural organisation of the content, which is not only helpful to reading but also benefit many important nlp tasks. for example, @xcite states that this kind of multi-paragraph division is one of the most fundamental tasks in discourse. however, many expository texts, like news articles, instruction manuals, or textbooks consist of long sequences of paragraphs with very little structural demarcation @xcite , and for these documents a subtopical segmentation can be useful. second, generating concise text descriptions of each sections further reduces the cognitive burden of reading the article @xcite . previous studies @xcite present that subtopic segments with their headings is an effective alternative to traditional summarization tasks. in this paper, we make two main contributions towards the development of segmentation-based news summarization systems. 1 first, we create and publicize a large-scale ## task formulation given a multi-paragraph article, the segmentationbased summarization task aims to: i) identify sections of the article to unveil its inherent sub-topic structure, where each section consists of neighboring paragraphs with a coherent topic, and ii) generate the heading-style summary for each section to concisely summarize the section. particularly, in one article, summaries of different sections should be coherent in content and consistent in style. formally, let d indicate a document consisting of paragraphs [p 1 , p 2 , ..., p m ]. meanwhile, summarization systems will generate the corresponding section summaries [y 1 , y 2 , ..., y n ]. ## systems for segmentation-based news summarization in this section, we present two different frameworks to tackle the segmentation-based summarization task. in pipeline approach, we first apply a segmentation model to identify potential sections, and then apply a generation model to produce the headings. in joint approach, one neural model is able to jointly segment an article and produce the summaries. to achieve this, we design a novel segmentation-aware attention mechanism, which allows the model to capture segmentation information when generating summaries. this new attention mechanism can also be considered as a non-invasive adaption for conventional transformer models. thus, to take the most advantage of existing pre-trained models, we propose segu-nilm and segbart which are respectively based on pre-trained unilm model and bart model. they can be initialized completely from pre-trained models and achieve substantial improvement on segmentation-based summarization. ## experiments in this section, we conduct experiments on seg-news dataset by comparing our proposed model with several strong baselines. ## conclusion in this work, we proposed a new task, segmentationbased news summarization. it aims to segment a news article into multiple sections and generate the corresponding summary to each section. this new task provides a novel alternative to digesting a news article. we built a new benchmark dataset segnews to study and evaluate the task. furthermore, we designed a segmentation-aware attention mechanism, which allows neural decoder to capture segmentation information in the source texts. we jointly train the model for generating summaries and recognizing news segments. experimental results on segnews demonstrate that our framework produces better segmentation-based summaries than competitive systems.