Composing Elementary Discourse Units in Abstractive Summarization (2020)

Abstract: In this paper, we argue that elementary discourse unit (EDU) is a more appropriate textual unit of content selection than the sentence unit in abstractive summarization. To well handle the problem of composing EDUs into an informative and fluent summary, we propose a novel summarization method that first designs an EDU selection model to extract and group informative EDUs and then an EDU fusion model to fuse the EDUs in each group into one sentence. We also design the reinforcement learning mechanism to use EDU fusion results to reward the EDU selection action, boosting the final summarization performance. Experiments on CNN/Daily Mail have demonstrated the effectiveness of our model.

## introduction abstractive summarization focuses on generating fluent and concise text from the original input document and has achieved considerable performance improvement with the rapid development of deep learning technology @xcite @xcite . in abstractive summarization, the recently popular and practical paradigm usually generates summary sentences by independently compressing or rewriting each pre-extracted sentence, which is from the source documents @xcite . however, a single document sentence usually cannot provide enough information that a summary sentence expresses, which is supported by the recent study of @xcite . they show that a high percentage of summary sentences include information from more than one document sentences, and composing a summary through only compressing sentences can cause performance degradation. simultaneously, in contrast to the brevity requirements of a summary, each document sentence usually offers trivial details and expresses a relatively independent meaning, posing difficulty of combining multiple sentences into one summary sentence. so we hope to seek a new summary composition unit which is more information-intensive and elementary than sentence. in this paper, we choose to use elementary discourse unit (edu) as the summarization unit, which is first proposed from rhetorical structure theory @xcite and defined as a clause. the finer granularity makes edu more suitable than sentence to be the basic summary composition unit @xcite . at the same time, benefited from the development of edu segmentation technology, which can achieve a high accuracy of 94% @xcite , it is feasible to automatically obtain edus from the text. next, the problems are: (1) which edus should be selected to compose a good summary? moreover, (2) how to well assemble the selected edus into a fluent summary? to solve the problems above, we need to extract the information-intensive edus from the source documents and effectively fuse the related edus into fluent summary sentences. with such an idea, inspired by @xcite 's work, we design an abstractive summarization method which is composed of two parts: edu selection and edu fusion. edu selection aims to extract informative edus and group them while edu fusion takes the grouped edus as input to generate a sentence. as the edu selection process lacks labeling training data, we apply the edu fusion results as the feedback to tune the edu selection model which in turn influences the edu fusion process. here, the actor-critic reinforcement learning algorithm is employed to train our edu-based summarization method. to the best of our knowledge, we are the first to propose a practical solution to compose edus in summarization. experiments show that compared to previous models, our edu based model achieves a significant improvement on the cnn/daily mail dataset. ## model our model is mainly composed of two modules: edu selection and edu fusion. edu selection aims to extract salient edus from the source document and group the closely related edus. here, we adopt a smart unified end-to-end method to implement both the extraction and grouping. next, edu fusion takes the edus in a group to generate a fluent and informative sentence. to train our method, we adopt reinforcement learning to leverage both the two modules. figure 1 shows the whole architecture of our method. ## learning we firstly pre-train the edu selection and edu fusion module separately and then use the pretrained model as initialization for reinforcement learning(rl). ## conclusions in this paper, we choose edu as the basic summary unit and propose a novel edu based summarization model edusum. in our model, the module of edu selection is designed to extract and group salient edus and the module of edu fusion to convert groups of edus into summary sentences. we also apply reinforcement learning to leverage edu selection and edu fusion for improving summarization performance. with such a design, edusum can fuse cross-sentence information and remedy the poor readability problem brought by edus. compared to previous work, this work has provided a feasible and effective method which makes full use of edus in summarization.