Extractive Summarisation for G erman-language Data: A Text-level Approach with Discourse Features (2022)

Abstract: We examine the link between facets of Rhetorical Structure Theory (RST) and the selection of content for extractive summarisation, for German-language texts. For this purpose, we produce a set of extractive summaries for a dataset of German-language newspaper commentaries, a corpus which already has several layers of annotation. We provide an in-depth analysis of the connection between summary sentences and several RST-based features and transfer these insights to various automated summarisation models. Our results show that RST features are informative for the task of extractive summarisation, particularly nuclearity and relations at sentence-level.

## introduction extractive summarisation involves directly using select phrases and sentences from a text as a summary, which still remains a strong method for producing summaries despite its simple nature @xcite . in this study, we examine the link between facets of rhetorical structure theory (rst) and the selection of content for extractive summarisation. rst is a framework which posits that every part of coherent text has a role and a function and represents texts in a hierarchical tree structure @xcite . the rst framework consists of the segmentation of the text into elementary discourse units (edus), which are then grouped into bigger segments -depending on the way they relate to each other -which forms the hierarchical structure of the text. the relations between these segments are defined, and a nucleus-satellite status is given. this nucleus-satellite allocation stems from the observation that within the majority of relations which hold between two segments, one segment tends to be more important than the other. this notion of importance seems to have an inherent link to summarisation and various studies have examined this link (see section 2). in this study we look at the role that these various aspects play in extractive summarisation for german-language texts and transfer these insights to different types of models. in this context, we introduce a new dataset of extractive summaries, analyse the rstbased features of these, collating the best features proposed over the last 20+ years and introducing a new document-based sentence embedding and use these in both linear and nuclear models. our results compare favourably to those from closely related work on english @xcite . we elaborate on this study and other related work in section 2, before describing our german-language extractive summarisation dataset in section 3. in section 4, we provide a detailed analysis of various rst-based features and examine how these features are distributed in the summaries. we describe our summarisation models and the results of our experiments in section 5 before discussing the results and providing some concluding remarks in sections 6 and 7. ## related work since the rst framework was proposed in the late 1980s @xcite , various studies have examined the link between discourse structure and summarisation, building on even earlier concepts of text-level structural analysis ('macrostructures') @xcite . a study from @xcite built on ideas proposed by @xcite and empirically analysed the link between the nuclearity aspect of rst and extractive summarisation on a small sample of five texts. @xcite concluded that there is a strong correlation between nuclei and what readers perceive to be the most important units in a given text, and implemented an automatic summarisation system using rst trees. @xcite conducted an analysis of the rst-dt corpus which contains newspaper articles with various annotation layers including rst and other discourse structures. @xcite specifically evaluated a subset of the corpus which consists of 150 extractive summaries, for which annotators were asked to select the most important edus. they analysed structural features derived from rst trees, such as the depth of a segment, in comparison to other discourse (e.g. the semantics of pdtb relations) and non-discourse features and found that the structural rst features were most useful for automatically selecting sentences for summaries. zhong et al. (2020) used automatically parsed discourse features for the task of sentence deletion for text simplification, a task which shares many similarities with extractive summarisation. they used rst-based features -such as local nuclearity, relations and the position of the sentence in the tree -in both a linear model and a neural model. other studies instead looked specifically at the role that discourse segmentation plays in extractive summarisation. @xcite automatically parsed a corpus of english language newspaper articles and compared the rst segmentation to manually annotated summary content units. molina @xcite also looked at discourse segmentation of spanish-language texts for the task of sentence compression, which they consider to be sentence-level summarisation. neural approaches have also been proposed for combining discourse structure with englishlanguage summarisation. xu et al. (2020) created a bert-based model which takes discourse units as input (as opposed to sentences) and also encodes automatically parsed rst trees in a cnn layer in the network, which resulted in an improvement to the state-of-the-art for english-language extractive summarisation. @xcite experimented with three different neural architectures and compare using sentences as input to discourse segments for the task of extractive summarisation. the models which use discourse segments score higher in an automatic evaluation. the link between summarisation and discourse structure is further explored by @xcite . they hypothesise that the link between the two may be bidirectional and analyse if summarisation can inform discourse structure by generating rst trees from the inner layers of a transformer-based summarisation model. ## data as far as we know, the potsdam commentary corpus (pcc) is the only german-language dataset with rst annotations @xcite . the corpus consists of 176 commentaries from a german regional daily newspaper, the märkische allgemeine zeitung. the commentaries have various layers of annotation, including part-of-speech, syntax and discourse structure but does not yet have summaries; we therefore created these ourselves. each extractive summary consists of 3 key sentences; we use the term 'key sentence' throughout this paper to refer to the sentences selected to be part of the summaries. the commentaries have an average length of 11.4 sentences, so the summaries represent ca. 26% of the average length (in sentences). the extractive summaries available for the rst-dt corpus, used for example in the study by @xcite , are of a similar size (the square root of the number of edus for each text) which makes the corpora easier to compare. we make the summaries publicly available foot_0 , and a sample annotated text can be seen in ## analysis to investigate a potential link between rst and extractive summarisation for german-language texts, we examine various rst-based features of the sentences that were chosen in the annotation task. this builds on the studies by @xcite and zhong et al. (2020) (as outlined in section 2). ## summarisation models we use these rst features and non-discourse features in linear and neural models to predict which sentences should be kept for an extractive summary. we frame extractive summarisation as a binary classification task, predicting whether each sentence should be included in a summary or not. our dataset consists of 167 texts and 1894 sentences; we use 30% of the dataset as a test set. ## discussion the results of the analysis, particularly section 4.4 and figure 3 , show that simply looking at nuclearity in isolation may not be sufficient for some downstream tasks, as we have background satellites and cause satellites featuring heavily in the summaries, for example. this is reflected in the results: the best combination of features for both the ffn and lr model is nuclearity (local and global, with most-nuclearity measuring global nuclearity) in combination with relations (and sentence embeddings). the bi-lstm model and our sentence embeddings with additional document context both perform worse than other models and features: this goes against our intuition that more text-level context is beneficial for such a task as extractive summarisation. we leave it to future work to examine if these embeddings could work in other settings, perhaps with larger datasets or pre-trained in a different manner. simple models (such as lr) perform well, which suggests that rst features (particularly nuclearity) are strong indicators of importance. combinations of rst features perform even better than sentence embeddings on their own. as can be seen in ## conclusion in this study we have introduced a new set of extractive summaries for the potsdam commentary corpus; texts which have already been annotated with various linguistic features such as discourse structure, co-reference and syntax. we have shown the connection between rst-based features and sentences chosen for extractive summaries and have transferred these to various models. our feature ablation experiments could provide useful insights for research on rst parsers for specific downstream tasks: by finding the aspect of rst which is most useful for a task such as summarisation, the parsers can be streamlined and will have less room for error. for example, one of our top models has sentencelevel nuclearity and relation information, showing that in this context, the additional rst tree structure is potentially not necessary. we hope that our dataset will enable research on the link between summarisation and other linguistic features. our analysis using manually annotated discourse structure also provides the necessary evidence to foster research on automatically parsed discourse structure and downstream tasks, such as summarisation.