Elaborative Simplification for G erman-Language Texts (2024)

Abstract: There are many strategies used to simplify texts. In this paper, we focus specifically on the act of inserting information or elaborative simplification. Adding information is done for various reasons, such as providing definitions for concepts, making relations between concepts more explicit, and providing background information that is a prerequisite for the main content. As all of these reasons have the main goal of ensuring coherence, we first conduct a corpus analysis of simplified German-language texts that have been annotated with Rhetorical Structure Theory (RST). We focus specifically on how additional information is incorporated into the RST annotation for a text. We then transfer these insights to automatic simplification using Large Language Models (LLMs), as elaborative simplification is a nuanced task which LLMs still seem to struggle with.

## introduction there are many strategies used to simplify texts. sentences can be shortened, split or paraphrased, complex words replaced with synonyms, and information can be reordered, dropped or inserted @xcite . in this paper, we focus specifically on the act of inserting information. inserting information is done for various reasons: providing definitions for concepts, making relations between concepts more explicit, and providing background information that is a prerequisite for the main content. these all should contribute to decreasing complexity and therefore ideally ensuring coherence; the semantic or pragmatic relationships that link units in a discourse to other units @xcite . readers need to recognise these relationships in order to make sense of the text, so a more coherent text should increase comprehension and also allow readers to recognise the communicative function of the text (cf. @xcite . in our study, we focus on german-language texts and aim to transfer insights from a detailed corpus analysis to automatic simplification models, to improve their ability in inserting information and therefore their overall ability at simplification. we use a corpus of parallel newspaper articles that have been annotated with rhetorical structure theory (rst). rst annotations provide information about how segments in a text are related to each other within semantic or pragmatic relations such as cause, background, or contrast @xcite . our corpus analysis examines how inserted information in simplified texts can affect the coherence, and also what purpose the additional information has. in order to utilise these discourse structure annotations for the task at hand, we first add a new layer of annotations by labelling the transformations that are applied to the original sentences to create the simplified sentences. one of these labels is 'insert complementary information' which we focus on in more detail. we examine the role that this inserted information plays in the overall rst annotation. we then transfer these insights to automatic simplification using large language models (llms), exploring the use of different prompts. in summary, our contributions are: we extend the apa-rst corpus @xcite to include transformation labels. we show results of an extensive corpus analysis, showing how new information is inserted in text simplification, and how this affects the coherence. we then explore models for document-level text simplification for german using the insights from our corpus analysis, with results comparable with the state of the art. in section 2 we present an overview on work that has looked at the insertion of new information in simplified text. in section 3 we present our annotations of alignment labels and fine-grained inserted information categories, before presenting our rst analysis. section 4 gives details on our models and experiments with them, and we reflect on our results and possible avenues for future work in section 5. we publish our annotations and models at https://github.com/fhewett/ germanelabsimplification . another related area of text simplification is conceptual complexity, defined as accounting for 'the background knowledge necessary to understand mentioned concepts as well as the implicit connections that the reader has to access between the mentioned concepts in order to fully understand a text' @xcite . our work is also related to the field of factuality (evaluation) of language model outputs: @xcite create a taxonomy of factual errors in automatic simplification, including 'information insertion' which is described as inserting 'irrelevant or erroneous content'. they differentiate between these insertion errors and useful insertions, such as 'defin[ing] jargon or provid[ing] explanatory content'. in the field of automatic summarisation, @xcite differentiate between intrinsic and extrinsic hallucinations, where the latter refers to 'adding information not directly inferable from the input document'. they find that 'over 90% of extrinsic hallucinations were erroneous' i.e. are 'neither faithful nor factual'. @xcite al. (2020) also find factual hallucinations to be 'acceptable if they lead to better summaries that are factual with respect to the document and the associated background knowledge'. this last point is particularly relevant to the task of simplification. in various guidelines on leichte sprache (ls)a highly simplified rule-based version of german -inserting factual information is allowed and also even desirable, in order to increase the level of comprehension on the one hand, and to allow readers to potentially learn new information on the other hand @xcite . the guidelines state that translators of ls are allowed to provide explanations, additional remarks, and (concrete) examples, in order to make abstract concepts or difficult words more comprehensible. @xcite does however state that translators, after adding these definitions, explanations and examples, should make sure that the text still has an argumentative flow. bredel (2016) state that additional explanations in texts in ls can hinder the flow of the text and potentially also cause problems on the text level. these aspects are the specific focus of the current study, i.e. what happens to the structure and coherence of the text overall when elaborative simplification is used. other corpus studies which focus on the transformation operations between non-simplified and simplified text often define an operation for inserting information. this category encompasses subcategories such as inserting eliciting information, complementary external information, spurious information, pre-requisite information, concrete examples of abstract concepts or phenomenona @xcite @xcite . this category has also been used in german-language corpus studies: @xcite manually align parallel texts with a category for additional information and @xcite find that phrases such as 'for example' or 'that means' feature heavily in simplified texts and are used for explaining difficult words, making abstract concepts more concrete and connecting the sentences of a text explicitly. ## corpus analysis the main corpus we work with is the apa-rst. the corpus consists of german-language newspaper articles, which are classified as being at b1 and a2 level, according to the common european framework of reference for languages (cefr), which is a scale from a1 (beginner language learner) to c2 (native speaker). there are 75 parallel articles in the corpus, with 25 at each level (original foot_0 , b1 and a2), covering various topics such as politics, culture and sport. the articles have been annotated with rst and manually aligned at sentence level; further information can be found in the original publication @xcite . due to the relatively small sample size, we extend our analysis to label 200 instances of the 'apa' subcorpus of deplain @xcite which features a larger number of newspaper articles from the same publisher as apa-rst. this subcorpus has been aligned at the sentence level, between the versions b1 and a2. ## automatic simplification models we use meta-llama-3-8b-instruct for our experiments as it is one of the most capable openweight llms at the time of writing and performs well in benchmarks. 3 additionally, llms that have been trained using strategies such as instructiontuning and rlhf (as is the case for llama-3) have been found to perform well in the task of automatic sentence simplification @xcite . we use meta-llama-3-8b-instruct out-of-the-box, and also use this base model to fine-tune on b1 texts and a2 texts. we then explore using different prompts which are influenced by the findings from 3 more information can be found on the model card on huggingface: https://github.com/meta-llama/llama3/ blob/main/model_card.md our corpus analysis. ## conclusion and outlook we have presented an in-depth analysis of elaborative simplification in german-language texts, using rst annotations and more fine-grained categories. we have experimented with using these insights to improve an llm's ability to produce elaborative simplifications. our fine-tuned model and our different prompts do encourage the model to insert additional information (see as our analysis showed, not all simplified texts contain additional information and certainly not all types of additional information. in most cases, just one type is necessary, i.e. for texts of a political nature, more background knowledge and the relation between the entities in the text may be more important for understanding the text. future work could investigate on selecting a prompt dependent on the input text. adding new information is not trivial; as can be seen in figure 1 , making relations more explicit, for example, can also slightly change the content of a text. in figure 1b , segment 9 leaves some room for interpretation, as 'disagreeing' is not specific, whereas segments 4 to 7 in figure 1a make this 'disagreement' very concrete. by keeping content more open and vague, it is easier to stay 'factual', showing that it is a fine line between making relations explicit and staying factual. overall, elaborative or additive simplification remains a challenging sub-task of automatic simplification. as shown by our manual evaluation, factuality and meaning preservation seem to represent separate requirements. we therefore advocate for factuality being included as a separate and additional evaluation criterion for text simplification, as up until now faithfulness and factuality seem to have been used interchangeably in the simplification literature, and simplifications are often (manually) evaluated for their meaning preservation (i.e. faithfulness). our experiments have been limited to fine-tuning and prompting approaches, but experiments which alter the training/fine-tuning paradigm and loss function could also be promising, as at the moment 'most summarization [and simplification] systems are trained to maximize the log-likelihood of the reference summary at the word-level, which does not necessarily reward models for being faithful' @xcite . as parts of our dataset are available online, we do not know if the data was part of the dataset used to pre-train llama-3.