Exploring G erman Multi-Level Text Simplification (2021)

Abstract: We report on experiments in automatic text simplification (ATS) for German with multiple simplification levels along the Common European Framework of Reference for Languages (CEFR), simplifying standard German into levels A1, A2 and B1. For that purpose, we investigate the use of source labels and pretraining on standard German, allowing us to simplify standard language to a specific CEFR level. We show that these approaches are especially effective in low-resource scenarios, where we are able to outperform a standard transformer baseline. Moreover, we introduce copy labels, which we show can help the model make a distinction between sentences that require further modifications and sentences that can be copied as-is.

## introduction simplified language is a variety of standard language characterized by reduced lexical and syntactic complexity, the addition of explanations for difficult concepts, and clearly structured layout. 1 among the target groups of simplified language are people with cognitive impairment and autism spectrum disorder, prelingually deaf and functionally illiterate people, and sometimes also foreign language learners and children @xcite . automatic text simplification (ats), the process of automatically producing a simplified version of a standard-language text, was initiated in the late 1990s @xcite and has since then been approached by means of rule-based and statistical approaches. as part of the rule-based paradigm, the operations carried out typically include replacing complex lexical and syntactic units by simpler ones. the statistical paradigm so far has mainly conceptualized the simplification task as a case of monolingual (sentencebased) machine translation (mt), i.e., as one of converting standard-language into simplified-language sentences using mt techniques @xcite . however, while in bilingual parallel texts used for mt, the relation between source and target sentences is mostly 1:1, ats usually requires n:m alignments with unaligned parts in-between. ats research has been documented for english @xcite , spanish @xcite , portuguese (aluisio and @xcite , french @xcite , italian @xcite , and other languages. research on german is still sparse but has gained momentum in recent years due to a number of legal and political developments in german-speaking countries, such as the introduction of a set of regulations for accessible information technology (barrierefreie-informationstechnik-verordnung, bitv 2.0) in germany, the approval of rules for accessible information and communication (barrierefreie information und kommunikation, bik) in austria, and the ratification of the united nations convention on the rights of persons with disabilities (un crpd) in switzerland. in this paper, we report on work in automatic simplification of standard german into three separate simplification levels (a1, a2, b1) using a sentence-based mt approach. we show that the use of source-side labels indicating the targeted level of simplification benefited performance. furthermore, pretraining the encoder and decoder on standard german also improved the performance of the ats models. in our experiments, we noticed that mt models have a tendency to copy the source segments. while copies are sometimes desirable, we want to avoid this in cases where the original segment could benefit from further simplification. we show that the use of special copy labels at training time can positively influence such behavior. in particular, the contributions of the paper at hand are the following: the remainder of this paper is structured as follows: section 2 describes existing datasets for text simplification for a variety of languages as well as established approaches to ats. section 3 describes our approach to multi-level text simplification for german. we discuss our experiments in section 4 and conclude in section 5 with further thoughts on improving ats for german and current challenges to overcome. ## discussion comparing our results to @xcite , whose experiments are similar to ours in terms of scope and data, it is clear that our baseline models are already quite strong. we find that source-side labels for target language levels generally improve bleu and sari scores and that the same is true for pretraining and fine-tuning. interestingly, combining labels and pretraining results in lower gains in both metrics on a1 for the model without copy labels, indicating that these two approaches cannot always simply be combined. we also note that the scores on cefr level a2 did not profit as much from the different strategies and we were not able to improve over the pretrained baseline model in terms of bleu by using labels. we attribute this to the relatively large amount of training data for a2, meaning that specifically dealing with this low resource setting was not needed for this cefr level. on the other hand, a1 and b2, for both of which there was substantially less data, benefit from labeling and pretraining. regarding the copying behaviour, we note higher numbers of direct source copies on the higher cefr levels. we attribute this to the fact that simplifications on these higher levels typically tend to be closer to the original text in terms of lexical and syntactic complexity. this means that there are more standard language segments without any ## conclusion and outlook we were able to demonstrate the advantages of different approaches to german multi-level ats. we established strong baselines on a generic simplification task across all cefr levels and were able to further boost model performance for specific levels of simplification using source-side labels and a pretraining/fine-tuning strategy. we tested fine-tuning with labeled multi-level models. these approaches were generally more effective on the cefr levels where we had more limited data, suggesting that they are especially useful in lowresource scenarios. we also investigated the use of copy labels at training time to mark segments where source and target segments are identical. at test time, this resulted in a lower number of copies overall and especially in the number of instances where the references differs from the source. this suggests that copy labels are a valid tool to reduce undesired copying behaviour in text simplification, though their influence on the quality of the output can likely only be determined by human evaluation. further work will be conducted on a more advantageous combination of the two approaches of apa+capito multi fine-tuned copy die praktische fahrradprüfung findet im grazer verkehrsgarten statt. the practical bicycle test takes place in the graz traffic garden.