A New Aligned Simple G erman Corpus (2023)

Abstract: “Leichte Sprache”, the German counterpart to Simple English, is a regulated language aiming to facilitate complex written language that would otherwise stay inaccessible to different groups of people. We present a new sentence-aligned monolingual corpus for Simple German – German. It contains multiple document-aligned sources which we have aligned using automatic sentence-alignment methods. We evaluate our alignments based on a manually labelled subset of aligned documents. The quality of our sentence alignments, as measured by the F1-score, surpasses previous work. We publish the dataset under CC BY-SA and the accompanying code under MIT license.

## introduction text in simple language benefits language learners, people with learning difficulties, and children that tend to have a hard time understanding original and especially formal texts due to grammar and vocabulary. text simplification describes the problem of generating a simplified version of a given text while conveying the same matter @xcite . this involves the reduction of lexical and syntactic complexity by various operations like deletion, rewording, insertion, and reordering @xcite . text simplification can further entail additional explanations for difficult concepts and a structured layout @xcite . to make language more inclusive, guidelines for simple versions of languages exist. in english, most notably, @xcite introduced "basic english". in german there are two prevalent kinds of simple language: "einfache sprache" (es) and "leichte sprache" (ls), both roughly translating to easy language @xcite . ls has strict rules, including the removal of subordinate clauses, the insertion of paragraphs after each sentence and the separation of compound nouns with hyphens. es is less restrictive and does not have a specific set of rules; instead, translators can work more liberally. however, the goal of both approaches is to improve the language's accessibility. there exists work on rule-based approaches for text simplification in german @xcite , but the problem of text simplification can also be defined as a monolingual translation task. then, the availability of data becomes a prerequisite in order to apply statistical machine learning models to it. especially sentence-aligned text constitutes the backbone of neural machine translation. to the best of our knowledge, only the work of @xcite presents a parallel sentence-aligned corpus in german created from public web data. our work addresses the lack of data for text simplification in german and thus creates an aligned corpus of easy language and corresponding german texts. as there is no german equivalent to the simple english wikipedia, which provides cross-lingual references between simple english and english articles, we had to rely on multiple sources offering a small number of articles in german as well as in some simplified version of it. our corpus consists of articles in "leichte sprache" from seven websites and "einfache sprache" from one extensive website. in the following, we will always talk about simple german whenever the distinction between those two forms of simplification is not relevant. following the description of our dataset and its collection process, we present the results of a comparison of different sentence-alignment methods. then, we select the best approach and obtain a sentence-aligned dataset that can potentially be extended by crawling further websites. see figure 1 to see examples of our sentence alignments. finally, we discuss the limitations of our dataset and future research. we share our code to build the dataset on github 1 . the repository contains a list of urls and scripts to reproduce the dataset by crawling the archived websites, parsing the text and aligning the sentences. we provide the fully prepared dataset upon request. ## related work there are various classification systems for language with different aims. the european council has defined six proficiency levels a1 to c2 based on the competencies of language learners and applicable to multiple languages (council of europe, 2020). yet, these are mainly intended to evaluate learners, not texts. for english, the lexile scale gives scores on reading proficiency, as well as text complexity, but has been criticized as carrying little qualitative meaning @xcite . a particularly early attempt at a "simplified", controlled english language is basic english @xcite . it is a subset of (rules and words of) english and aims at being easy to learn without restricting sentence length, complexity of content, or implicit context. as a result, even "easy" texts, as measured on one of the above scales, may fall short in comprehensibility and accessibility. we focus on german texts which follow specifically designed rules that aim at being more inclusive to certain target groups. ls (simple german) is designed for people with cognitive disabilities @xcite @xcite . es (plain german) targets the dissemination of expert contents to lay people and is less comprehensible (and hence less inclusive), but more acceptable to larger audiences @xcite . there are some sources of monolingual parallel corpora for different languages. english -simplified english corpora have been created, e.g. from the simple english wikipedia (which does not adhere to any fixed simplification standard) @xcite @xcite . using aligned articles from wikipedia has been criticized, as (i) simple wikipedia contains many complex sentences and (ii) sentence alignments are improbable, as the articles are often independently written @xcite . hence, an alternative corpus of five difficulty levels targeted at children at different reading levels has been proposed @xcite . spanish @xcite , danish @xcite , and italian @xcite corpora exist as well. when narrowing the research field down to the german language, only a few resources remain. @xcite crawl five websites that provide a total of 256 parallel german and simple german articles, spanning various topics. they provide sentence level alignments, and thus their result is the most similar dataset to ours that currently exists. they use a sentence alignment algorithm based on dynamic programming with prior paragraph alignment based on bag-of-word cosine similarities and report for their alignments an f1-score of 0.085 on the ground truth. @xcite introduce two sentence-aligned corpora gathered from the austrian press agency and from capito. here, the authors align the sentences of the original texts with their corresponding translation in level a1 and b1 of the common european framework of reference for languages (council of europe, 2020). the resulting simplifications are very different to the simplifications according to the rules of ls. @xcite extend this dataset by adding articles from a swiss news outlet which publishes "simplified" summaries alongside its content which, however, do not adhere to any simplification standard. here, sentence-level alignments are not provided. @xcite compile a corpus for simple german that mostly consists of unaligned simple german articles and 378 parallel article pairs, but without sentence-alignments. @xcite present an extensive document-aligned corpus by using the german children encyclopedia "klexikon". the authors align the documents by choosing corresponding articles from wikipedia, making it unlikely that specific sentences can be matched. as republishing may lead to legal ramifications, only the klexikon dataset is publicly available. overall, current german language text simplification datasets are rare, small, usually not publicly available, and typically not focused on inclusive simple german. ## dataset description as discussed, there are very few datasets tailored towards text simplification. our work addresses this lack of data for simple german. problems besides text simplification like automatic accessibility assessment, text summarization, and even curriculum learning would benefit from that data. we present a corpus consisting of 712 german and 708 corresponding simple german articles from eight web sources spanning different topics. they were collected from websites maintaining parallel versions of the same article in german and simple german. we made sure to only use freely ## dataset construction we now describe the process of data acquisition from the selection of the online sources over the crawling of the websites to the parsing of the text. to be transparent, we point out the problems and pitfalls that we experienced during the process. crawling starting point for the construction of the dataset was a set of websites. first, we identified a starting point for each website that offered an overview of all simple german articles. then, we created a crawling template for each website using the python library beautiful-soup4. the crawler always started from the articles in simple german. we first download the entire article webpages and later on parsed the text from the raw html-files. this process allows to return to the raw data to support unanticipated future uses. parsing we have ignored images, html-tags, and corresponding text metadata (e.g. bold writing, paragraph borders) for each article. in contrast to @xcite , where enumerations are removed since they may only contain single words or grammatically incorrect sentences, we decided to transform them into comma-separated text. enumerations are frequently used in simple german articles, and we argue that they may contain major parts of information. the most common challenge during crawling was an inconsistency in text location within a website, i.e. the structure of the html-boxes enclosing the main content. simply extracting @xmath0, as these regularly contained useless footer information. as only the main text was the targeted resource, the crawler's implementation needed to be unspecific enough to account for these deviations, but specific enough not to crawl any redundant or irrelevant text. another problem was the way in which the german articles and their corresponding translations in simple german were linked. the mdr, a statefunded public news organization, often showed inconsistent linking between articles. here one might expect a strict structure disallowing differences. however, the links were sometimes encapsulated within href, sometimes given as plain text or not at all. the referenced german article could even be a video, rendering both articles useless for our corpus. we discarded simple german articles whenever the original german source was unusable, i.e. unlocatable or in video format. the result of the data acquisition as described above is a dataset of articles in german with their corresponding articles in simple german. ## sentence alignment in the following section we compare different similarity measures and matching algorithms used to reach sentence-level alignment. we describe an article a as a list of sentences, i.e. @xmath1, . . . , s n ]. we define a s and a c as the simple and complex versions of the same article with a @xmath2. we consider a variant of the sentence alignment problem that receives two lists of sentences a s and a c and produces a list of pairs (s s i , s c j ) such that, with relative certainty, s s i is a (partial) simple version of the complex sentence s c j . we will approach this task in three steps: first (sec. 5.1), we transform the raw texts obtained in section 4 into lists of sentences and do some light pre-processing. next, we compute sentence similarity scores (sec. 5.2) for pairs of sentences from the aligned articles. finally, a sentence matching algorithm (sec. 5.3) takes the sentence lists and the respective inter-sentences similarities to calculate the most probable alignment. ## evaluation we combine both matching algorithms with all eight similarity measures using either a threshold of µ + 1.5 • σ or no threshold. this gives a total of 32 different alignment variants, the results of which we will discuss here. we select the best algorithm variant according to a two stage process. first, we analyse the results of the different alignment variants quantitatively. then, we perform two kinds of manual evaluation. for the first one we create a ground truth by manually aligning the sentences for a subset of articles. the second one focuses on the matches by manually labelling them as either correct or incorrect alignments. ## discussion the results for the sentence alignments presented in section 5 show that the more sophisticated similarity measures perform better in terms of both f1-score and manual alignment classification accuracy. the sbert similarity is the most sophisticated similarity measure yielding the highest f1 score. however, the precision and alignment classification accuracy of the maximum similarity with mst-lis is higher. generally, mst-lis benefits from its strong assumption on the order of information in both articles yielding a higher accuracy, but in return not finding all possible alignments. this can be traced back to our observation, that simple german articles often show a different structure. limitations our work presents a new dataset based on text data scraped from the internet. hence, the quality of the text depends on the quality of the available websites. most of our data stems from the three websites apo, koe and mdr providing a rich vocabulary in our corpus. while this vocabulary covers a variety of mixed topics, we cannot rule out any negative side effects of data imbalance. moreover, our dataset can only represent topics that were considered relevant to be translated into simple german by the respective website. in section 6.2 we presented the different guis that we used to either manually align the sentence pairs or evaluate a sample of sentence alignments. one drawback of the tool for the second evaluation method is that it focuses solely on the matched sentences and presents them isolated from their contexts. one can argue that evaluators using the tool would have to see the context in which the sentences appear in order to correctly classify partial matches. also, providing more information to the annotators might enable them to also correctly classify additional explanatory sentences. future work and use cases our corpus comprises data in ls and es, two types of simple german. a higher granularity of language difficulty could be achieved by incorporating texts originally directed at language learners that are rated, e.g. according to the european reference system (council of europe, 2020). our work presents a parallel corpus for german and simple german and should be continuously expanded. not only to increase its size, but mainly to increase the number of topics covered in the corpus. yet, as there are no efforts to start a single big corpus like a simple german wikipedia, web scraping from various sources stays the method of choice for the future. an additional option is to compute sentence alignments for existing article aligned corpora to include them in the dataset (e.g. @xcite . as for the sentence alignment algorithms, various extensions are imaginable. firstly, it might be interesting to allow one simple german sentence to be matched to multiple german sentences. also, the assumption of the mst-lis about the order of information is very strong, and recall might be improved by softening this assumption, e.g. by allowing matches that are at most n sentences away. other alignment algorithms that impose different biases on sentence order @xcite @xcite are interesting for further extensions. our dataset can be used to train (or fine tune) automatic text simplification systems (e.g. @xcite which then should produce text with properties of simple german. direct use cases for such simplification systems are support systems for human translators or browser plugins to simplify web pages. further research has shown that text simplification as a pre-processing step may increase performance in downstream natural language processing tasks such as information extraction @xcite , relation extraction @xcite , or machine translation @xcite . it remains an interesting direction for future research if simple german can help to further increase performance on such tasks. ## conclusion in this paper, we presented a new monolingual sentence-aligned extendable corpus for simple german -german that we make readily available. the data comprises eight different web sources and contains 708 aligned documents and a total of 10 304 matched sentences using the maximum similarity measure and the mst-lis matching algorithm. we have compared various similarity metrics and alignment methods from the literature and have introduced a variable similarity threshold that improves the sentence alignments. we make the data accessible by releasing a url collection foot_1 as well as the accompanying code for creating the dataset, i.e. the code for the text preprocessing and sentence alignment. our code can easily be adapted to create and analyze new sources. even the application to non-german monolingual texts should be possible when specifying new word embeddings and adjusting the pre-processing steps. we have obtained generally good results on our data. our corpus is substantially bigger than the one in @xcite (708 compared to 256 parallel articles) and our results of the best sentence alignment methods are better as well (f1-scores: 0.28 compared to 0.085). it is also bigger than the parallel corpus created in @xcite (378 aligned documents), which does not provide any sentence level alignment.