Overview of the G erm E val 2022 Shared Task on Text Complexity Assessment of G erman Text (2022)

Abstract: In this paper we present the GermEval 2022 shared task on Text Complexity Assessment of German text. Text forms an integral part of exchanging information and interacting with the world, correlating with quality and experience of life. Text complexity is one of the factors which affects a reader’s understanding of a text. The mapping of a body of text to a mathematical unit quantifying the degree of readability is the basis of complexity assessment. As readability might be influenced by representation, we only target the text complexity for readers in this task. We designed the task as text regression in which participants developed models to predict complexity of pieces of text for a German learner in a range from 1 to 7. The shared task is organized in two phases; the development and the test phases. Among 24 participants who registered for the shared task, ten teams submitted their results on the test data.

## introduction text forms an integral part of exchanging information and interacting with the world. along with the other types of content (e.g., image and video), textual content has been increased drastically in amount and importance during recent years. text complexity (in the following used interchangeably with text readability) is one of the factors which affects a reader's understanding of text @xcite . readability is concerned with the relation between a given text and the cognitive load of a reader to comprehend it. this complex relation is influenced by many factors, such as a degree of lexical and syntactic sophistication, discourse cohesion, and background knowledge @xcite . a readability score is the mapping of a body of text to a mathematical unit quantifying the degree of readability. it is the basis of readability assessment. readability assessment has diverse use cases and applications, such as helping people with disabilities and also facilitate choosing of learning material for second language learners @xcite . in this paper, we present the challenge and results from the task of german text complexity assessment in germeval 2022. the task includes developing natural language processing (nlp) models to automatically assign a complexity score in the range from 1 to 7 to german texts, where 1 represent an easy to understand (i.e., simple) text/sentence and 7 shows a complex text for german learners. in other words, the shared task is a text regression task in which the output is a continuous variable between 1 and 7. germeval is a series of shared task evaluation campaigns that focus on natural language processing for the german language. it started in 2014 with a shared task on german named entity recognition @xcite and continued in the years after with different tasks from lexical substitution @xcite to the task of identification of toxic, engaging, and factclaiming comments @xcite and german scene segmentation @xcite . the rest of the paper is organized as follow; section 2 presents recent research on text readability and complexity assessment and related tasks. an overview of the shared task and the data set, resources and the evaluation metrics that have been used in the shared task are presented in sections 3 and 4, respectively. we briefly review the submitted models and discussed the results in sections 5. finally, we conclude the paper and the german text complexity assessment shared task in section 6. in this section we provide an overview of related shared tasks in different languages, and also highlight a number of the recent approaches for the task of text complexity assessment. ## data set and evaluation in this section we discuss briefly the compiled data set for the competition and also overview the evaluation metrics that have been used to assess and ranked the submitted results. ## results in this section we present the baseline model and also survey the submitted models for the shared task. ## conclusion in this paper we described the germeval 2022 task on "complexity assessment of german text". the shared task is co-located with the conference on natural language processing (konvens) 2022. we presented the compiled data sets for the training and the test phases and the models proposed by the participants. the training and the test sets included 1,000 and 210 german sentences from wikipedia articles, respectively, with a readability/complexity score from 1 to 7. regarding the models, combining the traditional feature extraction models with state-of-the-art pretrained language models was the main trend in the submitted systems. although different teams used different feature set, pre-trained models and also different strategies to combine the outcomes of the models, there were similarities between the overall procedure from different participants. almost all of the submissions could outperform the transfer learning based model as the competition's baseline.