HHU plexity at Text Complexity DE Challenge 2022 (2022)

Abstract: In this paper, we describe our submission to the ‘Text Complexity DE Challenge 2022’ shared task on predicting the complexity of German sentences. We compare performance of different feature-based regression architectures and transformer language models. Our best candidate is a fine-tuned German Distilbert model that ignores linguistic features of the sentences. Our model ranks 7th place in the shared task.

## introduction texts are a basic form of human information exchange. too high of a text complexity, however, can result in text comprehension failures @xcite and therefore miscommunication. text complexity and readability assessment are a long known problem and several computational approaches and metrics have been proposed @xcite @xcite , relying on different linguistic features and primarily aiming at english. among other application objectives, an adequate, quantificational metric for text complexity can be of high benefit to the educational domain (as a means of providing textual material according to student levels), writing support systems (as feedback) or for other natural language processing tasks like estimating the complexity of the output of text simplification systems or chatbots. most related tasks focus either on the prediction of complex words @xcite or the assessment of readability levels @xcite . however, the goal of the 'text complexity de 2022' shared task is the prediction of an empirically determined complexity score called 'mean opinion score' (mos) for german sentences. overall, our best model is ranked on the 7th place out of 10. in the following, 1 our code is available at https://github.com/vipitis/hhuplexity we present the approach and results of our team "hhuplexity" in more detail. ## method our main approach is to combine hand-crafted features with text embeddings of language models. therefore, we have calculated several features as described in subsection 2.1. to compare the effect of these features in combination with language models, we follow two baseline approaches: i) training different regression models with the features (see subsection 2.2), and ii) fine-tuning language models without features (see subsection 2.3). afterwards, we combine the features with the language models in a multimodal model (see subsection 2.4). ## conclusion in our contribution to the shared task, we have compared predictions based on linguistic features with an approach based on transfer learning, i.e., finetuning a language model. we find that even though linguistic features achieve relatively high correlation with the mos scores, they are outperformed by a "simple" fine-tuned transformer language model.