(Dis)improved?! How Simplified Language Affects Large Language Model Performance across Languages (2025)

Abstract: Simplified language enhances the accessibility and human understanding of texts. However, whether it also benefits large language models (LLMs) remains underexplored. This paper extensively studies whether LLM performance improves on simplified data compared to its original counterpart. Our experiments span six datasets and nine automatic simplification systems across three languages. We show that English models, including GPT-4o-mini, show a weak generalization and exhibit a significant performance drop on simplified data. This introduces an intriguing paradox: simplified data is helpful for humans but not for LLMs. At the same time, the performance in non-English languages sometimes improves, depending on the task and quality of the simplifier. Our findings offer a comprehensive view of the impact of simplified language on LLM performance and uncover severe implications for people depending on simple language.

## introduction automatic text simplification (ats) is the task of rewriting a text using simpler vocabulary while preserving its original meaning. the goal is to increase readability and make information accessible to a broader audience. the primary target group is people with low literacy and mental disabilities, or language learners @xcite . however, previous work has shown that not only people from the target group but even the broad majority of people profit from simplified language @xcite . with this paper, we try to answer if the same holds true for large language models (llms). given that llms are approaching human-like capabilities @xcite , it is reasonable to hypothesize that they might also perform better with simplified input or at least show good performance and generalization on this language style. sentiment: neutral "currently , the plant operates on full capacity ." ground-truth: neutral currently, the plant works on full capacity. llama3.1 70b simplifier llama3.1 70b sentiment: positive @xcite . we test the generalization of llms like llama3.1 70b from original to automatically simplified data. the sentiment prediction on the original data sample is correct. however, if we use an automatic lexical simplifier that replaced the word "operates" with "works", llama misclassifies the sample as positive. to investigate this, we select six labeled datasets across three languages (english, german, and russian) and simplify their texts using nine pretrained simplification models and llms. then, we benchmark five large language models, including llama3.1 @xcite , aya expanse @xcite , and gpt-4o-mini, on both the original and simplified corpora. our results show a significant change in performance with a strong performance drop for english (see example in figure 1 ). this lack of generalization introduces a severe risk for people who rely on simplified language: if they input prompts or samples in simple language, llms may show a worse performance and make more mistakes than with standard english. especially for tasks with high societal impact, like fake news classification or news summarization, this increases discrimination for already vulnerable target groups. overall, our contributions can be summarized as follows: ## related work the impact of ats on nlp tasks has been studied for many years and for different nlp tasks @xcite @xcite . however, many of the older studies could not use transformers or even large language models and were based on statistical simplification. among the more recent studies, we identify two research directions: text simplification as data augmentation for pre-training or finetuning and text simplification as a pre-processing step to improve inference performance. to investigate the first direction, @xcite simplified the training data for lstm-and bert-based classification models and evaluated the simplification quality with bleu only. results show that different setups of data augmentation with simplification can improve the classifiers. however, they also show that simplifying the data at inference time results in a weaker performance than the original data. these results are in contrast to other studies that benchmarked simplification as inference preprocessing. @xcite tested google translator for japanese to english translations with sentence splitting and further rule-based simplifications. a human evaluation showed that the simplifications yielded strong improvements in the translation outputs. similarly, @xcite created an artificial simplification system through back translation and used this system to simplify the machine translation inputs of a lowresource-language translation system. they show improved translation quality across multiple languages. however, the performance changes of the target systems depend on the quality of the ats systems. as such, @xcite investigated how well ats systems preserve the meaning of the original texts. while human simplifications could improve the performance of a pre-trained question-answering model, automatic simplifications worsened the performance. our work tries to shed light on the contradicting findings of previous work. for this, we extend the existing research by covering more tasks, languages, and simplifiers. we paint a broader picture of the helpfulness of simplification as pre-processing, especially in times of flexible and powerful llms. a different research direction was chosen by @xcite , who used human-supervised simplification corpora to investigate how well models generalize between original and simplified data. they are the first ones to include llms in their investigations and show that models exhibit an incoherent behavior between original and simplified data. however, they only benchmarked gpt3.5turbo as llm, and their datasets do not contain ground-truth labels. while they assumed that the human-supervised datasets contain correct simplifications, they cannot measure the actual performance of the classification system without groundtruth labels. we try to overcome this weakness by using labeled datasets and benchmarking the performance of multiple llms on these datasets. in addition, we extend the investigation to the task of summarization and not only cover classification tasks. ## methodology our objective is to compare whether the performance of different llms changes when the input samples are simplified. for this, we take labeled datasets and simplify the inputs with existing simplifiers. then, we use pre-trained classification models or llms to predict the labels on the original and on the simplified inputs. finally, we calculate the accuracy and examine whether text simplification at inference can improve the models' performance. an overview of our approach is shown in figure 2 . our investigations cover three distinct languages with six different datasets, nine simplifiers, and six prediction models, including llms like gpt4o. all combinations were evaluated independently, and the models did not know if the input text was simplified or not to avoid bias. the different settings will be discussed in the following subsections. ## conclusion experiments across six datasets, nine ats systems, and three languages show that english llms exhibit a severe performance drop when switching from original to simplified language, uncovering a weak generalization to this language style. however, simplified texts can enhance performance at inference time for non-english languages. we thus encourage content creators to prioritize using simple language online as a way to improve llms' downstream performance and comprehension and to open their models to a broader audience.