E asy J on at TSAR 2025 Shared Task Evaluation of Automated Text Simplification with LLM -as-a-Judge (2025)

Abstract: This paper presents an approach to automated text simplification for CEFR A2 and B1 levels using large language models and prompt engineering. We evaluate seven models across three prompting strategies short, descriptive, and descriptive with examples. A two-round evaluation system using LLM-as-a-Judge and traditional metrics for text simplification determines optimal model-prompt combinations for final submissions. Results demonstrate that descriptive prompts consistently outperform other strategies across all models, achieving 46-65% of first-place rankings. Qwen3 shows superior performance for A2-level simplification, while B1-level results are more balanced across models. The LLM-as-a-Judge evaluation method shows strong alignment with traditional metrics while providing enhanced explainability.

## introduction a crucial component of inclusion lies in language accessibility, where barriers to text readability and comprehension remain significant @xcite . enhancing technological inclusion requires improving text readability, which shapes how easily diverse audiences can process, understand, and engage with written material @xcite @xcite . text simplification, whether achieved through manual strategies (e.g., via trained editors) or automated approaches (e.g., large language models), is a key method for enhancing readability and comprehension by adjusting content to the target audience @xcite . within this context, automated, readabilitycontrolled text simplification using large language models (llms) and refined prompt engineering represents an innovative approach. it offers a scalable, cost-efficient, and adaptive solution that can be tailored to specific readability levels and target audiences @xcite . the central challenge in automated, readabilitycontrolled text simplification lies in balancing reduced linguistic complexity with the preservation of meaning and fluency @xcite . achieving this often requires supervision mechanisms and nuanced adjustments rather than straightforward simplification @xcite . llms, particularly when guided by adequate prompt engineering strategies, offer a promising avenue by adapting text to specific proficiency levels while retaining semantic accuracy and naturalness @xcite @xcite . the evaluation of the quality of the output poses similar challenges and requires similar finesse @xcite . we employ a llm-as-a-judge @xcite approach to evaluate the simplifications alongside the metrics provided by the tsar team @xcite . our approach builds upon the simplification tool easyjon by @xcite which employs a llmas-a-judge approach to find the best performing model and prompt technique for simplifying text into german plain language leichte sprache. the aim of this study was to create the most target-level appropriate text simplification with the use of llms and non-complex prompt engineering as well as compare two different evaluation approaches of llm-based automated text simplifications. ## methodology the task was completed in two distinct phases: the simplification process, which involved multiple prompts and llms, and the subsequent evaluation of the appropriate simplifications for the final submission. for the simplification, we tested three prompts and seven models. for the evaluation, we employed an llm-as-a-judge system @xcite . the dataset was provided by the tsar 2025 shared task team @xcite . ## conclusion in this study, we aimed to create the most appropriate simplifications for cefr a2 and b1 target levels, as well as compare two evaluation approaches for llm-based automated text simplification. we conclude that the descriptive prompting strategy is optimal for obtaining cefr target level text simplifications. we also conclude that the evaluation strategy of using an llm-as-a-judge proves to be equally proficient as the traditional evaluation metrics for text simplification.qwen3 demonstrated superior performance for a2-level tasks, whereas mistral medium emerged as the top-performing model for b1 simplifications. notably, the performance distribution across models was more balanced for b1-level tasks compared to a2-level tasks. the primary advantage of employing an llm-as-a-judge approach lies in its explainability. unlike traditional metrics, an llm can identify, penalize, and provide detailed explanations for problematic simplifications. for instance, it can detect awkward phrasing, identify idioms requiring advanced comprehension despite simple vocabulary, and flag complex terms that are immediately clarified by an explanation. this approach offers significant utility for dataset creation, fine-tuning smaller models, and diagnosing model performance issues in an interpretable manner. the explainable feedback enables researchers to pinpoint specific simplification shortcomings, whether they stem from inappropriate phrasing, word choice, sentence structure, or other linguistic factors. such granular insights are crucial for iterative model improvement and understanding the nuanced challenges in text simplification. an example evaluation demonstrating this approach can be found in section a.5 in the appendix. the complete dataset is available on github foot_4 . examining additional automated readability and simplification assessment strategies could be a valuable area for research. comparing our prompt refinement strategy to the findings of further studies would also provide useful insights. furthermore, assessing text readability not only for target proficiency levels but also for specific audiences, such as individuals with cognitive impairments, neurodivergent individuals, or non-native speakers, may yield important insights.