Findings of the TSAR 2025 Shared Task on Readability-Controlled Text Simplification (2025)

Abstract: This paper presents the findings of the first Shared Task on Readability-Controlled Text Simplification at TSAR 2025. The task required systems to simplify English texts to specific target readability levels of the Common European Framework of Reference for Languages (CEFR). We received 48 submissions from 20 participating teams, with approaches predominantly based on large language models (LLMs), which included iterative refinement, multi-agent setups, and LLM-as-a-judge pipelines. For this shared task, we developed a new dataset of pedagogical texts and evaluated submissions using a weighted combination of semantic similarity and CEFR-level accuracy. The results of the participating teams demonstrate that while LLMs can perform substantially well on this task, dependable and controlled simplification often requires complex, multi-iterative processes. Our findings also suggest that the capabilities of current systems are beginning to saturate existing automatic evaluation metrics, underscoring the need for reevaluation and practicality.

## introduction text simplification consists of automatically rewriting a text to make it easier to read and understand while preserving meaning, supporting applications in education, accessibility, and second-language learning @xcite . many previous shared tasks have focused on lexical simplification @xcite @xcite or on complexity prediction @xcite @xcite . however, these tasks typically do not require control over output readability level, which is essential if simplification is to be adaptive to learner needs. in this work, we introduce a new shared task for readability-controlled text simplification in english, in which systems must simplify a source text to a specified cefr level (e.g., a2 or b1). in this way, the target complexity explicitly aligns with educational and pedagogical goals. our task builds on, but also departs from, earlier shared tasks in lexical simplification, in that participating systems must simplify short passages under a cefr constraint, rather than single words or phrases based on a target audience. to support this shared task, we curated a new cefr-based reference dataset of 100 paragraph-level english texts drawn from pedagogical reading materials for language learners. each source text was manually simplified by experienced english-language teachers to two lower target levels, resulting in a total of 200 reference simplifications. in addition, we trained a cefr evaluator model to estimate the readability level of system outputs automatically. the model was trained on cefr-annotated texts and fine-tuned to classify english texts into cefr levels with high reliability. both resources are publicly released to support further research. the shared task challenged participants to generate simplified versions of the same source texts at specified cefr targets, requiring systems to demonstrate both readability control and semantic fidelity. submissions were assessed using the cefr evaluator model to measure compliance with the target level and meaningbert @xcite to assess source and reference-wise meaning preservation. the resulting metric scores were combined into a single ranking using au-torank @xcite , an aggregation method that normalizes metric scales and mitigates the effect of outliers. ## related work the task of readability-controlled text simplification (rcts) aims to generate simplified text aligned with specific difficulty levels, often us-ing frameworks such as the common core standards or the cefr for reference @xcite @xcite . the main challenge with rcts is that it requires finergrained generation control across multiple versions of the exact text, tailored to different audiences. early approaches treated rcts as a supervised sequence-to-sequence problem requiring a parallel corpus annotated with readability @xcite , which then followed by more advanced techniques including the addition of low-level control tokens @xcite , lexical-based constrained decoding for better handling of complex words @xcite , and reinforcement learning to reward readability-aligned texts @xcite @xcite . despite the substantial progress, current methods remain heavily dependent on gold-standard parallel data. to address this, more recent works have explored techniques that take advantage of instruction-tuned llms' few-shot learning capabilities for shifting the readability levels of texts without the need for additional training data @xcite @xcite @xcite . however, achieving precise readability control while maintaining the quality of generated simplifications remains challenging, motivating further research into fine-grained level alignment. ## cefr evaluator model we detail the steps we followed to train the cefrbased evaluator model we will use to evaluate the cefr alignment of system submissions with goldstandard reference simplifications. ## evaluation setup we describe the evaluation pipeline, combining normalization procedures, weighting decisions, and ranking methodology, used to assess the performance of system submissions using the shared task data and a trained cefr evaluator model. following the autorank framework proposed in the wmt 2025 general machine translation shared task @xcite , we aggregate multiple evaluation metrics into a single overall ranking to increase robustness against outliers and improve interpretability. we present two forms of autorank rankings: one for all submitted runs and another for the best run per team. ## participants and system descriptions 6.1 overview we received overall 48 submissions by 20 participating teams. each team was allowed to submit outputs of up to three systems or runs. the most dominant strategy of the submissions was prompting (28 submissions), including evaluations with a cefr labeling system (14 submissions) or an llm as a judge (6 submissions). other strategies ranged from rule-based systems (4 submissions), agentic approaches (7), fine-tuning of llms (4), training of neuronal networks (3), and other approaches (2). ## results we report two separate rankings to provide a comprehensive view of system performance. comparing approaches that used only one llm, system submissions that used commercial ones like gpt-5 or gemini 2.5 performed substantially better in terms of average performance (ehimenlp, hit-you, or uniandes) than with system submissions that used open-weight models like llama or gemma (taskgen, archaeology, and starling). the best model, which uses only open-weight models, is taskgen, with an autorank score of 7.48 using llama-3.1-8b. on the other hand, system submissions that used multiple llms achieved even better results, including the top 5 submissions, where one is from ehimenlp and two from uol-upf and hit-you. all these top-scoring system submissions used at least four llm variants, except for easyjon, which has also used a collection of seven llms, but was only ranked 8.88. thus, it is clear that the selection of llms and the prompting technique largely affect the performance. likewise, we also observe a clear pattern from the system submissions where the use of commercial gpt-based models leads to a substantially stronger performance. based on the system submissions, prompting strategies achieved, on average, the best au-torank placement of 7.70, followed by agentic approaches with 8.48 and rule-based approaches with 13.00. comparing the prompting strategies, the approaches with an llm-as-a-judge performed slightly better, with an autorank placement of 6.33, than the approaches using a cefr labeling system for evaluation, with 6.70. due to the lack of training data, some participants used external resources, such as related simplification corpora, lexicons, and more informative descriptions of cefr levels to enhance their prompts. the teams also used reference-less metrics, such as readabilitybased formulas like flesch-kincaid and semantic similarity via meaningbert, to pre-evaluate their systems and feed the results back into their models in an iterative process. ## discussion are llms the only way forward for text simplification? the use of llms as a core resource across the majority of system submissions reflects a decisive shift away from traditional rulebased simplification methods to generative models. we observe a similar pattern with the proliferation of llms achieving state-of-the-art performance across general readability control benchmarks @xcite . this transition has significant implications for evaluation, shifting from measuring output quality to assessing real-world impact @xcite . as a where do we position synthetic data? while the shared task was ultimately successful in producing a new parallel cefr-based reference dataset for validation and testing, it is worth noting the substantial effort and financial support required for resource development. in line with this, we ask the question "is it time to use llm-generated cefr data to complement expert-produced data?" while adjacent nlp tasks like grammatical correction and essay scoring have benefited from the performance advantages of synthetic data, its practicality for readability and text simplification applications has only ever been explored recently @xcite @xcite . for text simplification that is anchored on a real-world language proficiency framework like cefr, there are both opportunities and risks. on the one hand, llms are capable of generating fluent text guided by cefr specifications, as evident in the top submissions of this shared task. this could potentially be valuable for low-resource languages across various domains, genres, and text types where expert-annotated resources are scarce. however, using synthetic data without careful validation checks risks producing noisy approximations of cefr levels, which may reinforce undesirable simplification patterns. as such, our stance on this is that we should establish a community-accepted framework to integrate synthetic data, involving steps such as filtering and expert validation stages. for the next iteration of this shared task, we propose a direction exploring how the use of llm-generated synthetic data can perform well on test and validation data generated by experts. cost-performance tradeoffs our leaderboard results show that top-ranked systems achieved strong performance by leveraging ensembles of multiple llms. however, these submissions come with substantial computational and financial costs. for example, the rank-one system ehimenlp combined six llms, including gpt-5, gpt-4.1, o3, gpt-oss-20b, qwen3-32b, and llama-3.3-70b-instruct to generate up to 120 candidate simplifications per input. while such methods clearly demonstrate the potential of ensemble-based techniques for producing precise readability-controlled simplifications, their heavy resource demands may limit their adoption in educational and resourceconstrained contexts where computational budget to run more than one commercial model is unavailable. future work should therefore explore approaches that strike a balance between achieving decent performance and computational efficiency. likewise, combining resource-aware evaluation procedures with performance-based metrics may encourage participants to propose innovative and computationally viable methods that are effective in real-world cefr-based text simplification settings. ## conclusion the tsar 2025 shared task introduced a benchmark for readability-controlled text simplification explicitly aligned with cefr levels. two main resources were developed as contributions to the community: a) a cefr-aligned dataset of pedagogical paragraph-level english texts simplified to a2 and b1 levels, and b) a cefr evaluator model finetuned to estimate text difficulty along the cefr scale. the results from participating teams indicate that although llms achieve strong performance on this task, achieving dependable and fine-grained control over simplification often relies on complex, iterative generation strategies. moreover, our analysis suggests that current systems are approaching the limits of existing automatic evaluation metrics, underscoring the need to adapt these metrics for greater robustness and practical relevance.