Uniandes at TSAR 2025 Shared Task Multi-Agent CEFR Text Simplification with Automated Quality Assessment and Iterative Refinement (2025)

Abstract: We present an agent-based system for the TSAR 2025 Shared Task on Readability-Controlled Text Simplification, which requires simplifying English paragraphs from B2+ levels to target A2 or B1 levels while preserving meaning. Our approach employs specialized agents for keyword extraction, text generation, and evaluation, coordinated through an iterative refinement loop. The system integrates a CEFR vocabulary classifier, pretrained evaluation models, and few-shot learning from trial data. Through iterative feedback between the evaluator and writer agents, our system automatically refines outputs until they meet both readability and semantic preservation constraints. This architecture achieved 4th position among participating teams, showing the effectiveness of combining specialized LLMs with automated quality control strategies for text simplification.

## introduction text simplification is a central task in natural language processing (nlp), aiming to make written content more accessible by reducing linguistic complexity while preserving meaning and fluency. in recent years, readability-controlled simplification has gained relevance, where the simplified output must conform to explicit proficiency levels defined by the common european framework of reference for languages (cefr). such control is important for applications in education, second-language learning, and inclusive communication, with recent work showing that instruction-tuned models can follow explicit readability targets @xcite . the tsar 2025 shared task on readability-controlled text simplification @xcite , organized as part of the emnlp conference, challenges participants to simplify english paragraphs originally written at the b2 level or above into target levels a2 or b1. official evaluation metrics include (i) compliance with the target cefr level, (ii) preservation of the original meaning, and (iii) similarity to human-written reference simplifications. while no training data is provided, organizers released 20 trial examples per target level for format familiarization, making this essentially a few-shot learning challenge that encourages creative combinations of linguistic resources, pretrained models, and novel system architectures. for this shared task, we developed an agentbased pipeline that leverages the modular nature of the simplification task. our approach integrates specialized agents for keyword extraction, text generation, and evaluation, orchestrated through an iterative refinement loop. the system incorporates diverse tools including a cefr vocabulary classifier that maps words to proficiency levels (a1-c1), pretrained evaluation models (cefr classifiers, meaningbert, bertscore), and few-shot examples from the trial data. this iterative feedback mechanism reduces the need for human annotation while ensuring outputs meet both readability and meaning preservation requirements. our system achieved competitive performance, ranking among the top-5 teams in the shared task. all system components, prompts, and implementation details are publicly available. 1 in this paper, we describe our system for the tsar 2025 shared task. section 2 reviews related work, section 3 presents our pipeline in detail, section 4 reports results and analysis, and section 5 concludes with key insights and directions for future work. ## related work readability-controlled generation has recently advanced through instruction tuning that targets fine-grained complexity levels, showing strong adherence to requested readability and competitive quality @xcite . for longer inputs, multiagent frameworks coordinate specialized roles to improve document-level coherence and thoroughness, offering an alternative to single-pass simplification @xcite . still, preserving meaning is a key challenge. paragraph-level human evaluation with reading-comprehension questions shows that even strong systems leave some questions unanswered, highlighting the need for direct meaning checks @xcite . ## discussion our experiments showed distinct trade-offs across model configurations. while run 2 achieved the best cefr compliance and run 1 demonstrated high precision and provided the most reliable balance between cefr compliance and meaning preservation. the hybrid run 3 struggled with level targeting. for meaning preservation, run 1 is more stable and shows less overfitting; run 3, despite having a higher mb-orig, has a lower mb-ref, indicating overfitting to the source text. after analyzing simpler agent configurations on trial data, several design improvements significantly increased performance. we found that using all 20 trial examples was counterproductive and wasteful of tokens, so limiting to 10 samples optimized both performance and efficiency. style examples enhanced output consistency, keyword extraction preserved domain-specific meaning through definitions, and cefr vocabulary classification prevented inconsistent term replacements. however, some configurations required many iterations to achieve correct cefr levels, suggesting convergence problems. our meaningbert threshold of 0.75 may have been conservative-higher thresholds could enforce stronger semantic preservation. furthermore, run 3 demonstrated that hybrid architectures with open models as writers can excel at semantic preservation despite weaker level control. this suggests that open models could be valuable as writer agents when paired with strong closed-model evaluators like gemini 2.5 pro, potentially offering cost-effective alternatives to fully proprietary systems. ## limitations key limitations include expensive and unpredictable token generation from iterative refinement, with some texts requiring many iterations and extended processing times. the cefr vocabulary coverage could be potentially missing some terms. the token consumption was highly variable across different texts, making cost prediction difficult. our cefr and semantic-similarity thresholds were intentionally conservative to favor coverage and reduce non-convergence. more ambitious (stricter) thresholds might yield higher precision in meaning preservation and level control, but at the cost of lower acceptance rates and longer refinement. exploring adaptive or curriculum-style thresholds is left for future work. ## lay summary this work builds a system that turns hard-to-read english paragraphs into easier ones for learners, at two target levels: a2 (elementary) and b1 (intermediate). the main goal is to make the text simpler without changing its meaning. to do this, the system uses three "agents" that work together in a loop: one finds important, difficult words that should be defined (not replaced), another writes a simpler version, and a third checks the result. if the check finds problems-like lost meaning or the level being too hard-the writer tries again. this repeat-and-improve cycle continues until the text is both simple enough and faithful to the original. to guide the writing, the system uses word lists tied to the cefr levels (a1-c1) and short example texts that show the expected style. to check quality automatically, it uses tools that (1) estimate the reading level, and (2) measure how closely the new text matches the original meaning. several model setups were tested and set clear passing rules before accepting any output. in a public competition on readability-controlled simplification, this approach ranked among the top teams (4th place). the results show that combining specialized roles with automatic checks can reliably simplify text while keeping its meaning. as limitations, some paragraphs need many rounds (which can be costly), and the vocabulary lists may not cover every word.