task G en at TSAR 2025 Shared Task Exploring prompt strategies with linguistic knowledge (2025)

Abstract: TaskGen ranked as 6th best team in the TSAR 2025 shared task for English text adaptation to a target CEFR level. Our experiments consisted of prompting a Llama-3.1-8B-Instruct model with linguistic descriptors of the target level, examples of adaptations and multi-step approaches. Our best run, 13th in the overall ranking, applied an ensemble strategy using a voting mechanism to find the most adequate among 10 texts, each produced by a different prompting strategy.

## introduction text simplification (ts) aims to reduce the linguistic complexity of texts while preserving their meaning and coherence. this facilitates accessibility for readers with low literacy and individuals with cognitive challenges @xcite , but it is also very useful to adapt texts for language learners. in that context, ts presents the additional requirement to target the learners' proficiency levels. the tsar 2025 shared task on readability-controlled text simplification (rcts) @xcite directly addresses this need by requiring participants to simplify english texts at b2 level or higher to specified target cefr levels (a2, b1). this setup not only evaluates systems on their ability to reduce complexity, but also on their capacity to produce texts adequate for language learners at different stages of proficiency. ## related work automatic readability assessment (ara) "refers to the task of modeling the reading and comprehension difficulty of a given piece of text, for a given target audience" @xcite . it has been widely studied across nlp, education and psychology, and is often applied to support tasks such as selecting suitable materials for l2 learners @xcite . traditional es to ara relied on readability formulae, such as flesch-kincaid grade level @xcite , smog grading @xcite and @xcite , among others. more recent work has explored feature-based machine learning and neural models @xcite @xcite @xcite . although central to understanding textual complexity, ara also serves as a foundation for downstream tasks such as text simplification (ts). early es to ts were rule-based or statistical, but recent progress has been driven by neural models @xcite @xcite . with the emergence of large language models (llms), ts research has increasingly swift to fine-tuning and prompt engineering @xcite @xcite . building on ts, readability-controlled text simplification (rcts) introduces explicit control over the target readability level, making it particularly valuable in educational contexts where texts must be aligned with learners' proficiency @xcite . most work on this area has followed supervised training es @xcite @xcite @xcite ; however, the drawback to these es is the limited number of datasets available. recent advances in llms and in their capacity for text generation have opened new directions for rcts, with studies exploring sentence-level @xcite and passagelevel simplification @xcite in zero-shot and few-shot settings. ## system description we explored a prompt-based approach to text adaptation, assessing the performance of different prompt variants: all prompt different strategies used llama-3.1-8b-instruct model (meta, 2024) with precision bfloat16 and decoding default parameters (nucleus sampling with top-@xmath0.9, @xmath1.6, and stochastic decoding enabled). we opted for llama-3.1-8b-instruct model because it is open-source, cost-effective and had performed well in cefrbased text classification @xcite . also, a previous version of the llama 3 8b family had proved successful for a similar task @xcite . ## results and discussion our second run, aiming to remove metainformation generated by the llm about the task itself, obtained lower scores for meaning preservation with respect to run 1. despite this drawback, it was our best run with respect to adequacy to the target cefr level, with an rmse of 0.561. this shows that the follow-up prompt for content adequacy of run 1 harmed compliance to the target cefr level. the ensemble strategy, our third run, produced a significant improvement on preservation of the meaning with respect to the original and the reference, although at the expense of worse accuracy with respect to the target cefr level, with the worst rmse for our three runs, at 0.628. nevertheless, high meaning preservation made this our best overall ranked submission. since adequacy to the target cefr level was the main weakness of our approaches, we conducted a more detailed analysis of this aspect. as shown in figure 1 , all three runs -particularly the thirdstruggled to adapt texts to the b1 level, while systematically succeeding at the a2 level. still, the errors were relatively minor, typically corresponding to a one-level difference, that is, texts that should have been adapted to b1 were adapted to a2 instead. we will examine these cases further to improve the system. in contrast, meaning preservation was significantly better for b1 than for a2, as illustrated in figure 2 , showing the probability density function (a smoothed version of a histogram) of the score for meaning preservation with respect to the original. thus, adaptation to lower levels seems to make meaning preservation more difficult. we found that run 3 was especially capable to adapt to different typologies of texts. for example, figure 3 shows how runs 1 and 2 could not produce an adequate result for a text that was particularly challenging, because it could easily be interpreted as an instruction in itself by the language model. however, run 3 produced a better adaptation than the others, even if there is still room for improvement. we believe that it is precisely the flexibility of the ensemble strategy to adapt to different typologies of texts that makes it our best ranked submission, ranking 13th best run and positioning taskgen as the 6th best team in the shared task. ## conclusions & future work taskgen submissions were the result of iterating on different prompting strategies, increasing their complexity based on the qualitative and quantitative analysis of performance on the trial dataset. results show that the ensemble with a voting strategy has proved effective. however, further research is required to improve adaptations to the b1 level, which were far worse than those for the a2 level. also the proposed methods rely on manual prompt engineering and ensembling of different prompts results which may limit the applicability of the proposed solution to other problems or scale. as future work, we aim to explore the specific contribution of the linguistic information used in the prompts, by analysing the individual effects of cefr descriptors, cefr-levelled wordlists and cefr-levelled morphosyntactic structures on the llm performance. we also plan to improve the voting heuristic by inferring the weight of each vote from empirical data, instead of manual inspection. to do that, we will work to obtain or build a bigger labelled corpus, so that empirically inferred votes are reliable. we also plan to examine more advanced prompting strategies, such as agentic and parallel prompting -where separate branches focus on cefr adequacy and semantic semantic fi-delity before combining. finally, we intend to conduct fine-tuning experiments guided by automatic evaluation metrics, including cefr-level classifiers and meaning preservation measures.