<figure><img src="image_1.png" /></figure>

HAL
open science

# UBOnlp Report at the SimpleText lab of CLEF 2025

Benjamin Vendeville, Liana Ermakova, Pierre de Loor, Jaap Kamps

► To cite this version:

Benjamin Vendeville, Liana Ermakova, Pierre de Loor, Jaap Kamps. UBOnlp Report at the SimpleText lab of CLEF 2025. CLEF 2025, Sep 2025, Madrid, France. hal-05305816

HAL Id: hal-05305816

https://hal.science/hal-05305816v1

Submitted on 9 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# UBOnlp Report at the SimpleText lab of CLEF 2025

Benjamin Vendeville<fnref n="1" />,<fnref n="2" />,<fnref n="3" />, Liana Ermakova<fnref n="2" />, Pierre De Loor<fnref n="3" /> and Jaap Kamps<fnref n="4" />

## Abstract

This paper presents the UBOnlp team's participation in the SimpleText lab at CLEF 2025, focusing on scientific text simplification and controlled creativity tasks. We evaluate the performance of GPT-4o using simple prompt-based approaches across multiple subtasks without specialized training or fine-tuning. For Task 1 (Text Simplification), we applied GPT-4o to both sentence-level and document-level simplification of scientific abstracts from the Cochrane-Auto corpus. Our system achieved competitive SARI scores (42.20 for sentence-level, 43.37 for document-level) while maintaining low complexity metrics, demonstrating effective simplification through content reduction rather than lexical substitution. For Task 2 (Controlled Creativity), we addressed spurious generation detection and error classification in simplified texts. Our approach showed strong performance in fluency error detection (F1 = 0.322, ranking first) and alignment error detection (F1 = 0.381, ranking third), but struggled with general spurious content detection, particularly in post-hoc scenarios without source documents. These results highlight both the potential and limitations of large language models for specialized text simplification tasks. While GPT-4o demonstrates capabilities in linguistic quality assessment, task-specific architectures remain superior for comprehensive error detection and generation control. Our findings contribute to understanding the practical applicability of general-purpose language models in scientific text processing workflows.

## Keywords

Automatic text simplification, Science popularization, Large Language Models

## 1. Introduction

With Internet becoming ever more common, people start relying more and more on it for communicating and learning. That includes learning about science and communicating on scientific research. However, science education has failed to keep up with these advances, leading people to misunderstand the science they are reading about online. With the advance of AI, there is a growing potential to build tools that address this issue. This is what the SimpleText lab at CLEF [1, 2, 3] aims to study. The 2025 edition of the SimpleText lab [4] is divided into 3 tasks:

* **Task 1: Text Simplification** simplify scientific text [5].

- **Subtask 1.1:** Simplify sentences.

- **Subtask 1.2:** Simplify abstracts.

* **Task 2: Controlled Creativity** identify and avoid hallucination [6].

- **Subtask 2.1:** Identifying creative generation.

- **Subtask 2.2:** Classifying information distortion.

- **Subtask 2.3:** Avoiding creative generation.

* **Task 3: SimpleText 2024 Revisited** selected tasks by popular request.

- **Subtask 3.1: Content Selection:** Retrieving passages to include in a simplified summary

- **Subtask 3.2: Complexity Spotting:** Identifying and explaining difficult concepts

CLEF 2025 Working Notes, 9 – 12 September 2025, Madrid, Spain

<figure><img src="image_3.png" /></figure>

benjamin.vendeville@univ-brest.fr (B. Vendeville); liana.ermakova@univ-brest.fr (L. Ermakova); deloor@enib.fr

(P. D. Loor); kamps@uva.nl (J. Kamps)

<figure><img src="image_4.png" /></figure>

0009-0003-5298-147X (B. Vendeville); 0000-0002-7598-7474 (L. Ermakova); 0000-0002-5415-5505 (P. D. Loor);

0000-0002-6614-0087 (J. Kamps)

<figure><img src="image_5.png" /></figure>

<footnote n="1">¹Université de Bretagne Occidentale, Brest, France</footnote>

<footnote n="2">²HCTI, Brest, France</footnote>

<footnote n="3">³Lab-STICC (UMR CNRS 6285), Brest, France</footnote>

<footnote n="4">⁴University of Amsterdam, Amsterdam, The Netherlands</footnote>
– **Subtask 3.3: Text Simplification**: Simplify Scientific Text

This paper will detail the participation of team UBOOnlp for tasks 1 and 2, where we used GPT-4o [7] to generate predictions. We will present the task and data provided, as well as the prompts we used for prediction.

# 2. Task 1: Text Simplification

## 2.1. Task Description

The goal of this task was to generate simplifications of scientific texts. It was divided into two subtasks: sentence-level simplification (Subtask 1.1) and document-level simplification (Subtask 1.2). This task used the Cochrane-Auto corpus, built from the Cochrane systematic reviews and their associated lay summaries. Cochrane-Auto consists of professionally written abstract-summary pairs aligned at sentence, paragraph, and document levels. The dataset was constructed by realigning biomedical abstracts and lay summaries at different levels of granularity-sentence, paragraph, and full document. The alignment is restricted to ensure accurate correspondences, enabling meaningful evaluation at each level. The dataset was split into training and test datasets:

* *train*: 4,171 sentences (Task 1.1) and 4,171 paragraphs (Task 1.2)

* *test*: 4,293 sentences (Task 1.1) and 217 abstracts (Task 1.2)

Participants were welcomed to use training data to train models, but we decided to use an untrained, prompt-based approach.

We evaluate system outputs using a range of standard and simplification-specific metrics provided by EASSE [8]. Flesch-Kincaid Grade Level (FKGL) [9] estimates the reading difficulty of a text based on average sentence length and syllables per word, returning a U.S. school grade level; higher values indicate more complex texts, with a theoretical lower bound of -3.40 and no upper limit.

BLEU [10] assesses n-gram overlap between generated and reference texts. Although originally developed for machine translation, it is commonly applied in simplification by treating standard and simplified English as distinct languages. Scores range from 0 (no overlap) to 1 (perfect match).

SARI [11] is specifically designed for text simplification, comparing the system output not only to references but also to the input. It evaluates the quality of additions, deletions, and words retained, with scores ranging from 0 to 100, where higher indicates better simplification.

To characterize structural transformations, we compute the compression ratio, which compares the token count of the output to that of the reference; higher values reflect more compressed outputs. Sentence splits count the number of input sentences divided into multiple ones in the output, with higher counts indicating more frequent segmentation.

We also use Levenshtein similarity to quantify the edit distance between the input and the output, where higher values denote greater surface similarity. The exact copy rate measures the proportion of output sentences that are identical to sentences in the input.

In addition, we track the proportion of additions and deletions, indicating the extent of lexical changes between input and output. Finally, lexical complexity is computed following Alva-Manchego et al. [8], by aggregating the third quartile of the log-frequency ranks of words, capturing the relative rarity of the vocabulary used.

For sentence-level simplification (Task 1.1) sentences were concatenated into abstract and evaluated as such. Furthermore, two different sets of references were used. One was based on the plain language summary (PLS) from the original Cochrane references and contained references for 217 abstracts, while the 2nd was made from Cochrane-auto and contained references for 37 abstracts.

## 2.2. Test Data

The provided test data for Task 1.1 was of the form:
```json
{
  "pair_id": "CD012520",
  "para_id": 0,
  "sent_id": 0,
  "complex": "We included seven cluster-randomised trials with 42,489 patient participants from 129 hospitals, conducted in Australia, the UK, China, and the Netherlands."
}
```

While test data for Task 1.2 was of the form:

```json
{
  "pair_id": "CD012520",
  "source": "Cochrane",
  "complex": "We included seven cluster-randomised trials with 42,489 patient participants from 129 hospitals, conducted in Australia, the UK, China, and the Netherlands. Health professional participants (numbers not specified) included nursing, medical and allied health professionals. Interventions in all studies included [...]"
}
```

## 2.3. Submission Description

Our goal for this task was to see the performance of state of the art models used in a simple way. Therefore, we decided to use GPT-4o to generate simplifications based only on a simple prompt and the source sentence. The decoding was made with a temperature of 0, and we used the following prompt:

```
prompt = f"""You are a classification expert for simplification errors. You need to simplify the following scientific text for the general public.
The goal is to make the provided text more easily understandable.
It is important to keep un easy vocabulary, a simple semantic structure, and to not have too much information density.
You also need to be informative and make the user understand important facts in the source.
----------
Source: "{source}" ""
```

The same prompt is used for both subtasks 1.1 and 1.2.

## 2.4. Results

### 2.4.1. Task 1.1

The evaluation of our run, along with scores of other participants, are presented in Table 1 and Table 2. We see our system being one of the best on SARI on sentence-level simplification while keeping one of the lowest FKGL and lexical complexity scores. Looking at the addition and deletion proportions, our model removed more content than other models, while adding less.

This suggests that our system adopts a more conservative rewriting strategy, favoring deletion over lexical addition. While this may help reduce complexity, it could also risk omitting important information.

On Cochrane-Auto aligned data, however, we observe a notable drop in our model's performance, especially on SARI and BLEU, while other systems such as DSGT *plan_guided_lla* remain closer to the PLS references. Interestingly, this drop coincides with a mismatch in sentence splitting behavior: while our model tends to preserve the original sentence boundaries, the PLS references in Cochrane-Auto may restructure content more, with significantly more sentence splits compared to those in the manually aligned references. This difference may have penalized our system, which performs better for sentence-level rewriting and performs well when reference simplifications follow similar segmentation. Despite this, our model maintains competitive scores on FKGL and lexical complexity, suggesting that it still
produces fluent and accessible output, albeit less aligned with the structural edits present in the PLS references.

<caption>**Table 1**</caption>

Scores of participants runs on Task 1.1 with references from the original plain language summaries from Cochrane.

<table><thead><tr><th>Team/Method</th><th>count</th><th>SARI</th><th>BLEU</th><th>FKGL</th><th>Compression ratio</th><th>Sentence splits</th><th>Levenshtein similarity</th><th>Exact copies</th><th>Additions proportion</th><th>Deletions proportion</th><th>Lexical complexity score</th></tr></thead><tbody><tr><td>Source</td><td>217</td><td>7.84</td><td>10.55</td><td>13.29</td><td>1.00</td><td>1.00</td><td>1.00</td><td>1.00</td><td>0.00</td><td>0.00</td><td>9.05</td></tr><tr><td>Reference</td><td>217</td><td>100</td><td>100</td><td>11.28</td><td>0.72</td><td>0.97</td><td>0.40</td><td>0.00</td><td>0.29</td><td>0.63</td><td>8.65</td></tr><tr><td>DSGT plan_guided_lla</td><td>217</td><td>42.98</td><td>6.33</td><td>7.82</td><td>0.48</td><td>0.99</td><td>0.46</td><td>0.00</td><td>0.18</td><td>0.71</td><td>8.50</td></tr><tr><td>UBOnlp GPT-4o</td><td>217</td><td>42.20</td><td>4.05</td><td>7.49</td><td>0.38</td><td>0.68</td><td>0.37</td><td>0.00</td><td>0.18</td><td>0.78</td><td>8.37</td></tr><tr><td>UM-FHS gpt-4.1-mini</td><td>217</td><td>42.13</td><td>9.52</td><td>7.56</td><td>0.74</td><td>1.52</td><td>0.61</td><td>0.00</td><td>0.26</td><td>0.53</td><td>8.54</td></tr><tr><td>UvA llama31</td><td>217</td><td>40.92</td><td>2.62</td><td>8.63</td><td>1.00</td><td>1.64</td><td>0.45</td><td>0.00</td><td>0.62</td><td>0.64</td><td>8.35</td></tr><tr><td>THM p2-gpt-4.1-nano</td><td>217</td><td>39.57</td><td>6.50</td><td>15.40</td><td>1.32</td><td>1.20</td><td>0.60</td><td>0.00</td><td>0.47</td><td>0.27</td><td>8.68</td></tr><tr><td>UM-FHS gpt-4.1-mini</td><td>217</td><td>39.16</td><td>11.95</td><td>12.23</td><td>0.67</td><td>0.82</td><td>0.60</td><td>0.00</td><td>0.14</td><td>0.50</td><td>8.76</td></tr><tr><td>Fujitsu llm_gpt3.5-t</td><td>217</td><td>38.84</td><td>3.05</td><td>5.04</td><td>0.35</td><td>1.02</td><td>0.44</td><td>0.00</td><td>0.11</td><td>0.75</td><td>8.96</td></tr><tr><td>UvA bartsent-cochran</td><td>217</td><td>38.71</td><td>6.01</td><td>11.34</td><td>0.31</td><td>0.46</td><td>0.45</td><td>0.00</td><td>0.00</td><td>0.72</td><td>8.81</td></tr><tr><td>UvA o-bartsent-cochr</td><td>217</td><td>38.53</td><td>8.57</td><td>11.99</td><td>0.37</td><td>0.49</td><td>0.51</td><td>0.00</td><td>0.01</td><td>0.67</td><td>8.78</td></tr><tr><td>UvA llama31</td><td>217</td><td>38.50</td><td>1.13</td><td>13.66</td><td>1.09</td><td>1.23</td><td>0.40</td><td>0.00</td><td>0.66</td><td>0.71</td><td>8.65</td></tr><tr><td>Fujitsu llm_45</td><td>217</td><td>38.49</td><td>2.06</td><td>5.32</td><td>0.31</td><td>1.00</td><td>0.40</td><td>0.00</td><td>0.09</td><td>0.79</td><td>8.90</td></tr><tr><td>THM p1-gpt-4.1-nano</td><td>217</td><td>38.24</td><td>6.59</td><td>15.03</td><td>1.28</td><td>1.18</td><td>0.63</td><td>0.00</td><td>0.45</td><td>0.25</td><td>8.69</td></tr><tr><td>UM-FHS gpt-4.1</td><td>217</td><td>37.93</td><td>9.46</td><td>8.82</td><td>0.76</td><td>1.22</td><td>0.64</td><td>0.23</td><td>0.22</td><td>0.46</td><td>8.54</td></tr><tr><td>UvA bartdoc-ca</td><td>217</td><td>37.14</td><td>7.23</td><td>11.43</td><td>0.39</td><td>0.49</td><td>0.52</td><td>0.00</td><td>0.01</td><td>0.63</td><td>8.85</td></tr><tr><td>THM pni1-gpt-4.1-na</td><td>217</td><td>35.26</td><td>5.23</td><td>15.49</td><td>1.94</td><td>1.72</td><td>0.54</td><td>0.00</td><td>0.59</td><td>0.12</td><td>8.68</td></tr><tr><td>EngKh biomedical_lla</td><td>217</td><td>33.16</td><td>7.30</td><td>10.76</td><td>1.18</td><td>1.53</td><td>0.65</td><td>0.00</td><td>0.37</td><td>0.25</td><td>8.75</td></tr><tr><td>THM c-gpt-4.1-nano</td><td>217</td><td>32.44</td><td>3.76</td><td>21.37</td><td>1.51</td><td>1.02</td><td>0.62</td><td>0.00</td><td>0.43</td><td>0.20</td><td>9.26</td></tr><tr><td>THM pn1-gemini-2.0-</td><td>217</td><td>32.27</td><td>5.80</td><td>7.92</td><td>1.28</td><td>1.94</td><td>0.66</td><td>0.00</td><td>0.46</td><td>0.20</td><td>8.68</td></tr><tr><td>AIIRLab llama3.1-8b</td><td>217</td><td>29.80</td><td>11.32</td><td>11.19</td><td>0.83</td><td>1.10</td><td>0.80</td><td>0.00</td><td>0.10</td><td>0.29</td><td>8.93</td></tr><tr><td>UM-FHS gpt-4.1-nano</td><td>217</td><td>28.89</td><td>10.35</td><td>9.90</td><td>0.83</td><td>1.19</td><td>0.78</td><td>0.35</td><td>0.13</td><td>0.30</td><td>8.77</td></tr></tbody></table>

### 2.4.2. Task 1.2

The evaluation of our system, UBOnlp GPT-4o, alongside those of other participants, is presented in Table 3 and Table 4. Our system demonstrates competitive performance, particularly on SARI, indicating effective simplification strategies. It produces longer outputs and performs frequent sentence splitting, reflecting a consistent approach focused on decomposing and elaborating complex information rather than merely shortening the text. This is further supported by the high compression ratio and addition proportion, suggesting that the model often introduces explanatory content-such as definitions-to enhance clarity. Despite these strengths, the lower BLEU scores point to a greater divergence from reference phrasing, potentially impacting perceived fluency and alignment. The system also performs well on FKGL and lexical complexity metrics, confirming its ability to adapt the vocabulary and structure to a simpler register.

These tendencies are confirmed in the evaluation against the Cochrane-Auto references, where the results remain broadly consistent-SARI scores decrease slightly, while BLEU improves marginally, highlighting the model's stable behavior across reference sets.
<table><caption>**Table 2**

Scores of participants runs on Task 1.1 with references from Cochrane-Auto.</caption><thead><tr><th>Team/Method</th><th>count</th><th>SARI</th><th>BLEU</th><th>FKGL</th><th>Compression ratio</th><th>Sentence splits</th><th>Levenshtein similarity</th><th>Exact copies</th><th>Additions proportion</th><th>Deletions proportion</th><th>Lexical complexity score</th></tr></thead><tbody><tr><td>Source</td><td>37</td><td>12.03</td><td>20.53</td><td>13.54</td><td>1.00</td><td>1.00</td><td>1.00</td><td>1.00</td><td>0.00</td><td>0.00</td><td>8.89</td></tr><tr><td>Reference</td><td>37</td><td>100</td><td>100</td><td>11.73</td><td>0.56</td><td>0.67</td><td>0.50</td><td>0.0</td><td>0.16</td><td>0.60</td><td>8.71</td></tr><tr><td>UM-FHS gpt-4.1-mini</td><td>37</td><td>43.34</td><td>13.93</td><td>7.46</td><td>0.78</td><td>1.58</td><td>0.63</td><td>0.00</td><td>0.28</td><td>0.50</td><td>8.50</td></tr><tr><td>DSGT plan_guided_lla</td><td>37</td><td>42.33</td><td>10.43</td><td>7.77</td><td>0.48</td><td>0.97</td><td>0.47</td><td>0.00</td><td>0.18</td><td>0.70</td><td>8.52</td></tr><tr><td>UvA o-bartsent-cochr</td><td>37</td><td>42.31</td><td>25.72</td><td>12.08</td><td>0.41</td><td>0.51</td><td>0.55</td><td>0.00</td><td>0.01</td><td>0.62</td><td>8.72</td></tr><tr><td>SINAI PRMZSTASK11V1</td><td>37</td><td>41.82</td><td>6.50</td><td>11.41</td><td>1.37</td><td>1.56</td><td>0.53</td><td>0.00</td><td>0.59</td><td>0.30</td><td>8.33</td></tr><tr><td>THM p2-gpt-4.1-nano</td><td>37</td><td>41.32</td><td>10.49</td><td>14.90</td><td>1.27</td><td>1.16</td><td>0.63</td><td>0.00</td><td>0.45</td><td>0.26</td><td>8.62</td></tr><tr><td>Scalar gpt_md_2_1</td><td>37</td><td>40.95</td><td>14.07</td><td>18.79</td><td>0.62</td><td>0.47</td><td>0.53</td><td>0.00</td><td>0.22</td><td>0.60</td><td>8.68</td></tr><tr><td>UBOnlp GPT-4o</td><td>37</td><td>40.74</td><td>7.53</td><td>7.39</td><td>0.46</td><td>0.80</td><td>0.41</td><td>0.00</td><td>0.23</td><td>0.73</td><td>8.31</td></tr><tr><td>THM p1-gpt-4.1-nano</td><td>37</td><td>40.42</td><td>11.02</td><td>14.66</td><td>1.23</td><td>1.13</td><td>0.65</td><td>0.00</td><td>0.42</td><td>0.24</td><td>8.61</td></tr><tr><td>UM-FHS gpt-4.1</td><td>37</td><td>38.84</td><td>14.04</td><td>8.51</td><td>0.79</td><td>1.26</td><td>0.68</td><td>0.30</td><td>0.22</td><td>0.41</td><td>8.49</td></tr><tr><td>UvA llama31</td><td>37</td><td>38.76</td><td>2.83</td><td>8.30</td><td>0.93</td><td>1.58</td><td>0.46</td><td>0.00</td><td>0.60</td><td>0.66</td><td>8.34</td></tr><tr><td>Fujitsu llm_gpt3.5-t</td><td>37</td><td>38.53</td><td>6.30</td><td>5.18</td><td>0.36</td><td>0.99</td><td>0.45</td><td>0.00</td><td>0.11</td><td>0.74</td><td>8.89</td></tr><tr><td>Fujitsu llm_45_judge</td><td>37</td><td>38.41</td><td>5.45</td><td>5.26</td><td>0.32</td><td>0.89</td><td>0.42</td><td>0.00</td><td>0.09</td><td>0.77</td><td>8.87</td></tr><tr><td>THM pni1-gpt-4.1-na</td><td>37</td><td>37.60</td><td>8.24</td><td>15.21</td><td>1.84</td><td>1.63</td><td>0.56</td><td>0.00</td><td>0.57</td><td>0.12</td><td>8.61</td></tr><tr><td>EngKh biomedical_lla</td><td>37</td><td>36.68</td><td>11.47</td><td>10.62</td><td>1.14</td><td>1.51</td><td>0.65</td><td>0.00</td><td>0.37</td><td>0.28</td><td>8.69</td></tr><tr><td>UvA llama31</td><td>37</td><td>36.45</td><td>1.22</td><td>13.04</td><td>1.07</td><td>1.31</td><td>0.41</td><td>0.00</td><td>0.66</td><td>0.70</td><td>8.61</td></tr><tr><td>THM pn1-gemini-2.0-flash.json Scalar BioBart</td><td>37</td><td>34.47</td><td>9.67</td><td>7.75</td><td>1.25</td><td>1.90</td><td>0.67</td><td>0.00</td><td>0.45</td><td>0.20</td><td>8.62</td></tr><tr><td>THM c-gpt-4.1-nano</td><td>37</td><td>33.94</td><td>5.81</td><td>21.56</td><td>1.49</td><td>0.99</td><td>0.63</td><td>0.00</td><td>0.44</td><td>0.22</td><td>9.22</td></tr><tr><td>DUTH Task11_bart-lar</td><td>37</td><td>27.59</td><td>12.01</td><td>8.67</td><td>1.69</td><td>2.90</td><td>0.66</td><td>0.00</td><td>0.46</td><td>0.09</td><td>8.61</td></tr><tr><td>DUTH Task11_flan-t5-</td><td>37</td><td>22.75</td><td>21.95</td><td>13.15</td><td>0.91</td><td>0.95</td><td>0.94</td><td>0.00</td><td>0.01</td><td>0.11</td><td>8.89</td></tr></tbody></table>

# 3. Task 2: Controlled Creativity

## 3.1. Task Description

In practice, when generating simplifications, organizers have found a high proportion and variety of spurious generation. The goal of this task is therefore to detect, classify and avoid *spurious generation*. In particular, we participated in the following subtasks 2.1 and 2.2.

### 3.1.1. Subtask 2.1

The goal of this subtask is to detect spurious generation. Participants were presented a system generated simplification, and had to classify it as *spurious* or not. in particular, two cases were studied: one (sourced) where participants had access to the source document of the simplification and one (posthoc) where they did not. The dataset was constructed from system simplifications retrieved from last year's submissions to the SimpleText lab and were automatically annotated based on token alignment, where if over 10% of the tokens in the generations were not aligned with the source, the generation was considered spurious. This created a high prevalence of the *spurious* label (90%). The train dataset was split into 13,341 sentences (posthoc) and 13,514 sentences (sourced) while the test dataset was split into 3,336 sentences (posthoc) and 3,379 sentences (sourced). Results are evaluated using Accuracy, Precision,
<table><caption>**Table 3**
Scores of participants runs on Task 1.2 with references from the original plain language summaries from Cochrane.</caption><thead><tr><th>Team/Method</th><th>count</th><th>SARI</th><th>BLEU</th><th>FKGL</th><th>Compression ratio</th><th>Sentence splits</th><th>Levenshtein similarity</th><th>Exact copies</th><th>Additions proportion</th><th>Deletions proportion</th><th>Lexical complexity score</th></tr></thead><tbody><tr><td>Source</td><td>217</td><td>7.84</td><td>10.55</td><td>13.29</td><td>1.00</td><td>1.00</td><td>1.00</td><td>1.00</td><td>0.00</td><td>0.00</td><td>9.05</td></tr><tr><td>Reference</td><td>217</td><td>100</td><td>100</td><td>11.28</td><td>0.72</td><td>0.97</td><td>0.40</td><td>0.00</td><td>0.29</td><td>0.63</td><td>8.65</td></tr><tr><td>LIA sumguid-all-w500</td><td>217</td><td>44.93</td><td>9.58</td><td>9.77</td><td>0.69</td><td>1.06</td><td>0.48</td><td>0.00</td><td>0.29</td><td>0.62</td><td>8.61</td></tr><tr><td>SINAI PRMZSTASK12V1</td><td>217</td><td>43.63</td><td>8.07</td><td>10.73</td><td>0.81</td><td>1.03</td><td>0.52</td><td>0.00</td><td>0.37</td><td>0.54</td><td>8.41</td></tr><tr><td>ASM MistralMinFKGL</td><td>217</td><td>43.51</td><td>8.26</td><td>11.85</td><td>0.63</td><td>0.82</td><td>0.48</td><td>0.00</td><td>0.22</td><td>0.62</td><td>8.78</td></tr><tr><td>LIA sumguid-styl-w50</td><td>217</td><td>43.17</td><td>5.92</td><td>6.87</td><td>0.49</td><td>1.03</td><td>0.39</td><td>0.00</td><td>0.25</td><td>0.75</td><td>8.50</td></tr><tr><td>UBOnlp GPT-4o</td><td>217</td><td>43.37</td><td>4.55</td><td>7.55</td><td>1.20</td><td>2.16</td><td>0.48</td><td>0.00</td><td>0.60</td><td>0.43</td><td>8.31</td></tr><tr><td>ASM MistralV7</td><td>217</td><td>43.10</td><td>7.64</td><td>12.68</td><td>0.66</td><td>0.82</td><td>0.48</td><td>0.00</td><td>0.23</td><td>0.62</td><td>8.86</td></tr><tr><td>AIIRLab Mistral_7b_b</td><td>217</td><td>42.57</td><td>7.47</td><td>9.26</td><td>0.50</td><td>0.82</td><td>0.48</td><td>0.00</td><td>0.16</td><td>0.66</td><td>8.56</td></tr><tr><td>LIA testLlama33</td><td>217</td><td>42.35</td><td>4.70</td><td>11.19</td><td>0.39</td><td>0.54</td><td>0.39</td><td>0.00</td><td>0.14</td><td>0.76</td><td>8.73</td></tr><tr><td>UM-FHS gpt-4.1-mini</td><td>217</td><td>42.13</td><td>9.80</td><td>7.65</td><td>0.69</td><td>1.44</td><td>0.60</td><td>0.00</td><td>0.23</td><td>0.55</td><td>8.57</td></tr><tr><td>UM-FHS gpt-4.1-nano-</td><td>217</td><td>41.01</td><td>7.15</td><td>10.64</td><td>0.48</td><td>0.66</td><td>0.41</td><td>0.00</td><td>0.15</td><td>0.69</td><td>8.58</td></tr><tr><td>UM-FHS gpt-4.1</td><td>217</td><td>38.88</td><td>10.00</td><td>8.97</td><td>0.67</td><td>1.07</td><td>0.59</td><td>0.18</td><td>0.20</td><td>0.52</td><td>8.53</td></tr><tr><td>UvA bartpara-cochran</td><td>217</td><td>34.97</td><td>12.70</td><td>12.13</td><td>0.55</td><td>0.70</td><td>0.68</td><td>0.00</td><td>0.01</td><td>0.49</td><td>8.86</td></tr><tr><td>Scalar gpt_md_2_1</td><td>217</td><td>34.61</td><td>0.02</td><td>9.26</td><td>0.09</td><td>0.13</td><td>0.13</td><td>0.00</td><td>0.02</td><td>0.93</td><td>8.81</td></tr></tbody></table>

Recall, F1 score, AUROC and AUPRC.

### 3.1.2. Subtask 2.2

The goal of this subtask is to detect and classify hallucinations with regards to a taxonomy of [12]. The taxonomy classifies errors in text simplifications into one or multiple of 14 different *errors* classes, grouped into 4 error groups:

* **A. Fluency** Is the answer provided in a correct form that a fluent speaker would speak?

* **B. Alignment** Is the format of the answer correct?

* **C. Information** Is the information provided accurate and relevant to the input?

* **D. Simplification** Does the response focus on simplification?

In addition, a "No Error" class is also considered. The training data is constructed from 42,392 synthetically generated simplifications containing targeted errors generated from past submissions to the SimpleText lab. The test data was constructed from 2,659 manual annotations of past submissions to the SimpleText lab. Results are evaluated on the four aggregated error categories, using both F1 score and AUC.

## 3.2. Test data

### 3.2.1. Subtask 2.1

The provided test datasets were provided in a json format as such:

* Subtask 2.1 Posthoc:
<table><caption>**Table 4**

Scores of participants' runs on Task 1.2 with references from Cochrane-Auto.</caption><thead><tr><th>Team/Method</th><th>count</th><th>SARI</th><th>BLEU</th><th>FKGL</th><th>Compression ratio</th><th>Sentence splits</th><th>Levenshtein similarity</th><th>Exact copies</th><th>Additions proportion</th><th>Deletions proportion</th><th>Lexical complexity score</th></tr></thead><tbody><tr><td>Source</td><td>37</td><td>12.03</td><td>20.53</td><td>13.54</td><td>1.00</td><td>1.00</td><td>1.00</td><td>1.00</td><td>0.00</td><td>0.00</td><td>8.89</td></tr><tr><td>Reference</td><td>37</td><td>100</td><td>100</td><td>11.73</td><td>0.56</td><td>0.67</td><td>0.50</td><td>0.0</td><td>0.16</td><td>0.60</td><td>8.71</td></tr><tr><td>LIA sumguid-all-w500</td><td>37</td><td>44.55</td><td>12.18</td><td>9.71</td><td>0.84</td><td>1.26</td><td>0.50</td><td>0.00</td><td>0.35</td><td>0.54</td><td>8.56</td></tr><tr><td>SINAI PRMZSTASK12V1</td><td>37</td><td>43.93</td><td>10.81</td><td>10.45</td><td>0.86</td><td>1.07</td><td>0.55</td><td>0.00</td><td>0.39</td><td>0.49</td><td>8.33</td></tr><tr><td>UM-FHS gpt-4.1</td><td>37</td><td>43.83</td><td>18.12</td><td>8.80</td><td>0.67</td><td>1.10</td><td>0.58</td><td>0.14</td><td>0.21</td><td>0.53</td><td>8.44</td></tr><tr><td>UM-FHS gpt-4.1-nano-</td><td>37</td><td>43.61</td><td>16.00</td><td>10.63</td><td>0.50</td><td>0.69</td><td>0.45</td><td>0.00</td><td>0.16</td><td>0.65</td><td>8.55</td></tr><tr><td>UM-FHS gpt-4.1-mini</td><td>37</td><td>43.53</td><td>14.11</td><td>7.48</td><td>0.72</td><td>1.49</td><td>0.62</td><td>0.00</td><td>0.25</td><td>0.52</td><td>8.52</td></tr><tr><td>ASM MistralMaxFRE</td><td>37</td><td>43.35</td><td>12.32</td><td>11.63</td><td>0.73</td><td>0.92</td><td>0.53</td><td>0.00</td><td>0.27</td><td>0.56</td><td>8.74</td></tr><tr><td>ASM MistralMinFKGL</td><td>37</td><td>43.24</td><td>12.27</td><td>11.63</td><td>0.73</td><td>0.93</td><td>0.53</td><td>0.00</td><td>0.27</td><td>0.56</td><td>8.75</td></tr><tr><td>ASM MistralV7CleanLi</td><td>37</td><td>42.93</td><td>11.38</td><td>13.77</td><td>0.78</td><td>0.84</td><td>0.51</td><td>0.00</td><td>0.29</td><td>0.56</td><td>8.80</td></tr><tr><td>UvA baseline-cochran</td><td>37</td><td>42.10</td><td>24.27</td><td>11.71</td><td>0.57</td><td>0.71</td><td>0.61</td><td>0.00</td><td>0.06</td><td>0.49</td><td>8.74</td></tr><tr><td>UBOnlp GPT-4o</td><td>37</td><td>41.56</td><td>5.45</td><td>7.22</td><td>1.14</td><td>2.08</td><td>0.50</td><td>0.00</td><td>0.58</td><td>0.43</td><td>8.25</td></tr><tr><td>AIIRLab llama_3.1-8b</td><td>37</td><td>41.07</td><td>8.61</td><td>9.22</td><td>0.46</td><td>0.70</td><td>0.43</td><td>0.00</td><td>0.20</td><td>0.72</td><td>8.44</td></tr><tr><td>LIA testLlama33</td><td>37</td><td>40.79</td><td>8.42</td><td>10.74</td><td>0.46</td><td>0.65</td><td>0.42</td><td>0.00</td><td>0.18</td><td>0.73</td><td>8.64</td></tr><tr><td>AIIRLab llama-8b</td><td>37</td><td>39.14</td><td>5.62</td><td>8.88</td><td>0.34</td><td>0.62</td><td>0.35</td><td>0.00</td><td>0.15</td><td>0.80</td><td>8.43</td></tr><tr><td>AIIRLab llama3.2-3b</td><td>37</td><td>39.14</td><td>5.62</td><td>8.88</td><td>0.34</td><td>0.62</td><td>0.35</td><td>0.00</td><td>0.15</td><td>0.80</td><td>8.43</td></tr><tr><td>UvA bartpara-cochran</td><td>37</td><td>37.89</td><td>27.43</td><td>12.22</td><td>0.62</td><td>0.77</td><td>0.74</td><td>0.00</td><td>0.01</td><td>0.41</td><td>8.78</td></tr><tr><td>UvA bartdoc-ca</td><td>37</td><td>37.25</td><td>19.54</td><td>11.97</td><td>0.51</td><td>0.61</td><td>0.62</td><td>0.00</td><td>0.02</td><td>0.52</td><td>8.77</td></tr><tr><td>UM-FHS gpt-4.1-nano</td><td>37</td><td>37.01</td><td>14.74</td><td>9.05</td><td>0.69</td><td>1.13</td><td>0.64</td><td>0.19</td><td>0.16</td><td>0.46</td><td>8.57</td></tr><tr><td>UvA llama31</td><td>37</td><td>36.98</td><td>3.99</td><td>7.61</td><td>0.79</td><td>1.59</td><td>0.39</td><td>0.00</td><td>0.46</td><td>0.77</td><td>8.48</td></tr><tr><td>Scalar gpt_md_2_1</td><td>37</td><td>34.39</td><td>1.01</td><td>10.56</td><td>0.14</td><td>0.19</td><td>0.20</td><td>0.00</td><td>0.03</td><td>0.88</td><td>8.67</td></tr></tbody></table>

```json
{
  "sentence": "I explained the complex terms directly within the simplified sentence:\n\n* 'Next-generation model' means a new and improved plan.",
  "anon_gen_id": "74704850//66348262//3"
},
```

* Subtask 2.1 Sourced:

```json
{
  "abs_id": "G01.1_1570837852",
  "sentence": "In this paper, we share our findings on how evolutionary algorithms and multi-agent systems can be used to understand a user's preferences while they interact with a digital assistant.",
  "gen_id": "11102757//G01.1_1570837852//1"
},
```

The Sourced data could be merged with the abstract data of the following format:

```json
{
  "query_id": "G11.1",
  "query_text": "drones",
  "doc_id": 2892036907,
```
"abs_id": "G11.1_2892036907",
"abs_source": "In the modern era of automation and robotics, autonomous vehicles are currently the focus of academic and industrial research. With the ever increasing number of unmanned aerial vehicles getting involved in activities in the civilian and commercial domain, there is [...]"
},

### 3.2.2. Subtask 2.2

The test data was provided as a json file as such:

{
  "source sentence": "Partners In Health (PIH) and its sister organization in Lima, Peru, Socios En Salud (SES), treat a majority of multidrug-resistant tuberculosis (MDR-TB) patients in Peru, in conjunction with the Peruvian National TB Program (NTP).",
  "simplified sentence": "Socios En Salud (SES) and its sister organization in Lima, Peru, treat a majority of multidrug-resistant tuberculosis (MDR-TB) patients in Peru, in conjunction",
  "snt_id": "G01.1_147704292_1",
  "simp_id": 783909.0,
  "No error": "",
  "A1. Random generation": "",
  "A2. Syntax error": "",
  "A3. Contradiction": "",
  "A4. Simple punctuation / grammar errors": "",
  "A5. Redundancy": "",
  "B1. Format misalignement": "",
  "B2. Prompt misalignement": "",
  "C1. Factuality hallucination": "",
  "C2. Faithfulness hallucination": "",
  "C3. Topic shift": "",
  "D1.1. Overgeneralization": "",
  "D1.2 Overspecification of Concepts": "",
  "D2.1. Loss of Informative Content": "",
  "D2.2. Out-of-Scope Generation": ""
}

## 3.3. Submission Description

In both subtasks, our goal was to try to measure performance of state of the art models used in a naive, simple way. In both subtasks, we relied on an untrained GPT-4o model using only a prompt with test data as input. The decoding was made with a temperature of 0.

### 3.3.1. Subtask 2.1

For this subtask, we used two slightly different prompts for the *sourced* and *posthoc* variations. For *posthoc* we used the following prompt:

prompt = f"""

You are an expert in detecting hallucinations in simplified scientific texts.

Hallucinations include:

- Information distortion: misrepresenting or oversimplifying facts in a misleading way.

- Spurious generation: adding information not supported by scientific content.

Your task: Analyze the simplified text and respond only with:
- True -> if the text likely contains a hallucination.

- False -> if the text seems accurate and faithful.

Respond with **only** `True` or `False`.

----------

Simplified Text:
{simplified}
"""

For the *sourced* variation, we used:

prompt = f"""

You are an expert in detecting hallucinations in simplified scientific texts.

Hallucinations include:

- **Information distortion**: when the simplified text misrepresents or alters the meaning of the source.

- **Spurious generation**: when the simplified text includes new information not present or supported in the source.

Your task is to compare the simplified text with the source and respond with:

- True -> if the simplified text contains hallucinations (of either type).

- False -> if the simplified text is faithful to the source.

Respond with **only** True or False.

----------

Source Text:
{source}

Simplified Text:
{simplified}
"""

### 3.3.2. Subtask 2.2

For the subtask 2.2, we used a prompt describing the taxonomy, as well as the format required, and included examples. The taxonomy is the definition of the errors as provided in [12] while *possible codes* are the codes corresponding to the error.

prompt = f"""

You are a classification expert for simplification errors. Whenever you answer, you MUST call the function *classify_simplification_errors* and pass exactly one JSON object of the form:

```json
{{ "labels": [<list of codes>] }} and nothing else-no prose, no extra keys.
```

The only valid key inside that object is "labels" (an array of strings).

TAXONOMY:
{taxonomy}

AVAILABLE CODES:
{POSSIBLE_CODES}

IMPORTANT RULES:

1. The function call must look exactly like:
   `classify_simplification_errors({{ "labels": [ ... ] }})`
   with no extra text before or after.

2. If **no error** applies, return exactly:
   `classify_simplification_errors({{ "labels": ["No"] }})`

3. If you choose any code *other than* "No," you **MUST NOT** include "No" in your array.
4. The list inside "labels" must be in **lexicographic order** (e.g. ["A1", "B2", "D22"], not ["B2", "A1", "D22"]).

5. If multiple codes apply, include all of them (still in lexicographic order). If only one applies, return only that one (e.g. ["A4"]).

EXAMPLES (you see INPUT -> function call OUTPUT):

- Example 1 (no error):

Input:

Source: "The cat meowed softly."

Simplification: "The cat meowed softly."

Output:

classify_simplification_errors({"labels": ["No"]})

- Example 2 (single-label):

Input:

Source: "The cat chattered."

Simplification: "The cat meowed."

Explanation: "chattered" -> "meowed" fixes a word-choice/typo error -> A2 Syntax error.

Output:

classify_simplification_errors({"labels": ["A2"]})

- Example 3 (multi-label):

Input:

Source: "Bob went to the market. He loves fruit."

Simplification: "Bob went market; he adores apples and wrote a poem about them."

Explanation:

- "went market;" has a grammar/syntax issue -> A2 Syntax error

- "he adores apples and wrote a poem about them" adds extra content (not part of simplification) -> D22 Out-of-Scope Generation

Output:

classify_simplification_errors({"labels": ["A2", "D22"]})

NOW CLASSIFY ONLY the pair below. Do not write any additional text-just invoke
classify_simplification_errors(...) once.

----------

Source: "{source}"

Simplification: "{simplification}"

"""

## 3.4. Results

### 3.4.1. Subtask 2.1

Results for this subtask are presented in table 5 and table 6 In the posthoc detection scenario, our GPT-4o approach ranked last among the participating teams. The results reveal a characteristic pattern: while our method achieved high precision (0.92), indicating that when it predicted spurious generation it was usually correct, it suffered from extremely low recall (0.21). This suggests our GPT-4o approach was overly conservative in identifying spurious content when operating without access to source documents. The low accuracy (0.27) and near-random AUROC (0.52) indicate that our approach struggled significantly with the posthoc detection task. Given that the dataset has a 90% prevalence of spurious examples, our low recall particularly hurt overall performance.

When source documents were available, our GPT-4o approach showed improved but still limited performance. The recall increased from 0.21 to 0.71, and accuracy improved from 0.27 to 0.70. This suggests that GPT-4o benefits significantly from having reference material to compare against when detecting spurious generation. However, our approach still ranked in the lower tier of submissions, with several teams achieving accuracy scores above 0.90 and F1 scores above 0.95.

The performance difference between our approach and top-performing methods (which achieved F1-scores above 0.95) suggests that task-specific model architectures, such as BERT-based classifiers
and ensemble methods, may still be more suitable for this type of detection task than general-purpose language models used in a zero-shot or few-shot manner.

<caption>**Table 5**</caption>

Results for CLEF 2025 SimpleText Task 2.1 Detecting Overgeneration: Test data, posthoc detection without sources, best five runs per team

<table><caption>**Table 6**</caption><thead><tr><th>Team/Method</th><th>count</th><th>Acc.</th><th>Prec</th><th>Rec</th><th>F1</th><th>AUROC</th><th>AUPRC</th></tr></thead><tbody><tr><td>SINAI basic-prefilter-all-true</td><td>3,336</td><td>0.91</td><td>0.91</td><td>1.00</td><td>0.95</td><td>0.55</td><td>0.91</td></tr><tr><td>DSGT bertclassifier</td><td>3,336</td><td>0.91</td><td>0.93</td><td>0.97</td><td>0.95</td><td>0.64</td><td>0.93</td></tr><tr><td>DSGT bert_nli_ilm_ensemble</td><td>3,336</td><td>0.90</td><td>0.93</td><td>0.97</td><td>0.95</td><td>0.64</td><td>0.93</td></tr><tr><td>DSGT bertnlillmensemble</td><td>3,336</td><td>0.90</td><td>0.93</td><td>0.97</td><td>0.95</td><td>0.64</td><td>0.93</td></tr><tr><td>DUTH Task21posthoc_et</td><td>3,336</td><td>0.90</td><td>0.92</td><td>0.97</td><td>0.95</td><td>0.62</td><td>0.92</td></tr><tr><td>DSGT Ilm</td><td>3,336</td><td>0.77</td><td>0.95</td><td>0.78</td><td>0.86</td><td>0.70</td><td>0.94</td></tr><tr><td>DSGT nli_entailment</td><td>3,336</td><td>0.45</td><td>0.95</td><td>0.41</td><td>0.57</td><td>0.61</td><td>0.92</td></tr><tr><td>SINAI improved-prefilter-confidence-95</td><td>3,336</td><td>0.35</td><td>0.95</td><td>0.29</td><td>0.44</td><td>0.57</td><td>0.91</td></tr><tr><td>UBOnlp GPT-4o</td><td>3,379</td><td>0.27</td><td>0.92</td><td>0.21</td><td>0.35</td><td>0.52</td><td>0.90</td></tr></tbody></table>

Results for CLEF 2025 SimpleText Task 2.1 Detecting Overgeneration: Test data, detection with sources.

<table><thead><tr><th>Team/Method</th><th>count</th><th>Acc.</th><th>Prec</th><th>Rec</th><th>F1</th><th>AUROC</th><th>AUPRC</th></tr></thead><tbody><tr><td>AIIRLab CrossEncoder</td><td>3,379</td><td>0.98</td><td>0.99</td><td>0.99</td><td>0.99</td><td>0.95</td><td>0.99</td></tr><tr><td>Mtest bartfinetuned</td><td>3,379</td><td>0.97</td><td>0.99</td><td>0.97</td><td>0.98</td><td>0.96</td><td>0.99</td></tr><tr><td>SINAI improved-prefilter-all-true</td><td>3,379</td><td>0.96</td><td>1.00</td><td>0.95</td><td>0.98</td><td>0.98</td><td>0.99</td></tr><tr><td>SINAI improved-prefilter-confidence-99</td><td>3,379</td><td>0.93</td><td>1.00</td><td>0.93</td><td>0.96</td><td>0.96</td><td>0.99</td></tr><tr><td>DSGT bertclassifier</td><td>3,379</td><td>0.91</td><td>0.93</td><td>0.98</td><td>0.95</td><td>0.65</td><td>0.93</td></tr><tr><td>DSGT bertnlillmensemble</td><td>3,379</td><td>0.91</td><td>0.93</td><td>0.97</td><td>0.95</td><td>0.68</td><td>0.93</td></tr><tr><td>DUTH Task21sourced_et</td><td>3,379</td><td>0.91</td><td>0.93</td><td>0.97</td><td>0.95</td><td>0.66</td><td>0.93</td></tr><tr><td>SINAI improved-prefilter-confidence-95</td><td>3,379</td><td>0.81</td><td>1.00</td><td>0.79</td><td>0.88</td><td>0.89</td><td>0.98</td></tr><tr><td>DUTH Task21sourced_ridge</td><td>3,379</td><td>0.77</td><td>0.94</td><td>0.79</td><td>0.86</td><td>0.68</td><td>0.93</td></tr><tr><td>DSGT Ilm</td><td>3,379</td><td>0.74</td><td>0.94</td><td>0.76</td><td>0.84</td><td>0.68</td><td>0.93</td></tr><tr><td>UBOnlp GPT-4o</td><td>3,379</td><td>0.70</td><td>0.95</td><td>0.71</td><td>0.81</td><td>0.69</td><td>0.93</td></tr><tr><td>RECAIDS T5</td><td>3,379</td><td>0.49</td><td>0.89</td><td>0.49</td><td>0.63</td><td>0.47</td><td>0.89</td></tr><tr><td>DSGT nli_entailment</td><td>3,379</td><td>0.35</td><td>0.92</td><td>0.31</td><td>0.46</td><td>0.53</td><td>0.90</td></tr><tr><td>AIIRLab LLMs</td><td>3,379</td><td>0.10</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.50</td><td>0.90</td></tr></tbody></table>

### 3.4.2. Subtask 2.2

Our system achieved the best F1 score (0.322) for fluency error detection, outperforming all competing systems including specialized fine-tuned models. This demonstrates GPT-4o's capabilities for identifying grammatical errors and fluency issues. It also showed strong performance in alignment error detection (F1 = 0.381, 3rd place), showing effective identification of format and structural issues. However, our system showed lower performance in "No Error" classification (F1 = 0.680) suggesting tendency toward false positives. Information and simplification error detection showed moderate results, indicating challenges with task-specific requirements.

The results highlight GPT-4o's strength in linguistic tasks while revealing limitations in specialized error detection, showing the usefulness of building task-specific error detection models.

## 4. Conclusion

This paper evaluated GPT-4o's effectiveness for scientific text simplification and controlled creativity tasks at CLEF 2025 SimpleText using straightforward prompt-based approaches without specialized
**Table 7**

<table><caption>Model Performance by Error Categories (Best Scores in Bold) for No error, Fluency (A), Alignment(B), Information (C), and Simplification (D) categories, with F₁ and AUC-PR.</caption><thead><tr><th rowspan="2">Team/Method</th><th colspan="2">No Error</th><th colspan="2">A</th><th colspan="2">B</th><th colspan="2">C</th><th colspan="2">D</th></tr><tr><th>F₁</th><th>AUC</th><th>F₁</th><th>AUC</th><th>F₁</th><th>AUC</th><th>F₁</th><th>AUC</th><th>F₁</th><th>AUC</th></tr></thead><tbody><tr><td>DSGT DebertaLlmensemble</td><td>0.763</td><td>0.561</td><td>0.283</td><td>0.133</td><td>0.354</td><td>0.173</td><td>0.301</td><td>0.156</td><td>0.374</td><td>0.224</td></tr><tr><td>AIIRLab paraphrase_mpnet</td><td>0.755</td><td>0.567</td><td>0.255</td><td>0.154</td><td>0.258</td><td>0.113</td><td>0.136</td><td>0.084</td><td>0.147</td><td>0.168</td></tr><tr><td>DSGT roberta</td><td>0.694</td><td>0.491</td><td>0.233</td><td>0.121</td><td>0.249</td><td>0.101</td><td>0.114</td><td>0.089</td><td>0.128</td><td>0.164</td></tr><tr><td>UBOnlp GPT-4o</td><td>0.680</td><td>0.505</td><td>0.322</td><td>0.150</td><td>0.381</td><td>0.192</td><td>0.250</td><td>0.122</td><td>0.292</td><td>0.136</td></tr><tr><td>DSGT llama</td><td>0.680</td><td>0.483</td><td>0.282</td><td>0.132</td><td>0.324</td><td>0.182</td><td>0.269</td><td>0.147</td><td>0.306</td><td>0.196</td></tr><tr><td>AIIRLab OpenChat</td><td>0.640</td><td>0.421</td><td>0.154</td><td>0.070</td><td>0.141</td><td>0.061</td><td>0.144</td><td>0.080</td><td>0.222</td><td>0.156</td></tr><tr><td>AIIRLab MajorityVoting</td><td>0.633</td><td>0.415</td><td>0.156</td><td>0.071</td><td>0.110</td><td>0.045</td><td>0.170</td><td>0.088</td><td>0.239</td><td>0.160</td></tr><tr><td>DSGT BERT</td><td>0.515</td><td>0.330</td><td>0.214</td><td>0.133</td><td>0.208</td><td>0.103</td><td>0.167</td><td>0.095</td><td>0.129</td><td>0.161</td></tr><tr><td>DUTH scibert</td><td>0.436</td><td>0.321</td><td>0.088</td><td>0.045</td><td>0.035</td><td>0.025</td><td>0.100</td><td>0.066</td><td>0.145</td><td>0.135</td></tr><tr><td>Mtest bartfinetuned</td><td>0.404</td><td>0.322</td><td>0.270</td><td>0.143</td><td>0.472</td><td>0.265</td><td>0.078</td><td>0.074</td><td>0.128</td><td>0.167</td></tr><tr><td>DSGT bert_llama_ensemble</td><td>0.404</td><td>0.322</td><td>0.231</td><td>0.137</td><td>0.253</td><td>0.107</td><td>0.116</td><td>0.088</td><td>0.128</td><td>0.163</td></tr><tr><td>RECAIDSTechTitans T5</td><td>0.404</td><td>0.322</td><td>0.022</td><td>0.046</td><td>0.000</td><td>0.026</td><td>0.004</td><td>0.065</td><td>0.000</td><td>0.136</td></tr><tr><td>DUTH logreg</td><td>0.404</td><td>0.322</td><td>0.000</td><td>0.044</td><td>0.000</td><td>0.026</td><td>0.000</td><td>0.064</td><td>0.000</td><td>0.136</td></tr></tbody></table>

training. Our results demonstrate both strengths and limitations of general-purpose language models for specialized text processing tasks. In text simplification, GPT-4o achieved competitive SARI scores (42.20 sentence-level, 43.37 document-level) through a conservative strategy that prioritized content reduction over lexical substitution. For controlled creativity, the model excelled in fluency error detection (highest F1 score among participants) and alignment error detection, but struggled with spurious generation detection, particularly in post-hoc scenarios without source documents. These findings highlight that while GPT-4o demonstrates strong linguistic capabilities for quality assessment tasks, task-specific architectures remain superior for comprehensive error detection and generation control. The substantial performance gap between our approach and specialized systems indicates that domain-specific fine-tuning or architectural modifications are necessary for optimal performance in critical applications. Future work should explore hybrid approaches combining the linguistic sophistication of large language models with the precision of specialized architectures. Our results underscore the importance of careful evaluation when deploying general-purpose language models in specialized domains where accuracy and reliability are essential.

## Acknowledgments

This research was funded by the French National Research Agency (ANR) under the projects ANR-22-CE23-0019-01 and ANR-19-GURE-0001 (program *Investissements d'avenir* integrated into France 2030).

## Declaration on Generative AI

During the preparation of this work, the author(s) used ChatGPT and Claude in order to: Grammar and spelling check, Paraphrase and reword, and Drafting content.

## References

[1] L. Ermakova, E. SanJuan, S. Huet, H. Azarbonyad, G. M. Di Nunzio, F. Vezzani, J. D'Souza, J. Kamps, Overview of the clef 2024 simpletext track: Improving access to scientific texts for everyone, in: Experimental IR Meets Multilinguality, Multimodality, and Interaction: 15th International Conference of the CLEF Association, CLEF 2024, Grenoble, France, September
9-12, 2024, Proceedings, Part II, Springer-Verlag, Berlin, Heidelberg, 2024, p. 283-307. URL: https://doi.org/10.1007/978-3-031-71908-0_13. doi:10.1007/978-3-031-71908-0_13.

[2] L. Ermakova, E. SanJuan, S. Huet, H. Azarbonyad, G. M. Di Nunzio, F. Vezzani, J. D'Souza, S. Kabongo, H. B. Giglou, Y. Zhang, S. Auer, J. Kamps, CLEF 2024 SimpleText Track, in: N. Go-harian, N. Tonellotto, Y. He, A. Lipani, G. McDonald, C. Macdonald, I. Ounis (Eds.), Advances in Information Retrieval, Springer Nature Switzerland, Cham, 2024, pp. 28-35. doi:10.1007/978-3-031-56072-9_4.

[3] L. Ermakova, S. Bertin, H. McCombie, J. Kamps, Overview of the clef 2023 simpletext task 3: Simplification of scientific texts, Overview of the CLEF 2023 SimpleText Task 3 (2023).

[4] L. Ermakova, et al., Overview of CLEF 2025 SimpleText Track: Simplify Scientific Texts (and Nothing More), in: J. C. de Albornoz, et al. (Eds.), Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings of the Fifteenth International Conference of the CLEF Association (CLEF 2025), Lecture Notes in Computer Science, Springer-Verlag, 2025.

[5] J. Bakker, et al., Overview of the CLEF 2025 SimpleText Task 1: Simplify Scientific Text, in: G. Faggioli, et al. (Eds.), Working Notes of the Conference and Labs of the Evaluation Forum (CLEF 2025), CEUR Workshop Proceedings, CEUR-WS.org, 2025.

[6] B. Vendeville, et al., Overview of the CLEF 2025 SimpleText Task 2: Identify and Avoid Hallucination, in: G. Faggioli, et al. (Eds.), Working Notes of the Conference and Labs of the Evaluation Forum (CLEF 2025), CEUR Workshop Proceedings, CEUR-WS.org, 2025.

[7] OpenAI, A. Hurst, A. Lerer, A. P. Goucher, Perelman, et al., GPT-4o System Card, 2024. doi:10.48550/arXiv.2410.21276. arXiv:2410.21276.

[8] F. Alva-Manchego, L. Martin, C. Scarton, L. Specia, EASSE: Easier Automatic Sentence Simplification Evaluation, in: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations, Association for Computational Linguistics, Hong Kong, China, 2019, pp. 49-54. doi:10.18653/v1/D19-3009.

[9] J. P. Kincaid, Jr. Fishburne, R. Robert P., C. Richard L., Brad S., Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel: Technical Report, Defense Technical Information Center, Fort Belvoir, VA, 1975. doi:10.21236/ADA006655.

[10] K. Papineni, S. Roukos, T. Ward, W.-J. Zhu, Bleu: A Method for Automatic Evaluation of Machine Translation, in: P. Isabelle, E. Charniak, D. Lin (Eds.), Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Philadelphia, Pennsylvania, USA, 2002, pp. 311-318. doi:10.3115/1073083.1073135.

[11] W. Xu, C. Napoles, E. Pavlick, Q. Chen, C. Callison-Burch, Optimizing Statistical Machine Translation for Text Simplification, Transactions of the Association for Computational Linguistics 4 (2016) 401-415. doi:10.1162/tacl_a_00107.

[12] B. Vendeville, L. Ermakova, P. D. Loor, Resource for Error Analysis in Text Simplification: New Taxonomy and Test Collection, 2025. doi:10.1145/3726302.3730304. arXiv:2505.16392.