# Overview of the CLEF 2023 SimpleText Task 1: Passage Selection for a Simplified Summary

Eric Sanjuan, Stéphane Huet, Jaap Kamps, Liana Ermakova

<figure><img src="image_1.png" /></figure>

## To cite this version:

Eric Sanjuan, Stéphane Huet, Jaap Kamps, Liana Ermakova. Overview of the CLEF 2023 SimpleText Task 1: Passage Selection for a Simplified Summary. Working Notes of the Conference and Labs of the Evaluation Forum (CLEF 2023), Sep 2023, Thessalokini, Greece. pp.2823-2834. hal-05181809

HAL Id: hal-05181809

https://hal.science/hal-05181809v1

Submitted on 23 Jul 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# Overview of the CLEF 2023 SimpleText Task 1: Passage Selection for a Simplified Summary

Éric SanJuan¹, Stéphane Huet¹, Jaap Kamps² and Liana Ermakova³

¹Avignon Université, LIA, France

²University of Amsterdam, Amsterdam, The Netherlands

³Université de Bretagne Occidentale, HCTI, France

**Abstract**

This paper presents an overview of the CLEF 2023 SimpleText Task 1: Content Selection, asking systems to retrieve scientific abstracts in response to a query prompted by a popular science article. Overall, the SimpleText track provides an evaluation platform for the automatic simplification of scientific texts. We discuss the details of the task set-up. First, the SimpleText Corpus with over 4 million academic papers and abstracts. Second, the Topics based on 40 popular science articles in the news and the 114 Queries prompted by them. Third, the Formats of requests and results, the Evaluation labels and Evaluation measures used. Fourth, the Results of the runs submitted by our participants.

**Keywords**

information retrieval, scientific documents, text simplification, scientific information retrieval, non-expert queries, press outlets, query-document relationships (Q-rels), popularized science

## 1. Introduction

This paper presents an overview of the first task in the SimpleText track on automatic simplification of scientific texts following up on the CLEF 2021 SimpleText Workshop [1] and CLEF 2022 SimpleText Track [2]. The main goal of the SimpleText track is to provide data and benchmarks to advance research in this area. This paper focuses on *Task 1: What is in (or out)? Selecting passages to include in a simplified summary*. This task is part of a pipeline with the two other SimpleText tasks, namely *Task 2: What is unclear? Difficult concept identification and explanation* and *Task 3: Rewrite this! Given a query, simplify passages from scientific abstracts*. For a comprehensive understanding of the other tasks, the overview papers of Task 2 [3] and Task 3 [4], as well as the Track overview paper [?], provide detailed information and insights.

Scientific literacy is a vital skill for individuals. It serves as a key component of critical thinking, enabling individuals to make objective decisions and assess the validity and significance of research findings. Scientific literacy helps to differentiate between reliable evidence and unsubstantiated claims and navigate the complex landscape of scientific advancements. Despite increasing digitization and open access to scientific literature, several barriers remain that impede non-experts from accessing unbiased scientific information from these texts. One

CLEF 2023: *Conference and Labs of the Evaluation Forum, September 18–21, 2023, Thessaloniki, Greece*

<figure><img src="image_3.png" /></figure>

eric.sanjuan@univ-avignon.fr (É. SanJuan); liana.ermakova@univ-brest.fr (L. Ermakova)

<figure><img src="image_4.png" /></figure>

https://simpletext-project.com/ (L. Ermakova)

<figure><img src="image_5.png" /></figure>

0000-0002-7598-7474 (L. Ermakova)

<figure><img src="image_6.png" /></figure>

© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

<figure><img src="image_7.png" /></figure>

CEUR Workshop Proceedings (CEUR-WS.org)
primary challenge is the difficulty in comprehending scientific literature, which arises from
its reliance on specialized knowledge and the utilization of complex terminology. As a result,
non-experts may face obstacles when attempting to understand and interpret scientific papers.
Retrieval of relevant yet credible and understandable scientific documents is still a challenge as
search engines virtually ignore documents' difficulty.

The rest of the paper is organized as follows. Section 2 provides details on the datasets utilized
and the evaluation metrics employed in the study. Section 3 offers an overview of the retrieval
approaches adopted by the participants, specifically focusing on the scientific text. In Section 4,
the official submissions' results are presented and discussed. Finally, Section 5 summarizes the
findings and outlines potential directions for future research.

## 2. SimpleText Task 1 Test Collection

This section provides an overview of the resulting test collection, detailing the corpus, the topics and queries, the input and output format, as well as the used evaluation measures.

### 2.1. Corpus: DBLP abstracts

The corpus utilized in this task is the Citation Network Dataset: DBLP+Citation, ACM Citation
network (12th version released in 2020¹)[5, 6] containing 4,894,081 papers (2020-04-09) with
abstract content and field of subjects, made available from Microsoft academic services[7]. An
ElasticSearch index is provided to participants with access through an API. A JSON dump of the
index is also released. Besides, document bibliographic information and abstract content in the
form of an inverted positional index, can be retrieved from OpenAlex²[8] using their document
ids as work references (W).

### 2.2. Topics: Press articles

Topics are a selection of press articles from the tech section of The Guardian newspaper (topics G01 to G20) and the Tech Xplore website (topics T01 to T20). URLs to original articles and textual content of each topic are provided to participants. All abstracts extracted from the document collection by participants are expected to be relevant to subjects addressed in the press articles.

#### 2.2.1. Queries as facets

Between one and four keywords queries are provided with each topic. It has been manually
checked that each query allows retrieving relevant passages that could be inserted as citations
in the press article.

<footnote n="1">¹https://www.aminer.org
²https://docs.openalex.org/</footnote>
<table><caption>**Table 1**
Statistics on qrels.</caption><thead><tr><th rowspan="2">Qrels</th><th rowspan="2">Topics</th><th rowspan="2">#Queries</th><th colspan="3">#Assessed abstracts</th><th rowspan="2">#Avg Ass.</th></tr><tr><th>0</th><th>1</th><th>2</th></tr></thead><tbody><tr><td>2022 test</td><td>G1-G20, T2,4,5,10-12,15-16,T18-20</td><td>72</td><td>192</td><td>187</td><td>107</td><td>6.8</td></tr><tr><td>2023 train</td><td>G1-G15</td><td>29</td><td>728</td><td>338</td><td>237</td><td>44.9</td></tr><tr><td>2023 test</td><td>G16-G20, T1-T5</td><td>34</td><td>2260</td><td>357</td><td>1218</td><td>112.8</td></tr></tbody></table>

### 2.2.2. Qrels

Quality relevance of abstracts w.r.t. topics is given in Simpletext_2023_task1_train.qrels, distributed to participants. This file extends the qrels released last year with a significant increase of the depth of judgments of abstracts per query (Table 1, lines 1 and 2). Thus, the average number of assessed abstracts by query has raised from 6.8 to 44.9. Relevance annotations are provided on a 0-2 scale (the higher the more relevant) for 29 queries associated with the first 15 articles from the Guardian.<fnref n="3" />

From runs submitted this year (see Section 3), a new qrels file was built using pooling on 10 topics different from the train file: the last 5 topics from the Guardian and the first 5 from Tech Xplore (Table 1, line 3). For pooling, only top 10 abstracts of each submitted run were considered for manual assessment. While the train file was released to participants to allow them to have supervised approaches, the test file is only used for the track evaluation.

## 2.3. Expected results

### 2.3.1. Ad-hoc passage retrieval

Participants had to retrieve, for each topic and each query, all passages from DBLP abstracts, related to the query and relevant to be inserted as a citation in the paper associated with the topic. Some passages could require simplification. We encouraged participants to take into account passage complexity as well as its credibility/influentialness.

### 2.3.2. Open passage retrieval (optional)

Participants were also encouraged to extract supplementary relevant queries from the titles or content articles and to provide results based on these supplementary queries.

### 2.3.3. Output format

Results had to be provided in a TREC style JSON or TSV format with the following fields:

**run_id** Run ID starting with : team_id_task_id_method_used, e.g. UBO_task_1_TFIDF

**manual** Whether the run is manual {0,1}

<footnote n="3">³Judgments made last year on a 0-5 scale were transformed with the following conversion rules: 0/1 → 0; 2/3 → 1; 4/5 → 2.</footnote>
<caption>**Table 2**</caption>

CLEF 2023 SimpleText Task 1 on content selection: example of output

<table><thead><tr><th>Run</th><th>M/A</th><th>Topic</th><th>Query</th><th>Doc</th><th>Rel</th><th>Comb</th><th>Passage</th></tr></thead><tbody><tr><td>ST1_task1_1</td><td>0</td><td>G01</td><td>G01.1</td><td>1564531496</td><td>0.97</td><td>0.85</td><td>A CDA is a mobile user device, similar to a Personal Digital Assistant (PDA). It supports the citizen when dealing with public authorities and proves his rights - if desired, even without revealing his identity.</td></tr><tr><td>ST1_task1_1</td><td>0</td><td>G01</td><td>G01.1</td><td>3000234933</td><td>0.9</td><td>0.9</td><td>People are becoming increasingly comfortable using Digital Assistants (DAs) to interact with services or connected objects</td></tr><tr><td>ST1_task1_1</td><td>0</td><td>G01</td><td>G01.2</td><td>1448624402</td><td>0.6</td><td>0.3</td><td>As extensive experimental research has shown individuals suffer from diverse biases in decision-making.</td></tr></tbody></table>

**topic_id** Topic ID

**query_id** Query ID used to retrieve the document (if one of the queries provided for the topic was used; 0 otherwise)

**doc_id** ID of the retrieved document (to be extracted from the JSON output)

**rel_score** Relevance score of the passage (in the [0-1] scale)

**comb_score** General score that may combine relevance and other aspects: readability, citation measures...(in the [0-1] scale)

**passage** Text of the selected passage

For each query, the maximum number of distinct DBLP references (doc_id field) was 100 and the total length of passages could not exceed 1000 tokens. The idea of taking into account complexity is to have passages easier to understand for non-experts, while credibility score aims at guiding them on the expertise of authors and the value of publication w.r.t. the article topic. For example, complexity scores can be evaluated using readability score and credibility scores using bibliometrics.

Here is an output format example:

An example of the output is shown in Table 2. For each topic, the maximum number of distinct DBLP references (_id JSON field) was 100 and the total length of passages was not to exceed 1,000 tokens.
## 2.4. Evaluation Metrics

Passage relevance has been assessed based on:

* lexical and semantic overlap of extracted passages with topic article content

* manual relevance assessment of a pool of passages (relevance scores provided by participants will be used to measure ranking quality)

* manual assessment by non-expert users of credibility and complexity

# 3. Participants' approaches

Five teams submitted 39 runs in total.

Elsevier (represented as *Elsevier*\* in Table 7) [9] made 10 submissions to Task 1. Their submissions focused on evaluating the performance of neural rankers, utilizing both zero-shot approaches and unsupervised fine-tuning techniques on scientific documents.

The University of Amsterdam (*UAmS*\*) [10] entered 10 submissions for Task 1. Initially, they contributed three baseline rankers aimed at enhancing the pool of judgments. These baseline rankers included an ElasticSearch run utilizing keyword queries (non-phrase), as well as a cross-encoder reranking approach applied to the top 100 and top 1,000 results obtained from ElasticSearch. They made four additional submissions that focused on evaluating the credibility of the retrieved results. These submissions took into consideration factors such as the recency and number of citations for each paper to assess their credibility. Finally, they submitted three runs specifically aimed at addressing the readability of the retrieved results.

The University of Maine (AIIR Lab, *maine*\*) [11] submitted 5 runs for Task 1. Their submissions involved experimenting with cross-encoder and bi-encoder models, comparing their performance to lexical models.

The University of Milano Bicocca (*unimib_DoSSIER*\*) [12] submitted 2 runs for Task 1. Their submissions encompassed domain-specific approaches for scientific documents, including probabilistic lexical ranking, hierarchical document classification, and pseudo-relevance feedback (PRF).

# 4. Results

## 4.1. Retrieval Effectiveness

Table 3 shows the results of the CLEF 2023 Simpletext Task 1, based on the 34 test queries. The main measure of the task is NDCG@10, and the table is sorted on this measure for convenience. Let us note that some participants used the possibility of having two different scores in their run. Since ranking made according to the relevance or combination scores may vary in this case, we add in the result table *rel* and *comb* for runs with two different scores.

A number of observations stand out. First and foremost, we see in general that the top of the Table is dominated by neural rankers; in particular, cross-encoders trained on MSMarco applied in a zero-shot way (or variants thereof), perform well for ranking scientific abstracts on NDCG@10 and other early precision measures. Traditional lexical retrieval models perform
reasonably but at some distance from the top-scoring runs, with the neural runs typically
re-ranking such a lexical baseline run.

Second, looking at more recall-oriented measures, such as MAP and bpref, the picture is more mixed. This is indicating some approaches privilege precision over recall, whereas other approaches seem to promote all recall levels.

Third, some submissions aimed to balance the topical relevance with the readability or credi-
bility of the results. We observe that these runs still achieve competitive retrieval effectiveness,
despite removing or down-ranking highly relevant abstracts that have for example a high text
complexity or are dated with low numbers of citations.

The document collection contains two different sets of topics. On the one hand, the Guardian topics (G) are built from articles related to societal issues: privacy, ubiquity, misinformation, etc. and are usually associated with general queries that must be disambiguated in the context of the articles. On the other hand, the Tech Xplore topics (T) are linked to an original scientific paper and deal with more technical facets: neural networks, indoor positioning system, RISC architecture, etc. Further analysis was performed on the behavior of systems according to these two sets and shown in Tables 4 and 5. A comparison of these results exhibits that more relevant abstracts were found for the T topics, with higher scores overall. However, ranking against NDCG@10 leads to a different ranking of systems. While ElsevierSimpleText_run8 still outperforms other systems on T topics, maine_CrossEncoder1 becomes the 1st system on G topics.

Going back to the training qrels released to participants, we observe as expected that super-
vised models learned on these data have the highest scores (Table 6). Runs submitted by the
University of Maine and to a lesser extent by the University of Amsterdam outperform others.
Runs by Elsevier also resorted to neural rerankers, but other training data were used. Only
G topics were included in the train qrels, which may explain why systems behave differently
between G or T topics used in the test qrels.

## 4.2. Analysis of Readability

Table 7 shows several statistics over to the top 10 results retrieved for the entire topic set for Task 1:

* citation analysis (impact factor based on ACM records and average number of references per document),

• textual analysis (document length and FKGL scores).

Let us note that when two different scores were provided for a run, only the combined one was
considered in this evaluation.

We make a number of observations.

First, it appears that the most effective ranking models tend to retrieve abstracts that are
not only longer, but also exhibit greater length variability. These retrieved abstracts often
have higher impact factors and extensive bibliographies. There also seems to be a discernible
difference between the lengths of abstracts retrieved by lexical-based systems compared to
those retrieved by neural-based systems.
<table><caption>**Table 3**
Evaluation of SimpleText Task 1 (Test qrels: G16-G20+T01-T05).</caption><thead><tr><th rowspan="2">Run</th><th rowspan="2">MRR</th><th colspan="2">Precision</th><th colspan="2">NDCG</th><th rowspan="2">Bpref</th><th rowspan="2">MAP</th></tr><tr><th>10</th><th>20</th><th>10</th><th>20</th></tr></thead><tbody><tr><td>ElsevierSimpleText_run8</td><td>0.8082</td><td>0.5618</td><td>0.3515</td><td>0.5881</td><td>0.4422</td><td>0.2371</td><td>0.1633</td></tr><tr><td>ElsevierSimpleText_run7</td><td>0.7136</td><td>0.5618</td><td>0.4103</td><td>0.5704</td><td>0.4627</td><td>0.2626</td><td>0.1915</td></tr><tr><td>maine_CrossEncoder1<sup>rel</sup></td><td>0.8106</td><td>0.5382</td><td>0.4456</td><td>0.5675</td><td>0.4908</td><td>0.3317</td><td>0.2810</td></tr><tr><td>maine_CrossEncoderFinetuned1<sup>rel</sup></td><td>0.7691</td><td>0.5559</td><td>0.4441</td><td>0.5542</td><td>0.4840</td><td>0.3433</td><td>0.2572</td></tr><tr><td>maine_CrossEncoder1<sup>comb</sup></td><td>0.7309</td><td>0.5265</td><td>0.4500</td><td>0.5455</td><td>0.4841</td><td>0.3337</td><td>0.2754</td></tr><tr><td>maine_CrossEncoderFinetuned1<sup>comb</sup></td><td>0.7338</td><td>0.4971</td><td>0.4000</td><td>0.4859</td><td>0.4295</td><td>0.3443</td><td>0.2385</td></tr><tr><td>ElsevierSimpleText_run5</td><td>0.6600</td><td>0.4765</td><td>0.3838</td><td>0.4826</td><td>0.4186</td><td>0.2542</td><td>0.1828</td></tr><tr><td>maine_CrossEncoderFinetuned2<sup>rel</sup></td><td>0.6588</td><td>0.4971</td><td>0.4088</td><td>0.4821</td><td>0.4254</td><td>0.3185</td><td>0.2242</td></tr><tr><td>ElsevierSimpleText_run2</td><td>0.7010</td><td>0.4676</td><td>0.4059</td><td>0.4791</td><td>0.4282</td><td>0.2528</td><td>0.1942</td></tr><tr><td>UAmS_CE100<sup>rel</sup></td><td>0.7050</td><td>0.4912</td><td>0.4044</td><td>0.4782</td><td>0.4236</td><td>0.2616</td><td>0.2011</td></tr><tr><td>ElsevierSimpleText_run6</td><td>0.6402</td><td>0.4676</td><td>0.3853</td><td>0.4723</td><td>0.4185</td><td>0.2557</td><td>0.1809</td></tr><tr><td>ElsevierSimpleText_run4</td><td>0.6774</td><td>0.4529</td><td>0.3794</td><td>0.4721</td><td>0.4116</td><td>0.2485</td><td>0.1898</td></tr><tr><td>ElsevierSimpleText_run9</td><td>0.5933</td><td>0.4735</td><td>0.3176</td><td>0.4655</td><td>0.3595</td><td>0.1758</td><td>0.1238</td></tr><tr><td>ElsevierSimpleText_run1</td><td>0.6821</td><td>0.4588</td><td>0.3824</td><td>0.4626</td><td>0.4071</td><td>0.2573</td><td>0.1823</td></tr><tr><td>maine_CrossEncoderFinetuned2<sup>comb</sup></td><td>0.7082</td><td>0.4706</td><td>0.3926</td><td>0.4617</td><td>0.4089</td><td>0.3259</td><td>0.2253</td></tr><tr><td>UAmS_CE1k_F Filter</td><td>0.6403</td><td>0.4765</td><td>0.3559</td><td>0.4533</td><td>0.3743</td><td>0.2727</td><td>0.1936</td></tr><tr><td>ElsevierSimpleText_run3</td><td>0.6502</td><td>0.4471</td><td>0.3779</td><td>0.4460</td><td>0.3994</td><td>0.2558</td><td>0.1785</td></tr><tr><td>UAmS_CE1k<sup>rel</sup></td><td>0.6329</td><td>0.4735</td><td>0.4044</td><td>0.4448</td><td>0.4049</td><td>0.2797</td><td>0.2051</td></tr><tr><td>maine_PI2TFIDF<sup>rel</sup></td><td>0.5791</td><td>0.4382</td><td>0.2853</td><td>0.4212</td><td>0.3313</td><td>0.2159</td><td>0.1410</td></tr><tr><td>UAmShaltic</td><td>0.6888</td><td>0.4324</td><td>0.3338</td><td>0.4103</td><td>0.3499</td><td>0.2395</td><td>0.1719</td></tr><tr><td>UAmS_CE100<sup>comb</sup></td><td>0.6779</td><td>0.3971</td><td>0.3456</td><td>0.4016</td><td>0.3642</td><td>0.2658</td><td>0.1792</td></tr><tr><td>maine_PI2TFIDF<sup>comb</sup></td><td>0.5626</td><td>0.4176</td><td>0.2809</td><td>0.4014</td><td>0.3218</td><td>0.2155</td><td>0.1364</td></tr><tr><td>UAmS_Elastic</td><td>0.6424</td><td>0.4059</td><td>0.3456</td><td>0.3910</td><td>0.3541</td><td>0.2501</td><td>0.1895</td></tr><tr><td>UAmShaltic</td><td>0.6429</td><td>0.4088</td><td>0.3382</td><td>0.3883</td><td>0.3468</td><td>0.2454</td><td>0.1833</td></tr><tr><td>UAmSCE1k<sup>comb</sup></td><td>0.6625</td><td>0.3971</td><td>0.3147</td><td>0.3723</td><td>0.3282</td><td>0.2123</td><td>0.1403</td></tr><tr><td>UAmSCE1k<sup>comb</sup></td><td>0.5880</td><td>0.4147</td><td>0.3515</td><td>0.3706</td><td>0.3398</td><td>0.2700</td><td>0.1865</td></tr><tr><td>UAmSCE1k<sup>comb</sup></td><td>0.5880</td><td>0.4147</td><td>0.3515</td><td>0.3706</td><td>0.3398</td><td>0.2700</td><td>0.1865</td></tr><tr><td>UAmS_EIF<sup>Read</sup>25</td><td>0.6076</td><td>0.3735</td><td>0.3074</td><td>0.3539</td><td>0.3190</td><td>0.2194</td><td>0.1522</td></tr><tr><td>UAmS_EIF<sup>Cred</sup>53Read</td><td>0.6088</td><td>0.3676</td><td>0.3059</td><td>0.3469</td><td>0.3153</td><td>0.2133</td><td>0.1456</td></tr><tr><td>maine_tripletloss<sup>rel</sup></td><td>0.5425</td><td>0.3500</td><td>0.2162</td><td>0.3439</td><td>0.2557</td><td>0.1296</td><td>0.0690</td></tr><tr><td>maine_tripletloss<sup>comb</sup></td><td>0.5502</td><td>0.3382</td><td>0.2176</td><td>0.3353</td><td>0.2561</td><td>0.1335</td><td>0.0696</td></tr><tr><td>unimib<sup>DoSSIER</sup>2</td><td>0.5201</td><td>0.2853</td><td>0.2515</td><td>0.2980</td><td>0.2683</td><td>0.1898</td><td>0.1141</td></tr><tr><td>unimib<sup>DoSSIER</sup>4</td><td>0.5202</td><td>0.2853</td><td>0.2441</td><td>0.2972</td><td>0.2632</td><td>0.1873</td><td>0.1111</td></tr></tbody></table>

Second, in terms of readability levels, the overwhelming majority of systems retrieve abstracts with an FKGL of around 14 — corresponding to university-level texts. This is entirely as expected since the corpus is based on scientific text, known to be written for experts with higher text complexity than for example newspaper articles.

Third, two systems retrieve abstracts with an FKGL of 11-12 — corresponding to the exit level of compulsory education, and the reading level of the average newspaper reader targeted by the use case of the track. These runs still achieved very reasonable retrieval effectiveness
<table><caption>**Table 4**
Evaluation of SimpleText Task 1 (Test G-only qrels).</caption><thead><tr><th rowspan="2">Run</th><th rowspan="2">MRR</th><th colspan="2">Precision</th><th colspan="2">NDCG</th><th rowspan="2">Bpref</th><th rowspan="2">MAP</th></tr><tr><th>10</th><th>20</th><th>10</th><th>20</th></tr></thead><tbody><tr><td>ElsevierSimpleText_run1</td><td>0.5602</td><td>0.2941</td><td>0.2441</td><td>0.3027</td><td>0.2680</td><td>0.1616</td><td>0.1155</td></tr><tr><td>ElsevierSimpleText_run2</td><td>0.5882</td><td>0.2765</td><td>0.2559</td><td>0.2942</td><td>0.2755</td><td>0.1622</td><td>0.1170</td></tr><tr><td>ElsevierSimpleText_run3</td><td>0.5387</td><td>0.2882</td><td>0.2471</td><td>0.2945</td><td>0.2681</td><td>0.1625</td><td>0.1159</td></tr><tr><td>ElsevierSimpleText_run4</td><td>0.5460</td><td>0.3000</td><td>0.2294</td><td>0.3186</td><td>0.2694</td><td>0.1614</td><td>0.1207</td></tr><tr><td>ElsevierSimpleText_run5</td><td>0.5145</td><td>0.3412</td><td>0.2559</td><td>0.3369</td><td>0.2896</td><td>0.1722</td><td>0.1222</td></tr><tr><td>ElsevierSimpleText_run6</td><td>0.5157</td><td>0.3412</td><td>0.2676</td><td>0.3423</td><td>0.3027</td><td>0.1733</td><td>0.1261</td></tr><tr><td>ElsevierSimpleText_run7</td><td>0.4958</td><td>0.3765</td><td>0.2588</td><td>0.3699</td><td>0.2952</td><td>0.1730</td><td>0.1222</td></tr><tr><td>ElsevierSimpleText_run8</td><td>0.6850</td><td>0.3706</td><td>0.2235</td><td>0.3948</td><td>0.3002</td><td>0.1672</td><td>0.1132</td></tr><tr><td>ElsevierSimpleText_run9</td><td>0.4102</td><td>0.3235</td><td>0.2118</td><td>0.2990</td><td>0.2366</td><td>0.1219</td><td>0.0816</td></tr><tr><td>UAmS_CE100<sup>rel</sup></td><td>0.6041</td><td>0.3353</td><td>0.2794</td><td>0.3185</td><td>0.2920</td><td>0.1863</td><td>0.1335</td></tr><tr><td>UAmS_CE100<sup>comb</sup></td><td>0.6817</td><td>0.3647</td><td>0.2765</td><td>0.3724</td><td>0.3136</td><td>0.2030</td><td>0.1482</td></tr><tr><td>UAmS_CE1k<sup>rel</sup></td><td>0.5104</td><td>0.3412</td><td>0.2824</td><td>0.3128</td><td>0.2851</td><td>0.1879</td><td>0.1280</td></tr><tr><td>UAmS_CE1k_Combine<sup>comb</sup></td><td>0.5796</td><td>0.3294</td><td>0.2853</td><td>0.3179</td><td>0.2933</td><td>0.1951</td><td>0.1455</td></tr><tr><td>UAmS_CE1k_F Filter<sup>comb</sup></td><td>0.4976</td><td>0.3471</td><td>0.2471</td><td>0.3294</td><td>0.2688</td><td>0.1906</td><td>0.1311</td></tr><tr><td>UAmSimm_EIF_Cred44</td><td>0.5899</td><td>0.3059</td><td>0.2265</td><td>0.2823</td><td>0.2482</td><td>0.1858</td><td>0.1276</td></tr><tr><td>UAmSimm_EIF_Cred44Read</td><td>0.6299</td><td>0.2882</td><td>0.2088</td><td>0.2666</td><td>0.2388</td><td>0.1703</td><td>0.1106</td></tr><tr><td>UAmSimm_EIF_Cred53</td><td>0.5373</td><td>0.2941</td><td>0.2235</td><td>0.2685</td><td>0.2453</td><td>0.1903</td><td>0.1376</td></tr><tr><td>UAmSimm_EIF_Cred53Read</td><td>0.5801</td><td>0.2588</td><td>0.1971</td><td>0.2469</td><td>0.2269</td><td>0.1725</td><td>0.1148</td></tr><tr><td>UAmSimm_Read25</td><td>0.5801</td><td>0.2647</td><td>0.2029</td><td>0.2555</td><td>0.2356</td><td>0.1799</td><td>0.1220</td></tr><tr><td>UAmS_Elastic</td><td>0.5373</td><td>0.2941</td><td>0.2294</td><td>0.2724</td><td>0.2519</td><td>0.1954</td><td>0.1432</td></tr><tr><td>maine_CrossEncoder1<sup>rel</sup></td><td>0.6947</td><td>0.4353</td><td>0.3529</td><td>0.4483</td><td>0.3973</td><td>0.2983</td><td>0.2441</td></tr><tr><td>maine_CrossEncoder1<sup>comb</sup></td><td>0.5892</td><td>0.4176</td><td>0.3647</td><td>0.4185</td><td>0.3930</td><td>0.3020</td><td>0.2390</td></tr><tr><td>maine_CrossEncoderFinetuned1<sup>rel</sup></td><td>0.7471</td><td>0.4471</td><td>0.3529</td><td>0.4448</td><td>0.3872</td><td>0.3019</td><td>0.2218</td></tr><tr><td>maine_CrossEncoderFinetuned1<sup>comb</sup></td><td>0.7765</td><td>0.4235</td><td>0.3353</td><td>0.4263</td><td>0.3848</td><td>0.3031</td><td>0.2068</td></tr><tr><td>maine_CrossEncoderFinetuned2<sup>rel</sup></td><td>0.5738</td><td>0.3882</td><td>0.3324</td><td>0.3582</td><td>0.3274</td><td>0.2659</td><td>0.1722</td></tr><tr><td>maine_CrossEncoderFinetuned2<sup>comb</sup></td><td>0.6959</td><td>0.3941</td><td>0.3353</td><td>0.3868</td><td>0.3542</td><td>0.2749</td><td>0.1842</td></tr><tr><td>maine_Pi2TFIDF<sup>rel</sup></td><td>0.5628</td><td>0.3882</td><td>0.2235</td><td>0.3318</td><td>0.2731</td><td>0.2235</td><td>0.1492</td></tr><tr><td>maine_Pi2TFIDF<sup>comb</sup></td><td>0.5216</td><td>0.3647</td><td>0.2265</td><td>0.3283</td><td>0.2690</td><td>0.2254</td><td>0.1477</td></tr><tr><td>maine_tripletloss<sup>rel</sup></td><td>0.4475</td><td>0.3471</td><td>0.1882</td><td>0.3069</td><td>0.2213</td><td>0.1279</td><td>0.0731</td></tr><tr><td>maine_tripletloss<sup>comb</sup></td><td>0.4433</td><td>0.3294</td><td>0.1882</td><td>0.2960</td><td>0.2196</td><td>0.1352</td><td>0.0745</td></tr><tr><td>uninib_DoSSIER_2</td><td>0.4349</td><td>0.1588</td><td>0.1353</td><td>0.1693</td><td>0.1534</td><td>0.1397</td><td>0.0601</td></tr><tr><td>uninib_DoSSIER_4</td><td>0.4351</td><td>0.1529</td><td>0.1324</td><td>0.1657</td><td>0.1514</td><td>0.1379</td><td>0.0584</td></tr></tbody></table>

(NDCG@10 0.37-0.45 in Table 3) while only retrieving abstracts with the desirable readability level.

## 5. Conclusion

In this CLEF lab track, a range of language models has been systematically examined and compared by participants in an attempt to elucidate their strengths and weaknesses when employed for accessing scientific information, especially amid the burgeoning science communication
<table><caption>**Table 5**

Evaluation of SimpleText Task 1 (Test T-only qrels).</caption><thead><tr><th rowspan="2">Run</th><th rowspan="2">MRR</th><th colspan="2">Precision</th><th colspan="2">NDCG</th><th rowspan="2">Bpref</th><th rowspan="2">MAP</th></tr><tr><th>10</th><th>20</th><th>10</th><th>20</th></tr></thead><tbody><tr><td>ElsevierSimpleText_run1</td><td>0.8039</td><td>0.6235</td><td>0.5206</td><td>0.6224</td><td>0.5462</td><td>0.3530</td><td>0.2491</td></tr><tr><td>ElsevierSimpleText_run2</td><td>0.8137</td><td>0.6588</td><td>0.5559</td><td>0.6641</td><td>0.5809</td><td>0.3434</td><td>0.2713</td></tr><tr><td>ElsevierSimpleText_run3</td><td>0.7618</td><td>0.6059</td><td>0.5088</td><td>0.5975</td><td>0.5308</td><td>0.3492</td><td>0.2411</td></tr><tr><td>ElsevierSimpleText_run4</td><td>0.8088</td><td>0.6059</td><td>0.5294</td><td>0.6257</td><td>0.5538</td><td>0.3357</td><td>0.2588</td></tr><tr><td>ElsevierSimpleText_run5</td><td>0.8056</td><td>0.6118</td><td>0.5118</td><td>0.6283</td><td>0.5477</td><td>0.3363</td><td>0.2435</td></tr><tr><td>ElsevierSimpleText_run6</td><td>0.7647</td><td>0.5941</td><td>0.5029</td><td>0.6024</td><td>0.5342</td><td>0.3380</td><td>0.2358</td></tr><tr><td>ElsevierSimpleText_run7</td><td>0.9314</td><td>0.7471</td><td>0.5618</td><td>0.7709</td><td>0.6302</td><td>0.3522</td><td>0.2609</td></tr><tr><td>ElsevierSimpleText_run8</td><td>0.9314</td><td>0.7529</td><td>0.4794</td><td>0.7809</td><td>0.5838</td><td>0.3071</td><td>0.2134</td></tr><tr><td>ElsevierSimpleText_run9</td><td>0.7764</td><td>0.6235</td><td>0.4235</td><td>0.6319</td><td>0.4823</td><td>0.2297</td><td>0.1659</td></tr><tr><td>UAmS_CE100<sup>rel</sup></td><td>0.6041</td><td>0.6471</td><td>0.5294</td><td>0.6379</td><td>0.5552</td><td>0.3369</td><td>0.2687</td></tr><tr><td>UAmS_CE100<sup>comb</sup></td><td>0.6740</td><td>0.4294</td><td>0.4147</td><td>0.4309</td><td>0.4148</td><td>0.3287</td><td>0.2101</td></tr><tr><td>UAmS_CE1k<sup>rel</sup></td><td>0.5104</td><td>0.6059</td><td>0.5265</td><td>0.5768</td><td>0.5247</td><td>0.3716</td><td>0.2822</td></tr><tr><td>UAmS_CE1k_Combine<sup>comb</sup></td><td>0.5965</td><td>0.5000</td><td>0.4176</td><td>0.4232</td><td>0.3862</td><td>0.3448</td><td>0.2276</td></tr><tr><td>UAmS_CE1k_FILTER<sup>comb</sup></td><td>0.7829</td><td>0.6059</td><td>0.4647</td><td>0.5773</td><td>0.4798</td><td>0.3548</td><td>0.2561</td></tr><tr><td>UAmSICIFF_Cred44</td><td>0.5899</td><td>0.5588</td><td>0.4412</td><td>0.5382</td><td>0.4516</td><td>0.2932</td><td>0.2161</td></tr><tr><td>UAmSICIFF_Cred44Read</td><td>0.6299</td><td>0.5059</td><td>0.4206</td><td>0.4780</td><td>0.4176</td><td>0.2542</td><td>0.1700</td></tr><tr><td>UAmSICIFF_Cred53</td><td>0.5373</td><td>0.5235</td><td>0.4529</td><td>0.5081</td><td>0.4483</td><td>0.3005</td><td>0.2289</td></tr><tr><td>UAmSICIFF_Cred53Read</td><td>0.5801</td><td>0.4765</td><td>0.4147</td><td>0.4468</td><td>0.4037</td><td>0.2541</td><td>0.1764</td></tr><tr><td>UAmSICIFF_READ25</td><td>0.5801</td><td>0.4824</td><td>0.4118</td><td>0.4523</td><td>0.4023</td><td>0.2589</td><td>0.1824</td></tr><tr><td>UAmS_Elastic</td><td>0.5373</td><td>0.5176</td><td>0.4618</td><td>0.5098</td><td>0.4565</td><td>0.3050</td><td>0.2358</td></tr><tr><td>maine_CrossEncoder1<sup>rel</sup></td><td>0.9265</td><td>0.6412</td><td>0.5382</td><td>0.6867</td><td>0.5842</td><td>0.3651</td><td>0.3180</td></tr><tr><td>maine_CrossEncoder1<sup>comb</sup></td><td>0.8725</td><td>0.6353</td><td>0.5353</td><td>0.6726</td><td>0.5752</td><td>0.3654</td><td>0.3119</td></tr><tr><td>maine_CrossEncoderFinetuned1<sup>rel</sup></td><td>0.7912</td><td>0.6647</td><td>0.5559</td><td>0.6636</td><td>0.5809</td><td>0.3848</td><td>0.2926</td></tr><tr><td>maine_CrossEncoderFinetuned1<sup>comb</sup></td><td>0.6912</td><td>0.5706</td><td>0.4647</td><td>0.5455</td><td>0.4742</td><td>0.3854</td><td>0.2703</td></tr><tr><td>maine_CrossEncoderFinetuned2<sup>rel</sup></td><td>0.7437</td><td>0.6059</td><td>0.5000</td><td>0.6059</td><td>0.5234</td><td>0.3710</td><td>0.2761</td></tr><tr><td>maine_CrossEncoderFinetuned2<sup>comb</sup></td><td>0.7206</td><td>0.5471</td><td>0.4500</td><td>0.5366</td><td>0.4636</td><td>0.3769</td><td>0.2663</td></tr><tr><td>maine_P12TFIDF<sup>rel</sup></td><td>0.5955</td><td>0.5118</td><td>0.3471</td><td>0.5106</td><td>0.3895</td><td>0.2084</td><td>0.1327</td></tr><tr><td>maine_P12TFIDF<sup>comb</sup></td><td>0.6036</td><td>0.4706</td><td>0.3353</td><td>0.4745</td><td>0.3746</td><td>0.2057</td><td>0.1251</td></tr><tr><td>maine_tripletloss<sup>rel</sup></td><td>0.6374</td><td>0.3529</td><td>0.2441</td><td>0.3809</td><td>0.2900</td><td>0.1313</td><td>0.0649</td></tr><tr><td>maine_tripletloss<sup>comb</sup></td><td>0.6572</td><td>0.3471</td><td>0.2471</td><td>0.3746</td><td>0.2926</td><td>0.1317</td><td>0.0646</td></tr><tr><td>uninib_DoSSIER_2</td><td>0.6053</td><td>0.4118</td><td>0.3676</td><td>0.4267</td><td>0.3832</td><td>0.2399</td><td>0.1681</td></tr><tr><td>uninib_DoSSIER_4</td><td>0.6053</td><td>0.4176</td><td>0.3559</td><td>0.4288</td><td>0.3750</td><td>0.2367</td><td>0.1639</td></tr></tbody></table>

landscape.

Task lab results involve the construction of a unique test set that pairs layperson queries with ideal scientific documents. The queries have been derived from a large-scale student experiment, providing a realistic representation of the nature of layperson queries. The test set has been used to train various participating systems, and the results have been enhanced with expert manual annotation of an additional pool of results.

This comprehensive exploration provides valuable insights into the effectiveness of each model and presents a critical comparison of their performances. The findings from this study
<table><caption>**Table 6**

Evaluation of SimpleText Task 1 (Train qrels: G01-G15).</caption><thead><tr><th rowspan="2">Run</th><th rowspan="2">MRR</th><th colspan="2">Precision</th><th colspan="2">NDCG</th><th rowspan="2">Bpref</th><th rowspan="2">MAP</th></tr><tr><th>10</th><th>20</th><th>10</th><th>20</th></tr></thead><tbody><tr><td>ElsevierSimpleText_run1</td><td>0.5294</td><td>0.2207</td><td>0.1931</td><td>0.2199</td><td>0.2296</td><td>0.3313</td><td>0.1815</td></tr><tr><td>ElsevierSimpleText_run2</td><td>0.5234</td><td>0.2621</td><td>0.2655</td><td>0.2366</td><td>0.2832</td><td>0.3952</td><td>0.2773</td></tr><tr><td>ElsevierSimpleText_run3</td><td>0.4897</td><td>0.2207</td><td>0.1845</td><td>0.2126</td><td>0.2177</td><td>0.3137</td><td>0.1694</td></tr><tr><td>ElsevierSimpleText_run4</td><td>0.4447</td><td>0.2655</td><td>0.2534</td><td>0.2437</td><td>0.2776</td><td>0.3894</td><td>0.2781</td></tr><tr><td>ElsevierSimpleText_run5</td><td>0.4156</td><td>0.2103</td><td>0.1862</td><td>0.2097</td><td>0.2277</td><td>0.3305</td><td>0.1742</td></tr><tr><td>ElsevierSimpleText_run6</td><td>0.3989</td><td>0.2000</td><td>0.1707</td><td>0.2000</td><td>0.2098</td><td>0.3091</td><td>0.1642</td></tr><tr><td>ElsevierSimpleText_run7</td><td>0.3512</td><td>0.2241</td><td>0.1828</td><td>0.1871</td><td>0.1986</td><td>0.3725</td><td>0.1498</td></tr><tr><td>ElsevierSimpleText_run8</td><td>0.2691</td><td>0.1828</td><td>0.1500</td><td>0.1526</td><td>0.1673</td><td>0.3585</td><td>0.1281</td></tr><tr><td>ElsevierSimpleText_run9</td><td>0.3238</td><td>0.1448</td><td>0.1241</td><td>0.1310</td><td>0.1319</td><td>0.1434</td><td>0.0906</td></tr><tr><td>UAmS_CE100<sup>rel</sup></td><td>0.5252</td><td>0.3034</td><td>0.2690</td><td>0.2947</td><td>0.3145</td><td>0.4012</td><td>0.3033</td></tr><tr><td>UAmS_CE100<sup>comb</sup></td><td>0.4371</td><td>0.3069</td><td>0.2466</td><td>0.2489</td><td>0.2795</td><td>0.3998</td><td>0.2838</td></tr><tr><td>UAmS_CE1k<sup>rel</sup></td><td>0.4608</td><td>0.2379</td><td>0.1948</td><td>0.2307</td><td>0.2421</td><td>0.3335</td><td>0.2001</td></tr><tr><td>UAmS_CE1k_Combine<sup>comb</sup></td><td>0.3182</td><td>0.1966</td><td>0.1897</td><td>0.1633</td><td>0.2005</td><td>0.3211</td><td>0.1714</td></tr><tr><td>UAmS_CE1k_F Filter<sup>comb</sup></td><td>0.4952</td><td>0.2414</td><td>0.1879</td><td>0.2431</td><td>0.2423</td><td>0.3249</td><td>0.1934</td></tr><tr><td>UAmS smCred44</td><td>0.6375</td><td>0.3655</td><td>0.3086</td><td>0.3658</td><td>0.3696</td><td>0.3531</td><td>0.3405</td></tr><tr><td>UAmS smCred44Read</td><td>0.5746</td><td>0.3379</td><td>0.2845</td><td>0.3079</td><td>0.3145</td><td>0.2724</td><td>0.2294</td></tr><tr><td>UAmS smCred53</td><td>0.5682</td><td>0.3759</td><td>0.3414</td><td>0.3729</td><td>0.4029</td><td>0.4284</td><td>0.4050</td></tr><tr><td>UAmS smCred53Read</td><td>0.5312</td><td>0.3379</td><td>0.3103</td><td>0.3053</td><td>0.3323</td><td>0.3089</td><td>0.2706</td></tr><tr><td>UAmS smRead25</td><td>0.5257</td><td>0.3310</td><td>0.3086</td><td>0.2970</td><td>0.3264</td><td>0.3082</td><td>0.2741</td></tr><tr><td>UAmS Elastic</td><td>0.5605</td><td>0.3655</td><td>0.3345</td><td>0.3627</td><td>0.3924</td><td>0.4226</td><td>0.4072</td></tr><tr><td>maineCrossEncoder1<sup>rel</sup></td><td>0.7102</td><td>0.4448</td><td>0.4086</td><td>0.4604</td><td>0.5017</td><td>0.4629</td><td>0.5064</td></tr><tr><td>maine_CrossEncoder1<sup>comb</sup></td><td>0.7165</td><td>0.4414</td><td>0.4155</td><td>0.4597</td><td>0.5084</td><td>0.4619</td><td>0.5023</td></tr><tr><td>maine_CrossEncoderFinetuned1<sup>rel</sup></td><td>0.9418</td><td>0.7517</td><td>0.6086</td><td>0.6861</td><td>0.7272</td><td>0.8730</td><td>0.7821</td></tr><tr><td>maine_CrossEncoderFinetuned1<sup>comb</sup></td><td>0.7959</td><td>0.6483</td><td>0.5552</td><td>0.5759</td><td>0.6324</td><td>0.8778</td><td>0.6645</td></tr><tr><td>maine_CrossEncoderFinetuned2<sup>rel</sup></td><td>0.8230</td><td>0.5207</td><td>0.4328</td><td>0.4883</td><td>0.5186</td><td>0.7296</td><td>0.5109</td></tr><tr><td>maine_CrossEncoderFinetuned2<sup>comb</sup></td><td>0.8346</td><td>0.5069</td><td>0.4379</td><td>0.4888</td><td>0.5270</td><td>0.7230</td><td>0.5042</td></tr><tr><td>maine_P12TFIDF<sup>rel</sup></td><td>0.2022</td><td>0.1379</td><td>0.1069</td><td>0.1146</td><td>0.1201</td><td>0.2125</td><td>0.0868</td></tr><tr><td>maine_P12TFIDF<sup>comb</sup></td><td>0.2191</td><td>0.1310</td><td>0.1069</td><td>0.1173</td><td>0.1226</td><td>0.2166</td><td>0.0891</td></tr><tr><td>maine_tripletloss<sup>rel</sup></td><td>0.5966</td><td>0.3793</td><td>0.2948</td><td>0.3461</td><td>0.3659</td><td>0.6041</td><td>0.3332</td></tr><tr><td>maine_tripletloss<sup>comb</sup></td><td>0.5629</td><td>0.3690</td><td>0.2966</td><td>0.3300</td><td>0.3611</td><td>0.6041</td><td>0.3292</td></tr><tr><td>uninib DoSSIER_2</td><td>0.4802</td><td>0.2310</td><td>0.2086</td><td>0.2492</td><td>0.2625</td><td>0.2568</td><td>0.2449</td></tr><tr><td>uninib DoSSIER_4</td><td>0.4462</td><td>0.2241</td><td>0.2069</td><td>0.2451</td><td>0.2596</td><td>0.2514</td><td>0.2384</td></tr></tbody></table>

have the potential to inform future research directions and aid the development of more user-friendly AI tools, leading to more accurate and effective retrieval of scientific literature for laypeople.

The comprehensive analysis of the CLEF 2023 SimpleText track leads to the overall conclusion that state-of-the-art models have made significant progress. However, it is evident that there is still substantial room for improvement in the field. This indicates that further advancements and refinements are necessary to enhance the performance and capabilities of the models.
<table><caption>**Table 7**
Text Analysis of SimpleText Task 1 output.</caption><thead><tr><th rowspan="2">Run</th><th rowspan="2">Impact</th><th rowspan="2">#Refs</th><th colspan="2">Length</th><th colspan="2">FKGL</th></tr><tr><th>Mean</th><th>Median</th><th>Mean</th><th>Median</th></tr></thead><tbody><tr><td>ElsevierSimpleText_run1</td><td>1.88</td><td>0.95</td><td>965.02</td><td>921.00</td><td>13.80</td><td>13.80</td></tr><tr><td>ElsevierSimpleText_run2</td><td>2.24</td><td>1.36</td><td>1017.57</td><td>981.00</td><td>13.98</td><td>13.90</td></tr><tr><td>ElsevierSimpleText_run3</td><td>1.80</td><td>0.94</td><td>951.64</td><td>912.00</td><td>13.71</td><td>13.75</td></tr><tr><td>ElsevierSimpleText_run4</td><td>2.10</td><td>1.21</td><td>1011.10</td><td>994.00</td><td>13.95</td><td>13.90</td></tr><tr><td>ElsevierSimpleText_run5</td><td>1.78</td><td>0.71</td><td>993.14</td><td>972.50</td><td>13.76</td><td>13.80</td></tr><tr><td>ElsevierSimpleText_run6</td><td>1.59</td><td>0.65</td><td>995.65</td><td>975.50</td><td>13.75</td><td>13.90</td></tr><tr><td>ElsevierSimpleText_run7</td><td>2.37</td><td>0.94</td><td>1101.23</td><td>1075.50</td><td>13.87</td><td>13.80</td></tr><tr><td>ElsevierSimpleText_run8</td><td>0.60</td><td>0.50</td><td>1089.90</td><td>1045.00</td><td>14.09</td><td>14.00</td></tr><tr><td>ElsevierSimpleText_run9</td><td>0.71</td><td>0.54</td><td>1016.96</td><td>991.00</td><td>13.66</td><td>13.70</td></tr><tr><td>UAmS_CE100</td><td>3.20</td><td>1.64</td><td>1028.78</td><td>975.00</td><td>14.59</td><td>14.50</td></tr><tr><td>UAmS_CE1k</td><td>2.41</td><td>1.24</td><td>1071.67</td><td>985.50</td><td>14.70</td><td>14.60</td></tr><tr><td>UAmS_CE1k_Combine</td><td>0.84</td><td>0.49</td><td>924.38</td><td>839.00</td><td>10.84</td><td>11.20</td></tr><tr><td>UAmS_CE1k_F Filter</td><td>1.09</td><td>0.62</td><td>988.00</td><td>913.50</td><td>12.40</td><td>12.70</td></tr><tr><td>UAmSicheCred44</td><td>3.32</td><td>1.62</td><td>973.03</td><td>970.50</td><td>13.60</td><td>14.50</td></tr><tr><td>UAmS_EIF_Cred44Read</td><td>1.85</td><td>1.34</td><td>799.29</td><td>851.00</td><td>13.18</td><td>14.20</td></tr><tr><td>UAmS_EIF_Cred53</td><td>2.89</td><td>1.49</td><td>938.41</td><td>932.00</td><td>13.73</td><td>14.40</td></tr><tr><td>UAmS_EIF_Cred53Read</td><td>1.70</td><td>1.28</td><td>774.76</td><td>823.00</td><td>13.29</td><td>14.30</td></tr><tr><td>UAmS_EIFRead25</td><td>1.60</td><td>1.25</td><td>767.70</td><td>819.00</td><td>13.09</td><td>14.20</td></tr><tr><td>UAmSElastic</td><td>2.84</td><td>1.45</td><td>922.36</td><td>917.00</td><td>13.49</td><td>14.30</td></tr><tr><td>maine_CrossEncoder1</td><td>4.22</td><td>2.86</td><td>961.17</td><td>923.00</td><td>14.64</td><td>14.60</td></tr><tr><td>maine_CrossEncoderFinetuned1</td><td>4.41</td><td>3.37</td><td>1003.75</td><td>988.00</td><td>15.01</td><td>14.80</td></tr><tr><td>maine_CrossEncoderFinetuned2</td><td>3.49</td><td>3.04</td><td>988.86</td><td>951.50</td><td>14.95</td><td>14.80</td></tr><tr><td>mainePI2TFIDF</td><td>3.35</td><td>2.58</td><td>893.29</td><td>894.00</td><td>14.03</td><td>14.00</td></tr><tr><td>maine_tripletloss</td><td>4.76</td><td>3.29</td><td>969.09</td><td>973.50</td><td>14.69</td><td>14.60</td></tr><tr><td>unimib_DoSSIER_2</td><td>1.44</td><td>1.33</td><td>1024.48</td><td>994.00</td><td>14.77</td><td>14.60</td></tr><tr><td>unimib_DoSSIER_4</td><td>1.44</td><td>1.33</td><td>238.63</td><td>212.00</td><td>15.11</td><td>15.00</td></tr></tbody></table>

## Acknowledgments

This research was funded, in whole or in part, by the French National Research Agency (ANR) under the project ANR-22-CE23-0019-01. We would like to thank Radia Hannachi, Silvia Araújo, Pierre De Loor, Olga Popova, Diana Nurbakova, Quentin Dubreuil, Aurianne Damoy, Angelique Robert, Julien Gaudin, and all other colleagues and participants who helped run this track.

## References

[1] L. Ermakova, P. Bellot, P. Braslavski, J. Kamps, J. Mothe, D. Nurbakova, I. Ovchinnikova, E. SanJuan, Overview of simpletext 2021 - CLEF workshop on text simplification for scientific information access, in: CLEF’21: Experimental IR Meets Multilinguality, Multimodality, and Interaction - 12th International Conference of the CLEF Association,
volume 12880 of Lecture Notes in Computer Science, Springer, 2021, pp. 432–449. URL: https://doi.org/10.1007/978-3-030-85251-1_27.

[2] L. Ermakova, E. SanJuan, J. Kamps, S. Huet, I. Ovchinnikova, D. Nurbakova, S. Araújo, R. Hannachi, É. Mathurin, P. Bellot, Overview of the CLEF 2022 simpletext lab: Automatic simplification of scientific texts, in: CLEF’22: Experimental IR Meets Multilinguality, Multimodality, and Interaction - 13th International Conference of the CLEF Association, volume 13390 of Lecture Notes in Computer Science, Springer, 2022, pp. 470–494. URL: https://doi.org/10.1007/978-3-031-13643-6_28.

[3] L. Ermakova, H. Azarbonyad, S. Bertin, O. Augereau, Overview of the CLEF 2023 SimpleText Task 2: Difficult Concept Identification and Explanation, in: [13], 2023.

[4] L. Ermakova, S. Bertin, H. McCombie, J. Kamps, Overview of the CLEF 2023 SimpleText Task 3: Scientific text simplification, in: [13], 2023.

[5] J. Tang, J. Zhang, L. Yao, J. Li, L. Zhang, Z. Su, Arnetminer: Extraction and mining of academic social networks, in: KDD’08, 2008, pp. 990–998.

[6] J. Tang, A. C. Fong, B. Wang, J. Zhang, A unified probabilistic framework for name disambiguation in digital library, IEEE Transactions on Knowledge and Data Engineering 24 (2012) 975–987.

[7] A. Sinha, Z. Shen, Y. Song, H. Ma, D. Eide, B.-j. P. Hsu, K. Wang, An overview of mi-crosoft academic service (mas) and applications, in: Proceedings of the 24th international conference on world wide web, ACM, 2015, pp. 243–246.

[8] J. Priem, H. Piwowar, R. Orr, Openalex: A fully-open index of scholarly works, authors, venues, institutions, and concepts, 2022. arXiv:2205.01833.

[9] A. Capari, H. Azarbonyad, G. Tsatsaronis, Z. Afzal, Elsevier at SimpleText: Passage Retrieval by Fine-tuning GPL on Scientific Documents, in: [13], 2023.

[10] R. Hutter, J. Sutmuller, M. Adib, D. Rau, J. Kamps, University of Amsterdam at the CLEF 2023 SimpleText Track, in: [13], 2023.

[11] B. Mansouri, S. Durgin, S. Franklin, S. Fletcher, R. Campos, AIIR and LIAAD Labs Systems for CLEF 2023 SimpleText, in: [13], 2023.

[12] O. E. Mendoza, G. Pasi, Domain Context-centered Retrieval for the Content Selection task in the Simplification of Scientific Literature, in: [13], 2023.

[13] M. Aliannejadi, G. Faggioli, N. Ferro, M. Vlachos (Eds.), Working Notes of CLEF 2023: Conference and Labs of the Evaluation Forum, CEUR Workshop Proceedings, CEUR-WS.org, 2023.