# SynCABEL: Synthetic Contextualized Augmentation for Biomedical Entity Linking

Adam Remaki, Christel Gérardin, Eulàlia Farré-Maduell, Martin Krallinger,

Xavier Tannier

<figure><img src="image_1.png" /></figure>

To cite this version:

Adam Remaki, Christel Gérardin, Eulàlia Farré-Maduell, Martin Krallinger, Xavier Tannier. SynCA-BEL: Synthetic Contextualized Augmentation for Biomedical Entity Linking. 2026. hal-05481323

HAL Id: hal-05481323

https://hal.science/hal-05481323v1

Preprint submitted on 28 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
arXiv:2601.19667v1 [cs.CL] 27 Jan 2026

# SynCABEL: Synthetic Contextualized Augmentation for Biomedical Entity Linking

Adam Remaki¹, Christel Gérardin¹, Eulàlia Farré-Maduell², Martin Krallinger² and Xavier Tannier¹

¹ Sorbonne Université, Inserm, Université Sorbonne Paris Nord, Limics, 75006 Paris, France

² Barcelona Supercomputing Center, Barcelona, Spain
adam.remaki@etu.sorbonne-universite.fr, christel.gerardin@aphp.fr,
{eulalia.farre,martin.krallinger}@bsc.es, xavier.tannier@sorbonne-universite.fr

## Abstract

We present SynCABEL (Synthetic Contextualized Augmentation for Biomedical Entity Linking), a framework that addresses a central bottleneck in supervised biomedical entity linking (BEL): the scarcity of expert-annotated training data. SynCABEL leverages large language models to generate context-rich synthetic training examples for all candidate concepts in a target knowledge base, providing broad supervision without manual annotation. We demonstrate that SynCABEL, when combined with decoder-only models and guided inference establish new state-of-the-art results across three widely used multilingual benchmarks: MedMentions for English, QUAERO for French, and SPACCC for Spanish. Evaluating data efficiency, we show that SynCABEL reaches the performance of full human supervision using up to 60% less annotated data, substantially reducing reliance on labor-intensive and costly expert labeling. Finally, acknowledging that standard evaluation based on exact code matching often underestimates clinically valid predictions due to ontology redundancy, we introduce an LLM-as-a-judge protocol. This analysis reveals that SynCABEL significantly improves the rate of clinically valid predictions. Our synthetic datasets, models, and code are released to support reproducibility and future research:

* HuggingFace Datasets & Models

* GitHub Repository

## 1 Introduction

Biomedical Entity Linking (BEL) is the task of mapping spans of text in biomedical documents to concepts in knowledge bases (KBs) such as the UMLS or SNOMED CT.

Early BEL approaches were largely context-free, relying on lexical matching or self-supervised models that exploited the KB alone, thus avoiding the need for context-dependent annotated training data. However, such methods struggle with pervasive lexical ambiguity: terms like “discharge” (release vs. secretion) or abbreviations common in clinical texts, such as “AS” (Aortic Stenosis vs. Ankylosing Spondylitis), may

refer to distinct concepts [Newman-Griffis et al., 2021; Xu et al., 2007].

To overcome these limitations, context-aware models emerged, leveraging surrounding text to resolve ambiguity. While superior when sufficiently trained, they introduce a major constraint: the need for large-scale, high-quality annotated datasets, which are scarce in the biomedical domain. Creating such resources is labor-intensive, requiring clinical experts to match mentions to ontology concepts. This scarcity severely limits generalization and creates a bottleneck for supervised BEL, as illustrated in Figure 1 where an ambiguous mention requiring context is absent from the training set.

<figure><img src="image_3.png" /><figcaption>Figure 1: Illustration of biomedical entity linking annotation scarcity: a context-free model fails without context, a supervised context-aware model trained on human annotations fails on unseen concepts, while a SynCABEL-augmented model leverages synthetic data to recover the correct concepts.</figcaption></figure>

To address this annotation scarcity issue, we introduce SynCABEL (Synthetic Contextualized Augmentation for Biomedical Entity Linking), a framework designed to enhance context-aware BEL by leveraging large language models (LLMs) to generate context-rich training instances for all candidate concepts in a KB. Our contributions are as follows:

* We release the first large-scale multilingual synthetic dataset for BEL in English, French, and Spanish.

* We show that SynCABEL matches full human-annotated training performance with substantially fewer human annotations.

* We demonstrate that combining recent decoder-only models with SynCABEL and guided inference achieves state-of-the-art performance on multiple BEL benchmarks.
Additionally, we developed an LLM-as-a-judge evaluation protocol for BEL that goes beyond exact code matching by assessing the semantic relationship between predicted and gold concepts, including equivalence, broader, narrower, and unrelated relations.

# 2 Related Works

## 2.1 BEL Systems

Research in BEL has evolved through several distinct paradigms, each addressing the problem with different strengths and limitations [Chen et al., 2025b].

**Rule-based Systems.** Early systems like cTAKES [Savova et al., 2010], and SciSpacy [Neumann et al., 2019] relied on heuristic string matching. While widely adopted, their reliance on static dictionaries limits generalization to predefined lexical variants, causing failures when a mention refers to an existing concept using a surface form not listed among its synonyms in the ontology.

**Context-free Bi-encoder Models.** Transformer-based models like SapBERT [Liu et al., 2021a; Liu et al., 2021b] and CODER [Yuan et al., 2022b] use self-supervised contrastive learning on synonyms. While efficient for embedding names in a shared space without annotated data, they ignore surrounding context, limiting their ability to resolve ambiguity.

**Prompting LLM.** LLMs have been used via prompting to simplify mentions before linking [Borchert et al., 2024; Vollmers et al., 2025], as re-rankers [Ye and Mitchell, 2025], or guided via constrained decoding without training [Lin et al., 2025]. However, prompting-based methods often yield inferior results compared to specialized EL models.

**Contextualized Bi-encoder Models.** These systems encode mentions with context and candidates in a shared space. Unlike context-free models, they explicitly learn to capture contextual information. ArboEL [Agarwal et al., 2022], for example, adapts BLINK [Wu et al., 2020] and introduces a graphical arborescence objective to model cross-document coreference through directed spanning trees, achieving state-of-the-art results on MedMentions [Mohan and Li, 2019].

**Generative Models.** These models treat entity linking as conditional text generation, directly producing concept identifiers. GENRE [Cao et al., 2021] introduced constrained decoding to ensure valid outputs and strictly improve memory efficiency by avoiding dense indexes. Currently, biomedical implementations like GenBioEL [Yuan et al., 2022a] are built on pretrained encoder-decoder architectures. The general-domain InsGenEL [Xiao et al., 2023] has extended this approach by fine-tuning a decoder-only model, though it is optimized for a joint mention detection and linking objective and not for entity linking only.

## 2.2 BEL Data Augmentation

Despite architectural differences, both retrieval and generative methods depend on annotated datasets which are currently covering only a fraction of the KB and lack variability.

This scarcity creates a bottleneck for developing robust, generalizable BEL systems, motivating alternative data creation strategies to improve scalability and coverage while reducing annotation costs.

**Weak and Distant Supervision.** Approaches using exact string matching in large corpora (e.g., PubMed, Wikipedia) to create training data [Zhang et al., 2022; Vashishth et al., 2021; Wang et al., 2023] are scalable but noisy due to synonym ambiguity and non-clinical contexts.

**Template-Based Synthetic Data.** To better capture entity semantics, Yuan et al. [2022a] used UMLS definitions as templates, filled with synonyms. This provides clean, concept-focused data but lacks the lexical and contextual diversity of natural language.

**LLM-generated Synthetic Data.** Xin et al. [2024] and Chen et al. [2025a] have demonstrated the effectiveness of LLMs for generating additional training examples, but restrict synthesis to concepts already observed during training. While this increases contextual diversity and improves generalization across known entities, it does not address annotation scarcity for entirely unseen concepts. Relatedly, Josifoski et al. [2023] leverage synthetic data to mitigate data scarcity, but their approach targets a different task setting, namely relational triple extraction.

## 2.3 Positioning of SynCABEL

SynCABEL advances BEL on two fronts. First, it introduces a data augmentation strategy that generates examples for the entire space of candidate concepts in the KB. It overcomes the limitations of prior work that either used templates for full coverage (low quality) or LLMs only for training-set concepts (limited coverage). Second, it is the first approach to fine-tune a decoder-only model specifically for BEL, allowing us to leverage recent improvements in large foundation models.

# 3 Methodology

## 3.1 Problem statement

Let $\mathcal{E}$ represent the set of candidate concepts from a given KB. Each concept $e \in \mathcal{E}$ is associated with a semantic group $g_e$ (e.g., Disorders) and multiple synonyms $S_e = \{s_e^1, \dots, s_e^{n_e}\}$. Given a mention $m$ with left context $c_l$, right context $c_r$ and semantic group $g_e$, our objective is to predict the correct concept $e_m$ that the mention $m$ refers to.

## 3.2 Generative BEL Training Objective

We adopt the autoregressive formulation from Cao et al. [2021]. The input sequence $x$ is defined as:

$$x = c_l[m] \{g_e\} c_r[SEP][m] \text{ is}$$

where $c_l, c_r$ denote context, $m$ the target mention, and $g_e$ its semantic group. The token $[SEP]$ and cue phrase “$m$ is” prompt the model to generate output sequence $y$ defined as:

$$y = e_m$$

where $e_m$ is the concept corresponding to mention $m$. We maximize the conditional likelihood:

$$p_{\theta}(y|x) = \prod_{i=1}^{N_y} p_{\theta}(y_i|y_{<i}, x)$$
where $N_y$ is the number of tokens in the output, and $y_i$ is the i-th token.

## 3.3 Adaptive Concept Representation for Training

A key challenge during training is representing the concept $e_m$ in natural language for the target output. We employ an adaptive method that selects the most appropriate synonym of a concept based on the input mention. As shown in Figure 2, the method proceeds in two steps. First, we preprocess each concept's list of synonyms by removing those that are ambiguous with other concepts in the same semantic group. For example, the term “discharge” is removed from the *Disorder* group because it can refer to multiple clinical symptoms. Then, for each remaining unambiguous synonym $s_e$, we compute the cosine similarity between the embeddings of the mention $m$ and $s_e$. The concept representation $e_m$ is defined as the synonym with the highest similarity score.

$$e_m = \arg \max_{s_e \in S_e} \frac{\mathbf{v}_m \cdot \mathbf{v}_{s_e}}{\|\mathbf{v}_m\| \|\mathbf{v}_{s_e}\|}$$

where $\mathbf{v}_m$ and $\mathbf{v}_{s_e}$ are the embedding vectors of the mention $m$ and synonym $s_e$, respectively. Importantly, the embeddings can be derived from any model, in our experiments, we compare different embedding models to assess their impact on linking performance.

<figure><img src="image_4.png" /><figcaption>Figure 2: Example of the disambiguation process used to select a natural representation $e_m$ = “Fluid Discharge” for a concept $e$ = “C0012621” and a mention $m$ = “discharge”.</figcaption></figure>

## 3.4 Synthetic Data Generation

To improve concept coverage while maintaining contextual realism, we generate synthetic training data using a structured prompt-based approach. For a given concept $e$, our method generates contextual sentences that capture diverse, natural ways of expressing $e$. As illustrated in Figure 3, this is achieved by constructing a structured prompt composed of three key components: (i) the task description that explicitly structures the generation process into two sequential tasks: first, selecting and generating mentions of the concept. Second, generating contextualized sentences for each generated mention, aligned with the style of the provided human-annotated dataset examples. (ii) Random contextual examples from a human-annotated training set (iii) Title, semantic

group, semantic type, definitions and synonyms of the target concept $e$ from the KB, and this structured prompt is provided to an LLM, which produces the final contextualized sentences. This structured prompt is formulated in the same language as the target dataset.

## 3.5 Training Data Composition

The synthetic examples generated by SynCABEL are combined with human-annotated training data to fine-tune the entity linking model (Figure 3). To preserve the characteristics of clinical reports while benefiting from the increased coverage of synthetic data, we jointly train on both sources and upsample human-annotated examples such that they constitute half of the training instances.

## 3.6 Guided Inference

Greedy decoding often yields valid entities absent from the KB. To address this, we adopt the guided inference mechanism from Cao et al. [2021], illustrated in Figure 3. First, the KB vocabulary is filtered by the target entity's semantic group (e.g., *Disorders*) and structured as a trie, where each path maps a synonym to a unique concept. Then, the model dynamically restricts token selection at each step to valid branches. For instance, after generating the prefix “A”, the trie permits only valid continuations like “the” (for *Atherosclerosis*) or “ortic” (for *Aortic*).

## 3.7 LLM-as-a-judge Evaluation

Standard entity linking evaluation relies on exact code matching and therefore may fail to capture the true clinical relationship between predicted and gold concepts. In practice, a prediction can be clinically correct despite a code mismatch due to ontology redundancy (e.g., multiple codes for the same concept) or annotation noise, where the prediction better reflects the mention than the human label. To capture these nuances, we introduce a four-class scheme (Figure 4): (i) **Correct**, clinically appropriate even without exact code match; (ii) **Broad**, relevant but overly general; (iii) **Narrow**, relevant but overly specific; and (iv) **No relation**, clinically unrelated. Labels are assigned automatically by an LLM using precise class definitions and five clinician-annotated examples.

# 4 Experiments

## 4.1 Datasets

### Human-Annotated Datasets

We rely on three human-annotated corpora for our experiments:

* **MedMentions-ST21pv (MM-ST21pv)** [Mohan and Li, 2019], a curated subset of the MedMentions corpus, consisting of 4,392 English PubMed abstracts. MedMentions is currently the largest publicly available human-annotated dataset for biomedical entity linking.

* **QUAERO French Medical Corpus (QUAERO)** [Névéol et al., 2014], the largest French dataset for biomedical entity linking, composed of two subsets: QUAERO-MEDLINE, derived from MEDLINE titles, and QUAERO-EMEA, extracted from documents published by the European Medicines Agency.
<figure><img src="image_5.png" /><figcaption>Figure 3: Overview of SynCABEL. LLM: large language model</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Figure 4: LLM-as-a-judge evaluation of predicted vs. gold concepts using four clinical-relation classes: Correct, Broad, Narrow, and No relation.</figcaption></figure>

* Spanish Clinical Case Corpus (SPACCC) [Miranda-Escalada et al., 2022; Lima-López et al., 2023a; Lima-López et al., 2023b], a manually annotated collection of clinical case reports derived from open-access Spanish medical publications. The corpus contains 1,000 clinical cases, with disease, procedure, and symptom mentions annotated using the SNOMED CT ontology.

MM-ST21pv and QUAERO were selected because they are the largest human-annotated biomedical entity linking datasets available in their respective languages and because they cover nearly all UMLS semantic groups. SPACCC was chosen as the largest Spanish BEL dataset and, more importantly, because it consists of clinical reports, making it particularly valuable for assessing the potential applicability of our approach in clinical settings. Table 1 summarizes the main characteristics of these datasets.

## Synthetic Datasets

We constructed three synthetic datasets (SynthMM, SynthQUAERO, and SynthSPACCC) to complement the human-annotated corpora. We employed Llama-3-70B [Grattafiori et al., 2024] to generate three training examples per concept, using prompts containing five random annotated examples from the target training set. While our method is applicable to all candidate concepts, for SynthMM and SynthQUAERO we restricted generation to the subset of UMLS concepts with available definitions (covering 6.5% and 4.5% of the KB, respectively) to maintain computational feasibility.

<table><caption>Table 1: Summary statistics of the human-annotated datasets. MM-ST21pv: MedMentions-ST21pv; SPACCC: Spanish Clinical Case Corpus</caption><thead><tr><th></th><th>MM-ST21PV</th><th>QUAERO</th><th>SPACCC</th></tr></thead><tbody><tr><td># Docs</td><td>4,392</td><td>2,422</td><td>1,000</td></tr><tr><td># Mentions</td><td>203,282</td><td>16,185</td><td>27,799</td></tr><tr><td># Concepts</td><td>25,419</td><td>5,045</td><td>9,254</td></tr><tr><td># Sem. Groups</td><td>14</td><td>10</td><td>3</td></tr><tr><td>Language</td><td>English</td><td>French</td><td>Spanish</td></tr><tr><td>Clinical cases</td><td>✗</td><td>✗</td><td>✓</td></tr></tbody></table>

<table><caption>Table 2: Summary of synthetic datasets.</caption><thead><tr><th>Dataset</th><th>SynthMM</th><th>SynthQUAERO</th><th>SynthSPACCC</th></tr></thead><tbody><tr><td># Mentions</td><td>460,878</td><td>396,914</td><td>1,813,463</td></tr><tr><td># Concepts</td><td>153,374</td><td>132,089</td><td>276,649</td></tr></tbody></table>

Table 2 summarizes the key statistics of these generated datasets. The full datasets, along with examples and all associated prompts are available on HuggingFace.

## 4.2 Implementation details

### Hardware and software

All experiments were conducted on a single NVIDIA H100 GPU (80GB). The codebase is implemented in Python using PyTorch and the Hugging Face Transformers library. For reproducibility, full implementation details, hyperparameters, and code are provided on GitHub.

### Knowledge bases

We used three KBs aligned with each dataset's annotation guidelines. For MM-ST21pv, we used UMLS 2017AA restricted to the ST21-pv scheme (21 semantic types and synonyms from 18 source ontologies). For QUAERO, we used UMLS 2014AA filtered to 10 semantic groups. For SPACCC, we relied on gazetteers derived from relevant branches of Spanish SNOMED-CT (July 31, 2021 release); since SPACCC comprises three subsets (diseases, symptoms, procedures), these categories served as semantic groups.
<table><caption>Table 3: Entity linking performance (Recall@1) on biomedical benchmarks. The highlighted row shows our contribution, where the base model is trained on an augmented dataset. The best results are shown in bold, the second-best results are underlined, and the “Average” column reports the mean score across the four benchmarks.</caption><thead><tr><th>Model</th><th>MM-ST21PV (english)</th><th>QUAERO-MEDLINE (french)</th><th>QUAERO-EMEA (french)</th><th>SPACCC (spanish)</th><th>Average</th></tr></thead><tbody><tr><td colspan="6">RULE-BASED (UNSUPERVISED)</td></tr><tr><td>SciSpacy [Neumann et al., 2019]</td><td>53.8</td><td>40.5</td><td>37.1</td><td>13.2</td><td>36.2</td></tr><tr><td colspan="6">CONTEXT-FREE BI-ENCODER (SELF-SUPERVISED)</td></tr><tr><td>SapBERT [Liu et al., 2021a]</td><td>51.1</td><td>50.6</td><td>49.8</td><td>33.9</td><td>46.4</td></tr><tr><td>CODER-all [Yuan et al., 2022b]</td><td>56.6</td><td>58.7</td><td>58.1</td><td>43.7</td><td>54.3</td></tr><tr><td>SapBERT-all [Liu et al., 2021b]</td><td>64.6</td><td>74.7</td><td>67.9</td><td>47.9</td><td>63.8</td></tr><tr><td colspan="6">CONTEXTUALIZED BI-ENCODER (SUPERVISED)</td></tr><tr><td>ArboEL [Agarwal et al., 2022]</td><td>74.5</td><td>70.9</td><td>62.8</td><td>49.0</td><td>64.2</td></tr><tr><td colspan="6">GENERATIVE ENCODER-DECODER (SUPERVISED)</td></tr><tr><td>mBART-large [Tang et al., 2020]</td><td>65.5</td><td>61.5</td><td>58.6</td><td>57.7</td><td>60.8</td></tr><tr><td>+ Guided inference [Cao et al., 2021]</td><td>70.0</td><td>72.8</td><td>71.1</td><td>61.8</td><td>68.9</td></tr><tr><td><strong>+ SynCABEL (Our method)</strong></td><td>71.5</td><td>77.1</td><td>75.3</td><td>64.0</td><td>72.0</td></tr><tr><td colspan="6">GENERATIVE DECODER-ONLY (SUPERVISED)</td></tr><tr><td>Llama-3-8B [Grattafiori et al., 2024]</td><td>69.0</td><td>66.4</td><td>65.5</td><td>59.9</td><td>65.2</td></tr><tr><td>+ Guided inference [Cao et al., 2021]</td><td>74.4</td><td>77.5</td><td>72.9</td><td>64.2</td><td>72.3</td></tr><tr><td><strong>+ SynCABEL (Our method)</strong></td><td>75.4</td><td>79.7</td><td>79.0</td><td>67.0</td><td>75.3</td></tr></tbody></table>

## 4.3 Results

### Benchmark Performance

We reproduced several state-of-the-art BEL systems from different paradigms: the rule-based **SciSpacy** [Neumann et al., 2019]; the context-free bi-encoders **SapBERT** [Liu et al., 2021a], **SapBERT-all** [Liu et al., 2021b], and **CODER-all** [Yuan et al., 2022b]; the contextualized bi-encoder **ArboEL** [Agarwal et al., 2022]; the generative encoder-decoder **mBART-large** [Tang et al., 2020] and decoder-only **Llama-3-8B** [Grattafiori et al., 2024]. For generative models (mBART-large and Llama-3-8B), we evaluated three configurations: (i) supervised fine-tuning on human-annotated data; (ii) adding guided inference adapted from GENRE [Cao et al., 2021]; and (iii) our proposed SynCABEL approach, which augments the training data with synthetic examples while retaining guided inference. For fair comparison, all baselines were constrained to the same KBs, and for each mention, only concepts from the matching semantic group were considered as candidates. In practice, this meant that for a given input, every baseline was provided with the same candidate set. This constraint could not be applied to SciSpacy, as its rule-based approach uses the entire KB and cannot be modified.

Table 3 presents entity linking performance (Recall@1) across four biomedical benchmarks: MM-ST21pv, QUAERO-MEDLINE, QUAERO-EMEA, and SPACCC. Models are grouped by architecture paradigm. Our SynCABEL framework with Llama-3-8B achieves the highest scores on all four benchmarks: 75.4 on MM-ST21pv, 79.7 on

QUAERO-MEDLINE, 79.0 on QUAERO-EMEA, and 67.0 on SPACCC. The mBART-large variant of SynCABEL also shows consistent improvements over its baseline across all datasets.

### Adaptive Concept Representation for Training

We evaluate our adaptive concept representation strategy against a static baseline using concept preferred titles. We consider two adaptive representations: character-level 3-gram TF-IDF [Neumann et al., 2019], following its effective use in GenBioEL [Yuan et al., 2022a] and CODER-all embeddings [Yuan et al., 2022b]. Experiments are conducted on UMLS-based datasets (MM-ST21pv, QUAERO-MEDLINE and QUAERO-EMEA) using mBART-large, with training limited to the original data (no augmentation) and guided decoding at inference.

<table><caption>Table 4: Recall@1 scores for different concept representation methods. mBART-large was supervised only on the human-annotated dataset (i.e., without data augmentation), and inference was performed using the guided inference method. The highlighted row shows the chosen representation method.</caption><thead><tr><th></th><th>MM-ST21PV</th><th>QUAERO-MEDLINE</th><th>QUAERO-EMEA</th></tr></thead><tbody><tr><td>Title (Static)</td><td>61.7</td><td>53.7</td><td>50.6</td></tr><tr><td>CODER-all</td><td>68.4</td><td>72.5</td><td>66.0</td></tr><tr><td>TF-IDF</td><td><strong>70.0</strong></td><td><strong>72.8</strong></td><td><strong>71.1</strong></td></tr></tbody></table>
As shown in Table 4, the TF-IDF-based representation consistently achieves the highest Recall@1 across all datasets, outperforming both the static baseline and CODER-all. We therefore adopt TF-IDF-based adaptive representations for all other experiments.

## Training Data Composition

We evaluate three training data composition strategies when combining human-annotated and synthetic data: (i) **Synthetic Pretrain**, which uses a two-stage training procedure, first training on synthetic data and then fine-tuning on human data, without resampling; (ii) **Combined**, which performs a single-stage training on the union of human and synthetic datasets as-is, without resampling; (iii) **Interleaved**, which performs a single-stage training on merged data while oversampling human samples so that each training step alternates between a human and a synthetic instance. These experiments use Llama-3-8B with guided inference.

<table><caption>Table 5: Recall@1 scores for different training data composition strategies. The highlighted row shows the chosen strategy.</caption><thead><tr><th></th><th>MM-ST21pv</th><th>QUAERO-MEDLINE</th><th>QUAERO-EMEA</th><th>SPACCC</th></tr></thead><tbody><tr><th>SPT</th><td>74.1</td><td>78.0</td><td>77.1</td><td>64.0</td></tr><tr><th>COMB</th><td>75.4</td><td>79.4</td><td>77.0</td><td>64.3</td></tr><tr><th>INT</th><td><strong>75.4</strong></td><td><strong>79.7</strong></td><td><strong>79.0</strong></td><td><strong>67.0</strong></td></tr></tbody></table>

Recall@1 results in Table 5 show that interleaved oversampling consistently outperforms other strategies across all datasets, highlighting the importance of preserving a strong human signal while leveraging synthetic diversity. We therefore adopt this strategy in other experiments.

## LLM-as-a-Judge Evaluation

To complement exact code matching, we employed GPT-5.2¹ as an automated judge to assess clinical validity on SPACCC, the only dataset containing full clinical reports. We evaluated all cases where the prediction of our best-performing model (Llama-3-8B) differed from the gold standard.<fnref n="2" />

A clinician labeled 150 mismatches across Disorders, Symptoms, and Procedures. GPT-5.2 achieved 66.2% agreement, with 79% precision for the 'Correct' label. Crucially, errors are conservative: only 12.4% of judge predictions overstated the human label (e.g., Narrow for No relation). This tendency to under-predict relevance implies the judge acts as a lower-bound estimator of validity.

Table 6 compares standard exact matching against the LLM-as-a-judge evaluation metric. While SynCABEL yields a 2.8% gain in exact matching, it achieves a larger 3.7% improvement in clinically valid predictions (rising from 68.9% to 72.6%). Given the judge's proven conservative bias, it shows SynCABEL helps the model capture correct semantics even when it misses the exact target code.

<table><caption>Table 6: Llama-3-8B performance on SPACCC with and without SynCABEL augmentation. Results are reported as percentages with 95% confidence intervals estimated via document-level empirical bootstrap, for exact code matching and each LLM-as-a-judge label.</caption><thead><tr><th>Label</th><th>w/o SynCABEL</th><th>w SynCABEL</th></tr></thead><tbody><tr><td>Exact Matching</td><td>64.2 (62.6-65.8)</td><td>67.0 (65.5-68.6)</td></tr><tr><td>Correct</td><td><strong>68.9 (67.4-70.4)</strong></td><td><strong>72.6 (71.2-74.1)</strong></td></tr><tr><td>Broad</td><td>13.0 (12.1-13.8)</td><td>11.0 (10.3-11.8)</td></tr><tr><td>Narrow</td><td>4.3 (3.8-4.7)</td><td>5.9 (5.3-6.5)</td></tr><tr><td>No relation</td><td>13.8 (11.1-16.7)</td><td>10.5 (7.6-13.2)</td></tr></tbody></table>

# 5 Discussion

## 5.1 Annotation Scarcity

To assess synthetic data impact, we stratify performance by whether concepts appeared in training. As shown in Table 7, SynCABEL consistently improves unseen concepts performance across benchmarks while preserving seen results, with notable gains on SPACCC and QUAERO-EMEA (+9.4 and +9.9 points). However, unseen performance remains below seen (e.g., 30.2 vs 83.0 on SPACCC), showing that SynCABEL reduces but does not fully close the annotation scarcity gap. For UMLS-based datasets (QUAERO, MM), generation was restricted to concepts with definitions, leaving some “unseen” test concepts absent from augmented data, a filter that limits gains compared to SPACCC, suggesting that extending generation to all concepts could further bridge this gap.

<table><caption>Table 7: Exact-match Recall@1 performance on three biomedical benchmarks comparing two variants of Llama-3-8B: one trained only on the corresponding human-annotated dataset with guided inference, and one further enhanced with SynCABEL. Results are reported overall and for subsets defined by seen/unseen concepts. 95% confidence intervals were estimated via the empirical bootstrap method at the document level.</caption><thead><tr><th>Subset</th><th>w/o SynCABEL</th><th>w/ SynCABEL</th></tr></thead><tbody><tr><td>MEDMENTIONS-ST21PV</td><td></td><td></td></tr><tr><td>Overall</td><td>74.4 (73.6-75.4)</td><td>75.4 (74.5-76.3)</td></tr><tr><td>Seen Concepts</td><td>81.8 (81.0-82.6)</td><td>82.2 (81.4-83.0)</td></tr><tr><td>Unseen Concepts</td><td>46.0 (43.9-48.2)</td><td>49.0 (46.7-51.3)</td></tr><tr><td>QUAERO-MEDLINE</td><td></td><td></td></tr><tr><td>Overall</td><td>77.5 (75.9-79.1)</td><td>79.7 (78.0-81.2)</td></tr><tr><td>Seen Concepts</td><td>90.5 (89.1-91.8)</td><td>89.7 (88.3-91.0)</td></tr><tr><td>Unseen Concepts</td><td>58.7 (55.8-61.6)</td><td>65.1 (62.1-68.1)</td></tr><tr><td>QUAERO-EMEA</td><td></td><td></td></tr><tr><td>Overall</td><td>72.9 (68.3-77.3)</td><td>79.0 (75.7-82.2)</td></tr><tr><td>Seen Concepts</td><td>89.0 (86.1-91.6)</td><td>92.0 (89.6-94.3)</td></tr><tr><td>Unseen Concepts</td><td>52.5 (46.1-59.3)</td><td>62.4 (56.4-68.7)</td></tr><tr><td>SPACCC</td><td></td><td></td></tr><tr><td>Overall</td><td>64.2 (62.6-65.8)</td><td>67.0 (65.5-68.6)</td></tr><tr><td>Seen Concepts</td><td>83.0 (81.8-84.2)</td><td>83.0 (81.8-84.2)</td></tr><tr><td>Unseen Concepts</td><td>20.8 (19.2-22.6)</td><td>30.2 (28.1-32.4)</td></tr></tbody></table>

<footnote n="1">¹Developed by OpenAI and accessed via their API.</footnote>

<footnote n="2">²We also explored the KB-based architecture to explicitly detect broader/narrower relations, but it did not yield reliable results, likely due to the complexity and brittleness of these KBs.</footnote>
## 5.2 Reducing Annotation Effort

We evaluate data efficiency by varying the fraction of human-annotated training data (0–100%), with subsets created by randomly sampling documents, and comparing SynCABEL augmentation to the human-annotated only baseline.

<figure><img src="image_7.png" /><figcaption>Figure 5: Data Efficiency Analysis. Exact-match Recall@1 performance as a function of human training data availability. The dashed line marks the performance ceiling of the standard baseline trained on human-annotated data. Stars (*) indicate where the SynCABEL-augmented model (Blue) matches or surpasses this ceiling. 0% implies training on synthetic data only. Error bars denote 95% bootstrap CIs.</figcaption></figure>

Figure 5 demonstrates that SynCABEL substantially reduces annotation needs: it reaches full-data performance using 60% of human annotations on MM-ST21pv and SPACCC, and 40% on QUAERO. The gains are largest in low-resource regimes and diminish as more human data becomes available, this trend is clear on SPACCC and MedMention, though it is less observable on QUAERO due to the larger confidence intervals inherent to its smaller size. Finally, models trained solely on synthetic data significantly underperform fully supervised baselines, confirming that human annotations remain essential for optimal results on human-annotated data.

## 5.3 Real-World Applicability

BEL plays a central role in enabling the secondary use of electronic health records. By transforming unstructured free-text into structured concepts, it supports medical research, clinical decision, and downstream tasks such as automated clinical coding, cohort identification, and patient recruitment for clinical trials [Remaki et al., 2025; French and McInnes, 2023]. To assess the viability of our models for such real-world deployment, we evaluate two dimensions: the utility

of confidence scores for ensuring high-precision predictions, and the computational efficiency required for scale.

**Prediction Confidence** For each prediction, our model outputs a confidence score, computed as the sum of token-level log-probabilities of the generated sequence. This score enables confidence-based filtering to trade precision for recall depending on application needs. On SPACCC, we empirically set a confidence threshold of 0.9 and evaluated performance on predictions exceeding this threshold.

It yields a precision of 80.8% (79.6–81.9), recall of 61.9% (60.2–63.5), and F1-score of 70.1% (68.5–71.6). These results illustrate how applying a strict confidence threshold can substantially boost precision for high-stakes applications while retaining reasonable recall.

**Inference Efficiency and Memory Footprint** We evaluate real-world deployability based on throughput and memory usage (Table 8). While SapBERT achieves the highest speed, it requires a massive 20.1 GB candidate index. ArboEL uses a smaller BERT encoder, reducing model size (1.2 GB) and candidate memory (7.1 GB), but throughput is lower (38.9 mentions/s) due to the cross-encoder re-ranking step required at inference. In contrast, generative models rely on a compact candidate trie (5.4 GB), independent of model size. The mBART configuration processes 51.0 mentions/s with a small model footprint (2.3 GB), whereas Llama-3-8B requires substantially more GPU memory (28.6 GB) and achieves lower throughput (19.1 mentions/s).

<table><caption>Table 8: Inference speed and memory footprint. Model: Model parameters size; Cand.: Candidate memory size.</caption><thead><tr><th></th><th>Model (GB)</th><th>Cand. (GB)</th><th>Speed (/s)</th></tr></thead><tbody><tr><td>SapBERT</td><td>2.1</td><td>20.1</td><td><strong>575.5</strong></td></tr><tr><td>ArboEL</td><td><strong>1.2</strong></td><td>7.1</td><td>38.9</td></tr><tr><td>mBART</td><td>2.3</td><td><strong>5.4</strong></td><td>51.0</td></tr><tr><td>Llama-3-8B</td><td>28.6</td><td><strong>5.4</strong></td><td>19.1</td></tr></tbody></table>

## 6 Conclusion and Future Work

We introduced SynCABEL, a framework designed to address annotation scarcity in BEL by generating synthetic, contextualized training examples for all candidate concepts in a target KB. Our approach achieves state-of-the-art performance on multilingual benchmarks, yielding significant recall improvements on unseen concepts and enhancing clinical validity as verified by our novel LLM-as-a-judge protocol. Although our results confirm that human annotations remain the most effective source of supervision, SynCABEL demonstrates that competitive performance can be achieved with substantially less of this costly data, maximizing the value of expert effort.

Future directions include extending the generation context beyond single sentences, extending to other languages and refining training with negative sampling like ANGEL [Kim et al., 2025]. We also aim to improve synthetic data quality by refining generation prompts and validating clinical coherence with intrinsic assessments by medical experts.
## References

[Agarwal et al., 2022] Dhruv Agarwal, Rico Angell, Nicholas Monath, and Andrew McCallum. Entity Linking via Explicit Mention-Mention Coreference Modeling. In Marine Carpuat, Marie-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz, editors, *Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pages 4644–4658, Seattle, United States, July 2022. Association for Computational Linguistics.

[Borchert et al., 2024] Florian Borchert, Ignacio Llorca, and Matthieu-P. Schapranow. Improving biomedical entity linking for complex entity mentions with LLM-based text simplification. *Database*, 2024, February 2024. Publisher: Oxford Academic.

[Cao et al., 2021] Nicola De Cao, Gautier Izacard, Sebastian Riedel, and Fabio Petroni. Autoregressive Entity Retrieval, March 2021. arXiv:2010.00904 [cs].

[Chen et al., 2025a] Haihua Chen, Ruochi Li, Ana Cleveland, and Junhua Ding. Enhancing data quality in medical concept normalization through large language models. *Journal of Biomedical Informatics*, 165:104812, May 2025.

[Chen et al., 2025b] Haihua Chen, Yuhan Zhou, Ruochi Li, Aryan Murthy Illa, Ana Cleveland, and Junhua Ding. A Comprehensive Survey on Medical Concept Normalization: Datasets, Techniques, Applications, and Future Directions, November 2025.

[French and McInnes, 2023] Evan French and Bridget T. McInnes. An overview of Biomedical Entity Linking throughout the years. *Journal of biomedical informatics*, 137:104252, January 2023.

[Grattafiori et al., 2024] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and Yang. The Llama 3 Herd of Models, November 2024. arXiv:2407.21783 [cs].

[Josifoski et al., 2023] Martin Josifoski, Marija Sakota, Maxime Peyrard, and Robert West. Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extraction. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, *Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*, pages 1555–1574, Singapore, December 2023. Association for Computational Linguistics.

[Kim et al., 2025] Chanhwi Kim, Hyunjae Kim, Sihyeon Park, Jiwoo Lee, Mujeen Sung, and Jaewoo Kang. Learning from Negative Samples in Biomedical Generative Entity Linking. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, *Findings of the Association for Computational Linguistics: ACL 2025*, pages 10714–10730, Vienna, Austria, July 2025. Association for Computational Linguistics.

[Lima-López et al., 2023a] Salvador Lima-López, Eulalia Farré-Maduell, Luis Gasco Sanchez, Jan Rodríguez-Miret, and Martin Krallinger. Overview of SympTEMIST at BioCreative VIII: corpus, guidelines and evaluation of systems for the detection and normalization of symptoms, signs and findings from text. In *Proceedings of the BioCreative VIII Challenge and Workshop: Curation and Evaluation in the Era of Generative Models*, November 2023.

[Lima-López et al., 2023b] Salvador Lima-López, Eulalia Farré-Maduell, Luis Gasco, Anastasios Nentidis, Anastasia Krithara, Georgios Katsimpras, Georgios Paliouras, and Martin Krallinger. Overview of MedProcNER task on medical procedure detection and entity linking at BioASQ 2023. *Working Notes of CLEF*, 2023.

[Lin et al., 2025] Zhenxi Lin, Ziheng Zhang, Jian Wu, Yefeng Zheng, and Xian Wu. Guiding Large Language Models for Biomedical Entity Linking via Restrictive and Contrastive Decoding. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors, *Findings of the Association for Computational Linguistics: EMNLP 2025*, pages 23745–23759, Suzhou, China, November 2025. Association for Computational Linguistics.

[Liu et al., 2021a] Fangyu Liu, Ehsan Shareghi, Zaiqiao Meng, Marco Basaldella, and Nigel Collier. Self-Alignment Pretraining for Biomedical Entity Representations. In *Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pages 4228–4238, Online, 2021. Association for Computational Linguistics.

[Liu et al., 2021b] Fangyu Liu, Ivan Vulić, Anna Korhonen, and Nigel Collier. Learning Domain-Specialised Representations for Cross-Lingual Biomedical Entity Linking. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Nagli, editors, *Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)*, pages 565–574, Online, August 2021. Association for Computational Linguistics.

[Miranda-Escalada et al., 2022] Antonio Miranda-Escalada, Luis Gasco, Salvador Lima-López, Eulalia Farré-Maduell, Darryl Estrada, Anastasios Nentidis, Anastasia Krithara, Georgios Katsimpras, Georgios Paliouras, and Martin Krallinger. Overview of DisTEMIST at BioASQ: Automatic detection and normalization of diseases from clinical texts: results, methods, evaluation and multilingual resources. In *Working Notes of Conference and Labs of the Evaluation (CLEF) Forum. CEUR Workshop Proceedings*, 2022.

[Mohan and Li, 2019] Sunil Mohan and Donghui Li. MedMentions: A Large Biomedical Corpus Annotated with UMLS Concepts, February 2019. arXiv:1902.09476 [cs].

[Neumann et al., 2019] Mark Neumann, Daniel King, Iz Beltagy, and Waleed Ammar. ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing. In Dina Demner-Fushman, Kevin Bretonnel Cohen, Sophia Ananiadou, and Junichi Tsujii, editors, *Proceedings of the 18th BioNLP Workshop and Shared*
*Task*, pages 319–327, Florence, Italy, August 2019. Association for Computational Linguistics.

[Newman-Griffis et al., 2021] Denis Newman-Griffis, Guy Divita, Bart Desmet, Ayah Zirikly, Carolyn P. Rosé, and Eric Fosler-Lussier. Ambiguity in medical concept normalization: An analysis of types and coverage in electronic health record datasets. *Journal of the American Medical Informatics Association: JAMIA*, 28(3):516–532, March 2021.

[Névéol et al., 2014] Aurélie Névéol, Cyril Grouin, Jeremy Leixa, Sophie Rosset, and Pierre Zweigenbaum. The QUAERO French Medical Corpus: A Ressource for Medical Entity Recognition and Normalization. In *Proc of Bio-TextMining Work*, pages 24–30, 2014.

[Remaki et al., 2025] Adam Remaki, Jacques Ung, Pierre Pages, Perceval Wajsburt, Elise Liu, Guillaume Faure, Thomas Petit-Jean, Xavier Tannier, and Christel Gérardin. Improving Phenotyping of Patients With Immunemediated Inflammatory Diseases Through Automated Processing of Discharge Summaries: Multicenter Cohort Study. *JMIR Medical Informatics*, 13(1):e68704, April 2025. Company: JMIR Medical Informatics Distributor: JMIR Medical Informatics Institution: JMIR Medical Informatics Label: JMIR Medical Informatics Publisher: JMIR Publications Inc., Toronto, Canada.

[Savova et al., 2010] Guergana K Savova, James J Masanz, Philip V Ogren, Jiaping Zheng, Sunghwan Sohn, Karin C Kipper-Schuler, and Christopher G Chute. Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications. *Journal of the American Medical Informatics Association*, 17(5):507–513, September 2010.

[Tang et al., 2020] Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. Multilingual Translation with Extensible Multilingual Pretraining and Finetuning, August 2020. arXiv:2008.00401 [cs].

[Vashishth et al., 2021] Shikhar Vashishth, Denis Newman-Griffis, Rishabh Joshi, Ritam Dutt, and Carolyn P. Rosé. Improving broad-coverage medical entity linking with semantic type prediction and large-scale datasets. *Journal of Biomedical Informatics*, 121:103880, September 2021.

[Vollmers et al., 2025] Daniel Vollmers, Hamada Zahera, Diego Moussallem, and Axel-Cyrille Ngonga Ngomo. Contextual Augmentation for Entity Linking using Large Language Models. In Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, and Steven Schockaert, editors, *Proceedings of the 31st International Conference on Computational Linguistics*, pages 8535–8545, Abu Dhabi, UAE, January 2025. Association for Computational Linguistics.

[Wang et al., 2023] Yi Wang, Corina Dima, and Steffen Staab. [novel] wikimed-DE: Constructing a silver-standard dataset for german biomedical entity linking using wikipedia and wikidata. In *The 4th Wikidata Workshop*, 2023.

[Wu et al., 2020] Ledell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, and Luke Zettlemoyer. Scalable Zero-shot Entity Linking with Dense Entity Retrieval. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, *Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)*, pages 6397–6407, Online, November 2020. Association for Computational Linguistics.

[Xiao et al., 2023] Zilin Xiao, Ming Gong, Jie Wu, Xingyao Zhang, Linjun Shou, and Daxin Jiang. Instructed Language Models with Retrievers Are Powerful Entity Linkers. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, *Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*, pages 2267–2282, Singapore, December 2023. Association for Computational Linguistics.

[Xin et al., 2024] Amy Xin, Yunjia Qi, Zijun Yao, Fangwei Zhu, Kaisheng Zeng, Xu Bin, Lei Hou, and Juanzi Li. LL-MAEL: Large Language Models are Good Context Augmenters for Entity Linking, July 2024. arXiv:2407.04020 [cs].

[Xu et al., 2007] Hua Xu, Peter D Stetson, and Carol Friedman. A study of abbreviations in clinical notes. In *AMIA annual symposium proceedings*, volume 2007, page 821, 2007.

[Ye and Mitchell, 2025] Christophe Ye and Cassie S. Mitchell. LLM as Entity Disambiguator for Biomedical Entity-Linking. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, *Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)*, pages 301–312, Vienna, Austria, July 2025. Association for Computational Linguistics.

[Yuan et al., 2022a] Hongyi Yuan, Zheng Yuan, and Sheng Yu. Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning. In Marine Carpuat, Marie-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz, editors, *Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pages 4038–4048, Seattle, United States, July 2022. Association for Computational Linguistics.

[Yuan et al., 2022b] Zheng Yuan, Zhengyun Zhao, Haixia Sun, Jiao Li, Fei Wang, and Sheng Yu. CODER: Knowledge-infused cross-lingual medical term embedding for term normalization. *Journal of Biomedical Informatics*, 126:103983, February 2022.

[Zhang et al., 2022] Sheng Zhang, Hao Cheng, Shikhar Vashishth, Cliff Wong, Jinfeng Xiao, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. Knowledge-Rich Self-Supervision for Biomedical Entity Linking. In *Findings of the Association for Computational Linguistics: EMNLP 2022*, pages 868–880, Abu Dhabi, United Arab Emirates, 2022. Association for Computational Linguistics.