# Fine-tuning Large Language Models (LLMs) for Entity Alignment in Knowledge Graphs (KG)

Bill Gates Happi Happi, Danai Symeonidou, Géraud Fokou Pelap, Pierre Larmande

<figure><img src="image_1.png" /></figure>

## To cite this version:

Bill Gates Happi Happi, Danai Symeonidou, Géraud Fokou Pelap, Pierre Larmande. Fine-tuning Large Language Models (LLMs) for Entity Alignment in Knowledge Graphs (KG). 2024. ird-04812722v2

HAL Id: ird-04812722

https://ird.hal.science/ird-04812722v2

Preprint submitted on 4 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution - ShareAlike 4.0 International License
# Fine-tuning Large Language Models (LLMs) for
Entity Alignment in Knowledge Graphs (KG)

Bill Gates Happi Happi¹[0000-0003-1199-7726],
Danai symeonidou⁴,
Géraud Fokou Pelap³[0000-0002-4024-1154], and
Pierre Larmande²[0000-0002-2923-9790]

¹ DIADE, University of Montpellier, IRD, CIRAD, Montpellier, France,
[bill.happi@ird.fr](mailto:bill.happi@ird.fr), [pierre.larmande@ird.fr](mailto:pierre.larmande@ird.fr)

² University of Dschang, Dschang, Cameroon, [geraud.fokou@univ-dschang.org](mailto:geraud.fokou@univ-dschang.org)

³ MISTEA, INRAE, Montpellier, France, [danai.symeonidou@inrae.fr](mailto:danai.symeonidou@inrae.fr)

**Abstract.** Searching for similar entities between diverse and heterogeneous data sources in knowledge graphs (KGs) has always been a major challenge. The advent of large language models (LLMs) has opened up new avenues for research and improvement of existing solutions. For example, fine-tuning an LLM was rapidly adopted due to its ability to specialize in specific tasks. This challenge notably involves capturing subtle similarities between the entities concerned, at the linguistic, syntactic, and semantic levels. In this article, we propose an approach to fine-tune GPT-2 and BERT language models to tackle the generalization of entity alignment problems on many different datasets by one single model. We also propose a protocol for entity alignment based on the Kolmogorov Arnold Network (KAN) to address the limitation of interpretability, redundancy, and computational cost of LLMs. The evaluations conducted in this study highlight the significant contribution of the fine-tuned GPT-2 language model, ensuring superior performance on entity alignment challenges compared to BERT and KAN.

**Keywords:** Knowledge Graph · Entity Alignment · RDF · Fine-tuning · Large Language Model.

## 1 Introduction

Entity alignment (EA) is a crucial task for data integration and interoperability. It aims to identify and relate semantically equivalent entities in different knowledge graphs (KGs). Generally, these entities are described as triples (subject-predicate-object) in RDF format, where subject and predicate are associated with uniform resource identifiers (URIs) and the object can be either a URI or a literal value. EA has traditionally been achieved through rule-based methods or manual matching within a single domain dataset. These methods can be tedious, expensive, and lack scalability (zou2017graph). Some modern EA approaches leverage deep learning and embedding techniques on individual datasets (i.e.,
pairs of KGs) to enhance the performances (lu2023review). Recently, we have witnessed an explosion of large language models (LLMs), which offer a promising avenue for many real-life problems. Their explosions owe themselves to the attention mechanisms in transformer architectures initiated by vaswani2017attention. LLMs have been pre-trained on massive corpora of text and code, and are capable of performing complex tasks of natural language processing. LLMs can also automatically understand patterns and semantic relationships from data (fernandez2023large), reducing the need for human intervention. Given the numerous advantages of LLMs in natural language understanding and manipulation, we leverage their ability to interpret entity descriptions in the KG and evaluate their accuracy in the EA process. Many techniques have been developed to adapt LLMs to be efficient for specific tasks such as Prompt Engineering, Reinforcement Learning Human FeedBack, and Fine-tuning (park2023study). Fine-tuning LLMs has gained attention in recent years due to their ability to learn from vast amounts of data and adapt to various tasks by capturing the complex relationships between inputs and outputs (brown2020language). However, the interpretability and computational cost limitations of the LLMs raise concerns about their widespread adoption (tan2024sparsity). On the other hand, a neural network with spline functions such as Kolmogorov-Arnold Network (liu2024kan) (KAN) offers a more transparent and interpretable approach liu2024kan. Spline functions, known for their flexibility in modeling complex mathematical relationships, can be effectively learned and adjusted by neural networks to capture intricate patterns between inputs and outputs. These capabilities of capturing complex relationships provided by LLMs or KAN in data intended to contribute to problem-solving for real applications motivate our intuition to build a single EA solution for datasets from various domains.

This paper investigates the path toward the generalization of the EA prob-
lems by fine-tuning GPT-2 (radford2019language) and BERT (devlin2018bert)
models to generate one single LLM model for EA over different and hetero-
geneous datasets. We opted for the LLM fine-tuning processes despite prompt
engineering caused by the limited context length of the data window size. We
evaluate this approach on eight diverse datasets. First, we fine-tuned and as-
sessed the performance of the models using individual training and testing data
(eight sets). Then, we merged the eight training datasets and fine-tuned the new
models on the combined data. We tested the resulting models on each corre-
sponding test dataset. Finally, we fine-tuned each model (GPT-2 and BERT)
with four training datasets. We tested their generalizability on four other test
datasets, allowing us to assess how well the model generalizes to unseen data.
We also performed the previous steps for the KAN architecture by developing
a from-scratch learning method specifically for KAN on EA problems. In Sec-
tion 2, we present the state of the art in traditional EA approaches and LLMs.
In Section 3, we detail our methodology to prepare data for fine-tuning LLMs
and training the KAN architecture model. In Section 4, we present the results
obtained on 8 datasets of various domains. Section 5 discusses the performance
of existing traditional approaches compared with fine-tuned GPT-2, BERT, and
from-scratch trained KAN models. Our contribution lies in exploring the path
toward the generalization of EA by applying:

- Fine-tuning pre-trained language models GPT-2 and BERT for EA tasks;

- Training a new model of KAN from scratch specifically for EA tasks;

- Evaluation of the generalization of the resulting EA models.

2 Related work

Traditional approaches have addressed the problem of EA using numerous strate-
gies due to its complexity (zeng2021comprehensive). Some EA approaches in-
volve the reduction of candidate entity pairs between source and target KG us-
ing common ontologies (DLinker (happi2022dlinker), Limes (ngonga2021limes),
Logmap (jimenez2011logmap)), followed by the syntactic comparison of literal
objects using similarity measures such as Jaro-Winkler (keil2019efficient), Longest
Common Sub-sequence (bergroth2000survey), N-gram (kim1994fast) to name a
few. In the machine learning domain, it is also common to build the vectors of en-
tities located in KGs (known as an embedding process) (zeng2021comprehensive)
to solve EA problems by exploiting the connections of the neighborhood of
entities in the respective graphs (zhu2021relation). These connections of enti-
ties participate in building strong specific characteristic vectors to each entity,
which are then combined and learned in pairs of positive and negative align-
ments (trisedya2019entity). Approaches based on graph embeddings have been
developed to exploit the deep connections between the entities to match by
learning and capturing structural information to identify similar entities (BERT-
INT (tang2020bert), GCN-Align (wang2018cross)). Current machine learning
approaches are founded on Multi-Layer Perceptrons (MLP) (cybenko1989approximation)
to capture complex relationships (shoeybi2019megatron). Recently, the Kolmogorov-
Arnold Network (liu2024kan), a new machine learning algorithm similar to MLP,
has shown high performance on small datasets. Unlike MLP which adjusts weights
with nonlinear activation functions from inputs from previous layers, the KAN
directly adjusts the parameters of spline functions, allowing it to perform well in
capturing non-linear relationships between the data. The meteoric rise of LLMs
has revolutionized natural language processing, enabling the generation of varied
digital content. Their remarkable performance in capturing subtle information,
sometimes invisible to humans, arouses admiration. The story around LLMs
started from transformer architectures from the work of vaswani2017attention.
These transformers contributed to the process of natural language and special-
ized models for specific tasks. The GPT-2 (radford2019language) model built
from transformer architecture aims to learn how to predict the next suitable
token for a sentence. GPT-2 has 1.5 billion parameters in its initial version and
was trained on millions of Web pages from the WebText corpus. Although it
was designed for seven specific tasks, it achieved results over the state of the art
at the time. BERT (devlin2018bert) uniquely tackles language understanding
by learning how to predict the masked token in a sequence. BERT starts by
learning from massive amounts of unlabeled text and analyzing the meaning of
words based on both the words before and after them (bidirectional) to create
a powerful foundation (pre-trained) model. This foundation can then be easily
adapted (fine-tuned) to perform specific tasks like classification after under-
standing relationships between words. Recently, LLM4OM (giglou2024llms4om)
leverages LLM prompting to achieve ontology matching. Our approach uses a
different strategy deep-seated on fine-tuning language models or incorporating
spline function learning for EA. These approaches have not yet been widely ex-
plored. We will delve deeper into this new direction in the following sections.
In the section below we detail our contribution to EA seated on fine-tuning the
GPT-2 model, BERT, and from-scratch training of the KAN architecture.

3 Methodology

<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Overview of the pipeline for entity alignment. NS denotes negative sampling, PS denotes positive sampling, op denotes optional, FT denotes Fine-tuning, N, and M represent the count of different domains to learn. The tokenization is required for KAN during the data labeling step. The "Building" layer aims to update the model weights and train KAN from new training data.</figcaption></figure>

This study tackles the challenges of generalizing EA models to unseen data. We investigate the impact of training data and diversity on the generalizability of EA models. We aim to develop strategies for training EA models that can perform well on a wide range of knowledge bases, even those with limited or noisy data. Figure 1 shows the entire process of the entity alignment task. To ensure the path towards the generalization, we formalize the entity alignment problem in Section 3.1. This formalization will be based on the knowledge graph structure commonly used in the semantic web paradigm. We perform the "Data Splitting" step on N different domain datasets to extract the valid alignments described in the knowledge graphs *KG1* and *KG2* to generate the test and training datasets with positive and negative sampling (PS or NS). Next, in
the "Data Labeling" step we format the textual inputs of the generated alignments (positives or negatives) with the binary output (1 or 0) for the models, which will serve as correspondences to be learned to fine-tune models (GPT-2 and BERT-BASE) during the "Building" step, before the inference on N+M domains where M is the count of unseen domains. We also propose a protocol for transforming textual inputs into reduced-dimensional vectors to evaluate the KAN architecture. This protocol uses a tokenizer coupled to the Principal Component Analysis (PCA) (greenacre2022principal) dimensionality reduction algorithm. For the sake of clarity, we formalize our contribution for a single dataset only, in the following section.

## 3.1 Preliminary

Consider an RDF data source (a graph) as a set of triplets defined by: $G = \{(s, p, o) \in (R \cup B) \times (R) \times (R \cup B \cup L)\}$, where R is the set of all resources in the form of IRIs (Internationalized Resource Identifier), B is the set of blank nodes, and L is the set of literals. We denote the set of RDF data graphs as D. Let S(G), P(G), and O(G) be the respective sets of subjects, predicates, and objects of a given dataset G. Let $\mathbb{N}$ be the set of natural numbers, including 0. Consider all the next numerical variables (belonging to $\mathbb{N}$), which may or may not depend on the expression of the function they define and the application context. Let $KG_1 = \{(s_i, p_k, o_k^i); 0 \le i < m, 0 \le k < k_{max}^i\} \in D$ be the set of source entities containing $m$ triplets and $k_{max}^i$ represents the maximum possible index of the object of the $i$-th subject, and $KG_2 = \{(t_j, p'_l, o_l^j); 0 \le j < n, 0 \le l < l_{max}^j\} \in D$ be the set of target entities containing $n$ triplets and $l_{max}^j$ represents the maximum possible index of the object of the $j$-th subject. The objects are not necessarily ordered. We declare $A_p = \text{owl:sameAs}$ as being the alignment predicate. We also consider the set $V_a = \{(s_{i'}, A_p, t_{j'})_c; \forall i', j' \in \mathbb{N}, s_{i'} \in S(KG_1), t_{j'} \in S(KG_2) \text{ with } KG_1 \text{ and } KG_2 \in D; 0 \le c < c_{max}\}$, where $s_{i'}$ and $t_{j'}$ are any alignable entities from their respective graphs. $V_a$ is the set of valid entity alignments containing fundamental valid ($V_a \subset D$ and $A_p$ is the entity linking predicate).

## 3.2 Data Sampling: training and testing

The valid data sampling process aims to generate sets of positive and negative alignments of entities for training and testing (PS and NS). This crucial step consists of extracting 70% of the alignments from the valid set for training, and the remaining 30% for testing. We first extract the 70% positive alignments (or PS) for training. Next, we generate an equivalent number of negative alignments (or NS) by randomly creating feature pairs between the source and target features, while ensuring that these pairs do not appear in the set of positive alignments. Finally, we merge the generated positive and negative alignment sets to obtain the complete training set. Similarly, we extract 30% remaining positive alignments for the test from the valid set. Then, we generate an equivalent number of negative alignments following the same procedure as for training. Merging the
generated positive and negative alignment sets results in the final test set. $\overline{A}_p$ means false alignments and $Rd(E)$ randomly select an element in $E$. The above description can be expressed mathematically by the following expressions:

1. $V_a = \{(s_{i'}, A_p, t_{j'})_c; \forall i', j', c \in \mathbb{N}, s_{i'} \in S(KG_1), t_{j'} \in S(KG_2); 0 \le c < c_{max}\}$;

2. $V_a^{30} = \{(s_{i'}, A_p, t_{j'})_{d_1}; \forall i', j', d_1 \in \mathbb{N}, s_{i'} \in S(V_a), t_{j'} \in O(V_a), 0 \le d_1 < c_1; c_1 = 0.3 \times c_{max}\}$;

3. $V_a^{70} = \{(s_{i'}, A_p, t_{j'})_{d_2}; \forall i', j', d_2 \in \mathbb{N}, s_{i'} \in S(V_a), t_{j'} \in O(V_a), 0 \le d_2 < c_2; c_2 = 0.7 \times c_{max}\}$;

4. $D_{test} = V_a^{30} \cup \{(s_i, \overline{A}_p, t_j)_{d_1}; \forall i, d_1 \in \mathbb{N}, s_i \in S(V_a^{30}), \exists j \in \mathbb{N}/t_j = Rd(O(V_a^{30})), 0 \le d_1 < c_1; c_1 = 0.3 \times c_{max}, (s_i, A_p, t_j) \notin V_a^{30}\}$;

5. $D_{train} = V_a^{70} \cup \{(s_i, \overline{A}_p, t_j)_{d_2}; \forall i, d_2 \in \mathbb{N}, s_i \in S(V_a^{70}), \exists j \in \mathbb{N}/t_j = Rd(O(V_a^{70})), 0 \le d_2 < c_2; c_2 = 0.7 \times c_{max}, (s_i, A_p, t_j) \notin V_a^{70}\}$.

## 3.3 Formatting Data Input and Output for the Model

**For LLMs** For optimal performance in language models, we aim to preserve the natural structure of input text, closely resembling how humans perceive it. This involves minimizing noise around relevant information and avoiding excessively long or truncated token sequences. By doing so, we can effectively build and fine-tune efficient language models. Given the similar entities or not $s_i$ and $t_j$ which are set of triples $KG_1 = \{(s_i, p_k, o_k^i); 0 \le k < k_{max}^i\}$ and $KG_2 = \{(t_j, p'_k, o_k^j); 0 \le k < k_{max}^j\}$. For $s_i$, use $Concat(s_i)$ will return the following sequence of predicate-objects not necessarily ordered:

"$s_i : s_i \ p_0 \ o_0^i; \ p_2 \ o_2^i; \ p_1 \ o_1^i; \ \dots \ p_{k_{max}^i-1} \ o_{k_{max}^i-1}^i$".

To emphasize the subject entity in the concatenated representation, prepending $s_i$: to the entity string concatenation emphasizes the representation of the subject. To label the data for the LLM, we associated $Concat(s_i) + Concat(t_j)$ with the output 0 to mean negative alignment or 1 to positive alignment. The tokenizer of each LLM will convert the concatenated input sequences to a matrix to adjust the model's weights during the fine-tuning process.

**For KAN architecture** Consider $s_i$ and $t_j$ formatted in $F = Concat(s_i) + Concat(t_j)$. $F$ is transformed into a numerical vector denoted by $v$ of fixed size $d$. $v$ can be padded to adapt the size of the inputs to the dimension expected by KAN. Then we apply the normalization function defined below (eq.2) to bring the values of $v$ into a value range suitable for the neural networks, like for KAN architecture. The tokenizer converts text into numerical representations and can create high-dimensional vectors (large and complex) from input strings ($I$ made of limited elements of printable symbols $V$). This tokenizer computes the unit vector for each symbol in $I$ and sums the vectors of the symbols present in $I$ to obtain the overall vector for $I$. These high-dimensional vectors (denoted by $h$ and belonging to the set of natural numbers $\mathbb{N}$) can make the classification process challenging for the KAN architecture. To address the challenge
of high-dimensional inputs, we employ a technique called Principal Component Analysis (PCA) (greenacre2022principal). PCA helps us reduce the complexity of the input data by transforming it into a lower-dimensional representation (denoted by $d_f$). This lower-dimensional representation captures the most important information from the original data, making it easier for KAN to perform the classification task.

$$ (1) \quad T_k : V^* \to \mathbb{N}^h \quad | \quad (2) \quad N : R^d \to R^d \quad | \quad (3) \quad V_s : R^d \to R^{d_f} \\ I \to Tokenizer(I) = [t_{k_0}, \dots, t_{k_{h-1}}], (t_{k_*} \in \mathbb{N}) \quad | \quad x \mapsto \frac{x-\min(x)}{\max(x)-\min(x)} \quad | \quad x \mapsto PCA(N(x), d_f) $$

Mathematically, we can express the flow of the transformation of F, by applying the following steps :

1. $v = T_k(F)$;

2. Input vector ($V_s(v)$) is associated with the output 0 (for NS) or 1 (for PS).

# 4 Experiments

To substantiate the claims made throughout this paper, we present the results of extensive experiments that showcase the efficacy of our path toward the generalization of the entity alignment approach. We chose 2 LLMs due to their light size and our calculation capabilities namely GPT-2 (124 Million parameters) and BERT-BASE (340 Million parameters) available at the Hugging Face<fnref n="4" /> platform. Additionally, we implemented KAN in Python programming language, to perform the classification process. The set of printable symbols comes from the Python library "strings". The experiments of classifications-based fine-tuning were performed on a computer of 16 processors with 32 GB of RAM, and 7.5 GB of GPU(1) RAM, the model name is 11th Gen Intel(R) Core(TM) i9-11950H clocked at 2.60GHz. The computer was under Ubuntu 20.04.4 LTS on kernel Linux 5.14.0-1059-oem with x86-64 architecture. The OAEI<fnref n="5" /> (Ontology Alignment Evaluation Initiative) community consistently offers challenges in instance matching in RDF format. We chose various domains of datasets from IM@OAEI (2010, 2011, 2016, 2019), each with unique complexity and specific characteristics.

## 4.1 Fine-tuning hyper-parameters

First, we use a small pre-trained "gpt2" model. GPT-2 is fine-tuned for 3 epochs on the datasets where 70% of alignments are used for training and 30% for testing. We choose a learning rate $lr=5e-5$, batch size=1 (1 to avoid the limited

<footnote n="4">⁴https://huggingface.co/</footnote>

<footnote n="5">⁵https://oaei.ontologymatching.org/</footnote>
memory issue), and gradient accumulation steps at 1. We use AdamW optimizer (kingma2014adam) with $\beta1 = 0.9$ and $\beta2 = 0.999$, $\epsilon = 10^{-8}$, weight decay=0.0. Second, we use a pre-trained "google/bert-base" model. BERT-BASE is fine-tuned for 2 epochs on splitting described above. We choose a learning rate $lr=2e-5$, batch size=1, and gradient accumulation steps at 4. We use AdamW optimizer with $\beta1 = 0.9$ and $\beta2 = 0.999$, $\epsilon = 10^{-6}$, weight decay=0.01 combined to the binary cross entropy loss function. Third, we have defined a KAN model composed of three layers of neurons: the input layer which has several neurons equal to the dimension of the input vectors $d$ (d=20); the hidden layer which contains $d + 1$ neurons and the final output layer which has only one neuron. The size of the interval grid is set to 10 and the maximum degree of the polynomial k of the splines is 3. While training, we used the LBFGS (ge2018parameter) optimization function over 20 iterations. After the training steps, we perform evaluation metrics to appreciate the impact of the process. The overall evaluation time is estimated at more than 24 hours.

## 4.2 Results

Experiments have been conducted to fine-tune several models either on individual or merged datasets. Results are shown in tables 1 and 2. Overall, GPT-2 fine-tuned specifically on individual datasets achieves high average precision but low F1-score. GPT-2 accuracy is increased for larger datasets when they are merged. Furthermore, it outperforms the BERT and KAN models.

**Fine-tuning on individual dataset** In Table 1, GPT-2 obtains a high precision but a low recall resulting in a low F1-score. This means the model can discriminate between positive and negative alignments but fails to find them all. However, the model perfectly identifies positive and negative alignments in the Person dataset. BERT also shows issues with the Restaurant and SL19 datasets leading to the F1-score valued at 0.0. Moreover, BERT performs less than GPT-2 on the Yago-Wikidata, Restaurant, SS19, and SL19 datasets. However, BERT does better than GPT-2 on the Doremus and Anatomy datasets. KAN has stable and high metric performances on three datasets, neither excelling nor failing in any specific alignment in the datasets but does more than others on NLL-DB, Yago-WIKI, Restaurant, and SL19.

**Fine-tuning on merged datasets** In Table 2, GPT-2 generally achieves the best F1-scores, indicating good overall performance compared to other models. GPT-2 excels in the Person, Restaurant, and SS19 datasets with F1-scores above 0.98. The Nell-DBPedia and Yago-Wikidata datasets, both part of the commonKG track, were previously evaluated by LLM4OM (giglou2024llms4om) achieving F1-scores of 0.9426 and 0.9219 respectively. These datasets yielded different results in our evaluation, with F1-scores of 0.927 for Nell-DBPedia and 0.955 for Yago-Wikidata. However, its performance decreased for Doremus and Anatomy. BERT shows poor performance, with F1-scores around 0.6 except
<table><caption>**Table 1.** Comparison of the models on individual fine-tuning. The underlined values (respectively in bold) are the best precision (respectively F for f1-score) associated with the models offering a better performance on each dataset.</caption><thead><tr><th>Models</th><th>Metrics</th><th>NLL-DB</th><th>Yago-Wiki</th><th>Person</th><th>Rest.</th><th>Doremus</th><th>Anatomy</th><th>SS19</th><th>SL19</th></tr></thead><tbody><tr><td rowspan="3">GPT-2</td><td>P</td><td>1.0</td><td>0.972</td><td><u>1.0</u></td><td><u>1.0</u></td><td><u>1.0</u></td><td><u>1.0</u></td><td><u>1.0</u></td><td>0.0</td></tr><tr><td>R</td><td>0.027</td><td>0.384</td><td>1.0</td><td>0.045</td><td>0.014</td><td>0.303</td><td>0.956</td><td>0.0</td></tr><tr><td>F</td><td>0.052</td><td>0.551</td><td><b>1.0</b></td><td>0.086</td><td>0.027</td><td>0.466</td><td><b>0.978</b></td><td>0.0</td></tr><tr><td rowspan="3">BERT</td><td>P</td><td>0.5</td><td>0.0</td><td><u>1.0</u></td><td>0.0</td><td><u>1.0</u></td><td><u>1.0</u></td><td><u>1.0</u></td><td>0.0</td></tr><tr><td>R</td><td>1.0</td><td>0.0</td><td>0.986</td><td>0.0</td><td>0.492</td><td>0.964</td><td>0.634</td><td>0.0</td></tr><tr><td>F</td><td>0.66</td><td>0.0</td><td>0.993</td><td>0.0</td><td><b>0.660</b></td><td><b>0.982</b></td><td>0.776</td><td>0.0</td></tr><tr><td rowspan="3">KAN</td><td>P</td><td>0.736</td><td>0.802</td><td>0.853</td><td>0.816</td><td>0.570</td><td>0.616</td><td>0.794</td><td><u>0.787</u></td></tr><tr><td>R</td><td>0.736</td><td>0.802</td><td>0.853</td><td>0.816</td><td>0.570</td><td>0.616</td><td>0.794</td><td>0.787</td></tr><tr><td>F</td><td><b>0.736</b></td><td><b>0.802</b></td><td>0.853</td><td><b>0.816</b></td><td>0.570</td><td>0.616</td><td>0.794</td><td><b>0.787</b></td></tr></tbody></table>

for the Doremus category where it completely fails (F1-score=0). Moreover, the BERT results having a precision valued at 0.500 and a recall valued at 1.0 show that it has difficulty distinguishing between positive and negative alignments. KAN obtains stable F1-scores around 0.5 for almost all the datasets. This indicates that it is not good at distinguishing between positive and negative alignments but does not exhibit blatant bias towards any dataset.

<table><caption>**Table 2.** Comparison of architectures on merged train samples of datasets and individual evaluation on each test sample. The underlined values (respectively in bold) are the best precision (respectively F for f1-score) associated with the models offering a better performance on each dataset.</caption><thead><tr><th>Models</th><th>Metrics</th><th>NLL-DB</th><th>Yago-Wiki</th><th>Person</th><th>Restaurant</th><th>Doremus</th><th>Anatomy</th><th>SS19</th><th>SL19</th></tr></thead><tbody><tr><td rowspan="3">GPT-2</td><td>P</td><td><u>1.0</u></td><td><u>0.977</u></td><td><u>1.0</u></td><td><u>1.0</u></td><td><u>0.746</u></td><td><u>0.710</u></td><td><u>1.0</u></td><td><u>0.963</u></td></tr><tr><td>R</td><td>0.864</td><td>0.934</td><td>0.993</td><td>0.966</td><td>0.704</td><td>0.605</td><td>0.965</td><td>0.931</td></tr><tr><td>F</td><td><b>0.927</b></td><td><b>0.955</b></td><td><b>0.996</b></td><td><b>0.983</b></td><td><b>0.724</b></td><td><b>0.653</b></td><td><b>0.982</b></td><td><b>0.947</b></td></tr><tr><td rowspan="3">BERT</td><td>P</td><td>0.500</td><td>0.500</td><td>0.500</td><td>0.500</td><td>0.500</td><td>0.0</td><td>0.483</td><td>0.502</td></tr><tr><td>R</td><td>1.0</td><td>1.0</td><td>1.0</td><td>1.0</td><td>1.0</td><td>0.0</td><td>1.0</td><td>1.0</td></tr><tr><td>F</td><td>0.666</td><td>0.666</td><td>0.666</td><td>0.666</td><td>0.666</td><td>0.0</td><td>0.651</td><td>0.669</td></tr><tr><td rowspan="3">KAN</td><td>P</td><td>0.526</td><td>0.598</td><td>0.536</td><td>0.533</td><td>0.570</td><td>0.527</td><td>0.605</td><td>0.579</td></tr><tr><td>R</td><td>0.526</td><td>0.598</td><td>0.536</td><td>0.533</td><td>0.570</td><td>0.527</td><td>0.605</td><td>0.579</td></tr><tr><td>F</td><td>0.526</td><td>0.598</td><td>0.536</td><td>0.533</td><td>0.570</td><td>0.527</td><td>0.605</td><td>0.579</td></tr></tbody></table>
# 5 Discussion

In Table 1, the low values of recall (and hence the F1-score) of the GPT-2 model could be due to a lack of regular or expected tokens and contexts in the training data. This lack of diversity in the data might not allow the model parameters to discriminate sufficiently during fine-tuning (wolfe2021low, mosbach2020stability). We observe overfitting of the model based on the BERT architecture (zhang2020revisiting), which results in a drop in precision and a jump in recall. This indicates that BERT has learned to identify positive alignments correctly but lacks precision in distinguishing positive from negative ones. Hence BERT is focused so much on the training data that it can no longer distinguish the subtleties of the new data. The model based on KAN architecture also presents an overfitting phenomenon (genet2024tkan), when evaluating the entire datasets. Indeed, KAN has difficulty distinguishing positive from negative alignments, despite a precision close to 1.0 when training on small data and 0.5 on large data. This could be explained by the input dimensionality reduction algorithm (PCA). KAN is accurate for low-dimensional spline functions and motivates the exploitation of PCA to reduce the input size (liu2024kan). In general, both GPT-2 and BERT-BASE struggled on the Doremus dataset. This could be attributed to the dataset being in French. The Doremus dataset, containing French text (e.g., factual statements about objects: "Created in Vienna on November 5, 1920, by Maria Olszewka and the composer"), yielded low individual F1-scores for the models, with GPT-2 scoring only 0.027. In Table 2, the merged evaluation for GPT-2 shows improved performance for Doremus dataset (0.724), which can be explained by a wide range of training data that allows it to adjust model weights and capture complex relationships, resulting in better performance. The results indicate low performance in the Anatomy dataset due to several factors. The Anatomy dataset is a medical dataset containing very short-length literal values data (e.g. of literal object: "visceral organ system") and, lacks extended context for better adjustment of model weights (entity described with less than 3 triples unlike everyone else). This lack of an anatomy dataset could be decisive for poor performance in individual and merged evaluations. Another reason is the scattering of training and testing data tokens (small groups of characters of variable size) of the model input values on each dataset. For the BERT model, this scattering and the data characteristics can lead to overfitting, as indicated by the metrics (0.0), and explain the difficulty distinguishing good alignments from bad ones. Hence the advantage of the GPT-2 model over others in the metrics of Table 2.

*Evaluation of our generalization on unseen datasets.* To evaluate the generalization of our models, we fine-tuned them on four datasets (Anatomy, Doremus, SS19, and SL19) and tested them on four others (Person, Restaurant, Yago-Wikidata, and Nell-DBPedia). After evaluation of the four unseen datasets, the results reported in Table 3, show that GPT-2 generalizes and does better than ex-
isting approaches (RIMOM (achichi2016results), Logmap<sup>6,7</sup>, AML (achichi2016results), Lily<sup>8</sup>). We reported the running time for each model in Table 4. GPT-2 takes more time than others to perform well on the generalization evaluations.

<table><caption>**Table 3.** Comparison between SOTA and our fine-tuned models on four datasets and predicted on four unseen datasets (Person and Restaurant). The values in the Table are the F1-score and the best tools are highlighted in bold. "A": absent on training data.</caption><thead><tr><th colspan="2">Dataset-Tool</th><th>RIMOM</th><th>Logmap</th><th>AML</th><th>Lily</th><th>GPT-2</th><th>BERT</th><th>KAN</th></tr></thead><tbody><tr><td colspan="2">Doremus</td><td>0.813</td><td>-</td><td><strong>0.862</strong></td><td>-</td><td>0.751</td><td>0.0</td><td>0.535</td></tr><tr><td colspan="2">Anat.</td><td>-</td><td>0.881</td><td>0.943</td><td>0.83</td><td><strong>0.962</strong></td><td>0.0</td><td>0.560</td></tr><tr><td colspan="2">SS19</td><td><strong>0.992</strong></td><td>0.841</td><td>0.864</td><td>0.991</td><td>0.952</td><td>0.0</td><td>0.55</td></tr><tr><td colspan="2">SL19</td><td><strong>0.995</strong></td><td>0.785</td><td>0.86</td><td><strong>0.995</strong></td><td>0.940</td><td>0.0</td><td>0.571</td></tr><tr><td colspan="2">Y-W (A)</td><td>-</td><td><strong>0.86</strong></td><td>0.0</td><td>-</td><td>0.66</td><td>0.0</td><td>0.615</td></tr><tr><td colspan="2">N-D (A)</td><td>-</td><td><strong>0.88</strong></td><td>0.0</td><td>-</td><td>0.68</td><td>0.0</td><td>0.697</td></tr><tr><td colspan="2">Person (A)</td><td>0.97</td><td>-</td><td>-</td><td>-</td><td><strong>0.990</strong></td><td>0.0</td><td>0.510</td></tr><tr><td colspan="2">Rest. (A)</td><td>0.81</td><td>-</td><td>-</td><td>-</td><td><strong>0.967</strong></td><td>0.0</td><td>0.5</td></tr></tbody></table>

<table><caption>**Table 4.** Running time (seconds) of models during the tests on the datasets. The less time-consuming model is in bold.</caption><thead><tr><th colspan="2">Dataset-Tool</th><th>NLL-DB</th><th>Yago-Wiki</th><th>Person</th><th>Restaurant</th><th>Doremus</th><th>Anatomy</th><th>SS19</th><th>SL19</th></tr></thead><tbody><tr><td colspan="2">GPT-2</td><td>8.31</td><td>14.16</td><td>22.09</td><td>6.93</td><td>8.45</td><td>58.87</td><td>13.25</td><td>57.64</td></tr><tr><td colspan="2">BERT</td><td>5.98</td><td>11.93</td><td>10.52</td><td>4.83</td><td>2.05</td><td>19.42</td><td>3.32</td><td>18.85</td></tr><tr><td colspan="2">KAN</td><td><strong>0.108</strong></td><td><strong>0.132</strong></td><td><strong>0.208</strong></td><td><strong>0.113</strong></td><td><strong>0.129</strong></td><td><strong>0.211</strong></td><td><strong>0.129</strong></td><td><strong>0.169</strong></td></tr></tbody></table>

## 6 Conclusion

This research investigated the potential of pre-trained and from-scratch-trained models for generalizing the EA task between RDF graphs. We explored two approaches: fine-tuning a pre-trained model (GPT-2 and BERT) and evaluating a specifically designed protocol for a new architecture (KAN) for this task. Our findings demonstrate that fine-tuning a large pre-trained model like GPT-2 with substantial training data leads to superior performance on the EA task. This

<footnote n="6">⁶https://hobbit-project.github.io/OAEI_2022.html</footnote>

<footnote n="7">⁷https://oaei.ontologymatching.org/2020/results/anatomy/index.html</footnote>

<footnote n="8">⁸https://hobbit-project.github.io/OAEI_2021.html</footnote>
improvement holds true not only for data from the training domain but also
for data from unseen domains. Conversely, BERT and KAN models exhibited
lower effectiveness, particularly when dealing with large datasets or data outside
the training domain. Building on these insights, our future work will focus on
leveraging pre-trained models like GPT-2 to develop more robust methods for
identifying different types of EA problems within datasets. We also aim to create
a reinforcement learning pipeline that utilizes unseen data to further enhance
the model's ability to handle novel alignment scenarios. Our code is available at
this url<sup>9</sup>.

7 Funding

This work was supported in part by the French National Research Institute for
Sustainable Development (IRD) and French National Research (ANR) through
the DIG-AI (ANR-22-CE23-0012) project.

8 Acknowledgments

This work was granted access to the HPC resources of IDRIS under the allocation
2024-A0160315119 made by GENCI. The authors thank the IRD supercomput-
ing infrastructure, I-Trop, and its staff for their support. The authors thank the
South Green Bioinformatics Platform for its IT support.

References

<footnote n="9">⁹https://github.com/BillGates98/Fine-Tuning-LLMs-4-EA-in-KG</footnote>