Leveraging W ikidata for Biomedical Entity Linking in a Low-Resource Setting: A Case Study for G erman (2024)

Abstract: Biomedical Entity Linking (BEL) is a challenging task for low-resource languages, due to the lack of appropriate resources: datasets, knowledge bases (KBs), and pre-trained models. In this paper, we propose an approach to create a biomedical knowledge base for German BEL using UMLS information from Wikidata, that provides good coverage and can be easily extended to further languages. As a further contribution, we adapt several existing approaches for use in the German BEL setup, and report on their results. The chosen methods include a sparse model using character n-grams, a multilingual biomedical entity linker, and two general-purpose text retrieval models. Our results show that a language-specific KB that provides good coverage leads to most improvement in entity linking performance, irrespective of the used model. The finetuned German BEL model, newly created UMLS _{Wikidata} KB as well as the code to reproduce our results are publicly available.

## introduction bel is the task of disambiguating text spans by linking them to a unique identifier in a biomedical knowledge base @xcite . for instance, the umls @xcite entity having the concept unique identifier (cui) c0007765 is usually mentioned in english under the name cerebellum but can be mentioned in german using either kleinhirn, cerebellum or zerebellum. each entity in the kb has an entity name and one or multiple aliases associated with it, in multiple languages, as shown in fig. 1 . in this work, we refer to such names as entity mentions. the task of biomedical entity linking is to recover the unambiguous entity identifier from a kb given either of the names that can be used to refer to an entity. the task can be performed with context -where 1 german-bio-entity-linking github repository the name is provided together with the surrounding text, or without context -where only the name itself is provided for the disambiguation. in this paper we tackle the problem of bel for entity mentions without context. a wide range of models using ruled-based and deep learning approaches for bel have been proposed for english, for which many data resources are available @xcite . however, the indomain bel datasets, kbs, and models are scarce for low-resource languages. multilingual biomedical models such as sapbert @xcite have been proposed and evaluated on cross-lingual bel benchmarks like xl-bel @xcite . this benchmark, however, is only intended for evaluation purposes, as it includes only 1,000 samples per language. wang et al. (2023) proposed a comprehensive german bel benchmark, wikimed-de-bel, which has, however, not yet been used for evaluating bel models. we adapt several models from the literature to bel on german, and evaluate them on this new benchmark. a problematic aspect when training a bel model for german is the lack of a biomedical kb with entity names and descriptions in german. the unified medical language system (umls) @xcite , the most comprehensive biomedical thesaurus available to date, which is the standard kb in bel for english, only contains 1.6% entities with german descriptions @xcite . we propose a solution to this problem by building a german biomedical kb using umls information harvested from wikidata @xcite , an approach that leads to better entity coverage and can be extended to further languages. ## methodology the first step in the evaluation of each of the selected models is to create vector representations for all kb entities using each model in turn and then store the obtained entity representations in a faiss index @xcite for efficient retrieval. the linking step for all the models involves first creating a vector representation for the entity mention using the selected model and then finding the k nearest neighbors from the kb by comparing the mention vector to the kb vector representations stored in the corresponding faiss index using cosine similarity. mentions are linked to the 5 nearest neighbors for all the models. we further fine-tune the sapbert-umls model 8 , which is already trained on multilingual umls pairs, on umls w ikidata . we use the same procedure as described in section 4 and train for 5 epochs using a batch size of 256. the fine-tuned model is available on hugging face model hub 9 the only hyperparameter of the scispacy model is the size of the character n-grams to be used. we use the 3-grams that appear in 10 or more entities in the target kb. we only use the dense representations from the m3 embedding model. for performance reasons, the maximum sequence length of all the embedding models is set to 40 tokens. ## results the evaluation metric precision@k (p@k) indicates the percentage of samples where the correct entity is found in the top k kb entities predicted by a model. tables 2 and 3 report the p@1 and p@5 obtained by the various models when linking against the umls w ikidata and the umls sapbert , respectively. as a general trend, the sparse, scispacybased n-gram models score lower than the embedding models. the difference is more pronounced when using the umls sapbert kb (in sapbert fine-tuned on umls w ikidata offers good, consistent performance: it performs on par with the jina model when using the umls w ikidata kb (see kb because of its larger size and because it provides better coverage for the german entities in the two evaluation datasets. moreover, the scores for wikimed-de-bel are significantly higher than for xl-bel when using the umls w ikidata kb but the opposite is true when using the umls sapbert kb. the reason for this behaviour is discussed next. ## kb coverage the results obtained for the different dataset/kb combinations are drastically different. the precision is above 0.75 for wikimed-de-bel using the umls w ikidata kb, but below 0.20 when using the umls sapbert kb. if a mention's cui is not present in kb then the model cannot link to it. therefore, we check the upper limit for the metric scores by calculating the dataset coverage for the two kbs. another problematic setup is when the a particular name of an entity or alias is not present in the 205 kb, even its cui is in kb. therefore, we compute the (mention, cui) pair coverage by looking at the percentage of (mention, cui) pairs present in the respective kb. ## conclusion the unavailability of knowledge bases, datasets and, subsequently, models makes bel a challenging task for low-resource languages. to this end, we propose an approach to create a kb for german bel, umls w ikidata , using a methodology that can be easily applied to further low-resource languages. we further compare four different models with various representations and trained on different languages. our results show that creating a dedicated, large-scale knowledge base in the target language leads to the most improvement for doing entity linking in that language, independently of the used model. the best bel results for german are obtained using the language-specific umls w ikidata knowledge base.