REAL : A Retrieval-Augmented Entity Linking Approach for Biomedical Concept Recognition (2024)

Abstract: Large Language Models (LLMs) offer an appealing alternative to training dedicated models for many Natural Language Processing (NLP) tasks. However, outdated knowledge and hallucination issues can be major obstacles in their application in knowledge-intensive biomedical scenarios. In this study, we consider the task of biomedical concept recognition (CR) from unstructured scientific literature and explore the use of Retrieval Augmented Generation (RAG) to improve accuracy and reliability of the LLM-based biomedical CR. Our approach, named REAL (Retrieval Augmented Entity Linking), combines the generative capabilities of LLMs with curated knowledge bases to automatically annotate natural language texts with concepts from bio-ontologies. By applying REAL to benchmark corpora on phenotype concept recognition, we show its effectiveness in improving LLM-based CR performance. This research highlights the potential of combining LLMs with external knowledge sources to advance biomedical text processing.

## introduction biomedical concept recognition (cr) aims to identify and link textual mentions of biomedical concepts to entries in expert-curated knowledge bases and ontologies. cr combines two subtasks from the standard information extraction pipeline: entity recognition (ner) and entity linking (el), sometimes referred to as named entity disambiguation (ned) or grounding. ner aims to detect strings in text that refer to classes of biomedical entities, such as phenotypes, diseases, or genes. el maps those strings to terms in an ontology, such as human phenotype ontology (hpo) @xcite for phenotypic features, mondo @xcite for disease terms, and hgnc @xcite for human genes. automated cr methods represent an active research area and are essential for a range of downstream biomedical applications. in genomic medicine, for instance, accurately recognizing phenotype concepts from freetext medical notes is the starting point to improve genetic disease diagnostics @xcite . state-of-the-art cr systems rely on fine-tuning transformer-based language models pretrained on biomedical texts, such as biobert @xcite , and have restricted scope, targeting a single or few application domains @xcite . the major limitation of these approaches is the need for domain-specific training with expert-labeled corpora, which is not always feasible due to the scarsity of annotated data in the biomedical field @xcite . on the other hand, general-purpose large language models (llm), such as openai's generative-pretrained transformer (gpt), have demonstrated remarkable zero and few-shot learning abilities, offering significant potential for biomedical nlp. recent studies have shown promising results when using llms in clinical information extraction without domainspecific training @xcite . however, challenges persist regarding factual accuracy in generated responses, hindering their usability for knowledge-intensive tasks in specialized domains @xcite . to address these challenges, retrieval-augmented-generation (rag) @xcite has been recently proposed as a technique to enhance llms with relevant information retrieved from external knowledge bases through semantic similarity calculation. our study aims to explore the application of the rag paradigm in the context of biomedical cr. to this end, we developed real, a retrieval-augmented entity linking approach for ontologybased cr. to overcome the limitation of training dedicated ner and el models, our approach lever-ages prompting techniques with general purpose llms to handle both tasks in a unified pipeline. given a text, real first identifies mentions of concepts belonging to some target biomedical domain through a zero-shot ner, and then associates these mentions to terms in the domain ontology using retrieval-enhanced entity linking. by embedding the mention and ontology concepts into a common dense space, the retrieval mechanism provides the llm with a selection of candidates from a bio-ontology identified through nearest neighbor search. by synergistically combining the retrieval mechanism with prompt-engineering, real aims to leverage up-to-date knowledge with existing knowledge bases, thereby improving the accuracy and reliability of llm-based cr. we summarize our contributions as follows: our approach is versatile and can be easily adopted in various application domains without requiring domain-specific training. ## methodology the ontology-based cr problem can be formally presented as two consecutive tasks, ner and el, as follows. let o denote a set of concepts {c 1 , . . . , c n } defined in the domain ontology used for text annotation. given a text t , the ner task identifies textual mentions of biomedical entities from the target domain, m 1 , . . . , m h . then, the el task consists in assigning each entity mention m i to a concept @xmath0. as shown in figure 1 , real implements cr in a pipeline consisting of three main phases: ontology indexing, ner, and retrieval-enhanced entity linking. the ontology indexing is executed only once during the pre-processing to convert the concepts in o into a searchable index. the main workflow starts with the zero-shot ner (in the left bottom of the figure), where we prompt the llm to extract instances of a specified entity type from t and generate a short definition for each of them. in the retrieval-enhanced entity linking phase (right bottom part of the figure), we search in the ontology index the top-k most similar concepts to the embedding of m. they are the candidates for entity linking and the best matching is identified by properly instructing an llm prompt. details of the approach are provided in the remainder. ## concluding remarks in this work, we introduced a novel approach for ontology-based concept recognition, that leverages rag to harness general-purpose llms for automatic annotation of biomedical texts with classes from domain ontologies. the approach does not require domain specific training, but relies on promptengineering for both ner and el tasks, integrating a retrieval mechanism to dynamically source domain knowledge from biomedical ontologies. we discussed the effectiveness of our approach on clinical phenotyping and phenotype annotation with experiments conducted on hpo gsc+ and bioc gs benchmark corpora. ongoing efforts focus on refining the prompt design to enhance performance and consider the integration with other genai providers. using gpt models through ope-nai's api hinders the reproducibility of the results, which represents the main limitation of the current implementation. we plan to address this issue using a local installation of open-source llms. furthermore, future research activities include conducting a comprehensive cross-domain evaluation to assess the generalizability of the proposed solution to diverse application domains.