DOI: 10.57760/sciencedb.dmh.00547
title: Development and Validation of PhenoRAG: A Visualization Tool for Automated Human Phenotype Ontology Term Annotation Based on Large Language Models and Retrieval-Augmented Generation Technology
title: 基于大语言模型和检索增强生成技术的人类表型本体术语自动注释可视化应用PhenoRAG的开发和验证

description: Objective To develop a user-friendly visualization application for the automatic annotation of Human Phenotype Ontology (HPO) terms based on large language models and retrieval-augmented generation (RAG) technology, and to validate its performance in an authoritative case dataset.Methods Integrating the domestic open-source large language model DeepSeek-V3 with RAG technology, an interactive web application was deployed on the Streamlit cloud platform. Using only the latest official HPO dataset as the data source, the lightweight sentence-embedding model BAAI/bge-small-en-v1.5 was employed to construct a FAISS vector index. During the online phase, a four-step closed-loop process is automatically completed: multilingual translation, phenotype phrase extraction, RAG candidate retrieval, term mapping, and official database validation. 121 English case reports publicly released by BMJ Case Reports and Oxford Medical Case Reports (with a gold-standard HPO set of 1,794 terms) were selected for application validation. Precision, recall, and F1 score were calculated and compared horizontally with traditional dictionary tools, standalone large language models, and the similar application “RAG-HPO.”Results An HPO term automatic annotation visualization application named PhenoRAG, based on large language models and RAG technology, was successfully developed. Users can access it directly via a web link. Across the 112 cases, a total of 2,150 HPO terms were generated; 2,064 (96.0 %) were fully validated by the official database, with a hallucination rate of 1.3 % and an HPO ID–name mismatch rate of 2.7%. After deduplication, 1,906 terms remained for testing. The overall precision was 63.65 %, recall was 67.34%, and F1 was 65.44%, significantly outperforming traditional annotation tools (F1 0.45–0.49). Although PhenoRAG’s F1 was lower than that of RAG-HPO (F1 = 0.78), which relies on a manually constructed synonym database of 54,000 entries plus the HPO dataset, it requires no additional dictionary maintenance and can be used without any background in computer programming.Conclusion Without constructing a synonym database, PhenoRAG achieves high-accuracy automatic mapping from clinical text to standard HPO terms. It features a low usage threshold, free access, and a Chinese-language interface, and can directly serve rare disease diagnosis, genetic counseling, and research scenarios in China and worldwide, warranting further clinical promotion and multicenter validation.
description: Objective To develop a user-friendly visualization application for the automatic annotation of Human Phenotype Ontology (HPO) terms based on large language models and retrieval-augmented generation (RAG) technology, and to validate its performance in an authoritative case dataset.Methods Integrating the domestic open-source large language model DeepSeek-V3 with RAG technology, an interactive web application was deployed on the Streamlit cloud platform. Using only the latest official HPO dataset as the data source, the lightweight sentence-embedding model BAAI/bge-small-en-v1.5 was employed to construct a FAISS vector index. During the online phase, a four-step closed-loop process is automatically completed: multilingual translation, phenotype phrase extraction, RAG candidate retrieval, term mapping, and official database validation. 121 English case reports publicly released by BMJ Case Reports and Oxford Medical Case Reports (with a gold-standard HPO set of 1,794 terms) were selected for application validation. Precision, recall, and F1 score were calculated and compared horizontally with traditional dictionary tools, standalone large language models, and the similar application “RAG-HPO.”Results An HPO term automatic annotation visualization application named PhenoRAG, based on large language models and RAG technology, was successfully developed. Users can access it directly via a web link. Across the 112 cases, a total of 2,150 HPO terms were generated; 2,064 (96.0 %) were fully validated by the official database, with a hallucination rate of 1.3 % and an HPO ID–name mismatch rate of 2.7%. After deduplication, 1,906 terms remained for testing. The overall precision was 63.65 %, recall was 67.34%, and F1 was 65.44%, significantly outperforming traditional annotation tools (F1 0.45–0.49). Although PhenoRAG’s F1 was lower than that of RAG-HPO (F1 = 0.78), which relies on a manually constructed synonym database of 54,000 entries plus the HPO dataset, it requires no additional dictionary maintenance and can be used without any background in computer programming.Conclusion Without constructing a synonym database, PhenoRAG achieves high-accuracy automatic mapping from clinical text to standard HPO terms. It features a low usage threshold, free access, and a Chinese-language interface, and can directly serve rare disease diagnosis, genetic counseling, and research scenarios in China and worldwide, warranting further clinical promotion and multicenter validation.