Automatic Glossary of Clinical Terminology: a Large-Scale Dictionary of Biomedical Definitions Generated from Ontological Knowledge (2023)

Abstract: Background: More than 400.000 biomedical concepts and some of their relationships are contained in SnomedCT, a comprehensive biomedical ontology. However, their concept names are not always readily interpretable by non-experts, or patients looking at their own electronic health records (EHR). Clear definitions or descriptions in understandable language or often not available. Therefore, generating human-readable definitions for biomedical concepts might help make the information they encode more accessible and understandable to a wider public. Objective: In this article, we introduce the Automatic Glossary of Clinical Terminology (AGCT), a large-scale biomedical dictionary of clinical concepts generated using high-quality information extracted from the biomedical knowledge contained in SnomedCT.Methods: We generate a novel definition for every SnomedCT concept, after prompting the OpenAI Turbo model, a variant of GPT 3.5, using a high-quality verbalization of the SnomedCT relationships of the to-be-defined concept. A significant subset of the generated definitions was subsequently evaluated by NLP researchers with biomedical expertise on 5-point scales along the following three axes: factuality, insight, and fluency. Results: AGCT contains 422,070 computer-generated definitions for SnomedCT concepts, covering various domains such as diseases, procedures, drugs, and anatomy. The average length of the definitions is 49 words. The definitions were assigned average scores of over 4.5 out of 5 on all three axes, indicating a majority of factual, insightful, and fluent definitions. Conclusion: AGCT is a novel and valuable resource for biomedical tasks that require human-readable definitions for SnomedCT concepts. It can also serve as a base for developing robust biomedical retrieval models or other applications that leverage natural language understanding of biomedical knowledge.

## introduction to unlock the value of in-hospital data while preserving patients' right to privacy, federated learning might become a corner stone for retrospective studies in the healthcare domain @xcite . inter-hospital data interoperability is, however, one of the pre-requirements of federated learning @xcite and is getting attention. this has resulted in a stronger appetite for more structured and more standardized coding of patient journeys through the medical services @xcite ). an issue with these standardized codes from ontologies is their usage of a highly-specialized terminology @xcite . this diminishes their suitability for non-experts or patients wishing to consult their personal clinical data. efforts to produce simpler-to-understand definitions of biomedical concepts have been welldocumented (e.g. by mayo clinic) but they are usually of limited scope due to the time and cost involved in their creation, requiring heavy prioritization of the efforts @xcite . early efforts to bridge the gap between ontologies and textual definitions by @xcite and @xcite remained insufficient. but in recent years, the fluency of text generated using large language models has reached extremely high levels @xcite and so has their ability to convert graph-level information into textual descriptions @xcite . in this study, we set out to investigate the suitability of commercially-available language models to generate at scale medically-accurate descriptions of clinical concepts in 2023. to this end, we introduce the automatic glossary of clinical terminology (agct), a large-scale biomedical dictionary containing more than 400,000 biomedical concept definitions generated using gpt-3.5 @xcite on the basis of the biomedical knowledge contained in the snomedct ontology. ## methodology each definition was obtained after prompting the gpt-3.5 model with a long prompt containing a summarized verbalization of the snomedct relationships of the to-be-defined concept, as well as precise instructions for the generation of a relevant biomedical concept definition (see annex b). an important aspect of the generation process is its data curation. for every token generated by the model, four times more tokens were provided to the model in its prompt, on average. our hypothesis was that this saturation of information should ensure that the generated definitions are well-informed and therefore factual, coupled with the already impressive biomedical knowledge of gpt-3.5 @xcite . to confirm our intuition, 175 of the generated definitions were subsequently sent to be rated by nlp practitioners with biomedical expertise on three scales: factuality, insight, and fluency. each of these scales is measured using a 5-point rating system, detailed further in figure 3 . to assess factuality, annotators relied on several internet search queries for each definition, to verify each of the points mentioned in the definition. this was necessary as the information contained in the definition was not always readily available from a single initial search foot_0 . ## conclusion in this paper, we introduced a dataset of more than 400,000 computer-generated definitions for snomedct concepts, along with an quality control procedure applicable to biomedical definitions consisting of three axes (factuality, insight, and fluency) and a strict quality level classification based on these three axes. our quality control demonstrated that this dataset is suitable to serve as a base for several biomedical pre-training tasks, for instance the development of robust biomedical retrieval models, and might act as a bronze standard for evaluating the inherent knowledge of biomedical concepts of large language models by rating the definitions they generate in the absence of a snomedct-sourced prompt. the usage of the definitions in user-facing scenarios is however not yet within reach.