LAB - KG : A Retrieval-Augmented Generation Method with Knowledge Graphs for Medical Lab Test Interpretation (2025)

Abstract: Laboratory tests generate structured numerical data, which a clinician must interpret to justify diagnoses and help patients understand the outcomes of the tests. LLMs have the potential to assist with the generation of interpretative comments, but legitimate concerns remain about the accuracy and reliability of the generation process. This work introduces LAB-KG, which conditions the generation process of an LLM on information retrieved from a knowledge graph of relevant patient conditions and lab test results. This helps to ground the text-generation process in accurate medical knowledge and enables generated text to be traced back to the knowledge graph. Given a dataset of laboratory test results and associated interpretive comments, we show how an LLM can build a KG of the relationships between laboratory test results, reference ranges, patient conditions and demographic information. We further show that the interpretive comments produced by an LLM conditioned on information retrieved from the KG are of higher quality than those from a standard RAG method. Finally, we show how our KG approach can improve the interpretability of the LLM generated text.

## introduction artificial intelligence (ai) has become increasingly influential in the medical field, offering transformative potential in various applications such as medical data summarisation @xcite and diagnostics @xcite . the data generated in clinical care, from electronic health records (ehrs) to laboratory tests, present both an opportunity and a challenge. in principle, using such data efficiently and intelligently has the potential to create efficiencies for healthcare professionals which allow them to improve patient experiences and outcomes. laboratory diagnostics generate substantial amounts of structured numerical data, which can be difficult for patients and clinicians to inter-pret effectively. ai models have the potential to provide interpretative comments and personalised explanations of laboratory results, improving the laboratory-clinical interface, and improving patient understanding @xcite . however, there are critical considerations when using ai models in the medical domain, including issues such as hallucinations, inaccuracies, and non-determinism. these issues can lead to incorrect or harmful results in healthcare @xcite , and the errors can often be difficult to identify during model evaluation and to characterize a priori. these problems call for approaches to improve the reliability and accuracy of ai systems in medicine. integrating knowledge graphs (kgs) with llms through retrieval-augmented generation (rag) can be a promising strategy. kgs provide structured, interconnected data that can ground llm outputs in factual information, reducing hallucinations, and improving the accuracy of aigenerated content @xcite . by combining the llm's generative capabilities with the kg's factual grounding, ai systems can be more reliable and explainable. in this work, we aim to improve laboratory test interpretation generation by combining rag with a knowledge graph, referred to as the lab-kg approach. traditional rag methods rely on embedding similarity between the user's query and a set of documents or knowledge base entries. they retrieve relevant information to condition the language model's generation process. however, the reasoning behind the generated interpretations often remains a black box, our lab-kg approach uses both the internal knowledge of llms and lab test examples to build a knowledge graph that explicitly captures the relevance between each test result and the patient's condition. this allows for more explainable and transparent interpretation generation. our contributions are threefold: ## previous work the application of ai to the task of laboratory test interpretation is an area of growing interest. traditional methods of providing interpretative comments on laboratory reports have been recognised as essential to improving the laboratory-clinical interface @xcite . several studies have been applied to use ai and natural language processing models to interpret laboratory test results. @xcite evaluated the performance of chatgpt and other ai models in understanding laboratory medicine test results. whilst the ai models could recognise laboratory tests and detect deviations from reference intervals, their interpretations were often superficial and incorrect. the models sometimes failed to differentiate between slight and severe deviations and did not provide meaningful suggestions for follow-up diagnostics. @xcite evaluated the thyroid function test result interpretation by biochemist, chatgpt, and google bard. the ai tools correctly interpreted only a fraction of the cases, showing the limitations of current ai models in complex medical interpretation tasks. @xcite assessed the performance of various chatbots as assistants for problem-solving in clinical laboratories. their study showed that ai applications had good performance in identifying cases and responding to questions related to preanalytical, analytical, and postanalytical errors. however, the chatbots' accuracy varied, and there were concerns about their reliability and safety in clinical settings. an early work by @xcite introduced a concept graph engine (cg-engine) that generates patient-specific personalised disease rankings based on laboratory test data, using the unified medical language system (umls) as a medical knowledge base. the cg-engine constructs a concept graph connecting laboratory tests to diseases and computes weights based on relation types, semantic types, and other attributes. while their approach utilises a knowledge base to connect lab tests and conditions, it relies on pre-existing medical ontologies that may differ from the actual data terminology. despite these advancements, a major challenge with llms in the medical domain is their tendency to produce hallucinations and inaccurate information. retrieval-augmented generation (rag) techniques have been proposed to mitigate these issues, where llms are augmented with external knowledge sources to ground their outputs in factual data. @xcite developed almanac, an llm framework augmented with retrieval capabilities from curated medical resources for medical guidelines and treatment recommendations. their results showed significant performance improvements compared to standard llm pipelines. in the domain of laboratory test interpretation, @xcite built a dataset by collecting and annotating interpretations of textual lab results from health articles. they evaluated transformer-based language models for recognizing key terms and mapped them to concepts in major controlled terminologies. in healthcare generally, integrating llms with knowledge graphs can improve the reliability and accuracy of ai models. @xcite discussed the potential of combining llms with kgs as medical information curators. by providing a structured representation of medical knowledge, kgs can help llms generate more accurate and verifiable outputs, reducing the risk of misinformation and enhancing patient safety. our work builds upon these approaches by constructing a knowledge graph that combines llm internal knowledge with examples to associate lab tests with conditions. this allows for improved accuracy in lab test interpretation generation and provides explainability through the graph structure. ## method given a set of patient full blood test csv files, we build a lab-kg with the help of gpt-4o-mini to find the relationships between the test results and conditions. a condition in our context refers to a specific medical finding or diagnosis derived from laboratory test results. for instance, "mild normochromic normocytic anaemia" indicates a type of anemia characterized by red blood cells that are of normal size (normocytic) and normal hemoglobin content (normochromic). clinicians use those conditions to determine the appropriate follow-up and management for the patient. the new patient test result is compared with the lab-kg to find the relevant condition. an overview of this process is in figure 1 . an example of a transcribed report is shown in ## implementation and evaluation to the best of our knowledge, there are very few publicly accessible datasets providing detailed laboratory test reports along with associated clinical interpretations. we utilised a dataset from mendeley data @xcite , which includes 260 clinical laboratory test reports issued by redundant test result redundant test result figure 4 : illustration of explainability in the kg. initially, "mild microcytosis" was connected to both "low mch" and "low mcv." when querying patient id 283, "mild microcytosis" was not retrieved because the kg incorrectly required both "low mch" and "low mcv" for retrieval. after removing the unnecessary connection to "low mch," "mild microcytosis" was successfully retrieved for the patient. we used the cypher query language in python to build the kg in neo4j community edition. gpt-4o-mini was used as the default llm. once the graph was built on training examples, it included 38 patient examples, 37 conditions, 78 test results, 459 nodes and 2287 relationships. our evaluation metrics include medcon @xcite , bleu-3 @xcite , rouge-1/2/l @xcite , bertscore @xcite , meteor @xcite , recall, precision, and f1 score. the llm helps to preprocess the condition result, aligning semantically equivalent conditions (e.g., "mild anemia" vs. "anemia") between the generated and target, so that the extracted sets of conditions are comparable. the calculation of recall and precision itself remains a standard statistical comparison after the conditions have been extracted and aligned by the llm. the f1 score is based on recall and precision. we use bleu-3 instead of bleu-4 because the results can be very short, such as "normal blood picture," which bleu-4 would omit. medcon is a metric for evaluating medical condition extraction from generated texts, considering semantic similarity and clinical relevance. the kg and kg with contributes_to relationships inferred without examples (referred to kg * below) are compared for those metrics, together with other methods listed below. in all our experiments, we performed five-fold cross-validation, with test data sizes of 10, 10, 9, 9, and 9 in each fold. we use medcon to select the threshold for the confidence score in each fold and from a list of values ranging from 0.1 to 0.9, with step 0.05. the best threshold values are stable across folds (0.55, 0.6, 0.55, 0.5, 0.5 for kg and 0.45, 0.35, 0.35, 0.35, 0.35 for kg *). the median values 0.55 and 0.35 are selected as the final threshold values for all the folds in kg and kg * respectively. we compared the performance of the lab-kg approach with several baseline methods, including: ## conclusion in this paper, we integrate the knowledge graph with rag and llm to improve the interpretation of laboratory test results with limited examples, providing an explainable framework clinicians can understand. the evaluation demonstrated that the lab-kg method outperforms llm prompt engineering, text embedding-based retrieval, and random forests. the combination of strict matching and confidence score-based matching with kg allows us to retrieve the most clinically relevant interpretations. the kg with the relationship between condition and test result built without examples also performs well, especially in the strict match case, demonstrating its accurate internal knowledge. we observed that in some cases, the relevance inferred by the llm without examples was better than when examples were provided. this suggests the potential to combine the llm's internal knowledge more effectively with examples to optimise performance. when multiple conditions are present in the example, the llm sometimes struggles to differentiate the test results associated with each condition. providing separate conditions in examples or generating synthetic data could help mitigate this issue. the strength of our findings may be limited with only 47 blood test reports. expanding the dataset and applying lab-kg to other laboratory tests are essential steps for validating lab-kg. the llm's internal knowledge may not be up to date and limited, and integrating our knowledge graph with external medical ontologies like snomed ct is for future exploration. strict match, the new patient only has the test result mcv_abnormal (low), which is not present in the example patient who has both mcv_abnormal (low) and rbc count_abnormal (high). the internal llm did not connect rbc count_abnormal to that condition, and it can retrieve that condition by strict match. the llm's decision is affected by the noise of the dataset, which causes this misidentification. the graph for this query is figure 5 in appendix. a.5 correlation between f1 score and other metrics metrics medcon bleu bert_score meteor rouge1 rouge2 rougel corr 0.95 0.75 0.94 0.91 0.91 0.9 0.85 corr * 0.97 0.66 0.91 0.87 0.84 0.9 0.74