Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrieval

Authors: Pengcheng Jiang, Cao Xiao, Minhao Jiang, Parminder Bhatia, Taha Kass-Hout, Jimeng Sun, Jiawei Han
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
Keywords: EHR Prediction, Large Language Models, Knowledge Graphs

Abstract:
Large language models (LLMs) have demonstrated significant potential in clinical decision support. Yet LLMs still suffer from hallucinations and lack fine-grained contextual medical knowledge, limiting their high-stake healthcare applications such as clinical diagnosis. Traditional retrieval-augmented generation (RAG) methods attempt to address these limitations but frequently retrieve sparse or irrelevant information, undermining prediction accuracy. We introduce KARE, a novel framework that integrates knowledge graph (KG) community-level retrieval with LLM reasoning to enhance healthcare predictions. KARE constructs a comprehensive multi-source KG by integrating biomedical databases, clinical literature, and LLM-generated insights, and organizes it using hierarchical graph community detection and summarization for precise and contextually relevant information retrieval. Our key innovations include: (1) a dense medical knowledge structuring approach enabling accurate retrieval of relevant information; (2) a dynamic knowledge retrieval mechanism that enriches patient contexts with focused, multi-faceted medical insights; and (3) a reasoning-enhanced prediction framework that leverages these enriched contexts to produce both accurate and interpretable clinical predictions. Extensive experiments demonstrate that KARE outperforms leading models by up to 10.8-15.0\% on MIMIC-III and 12.6-12.7\% on MIMIC-IV for mortality and readmission predictions. In addition to its impressive prediction accuracy, our framework leverages the reasoning capabilities of LLMs, enhancing the trustworthiness of clinical predictions.

Decision: accept (poster)
Reviewer scores: 5, 6, 6, 3, 8

Reviews and discussion:
<thread>
creation date:
1730291704320

thread_type:
official_review

<thread_object>
creator: reviewer_okgq

<content>
summary:
In this manuscript, the authors propose KARE, a knowledge graph-enhanced large language model designed for predicting patient mortality and readmission, with an added aim to mitigate LLM hallucinations. The paper introduces a novel approach that effectively combines knowledge graphs with LLMs through a clustering method. Additionally, the authors present a reasoning-chain mechanism to enhance the LLM's inference capabilities and provide interpretable prediction results. Experimental results demonstrate that KARE achieves a marginal improvement in mortality and readmission predictions on the MIMIC-III and MIMIC-IV datasets. However, several major and minor issues need to be addressed. If author could well address my concern, I would like to update my ratings,

strengths:
* The paper is well-written and structured.
* The authors propose a novel knowledge-graph learning framework combined with LLMs.
* The integration of knowledge augmentation and knowledge graphs is both novel and interesting.
* The experimental evaluation is thorough, conducted on both the MIMIC-III and MIMIC-IV datasets.

weaknesses:
* The excessive use of symbols makes certain sections difficult to follow.
* The paper lacks a detailed description of longitudinal EHR processing with LLMs.
* The improvement in mortality prediction performance achieved by KARE is marginal.
* There is a lack of sensitivity analysis for hyperparameters, such as the number of top co-occurring concepts, top documents, and the criteria for selecting optimal values. Additional hyperparameters, such as maximum sequence length and maximum path count, should also be discussed, or at least identified explicitly for clarity.
* The paper would benefit from a more in-depth interpretability analysis to clarify the relationship between the generated reasoning, labels, and the knowledge graph.
* The paper lacks an evaluation of LLM hallucinations, despite the stated aim to address hallucination issues in clinical decision-making.
* An analysis of model parameters and time consumption is missing, which could provide valuable insights into the model’s computational efficiency and practical applicability
* There is a lack of references to related work.

  [1] . Kang, M., Lee, S., Baek, J., Kawaguchi, K., & Hwang, S. J. (2024). Knowledge-augmented reasoning distillation for small language models in knowledge-intensive tasks. Advances in Neural Information Processing Systems, 36. 

  [2]. Niu, S., Ma, J., Bai, L., Wang, Z., Guo, L., & Yang, X. (2024). EHR-KnowGen: Knowledge-enhanced multimodal learning for disease diagnosis generation. Information Fusion, 102, 102069.

  [3]. Kwon, T., Ong, K. T. I., Kang, D., Moon, S., Lee, J. R., Hwang, D., ... & Yeo, J. (2024, March). Large language models are clinical reasoners: Reasoning-aware diagnosis framework with prompt-generated rationales. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 38, No. 16, pp. 18417-18425).

questions:
* In Section 3.2, how does the method handle cases where the same patient has more than two visits with different labels? Is there a specific design to distinguish similar symptoms in different patients and multiple visits from the same patient?
* In Section 3.2, how is the effectiveness of the relevance score (i.e., Formula (3)) validated? Would an ablation study on the components of Relevance(C_k) help to clarify this?
* How does the method utilize patient longitudinal visit information?
* For the baseline ML methods, it’s mentioned that most are implemented using PyHealth. Could you clarify the backbone model for each ML method? Also, is there a fair configuration in place for implementing language model-based encoders, such as ClinicalBERT, across these methods? 
* The experimental results for mortality prediction show that the baseline ML methods, such as ConCare and TCN, perform closely to KARE, with some evaluation metrics even exceeding KARE’s. How should these results be interpreted?
* In experiment, sensitivity and specificity are indeed important metrics, but F1 score and accuracy are also widely used for assessing diagnostic accuracy. Interestingly, in ablation study, excluding similar patients often yields better or comparable results than including them, especially for mortality prediction on the MIMIC-III dataset. Do these results support the methodological choices made in Section 3.2?
* As you state that your base model is fine-tuned on Mistral-7B-Instruct-v0.3, I wonder about the performance of fine-tuning an LLM (Mistral or Llama) with RAG. Can your model still show substantial improvements compared to other LLM-based models beyond just zero-shot or few-shot settings? I am concerned that most of the current improvements may be attributed primarily to supervised fine-tuning.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730342120336

thread_type:
official_review

<thread_object>
creator: reviewer_thia

<content>
summary:
The paper introduces KARE, a novel framework designed to enhance clinical decision support by addressing the limitations of Large Language Models (LLMs) in healthcare. While LLMs show potential, they suffer from hallucinations and lack the fine-grained medical knowledge necessary for high-stakes applications like diagnosis. Traditional retrieval-augmented generation (RAG) methods often retrieve sparse or irrelevant data, undermining accuracy. KARE improves upon this by integrating a multi-source knowledge graph (KG) with LLM reasoning. The KG is built from biomedical databases, clinical literature, and LLM-generated insights, structured using hierarchical community detection for precise information retrieval. Key innovations include dense medical knowledge structuring, dynamic retrieval of multi-faceted medical insights, and reasoning-enhanced predictions. KARE outperforms existing models in MIMIC-III and MIMIC-IV datasets, improving prediction accuracy by up to 15%, while also enhancing the interpretability and trustworthiness of clinical predictions.

strengths:
1. The research team has achieved impressive results, exceeding established benchmarks. This progress marks a significant step forward in improving predictive model accuracy in medical data analysis.
2. The authors' workload is immense, and the experimental details are thoroughly outlined, which I greatly appreciate.

weaknesses:
1. By reviewing your code and the details in the article, I can see that your workload is immense, however, the contribution of this article is incremental. My understanding is that it is essentially a combination of GraphRAG and GraphCare [1]. Furthermore, many key baselines were not cited. Since the authors mentioned that this paper focuses on RAG for EHR, some essential RAG algorithms should have been introduced, such as MedRetriever [2], and commonly used GraphRAG algorithms like KGRAG [3]. 
2. In the experiment or appendix section, I did not clearly see the formulas for Sensitivity and Specificity, nor were there any corresponding references, which is quite confusing to me. Moreover, using Accuracy as a metric in cases of highly imbalanced labels is unreasonable. For instance, in the MIMIC-III Mortality Prediction task, the positive rate is 5.42%. If I predict that all patients will survive, I can still achieve an accuracy of 94.58%. Previous works, such as GraphCare [1], have adopted AUROC and AUPRC as evaluation metrics.
3. The article is overly long and filled with detailed content, making it easy for readers to miss important points.

- [1] GraphCare: Enhancing Healthcare Predictions with Personalized Knowledge Graphs. ICLR 2024
- [2] MedRetriever: Target-driven interpretable health risk prediction via retrieving unstructured medical text. CIKM 2021
- [3] Biomedical knowledge graph-enhanced prompt generation for large language models. Arxiv 2023

questions:
1. The authors used Claude 3.5 Sonnet as an expert model to generate training samples and augment knowledge graph. However, since Claude is a general-purpose model, could it lack some medical knowledge, potentially leading to biased training samples and cumulative errors? As mentioned in your summary: "Yet LLMs still suffer from hallucinations and lack fine-grained contextual medical knowledge, limiting their high-stakes healthcare applications such as clinical diagnosis." In the experiment section, there are many related LLMs in the medical domain. It would be better if the researcher could compare KARE with more related LLM-based baselines referred in [4].
2. I didn't see any examples of training samples in the code you provided. Can you provide us with some examples?
3. This parameter design is very challenging. In KARE, there are many hyperparameters, including but not limited to those in graph generation, summarization, model training, and model testing. Any slight change can lead to significant deviations in the model's results. Could you elaborate on how you adjust the parameters in such a large hyper-parameter space?


- [4] https://huggingface.co/blog/leaderboard-medicalllm.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730534094440

thread_type:
official_review

<thread_object>
creator: reviewer_pyh3

<content>
summary:
This paper introduces KARE, a framework integrating LLM reasoning with KG retrieval to improve healthcare predictions. KARE combines structured multi-source medical knowledge with dynamic, patient-specific context augmentation to provide accurate and interpretable clinical predictions. Evaluated on MIMIC-III and MIMIC-IV datasets for mortality and readmission predictions, KARE demonstrates improved accuracy and interpretability over conventional models.

strengths:
- Pros
  - Interesting topic for enhancing healthcare predictions by combining the reasoning capabilities of LLM.
  - Clear and well-motivated reasoning in the paper. 
  - Comprehensive ablation studies validate the contributions of each model component.
  - Well-written and structured.

weaknesses:
- Cons
  - The novelty of this paper is relatively limited, appearing to be incremental compared to GraphRAG [1].
  - In Section 3.3.1, the authors select the reasoning chain with the highest confidence as training data. However, according to conclusions from some existing studies [2,3], the reasoning chain with the highest confidence is not necessarily the most reliable.
  - MedRetriever [4] also adopts a retrieval-augmented approach for healthcare prediction, but this paper lacks a comparative analysis with MedRetriever.
  - The definitions and calculation methods for Sensitivity and Specificity need to be clarified more thoroughly, and metrics such as AUROC and AUPRC should be added.
  - Although Amazon Bedrock provides strict compliance standards and privacy protection measures, relying on it to generate reasoning chains for distillation may limit the generalizability of this approach in real healthcare scenarios with high privacy protection requirements.

[1] Edge D, Trinh H, Cheng N, Bradley J, Chao A, Mody A, Truitt S, Larson J. From local to global: A graph rag approach to query-focused summarization. arXiv preprint arXiv:2404.16130. 2024 Apr 24.

[2] Yang, Haoyan, et al. "Can We Trust LLMs? Mitigate Overconfidence Bias in LLMs through Knowledge Transfer." arXiv preprint arXiv:2405.16856 (2024).

[3] Xiong, M., Hu, Z., Lu, X., LI, Y., Fu, J., He, J. and Hooi, B., Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs. In The Twelfth International Conference on Learning Representations.

[4] Ye M, Cui S, Wang Y, Luo J, Xiao C, Ma F. Medretriever: Target-driven interpretable health risk prediction via retrieving unstructured medical text. InProceedings of the 30th ACM International Conference on Information & Knowledge Management 2021 Oct 26 (pp. 2414-2423).

questions:
See Weaknesses Above.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730699302130

thread_type:
official_review

<thread_object>
creator: reviewer_tzi7

<content>
summary:
The paper presents KARE, a framework that enhances healthcare predictions by combining knowledge graph (KG) community-level retrieval with large language model (LLM) reasoning. It addresses LLM limitations like hallucinations and inadequate medical knowledge, which can affect clinical diagnosis. KARE builds a comprehensive knowledge graph from biomedical databases, clinical literature, and LLM insights, organized through hierarchical community detection and summarization to improve retrieval precision and relevance. Key innovations include: (1) a dense medical knowledge structuring approach for accurate information retrieval; (2) a dynamic retrieval mechanism that enriches patient contexts with multi-faceted insights; and (3) a reasoning-enhanced prediction framework that produces accurate and interpretable clinical predictions.

strengths:
1. I really enjoyed the LLM-Driven Reasoning, producing both accurate and interpretable clinical predictions that enhance trust.
2. The model's multi-source knowledge graph ensures relevant information retrieval, addressing LLM limitations like hallucinations and sparse data.
3. KARE's retrieval mechanism enriches patient data with multi-faceted insights, enhancing EHR representation learning

weaknesses:
1. Methodology Clarity: Some aspects of the methodology lack clarity, such as how the different knowledge graphs (KGs) are connected in Equation (1). Specifically, the approach for creating edges between nodes in different KGs, like G^KG and G^BC, is not well explained.

2.  The experimentation could be broadened to include more general tasks, such as diagnosis prediction or drug recommendation. Was there a reason these broader tasks were not considered?

3.  The results lack standard deviations or confidence intervals, which would help indicate the reliability of the reported performance.

4. Ablation Study Design: The ablation study could be more informative if it involved removing each feature individually, rather than adding features one at a time.
5. The provided anonymous GitHub code link cannot be opened.

questions:
please refer to the weaknesses
</content>
</thread_object>

</thread>
<thread>
creation date:
1730700201399

thread_type:
official_review

<thread_object>
creator: reviewer_83dm

<content>
summary:
The paper presents KARE, a framework designed to improve healthcare predictions by combining KG community-level retrieval with LLM reasoning. It builds a medical KG from diverse sources, organizes it into meaningful communities, and dynamically augments patient data with relevant information. Extensive tests on MIMIC-III and MIMIC-IV datasets demonstrate significant performance improvements for mortality and readmission prediction.

strengths:
1. The proposed method improves the interpretability and trustworthiness of clinical predictions by generating reasoning chains.
2. The experiments are extensive. The framework is compared to many baselines and shows a sufficient performance gain in both tasks.
3. The visualizations are well-structured and the writing is easy to follow.
4. The problem setting is clearly defined.

weaknesses:
1. The framework is overly complex, making it difficult to optimize and reproduce. Moreover, any noise introduced in the earlier steps (e.g., entity and relation extraction, LLMs for relationship suggestions and reasoning chains generation, etc.) could affect subsequent stages and degrade the final performance.
2. Given its complexity, the framework's efficiency should be evaluated. It involves building three different KGs for each medical concept, conducting community detection 25 times, and generating multiple summaries for each community (~60k communities in total). This process likely requires significant time and resources, making it inefficient.
3. The contribution is a bit limited compared to GraphRAG; the primary difference is mentioned in line 213 that they run GraphRAG multiple times for better diversity.
4. Acronyms like LLM, KG, EHR, and RAG are introduced multiple times throughout the paper.
5. It would be better to include case studies for the three KGs generated by the biomedical KG, biomedical corpus, and LLMs.

questions:
1. In Section 3.1.1, three different KGs are generated for each medical concept, and the final KG is to integrate the three KGs together. How do you handle the conflicts among the three KGs?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732300732185

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Response to All Reviewers

comment:
Dear Reviewers,

We sincerely appreciate your thorough feedback and constructive suggestions. Based on your comments, we have made comprehensive revisions to our paper, with all changes highlighted in blue.

First, we thank the reviewers for recognizing the key strengths of our work:

* The framework effectively addresses critical challenges in healthcare predictions (`All Reviewers`)
* The experimental evaluation is comprehensive and thorough (Reviewers `83dM`, `pYH3`, `thia`, `oKgq`)
* The LLM-driven reasoning significantly enhances both accuracy and interpretability (Reviewers `83dM`, `pYH3`, `tZi7`, `oKgq`)
* The integration of knowledge graphs with LLM reasoning presents a novel and promising approach (Reviewers `tZi7`, `oKgq`)
* The visualization and presentation demonstrate clarity and strong structure (Reviewers `83dM`, `pYH3`, `oKgq`)

The revised paper incorporates all reviewers' suggestions, including enhanced analysis explanations, new experimental results, and additional experimental details.

Key updates in our revision include:

* Integration of additional related works as suggested by reviewers (`pYH3`, `thia`, `oKgq`)

* Analysis of metric selection rationale (excluding AUROC/AUPRC) and detailed definitions of Sensitivity and Specificity in **Appendix E** (`pYH3`, `thia`)

* Addition of MedRetriever performance results in Table 2 (`pYH3`, `thia`)

* Inclusion of KG construction case studies from different sources in **Appendix B.4** (Figures 7, 8, and 9) (`83dM`)

* Performance with standard deviation in **Appendix H** (`tZi7`)

* Human evaluation results of KARE-generated reasoning chains in **Appendix I** (`thia`, `oKgq`)

* Comparative analysis of model parameters and training time in **Appendix J** (`oKgq`)

  

**We have addressed each reviewer's concerns and questions thoroughly in our detailed individual responses**. 

As the reviewer-author discussion phase concludes shortly, we welcome your review of our responses and any additional feedback for improvement. We greatly appreciate your continued participation in this discussion.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper introduces KARE, a novel framework designed to enhance clinical decision-making by combining knowledge graph (KG) community retrieval with reasoning capabilities of large language models (LLMs). Traditional retrieval-augmented generation (RAG) models often retrieve sparse or irrelevant data, hindering healthcare predictions. KARE overcomes these limitations by structuring a multi-source KG from biomedical databases, clinical literature, and LLM-generated insights, then leveraging hierarchical graph community detection to retrieve precise and contextually relevant information.

Key Contributions:  
1. Dense Medical Knowledge Structuring: Enables accurate retrieval of context-specific medical data.  
2. Dynamic Knowledge Retrieval: Enriches patient-specific contexts with detailed and relevant insights.  
3. Reasoning-Enhanced Prediction Framework: Combines enriched contexts with LLM reasoning to deliver interpretable and precise clinical predictions.

Results:  
- Outperforms leading models in mortality and readmission predictions on the MIMIC-III (by 10.8-15.0%) and MIMIC-IV (by 12.6-12.7%) datasets.  
- Demonstrates improved prediction accuracy and trustworthiness due to LLM-driven reasoning.

Innovations:  
- Hierarchical KG community organization for effective retrieval.  
- Multi-faceted insights addressing LLM hallucination and sparse retrieval issues.

Additional Notes:  
The paper includes extensive experiments, human evaluations, and detailed responses to reviewer concerns. Discussions highlight the importance of choosing appropriate metrics, such as sensitivity and specificity, over AUROC/AUPRC for LLM-based predictions. Despite some critical reviews, KARE is presented as a significant step forward in leveraging AI for clinical applications.

The paper introduces KARE, a framework that combines knowledge graph (KG) retrieval and large language model (LLM) reasoning to enhance healthcare predictions. KARE addresses LLM limitations, such as hallucinations and insufficient medical knowledge, by integrating structured data from biomedical sources, clinical literature, and LLM insights into a unified KG. The KG is organized through hierarchical community detection and summarization, enabling precise and contextually relevant retrieval for improved predictions.

Key Features:  
1. Dense Medical Knowledge Structuring: Ensures accurate and relevant information retrieval by embedding entities and relations into a shared semantic space.  
2. Dynamic Context Augmentation: Enhances patient-specific data with multi-source insights, improving electronic health record (EHR) representation learning.  
3. LLM-Driven Reasoning Framework: Produces accurate, interpretable predictions to boost trust in clinical applications.

Innovations:  
- Integration of multi-source medical knowledge with KG community detection.  
- A dynamic retrieval mechanism enriching patient data with multi-faceted insights.  
- An interpretable, reasoning-based prediction framework for critical tasks like mortality and readmission prediction.