Multimodal Clinical Reasoning through Knowledge-augmented Rationale Generation
Authors: Shuai Niu, Jing Ma, Liang Bai, Zhihua Wang, Richard Yi Da Xu, Yunya Song, Xian Yang


Reviewer paper summaries: This paper introduces ClinRaGen, a novel model that leverages the reasoning abilities of Large Language Models (LLMs) to enhance disease diagnosis and clinical rationale generation in smaller language models (SLMs). The authors address the challenge of applying LLMs, which are often computationally expensive, to the knowledge-intensive task of interpreting Electronic Health Records (EHRs) for accurate disease diagnosis.

The paper might be misunderstood as solely focusing on improving diagnostic accuracy. However, its primary contribution lies in bridging the gap between accurate diagnosis and explainability by enabling SLMs to generate comprehensive, human-understandable rationales for their diagnoses.
Summary of strengths: - The paper introduces ClinRaGen, a novel model that improves the ability of SLMs to generate clinical rationales for disease diagnoses from electronic health records (EHRs).  ClinRaGen employs a novel knowledge-augmented attention mechanism and a sequential multimodal rationale distillation paradigm.
- The authors evaluated ClinRaGen on two publicly available datasets, MIMIC-III and MIMIC-IV, and showed that it outperforms several baseline models in terms of both diagnostic accuracy and the quality of the generated rationales. 
- The authors conducted ablation studies to evaluate the contribution of each component of ClinRaGen and showed that rationale distillation from LLMs, the integration of external medical knowledge, and the use of multimodal data are all important for achieving good performance.
-
Summary of weaknesses: - Limited Computational Resources: The authors acknowledge using limited computational resources, restricting them to the Flan-T5 small model, which has an input limit of 512 medical note tokens. This limitation prevented the processing of longer EHRs and the exploration of larger, potentially more effective language models like Flan-T5-large, LLaMA, and LLaMA2 . utilizing a larger model with increased capacity could potentially help understand the generalizability of model better.  

- The study currently focuses on only two modalities: medical notes and laboratory test results. While the authors acknowledge this limitation and express intentions to incorporate additional modalities in future work, the current scope is limiting. Incorporating other relevant data, such as discrete clinical events and medical images, could potentially enhance the model's reasoning capabilities and practical applicability in real-world clinical settings.

- The authors have not used any domain-specific models. It would be interesting to see the performance of the approach when the base model is fine-tuned on the domain data.
Comments, suggestions and typos: - Provide more details about the rationale generation process with ChatGPT. While the sources outline the prompts used for rationale generation with ChatGPT-3.5 Turbo, providing a more detailed explanation of the rationale generation process, including the specific instructions and parameters used, would enhance the reproducibility of the research.

- Elaborate on the selection of the 50-document limit for PubMed knowledge augmentation. The sources mention retrieving abstracts of 50 documents per target disease from PubMed for knowledge augmentation.
Overall assessment scores: 3