C o D , Towards an Interpretable Medical Agent using Chain of Diagnosis (2025)

Abstract: The field of AI healthcare has undergone a significant transformation with the advent of large language models (LLMs), yet the challenges of interpretability within these models remain largely unaddressed. This study introduces **Chain-of-Diagnosis (CoD)** to enhance the interpretability of medical automatic diagnosis. CoD transforms the diagnostic process into a diagnostic chain that mirrors a physician’s thought process, providing a transparent reasoning pathway. Additionally, CoD outputs the disease confidence distribution to ensure transparency in decision-making. This interpretability makes model diagnostics controllable and aids in identifying critical symptoms for inquiry through the entropy reduction of confidences. With CoD, we developed **DiagnosisGPT**, capable of diagnosing 9,604 diseases for validating CoD. Experimental results demonstrate that DiagnosisGPT outperforms other LLMs on automatic diagnostic tasks across three real-world benchmarks. Moreover, DiagnosisGPT provides interpretability while ensuring controllability in diagnostic rigor.

## introduction in ai healthcare, automatic diagnosis @xcite @xcite , which aims to provide convenient medical care and assist in diagnosis, is one of the most promising applications and is garnering increasing attention @xcite @xcite . however, it is complex, challenging the agent with multi-step decision-making abilities @xcite . specifically, it relies on interactions between an agent and a patient, where the agent inquires about the necessary symptoms for † equal contribution. * corresponding authors. diagnosis, as illustrated in figure 1 . this is consistent with real-world diagnostic procedures, where doctors inquire about symptoms to make a diagnosis @xcite . compared to prior methods, large language models (llms) offer a promising path due to their superior reasoning and dialogue abilities @xcite . these capabilities enable them to address a wide range of diseases and interact effectively with patients @xcite . in this paper, we explore the use of llms for automatic diagnosis. in our preliminary experiments, we find that llms, like gpt-4, tend to make arbitrary diagnoses without sufficient inquiry. without interpretability, it is unclear if the decisions meet sound analytical and ethical standards @xcite . interpretability is crucial to mitigate such arbitrary decisions. on the other hand, llms demonstrate poor symptom inquiry capabilities, consistent with findings from @xcite . optimizing this multi-step decision-making process without interpretability is challenging, akin to monte carlo sampling. in response to these limitations, we propose the chain of diagnosis (cod) to enhance the interpretability of llms. cod provides transparency for the diagnostic process. it transforms the blackbox decision-making process into a diagnostic chain that mirrors a physician's thinking process through five steps. for decision transparency, cod outputs a confidence distribution, where higher confidence indicates a stronger belief in diagnosing a specific disease. this allows for control over the llm's decisions using a confidence threshold. additionally, diagnostic uncertainty can be quantified by the entropy of these confidence levels. the goal of entropy reduction can aid in eliciting more effective symptoms for inquiry. to implement cod, this paper proposes constructing cod training data from synthetic patient cases generated from disease encyclopedias. this approach mitigates concerns about patient privacy and allows for scalability in validating cod. we constructed a training dataset containing 48,020 cod instances, leading to the development of our model, diagnosisgpt, capable of diagnosing 9,604 diseases. experiments demonstrate that di-agnosisgpt outperforms other llms with controllable diagnostic rigor. moreover, it achieves over 90% accuracy across all datasets with a diagnostic threshold of 0.55, underscoring the reliability of its confidence levels. it is important to emphasize that diagnosisgpt is developed exclusively for validating cod and has no clinical application. our contributions are summarized as follows: 1) we introduce the chain-of-diagnosis (cod) method, designed to enhance interpretability of llms in disease diagnosis; 2) we propose to synthesize patient cases using disease encyclopedias to validate cod, avoiding privacy and ethical concerns; 3) using cod, we built diagnosisgpt that can support automatic diagnosis for 9,604 diseases. experiments demonstrate the excellent interpretability and multi-turn decision-making capabilities of diagnosisgpt; 4) we present dxbench, a real-world diagnostic benchmark with 1,148 real cases covering 461 diseases, to expand the scope of existing automatic diagnosis evaluations. ## methodology: chain of diagnosis as depicted on the left side of figure 2 , the cod outputs a diagnostic chain, that mirrors a physician's diagnostic thinking. to implement the cod, we construct cod training data based on patient cases to fine-tune llms to perform cod, as shown on the right side of figure 2 .