A Layered Debating Multi-Agent System for Similar Disease Diagnosis (2025)

Abstract: Distinguishing between extremely similar diseases is a critical and challenging aspect of clinical decision-making. Traditional classification, contrastive learning, and Large Language Models (LLMs) based methods fail to detect the subtle clues necessary for differentiation. This task demands complex reasoning and a variety of tools to identify minor differences and make informed decisions. This paper probes a novel framework that leverages LLMs and a multi-agent system to achieve accurate disease diagnosis through a process of repeated debate and reassessment. The approach aims to identify subtle differences between similar disease candidates. We structure patient information and integrate extensive medical knowledge to guide the analysis towards discerning these differences for precise diagnosis. Comprehensive experiments were conducted on two public datasets and two newly introduced datasets, JarvisD2-Chinese and JarvisD2-English, to validate the effectiveness of our method. The results confirm the efficacy of our approach, demonstrating its potential to enhance diagnostic precision in healthcare.

## introduction in recent years, ai-assisted clinical diagnosis has significantly enhanced the efficiency and accuracy of medical assessments. swift and precise disease prediction is crucial for timely and effective treatment, ultimately saving lives. diagnosing diseases that present with prominent symptoms is relatively straightforward. however, diagnosing conditions that exhibit very similar symptoms is more challenging and carries a higher risk of misdiagnosis. in clinical practice, when faced with the potential for misdiagnosis (also known as similar diseases), medical experts employ a method known as "differential diagnosis". this involves compiling a com-* equal contribution † corresponding author prehensive list of all possible diseases that could cause the observed symptoms and systematically narrowing down this list through further medical examinations until the most likely disease is identified. for instance, cardiovascular diseases like myocarditis, heart failure, and myocardial infarction share symptoms such as chest pain, shortness of breath, fatigue, and palpitations, but have distinct causes and treatments. accurate diagnosis is crucial to prevent serious complications. a key differentiator is the duration of symptoms: heart failure is long-term, while myocardial infarction and myocarditis have different temporal patterns. diagnosing these conditions requires extensive medical knowledge and expert reasoning to identify subtle differences. traditional methods for disease diagnosis include classification based methods that predict diseases using trained classification networks @xcite @xcite ; contrastive learning based methods that separate diseases using contrastive learning strategies @xcite @xcite ; large language models (llms) based methods that conduct disease diagnosis through pre-training or prompt learning based on llms @xcite @xcite @xcite @xcite . however, these methods may fail to capture the subtle clues necessary for differential diagnosis, as these clues are often too subtle to detect and many require consequential decision-making. in this paper, we propose a novel framework that leverages multiple llm-based agents working collaboratively to achieve accurate disease diagnosis (denoted as mlad). the key insight of mlad lies in identifying subtle distinctions between similar disease candidates through a cycle of iterative debating and reflecting, all guided by comprehensive medical knowledge to facilitate ef-fective differential diagnosis. the process involves engaging agents specialized in different disease domains to present their perspectives, participate in debate, and reflect on the diagnosis. the process continues until the agents' diagnoses converge. furthermore, we employ a highly effective structured mechanism, imap @xcite , to restructure patient information, emphasizing crucial information like symptoms and lab results. throughout the procedure, the agents have access to various resources, such as medical knowledge graph searches, to assist in pinpointing the correct diagnosis. to evaluate mlad, we first compare its performance on two publicly available medical exam datasets in both english and chinese. to address the lack of challenged similar disease options and potential data leakage in public datasets, we enhanced two public datasets by revising the options to create a more robust similar disease diagnosis dataset. to generate options that include more differential diagnoses, we consider candidates derived from various sources such as medical knowledge graph, llms and icd-10 foot_0 . in summary, our contributions can be outlined as follows: ## methods the key insight of mlad lies in its ability to uncover subtle differences between similar diseases through iterative debate and reflection, guided by essential medical knowledge and tools. as illustrated in figure 1 , mlad begins with an initialization phase that highlights the input text with patient information using imap-a data structure for key information extraction introduced by @xcite . it also equips the llm-based agents with different disease backgrounds. the process then moves into the debating phase, which includes an inner-group discussion among agents with the same diagnosis to consolidate their reasoning, followed by an inter-group debate to compare differing diagnostic views. subsequently, the tool utilization phase allows agents to use resources like search engines to acquire additional medical knowledge and evaluate the perspectives of other agents. after this, all agents are given the opportunity to reflect on their points and re-evaluate their diagnoses. this cycle continues until a consensus on the diagnosis is reached. the detailed process is as follows. ## conclusion this paper proposes a collaborative framework named mlad, which utilizes multiple llm-based agents for accurate differential diagnosis. the method involves iterative debating and reflecting, guided by extensive medical knowledge, to identify subtle distinctions between similar diseases. empirical results on two public datasets and two newly introduced challenging dataset demonstrate the effectiveness of mlad. especially, mlad outperforms other methods on the challenging dataset and demonstrates strong generalizability in differentiating similar diseases.