MoodAngels: A Retrieval-augmented Multi-agent Framework for Psychiatry Diagnosis
Authors: Mengxi Xiao
Ben Liu
He Li
Jimin Huang
Qianqian Xie
Xiaofen Zong
Mang Ye
Min Peng
Venue: NeurIPS 2025 poster ()
Keywords: psychiatry diagnosis
multi-agent framework
mental health
Primary area: applications

Abstract: The application of AI in psychiatric diagnosis faces significant challenges, including the subjective nature of mental health assessments, symptom overlap across disorders, and privacy constraints limiting data availability. To address these issues, we present MoodAngels, the first specialized multi-agent framework for mood disorder diagnosis. Our approach combines granular-scale analysis of clinical assessments with a structured verification process, enabling more accurate interpretation of complex psychiatric data. Complementing this framework, we introduce MoodSyn, an open-source dataset of 1,173 synthetic psychiatric cases that preserves clinical validity while ensuring patient privacy. Experimental results demonstrate that MoodAngels outperforms conventional methods, with our baseline agent achieving 12.3\% higher accuracy than GPT-4o on real-world cases, and our full multi-agent system delivering further improvements. Together, these contributions provide both an advanced diagnostic tool and a critical research resource for computational psychiatry, bridging important gaps in AI-assisted mental health assessment.

MoodAngels: A Retrieval-augmented agntig
Framework for Psychiatry Diagnosis
Mengxi 1,2 Ben Liu He Li Jimin Huang", Qianqian Xie!:?,
Xiaofen Zong",, Mang Ye Min Pengl.2*
Sccooo of Artificial Intelligence, Wuhan University
2 Center for Language and Information Research, Wuhan University
School of Computer Science, Wuhan University
*Department of Psychiatry, Renmin Hospital of Wuhan University
"The Fin AI Taikang Center for Life and Medical Sciences, Wuhan University
Abstract
The application of AI in psychiatric diagnosis faces significant challenges, includ- ing the subjective nature mental health assessments, symptom overlap across disorders, and privacy constraints limiting data availability. To address these is- sues, we present MoodAngels, the first specialized multi-agent framework for mood disorder diagnosis. Our approach combines granular-scale analysis of clin- ical assessments with a structured verification process, enabling more accurate interpretation of complex psychiatric data. Complementing this framework, we introduce MoodSyn, an open-source dataset of 1,173 synthetic psychiatric cases that preserves clinical validity while ensuring patient privacy. Experimental results demonstrate that MoodAngels outperforms conventional methods, with our base- line agent achieving 12.3% higher accuracy than GPT-4o on real-world cases, and our full multi-agent system delivering further improvements. Evaluation in the MoodSyn dataset demonstrates exceptional fidelity, accurately reproducing both the core statistical patterns and complex relationships present in the original data while maintaining strong utility for machine learning upplications. Together, these contributions provide both an advanced diagnostic tool and critical research re- source for computational psychiatry, bridging important gaps in Al-assisted mental health assessment.
Introduction
Mental discases [ with their high prevalence and profound socictal impact, pose major public health challenge by severely impairing quality of life. Accurate diagnosis [2] is essential for timely intervention and effective treatment [. yet the complexity and variability of symptoms make it particularly difficult [4) highlighting the need for advanced diagnostic tools to aid clinicians. Among mental diseases, mood disorder, including conditions like depression and bipolar disorder, is critical due to its high prevalence and the significant overlap of symptoms with other psychiatric conditions [S]. Correctly diagnosing mood disorders is crucial, it influences the diagnostic process for other disorders [6]: for example, symptoms like difficulty concentrating may signal neurodevelopmennall disorders only if they occur outside of depressive episodes. Given their prevalence and severe consequences, including suicide risk and chronic disability, mood disorders represent a significant burden on individuals and healthcare systems.
Correspondence to: Min Peng <pengm @ whu.edu.cn>, Mang Ye cyemmnn@@wu..ce. Xiaofen Zong <zongxiaofen @ whu.edu.cn>
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

External Data
Diagnostic Tools
Multi-Angels
e Inegaiiaron
Spmppton matching e %
Client Duta
metial recrd
@, diegnostic feril
[
performance
oelertion
ahfikrentia crisaii
jekited syripterss with correggondngg dioeases
DSM-5
Krunlleige Bund
Individaal Diagnoda
AnarLE
Anrel.D
AraOL C
modical record
The cliss (barddoen'' Jaae] inced disorder The ressoes
Clincal Data
scals performarees
sinillr cases
Judge
Data Store
Figure 1: The MoodAngels framework. Diagnostic agents include Angel.R, Angel., Angel. and multi-Angels.
While large language models (LLMs) and LLM-based agents demonstrate strong capabilities in medical domains through robust textual analysis and decision-making, their application to psychiatry faces unique challenges. General medical diagnostic agents typically rely on concrete medical records or biological test results 7 [] resources largely unavailable in psychiatric practice. These limitations interact with the field's inherent uncertainties, including significant symptom overlap across disorders [TI and the absence of definitive biomarkers [12 [3 that are standard in other medical specialties, creating fundamental barriers for Al implementation. These diagnostic difficulties are intensified by psychiatry's reliance on nuanced interpretation of subjective clinical data, which contrasts sharply with the structured evidence typically used to train LLM-based agents 77] [). Moreover, the situation faces additional complications from data accessibility constraints. Clinical diagnostic information, while rich in potential insights, contains sensitive patient data that cannot be publicly shared, creating critical bottleneck for Al-driven psychiatric research. Taken together, this dual challenge of diagnostic complexity and data scarcity underscores the urgent need for specialized systems capable of emulating clinicians' probabilistic reasoning under conditions of imperfect information [4].
To address these challenges, we propose Anoel (Figure 1 the first ettrieval-augment multi-agent framework for mood disorder diagnosis, which enhances the diagnostic process through granular-scale analysis and multi-step verification. The system tackles the inherent challenges of psychiatric diagnosis, reliance on potentially unreliable self-reported and clinician-estimate scales (with optional medical records), by decomposing traditional scale scoring into item-level analysis. Using Pearson correlation, we identify the top 5% most statistically significant questions for mood disorders, categorizing them into five diagnostic groups (depression, suicidal idcation, energy/interest loss, anxicty, and insomnia). By analyzing consistency within these groups, MoodAngels can resolve diagnostic discrepancies (e.g., when self-reported depressive symptoms conflict with clinical observations) through additional behavioral marker validation.
To address symptom overlap, we structure DSM-5 diagnostic criteria 15 into retrievable knowledge base, augmented with anonymized clinical data to incorporate expert judgment and handle ambiguous cases. We develop three diagnostic variants to balance historical reliance with individual variability: Angel.R (no reference), Angel.D (case display), and Angel.C (comparative analysis). Our final multi- Angels model synthesizes their independent diagnoses through debate, combining computational efficiency with clinical nuance.
Beyond diagnostic frameworks, clinical data scarcity presents another research challenge. Despite longstanding interest in depression and bipolar detection, existing methods predominantly analyze social media posts [16-18], which may be distorted by social norms or exaggeration. To enable accurate Al-driven psychiatric diagnosis and early detection, we construct the open-source synthetic dataset MoodSyr with 1,173 synthetic psychiatric cases, containing: selected five groups of top- related scale data, total scores from common mental disorder scales, and mood disorder labels. Through comprehensive evaluation of quality, ML efficiency, and privacy protection, we demonstrate that this synthetic dataset maintains high fidelity while safeguarding visitor confidentiality.
"Code and synthetic data sample available AnoeAs

---

We evaluate the effectiveness of our framework on 561 real-world clinical cases and 140 synthetic cases. Experimental results on both real cases and synthetic data demonstrate the superiority of our diagnostic process, with all versions of our agents outperforming bare L.Mss that rely solely on client performances as context. Notably, on real data, even our raw agent, Angel.R, achieves a 12.3% higher diagnostic accuracy than our backbone LLM, GPT-4o. The multi-agent framework further enhances performance, surpassing the accuracy of all single-agent variants and showing considerable improvement hard cases. Ablation studies confirm the individual contributions of our medical record analysis and scale selection processes, highlighting their critical roles in improving diagnostic precision.
Our contributions are summarized follows:
We propose the first psychiatry diagnosis agent framework, MoodAngels, specifically targeting mood disorders. Our agents effectively address challenges such as the lack of objective diagnostic tools, symptom overlaps, and clients misjudging or misrepresenting their mental states, achieving high diagnostic accuracy.
We present MoodSyn, open-source dataset of 1,173 synthetic psychiatric cases that provides clinically plausible alternatives to sensitive patient data while preserving key statistical patterns for mood disorder detection research.
Experimental results validate the effectiveness of MoodAngels, with all agent variants outperforming bare L.MMs. Notably, on real-world data, even our raw agent surpasses GPT- 4o by 12.3% in accuracy. while the multi-agent framework further enhances performance. Ablation studies highlight the contributions of the granular-scale analysis and multi-step verification, underscoring their importance in achieving robust diagnostics.
2 Retrieval-augmented Multi-agent Framework
In this section, we propose a -arievee--egeeeted multi-agent framework tailored for the diagnosis of mood disorders integrating structured clinical knowledge with dynamic decision-making The framework consists of three specialized diagnostic agents, each performing an independent diagnostic process based on reorganized medical records and selected diagnostic scales. These agents employ two core analytical tools: a Granular-S Scalle Analysis module that matches symptoms to diagnostic criteria fine-grained level, and a Case-Basee Retrieval module that retrieves relevant prior cases from a curated knowledge base. The diagnostic opinions of the agents are then synthesized through a structured debate mechanism to form final judgment.
The key innovations of MoodAngels focus on the depth of symptom analysis and the design of diversified diagnostic reasoning processes. Unlike conventional methods that rely solely total scale scores, the Granular- more personalized and accurate understanding of individual differences. This aspect is particularly important in psychiatric assessment, where such variability is often overlooked. In addition, although case retrieval is commonly applied in general medical diagnosis, psychiatric evaluation requires more cautious application due to its subjectivity and high inter-individual variability. To address this challenge, the three diagnostic agents are designed with different levels of reliance on historical cases. This ensures that prior knowledge contributes meaningfully to the diagnostic process without dominating it. The complementary perspectives of the agents are ultimately integrated through structured debate, improving both the robustness and interpretability of the final diagnostic outcome.
Analysis method captures intra-scale symptom patterns, which supports a
2.1 Granular-scale Analysis
The primary challenge in constructing the diagnostic framework involves accurately identifying infor- mation that reliably reflects the visitor's actual condition. This difficulty arises because psychiatric diagnoses depend self-reported data and clinician-estmmated scales (supplemented by optional medical records), which carry inherent risks of misjudgment or misrepresentation.
To address this challenge, we introduce an innovative granular-scale analysis approact Rather than evaluating scales solely through total scores, we decompose them into individual item-level
'The problem definition is provided in Appendix B.1
Wee provide simple descriptions of scales and the selection process of relevant questions in Appendix B.2
3

---

responses. We employed comprehensive set of clinical scales to assess various aspects of mental health, including eight self-reported scales and five Siiiicinn-evaluater scales. These tools provide valuable insights into both subjective experiences and symptoms, as well as clinical observations and structured evaluations of clients' mental states and behaviors. All scales are routinely used in hospital settings for clinical diagnosis, ensuring the reliability and validity of the collected data.
To identify the most relevant questions for mood disorder diagnosis, we computed the Pearson correlation between each question's 's score (and total score) and the presence of a mood disorder, selecting the top 5% with the highest correlations. These questions naturally clustered into key symptom groups: depressive mood, loss of interest, anxiety. insomnia, and suicidal tendencies. These groups enhance diagnostic robustness through pross-validatte and comprehensive symptom coverage. To further refine our framework, we included clinically significant PHQ-9 questions, such as phq9_ __ (depressed mood) and phq9_0 (loss of interest), even if their correlation scores were slightly below the threshold, ensuring nuanced and reliable diagnostic process.
By evaluating response consistency within each symptom group, we derive more accurate inferences about the visitor's 's probable condition. For instance, when a visitor reports frequent depressive symptoms on self-assessment scale but clinicians observe no corresponding depressive signs, this discrepancy directs MoodAngels to investigate additional behavioral markers for diagnostic validation.
2.2 Retrieval Datastore
Since overlapping symptoms may correspond to multiple disorders, we extract and structure diagnostic and differential criteria from the Diagnostic and Statistical Manual of Mental Disorders: DSM-5 [13), a widely recognized authority in psychiatry, to build a retrievable knowledge base. The knowledge base construction process is detailed in Appendix [B.322
To prevent AnodAngels from making arbitrary decisions based solely on symptom presentation, we also incorporate clinicians' diagnostic expertise by including anonymized clinical data for retrieval. These experiences are also beneficial when a visitor's s symptoms are ambiguous. for historical diagnostic precedents may offer additional interpretive insights. The clinical data used in study consists of anonymized real-world hospital cases, totaling 2804 entries. We partitioned the dataset such that 80% of the cases are used historical cases for retrieval, while the remaining 20% serve as the test set. All clients in the dataset have completed scale assessments, although clients without diagnosed conditions do not have medical records available, and our agents are not pre-informed about this distinction. The dataset statistics are summarized in Table
Table I: Dataset statistics of real-world clinical cases.
Normal
Mood Disorder
Other Discases
Total
cases for retrieval
1259
759
225
2243
cases for test
315
190
56
561
2.3 Diagnostic Agents
To mitigate overreliance on past cases (which could overlook individual variability in psychiatric diagnoses), we develop three diagnostic variants with differing levels of historical dependence: Angel.R (no reference to previous cases), Angel.D (displays retrieved cases as context), and Angel.C (compares each retrieved case with the current query and returns an analysis as context). Prompts of this angels are provided in Appendix [.I By aggregating independent diagnoses from these three agents and facilitating debate among their conclusions, our final diagnosis model, multi-Angels, bridges the gap between computational decision-making and the nuanced understanding essential for accurate psychiatric evaluations. The following parts introduce the main components of our agents:
Symptom Matching, To align client symptoms in medical records with DSM-5 diagnostic criteria, we process records and compute relevance between records and criteria using dense vector encoding The BGE-M3 embedder [19 is employed for its strong semantic embedding capabilities. We retrieve
'The processing steps of medical records are detailed in Appendix C.2.1 and the formulation of symptom matching is provided in Appendix C.2.2

---

the top-5 most similar criteria, returning their text, classification, and similarity scores. The tool does not diagnose but provides results for agent analysis, ensuring decisions integrate quantitative data and clinical expertise, mitigating over-reliance on single metrics.
For cases with overlapping symptoms, additional instruction prompts the agent consider differential diagnosis, guiding systematic evaluation of potential conditions. This enhances the agent's ability distinguish between mood disorders and other diseases.
Scale Performance Analysis. We diagnose the presence of mood disorder using 16 key questions selected in Section 2.1 the most mood-relevant items. Client performances converted from numeric scores to textual descriptions based on question content and options. For agent interpretability. performances reorganized into coherent descriptive paragraphs, enhancing analysis effectiveness (examples are provided Appendix C.3)
Similar Cases Retrieval. To leverage clinical experience from similar cases, we develop two optional tools for retrieving medical records and scales with similar performance. After performing similarity retrieval, our tools generate different outputs tailored to the type of diagnosis agent in use. For Angel.R, this tool intentionally excluded to minimize potential interference from the diagnostic outcomes of other cases. For Angel.D, the tool directly returns the retrieved cases for reference, enabling the agent to review and draw insights from them. For Angel.C, the tool conducts a detailed comparison of similarities and differences among the retrieved cases and returns analysis text summarizing the findings. Consistent with the symptom matching tool, we employ BGE-M3 as the retriever. This approach ensures that the tool adapts to the specific needs of each diagnosis agent, enhancing the diagnostic process while maintaining flexibility and precision.
Multi-agent Diagnosis. To integrate insights from all Angels and improve diagnostics, Angel.R, Angel.D, and Angel.C first provide independent decisions and reasoning. A Judge Agent consolidates their inputs. If consensus is reached, the Judge outputs the diagnosis and reasoning. For disagreements, two Debate Agents are introduced: a Positive Agent, supporting a mood disorder diagnosis, and Negative Agent, opposing it. Both Debate Agents and the Judge access symptom matching results, scale performances, relevant cases, and the Angels' diagnosis and reasoning. In each debate round, the Positive Agent speaks first, followed by the Negative Agent. After each round, the Judge evaluates the arguments and decides whether to conclude the dehate. If concluded, the Judge delivers the final diagnosis and supporting reasons. More details about the judge and complete debate example presented in Appendix C.4.1
3 MoodSyn Dataset
The MoodSyn dataset addresses the critical need for clinically valid yet privacy-preserve data in computational psychiatry by introducing open-source collection of 1,173 synthetic cases, the statistics shown in Table Each case captures the essential characteristics psychiatric assessments through 25 carefully selected features, including 16 diagnostic questions, 8 standard scale scores, and expert-verified mood disorder labels. A data example is shown in Appendix D.I
Table 2:
Descriptive statistics of the synthetic MoodSyn dataset. Positive counts represent cases with mood disorder diagnoses, while negative counts indicate an absence of mood disorders.
positive amounnt
negative amount
Total
cases for retieval
687
419
1106
cibes for test
73
67
140
Dataset Construction. The MoodSyn dataset constructed on an advanced synthesis pipeline built upon the TabSyn framework (20] which integrates variational autoencoders with diffusion models through several technical innovations. The architecture employs adaptive tokenization and hierarchical encoding to maintain the complex relationships between clinical features, while condi- tional denoising process preserves the nuanced symptom patterns characteristic of mood disorders. During generation, dynamic loss weighting ensures the mathematical consistency between individual question responses and their corresponding scale scores, and context-aware sampling maintains clinically meaningful feature correlatinns. The construction process is detailed in Appendix D.I
5

---

Dataset Evaluation. Following Zhang et al. [20], we conduct a comprehensive evaluation of MoodSyn across five critical dimensions: statistical density, data quality, machine learning efficacy, privacy preservation, and logistic detectability (detailed in Appendix D.2). The results demonstrate exceptional fidelity to real clinical data, with MoodSyn accurately reproducing both core statistical patterns and complex symptom relationships while maintaining strong utility for machine learning applications. Specifically, our evaluation reveals high-density preservation of univariate and multi- variate distributions matching the original data's feature patterns, robust performance in downstream diagnostic prediction tasks comparable to real data, and near-indistn sesa--iddiiinguisaabbee tests. Furthermore, MoodSyn provides stronger privacy guarantees than traditional anonymization approaches through its synthetic generation process. This unique combination of statistical fidelity, clinical validity, and privacy protection establishes MoodSyn both a reliable analytical surrogate and a valuable resource for advancing AT applications in mental health research.
in rigorous detection
4 Experiment
4.1 Experimental Settings
Datasets. We assess the effectiveness of our agent framework on the test set described in Table I which comprises 561 cases. These include 315 normal cases, 56 cases of mood disorders, and 190 cases of other mental disorders. Cases without mental disorders contain only scale data and no medical records, whereas the remaining cases include both medical records and scale data. All cases are derived from real clinical data collected from our corporate hospital, ensuring the evaluation reflects practical diagnostic scenarios.
Baselines. To evaluate the effectiveness of our agent, we compare it against five baseline LLMS: -aaaMA8-----nss
[21], Misstrll-B--Instrettvv [22), GPT-4o (gpt-40-2024-06
23
DeepSeek-Ve V3 (24,, and medgemma-27b-tex-t. [25). Each LLM is provided with a standardized input consisting of: (1) A combined medical record, presented as a unified narrative. (2) A summary of scale test results, listing the client's scores across multiple psychological scales along with their clinical implications. The LLMs are prompted to determine whether the client has mood disorder (including depression or bipolar disorder) and to provide a structured explanation in JSON format. An example of the prompt used for baseline models is shown in Figure g
Hy pyperparaaeters. The retrieval number l for Angel.D and Angel.C is set to k 5 by default. For each model, we employ default parameter settings, utilizing official models for open-source LLMs obtained from Hugging Face or the API from the official website. These testing procedures take place on computational infrastructure consisting of two NVIDIA A800 Tensor Core GPUs, equipped with 80GB of memory.
Evaluation Metrics. Following previous methods [66, we utilize Recall, Accuracy (ACC), Matthews Correlation Coefficient (MCC) (26), and Macro Fl evaluate the performance of various versions of Anoels and baseline LLMs on the mood disorder diagnosis.
More details about baseline prompts and the selection of retrieval number k are provided in Ap- pendix
4.2 Analysis of Experimental Results
Our diagnostic framework achieves superior performance through innovative components that trans- form psychiatric assessment. Unlike conventional approaches limited by total score interpretation of standardized scales, MoodAngels implements granular item-level analysis that identifies and groups mood-relevant questions, overcoming the inherent information loss of aggregate scoring. This technical advancement enables more precise detection of symptom patterns that would otherwise be obscured in traditional scale processing, The framework's 's structured diagnostic process represents another significant innovation, replacing the direct diagnosis generation used by baseline models with rigorous multi-step verification system. While Angel.R establishes core functionality through DSM- 5 criteria referencing, our more advanced variants incorporate clinical case retrieval and inter-agent debate, substantial departure from the direct inference approach employed by bare LLMss.
The experimental results presented in Tables 3 and demonstrate the clear advantages of this approach. We further observed that the generated reasoning demonstrates high quality, as detailed

---

All MoodAngels variants show substantial improvements over baseline LLMs, with even our foundational agent achieving 12.3% greater accuracy than GPT-4o, underscoring the effectiveness of our redesigned diagnostic process. A deeper examination of the agent comparisons reveals important insights about psychiatric diagnosis. The performance advantage of Angel. over Angel.R confirms the clinical value of historical cases as reference points, while the slight dip observed with Angel.C serves as a caution against over-reliance on past cases. Particularly telling are those instances where only one agent succeeds in making the correct diagnosis while others fail, highlighting both the complexity of psychiatric assessment and the complementary nature of different diagnostic approaches.
in Appendix E.2.1
We also conduct subgroup analysis on gender and age, confirming no significant bias in predictions (detailed in Appendix [.2.2). These findings collectively validate our multi-agen framework's s design, which synthesizes granular symptom analysis, structured diagnostie protocols, and balanced clinical experience integration through its collaborative dehate mechanism. The framework's 's ultimate performance superiority emerges from this sophisticated combination of innovations, each addressing specific limitations in conventional psychiatric assessment methods while working in concert to achieve more accurate and reliable diagnoses.
Table 3; Diagnosis performances of all methods on real data.
Model
Recall
ACC
MCC
Macro
le-context learning
LLaMA3-8B-nnsruct
0.393
0.506
0.315
0.492
Mistral 7B v0..
0.456
0.597
0.411
0.397
GPT-4o
0.631
0.797
0.639
0.792
Despsook-V3
0.703
0.847
0,716
0.841
medgemma
0.67
0,743
0.57
0.717
GPT-40 based agents
0.840
0,920
0.829
0.913
Angel.D 4o
0.845
0,923
0.837
0.917
Angel.C
0.848
0,914
0.814
0.906
multi Malli-Angelsaaee GPT
4a
0.881
0,925
0.834
0.917
Deepreek V3
agents
Angel.R. Decpucek
V3
0.863
0.927
0.841
0.920
Angel. Decyacek
V3
0.864
0.920
0.823
0.911
Angel.C Deepeesk-V3 V3
0.858
0.922
0.829
0.914
multi-Angels Decpseck
V3
0.866
0,923
0.832
0.916
Table 4: Diagnosis performances of all methods on synthetic data.
Method
Recall
ACC
MCC
Macro Fl
in co-iontent learning
LLaMA3-8B 8B Instruct
0,557
0.564
0.221
0.435
Mistrl-7B-llnttmee-00333
0.589
0.636
0.375
0.563
Deepweek V3
0.761
0,821
0.664
0.816
modgemma
0.535
0.704
0.525
0.704
Deepreek V3 baved agents
Despacek Angel.&DDsspackk
13
0.778
0.601
0.798
Angel Deepasek-V3
0.787
0.615
0.805
Angel.C neepecek-
0.816
0.821
0.642
0.821
multi-Angels pespreckk V3
0.824
0.821
0.642
0.821
4.3
Ablation Study
In this section, we adapt Angel.R on real data to test the efficiency of processed medical records and selected seales. Additionally, we evaluate the stability of our framework by changing parameter settings in Appendix E.3
7

---

4.3.1 Medical Record Format
In this section, we investigated whether structured medical records improve the performance of the agents. Initially, we tested different formats in the symptom matching process, which extracts diagnostic criteria from the DSM-5 based on a medical record. Comparing Setting and Setting 2, we observed only a slight decline the agents' performance. Upon further analysis of the diagnostic criteria retrieved for specific case, found that the extracted criteria were highly relevant and similar. In most instances, the diagnostic criteria extracted under both experimental settings were identical, demonstrating the robustness and inclusivity of our chosen retriever.
However, when medical records were presented to the agent in different formats, the agent's 's diagnostic performance showed significant variation. Comparing Setting 2 and Setting 3, where the same record format was used the symptom matching step but different formats were returned to the agent, we identified cases where the diagnosis was correct in Setting 2 but incorrect in Setting 3. We found that a continuous, narrative-style medical record contains more detailed descriptions, which better reflect the severity of the client's 's condition. In contrast, structured medical records tend to focus on objective facts, potentially overlooking nuanced symptoms. This makes it more challenging to identify patients with atypical presentations of mood disorders. For example, a patient with severe depression may not exhibit obvious signs of low mood or mania but instead feel a sense of hopeless calmness and express suicidal ideation. This "hopeless calmness" is clearly conveyed in narrative-style medical record but may be less apparent in a structured format.
In summary, while structured medical records offer consistency and objectivity, they may lack the nuanced details present in narrative-style records, which are crucial for identifying atypical or subtle symptoms of mood disorders. This highlights the importance of balancing structured data with rich, descriptive narratives to ensure accurate and comprehensive psychiatric diagnosis.
Table 5: Diagnosis accuracy of different medical record formats in symptom matching and agent processing steps. The numbers indicate different combinations of experimental settings for ease of reference. In the experimental analysis, these settings are directly referred to Setting 1, Setting 2 and Setting 3.
Symptom Matching
to Agent
unstructured
unstructured
0.920
0.829
2
structured
unstructured
0.918
0,002]
0.822
0.007.4
0.006]
0.015L
4.3.2 Selection of Scales
In this section, we compared the diagnostic accuracy of Angel.R when using two different input strategies: providing 16 selected questions most relevant to mood disorders from the 13 scales, and providing the total scores of all 13 scales without any filtering. The results in Table a significant performance drop of 6.8% in accuracy when using the unfiltered total scores, highlighting the importance of question selection. This substantial decline underscores that the inclusion of less relevant or noisy data from unrelated questions can adversely impact the agent's diagnostic precision. By focusing on the most pertinent questions, our agent achieves more reliable and accurate mood disorder assessments.
Table 6: Diagnosis accuracy of selected and unselected scales used in the agent processing steps. The numbers indicate different combinations of experimental settings for case of reference. In the experimental analysis, these settings are directly referred to as Setting 1 and Setting 4.
Scale Used
ACC
MCC
selected
0.829
unselocted
0.852
0.0681
0.708
0.121L
4.4 Case Study
To demonstrate the robustness and diagnostic capabilities of our agents, we present two critical scenarios: inter-scale conflicts and overlapping symptoms. In inter-scale conflict cases, clients
8

---

may misjudge their condition in self-reported scales or, due to certain personality traits or stress responses, may not fully disclose their true feelings to clinicians. In such situations, our approach of grouping and analyzing similar questions across different scales proves invaluable, enabling the agent to identify inconsistencies and arrive at more accurate diagnosis. Additionally, we examine cases with overlapping symptoms, where single symptom may be indicative of multiple diseases, significantly complicating the diagnostic process. In these instances, simple symptom matching alone may fail to pinpoint the accurate disease. However, by leveraging historical case experiences and employing a multi-agent debate framework to explore various possibilities, MoodAngels achieves a more nuanced and precise diagnosis, bringing it closer to the underlying truth. These case studies highlight the strengths of our method in handling complex real-world diagnostic challenges. The intuitive presentations of these cases are shown in Appendix E.4
Inter-Scale Conflicts. We analyze a case where self-reported scales conflict with clinician-evaluate performances. Although the client scored 11 on the PHQ-9, suggesting moderate depression, self- reports are subjective. Clinician evaluations across multiple professional scales (e-g., HAMD, HAMA, BPRS) revealed no depressive symptoms energy decline, providing more objective diagnostic assurance. Historical cases with similar self-reported depression but no climician-confirmed symploms further support this conclusion. While the client reported mild to moderate anxiety, which correlates with mood disorders, was insufficient for a diagnosis, especially given the clinician's 's negative findings. The absence of Sunnccan-ccofirmec symptoms led our agent to conclude no mood disorder, demonstrating its ability to resolve inter-scale conflicts through comprehensive analysis of grouped questions and historical data.
We analyze a particularly challenging diagnostic scenario with overlapping symptoms. While the client's self-reports indicated negative emotions, suicidal tendencies, and loss of interest, the clinician found none of these. However, the medical record confirmed the self-reports, also noting delusions and self-talk. Symptom matching with DSM-5 criteria further complicated the diagnosis, as the top five matched disorders spanned Mood Disorder, Personality Disorder, and Neurocognitive Disorder, none of which included the actual diagnosis of Schizophrenia. Only through historical case retrieval and multi- debate did the Judge Agent identify that the client had concealed symptoms during clinical interviews, ultimately leading to the correct Schizophrenia diagnosis. This case highlights the difficulty of diagnosing complex cases with overlapping symptoms and underscores our agent's strength in integrating diverse data sources to uncover hidden diagnostic patterns.
Overlapping Symptoms.
4.5 Error Analysis
We examined cases where the three single angels disagreed and instances where the multi-agent debate still resulted in incorrect diagnoses. These cases typically involved borderline symptoms or conflicts between medical records and scale performances. Conflicts between medical records and scale performances are particularly challenging, they often require additional information make a more confident diagnosis.
Borderline Cases. Some clients' self-reported and clinician-evaluatedd scale performances were highly consistent, both indicating mild degrees depression, anxiety, insomnia, and loss of interest. After discussion with our coauthor expert, we concluded that while the client does not currently meet the diagnostic threshold for major depressive disorder, they fall on the borderline between normal and mild depression. Such cases warrant close attention and monitoring.
Conflicts between Medical Record and Scale Performances. In some cases, the medical record indicates severe symptoms, while both self-reported and clinician-evaluater evaluated scales show minimal or no symptoms related to mood disorders, with most relevant questions scoring zero. This discrepancy may arise when the client is in remission from a severe mood disorder, when medication temporarily suppresses negative emotions, or when the client has recovered. Recognizing this challenge, we enhanced our agents to flag such cases for clinicians, prompting further investigation. While the agents may misjudge the presence of a mood disorder these scenarios, this additional signal ensures that critical cases are not overlooked, thereby improving diagnostic reliability and clinical utility.

---

Conclusion
In conclusion, we propose MoodAngels, the first specialized multi-agent framework for mood disorder diagnosis that addresses key challenges in psychiatry through granular-scale analysis and structured multi-step verification, achieving superior aceuracy over existing methods. Our framework is complemented by MoodSyn, an open-source synthetic dataset of 1,173 clinically validated cases that enables research while preserving privacy. Experimental results demonstrate the effectiveness of our approach, with the baseline agent outperforming GPT-4o by 12.3% and the full multi-agent system achieving even greater accuracy. These contributions advance Al applications in mental health by providing both effective diagnostic framework and valuable rescarch resource that addresses critical gaps in computational psychiatry.
Ethical Statement
All medical record de-identification was conducted in an offline environment, ensuring no data was processed on external or networked systems. To further protect privacy, the medical records presented in this paper have undergone event and symptom obfuscation, along with partial modification. Additionally, all cases used in this study were included with the explicit consent of the clients, strictly for academic research purposes.
Evaluators' Background. Our evaluation team consists of coauthors who are experts in the field, led by a professional attending physician and professor with over 20 years of clinical experience. This lead evaluator is affiliated with one of the most authoritative hospitals in our country, bringing unparalleled expertise and credibility to the evaluation process. Their deep understanding of psychiatric disorders and extensive clinical background ensured a rigorous and reliable assessment of the knowledge base.
Limitations
Our study is limited by the available data, which only includes client medical records and scale scores, restricting ability fully replicate the comprehensive diagnostic process employed by clinicians. Clinician diagnoses typically involve around one hour patient interviews, during which additional clinician-evaluated scales are completed and more granular judgments are made based on real-time interactions. While our approach cannot fully capture this in-depth process, it still provides valuable insights by leveraging existing data support mood disorder diagnosis, offering promising tool for assisting clinicians in cases where complete diagnostic information may not be available.
Acknowledgments
This work is partially supported by Key Project of the National Natural Science Foundation of China (U23A20316). National Key Research and Development Program of China (2024YFC3308400) and CCF-Tencent Rhino-Bird Open Research Fund (CCF-Tencent RAGR20250115). We grate- fully acknowledge all clinicians who participated in the controlled reader study for their valuable contributions to model evaluation.
References
[1] Dominic Murphy and Robert L Woolfolk. The harmful dysfunction analysis mental disorder. Philosophy, Psychiatry, & Psychology, 7(4):241-255 2000.
[2] Steven E Hyman. The diagnosis of mental disorders: the problem of reification. Annual review of clinical psychology, 6 6(1):155-179 2010.
[3] Robert. J Haggerty and Patricia J Mrazek. Reducing risks for mental disorders: Frontiers for preventive intervention research. National Academies Press, 1994.
[4] Carolyn E Schwartz, Robert M Kaplan, John P Anderson, Troy Holbrook, and M Wilson Genderson. Covariation of physical and mental symptoms across illnesses: Results of factor analytic study. Annals of Behavioral Medicine, 21(2):122-127, 1999.
10

---

[5] Jeffrey Rakofsky and Mark Rapaport. Mood disorders. CONTINUUM: Lifelong Learning in Neurology, 24(3):804-88) 827, 2018.
[6] Eiko Fried, Claudia D van Borkulo, Angélique OJ Cramer, Lynn Boschloo, Robert Schoevers, and Denny Borsboom. Mental disorders networks of problems: review of recent insights. Social psychiatry and psychiatric epidemiology, 52:1-10, 2017.
[7] Junkai Li, Yunghwei Lai, Weitao Li, Jingyi Ren, Meng Zhang, Xinhui Kang, Siyu Wang, Peng Li, Ya- Ra-Qin Zhang, Weizhi Ma, et al. Agent hospital: A simulacrum of hospital with evolvable medical agents. arXiv preprint arXiv:2405.02990
2024.
[8] Haochun Wang, Sendong Zhao, Zewen Qiang, Nuwa Xi, Bing Qin, and Ting Liu. Beyond direct diagnosis: Llm-based multi-specialist agent consultation for automatic diagnosis. arXiv preprint urXiv:2401.16100 2024.
[9] Omid Kohandel Gargari, Farhad Fatehi, Ida Mohammadi, Shahryar Rajai Firouzabadi, Arman Shafiee, and Gholamreza Habibi. Diagnostic accuracy of large language models in psychiatry. Asian Journal of Psychiatry, 100:104168, 2024.
[10] Daniel F Gros, Matthew Price, Kathryn M Magruder, and B Christopher Frueh. Symptom overlap in posttraumatic stress disorder and major depression. Psychiatry research, 196(2-3); 267-270, 2012.
[11] Miriam K Forbes. Implications of the symptom-level overlap among dsm diagnoses for dimensions of psychopathology. Journal of Emotion and Psychopatholoy, 1(1):144-11, 2023.
[12] John A Bilello. Seeking an objective diagnosis of depression. Biomarkers in medicine, 10(8): 861-875, 2016.
[13] Shitij Kapur, Anthony G Phillips, and Thomas Insel. Why has taken so long for biological psychiatry to develop clinical tests and what to do about it? Molecular psychiatry, 17(12): 1174-1179, 2012.
[14] Yu He Ke, Rui Yang, Sui An Lie, Taylor Xin Yi Lim, Hairil Rizal Abdullah, Daniel Shu Wei Ting, and Nan Liu. Enhancing diagnostic accuracy through multi-agent conversations: Using large language models to mitigate cognitive bias. arXiv preprint arXiv:2401.14589, 2024.
(15) DSMTF American Psychiatric Association, DS American Psychiatric Association, et al. Di- agnostic and statistical manual of mental disorders: DSM-5, volume 5, American psychiatric association Washington, DC, 2013.
[16] Nawshad Farruque, Randy Goebel, Sudhakar Sivapalan, and Osmar R Zaïane. Depression symptoms modelling from social media text: an Ilm driven semi semi-supervised learning approach. Language Resources and Evaluation, pages 1-29, 2024.
[17] Yuxi Wang, Diana Inkpen, and Prasadith Kirinde Gamaarachchige. Explainable depression detection using large language models social media data. In Proceedings of the 9th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2024), pages 108-126, 2024.
[18] Daeun Lee, Hyolim Jeon, Sejung Son, Chaewon Park, Ji hyun An, Seungbae Kim, and Jinyoung Han. Detecting bipolar disorder from misdiagnosed major depressive disorder with mood- multi-task learning. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume I: Long Papers), pages 4954-4970, 2024.
[19] Jianly Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. Bge m3- embedding: Multi-lingual, caltiffunctionaaity, multi-granularity text embeddings through self-knowledge distillation, 2024. URL httt.tttt:///rriiii http//axxivvorr..a////// wtggabs////e 2402. 22t.//22222.212166.
[20] Hengrui Zhang, Jiani Zhang, Balasubramaniam Srinivasan, Zhengyuan Shen, Xiao Qin, Christos Faloutsos, Huzefa Rangwala, and George Karypis. Mixed-type tabular data synthesis with score-based diffusion in latent space. The twelfth International Conference on Learning Representations, 2024.

---

[21] Aaron Grattafioni, Abhimanyu Dubey, Abhinav Jauhri, et al. The Ilama herd of models, 2024.
URL luttps://arxiv. http:///aaxii..o//ab/////// 21783
[22] Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023. URL org/ abs/ 0alos/231000600
[23] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint 1rXiv:2303.08772 2023.
[24] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng. Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024.
[25] Andrew Sellergren, Sahar Kazemzadeh, Tiam Jaroensri, Atilla Kiraly, Madelcine Traverse. Timo Kohlberger, Shawn Xu, Fayaz Jamil, Cian Hughes, Charles Lau, et al. Medgemma technical report. arXiv preprint arXiv:2507.05521, 2025.
[26] Brian W Matthews. Comparison of the predicted and observed secondary structure of 14 phage lysozyme. Biochimica et Biophysica Acta (Bote)Protein Structure, 405(2):442 1975.
[27] Kailai Yang, Shaoxiong Ji, Tianlin Zhang, Qianqian Xie, Ziyan Kuang, and Sophia Ananiadou. Towards interpretable mental health analysis with large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6056-6077, 2023.
[28] Kailai Yang, Tianlin Zhang, Ziyan Kuang, Qianqian Xie, Jimin Huang, and Sophia Ananiadou. Mentallama: interpretable mental health analysis on social media with large language models. In Proceedings of the ACM Web Conference 2024, pages 4489-4500, 2024.
[29] Zhuang Chen, Jiawen Deng, Jinfeng Zhou, Jincenzi Wu, Tieyun Qian, and Minlie Huang. Depression detection in clinical interviews with empowerdd structural element graph. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume I: Long Papers), pages 8174-8187, 2024.
[30] Ana-Maria Bucur. Leveraging I1m- -lnegeeertted data for detecting depression symptoms on social media. In International Conference of the Cross-Language Evaluation Forum for European Languages, pages 193-204. Springer, 2024.
[31] Benyou Wang, Qianqian Xie, Jiahuan Pei, Zhihong Chen, Prayag Tiwari, Zhao Li, and Jie Fu. Pre-trained language models biomedical domain: systematic survey. ACM Computing Surveys, 56(3):1-52, 2023.
[32] Qianqian Xie, Qingyu Chen, Aokun Chen, Cheng Peng, Yan Hu, Fongei Lin, Xueqing Peng, Jimin Huang, Jeffrey Zhang, Vipina Keloth, et al. Medical foundation large language models for comprehensive text analysis and beyond. npj Digital Medicine, 8(1):141, 2025.
[33] Qianqian Xie, Weiguang Han, Xiao Zhang, Yanzhao Lai, Min Peng, Alejandro Lopez-Lira, and Jimin Huang, Pixiu: A comprehensive benchmark, instruction dataset and large language model for finance. Advances in Neural Information Processing Systems, 36:33469-33484, 2023.
[34] Qianqian Xie, Weiguang Han, Zhengyu Chen, Ruoyu Xiang, Xiao Zhang, Yueru He, Mengxi Xiao, Dong Li, Yongfu Dai, Duanyu Feng, et al. Finben: A holistic financial benchmark for large language models. Advances in Neural Information Processing Systems, 37:95716-955 957433, 2024.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and the last paragraph of the introduction summarize contri- butions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
In the limitation section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
13

---

This paper is an application study without new theoretical assumptions pro- posed.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
We provide all code and synthetic data in supplemental materials.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
14

---

Answer: [Yes]
Justification:
We provide all code and synthetic data in the supplemental meterial.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
il for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ cppppi.ccppulli//uuuia wwwiiicgguie//eeeeeeeeeic CoodeSubeiiieeee ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We mentioned them in experimental settings.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: In error analysis section.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
15

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
In experimental settings.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /tte::///euip...oeoe httpr//ps..............cccceeead
Answer:
[Yes]
Justification: In ethical statement section.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
In contribution paragraph of introduction.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
16

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification:
We provide usage declaration in supplemental materials.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: We don't use any existing dataset in the paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppsiipteeeee www.rrewwthhhe....c. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
17

---

Answer: [Yes]
Justification:
We provide it in supplemental materials,
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification:
In supplemental materials.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification:
We upload the IRB approvals in supplemental materials.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes]
18

---

Justification:
The LLMs are served as our backbone model.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy httppses//earrppe.cccc/ooorrrers for what should or should not be described.
19

---

A Related Work
A.1 LLM-based Depression and Bipolar-disorder Detection
Recent advancements in depression detection 27 28 have primarily focused on leveraging LLMs to address the challenges of psychiatric diagnosis. Chen et al. [29 proposed structuring clinical interviews into directed acyelic graph to enable automatic diagnosis, though this approach may struggle with cases where clients misrepresent their conditions. Social media platforms have also become a valuable resource for depression detection due to the abundance of user-generated content. Farruque et al. [16] fine-tuned a pre-trained language model on specific datasets to detect depressive symptoms from self-disclosed tweets, while Wang et al. [17] employed LLMs to identify depression- related text and predict depression levels from Reddit posts. Additionally, Lee et al. [18] utilized historical mood swings from users' past social media activities detect bipolar disorder. Efforts to enhance data quality and model performance have further explored synthetic data generation; for instance, Bucur (30] used L.LSs generate synthetic data for BDI-II symptoms, enriching datasets with semantic diversity and emotional experiences unique to Reddit posts. These approaches collectively highlight the potential of LLMSs and social media data in advancing depression detection, though challenges remain in ensuring accuracy and addressing client misrepresentations.
A.2 Medical Diagnosis Agents
Recent studies have begun to explore the potential of LLMs 31 in medical diagnosis, though these approaches often face limitations when applied to psychiatric contexts. Li et al. 7 proposed general disease diagnosis framework that leverages medical examination reports, such as blood tests and cell staining, to retrieve similar cases and rules. A doctor agent then evaluates the information to make a diagnosis, and correctly diagnosed cases are added to a historical database for future retrieval. However, this method is unsuitable for psychiatric diagnosis due to its lack of patient data anonymization, which could lead to privacy concerns and patient resistance. Additionally, psychiatric diagnosis lacks objective diagnostic tools and often involves overlapping symptoms, making rule- based approaches ineffective. Similarly, Wang et al. [8] enhanced agent expertise using external knowledge from the National Institutes of Health for conditions like diarrhea and bronchitis. Their agent-specialist outputs probability distribution over possible diagnoses based on patient-reported symptoms. However, this approach struggles with psychiatric cases where clients may misrepresent or inaccurately estimate their mental state, highlighting the need for more nuanced frameworks tailored to mental health diagnoses.
B Details of MoodAngels Framework
B.1 Problem Definition
Mood disorder diagnosis is a binary classification task aimed at determining whether a client has a mood disorder, such as depression or bipolar disorder, based on given case of clinical data. The input for each case includes structured clinical information, which consists of medical records (which may be absent for first-time visitors) and scale performance data (which is always available). The output of the task is a diagnostic result, represented as a binary decision (yes or no), along with supporting reasons that justify the diagnosis. Formally, for given case = (M,S). where M represents the medical records and S represents the scale performance data, the goal is to determine f(C) = (g. r). Here, ğ E {0,1} is the predicted diagnosis (1 indicating the presence of a mood disorder and O indicating its absence), and is the reasoning that supports the prediction. This task is particularly challenging due to the potential absence of medical records for some cases and the need to provide interpretable reasoning for the diagnostic decision.
Currently, our framework performs binary classification to determine whether a mood disorder is present. While we have extracted the DSM-5 differential criteria for more fine-grand inn-ggrined distinctions, we do not attempt subcategory classification. This is because psychiatric diagnosis requires careful deliberation; in clinical practice, clinicians typically first identify broad diagnostic direction before narrowing it down through further assessments. Similarly, our framework is designed to support this initial diagnostic stage.
20

---

B.2
Granular-scale Analysis
B.2.1
Scale Descriptions
Self-evaluated scales:
(I) CTQ (Childhood Trauma Questionnaire). A retrospective self-report inventory that measures the severity of childhood trauma across five domains: emotional, physical, and sexual abuse, well as emotional and physical neglect. The scale content is available at CTQQ
(2) DAS (Dysfunctional Attitude Scale). A self-report measure that assesses maladaptive cognitive patterns and beliefs associated with depression. The scale content is available at DAS
(3) GAD-7 (Generalized Anxiety Disorder-7). A self-report scale used to assess the severity of generalized anxiety disorder symptoms over the past two weeks. The scale content available at GAD-7
(4) HCL-32 (Hypomania Checklist-32). A self-report questionnaire designed identify symptoms of hypomania, often used the assessment of bipolar disorder. The scale content is available at HCL-32
(5) MDQ (Mood Disorder Questionnaire). self-report screening tool used identify symptoms of bipolar spectrum disorders. The scale content is available at MDQ
(6) NSSI (Non-Suicidal Self-Injury Assessment). A self-report measure designed assess the frequency, methods, and motivations behind non-suicidal self-injurious behaviors. The scale content available at NSSI
(7) PHQ-9 (Patient Health Questionnaire-9), A self-admminitered tool used to screen for and measure the severity of depression based on the DSM-5 criteria. The scale content is available at PHQ-9
(8) SHAPS (Snaith-Hamiltee Pleasure Scale). A self-report measure designed to assess anhedonia, or the inability to experience pleasure, commonly associated with depression. The scale content is available at (HAPSS
Clinician-rated scales:
(9) BPRS (Brief Psychiatric Rating Scale). A climician-administeeee scale that evaluates broad range of psychiatric symptoms, including psychosis, depression, and anxiety. The scale content is available at (PRSSS
(10) HAMA (Hamilton Anxiety Rating Scale). A the severity of anxiety symptoms based on observed and reported behaviors. The scale content available at HAMA
scale used to measure
(11) HAMD (Hamilton Depression Rating Scale). clinician-rated tool that assesses the severity of depressive symptoms, widely used in clinical and research settings. The scale content is available HAMD-24 In this study, we employ the Chinese version of the HAMD-24, which features different question order compared to the English version.
(12) MCCB (MATRICS Consensus Cognitive Battery). A comprehensive, mumnician-administerr battery of tests designed to measure cognitive functioning in individuals with schizophrenia and other psychiatric disorders. The scale content is available at MCCB
(13) YMRS (Young Mania Rating Scale). clinician-rated scale used to assess the severity of manic symptoms in individuals with bipolar disorder. The scale content is available at YMRS
B.2.2 Relevant Question Selection
We calculated the correlation scores between each question and the total score across the 13 scales mentioned above, well their correlation with the presence of mood disorders, using training set of 2,243 cases. The results are presented below:
21

---

Question
Correlation Score
P-p-value
Question ID
Correlation Score
PC p-valee
J. C7Q
das Q31
0.3770
4.3e:76
clq_Q1
0.1771
3.9c-17
das _
0.3108
4.9c-51
c2q_Q2
0.1259
2.5e-09
das Q33
0.2986
4.8c-47
cs
0.2428
3.10-31
das Q34
Q4
0.2000
1.6e-11
das_ _35_
0.1615
1.8c-14
0.0914
1.6c-05
das
0.2307
2,9e-88
c6
0.1797
1.3c-17
das_ Q37
0.2021
6.1e-22
c7
0.2371
8.0e-30
das_ _s8_
0.2937
1.6e-15
Q8
0.4358
7.56-104
das
0.2404
1.2o-30
cig Q9
0,1150
5.3e-08
das_ des_Q40
0.2141
1.7e-24
eqg_Q10
0.3248
7.60-56
das total. score
0.4389
2.0e- 105
eq
0.2805
1.6c-11
Q12
0,2726
3.
gad7_Qu
0.4377
1.1e-104
cig
0,2751
6.2o-40
gad7_Q2
0.4492
8.3e-111
cuq Q14
0.3973
4.6e 85
gad7_Q3
0.4393
1.6e-105
Q15
0.2841
1.4e 42
gad7 __Q
0.4522
2.0e-112
ciq
0.2860
3.70-43
gad7_Q5
0.3891
3.3e-81
ciq Q17
c1
0.1606
6.3e-10
gad7_Q6
0.4661
2.9c-120
19_Q18
0,3640
1.0c-70
gad7 __0
0.3493
8.9e-65
csp_119
0.2959
gad7_total__se
0.5116
1.7c-148
c20
ciq Q20
0.1276
1.5e-00 09
4. HCL-32
csp 021
0,0810
1.3c-04
het32_Q1
0.0179
4,0c-01
cup
he132_Q2
.0.1972
2.9e:21
023
0.1246
3.7c409
hel32_Q3
.0.2632
1,3e.36
cug Q24
0,1690
9.9c-16
he132_Q4
-0,2292
6.50-88
c25
0.3855
he132_Q5
.0.1799
1.2e 17
c2q_
0,1815
6.3c-18
be132_06
-0,1501
1.10-12
etp Q27
0.0478
2.4e-02
Q7
e2q_Q28
0.3101
8.0e551
he132_Q8
0.2175
3.
2. DAS
he132_Q9
-0,0151
4,8c-01
das_Q1
0.2455
6.9e-32
hc132_Q10
.0.2148
das_Q2
0.0296
1.6c-01
be132_Q11
-0.1509
8.20-13
das
0,2930
2.7e-45
be132_Q12
-0,2202
7.4e-26
das_ Q4
0,3332
8.10-59
hc132_Q13 _12
.0.2187
das _5
0.2801
2.2e
be132_Q14
-0,0594
5.0c-03
das Q6
0,1511
7.7e-13
he132_00
.0,2313
2.0e 28
das_ Q7
0.3111
4.10-51
be132_Q16
0.0177
4,0c-01
das Q8
0.3161
8.1e.53
be132_Q17
-0.0184
3.9c-01
das_
0.3706
2.30-73
he132_00 _18
.0.1769
4,2e 17
das Q10
0.3823
2.4-788
be132_Q19
-0,2283
1.0c-27
das, Q11
0.3618
8.6e: 70
bel32, Q20
.0.1844
1.8e-1 18
Q12
0.1682
1.40-15
be132_
0.1239
4,5e 09
dax Q13
0.2901
4.1
be132_Q22
-022486
1.0c-32
das_ das_Q14 Q14
0.3695
6.10-73
he132_ Q23
0.0968
4.7e
das_ Q1__115
0.3308
6.0c-88
be132_Q24
-0.1985
3.20-21
dax Q16
0.3399
2.8e-61
he132_Q25
0.2063
8.0e
das_ Q17
-02876
1.2c-43
hel32_ _26
0.1971
6.2e 21
Q18
4.4e
be132_Q27
0.1479
2.3e-12
dax Q19
0.3015
5.4e 48
he132_Q28
.0.2751
6.2e 40
das_ Q2_
0.2773
1.4e400
he132_Q29
0.0465
2.8c-02
Q21
0.1974
be132_Q30
-0.0205
3.3c-01
das Q4_
0.2228
2.0x226
hc132_Q31
0.0117
5.8e 01
Q23
0.3144
3.0e-52
be132_Q33
0.0905
1.9c-05
Q24
0.0429
hc132 Jotal_score
.0.0788
das Q2_
-0.0428
4,3o002
MDQ
Q26
0.3093
1.6e-50
mdq_ Q1
0.1832
Q2?
0.3224
5.70-55
mdq_Q2
0.2916
7.3e-45
das, Q28
0.2095
1.7o-23
mdq_Q3
-0.1594
3.90-14
Q29
0.1205
1.2e 08
mdq__Q4
0.2980
das_ Q30
0.0003
9.9c-01
mdq_Q5
0.0515
1.5e-02
22

---

Question ID
Correlation Score
PC p-value
Question ID
Correlation Score
PC p-valle
md
-0.0556
8.8c-03
bprs_02 _
2.7e-127
mdq_Q7
0.2141
1.8e-24
bprs_Q6
0.1560
1.1e-13
mdq_Q8
-0.1568
1.0c-13
Q27
0.0003
9.9e-01
m9
40.1477
9.7e-12
bprs_Q6
mdg _i0
0.0857
5.2e-05
bprs_Q9
0.6200
1.8e-288
miq_Q11
0,0907
1.8c-05
hpes_Q1 _1s_Q10
0.1341
1.8c- 10
mdq_Q12
bpn__11
0.3040
3.4e-49
mdg mdq_Q13 Q13
0.2920
5.6c-45
Opes_Q12
0.1088
2.4e-07
mág total score
0,1742
1.3e-16
bpss_Q13 Q13
0.3225
1.8e-55
6. NSS7
bpm__0
0.0363
8.5e-02
nisi_Q Q1
0,4597
4.0c-116
hpss_Q15
0.0224
2.9c-01
nssi, Q2
bprs_Q16
0.0859
4.6c-05
nesi Q3
0.0221
3.0e-11
bpm__117
0.2906
6,5e-45
nasi 14
0,1636
9.5c-15
hprs__11
-0.0506
1,7e-02
nssi Q5
1.4
opns_total_sooe
0.4724
4.2c-125
ussi_ Q6
0.2471
3.8c-32
10. HAMA
nesi Q7
0,3085
5.4e-50
hama,_
0.4227
6.2c-98
nssi Q8
6.5e-51
hama, Q2
0.4177
1.9e995
nssi_ 1
0.4045
6.8c-88
hara, Q3
0.2960
1,3e-46
Q10
0.3417
1.3e-61
Q4
0.5099
8.7c-149
mssiQ11 Q01
0,2181
3.10-25
harn, Q5
0.4987
2.2e-141 141
nssi Q12
0,2392
3.7e-30
hama__Q Q6
0.6304
7.4e-249
Q13
0,2664
2.8e-37
hama__7 Q7
0.2840
6.9c-43
mssi_11 Q14
0.2641
1.36-36
harn, Q8
0.2837
8.2c 43
nssi Q15
0.0729
6.0e004
hama__Q
0,3875
2.7c-81
nssi_Q16
-0.0225
1.0c-44
hama, Q10
0.3994
[ssi_Q1
0.1995
2.76-21
hama Q11
0.3641
2.7c-71
nssi Q18
0,0267
2.1e-01
hama Q12
0.1479
1.9e-12
7. PHQ-9
hama Q13
0.2420
phu9__1
0.5008
9.0e-143
hama Q14
0.1529
3,3e-13
phq9_Q2
0.5006
1.2c-142
hama, total.
0.5513
1.2e-178
phu9_Q3
0.4842
3:2e-132
HAMD
phug__00 _4
0.5097
1.2
hamd_Qq Qq
0.6021
8.16-219
pha9_Q5
0.4484
1.9c-111
phug_
0.4877
1.9e- 134
ha
0.6024
4.1c-219
phqg_Q7
0,4578
1.1c-116
hamd Q4
0.5247
4.3
phq9_Q8
0.4493
6.6c-112
bamd Q5
0.4385
7.26-105
phug_
0.5057
5.4e 146
hamd, Q6
0,3932
6,9c-83
2bq9_total_sooe
0,5920
2.7c-212
hamd
0.6079
2.8e 223
8 SHAPS
bamd Q8
0.3943
2.46-83
shaps_Q1
0,1860
8.4--19
hamd Q9
0.3444
9.0e-63
shups_00
0.2637
8.5e.37
hand [a0d
0.4053
2.1e.88
shaps_Q3
0.2561
1.0e-44
hamd Q11
0.4846
6.5c-131
shaps_ QQ
0,2186
1.6e-25
hamd Q12
0.4143
1.1e
shaps Q5
0.1925
4.7e-20
band @ind_Q13
0.4448
3.30-108
shaps Q6
0,1774
3.1c-17
hand Q14
0.1405
3.0e
shaps_ Q7
0,2355
1.8c-29
hamd Q15
0.1536
3.6e
shaps Q8
band [16
0.1968
8.5e-21
shaps Q9
0,2753
4.7c-00
hand Q17
0.2599
1.5e
shaps_Q10
0,1845
1.6e-18
hamd Q18
0.2671
1,50-37
shaps_Q1 Q11
0,2522
1.1o-33
band
0.3108
7.6c-51
shaps_Q12 Q12
0,2314
1.7e-28
hand Q20
0.2943
1.6e 45
shaps_Q11
0.1941
2.3e-20
bamd _21
0.2604
1.10-35
shaps_Q14
4.7c-14
0.5236
shaps Lotal Lotal score
0,2675
7.4e-38
hamd Q23
0.4595
3.20-116
band Q24
0.4293
3.9c-100
bprs_Q
0.1675
LAc-15
hand, Jotal, score
bprx_Q2
0.4255
2.2c-99
12. MCCB
bpps_0t Q3
8.9c-07
TMT-A
0.2300
bpes_Qu Q4
0.0232
2.7e-01
TMT-B
0.1530
4.20-13
23

---

Question
Correlation
p-value
BACS SC
-0.2589
2.3e-35
HVLI-R.1
-0.0672
1.5e-03
HVLT-R2
0.0157
4.6e-01
HVLT-R-3
0.0536
1.le-02
WMS-LII SS
-0.1140
7.1e-08
NAR Mares
-0.1402
3.1e-11
BVMI-R.1
-0,0217
3.1e-01
BVMI-R.2
-0,0255
2.3e-01
BVMT-R.3
-0.0510
1.6e-02
Fluency
-0,0489
2.1e-02
MSCEIT MR
-0.2084
CPT-IPI
-0.2915
8.8e-45
CPT-P2
-0.3012
8.2c-48
-0.2747
13. YMRS
ymrs_ Q1
1.1e:20
ymrs_Q2
0.2100
5.3c-18
ymrs_
0,0520
3,4e-02
ymas_
2.6e-15
ymus_ Q5
0.3067
1.90-37
ymrs_
0,2548
5.3e:26
ymrs_
ymus_
0.1106
6.4c-06
ymrs_
0.1501
8.0e 10
ymrs_ _10
-0.0730
2.9c-03
ymns Q1I
0.0900
2.4e.04
0.3196
1.0e.40
To ensure the selection of highly relevant questions, identified the top 5% of questions with the highest correlation coefficients. The distribution correlation scores and the selection threshold are illustrated in Figure 2 Question IDs with correlation scores above the threshold are highlighted in purple, and their corresponding correlation scores are bolded for emphasis. Upon reviewing the original questions selected, we observed that they naturally clustered into five distinct categories: five questions related to depressive mood, two related to suicidal tendencies, three related to loss of interest, two related to anxiety, and two related insomnia.
35
30
10
-0.4
Questions
Figure 2: The correlation distribution and threshold of related questions.
Notably, for questions related to depressive mood and loss of interest, we observed variations in how similar questions were phrased across different scales. To ensure robustness, we consulted our domain expert co-author and decided include two additional questions from the self-reported PHQ-9 depression scale: phq9 Q2 (depressed mood over the past two weeks) and phq9_ __Q (loss of interest over the past two weeks). We highlight these two question in teal. Although their correlation scores (0.5006 and 0.5008, respectively) were slightly below the threshold of 0.5032,

---

their inclusion provides valuable insights and enhances the diagnostic framework. This decision underscores the importance of multi-scale assessment in minimizing misjudgments of clients' mental states, reducing the impact of short-term emotional fluctuations, and mitigating potential inaccuracies due to inconsistent responses. By repeating similar questions across self-reported and clinician-rated scales, we can achieve more comprehensive and reliable evaluation. The detailed content and options of selected questions are listed in Table 7
25

---

Table 7: Question content and options of selected scale questions.
Question
Correlation
Question Conteut
Options
Depression revitre Performances
hamd_total_ae soorl_score
0,6348
total score Hamilton Depression
no depression, may have depression, 17-23=
Raiing Scale (HAMD).
must have depression, 24-76 = severe depresssion
homa_
0.6304
Depressed mood. Loss interest, lack of pleasure in hubbies, depression, early waking diurnal swing.
Q- Not preseur, Mild Medentte Severe, Very severe.
bpri_C99
0,6200
DEPRESSIVE MOOD. Despondency sadness. Rate only degrec spondasey; do not on the basis inferences concerning depression based upen general retundation and somaric com- plaints.
o not assessed, 1 not present, = very mild, 3 mild, -mddente, modirately severe, 6- severe, extemmly severe
hamd_Q
0.8021
DEPRESSED MOOD (sadness, hopeless, helpless, worthless]
O Absent. 1= These feeling indicated only
questioning.. 2- These feeling states spontanccusly reporsed verbally. = Communicates feeling states non verbally, La. through facial expression, posture, voice and tendency to weep. 4- Paseen rpprrts virtaally only fecling states hicher spectancous verbal non verbal. communication
phup_ spore
0,592
The total score of Patient Health Questionnire 9 (PHQ 9).
minimal deprossion, mild depression, 10-14 modersse depression. 19 = moderately severs depeession, 20-27 severt depression
phup.QQ
0.5006
Over the last 2 weeks, how often have you boen buthered any of the following problems?? Feeling deprossed, hopelesa.
a- Not at all, -everal days, 2 More than half the days, 3 Nearly every day
Suicide related Performances
hamd_Q3
0.6024
SUICIE
(1 Absent. = Feels life is not worth living. = Wishes he'she were dead or any thoughts of possible death to seli. 3 Ideas gestures suicide. Attempts suicide (aay serious attempt rate 4).
phyp_ Q9
0,5007
Over the 2 weeks, how you been bothered by any of the following problems? Thoughts that you would be better all dead, or of burling yourself.
Several days, 3 Neazly every day
half days,
Energy platerskltttee--eeiiee Peffrmances
hand__7
0.667
Work and Activities
No difficulty, Thoughts and fealings incapecity, fatigue, weakiness reluted activitics, work, hobbies, only reparied whas asked. 2 Spontancessly reports loss of interest in activities, work or hobbies, ciffier directly or indireclly such feding listlss, indecisive, needing to josh themselves in acnual time spent in activities or decrease in producavity. in hoepital setting, rite if the puient does spend at theee
angage activitios 3 Decresse
activities, exclosive 4- Stappe working due the current illness, hospital serting. rate if the patient engages in no aetivities except ward chores or if the patien falk to perform ward chores umassisted.
chores,
hond_Q22
0.5236
Feelings of tradequacy Reduced Abil- ity
Absear, Subjective feelings of inadquuccy only elicited questioning. 2 Patient spontaneously reports feelings of inadequacy). 3 = Needs encouragement, guad- anoe, and reassurance to complete daily tasks or personal hygaene, 4 Roquires assustance from others for dressing, grooming, caring, making the hel, or personal hygiene. days.
phyp_04
0,5097
Over the weeks, how often you been bothered by any of the following problems? Feeling tired or having littl energy.
o Not at Several days, 2 More = Neazly every day
phyP_Q1
0,5008
Over the last 2 weeks, how often have you bothered any of following problems? Little interes or pleasure in de- ing ihings.
0= Not at all, =s Several days, = More tham half the days, 3 Nearly every day
Ansicty- reiated Performunces
hooma Jotal score
0.5513
The total seare of Hamsiton Auxiety Rat- ing Scale (HAM-A).
(16 no anxiety, 7-13 - may have naxiety, 14-30 - imsst have anxiery, 21-28 have obvious anxiory, 29.36 severe asxiety
2od7_tomal
0.5185
The total score of Generalized Anxieey Disorder (GAD-)))
anxiety, 5-9 mild ansiety, 10-14 moderate anxiety, 15:21 = severe ansiety
Jusomnia related Performances
hamd Q4
0.5247
INSOMNIA: EARLY IN THE NIGHT
O= No difficulty willing I Complains of occusioral difficulty falling teleep, ie. more than half hour. 2
Complaints nightly diffeeslty falling aslecp.
homa, Q24
0.5099
Irscomnin. Difficulry falling asleap, hm-- slzep. unsatisfylin sleep and futigue on waking, dreams, nightmares, night teo- roes.
Not preseut, Mild, Mederate, Severe, 4 Very severe.
26

---

B.3 Retrieval Datastore
B.3.1 Mental Disorders
In this section, we provide brief overview of 18 common mental disorders, along with examples of their typical symptoms as outlined in the DSM-5, important to note that mental disorders are complex, and their manifestations often extend beyond the descriptions provided here. The following content intended offer a general understanding of these conditions and should not be considered exhaustive or definitive.
Neurodevelopmeental Disorders. The neurodevclopmental disorders are group conditions with onset in the developmental period. The range of developmental deficits varies from very specific limitations of learning or control of executive functions global impairments of social skills or intelligence. For example, individuals with neurodevelopmental disorders may have difficulties in speech and language development, problems with social communication and understanding social cues, repetitive behaviors, and restricted interests.
Schizophrenia Spectrum and Other Psychotic Disorders. Schizophrenia spectrum disorders characterized by a range of symptoms that affect thinking, perception, emotional regulation, and behavior. For example, individuals with auditory hallucinations would hear voices that others do not.
Bipolar and Related Disorders. Bipolar and related disorders are characterized by significant mood swings that include episodes of mania or hypomania (elevated mood) and depression. For example, individuals with mania may experience an abnormally elevated, expansive, or irritable mood, increased energy, reduced need for sleep, grandiosity, impulsivity, and excessive engagement in risky behaviors.
Depressive Disorders. Depressive disorders are characterized by persistent feelings of sadness, hopelessness, and a loss of interest or pleasure in most activities. For example, individuals with depression may experience changes in appetite or weight, sleep disturbances (either insomnia or excessive sleeping), fatigue, feelings of worthlessness or excessive guilt, difficulty concentrating, and thoughts of death or suicide.
Anxiety Disorders. Anxiety disorders are characterized by excessive fear, worry, or nervousness that is disproportionate to the actual threat situation. For example, individuals with generalized anxiety disorder (GAD) experience chronic, uncontrollable worry about various aspects life, such as work, health, or social interactions.
Obsessive- Compsuii-o and Related Disorders Obsessive-Compulsive and Related Disorders characterized by the presence of obsessions (intrusive, unwanted thoughts) and/or compulsions (repetitive behaviors mental performed to reduce anxiety). For example, individuals with Obsessive- Corssivv--omuusivee Disorder (OCD) experience persistent, distressing obsessions and feel com- pelled perform rituals or routines alleviate the anxicty caused by these thoughts.
Trauma- and Stressor-Related Disorders. Trauma- and Stressor-
Disorders are charac- terized by the emotional and psychological response to traumatic stressful events. For example, individuals with -rss-Trramatic Stress Disorder (PTSD) intrusive memories of the trauma (flash- backs, nightmares), emotional numbness, avoidance of reminders of the event, hypervigilance, and heightened arousal, such as irritability, difficulty sleeping, and exaggerated startle responses.
Dissociative Disorders. Dissociative disorders are characterized by disruptions or breakdowns in memory, consciousness, identity, or perception. For example, individuals with Dissociative Identity Disorder (DID), previously known Multiple Personality Disorder, exhibit two more distinct identities or personality states, each with its own pattern of thinking, feeling, and behaving. These identities may take control of the person's 's behavior and are often accompanied by gaps in memory awareness of time.
Somatic Symptom and Related Disorders Somatic Symptom and Related Disorders are character- ized by the presence of physical symptoms that cause significant distress or impairment, which are not fully explained by medical condition. For example, individuals with IlIness Anxiety Disorder (formerly known as hypochondriasis) may experience excessive worry about having or developing a serious illness, despite having little physical symptoms.
27

---

Feeding and Eating Disorders Feeding and Eating Disorders are characterized by persistent distur- bances eating behaviors and related thoughts or emotions that negatively impact physical health, emotional well-being, and daily functioning. For example, individuals with Anorexia Nervosa have an intense fear of gaining weight and engage in restrictive eating, leading to significantly low body weight. They may also have distorted body image, perceiving themselves as overweight even when underweight.
Elimination Disorders Elimination Disorders are characterized by the inappropriate elimination of urine or feces, which are not due to a medical condition. For example, individuals with enuresis refers to the repeated involuntary or intentional voiding of urine, typically at night (bedwetting), beyond the expected age for bladder control.
Sleep- Wake Disorders Sleep- Wake Disorders are characterized by disturbances in the quality, timing, and amount of sleep, leading to significant impairment in daytime functioning and distress. These disorders are not attributable to the physiological effects of a substance or another medical condition. For example, individuals with insomnia disorder experience persistent difficulty falling asleep, staying asleep, or achieving restorative sleep, despite adequate opportunity for sleep, resulting in fatigue, mood disturbances, and impaired cognitive or social functioning.
Sexual Dysfunctions Sexual Dysfumctions are characterized by a clinically significant inability to participate in or experience satisfaction from sexual activity, often causing marked distress or interpersonal difficulties. These dysfunctions are not better explained by a nonsexual mental disorder, relationship distress, or the effects of a substance medical condition. For example, individuals with erectile disorder experience a persistent or recurrent inability to attain or maintain an adequate erection during sexual activity, leading significant distress interpersonal strain.
Gender Dysphoria Gender Dysphoria involves a strong and persistent discomfort with one's 's assigned gender at birth and a desire to be treated as the opposite gender. This condition is marked by significant distress or impairment in functioning due to the incongruence between experienced or expressed gender and assigned gender. For example, a person assigned female birth may experience intense discomfort with their body, wishing to transition to a male gender identity, often leading to emotional and social distress.
Disruptive, Impulse-Control. and Conduct Disorders These disorders are characterized by persis- tent patterns of behavior where the rights of others or societal norms are violated. Individuals with these disorders often exhibit aggressive, antisocial, or impulsive behaviors that disrupt their social, academic, or occupational functioning. For example, child with conduct disorder may frequently engage theft, aggression toward others, or deliberate property destruction, showing little empathy or remorse for their actions.
Substance-Related and Addictive Disorders Substance-Relan and Addictive Disorders encompass a range of problems caused by the use of substances (c.g., alcohol, drugs) or behaviors (c.g., gambling) that lead to addiction. These disorders are defined by a pattern of substance use or behavior that leads to significant impairment, including physical, social, or psychological problems. For example, an individual with alcohol use disorder may find themselves drinking excessively despite negative consequences, such as relationship issues or health problems, and may experience withdrawal symptoms when not drinking.
Neurocognitive Disorders Neurocognitive Disorders involve a decline in cognitive function that represents significant change from previous level of functioning. These disorders can affect memory, learning, attention, executive function, and perception. For example, Alzheimer's 's disease common neurocognitive disorder where individuals experience progressive memory loss and confusion, often leading to difficulty performing everyday tasks.
Personality Disorders Personality Disorders characterized by enduring patterns of behavior, cognition, and inner experience that deviate markedly from the expectations of an individual's culture. These patterns inflexible and pervasive, leading to distress impairment in social, occupational, or other important areas of functioning. For example, individuals with borderline personality disorder may have intense and unstable relationships, fear of abandonment, and difficulty regulating their emotions, often leading to impulsive actions or self-harning behaviors.
28

---

B.3.2 Knowledge Base Construction
To gather professional diagnostic criteria, we rely on the Diagnostic and Statistical Manual of Mental Disorders: DSM-5 [5]. comprehensive and authoritative guide widely used by clinicians and researchers. The DSM-5 outlines the standardized criteria for the classification and diagnosis of mental disorders, providing detailed descriptions of symptoms, diagnostic features, and associated conditions. As a crucial tool in psychiatry, ensures consistency and accuracy in mental health diagnosis, making it an essential reference for both clinical practice and research.
Our knowledge base is built by extracting diagnostic criteria and symptoms from the DSM-5, focusing on mood disorders and other conditions. We reference the diagnostic criteria section to identify required symptoms and their severity, and the differential diagnosis section to distinguish mood disorders from similar conditions. This ensures accurate diagnosis while addressing symptom overlaps.
Diagnostic Criteria Extraction. Symptoms in the DSM-5 are listed individually. To facilitate comparison with client records, reframe these using GPT-4o for brevity and clarity, ensuring independent, complete criteria. We extracted symptoms for 18 common mental disorders, including all major symptoms of "bipolar and related disorders" and "depressive disorders. The extraction prompt is provided in Table
Table 8: Prompts of diagnostic criteria extraction.
To facilitate comparison between diagnostic criteria and medical records. please summarize the criteria for (disease name) from DSM-5 diagnostic manual in point-by-point fornat. point should fully describe symptom, explicitly include reference information where nccessary, and retain other relevant restrictions. Below the description of the diagnostic criteria for (disease nane) from the DSM-5: (the original criteria).
For example, one symptom of bipolar disorder is as follows:
"A distinct period of abnormally and persistently elevated, expansive, irritable mood and abnor- mally and persistently increased goal-directed activity or energy, lasting at least week and present most of the day, nearly every day (or any duration if hospitalization is necessary)."
The extracted symptom description would be:
"Manic episode: Elevated, expansive, or irritable mood, accompanied by increased energy and activity, lasting at least week and present most of the day, nearly every day."
Differential Criteria Extraction. We decompose complex differential diagnosis into distinct expressions using GPT-4o. This process ensures precise differentiation between mood disorder and other diseases. The extraction prompt is provided in Table
Table 9: Prompts of differential criteria extraction.
Please split the following differential criteria into two separate dingnostic criteria. Each point should fully descrihe a symptom, explicitly include reference information where nocessary, and retain ago other relevant restrictions. Below are the differential criteria to be split: the original criteris.
For example, differential symptom for bipolar disorder is described as:
" Attention-eeffcculyypratttiii disorder. This disorder may be misdiagnosed as bipolar disorder, especially in adolescents and children. Many symptoms overlap with the symptoms of mania, such as rapid speech, racing thoughts, distractibility, and less need for sleep. The 'double counting" of symptoms toward both ADHD and bipolar disorder can be avoided if the clinician clarifies whether the symptom(s) represent a distinct episode."
This description was refined into two distinct expressions:
disorder: rapid speech, racing thoughts, distractibility, and less need for sleep in adolescents and children."
iiiention-deffiiihyypeeeee
"Bipolar disorder: rapid speech, racing thoughts, distractibility, and less need for sleep in adults." "
29

---

B.3.3 Knowledge Base Evaluation
We conducted comprehensive evaluation of the knowledge base to ensure the accuracy and com- pleteness of the symptom descriptions for all entries, with particular focus on the sections covering "bipolar and related disorders" and "depressive disorders." The evaluation involved assessing the correctness of each symptom description and verifying the completeness of the content within these two critical sections. Any inaccuracies gaps identified during this process were manually revised ensure the highest quality of information.
30

---

C Agent Details
C.1 Three Diagnostic Variants
In this section, we provide prompts of Angel.R, Angel.D and Angel. in Table I0
Table 10: Prompts of Angel.R, Angel. and Angel.C.
Role: You are psychiatry diagnosis export.
Objective: Diagnose whetber the visitor (digit__id) has mood diserder (including depression and bipolar disorder), which is characterized by depressive or manic symptoons.
Constraints:
1. You may only use the following actions.
must act prosctively. Always keep this mind when planning actions.
Available Actions:
Three varionts of anggels are different actions, Angel R: A.2.7:AA 1,2.3,477 Angel.C: CC: 1,2,5,6,7.
1. loggle_ bisitor_rccord Retrieves stractured medical records for the visitor, available. Also extracts top-5 symptoms related to DSM-5 diagnostic criteria. Args: ("name": : "digitii jdd.."" "description" VVisstor ID, must be provided in the query' "type" sypee""unn "int" the top 5% most mondd disorder-relatedd questions from psychological scales. The correlation score represents the statistical relevance to mood disonders. Args: { I"name" "'iiii id", "description" "VVisiooe ID, must he provided the query". "type" "iint"
2. get_ scale performances: Retrieves the visitor's performance
3. previous cases, cases_disppay: If the visitor has medical record information, this tool compares the case with post stractured medical rocords in database and extracts the top 5 most similar cases. Duc to the personalized nature of psychiatric diagnosis, extracted similar
for reference only. Args: ("name": : digitid"". "descriptinn" "Visitor ID, must be provided the
query", "type": "ini")
4. previous scales_ display: This tool compares the visitor's 's scale performance with past scale performances the datahase, and extracts the top 5 must similar cases. Due to the personaizzd nature of psychiatric diagnosis, the extracted similar cases are reference only. Args: I"name": 'digit id", "description"" "Visitor must he provided
query"; "type": "int")
5. previous cases_ analysis: If the visitor has medical record information, this tool compares the case with past structured medical records in the database, extracts and andyyes the top 5 most similar cases. Dus to the personalized nature of psychiatric diagnosis, are for reference only. Args: ("name": "digit id", "description" "Visitor ID, musl be provided in the query", "type": "int")
the extructed similar
previous scales, analysis: This tool returns comparison the visitor's scale performance with past scale performances in the database, extracts and analyzes the top 5 most similar cases. Due to the personalized nature of psychiatric diagnosis, the extracted similar cases are for reference only, Args: ("name": : "digit_id", jd", 'description.. "Visitor ID, must provided query". "type": "int")
7. Hinish:
Completes the diagnosis process. Arga: "aanswrr" yyes" or 'ao' (indicating whether the visitor has a mood disorder)". "reasons"""A summary of key
supporting the decision.") "I
Resources: You are large language model mained on vast textual data, including factual medical knowledge.
Best Practices:
I. Always refer to similar cases (medical records or scale data) for diagnosis. However, do not over-rely past diagnoses due to the personalized nature of mental health conditions.
2. Consider all available information (medical records & scale performance), Always examine similer records (if available) and simular scale data for reference. Do not rely solely on one dain source.
3. Prioritize clinical evaluation
self reported scales in of comtradictions. Self reported terms like "occasionally'. "sometimes", and "frequently' are subjective, making it difficult for visitors to assess their condition accurately.
4. Only consider symptoms selevant to mood disorders. The visitor may have other mental disorders that not classified as mood disorders.
5. Mood disorders include only depression and bipolar disorder, characterized by depressive or manic symptoms.
Response Format:
Generate a JSON string following the structure below. Do not include any exima text or explanation. I ("action": : ["name": "action name". "args": ('args name": : "orgs value")), "thouphts": ("plan": "Briefly descrihe the list of short-term and long-term plans". "criticism": ; Consttuctive self-criticism", "obeevvatoon: "Summary of the current step returned to the user", "reasoning" 'Reasoning behind decision") )))
C.2 Symptom Matching
C.2.1 Medical Record Processing
To ensure patient privacy, the medical records presented in this section have been anonymized, with all event details adapted while preserving the patient's actual symptoms. The original medical record (adapted) and the corresponding processing steps are illustrated in Figure 3
Effective psychiatric diagnosis relies on comprehensive and well-structured patient information. However, raw medical records often contain scattered, redundant, or sensitive details that can hinder accurate analysis. To address these challenges, we designed a systematic medical record processing pipeline that ensures data security, enhances temporal reasoning, and improves symptom extraction for LLM-bbased agents. This pipeline consists of four key steps: extracting essential diagnostic elements, refining present illness descriptions to prevent data leakage, integrating structured information for coherence, and an optimal step of reorganizing records into a structured format for precise symptom matching. The processing prompts are provided in Table II]
31

---

Step Raw Data Extraction.
Step Present lllnes Processing.
Age
Age
Occupation colagge dusker
Occapation collgge stadent
Viat bate: 2013
fate 3555 10
Chief complaiet Unnoiiral intthiiiy for yours
Chief Complaim Emprionss inssiiiy fir yoar
Present illses En 20:202066 the prilent begas experiencing emotioral instailiity due academic doess, ofee crsing,
Present illress Apunsinatlly Sinee years the gatlerr began experiencing enstional itstability due to aendemic ofte crying, feiling imiable and having onthrrs of arger.
freling iritable, and baving outhorete angee diegsosed with lussicty disoede' another houittal lncitalograr
with 'anaiaty disndd? another valprozee Despiic traiment sympomm fluctutted Alauu koyarr aga ssught ouppotor cane our hrqpissl proserbedd "basetrigiee litiinn on boozte and aipppaazole. which helped eubilize his condition so some extere Hhres months apa following a confiiice with a clese friend, his syuution wersesed, He rxbibited crsotional instability.
mediccatoos
koppita
Deapitc teatmert sympioms llcturedd 20224188 snught vusputienn cam at our hoepitsl was preserthed "lanooigine; lihiiee caberste and alpiprazol: which Islped stabiline his condition to some exters. In 202222 fullowig conflier with a close hiiend his sympooms worsened. Hi echibited cmntional insubility frequett esing, intubility reconcile ergaged atceppiing to janp from beight (peceented fanily) H now wroking fammee cesiment our ctrpatmnnt clinic and admited ino our hosttall an hipolar dicolal Since the arsec of symetome, the patient has romsined alert, with a sthhte appetiie, siightty distubedd slees, secural bowel and bladder fanciion, egnifiane changos
swings repeatell: the frieed declined
using, inbability attempied
snings repertslly
friend. When oociading
lncluding oventasing
potiztt ongagal ovendosing or redication and atteupting jump from height iprevented by family). Hi is new reeking forthet Inameee at or oppotert oliiicae ahsantd
hostiaa
achipilar discedm Sinoe the ovor of oyympton the patire las oettained alet, with stalte appetiie slighty distarbed
lovela wright.
numaal bowel and blalder function, and agnificane changes in evegy leves weigtt
Step 3. Key Elements Combisation.
Step 4. Medical Record Stracturing
Reyaamald camently collegs complein Enctional intibblity for 3 years. Histary of present illness repocted by the paticat's family: Appessinaicly these year age, patient boger experencing enotional instability due to condemic st ecess, ofter crying. deiling iniiahll,, and having onthurcte angen He was dizgaosed with preseribel medications each Sacislapeam valgmaats Depiie mommont his cyrpuome flcttote Rhoer yearags sougle
'amptomm" {f
insiiity incsabiiity patient reported
Rapid crying fieling leggroeed irizaiill..
oubaast arges,
"Ansiitt spmplome: Experieneed anciety dua seaderric strea, ofee acommpanied by enotioaal cuthursts and attempte recencle inteperorall confliee.
dionde another
and
*talthemm butorione The potient overlwed on medication duse tino and aticapted to jume fore heght (prevested by
hespital rnd pessusled Samotiaine, litiiam carbersie and arpopeantic which hclpol stabilize his condition to soare exterr. There monils ago Sollowing confliet with close friend, his seceaaed
fandy''
"Sleop destrroncce Siihtly cieuthed shopp pattora'
Backgroend [
"DSagmosi yoarardd curverly collegy
musiillty, friquet crynng iniubillyy ragid nood swings, and repeatelly aitenipte recocii with the friend. Whes far friend declinod, the patiest eagaged
with diander asotier
intially
clmnc."
"Mladiation hitzey: Exciulopean, vulpmate, Lemotngine, liliur caboraae, and aigiaaaeee"
including ovendasing medication attempiing jump from haght (pervetied by smiiy) He how seeking funterr meamnett at our coppatient klinc the of
"Syngten ressmence Syrppons worsesed theee meeths ugo Salaning a conflst with dose frceal"
the patieut remanne illn asble slpgtit disurbed deep, asemal howrl and blariter function, and ss significant changes levels seiglt
"Semnal and plasioal iuse Sine the snitt, the patioul has remained alert, with aobbi appetite, nermal howel bladder finction, and signifiom chango in enengy lovels weight."
Figure 3: An example of the original medical record (adapted) and its corresponding processing steps.
Table 11: Prompts of medical record processing.
Step -+ Step Replace specific dates the present illness history relative time expressions based on the patient's admission date. Avoid using exact dates after the replacement. Step 2 - Step 3: Organize the patient's 's information into a single, coherent paragraph without altering the content of the present illncss section. Step 3 + Step 4: Structure the medical record by extracting and summarizing the patient's is symptoms and background. List each item separately, using" "-" delimiter. Respond only in ISON format.
Step 1. Raw Data Extraction. To construct a comprehensive and secure data store, we first obtained anonymized client information from the hospital database. To capture the full context of clients' backgrounds, which are critical for accurate diagnosis, we extracted the following key elements from the medical records: (1) Gender: Essential for identifying gender-specife symptoms (e.g., menstrual- related mood changes). (2) Age: Some disorders manifest at specific life stages, such as adolescence. (3) Occupation: Helps differentiate disorder-related symptoms from external factors (e.g.. shift work causing sleep issues). (4) Visit Date: Provides reference for interpreting time-sensitive information. (5) Chief Complaint: A concise summary of primary symptoms. (6) Present illness: A detailed symptom history, including medication usage.
32

---

Step 2. Present Illness Processing, To prevent data leakage in agent diagnosis, removed explicit disorder labels from the present illness section. Additionally, absolute dates were converted into relative timeframes based on the visit date (e.g.. "9 months ago" instead of specific date), enhancing data privacy while facilitating temporal reasoning for symptom progression analysis.
Step 3. Key Elements Combination. To generate coherent input for LLM-based analysis, we integrate the structured elements from Step with the processed present illness data from Step ensuring comprehensive contextual understanding.
Step 4. Medical Records Structurizing. Symptoms relevant to diagnosis often dispersed across different parts of medical records. For example, "suspecting others and perceiving malicious intent" and "engaging in defensive behaviors" may appear separately, making direct symptom matching difficult. To address this, we reorganize medical records into a structured format, presenting key symptoms in ISON. We will compare whether coherent text structured representation is more effective for symptom matching and analysis in the ablation study (Section 4.3.
C.2.2 Symptom Matching Formulation
Symptom matching is calculate the similarity of embedded medical records and diagnostic criteria. The calculation involves transforming the structured medical record into hidden states H, via text encoder, expressed e, norm H, [0]). Similarly, the embedding of each diagnostic criterion e is obtained as é norm [O)) The similarity score of dense embedding Sdense between the structured record and each criterion is calculated by the inner product of the two embeddings e, and ee, expressed as Sdense + (ee: ex)). This score provides a quantitative measure of the semantic aligmment between the client's symptoms and the DSM-5 diagnostic criteria.
C.3
Scale Performance Analysis
Client performances are converted from numeric scores to textual descriptions based on question content and options, with an example of question provided in Figure and an example of all related questions is provided in Figure
"phq9 Q2 score": 2
Over the past two weeks, how often have you been bothered by any following problems?
Question Content
Feeling down, depressed, or hopeless.
Not at all
I= Severul days
More than half the days
Nearly every day
[seff-reported] In the second question of the PHQ-9 questionnaire, the client self-reported over past two weeks, they feel down, depressed, or hopcless more than half the days. The correlation score between this question and the presence of mood disorder is 0.5006.
Description
Figure 4: An example of generating scale performance description from the score value of a relevant question.
33

---

Scale Performance Descriptions
The followiing are the perforaance of highly relevast questions in the [self-reporte]] and [cliaicion- evalusted] scales, mood disorders a statistical sense:
the Pessson correlation coefficient
question and the presence
Depresiion-rclated Performaanes:
[clbutcion evalnnedd] in the Hamilton Depression Kating Scale (HAMI) filled out by the clinscian during the coroonaation, the client scoral 15 pointe (out of 76 points), indienting that the elient may have degressive moed disarder 0.6348.
"hand Jotal scure": 15
symptoms. The conclation
this
(=bieisioo-eubbotee In the sixth question of the HAMA questionnzire, the cliniciss assessed the elient's depeessivve mund (Loss of interest, lack pleasure hobbies, depression, early waking, diumal swing.) as mild. correlation
"bama, Q6 score::
this question
presence mood disorder 0.6304.
[eliteran- Leliiccaa--vvafiatee evafnateed In the minth question of the HPRS seale. the clinician assessed the client's DEPRESSIVE MOOD (Despondeney mood, sadnese. Rate only dogros despondese)) not present. The correlation seore between question and the presence mood disceder 0.62.
"bprs_09_ "eoorrere
[choictan evaluared] the finst question of the HLAD questionairs, the clinician assessed the client's depressive
'hamd, soore": 1:
feeling spuntancously repored verbaaly. correlation this question and the presence of moed disarder 0.5021.
QJestionnaire-9 (PHQ-9), 9), scared 8 points (out total of 27 points), indieating that the elient may have mild depression. The correlation score between this and the presence of mood diserder is 11.92.
[aetfeeearredd la self-ruttd Putient
"phes totarl score': 8
[ellfeegoored) la the second question of the PHQ-9 questionsirr,, the elient self-reported that over the past two weeks, they fiel sod, deprossed, hopuless more than half the days, The cornelttinn betwern this question and the presence mood disorder is 0.5006.
"phop_02 scare"
Suicide-rde clated Performances:
[elbutston evaihatee) the third question of the HAMD questionnaire, the cliniciaa assessed client's suicidal tendencies none. The corelation score between this question presence mood diserder is 0.6024
"hamd Q3 score":
question of PlIQ-9 9 questiomaire, the client self repurted over the past two weeks, they did not at all think that death or harming themselves in some way was a solution. The correlation score betw een this question and the presence of mood disorder 0.5057.
reportee))
"ybu9 Q9 soore":
Energy & rellaed Performances:
[elinician-esudln exalosted] of] In the 7th quesiion the HAMD questiommire, the clinician nitedd the client's work and interests as: Thoughts and feelings of incupacity. faligue, weakness related activities, work. hobbies, only reported when aoked. The conelation score between this question and the presenze of ssood disorder is 0.607.
"hamd_g QT_ "_sorrrr"
question the HAMD questionnaire, clinician rated clieut's Foelings of Inadequary or Reduced Ability so: pation spontancoudly reports foclings of inddequecy. The cornelation score between this question and the presence of muud disorder 0.5236.
celaiciin-evsslaace tn
"hand 022 scoce': 2
rserrrea In the 4th question of the PII questiommare, the client self reported that over the past two weeks, they foel tired having little energy than half days. correlation between this question the presence mood disorder 0.5097.
"phot_Os Q4 seace": 2
[setfrogonted] the question PHQ-9 qusstiommire, the client self-repoted that the past weeks, they have little interest or pleasure doing things more than half the days. The correlation seore between this question and the presence of mood disorder is 0,5008.
"plqp_Qt QL_ Ltp_____sseee__
ssore"
Amslety-relate related Performances:
[efinicion-cww Auotod] ln the Homilton Anxiety Rating Scale (HAMA) filled out by the doctor during the consultation, the client scored Il points (out of a total of 56 points), indicuting that the cliemi may have anxiety. The correlation between this question and the presence of meod disorder 0.5513.
Thoma Mhama_total__aao score": 11
[setfrggoried] the self-asessed Generalized Ansidty Disonder-7 (GAD-T), the client scored 8 points (out of 21 points), indicating that the client has mild unsiety. The correlation score between this question and the presence
"gad7_ Jocal sooreh: & "hamd Q4 score": 0
0,5185.
Insomia-related Purformances:
[clinicizo evaluated] In the fourth question of the HLAD questionnaire, the clinician ascessed the client's difficulty filling teloop. The conrelation
INSOMNLA: EARLY IN NIGHH question the presence mood diserder is 0.5247.
this
[clinciinn-evd eo evadetee)) In fourth question the HAMA questionnaire, the clinician assessed the client's Insomia (Difficulty falling asleep. broken sleep, unsatiafying sleep and fatigue waking. dreams, nightmares, night terroes) as mild. The correlation score between this question and the presence of mood disorder is 5099.
hama Q4 [carroo""" score":
Figure 5: full example of scale performance descriptions derived from the score values of mood disorder-related questions.
34

---

C.4 Multi-agent Diagnosis
In this section, we provide prompts of two debate agents in Table 12 and prompts of the judge in Table
Table 12: Prompts of Debate Agents.
Role: You psychialry diagnosis expert. You believe the current visilor has (does not have) mood disurder. Present your argument and persuade the opposing side.
Visitor Information: This is challenging casc, and three agents have already provided their diagnoses:
1. The agent without prior case retrieval believes the visitor {has (does not have]) mood disorder, Its reasoning:
{Reasons given by Angel.83
2. The ageat who retrieved past cases but did not analyze them believes the visitor (has (does not have)) mood disorder.
Their reusoning: (Reasons given by Angel.D)
3. The agent who retrieved and analyzed past believes the visitor (has (does not have)) mood disorder. Their reasoning: ERaasona given by Angel.c) CC
Original Case Information:
1. Visitor's Medical Record: [The visitor's medical recond in Step 3]
2. Visitor's ss Scale Performances: (The visitor's s performances on selected scales)
3. Reterence from Past Medical Records: (Top-5 similar medical records with analysis)
4. Reference from Pasi Scale Performancss:
[Top-5 similar scale performances with unalysis)
Constraints:
1. Stick to your stance, Every case presented for discussion is inherently controversial, meaning your viewpoint is strongly justified.
2. You cannot interuct with physical objects. If an action is absolutely necessary, request the user to perform it. If the user refuses and no alternative exists, terminate the process to avoid wasting time effort.
Resources;
1. You are large language model trained on extensive textual datn, including factual medical knowledge.
2. All relevant visitor information provided. Make fall use of it.
Best Practlees:
information (medical records scale performances). Examine similar medical records and similar scale performances as references. Do not base conclusions on a single source.
Consider available
2. Due to the personalized nature of psychiatric diagnosis, do not overly rely on similar case diagnoses.
3. reportt questionnaire results may contradict clinician evalunaions. In such cases, prioritize the clinician's assessment
Terms like "rccasionally," "sometimes, and "frequently in self-reports are subjective and may lead to misinterpretation of the visitor's condition.
4. Only symptoms related to moed diserders should influence your diagnosis. The visitor may have other mental diserders that are not classilied as mood disorders.
5. Mood disorders include only depression and bipolar disorder, characterized by depressive or manic symptoms.
Response Format:
Generate JSON string the following format. Do not include any text explanation.
"Your reasoning for why the visitor has (ar does not have) a mood disorder, along with rehuttal to the opposing argument.", ", "thoughts": ('plan": 'Briefly outline short-term and long-term plans", "'ritticism" "Constructive self-eriticism")
I"response": :
C.4.1 Debate Details
Our debate stage is intentionally designed to simulate the classical four-stage structure commonly used in formal debates, with minimum of 2 rounds and maximum of 4. At the beginning of each round, the prompt guides the Pro and Con agents through a structured process:
Round Opening Statements: Both Pro and Con agents present their initial diagnostic positions based on the available information.
Round 2 - Rebuttals: Each agent critically responds to the opponent's arguments, defending their own stance and challenging the other side's reasoning.
Round 3 Free Debate (Optional): If the Judge Agent identifies unresolved or ambiguous points after the first two rounds, it initiates a more open-ended exchange to further clarify conflicting perspectives.
Round 4 Closing Statements (Optional): If the discussion remains inconclusive or highly contentious, both agents are given one final opportunity to summarize their most compelling arguments in concise manner.
After each round, the Judge Agent evaluates whether the debate should continue, provides reasoning, and reflects the arguments. At the end of the debate, the judge delivers diagnosis with reasons. The example in Figure @demonstrates a case where the debate concluded in three rounds.
This flexible structure allows the system to adapt the level of deliberation to the complexity of the case, balancing efficiency with thoroughness clinical decision-making The minimum of two rounds ensures that both sides have the opportunity to present and respond to each other's 'ss views fiiss-this
35

---

The client disorder.
The clieni shows mood insability with shills from depression to (hypo/munia, consisteni with bipolar disorder. Past hospitaiiationn for mood disonder and long tenn use mood stabilzzrrs support this, activity, energy align hypocmanic episodes per
AngelR
Symptoms like initablity,
The elient has mood disorder.
instabiliny, including hallocinations inuppropriate laughter. Despiie clinician-rated scales showing no depression, self-reports suggest moderate depression.
The echibits
Angel
Similar
diagnosed as bipolar majur depression, reinforcing possibility.
6.6
The client doesn't have mood disonder.
Though ceferesorts depressive auxicty symptoms, clinician ratings show signs depressions, ansiery, insomnin.
Angel
Mood fluciuation hisurry noted, but
do support aclive moed disorder diugnocis. having schizophrenia.
ballucinations inappcopriate laughter,
Buth sides should claim the disgnusis and ressons in this round.
believe
disorder.
The client's records show long-term mood lluctuations, niniig from depressive to elevsted states, which characteristic of' bipular disceder.
previously hospitalized
has on
medication, indicating a strong
moed disorder symptoms.
Although clinician-rated essessments indicate significant depression maxiety, the client's self-repoct showed modemiee depression, may better reflect their subjective emotional state.
don't think this elient has mood disorder.
assessments Although the client initially improved with mood-related medication, new symptoess appeared despire contimeed medication suggesting possible different mental condition. diagnoses, rolying sobily on past hospetalizatiion
signs depression, ansicty, suicidal tendencies key
cases diagnosis.
conclusive
Rehurtals missing, Continue.
Round 2
Both sides should refute the opposite side.
Selfreporied symptoms indicate depression and suicidality, with PIQQ-9 score of 19 and daily suicidal idiatioo. These are indicasors mood disorders, if clinicaan cated scales show minimal symptoms. disarders. disorders present episodicailyy
symptams during contimuous modication with partial response to teatment.
Historical diagrosis and
response support disonder. The client previously hospitalized affective disunder and showed improvement with mood siaiilians.
Mood Flactustions
nor confiem a mood disarder. observed
be part other psychintric conditions
such bunderline penoonlity trails or stres--iddued responses. Symplom variability requires broader differential dingnosii. Past hospinalization and medication do confirm condition not definitive evidence of present mood disorder, especially when recent cliniciun-rail rated assessments show no setive depressive anxious symptoms.
diagnasis. Historical treatment for moed related
interpreted cautiously. While PHQQ-9 indicates moderate depression, clinician-ddmiisstedd scales (HAMD, HAMA, BPRS) consistently show no depcessive, anxious, or suicidal signs. Clinical observaiion remains the gold standard.
Self-report
Cestain discussed. Continue.
Keund 3
Please discuss the passibility of performancz conflict betwees self-report and clinicias assesement, in one sentence.
client' symptoms suppressed by medicine.
The elient may schixophrznia.
W
All symptoms discussed. Debase ends.
elient docen't moed disorder. administered
Although the client was previously hospitalized and preseribed modication related moud disorders, multiple clinician-
indicae typical symploms depression ansiety. Morawer, cliem'' symptoms allevisted by mood disorder medications, raising the possibility of other psychiatric cocditions with overlagping features, such as schizophrenia.
Figure 6: A debate example where only Angel.C was correct and the other two were wrong, and the Judge made a correct decision by debate.
36

---

Table 13: Prompts of the Judge.
Role:
You are a psychiatry diagnosis expert acting as the judge in this consullation. Your role to determine whether each debate round should concluie and make the final diagnosis.
Visitor Informaaiio::
This challenging case, and thrce agents have alrcady provided their diagnoses:
Agent without prior case retrieval believes the visitor (is/isn', i, diagnosis by AngelRR) diagnosed with mood disorder. Their reasoning: [Reasons given by AngelRR
Agent who retrieved past cases but did analyze them believes the visitor (is/isa't i diagnosis Angel.D) diagnosed with mood disorder. Their remsoning: [Reasons given by Angel.D) Agent who retrieved and analyzed past helieves the visitor [iw/ssn't diagnosis by Angel.C) diagnosed with mood disorder. Their reasoning: |Reasons given by Angel.C)
Original Case Information:
Visitor's Medical Record: (The visitor's 's medical record in Step 3]
Visitoe's Scale Performances: (The visitor's perfornances selected scales]
Reference from Past Medical Records: (Top-5 similar medical records with analysis]
Reference from Past Scale Performances: [Top similar scale performances with analysis]
Constraints:
Palfill your rolc as judge and base your decision solely on the arguments presented by both sides.
You cannot interact with physical objects. action absolutely necessary to complete If the user refuses and no alternative exists, terminate the process avoid wasting time and effirt.
request user pertorm
Resources:
You large language model trained extensive textual data, including factual medical knowledgee All relevant visitor information is provided. Make full use of it.
Best Practices:
Consider all available visitor information (medical records & scale performances). Examine both similar medical records and similar scale performances as references. not base conclusions on a single source.
Due to the personalized nature of psychiatric diagmosis, do not overly rely on similar case diagnoses.
Self-reported questionnaire results may contradict clinician evaluations. such cases, prioritize the clinician's assessment. Terms
like "occasionally. "sometimes," and "Irequently in self reports subjective and may lead to misinlerpeetation of the visitor's condition.
Only symptoms related mood disorders should influence your diagnosis. The visitor may have other mental disorders that not classified as mood disorders.
Mood disonders
only depression bipular disorder, churacterized by depressiive or munic xy symptoms..
Response Format:
Generate a JSON string in the following format. Do not include any extra text or explanation.
!"Judge": "Do belleve the debate should cad? Answer only 'yes' 'no". "diagnose": "Do you believe the visitor has mood disorder? Answer only 'yes' or "no'.. "thoughts"; I "plan": "Briefly outline short-term and long-term plans." "criticism", "Constructive self-criticism." "judge_ reasons". "Your reasoning for why debatc should or should end.", "reasoning": "Your reasoning for whether the visitor does or does not have mood disorder."
establishes a meaningful diagnostic exchange. If the Judge Agent determines that all relevant points have been sufficiently addressed by the end of Round 2, the debate can be concluded early. However, when the case involves ambiguity or conflicting symptom interpretations, the Judge may trigger additional rounds to promote deeper reasoning.
In practice, we found most cases concluded after just two rounds of debate, as the key diagnostic arguments had already been clearly presented and addressed. The remaining cases required only one additional round, with all debates concluding within three rounds. This observation supports the efficiency of our debate framework in ensuring thorough reasoning when needed, while avoiding unnecessary verbosity or repetition.
D Synthetic Data
Usage Declaration. This work and its contents, including the synthetic mental health diagnostic dataset and diagnostic agent, are provided for academic and research purposes only. None of the material constitutes medical, clinical, therapeutic advice. The synthetic data is artificially generated and does not represent real patient information. No warranties, express or implied, are offered regarding the accuracy, reliability, suitability of the dataset, agent, any related content. The authors and contributors are not responsible for any errors, omissions, or consequences arising from the use of this repository.
This is not a substitute for professional medical or psychological evaluation. Users should consult qualified healthcare providers for clinical assessments or treatment decisions. By using this repository, you acknowledge that the diagnostic agent is experimental and should not guide real-world medical decisions.
37

---

The use of the software, data, or any derived materials is entirely at the user's s own risk. By accessing or utilizing this repository, you agree to indemnify, defend, and hold harmless the authors, contributors. and affiliated entities from any claims or damages.
D.1 Synthetic Data Construction
We synthesize our dataset using a structured pipeline that begins with data preparation, followed by model training, and concludes with rigorous post-processing. First, we construct the input data by combining 16 mood disorder-related questions from Table 7 8 baseline assessment scale total scores, and binary mood disorder labels (1 for presence, 0 for absence), resulting in 25 features per case. This tabular format serves as the foundation for our synthesis process using the TabSyn framework [20].
The training phase consists of two key components. We first pretrain VAE encode the tabular data into a continuous latent space, employing column-wise tokenizers and Transformer based encoders/decoders handle mixed data types. The model optimizes an adaptively weighted ELBO loss, dynamically balancing reconstruction accuracy against KL divergence regularization to preserve inter-column dependencies. Subsequently, we train diffusion model in this latent space, where the forward process gradually adds Gaussian noise following linear schedule, while the reverse process learns to denoise samples through score-based SDE.
Our synthetic data generation initially follows a 1:1 ratio with the original dataset's ' predefined splits (for both retrieval and test sets). The generation process involves iterative denoising of Gaussian priors to produce latent vectors, which are decoded into tabular form using the VAF's detokenizer, applying linear inverse transformations for numerical features and Softmax sampling for categorical variables. Through rigorous post-processing. including value rounding, logical consistency checks (e.g., ensuring question-level sums never exceed scale totals), and illogical case removal.
An example of our synthetic data is shown below.
"HANA 24 Score": 3,
"HANA 96 Score": 3,
"HAMA Tatal Scora": 28,
Score": 18.
*РН2Я Q1 Score": 3,
*PиΩ9 Q2 Scora": 2,
"PH29 94 Score":
99 Score": 0,
"PH29 Iotal Score": 14.
"HAND q1 Score": 2,
"HAND 03 Score":
q4 Score":
"HAND q7 Score": 2,
"HAND 422 Score":
"HAND
Score":
99 Score":
"PS91 Total Score"; 11,
"SHAPS Total Score": 29,
"HCL32 Total Score";
"DAS Total Score": 122,
"ESRS Total Score"; 40,
"ypq Total Score": 11,
"EFRS Total Score":
Total Score": 5,
"Zood Disorder";
D.2 Synthetic Data Evaluation
Building on the evaluation framework Zhang al. (20]. we conduct comprehensive evaluation of our synthetic data across five critical dimensions: statistical density, data quality, machine learning efficacy, privacy preservation, and logistic detectability. The results demonstrate that our synthetic dataset achieves exceptional fidelity, accurately reproducing both the core statistical patterns and complex relationships present in the original data while maintaining strong utility for machine learning applications. Notably, our evaluation shows the synthetic data achieves: (1) high-density preservation of univariate and multivariate distributions, (2) robust performance in downstream ML tasks, and (3) near- saaa-indistingushabbbiii collective findings position our synthetic data as a reliable surrogate for the original dataset across analytical and modeling use cases.
from real data according to rigorous detection tests. These
38

---

D.2.1 Density Evaluation
The quality of synthetic data hinges on ability to accurately replicate both individual feature distributions and the complex relationships between variables in real-world data. The metrics, derived from Shape, Trend, Coverage, and overall Density score, collectively measure the fidelity and utility of synthetic data for downstream applications.
Our synthetic dataset demonstrates remarkable fidelity in this regard, achieving an excellent overall density score of 0.86, well above the 0.8 threshold considered indicative of high-quality synthetic data. The component scores reveal particularly strong performance in capturing feature relationships, with Column Pair Trends reaching 0.93, while maintaining solid 0.79 fidelity in individual Column Shapes. These results demonstrate that our synthesis process successfully preserves the nuanced statistical patterns of the original data, with especially robust preservation of the critical multivariate relationships that are often most challenging to replicate.
Metric descriptions and our detailed scores are illustrated in the following parts.
Shape (Column Distribution Shape). The Shape metric evaluates how well synthetic data replicates individual feature distributions from the real data using two complementary measures: KSCom- plement compares cumulative distribution functions to assess overall shape alignment (including normality and skewness), while TVComplement evaluates precise probability mass matching for categorical data. Higher scores (closer to 1.0) indicate better preservation of the original data's 's statistical properties. As shown in Table 14 our synthetic data demonstrates strong distributional fidelity across all columns.
Table 14: Score of column distribution. KS is short for Kolmogorov-Smirmon complement; TV is short for Total Variation complement.
Column
9
10
11
12
Metric
KS
KS
KS
KS
KS
KS
KS
KS
KS
KS
KS
KS
KS
Score
0.76
0.74
0.68
0.79
0.79
0.84
0.81
0.85
0.72
0.77
0.80
0.80
0.78
Column
13
14
15
16
17
18
19
20
21
22
23
24
Average
Metric
KS
KS
KS
KS
KS
KS
KS
KS
KS
KS
TV
Score
0.82
0.67
0.78
0.88
0.80
0.90
0.84
0.87
0.90
0.67
0.83
0.73
0.79
Trend (Column Pair Trends). This metric evaluates how faithfully synthetic data reproduces the statistical relationships between variables. For numerical features (0-23), assess linear and nonlinear correlations (CorrllaionnSmilaritt)) examining whether directional patterns (e.g.. positive/negative associations) are preserved. For categorical feature 24, we measure dependency preservation through joint probability distributions "ContingencySimilariy 1.0) indicate the synthetic data successfully maintains the original data's 's multivariate patterns, as demonstrated in Figure 7
High scores (closer to
Coverage. The Coverage metric evaluates how comprehensively synthetic data captures the full spectrum of variability in real data. For numerical features, RangeCoverage assesses whether extreme values and distribution tails are properly reproduced, while CategoryCoverage verifies the faithful representation of categorical frequencies and combinations. High scores (closer to 1.0) indicate the synthetic data successfully encompasses the complete range of real-world scenarios present in the original dataset, as shown Table [5]
Table 15: Score of column distribution. RC is short for RangeCoverage; CC is short for Category erage.
Column
5
6
7
8
9
10
11
12
Metric
RC
RC
RC
RC
RC
RC
RC
RC
RC
RC
RC
RC
RC
Score
1.00
1.00
0.98
1.00
1.00
1.00
1.00
1.00
0.96
0.75
1.00
1.00
0.75
Column RC
16
18
19
20
21
22
23
24
Metric
RC
RC
RC
RC
RC
RC
RC
RC
RC
RC
CC
Score
1.00
0.78
0.83
0.95
0.95
0.94
0.98
1.00
0.86
0.90
0.90
1.00
39

---

0:5
6.9
6.31
8:39
0.00
0.51
0.:00
1.90
0:90
0.912
00:51
0.85
0.92
0.90
3.90
10.49
0..55
6:92
0.95
0.95
0.99
0,93
555
6.39
2.94
0.04
0.55
6:36
1.95
0.97
2.90
00.53
6,36
0.97
0..96
3.50
10.99
00.55
6:37
2.09
0.50
2.19
0.55
00.39
6.94
8:44
8:95
01996
8:44
0.99
3:99
0.99
0.99
0.99
0.99
0.97
8:35
8.97
0.93
9:97
0U995
01:05
6:96
0.95
0.95
3.94
0.99
1:00
6:35
3.9
0.99
3.95
0.00
0.94
0.55
0:33
9.94
0.92
9:95
0.900
6:33
9.92
9.95
3.95
0.99
0:59
6:37
0.97
1X00
3.94
0.99
8:45
6.99
0.99
11.91
8:44
0.90
0:99
18.99
11:09
0.99
1.99
8:45
11.95
0.99
PM
0.99
08.96
0.97
0.99
5:95
8:47
5.94
9.97
3:95
0.966
0.96
0.99
18.95
16.00
0.99
11.99
-17
Thood Scoee
E:47
0.97
18.96
1:09
0.91
2.15
1L:55
0.59
1U.99
EUSY
0.50
0.46
0.56
2.09
0.96
10
11
18
21
Figure 7: Column pair trend similarity scores. Features 0-23 are evaluated using CorreltiooSSimllriyy. while feature 24 is assessed with SantingencySimilaigg
Density (Overall Data Fidelity) The Density score provides comprehensive evaluation of synthetic data quality by combining distributional accuracy (Shape) and relational preservation (Trend) into unified metric. Calculated the harmonic mean of these components, our synthetic data scored 0.86, demonstrating exceptional statistical alignment with the source data, not only maintaining faithful individual feature distributions but also accurately reproducing the complex web of relationships between variables.
D.2.2 Quality Evaluation
Automatic Evaluation We evaluate synthetic data fidelity using the Alpha-Precision metric, which assesses two critical dimensions of data quality:
Pattern Accuracy (a=0.72): Measures how precisely the synthetic data reproduces the core statistical patterns of real data, ensuring generated samples are realistic and representative.
Diversity Coverage (8=0.44)): Evaluates the synthetic data's 's ability to capture the full variability of real-world scenarios, including less frequent but important edge cases.
These results indicate our synthetic data successfully captures the main data distribution (a) while showing room for improvement in representing rare cases (3). The strong a-score suggests ex- cellent utility for applications requiring faithful pattern reproduction, while the 3-score highlights opportunities to enhance coverage of the data manifold's full extent.
Clincian Audit We randomly sampled 50 cases (25 real and 25 synthetic) and asked three licensed clinicians to distinguish between them. Their classification accuracy was as follows:
Table 16: Human Evaluation of MoodSyn.
Clinician
Correct Real Real Misclassified
Correct Synthetic Synthetic Misclassified
1
18
20
2
21
4
22
3
3
19
18
The three clinicians achieved accuracy rates 0.80, 0.82, and 0.72 when classifying synthetic data real, demonstrating the synthetic data's convincing resemblance to real clinical data. We measured
40

---

antor-ano agreement using Fleiss' Kappa, which showed moderate agreement among raters (N 0.54).
D.2.3 Machine Learning Efficiency Evaluation
This evaluation assesses the practical utility of synthetic data by measuring how effectively it can train machine learning models to perform real-world classification tasks. We employ comprehensive set of metrics to evaluate model performance when trained on synthetic data and tested on real data: the Fl score (both standard and weighted) evaluates the balance between precision and recall, AUROC measures the model's 's discriminative ability across all classification thresholds, and accuracy assesses overall prediction correctness.
As shown in Table 17 the results demonstrate our synthetic data's strong capability to train performant models, with consistently high scores across all metrics indicating successful knowledge transfer from synthetic to real data scenarios. These findings validate that models trained on our synthetic data can effectively generalize to real-world applications while maintaining robust classification performance.
Table 17: Machine Learning Efficiency (MLE) evaluation using XGBoost classifier.
Metric
binary fl
roc auc
weighted fl
accuracy
best f'l scores
0.884
0.9767
0.9037
0.9247
best weighted scores
0.884
0.9767
0,9037
0,9247
best auroc scores
0.8852
0.9735
0.9044
0.9247
best
0.884
0.9767
0.9037
0.9247
best avg scores
0.884
0.9767
0.9037
0.9247
D.2.4 Privacy Evaluation
We evaluate privacy protection using the Distance Closest Record (DCR) metric by first prepro- cessing data: numerical features are normalized by their range, and categorical features are one-hot encoded to standardize feature scales for consistent distance calculations. For each synthetic record, the L.1 distance (sum of absolute differences) to all real records and all test records is computed, and the minimum distance for each synthetic record in both groups is identified. The DCR score is then calculated as the proportion of synthetic records where the closest real record is closer than the closest record. A score approaching 0.5 indicates that synthetic data is equidistant real and test datasets on average, suggesting stronger privacy protection by reducing the risk of re-identifying specific real individuals from the synthetic data.
Given that the real data has been desensitized, although we scored 0.87 for the synthetic data, it doesn't t have the potential for privacy leaking. The synthetic data effectively preserves the distributional characteristics of the desensitized real data, such as the value ranges of numerical features and the probability distributions of categorical features. This is advantageous for data applications like model training, as it indicates that the synthetic data has strong "realism" or the ability to mimic the desensitized real data closely. This alignment ensures that the synthetic data retains the statistical utility of the original dataset while avoiding privacy risks associated with identifiable information, making it valuable substitute for downstream analytical tasks.
D.2.5 Logistic Detection Evaluation
The Logistic Detection metric evaluates how well synthetic data mimics real data by measuring its distinguishability. perfect synthetic dataset should be virtually indistinguishable from real data.
Our synthetic data achieved an excellent score 0.53, very close to the ideal 0.5 (random guessing). This demonstrates that synthetic data maintains high fidelity to the statistical properties of real data, making it highly suitable for downstream applications where data realism is crucial.
41

---

E
Experiment
E.1
Experimental Settings
Baseline Prompts. For prompts in bare LLMs, we employ the key element combination format derived from medical records (Step in Figure along with the overall performance metries (total score and corresponding descriptions) of each available scale. It is important to note that some scales do not rely on a total score assess performance; instead, they use individual questions or specific question combinations to evaluate a particular dimension (e.g., CTQ scale and MCCB tests). Additionally, certain scales include not only multiple-choice questions but also open open-ended questions, making impossible to calculate total score (e.g.. NSSI scale). However, since several scales closely related mood disorders utilize a fully multiple-choice format with a total score calculation, the information provided to the baselines remains comprehensive. A prompt template with example is shown in Figure
Baseline Prompt Template
Hased following infommation, assess whether the elient may have a mood disorder, which includes depression and bipolar disordor, characterized depressive manic symptoms. Please provide detailed analysis JSON format:
I"Diagnosis,; : yes or no. "Reasons": surmmarize and explain each point)
The client's medical record is follows:
[Medical Record (she format Step 3/)
The client's scale test results
follows:
Dysfunctional Attitudes Scale (DASj: 159/280, indicuting cognitive distortions in his views of people and situations.
Generalized Anxiety Disorder=? (GAD-7): 2/21, indicating anxiety symptoms.
Hypomania Chocklist (HCI--22) 11/52, indicating no hypomanic symptoms.
Mood Disorder Questionmair (MDQ): 7/13, suggesting possible bipolar disorder.
Patient Health Questionmire--) (PHQ-9y: 13/27, suggesting moderate depression.
Snaith-Eamilton Pieasure (SHAPS): 40/56, indicating anhedonia.
Brief Psychiatne Rating Seale (BPRS): 24/126, indicating no psychotic symptoms.
Hamnlloon Amsicty Rating
(HAMA): 3/56, indicating amsicty.
Hamilton Depression Rating Scale (HAMM)): indicating no depressive symptoms.
Young Mania Rating Scale (YMRS): indicating mild symptoms.
Figure 8: An example of baseline prompt.
Retrieval number. Table 18 evaluaaee how retrieval number is affects model performance, and k yields the optimal balance across evaluation metrics. Note that the optimal value of may vary across datasets and should be adjusted accordingly when using the MoodAngels diagnostic framework.
Table 18: Performances with different retrieval numbers.
Setting
ACC
MCC
An=e
0.923
0.837
Angel.Dx=3
0.906
0.794
Angel.Dx=11
0.923
0.733
Angel.R(k=D)
0.920
0.829
42

---

E.2 Experimental Results
E.2.1 Human Evaluation of Generated Reasons
The debate agents generate reasons for their position by points. We evaluated their performances by three licensed psychiatry experts our co-author clinician's s team. Metrics are clinical coherence (0-2 points), information completeness (0-2 points), and clarity of expression (0-2 points), as detailed in Table 19
The assessment results are shown in Table 20
Table 19: Evaluation criteria of generated reasons.
Clinical Coherence: whether reasoning aligns with clinical logic and evidence.
2 Fully coherent and professionally sound;
1 Minor logical flaws;
Major errors or conflicts.
Information Completeness: inclusion of key diagnostic elements (e.g., symptoms, scales, course, exclusions,
impairment).
2 Comprehensive and sufficient;
1 Missing important elements;
0 = Lacks key diagnostic info.
Clarity and Expression: clarity and professionalism of the explanation.
2 Clear and professionally worded;
Some ambiguity or terminology misuse;
0 Unclear or confusing.
Table 20: Evaluation results of generated reasons.
Model
coh1
com1
clarity1
coh2
com2
clarity2
cok3
com3
clarity3
avp. coh
.oom
ing. clarity
GPT-4o
0.9
0.7
0.2
0.9
0.6
0.13
0.5
0.9
AngeLR
0.9
0.6
1
1.03
1.17
AngelD
0.9
0.6
0.9
1.03
0,73
AngeliC
L11
1.13
multi-Angels
1.1
We report Fleiss' Kappa the measure of inter-annotator agreement (IAA), and the results shown in Table The results demonstrate the high quality of our generated outputs. The higher inter-annotator agreement suggests that the outputs of our multi-agent system consistent, clinically aligned, and better recognized by experts.
Table 21: Evaluation IAA of generated reasons.
Model
coherence
completeness
clarity
GPT-4o
0.6
0.26
0.13
Angel.R 0.55
0.37
0.19
Angel.D
0.69
0.28
0.25
Angel.C
0.69
0.33
0.27
multi-Angels
0.67
0.45
0.54
43

---

E.2.2 Subgroup Performance Analysis
We evaluate the performance of Angel.R across demographic subgroups the patient subset, containing 190 cases of mood disorder and 56 cases of other diseases. The performances are shown in Table 22]
For gender subgroups, despite nominal accuracy difference (A 0.088), the p-value (0.083 0.05) and comparable MCC difference (A 0.018) suggest no statistically significant gender bias. The slightly higher female accuracy likely reflects dataset characteristics rather than model bias.
For age subgroups, no significant accuracy difference (p=0.1 is observed between adults and children, with MCC values differing by only 0.048. It indicates that our model generalizes well across age groups without age-specific calibration.
Table 22: Subgroup Performances.
Group
ACC
MCC
All Patients
0.845
0.489
Gender Subgroups
Male
0.784
0.466
Female 0.872
0.484
Age Subgroups
Adult 0.795
0.500
Female
0.873
0.452
E.3
Ablation Study
In this ablation study, evaluate the robustness of our framework by changing input order and temperature.
Input Order Sensitivity. We use the random. shuffle( function to randomize the input order during the debate stage, and the results are shown in Table 23
Table 23: Performances with different input orders.
Setting
ACC
MCC
multi-Angels (ordered input)
0.925
0.834
multi-Angels (shufiled input)
0.927
0.838
Temperature Sensitivity. We vary the temperature of GPT-4o assess the output stability of Angel.R, and the results are shown in Table 24
Table 24: Performances with different LLM temperatures.
Setting
ACC
MCC
Angel. R. Rtemp= (default)
0.920
0.829
Angel.Ru Rtemy=0.7
0.922
0.836
Angel.R
Rtemp=0.3
0.923
0.836
Angel.Rrempzo
0.911
0.808
These results indicate that variations in input order and temperature have minimal impact on diagnostic performance, demonstrating the stability and robustness of our framework.
44

---

E.4
Case Study
For the inter-scale conflict and borderline cases, we selected two representative scale-only cases. The scale performance data are derived from real cases at our corporate hospital. These cases are illustrated in Figure and Figure T0
For the overlapping symptoms cases presented in this paper, which include medical records, we have anonymized all event details while retaining the patient's actual symptoms. The case is depicted in Figure T1] In our experiments, we exclusively used fully anonymized real-world cases to evaluate MoodAngels' diagnostic capabilities in practical application scenarios.
Case with Inter-Scale Conflicts
A Case with Borderline Performances
[elirician-evafuune evainated]
[sefFrepurred]
aeiiiician-vvlluiiee
[ee//reporiee]
Depression-related Performances: (confllets)
Depression-related Performances:
"hamd lotal_ score": 2
"phqq__ @oallsccree"
"bamd total score": 10
"phqo "phq9_total
score" total_score"
"hama 20l_stere"
"phq9_Q2 score":
"hama Q6 soore":
"pha9 Q2. score":
"bprs Q9 soore":
"bprs_ score":
"handd Q1
"hamd Q1 score": ]
depressed mood observed.
Moderate depression.
depression.
Suicide-related Performances:
Suicide-related Performances: (consistent)
"hamd Q3 score": 0
"phug Q99 score":
"hamd Q3 score": 0
"pbq9_ 09 scone"
suicide tendency.
No suicide tendency.
Energy & Interest-related Performances: (mild conflicts)
Energy Interest-relteee Performances:
_0___0___o_ score":
"phq9 Q4_0000ssss score":
"band score":
"phap Q4 score":
"hamd Q22 score":
"phq9_g Sap____saaaa score":
"hamd Q22 score":
"phq9 Q1 score":
No interest decline.
Mild interest decline.
Mild interest decline.
Anxiety-related Performances: (consistent)
Anxiety-related Performances:
Jotal_ score":
"gad? score":
"hama total score":
"gad7 total score":
Mild anxiety.
Mild anxiety.
Insomnia-related Performances: (comsistent)
Insomnia related Performances:
"harn Q4 score":
"hand score":
"hama Q4 score":
"bama, Q4 score":
insomnia.
Mild insomnia.
Figure 9: A case with inter-scale conflicts, with Figure 10: A case with borderline performances, only scale performances provided. In this case, with only scale performances provided. In this the client doesn't 't have any mental diseases. case, the client doesn't it have any mental diseases.
45

---

Scale Performances
Medical Record
[olaicimm-nnlaatee]]]
[snloegortto]
The climnt is an 18-yer--old female student
Chief Complaint Absermal behavior and speech for over montls, worsening the pist week.
Depressio-vviated Performances: (Conflictss
Hissory of Present Illuess: Approximaiely months the patient experienced meed, reduced motivation, and a lack of Interest in daily acrivitles, She reported persistent feelings of sadmess, a sense of life being measing gless, and passive thoughts about wasting live (wilheut any expressed warries about future described ongoing cognitive confusion. The patient sometimes reforring to family member in an inconvetcional way and identifyingg childhood aequiittance romantic being watched
"hand Jotal, scorc":5
"phet_ioo Josal secre";
accident and began exhibiting
"hana Q6 score". 0
"plo9_02,
soore" [M4__02_soone"
"bpes Q9 scone":
attenpts), She occasionally
Ql_score
She also described feelings of inscalled surveillanee device
No depression.
Moderaee depression.
times, she was observed talking and laughing to heseelff appetile decreased, and she exps essed desire to lose weightt despite being of normal build. She was previously evalunsed at local bospital, where condition preseribed medication. she discontinued medication on own. Over pasi week. her symptoms have recurred. prompting her family to seek forther evaluation and treamment our facility Since the onset of symptomss, the patient has maintained adequate sleep patterns, regullar howel bladkerr habits. Siee has experinned some weight loss.
Suieide-relltte Performances: (C (Conflicts)
diagnosed with psychiatric ingeovement.
score":
"phop 2
No sricide sendaney.
Suicide lendency hall days.
levels,
Energy & Interest-reeatte Performances: (Conflictt)
Symptom Matching DSM-5
"hhemd Q7 scone" O
"pbc" Q4 score":2
("Sympton"" "Signi changes in passonalily and behavior, accompanied by decline is language abilities and memorr impairment, commonly seen in younger patients". "
"loand Q22 score" O
"plup200 _l soore")
"Disease" "Pronisemppral Neurocognitvve Disander", "Canegory": "Neumocogntive Disunders', "Relevance": (.6644
No interest decline
Intorest declines in half days.
('Syuplom" "Symptons fins appearal before the age of 10 and cannot be explnised by other mental disordes (such as aution, post- trumatic disceder. substance shuse.. "Diseaze": "Dismppiive "Mood Disorders" "Relevance" 0,626 ),
Dysregulation Disorder", "Canegory":
Ausiety-rdlated Performances:
"hama total score": O
"gad7 total score":
""Sympton": "Extreme sensilivity to social situtions, evoidance social activities, feelings of inferiority. and excessive werry negitive exalatioo..
No anciety.
"Disesse": "Avoidant Personality Disorder", "Category": "Penorality Diserdes", "Relevance", 0.6196 J.
""Sympoom": "Hypomanic cpisede: Lasting least days, characterized abnormally elevated or irritable mood, accomponied by incressed merry or activityy..
Insomala-rehated Performances:
"Disesse" 'Rigolar Disonder", "Categery" " "Mood Disordans" "Rolevanz"" 0.143 ).
"hamd Q4 scare":0
'hama, score":
"Symptom": " "Decline cognitive abilisies, closely related the neuropathological effects of IIV infection, often accompanied by behaviona and emesional distrbsnnes"" "Disease": : "Neurocogsitive Disorder Duc HTV Infection", "Cologory "Neurocogiiive Disadars", "Radevinc" 0.6094)
No insomaia
Top-5 Related Cases
Parient Sympioers:
Fanationaal instability, social impaimment, negative behavions, Iullucinations, dehoionn
Diagsosis The putient lass mood discrdes, specifically Bipolar I Diseeder, curcuily presenting wilh depesssive opicdes and mixed features.
Samilartty Query: 0.8276
Parient Sympioms: Low mood, negative thonghts, palpitaionn, iniiaiiity, and aggnesive tendencies. Diagrosis: The potient hos mood discrder, specifically Mujor Deprassive Disarder with psychotic featunos Sauilarit Quary. 0.8228
Putient Sympions: Emotional instabiliiy, ausiety and intubiiiyy, physical discondort, social inpairment. Diagnosis: The patient bas mood discrder, specifically Major Depressive Disorder with anxious discess. Similrity Query: 0.822
Purient Symptoms: Low mood, self harm sendencies, appetine and sleop disturbances.
Diagnosic: patient mood discrder, specifically Dipolar Disonder, currenll experiencing a major depeessive episods. Simibrity Query: 0. 8203
Patient Sympiooss: Emntiionaa instability, loss of interest, hypersernitivityy auditory Iallucinations, plysical discomffert. mood disonder; diagnosis Schazophrenia, finst epssode.
Diagnosis: The patlent
Samilarity Queey:
Figure 11: An adapted case with overlapping symptoms. The top-5 related cases are simplified for intuitive presentations. In this case, the client has a delusional disorder on the schizophrenía spectrum (not a mood disorder).
46


Reviews:
[{"review_id": "phurhIsGdq", "reviewer": "Reviewer_HsoZ", "summary": "This paper presents MoodAngels, a multi-agent framework for psychiatric diagnosis, particularly for mood disorders. It combines fine-grained analysis of DSM-5 scales, a retrieval-augmented knowledge module, and a structured multi-agent debate system (Angel.R/D/C, Judge, and Debate Agent). The authors also introduce MoodSyn, a synthetic benchmark dataset built via a combination of generative techniques to simulate high-quality psychiatric cases while preserving privacy. Evaluations on real hospital data demonstrate that the system outperforms strong baselines, including GPT-4o.", "questions": "1. Could you provide a detailed example of a case where Angel.R, Angel.C and Angel.D disagree, and show how the Debate Agent arguments are structured and evaluated by the Judge?\n\n2. How does the Judge determine when a debate should end? Is there a scoring or threshold-based mechanism?\n\n3. Have you evaluated the consistency of final decisions under changes to input phrasing, order of Angel outputs, or prompt style?\n\n4. Is the Debate Agent using scripted arguments or generating open-ended reasoning? How interpretable are the generated explanations?\n\n5. Does the Judge module retain any clinical rules (e.g., DSM-5 heuristics), or is it entirely language-model-based?", "limitations": "Partially addressed. The synthetic dataset's limits are discussed, and the authors recognize the challenge of capturing full clinical nuance. However, the paper would benefit from more detailed reflection on the debate module’s sensitivity to randomness and prompt design, as well as potential unintended bias amplification across agents.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "This paper is timely and ambitious. It addresses core limitations in existing medical LLMs, namely: limited interpretability, lack of structure in multi-perspective reasoning, and the absence of privacy-aware psychiatric benchmarks. The design of agent specialization (retrieval, contrastive, and isolated judgment) is thoughtful and clinically grounded, reflecting patterns observed in real-world differential diagnosis.\n\nHowever, the core decision aggregation mechanism—namely, the Judge and Debate Agent modules—remains underexplained from an interpretability and implementation perspective. While the paper outlines the overall pipeline (i.e., initial vote → conflict check → debate → final decision) (lines 173-181), it does not show how the Judge actually resolves conflicts or evaluates arguments. For instance, what mechanisms (prompt patterns, learned scoring functions, or rule-based criteria) are used by the Judge to choose between competing perspectives? How are the debate rounds terminated, and how sensitive is the final outcome to minor prompt variations or phrasing changes?\n\nMoreover, no concrete case studies or dialogue traces are provided to help the reader understand what types of reasoning emerge during debates. This makes it difficult to assess whether the system is simply ensembling diverse outputs or performing meaningful deliberation. In recent literature on medical LLM auditing (e.g., checklists in MedQA, Dialogue-Based Evaluation in MedAlign), providing interpretable reasoning paths has become a standard expectation. Including such examples would significantly enhance trustworthiness and traceability.\n\nThe paper could also benefit from a more thorough evaluation of system robustness—e.g., does MoodAngels produce consistent diagnoses under slight perturbations (e.g., varying agent order, temperature, phrasing)? Multi-agent LLM systems are known to be brittle, and demonstrating stability would strengthen the case for real-world deployment.", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for your thoughtful questions, and we have revised our paper to address them and presented examples of our framework's output.\n\n**Q1.** A debate case, where only Angel C was correct and the other two were wrong, and the Judge made a correct decision by debate.\n\n[Angel R's final output] The client has a mood disorder. \n- The client shows mood instability with shifts from depression to (hypo)mania, consistent with bipolar disorder. \n- Past hospitalization for mood disorder and long-term use of mood stabilizers support this. \n- Symptoms like irritability, increased activity, and energy align with hypomanic episodes per DSM-5.\n\n[Angel D's final output] The client has a mood disorder. \n- The client exhibits emotional instability, including hallucinations and inappropriate laughter. \n- Despite clinician-rated scales showing no depression, self-reports suggest moderate depression. \n- Similar cases were diagnosed as bipolar or major depression, reinforcing this possibility.\n\n[Angel C's final output] The client doesn't have a mood disorder.\n- Though the client self-reports mild depressive and anxiety symptoms, clinician ratings show no signs of depression, anxiety, or insomnia. \n- Mood fluctuation history is noted, but current assessments do not support an active mood disorder diagnosis. \n- For hallucinations and inappropriate laughter, consider the client having schizophrenia.\n---\n[Judge] Round 1 begins. Both sides should claim the diagnosis and reasons in this round.\n\n[Positive] I believe this client has a mood disorder.\n- The client's records show long-term mood fluctuations, ranging from depressive to elevated states, which is characteristic of bipolar disorder.\n- The client was previously hospitalized for a mood disorder and has been on mood-related medication, indicating a strong link to mood disorder symptoms.\n- Although clinician-rated assessments did not indicate significant depression or anxiety, the client's self-report showed moderate depression, which may better reflect their subjective emotional state.\n\n[Negative] I don't think this client has a mood disorder.\n- Clinician-rated assessments show no signs of depression, anxiety, or suicidal tendencies—key indicators of mood disorders.\n- Although the client initially improved with mood-related medication, new symptoms appeared despite continued medication use, suggesting a possible different mental condition.\n- Similar cases have inconsistent diagnoses, so relying solely on past hospitalization records is insufficient for a conclusive diagnosis.\n\n[Judge] Rebuttals missing. Continue.\n\n---\n\n[Judge] Round 2 begins, both sides should refute the opposite side.\n\n[Positive]\n- Self-reported symptoms indicate depression and suicidality, with a PHQ-9 score of 19 and daily suicidal ideation. These are core indicators of mood disorders, even if clinician-rated scales show minimal symptoms.\n- New symptoms during continuous medication do not rule out mood disorders. Mood disorders often present episodically or with partial response to treatment.\n- Historical diagnosis and treatment response support a mood disorder. The client was previously hospitalized for an affective disorder and showed improvement with mood stabilizers.\n\n[Negative]\n- Mood fluctuations alone do not confirm a mood disorder. The observed shifts could be part of other psychiatric conditions such as borderline personality traits or stress-induced responses. Symptom variability requires a broader differential diagnosis.\n- Past hospitalization and medication use do not confirm the current diagnosis. Historical treatment for a mood-related condition is not definitive evidence of a present mood disorder, especially when recent clinician-rated assessments show no active depressive or anxious symptoms.\n- Self-report scores must be interpreted cautiously. While PHQ-9 indicates moderate depression, clinician-administered scales (HAMD, HAMA, BPRS) consistently show no depressive, anxious, or suicidal signs. Clinical observation remains the gold standard.\n\n[Judge] Certain points are not discussed. Continue.\n\n---\n\n[Judge] Round 3. Please discuss the possibility of a performance conflict between self-report and clinician assessment, in one sentence.\n\n[Positive] The client's symptoms are suppressed by medicine.\n\n[Negative] The client may have schizophrenia.\n\n[Judge] All symptoms discussed. Debate ends.\nThe client doesn't have a mood disorder. Although the client was previously hospitalized and prescribed medication related to mood disorders, multiple clinician-administered scales did not indicate typical symptoms of depression or anxiety. Moreover, the client's current symptoms are not alleviated by mood disorder medications, raising the possibility of other psychiatric conditions with overlapping features, such as schizophrenia.\n\n**Q2.** We designed the prompt of the Judge Agent to simulate a debate format, with a minimum of 2 rounds and a maximum of 4. At the beginning of each round, the prompt guides the Pro and Con agents through a structured process: Round 1 for opening statements, Round 2 for rebuttals, Round 3 for clarifying unresolved points, and the final round for closing arguments. After each round, the Judge Agent evaluates whether the debate should continue, provides reasoning, and reflects on the arguments. At the end of the debate, the judge delivers a diagnosis with reasons. The example in Q1 demonstrates a case where the debate concluded in three rounds.\n\n**Q3.** Thanks for your suggestion. We are testing on changing the temperature of the angel and the different prompt order of the debate part. We will display the result as soon as we complete the experiment.\n\n**Q4.** The debate agents generate reasons for their position by points. We evaluated their performances by three licensed psychiatry experts on our co-author clinician’s team. Metrics are clinical coherence (0-2 points),  information completeness (0-2 points), and clarity of expression (0-2 points). \n\nClinical Coherence: whether reasoning aligns with clinical logic and evidence.\n- 2 = Fully coherent and professionally sound\n- 1 = Minor logical flaws\n- 0 = Major errors or conflicts\n\nInformation Completeness: inclusion of key diagnostic elements (e.g., symptoms, scales, course, exclusions, impairment).\n- 2 = Comprehensive and sufficient\n- 1 = Missing important elements\n- 0 = Lacks key diagnostic info\n\nClarity and Expression: clarity and professionalism of the explanation.\n- 2 = Clear and professionally worded\n- 1 = Some ambiguity or terminology misuse\n- 0 = Unclear or confusing\n\nHere are the assessment results:\n|Model|coh1|com1|clarity1|coh2|com2|clarity2|coh3|com3|clarity3|avg_coh|avg_com|avg_clarity|\n|:-------|:-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|\nGPT-4o|0.5|0.9|0.7|0.2|0.5|0.9|0.6|1|1.1|0.43|0.5|0.9|\nAngel.R|0.9|1.1|0.9|0.6|0.7|1.2|1|1.3|1.3|0.87|1.03|1.17|\nAngel.D|0.8|1.2|0.9|0.6|0.6|1|0.9|1.3|1.2|0.83|1.03|0.73|\nAngel.C|0.9|1.2|1|0.7|0.8|1.1|1.1|1.4|1.3|1.27|1.4|1.13|\nmulti-Angels|1.1|1.3|1.1|0.8|0.8|1.1|1.1|1.1|1.1|1.3|1.3|1.1|\n\nWe report Fleiss' Kappa as the measure of inter-annotator agreement (IAA).\n|Model|coherence|completeness|clarity|\n|:-------|:-----:|-----:|-----:|\nGPT-4o|0.6|0.26|0.13|\nAngel.R|0.55|0.37|0.19|\nAngel.D|0.69|0.28|0.25|\nAngel.C|0.69|0.33|0.27|\nmulti-Angels|0.67|0.45|0.54|\n\nThe results demonstrate the high quality of our generated outputs. **The higher inter-annotator agreement suggests that the outputs of our multi-agent system are more consistent, clinically aligned, and better recognized by experts.**\n\n **Q5.** Thank you sincerely for the thoughtful suggestion. Our judge is entirely language-model-based and is designed to simulate a debate adjudicator rather than proposing new diagnostic hypotheses. The DSM-5 alignment is primarily handled in earlier stages by the three independent diagnostic agents, whose outputs serve as the “debate material.” In this setting, the Judge is more like a referee than a debater, who does not gather evidence or match symptoms directly to criteria."}, {"review_id": "pm5q53tiA1", "reviewer": "Reviewer_EzJT", "summary": "The paper proposes a multi-agent psychiatric diagnostic framework for mood disorders that utilizes retrieval-augmented generation to incorporate the context of analogous cases. The architecture is composed of distinct agents: Angel.R, which operates without reference to previous cases; Angel.D, which presents retrieved cases as contextual information; and Angel.C, which conducts a comparative analysis between each retrieved case and the current query, returning this analysis as context. In the event of diagnostic disagreements, supplementary agents engage in positive and negative argumentation to facilitate resolution. Concurrently, the paper introduces MoodSyn, an open-source dataset comprising 1,173 synthetic cases. The framework demonstrates superior performance, surpassing that of GPT-4o by 12.3%.", "questions": "- Given the potential for performance improvements, are there plans to evaluate models that have been fine-tuned or explicitly designed for this clinical context?\n\n- Was the choice to use a combination of variational autoencoders and diffusion models for data generation benchmarked against other approaches? If so, what was the rationale for selecting this specific method over other potential alternatives?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- The assertions made within the paper are well-substantiated, with the experimental outcomes providing robust corroboration for the claims. \n- The study is presented with notable clarity and is both well-written and coherently structured.\n- The authors have provided code and rich appendices, which significantly strengthen the paper's reproducibility and utility.\n\n\nWeaknesses:\n- The primary weakness is the limited novelty of the proposed framework. As it primarily combines existing components, the contribution may not meet the high bar for novel methods expected at NeurIPS. Given its strong application, the work might be better suited for a top-tier healthcare informatics venue (e.g., ICHI, CBMS) or a journal, where the applied contribution would be more prominently recognized.\n\n- A discussion on the potential benefits of using specialized, fine-tuned models would be a valuable addition. Exploring this could offer a more comprehensive understanding of the framework's place within the existing landscape of NLP methods.", "was_revised": true, "final_justification": "I have raised my score.\n\nThe clarification on the specific methodological innovations was helpful, but the new results demonstrating the limitations of current medical Large Language Models (LLMs) were particularly compelling. These findings significantly strengthen the paper's contribution and impact.\n\nThe rebuttal has convinced me that the paper is stronger than I had initially perceived. Consequently, I have raised my score to reflect this positive reassessment.", "rebuttal": "**W1. Clarification on Methodological Innovations** We appreciate the reviewer’s feedback and would like to clarify that our framework is not merely a combination of existing components, but introduces several key innovations tailored to the unique challenges of psychiatric diagnosis:\n- Granular Scale Decomposition: Instead of treating scales as fixed global scores, we restructure them into item-level diagnostic signals. This mitigates the limitations of self-reported or clinician-rated scale scores by enabling finer-grained and more interpretable analysis.\n- Retrieval-Augmented Symptom Matching: Unlike generic RAG applications, we apply retrieval specifically to address the symptom overlap problem in psychiatric diagnosis, leveraging structured DSM-5 criteria and expert-annotated examples to improve diagnostic precision in ambiguous cases.\n- Debate-Inspired Multi-Agent Reasoning: We introduce a novel debate mechanism, inspired by formal debate tournaments, in which independent diagnostic agents present their reasoning and a non-intervening judge agent synthesizes the outcome. This design enables transparent multi-perspective evaluation and handles difficult or conflicting cases with human-like deliberation.\n\nTogether, these innovations go beyond simple component reuse and are purpose-built for the nuanced reasoning required in mental disorder diagnosis.\n\n**W2/Q1. Medical LLM as backbone** We are testing the results of medical llm as backbone, and we will report the result in this rebuttal as soon as we finish the experiment.\n\n**Q2. Synthetic data generation methods** We appreciate the insightful suggestion regarding synthetic data. While our primary contribution lies in the diagnostic framework, the synthetic dataset was developed as an additional effort to address the challenges of data privacy and the lack of high-quality public datasets in psychiatric diagnosis. Currently, we employ an off-the-shelf method for case generation, and we are actively exploring ways to incorporate more diverse datasets and multimodal inputs to further advance AI-driven psychiatric diagnosis."}, {"review_id": "L6LDoFUDf8", "reviewer": "Reviewer_6o25", "summary": "The paper introduces a retrieval-augmented multi-agent system for psychiatric diagnosis. Specifically, it proposes a granular item-level analysis of psychiatric scales by integrating DSM-5 structured retrieval and generates a synthetic dataset (MoodSyn) to address data privacy issues. This study is validated on a hybrid dataset with both real-world cases and synthetic cases.", "questions": "See above.", "limitations": "The study lacks clinical justification and validation.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n- The challenge of AI-assisted mood disorder diagnosis is important.\n\n- The reviewer acknowledges that the authors took privacy concerns into consideration.\n\n\nWeaknesses:\n\n1. Insufficient Clinical Validity of the Proposed Framework\n- There is no involvement of licensed clinical professionals in validating the diagnostic outputs, aside from general references to coauthors.\n\n- The design of the Granular-scale Analysis lacks justification regarding its clinical validity and reliability.\n\n- Mood disorders are highly dependent on clinical interviews and patient interaction; thus, they cannot be reliably diagnosed from scale scores and written notes alone.\n\n2. Lack of Clinical Realism in the Synthetic Dataset\n- No human evaluation by licensed mental health professionals or comparison with gold-standard clinical diagnoses is provided.\n\n- Statistical similarity $\\neq$ clinical adequacy: Diagnostic decisions depend on subtle symptom patterns and context, not just distributional alignment.\n\n- There is no evidence that the synthetic dataset improves model performance on downstream or real-world tasks.\n\n- The dataset's clinical representativeness is unclear, and no comparisons are made to existing synthetic or anonymized psychiatric datasets.\n\nOverall, the authors fail to evaluate the realism, practical utility, and clinical usefulness of the synthetic dataset.\n\n3. Unfair Baseline Comparisons\n\n- GPT-4o and other baselines are neither (1) designed for psychiatric diagnosis nor (2) fine-tuned for it, making the performance comparisons methodologically unsound and misleading.", "was_revised": true, "final_justification": "The reviewer is still not fully convinced about the clinical realism and practical implementation of the study in clinical trials.", "rebuttal": "**W1.1 Clinical validation of diagnostic output** We evaluated their performances by three licensed psychiatry experts on our co-author clinician’s team. Metrics are clinical coherence (0-2 points),  information completeness (0-2 points), and clarity of expression (0-2 points). \n\nClinical Coherence: whether reasoning aligns with clinical logic and evidence.\n- 2 = Fully coherent and professionally sound\n- 1 = Minor logical flaws\n- 0 = Major errors or conflicts\n\nInformation Completeness: inclusion of key diagnostic elements (e.g., symptoms, scales, course, exclusions, impairment).\n- 2 = Comprehensive and sufficient\n- 1 = Missing important elements\n- 0 = Lacks key diagnostic info\n\nClarity and Expression: clarity and professionalism of the explanation.\n- 2 = Clear and professionally worded\n- 1 = Some ambiguity or terminology misuse\n- 0 = Unclear or confusing\n\nHere are the assessment results:\n|Model|coh1|com1|clarity1|coh2|com2|clarity2|coh3|com3|clarity3|avg_coh|avg_com|avg_clarity|\n|:-------|:-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|-----:|\nGPT-4o|0.5|0.9|0.7|0.2|0.5|0.9|0.6|1|1.1|0.43|0.5|0.9|\nAngel.R|0.9|1.1|0.9|0.6|0.7|1.2|1|1.3|1.3|0.87|1.03|1.17|\nAngel.D|0.8|1.2|0.9|0.6|0.6|1|0.9|1.3|1.2|0.83|1.03|0.73|\nAngel.C|0.9|1.2|1|0.7|0.8|1.1|1.1|1.4|1.3|1.27|1.4|1.13|\nmulti-Angels|1.1|1.3|1.1|0.8|0.8|1.1|1.1|1.1|1.1|1.3|1.3|1.1|\n\nWe report Fleiss' Kappa as the measure of inter-annotator agreement (IAA).\n|Model|coherence|completeness|clarity|\n|:-------|:-----:|-----:|-----:|\nGPT-4o|0.6|0.26|0.13|\nAngel.R|0.55|0.37|0.19|\nAngel.D|0.69|0.28|0.25|\nAngel.C|0.69|0.33|0.27|\nmulti-Angels|0.67|0.45|0.54|\n\nThe results demonstrate the high quality of our generated outputs. **The higher inter-annotator agreement suggests that the outputs of our multi-agent system are more consistent, clinically aligned, and better recognized by experts.**\n\n**W1.2 Literature Basis for Item-Level Analysis** Traditional psychiatric diagnostic approaches have predominantly relied on total score interpretations of assessment scales, potentially obscuring crucial symptom-level heterogeneity. Recent advances in depression research have highlighted the importance of granular-scale analysis to overcome this limitation. [1] demonstrated how item-level variability in depression scales (e.g., distinguishing somatic vs. cognitive symptoms) challenges conventional unidimensional scoring. Beyond this, [2] identified core depressive symptoms through factor analysis of individual Hamilton Scale items, while van [3] conclued clinically meaningful MDD subtypes using PHQ-9 item clustering. Through close collaboration with clinical experts, our MoodAngels framework extends this paradigm by systematically grouping scale items to reduce assessment bias and enhance diagnostic precision.\n\n[1] Measuring depression over time… Or not? Lack of unidimensionality and longitudinal measurement invariance in four common rating scales of depression. In *Psychological Assessment*, 28(11), 1354–1367.\n\n[2] An Item Response analysis of the Hamilton Depression Rating Scale using shared data from two pharmaceutical companies. In *Journal of Psychiatric Research*, 38(3), 275-284.\n\n[3] Data-driven subtypes of major depressive disorder: a systematic review. In *BMC medicine*, 10(1), 156.\n\n**W1.3 Dependence on Clinical Interviews** Our framework incorporates medical records and five clinician-administered scales, all of which rely on structured clinical interviews. Rather than replacing clinical interactions, our system is designed to assist clinicians after the full interview process is completed, by supporting diagnostic reasoning based on the comprehensive set of information gathered from the patient.\n\n**W2.1/W2.3/W2.4 Human Evaluation of Synthetic Dataset** Our synthetic dataset consists of scale-based numerical data. We initially filtered the data by ensuring the sum of subscores did not exceed the total score. Additionally, we evaluated how closely the synthetic data resembled real data using automated metrics (see Line 877, achieving a score of 0.87). For human evaluation, we randomly sampled 50 cases (25 real and 25 synthetic) and asked three licensed clinicians to distinguish between them. Their classification accuracy was as follows:\n\n|Clinician|Correct Real|Real Misclassified|Correct Synthetic|Synthetic Misclassified|\n|:-----|:------:|-------:|-------:|-------:|\n1|18|7|20|5|\n2|21|4|22|3|\n3|19|6|18|7|\nThe accuracy rates for identifying synthetic data as \"real\" were 0.80, 0.88, and 0.72 across the three clinicians, demonstrating that the synthetic data could convincingly mimic real data.\n\n**W2.2 Pattern Accuracy of Synthetic Data** In line 847, we evaluated Pattern Accuracy (α=0.72), which measures how precisely the synthetic data reproduces the core statistical patterns of real data, ensuring generated samples are realistic and representative.\n\n**W2.4 Comparisons to existing psychiatry datasets** Current scale data are mainly one self-reported scale for the depression detection task. We found no existing dataset enough for mood disorder diagnosis. Therefore, besides our main contribution of designing a diagnostic framework and testing its performance on real clinical data,  we constructed a synthetic dataset to prompt academic research in the field.\n\n**W3. Medical baselines** We tested the result of medgemma-27b-text-it, which is an advanced LLM provided by Google. We found that in real cases, it tends to diagnose nearly every case with mood-disorder-related symptoms in the medical record as mood disorder, without having the sensitivity of overlapping symptoms in other mental disorders.\n\nResult on synthetic data: recall: 0.67, acc: 0.743, mcc: 0.57, macro f1: 0.717. \n\nResult on real data: recall: 0.535, acc: 0.704, mcc: 0.525, macro f1: 0.704."}, {"review_id": "RpU3Vejedl", "reviewer": "Reviewer_jxzA", "summary": "This paper introduces MoodAngels, a retrieval-augmented multi-agent framework designed for mood disorder diagnosis. The framework addresses challenges like the subjective nature of assessments and symptom overlap. Key components of MoodAngels include granular-scale analysis of clinical assessment scales, a retrieval datastore incorporating DSM-5 diagnostic criteria and anonymized real-world clinical cases, and a suite of specialized diagnostic agents with varying reliance on historical data, culminating in a multi-agent system that uses a debate mechanism among agents and a judge agent for final diagnosis. The other contribution is the development of a open-source synthetic dataset, MoodSyn, which includes 1173 synthetic psychiatric cases for psychiatric AI research. The authors evaluate their method on real-world clinical data and MoodSyn, comparing it against several baseline LLMs. Results indicate that MoodAngels outperforms baseline LLMs in diagnostic accuracy on the real-world data.", "questions": "* The improvement from Angel.R to the full multi-Angels system is modest (slight improvement in GPT-4o based agents, and no better in Deepseek-V3 based agents). Meanwhile Angel.R outperforms the much more complicated variant Angel.C on real data. Does the added complexity justify this small gain (or no gain), or are there some possible qualitative benefits (e.g., robustness on specific case types) not captured by aggregate metrics? Would be nice to discuss it a bit.\n* Were any statistical significance tests performed for the results in Table 3 and Table 4? If so, what were the findings?\n* For the similar cases retrieval, why are the top-5 most relevant cases retrieved for Angel.D and Angel.C? Is this number fixed and how sensitive is performance to this number?\n* The BGE-M3 embedder is used for symptom matching and similar case retrieval. Was its effectiveness compared to other embedding models for the psychiatric text?\n* If there's no consensus after a certain number of rounds in the multi-agent debate, how does the Judge Agent make a final decision? Does such case mean the diagnosis is unspecified or uncertain?\n* How was the quality and consistency of these GPT-4o generated DSM-5 knowledge ensured and evaluated?\n* The real-world dataset has 56 mood disorder cases in the test set, what is the diagnosis distribution for these cases? \n* In the ablation analysis, the unstructured medical records for both symptom matching and return to agent yielded the best results. Why the authors also mentioned structuring EHR is “an optimal step of reorganizing records into a structured format for precise symptom” in Appendix E.1?\n* Some mood disorders can’t be clearly distinguished from each other. How did you deal with such cases? E.g. bipolar vs. MDD, bipolar vs. ADHD, or schizoaffective disorder vs. bipolar with some psychotic features.\n* Note that usually we don’t use visitor/client but patient for the person who visits the clinic.", "limitations": "Limitations are mentioned and discussed in the appendix. The potential negative societal impact has not been explicitly discussed in this work. It would be nice to discuss the societal impact, specifically the caveats and warnings of using AI for psychiatric diagnosis. Meanwhile it's also good to point out what aspects of psychiatric diagnosis may not be captured well with such proposed method. e.g., if having subgroup analysis for each type of mood disorder, the authors may point out what types of mood disorder are suitable for using the method, what what types are not.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths\n* The authors’ proposed system, MoodAngel, is a tailored multi-agent system designed specifically for the complexities of psychiatric diagnosis, incorporating granular data analysis, knowledge retrieval, and debate reasoning.\n* The paper introduces MoodSyn, a synthetic dataset for mood disorders, along with a detailed account of its generation and a rigorous multi-faceted evaluation. This is a valuable resource for the community.\n* The framework integrates multiple important components: item-level scale analysis, DSM-5 and case-based retrieval, and a multi-agent debate architecture. The data processing steps, although in the Appendix, are also detailed and thoughtful.\n* The MoodAngels agents, particularly Angel.R, demonstrate a substantial improvement on real-world clinical cases.\n* The paper is supported by detailed appendices covering ethical considerations, related work, data processing, synthetic data evaluation, and agent details, which greatly enhance the transparency and depth of the work.\n* AI for psychiatry is very challenging due to data subjectivity, privacy, and lack of objective markers. This work makes a concerted effort to tackle these issues.\n\nWeaknesses\n* While Angel.R shows significant uplift over baselines, the incremental performance gain from Angel.R to the full multi-Angels system (with debate) is relatively small. This raises questions about the cost-benefit of the added complexity. Specifically the paper does not discuss why the proposed multi-agent framework do not show a clearer advantage compared with the simple agent setup, or in some cases perform slightly worse. It would be better to add some discussion about this.\n* The experimental results in Tables 3 and 4 are presented without statistics such as confidence intervals or p-values. Given some small differences between agent variants, statistical tests would be necessary to confirm the significance of these differences.\n* While the ablation study examines medical record format and scale selection for Angel.R, an ablation on the impact of the retrieval component (e.g., w/o DSM-5, w/o similar cases) or the debate mechanism (e.g., no debate, just majority vote among agents) itself would be beneficial.\n* The paper mentions many default parameters for the study. The impact of choices like the number of retrieved cases (top-5), or thresholds in granular-scale analysis (top-5% correlation), is not discussed.", "was_revised": true, "final_justification": "Why not higher\n* The central claim about the superiority of the complex multi-agent debate framework is marginal from empirical evidence.\n* Tasking the binary classification (disorder present/absent) rather than differentiating between specific mood disorders (e.g., Bipolar I vs. MDD) makes the task less complex but also reduces the perceived significance from the clinical use perspective.\n\nWhy not lower\n* Angel.R model itself is a novel and effective system that significantly outperforms strong LLM baselines on a challenging real-world dataset.\n* Open-source MoodSyn dataset is a significant service to the psychiatric AI research community.\n* The methodology and rebuttal are well-detailed with its capabilities and limitations.", "rebuttal": "We really acknowledge your thoughtful questions, and we revised our paper to clarify them and add experiments on the ablation study and analysis.\n\n**W1/Q1 Clinical Significance of Incremental Performance Gains**\nWhile the improvement from Angel.R to the multi-agent system may appear modest in statistical terms, even marginal gains in diagnostic accuracy hold substantial clinical value. In psychiatric practice, where each case carries significant implications for patient care and treatment outcomes, these incremental advancements can translate into meaningful benefits for a broader population of individuals seeking mental health support.\n\n**W2/Q2 Statistical Significance Test Reults** \n- Interpretation of p-values: All models except llama3-8b-instruct on real cases have p-values far less than 0.05, indicating that their predictive performance is significantly better than random guessing.\n\n- 95\\% Confidence intervals (CI): The 95% confidence intervals for each model are relatively narrow, suggesting that the results have high statistical reliability.\n\nResults of real cases:\n| Model | ACC |95\\% CI | p value |\n|:-------|:-----:|-----:|-----:|\n| llama3-8b-instruct | 0.506 | [0.464, 0.548] | 0.80 |\n| mistral7b-instruct-v0.3 | 0.597 | [0.555, 0.638] | 0.00 |\n| MedGemma-27b | 0.704 | [0.664, 0.742] | 1.58e-22 |\n| gpt-4o | 0.797 | [0.761, 0.829] | 1.46e-47 |\n| deepseek-v3 | 0.847 | [0.814, 0.876] | 3.55e-66 |\n| [gpt]Angel.R | 0.920 | [0.894, 0.941] | 2.00e-102 |\n| [gpt]Angel.D | 0.923 | [0.898, 0.944] | 1.47e-104 |\n| [gpt]Angel.C | 0.914 | [0.888, 0.936] | 2.65e-99 |\n| [gpt]multi-angels | 0.925 | [0.900, 0.946] | 1.22e-105 |\n| [deepseek]Angel.R | 0.927 | [0.902, 0.947] | 9.80e-107 |\n| [deepseek]Angel.D | 0.920 | [0.894, 0.941] | 2.00e-102 |\n| [deepseek]Angel.C | 0.922 | [0.896, 0.942] | 1.74e-103 |\n| [deepseek]multi-angels | 0.923 | [0.898, 0.944] | 1.47e-104 |\n\nResults of synthetic cases:\n| Model | ACC |95\\% CI | p value |\n|:-------|:-----:|-----:|-----:|\n| llama3-8b-instruct | 0.564 | [0.478, 0.648] | 0.15 |\n| mistral7b-instruct-v0.3 | 0.636 | [0.550, 0.715] | 0.00 |\n| MedGemma-27b | 0.743 | [0.662, 0.813] | 7.60e-09 |\n| deepseek-v3 | 0.821 | [0.748, 0.881] | 5.41e-15 |\n| [deepseek]Angel.R | 0.800 | [0.724, 0.863] | 4.25e-13 |\n| [deepseek]Angel.D | 0.807 | [0.732, 0.869] | 1.04e-13 |\n| [deepseek]Angel.C | 0.821 | [0.748, 0.881] | 5.41e-15 |\n| [deepseek]multi-angels | 0.821 | [0.748, 0.881] | 5.41e-15 |\n\n**W3. Ablation Study on Retrieval Components** The strong performance of majority voting validates the diagnostic capability of individual agent judgments. However, the debate process remains essential for ensuring comprehensive symptom analysis through structured deliberation, maintaining output stability via evidence-based adjudication, and providing interpretable reasoning traces for clinical validation.\n\n| Setting| ACC | |MCC | |\n|:-------|:-----:|-----:|-----:|-----:|\nAngel.R|0.927||0.841|| \nAngel.R w/o DSM5 Symptom Matching|0.923|0.004↓|0.83|0.011↓|\n\n| Setting|ACC|| MCC||\n|:-------|:-----:|-----:|-----:|:-------|\nmulti-angels debate |0.925 ||0.834||\nmulti-angels majority vote| 0.923|0.002↓|0.835|0.001↑|\n\n**W4. Top-5% Scale-Item Threshold Selection**\nIn designing our threshold criteria, we carefully considered this important aspect. Through a review of established psychiatric methodologies [1], we observed that prior work typically derives thresholds through a combination of clinical expertise and empirical data distribution analysis. To ensure clinical validity, we first computed Pearson correlation coefficients for each scale item, then engaged in extensive discussions with our clinician co-authors to determine appropriate thresholds. Notably, based on their expert recommendations, we incorporated additional items (PHQ-9 Q1 and Q2) that demonstrated particular clinical relevance, as detailed in lines 697-700 of our manuscript.\n\n[1]Unraveling the distinction between depression and anxiety: A machine learning exploration of causal relationships.\n\n**W4/Q3. Retrieval Top-k Similar Cases Threshold Selection** \nWe will include an ablation study that varies the top-k value (0, 1, 3, 5) for case retrieval, and we will report the results in the rebuttal once testing is complete. Notably, we observed that Angel.R already performs well even without retrieval (k=0), which further supports the robustness and quality of our overall framework.\n\n**Q4. Choice of Retriever** We chose BGE-M3 for its strong semantic representation capabilities, which align well with our goal of capturing nuanced symptom similarities in clinical texts. While we agree that exploring alternative retrievers could offer additional insights, our current focus has been on validating the overall effectiveness of the framework. Notably, as shown in W3, even in the absence of any retrieval component (Angel.R w/o DSM-5 Symptom Matching), the model still achieves promising results, suggesting the robustness of the reasoning process. We hope this addresses your concern, and we truly value your encouraging feedback.\n\n**Q5. Judge's Criteria for Ending the Debate.** We designed the prompt of the Judge Agent to simulate a debate format, with a minimum of 2 rounds and a maximum of 4. At the beginning of each round, the prompt guides the Pro and Con agents through a structured process: Round 1 for opening statements, Round 2 for rebuttals, Round 3 for clarifying unresolved points, and the final round for closing arguments. After each round, the Judge Agent evaluates whether the debate should continue, provides reasoning, and reflects on the arguments. At the end of the debate, the judge delivers a diagnosis with reasons. \n\n**Q6. Quality of Knowledge Base.** The diagnostic and differential criteria from the DSM-5 were originally presented as bullet points, as shown in Appendix C1. We used GPT-4o to revise these items into concise, self-contained statements, rather than generating new diagnostic content. This step was necessary because some original entries contained pronouns or overly detailed explanations that were not helpful for symptom matching between the knowledge base and medical records. After constructing the knowledge base, each extracted item was reviewed and verified by our domain expert.\n\n**Q7. Distribution of real cases** We are calculating the detailed types, and we will report the result in this rebuttal.\n\n**Q8. Medical Record Structuring** We designed medical record structuring as an optional component to ensure the flexibility of our framework, particularly in scenarios where medical records lack clarity or where input token limitations need to be considered.\n\n**Q9. Distinguishing Mood Disorders.** Currently, our framework performs binary classification—determining whether a mood disorder is present. While we have extracted the DSM-5 differential criteria for more fine-grained distinctions, we do not attempt subcategory classification. This is because psychiatric diagnosis requires careful deliberation; in clinical practice, clinicians typically first identify a broad diagnostic direction before narrowing it down through further assessments. Similarly, our framework is designed to support this initial diagnostic stage.\n\n**Q10. Terminology for Individuals Seeking Help** Thanks for raising this important point. We also place great importance on using professionally appropriate terminology. For individuals who have not been formally diagnosed with a mental disorder, we use terms such as client or visitor to show respect and avoid assumptions. Once a diagnosis is confirmed, we refer to them more specifically as patients. For example, as in line 276, where we state “a patient with severe depression.”\n\n**L1. Usage Declaration of the Diagnostic Framework** Thanks. We will revise our paper and GitHub repo to add a usage declaration."}]

Author rebuttal:

Meta-review: **Summary**
This paper presents a novel multi-agent framework, MoodAngels, designed for psychiatric diagnosis of mood disorders, and introduces a synthetic dataset, MoodSyn, to address data privacy challenges. 

**Strengths**
* This work introduces a novel synthetic dataset, MoodSync, comprising 25 clinical features—including 16 diagnostic questions, 8 standardized scale scores, and expert-verified labels—that provides clinically valid and privacy-preserving cases for mood disorder analysis, enabling robust AI development in computational psychiatry.
* The paper also presents a novel framework addressing critical challenges in AI-assisted psychiatry, such as symptom overlap and interpretability. Specifically, the paper authors develop a tailored, clinically-grounded multi-agent system (e.g., Angel.R, Angel.D, Angel.C) that integrates granular-scale analysis, DSM-5-informed retrieval, and structured debate mechanism. 

**Weaknesses**
* The empirical performance gain from the complex debate mechanism over the simpler (but highly effective) Angel.R or a majority vote is modest, though its value for interpretability is recognized.
* The task is framed as a binary classification (mood disorder present/absent) rather than fine-grained differential diagnosis between specific disorders, which is a necessary scope limitation but reduces immediate clinical application.


**Reason for Accept/Reject**

*Borderline Accept* - The paper presents two novel and methodologically sound contributions—a privacy-preserving synthetic dataset and an interpretable multi-agent framework—that meaningfully advance the field of AI in psychiatry. The authors also successfully address key reviewer concerns during rebuttal by demonstrating ethical rigor and providing additional analyses, though the practical clinical impact may be currently limited by the binary classification task.

**Rebuttal Discussion**
Initial reviews were mixed, but through the rebuttal phase of auxiliary experimental results and clarification to concerns, the authors addressed the majority of raised issues, and this resulted in three reviewers (HsoZ, EzJT, jxzA) updating their scores positively, leading to a consensus for acceptance.

In address of ethical concern consideration, the paper includes a clinician audit of synthetic data and a detailed subgroup performance analysis (by age and gender) that confirmed no significant bias, as requested by the ethics reviewer.
Decision: Accept (poster)