Does Rationale Quality Matter? Enhancing Mental Disorder Detection via Selective Reasoning Distillation (2025)

Abstract: The detection of mental health problems from social media and the interpretation of these results have been extensively explored. Research has shown that incorporating clinical symptom information into a model enhances domain expertise, improving its detection and interpretation performance. While large language models (LLMs) are shown to be effective for generating explanatory rationales in mental health detection, their substantially big parameter size and high computational cost limit their practicality. Reasoning distillation transfers this ability to smaller language models (SLMs), but inconsistencies in the relevance and domain alignment of LLM-generated rationales pose a challenge. This paper investigates how rationale quality impacts SLM performance in mental health detection and explanation generation. We hypothesize that ensuring high-quality and domain-relevant rationales enhances the distillation. To this end, we propose a framework that selects rationales based on their alignment with expert clinical reasoning. Experiments show that our quality-focused approach significantly enhances SLM performance in both mental disorder detection and rationale generation. This work highlights the importance of rationale quality and offers an insightful framework for knowledge transfer in mental health applications.

## introduction detecting mental health issues at an early stage is crucial for initiating timely interventions that can significantly improve treatment outcomes. as online communities continue to grow, researchers in nlp have developed methods to screen usergenerated content for signs of depression, anxiety, and other mental illnesses @xcite @xcite been depressed since the 5th grade. now i'm 28 year old. i've tried multiple conseling, medications, and therapy. i'm trying to wean myself off effexor now because i'm just tired of the side effects. going through hell at the moment. i'm just tired. is there a light at the end of this (...) ## method this section introduces our proposed framework for improving the effectiveness of reasoning distillation by selectively focusing on high-quality rationales for major depressive disorder detection, as illustrated in figure 2 . this framework includes rationale generation, quality evaluation, and qualitybased selection. ## analysis of quality-based selection 6.1 clinical relevance improvement in order to investigate how our quality-based selection approach enhances the clinical relevance of rationales, we compared the distribution of semantic similarity scores between rationales generated by different teacher models and symptom descriptions from the dsm-5 diagnostic criteria for depression. we utilized bertscore @xcite to compute the semantic similarity, making it suitable for assessing the alignment between rationales and clinical descriptions. we visualized these distributions in figure 4 . the histograms within each panel show the frequency of different bertscore values for teacher-generated rationales with and without our quality-based selection. in most cases, applying quality-based selection results in a noticeable shift of the semantic similarity score distribution towards higher values compared to the w/o selection condition. this suggests that our method improves the clinical relevance of the rationales by increasing their semantic similarity to the dsm-5 criteria, indicating that they convey more precise and relevant information about the associated symptoms. this improvement is essential because it ensures that the rationales used for reasoning distillation are clinically meaningful and informative, leading to enhanced student models through fine-tuning with them. ## conclusion this paper empirically investigated how the quality of rationales, especially their relevance to domain knowledge, impacts the rationale distillation process. we proposed a quality-focused framework that fine-tunes smaller language models with rationales exhibiting high domain relevance, achieved through a process of evaluating and selecting rationales generated by teacher models. our experiments demonstrated that our selective reasoning distillation significantly improves both detection accuracy and clinical interpretability in major depressive disorder detection. these results shed light on the importance of refining distillation data to enhance the model performance and interpretability, especially in mental health applications requiring high-quality domain knowledge. this work provides a promising direction for enhancing the performance and interpretability of smaller language models in mental health applications. our selective distillation framework, which prioritizes high-quality, domain-specific knowledge, offers a valuable approach, especially in domains where accurate and reliable explanations are crucial.