D4: a C hinese Dialogue Dataset for Depression-Diagnosis-Oriented Chat (2022)

Abstract: In a depression-diagnosis-directed clinical session, doctors initiate a conversation with ample emotional support that guides the patients to expose their symptoms based on clinical diagnosis criteria. Such a dialogue system is distinguished from existing single-purpose human-machine dialog systems, as it combines task-oriented and chit-chats with uniqueness in dialogue topics and procedures. However, due to the social stigma associated with mental illness, the dialogue data related to depression consultation and diagnosis are rarely disclosed. Based on clinical depression diagnostic criteria ICD-11 and DSM-5, we designed a 3-phase procedure to construct D ^4 : a Chinese Dialogue Dataset for Depression-Diagnosis-Oriented Chat, which simulates the dialogue between doctors and patients during the diagnosis of depression, including diagnosis results and symptom summary given by professional psychiatrists for each conversation. Upon the newly-constructed dataset, four tasks mirroring the depression diagnosis process are established: response generation, topic prediction, dialog summary, and severity classification of depressive episode and suicide risk. Multi-scale evaluation results demonstrate that a more empathy-driven and diagnostic-accurate consultation dialogue system trained on our dataset can be achieved compared to rule-based bots.

## introduction given the increasing worldwide health threat brought by depression, researchers have been exploring effective methods for depression detection and diagnosis. besides automatic depression detection from posts on social media @xcite , speech @xcite and multimodality @xcite , the dialogue system is considered an effective tool for largescale depression detection @xcite . it is believed that conversation agents could reduce the concealment of sensitive information such as suicidal thoughts caused by social expectations @xcite and the emotional hindrance due to the pressure of being judged in face-to-face conversation @xcite . in past research, chatbots initiated for depression diagnosis are generally implemented based on self-rating scales @xcite or diagnostic criteria @xcite . the final diagnosis results are obtained by asking fixed questions on the scale and corresponding the user's answers to each question to the scale options. these chatbots present good sensitivity and specificity in diagnosis and are more attractive and acceptable @xcite than the original self-rating scales. nevertheless, the fixed dialogue flow limiting the user's expressions to specific answers can not realize personalized consultation and give emotional support at an appropriate time, for which there still exists a big gap between the conversation experience current depression diagnosis agents provide and the face-to-face interview in the process of clinical diagnosis. interview-based clinical diagnosis in psychiatry is a complex procedure with the purpose of collecting and summarizing key symptom infor-mation about one patient while providing a chatlike conversation experience. in clinical practice, psychiatrists communicate with patients and offer diagnosis results based on practical experience and multiple diagnostic criteria. the most clinicallyadopted criteria involve icd-11 (the world health organization, 2022), dsm-5 (american psychiatric association, 2013), etc., which define core symptoms for the depression diagnosis. at the same time, psychiatrists provide emotional support such as empathy and comfort during the consultation to better prompt patients' self-expression. the practice of clinical depression diagnosis displays the possibility of the depression diagnosis dialogue system in further improving the accuracy of diagnosis and user engagement. accordingly, the depression diagnostic conversation belongs to a distinguished dialogue from previously defined dialogue typologies, which is a combination of task-oriented dialogue and chitchat. such a compound dialogue type could be defined as task-oriented chat as shown in figure 1 . this type of dialogue requires multiple assessments regarding task completion and chit-chat experience, which are extremely challenging and still under-investigated. as a specific domain of task-oriented chat, the depression diagnosis dialogue has a clear purpose of the task-oriented dialogue aiming at medical diagnosis: to collect the patient's symptom information and draw a diagnosis conclusion while simultaneously bearing the needs of a chit-chat dialogue with emotional support: to start a user-oriented chat and provide emotional support from time to time. currently, no datasets are specified for depression diagnosis, mainly due to the social stigma associated with clinical privacy and the complexity of the diagnosis process. to construct a clinically sound and empathetic depression-diagnosis-oriented dialogue system close to clinical practice, we conduct dialogue collection through consultation dialogue simulation. we devise a three-phase approach to collect depression diagnostic dialogues (see figure 2 ). p1: to simulate medical records, we collect actual patients' portraits with a consultation chatbot web app that asks users fixed questions abstracted from clinical depression diagnosis criteria icm-11 and dsm-5. p2: to restore psychiatric consultation conversations, we employ workers to conduct the consultation dialogue simulation based on the collected portraits. the workers are divided into pa-tients and doctors for separate training by professionals. the doctor actor is required to obtain fixed symptom information involved in the diagnostic criteria in the chat, while the patient actor needs to express according to the symptoms in the portrait. p3: to reinforce the clinical setting, professional psychiatrists and psychotherapists supervise the whole process and filter out unqualified dialogues. in addition, they provide diagnosis summaries based on the portrait and dialogue history. we further annotate the conversation procedure with 10 topic tags and the symptom summaries with 13 symptom tags (grouped by core depressive symptoms listed in dsm-5 and icd-11). in this way, we propose d 4 : a chinese dialogue dataset for depression-diagnosis-oriented chat. the key contribution of this paper is as follows: ## data collection to maximize doctor and patient authenticity in a diagnosis dialogue, we devise a 3-phase collection paradigm (see figure2) instead of the commonlyadopted vanilla crowdsourcing scheme: p1. we collected natural populations' portraits (in particular actual depressive patients) to form pre-diagnosis records; p2. simulated natural diagnostic consultation dialogues based on the portraits; p3. psychiatrists proofread dialogue history and prescribed professional symptom summaries. age: 30 gender: 女 marital status: 已婚 occupation: 证券公司 reason: 生活不受控制无能为力 symptom description: 0. 持续的情绪低落，且有晨重夜轻特点 1. 兴趣低下，对所有事情兴趣丧失 2. 注意力难以集中，记忆力变差 3. 缺乏自信心，自我价值感低，有自罪感，无望感 8. 有自杀想法 age: 30 gender: female marital status: married occupation: securities company reason: my life is uncontrollable and powerless symptom description: 0. persistent depression, more severe in the morning 1. low interest, loss of interest in everything 2. difficulty concentrating, memory deterioration 3. lack of self-confidence, low self-worth, guilt, hopelessness 8. thoughts of suicide 医生你好，我最近感觉很丧 hi doctor, i feel very bad recently 你好，请问你有什么问题呀？ hi what can i do for you? 感觉自己很没用，干什么都不行 i'm feeling really useless and couldn't do anything well 嗯嗯，你处于这种状态有多久了？ well how long have you been feeling this way? 快一个月了 about a month (社会功能) 确实有一段时间了，最近有在正常上班吗？ (social function) it's been a while, so have you been at work recently? 我在正常上班，但是工作效率很低，还觉得很疲惫 i go to work daily but it is quite inefficient and sometimes i feel exhausted (精神状态) 除了觉得累，最近的注意力记忆力有什么变化吗？ (mental state) how do you feel about your attention and memory apart of feeling exhausted? 很难集中注意力，也记不住事情 it's hard to focus and remember things 是因为什么原因呢？ what do you think that might be the reason? (共情安慰) 你有比较明显的抑郁症状，但现在还在正常工作，虽然 压力很大，仍旧坚持，且能主动寻求帮助，建议到专科医院在进一 步检查。 (empathy) you seem to have the symptoms of depression, but you are still working normally. although you are under a lot of pressure, you still persist and can seek help. it is recommended to go to a specialist hospital for further examination. depressive episode risk: 中度 suicide risk: 轻度 symptom summary: 来访者近一个月心情低落，尤其是晚上，有焦虑情绪，兴趣 下降，记忆力下降，愉快感下降，信心下降，自我价值感低， 无望感，精神运动性激越。有自杀念头。社交功能受影响。 建议引起重视，去专科门诊进一步诊断. depressive episode risk: moderate suicide risk: mild symptom summary: the patient has been depressed for the past month, especially at night and has anxiety, loss of interest, memory deterioration, decreased pleasure, low confidence and selfworth, hopelessness, psychomotor agitation and suicidal thoughts with social function affected. it is recommended that the patient go to a specialist clinic to confirm. p1: human-to-machine portrait collection p2: human-to-human dialogue collection p3: professional diagnosis collection ## conclusion in this paper, we designed a 3-phase data collection and constructed a close-to-clinical-practice and up-to-clinical-standard depression diagnosis dataset with 1,339 conversations accompanied by psychiatrists' diagnosis summaries. further, we conduct experimental validation on multiple tasks with state-of-art models and compare the results