Empowering Psychotherapy with Large Language Models: Cognitive Distortion Detection through Diagnosis of Thought Prompting
Authors: Zhiyu Chen
Yujie Lu
William Yang Wang
Venue: EMNLP 2023 Findings (2023)
Keywords: Psychotherapy
Cognitive distortion
Cognitive behavior therapy
Large language models
TL;DR: We unveil the power of LLM to assist real professional psychotherapy by proposing the diagnosis of thought prompting for cognitive distortion detection.

Abstract: Mental illness remains one of the most critical public health issues of our time, due to the severe scarcity and accessibility limit of professionals. Psychotherapy requires high-level expertise to conduct deep, complex reasoning and analysis on the cognition modeling of the patients. In the era of Large Language Models, we believe it is the right time to develop AI assistance for computational psychotherapy. We study the task of cognitive distortion detection and propose the Diagnosis of Thought (DoT) prompting. DoT performs diagnosis on the patient's speech via three stages: subjectivity assessment to separate the facts and the thoughts; contrastive reasoning to elicit the reasoning processes supporting and contradicting the thoughts; and schema analysis to summarize the cognition schemas. 
The generated diagnosis rationales through the three stages are essential for assisting the professionals. Experiments demonstrate that DoT obtains significant improvements over ChatGPT for cognitive distortion detection, while generating high-quality rationales approved by human experts.

Empowering Psychotherapy with Large Language Models:
Cognitive Distortion Detection through Diagnosis of Thought Prompting
Zhiyu Chen', Yujie Lu', and William Yang Wang"
'Carnegie Mellon University
2 University of California, Santa Barbara
zhiyuc@andrew. cmu edu, {yujielu, william ucsb. edu
Abstract
To mitigate such situations, there have been consistent efforts on developing automated sys- tems for mental health support, such as sentiment analysis (Rathje et al., 2023) and empathetic chat- bots (Welivita et al., 2021; Sharma et al., 2020b; Saha et al., 2022). However, existing works mostly take shallow attempts in heuristic manner, e.g.. analyzing emotions and generating comforting sponses. There is still a significant gap for such sys- tems to contribute real professional psychother- apy, which requires deep studies of the patient's 's thinking patterns, the establishment of cognition models, and the methods to reconstruct the cog- nition models. These procedures constitute the core pillars in common classic therapy paradigms like bechaviv therapy (CBT) (Rothbaum et al., 2000; Wright et al., 2017; Beck, 2020) and acceptance and commitment therapy (ACT) (Har- ris, 2006; Hayes et al., 2011). Most data sources recording the interactions between the patients and licensed professionals are confidential, making it even more challenging to build professional assis- tance for psychotherapy.
Mental illness remains one of the most criti- cal public health issues of our time, due to the severe scarcity and accessibility limit of pro- fessionals. Psychotherapy requires high-level expertise to conduct deep, complex reasoning and analysis the cogmition modeling of the patients. In the era of Large Language Mod- els, we believe Al assistance for computational psychotherapy. We study the task of cognitive distortion de- tection and propose the Diagnosis Thought (DoT) prompting. DoT performs diagnosis the patient's speech via three stages: subjec- rivity assessment to separate the facts and the thoughts; contrastive reasoning to elicit the rea- soning processes supporting and contradicting the thoughts; and schema analysis to summa- rize the cognition schemas. The generated di- agnosis rationales through the three stages are essential for assisting the professionals. Experi- ments demonstrate that DoT obtains significant improvements over ChatGPT for cognitive dis- tortion detection, while generating high-quality rationales approved by human experts.
the right time develop
Recent progress in Large Language Model (LLM) development uncovers its astonishing abil- ity in various textual reasoning tasks in zero-shot setting (Kojima al., 2022: Bang al., 2023; Chen, 2022). For the psychology domain, Chat- GPT and GPT-4 present very promising perfor- mance in the classic Sally-Anne test (Baron-Cohen et al., 1985; Bubeck et al., 2023), evaluating the basic theory of mind capabilities to attribute mental states such as beliefs, emotions, desires, etc. We believe it is promising to further exploit such ability to complex mental reasoning and analysis. It is the right time start developing professional, targeted, and systematic AI assistance for psychotherapy.
1 Introduction
About one in eight people worldwide suffer from mental disorders (World Health Organization, 2022). However, such mental health conditions are severely underserved, due to a number rea- sons including the scarcity of mental health pro- fessionals, poor quality of services, unaffordable cost, and social stigma (White and Dorman, 2001; Sharma et al., 2020b). Treatment coverage for men- tal health service use ranges from 33% in high- income locations to only 8% in low- and lower middle-income countries (Moitra et al., 2022). A recent study from the American Psychological As- sociation (APA)' found that six in ten psychologists "no longer have openings for new patients."
In this paper, we take the first step by study- ing the task of cognitive distortion detection, the first core procedure in cognitive behavior therapy (CBT) (Beck, 2020). Inspired by how psychother-
http:///hh/aaaaaaaaiiaaaaaoioeo/ wwtp.///ccccyyy/22//////eeeeeeee shorttge-usppyycholgiisssssa
'htpss//www.thhqurddian

---

Cognilive Distortion Type
Interpretation
Examppe Distorted Speech
Personalizing taking up blame situaaion, that reality involved many factoes and was out of the person's control.
My son is preity quiet today. wonder what I cld to upsci him.
Persomalizatioo
Mind Reading
Suspecting
are thinking
motivations behind actions.
My house over, they must
when my friends cume
Looking at sitation as cither black white or thinking that there are only two
All-or-nothing thinking
possible outcores situation.
IfT cannot Ph.D. then I Fam total failtrre.
Table Three example common cognitive distortion types, taken from (Beek, 2020; Shreevastava Foltz, 2021).
apy professionals perform nuanced diagnosis over the patient's speech, we propose the Diagnosis of Thought (Do'T) prompting. In DoT, we diagnose the patient's speech through three stages: (1) sub- jectivity assessment, (2) contrastive reasoning, and (3) schema analysis. In subjectivity assessment, we distinguish the patient's subjective thoughts from the objective facts; In contrastive reasoning, we elicit the reasoning processes supporting and op- posing the patient's thoughts; Finally, in schema analysis, summarize the underlying thought schema and map the cognitive distortion types. We conduct comprehensive experiments using the recent top- top-perforiing LLMs. In zero-shot setting, DoT obtains over 10% and 15% relative improve- ments for distortion assessment and classification, respectively, on ChatGPT. Meanwhile, the gener- ated rationales through the three stages grant full interpretability for the diagnosis process, whose quality is further approved by human experts.
adaptive negative thoughts and summarize their un- derlying schemas, formally known cognitive dis- tortions. There are generally 10-20 common, well- studied types of cognitive distortions. We present a few types with examples in Table 1, and refer
the readers to Appendix for the full list. Once accurately identify these cognitive distortions, CBT therapists will guide the patients to justify and cor- rect these distortions, so as to gradually reconstruct their cognition models.
In the real psychotherapy process, there's a sig- nificant amount of textual information including therapeutic conversations and diaries, etc. Such information is often long, highly fragmented, and disorganized, containing multiple types of distor- tions beyond the toy examples in Table 1. The task of cognitive distortion detection aims to automati- cally detect the distortion types given such textual information from the patients, in order to assist therapists to enhance their efficiency and productiv- ity. Meanwhile, such detectors can also potentially serve as self-assisting tools for the patients to di- agnose their thoughts and conduct CBT practice, upon meeting the robustness and safety require- ments. Formally, cognitive distortion detection consists of two steps: 1) Distortion assessment to predict whether the given speech contains cogni- tive distortions, and 2) Distortion classification to predict the spe- cific distortion types, as a multiclass classification problem.
We unveil the great potential of empowering pro- fessional psychotherapy with LLM. This explo- ration serves as a catalyst for larger initiative; we extend an invitation to both AI and psychotherapy communitics to come together in a collaborative ef- fort. Our ultimate goal is to construct professional, safe, AI-driven assistance that substantially en- hance mental health support systems.
binary classification problem;
2 Cognitive Distortion Detection
Cognitive Behavior Therapy (CBT) is well- established therapy paradigm primarily on depres- sion and anxiety disorders (Beck, 2020). In CBT, given the patients' speech or written content, we establish the cognitive model by building the inter- actions among the situations, thoughts, and emo- tions. Patients with mental disorders, such as de- pression or anxiety, tend to form negative thoughts very rapidly and unconsciously, leading to nega- tive emotions which further strengthen their overall negative views and beliefs about the world.
3 The Diagnosis of Thought Prompting
On discussing with psychotherapy professionals, we identify the following key stages to diagnose the patient's speech to detect cognitive distortions. Subjectivity Assessment. The patient's speech consists of mixture of reality (objective facts) and interpretations/opinioos (subjective thoughts). In order perform deep analysis of distorted thinking. we first need to find out which parts of the speech are objective facts and which parts are subjective thoughts. After such an assessment, we summa-
To break this vicious circle, in a typical CBT process, the first core step is to identify those mal-

---

Reasoning process supporting the thoughts: People often stay quict someone who hes annoyed them. The advisor likely to get angry with speakior.
Objective facts: advisor was quiet in the meeling with the speeker today.
Undertying thought
Patient speech: My advisor was quiet meeting today, must have done somethhing wrong...
scheme:
The speaker
Distortion
taking up the blame for the situation that his advisor didn't meeling. This in really may involve many the speakar's control.
assessment:
Yas
Subjective thoughts:
Reasoning process
Distortion type:
contradicting the thoughts: may be many reasons making the advisor quiat, e.g. he was just getling loo fired. more, reason the speaker made him unhappy.
The speaker thinks he must have done something wrong that upsets advisor.
of
Personalization
say
Subjectivity assessment
G Contrastive reasoning
Schema analysis
Figure 1: The Diagnosis of Thought framework. We strategically prompt the LLM go through the three diagnosis stages: subjectivity assessment, contrastive reasoning, and schema analysis.
rize the objective facts into the situations as the evidence base diagnose the subjective thoughts. Contrastive Reasoning. This stage aims to dis- cover how the patient ascertains the veracity of their subjective thoughts. Based on the situation, we deduct the reasoning processes that supports and contradicts the patient's thoughts respectively. By contrasting two different interpretations based on the same situation, we can identify the thought schemas more clearly.
the vast amount of speech.
Experimental Results
4.1 Dataset and experimental settings
We experiment the cognitive distortion detection dataset proposed by Shrecvastava and Foltz (2021), which is annotated by experts based on the Ther- apist QA dataset?. The dataset consists of 2,531 examples of patient speech annotated with ten com- mon types of cognitive distortions, as specified in Appendix A. 63. 1% of the examples have cogni- tive distortions, which are annotated with the two dominant ones. We follow the same train-test split in (Shreevastava and Foltz, 2021). For automatic evaluations, we report F-1 for distortion assessment and weighted F-1 for distortion classification.
This stage aims to study why the patient forms the specific reasoning process. The term "schema" refers to the cognitive struc- tures that organize our knowledge, beliefs, and ex- pectations. Understanding what schemas patient is relying mode and distortions.
Schema Analysis.
reveal much about their cognitive
We propose the Diagnosis of Thought (DoT) prompting, guiding the LLM through the above three stages to diagnose the patient's speech. Fig- ure illustrates our framework. We sequentially prompt LLM with three questions for the three stages. After the LLM finishes the generation for all stages, we prompt with another two questions asking distortion assessment and classification. See Appendix B for all the prompts we use.
4.2 Automatic Evaluation Results
We experiment using three of the recent represen- tative LLMs: ChatGPT (gpt-3.5-turbo)) Vicuna*, and GPT-4 (OpenAL, 2023). For all methods, we first prompt with a general instruction specifying the task and the target distortion types. We com- pare DoT prompting with 1) Directly generat- ing the results, and 2) Zero-Shot CoT prompting (ZCoT) (Kojima et al., 2022). For Vicuna, both ZCoT and DoT exceed the token limits for many examples; we omit the results. We average over five runs and report mean and standard deviation for all experiments. Table 2 shows our main experiment results. As we can see, under the zero-shot setting, due to the challenge of this task, Vicuna and Chat- GPT still fall behind the supervised full-training models. The proposed DoT prompting significantly
With DoT prompting, we obtain fully inter- pretable diagnosis rationales for detecting cogni- tive distortions. Such interpretability is vital for professionals in real applications. Serving as the diagnosing assistance tool, the diagnosis rationales are essential for the professionals to justify the re- sults. More importantly, they can potentially learn the nuanced thought patterns and schemas derived from the patient's speech through the rationales. This is crucial for the professionals to establish the patient's 's cognition models more efficiently from
http:///www..kgggl..o/m/mmaaaaae
http:///oopppaacccooo//ooo////// cowbblog/chatggp hhtp::/lmncy..og//bg///222200000
hhtps:///ppeaiii

---

Distortion Assessment
Distortion Classification WWighttd F-1)
ChROPT-boT
24
Methods
(F-1)
0.5
Full training"
75.0
24.0
3.2
Vicuna
73.81a.95
11.230.78
ChatGPT
73.470.58
19.241.00
ChatGPT + ZCoT
77.101.21
20.211.02
atrondig
marde Sit
evoie abaerent
enstbra Anstriiy
nating
Roteen delin
ChatGPT + DoT
81.19h.11
22.250.70
GPT-4 GPT-4 + TCoT
83.04a.51
33.866.83
Figure 2: Distortion classification per class F-1.
81-97
33.221.36
GPT-4 DoT
82.77a.81
34.641.00
Subjectivity Assessment
Contrastive Reasoning
Schema
Methods
Quality
Analysis
Table 2:
Main Results (Standard deviation in subscript). (*) results copied from (Shrecvastava and Foltz, 2021).
Comprehensive
68.3%
45.0%
50.5%
CharGPT +DoT
Partially good
24.3%
30.5%
27.0%
Invalid
7.0%
21.5%
22.5%
Distortion
Distortion
Comprehensive
84.3%
76.5%
Methods
Assessment (F-1)
Classification (Weighted F-1)
GPT4 +DoT
Partially good
13.3%
25.5%
18.5%
Invalid
2.0%
5.0%
5.0%
CharGPT
73.47o.m
19.24
CharGPT +
79.62,.12
18.7 18.7
Table
Human evaluation results for diagnosis rationales.
CharGPT + S2
80.70, 70p.48
20. 11.02
ChatGPT + S1 S2 S3
11.19
22. 22.25g.70
Table 3: Ablation studics on each stage of DoT: we denote Subjectivity Assessment us S1, Contrastive Reasoning as $2, and Schema Analysis as S3.
obtains large improvement, as the summarized un- derlying thought schema match the definition of the distortion types more accurately. We also analyze the distortion classification results for each distortion type, with results shown in Figure 2.
boosts the performance of ChatGPT, with the distor- tion assessment score surpassing the full-training performance and the distortion classification score approaching the full-training performance. The im- provement over ZCoT demonstrates the superiority of our proposed strategic stages compared with ra- tionale generation with no guidance. Astonishingly, GPT-4 surpasses the full full-training performance even by a large margin for distortion classification. On the one hand, this may demonstrate the powerful ability of GPT-4 on this task- it can already reach potential upper bound regardless with DoT pro- moting not; On the other hand, since the dataset we use is publicly released on Kaggle and its orig- inal data source is from online forums, we do not exclude the possibility of data contamination. Ablation Studies and Analysis. To better under- stand how each stage of DoT contributes to overall performance, we conduct ablation studies using ChatGPT shown in Table 3. All three stages ob- tain improvements for distortion assessment and distortion classification. For distortion assessment, subjectivity assessment (S1) gains the largest im- provement, as separating objective facts and subjec- tive thoughts a strong trigger to assess distortions. For distortion classification, Schema Analysis (S3)
4.3 Human Evaluation Results
As there is no reference available for the diagno- sis rationales, employ psychotherapy experts to assess the quality of the generated rationales. We hired psychotherapy professionals from Up- Work e.g., certified clinicians, counseling psy- chology Ph.D. students, etc. We discussed with each hire to reach an agreement on the payment following a fair wage rate.
Specifically, we present the patient's speech, all the prompts, and the generated rationales for all stages human experts. For the generated tionales of each stage, we instruct the experts choose between: 1) Comprehensive. (Correct and comprehensive.); 2) Partially good. (Reasonable but not comprehensive) 3) Invalid. (Not reason- able.) Table 4 shows the evaluation results on 100 examples for DoT over ChatGPT and GPT-4. We employ two experts for each evaluation and report the average; The agreement rates (the percentage of examples the two experts gave the same rating) for both evaluations are over 80%. The diagnosis rationales generated for all stages shows decent
www..ppork.comm

---

quality verified by the experts, See Appendix for more generation examples.
matic and human expert evaluation. We believe very important future research direction investigate the general cognitive abilities of LLM and make connections to the field of psychology and cognitive science. Theory-wise, we are eager to explore to what extent the LLM can simulate the human cognitive functions, so as to determine the role that language plays in the overall human cognition. Application-wise, straightforward and encouraging direction should be the integration for assisting mental health treatment, as the motivation and goal of this work.
5 Related Work
Due the verbal and textual nature of psychother- apy procedures, there have been consistent efforts to employ NLP techniques to assist the mental health domain (Althoff et al., 2016; Abd-Alrazaq et al., 2021, 2019; Valizadeh and Parde, 2022). However, with the knowledge gap between the two communities, most existing works take shallow attempts without deep investigation of the profes- sional psychotherapy knowledge. A majority of the previous studies targets identification of com- mon mental health issues, such as depression and anxiety, from textual contents (Cohan et al., 2018; MacAvaney et al., 2018; Harrigian et al., 2020; Ji et al., 2022; Zanwar et al., 2023; Juhng et al., 2023). Another major category of works study therapeu- tic conversations with focus on emotional/empa- thetic supports (Halder et al., 2017; Sharma et al., 2020a; Atapattu et al., 2022; Mishra et al., 2023), and discourse structures (Cao et al., 2019; Hsu et al., 2023). few more recent works have started to investigate deeper professional psychotherapy knowledge, e.g., cognitive distortion detection in CBT (Shreevastava and Foltz, 2021; Ding et al., 2022; Lybarger et al., 2022), identifying and re- framing unhelpful thoughts (Ziems et al., 2022; Maddela al., 2023). Early famous system Eliza took the initial attempts emulate a Rogerian psychotherapist. Due to its rule-based nature, the responses are often reflections or rephrasings of the user's statements. users deviate too much from expected inputs or probe its capabilitics, the program may produce irrelevant or nonsensical re- sponses.
6 Conclusion and Future Work
This work delved into the integration of large language models (LLMs) within the realm of psychotherapy. We introduced the Diagnosis of Thought (DoT) framework, which strategically prompts the LLM to produce diagnosis rationales pertinent to the detection of cognitive distortions. We believe that there exists substantial potential for leveraging L.MMs within numerous facets of mental health support that are currently under-explored. Our findings, thus, not only illuminate a path to- wards more efficient therapy methods but also open doors for future investigations to push the bound- aries of Al's role in mental health treatment.
In the domain of Al-enhanced psychotherapy. particularly when harnessing the power of LLMs, there are paramount challenges surrounding safety, robustness, and ethical responsibilities. LLM is known to generate hallucinations and biases. While one potential usage of our system is to serve as a self-ddaagnosig/aassitttt
tool for patients, e.g., to help the patients to recognize their own thinking patterns and learn how the cognitive distortions developed, such tools should not be deployed for direct patient use without the supervision of pro- fessionals. As we navigate the potential LLM in psychotherapy, our foremost priority should re- main in building systems that are safe and responsi- ble. Concurrently, the formulation of robust ethical guidelines tailored to this new era becomes indis- pensable.
Recent progress in large language model reveals that ChatGPT and GPT-4 present very promising performance in the classic Sally-Anne test (Baron- Cohen al., 1985; Bubeck et al., 2023), evaluating the basic theory of mind capabilities to attribute mental states such as beliefs, emotions, desires, etc. However, latter works also point out the robust- ness issue of such ability (Shapira et al., 2023). In work, we are inspired by such enhanced abil- ity and investigate the application of diagnosing patients' thoughts in psychotherapy. Our experi- ments on the real dataset, not anecdotal examples, show very promising performance both in auto-
Limitations
One of the biggest obstacles to studying Al for psychotherapy is the lack of available, high-quality data. Most of the datasets recording the informa- tion of the interactions between the patients and licensed professionals are confidential due to pri-

---

vacy concerns. The dataset used in this work the only one we can find publicly available for cogni- tive distortion detection, as its data source is from public online forums. In addition to the patient's s speech, the patient's demographic information also plays an important role in analyzing the thought process leading to cognitive distortions. In this dataset, some of the patient speech include few demographic details, but pretty minimal. Beyond the task of cognitive distortion detection, the same data constraint issue exists for all other therapy stages and paradigms. This also motivates us to work on building siivacy-preserve systems as a crucial future direction.
being of individuals seeking help.
For all the annotations and human evaluations presented in this work, we hired psychotherapy pro- fessionals from UpWork e.g., certified clinicians, counseling psychology Ph.D. students, etc. We dis- cussed with each hire to reach an agreement on the payment following a fair wage rate. The average hourly rate for all the experts is $80.
This project is approved by our Institutional Re- view Board (IRB). The data annotation is classified as exempt by our IRB.
References
Alaa A Abd-Alrazaq, Mohamnad Alajlani, Ali Abdal- lah Alalwan, Bridgette M Bewick, Peter Gardner, and Mowafa Housch. 2019. An overview of the features of chatbots in mental health: A scoping re- view. International Journal of Medical Informatics, 132:103978.
Ethics Statement
The system built in this work are never meant to replace psychotherapy professionals: all system- produced results need be verified by licensed professionals. The system should not be deployed for direct patient use without the supervision of licensed professionals.
Alaa A Abd-AIrazaq, Mohannad Alajlani, Nashva Ali, Kerstin Denecke, Bridgette M Bewick, and Mowafa Househ. 2021. Perceptions and opinions patients about mental health chatbots: scoping review. Jour- nal of medical Internet research, 23 13882887
There has been lot of discussions regarding the use of LLMs for AT-driven therapy purpose. Chat- bots will find their most effective role in medicine within the realm of mental health, an opinion? quoted from Thomas Insel, former director of the National Institute of Mental Health. In the mean- time, we should always take caution for such devel- opment. Our foremost priority should be building safe and responsible applications while establish- ing ethical and regulation standards for this new era. Under the current status, the partnership between professionals and AI systems is promising direc- tion for the goal to ease the burden of professionals, which is the motivation of this work.
Tim Althoff, Kevin Clark, and Jure Leskovec. 2016. Large-scale analysis of counseling conversations: An application of natural language processing mental health. Transactions of the Association for Computa- tional Linguisties, 1.463-4766
Thushari Atapattu, Mahen Herath, Charitha Elvitigala, Piyanjali de Zoysa, Kasun Gunawardana, Menasha Thilakaratne, Kasun de Zoysa, and Katrina Falkner. 2022. EmoMent: An emotion annotated mental health corpus from two South Asian countries. In Proceedings of the 29th International Conference on Computational Linguisties, pages 6991-7001, Gyeongju. Republic of Korea. International Com- mittee on Computational Linguistics.
"Diagnosis of Thought" is an innovative ap- proach that harnesses the capabilities of large lan- guage models to enhance the field of psychotherapy. It is imperative to understand that this tool is not designed for censorship or any form of invasive surveillance. Instead, it aims to augment the skills of therapists by providing them with advanced, data-driven insights into their clients' thoughts and emotions. Through the efficient processing of lin- guistic cues and patterns, these models aid mental health professionals in formulating more precise diagnoses and crafting tailored therapeutic inter- ventions that could significantly enhance the well-
Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wen- liang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. 2023. A multi- task, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint 232.:2302.04023
Simon Baron-Cohen. Alan M Leslic, and Uta Frith. 1985. Does the autistic child have a "theory of mind"? Cognition, 21(1):37-46.
Judith S Beck. 2020. Cognitive behavior therapy: Ba- sics and beyond. Guilford Publications.
Sébastien Bubeck, Varun Chandrasekaran, Ronen El- dan, Johannes Gehrke, Eric Horvitz, Ece Kamar,
8 'www.upwor..com

---

Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lund- berg, et al. 2023. Sparks of artificial general intelli- gence: Early experiments with gpt-4. arXiv preprint arXiv:2303.122 12712.
Shaoxiong Ji, Tianlin Zhang, Luna Ansari, Jie Fu, Prayag Tiwari, and Erik Cambria. 2022. Mental- BERT: Publicly available pretrained language models for mental healthcare. In Proceedings of the Thir- leenth Language Resources and Evaluation Confer- ence, pages 7184-7190, Marseille, France. European Language Resources Association.
Jie Cao, Michael Tamana, Zac E Imel, Eric Poitras, David C Atkins, and Vivek Srikumar. 2019. Observ- ing dialogue in therapy: Categorizing and forecasting behavioral codes, arXiv preprint 1rxivv:907.000
Swanie Juhng, Matthew Matero, Vasudha Varadarajan, Johannes Eichstaedt. Adithya V Ganesan, and H. An- drew Schwartz. 2023. Discourse-level representa- tions can improve prediction of degree of anxicty. In Proceedings of the st Annual Meeting of the As- sociation for Computational Linguistics (Volume 2: Short Papers), pages 1500-1511.
Wenhu Chen. 2022. few (1)-shot table reasoners. arXiv preprint arXiv:2210.06710.
Large language models
Arman Cohan, Bart Desmet. Andrew Yates, Luca Sol- daini, Sean MacAvaney, and Nazli Goharian. 2018. SMHD: a large-scale resource for exploring online language usage for multiple mental health condi- tions. In Proceedings of the 27th International Con- ference on Computational Linguistics, pages 1485- 1497, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yu- taka Matsuo, and Yusuke Iwasawa. 2022. Large lan- guage models are zero-shot reasoners. arXiv preprint arXiv:2205. /916.
Kevin Lybarger, Justin Tauscher, Xiruo Ding, Dror Ben- Zeev, and Trevor Cohen. 2022. Identifying distorted thinking in patient-therapist text message exchanges by leveraging dynamic multi-turn context. In Pro- ceedings of the Eighth Workshop on Computational Linguistics and Clinical Psychology, pages 126-136.
Xiruo Ding, Kevin Lybarger, Justin Tauscher, and Trevor Cohen. 2022. Improving classification of infrequent cognitive distortions: Nomain-specifee model vs. data augmentation. In Proceedings of the 2022 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop, pages 68-75, Hybrid: Scattle, Washington + Online. Association for Computational Linguistics.
Sean MacAvaney, Bart Desmet, Arman Cohan, Luca Soldaini, Andrew Yates, Ayah Zirikly, and Nazli Go- harian. 2018. RSDD-time: Temporal annotation of self-reported mental health diagnoses. In Proceed- ings of the Fifth Workshop on Computational Lin- guistics and Clinical Psychology: From Keyboard to Clinic, pages 168-173, New Orleans, LA. Associa- tion for Computational Linguistics.
Kishaloy Halder, Lahari Poddar, and Min-Yen Kan. 2017. Modeling temporal progression of emotional status in mental health forum: A recurrent neural net approach. In Proceedings of the 8th Workshop on Computational Approaches Subjectivity, Sen- riment and Social Media Analysis, pages 127-135, Copenhagen, Denmark. Association for Computa- tional Linguisties.
Mounica Maddela. Megan Ung, Jing Xu, Andrea Madotto, Heather Foran, and Y-Lan Boureau. 2023. Training models to generate, recognize. and reframe unhelpful thoughts. arXiv preprint 2rXiv:3007.02766
Kshitij Mishra, Priyanshu Priya, and Asif Ekbal. 2023. PAL to lend a helping hand: Towards building an emotion adaptive polite and empathetic counseling conversational agent. In Proceedings of the 61 An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12254- 12271, Toronto, Canada. Association for Computa- tional Linguistics.
Keith Harrigian, Carlos Aguirre, and Mark Dredze, 2020. Do models mental health based on social media data generalize? In Findings of the Associa- tion for Computational Linguistics: EMNLP 2020, pages 3774-3788, Online. Association for Computa- tional Linguistics.
Russell Harris. 2006. Embracing your demons: An overview of acceptance and commitment therapy. Psychotherapy in Australia, 12(4):70-6.
Modhurima Moitra, Damian Santomauro, Pamela Y Collins, Theo Vos, Harvey Whiteford, Shekhar Sax- ena, and Alize Ferrari. 2022. The global gap in treat- ment coverage for major depressive disorder in 84 countries from 2000-2019) A systematic review and bayesian meta-regresiion analysis. PLoS Medicine, 1902)1c0039901
Steven C Hayes. Kirk Strosahl, and Kelly G Wilson. 2011. Acceptance and commitment therapy: The process and practice of mindful change. Guilford press.
Shang-Ling Hsu, Raj Sanjay Shah, Prathik Senthil, Zahra Ashktorab, Casey Dugan, Werner Geyer, and Diyi Yang. 2023. Helping the helper: Supporting peer counselors via ai-empowered practice and feed- back. arXiv preprint 2rX55:2305.0899
OpenAl. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.0877
Steve Rathje, Dan-Mircea Mirea, Ili Sucholutsky, Raja Marjich, Claire Robertson, and Jay J Van Bavel. 2023.

---

Gpt an effective tool for multilingual psychological text analysis.
strategies for explainable mental disorder detection. In Findings of the Association for Computational Linguisties: ACI. 2023, pages 8926-8940, Toronto, Canada, Association for Computational Linguistics.
Barbara Olasov Rothbaum, Elizabeth A Meadows, Pa- tricia Resick, and David W Foy. 2000. Cognitive- behavioral therapy.
Caleb Ziems, Minzhi Li, Anthony Zhang. and Diyi Yang. 2022. Inducing positive perspectives with text reframing, arXiv preprint arXiv:220.002952
Tulika Saha, Saichethan Reddy, Anindya Das, Sriparna Saha, and Pushpak Bhattacharyya. 2022. A shoulder cry on: Towards a motivational virtual assistant for assuaging mental agony. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies. pages 2436-2449.
Appendix A: Cognitive Distortion Types
We list all the ten cognitive distortion types stud- ies in this work Ballowing-(Shreeaaann 2021) in Table 5. Note that these ten types be- ing studied are some common, well-studied ones. As psychotherapy develops, more fine-grained and new distortion types are expected to be unveiled. For the cognitive distortion detection dataset pro- posed by Shrecvastava and Foltz (2021), there are 2,531 examples in total. We follow the original 80% 20% train-test split. The input patient speech has an average length of 167.3 tokens, and the ten distortion types are roughly equally distributed.
and Foltz,
Natalie Shapira, Mosh Levy, Seyed Hossein Alavi, Xuhui Zhou, Yejin Choi, Yoav Goldberg, Maarten Sap, and Vered Shwartz. 2023. Clever hans or neural theory of mind? stress testing social rea- soning in large language models. arXiv preprint arXiv:2305. 14763.
Ashish Sharma, Adam Miner, David Atkins, and Tim Althoff. 2020a. A computational approach un- derstanding empathy expressed in text-based mental health support. In Proceedings of the 2020 Confer- ence on Empirical Methods in Natural Language Processing (EMNLP), pages 5263-5276, Online. As- sociation for Computational Linguisties.
Appendix B: Prompt Details
For all our experiments without using DoT, we first prompt the LLM with the following general instructions:
Ashish Sharma, Adam S Miner, David C Atkins, and Tim Althoff. 2020b. A computational approach to un- derstanding empathy expressed in text-based mental health support. arXiv preprint arXiv:20099 08841.
Given a speech of a patient, our task is to 1) iden- tify if there is cognitive distortion in the speech; 2) Recognizing the specific types of the cognitive dis- tortion. Here we consider the following common distortions: (followed by the descriptions and ex- amples for all ten prompts in the dataset metadata of (Shreevastava and Foltz, 2021).
Sagarika Shreevastava and Peter Foltz. 2021. Detecting cognitive distortions from therapit interac- tions. In Proceedings of the Seventh Workshop on Compusaatoonl Linguistics and Clinical Psychology: Improving Access, pages 151-158.
Mina Valizadeh and Natalie Parde. 2022. The ai doctor is in: A survey of taik-oriented dialogue systems for healthcare applications. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers), pages 6638- 6660.
For all our experiments using DoT, we first prompt the LLM with the following general in- structions:
Given speech of a patient, our task is to 1) fin- ish a few diagnose of thought questions to analyze the thought patterns of the patient. Then based on the diagnose of thought analysis, 2) identify if there is cognitive distortion in the speech; 3) Recognizing the specific types of the cognitive dis- tortion. Here we consider the following common distortions: (followed by the descriptions and ex- amples for all ten prompts in the dataset metadata of (Shreevastava and Foltz, 2021).
Anuradha Welivita, Yubo Xie, and Pearl Pu. 2021. A large-scale dataset for empathetic response gener- ation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1251-1264.
Marsha White and Steve M Dorman. 2001. Receiving social support online: implications for health educa- tion. Health education research, 16(6))693-707.
Jesse H Wright, Gregory K Brown, Michael E Thase, and Monica Ramirez Basco. 2017. Learning cognitive-behavior therapy: An illustrated guide. American Psychiatric Pub.
Then for methods without using DoT, we prompt the following two questions:
Please first answer: if there is cognitive distor- tion in the speech; Answer "yes" or 'no'; Please then answer: Recognizing the specific types of the cognitive distortion in the speech. There may be
Sourabh Zanwar, Xiaofei Li, Daniel Wiechmann, Yu Qiao, and Elma Kerz. 2023. What to fuse and how to fuse: Exploring emotion and personality fusion

---

Coguitive Distortion Type
Interpretation
Esample Distorted Speech
Pemoralizzaion
Personalizing or taking up the blame for situatios, that as reality unvlved many fuctiers and was oun of the perso's control.
Myy son is pectty quist today. I wouder what 1did upeat him.
Mind Reading
Suspectng what
dlinking
ausivations behind
they think slob!
when friends
temible pool crowned.
Overgencealiazion
Majer conchucions dhown limited information.
should go
water again.
Allomauthing thinking
possiblc outerme ao situation.
Lanking a sitution cither black or white thinking there only two
cemm get my Ph.
fiilane,
Finetional reasoning
Letiig one's 's feeling about somethug overrule facts to the coutrary.
Eves thoug3 Steve is here work late every day, know i wock harder tua anyone else any job.
Labeling
Giving someone or something lobel without fidling out more about inthem.
My daughter would never de snything I dissopprowed of.
Magnificttion
Empiraskiing the negatiee or playing down posiiive of sination.
know I'll probubly fuil class.
profesour said be made corrcciions
Placing all
only. segutives situation.
hushand wishes loney
housekorpingg
statements appear as
mies about how person should behove, about spesker theacllee other.
Should statoments
NUT necessary
'sould' syuoayme
Ishooldd get A's good studant.
tought to, eustett.) be preseat in the statements contuining this distottion.
As the name siggees, this distoction about expecting things happen cortain way. oc assuming that cling will go budly. Counerimtacvell. this distortin does not always have future lonse.
Forune telling
1 was afraid of job interviwws so docated to start any own thing.
Table 5: Common cognitive distortion types and example speech, taken from (Beck, 2020; Shreevastava and Foltz, 2021).
one type of cognitive distortion or multiple types involved. If there are multiple types, please give the top 2 dominant ones. Please only give the dis- tortion type names separated by comma.
For methods using DoT, before asking this two questions, we provide the following DoT prompts: Based on the patient's s speech, finish the follow- ing diagnosis of thought questions: 1. what is the situation? Find out the facts that are objective; what is the patient thinking or imagining? Find out the thoughts or opinions that are subjective. 2. what makes the patient think the thought is true or is not true? Find out the reasoning processes that support and do not support these thoughts. 3. why does the patient come up with such reasoning process supporting the thought? What's the underlying cognition mode of it?
Appendix C: Generation Examples
We present some generation examples of GPT-3.5 and GPT-4 in Figure 3.

---

Patient Message: Lately have been extremely unhappy for no epparent reeson. feel emply almost all the time. is like nothing can give me genuine joy anymore. also feel dlone a though do have friends and dlose friends, Whenever imagine fature sil see is full obstacles. have become consumed in sadness not know how anymore. confused not understand myself anymore.
Diagnosis of Thought (ChatGPT):
Diagnosis of Thought (GPT4):
Objective
feeling actermely untepoy amppy lnaey.
Objective Fast
The patient has been ecremnty unhoppy recently The patient frels stone move the timw. tw pat frande close
Subjective
Nothing anymonn..
joy future
Thought:
Subjective
They bullove notting bring trem gunuine anymars They therr hetur full ctstectee They teed consummed
Thought:
therrsatves
Ressoning
palent may thoughts exper encing thesw feullngs and amotionn Braitand.
Reasoning
The persitent fooling empiinera and lack miihtt maki the
Proceus
Process
potoent love
siluatin is unchang
Supporting the
Supporting
making
Thoughts:
Thoughts:
harder hopelll about
Foel ig clone even when sumbuned mounddd by heeds could couse them
Reasoning
There be evidence supoort the nothing that future obstaddas pastent may using omotional reasoning support bellefs.
tiin dua thik Il endsh go are not Lulllinga moaninglul
Contradicting
Reasoning
Emmetonn and feelings nee temporary and change oonr time,
Thoughts:
that
will feel this way. The
Contradicting
they
suggests
Cognition
The patentt may be expe perienning cognttiv mode negativity they
Thoughts:
meeninght conmection eostss Dcstacdes matural part life, but
Schema:
Inading them this negative expertencce thoughs of thei life and future, additinalls palient be shruggling with depressioo which can impari the way they persoisn vn themeahues and the world around thom.
they are not the only things prosent onn'n future pos live events and
wct aetemetet ments are also losty noour
thter the postivn sxpectss
Cognttion
Ernertore reascring: ceringg they
leetings untappamees,
Schema:
emptiness, lone
reflections their reality
titvinng out expertencne memories.
Forlnn fulling are exquming ig their future will bu full obetecten bnowd on their cumment amotions and thoughht.
Patient Message: So have na close friends the moment probably know some people enough just to say hi but no close one and whenever make any they go away I don't know if its my parsonality the mason or the way behava and the only friends can gain are school frlends since there no other place Thanks helping.
go can't really stand myself
Diagnosis Thought (ChatGPT):
Diagnosis of Thought (GPT4):
Objective Fach:
The parkent clzen triends momentt and tants The
Objjective Fact:
The palient our cameerlyy does not have closs friend and mairty inteads
with achool trrenss. they feel tonely and tank help.
Subjective
The petaent leats that they are resson for their Frandde leentigg and causing
Subjective
The pataent beknows that their perscesiity beterver right be
Thought:
parsonallty
Thought:
reeaon this ancaitip don't leat, and they are inited opportunties
make now filer
Ressoning
The petaent celaeves their thorugta true because they have not fiiendsi ips and
Reasoning
The patient night have reporfeneed falled filendal ips and notieed a
Process
able maintain
Proceus
pattamm of peocle leaving thei lif the serds them question if their
Supporting
repeating they may using past boleef
support
Supporting the
personality or behavior is the couse.
Thoughts:
Thoughts:
Reasoning
Them may be other rensone for the loss of thene tranniih pe that the is consi doring.
Reasoning
The patient not provided
examples
Process
Procees
personsiity or behoviorr the meseon behend the trdc clone
Contradicting
Contradicting the
Fiondshrip.. itss possible there are external faclors siuutional ciroumatanone that ied to the loss treenioh po.
Thoughts:
Thoughts:
Cognttion
dlstortions such overgeneral zadion assuming that fiiendshipo and up the same way and meets string by thesiing only on the negelivn seppects of the pasi Fiendilipps, ip. the undorlying sogrition made could be negel ve with maga well-adwm..
The porkent esperiencing
Cognition
The patient mightt be enpaging personsliinn iisiiiiiin and
Schema:
ove peegeeeraltzttoo cognitive distotione. persorolizatio becouse they
suent laking blems fiiendar ips lexting,
or
evergerecaization
informston [ta, the exsumption that persoraiity iehhwsor
sole roason for the laok of close Mlonds), this underfying oogn cignition mode could stemn from low sell-esewm Heer of ni repection, pawv mus negatow experfences in social siuations.
Figure 3: Showcases of Diagnosis of Thought using ChatGPT and GPT4 given the Patient Message.


Reviews:
[{"review_id": "eGMvG4721d", "reviewer": "Reviewer_eTZW", "summary": "", "questions": "A: Are you releasing data or code?\n\nB: How exactly did you measure the agreement between annotators? What were the exact guidelines for the experts? What was the procedure? to get an agreement?\n\nC: How can you explain the results?  why GPT4+DoT is lower than GPT4 in Distortion assessment? and why GPT4+DoT is lower than GPT-4 in Distortion assessment but higher in Distortion Classification? What does it mean with respect to the interpretability?\nWhy ChatGPT+DoT is better in DA than DC but GPT4 results are opposite.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the comments and suggestions. We would like to respond to your concerns as follows:\n\n### 1. Regarding “EMNLP stage is not the right place… ” \nWe believe the interdisciplinary nature, as well as the nature of creating/exploring new research directions, should not be the reason to block research work from major conferences and only limit it to workshops. \n\nAs we demonstrate in the introduction, mental health condition worldwide has been growing extremely critical in recent years. Given the severe shortage of therapists and the verbal/textual nature of psychotherapy, we believe NLP for psychotherapy should be one of the most important next research focuses to benefit billions of lives. NLP for psychotherapy hasn’t been a popular research field in the past due to its challenge, and hasn’t established a large community like other interdisciplinary AI fields (e.g., AI for drug synthesis). And now, with the promising ability shown by LLMs, we believe it’s the correct time to raise attention and efforts in this direction to significantly strengthen this new community. This is one of the most important purposes of this work. The impact of workshops is far smaller than major conferences. We believe we should not limit valuable new fields to workshops that slow down their growth. \n\nIt is not accurate to say, “The main innovation is for the psychotherapy domain.” Our proposed DoT framework targets deep analysis of the patient's speech, involving generating entailments/contradictions, commonsense understanding, emotion understanding, and schema summarization, covering a wide range of tasks in the language analysis/generation domain. \nNonetheless, for any interdisciplinary research direction, it is hard and also not beneficial to draw a rigid boundary between the knowledge from two sides. As mentioned above, psychotherapy mostly involves the understanding and generation of speech and written content. It overlaps with language research in nature. We believe new directions and diversities should be embraced in order for NLP to thrive in the new era. \n\n### 2. Regarding “this type of work must be assessed by clinicians too,” \nin section 3, we stated that the DoT framework is established upon consultation and discussion jointly with psychotherapy professionals. Also, please see our Ethics Statement section for details regarding hiring psychotherapy experts. Specifically, we hired psychotherapy experts from UpWork (www.upwork.com, a platform for hiring professionals). The experts we hired are certified clinicians, therapists, and counseling psychology Ph.D. students. They all have expertise in cognitive behavior therapy (the target therapy paradigm of cognitive distortion detection). We discussed with each of them to reach an agreement on the payment and signed the contract. The average hourly rate for all the experts is $80. The involvement of all these experts in this work guarantees the high quality, correctness, as well as reliability of the domain expertise. \n\n### 3. Regarding Bubeck et al. (2023) paper on the Sally-Annie test\nShapira et al. (2023) paper concludes that “LLMs demonstrate enhanced, but not robust theory-of-mind ability.” In our work, we are inspired by such enhanced ability and investigate the application of diagnosing patients’ thoughts in psychotherapy. Our experiments on the real dataset (not anecdotal examples) show very promising performance both in automatic and human expert evaluation. Whether LLM can learn the general theory-of-mind ability and the mechanism of learning are challenging research questions in themselves - we would like to leave them for future work. Shapira et al. (2023) paper was published only one month before the EMNLP deadline, and we had not read it by that time. We will add more discussions regarding this in the revision. \n\n### 4. Regarding “lacking relevant related-work-literature-review… ” \nDue to the space constraint of the short paper, we mostly discuss the related work in the introduction of the current version. We will add a related work section and discuss all the papers you mentioned on the extra page if accepted. \n\n### 5. Regarding “The connection to the field of interpretability”\nWe would like to clarify that this paper is submitted to the tracks “Theme Track: Large Language Models and the Future of NLP” and “NLP Applications,” not the “Interpretability for NLP” track. In our paper, we use the term “Interpretability” to describe the benefits of the generated diagnosis rationales - with such rationale, the diagnosis process and the results can be interpreted and verified by human experts, as well as providing assistance to the patients to do thought analysis, as stated in section 3. \n\n### 6. Regarding Vicuna results\nFor ZCoT and DoT, over 95% of examples go beyond the Vicuna token limit (2048) before generating the DA and DC results. So we remove these two experiments for Vicuna and only present the results for vanilla Vicuna zero-shot performance (directly generating the DA and DC results). \n\n### 7. Regarding “an improvement of results using GPT4 on an existing task… ”\nFor the methodology part of this work, our contribution is the DoT prompting, which can be applied to any advanced LLM with sufficient input length. We use GPT-3.5 and GPT-4 as underlying LLM in our experiments. \n\n### 8. Regarding “ablation studies for GPT4”, “connection between the explanations and the results of the final classification”\nWe demonstrate major improvements in both distortion assessment (DA) and distortion classification (DC) for all three stages of DoT for GPT-3.5. For GPT-4, the improvement is marginal. One possibility is data contamination - the dataset we use is publicly released, so GPT-4 may have already seen it in its training data and achieved a near upper-bound performance no matter with DoT or not. With generating explanations for all three DoT stages, we demonstrate significant improvements for GPT-3.5. Also, the good quality of the explanation itself is very valuable for assisting psychotherapy, as we stated in section 3. \n\n### 9. Answer to Question A\nYes, we will release all our data and code upon acceptance.\n\n### 10. Answer to Question B\nWe employ two experts for each evaluation (the cost of recruiting psychotherapy experts is high). The agreement rate is the percentage of examples the two experts gave the same rating. Regarding the guidelines for the experts, section 4.3 describes our evaluation procedure. For each hired expert, we first present the task instructions, have discussions, and ask them to practice a few examples to make sure they understand the task. Specifically, for each example, we present the patient’s speech, all the prompts, and the generated rationales for all stages. We ask the expert to rate the rationale generated for each stage based on the rating criteria described in section 4.3. We will add more details regarding the guidelines in the revisions. \n\n### 11. Answer to Question C\nAssessing the distortion is easier than the more fine-grained classification, also with less variance. So, the improvement in DA is greater than DC for ChatGPT+DoT. However, for GPT-4, as we explained previously, we did not observe a significant improvement for DA and DC using GPT-4+DoT, due to the possibility of data contamination. In this case, the addition of DoT will possibly introduce more interference into the “pattern” that GPT-4 memorized in data contamination, leading to slight performance decrease and instability. But for DC, since it’s more challenging and variant than DA, DoT still brings slight improvement. \n\n### 12. Regarding Typos and suggestions\nYes, Figure 3 is the content of Appendix C. We will address all your suggestions in the revisions. \n\n### 13. Regarding Ethical concerns\nPlease see our Ethics statement section, as well as our response point #2 for details of hiring psychotherapy experts.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The results of the new models improve the performance of the task over a model trained in the previous literature.\n\nThe paper is written clearly and with good examples.\n\nAlthough it is a small dataset (only 100 samples), human tagging work could possibly be useful in future works.\n\n---\n\nAfter reading the authors' response -\nThank you for answering.\n\nI did miss the paragraph on ethics where it is described that the authors employed psychotherapists to label the data.\n\nIndeed, I also missed that the authors sent the paper to \"Theme Track: Large Language Models and the Future of NLP\" or \"Submission Track 2: NLP Applications\", NLP Application sounds more appropriate.\n\nI still think that more significant work needs to be done in related work (the authors agreed, but they did not provide such a paragraph, nor do I believe that the entire literature can be covered in such a short time)  and that the scientific innovation here is in the application and not in the method.\nI still think that an expert from the field of cognitive-distortion-detection should be involved in building such an application (PhD students in psychology are not considered experts).\nAnd if this is an application intended for clinicians, then clinicians should be part of the authors or\\also clinicians should be involved in the review process.", "reasons_to_reject": "There are flaws that are detailed below. The main concern is that the EMNLP stage is not the right place given the scientific innovation, lacking relevant related-work-literature-review, and the scope of the work.\nAlso, since the authors are building a system for psychotherapy usage, and the main innovation is for the psychotherapy domain, this type of work must be assessed by clinicians too, I highly recommend to re-submit this work to the CLPsych workshop within ACL (that contains reviewers from both domains psychotherapy and NLP and also can tolerate low scientific innovation).\n\nline 46 \"However, existing works mostly take shallow attempts in a heuristic manner, e.g., analyzing emotions and generating comforting responses.\" \n\nThere are several researchers who work with technological startups that combine psychotherapy experts with advanced automated methods among them:\nZac Imel (Lyssn.io), Jonathan D. Huppert (expert in CBT and researching the field of automated tasks), Shiri Sadeh-Sharvit (Eleos Health)\n\nsee also\nCao, J., Tanana, M., Imel, Z. E., Poitras, E., Atkins, D. C., & Srikumar, V. (2019). Observing dialogue in therapy: Categorizing and forecasting behavioral codes. arXiv preprint arXiv:1907.00326.\n\n\nline 68: \"For the psychology domain, ChatGPT and GPT-4 present very promising performance in the classic Sally-Anne test (Baron-Cohen et al., 1985; Bubeck et al., 2023)\"\n\nBubeck et al (2023) paper received severe criticism from the scientific community and in particular on the subject of Sally-Anne test (Baron-Cohen et al., 1985)\nSee:\nShapira, N., Levy, M., Alavi, S. H., Zhou, X., Choi, Y., Goldberg, Y., ... & Shwartz, V. (2023). Clever Hans or neural theory of mind? stress testing social reasoning in large language models.\n\nSee also the section \"Artificial intelligence\" under \"Sally-Anne test\" in Wikipedia, they relate to Bubeck et al (2023) claim and cite more scientific references:\nhttps://en.wikipedia.org/wiki/Sally%E2%80%93Anne_test\n\n\nThe scientific innovation here does not come from the field of interpretability nor from the field of NLP (generally known methods are used to perform classification, there is nothing new here), the innovation is more suitable for the psychotherapy community. I recommend submitting the work to the CLPsych workshop.\n\nThe connection to the field of interpretability is weak. There is no reference to the reliability of the interpretation. there are no references at all for the interpretability-related work.  How is it different/similar to previous works?\n\nThe literature review and related work is weak (both from the psychotherapy research and interpretability or NLP).\n\nline 230:\n\"For Vicuna, both ZCoT and DoT exceed the token limits for many examples; we omit the results.\"\nhow many samples have you omitted? did you score them as wrong answers for the evaluation?\n\nIt is worth mentioning already in the introduction that this is not a new task but an improvement of results using GPT4 on an existing task that has already been published in Kagel earlier.\n\nWhere are the ablation studies for GPT4\n\nIn human evolution, you proved that the explanations make good sense.\nBut you did not prove that there is a connection between the explanations and the results of the final classification.", "questions_to_authors": "A: Are you releasing data or code?\n\nB: How exactly did you measure the agreement between annotators? What were the exact guidelines for the experts? What was the procedure? to get an agreement?\n\nC: How can you explain the results?  why GPT4+DoT is lower than GPT4 in Distortion assessment? and why GPT4+DoT is lower than GPT-4 in Distortion assessment but higher in Distortion Classification? What does it mean with respect to the interpretability?\nWhy ChatGPT+DoT is better in DA than DC but GPT4 results are opposite.", "paper_topic_and_main_contributions": "The authors used an existing labeled dataset from the field of Psychotherapy/CBT/cognitive-distortion-detection and classified the dataset with GPT4, GPT3.5, and Vicuna.\nThe authors suggested a new prompting mechanism with 3 stages, that extract information beyond the classification itself and use it to refine the classification. The authors also perform ablation tests and human evaluations for theIntermediate information extracted.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "Yes", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "Cao, J., Tanana, M., Imel, Z. E., Poitras, E., Atkins, D. C., & Srikumar, V. (2019). Observing dialogue in therapy: Categorizing and forecasting behavioral codes. arXiv preprint arXiv:1907.00326.\n\nShwartz, V., West, P., Bras, R. L., Bhagavatula, C., & Choi, Y. (2020). Unsupervised commonsense question answering with self-talk. \n\nYao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., & Narasimhan, K. (2023). Tree of thoughts: Deliberate problem-solving with large language models. arXiv preprint arXiv:2305.10601.\n\nShapira, N., Levy, M., Alavi, S. H., Zhou, X., Choi, Y., Goldberg, Y., ... & Shwartz, V. (2023). Clever Hans or neural theory of mind? stress testing social reasoning in large language models.", "Typos_Grammar_Style_and_Presentation_Improvements": "Typos and suggestions\n* The font in Table 1 is too small, making it hard to read\n* Table 5 \"o, or\" --> \", or\"\n* Appendix B could be written in a clearer way (using a table, explaining what is the blue lines and what are the rest)\n* Appendix C's content is missing (I guess you meant that it includes Figure 3 (?) write it also in the text.\n* Add the annotator's label for the examples you added in Appendix C.", "Justification_for_Ethical_Concerns": "Ethical concerns:\nThe \"experts\" that were mentioned in the paper\nWere there involved clinicians with expertise in \"cognitive distortion detection\"?\nDid they were paid? How much did they were paid? or were they part of this research (co-authors)?\nHow they were found? who are they?\n\nBecause the paper deals with the field of mental health, it is worthwhile to involve experts from this field as part of the research or at least send the work to clinical reviewers too."}}, {"review_id": "nadfSMpCnQ", "reviewer": "Reviewer_sC35", "summary": "", "questions": "A. line 237: any details about the full-training model?\n\nB. any details about the experts for human evaluation? How many experts assessed the outcome? Who are they? How did you compute the inter-agreement? ", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the comments and support for this work. \n\n### 1. Regarding “lack of technical contributions”\nOur proposed DoT framework comes from the consultation and study jointly with psychotherapy experts. The framework is concise and elegant - all stages in DoT address the key insights and challenges in diagnosing the patients’ thoughts without any redundancies. Each stage brings significant improvement for distortion assessment and classification, and the generated rationales are highly valuable for psychotherapy professionals to interpret, analyze, as well as learn more insights into the patients’ thought patterns. Given the critical condition of mental health issues and the severe shortage of therapists today, we believe it is imperative, or at least highly valuable, to call on efforts for NLP in psychotherapy, which can benefit billions of lives. This is one of the most important purposes of this work. We are happy that the reviewer acknowledges the vision and contribution of this work. \n\n### 2. Answer to Question A:\nThe full-training model is the best-performing one from Shreevastava et al., 2021 paper. They use sentence-BERT combined with the K-Nearest Neighbors method for classification. \n\nSagarika Shreevastava and Peter Foltz. 2021. Detecting Cognitive Distortions from Patient-Therapist Interactions.\n\n### 3. Answer to Question B:\nSee our Ethics Statement section for details regarding the hired experts. Specifically, we hired psychotherapy experts from UpWork (www.upwork.com, a platform for hiring professionals). The experts we hired are certified clinicians, therapists, and counseling psychology Ph.D. students. They all have expertise in cognitive behavior therapy (the target therapy paradigm of cognitive distortion detection). We discussed with each of them to reach an agreement on the payment and signed the contract. The average hourly rate for all the experts is $80. We employ two experts for each evaluation - four experts in total (two for results of GPT-3.5 and two for GPT-4). The agreement rate is the percentage of examples the two experts gave the same rating. We will add more details to the Ethics Statement section in the revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper presents an impressive illustration of how large language models can be applied effectively. The application addresses a compelling and significant issue related to psychotherapy, and the proposed prompting method is thoughtfully grounded in domain expertise.", "reasons_to_reject": "While the lack of technical contributions may be a concern for this conference, personally, I don't view it as a problem for this paper.", "questions_to_authors": "A. line 237: any details about the full-training model?\n\nB. any details about the experts for human evaluation? How many experts assessed the outcome? Who are they? How did you compute the inter-agreement? ", "paper_topic_and_main_contributions": "This paper proposes to use the Diagnosis of Thought (DoT) prompting with large language models (LLMs) to detect cognitive distortion types of patients for psychotherapy. The examples of cognitive distortion include suspecting that others are hiding intentions, taking up the blame for a situation, or looking at a situation in a black-and-white way. Detecting this cognitive distortion types could help both psychotherapists and patients. The proposed DoT method is to prompt an LLM with three questions for the three stages of (1) subjective assessment, (2) contrastive reasoning, and (3) schema analysis. In the subjective assessment stage, we identify objective facts and subjective thoughts in the patient's speech. The contrastive reasoning stage a provides reasoning  process to support or contradict the patient's subjective thoughts. In the schema analysis stage, we want to figure out the cognitive structure of the patient is relying on. The paper shows evaluations on a public cognitive distortion detection dataset, and show that the proposed prompting method is more effective than other methods (directly generating the results and zero-shot CoT prompting). The ablation study was performed to study the effect of including each stage. Then, human evaluation was also performed to assess the quality of generated rationales. These generated rationales provides interpretability/explainability to users when the system is deployed in field. ", "excitement": 4, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "7OjFi7oYFU", "reviewer": "Reviewer_7GRj", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the comments and suggestions. We would like to address your concerns as follows:\n\n### 1. Regarding “...not sufficiently novel contribution”. \nOur proposed DoT framework comes from the consultation and study jointly with psychotherapy experts. The framework is concise and elegant - all stages in DoT address the key insights and challenges in diagnosing the patients’ thoughts without any redundancies. Each stage brings significant improvement for distortion assessment and classification, and the generated rationales are highly valuable for psychotherapy professionals to interpret, analyze, as well as learn more insights into the patients’ thought patterns. Furthermore, DoT, as well as its related work like \"Chain of Thought (CoT)\" (Wei et al., 2022), are essentially studies towards cognitive science and psychology for LLMs rather than prompt engineering. We explore the abilities that LLMs elicit like the way humans think and reason, and study how such cognition abilities emerged (Physics of LLM), or extrapolate such abilities into real-world applications (like this work). Our work is different from and goes beyond CoT in that we design strategies to elicit more complex and fine-grained cognition abilities based on psychotherapy domain knowledge. The advanced cognition abilities we demonstrate can potentially revolutionize the field of AI for psychotherapy. \n\nGiven the critical condition of mental health issues and the severe shortage of therapists today, we believe it is imperative, or at least highly valuable to call on efforts for NLP in psychotherapy, which can benefit billions of lives. We believe the contribution of this work is far beyond a \"case study\" - with the proposed DoT method, we unveil the power of LLM in the deep analysis of patients' thoughts, which can potentially automate and assist all stages of the psychotherapy process. We start from CBT since it is the only therapy paradigm we found publically available dataset - However, we believe the cognition ability of LLMs we elicited, as well as the effectiveness of our method, can easily be extended to a wide range of therapy paradigms. \n\nPsychotherapy mostly involves the understanding and generation of speech and written content since it is conducted primarily based on communications between therapists and patients. It overlaps with language research in nature. Our paper demonstrates the great promise and vision for the new era of the research direction of NLP for psychotherapy. We believe our contribution meets the standard of a short paper in the submitted tracks “Theme Track: Large Language Models and the Future of NLP” and “NLP Applications.” \n\nChain-of-Thought Prompting Elicits Reasoning in Large Language Models, Wei et al., 2022. In NeurIPS. \n\n### 2. Regarding “Diagnosis\" is kind of over-claim”\nIn the proposed DoT framework, we demonstrate that GPT-3.5 and GPT-4 can generate diagnosis rationales with promising performance verified by human experts. Note that our method is intended to serve as assistance for psychotherapy professionals to perform diagnosis, but is never meant to replace the professionals. We enlighten the vision and promise that LLM can potentially generate diagnosis of patients’ thoughts. Regarding the mechanism as well as the robustness of such ability, how to further improve the performance, and how to build real-world systems to assist professionals, we believe they are all valuable future directions starting from this paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The utilization of LLM in this context of mental healthcare could lead to faster and more accurate identification of cognitive distortions, enhancing the effectiveness of therapy sessions.\n2. The DoT method involves a three-stage process for cognitive distortion detection, which proposed a good way of prompt engineering to elicit the capabilities of LLMs. ", "reasons_to_reject": "1. This paper is a case study that utilize LLMs to generate texts in the field of mental health. The main contribution is proposing a prompt based on CBT, which is good but not sufficiently novel to appear in such a competitive conference.\n2. \"Diagnosis\" is kind of over-claim. There is no clear evidence that LLMs can really diagnose. ", "questions_to_authors": "", "paper_topic_and_main_contributions": "The proposed DoT approach addresses the pressing need for more accessible and efficient mental health support. By leveraging Large Language Models, it aims to provide valuable AI assistance to mental health professionals, potentially bridging the gap between supply and demand for psychotherapy services. By leveraging the power of LLMs, the approach demonstrates potential improvements over existing methods and provides valuable rationales for both professionals and patients. \n", "excitement": 2, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper examines the automatic identification of cognitive distortions with “diagnosis of thought” prompting. They use a three stage technique: first assessing the subjectivity of statements, second looking at statements that support or contradict the statements, and lastly an analysis of schemas, which means matching the statements to a cognitive distortion type. They show that using ChatGPT and GPT-4 they can improve distortion detection with this prompting approach. The two of the reviewers are not excited by this work, seemingly because of the novelty of the prompting method. The application area is exciting and does have potentially high impact. Reviewer eTZW makes some helpful suggestions for related work that should be expanded upon. The human evaluation and consultation with experts is valuable in this paper. I disagree with reviewers in that their perceived lack of novelty should not lead to a low soundness score, so I think this paper is appropriate for findings.
Decision: Accept-Findings