Detecting Cognitive Distortions from Patient-Therapist Interactions (2021)

Abstract: An important part of Cognitive Behavioral Therapy (CBT) is to recognize and restructure certain negative thinking patterns that are also known as cognitive distortions. The aim of this project is to detect these distortions using natural language processing. We compare and contrast different types of linguistic features as well as different classification algorithms and explore the limitations of applying these techniques on a small dataset. We find that pre-trained Sentence-BERT embeddings to train an SVM classifier yields the best results with an F1-score of 0.79. Lastly, we discuss how this work provides insights into the types of linguistic features that are inherent in cognitive distortions.

## introduction cognitive behavioral therapy (cbt) is one of the most common methods of psycho-therapeutic intervention to treat depression or anxiety. due to the covid-19 pandemic, mental health issues are on the rise. at the same time, more and more interactions are now held virtually. furthermore, mental health issues are not limited to the one-hour-perweek window that patients usually get with their therapists. this has led to a growth in the demand for digitally accessible therapy sessions. as mental health care is often inaccessible to people, there is a need for innovative ways to make it more widely available and affordable @xcite . one possible solution is to develop an automated system that could serve by performing some ancillary tasks more efficiently. towards that, natural language processing (nlp) and machine learning (ml) algorithms are now gaining widespread popularity and are being implemented in many fields where language is used. while we are far from a chatbot replacing a therapist's nuanced skillset, having easy access to an intelligent support system can help fill in these gaps. one of the major aspects of cbt is to recognize and restructure certain types of negative thinking patterns. some established negative thinking patterns are commonly observed in patients dealing with anxiety or depression. these cognitive distortions arise due to errors in reasoning @xcite . the aim of educating the patient about these distortions during cbt is to equip the patient with the right tools to detect errors in their own thought processes. once the patient is aware of the error in their reasoning, they can start to work on restructuring how to perceive the same situations in a healthier way. ## methods and dataset one of the most common roadblocks in using artificial intelligence for clinical psychology is the lack of available data. most of the datasets that have patients interacting with licensed professionals are confidential and therefore not publicly available. here, we use a dataset, named therapist q&a, obtained from the crowd-sourced data science repository, kaggle foot_0 . the dataset follows a question and answer format and the identity of each patient is anonymized, to maintain their privacy. each patient entry usually consists of a brief description of their circumstance, symptoms, and their thoughts. each of these concerns is then answered by a licensed therapist addressing their issues followed by a suggestion. since the patient entry is not just a vague request and it provides some insight into the situation as well as their reaction to it, it can be used to detect if they were engaging in any negative thinking patterns. ## feature selection to address the different aspects of language, feature selection was divided into two categories -semantic and syntactic features. two different training approaches were implemented for each of these categories. a brief description of each training method is given below. ## conclusion in this work, we compare and contrast the performance of five classification algorithms in detecting cognitive distortion. we find that the task of determining whether or not an input indicates distorted thinking is computationally feasible, wherein semantic as well as syntactic features perform equally well. the order of words was found to have no impact on the results. entries with cognitive distortions tend to be more subjective than non-distorted entries. the best classification results were obtained by svm using pre-trained s-bert embeddings with an f1 score of 0.79. regarding the task of identifying the type of distortion, we found that semantically similar entries do not always get categorized as the same distortion. some of the distortions are easier to classify than others, e.g. 'should statements', 'mind reading', 'fortune-telling' etc. none of the implemented ml techniques obtained an f1 score higher than 0.30 on the classification of each type of cognitive distortion. a challenging aspect of this research is getting a standardized inter-annotator agreement. one reason for that is a lack of clear distinction in psychology literature itself wherein some of these distortions are sometimes grouped as one. another reason for this could be the presence of multiple distortions in a single patient entry @xcite . as with the clinical application of detection algorithms, there are some ethical risks to keep in mind. if the algorithm is implemented as an unregulated flagging system, the false negatives would go undiagnosed and the false positives would be put through an unnecessary position of secondguessing their cognitive capabilities. however, 100% accuracy of classification from a single interaction (as used for training here) may not be needed for such clinical applications. if this were to be implemented in a dialogue system, an ongoing conversation with the participant will serve to make the system more accurate and personalized. as the main goal is to develop effective feedback to help any participants, having less than perfect predictions is still valuable in informing the types of feedback that an automated clinical tool could provide to the participant. lastly, we discuss several applications of this work in the mental healthcare sector. it could be used to flag or screen people for referrals to mental health care providers. likewise, it could also be used in tandem with the diagnosis to establish an estimate of the severity of the anxiety or de-pression. this approach might also be useful in detecting delusions or paranoia as well as suicide risk in natural language. lastly, the measure of a patient's distorted thinking can be used as an indicator of remission which can be used to determine which therapy techniques (or therapists, from the perspective of insurance companies) are more effective. in conclusion, this tool can be adapted for applications in mental health screening, diagnosis, and tracking treatment effectiveness.