Detection and Positive Reconstruction of Cognitive Distortion Sentences: M andarin Dataset and Evaluation (2024)

Abstract: This research introduces a Positive Reconstruction Framework based on positive psychology theory. Overcoming negative thoughts can be challenging, our objective is to address and reframe them through a positive reinterpretation. To tackle this challenge, a two-fold approach is necessary: identifying cognitive distortions and suggesting a positively reframed alternative while preserving the original thought’s meaning. Recent studies have investigated the application of Natural Language Processing (NLP) models in English for each stage of this process. In this study, we emphasize the theoretical foundation for the Positive Reconstruction Framework, grounded in broaden-and-build theory. We provide a shared corpus containing 4001 instances for detecting cognitive distortions and 1900 instances for positive reconstruction in Mandarin. Leveraging recent NLP techniques, including transfer learning, fine-tuning pretrained networks, and prompt engineering, we demonstrate the effectiveness of automated tools for both tasks. In summary, our study contributes to multilingual positive reconstruction, highlighting the effectiveness of NLP in cognitive distortion detection and positive reconstruction.

## introduction the rapid pace of modern life and the prevalence of high-pressure lifestyles have led to a rise in mental health issues among individuals. according to a report by the world health organization (who), depression stands out as a significant contributor to both physical and mental disorders globally @xcite . moreover, in response to the escalating concern, the who recently launched a new commission on social connection in november 2023, recognizing the urgency of combating loneliness as a critical health hazard @xcite . these indicators strongly suggest that within an * corresponding author: ni.shiguang@sz.tsinghua.edu.cn environment marked by increased instability and diminished personal agency, there is a heightened risk for individuals to experience negative emotions, closely linked with negative thought patterns. these patterns, identified as cognitive distortions @xcite , represent rigid and inaccurate ways of thinking. such detrimental thoughts reinforce self-negation and perpetuate negative emotions @xcite . these distorted thoughts, often irrational and reinforced over time, become deeply ingrained. habitual adoption of such patterns makes them challenging to recognize. consequently, they are profoundly destructive, when individuals perceive these distortions as truths. cognitive restructuring is the main method to work on cognitive distortions in a clinical setting @xcite . the process can be broadly categorized in two key phases: first, recognizing negative thinking patterns and mitigating their impact, this process requires individuals to write down their negative thoughts and identify any underlying cognitive distortions in those thoughts. then, the cognitive restructuring process is designed to help cultivate a more objective view of the situation. yet, self-examining and rectifying distorted thinking can prove challenging for individuals @xcite . often, intervention and guidance by trained counselors are necessary to identify cognitive distortions and erroneous beliefs. professional counselors can then follow with cognitive restructuring, drawing from their expertise to aid individuals in fostering new perspectives. this approach significantly relies on the counselor's unique skills and individual style. however, due to the high price of psychological counseling (sin-glecare @xcite and the reluctance to speak with a therapist, most struggling people find it challenging to get the support they need. even if they can afford the high cost of a consultation, they may have to wait months for an appointment @xcite . therefore, in recent years, research has begun to turn its attention to natural language processing (nlp) methods to assist individuals in reconstructing negative thoughts. in recent years, the integration of nlp into the field of psychology has been gaining momentum. these nlp models, initially trained on large datasets for general purposes, are now being finetuned to address specific needs in mental health @xcite @xcite @xcite . the emerging success of nlp in mental health is exemplified by applications @xcite @xcite , demonstrating its potential for widespread, accessible psychological support. this development represents a significant advancement, offering a new avenue for cognitive restructuring and mental wellness, seamlessly connecting the high demand for mental health support with innovative, technology-driven solutions. in this rapidly-evolving landscape, the scarcity of language-specific datasets poses a significant challenge. currently, available datasets in the field of mental health nlp predominantly focus on the english language. recognizing the linguistic and cultural intricacies inherent in mental health expressions, it becomes evident that dedicated datasets in other languages is essential for evaluating and fine-tuning nlp models. in this study, we build and openly share the first mandarin dataset foot_0 for both phases of the cognitive restructuring process, encompassing both cognitive distortion detection and positive reframing. additionally, we benchmark different nlp models and training strategies for each task. our main contributions are: ## related works previous studies have proposed various approaches to build nlp pipelines on the detection, categorization of distortions, and positive reframing. we detail here the challenges regarding corpus annotation, in particular labeling consistency, and the need for multilingual research. ## positive reconstruction framework this section presents five positive reconstruction strategies based on the broaden-and-build theory which we describe in this section. utilizing principles derived from this theory can enhance the theoretical underpinnings of both present and forthcoming endeavours on the positive reframing task. these strategies attempt to reframe negative thoughts from a positive perspective, fundamentally reorganizing rigid and harmful cognitive distortion beliefs. the theory posits @xcite that positive emotions broaden an individual's scope of attention, cognition, and action. additionally, positive emotions can undo the effects of sustained negative emotions, known as the undoing hypothesis. they construct positive psychological traits, providing immediate and long-term benefits to individuals. our study will propose positive reconstruction strategies based on the key positive emotions outlined in the theory, as shown in ## dataset construction previous research in cognitive distortion classification has revealed the challenges in achieving robust outcomes, necessitating ample data and highquality annotation. hence, we describe here the collection and annotation of a chinese corpus with binary labels for the presence or absence of cognitive distortion, as well as positive reframing with five different strategies. all collected and annotated data are shared online 1 . ## cognitive distortion detection in our study addressing the detection of cognitive distortions, we trained a binary classification model using a corpus of 4,001 entries annotated by individuals with a background in psychology. the sentences are fed into the neural network for a binary classification task to detect cognitive distortions. the dataset was randomly divided into a training set and a test set in a 4:1 ratio. given the imbalanced nature of our dataset, where only approximately 28% of the data represented distorted cognitions, we utilized focal loss to enhance the model's performance (with hyperparameters α = 0.25 and γ = 2). we trained for 50 epochs with the adamw optimizer and a learning rate equal to 10 -5 , determined using a coarse grid search. we employ roberta-wwm-ext as pre-trained model @xcite . two main strategies have been implemented to explore the performance of this language model for our particular task. first, transfer learning freezes all the weights in the pre-trained roberta-wwm-ext network and augments it with a linear layer, a multilayer perceptron, or an lstm model on the sequential robertawwm-ext output. this last strategy is inspired by previous studies @xcite which have shown the potential of hybrid roberta-lstm for sentiment analysis. the multilayer-perceptron has 2 hidden layers of dimensions 512 and 256, with dropout rate at 0.2, and the lstm hidden dimension is set to 256. the second strategy is fine-tuning, in which we tune the weights of the whole roberta-wwm-ext network with a linear classification readout for our classification task. this is more computationallydemanding and one has to be careful about catastrophic forgetting but it enables the network to learn better embeddings for the task at hand. results are presented in ## positive reconstruction the emergence of large-scale language models has revolutionized various tasks, enabling their completion through prompt engineering @xcite . this led us to wonder whether the performance of prompt engineering parallels or surpasses that of fine-tuning for positive reconstruction generation. ## discussion in conclusion, this study demonstrated the efficacy of nlp techniques in detecting and reconstructing cognitive distortions. we introduced a positive reconstruction framework, grounded in positive psychology theory, and developed a dataset based on its five strategies. this framework was applied to fine-tune large language models, and its performance was compared with that of prompt engineering. our research indicates that fine-tuning these models is more effective than prompt engineering. in future work, a key focus will be conducting extensive user testing experiments to determine whether sentences reconstructed with a positive approach can effectively alleviate negative emotions or stimulate positive ones in users. additionally, exploring the vast potential of prompt engineering to identify the most effective prompts for optimal resource utilization will be a priority. also, further research into how the size of large language models influences their fine-tuning performance in specific domain tasks will be crucial, offering a comprehensive understanding of their application in practical scenarios.