K o ACD : The First K orean Adolescent Dataset for Cognitive Distortion Analysis via Role-Switching Multi- LLM Negotiation (2025)

Abstract: Cognitive distortion refers to negative thinking patterns that can lead to mental health issues like depression and anxiety in adolescents. Previous studies using natural language processing (NLP) have focused mainly on small-scale adult datasets, with limited research on adolescents. This study introduces KoACD, the first large-scale dataset of cognitive distortions in Korean adolescents, containing 108,717 instances. We applied a multi-Large Language Model (LLM) negotiation method to refine distortion classification, enabling iterative feedback and role-switching between models to reduce bias and improve label consistency. In addition, we generated synthetic data using two approaches: cognitive clarification for textual clarity and cognitive balancing for diverse distortion representation. Validation through LLMs and expert evaluations showed that while LLMs classified distortions with explicit markers, they struggled with context-dependent reasoning, where human evaluators demonstrated higher accuracy. KoACD aims to enhance future research on cognitive distortion detection. The dataset and implementation details are publicly accessible.

## introduction negative thoughts @xcite are a natural part of human cognition, often helping individuals recognize potential dangers, prepare for challenges, or engage in self-reflection. however, when these patterns become rigid and excessive, they can lead to emotional distress and contribute to mental health issues such as depression and anxiety disorders. adolescents, in particular, may be more vulnerable to these maladaptive thought patterns due to their ongoing cognitive and emotional development. globally, one in seven children and adolescents (about 166 million) suffers from mental illness, with 42.9% experiencing anxiety and depression @xcite . the rising prevalence of these conditions during adolescence has become a serious global concern. since this stage is crucial for self-identity formation and emotional regulation @xcite , understanding how negative thought patterns emerge and persist is essential for early intervention and prevention. in particular, depression is frequently associated with habitual negative thinking, known as cognitive distortion @xcite . adolescents experiencing these distortions may instinctively blame themselves when something goes wrong, thinking, 'i messed up again' or 'i completely failed.' these persistent negative thoughts trigger emotional distress, reinforcing cycles of depression @xcite . identifying and analyzing these patterns is essential for developing effective coping strategies. to achieve this, building a comprehensive dataset of adolescent cognitive distortions is necessary to enable more targeted and impactful mental health interventions. our key contributions are as follows: ## validating synthetic data with clustering to verify the validity of the synthetic data we created, we performed clustering based on two criteria: (1) topics that trigger negative emotions in adolescents and (2) negative emotions and symptoms outlined in the dsm-5 foot_1 , a widely used framework for assessing and diagnosing mental disorders @xcite . ## evaluation we evaluate the quality of two types of synthetic data: data generated from clearly identified cognitive distortions (section 3.5.1) and data generated to address cognitive distortion imbalances (section 3.5.2). the evaluation was conducted independently using three evaluation criteria, with both llms and human assessments. ## conclusion and future work we developed koacd, a dataset of cognitive distortions in korean adolescents, overcoming the limitations of small-scale datasets focused on english-speaking adults. koacd offers a balanced representation of cognitive distortions through the creation of synthetic data. to our knowledge, it is the first dataset specifically designed for korean adolescents. we introduced a multi-llm negotiation method to improve the objectivity and accuracy of the synthetic data. by using multiple llms to negotiate and refine cognitive distortion labels, we minimized biases and enhanced data quality. expert and llm evaluations confirmed that llms performed well when clear linguistic cues were present, while human evaluators showed higher accuracy in context-dependent situations. discrepancies between llms and human evaluations highlighted the llms' reliance on superficial linguistic patterns. future work will focus on fine-tuning models with adolescent-specific data to enhance contextual understanding of cognitive distortions. additionally, we aim to improve llm performance by developing algorithms that better distinguish cognitive distortions, mitigating biases toward specific types and enhancing both balance and accuracy in detection.