Related Works
Significant research efforts have been devoted to developing sophisticated conversational models capable of understanding human emotions and generating empathic responses. The detection of sentiment and emotions has been recognized as crucial for the development of empathetic chatbots, as highlighted in Felbo et al. ( 2017 ); Xu et al. ( 2018 ); Shin et al. ( 2019 ); Zhou et al. ( 2020 ) . These works underline the importance of integrating emotional understanding capabilities into automatic dialogue systems to enhance human-computer interaction. In Morris et al. ( 2018 ) , authors demonstrated the feasibility of using corpus-based approaches to enable conversational agents to simulate subtle empathy.
Recent research has focused on developing personalized conversational systems that can maintain coherence and user engagement throughout interactions Madotto et al. ( 2019 ); Cho et al. ( 2022 ) . These systems aim to create more natural and personalized dialogue experiences by adapting their responses to specific user characteristics and conversation contexts. Furthermore, the comprehensive scoping review by Abd-Alrazaq et al. ( 2021 ) sheds light on patient perceptions of mental health chatbots, revealing a positive outlook but emphasizing the need for enhanced linguistic capabilities and personalized interactions.

Recently, there has been a significant rise in the application of NLP techniques within the field of psychology Le Glaz et al. ( 2021 ) . This growing interest stems from the ability of NLP to extract valuable linguistic markers from both spoken and written communication, offering crucial insights into various mental health disorders Agurto et al. ( 2023 ); Corcoran et al. ( 2020 ); Corona Hernández et al. ( 2023 ); He et al. ( 2024 ) .

Research has shown, for example, that measures of language coherence can serve as strong predictors of psychotic symptoms in individuals at high clinical risk Just et al. ( 2020 ) . Clearer language production deficits are typically observed during the first episode of psychosis Gargano et al. ( 2022 ) . One of the core symptoms, language disorganization, can be evaluated by analyzing the coherence and logical consistency of speech. For example, topic models Blei et al. ( 2003 ) have been used to assess psychotic symptoms during patient interviews.
In this context, the use of markers proves valuable for identifying differences within patient groups in an interpretable way. Our method aligns with the trend of leveraging the representational power of large language models to create useful markers for identifying trends within populations. This approach extends the current research in NLP applications for mental health, where language patterns serve as indicators of psychological states. By using emotion embeddings as markers, our method offers a novel way to quantify and analyze the emotional content of language, offering a computational framework for understanding mental health through affective patterns. Similar to how language coherence and organization have been used to predict psychotic symptoms, these emotional markers could potentially serve as early indicators or diagnostic aids for a range of mental disorders.

Building upon these foundational works, this study draws inspiration from CAiRE’s empathetic neural chatbot model by Lin et al. ( 2020 ) , and the innovative approach of using grayscale labels for emotion recognition as suggested by “ The Emotion is Not One-hot Encoding ” by Lee ( 2022 ) .