Can Large Language Models be Good Emotional Supporter? Mitigating Preference Bias on Emotional Support Conversation (2024)

Abstract: Emotional Support Conversation (ESC) is a task aimed at alleviating individuals’ emotional distress through daily conversation. Given its inherent complexity and non-intuitive nature, ESConv dataset incorporates support strategies to facilitate the generation of appropriate responses. Recently, despite the remarkable conversational ability of large language models (LLMs), previous studies have suggested that they often struggle with providing useful emotional support. Hence, this work initially analyzes the results of LLMs on ESConv, revealing challenges in selecting the correct strategy and a notable preference for a specific strategy. Motivated by these, we explore the impact of the inherent preference in LLMs on providing emotional support, and consequently, we observe that exhibiting high preference for specific strategies hinders effective emotional support, aggravating its robustness in predicting the appropriate strategy. Moreover, we conduct a methodological study to offer insights into the necessary approaches for LLMs to serve as proficient emotional supporters. Our findings emphasize that (1) low preference for specific strategies hinders the progress of emotional support, (2) external assistance helps reduce preference bias, and (3) existing LLMs alone cannot become good emotional supporters. These insights suggest promising avenues for future research to enhance the emotional intelligence of LLMs.

## introduction emotional support conversation (esc) aims to alleviate individuals' emotional intensity and provide guidance for navigating personal challenges through engaging dialogue @xcite @xcite . effective emotional support involves not just providing helpful emotional support but also avoiding poorquality emotional support, which can exacerbate an already stressful situation and may contribute to numerous psychological, relational, and physical problems @xcite . however, providing emotional support is a complex and not intuitive task, often challenging even for humans @xcite . therefore, based on hill's helping skills theory @xcite , @xcite propose a framework for emotional support that generally follows three stages (@xmath0, with a total of eight support strategies corresponding to each stage, where support strategies consist of various conversational methods for the generation of the following response, such as reflection of feelings, self-disclosure. recently, large language models (llms), based on their remarkable conversational ability, have been widely used in various dialogue systems @xcite @xcite . in particular, there is a growing interest in leveraging llms for providing emotional support @xcite , as it takes place in daily conversations rather than in professional counseling @xcite . however, llms that demonstrate outstanding capabilities often struggle with providing emotional support @xcite . as esc task consists of strategy selection and strategy-constrained response generation, selecting the appropriate strategy is crucial for effective emotional support, thereby we anticipate that llms may struggle with predicting strategies. as expected, we find that llms lack proficiency in predicting the accurate strategy 1 . to understand the reasons behind this, we examine the distribution of how often llms select each strategy and observe high preference for certain strategies (i.e., preference bias), as shown in figure 1 . motivated by these, this work is guided by three research questions: rq1: does the preference affect providing emotional support? (section 4.2) initially, we assess the proficiency of various llms, identifying both the strategies and the stages where each model excels and struggles. our findings reveal that they exhibit better performance with strategies that have higher preference and in stages where these high preference strategies are used. since excessive preference for a specific strategy can negatively affect the performance of other strategies, and low performance at a particular stage might hinder the progress of emotional support, we emphasize the importance of low preference bias for robustly predicting strategies across all three stages. rq2: how to mitigate the preference bias on llms? (section 5.2) to understand how to alleviate the preference bias, we apply two groups of methods to llms, based on contact hypothesis @xcite , which posits that contact between different groups can reduce their bias. we find that llms align with contact hypothesis, indicating that reducing preference bias is challenging for llms themselves so that external assistance is necessary. as a result, when mitigating preference bias, llms consistently perform well in predicting strategy across all three stages. this can effectively prevent poor-quality emotional support, which is more crucial than providing appropriate emotional support, given its potential to exacerbate an already 1 the detailed results are shown in appendix a.1 stressful situation. rq3: does improving preference bias indeed help to become a better emotional supporter? (section 5.3) to precisely evaluate whether responses provide helpful emotional support, we build a comprehensive set of criteria formulated in collaboration with psychologists. within these criteria, we analyze whether enhancements in preference bias translate into actual improvements in the quality of emotional support, considering both the advantages of low preference bias and the drawbacks of high preference bias. in human evaluations based on the criteria, lower preference bias is associated with higher scores, while higher preference bias leads to an increased number of poorquality responses. to summarize, our contributions are as follows: ## methodological study: mitigating preference bias according to findings from the previous section, our focus shifts to offering insights into effective approaches for llms to reduce their preference bias. we utilize two models, chatgpt and llama2-70b, each serving as a representative of closedsource and open-source llm respectively. ## discussion and conclusions this work conducts a strategy-centric analysis to delve into why llms struggle with providing emotional support, relying on the importance of strategy in emotional support. our results show that as llms exhibit preference bias towards certain strategies, they lack robustness in predicting strategy across the three stages of emotional support, where struggling in a particular stage may hinder the progress to the next stage. we empirically demonstrate that llms are aligned with the psychological contact hypothesis just like humans, indicating that external assistance can mitigate the preference bias in llms, which they can not do themselves. we highlight that mitigating the preference bias strengthens robustness in selecting appropriate strategies across the stages, leading to overall improvement in the quality of emotional support and a significant reduction in the number of poor-quality responses. we hope that this work will become a promising step for future work to enhance the emotional intelligence of llms.