Can Large Language Models be Good Emotional Supporter? Mitigating Preference Bias on Emotional Support Conversation (2024)

Abstract: Emotional Support Conversation (ESC) is a task aimed at alleviating individuals’ emotional distress through daily conversation. Given its inherent complexity and non-intuitive nature, ESConv dataset incorporates support strategies to facilitate the generation of appropriate responses. Recently, despite the remarkable conversational ability of large language models (LLMs), previous studies have suggested that they often struggle with providing useful emotional support. Hence, this work initially analyzes the results of LLMs on ESConv, revealing challenges in selecting the correct strategy and a notable preference for a specific strategy. Motivated by these, we explore the impact of the inherent preference in LLMs on providing emotional support, and consequently, we observe that exhibiting high preference for specific strategies hinders effective emotional support, aggravating its robustness in predicting the appropriate strategy. Moreover, we conduct a methodological study to offer insights into the necessary approaches for LLMs to serve as proficient emotional supporters. Our findings emphasize that (1) low preference for specific strategies hinders the progress of emotional support, (2) external assistance helps reduce preference bias, and (3) existing LLMs alone cannot become good emotional supporters. These insights suggest promising avenues for future research to enhance the emotional intelligence of LLMs.

heading: 1 Introduction
text: Emotional support conversation (ESC) aims to alleviate individuals’ emotional intensity and provide guidance for navigating personal challenges through engaging dialogue (Langford et al., 1997; Greene, 2003; Heaney and Israel, 2008). Effective emotional support involves not just providing helpful emotional support but also avoiding poorquality emotional support, which can exacerbate
∗Equal contribution
an already stressful situation and may contribute to numerous psychological, relational, and physical problems (Burleson, 2003). However, providing emotional support is a complex and not intuitive task, often challenging even for humans (Burleson, 2003). Therefore, based on Hill’s Helping Skills Theory (Hill, 2009), Liu et al. (2021) propose a framework for emotional support that generally follows three stages (Exploration → Comforting → Action), with a total of eight support strategies corresponding to each stage, where support strategies consist of various conversational methods for the generation of the following response, such as reflection of feelings, self-disclosure.
Recently, large language models (LLMs), based on their remarkable conversational ability, have been widely used in various dialogue systems (Ji
15232
et al., 2023; Friedman et al., 2023; Lee et al., 2023). In particular, there is a growing interest in leveraging LLMs for providing emotional support (Chen et al., 2023a; Zheng et al., 2023b), as it takes place in daily conversations rather than in professional counseling (Liu et al., 2021). However, LLMs that demonstrate outstanding capabilities often struggle with providing emotional support (Chen et al., 2023b; Farhat, 2023). As ESC task consists of strategy selection and strategy-constrained response generation, selecting the appropriate strategy is crucial for effective emotional support, thereby we anticipate that LLMs may struggle with predicting strategies. As expected, we find that LLMs lack proficiency in predicting the accurate strategy1. To understand the reasons behind this, we examine the distribution of how often LLMs select each strategy and observe high preference for certain strategies (i.e., preference bias), as shown in Figure 1.
Motivated by these, this work is guided by three research questions:
RQ1: Does the preference affect providing emotional support? (Section 4.2) Initially, we assess the proficiency of various LLMs, identifying both the strategies and the stages where each model excels and struggles. Our findings reveal that they exhibit better performance with strategies that have higher preference and in stages where these high preference strategies are used. Since excessive preference for a specific strategy can negatively affect the performance of other strategies, and low performance at a particular stage might hinder the progress of emotional support, we emphasize the importance of low preference bias for robustly predicting strategies across all three stages.
RQ2: How to mitigate the preference bias on LLMs? (Section 5.2) To understand how to alleviate the preference bias, we apply two groups of methods to LLMs, based on Contact Hypothesis (Allport et al., 1954), which posits that contact between different groups can reduce their bias. We find that LLMs align with Contact Hypothesis, indicating that reducing preference bias is challenging for LLMs themselves so that external assistance is necessary. As a result, when mitigating preference bias, LLMs consistently perform well in predicting strategy across all three stages. This can effectively prevent poor-quality emotional support, which is more crucial than providing appropriate emotional support, given its potential to exacerbate an already
1The detailed results are shown in Appendix A.1
stressful situation. RQ3: Does improving preference bias indeed help to become a better emotional supporter? (Section 5.3) To precisely evaluate whether responses provide helpful emotional support, we build a comprehensive set of criteria formulated in collaboration with psychologists. Within these criteria, we analyze whether enhancements in preference bias translate into actual improvements in the quality of emotional support, considering both the advantages of low preference bias and the drawbacks of high preference bias. In human evaluations based on the criteria, lower preference bias is associated with higher scores, while higher preference bias leads to an increased number of poorquality responses.
To summarize, our contributions are as follows: • We introduce that a wide range of LLMs ex-
hibits different preference for strategies. • We propose a new suite of metrics that fo-
cus on strategies: proficiency, preference, and preference bias. • We emphasize the crucial role of preference bias in robustly providing effective emotional support across the stages. • We showcase that LLMs align with Contact Hypothesis, which indicates that external assistance can help address preference bias. • We construct a comprehensive set of criteria to precisely evaluate whether responses provide helpful emotional support. • Through extensive human evaluation, we demonstrate that mitigating preference bias is crucial for decreasing the proportion of poorquality responses and, consequently, for effective emotional support.

heading: 2 Preliminaries & Related Work

heading: 2.1 Emotional Support Conversation
text: Liu et al. (2021) propose the task of emotional support conversation and release the dataset ESConv, covering a wide range of situations. The ESC centers on the interaction between a user experiencing emotional distress (help-seeker) and a system designed to provide comfort (supporter), aiming to alleviate the user’s emotional intensity. As ESC primarily focuses on providing emotional support, it differs from professional counseling and instead emphasizes support within a social context, such as interactions with friends or family.
The procedure of emotional support in ESConv
generally follows three stages (Exploration → Comforting→ Action). While it does not necessarily follow this sequence of stages, providing emotional support often requires progressing through multiple stages. Therefore, it is crucial to be able to provide appropriate responses in all stages, as poor performance in a particular stage could hinder the progress of the conversation. Further details about ESConv are in Appendix B.

heading: 2.2 Incorporating Strategies into ESC Systems
text: Prior researches on building ESC systems primarily emphasize the integration of support strategies, in conjunction with elements such as emotion, semantics (Zhao et al., 2023b), and persona (Cheng et al., 2023). Some latent studies focus on modeling the user’s state along with the strategies (Cheng et al., 2022; Jia et al., 2023). Notably, Deng et al. (2023) incorporate generative commonsense knowledge model (Hwang et al., 2020) with strategy prediction as an auxiliary task to provide better emotional support. However, many of these approaches involve modifications to the model’s architecture or tuning the pre-trained parameters, a process not typically feasible with LLMs.

heading: 2.3 Emotional Support from LLMs
text: With the emergence of LLMs, there has been an increased amounts of research exploring LLMs as emotional supporters. Recent studies have attempted to replace the fine-tuning approach by prompting LLMs via in-context learning to leverage LLMs as ESC systems (Chen et al., 2023a; Zheng et al., 2023b). Despite their potential, recent studies have demonstrated limitations in LLMs’ ability to provide emotional support (Chung et al., 2023; Farhat, 2023; Eshghie and Eshghie, 2023; Song et al., 2024). Specifically, Song et al. (2024)
find that users may experience discomfort or concern due to the lack of responsibility in LLMs’ recommendations for emotional support response. However, even though the majority of ESC research has focused on leveraging support strategies in their methods, a comprehensive analysis focused on strategy in LLMs has been under-explored.

heading: 3 Evaluation Setup

heading: 3.1 Task and Focus
text: Task: emotional support response generation. The effectiveness of machine-generated responses in providing emotional support is highly dependent on selecting an appropriate strategy. We formulate the emotional support response generation task as generating a response over a support strategy. Formally, given the dialogue background I, a pre-chat survey from the seeker (e.g., emotion, situation), and the dialogue context C, the model θ first predicts the strategy S , and then generates the responseR based on I, C, and S:
S ∼ Pθ(·|I, C) (1) R ∼ Pθ(·|I, C,S) (2)
Focus: strategy-centric analysis. Among the various reasons why LLMs struggle with providing emotional support, this work focuses on strategy, which is the key factor within the ESC systems. To emphasize the validity of strategy-centric analysis, we explore the potential of response quality when generated upon the ground-truth strategy. As a result, in Figure 2, if the model can predict strategies correctly, there is significant room for improvement in the quality of emotional support response.

heading: 3.2 Evaluation Set
text: For comprehensive analysis, we construct three test sets Dt based on stages from ESConv, as demonstrated in Table 1. Firstly, we randomly truncate the dialogues into 5-15 turns samples. We then annotate each sample with a stage and classify the samples according to their stage. Additionally, we minimize the proportion of the strategy Others to reduce responses less relevant to emotional support. Finally, we remove some samples to ensure no overlap of conversations in each test set, and a more detailed explanation of data construction is in Appendix B.2.

heading: 3.3 Metrics
text: Proficiency. We define proficiency as how well the model selects the correct strategy. The proficiency for strategy (qi) is quantified as the F1 score for strategy i. To precisely analyze the model’s proficiency, we utilize two types of F1 scores, both of which stem from the proficiency qi of each strategy: (1) the macro F1 score Q, and (2) the weighted F1 score. The macro F1 score (Q) represents the overall proficiency of the model across the strategies, which is evaluated over the entire test sets (D). In contrast, we employ the weighted F1 score to assess the model on a test set (Dt) consisting only of data corresponding to a specific stage.
Preference. We define preference as how much the model prefers certain strategies over others. To quantify the preference for each strategy in LLMs, we employ the Bradley-Terry model (Bradley and Terry, 1952), which is widely used in human preference modeling (Rafailov et al., 2023). Following Newman (2023), we formally derive the preference p for strategy i as follows:
p′i =
∑ j(wijpj)/(pi + pj)∑
j wji/(pi + pj) (3)
where wij represents the number of times the model predicts strategy i when the ground-truth strategy is j. All of the preference pi are initialized as 1 and updated through iteration of the Eq (3)2, where p′i represents the preference in the next iteration. After the final iteration, we scale the total sum of pi to 8 ( ∑ pi = 8) so that the average p̄ becomes 1, indicating a strong preference for strategy i if pi > 1.
2The details are demonstrated in Appendix C.2.
Preference Bias. We also define a standard deviation of preferences pi across the strategies as preference bias B.
B =
√∑N i=1(pi − p̄)2
N (4)
where a higher value for B indicates that the model exhibits a clear preference for both preferred and non-preferred strategies.

heading: 4 Proficiency and Preference of LLMs on Strategy

heading: 4.1 Models & Implementation Details
text: Based on their availability, we categorize LLMs into the following two groups: (1) Closed-source models which are available via APIs, such as ChatGPT and GPT4 (OpenAI, 2023b); (2) Open-source models accessible through parameters, including LLaMA2-7B/70B (Touvron et al., 2023), Tulu70B (Ivison et al., 2023), Vicuna-13B (Zheng et al., 2023a), Solar-10.7B (Kim et al., 2023) and Mistral7B (Jiang et al., 2023).
In the prompt, we include strategy descriptions to enhance the understanding of each strategy and randomly selected 2-shot examples due to challenges in adhering to the desired output format with open-source models. To facilitate comparison, we also provide 2-shot examples of the closedsource model. More details about models are in Appendix C.3 and about the prompt are in Appendix C.4.

heading: 4.2 RQ1: Does the preference affect providing emotional support?
text: Proficiency of LLMs. Figure 3a illustrates the proficiency Q of each LLM (red line). Not surprisingly, GPT-4 records the highest score in proficiency Q, indicating that it has the overall highest ability to align with strategies, and smaller models tend to achieve lower scores. However, even among models of similar sizes, LLMs exhibit different performances, with smaller models like Solar and LLaMA2-7B showing relatively good proficiency.
The performance varies depending on the test set. Figure 3a also exhibits the performance of LLMs on each test set, with distinct shapes representing different test sets Dt. Most LLMs achieve high scores on D2 or D3, while scoring mostly lower on D1. This indicates that LLMs exhibit
relatively better performance in comforting or action but struggle with exploration stage, suggesting that they may provide poor-quality emotional support in specific situations, especially during the exploration stage. Generally, emotional support progresses through stages from exploration to comforting and action, thereby providing poor-quality response in the exploration stage (D1) may hinder the transition to the next stage, making it difficult to offer effective emotional support. As a result, we can conclude that even though LLMs may achieve a high score in proficiency Q, this does not necessarily guarantee providing helpful emotional support.
Preference bias affects robustness. Figure 3b illustrates that each LLM exhibits different preferences for strategies (pi) and the average preference of strategies belonging to each stage, along with preference bias (B). We observe a strong average preference in stages that exhibit higher performance in Figure 3a. Especially, GPT-4 exhibits low preferences for the exploration stage, which aligns with the lower performance on D1. In contrast, LLaMA2-70B demonstrates relatively uniform preferences for strategies, leading to robust performance across Dt. Through these observations, we can conclude that despite a high proficiency Q, significant preference bias can result in lower performance at specific stages, hindering robustness, which means consistent performance in predicting strategy across all three stages.

heading: 5 Methodological Study: Mitigating Preference Bias
text: According to findings from the previous section, our focus shifts to offering insights into effective approaches for LLMs to reduce their preference bias.
We utilize two models, ChatGPT and LLaMA270B, each serving as a representative of closedsource and open-source LLM respectively.

heading: 5.1 Methods
text: Based on the Contact Hypothesis, which suggests that bias between two groups can be reduced through intergroup contact, we hypothesize that external assistance for LLMs might help alleviate their preference bias. Therefore, we categorize available methods for LLMs into two groups: (1) self-contact and (2) external-contact.
Self-contact approaches. We define self-contact as methods that rely solely on LLMs’ abilities without external interaction. We utilize three selfcontact methods: (1) Direct-Refine, refining the initially generated response by the model itself; (2) Self-Refine, refining the initially generated response through self-feedback; (3) Emotional-CoT, which generates user states as a reasoning path for response generation, following Wei et al. (2022).
External-contact approaches. External-contact involves methods where LLMs not only utilize their internal knowledge but also receive assistance from external knowledge. Similar to KEMI (Deng et al., 2023), one of the state-of-the-art model in ESC task, we leverage commonsense knowledge, COMET. Furthermore, we fine-tune LLaMA2-7B as a strategy planner, a model for planning the next strategy the supporter should take based on the dialogue context. LLMs then respond based on the strategy generated by the strategy planner. Finally, we expand the number of examples (n) in the prompt by selecting them randomly (n = 4). Details about the methods are in Appendix C.5.

heading: 5.2 RQ2: How to mitigate the preference bias on LLMs?
text: Methods with negative effects. Table 2 reports changes in proficiency Q and preference bias B across the various methods. Several methods exhibit negative effects on LLMs’ proficiency and preference bias. Specifically, the results of selfcontact methods present a noticeable pattern in which proficiency declines and preference bias becomes more pronounced. This pattern implies that, similar to humans, when LLMs have bias, thinking alone can deepen those bias, indicating that self-contact methods do not contribute to enhancing their capabilities to become better emotional supporters. Moreover, the degradation of auto-
mated metrics (B-2, R-L) on self-contact stems from lower proficiency and increased preference bias, which leads to poor performance, especially in stages that are less proficient. To further investigate the negative impact of self-contact, we measure the results of Direct-Refine and Self-Refine under an iterative refinement setting to further analyze the preference of each strategy (pi). In Figure 4, we observe a trend where, as the iterations continue, there is a growing preference for strategy that is initially preferred (i.e., pi > 1). In contrast, the preference for strategies that are initially dispreferred (i.e., pi < 1) tends to diminish over successive iterations. As this trend continues, LLMs may struggle more in stages that include strategies with lower preference, and during these stages, they gradually provide poor-quality emotional support.
LLMs align with contact hypothesis. As shown in Table 2, the application of external-contact methods mostly results in a reduction of preference bias on both closed- and open-source LLMs. Particularly, receiving assistance from a fine-tuned strategy planner (w/ Strategy Planner) or having more examples (w/ Example Expansion) seems to be more helpful than relying on commonsense knowledge. These external-contact methods commonly enable LLMs to receive knowledge they cannot generate independently. Utilizing the strategy planner or expanding more examples offers direct knowledge related to strategy, whereas incorporat-
ing commonsense knowledge transfers it indirectly. In summary, external assistance, particularly when directly informing about strategies, plays a crucial role in enhancing both proficiency and preference bias in LLMs. Further analysis on the impact of external-contact is provided in Appendix G.2.
Methodological impacts on providing emotional support. Figure 5 illustrates the results for each test set Dt when applying self-contact (gray background) and external-contact (yellow background) to both ChatGPT and LLaMA2-70B. As observed earlier, applying self-contact, which reduces proficiency and intensifies preference bias, leads to an increased gap between Dt. This substantial gap between Dt indicates a decrease in robustness across various stages of emotional support, and in less proficient stages, they may provide poor-quality responses, which might worsen the seeker’s situation and intensify distress. In particular, all self-contact approaches significantly reduce performance on the exploration stage (D1), which can create challenges in progressing to subsequent stages, ultimately hindering the achievement of the goals in emotional support. On the other hand, externalcontact reduces the overall gap between different Dt, particularly exhibiting significant improvement on ChatGPT. This reduction contributes to robust performance in selecting strategy across the stages, which is crucial for effective emotional support.
Effect of examples in the prompt. To assess the efficacy of examples in the prompt, we initially investigate a trend associated with the number of examples (n). Figure 6a demonstrates that proficiency and preference bias improve when using randomly selected examples. However, while proficiency Q converges as n increases, preference bias B worsens significantly with larger values of n (n > 8), indicating that too many examples may be detrimental. Additionally, to understand the impact of different types of strategies employed in the examples, we include the various combinations of strategies within 2-shot examples. Intriguingly, Figure 6b reveals consistent results across the diverse combinations. In summary, providing the appropriate number of examples may enhance preference bias, whereas the type of strategies within each example does not matter. Further analysis of each preference pi based on n is in Appendix F.3.
Various models as a strategy planner. In our previous experiments, a trained LLaMA2-7B serves as a strategy planner, yielding improved outcomes. To explore the potential of various models as a strategy planner, we ablate with several language models, including Mistral and encoder-based models such as BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019). As shown in Table 3, we find that using LLMs as the backbone model for the strategy planner leads to notable enhancements in proficiency and preference bias. Moreover, while encoder-based models achieve performance comparable to LLMs, they exhibit relatively higher preference bias, indicating weaker robustness and potentially providing poor-quality emotional support. We also leave the exploration of training a strategy planner with more diverse and systematic methods for future work. Additionally, a more ablation study on directly fine-tuning LLMs as emotional supporters is provided in Appendix F.4.

heading: 5.3 RQ3: Does improving preference bias help to become a better emotional supporter?
text: Criteria of human evaluation. To precisely assess whether responses provide helpful emotional support, we build a comprehensive set of criteria formulated in collaboration with psychologists in terms of emotional support, based on the perspective of seeker’s satisfaction (Sat.). As emotional support aims to appropriately assess the user’s state and reduce emotional intensity, we fine-grain this perspective and finally construct three smaller criteria to enable a more elaborate assessment: (1) Acceptance: Does the seeker accept without discomfort; (2) Effectiveness: Is it helpful in shifting negative emotions or attitudes towards a positive direction; (3) Sensitivity: Does it take into consideration the general state of the seeker. Furthermore, to clarify the capability of LLMs to align strategy and responses, we include Alignment.
We randomly sample 100 dialogues from three test sets (Dt), ensuring diversity (e.g., strategy), and three annotators are required to determine the Win/Tie/Lose for each comparison in Table 4. Additionally, we ask three annotators to evaluate each sample on a 1-5 Likert scale, providing specific rubrics for each score to ensure detailed assessments on the quality of responses (Table 5). We include more details on the human evaluation, including the results of Alignment, in Appendix E.
Benefits of mitigating preference bias. Table 4 presents a comparative human evaluation between the results of various methods on ChatGPT and the results of vanilla ChatGPT. Consistent with our previous findings, external-contact outperforms self-contact (i.e., Self-Refine) in terms of overall
seeker’s satisfaction (Sat.). Concretely, when comparing the w/ COMET with Self-Refine, which have similar proficiency but significant differences in preference bias, the overall seeker’s satisfaction score is higher for w/ COMET with lower preference bias. Furthermore, among the external-contact methods, responses generated through the strategy planner, which exhibits the most significant improvements in preference bias, are the most helpful in reducing the seeker’s emotional intensity. Consequently, we can confirm that it is crucial to mitigate preference bias to enhance robustness in predicting strategy, thereby providing effective emotional support.
Drawbacks of aggravating preference bias. To understand the negative impact of severe preference bias, we investigate the proportion of responses that could worsen the seeker’s situation or distress (i.e., rated below 3). Table 5 demonstrates that the proportion of poor-quality emotional support significantly increases in self-contact (i.e., DirectRefine, Self-Refine), which exacerbates preference bias. This confirms that the aggravation in preference bias sharpens the contrast between proficient and less proficient stages, leading to providing more poor-quality responses in the less proficient stages. Additionally, the decrease in the proportion of poor-quality responses in external-contact (i.e., w/ Strategy Planner), where preference bias diminishes, supports this conclusion. As a result, high preference bias disturbs robustness, leading to an increased number of poor-quality responses. This demonstrates that low preference bias reduces the number of poor-quality responses and, consequently, is crucial for effective emotional support.

heading: 6 Discussion and Conclusions
text: This work conducts a strategy-centric analysis to delve into why LLMs struggle with providing emotional support, relying on the importance of strategy in emotional support. Our results show that as LLMs exhibit preference bias towards certain
strategies, they lack robustness in predicting strategy across the three stages of emotional support, where struggling in a particular stage may hinder the progress to the next stage. We empirically demonstrate that LLMs are aligned with the psychological Contact Hypothesis just like humans, indicating that external assistance can mitigate the preference bias in LLMs, which they can not do themselves. We highlight that mitigating the preference bias strengthens robustness in selecting appropriate strategies across the stages, leading to overall improvement in the quality of emotional support and a significant reduction in the number of poor-quality responses. We hope that this work will become a promising step for future work to enhance the emotional intelligence of LLMs.

heading: Limitations
text: This work has the following limitations: (1) As aforementioned in Section 3.2, Cheng et al. (2022) demonstrate that the strategy Others are not helpful in enhancing the response generation and may not be fully fine-grained. This can potentially prevent obtaining sufficient insights by obscuring more detailed preferences of the model; (2) We include 2-shot examples for open-source LLMs as they often struggle to adhere to the desired output format (e.g., wrong strategy that is not among the eight provided). Since we demonstrated improvement when prompting with n-shot examples in Section 5.2, the actual proficiency and preference bias of opensource LLMs may be worse than the scores we published; (3) Understanding the reasons for preference bias is challenging not only for closed-source LLMs but also for open-source LLMs, as it is difficult to precisely grasp the relationships between strategy, training data, methods and model architecture; (4) We have observed that even when using an oracle strategy in LLMs (Table 8), responses that increase emotional intensity still exist (3.8%). This indicates a lack of ability to generate appropriate responses for emotional support, even when the strategy is perfectly selected. Therefore, future work might consider both correctly predicting the strategy and generating helpful responses based on the predicted strategy; (5) While we confirm that LLMs generally generate well-aligned responses with the strategy (Figure 16), it is evident that there are some cases where they are not aligned, thereby future work should recognize this misalignment.

heading: Ethical Considerations
text: The ESConv a dataset used in this work is a publicly available and well-constructed benchmark for emotional support conversation, which is collected by employed crowd-sourced workers, with the sensitive and private information filtered during the dataset construction. All participants in our human evaluation are volunteered, transparently informed of our research intent, and paid reasonable wages.
It is worth mentioning that the term "emotional support" in this paper mainly refers to support within a social context, such as interactions with friends or family in daily conversation, rather than professional counseling or diagnosis. Moreover, as LLMs can generate sensual, harmful, biased, offensive, or violent content, using them as emotional support systems requires particular caution to avoid such content from appearing to users. And it also requires considerable further efforts to construct a safer system, which is capable of detecting users who have tendencies of self-harming or suicide.

heading: Acknowledgements
text: This work was supported by Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korean government (MSIT)(No.RS-2020-II201361, Artificial Intelligence Graduate School Program (Yonsei University)) and (No.RS-2021-II212068, Artificial Intelligence Innovation Hub) and (No.RS2022-II220077,AI Technology Development for Commonsense Extraction, Reasoning, and Inference from Heterogeneous Data). Jinyoung Yeo is a corresponding author.

heading: A Details of Preliminary Studies
text: For the preliminary study, we prompt gpt-4-0613 and gpt-3.5-turbo-1106 to predict a strategy and generate a strategy-constrained response in 0-shot setting, and LLaMA2-7B in 2-shot setting as it struggles with adhering to desired output format. We utilize a total of 4,833 samples across various strategies, and the strategy distribution of samples is reported in Table 6 (Ground-Truth). We provide the prompt used for the test in Table 12.

heading: A.1 Analysis of LLMs on ESC
text: Performance in Selecting Correct Strategy. Table 7 indicates that LLMs have limited proficiency in accurately predicting strategy, showing performance similar to random selection.
Preference for Strategy. To further analyze the reason behind the low performance, we investigate the distribution of how often LLMs select each strategy. Table 6 includes the proportions of strategy selected by LLMs and their preferences (pi) for each strategy. We have observed that all LLMs have a strong preference for the strategy Affirmation and Reassurance and each LLM has its preferred strategies with various degrees of preference.
A.2 Importance of Strategy To comprehend the importance of strategy in emotional support conversation tasks using LLMs, we
examine gpt-3.5-turbo-1106 and LLaMA2-70B under the following settings: response generation (a) without strategy, (b) with randomly selected strategy, (c) with strategy predicted by itself, and (d) with ground-truth strategy.
Figure 2 and Table 8 represent that the responses based on correct strategy (ground-truth strategy) outperforms those generated without strategy. Furthermore, although LLMs exhibit low performance in strategy prediction, the responses conditioned on predicted strategy achieve performance similar to those without strategy, emphasizing that there is significant room for improvement in the quality of emotional support responses with LLMs.

heading: Models Strategy Q R-L. Sat.

heading: B ESConv Dataset

heading: B.1 Definitions of Stages
text: Grounded on Hill’s Helping Skills Theory (Hill, 2009), Liu et al. (2021) propose three stages of emotional support:
1. Exploration: Explore to identify the seeker’s problem.
2. Comforting: Comfort the seeker through expressing empathy and understanding.
3. Action: Help the seeker solve the problems.
Although it is suggested that ESC target these stages in the order: (1) Exploration → (2) Comforting→ (3) Action, this sequence can be flexibly tailored to individual needs, as conversations, in practice, do not always follow a fixed order.

heading: B.2 Definitions of Strategies
text: Liu et al. (2021) also propose a specific set of conversational skills corresponding to each stage. In ESConv, they annotate eight types of support strategies:
• Question: Asking for information related to the problem to help the seeker articulate the issues that they face. • Restatement or Paraphrasing: A simple, more concise rephrasing of the seeker’s statements that could help them see their situation more clearly. • Reflection of Feelings: Articulate and describe the seeker’s feelings to show an understanding of the situation and empathy. • Self-disclosure: Divulge similar experiences that you have had or emotions that you share with the help-seeker to express your empathy. • Affirmation and Reassurance: Affirm the seeker’s ideas, motivation, strengths, and capabilities to provide reassurance and encouragement. • Providing Suggestions: Provide suggestions about how to get over the tough and change the current situation, but be careful to not overstep and tell them what to do. • Information: Provide useful information to the help-seeker, for example with data, facts, opinions, and resources. • Others: Use other support strategies that do not fall into the above categories.

heading: C Experiments Details

heading: C.1 Evaluation Sets
text: In this study, we systematically partition the ESConv dataset into three distinct test sets, denoted as D1 (Exploration), D2 (Comforting), and D3 (Action), to facilitate stage-specific assessments. To prevent utterance duplication, we split the 1,300 dialogues within the ESConv dataset into three sets
and randomly allocate them to Dt. We slice each dialogue comprising 5 to 15 turns to generate instances. The determination of the stage for the label response of each instance is based on the majority stage indicated by surrounding strategies within a window size of 4. In cases where the randomly assigned stage of Dt differs from the determined stage, the instance is excluded from the respective test set. Furthermore, to maintain the relevance of the test sets to emotional support contexts, we restrict the slicing process, ensuring that the frequency of the Others strategy does not exceed 5%. Detailed statistics of the test sets are provided in Table 1 and Table 9.

heading: C.2 Preference Metric
text: Bradley-Terry Model. The Bradley-Terry model (BT model) serves as a probability model for pairwise comparisons between individuals or objects. Its utility spans a broad spectrum of areas, notably in ranking competitors in sports, chess, and other competitions. Beyond these traditional domains, the BT model extends to the realm of machine learning, facilitating multi-class probability estimations by incorporating pairwise classification results. Recently, Rafailov et al. (2023) employed the BT model for optimizing preference alignment of LLMs, known as direct preference optimization.
Preference Evaluation with the Bradley-Terry Model. In this study, we employ BT modeling to assess the preference of LLMs across the strategies. The probability P (i > j), representing the preference for strategy i over ground-truth strategy j, is formally defined as:
P (i > j) = pi
pi + pj (5)
where we assign a numerical score si to each strategy i and define pi = esi , enabling the expression of P (i > j) in terms of these scores. Zermelo (1929) characterizes the parameter pi as playing strengths. In scenarios involving a series of pairwise competitions among N competitors (specifically, 8 strategies in our case), estimating these strengths becomes relatively straightforward.
The likelihood of the preference (P) with the Bradley-Terry model is given by the equation:
P = ∏
ij
[P (i > j)]wij = ∏
ij
( pi pi + pj )wij
where wij represents the total number of times where strategy i is preferred over strategy j. This
leads to the log-likelihood:
logP = ∑
ij
wij log pi
pi + pj
= ∑
ij
wij log pi − ∑
ij
wij log (pi + pj)
Zermelo (1929) showed that this expression has only a single maximum, differentiating with respect to pi for any i and setting the result to zero:
1
pi
∑
j
wij − ∑
j
wij + wji pi + pj = 0 (6)
Iterative Algorithms. Following the efficient algorithm proposed by Newman (2023), Eq 6 can be rearranged as:
1
pi
∑
j
wij pj pi + pj − ∑
j
wji pi + pj = 0 (7)
pi =
∑ j(wijpj)/(pi + pj)∑
j wji/(pi + pj) (8)
Finally, Eq 8 results in the iterative algorithm for the Bradley-Terry model to calculate the preference pi for each strategy i.
For this iterative algorithm, we initially set all values (pi) to 1 and iteratively update these estimates over k iterations, where in this study we utilize 20 iterations for estimation. Subsequent to each iteration, it is necessary to normalize the values by dividing them by their geometric mean to ensure stability and convergence of the algorithm. This normalization step is represented as:
pi ← p′i(
Πj=1p′j )1/N (9)
where N is the total number of strategies. After the final iteration, the converged p values indicate the final preference pi for strategy i.

heading: C.3 Models
text: ChatGPT / GPT-4. ChatGPT and GPT-4 (OpenAI, 2023a,b) are among the most widely used LLMs, demonstrating state-of-the-art performance in numerous applications. However, as they are closed-source LLMs, they are available exclusively through APIs. Thereby, we employ gpt-3.5-turbo1106 for ChatGPT and gpt-4-0613 for GPT-4 in this work.
LLaMA2. LLaMA2 (Touvron et al., 2023) is a prestigious open-source LLM that is widely employed as a foundation model for various opensource LLMs. The model size ranges from 7B to 70B parameters. In this work, we implement both the 7B (Llama-2-7b-hf ) and the 70B (Llama2-70B-hf ) versions, allowing for an exploration of the effects of model size on performance.
Tulu. Tulu is a model with 70B parameters, based on LLaMA2 models fine-tuned on V2 mixture (Ivison et al., 2023). The employ the tulu-270b version in our experiments to assess its capabilities within the context of our study.
Vicuna. Vicuna is a 13B language model from LLaMA-13B model fine-tuned with high-quality conversation datas (Zheng et al., 2023a). We incorporate the vicuna-13b-v1.5 version into our experiments to evaluate its performance.
Solar. Solar is an LLM with 10.7B parameters, employing the depth up-scaling (DUS) method as its scaling method (Kim et al., 2023). This approach contributes to its performance exceeding other LLMs, including those utilizing mixture-ofexperts (MoE) methods. We use the SOLAR-10.7BInstruct-v1.0 version in this work.
Mistral. Mistral is a 7B LLM that leverages grouped-query attention (GQA) and sliding window attention (SWA) for faster inference and reduced inference cost (Jiang et al., 2023). It claims superior performance over the LLaMA213B model and even the LLaMA-34B model across various evaluation benchmarks. We employ the Mistral-7b-Instruct-v0.2 version.

heading: C.4 Prompts Details
text: The prompts employed in our experiments are shown in Table 12. To ensure a clear understanding of the task, Task description and strategy description are prompted to LLMs. Furthermore, in
addition to the dialogue context, we also incorporate dialogue background, which encompasses the seeker’s problem, emotion, and situation gathered from a pre-chat survey. Depending on the method employed, various types of information, such as feedback, rationale, commonsense knowledge, and few-shot examples, are also included as supplementary inputs.
Random few-shot samples. To prevent potential biases in strategy induced by few-shot learning, we randomly select examples. During the experiments, for each data instance, we randomly select exemplars with non-overlapping strategies and incorporate them into the prompt. This approach ensures that the influence of few-shot samples on strategy prediction is minimized by diversifying the strategies presented to the model. However, we figure out in Section 5.2 and Figure 6b that the types of strategies included in the prompt as examples do not significantly impact on the results in the end.

heading: C.5 Methods Details
text: Direct Refine. Direct refine is a straightforward refinement method, wherein we instruct the model to revise its initial response to incorporate emotional support elements.
Self-Refine. Self-refine, a method introduced by Madaan et al. (2023), initiates by generating feedback emphasizing emotional support from the initial response. Subsequently, it refines the response based on this feedback.
Emotional-CoT. Building upon the success of Chain-of-Thought (CoT) prompting (Wei et al., 2022), we employ CoT to first generate the user state, which then guides the generation of strategy and response.
w/ COMET. To incorporate external commonsense knowledge for providing emotional support, we integrate the COMET model (Hwang et al., 2020), specifically COMET-BART3, while leveraging five relation types (i.e., xReact, xIntent, xNeed, xEffect, and xWant). Following Chae et al. (2023), we implemented a retriever using ComFact (Gao et al., 2022b) to align the dialogues with the knowledge from COMET. Among the inferences generated by COMET, we apply the retriever (DeBERTalarge4) and filter inferences that are non-relevant
3https://github.com/allenai/comet-atomic-2020 4https://github.com/silin159/comfact
to the dialogue context. Subsequently, we convert the remaining inferences into natural language and augment to LLMs, which is shown in Table 12.
w/ Strategy Planner. Strategy planner is a classification model that is fine-tuned to predict the strategy based on dialogue background and context. Thereby, we formulate w/ Strategy Planner as follows: given the dialogue background I, and dialogue context C, the strategy planner model θ′ predicts the strategy Ŝ . Then, LLM θ generates the responseR, leveraging I, C, and Ŝ.
Ŝ ∼ Pθ′(·|I, C) (10) R ∼ Pθ(·|I, C, Ŝ) (11)
D Implementation Details
All experiments are conducted on 8 NVIDIA GeForce RTX 3090 GPUs and 2 NVIDIA A100 80GB PCIe GPUs.
Fine-tuning. Since the test sets are constructed by dividing the dialogues in ESConv into three without overlap, to evaluate each test set with a trained model, we construct a train/valid set from dialogues corresponding to the other two sets and train the model on it.
For training, we employ QLoRA (Dettmers et al., 2023) to effectively fine-tune a model, incorporating 4-bit quantization and specifying the dimension of low-rank metrices as 64 and alpha as 16. The DeepSpeed library5 is utilized to facilitate the training, with a learning rate of 5e-5 over 5 epochs, resulting in approximately 8 hours of training. For encoder-based models like BERT and RoBERTa, we train them to classify among 8 categories (corresponding to the number of strategies), with training extending up to a maximum of 20 epochs.
Inference. For generating responses, we follow the default settings provided by OpenAI for topp sampling and temperature, with p = 1.0 and T = 0.7. To achieve higher throughput during inference, we leverage the vLLM library6.
Terms and License. For our implementation and evaluation, we use Huggingface library7 and vLLM library. Both libraries are licensed under Apache License, Version 2.0. We have confirmed that all of the artifacts used in this paper are available for non-commercial scientific use.
5https://www.deepspeed.ai 6https://docs.vllm.ai 7https://huggingface.co/

heading: E Details on Human Evaluation

heading: E.1 Human Evaluation Criteria
text: With automatic metrics, it is challenging to precisely assess the emotional support quality of responses (Mehri and Eskenazi, 2020; Gao et al., 2022a). Furthermore, conventional criteria commonly used for general dialogue are not specifically designed to evaluate whether a response provides emotional support. Hence, in collaboration with four psychologists, we develop a specific set of criteria focused on assessing whether a response provide effective emotional support from various perspectives of the seeker.
Seeker’s Satisfaction (Sat.), focusing on the quality of emotional support, comprises three detailed criteria. Moreover, we add Alignment to assess how well the generated response aligns with the predicted strategy. Consequently, we focus on these four criteria:
• Acceptance: Is the response accepted by the seeker without discomfort or resistance?
• Effectiveness: Is it expected that the response would mitigate or shift the seeker’s negative emotional state or attitude toward a more positive direction?
• Sensitivity: Does the response take into consideration the seeker’s state (mood, needs, resources, culture, attitude, etc.)?
• Alignment: Is the response fitting for the chosen strategy?
E.2 Implementations of Human Evaluation
We employ human evaluation, outsourcing the task to assess response quality on Amazon Mechanical Turk (AMT). Figure 9 shows the interface employed for comparative evaluations (Win/Lose/Tie) between two responses. Figure 10 and 11 depict the interface employed to rate our four criteria using 5-point Likert scale. Detailed instructions and rubrics for each score are included to ensure precise evaluation. For each evaluation, we ask three human annotator to assess 100 samples each based on four specified criteria. We compensate each data piece in the human evaluation with a payment of $0.07.

heading: F Additional Analysis

heading: F.1 LLMs’ Proficiency for Each Strategy
text: Building upon the findings where LLMs generally tend to demonstrate a low proficiency, as shown in Figure 3a, we further delve into the proficiency of each strategy on LLMs. As illustrated in Figure 8, there are notable differences in proficiency depending on the strategy. In particular, each LLM tends to exhibit higher proficiency in strategies with higher preference, observed in Figure 3b.

heading: Models Params Pearson Correlation

heading: F.2 Relation between Proficiency and Preference
text: In Figure 3, we observe that LLMs achieve higher scores on test sets aligned with strategies that they prefer more, raising the question of how this preference influences the proficiency. To explore the relationship between preference pi and proficiency qi, we calculate the Pearson correlation between pi and qi for each strategy. As a result, Table 10 reports a strong positive correlation between preference and proficiency for most LLMs, suggesting that a high preference pi for strategy i leads to a high proficiency qi. Ultimately, this confirms that LLMs perform better in stages containing preferred strategies.

heading: F.3 Preference for Strategies by the Number of Examples.
text: In Figure 6a, we observed improvements in proficiency and preference bias when prompting ChatGPT with few examples. However, we also found that as the number of examples increases, preference bias significantly worsens. To delve deeper into the reasons behind this, we examine the changes in preference for each strategy as the number of examples increases. As demonstrated in Figure 7, the preference for Affirmation and Reassurance gradually diminishes, while the preference
for Others gradually increases. The strong preference for the Others, as the number of examples increases, eventually exacerbates preference bias. Consequently, the strong preference for the Others disrupts the selection of alternative strategies, hindering the enhancement of proficiency as the number of shot examples increases.

heading: F.4 Supervised Fine-tuning on ESC Task
text: To explore the possibility of fine-tuning the model itself as an emotional supporter in addition to finetuning the strategy planner, we train the LLaMA27B to generate emotional support responses. Table 11 shows that fine-tuning the model leads to significant improvements in emotional support quality.
We also ablate to examine the effectiveness of strategy on fine-tuned models. As a result, Table 11 demonstrates that fine-tuning the model on a dataset with strategies yields a higher quality of emotional support compared to training on a dataset that does not include strategies.

heading: G Case Study

heading: G.1 Responses of LLMs by Stages
text: In Figure 12, 13, and 14, we present examples of generated responses in each stage from LLMs. During the Exploration stage (Figure 12), it is observed that LLMs, excluding the LLaMA2 family, tend to express empathy prematurely before sufficient exploration, potentially causing discomfort for the seeker. These findings correlate with the LLaMA2 family’s high preference for Question, exhibiting a lower preference compared to other models , as illustrated in Figure 3b. Furthermore, these results correspond to earlier findings discussed in Appendix F.1. In the Comforting stage (Figure 13),

heading: Methods Q ↑ B ↓ B-2 R-L
text: each model demonstrates suitable responses, primarily due to the high preference for Affirmation and Reassurance in most LLMs. Lastly, in the Action stage (Figure 14), GPT-4 and ChatGPT exhibit the superior performance compared to others, particularly excelling in generating informative responses, aligning with the observations in Zhao et al. (2023a) and Chen et al. (2023b). Psychologists who assess the overall responses of LLMs also comment as follows:
“ChatGPT exhibits a tendency to excessively employ affirmations. In contrast, LLaMA2, despite its overall lower proficiency, displays notable strength in effectively handling open-ended questions.”
These results are aligned with the findings we identify through our case study.

heading: G.2 Comparison between Self-Contact and External-Contact
text: While self-contact methods negatively impact on performance, external-contact methods exhibit a noticeable enhancement. A detailed case study presented in Figure 15 supports this findings, where the response of self-contact methods fall short of meeting the seeker’s expectations, while the external-contact methods effectively address the seeker’s question by drawing upon personal experiences.

heading: G.3 Misalignment between Strategy and Response
text: A possible concern is that LLMs might lack ability to generate responses aligned with strategies. Therefore, we conduct an empirical case study to figure out this misalignments. In Figure 16, ChatGPT generates a response that is not aligned with the strategy Information predicted by external strategy planner. This may be due to knowledge conflicts, i.e., ChatGPT does not consider it appropriate to use the Information for the next response, despite being forced to generate a response aligned with the strategy Information. In conclusion, while
external assistance has potential to enhance performance, it is crucial to acknowledge that not all approaches yield positive impacts.

heading: Future Work
text: Therefore, future work might consider both correctly predicting the strategy and generating helpful responses based on the predicted strategy; (5) While we confirm that LLMs generally generate well-aligned responses with the strategy (Figure 16), it is evident that there are some cases where they are not aligned, thereby future work should recognize this misalignment