Improving Multi-turn Emotional Support Dialogue Generation with Lookahead Strategy Planning (2022)

Abstract: Providing Emotional Support (ES) to soothe people in emotional distress is an essential capability in social interactions. Most existing researches on building ES conversation systems only considered single-turn interactions with users, which was over-simplified. In comparison, multi-turn ES conversation systems can provide ES more effectively, but face several new technical challenges, including: (1) how to adopt appropriate support strategies to achieve the long-term dialogue goal of comforting the user’s emotion; (2) how to dynamically model the user’s state. In this paper, we propose a novel system MultiESC to address these issues. For strategy planning, drawing inspiration from the A* search algorithm, we propose lookahead heuristics to estimate the future user feedback after using particular strategies, which helps to select strategies that can lead to the best long-term effects. For user state modeling, MultiESC focuses on capturing users’ subtle emotional expressions and understanding their emotion causes. Extensive experiments show that MultiESC significantly outperforms competitive baselines in both dialogue generation and strategy planning.

## introduction almost every human has experienced emotional distress, even if not suffering from any mental disorders. frequently, people deal with the distress by seeking emotional support (es) from social interactions @xcite . nevertheless, es from family and friends is not always available @xcite . with the potential of providing more people with in-time support, developing emotional support conversation (esc) systems has attracted much attention. however, since early es datasets are constructed figure 1 : an example of an emotional support conversation between the support-seeker (left) and the supporter (right). the support strategies adopted by the supporter are presented in red italics before the utterances. by crawling post-response pairs from online forums, they only contain single-turn conversations @xcite . thus, most of the existing research on esc also only considers single-turn interactions with the user @xcite @xcite , which is over-simplified and has limited support effects. it was not until recently that @xcite released the first large-scale multi-turn es dataset, esconv. they also designed an esc framework, suggesting the conversation procedures and support strategies for multi-turn esc. compared to the single-turn scenario, developing multi-turn esc systems faces several new challenges. one significant challenge is support strategy planning. as pointed out in the psychological literature, particular procedures and strategies are indispensable for effective emotional support @xcite . as in fig. 1 , the supporter strategically soothes the support-seeker by first caringly inquiring about the situation, then resonating with the seeker's feelings, and finally providing suggestions to evoke positive emotions. notably, strategy planning in esc should be conducted on a long planning horizon. that is, instead of merely considering the dialogue history or foreseeing the immediate effect after using the strategy, the system should further look ahead, to consider how much the adopted strategy would contribute to reducing the user's emotional distress at a long run. though some strategies may not directly provide comfort, they are still essential for reaching the long-term dialogue goal, such as greetings at the beginning of the conversation and inquiring about the user's experiences. another challenge for multi-turn esc is how to dynamically model the user's state during the conversation. prior works on emotion-related dialogue tasks mainly detect the user's coarse-grained emotion type to enhance dialogue generation @xcite @xcite . however, such practice is not completely appropriate for esc. the reason is that the user's emotion in esc almost stays the same type, such as being sad, throughout the conversation. instead, it often changes subtly in terms of emotion intensity. besides, effective es requires more than only identifying the user's emotion. a thorough understanding of the user's situation is also essential. in this paper, we propose a multi-turn esc system multiesc to address the above issues. for strategy planning, we draw inspiration from the a * search algorithm @xcite and its recent application in constrained text generation @xcite , which addressed the challenge of planning ahead by incorporating heuristic estimation of future cost. in multiesc, we develop lookahead heuristics to estimate the expectation of the future user feedback to help select the strategy that can lead to the best long-term effect. concretely, we implement a strategy sequence generator to produce the probability of the future strategy sequences, and a user feedback predictor to predict the feedback after applying the sequence of strategies. for user state modeling, multiesc captures the user's subtle emotion expressed in the context by incorporating external knowledge from the nrc vad lexicon @xcite . moreover, it identifies the user's emotion causes (i.e., the experiences that caused the depressed emotion) to more thoroughly understand the user's situation. in summary, our contributions are as follows: emotional support conversation systems. since early es datasets were mainly composed of single-turn conversations @xcite , most research on developing esc systems only considered the simplified scenario of single-turn interactions with the user @xcite . the few works that developed multi-turn es chatbots rely on predefined templates and handcrafted rules (zwaan et al., 2012), which suffer from limited generality. it was not until last year that liu et al. (2021) released the first multi-turn esc dataset esconv. following liu et al. (2021), peng et al. (2022) and tu et al. (2022) recently explored data-driven multi-turn esc systems. peng et al. (2022) proposed a hierarchical graph network to capture both the global context and the local user intention. they did not consider strategy planning, which is critical in multi-turn esc. tu et al. (2022) proposed to enhance context encoding with commonsense knowledge and use the predicted strategy distribution to guide response generation. nevertheless, their method of strategy prediction, directly implemented with a vanilla transformer encoder, was relatively preliminary and did not consider any user-feedback-oriented planning as we do. empathetic response generation. empathetic response generation (erg) (rashkin et al., 2019) ## preliminaries esconv. our research is conducted on esconv. it is a long conversation dataset, with an average of 29.8 utterances in each dialogue. it also includes rich annotations, such as the strategies adopted by the supporter and the user feedback scores. there are overall eight types of strategies (e.g., question, reflection of feelings and self-disclosure). the user feedback score indicates how much the user's emotional distress is reduced during the conversation. they are marked by the support-seekers on a likert scale with five levels after every two turns. more data statistics are provided in the appendix. ## methodology as shown in fig. 2 , our proposed system multi-esc consists of four modules. the dialogue encoder first converts the dialogue history h t into the embeddings h t . at the same time, the user 1 we suppose that escs are always initiated by the system (or the supporter). state modeling module extracts the user state information, producing the embeddings u t . then, given h t and u t , the strategy planning module selects the strategy s t . finally, the utterance decoder generates the utterance x t , adopting the strategy s t . ## conclusion in this paper, we explored the task of developing multi-turn emotional support conversation (esc) systems, with focus on how to strategically plan the conversation procedure. to this end, we proposed a novel esc system, multiesc, that conducts strategy planning with lookahead heuristics to estimate the long-term effect of the adopted strategy on the user. moreover, we also proposed some effective mechanisms to dynamically model the user's state in multi-turn escs. the empirical results showed that multiesc achieves significant improvement compared with a set of strong baselines in both generation quality and strategy planning.