From Chatter to Matter: Addressing Critical Steps of Emotion Recognition Learning in Task-oriented Dialogue (2023)

Abstract: Emotion recognition in conversations (ERC) is a crucial task for building human-like conversational agents. While substantial efforts have been devoted to ERC for chit-chat dialogues, the task-oriented counterpart is largely left unattended. Directly applying chit-chat ERC models to task-oriented dialogues (ToDs) results in suboptimal performance as these models overlook key features such as the correlation between emotions and task completion in ToDs. In this paper, we propose a framework that turns a chit-chat ERC model into a task-oriented one, addressing three critical aspects: data, features and objective. First, we devise two ways of augmenting rare emotions to improve ERC performance. Second, we use dialogue states as auxiliary features to incorporate key information from the goal of the user. Lastly, we leverage a multi-aspect emotion definition in ToDs to devise a multi-task learning objective and a novel emotion-distance weighted loss function. Our framework yields significant improvements for a range of chit-chat ERC models on EmoWOZ, a large-scale dataset for user emotions in ToDs. We further investigate the generalisability of the best resulting model to predict user satisfaction in different ToD datasets. A comparison with supervised baselines shows a strong zero-shot capability, highlighting the potential usage of our framework in wider scenarios.

## introduction emotion recognition in conversations (erc) is a crucial task in conversational artificial intelligence research because it lays the foundation for affective abilities in computers such as empathetic response generation @xcite . over years, it has shown values in downstream applications such as opinion mining @xcite and humanlike dialogue modelling @xcite . dialogue systems can be broadly categorised into two categories: (1) chit-chat or open-domain (a) chit-chat dialogue from @xcite (b) task-oriented dialogue from @xcite figure 1 : comparison of dialogues about holiday in chit-chat dialogues and task-oriented dialogues. recently, the difference between chit-chat and tod systems have been blurred by the utilisation of pre-trained language models as back-bone to both types of systems. however, emotions in tods and chit-chat dialogues play different roles and are therefore expressed differently @xcite . this highlights the need for dedicated emotion modelling methods for each system. as illustrated in figure 1 , in chit-chat dialogues, speakers make use of emotions to facilitate communication by, for example, raising empathy as a result of emotion-eliciting situations or topics. on the other hand, emotions in tods are centred around the user's goal, and therefore emotion cues lie in both the user's wording and the task performance. while many large-scale corpora for emotions in chit-chat dialogues exist @xcite @xcite @xcite , there are considerably fewer resources for emotions in tods. emowoz, which evolved from multiwoz, a widely used tod dataset, is one notable exception @xcite . it contains a novel emotion description that is designed for tods and inspired by the ortony-clore-collins (occ) model @xcite . emotion is described in terms of three aspects: valenced (positive or negative) reactions towards elicitors (operator, user, or event) in a certain conduct (polite or impolite). however, due to the nature of tods, the occurrence of some emotions (e.g. users expressing feelings about their situations) are very rare, leading to a class imbalance in the corpus. similarly, advancements on the erc task are mainly focused on chit-chat dialogues, involving an array of diverse factors from speaker personality @xcite to commonsense knowledge @xcite . nevertheless, since these models are designed for chit-chat dialogues, they overlook how emotions are triggered and expressed with respect to goal completion in task-oriented context. the work of @xcite is among one of the earliest and very few to address emotion detection in tods but uses generic unigram models instead of dedicated approaches. in this work, we tackle critical steps of erc in tods from three angles: the data, the features, and the learning objective. in particular, data: we address the poor erc performance of particularly rare emotions in tods via two strategies of data augmentation (da), features: we leverage dialogue state information and sentiment-aware textual features, objective: we exploit the three aspects of emotions, namely valence, elicitor, and conduct, in two ways: as a multi-task learning (mtl) objective and to define a novel emotion-distanceweighted loss (emodistloss). to the best of our knowledge, our work is the first to provide dedicated methods for emotion recognition in tods. our experiments and analyses show that our framework leads to significant improvements for a range of chit-chat erc models when evaluated on emowoz. we further investigate the generalisability of the best resulting model to predict user satisfaction in various tod datasets under zero-shot transfer. our model achieves comparable results as supervised baselines, demonstrating strong zero-shot capability and potential to be applied in wider scenarios. ## emotion recogniser for task-oriented dialogues (ertod) in this section, we propose our ertod framework that adapt chit-chat erc models to the taskoriented domain, as illustrated in figure 2 . ## conclusion in this work, we propose ertod, a framework to address three critical steps in learning and effectively adapt chit-chat erc models to recognise emotions in tods. we propose two strategies of da for different emotions to improve erc performance in tods on rare emotions. we further leverage dialogue state and sentiment-aware embeddings for a richer feature representation. in addition, we apply mtl and devise a novel loss function, emodistloss, which take the similarities between emotions into account. our framework significantly improves existing chit-chat erc models' performance in recognising user emotions in tods. by further applying our best resulting model to perform the task of user satisfaction prediction, we show that our method generalises well on other similar valence-related classification tasks in tods. as more sophisticated and powerful dialogue systems such as chatgpt arise, there is an urge to recognise, understand and handle the emotion of the user, especially in the age where online abuse is omnipresent. the long-term aim of this work is to obtain valuable insight for downstream tod modelling tasks. this allows further investigation of emotion regulation strategies on the system side to improve task performance and user satisfaction, and to prevent undesirable user behaviours.