Infusing Emotions into Task-oriented Dialogue Systems: Understanding, Management, and Generation (2024)

Abstract: Emotions are indispensable in human communication, but are often overlooked in task-oriented dialogue (ToD) modelling, where the task success is the primary focus. While existing works have explored user emotions or similar concepts in some ToD tasks, none has so far included emotion modelling into a fully-fledged ToD system nor conducted interaction with human or simulated users. In this work, we incorporate emotion into the complete ToD processing loop, involving understanding, management, and generation. To this end, we extend the EmoWOZ dataset (Feng et al., 2022) with system affective behaviour labels. Through interactive experimentation involving both simulated and human users, we demonstrate that our proposed framework significantly enhances the user’s emotional experience as well as the task success.

## introduction in recent years, conversational artificial intelligence (ai) has become increasingly prevalent in various domains, providing users with interactive and personalised experiences. emotions play a crucial role in human communication and can influence the way individuals perceive, process, and react to information @xcite . consequently, incorporating emotions into conversational ai has emerged as a promising avenue for improving user experience and creating more human-like interactions @xcite . task-oriented dialogue (tod) systems, an important genre of conversational ai, are designed to assist users in fulfilling specific tasks or queries. in contrast to chit-chat or open-domain dialogue systems, which focus on creating engaging and entertaining conversations, tod systems interact with users in a more structured way with a clear objective under specific domains (jurafsky and martin, * these authors contributed equally to this work. in tod, emotion is centred around the user goal, making it more contextual and subtle @xcite . a recent study has shown that the valence of user emotion in tod positively correlates with dialogue success @xcite . this observation aligns with a number of emotional theories. for example, the appraisal theory of emotion argues that emotion is the result of our evaluation of a situation @xcite . in relation to a tod user goal, it is straightforward to see how task fulfilment would lead to positive emotions and failures to negative ones. similarly, the ortony-clore-collins (occ) model of emotion states that emotion is the result of elicitation by events, agents, and objects @xcite . @xcite have drawn the connection between the occ model and user emotions in tod. therefore, besides inferring emotional states from dialogue utterances, an agent also needs to reason about emotion-generating situations and to utilise this information to achieve dialogue success. the integration of emotion into the full tod pipeline has been a long-standing interest @xcite . yet, early works explored analytical solutions in constrained set-ups, which hindered their applications in more complicated scenarios. recently, a number of resources emerged for studying user affect in tods, e.g. emotion, sentiment, or satisfaction @xcite . this has motivated efforts to model user emotion via data-driven approaches, such as emotional user simulation @xcite and user emotion recognition @xcite . however, to the best of our knowledge, no work so far has com-bined these emotional aspects into a fully-fledged dialogue system and an interactive pipeline where emotions play a role in understanding, generation, as well as management of the conversation. to achieve this, we need to endow the dialogue system with the ability to respond with an affective behaviour, closing the emotional loop between the user and the system in tods. towards this goal, we make the following contributions: ## related work in this section, we discuss related works on incorporating emotion in each stage of tod pipeline: understanding, management, and generation. these stages are modelled explicitly with multiple models in modular systems and implicitly with a unified model in end-to-end systems @xcite . ## emowoz 2.0: a fully emotion-annotated tod dataset to study emotion in real-world interactions between users and human operators in the tod setting, we extend emowoz @xcite by further annotating the affective behaviour of the system, which is acted by human-beings. we call this dataset with extended labels emowoz 2.0. * in tods, the user and the system play different roles. users may express a wide range of emotions during interactions based on their goals and experiences with the operator. the system is responsible for managing and facilitating the conversation and is supposed to behave professionally and politely to achieve the goal. therefore, it is necessary to consider different sets of affective behaviours in the user and the system respectively. we refer to the concept of the operator's affective behaviour as affective conduct, or conduct for short. annotation scheme according to studies on customer satisfaction in business @xcite , competent operators in tod try to guide user emotion towards positive valence by making use of subtle emotion in their response while providing correct information. by considering the set of user emotions in emowoz and the occ emotion model (detailed justification in appendix a.1), we arrive at five affective conduct classes: annotation set-up we annotated the conduct for all operator utterances in the multiwoz subset of emowoz. machine-generated system responses in the dialmage subset came from a template nlg, which we considered to have neutral conduct because those templates aimed to express actions concisely rather than conveying emotions by design. we followed the data collection and quality assurance set-up of emowoz and conducted the annotation via the amazon mechanical turk platform. details and an illustration of the annotation interface can be found in appendix a.2. annotation quality each utterance has been annotated by at least three annotators. the interannotator agreement as measured with fleiss' kappa is 0.647, suggesting substantial interannotator agreement. the annotator confusion matrix and label distribution can be found in appendix a.3 and a.4, respectively. ## infusing emotions into tod systems we propose to incorporate emotion into the full interactive tod pipeline, which is primarily comprised of three stages: understanding, management, and generation. we aim for understanding to accurately recognise the user's emotion in addition to the task-centred dialogue state. for dialogue management, we make use of emotion for optimal action selection. lastly, we additionally condition the natural language generation on the system conduct to generate more diverse and emotion-aware responses. these are realised in each modular system component individually (section 4.1 to 4.3) and as intermediate tasks in the unified model in end-to-end systems (section 4.4). * ## conclusion in this work, we incorporate emotion into the complete tod processing loop, involving understanding, management, and generation. to achieve this, we first enrich the emowoz dataset with system conduct labels to construct emowoz 2.0. we then build modular and end-to-end tod systems, as well as emotional user simulators with the newly collected dataset. we train the modular system policy via rl with the emotional user simulator and the end-to-end system via sl on emowoz 2.0. through interactive evaluation with both simulated and human users, we show that incorporating emotion into tod systems can improve user's emotional experience as well as task success. there is still a long way to go from our work to the perfect emotional tod system. yet, we show our method as a promising avenue to achieve this ultimate goal. in our study, we directly translate user emotion labels into valence scores on a linear scale as a reward for rl. we believe that utilising the full set of user emotion labels for diverse reward would be a promising future direction. we hope that with our work, we can motivate future research efforts to look at user experience beyond task success for tods and bring about insights to other task-oriented conversation settings. we would also like to highlight the opportunities in further improving llm-based end-to-end tod systems via rl, combining established approaches for policy training in modular systems and recent advancements in llm research in other applications.