Emo-Gen BART - A Multitask Emotion-Informed Dialogue Generation Framework (2024)

Abstract: This paper is the model description for the Emo-Gen BART dialogue generation architecture, as submitted to the SCI-CHAT 2024 Shared Task. The Emotion-Informed Dialogue Generation model is a multi-task BARTbased model which performs dimensional and categorical emotion detection and uses that information to augment the input to the generation models. Our implementation is trained and validated against the IEMOCAP dataset, and compared against contemporary architectures in both dialogue emotion classification and dialogue generation. We show that certain loss function ablations are competitive against the state-of-the-art single-task models.

## introduction the realm of human conversation is intricately woven with emotions, a fundamental aspect that significantly influences the dynamics of communication @xcite . in contemporary research within natural language processing (nlp) and human-computer interaction (hci), the development of emotion-aware conversational agents has emerged as a focal point. various methodologies have been employed to handle emotions in conversation, with categorical labels and dimensional ratings being prominent avenues. these labels often find their roots in established emotion theories, such as ekman's @xcite ) or plutchik's @xcite , as evidenced in datasets like iemocap @xcite and dailydialog @xcite . additionally, alternative corpora and models adopt unique lists of emotion words, exemplified by the empathetic-dialogues dataset @xcite . the "dimensional" approach to handling emotion involves the utilization of characteristics inherent in emotional speech @xcite . a noteworthy model in this context is the valence-arousal-dominance (vad) model, which assesses the positive or negative sentiment, the degree of excitation, and the level of control exerted by the stimulus @xcite . this model has become a cornerstone in understanding and quantifying the nuanced dimensions of emotions expressed in conversational interactions. as we delve into the intricacies of emotionaware conversational agents, the utilization of both categorical and dimensional frameworks provides a comprehensive understanding of the emotional landscape within human-machine dialogues. in the domain of emotion-aware or empathetic conversational agents, diverse methodologies have been employed to augment systems' understanding and responsiveness to emotional cues. some methods incorporate input augmentation techniques, thereby exposing the conversational agent to various emotional expressions to enhance learning robustness @xcite . simultaneously, alternative approaches integrate common-sense or pragmatic information, drawing upon broader contextual knowledge to enrich the agent's comprehension of emotions within a given conversation @xcite . our system, emo-gen bart is a modification on bart architecture @xcite . our approach uses bart's emotion decoder attention representation to perform emotion classification as well as dimensional emotion detection. we then augment that representation to reinforce signals associated with emotion information. our strategy implements emotion classification and regression and combines their loss with the emotioninformed generation task. when accounting for contextual information through the conversation, we find that this method makes it competitive with state-of-the-art conversational agents. figure 1 : the emo-gen bart is a variation over the standard bart architecture with a bidirectional bert-like encoder and an autoregressive decoder. note that during fine-tuning for conditional generation, the input sentence is provided to both the encoder and the decoder. using the decoder output, we perform multiple emotion detection tasks. the final generation layer uses the decoder attention as well as the multitask attention. ## model architecture and implementation emo-gen bart, a customized version of the bart language model, is specifically tailored for conditional dialogue generation. the architecture of bart involves a bidirectional encoder processing tokenized and masked input sentences. during fine-tuning, this encoder utilizes the denoised input along with the encoder representation to generate subsequent sentences. for conditional generation tasks, a randomly initialized encoder precedes the bidirectional encoder during training. during fine-tuning, emo-gen bart modifies the bart architecture by extracting the last hidden layer, employed to predict emotion class and valence-arousal-dominance (vad) attention scores, illustrated in figure 1 . emotion-aware information is incorporated by concatenating the multitask and decoder attention outputs before the generation phase. ## experiments in this section, we describe the dataset, experimental setup, and hyperparameter information for reproducing these experiments. ## conclusion and future work in this paper, we introduce emo-gen bart, an architecture that employs a modified bart language model to enhance the capabilities of emotion-aware conversational agents. our approach integrates a multitask attention framework, acting as an emotion capsule, to improve the model's proficiency in identifying emotional cues during dialogue generation. we find that this approach of accounting for several tasks including emotion classification and regression, can inform the model and improve upon baseline results. we use only a single model variation where all the loss functions are weighted equally, however model ablations which form a hyperparameter relationship between the various tasks. finally, with multitask setups which change the nature of the architecture itself, it would be interesting to leverage llm predictions using dataset specific signals.