Understanding Emotion Valence is a Joint Deep Learning Task (2023)

Abstract: The valence analysis of speakers’ utterances or written posts helps to understand the activation and variations of the emotional state throughout the conversation. More recently, the concept of Emotion Carriers (EC) has been introduced to explain the emotion felt by the speaker and its manifestations. In this work, we investigate the natural inter-dependency of valence and ECs via a multi-task learning approach. We experiment with Pre-trained Language Models (PLM) for single-task, two-step, and joint settings for the valence and EC prediction tasks. We compare and evaluate the performance of generative (GPT-2) and discriminative (BERT) architectures in each setting. We observed that providing the ground truth label of one task improves the prediction performance of the models in the other task. We further observed that the discriminative model achieves the best trade-off of valence and EC prediction tasks in the joint prediction setting. As a result, we attain a single model that performs both tasks, thus, saving computation resources at training and inference times.

## introduction speakers express their emotions in the language in different modalities (e.g. speech and/or text) and interaction contexts (e.g. dyadic interactions or social media posts). a type of document imbued with emotions conveyed through the recollection of personal events experienced by the speaker is the personal narrative. personal narratives (pn) have been recently studied to promote healthier mental health by modelling the patients' life events and monitoring emotional states @xcite @xcite . monitoring the narrators' emotional states in pns is achieved through valence analysis and the identification of related emotion carriers. valence analysis addresses the identification of emotion levels ranging from pleasantness to unpleasantness generated by an event or a stimulus (russell, figure 1 : example of two functional units (fu1, fu2) by @xcite . in each unit, the emotionladen words convey an explicit emotion while the emotion carriers are implicit manifestations of emotions even though they represent neutral emotion at the surface level. the two elements of valence and ec are interdependant since valence represents the intensity of the experienced emotions while the ecs are the means through which emotions are expressed and conveyed throughout the pn. consequently, when narrators recount an event that activated their emotional state, the intensity of such emotion is manifested as valence while the expression of the emotion is through the recollection of the event and/or the participants characterising the activation of the emotional state. in this work, we explore the natural interdependency of valence and the related ecs in pns. this inter-dependency is characterised by the relations between the presence or absence of ecs and neutral or non-neutral valence. namely, the presence of ecs in a fu implies a non-neutral valence, while neutral valence for a fu implies the absence of ecs. moreover, the polarity of the valence might be related to the presence of domain-specific ecs. for instance, the ecs "the office" or "boss" might appear more frequently with a negative valence as opposed to "vacation" or "children". we investigate the contribution of this interdependency in the prediction of the valence and the related ecs in a multi-task learning (mtl) approach. mtl is to train a single model on multiple related tasks to achieve inductive transfer between the tasks, which is to leverage additional information sources while learning the current task. inductive transfer enhances generalisation by introducing an additional source of inductive bias used by the learner to prefer a hypothesis over the other hypothesis @xcite . we experiment with two mtl approaches i.e. joint and two-step prediction. while in the joint approach, the labels of the valence and ec prediction tasks are predicted simultaneously, in the two-step approach, the prediction of one label is used as the context for the prediction of the other task. we investigate whether this inter-dependency can be learned by pre-trained language models (plm). plms have prevailed over the other deep neural models in sentiment analysis @xcite , and they have been effectively used as a backbone of mtl architectures achieving state-of-the-art performance in intent classification and slot filling @xcite , dialogue state tracking @xcite and aspect-based sentiment analysis @xcite . we experiment with discriminative alberto (bert) as well as generative geppetto (gpt-2) models. in particular, for the discriminative model, we combine the two architectural solutions for valence and ec prediction tasks proposed by @xcite . while for the generative model, we design two prompts for valence and ec prediction and one prompt for the two-step prediction setting. in these experiments, we evaluate both models for each mtl setting, where the baseline is the performance of the model fine-tuned on every task separately. moreover, we compute the upper bound in the two-step prediction setting by replacing the first prediction with the ground truth. this upper bound also represents the level of inter-dependency between the two tasks. in this work, we use a corpus of pns in italian with valence and ecs annotated at the functional unit level. the contributions of this paper can be summarized as follows: valence & sentiment analysis the values of valence have been studied both in a continuous space @xcite , and discrete space with a likert scale @xcite , ranging from negative (unpleasant) to positive (pleasant). using the discrete approach, valence can be assessed with different levels of granularity as the narrative level and functional unit level. a functional unit (fu) is the minimal span of text expressing a dialogue act @xcite . narrative-level valence analysis provides a general yet flat perspective of the narrators' emotional state @xcite , meanwhile, the sentencelevel and fu-level analysis provide a detailed perspective as it highlights the variations and fluctuations of the valence throughout the narrative @xcite . a common practice in developing models for emotion analysis is to model valence analysis as sentiment analysis by mapping the valence values into three sentiment classes, i.e. positive, negative, and neutral @xcite . however, valence differs from sentiment as the latter identifies the polarity of attitudes or beliefs, such as hating or liking, towards a person (e.g. a politician) or an object (e.g. product or a movie) @xcite . meanwhile, valence represents the level of emotions in such as anger or happiness. emotion carrier and valence: emotion carriers are closely related to emotional valence as they explain the valence. mousavi et al. (2022) studied the correlation between the sequence tokens and the predicted valence (sentiment). the authors observed that the model focuses more on emotionladen words (explicit), whereas humans identify the emotion carriers to explain the valence. multi-task learning: multi-task learning (mtl) has been used for affective computing in aspect-based sentiment analysis @xcite @xcite , and emotion classification and emotion cause extraction @xcite . mtl has been studied using discriminative models (bert) for entity-relation extraction @xcite , as well as generative models (gpt-2) for task-oriented dialogues @xcite , dialogue state tracking @xcite , and task-oriented response generation @xcite . ## approach the inter-dependency of the valence and ecs results in the co-occurrence of both elements in the same utterance as the valence represents the level of emotion and the ecs are expressions through which the emotion is conveyed. in other words, the neutral valence of an utterance implies the absence of ecs, while the presence of ecs indicates a valence polarity for the same utterance. furthermore, the ecs in an utterance can provide insights into the polarity of the utterance valence since certain ecs are more often associated with negative emotions (such as deadline) or positive ones (such as graduation). we investigate the natural inter-dependency of valence and ecs via the mtl approach, which allows the models to leverage additional information learned from other tasks while learning the current task, improving the generalization. we experiment with joint training, where the two labels of valence positive negative intersection "perfume" "to tackle" "work" "vacations" "administration" "home" "yoga" "dentist" "lunch" "a stroll" "be late" "today" "freedom" "charged with" "feeling" and ecs are predicted simultaneously, and twostep prediction, where one of the labels is predicted and used by the model to condition and guide the prediction of the second label. ## models we experiment with discriminative (bert) and generative (gpt-2) models and investigate the performance of the two models for the joint and twostep prediction of valence and ecs. the joint and two-step prediction settings along with the two architectures are depicted in figure 2 . all the hyperparameters and model settings are reported in appendix 8 to achieve the reproducibility of the results. ## discussion the inter-dependency between valence prediction and ec prediction tasks is quantified in terms of performance by the upper bound computed by substituting the first step prediction with the ground truth. we observe that the performance of both tasks is enhanced. furthermore, we observe that the proposed mtl approaches, i.e. two-step and joint prediction, are effective in exploiting such inter-dependency. in particular, the two-step prediction boosts the performance of the generative model, while joint prediction improves the performance of the discriminative model. moreover, we observe that these improvements affect mainly the ec prediction task. this is because, compared to valence prediction, the ec prediction task is objectively more challenging for the models due to the unbalanced distribution and the sparsity of the ecs (some ecs are personal w.r.t the narrator). thus, the predictions of the ec tasks are too noisy to be exploited by the models to improve the performance of valence prediction. indeed, the worst performance for valence prediction is achieved by both models in the two-step setting in which the first task is the ec prediction (i.e. @xmath0.). regarding the comparison between discrimina-tive and generative models, the best trade-off between valence and ec prediction tasks is achieved by alberto, although geppetto fine-tuned with the single-task setting achieves the best performance on valence prediction. ## conclusions in this work, we studied the inter-dependency between valence and ecs in personal narratives. for valence and ec prediction tasks, we compared taskspecific models with two mtl settings, namely joint and two-step prediction. we experimented with discriminative and generative plms. the results indicate that plms fine-tuned with mtl settings achieve improved performance by exploiting the inter-dependency between valence and ec prediction tasks. in particular, the two-step setting is more effective for the generative model, while the joint setting best fits the discriminative model. furthermore, the generative model outperforms the discriminative model on the valence prediction task, while the discriminative model achieves better results on ec prediction and the best tradeoff between the valence and ec prediction tasks. consecutively, one discriminative model performs the two tasks, reducing the demand for computational resources at training and inference time and, therefore, lowering carbon emissions in the environment.