Improved Text Emotion Prediction Using Combined Valence and Arousal Ordinal Classification (2024)

Abstract: Emotion detection in textual data has received growing interest in recent years, as it is pivotal for developing empathetic human-computer interaction systems.This paper introduces a method for categorizing emotions from text, which acknowledges and differentiates between the diversified similarities and distinctions of various emotions.Initially, we establish a baseline by training a transformer-based model for standard emotion classification, achieving state-of-the-art performance. We argue that not all misclassifications are of the same importance, as there are perceptual similarities among emotional classes.We thus redefine the emotion labeling problem by shifting it from a traditional classification model to an ordinal classification one, where discrete emotions are arranged in a sequential order according to their valence levels.Finally, we propose a method that performs ordinal classification in the two-dimensional emotion space, considering both valence and arousal scales.The results show that our approach not only preserves high accuracy in emotion prediction but also significantly reduces the magnitude of errors in cases of misclassification.

## introduction emotion prediction from textual data has increasingly become important in natural language processing (nlp), as it lays the foundations for interactive and personalized computing; from enhancing the empathetic responses of chatbots to providing emotion-aware prompts in text-to-speech (tts) systems. the ability to accurately infer emotional states from text remains challenging, due to the absence of relevant cues which are only present in speech, such as tone and pitch. emotions are not always explicitly stated in the text, and intended emotion may be classified ambiguously, even by humans. traditional classification models treat emotions as discrete classes, offering a binary or multi-class output that may not fully capture the spectrum of human emotions @xcite @xcite @xcite . in this paradigm, the model does not account for the similarities among classes, e.g. the misclassification of sadness for joy is equivalently wrong as that of sadness for depression. in downstream applications like tts, such errors can lead to a substantial misrepresentation of the intended emotional tone and an unnatural outcome, e.g. uttering sad content with an excited voice. ## data we used the isear, wassa-21 and goemotions datasets in our study, which are publicly available and are commonly used in relevant works. isear dataset @xcite ) is a balanced dataset constructed through cross-culture questionnaire studies. it contains 7666 sentences classified into seven distinct emotion labels: joy, anger, sadness, shame, guilt, surprise, and fear. wassa-21 was part of the wassa5 2021 shared task on empathy detection and emotion classification. the dataset contains essays in which authors expressed their empathy and distress in reactions to these news articles. goemotions was presented in @xcite . the original dataset contains about 58k reddit comments with human annotations mapped into 27 emotions or neutral. to make our model comparable to other approaches we pre-processed our datasets following (adoma et al., 2020) for isear and @xcite for wassa-21 and goemotions keeping only that follows ekman's emotions @xcite . ## baseline model in constructing the baseline model, we conducted additional experiments focusing on the classification head. our classification head consists of two convolutional neural network (cnn) layers with kernel sizes @xcite and @xcite the number of filters respectively. the encoded information is compressed using mean pooling and the resulting vector undergoes a 3-layer feedforward neural network (ffnn) @xcite with softmax in the end. experiments followed these hyperparameters: @xmath0, learning @xmath1.6e-5, @xmath2, @xmath3, adamw optimizer. acknowledging that even with the state-of-theart approaches, models inevitably commit errors, we have introduced an ordinal classification approach aimed at reducing significant misclassifications on emotion recognition task. ## ordinal classification following the previous approach, we fine-tuned our model utilizing a standard cross-entropy loss where each label is discrete. an inherent limitation of the cross-entropy loss lies in its treatment of misclassifications as nominal rather than ordinal. in this context, misclassifying a "positive" as a "very positive" is no worse (in terms of loss) as "very negative". however, following this methodology is not optimal when we refer to emotions, e.g. misclassifying joy as excitement, is different from a misclassification to sadness. to address this, we arrange the emotions in an ordinal manner based on their valence level as illustrated in figure 1 . in order to minimize the gaps between labels in our model, we replaced the discrete one-hot representations of emotions with ordinal ones. by employing mean square error (mse) loss during training, our model focuses on narrowing the gap between target and prediction distances, emphasizing not only the correct classification but also the overall reduction of discrepancies. we experimented further by using regression loss instead of ordinal loss, however, the initial results favored the latter. following the ordinal classification, our baseline model achieved competitive performance on the three datasets, having a quicker convergence in every case on wassa-21 dataset the ordinal model did not achieve a macro-f1 score comparable to the baseline, despite maintaining an equally high accuracy. this can be attributed to the fact that the dataset was unbalanced and mse did not have a mechanism to handle it. we further examined the isear dataset for its balance, featuring a substantial number of examples for each emotion category. the ordinal classification forces the model to make less severe mistakes, by penalizing higher misclassifications that are very far from the ground truth regarding the valence order. even if the accuracy and f1-score are similar to the base model, the effectiveness of ordinal can be seen through the confusion matrices in figures 2 and error distances histogram in figure 3 . in the first case, the baseline confusion matrix (figure 2a ) makes more severe misclassifications that are far from the main diagonal. in contrast, on ordinal confusion matrix (figure 2b ) the misclassifications tend to distant the upper right and the down left corners, where the misclassification error is max, and gather around the diagonal. moreover, this phenomenon can be observed through error distances histogram figure 3 , in which we count the number of misclassification errors for each case. the misclassification error is defined as the distance between the target and the prediction on valence scale (i.e if the target was sadness and the prediction was anger the misclassification-error is 2 and if the prediction was fear the misclassification-error would be 5). the histograms show that the ordinal approach prefers to make misclassifications with distances of 1 rather than errors with distances larger than 3. ## 2d ordinal classification however, expressing a broader range of emotions proves challenging when relying solely on valence levels, as certain emotions may share similar valence values (e.g., both excitement and amusement emotions describe a very positive state). to enhance the expressiveness of our model and encompass a wider variety of emotions we introduced a second dimension to our problem: the arousal scale. based on russell's circumplex space model (russell, 1980), @xcite , we mapped a subset of 23 emotions to a 2d cartesian coordinate system, where the emotions are represented as points and the x-and y-axis are valence and arousal, respectively @xcite figure 4 . to extend the ordinal approach on both dimensions, we separated the emotion space into a 5 × 5 grid space, where each emotion belongs in a unique cell (e.g., in figure 4 grief and pride are mapped to (0, 0) and (3, 2) cells respectively). we adapt our model for 2d classification task by maintaining the valence classifier and introducing a supplementary classifier head for predicting the arousal level of each emotion. our model is trained to classify the given text in two manners, valence and arousal following the ordinal approach presented before. both heads are trained simultaneously by combining their losses. the anticipated valence and arousal levels serve as the coordinates within the emotion grid. to evaluate our 2d ordinal approach we utilized goemotion dataset, which offers a broad spectrum of emotions. among the 27 emotions available, we incorporated 23, ensuring that each grid cell corresponds to, at most, one emotion label. both approaches followed the previously outlined hyperparameter set during training. the results are presented in ## conclusion in this paper we presented a novel approach to emotion prediction from textual data, recognizing the nuanced similarities and distinctions among various emotions. initially, we introduced a roberta-cnn model for standard emotion classification as our baseline. by arranging emotions based on valence levels we shifted from traditional classification to ordinal. further innovation introduces ordinal classification in the two-dimensional emotional space, considering both valence and arousal scales. the proposed methodology enhances the model's performance by providing more meaningful predictions, taking into account the correlations between emotions. future directions involve extending research to diverse datasets, exploring alternative models, and experimenting with different emotion ordering schemes. an interesting direction also involves interpreting the model's components in order to better understand the importance of each feature in order to improve the existing method.