INGEOTEC at S em E val-2024 Task 10: Bag of Words Classifiers (2024)

Abstract: The Emotion Recognition in Conversation subtask aims to predict the emotions of the utterance of a conversation. In its most basic form, one can treat each utterance separately without considering that it is part of a conversation. Using this simplification, one can use any text classification algorithm to tackle this problem. This contribution follows this approach by solving the problem with different text classifiers based on Bag of Words. Nonetheless, the best approach takes advantage of the dynamics of the conversation; however, this algorithm is not statistically different than a Bag of Words with a Linear Support Vector Machine.

## introduction sentiment analysis has been very useful for emotion detection in digital text. for example, in the analysis of a customer review, a sentiment analysis system can find whether the review is positive or negative. today, this way of finding out what the sentiment expressed in the digital text has been popular due to its potential to have a feeling over what people are writing about. recent studies have been conducted towards sentiment analysis, not only in a one-party text (text written by one person/source) but also within a multi-party conversational text @xcite @xcite , known as emotion recognition in conversation (erc). erc refers to the emotion detection of each of the phrases/utterances within a dialogue. for instance, in a conversation between two people (speaker 1, sp1, and speaker 2, sp2) saying the following, sp1: "i had an awful day", sp2 replies "oh no, what happened?". sp1 may have a "sad" emotion and sp2 may be also "sad". however, following the conversation sp1 answers this work is licensed under a creative commons attribution 4.0 international licence. licence details: http://creativecommons.org/licenses/by/4.0/ . to date, several studies have investigated erc using different approaches @xcite as it is summarized by @xcite . @xcite presented a framework for emotion detection in conversations using a recurrent neural network (rnn) based memory network with multi-hop attention modeling. @xcite is a method based on an rnn that maintains information of each party separately and this information is used for emotion classification. most recent studies have been focused on more elaborated proposals about emotion detection based on the context and the common sense knowledge within the conversation @xcite . recently, @xcite presented a self-supervised model to better understand the semantics within the text associated with the order of the utterances. in this paper, we present a model given the task 10: emotion discovery and reasoning its flip in conversation(subtask 1) of the semeval-2024 workshop @xcite which consists of applying emotion recognition in conversation (erc) to hindi-english code-mixed conversations. we propose to solve the challenge as a classification problem, using a bag of words (bow) for the representation. despite the usage of bow is not so common anymore due to the current usage of more sophisticated techniques as deep learning, our work is built on the previous work presented in @xcite , leading to a unique and customized bow for solving this specific problem. the rest of the paper is organized as follows: section 2 presents the background; section 3 introduces a description of the model; finally section 4 shows a brief analysis of the results and section 5 concludes the paper. kumar2024semeval 2 background emotion recognition has been a popular research field using artificial intelligence. according to @xcite , several methods are applied for emotion recognition including facial expression recognition, physiological signals recognition, speech signals variation, and text semantics, among others. specifically, in this work, we focus on emotion recognition written in a digital text. previously, emotion recognition in the text has been focused on the selection of emotional keywords @xcite and the classification of their emotional state within a conversation. however, this keyword-based method presented some limitations as ambiguity and the lack of semantic and syntactic information. emotion recognition in a conversation has shown to be a challenge due to the way emotions change over time. other machine learning method have been applied as icon @xcite and dialoguernn @xcite , both using rnn. icon @xcite generates memories from the conversation to generate a good context for predicting emotions within a conversational video. dialoguernn @xcite presented a model involving three aspects in a conversation: the speaker, the context of the previous phrases, and the emotion, according to them, taking into account these key aspects leads to a much better context representation. commonsense knowledge is something difficult to pass over a machine, for that reason new approaches added external knowledge to help the machine to have a context @xcite ) @xcite for solving new challenges as having an empathetic dialogue system @xcite . ## system overview the subtask emotion recognition in conversation can be posed as a supervised learning problem. without considering that emotions are part of a conversation, the problem can be seen as finding the mapping between an utterance and its associ-ated emotion, i.e., it is a classification problem. in order to use the majority of classifiers, one needs to transform the utterance into a format amenable to the classifier selected. generally, the representation acceptable for the majority of traditional classifiers is vectors. perhaps one of the most studied representations that transform a text into a vector is the bag of words (bow); it is not so common anymore because it has been overcome by the use of deep learning techniques such as the attention mechanism @xcite . however, our participation is based solely on the use of bow, following a similar approach used in previous competitions see @xcite , and complementing it with an approach tailored for this specific subtask. the realm of the bow representation is that each token t of a text is associated with a vector v @xmath0. in this contribution, the i-th component of v t corresponds to the token's inverse-document-frequency (idf) estimated in a collection of hindi tweets (9.5 million), and the rest of the components of v t are zero, i.e., ∀ j̸ =i v @xmath1. the set of all tokens is fixed, and these correspond to the vocabulary. the vocabulary is fixed to containing only 2 17 = d elements, and this corresponds to the most frequent tokens found in the collection of tweets. furthermore, given that only one component of each vector is different from zero, the set of all the vectors constituted a basis, and each text is represented in this vector space. additionally, any token that is not found in the vocabulary is discarded from the representation. using this notation, a text x is represented by the sequence of its tokens, i.e., (t 1 , t 2 , . . .); the sequence can have repeated tokens, e.g., t @xmath2. then each token is associated with its respective vector v (keeping the repetitions), i.e., (v t 1 , v t 2 , . . .). finally, the text x is represented as: the second representation is inspired by a selfsupervised technique, particularly the procedure of masking tokens in a text and then developing an algorithm to predict the masked tokens. the idea is pursued by creating m binary classification problems where the task is to predict the presence of a particular token; in this case, the tokens are words (defined as a string surrounded by spaces or punctuation symbols) or emojis. the words are selected based on their frequency; the most frequent words are not considered, and the words considered started when the plot of rank vs. frequency settles, i.e., it is when the flat part starts. on the other hand, all the emojis are considered. however, only the words and emojis where there are more than 1024 positive examples in the collection are kept. in total, there are 176 tweets and 2048 words. there are 2,224 binary classification problems; each is solved using a bow representation where the classifier is a linear support vector machine. consequently, there are m binary text classifiers, i.e., (c 1 , c 2 , . . . , c m ). the utterance is represented using the decision values of the m binary classifiers; that is, the text lives in r m . as can be seen, each component is associated with either a word or emoji, and its value indicates the likelihood of its presence in the text. we refer to this representation as dense. finally, the classifier used with the dense representation is again a linear support vector machine. after the competition ended, we decided to include in the comparison a procedure to combine (using stacking @xcite ) the bow and dense representation, namely stackbow. the idea is to make a convex combination of the class probabilities predicted by these two classifiers. the approach is to use the training set with k-fold crossvalidation to estimate the decision function of these two classifiers on the training set. these decision functions are then transformed with softmax to obtain probabilities, and then an optimizer is used to estimate the convex combination. there are 8 emotions, so the optimizer needs to find 8 coefficients, each corresponding to a class. for example, let p @xmath3, p @xmath4, and β ∈ r 8 are the estimated coefficients, then the prediction of the stackbow is β ⊙p b +(1-β)⊙b d where ⊙ is the pointwise product. the last system, ingeotec, corresponds to an approach that takes advantage of the conversation dynamics. it considers the current utterance, the previous, and the next. in the extremes, either the next or the previous are empty utterances. let x, x p , and x n be the dense representation, then it is computed the similarity between the current utterance (x) and the previous and next utterance, as follows: s @xmath5, and s @xmath6. at first, ρ is a vector of ones, so s p and s n are the cosine similarity because the dense representations have unit length. using s p and s n , the contribution of each representation is computed by converting the similarity to a probability; this is done with the softmax as @xmath7, s p , s n ). using s another representation is created which is the convex combination between x, x p , and x n , i.e., x @xmath8. using x and x s , the dense representation used is the concatenation of them, i.e., @xmath9, x s ]. the final dense representation, w, is used in a linear equation combined with softmax to predict the probabilities of each class. the probabilities obtained in the previous step are combined, using a convex combination, with the decision function of a bow classifier (transformed with softmax). it is important to mention that the parameters, ρ, the coefficients to create the final convex combination, and the parameters of the linear equation of the dense representation w are optimized with gradient descent. ## results the systems' performance analysis starts with the information presented in ## conclusion we have described the algorithms tested on the emotion recognition in conversation task. most of the approaches treat this problem by looking at each utterance separately. the only system taking advantage of the dynamic of the conversation is the ingeotec system; this system is the one having the best performance. nonetheless, as figure 1b shows, it is not statistically different than a bow classifier. the bow classifier is the simplest model one can start experimenting with.