Zero at S em E val-2025 Task 11: Multilingual Emotion Classification with BERT Variants: A Comparative Study (2025)

Abstract: Emotion detection in text plays a very crucial role in NLP applications such as sentiment analysis and feedback analysis. This study tackles two tasks: multi-label emotion detection, where the goal is to classify text based on six emotions (joy, sadness, fear, anger, surprise, and disgust) in a multilingual setting, and emotion intensity prediction, which assigns an ordinal intensity score to each of the above-given emotions. Using the BRIGHTER dataset, a multilingual corpus spanning 28 languages, the paper addresses issues like class imbalances by treating each emotion as an independent binary classification problem. The paper first explores strategies such as static embeddings such as GloVe with logistic regression classifiers on top of it. To capture contextual nuances more effectively, we fine-tune transformer based models, such as BERT and RoBERTa. Our approach demonstrates the benefits of fine-tuning for improved emotion prediction, while also highlighting the challenges of multilingual and multi-label classification.

## introduction emotion detection in text is a fundamental task in natural language processing (nlp), which has various important applications like sentiment analysis, feedback analysis, and chatbots. understanding the emotions present in textual data is important in digital communication, where body language and facial expressions will not available. the increased textual interactions on social media, news, and online forums have emphasized the need for accurate emotion detection systems. recent breakthroughs and results in nlp have significantly enhanced the performance of emotion detection models @xcite . current-age transformer architecture-based models @xcite use large-scale language representation learning to model intricate semantic and syntactic relations in the text. in this paper, we address the task of multi-label emotion detection @xcite as part of a shared workshop challenge. the task consists of two tracks: the model needs to determine whether each emotion is either present (1) or absent (0) for each sample. class imbalance is a common challenge when collecting data for emotion classification, mostly due to the natural distribution of real-world occurrences. the methodology discussed in this paper tries to adequately address the class imbalance and multi-label generated nature of the data. rather than treating the task as a multi-label classification, we split it into separate independent binary classification tasks, where one classifier is trained for each emotion. this method ensures that the model learns to classify each emotion separately, thus trying to avoid class imbalance problems. we also finetune transformer models such as as bert @xcite , roberta @xcite , and other such models to leverage their contextual representations and enhance prediction performance. through the exploration of both static embeddings and transformer models fine-tuned from scratch, we want to compare the efficiency of various feature representations in multi-label emotion classification and intensity estimation. past studies in this area have explored the topics of sentiment analysis and emotion detection using multiple different types of approaches. @xcite analyzed many malayalam youtube comments using ml models and deep learning techniques. the methodology provided in @xcite proposes a hybrid bert model for the task, which shows improved contextual understanding. @xcite uses word embeddings for sentiment analysis, clearly showcasing the role of pretrained representations. these works provide some important information about the evolution of emotion detection techniques. further sections provide a more detailed overview of the dataset, methodology, and experimental results. pet, and labels that indicate the presence of a particular emotion. instances are multi-labeled and are labeled from 0 to 3 depending on the intensity of the emotion present in the sentence. for the initial task, these labels are simplified to depict the presence (0) or absence (1) of a specific emotion and intensity is ignored. we shall primarily use the english, spanish, russian, and romanian datasets with the following emotions: anger, fear, joy, sadness, surprise and disgust. the english dataset does not have a label for disgust. upon analyzing the class distribution, we observe an imbalance between positive and negative samples for nearly every emotion as depicted in fig 1 . the substantial variation in the class distribution comes from the method of choosing data from the named sources and also the amount of available data on platforms such as those. this is not only present across emotions in a language, but also across languages as expected. ## methodology due to the limited size of the dataset, it is rare to encounter samples that encompass all possible combinations of emotions across all the classes, as certain combinations may be inherently less likely to co-occur than others or may not be represented at all due to the natural distribution of real-world occurances. to address the observed class imbalances in the dataset, we split the classification task into independent binary prediction tasks to detect the presence of each emotion separately by utilizing distinct classifiers. the predictions are concatenated to obtain the final emotion representation vector. ## results the evaluation of the models was conducted using different metrics based on the task at hand. for binary emotion classification task, accuracy and f1-score were used. accuracy provides an overall measure of the correct predictions, but since emotion classes are imbalanced in the present dataset, f1-score is a more informative metric as it takes into account both precision and recall. for the task of emotion intensity prediction, the performance is measured using pearson correlation coefficient (r). this metric evaluates the linear relationship between the predicted values and the actual intensity values. ## conclusion the evaluation of transformer-based architecture models for the tasks of emotion detection and intensity prediction, highlights the advantages of finetuning over using static embeddings. bert-based models, especially bert-large, consistently outperformed other models in both the tasks, achieving the highest f1-scores and pearson correlations on average. roberta also demonstrated competitive performance, particularly in the binary classification task, due to its optimized pre-training approach. in multilingual classification, mbert facilitated cross-lingual generalization, though performance did vary depending on language representation in the pre-training corpus. across all tasks, larger models with deeper contextual representations provided the better results, reinforcing the impact of the size of models and training methods. these findings bring out the ef-fectiveness of transformer models in emotion classification and suggest that more advancements in model architecture and quality of pre-training data could yield even better results.