IEGPS - CSIC at S em E val-2025 Task 11: BERT -based approach for Multi-label Emotion Detection in E nglish and R ussian texts (2025)

Abstract: This paper presents an original approach for SemEval 2025 Task 11. Our study investigates various strategies to improve Text-Based Multi-label Emotion Detection task. Through experimental endeavors, we explore the benefits of contextualized vector representations by comparing multiple BERT models, including those specifically trained for emotion recognition. Additionally, we examine the impact of hyperparameters adjustments on model performance. For Subtask A, our approach achieved F1 scores of 0.71 on the English dataset and 0.84 on the Russian dataset. Our findings underscore that (1) monolingual BERT models demonstrate superior performance for English, whereas multilingual BERT models perform better for Russian; (2) pretrained emotion detection models proving less effective for this specific task compared to models with reduced vocabulary and embeddings focused on specific languages;(3) exclusive use of BERT-based models, without incorporating additional methods or optimization techniques, demonstrates promising results for multilabel emotion detection.

## introduction the emotion mining framework @xcite represents a recently established specialized task that allows for evaluation and identification of emotional tone conveyed in textual data. traditionally, sentiment analysis classified textual content as positive, negative, or neutral @xcite . however, emotion mining techniques enable the assignment of a broader range of emotions to texts @xcite , such as those defined by the ekman @xcite , which identifies six basic emotions (sadness, joy, disgust, surprise, anger, and fear) and the absence of emotion. with rise of large language models (llm), this emotion classification has been also widely applied in domains like marketing @xcite , health @xcite , and media @xcite , helping researchers and organizations understand emotional aspects, make informed decisions, and improve services. to date, emotion analysis has shown promising results, especially in english, due to abundant resources @xcite . semeval 2025 task 11 @xcite seeks to bridge the gap in text-based emotion identification across 32 languages, focusing on emotion perception-how most people interpret a speaker's emotions from text @xcite . this is challenging as perceived emotions may differ from actual emotions due to cultural and individual factors @xcite . it is also important to highlight that subtask a accounts for the complexity of expressed emotions in text by allowing multi-label emotion classification. this approach not only enhances the accuracy of emotion extraction but also acknowledges that some emotions can emerge from the interaction of two or more emotions, as described by @xcite . based on these outcomes, our study seeks to advance the field of text-based emotion detection by providing the following contributions: this paper is structured as follows: section 2 reviews state-of-the-art models for text-based emotion identification. section 3 discusses dataset insights and our approach to architecture selection. section 4 presents the experimental results. section 5 provides a qualitative analysis and error categorization of the best-performing models. finally, section 6 concludes the paper. ## background several antecedent works have addressed the problem of emotion annotation and identification in textbased sources. a foundational contribution within the semeval framework is by @xcite , who introduced multi-label emotion annotation across 174,356 tweets in english, arabic, and spanish, covering twelve emotion categories (e.g., "anger" included related emotions like annoyance and rage). for english tweets, the best results were achieved using an ensemble of pretrained models, feature extraction, and traditional classifiers (e.g., svm, logistic regression), reaching a pearson's r of 79.9. building on this, @xcite improved performance through ensemble methods combined with transfer learning. since then, particularly in semeval 2020, 2023, and 2024, bert models @xcite and transfer learning have become the dominant approaches for emotion detection, consistently achieving high f1 scores (see ## datasets and model the dataset by @xcite contains english and russian short text snippets. we provide a statistical overview for each dataset in ## experiment results using the architecture detailed above, we experimented with 15 different models for both english and russian datasets. as shown in these development dataset evaluation results were subsequently tested on the final test dataset described in section 3. the final f1 macro scores achieved were 0.71 for english and 0.8418 for russian, demonstrating the effectiveness of our chosen models and approach across both languages. ## discussion the scores illustrated in figure 2 represent the accuracy of our model in correctly predicted emotions. in english, fear has the highest accuracy (82%), followed by joy and sadness, while surprise and anger perform slightly lower. the russian dataset shows higher overall accuracy, with joy at 91% and fear 88%, while disgust is the less accurate (77%). these results could be attributed to both the model's performance and the dataset used for finetuning. nonetheless, as discussed in section 3, fear is the most overrepresented emotion in the english dataset, which aligns with our model strong performance predicting it, as shows the figure 2 . on the contrary, anger, being underrepresented in the english dataset, exhibits the lowest accuracy. this indicates that the model is moderately sensitive to class imbalance-performance on minority emotions drops, especially in the english dataset where fear dominates. the russian dataset is more balanced, allowing the model to maintain stable performance across different classes. as future research, we aim to improve this dataset by incorporating a more diverse range of text snippets, which, we hope, will benefit the model. finally, we analyzed the accuracy of our model in performing multilabel emotion classification based on textual data. as shown in ## conclusions to the best of our knowledge, this study presents several novel contributions to the field of emotion detection in english and russian, which have not been explored in previous works: first, the evaluation and fine-tuning of various bert-based models for emotion detection reveal that monolingual bert models achieve superior performance for english, while multilingual bert models perform better for russian. notably, a bert model with a reduced vocabulary and embeddings specifically tailored to russian and english demonstrated the highest accuracy for the russian dataset. compared to the baseline by @xcite , which used only finetuned multilingual roberta on each language's training data, our approach-fine-tuning the classifier layer with both multilingual and monolingual bert models-achieved better results. we ranked 44th in track a for english and 25th for russian. second, the architecture of our model follows state-of-the-art principles used in emotion detection tools. while prior studies suggest that incorporating additional methods and techniques alongside bert-based models enhances emotion detection performance, our experiments reveal that bert models alone achieve sufficient accuracy. in summary, we have contributed to the area of emotion detection in english and russian by reutilizing the available bert-based models, refining them for this specific task, which has provided positive results (71% for english and 84% for russian) showing the high accuracy in multilingual emotion labeling and outperforming results, reported in previous works. as for limitations, we have not yet extensively explored llms due to resource constraints, though we recognize their potential for improving contextual understanding in multilabel emotion classification. while this work focused on transformer-based models, we acknowledge that exploring alternative classifier architectures (e.g., conv1d + mlp) and traditional methods (e.g., svm, xgb) could enhance accuracy. we also aim to apply regularization and data augmentation techniques to reduce overfitting and improve generalization in future work.