LATE - GIL - NLP at S em E val-2025 Task 11: Multi-Language Emotion Detection and Intensity Classification Using Transformer Models with Optimized Loss Functions for Imbalanced Data (2025)

Abstract: This paper addresses our approach to Task 11 (Track A and B) at the SemEval-2025, which focuses on the challenge of multilingual emotion detection in text, specifically identifying perceived emotions. The task is divided into tracks, we participated in two tracks: Track A, involving multilabel emotion detection, and Track B, which extends this to predicting emotion intensity on an ordinal scale. Addressing the challenges of imbalanced data and linguistic diversity, we propose a robust approach using pre-trained language models, fine-tuned with techniques such as extensive and deep hyperparameter optimization, along with loss function combinations to improve performance on imbalanced datasets and underrepresented languages. Our results demonstrate strong performance on Track A, particularly in low-resource languages such as Tigrinya (ranked 2nd), Igbo (ranked 3rd), and Oromo (ranked 4th). This work offers a scalable framework for emotion detection with applications in cross-cultural communication and human-computer interaction.

## introduction emotions are central to human communication, yet they are inherently complex and often difficult to express or interpret accurately through text, @xcite . while we regularly communicate our emotions, the way people perceive emotions in a text can be highly subjective, influenced by individual experiences, cultural backgrounds, and context, @xcite . the task 11: bridging the gap in text-based emotion detection, @xcite , focuses on this challenge, dividing it into two tracks: track a, which involves multilabel emotion detection, and track b, which extends this to predicting emotion intensity on an ordinal scale (level 0 to 3). the task is particularly challenging due to imbalanced data and the diversity of languages ( 28 for track a and 11 for track b), each with unique linguistic and cultural nuances, @xcite . this work addresses these challenges by leveraging pre-trained language models from hugging-face, fine-tuned through an extensive search for optimal hyperparameters and custom loss functions tailored to emotion detection. our approach combines models with advanced techniques such as cross entropy loss, focal loss, and label smoothing, improving f1 score and pearson correlation metrics, especially for underrepresented languages and imbalanced datasets. key contributions include: our method provides a robust framework for multilingual emotion detection. it has potential applications in cross-cultural communication and human-computer interaction. the main codes used to address this task are available in the github foot_0 repository of our research group. ## related work emotion detection and classification have been widely studied in natural language processing (nlp), with significant progress driven by deep learning architectures, transfer learning, and multimodal approaches @xcite . early methods relied on lexiconbased techniques and traditional machine learning models, but recent advancements leverage largescale pre-trained models and hybrid neural architectures to capture complex linguistic patterns more effectively. the field of emotion detection has evolved from lexicon-based approaches to modern deep-learning models capable of capturing contextual nuances. the adoption of deep learning significantly improved performance by allowing models to learn representations directly from data. early neural network architectures, particularly recurrent models like long short-term memory (lstm) networks, enhanced the ability to capture sequential dependencies in text. however, these models required substantial labeled data and computational resources. the introduction of transformer-based models, such as bert, roberta, and xlm-r, further advanced emotion detection by leveraging self-attention mechanisms to model complex linguistic structures, @xcite . pretrained on large-scale corpora, these models set new benchmarks in emotion detection, outperforming earlier architectures. more recently, instructiontuned models, such as gpt-4 and t5, have shown promise in classifying emotions in ambiguous or contextually complex text, @xcite . while multimodal approaches integrating textual, auditory, and visual data have gained traction, textbased models remain widely used due to their efficiency and accessibility. while emotion detection focuses on identifying whether an emotion is present in a given text, an equally important challenge is determining its intensity. emotion intensity classification has gained increasing attention in nlp, with notable advancements in deep-learning architectures and transfer learning. the advent of large-scale pre-trained transformers has further advanced emotion intensity classification. models such as bert and its derivatives have been fine-tuned on emotionlabeled datasets, achieving state-of-the-art results, @xcite . more recently, frameworks such as deepemotex have demonstrated the effectiveness of fine-tuned transformer-based models for multi-class emotion classification, significantly outperforming conventional deep learning models, @xcite . a major challenge in emotion classification is class imbalance, where certain emotions are signifi-cantly underrepresented in datasets. to address the imbalance, recent work has explored hierarchical classification and weighted loss functions to improve model performance in multilingual settings. in the wassa 2024 shared task, vázquez-osorio et al. @xcite proposed a two-stage hierarchical classification approach. the first stage classified tweets into four broad categories, while the second stage further distinguished between underrepresented emotions. additionally, they employed focalloss and weighted crossen-tropyloss to emphasize minority classes during training. their approach, implemented with a finetuned deberta-v3-large model, resulted in improved performance, ranking among the top 15 submissions. these findings highlight the effectiveness of hierarchical classification and adaptive loss functions in handling imbalanced emotion datasets. recent work on multilingual emotion detection @xcite highlights challenges such as class imbalance and low-resourced languages. therefore, our approach introduces custom loss functions for imbalanced data, which is critical for some languages with underrepresented data. ## system overview our system is based on fine-tuning pre-trained transformer models for multilingual emotion recognition and intensity prediction. we designed a twostage pipeline tailored to the task's requirements of track a and track b. in track a, we focused on detecting the presence of emotions in text through multilabel classification. in track b, we extended this setup by incorporating an additional step to predict the intensity of each detected emotion. all models were language-specific and selected based on empirical performance. for track a, we fine-tuned a multilabel binary classification model for each language to determine whether each emotion was present in the text. track b expanded on track a's process, incorporating a two-step approach to predict emotion intensity. the initial step, the same as in track a, employed the multilabel classification model to identify emotions. in the next phase, if an emotion was detected (label 1), a separate model estimated its intensity on a scale from 1 to 3. all our models were based on pre-trained transformer models, which were selected as the most suitable for each language. ## results macro f1 score was the official metric for track a, and as we can see, for both tracks, the macro f1 score and the pearson correlation vary significantly across languages. given that for each language, we fine-tuned a language-specific model, we can observe that the results are highly dependent on the base model. for hindi, english, russian, and spanish, the scores are considerably higher compared to languages like amharic, algerian arabic, or ukrainian. other factors to take into consideration that can influence the performance of the classification models include the size of the dataset and class imbalance (see appendix b for details on class distribution for track a). unlike top teams reported in the task paper @xcite , such as pai and chinchunmei, who rely on large llm ensembles, contrastive learning, and prompt engineering, our approach focuses on robust fine-tuning of pre-trained models using hyperparameter optimization and tailored loss functions for imbalanced and low-resource data. without external augmentation or instruction tuning, our method achieved competitive results, ranking in the top 10 in multiple languages and significantly outperforming baselines in 17 languages, highlighting the strength of our optimization-based strategy (see ## ethical considerations predicting perceived emotions and their intensity is inherently subjective and influenced by cultural and individual differences. biases may arise from the dataset, the annotation process, or the model itself. as noted in @xcite , the dataset focuses on perceived emotions (what most annotators believe the speaker may have felt and their intensity) rather than determining the true emotional state of the speaker. therefore, our prediction models should not be used for high-stakes decisions, nor should their outputs be interpreted as definitive assessments of the speaker's actual emotions or intensity. for more ethical considerations and details on the data annotation process, see @xcite . ## conclusion in this paper, we presented our approach to the emotion detection and intensity classification task at semeval-2025. our approach leveraged fine-tuned models based on pre-trained transformer models. we achieved our best results by incorporating some custom loss functions for certain languages and emotions, demonstrating their effectiveness in handling imbalanced data. however, the performance varied significantly across languages, underscoring the importance of further analyzing and exploring additional techniques and architectures.