NITK - VITAL at S em E val-2025 Task 11: Focal- R o BERT a: Addressing Class Imbalance in Multi-Label Emotion Classification (2025)

Abstract: This paper presents our approach to SemEval Task 11, which focuses on multi-label emotion detection in English textual data. We experimented with multiple methodologies, including traditional machine learning models, deep learning architectures, and transformer-based models. Our best-performing approach employed RoBERTa with focal loss, which effectively mitigated class imbalances and achieved a macro F1-score of 0.7563, outperforming other techniques. Comparative analyses between different embedding strategies, such as TF-IDF, BERT, and MiniLM, revealed that transformer-based models consistently provided superior performance. The results demonstrate the effectiveness of focal loss in handling highly skewed emotion distributions. Our system contributes to advancing multi-label emotion detection by leveraging robust pre-trained models and loss function optimization.

## introduction understanding emotions in text is a crucial aspect of natural language processing (nlp) with applications in sentiment analysis, mental health monitoring, and human-computer interaction. emotions are inherently complex, nuanced, and subjective, making their automatic detection a challenging task. the semeval 2024 task 11 focuses on multi-label emotion detection, where the goal is to determine the perceived emotions conveyed by a speaker in a given text snippet. this task is particularly challenging due to variations in individual emotional expression, cultural differences, and the ambiguity of language. @xcite in this paper, we present our approach for track a (english language), where we predict whether a given text snippet expresses one or more of the following emotions: joy, sadness, fear, anger, or surprise. our method leverages deep learning-based models, incorporating pre-trained transformer architectures such as bert, roberta, and t5, along with fine-tuning strategies tailored for multi-label classification. through our participation in the task, we observed several key insights. firstly, contextual embeddings significantly improve performance compared to traditional machine learning methods. secondly, class imbalance poses a challenge, as certain emotions are underrepresented in the dataset, leading to biased predictions. despite these challenges, our system achieved competitive performance, ranking among the top-performing models in the competition. the rest of this paper is structured as follows: section 2 discusses related work in emotion detection, section 3 details our system architecture, section 4 presents the experimental setup, section 5 analyzes the results, and section 6 concludes with future directions. ## related works a key challenge in multi-label emotion detection is class imbalance, where certain emotions are underrepresented. to address this, lin et al. @xcite introduced focal loss, which dynamically adjusts the cross-entropy loss to focus on harder examples while down-weighting easier ones. though originally developed for object detection, its principle of emphasizing difficult samples has inspired applications in other domains. jiang et al. @xcite , for instance, extended this idea to frequency components in image reconstruction. this relevance extends to emotion detection, where rare emotions are similarly difficult to capture and benefit from targeted loss optimization. in textual emotion detection, researchers have explored methods that incorporate emotion interdependencies. chochlakis et al. @xcite leveraged label correlations through pairwise constraints as regularization terms, helping models better capture co-occurring emotions. alhuzali and ananiadou @xcite proposed spanemo, reframing emotion classification as a span-prediction task to capture overlapping emotional cues within text. additionally, choudhary and chakraborty (choudhary and chakraborty, 2020) combined deep learning features with handcrafted features to improve the granularity of emotion recognition, showing that hybrid approaches can enhance model interpretability and accuracy. in multilingual and code-mixed contexts, gupta et al. @xcite introduced sentimoji, a dataset designed for multi-label emotion and sentiment classification in diverse linguistic settings. their work highlights the complexities introduced by language mixing and the need for adaptable models. although transformer-based architectures like roberta have shown promise in emotion detection tasks, few studies have explored the integration of focal loss with such models. our approach distinguishes itself by combining focal loss with a fine-tuned roberta and a multi-head attention layer, leading to improved detection of infrequent emotions and better overall performance. ## data the dataset used for this task consists of 2768 text samples, each labeled with one or more emotions. the dataset contains 7 columns, including the text snippet and five binary labels representing the presence or absence of the following emotions: joy, sadness, fear, anger, and surprise. the text data serves as the primary input for classification. the evaluation metric for this task is macro-averaged f1-score (f1-macro), which ensures balanced performance across all emotion categories, regardless of class imbalance. @xcite 4 system overview ## results and analysis we evaluated our models using the macro-f1 score. ## conclusion and future work this work explored traditional machine learning, deep learning, and transformer-based models for multi-label emotion detection. pre-trained transformers like bert and roberta notably boosted performance, with focal loss effectively addressing class imbalance-leading to the highest macro f1-score with roberta. future work will focus on data augmentation methods, such as back-translation and synonym replacement, to improve generalization and handle class imbalance. exploring ensemble approaches that combine multiple transformer models may further enhance performance. finally, deploying the best-performing model in real-world applications like mental health monitoring could provide valuable insights.