xiacui at S em E val-2025 Task 11: Addressing Data Imbalance in Transformer-Based Multi-Label Emotion Detection with Weighted Loss (2025)

Abstract: This paper explores the application of a simplified weighted loss function to Transformer-based models for multi-label emotion detection in SemEval-2025 Shared Task 11. Our approach addresses data imbalance by dynamically adjusting class weights, thereby enhancing performance on minority emotion classes without the computational burden of traditional resampling methods. We evaluate BERT, RoBERTa, and BART on the BRIGHTER dataset, using evaluation metrics such as Micro F1, Macro F1, ROC-AUC, Accuracy, and Jaccard similarity coefficients. The results demonstrate that the weighted loss function improves performance on high-frequency emotion classes but shows limited impact on minority classes. These findings underscore both the effectiveness and the challenges of applying this approach to imbalanced multi-label emotion detection.

## introduction emotions conveyed through language can manifest via facial expressions, speech, and written text. unlike facial expressions, which can display a wide spectrum of emotions, or speech, which can express multiple emotions while retaining the same verbal content, emotions in text are particularly challenging to analyse and interpret @xcite . textual emotions are especially difficult to analyze due to their subtlety, complexity, and inherent ambiguity in how emotions are expressed. for machines, detecting emotions in text is a particularly formidable task, given the nuanced and often context-dependent nature of emotional language @xcite . developing an artificial intelligence (ai) system capable of identifying emotional content is even more challenging, as the machine's role is primarily to support human interpretation, offering insights rather than definitive conclusions-especially given the subjectivity of emotional perception and the likelihood of disagreement among human interpreters @xcite . this task becomes even more complicated with short texts, which often communicate a blend of emotions simultaneously, making it a critical area of study for advancing both ai and emotional intelligence research. this paper focuses on developing a system capable of identifying the presence of one or more emotions within short texts as a multi-label classification problem, aiming to advance the accuracy and utility of automated emotion detection. to achieve this, we develop a transformer-based model, a class of architectures that has recently gained prominence in natural language processing (nlp) for tasks such as sentiment analysis and machine translation. while prior research has extensively explored traditional machine learning (ml) and deep learning (dl) approaches, including convolutional neural networks (cnns) and long short-term memory (lstm) networks @xcite , this work focuses on the potential of transformers to address the unique challenges of multi-label emotion detection. data imbalance is a common challenge in classification tasks, often leading to poor performance in detecting minority classes. traditional approaches to address this issue typically involve resampling techniques, such as oversampling minority classes or undersampling majority classes @xcite . however, in multilabel classification tasks, where a single instance can simultaneously belong to both minority and majority classes, these methods become less effective. resampling in such scenarios complicates the mapping of instances to multiple output labels, as increasing or decreasing the occurrence of one label may inadvertently affect others @xcite . in this paper, we investigate the application and impact of class weighting as an alternative strategy for handling data imbalance in multi-label classification, particularly when applied to transformer-based models. a key list of our contributions is summarised as follows: ## background significant amounts of studies have been carried out on automatic emotion recognition to help the process of manually checking emotions for various purposes. previous studies have explored various machine learning and deep learning approaches to address the challenges of capturing multiple emotions in concise text. @xcite proposed a framework using binary relevance and classifier chains to handle multi-label classification, demonstrating its effectiveness on social media datasets. @xcite proposed leveraging cnn and lstm to tackle emotion detection, highlighting the potential of deep learning models in capturing complex emotional cues in text. transformer-based models such as bert, roberta, and bart have significantly advanced the field of emotion detection by leveraging their ability to capture contextual relationships in text. bert (bidirectional encoder representations from transformers) @xcite introduced a bidirectional attention mechanism, enabling it to understand the context of words from both left and right, which is particularly useful for detecting nuanced emotional cues in text. roberta (robustly optimized bert pretraining approach) @xcite builds on bert by optimizing its pretraining process, using larger datasets and longer training times, resulting in improved performance on emotion classification tasks. bart (bidirectional and auto-regressive transformers) @xcite , on the other hand, combines bidirectional and autoregressive pretraining objectives, making it effective for both understanding and generating emotionally rich text. these models or variants have been widely adopted in emotion detection due to their ability to handle complex linguistic patterns and their state-of-the-art performance on benchmark datasets @xcite @xcite . many methods fail to address label imbalance, leading to biased models @xcite @xcite . @xcite proposed a multimodal transformer-based approach for multi-label emotion detection, modeling both modality and label dependencies. similar label dependency techniques have also been proposed in several previous studies @xcite @xcite . however, emotions are not always correlated and can conflict (e.g., joy and anger), making dependency assumptions problematic. their model mitigates imbalance with a weighted loss function and a conditional set generation mechanism, but is computationally expensive due to beam search and permutation-based training. in this paper, we employ a similar weighted loss function that adjusts class weights directly, reducing complexity and resource demands, to the transformer-based models. ## methods we consider the emotion detection task as a multilabel classification problem. multi-label classification involves assigning multiple labels to each input instance. section 3.1, we provide the definition of the problem with notation. unlike traditional classification tasks, where each input is associated with a single label, multi-label classification requires the model to predict a subset of labels from the label space. section 3.2, we introduce the transformerbased architecture including critical components and how we apply instance weighting in the loss function for the defined multi-label classification problem. ## experimental setup the task is framed as a multi-label classification problem. the proposed system is developed using the brighter dataset @xcite , which includes multi-label emotion annotations across 28 languages. to address the challenge of data imbalance, we focus our experiments on the english subset as a representative example. the proposed approach is designed to be generalizable and can be extended to other languages, such as german, with minimal adaptation.