Exploration Lab IITK at S em E val-2025 Task 11: Bridging the Gap in Text-Based Emotion Detection (2025)

Abstract: This paper presents our approach to SemEval-2025 Task 11 (Track A): Bridging the Gap in Text-Based Emotion Detection, with a focus on multi-label emotion classification for the English dataset. Our methodology leverages an ensemble of transformer-based models, incorporating full fine-tuning along with additional classification layers to enhance predictive performance. Through extensive experimentation, we demonstrate that fine-tuning significantly improves emotion classification accuracy compared to baseline models. Furthermore, we provide an in-depth analysis of the dataset, highlighting key patterns and challenges. The study also evaluates the impact of ensemble modeling on performance, demonstrating its effectiveness in capturing nuanced emotional expressions. Finally, we outline potential directions for further refinement and domain-specific adaptations to enhance model robustness.

## introduction the increasing availability of electronic documents in the digital era has provided a valuable resource for analyzing human expressions and improving various applications. understanding this plays a crucial role in multiple domains, including user experience enhancement, social media analysis, mental health monitoring, and market research. increasing scholarly attention has been directed toward extracting user perspectives on various events by analyzing textual content. the computational identification and classification of opinions within text has been recognized as a fundamental step in data mining. researchers have traditionally focused on determining whether a given text conveys a positive, negative, or neutral stance toward a specific subject or product @xcite . more recently, studies have expanded to incorporate multidimensional emotional annotations @xcite @xcite capturing sentiments such as joy, fear, anger, etc. emotion expression in language is inherently nuanced and complex, presenting challenges for emotion recognition. despite its significance, research in this field has predominantly focused on highresource languages, leading to significant disparities in dataset availability and model performance for low-resource languages. to address this gap, the organizers of semeval-2025 task 11 have curated a specialized dataset, brighter @xcite to bridge further the gap in emotion recognition research in underrepresented languages, facilitating more inclusive and effective nlp models. it is a collection of multilabel emotion-annotated datasets spanning 28 languages. this dataset emphasizes low-resource languages from africa, asia, eastern europe, and latin america, incorporating diverse textual sources annotated by fluent speakers. the shared task consists of three tracks: multi-label emotion classification (track a), emotion intensity prediction (track b), and text cross-lingual emotion detection (track c). in this paper, we present our system developed for track a of the task @xcite , multilabel emotion classification. transformers have been proven to be the most successful in understanding the contextual and semantic information of the text @xcite . we selected top-performing models from preliminary experiments, including cardiffnlp/twitter-roberta-large-emotion-latest @xcite , samlowe/roberta-basegoemotions @xcite and emanuel/twitteremotion-deberta-v3-base @xcite , to build an ensemble leveraging these models, combining their strengths for improved emotion classification, and performed full fine-tuning. since these models had more emotion categories than our dataset, we incorporated additional classifier layers, adapting to our task. our system demonstrated good performance across track a. we achieved competitive results, ranking 42 on the english dataset among the 75 participating teams. dataset imbalance presents a challenge in these methods. robust methods using undersampling and oversampling techniques need to be used to overcome this challenge. further, extensive data analysis is always beneficial. please find the code herehttps://github.com/ tafazzul-nadeem/tbed-cs779-iitk . ## background the landscape of human emotions has been described through various taxonomies and frameworks. ekman @xcite ) identified six core emotions expressed through facial expressions that are universally recognized across cultures: joy, sadness, anger, surprise, disgust, and fear. later, finer-grained taxonomies have been developed, capturing a broader range of up to 600 emotions and employing machine learning to cluster emotion concepts @xcite . these frameworks highlight the complex, culturally influenced nature of emotions expressed through vocalization @xcite , music, and facial expressions. previous approaches to text-based emotion detection have primarily utilized machine learning (ml) techniques. for instance, wikarsa et al. and ameer et al. @xcite focused on multilabel emotion classification for code-mixed sms messages in roman urdu and english, employing classical ml methods (svm, j48, naive bayes, etc.) and deep learning models (lstm, cnn, etc.) on a new dataset. their results indicated that classical ml methods outperformed both ml and deep learning models. similarly, @xcite developed a model combining bi-lstm, self-attention, and cnn for emotion detection, finding that word embeddings significantly improved performance. their experiments in the isear, semeval-2018 @xcite , and semeval-2019 datasets demonstrated that the isear dataset produced the best precision and recall. in recent years, research on text-based emotion detection has increasingly utilized transformer-based pre-trained language models. for instance, acheampong et al. (acheampong et al., 2020) conducted comparative analyses of models such as bert @xcite , roberta @xcite , distilbert @xcite , and xl-net @xcite for emotion recognition using the isear dataset. asalah et al. @xcite proposed an ensemble deep learning approach for emotion detection in textual conversations, cnn-based model and transformer-based models, including bert, roberta, and xlnet. they utilize hard and weighted majority voting methods to enhance prediction accuracy. their method demonstrates superior performance, achieving a micro-averaged f1-score of 77.07% on the semeval-2019 task 3: emocontext dataset (chatterjee et al., 2019), outperforming previous baseline results. we performed experiments with the track a dataset @xcite , where the training set contains 2768 samples with five binary emotion labels (joy, sadness, fear, anger, and surprise). dev set contains 116 samples, and 2767 samples are available for inference. ## analysis the co-occurrence matrix of the labels for the english dataset is plotted to gain insights about the correlation between the labels, as shown in figure 1 . we have also visualized the box plot for the number of @xmath0. ## proposed approach since transformer-based models have shown promising results in classification tasks lately, we decided to first find some suitable models for the multilabel classification task. from recent works @xcite @xcite @xcite , we shortlisted the following huggingface models, ensembled them, and fully fine-tuned the ensemble: cardiffnlp/twitter-roberta-large-emotion-latest @xcite , samlowe/roberta-base-go_emotions @xcite and emanuel/twitteremotion-deberta-v3-base @xcite . the ensemble model, along with the standalone parts, was evaluated on the training data for the english language with full fine-tuning. the results are shown in ## experiments the following sections provide a comprehensive overview of our model development process and the final model used for multi-label emotion detection. the first section traces the evolution of our approach, detailing the various models and finetuning strategies we experimented with. this includes trials with both smaller and larger models, full fine-tuning, classifier adaptations, and specialized training settings such as entailment-based approaches. the second section focuses on our final approach, which represents the culmination of our iterative experimentation. we describe its architecture, training methodology, and the rationale behind selecting this configuration as our best performing system. the final model integrates insights gained from our earlier experiments, leveraging an ensemble of fine-tuned models to achieve optimal performance. ## results the results of our experiments are summarized in in our final approach, to further boost performance, we created an ensemble of pre-trained models trained on different emotion datasets and fully fine-tuned the ensemble with added classifier layers at the top. the models used were cardiffnlp/twitterroberta-large-emotion-latest, samlowe/robertabase-go_emotions, and emanuel/twitter-emotiondeberta-v3-base. the ensemble achieved the best overall performance with a macro-f1 of 0.7344 in the test dataset, demonstrating the advantage of leveraging multiple models for emotion classification. ## conclusion in this challenge, we explored various transformerbased models for multi-label emotion detection, evaluating their performance on track a using multiple fine-tuning strategies and model ensembles. our results demonstrate that model architecture and fine-tuning approach significantly impact performance. our findings highlight the advantages of leveraging multiple pre-trained models and ensembling techniques for emotion classification tasks. future work could explore additional architectures, data augmentation methods, and domain adaptation techniques to further enhance model performance and generalizability across different datasets. our findings show that fine-tuning smaller models can sometimes perform as well as, or even better than, larger models. this means it is possible to improve accuracy without the requirement of very big models like llms.