Q i MP at S em E val-2025 Task 11: Optimizing Text-based Emotion Classification in E nglish Beyond Traditional Methods (2025)

Abstract: As human-machine interactions become increasingly natural through text, accurate emotion recognition is essential. Detecting emotions provides valuable insights across various applications. In this paper, we present our approach for SemEval-2025 Task 11, Track A, which focuses on multi-label text-based detection of perceived emotions. Our system was designed for and tested on English language text. To classify emotions present in text snippets, we initially experimented with traditional techniques such as Logistic Regression, Gradient Boosting, and SVM. We then explored state-of-the-art LLMs (OpenAI o1 and DeepSeek V3) before developing our final system, a fine-tuned Transformer-based model. Our best-performing approach employs an ensemble of fine-tuned DeBERTa-large instances with multiple seeds, optimized using Optuna and StratifiedKFold cross-validation. This approach achieves an F1-score of 0.75, demonstrating promising results with room for further improvement.

## introduction semeval-2025 task 11 @xcite focuses on detecting perceived emotion in short text snippets. emotion detection has valuable applications in fields such as healthcare, education, finance @xcite , customer service, and other applied domains @xcite . our participation in this task was centered on the english language, where we explored various approaches before ultimately adopting debertalarge @xcite . this task provides insights into how both humans and machines perceive emotions in written text, particularly in context-free scenarios. our focus is on track a, which involves identifying the presence of emotions in sentences without their intensity. as multimodal research continues to advance, integrating text, visual, and audio modalities, text remains a critical component of emotion-recognition systems @xcite . improving emotion detection in text-based snippets not only enhances classification accuracy but also better informs the selection of key triggers for emotional shifts. furthermore, text-based communication remains the dominant form of online interaction. we approach this problem with a plethora of natural language processing techniques, from traditional methods to modern large language models (llms) such as deepseek v3 @xcite and openai o1 @xcite . the dataset for english language for this task is heavily imbalanced, making classification challenging. performance is evaluated using the f1-score, which allows for a balanced identification of emotion's presence and absence. we define traditional methods as those relying on rule-based, feature-based, lexicon-based, or static-embedding classifiers, such as models using tf-idf, bag-of-words, or pre-trained word embeddings like glove, combined with algorithms such as svm, logistic regression, mlp, or gradient boosting. beyond traditional refers to transformerbased, pre-trained, or prompt-based models-such as bert, deberta, openai o1, and deepseek v3, which leverage deep contextual representations, large-scale transfer learning, and, in the case of models like openai o1 and deepseek v3, zeroshot inference capabilities. @xcite to streamline experimentation, we initially used bert-base @xcite before applying our optimizations to our best-performing model, deberta-large. this allowed us to iterate efficiently while ensuring improvements translated effectively to our final model. throughout this task, we encountered several counterintuitive challenges. these challenges are discussed in section 3, along with our hypotheses regarding their causes. our final system achieved an f1-score of 0.7537, placing us 27th out of 96 participating teams in the english track. an analysis of our model's performance reveals notable class detection disparities. anger, the least common emotion in our dataset, suffers from underdetection with the lowest recall (61.04%), meaning 38.96% of anger instances were missed. conversely, fear, the most prevalent emotion, exhibits over-prediction tendencies, achieving excellent recall (91.20%) but the lowest specificity (64.77%). despite these challenges, our model demonstrates strong overall performance, achieving multi-label subset accuracy of 47.04%, meaning the exact combination of emotions was correctly predicted in nearly half of all cases. ## conclusion our proposed approach, leveraging debertalarge with multiple seeds, ensemble methods, optuna hyperparameter optimization, and stratified-kfold cross-validation, achieved an f1-score of 0.7537, surpassing the baseline provided by the task organizers. while our model demonstrated strong performance, our final ranking suggests room for improvement. this study explored traditional emotion recognition techniques, llms, and transformer-based approaches, highlighting the successful application of advanced ensemble methods to task 11 at semeval-2025. future improvements may focus on exploring computationally efficient alternatives to debertalarge for better scalability, expanding and balancing training data to reduce class imbalance issues, and implementing more robust labeling strategies, accounting for shortcomings of the current model discussed in section 5. while our research explored several widely used approaches, we recognize that other promising techniques could achieve comparable or better results, potentially with lower computational costs. these include lexicon-based approaches (nrc vad @xcite , senticnet @xcite ), alternative transformer models (siebert ( rozado et al., 2022) lei et al., 2024) ), more traditional machine learning methods (lstm @xcite ), and explainability techniques like shap @xcite . although our initial attempts at re-weighting and naïve augmentation did not yield significant results, future work could explore multi-label aware oversampling (e.g., mlsmote @xcite or similarity-based oversampling (karaman et al., 2024)), adaptive batch-level strategies, such as loss-driven batch selection @xcite , and deferred re-weighting schedules @xcite , that apply stronger class weights only after an initial warm-up phase, to mitigate the severe class imbalance. moreover, addressing dataset biases remains crucial. generalization is particularly challenging in english, where linguistic and cultural diversity influences both text production and emotional perception. future work should extend beyond english to encompass multilingual and cross-cultural perspectives, incorporating sociolinguistic and anthropological insights. additionally, integrating non-verbal elements like emojis into text-based emotion recognition may improve model robustness and real-world applicability.