Emotion Train at S em E val-2025 Task 11: Comparing Generative and Discriminative Models in Emotion Recognition (2025)

Abstract: The emotion recognition task has become increasingly popular as it has a wide range of applications in many fields, such as mental health, product management, and population mood state monitoring. SemEval 2025 Task 11 Track A framed the emotion recognition problem as a multi-label classification task. This paper presents our proposed system submissions in the following languages: English, Algerian and Moroccan Arabic, Brazilian and Mozambican Portuguese, German, Spanish, Nigerian-Pidgin, Russian, and Swedish. Here, we compare the emotion-detecting abilities of generative and discriminative pre-trained language models, exploring multiple approaches, including curriculum learning, in-context learning, and instruction and few-shot fine-tuning. We also propose an extended architecture method with a feature fusion technique enriched with emotion scores and a self-attention mechanism. We find that BERT-based models fine-tuned on data of a corresponding language achieve the best results across multiple languages for multi-label text-based emotion classification, outperforming both baseline and generative models.

## introduction the task of emotion recognition involves identifying emotions in text or speech. track a of semeval 2025 task 11 @xcite focuses on multi-label emotion recognition in social media texts across several languages. following the definition of the universal emotions introduced by @xcite , the task involves classifying the texts for the following six basic emotions: anger, disgust, fear, joy, sadness, and surprise. in this paper, we present our work submitted to the shared task for the following languages: english, algerian and moroccan arabic, brazilian and mozambican portuguese, german, spanish, nigerian-pidgin, russian, and swedish. large language models (llms) have achieved remarkable results on a wide range of applications @xcite . as these models become increasingly integrated into real-world settings covering diverse domains, llms are expected to exhibit human-like behaviour for proficient social interactions. this served as a motivation to include llms in our investigated approaches, comparing their performance to discriminative pre-trained language models (plms). accordingly, our approaches fall under two main tasks, classification and generation, as we work with both discriminative and generative plms. we explore a wide range of techniques, including zero-and few-shot prompting, as well as fine-tuning and few-shot fine-tuning. we also propose a novel approach to extending the bert architecture with feature fusion and selfattention, incorporating token-level emotion scores statistically calculated on the train set. in our conducted experiments, non-causal models demonstrated superiority over causal ones. we also observed that imbalanced data has a high impact on a model's performance, notably biasing outcomes toward the detection of 'fear' in the english setup. our code is available online. foot_0 2 related work emotion recognition. @xcite and @xcite addressed emotion recognition in the semeval challenges, tackling tasks such as affective text exploration, bridging emotional and sentiment aspects, and inferring speakers' emotions from tweets. more recently, @xcite examined an architecture with discourse-and speaker-aware modules within graph attention networks, which outperformed the state-of-the-art (sota) in the task of emotion recognition in conversations. nag et al. (2023) explored several deep learning techniques to address different emotional intelligence (ei) tasks, including emotion recognition. zhao et al. (2024) tackled the issue of catastrophic forgetting in llms, which was previously reported by luo et al. (2023) when integrating ei. plms have proven to be highly effective on various nlp benchmarks @xcite @xcite @xcite @xcite . with respect to the current task, we considered the following approaches: fine-tuning. recent advances have been made in fine-tuning approaches by adapting plms with minimal parameter updates. for example, peft @xcite and @xcite techniques significantly reduce the computational requirements while maintaining high performance. in-context learning. zero-and few-shot icl offer the advantage of not modifying the model parameters. @xcite 's survey provides a taxonomy of icl that demonstrates various ways to apply pre-trained language models in nlp tasks. @xcite highlight the effectiveness of few-shot icl reaching sota performance. few-shot fine-tuning. @xcite presented a method of few-shot fine-tuning that is between icl and full fine-tuning. their approach involves using a small number of labelled examples in the input during the fine-tuning stage (resembling few-shot learning). feature fusion with self-attention. the feature fusion (ff) method implies a combination of multiple feature representations, such as embeddings. recent works @xcite showed implementations of ff under self-attention that enhanced model performance in named entity recognition. @xcite explored the combination of self-attention and word-level features that improve speech emotion recognition. ## methodology in this section, we describe the system overview by examining both groups of approaches for classification and generation tasks in detail. ## results for discriminative models, we use bert-based language-specific models as well as xlm-r, covering fine-tuning, ffsa, and cl across all languages. for generative models, we conduct preliminary experiments on the english language, where the best setup was found to be using llama-3 along 5 https://huggingface.co/docs/bitsandbytes/ 6 for all prompt-related experiments, the model was prompted in english with a language-specific example. with prompt#1 (see appendix a.4). in ## discussion in order to further understand our experiment results on the english development set, we analysed confusion matrices of models with finetuning and feature fusion with self-attention (ffsa) approaches (see figures 7a and 7b in appendix a.6.1). we observed that both models are overfitting by choosing 'fear' in most of the con- we believe multiple factors could be affecting the performance of models across languages, including data imbalance, sentence length and dataset size. mozambican portuguese and algerian arabic demonstrate the lowest results, likely due to being low-resource languages with relatively small datasets. in contrast, for nigerian-pidgin, despite typically being a low-resource language, nigerian-pidgin xlm-r performs relatively well. we believe this could be due to being well-resourced in this set-up (see regarding error analysis for roberta, expected calibration error (ece) of the approaches of both fine-tuning and fine-tuning with curriculum learning and data preprocessing have 8.7% and 8.5% ece, respectively. this indicates that these models are well-calibrated but still have miscalibration. as for the feature fusion with self-attention approach, the model is on a threshold with 10.3% ece, meaning the model's confidence levels might be overconfident or underconfident, compared to actual outcomes. comparing selected predictions of those models in figures 8 9 10 in appendix a.6.2 show that fine-tuned roberta demonstrates some overconfidence with a high probability of incorrect labels, particularly when it comes to fear, possibly related to a data imbalance. ## conclusion this paper presents our contribution to the semeval-2025's multi-label text-based emotion recognition task. in our work, we compare the emotion-detecting abilities of causal and noncausal models along with investigating multiple techniques such as curriculum learning, instruction and few-shot fine-tuning, as well as feature fusion with emotion scores (ffsa). fine-tuned language-specific bert-based models and xlm-roberta-large gave the best results across multiple languages, outperforming baseline and generative models. for future work, we believe an interesting direction would be using data augmentation to address the lack of perceived emotion detection datasets. additionally, we can explore improving the ffsa method using emotion lexicons.