Tewodros at S em E val-2025 Task 11: Multilingual Emotion Intensity Detection using Small Language Models (2025)

Abstract: Emotions play a fundamental role in the decision-making process, shaping human actions across diverse disciplines. The extensive usage of emotion intensity detection approaches has generated substantial research interest during the last few years. Efficient multi-label emotion intensity detection remains unsatisfactory even for high-resource languages, with a substantial performance gap among well-resourced and under-resourced languages. Team {textbf{Tewodros}} participated in SemEval-2025 Task 11, Track B, focusing on detecting text-based emotion intensity. Our work involved multi-label emotion intensity detection across three languages: Amharic, English, and Spanish, using the (afro-xlmr-large-76L), (DeBERTa-v3-base), and (BERT-base-Spanish-wwm-uncased) models. The models achieved an average F1 score of 0.6503 for Amharic, 0.5943 for English, and an accuracy score of 0.6228 for Spanish. These results demonstrate the effectiveness of our models in capturing emotion intensity across multiple languages.

## introduction the modern digital era allows users to freely express their feelings, attitudes, and opinions through websites, microblogs, and social media platforms @xcite @xcite @xcite . this has increased interest in extracting user sentiments and emotions towards events for different purposes, including social media monitoring, product analysis, political promotions, customer feedback analysis, and marketing research @xcite @xcite @xcite . language is not just a medium of communication but also a way to express emotions, sentiments, and their intensity @xcite . the task of emotion classification within nlp stands as a complex challenge that involves assigning emotional labels to texts to reveal the precise mental state of writers/users @xcite @xcite . the challenge of emotion detection exceeds sentiment analysis @xcite because emotions span a wide spectrum and single texts can contain multiple feelings, while cultural and linguistic differences impact interpretation and transferability @xcite @xcite . multi-label emotion classification (mlec) @xcite @xcite involves analyzing complete emotional expressions within written content, thereby demonstrating its value as a complex yet fundamental natural language processing (nlp) task because one text may convey various simultaneous emotions. multi-label classification differs from single-label by enabling instances to possess different mixture levels of emotions from the complete emotion set @xcite . different machine learning (ml) algorithms @xcite such as naive bayes (nb), k-nearest neighbors, and support vector machines (svm) have been applied to resolve emotion classification problems, often incorporating linguistic and contextual features for better performance. the detection method of emotions in coarsegrained systems only identifies emotions and ignores their intensity level. traditional emotion classification approaches can determine whether a sentence expresses happiness or sadness but do not quantify how intense the emotion is @xcite . fine-grained emotion intensity detection aims to capture these variations, which is crucial for distinguishing sentences with the same emotion but different intensities. detecting emotion intensity requires identifying intensity words and other linguistic factors that influence the degree of emotion expressed in the text @xcite . despite growing research in this domain, most studies focus on data-rich languages due to the unavailability of datasets for data-scarce languages @xcite . this gap has led to limited advancements in emotion intensity detection for languages spoken in linguistically diverse regions such as africa and asia, which together account for over 4,000 languages @xcite . the workshop organizers launched semeval-2025 task 11: bridging the gap in text-based emotion detection (track b) as a response to address the current research gap. the task serves as a research platform that allows scientists and researchers to build and evaluate multi-label emotion intensity detection methods while targeting the gap between languages with varying linguistic resources. ## literature review the process of emotion classification (ec) analyzes verbal and nonverbal indicators, including text and facial expressions, together with body language and speech, to determine a subject's emotional state @xcite . the main goal of ec consists of detecting emotions through categorization across text expressions, including anger, disgust, and fear, together with happiness, sadness, and surprise emotions. psychologists argue about essential emotions, but different psychological frameworks propose between six and twenty emotions as core @xcite @xcite . several dimensional models of emotion have been developed, yet only a limited number persist as dominant frameworks. the circumplex model @xcite features eight emotional groups established through 28 emotion words, and the positive-and negative-activation (pana) model demonstrates an emotion ranging from high positive to low positive activation @xcite . most researchers in emotional-based research continue to use ekman's model @xcite to divide emotions into six core categories that include joy, surprise, happiness, anger, sadness, disgust, and also fear @xcite . the research community has created text mining solutions to analyze emotions on social media @xcite , especially twitter, through naïve bayes machine learning strategies @xcite . the tagging of emotions in online news through multi-source systems is addressed by researchers who introduce a two-layer logistic regression model in their approach @xcite . research in emotion classification underwent several developments by integrating word and character n-grams (mohammad, 2012) with sentiment and emotion lexicons @xcite as well as neural network models @xcite . there has been an increased emphasis in nlp research @xcite that incorporates large language models (llms) for emotion identification, mainly within data-rich and data-scarce languages @xcite @xcite . emobench represents a new assessment method that tests llms for detecting emotional origins across english and chinese languages, as described by @xcite . @xcite proposed emollms, fine-tuning open-source llms for affective analysis and emotion prediction. the researchers @xcite applied mt5 model fine-tuning before conducting evaluations of both flan and chatgpt through few-shot prompting for multi-label emotion classification. mostly used emotion classification datasets exist, including: @xcite ) semeval-2024 task 3, @xcite ) semeval task 11, @xcite brighter, @xcite multilingual emotion prediction (xlm-emo), @xcite wassa 2023 shared task 2, @xcite romanian emotion dataset (redv2), @xcite goemotions, and balanced multi-label emotional tweets (bmet) @xcite . the detection of emotion intensity in text @xcite has become a core capability of the multi-label emotion classification (mlec) process to identify emotional levels @xcite . the semeval-2018 task 1 @xcite together with the multimodal multilabel emotion, intensity, and sentiment dialogue dataset (meisd) @xcite and @xcite demonstrate examples of emotion classification. the development of emotion analysis through deep learning techniques signifies an increasing preference for sophisticated emotion detection methods. distinguishing sentences from the same emotion category requires examining their emotional strength because intensity plays a vital role in these cases @xcite @xcite . the intensity of individual words stands as an approach to measure sentence-level intensity since words that share related meanings push emotional strength either upward or downward @xcite . the detection of emotional intensity in text documents remains a subject that researchers have studied comparatively less than sentiment intensity @xcite @xcite @xcite . this gap in research has been addressed through several notable studies. the semi-supervised approach defines an adjective intensity scale through contextual analysis of highintensity words @xcite . sciences have studied automated approaches for emotional intensity tagging in sentences with wordnet affect alongside word sense disambiguation @xcite . these research techniques aim to establish quantitative methods that identify emotional intensity levels across a given textual content. the first major attempt to introduce emotion intensity annotation occurred when @xcite participated in semeval-2007 shared task competitions. the study employed a 0 to 100 continuous scale through which annotators rated emotions present in newspaper headlines. the development of rating emotion intensity at a granular level still encounters specific collection difficulties. the process faces major problems because different annotators tend to rate the same piece of text with substantially varying scores (one person assigned 79 while another only gave 62). researchers have made important progress in emotion classification, but their work mostly concentrates on high-resource languages @xcite @xcite , whereas the investigation of emotion detection within ethiopian languages along with other low-resource languages remains scarce @xcite . benchmark datasets primarily emerge for english together with popular languages, which impede proper generalization of research outcomes across diverse linguistic settings @xcite @xcite . the current emotion datasets derive from single-source text corpora, which affect their representativeness, while state-of-the-art llms for both multi-label and multilingual emotion classification remain underexplored domains @xcite . our team extends this research on track b of semeval 2025 task 11 to address the existing gaps in emotion intensity detection. the assessment task requires emotion annotation with perceived levels of intensity, which range from 0 for no emotion to 3 for high intensity according to sadness, fear, and so on. to detect emotion intensity in highand low-resource languages, this work investigates methods and datasets and demonstrates results. extending from the extant literature, it is our intention to contribute by culturally informed approaches to improve the efficiency of this task. ## methodology the research methodology included data preprocessing, feature extraction, and text classification procedures on textual data collected from diverse sources in english, spanish, and amharic languages. the data was divided into three distinct sets: training, development, and testing. each sample was labeled with one of six emotional states: surprise, anger, joy, fear, disgust, joy, and sadness. we loaded the data through pandas to examine its structure while confirming essential attributes exist. we determined emotion frequencies through value_count functions. the feature engineering process involved countvectorizer's bigram tokenization technique alongside a selection of the top 90 bigrams per language for improving input representation. the model design included provisions that allowed it to detect important linguistic patterns regardless of the text's language. the model architecture incorporated afro-xlmr-large-76l for amharic, deberta-v3-base for english, and bert-base-spanish-wwm-uncased for spanish. different tokenization methods applied to textual data through model-specific tokenizers resulted in hugging face datasets for training purposes. the training speed was accelerated by using @xmath0. the training process employed 16 samples per batch and set the learning rate at 2e-5. a trainer class and crossentropyloss module enabled the computation of class weights for balancing class distribution in the training process. the training process lasted for 20 epochs through early stopping to avoid overfitting. multiple performance metrics, including accuracy, macro f1-score, and per-label f1-score, served to assess model performance during evaluation. ## dataset analysis the semeval 2025 task 11 dataset, track b (emotion intensity detection), originates from the paper @xcite . the dataset provides labeled intensity scores for joy, sadness, fear, anger, surprise, and disgust. our analysis focuses on examining three languages: english, spanish, and amharic, which represent a mix of high-resource and low-resource linguistic contexts. the dataset is compiled from diverse sources, including news portals x (formerly twitter), youtube, and facebook, capturing a mix of formal and informal emotional expressions. social media data poses distinct challenges, such as the use of abbreviations, informal phrasing, and extreme variations in emotional intensity, while news articles typically present structured and neutral emotional tones. ## experimental setup experimental setup our experiment utilized a training-validation split on the dataset to ensure a balanced distribution of emotion intensity labels for the six perceived emotions: joy, sadness, fear, anger, surprise, and disgust. language-specific preprocessing techniques were applied to address platform-specific variations and the models were trained in a high-performance computing environment for efficient training. for model training, we fine-tuned afro-xlmr-large-76l, deberta-v3-base, and bert-base-spanish-wwm-uncased, leveraging their capabilities in multilingual and language-specific emotion detection. each model was initialized with pre-trained weights and finetuned on the dataset to capture nuanced emotional patterns across the six emotion categories. to evaluate model performance, we developed a comprehensive evaluation pipeline, using accuracy, f1score, and pearson correlation as key metrics to assess the effectiveness of emotion intensity detection. generalization was tested on unseen test data from different platforms and contexts. additionally, we also integrated language-agnostic embeddings to enhance robustness across multiple languages. model hyperparameters were optimized through experimental tuning to balance precision and computational efficiency in detecting joy, sadness, fear, anger, surprise, and disgust across varied textual contexts. ## results our evaluation focuses on measuring the model's effectiveness in detecting emotion intensity across the three languages: english, spanish, and amharic. we used pearson correlation (r) as a key metric to evaluate the alignment between predicted emotion intensity scores and gold-standard annotations. ## discussion the f1 scores of afroxlmr-base reached their highest levels across all languages, especially amharic and spanish, which proves its powerful multi-language capabilities. deberta succeeded within english datasets but experienced difficulties working with languages with fewer available resources. the bert-based spanish model performed effectively for spanish tasks but displayed a weak translation ability between languages. the models' performance suffered mostly because of class imbalance when identifying rare emotions, including disgust and surprise. transformer models demonstrated superior performance than traditional deep learning approaches, as afroxlmr achieved the best results in precision and recall metrics. the research agenda should encompass emotion-based pre-training and approaches to address imbalanced classes. ## conclusion this research work used transformer models that were fine-tuned specifically for amharic, english, and spanish to understand emotional intensity across the three languages. the research process included text preprocessing along with feature engineering before training afro-xlmr-large-76l, deberta-v3-base, and bert-base-spanish-wwm-uncased models. the tested models exhibited superior performance for detecting emotion intensity variations in amharic with an f1 score of 0.6503, followed by spanish with an f1 score of 0.6228, and english with an f1 score of 0.5943. while our models perform well across multiple languages, detection of underrepresented emotions and low-resource languages remains a challenge. future work should focus on exploring data augmentation techniques and developing adaptation frameworks to achieve better results with multilingual fusion approaches. these advancements will further enhance emotion intensity detection, ensuring more robust and accurate predictions across diverse linguistic and cultural contexts.