Team A at S em E val-2025 Task 11: Breaking Language Barriers in Emotion Detection with Multilingual Models (2025)

Abstract: This paper describes the system submitted by Team A to SemEval 2025 Task 11, “Bridging the Gap in Text-Based Emotion Detection.” The task involved identifying the perceived emotion of a speaker from text snippets, with each instance annotated with one of six emotions: joy, sadness, fear, anger, surprise, or disgust. A dataset provided by the task organizers served as the foundation for training and evaluating our models. Among the various approaches explored, the best performance was achieved using multilingual embeddings combined with a fully connected layer. This paper details the system architecture, discusses experimental results, and highlights the advantages of leveraging multilingual representations for robust emotion detection in text.

## introduction human emotions are intricate and multidimensional, resisting simplistic classification due to their fluid, overlapping nature. as @xcite noted, affective states rarely occur in isola-tion; they coalesce and evolve dynamically, challenging reductionist labelling approaches. this complexity underpins multi-label emotion detection, where texts or behaviours often encode layered sentiments @xcite . the benefits of accurately deciphering these nuances span domains from early mental health screening and tailored interventions @xcite to enhanced consumer sentiment analysis in ai systems @xcite ). yet, current recognition systems often treat emotions as mutually exclusive, contrary to psychological frameworks; works by @xcite and @xcite view emotions as interconnected constructs with gradational intensities, a perspective supported by @xcite , who shows that joy and love correlate more strongly than, say, anger and sadness. another gap is the treatment of emotional intensity, which ranges from subtle to profound expressions @xcite . most systems neglect these gradations by focusing on binary classifications, limiting real-world applicability in clinical or market settings. moreover, linguistic and cultural disparities evident in divergent emotion lexicons and display rules @xcite render monolingual models inadequate, with culture specific metaphors or untranslatable terms risking misclassification. thus, frameworks that jointly model multi-label emotions, intensity spectra, and cross-cultural variations are essential for advancing emotion-aware technologies. as part of semeval-2025 task 11: bridging the gap in text-based emotion detection @xcite , we propose a multilingual framework integrating multilingual embeddings to capture shared semantic and affective features alongside intensity-sensitive architectures for detecting gradational nuances. the remainder of the paper is organized as follows: section 2 reviews existing methods in multi-label emotion detection and their limitations; section 3 introduces the multilingual dataset; section 4 details our model's approach to disentangling overlapping emotions; section 5 compares our method with state-of-theart baselines; and section 6 presents experimental outcomes and performance analysis. finally, section 7 discusses implications for affective computing and future directions, including multimodal data integration and low-resource language adaptation. ## related work the evolution of multilingual emotion detection systems has been shaped by three interconnected pillars: (1) the creation of high-quality datasets, (2) innovations in cross-lingual transfer methodologies, and (3) architectural advancements in multilingual models. this progression reflects a paradigm shift from monolingual benchmarks to language-agnostic scalable frameworks capable of capturing emotional nuance across linguistic boundaries. early research established rigorous baselines through carefully curated monolingual datasets. the goemotions corpus @xcite , a seminal resource comprising 58,000 english reddit comments annotated with 27 emotion categories, underscored the importance of multi-rater consensus and quality control in emotion labelling, achieving an f1-score of 0.46 through bertbased fine-tuning combined with principal preserved component analysis (ppca). although this work laid the groundwork for data-driven approaches, it also exposed a key limitation: the lack of multilingual comparability inherent to single-language corpora. to overcome this, subsequent studies focused on knowledge transfer from high-resource to low-resource languages. wang et al. (2024b) pioneered a knowledge distillation framework that aligns multilingual representations (e.g., xlm-roberta) with english-centric models (e.g., roberta) using translation-weighted data, reducing the performance gap between monolingual and multilingual systems by 23%. complementary work by @xcite compared cross-lingual strategies-including multilingual embeddings (mbert), translated corpora, and parallel text alignment for arabic and spanish emotion detection, finding that target-language fine-tuning outperforms direct transfer by 14% f1score while affirming the indispensability of crosslingual methods for under-resourced languages. parallel efforts have optimized model architectures for improved multilingual generalization. @xcite developed xlm-emo, a social media-oriented model trained on 19 languages using xlm-roberta, which achieved state-ofthe-art zero-shot performance in low-resource settings and demonstrated that unified architectures can capture shared affective features without language-specific tuning. meanwhile, gupta (2021) improved robustness via virtual adversar-ial training (vat), enforced consistency between original and perturbed inputs to boost cross-lingual f1-scores by 8% in arabic and spanish. further breakthroughs leverage the semantic richness of large language models: cheng et al. (2024) introduced the teii framework, which iteratively refines predictions by combining gpt-3.5 and gpt-4 and employs explanation-driven fine-tuning on translated emotion lexicons to reduce cross-lingual prediction variance by 37%. this approach aligns with findings from navas alejo et al. (2020), who demonstrated that unsupervised machine translation better preserves emotional intensity gradients, especially for morphologically rich languages like catalan. despite these advances, critical gaps remain in reconciling performance disparities across languages. as noted in @xcite , even state-of-the-art multilingual models exhibit 'linguistic bias', with performance degrading for languages typologically distant from english. moreover, the common practice of treating emotion intensity as static rather than contextual oversimplifies the complex nature of affect, as argued by psycholinguistic evidence @xcite . our work addresses these limitations by focusing on (1) culture-aware multilingual representation learning and (2) dynamic intensity modelling, thereby advancing beyond the current paradigm of static cross-lingual transfer. ## dataset in our study, we leverage the brighter dataset @xcite to explore cross-lingual emotion recognition. brighter is a large-scale, manually curated resource designed to bridge the gap in emotion recognition for low-resource languages. it comprises nearly 100,000 text instances gathered from diverse sources, including social media posts, personal narratives, speeches, literary texts, and news articles across 28 languages from various language families. each text instance is annotated by native speakers with one or more emotion labels (anger, sadness, fear, disgust, joy, surprise, and a neutral category) along with corresponding intensity ratings on a four-point scale (0 indicating no emotion up to 3 indicating high intensity). the dataset's annotation process involves rigorous preprocessing steps such as deduplication and noise removal, followed by quality control measures like the split-half class match percentage (shcmp) to ensure high reliability in labelling. this comprehensive dataset not only enriches the training resources available for multilingual emotion recognition models but also serves as a valuable benchmark for evaluating performance across both high and low-resource languages. furthermore, we complement our approach for languages with particularly scarce resources, such as amharic and afan oromo by incorporating data from the ethioemo dataset @xcite . ethioemo is specifically tailored for ethiopian languages and provides robust multi-label emotion annotations derived from sources like news headlines, twitter posts, youtube comments, and facebook data. by integrating these datasets, our work benefits from enhanced linguistic diversity and improved reliability in emotion classification, especially for under-represented languages. the dataset splits are as follows: the hindi corpus comprises a total of 3,666 instances, with 2,556 instances allocated for training (approximately 70%), 100 instances for development (around 2.7%), and 1,010 instances for testing (roughly 27.5%). similarly, the english corpus consists of 5,651 instances, with 2,768 instances used for training (approximately 49%), 116 instances for development (about 2%), and 2,767 instances for testing (roughly 49%). ## methodology our methodology integrates multilingual representation learning with multi-label classification to address cross-lingual emotion detection. we refer to our proposed model as transfermodel_fc_embeddinge5 throughout this paper. central to this approach is the multilingual e5 text embedding framework @xcite , which undergoes a two-stage training process to align semantic representations across languages. first, weakly supervised contrastive pre-training on ∼1 billion multilingual text pairs (sourced from wikipedia, mc4, nllb, and others) optimizes cross-lingual alignment using infonce loss with large batch sizes (32k) to maximize negative sample diversity. this is followed by supervised fine-tuning on high-quality labeled datasets (ms marco, nq, triviaqa), augmented with mined hard negatives and knowledge distillation from a cross-encoder teacher. we employ the instruction-tuned me5-large-instruct variant, pre-trained on 500k gpt-3.5/4-generated synthetic instructions across 93 languages, to enhance task-specific adaptability. building upon this foundation, our emotion detection architecture processes input text through the multilingual e5 tokenizer, standardizing sequences to 150 tokens to balance computational efficiency and semantic retention. the model generates contextualized embeddings via multilingual-e5-large-instruct, with the [cls] token serving as a sequence-level semantic summary @xcite . a dropout layer (@xmath0.3) regularizes the 1024-dimensional [cls] embedding before projection into the emotion space through a fully connected layer. sigmoid activations independently estimate probabilities for 5-6 emotion labels (dataset-dependent), explicitly modelling label co-occurrence inherent to multi-label scenarios. to optimize performance, we train the system using binary cross entropy (bce) with label smoothing (α = 0.1), mitigating overconfidence in sparse annotations. the adamw optimizer @xcite (learning @xmath1, β 1 = 0.9, β 2 = 0.999) processes mini-batches of 16 samples, with gradient clipping (max @xmath2.0) stabilizing updates. early stopping monitors the development set macro f1 score (@xmath3, preserving generalizability by halting training during performance plateaus. during inference, emotion probabilities are thresholded at 0.5 (adjustable per application needs) to yield binary predictions. evaluation prioritizes macro-averaged f1, which aggregates per-class true/false positives and negatives across all batches to penalize bias toward frequent labels a critical safeguard for imbalanced multi-label datasets. results are averaged over five random seeds to account for initialization variance, ensuring reproducibility. by unifying multilingual semantic alignment with modular classification components, transfermodel_fc_embeddinge5 addresses the dual challenges of cross-lingual emotion detection, preserving affective nuance across languages while disentangling overlapping emotional states. ## experiments to complement our transformer-based system described in section 4, we implemented a baseline multi-label emotion classification pipeline that integrates classical machine learning classifiers with pre-trained sentence embeddings. in our experiments, we compare two variants that differ solely in the choice of embedding model. our setup uses two csv files containing text samples and six emotion labels (anger, disgust, fear, joy, sadness, and surprise) for both training and testing. texts are converted into normalized embeddings using a helper function that leverages sentencetransformer models with the @xmath4. since raw embeddings from our language models exhibit variability across dimensions and may not be centered around zero-factors that can obscure underlying semantic information we apply a two-step normalization process. first, we perform l2 normalization to ensure each embedding vector has a unit norm, emphasizing the semantic direction rather than its magnitude. in our implementation, one branch uses the labse model @xcite while the other employs the multilingual e5 large model @xcite . second, we apply zscore normalization (standard scaling) using scikitlearn's standardscaler @xcite to adjust features to a mean of zero and a standard deviation of one, thereby mitigating scale differences. after normalization, we extract the six emotion labels to facilitate multi-label classification. four classifiers are then trained: support vector machine (with an rbf kernel and probability estimates), gaussian naïve bayes, logistic regression (with increased iterations), and random forest (regularized by limiting tree depth and controlling split criteria). these classifiers are wrapped using scikit-learn's multioutputclassifier, ensuring that the multi-label nature of the task is properly addressed. evaluation is performed on both the training and testing set using detailed classification reports and macro f1 scores to gauge performance across all emotion classes. for real-time prediction, a dedicated function processes new text inputs by generating embeddings, applying the same scaling procedures, and predicting emotion labels. the output is returned as a dictionary mapping each emotion to a binary prediction. finally, our experimental design facilitates a direct comparison between the two embedding models: labse, which provides robust, language-agnostic sentence representations @xcite , and multilingual e5 large, which may offer richer semantic embeddings @xcite . this unified approach enables a systematic analysis of the impact of embedding choice on multi-label emotion detection performance, reinforcing the potential of multilingual representations for robust cross-lingual emotion analysis. ## results in this section, we report the evaluation results of our approach to the multi-label emotion detection task (track a) across 13 languages. our model, transfermodel_fc_embeddinge5, built upon multilingual e5 embeddings and a fully connected output layer, was evaluated on its ability to predict six emotion categories (anger, disgust, fear, joy, sadness, and surprise) using both micro and macro f1 scores as evaluation metrics. comparison with top systems. in comparison with the top two performing teams for each language, our approach did not secure the top spot but remained competitive across most languages. for example: our system achieved its strongest results in russian (0.8831), closely trailing the 2nd-ranked team (0.9008), demonstrating competitive performance. in hindi (0.8901) and marathi (0.8657), team a secured scores within 3-4% of the 1st-place teams, highlighting robustness in these languages. while not topping the leaderboard, these narrow gaps reflect effective alignment with top-tier approaches. notably, languages like arabic and chinese showed larger performance drops, emphasizing the need for targeted improvements. ## conclusion in this paper, we proposed transfermodel_fc_embeddinge5, a novel approach to multilingual emotion detection that integrates multilingual e5 embeddings with a fully connected classification layer. our experiments on the brighter dataset show strong macro f1 scores for languages like hindi, russian, and spanish, while also highlighting challenges in arabic, chinese, and oromo due to linguistic and cultural diversity. our model effectively captures emotional nuances, accounting for variations in expression and intensity across languages. this work advances affective computing by demonstrating that multilingual embeddings within a structured classification framework enhance cross-lingual emotion detection. it also lays a foundation for future research on breaking language barriers in sentiment analysis. robert plutchik. 1980. chapter 1 -a general psychoevolutionary theory of emotion. in robert plutchik and henry kellerman, editors, theories of emotion, pages 3-33. academic press. liang wang, nan yang, xiaolong huang, linjun yang, rangan majumder, and furu wei. 2024a. multilingual e5 text embeddings: a technical report. preprint, arxiv:2402.05672. yuqi wang, zimu wang, nijia han, wei wang, qi chen, haiyang zhang, yushan pan, and anh nguyen. 2024b. knowledge distillation from monolingual to multilingual models for intelligent and interpretable multilingual emotion detection. in proceedings of the 14th workshop on computational approaches to subjectivity, sentiment, & social media analysis, pages 470-475, bangkok, thailand. acl. the flowchart depicts the end-to-end pipeline starting from the input text, which is first processed using the multilingual e5 tokenizer. the resulting embeddings are passed through a dropout layer and then into a fully connected layer with sigmoid activations to perform multi-label emotion classification. this modular setup allows efficient handling of semantic nuances across languages and emotion co-occurrence patterns in the dataset.