GT - NLP at S em E val-2025 Task 11: E mo R ationale, Evidence-Based Emotion Detection via Retrieval-Augmented Generation (2025)

Abstract: Emotion detection in multilingual settings presents significant challenges, particularly for low-resource languages where labeled datasets are scarce. To address these limitations, we introduce EmoRationale, a Retrieval-Augmented Generation (RAG) framework designed to enhance explainability and cross-lingual generalization in emotion detection. Our approach combines vector-based retrieval with in-context learning in large language models (LLMs), using semantically relevant examples to enhance classification accuracy and interpretability. Unlike traditional fine-tuning methods, our system provides evidence-based reasoning for its predictions, making emotion detection more transparent and adaptable across diverse linguistic contexts. Experimental results on the SemEval-2025 Task 11 dataset demonstrate that our RAG-based method achieves strong performance in multi-label emotion classification, emotion intensity assessment, and cross-lingual emotion transfer, surpassing conventional models in interpretability while remaining cost-effective.

## introduction emotions are the unseen threads that weave together human communication, influencing decision-making, social interactions, and personal well-being @xcite . with the rapid advancement of large language models (llms) and the increasing sophistication of aidriven text analysis @xcite @xcite , understanding emotions embedded within text has become critically important across multiple domains including mental health monitoring, customer engagement, and human-computer interaction @xcite . while llms have significantly improved 1 https://github.com/daniel-saeedi/ semevaltask11_emorationale sentiment-aware applications, their effectiveness across diverse linguistic and cultural contexts remains inconsistent @xcite . existing research has primarily focused on highresource languages, leaving significant gaps in emotion detection for low-resource languages due to a lack of annotated datasets. this disparity hinders the development of inclusive and effective emotion-aware applications @xcite . to address these shortcomings, brighter (bridging the gap in human-annotated textual emotion recognition datasets for 28 languages) takes a major step forward, particularly for low-resource languages to collect multilabeled emotion-annotated datasets particularly for 28 different low-resource languages @xcite . this paper presents emorationale, an ai-driven framework designed for the semeval-2025 shared task "bridging the gap in text-based emotion detection" @xcite , addressing challenges in multilabel emotion detection, emotion intensity assessment, and cross-lingual emotion detection. leveraging retrieval-augmented generation (rag), our system integrates the retrieval capabilities of vector databases such as faiss @xcite with the generative power of llms like the multilingual minilm sentence transformer @xcite to incorporate relevant contextual information and bridge gaps in low-resource languages. by combining few-shot prompting with retrieval-augmented generation, emorationale produces accurate emotion predictions accompanied by explicit, evidence-based reasoning, with extensive ablation studies demonstrating that incorporating in-context examples significantly boosts performance-surpassing traditional fine-tuning approaches such as those based on roberta @xcite . moreover, our framework exhibits notable cross-lingual transfer capabilities and cost-effectiveness, paving the way for more inter-pretable and efficient emotion recognition systems in multilingual settings. ## background emotion detection in text is a multifaceted area of research within natural language processing (nlp) that seeks not only to identify basic sentiment but also to capture the rich spectrum of human emotions. previous research mainly focused on sentiment analysis-differentiating between positive and negative affect (dang et al., 2020)-but subsequent research has expanded the scope to include discrete emotions such as joy, sadness, anger, fear, surprise, and disgust @xcite ) @xcite . while llms trained on highresource languages such as english benefit from extensive corpora, their performance in low-resource settings is hindered by data scarcity, inadequate tokenization, and the difficulty of capturing nuanced emotional cues unique to these languages. bert @xcite has revolutionized nlp by providing deep contextualized representations that capture subtle semantic nuances in text @xcite . fine-tuning bert on emotionlabeled datasets enhances its effectiveness in tasks such as multi-label emotion classification and emotion intensity assessment by enabling the model to learn complex emotion-specific features @xcite . these capabilities have been further enhanced through the adoption of multilingual variants of bert, which facilitate cross-lingual emotion detection by leveraging shared representations learned from extensive and diverse corpora @xcite . recent advancements in retrieval-augmented generation (rag) @xcite have introduced a promising avenue for enhancing emotion recognition systems. rag frameworks leverage retrieval mechanisms to access semantically relevant examples from external knowledge bases, integrating this contextual information with the generative capabilities of large language models (llms) to improve accuracy and robustness. additionally, chain-of-thought (cot) prompting has significantly enhanced llm reasoning and interpretability, enabling models to decompose complex nlp tasks into structured intermediate steps. this approach has demonstrated notable improvements in various applications, including emotion detection, by guiding models to explicitly articulate their decision-making process @xcite . this evolution underscores the potential of rag and cot-based methodologies in advancing emotion-aware ai systems and bridging gaps in interpretable affective computing @xcite . ## tasks in semeval task 11 @xcite , participants are invited to tackle one or more challenges that address the complex and nuanced nature of emotional expression in text. below, we detail the three tracks of the shared task. in track a, the objective is to determine the set of emotions that the speaker conveys in a given text snippet. each snippet is labeled with one or more of the following emotions: joy, sadness, fear, anger, surprise, and disgust. track b builds on the multi-label classification framework by incorporating emotion intensity. in this task, for each text snippet paired with a target emotion, the objective is to predict an ordinal intensity value that indicates the strength of the expressed emotion. intensity is scaled from 0 (none) to 3 (high). track c addresses the challenge of cross-lingual emotion detection. in this track, participants are provided with a labeled training set in a source language and are required to predict the perceived emotion labels for texts in a different target language. the label set includes the same six emotions as in track a. ## system overview emorationale employs a retrieval-augmented generation (rag) framework to classify emotions in text. our system is composed of three main components: a vector store of labeled training data, a large language model (llm) prompt that incorporates relevant examples, and an llm-based inference step. ## experimental setup t4 gpu from colab pro for the multilingual minilm sentence transformer. we also conducted evaluations using deepseek r1(deepseek-ai et al., 2025) with the figure 2: average performance scores across emotions for all three tracks in the semeval 2025 task 11. track a and track c use f1-scores while track b uses pearson correlation coefficients. joy consistently achieves the highest scores (≈ 0.73) across all tracks, while surprise shows the lowest performance (≈ 0.49). anger and sadness also demonstrate strong performance, while disgust exhibits the second-lowest scores among the emotions. 685b model for track a, deepseek v3 (deepseek-ai, 2024) for track b, and claude sonnet 3.5 (anthropic) for track c. the total cost for development and evaluation on the test set amounted to @xmath022.22. however, one notable shortcoming of the deepseek api was its unreliability, frequently experiencing interruptions and errors due to high demand. ## results we evaluated our system on all three tracks of semeval-2025 task 11: multi-label emotion detection (track a), emotion intensity (track b), and cross-lingual emotion detection (track c). below, we detail our overall performance, present findings from ablation studies, and discuss the impact of our retrieval-augmented generation (rag) approach. across all three tracks, models consistently exhibited lower performance for the emotions surprise (approximately 0.49) and disgust (approximately 0.53), as illustrated in figure 2 . detecting surprise and disgust poses significant challenges, primarily because these emotions often depend on nuanced, context-specific, and culturally influenced expressions, which are difficult to interpret accurately through textual analysis alone. unlike more ex-plicit emotions such as anger or joy, surprise and disgust typically lack distinct verbal cues and thus require additional contextual understanding @xcite . furthermore, emotions like surprise can convey either positive or negative sentiments, while disgust may frequently be misinterpreted as anger or fear. such inherent ambiguities and overlaps further complicate accurate emotion classification for language models. ## conclusion and future work in this work, we introduced emorationale, a retrieval-augmented generation (rag) framework for explainable emotion recognition that addresses the challenges posed by multilingual and nuanced emotional expressions in text. by integrating robust sentence embeddings with a faiss vector store and leveraging few-shot in-context learning, emorationale offers significant advantages in interpretability and cross-lingual transfer-even if it may not achieve the highest raw predictive metrics. the framework provides explicit, evidence-based reasoning for its predictions while effectively generalizing emotion recognition across languages using only a few in-context examples-a feat that conventional fine-tuning approaches, such as robertabase, often struggle to achieve.