H oward U niversity- AI 4 PC at S em E val-2025 Task 11: Combining Expert Personas via Prompting for Enhanced Multilingual Emotion Analysis (2025)

Abstract: For our approach to SemEval-2025 Task 11, we employ a multi-tier evaluation framework for perceived emotion analysis. Our system consists of a smaller-parameter-size large language model that independently predicts a given text’s perceived emotion while explaining the reasoning behind its decision. The initial model’s persona is varied through careful prompting, allowing it to represent multiple perspectives. These outputs, including both predictions and reasoning, are aggregated and fed into a final decision-making model that determines the ultimate emotion classification. We evaluated our approach in official SemEval Task 11 on subtasks A and C in all the languages provided.

## introduction semeval-2025 task 11 @xcite focuses on detecting perceived emotion in a given text. understanding emotion in natural language is an inherently complex task as the author not only expresses an emotion, but each reader may perceive a different emotion based on linguistic, cultural, and contextual factors. in natural language processing, emphasis is traditionally placed on the emotion explicitly expressed in the given text @xcite ; however, perceived emotion detection aims to predict the emotion evoked in the audience, which may ultimately differ from the emotion portrayed by the author. these challenges are amplified in multilingual settings, where variations in word connotations, tone, and idiomatic expressions often lead to subjective and inconsistent interpretations of emotion @xcite . to address these challenges, semeval-2025 task 11 introduces a multilingual perceived emotion detection task 1 , to bridge the gap in nlp systems' ability to handle perceived emotion. this task consists of 3 tracks, 1 task data available at: ## related work detecting emotion in text has been a significant area of research in natural language processing (nlp). previous studies have explored various approaches, including lexicon-based methods, machine learning techniques, and using deep learning models @xcite . for example, mohammad (2018) developed the nrc emotion lexicon, a resource widely used for text emotion analysis. more recent approaches have harnessed the capabilities of transformerbased models such as bert for emotion detection tasks, demonstrating improved performance in emotion detection across multiple languages @xcite . several studies have attempted to address the challenge of multilingual emotion detection, @xcite introduced xlm-emo, a multilingual emotion classification model that performs well across 32 languages. however, researchers have highlighted that using machine translations in multilingual datasets can be problematic as it has the potential to overlook language-specific characteristics of emotion verbalization @xcite @xcite . our work builds on recent advancements in collaborative strategies for large language models, as explored by @xcite . these approaches leverage the strengths of multiple perspectives to enhance overall performance and robustness in complex nlp tasks, particularly in emotion analysis (ea). ## system overview our multi-tier evaluation framework foot_0 for perceived emotion detection combines specialized expert models with a larger aggregator model, all locally hosted using ollama foot_1 . this architecture is designed to capture nuanced emotional perceptions across diverse linguistic and cultural contexts. ## experimental setup technical implementation all models (llama 3.2 3b (ai@meta, 2024) and deepseek r1 32b (deepseek-ai, 2025)) are locally hosted using ollama. we utilized the ollama python library for programmatic interaction with these models. expert personas are implemented by modifying the prompts and roles assigned to each llama 3.2 3b model instance. ## results and analysis our multi-tier evaluation framework for perceived emotion detection demonstrated varying performance across different languages and emotions in track a of the semeval-2025 task 11. below is a detailed analysis of the results. note: due to space constraints, the complete results are provided in appendix c ## ethical considerations bias and fairness: our framework showed varying performance across different languages, potentially leading to unequal treatment of speakers of different languages. this could result in bias against speakers of low-resource languages or languages not well-represented in the training data. secondly, the reliance on pre-trained models like llama 3.2 3b (ai@meta, 2024) and deepseek r1 32b (deepseek-ai, 2025) may inherit biases present in their training data, potentially amplifying societal biases related to emotion expression across different cultures. ## conclusion in this paper, we presented a multi-tier evaluation framework for perceived emotion detection in text, which demonstrated mixed performance across multiple tracks of the semeval-2025 task 11. our system leverages a combination of specialized expert models based on llama 3.2 3b (ai@meta, 2024) and a powerful aggregator model using deepseek r1 32b (deepseek-ai, 2025), all locally hosted using ollama. we showed that this approach can effectively capture nuanced emotional perceptions across diverse linguistic and cultural contexts, particularly excelling in indo-european languages like hindi and romanian. our framework demonstrated strength in detecting emotions like joy and anger across multiple languages, suggesting these emotions may have more universal linguistic markers. the system's performance varied significantly across language families, with indo-european languages generally outperforming others. this highlights our approach's potential for nuanced emotion detection while underscoring challenges in cross-lingual analysis. for future work, we aim to enhance the system's ability to handle linguistic diversity and improve performance on underrepresented languages and emotions. addressing these challenges moves us closer to developing robust, multilingual emotion detection systems capable of capturing the complexities of human emotions across diverse cultural contexts.