Empaths at S em E val-2025 Task 11: Retrieval-Augmented Approach to Perceived Emotions Prediction (2025)

Abstract: The paper introduces EmoRAG, a retrieval-augmented emotion detection system designed for the SemEval-2025 Task 11. It uses an ensemble of models, retrieving similar examples to prompt large language models (LLMs) for emotion predictions. The retriever component fetches the most relevant examples from a database, which are then used as few-shot prompts for the models. EmoRAG achieves strong, scalable performance across languages with no training at all, demonstrating effectiveness in both high and low-resource languages.

## emorag first, we overview the emorag pipeline. next, we detail the procedures for the database, retriever, generators, and aggregation model. ## experiments the main three components of emorag are the retriever, the generator models, and the aggregation model. retrievers we experimented with an n-gram based retriever and a sentence embedder-based retriever bge-m3 foot_0 (chen et al., 2024). the ngram based retriever is hypothesized to perform better for low-resource languages due to its reliance on surface-level text features. we have used the n-gram retriever from the langchain module foot_1 (chase, 2022). the number of retrieved examples (k) is fixed to 30 for low-resource languages and 100 for highresource languages. the reason for this is that low-resource languages consume more tokens and thus more compute. decoder models the emorag system employed four different llms: llama-3.1-70b foot_2 (grattafiori et al., 2024), qwen2.5-72b-instruct foot_3 (yang et al., 2024), gpt-4o-mini-2024-07-18 (hereafter referred to as gpt-4o-mini), and gemma-2-27b-it foot_4 @xcite . the llm system prompt is in english and only specifies the language of the input text. we experimented with prompts in the target language for which predictions were made but found that english prompts yielded better results. we provide the whole system prompt in the appendix a. aggregation strategies we tested five aggregation strategies to combine predictions from different models: ## results the results of our experiments are summarized in the the emorag system demonstrated strong performance across a wide range of languages, with the majority_vote_by_label_f1 aggregation strategy generally yielding the best results. performance across languages the system achieved high f1-micro and f1-macro scores in high-resource languages such as english, spanish, and russian, with scores exceeding 0.80. in low-resource languages, the performance was more variable, but the system still achieved competitive results, particularly with the use of the n-gram retriever. best models for each language, the bestperforming model and aggregation strategy were selected based on the development set results. the majority_vote_by_label_f1 strategy was often the best choice, indicating the effectiveness of leveraging label-specific f1 scores for aggregation. ## conclusion this paper presents the emorag system submitted to semeval-2025 task 11. our system achieved strong performance across multiple languages, demonstrating its effectiveness in multilingual emotion recognition. emorag introduces a novel pipeline that integrates rag with llms and an adaptive aggregation mechanism. the combination of diverse retrievers and model-specific aggregation strategies enables flexible and robust emotion detection, particularly for low-resource languages. we believe this approach holds significant potential for improving multilingual nlp tasks by leveraging retrieved examples to enhance model predictions. our future research will be focused on refining retrieval methods, exploring alternative rag techniques, and investigating the use of smaller, more efficient models to improve scalability and accessibility across different com-2003 putational environments.