OPI - DRO - HEL at at S em E val-2025 Task 11: Few-shot prompting for Text-based Emotion Recognition (2025)

Abstract: This paper presents our system, developed as our contribution to SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Detection task, in particular track A, Multi-label Emotion Detection subtask. Our approach relies on two distinct components: semantic search for top N most similar inputs from training set and an interface to pretrained LLM being prompted using the found examples. We examine several prompting strategies and their impact on overall performance of the proposed solution.

## introduction emotions are inherently complex and multifaceted, influencing daily interactions while often remaining challenging to articulate and interpret. individuals employ language in intricate and nuanced ways to convey emotions, with expression and perception varying across linguistic and cultural contexts, as well as within the same societal or social group. this paper presents our system proposed as a solution to task 11, track a of semeval-2025 competition, which focuses on detecting perceived emotions, i.e., what emotion most people think the speaker might have felt given a sentence or a short text snippet @xcite . the recognition of emotions by machine learning (ml) systems has been an active area of research for several decades, with approaches evolving from rule-based models to neural networks such as long short-term memory (lstm) @xcite . the advent of large language models (llms) has introduced significantly more complex architectures, which demonstrated efficacy in classical natural language processing downstream tasks but also phenomenons such as emergent abilities @xcite . the effectiveness of llms in emotion detection is further supported by benchmarks such as semeval-2024 task 10, where llms were widely adopted by participants @xcite . given their demonstrated performance, llms are now regarded as state-of-the-art solutions in this domain. consequently, this study leverages 70b variant of deepsek r1 llm to solve given task using several prompting strategies such as chainof-thought and few-shot-prompting with examples for in-context-learning being provided by an information retrieval subsystem based on embeddings generated by a fine-tuned roberta encoder @xcite . ## task and dataset the task at hand is an instance of classical multilabeled text classification task with the set of labels spanning six categories of perceived emotions: anger, sadness, fear, disgust, joy, surprise, which align with ekman's six basic emotions. text snippets were mostly extracted from social media webpages such as reddit, youtube and twitter among others. for instance, the sentence "that was the last time anyone saw her." was annotated with "fear" and "sadness". the organizers provided a separate dataset for each of 28 different languages from 7 language families, each dataset was further divided into train, dev and test splits. in this study, we focused on developing a solution for the english subset of the track a dataset, which consists of 2,768 training examples, 116 development examples, and 2,767 test examples. an analysis of the label distribution, as presented in the task dataset description paper @xcite , suggests that class imbalance may introduce additional challenges. specifically, the most frequent class, fear, appears in 3,218 instances, whereas the least frequent class, anger, is present in only 671 examples. additionally, only 545 instances do not belong to any of the six predefined emotion categories, classifying them as neutral. the official evaluation metric selected by the organizers was the macro-averaged f1-score, computed based on the predicted and gold-standard labels. ## experiments in the following section, we present results exclusively on the test dataset to ensure a consistent and reliable point of reference, which most closely aligns with the final ranking. all reported results are based on the official macro-averaged f1-score; therefore, unless otherwise specified, this metric should be assumed by default. ## conclusions and limitations consequently, we selected the unstructured fewshot approach with roberta-based semantic search as our final submission. this approach achieved a macro-averaged f1-score of 0.7356, ranking 32nd out of 75 teams and outperforming the official baseline solution, which scored 0.7083. as discussed in the previous section, we believe that this ranking could be improved with minimal modifications to the overall system while maintaining the existing framework. specifically, replacing deepseek-r1-70b with a larger model, optimizing the number and potentially the order of few-shot examples, and further fine-tuning roberta for the metric learning task could yield performance gains. furthermore, given the sensitivity of large language models (llms) to prompt formulation, refining prompt design presents an additional avenue for optimization and future research.