M c G ill- NLP at S em E val-2025 Task 11: Bridging the Gap in Text-Based Emotion Detection (2025)

Abstract: In this paper, we present the results of our SemEval-2025 Emotion Detection Shared Task Track A which focuses on multi-label emotion detection. Our team’s approach leverages prompting GPT-4o, fine-tuning NLLB- LLM2Vec encoder, and an ensemble of these two approaches to solve Track A. Our ensemble method beats the baseline method that fine-tuned RemBERT encoder in 24 of the 28 languages. Furthermore, our results shows that the average performance is much worse for under-resourced languages in the Afro- Asiatic, Niger-Congo and Austronesia with per- formance scores at 50 F1 points and below.

## introduction emotion detection is the task of identifying and categorizing emotions expressed in textual data @xcite . given a piece of text, such as sentence, document, sentence, or summary, the goal could be to determine the underlying emotion conveyed by the speaker or that invoked in the listener. emotions play an essential role human interaction and interpersonal relationships @xcite . in actual speech, speakers often give many non-verbal cues such as facial expressions or hand gestures to convey their emotions. even then emotions of the speaker can be hard to detect and this problem becomes even harder for text because of the subtle cues, lack of emoticons, sarcasm or satire, or complexity and ambiguity of language @xcite . people use text on social media such as reddit and there is a growth of textual dialogue with growing prominence of these platforms @xcite . most human-llm interaction is in text and hence emotion detection in text is important for nlp models to respond appropriately. nandwani and verma (2021) describe a wide variety of cases for businesses, healthcare sector, education sector, etc. that have a need for accurate emotion detection. wide array of approaches have been tried to solve emotion detection. @xcite provide a list of state of the art approaches from 2015 to 2020 including rule-based, machine learning based, and hybrid approaches that combine the two. in semeval2018 @xcite , @xcite use bert @xcite to learn contextualized word representation and combine it with a linear layer to get a single score for each token to solve multi-label emotion detection in english, arabic, and spanish. for the same task @xcite use bi-directional lstms as encoders and decoders. @xcite experiment with various machine learning, deep learning, and transformer based architectures and get best performance from xlm-r @xcite for emotion classification on bengali text. plaza-del arco et al. (2022) explore emotion classification in zero shot learning setup with natural language inference (nli). in more recent times, llms are being used for all sorts of nlp tasks and at semeval-2024 task 3 @xcite , a third of the teams used llms. team petkatz @xcite that ranked second, fine-tuned gpt-3.5 for emotion classification. our team used a combination of gpt-4o (ope-nai, 2024) given the success of @xcite which integrates machine translation (mt) encoders into llm backbones. mt-llms preserve the multilingual representation alignment from mt encoder, allowing low resource languages to tap into the knowledge of english-centric llms. for gpt-4o we use few-shot prompting technique @xcite and with nllb-llm2vec we fine-tune lora adapters for each language with the training and dev set of the data provided. ## task description the task @xcite focuses on identifying and quantifying perceived emotion of the speaker based on a given sentence or a short text snippet. this means determining the emotion that the speaker appears to express, rather than the emotions invoked in the listerner, or anyone else mentioned in the text, or the true emotion of the speaker. the emotions being looked at for this task are from ekman's six basic emotions @xcite )joy, sadness, fear, anger, surprise, or disgust. the definition of these emotions is provided in muhammad et al. (2025a) , which we present here: the shared tasks has three tracks of problems -track a, track b, and track c. track a is for multi-label emotion detection where given a piece of text we evaluate the presence of each of the six emotions described above and give a binary classification of 0 or 1 for each of them. track b is for emotion intensity detection where each of the emotions can have ordinal values from 0 to 3. 0 meaning no emotion, 1 for low degree of emotion, 2 for moderate degree of emotion, and 3 for high degree of emotion. track c is for cross-lingual emotion detection where given labeled training data in a language, predictions need to be done for text instances in another language. some languages in all the 3 tasks include five emotions, excluding the emotion disgust. we focus on a single track, so our submission is only for track a which had 28 languages in the dataset @xcite : afrikaans (afr), algerian arabic (arq), amharic (amh), portuguese (brazilian) (ptbr), mandarin chinese (chn), emakhuwa (vmw), english (eng), german (deu), hausa (hau), hindi (hin), igbo (ibo), kinyarwanda (kin), spanish (latin american) (esp), marathi (mar), moroccan arabic (ary), portuguese (mozambican) (pt-mz), nigerian-pidgin (pcm), oromo (orm), romanian (ron), russian (rus), somali (som), sundanese (sun), swahili (swa), swedish (swe), tatar (tat), tigrinya (tir), ukrainian (ukr), yoruba (yor). ## system overview our system for solving track a consists of gpt-4o, nllb-llm2vec and a method of ensemble to combine the best results from the two. we describe each of these below: ## results and discussion the results from the two approaches, and the ensemble which is our final ranked submission, are shown in the we computed the results for all the 28 languages. comparing our results to other teams, on 12 languages out of 28, our team is in the top 10 rankings for that language. our ensemble method does better than the baseline rembert ( chung there are a few things we could've done to look for better performance. we have the same training parameters for nllb-llm2vec for each language and we might have benefited from tuning parameters at a language level. we could've also increased the number of epochs of training for better performance. ## conclusion in this work, we present our approach for track a of semeval-2025 task 11 for multi-label emotion detection in text on 28 languages. we experiment with two methods -few-shot prompting with gpt-4o and fine-tuning nllb-llm2vec with lora adapters, and present an ensemble of these two for our best results. our model outperforms baseline results in most languages. we also see that average f1 scores are at 50 f1 points and below for underresourced languages in the afro-asiatic, niger-congo and austronesia. we also present improvements and analysis on the performance of nllb-llm2vec for this task with better hyperparameter tuning on the dev set. this leads to nllb-llm2vec beating baseline results in 19 languages, and its performance gets close to gpt-4o performance on average macro f1 score across languages. further work can be done to improve the ensemble technique to work more fluently and combine the results of gpt-4o and nllb-llm2vec based on the dev set. this would invariably provide better results than our ranked submission.