Chinchunmei at S em E val-2025 Task 11: Boosting the Large Language Model’s Capability of Emotion Perception using Contrastive Learning (2025)

Abstract: The SemEval-2025 Task 11, Bridging the Gap in Text-Based Emotion Detection, introduces an emotion recognition challenge spanning over 28 languages. This competition encourages researchers to explore more advanced approaches to address the challenges posed by the diversity of emotional expressions and background variations. It features two tracks: multi-label classification (Track A) and emotion intensity prediction (Track B), covering six emotion categories: anger, fear, joy, sadness, surprise, and disgust.In our work, we systematically explore the benefits of two contrastive learning approaches: sample-based (Contrastive Reasoning Calibration) and generation-based (DPO, SimPO) contrastive learning. The sample-based contrastive approach trains the model by comparing two samples to generate more reliable predictions. The generation-based contrastive approach trains the model to differentiate between correct and incorrect generations, refining its prediction. All models are fine-tuned from LLaMa3-Instruct-8B. Our system achieves 12th place in Track A and 7th place in Track B for English, while ranking among the top-tier performing systems for other languages.

## introduction text-based emotion detection (tbed) has long been a prominent research area in nlp, with widespread applications in social media analysis @xcite @xcite , mental health treatment @xcite , and dialogue systems @xcite @xcite . depending on how emotions are defined, tbed can be broadly categorized into two approaches: (1) classification-based methods, where emotions are categorized into discrete labels @xcite . (2) scoring-based methods, where emotions are treated as interrelated entities with varying intensity levels @xcite . however, due to the nuanced and complex nature of emotional expression, tbed faces several key challenges @xcite : (1) the distinction between different emotions is often subtle, and emotions are often conveyed implicitly-through metaphors or situational cues rather than explicit words. (2) cultural and linguistic differences influence emotion perception. these challenges make tbed difficult to rely solely on predefined lexicons. a robust tbed system must integrate cultural context, linguistic diversity, background knowledge, and advanced semantic understanding. semeval-2025, task 11, titled "bridging the gap in text-based emotion detection" @xcite , introduces a multilingual benchmark covering 28 languages @xcite . the competition consists of track a (multi-label classification) and track b (intensity prediction). the goal is to identify the speaker's perceived emotion in a given sentence. emotion categories follow ekman's framework @xcite , encompassing six basic emotions: anger, fear, sadness, joy, disgust, and surprise. task b further introduces four intensity levels for each emotion. this competition setup encapsulates both primary tbed methodologies while incorporating challenges in multilingual and fine-grained emotion recognition. to participate in both tracks and support all languages predictions, we adopt the generative large language model (llm). this decision is driven by its robust multi-task integration capabilities and strong support for cross-linguistic applications. in this paper, we explore two alternative types of approaches -sample-based contrastive learning and generation-based contrastive learning -to address the complexity of emotional expression and the ambiguity of sentiment labels. the samplebased approach leverages contrastive reasoning calibration technology (crc) @xcite , which enhances prediction reliability by generating multiple predictions through sample comparisons and aggregating them via majority voting. the generation-based approaches employ preference optimization techniques @xcite @xcite , refining the model's comprehension of sentiment labels by increasing the log probability of correct outputs while reducing that of incorrect ones. given computational constraints, we only explore dpo @xcite and simpo @xcite , two widely acknowledged methods that do not require the reward model. our key contributions are as follows: ## methodology to reduce our workload, we integrate both track a and track b into the same model by using different prompt templates for each. this unified approach enables the model to dynamically switch between prediction tasks as needed. in our explorations, we designed three distinct tasks: figure 1 illustrates the training workflow of these three tasks. ## experiment in this competition, we participate in @xcite , selected for its strong multilingual capabilities and acceptable computational cost. to validate its multilingual capabilities in each language, we check the consistency between original and recovered text using its tokenizer to encode and decode the text content. this experiment confirmed that the model supports all competition languages. we first evaluate the impact of multilingual data on english-only performance in the standard prediction (sp) task. then, we experiment with crc, dpo, and simpo on the english dataset to examine the effectiveness of sample-based and generationbased contrastive learning for tbed. the crc model is trained via sft on a combination of sp and crc datasets. dpo and simpo models are obtained by applying preference tuning to the englishonly sp model. to reduce training costs, we use lora @xcite for all fine-tuning tasks, with its rank and alpha set as 8 and 16. each training lasts for 3 epochs with a batch size of 128. regarding the learning rate (lr), sp and crc employ 4e-4, dpo adopts 5e-6, and simpo uses 1e-6. all models are trained using the adamw optimizer, with β 1 = 0.8, β 2 = 0.99. all lr scheduler employs cosine decay with a warmup ratio of 0.1. for crc inference, we set @xmath0. all experiments are conducted using 8-gpu distributed training. due to limited computational resources, different gpus are used across various training runs. however, all gpus are based on the ada lovelace or hopper architecture, allowing us to leverage a wide range of existing acceleration techniques to enhance training efficiency. ## results tables 1 and 2 present the performance of all models on the english test set. ## conclusion the semeval-2025 organizers introduce a highly challenging text-based emotion detection dataset, covering multi-label classification and intensity prediction across 28 languages. the dataset reflects the complexity of emotional expression and the diversity introduced by different linguistic and cultural backgrounds. we explore three types of approaches in this competition. the baseline approach is the standard prediction task. the two enhanced types of approaches were sample-based contrastive learning and generation-based contrastive learning. for the sample-based approach, we try crc. for the generation-based approach, we explore both dpo and simpo. they all aim to improve model prediction through contrastive training-one by comparing samples and the other by discriminating between correct and incorrect generations. our experiments reveal that different languages reflect distinct cultural backgrounds, so multilingual training does not improve english emotion detection. meanwhile, due to the inherent ambiguity of sentiment expressions, sample-based contrastive learning raises additional uncertainty, ultimately reducing prediction accuracy. on the other hand, generation-based contrastive learning provides consistent improvements in intensity prediction, though its effectiveness varies significantly across different techniques. notably, reference model constraint is crucial in stabilizing generationbased contrastive optimization process. it prevents excessive deviation from the original model distribution, and preserves key capabilities such as structured output generation.