PAI at S em E val-2025 Task 11: A Large Language Model Ensemble Strategy for Text-Based Emotion Detection (2025)

Abstract: This paper describes our system used in the SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Detection. To address the highly subjective nature of emotion detection tasks, we propose a model ensemble strategy designed to capture the varying subjective perceptions of different users towards textual content. The base models of this ensemble strategy consist of several large language models, which are then combined using methods such as neural networks, decision trees, linear regression, and weighted voting. In Track A, out of 28 languages, our system achieved first place in 19 languages. In Track B, out of 11 languages, our system ranked first in 10 languages. Furthermore, our system attained the highest average performance across all languages in both Track A and Track B.

## introduction the objective of task 11 @xcite is to determine, within different linguistic contexts, what emotion most people would perceive the speaker to be feeling based on a sentence or short text snippet uttered by the speaker @xcite . track a of task 11 includes 28 languages, while track b consists of 11 languages. the task requires detecting the presence of the following emotions and assessing their intensity: joy, sadness, fear, anger, surprise, and disgust. given the highly subjective nature of emotion detection in textual content, different annotators may provide varying answers regarding whether a certain emotion is present in the text (or the degree to which it is present). similarly, for large language models (either untrained or only lightly fine-tuned), different models may output differing judgments for the same textual content. therefore, * equal contributions bridging the gap between these two sources of subjectivity-annotator variability and model inconsistency-becomes the central focus and optimization goal of our system. considering the powerful capabilities of large language models and the potential for catastrophic forgetting resulting from improper training, we select both the original and lightly fine-tuned versions of several models as base models. we then employ ensemble strategies such as neural networks, decision trees, linear regression, and weighted voting to combine the outputs of multiple base models, ultimately providing the final prediction. the rationale behind using a ensemble strategy is that the independent predictions made by multiple base models resemble the behavior of multiple annotators independently labeling data, each with their own judgment tendencies. we hypothesize that, when there is sufficient divergence between the prediction results of different base models, appropriate ensemble strategies can better capture the annotators' labeling outcomes. ## results due to limited time and gpu resources, we initially conducted experiments and exploration only on the eng and ptbr languages ( in devlopment dataset, compared to the performance metrics of single-path large language models (either untrained or lightly fine-tuned), the fusion strategy consistently provides an additional improvement of 0.01 to 0.02 on top of the optimal single-path model's metrics. after the release of the test dataset, we plan to apply the same strategy to the 28 languages in track a and the 11 languages in track b. in the final test dataset, we achieved first place in 19 out of 28 languages in track a( ## conclusion similar to the emotion detection task discussed in this paper, strongly subjective tasks are prevalent in industry. at both ends of such tasks, on one side, users or annotators have their own subjective judgment criteria, and on the other, language models, due to the nature of their training data, also in task 11, the ensemble strategy we employed is based on traditional nlp algorithmic solutions. if similar tasks arise in the future, we aim to explore whether there are applicable solutions within the llm domain, such as the moe strategy. additionally, in the dev dataset, we found that transferring the more fine-grained annotation results from track b to track a could further improve the performance metrics of track a. however, due to time constraints, we were unable to test this approach on the test dataset, presenting an opportunity for future exploration.