QMUL at S em E val-2025 Task 11: Explicit Emotion Detection with E mo L ex, Feature Engineering, and Threshold-Optimized Multi-Label Classification (2025)

Abstract: SemEval 2025 Task 11 Track A explores the detection of multiple emotions in text samples. Our best model combined BERT (fine-tuned on an emotion dataset) predictions and engineered features with EmoLex words appended. Together, these were used as input to train a multi-layer perceptron. This achieved a final test set Macro F1 score of 0.56. Compared to only using BERT predictions, our system improves performance by 43.6%.

## introduction semeval 2025 task 11 track a is about determining what emotion most people will think is reflected in a short text snippet @xcite . this is about the perceived emotion by a reader, not about how someone is truly feeling. this is important because the individual's actual emotional state of being is difficult to define with absolute certainty @xcite . the task consists in identifying the presence of five emotions, i.e. joy, sadness, fear, anger, and surprise. the main chellenges include varying lengths of texts, and imbalance of emotions. we approached this by stacking wordpiece tokenisation, preprocessing, emolex words and bert predictions as features, which then we pass to mlps. upon quantitative evaluation on the development set, we found that using separate models for each emotion and dynamic thresholding based on each emotion was the most effective system. our code is openly available 1 . ## background and related work emotion analysis, a subfield of sentiment analysis, seeks to identify nuanced emotional states in text rather than broad polarity @xcite . early work focused on lexicon-based methods, such as emolex (nrc emotion lexicon), which maps 1 http://github.com/angelinewang/semeval-task-11-track-a words to primary emotions and remains foundational for explicit emotion representation @xcite . while lexicons like emolex provide interpretability, their static nature struggles with contextual nuances. @xcite showed emotion co-occurrence patterns (e.g., anger-disgust) to refine multi-label predictions, but their work relied on rigid lexicon counts rather than context-aware scoring. the shift toward multi-label detection addresses the limitation of single-label classification, as text often expresses overlapping emotions @xcite . semeval tasks have driven progress with top systems hybridizing lexicons and neural models. for example, @xcite combined lexicon-derived features with bert for multimodal classification, while @xcite optimized thresholds for llm-based emotion detection. although weighted losses provide gains @xcite , few studies address sensitivity-adjusting boundaries for imbalance. traditional approaches like countvectorizerbased lexicon scoring @xcite treat emotion-linked words equally, ignoring discriminative power. recent advances in hybrid paradigms highlight the need for weighted lexical integration and threshold optimization. our work bridges this by integrating emolex with bert, using positive weight calculation to amplify discriminative terms (e.g., "devastated" for sadness) and emotion-specific threshold optimization to balance precision and recall-advancing methods from generic sentiment analysis @xcite to nuanced multi-emotion detection. mad et al., 2025). this diverges from previous years' tasks, which focused on the speaker emotion @xcite . this shift notably introduces cultural relativity due to annotators' diverse backgrounds, pragmatic ambiguity, and multiperspective modeling (the need to predict majority perceptions rather than 'true' emotions). ## experimental setup input features. we assessed performance with three different variations of input features: (i) with inputs that consisted of bert prediction logits only, (ii) with bert predictions stacked on top of the engineered features, and (iii) with the addition of emolex words appended to the text that went into engineered features. classifier configurations. for each of the three variations above, we evaluate three dfiferent configurations of mlp classifiers: (i) a single mlp with a threshold of 0.5, (ii) a different mlp for the detection of each emotion, with a threshold of 0.5, and (iii) due to the imbalance of emotion data (figure 2a ), we assessed whether it might be useful to set differing detection thresholds for each emotion on top of just emotion-specific mlps. these configurations help us evaluate the extent to which having different classifiers for each emotion helps. ## results for experiments with a single mlp and a 0.5 emotion detection threshold across all emotions, using only bert predictions as input features achieved a macro f1 score of 0.6080 on the development set and 0.5418 on the test set. using stacked engineered features on top of bert predictions achieved an improvement leading to a macro f1 of 0.6114 on the development set and 0.5624 on the test set. the addition was emolex words to the engineered features led to a drop in macro f1 on the development set with a score of 0.6075, but an increase in test set performance to 0.5638-this ended up being our best model over all others on test set. the variant with emotion-specific mlps (but still with 0.5 detection thresholds), using only bert predictions as input led to a model with a macro f1 of 0.6162 on the development set (beating the performance of all preceding models) and 0.5434 on the test set. stacking engineered features further increased the macro f1 on the development set to 0.6476, and led to the a test set performance of 0.5457. appending emolex words to the engineered features further increased the macro f1 on the development set to 0.6491, and led to a test set performance of 0.5538. finally, using emotion-specific detection thresholds for each emotion-specific mlps, when using bert predictions as input, the development set performance continued to increase to 0.6568, but test set performance went down to 0.5364. using engineered features in addition to bert predictions led to an increase in development set performance with macro f1 of 0.6816, with test set performance hovering at 0.5542. finally, appending the emolex words to the engineered features led to a dip in development set macro f1 performance to 0.6433, with a corresponding test set performance of 0.5558. in general, we see that engineered features (without emolex words) always improved the performance on both the development and test set for all variations. on the test set, adding emolex words also consistently improved the performance of our models across all variations. on the other hand, emolex words resulted in a decrease in development set macro f1 performance with emotion-specific mlps and emotion-specific detection thresholds. it seems that emolex either does not create an impact or creates a slight negative impact when looking at development set performance. this shows that emolex words were non-specific to the validation set but allowed for better generalisability to unseen test data, which presumably included more of these words, and were perhaps indicative of classifications, which was maybe not the case for the smaller validation set. it is worth remembering that the test set is more than 20 times bigger than the development set. thus, the size of the test set makes it more complex, challenging and thorough than the validation set, hence the generalisation of best models based on development set performance is not good. class-wise performance correlated with label frequency: the majority class (fear) had the highest f1, while the rarest (anger) showed lower recall (sparse training data). joy performed well (moderate frequency), likely aided by anti-correlation with other emotions. threshold optimisation partially addressed imbalance, but minority classes still lagged due to limited training data. in summary, the results ( ## conclusion our best model combined bert predictions and engineered features (including emolex), which were used as input to an mlp. the detection was optimal using a 0.5 threshold, achieving a test macro f1 of 0.564. this configuration generalized better than emotion-specific mlps, likely due to capturing inter-class correlations. key challenges included dataset label imbalance (e.g., dominance of "fear") and performance drops between validation and test sets. future work should explore synthetic data generation for minority emotions and newer bert variants. this approach advances multi-hot emotion detection, with applications in opinion analysis and targeted sentiment modeling.