Trans-Sent at S em E val-2025 Task 11: Text-based Multi-label Emotion Detection using Pre-Trained BERT Transformer Models (2025)

Abstract: We have introduced Trans-Sent, a Transformer-based model designed for multi-label emotion classification in SemEval-2025 Task 11. The model predicts perceived emotions such as joy, sadness, anger, fear, surprise, and disgust from text across seven languages, including Amharic, German, English, Hindi, Marathi, Russian, and Romanian. To handle data imbalance, the system incorporates preprocessing techniques, SMOTE oversampling, and feature engineering to enhance classification accuracy. The model was trained using the BRIGHTER and EthioEmo datasets, which contain diverse textual sources, such as social media, news, literature, and personal narratives. Traditional machine learning models, including Logistic Regression and Decision Trees, were tested but proved inadequate for multi-label classification due to their limited ability to capture contextual and semantic meaning. Fine-tuned BERT models demonstrated superior performance, with Russian achieving the highest ranking (9th overall), while languages with complex grammar, such as German and Amharic, performed lower. Future enhancements may include advanced data augmentation, cross-lingual learning, and multimodal emotion analysis to improve classification across different languages. Trans-Sent contributes to NLP by advancing multi-label emotion detection, particularly in underrepresented languages.

## introduction research on natural language processing is getting a lot of attention recently. nevertheless, the majority of research is still limited to a few languages with plenty of available training data. emotions are both recognizable and mysterious. we all express and control our emotions on a daily basis. however, these are intricate, subtle, and occasionally challenging to describe @xcite . some basic emotions are -joy, sadness, anger, fear, surprise, and disgust etc. emotion recognition is a broad term encompassing tasks like detecting a speaker's emotions, identifying emotions in text, and recognizing emotions evoked in a reader. since people express and perceive emotions in complex, variable ways, it is impossible to determine one's feelings with absolute certainty @xcite . here the goal of this task is to determine perceived emotion i.e. what emotion most people will think the speaker may be feeling given a sentence or short text snippet uttered by the speaker. two popular methods for processing categorical data in machine learning are multi-class and multilabel classification. in multi-class classification, each instance is assigned to only one category from a predefined set of mutually exclusive classes. one instance of a multi-class problem is sentiment analysis, which involves classifying a text as either good, negative, or neutral. labels are not mutually exclusive in multi-label classification, however, since an instance can be a part of more than one category at the same time @xcite . this is especially helpful for jobs like emotion recognition, where a single text can simultaneously convey several emotions, like surprise and joy. multi-label classification frequently uses sigmoid activation, whereas multi-class classification usually uses softmax activation. in this paper, we have introduced trans-sent, a transformer-based sentiment extraction model to extract multi-label emotion from the text data. it uses various pre-trained bidirectional encoder representations from transformers or bert models for individual language to generate emotions @xcite and classify the emotion-labels. some preprocessing measures are used on training data, then it is oversampled, as mostly the dataset is imbalanced for various emotion-labels. then, feature extractions and feature engineering are performed to extract sentence-level contexts. finally, different pretrained bert models are used to extract emotions. ## dataset brighter @xcite is a largescale collection of multi-label emotion recognition datasets covering 28 languages, with a strong focus on low-resource languages from africa, asia, eastern europe, and latin america. unlike many ## system overview trans-sent, our system is built on bert (bidirectional encoder representations from transformers), fine-tuned for multi-label emotion classification. initially, we experimented with traditional machine learning techniques @xcite , using tf-idf embeddings with models such as logistic regression, decision trees, and random forest etc. to handle the multi-label nature of the task, we applied label powerset and classifier chaining @xcite techniques, which, despite providing structured predictions, were ineffective due to their inability to capture the semantic meaning and context of words. the architecture of our proposed trans-sent model is shown on figure 1 . the primary issue we encountered was that traditional models relied on word frequency-based representations, which struggled with incomplete or ambiguous text samples. given that emotions are highly context-dependent and often co-occur, we needed a model capable of understanding semantic nuances and bidirectional dependencies. this led us to transition to a bert-based approach, leveraging the pre-trained models like bert-baseuncased, ai-forever/rubert-base etc and fine-tuning it for our specific task. the methodology behind working of trans-sent is explained below. ## experimental setup to fine-tune the bert-based model for multilabel emotion classification, we employed a structured training strategy designed to optimize performance while preventing overfitting. the optimization process was carried out using the adamw optimizer, an improved variant of adam that includes weight decay correction to mitigate over-regularization of important parameters. the learning rate was set to 2e-5, a commonly used value for fine-tuning transformer-based models, ensuring a balanced trade-off between convergence speed and stability. a batch size of 8 was chosen, considering the computational constraints of training large-scale transformer models while maintaining a sufficient number of samples for each gradient update. to ensure effective learning, the model was trained for 3 epochs, as preliminary experiments indicated that performance plateaued beyond this point, and additional epochs led to minimal improvement while increasing the risk of overfitting. weight decay was set to 0.01 to regulate the model's parameters and reduce the effect of less significant features, thereby enhancing generalization. ## conclusion we introduced trans-sent, a transformer-based model for multi-label emotion detection in semeval-2025 task 11, leveraging pre-trained bert models to classify joy, sadness, anger, fear, surprise, and disgust across seven languages. our approach combined data preprocessing, smotebased oversampling, and fine-tuning, achieving competitive results, with russian ranking ninth overall. performance varied across languages due to morphological complexity, availability of pretrained models, cultural nuances, and tokenization challenges. while russian, marathi, and hindi performed well, german and amharic faced difficulties due to grammatical complexity and ambiguous sentence structures. we have also highlighted the impact of data imbalance, as some emotions appeared more frequently than others, influencing classification accuracy. traditional machine learning models struggled with the nuances of multi-label classification, reaffirming the effectiveness of transformer-based architectures. despite its strengths, trans-sent has room for improvement. future work could explore better data augmentation, cross-lingual learning, and ensemble models to refine classification. additionally, incorporating context-aware modeling and multimodal data could enhance accuracy. expanding to low-resource languages through domain-specific fine-tuning is another promising direction. in international conference on machine learning (pp. 3069-3077). pmlr.