zhouyijiang1 at S em E val-2025 Task 11: A Multi-tag Detection Method based on Pre-training Language Models (2025)

Abstract: In order to effectively predict the speaker’s informing emotion from text fragments, we propose a transfer learning framework based on the BERT pre-training model through deep semantic feature extraction and cascade structure of dynamic weight linear classifier. In the speaker informing emotion prediction task, a 0.70 F1 score is achieved, illustrating the effectiveness of cross-domain emotion recognition.

## introduction our team participated in track a multi-tag emotion detection in task 11, "bridging the text-based emotion detection gap". emotion detection (sentiment analysis, sa), as an important research direction in the field of natural language processing (nlp), mainly uses computers to automatically process large-scale comment texts, identify the text information contained in the text, and mine the emotional tendencies expressed in people's texts, so it is also called opinion mining @xcite . identifying emotion categories in the text is an important task in nlp and its applications @xcite . through the analysis of various forms of information in the dialogue, we can more accurately identify the sources and causes of emotion. this is significant in various fields, including psychology, human-computer interaction, and emotional computing. it is helpful to develop more intelligent and human-centered techniques and systems, improve the efficiency and quality of communication, and promote better understanding and communication between individuals @xcite . for this reason, this study constructs a model that can predict multiple tags simultaneously by building a bert model, combined with the needs of multi-tag classification @xcite . the model training process includes key steps such as data loading process optimization, adaptive optimizer setting, and binary cross entropy loss function configuration, and introduces dynamic learning rate scheduling strategy and early stop mechanism (if the macro-f1 value of 5 consecutive rounds of verification sets is not increased, the training is terminated) to balance the model efficiency and generalization ability. this article will describe in detail how to use bert for multi-tag emotion detection @xcite . ## background as the core task of nlp, emotional analysis shows great application value in the fields of public opinion monitoring, mental health assessment (tausczik and pennebaker, 2010), and intelligent customer service. industry analysis shows its market size will exceed 28 billion us dollars in 2025. early multi-tag emotion detection relies on text template matching or shallow machine learning models such as svm, but these methods are limited by the semantic representation defects of artificial feature engineering @xcite , so it is difficult to capture emotional interactions in complex contexts (such as "irony in humor"). although the introduction of rnn and cnn's deep learning model improves context awareness, its one-way or limited window semantic modeling still cannot solve the coupling problem of long-distance dependent emotional cues @xcite . the pretraining language model represented by bert @xcite implements deep bidirectional context coding through full-stack transformer architecture, which significantly improves the base linearity of multitag tasks. ## system overview in this section, we introduce the method of multitag detection, and we use the bert pre-training model for text sequence processing and calculation. our method is to input the original text into bert and then use the pre-trained token embedding knowledge @xcite and the self-attention structure of bert to directly transform the text into the corresponding feature vector, in which the first bit [cls] of the vector is used alone for downstream classification tasks. we predict emotion from a text fragment first to get two sentences that belong to the context, and we add some special token to the two consecutive sentences: [cls] the last sentence, [sep] the next sentence. [sep]. as shown in the following figure, token embedding is the embedding matrix of the word vector; segment embedding is the boundary between the upper and lower sentences; and position embedding is the position embedding, which can be added by the alignment of the three embedding elements. ## experimental setup this study conducted model validation on the multilingual emotion analysis benchmark dataset from semeval, with the english subset selected as the central evaluation targetcite @xcite . the training set comprises 15,000 manually annotated social media short texts (average length 28 words), while the validation and test sets contain 3,000 and 2,000 samples, respectively, covering fine-grained annotations of five fundamental emotions: anger, fear, joy, sadness, and surprise. to mitigate data bias, a stratified random sampling strategy was employed to ensure a proportional representation of each emotional category across all three datasets (positive case proportions ranging approximately @xcite . representative examples of dataset inspection results are presented in the model architecture is based on the bertbase-uncased pre-trained language model. the original pooling layer was removed, and a 768dimensional context vector extracted from the [cls] token position in the final layer serves as the global semantic representation. a fully connected network is used in the output layer to predict fivedimensional emotion probabilities, with weights initialized using the xavier normal distribution to accelerate convergence. during training, the parameters of the first 8 bert layers were frozen while fine-tuning the higher-level network layers, combined with a mixed precision training strategy to reduce gpu memory consumption. the optimizer utilizes the adam algorithm with an initial learning rate of 3e-5 and regularization of l2 (@xmath0.001) @xcite , accompanied by a step-based learning rate scheduling policy (step @xmath1, decay @xmath2.1). a dynamic early stopping mechanism monitors the validation set macro-f1 score, terminating training after 5 consecutive epochs without performance improvement to prevent overfitting @xcite . the performance of the model on this dataset is in ## conclusion this article describes our participation in the semeval-2025 competition. we participate in track a multi-emotion tag detection, and our method uses the bert pre-training model for text sequence processing and calculation. in the training process, we combine the optimizer, learning rate schedule, early stop mechanism, and other technical means. through the reasonable selection of hyperparameters, loss function and optimization strategy, we can effectively train a high-quality text classification model on a given data set, and can quickly evaluate the performance of the model in practical application. with the help of bert's ability to understand context semantics, we improve the accuracy of emotion analysis and provide a new idea for the follow-up study of daily text emotion analysis. in future research, we can continue to improve the performance of the emotion analysis method from the following aspects: consider the integration of focus technology to improve the attention of the bert model to some core terms. these in-depth studies will help to grasp the speaker's perceived emotion.