TIL e N at S em E val-2025 Task 11: A Transformer-Based Model for Sentiment Classification Applied to the R ussian Language (2025)

Abstract: We present our approach to tackle the Sentiment Classification Task. The task was divided into 3 categories: 1) Track A: Multi-label Emotion Detection 2) Track B: Emotion Intensity, and 3) Cross-lingual Emotion Detection. We participate in subtasks 1 and 2 for the Russian language. Our main approach is summarized as using pre-trained language models and afterwords working with fine-tuning aside the corpora provided. During the development phase, we had promising outcomes. Later during the test phase, we got similar scores to the Semeval baseline. Our approach is easy to replicate and we proportionate every detail of the process performed.

## introduction nowadays, a significant portion of human communication takes place through text-based platforms such as social media, emails, and messaging applications. understanding the emotions embedded in these interactions is crucial for enhancing user experience, analyzing public opinion, and even detecting mental health issues. text-based emotion detection (tbed) is a field within natural language processing (nlp) that aims to identify and classify emotions in written text using machine learning and artificial intelligence techniques. this technology has diverse applications, ranging from sentiment analysis in social media to personalized content recommendations and customer service improvement. however, accurately detecting emotions in text remains a challenging task due to the inherent ambiguity of language, subjectivity in emotional expression, and cultural differences @xcite . semeval-2025 task 11 seeks to identify the perceived emotion most people would associate with a speaker based on a given sentence or short text snippet. this task prioritizes interpreting emotions rather than the speaker's actual emotional state, the emotions elicited in the reader, or those of other individuals referenced in the text. its significance lies in enhancing the understanding of how emotions are conveyed and perceived in written language, considering the influence of cultural context, individual variations in emotional expression, and the inherent constraints of text-based communication @xcite . this task consists of three tracks: ## system overview we participated in tracks, a and b for the russian language since we got promising results during the development phase. both tracks were tackled using the same approach. we used the pretrainned model rubert based on bert for the russian language @xcite . this model is cased and it was trained with 12-layer, 768-hidden, 12-heads and 180m parameters. the corpora used for training consist of the russian part of wikipedia and news data. the authors used the training data to build a vocabulary of russian subtokens and took a multilingual version of bert-base as an initialization for rubert. after loading the pre-trained model, we finetuned it using the task data @xcite provided by the organizers. ## experimental setup as mentioned in the previous section, we consider the new corpora for training the text snippets of train + dev to train our models. ## results after the test stage, we had the following results. for the official ranking, organizers used the macro f1 score and we were positioned in the 31st place on the final ranking for this task. the average pearson r was the score used by the organizers to rank official results. the position obtained for this task was 17. the model a exhibited rapid convergence throughout training. starting with an initial loss of 0.1192 in epoch 1, the loss quickly decreased to 0.0315 by epoch 2 and further to 0.0199 in epoch 3, demonstrating efficient early learning. minor fluctuations were observed in subsequent epochs, but the overall trend remained strongly downward. notably, after epoch 5, the model consistently achieved very low loss values, reaching as low as 0.0054 by epoch 10. these results highlight the model's ability to effectively capture the data distribution and maintain stable performance throughout the later stages of training. in model b, the training loss shows a general trend of effective learning, especially in the early epochs. initially, there is a significant decrease in loss from 0.15 to 0.08 by the second epoch and further to 0.018 by the third, indicating rapid model convergence at the start. although some fluctuations are observed in later epochs (e.g., an increase at epoch 4 and a notable spike at epoch 7), the overall trend suggests that the model adapts well and corrects itself quickly, as seen by the drop to 0.031 at epoch 8 and reaching a near-zero loss of 0.0006 at epoch 9. the final loss at epoch 10 (0.062) remains low, demonstrating that the model maintains good performance and stability across training. both ranking results were positioned below the semeval baseline scores. however, we were closer to this baseline in track a. the distance from the baseline was short, i.e. 0.0168 for track a and 0.0695 for track b. ## conclusion we present a simple way to make predictions having multilabel classification for sentiment analysis. the system approach, actually uses the same process applied to both tasks, changes were made on the track b corpus to be considered as a multi-label binary classification task. the approach is based on a pre-trained language model for a specific idiom, and after performing some fine tuning we adjust the weights for the predictions. in this way, we consider that our approach is easy to replicate when you have some specific resources, especially on the pre-trained fact. considering this, we tried to participate in some other languages and due to the low digital resources we couldn't get good results. this problem is also visible in the baseline results provided by the organizers, for example in afrikaans language the baseline score is 0.3741. even though we couldn't overcome the semeval baseline, the results weren't disappointing. as a future work, we will perform additional aggregation to the corpus by applying paraphrasing, or even adding the track b corpus to track a, affirming the fact that all the intensity classes marked with values different from zero, means the presence of that sentiment as a value of 1 in track a.