clujteam at S em E val-2025 Task 10: Finetuning S mol LM 2 with Taxonomy-based Prompting for Explaining the Dominant Narrative in Propaganda Textt (2025)

Abstract: XAI has been a long-standing goal of AI. Explaining why a text can be considered to have a dominant narrative, where the narrative is known, is of great importance for dealing with propaganda in news. This paper reports on the participation of the system clujteam in Subtask 3 of Task 10 of Semveal 2025. The system obtained 7th place with a value of 0.72464 for F1macro, at 0.026 distance from the 1st place. The key components of the solution are the011given taxonomy and supervised fine-tuning of SmolLM2.013

## introduction unfortunately, propaganda detection has become a crucial task in today's world. understanding the dominant narrative and subnarratives, as well as identification of the different roles an entity can play, are two key aspects of propaganda identification. task 10 of semeval 2025 @xcite , multilingual characterization and extraction of narratives from online news, introduces 3 main subtasks in five languages: entity framing, narrative classification, and narrative explanation. this paper reports the results obtained by the system clujteam in the official competition for subtask 3 in english. additionally, post-competition experiments are reported for subtask 1. subtask 3 is framed as a text-generation task where the input is a news text and a dominant narrative, and in some cases, the dominant subnarrative. the expected text must capture the explanation for having this subnarrative/narrative as the dominant one from the taxonomy. the evaluation relies on bertscore and the primary metric is f1macro. ## system overview our solution for subtask 3 is based on supervised fine-tuning (sft) of a small language model using the data provided in the competition. specifically, we use smollm2 1.7b-instruct and 360m-instruct @xcite , which offer a favorable balance between model size and performance. ## results in the official leaderboard of subtask 3 of task 10, our system clujteam is positioned in the 7th place with a value of 0.72464 for f1macro. the best system obtained a value of 0.75040. as expected, the smollm2 1.7b obtained the best results, even though mixed precision is used the first observation is that the 360m model performance is not significantly lower than that of the 1.7b model. when comparing the performance of the 360m model to that of the 1.7b model, it's important to note that the sft for the larger model was conducted using mixed precision and an 8-bit optimizer. the second observation is that the 360m model achieves acceptable performance despite the limited size of both the training dataset and the model itself. this is justified by the fact that all versions of smollm2 are prepared for text summarization and rewriting. the third observation is that the taxonomydependent prompt does not improve the performance of the 360m model, but rather slightly the opposite. we can just assume that this is due to the size of the model and the increased length of the system prompt. the 360m model obtained better performance when the taxonomy-independent prompt is used, at least according to the used metrics that rely on bert score. nonetheless, the text generated by the models that use the taxonomydependent prompt is more meaningful. the fourth observation, and the most important in our view, is that the taxonomy-dependent prompt increases the performance for the 1.7b model. the model using taxonomy-dependent prompts is significantly more precise than the one with a general prompt (0.723 vs 0.709 on the test set). the fifth observation is that even though the metrics do not show a significant difference between the quality of the sft with the taxonomydependent prompt compared to sft with the independent one, the generated text tends to be more meaningful when the definition of the narrative statements is included. for example, the following text is generated by the taxonomydependent model: the text suggests that the climate agenda has hidden motives, such as depopulation and population control. the author argues that the climate agenda is a tool for globalists to control the population and implement their plans for a one-world government. the text also criticizes the media. the taxonomyindependent model explains the dominant narrative of the same text with: the text discusses the climate agenda and its hidden motives. the text also talks about the agenda of powerful groups and their hidden plots. the text also talks about the agenda of the powerful groups and their hidden motives. ## post-competition experiments in this section we describe our post-competition experiments for subtask 1 using the same sft on smollm2. the input for subtask 1 is a news article and a list of entity mentions; it is required to classify the role and the subrole of the given entities using a predefined taxonomy of fine-grained roles. the subrole is not unique, so it is a multilabel classification. the same entity mention can occur several times in one text, not necessarily with the same role or subrole. ## conclusions the paper describes the participation of clujteam system at subtask 3 of semeval task 10. our experiments indicate that taxonomies defined for the narratives and subnarratives play a key role in obtaining a model that generates precise explanations in resource-constrained settings.