GPLSICORTEX at S em E val-2025 Task 10: Leveraging Intentions for Generating Narrative Extractions (2025)

Abstract: This paper describes our approach to address the SemEval-2025 Task 10 subtask 3, which is focused on narrative extraction given news articles with a dominant narrative. We design an external knowledge injection approach to fine-tune a Flan-T5 model so the generated narrative explanations are in line with the dominant narrative determined in each text. We also incorporate pragmatic information in the form of communicative intentions, using them as external knowledge to assist the model. This ensures that the generated texts align more closely with the intended explanations and effectively convey the expected meaning. The results show that our approach ranks 3rd in the task leaderboard (0.7428 in Macro-F1) with concise and effective news explanations. The analyses highlight the importance of adding pragmatic information when training systems to generate adequate narrative extractions.

## introduction understanding natural language goes beyond recognizing objects and their relations within a sentence or a news article. the way information is presented-or omitted-is shaped by the author's communicative intentions and implicit biases. to fully grasp the narrative conveyed by a text, one must move beyond a surface-level reading and incorporate assumptions about the author's intent, as well as general knowledge about the topic. with the rise of modern misinformation, automated approaches have become crucial in combating information warfare. however, these systems come with their own biases, making explainability a key factor in building trust. semeval-2025 task 10 addresses this challenge by combining the identification and explanation of narratives in text. our work focuses on subtask 3, which involves generating narrative extractions given a news article. large language models (llms) have been shown to encode general knowledge @xcite , and analytical capabilities @xcite to identify implicit narratives of a text. additionally, they hold promise for explainability, as they can be prompted to generate explanations @xcite . however, their explanations still fall short of human-level quality @xcite . our approach enhances llm-generated explanations as a form of controllable abstractive summarization @xcite using knowledge injection and standard fine-tuning to align the model's outputs with the required task format. a key contribution of our work is leveraging a curated dataset with labeled communicative intentions to assist the model's analytical capabilities based on speech act theory @xcite . we hypothesize that injecting this knowledge about intentions can control the model's focus and improve explanation quality. our approach ranked 3rd in the competition. ## system overview we propose a three-step approach for addressing the narrative extraction task foot_0 . our methodology begins by retrieving the narrative intention of the input article, as each article in the dataset exhibits a distinct underlying intention. identifying these intentions is crucial for shaping the generated explanations. next, we employ a prompt engineering strategy to construct an input prompt by combining the extracted intention with the corresponding dataset entry. finally, we inject the knowledge by fine-tuning instruction-tuned models to generate the corresponding explanations. the overall approach is illustrated in figure 1 and will be described in detail in the following sections. ## experimental setup in this section, we outline the experimental setup utilized in our experiments. ## results in this section, we report the obtained results through our experimentation, and our official results in the semeval 2025 task 10 subtask 3. ## analysis of the efficacy of injecting the intentions in this section, we aim to demonstrate that incorporating communicative intentions extracted from the text enhances the model's performance. to validate this, we fine-tuned the bestperforming model, flan-t5 in its larger configuration, without integrating the extracted intentions. we then compared its performance against the results obtained when intentions were included. tables 5 and 6 present the classification performance on the validation and test sets, respectively, using the bertscore metric. results show that the classification performance for the models enhanced with intentions is consistently higher than for the models without intent across both the validation and test sets. the intentions help to achieve higher precision, recall, and macro-f1 scores, indicating that those models are more confident and accurate in generating the explanations of the articles. ## post-competition after the conclusion of the competition, we entered a phase where we could systematically evaluate our approaches using the test set. during this process, we found it particularly insightful to explore the impact of different hyperparameter configurations on our best-performing approach. specifically, during the competition, we observed that certain hyperparameter settings yielded higher precision on the bertscore metric in the validation set, besides at the cost of reduced recall compared to the configuration used in our final submission, which was more balanced. this effect was particularly pronounced when adjusting the learning rate. the fine-tuned approach with a learning rate of 4e -5 outperforms our official competition submission, achieving a macro-f1 score of 0.75040 on the bertscore metric. notably, this configuration surpasses our official results not only in precision but also in recall. this configuration would have secured us 1st place in the competition, matching the score of the top-performing team. ## conclusions in this paper, we present our approach for the semeval-2025 task 10 subtask 3 in english, focused on the generation of explanations of articles containing disinformation. our method provides narrative intention knowledge to the model within a fine-tuning and hyperparameter-tuning process. our submission involves the fine-tuned flan-t5 large model, with which we ranked 3rd in the competition, achieving a bertscore f1 of 0.74280, only 0.0076 lower than the top-performing method. the results show the importance of intention analysis in exposing misinformation initiatives. the selected 13 global intention categories extracted from the scheme of @xcite have been proved to be beneficial for the task, as shown in section 6. in addition, this work has provided an analysis to determine the best-performing model configuration for the task. flan-t5 models have demonstrated satisfactory results. to effectively detect articles that disseminate disinformation, identifying the author's intent is crucial. however, our approach can be further enhanced by integrating additional perspectives. for instance, incorporating a propaganda strategy taxonomy could help identify specific linguistic tools employed by the author to propagate a particular narrative. furthermore, advanced techniques such as few-shot prompting with llama 3.1, sentiment analysis of dominant narratives, and the chain of thought reasoning framework could be utilized to improve the accuracy and depth of the analysis.