S em E val 2025 Task 10: Multilingual Characterization and Extraction of Narratives from Online News (2025)

Abstract: We introduce SemEval-2025 Task 10 on Multilingual Characterization and Extraction of Narratives from Online News, which focuses on the identification and analysis of narratives in online news media. The task is structured into three subtasks: (1) Entity Framing, to identify the roles that relevant entities play within narratives, (2) Narrative Classification, to assign documents fine-grained narratives according to a given, topic-specific taxonomy of narrative labels, and (3) Narrative Extraction, to provide a justification for the dominant narrative of the document. To this end, we analyze news articles across two critical domains, Ukraine-Russia War and Climate Change, in five languages: Bulgarian, English, Hindi, Portuguese, and Russian. This task introduces a novel multilingual and multifaceted framework for studying how online news media construct and disseminate manipulative narratives. By addressing these challenges, our work contributes to the broader effort of detecting, understanding, and mitigating the spread of propaganda and disinformation. The task attracted a lot of interest: 310 teams registered, with 66 submitting official results on the test set.

## introduction the internet has opened vast possibilities for creating direct communication channels between producers and consumers of information, potentially leaving the latter exposed to deceptive content and attempts at manipulation. huge audiences can be affected online, and major crisis events are constantly subjected to the spread of harmful disinformation and propaganda. this creates a growing demand for tools that assist media experts in analyzing the news ecosystem, detecting manipulation attempts, and studying how media worldwide engage with topics of global interest, including the arguments and techniques used to influence public opinion. to foster research and development in this direction, a number of shared tasks have been organized over the years. this includes semeval-2020 task 11 on detection of persuasion techniques in news articles @xcite ; semeval-2021 task 6 on detection of persuasion techniques in texts and images @xcite ; constraint 2022 shared task on detecting the hero, the villain, and the victim in memes @xcite ; semeval-2023 task 3 on detecting the category, the framing, and the persuasion techniques in online news in a multi-lingual setup @xcite ; semeval-2024 task 4 on multilingual detection of persuasion techniques in memes @xcite ; and clef 2024 task 3 on persuasion techniques @xcite . our new task, named multilingual characterization and extraction of narratives from online news expands on the previously mentioned tasks to explore new dimensions in the context of news analysis. the task focuses on the identification of narratives in the news, identification of entity roles, classification into dominant and sub-dominant narratives, 1 and the justification of the choice of dominant narrative. we cover news articles from two domains-ukraine-russia war (urw) and climate change (cc)-in five languages: bulgarian, english, hindi, (european) portuguese, and russian, making this a multi-lingual multi-faceted task. by systematically identifying and analyzing narratives across multiple languages and domains, this task contributes to a deeper understanding of how disinformation is framed and disseminated, providing valuable insights into how specific viewpoints gain traction and influence public perception. by detecting recurring and evolving narratives, this work lays the foundation for more effective countermeasures against disinformation, supporting journalists, fact-checkers, and policymakers in mitigating its societal impact. the paper is organized as follows. section 2 introduces the three subtasks. section 3 surveys related work. section 4 describes the dataset and its creation process. section 5 gives an overview of the evaluation. section 6 presents the results of the competition and comparison of the participant systems. section 7 concludes with a summary of the task. ## the tasks in this section, we describe the three subtasks of semeval 2025 task 10: (1) entity framing; (2) narrative classification; and (3) narrative extraction. subtask 1 (st1) entity framing: given a news article, such as in figure 3 (top), and a list of mentions of named entities (nes) contained therein, assign to each mention one or more roles from a predefined taxonomy of fine-grained roles. formally, let r be a tree structure with k nodes that represents the taxonomy of roles. let s be a string of length |s| characters which contains the article. the goal of entity framing is to learn a function f : (s, [i, j]) → {-1, +1} k protagonist guardian: heroes or guardians who protect values or communities, ensuring safety and upholding justice. martyr: individuals who sacrifice their well-being, or even their lives, for a greater good or cause. peacemaker: individuals who advocate for harmony, resolving conflicts and bringing about peace. rebel: revolutionaries who challenge the status quo and fight for significant change or liberation. underdog: entities who, despite a disadvantaged position, strive against greater forces and obstacles. virtuous: individuals portrayed as righteous, fair, and upholding high moral standards. ## related work we next discuss work related to the three subtasks considered in this paper. ## participants and results this section provides the official results on all three subtasks. for complete information with suplementary metrics, please see the official leaderboard on the test data. foot_6 ## conclusions and future work this paper describes semeval-2025 task 10 on multilingual characterization and extraction of narratives from online news. the task attracted a lot of attention: 310 teams registered for the task, of which 66 made an official submission on the test set, of which 40 submitted a task description paper. in future work, we envisage exploiting the datasets we created for of this task to explore and elaborate solutions for other related tasks-e.g., narrative classification at the paragraph level, unsupervised discovery of a taxonomy of narratives, detection of entities central to a narrative, and predicting the dominant narrative in a document based on an explanatory text.