W ord W iz at S em E val-2025 Task 10: Optimizing Narrative Extraction in Multilingual News via Fine-Tuned Language Models (2025)

Abstract: This paper presents our WordWiz system for SemEval-2025 Task 10: Narrative Extraction. We employed a combination of targeted preprocessing techniques and instruction-tuned language models to generate concise, accurate narrative explanations across five languages. Our approach leverages an evidence refinement strategy that removes irrelevant sentences, improving signal-to-noise ratio in training examples. We fine-tuned Microsoft’s Phi-3.5 model using both Supervised Fine-Tuning (SFT). During inference, we implemented a multi-temperature sampling strategy that generates multiple candidate explanations and selects the optimal response using narrative relevance scoring. Notably, our smaller Phi-3.5 model consistently outperformed larger alternatives like Llama-3.1-8B across most languages. Our system achieved significant improvements over the baseline across all languages, with F1 scores ranging from 0.7486 (Portuguese) to 0.6839 (Bulgarian), demonstrating the effectiveness of evidence-guided instruction tuning for narrative extraction.

## introduction the democratization of information on the internet has posed significant challenges in discerning and interpreting manipulative content. news ecosystems, particularly during crises, often serve as contested spaces where disinformation and propaganda narratives vie for public credibility and attention. consequently, the automated identification and characterization of narratives in multilingual news sources constitute a vital endeavor for content moderators, fact-checkers, and media literacy initiatives. the semeval 2025 task on multilingual characterization and extraction of narratives from online news addresses this issue through three subtasks spanning five languages english, bulgarian, hindi, portuguese, and russian @xcite . our research centers on subtask 3: narrative extraction, which entails producing a concise explana-tion (maximum 80 words) that substantiates a given dominant narrative using textual evidence from a news article. this text-to-text generation task necessitates both the precise detection of pertinent evidence and a coherent summary that aligns with the narrative classification. our methodology leverages recent advancements in large language models, employing two complementary approaches. first, we utilized supervised fine-tuning (sft) @xcite on pre-trained models, including phi-3.5, llama-3, and llama-3.1-8b. second, we applied direct preference optimization (dpo) @xcite to enhance model outputs by incorporating human preference data. to optimize performance across diverse linguistic contexts, we developed a robust preprocessing pipeline that eliminates duplicate sentences, standardizes text, and augments training examples with narrative-specific details. during inference, our system generates multiple candidate explanations by varying temperature parameters and selects the most apt one via semantic similarity scoring. experimental findings reveal that, although larger models frequently exhibit robust performance, the meticulously fine-tuned phi-3.5 model consistently yielded superior results across most languages. we delineate critical factors for effective narrative extraction, including the identification of relevant evidential sentences, the preservation of cross-lingual consistency in explanation structure, and the equilibrium between textual fidelity and succinct summarization. our system markedly outperforms baseline models across all evaluated languages. ## background semeval 2025 task 10 posed a complex information extraction challenge across five languages: english, bulgarian, hindi, portuguese, and russian. the dataset comprised news articles primarily covering two domains climate change (cc) and the ukraine-russia war (ua) with each article annotated for its dominant narrative and, when applicable, sub-narrative classifications. the training set contained a substantial number of multilingual examples, while the development set was comparatively smaller. both adhered to the same data structure: article text, narrative label, sub-narrative label, and gold-standard explanations. the task presented significant challenges in identifying evidence supporting the assigned narratives, particularly given the substantial variation in article length. the gold-standard explanations highlighted key textual evidence aligning with the assigned narrative without explicitly referencing its classification. our dataset analysis indicated that effective explanations required capturing the rhetorical strategies, entities, and argumentation patterns characteristic of each narrative type while maintaining linguistic and structural appropriateness for each language. the task organizers provided a baseline system utilizing zero-shot prompting with the microsoft phi-3-small-8k-instruct model. this approach relied exclusively on the model's pre-trained capabilities to interpret prompts and generate explanations, without any fine-tuning. research on information and event extraction from news articles remains in its early stages. sentence-level methods, frequently falling under the umbrella of timeline summarization (tls), typically generate linear sequences representing a single story's progression. recent variations on tls aim to capture more complexity, such as comparative timelines highlighting contrasting events between datasets @xcite or multi-timeline summarization (mtls) which extracts distinct parallel storylines from a corpus @xcite . alternative paradigms like summarize dates first reverse the typical tls pipeline by summarizing individual dates before selecting relevant ones @xcite . document and clusterlevel methods often employ more complex graph structures to represent interactions between multiple storylines and events, moving beyond simple linearity to capture convergences and divergences. recent advancements have incorporated diverse techniques and representations. for sentence-level extraction, methods like wilson utilize pagerank and bert embeddings within a divide-andconquer framework for date selection and summarization @xcite , while specialized approaches like texsl construct spatio-temporal storylines for disaster events using neural embeddings and integer linear programming @xcite . at the document level, optimization techniques continue to be refined, building upon earlier 'connect the dots' concepts to maximize coherence and other criteria, sometimes incorporating entity information and temporal decay factors @xcite . cluster-level approaches represent events as groups of documents, using techniques like temporal event maps (tems) based on lda and mutual information metrics @xcite , event phase oriented news summarization (epons) using structural clustering and random walks @xcite , or the iterative construction of story forests using classifiers and tree-based operations @xcite . ## methodology our approach to narrative extraction integrates two complementary methodologies. these methodologies leverage the strengths of large language models (llms) while addressing their limitations in generating concise, evidence-based explanations. ## experimental setup our experimental framework was designed to evaluate the effectiveness of our approach across multiple languages and model architectures. ## results the experimental results demonstrate the effectiveness of our approach across multiple languages and model architectures, consistently outperforming baseline systems. ## conclusion this paper presents the wordwiz team's solution for extracting narrative explanations from multilingual news articles. by combining advanced preprocessing techniques with two complementary fine-tuning strategies (sft and dpo) our system demonstrates substantial improvements over the baseline model across all five competition languages. the success of our approach underscores the effectiveness of targeted text preprocessing, which facilitates cleaner, more focused input for model training. our structured prompting strategy effectively guided model generation toward taskrelevant outputs. the multi-candidate generation with temperature-based sampling enabled the exploration of diverse response possibilities, while the selection of candidates based on narrative relevance ensured that final explanations were wellaligned with the intended narrative characterization. furthermore, our results suggest that, for specialized applications, model architecture and pre-training approach can be more critical than model size. future research could explore the extraction of evidence from news text to support narrative extraction. incorporating more advanced evidence extraction techniques could further enhance the grounding of explanations in the source material. extending our system's multilingual capabilities to encompass low-resource languages and investigating cross-lingual transfer learning may also expand the system's applicability to a broader range of global contexts.