T artan T ritons at S em E val-2025 Task 10: Multilingual Hierarchical Entity Classification and Narrative Reasoning using Instruct-Tuned LLM s (2025)

Abstract: In today’s era of abundant online news, tackling the spread of deceptive content and manipulative narratives has become crucial. This paper details our system for SemEval-2025 Task 10, focusing on Subtasks 1 (Entity Framing) and 3 (Narrative Extraction). We instruct-tuned quantized Microsoft’s Phi-4 model, incorporating prompt engineering techniques to enhance performance. Our approach involved experimenting with various LLMs, including LLaMA, Phi-4, RoBERTa, and XLM-R, utilizing both quantized large models and non-quantized small models. To improve accuracy, we employed structured prompts, iterative refinement with retry mechanisms, and integrated label taxonomy information. For subtask 1, we also fine-tuned a RoBERTa classifier to predict main entity roles before classifying the fine-grained roles with Phi-4 for the English language. For subtask 3, we instruct-tuned Phi-4 to generate structured explanations, incorporating details about the article and its dominant narrative. Our system achieves competitive results in Hindi and Russian for Subtask 1.

## introduction the internet has facilitated direct communication between information producers and consumers, making it easier for deceptive content and manipulative narratives to spread. to address this challenge, @xcite introduces the "multilingual characterization and extraction of narratives from online news". the task spans over five languages -bulgarian, english, portuguese, hindi, and russian. the task has three subtasks -entity framing, narrative classification and narrative extraction. in subtask 1, the dataset has news articles which have entities that need classification into main roles and then further into corresponding fine-grained roles. each article may have multiple instances of the entity but the task entails classification on a specific occurence. similar to subtask 1, subtask 2 focuses on assigning a single dominant narrative and one or more associated sub-narratives to a given news article. subtask 3 involves generating a concise explanation that supports the dominant narrative of a news article, given the article and the narratives. we participated in subtasks 1 and 3, by instructtuning microsoft's @xcite model and various prompt engineering techniques specific to these subtasks. our approach experimented with multiple large language models (llms), specifically llama @xcite , phi-4, roberta @xcite , and xlm-r @xcite . we utilized both quantized large models and non-quantized small models, discovering that the former performed better. to enhance classification accuracy, we structured prompts to enforce output format consistency, used an iterative refinement methodology with retry mechanisms to reduce llm generation errors, and incorporated label taxonomy information in prompts to improve performance. for subtask 1, we also fine-tuned a roberta classifier to predict main entity roles (with 72% accuracy) before refining for fine-grained role classification with phi-4 for the english language. for other languages, prompt engineering with instruction tuning to generate both main and fine-grained roles performed best. for subtask 3, we instruct-tuned phi-4 to generate structured explanations, incorporating details about the article and its dominant narrative. the need for scalable solutions capable of processing millions of articles daily necessitates an approach that prioritizes efficiency and minimizes resource consumption. we aim to build a system that effectively addresses the challenges of the task while maintaining computational efficiency. ## background the ## system overview this section presents the key algorithmic and modeling decisions behind our system. we rely exclusively on the dataset provided by the organizers and systematically examine a range of approaches, including large and small language models, in-context learning (icl) paradigms (zero-shot and one-shot), and instruction-tuning methods. due to compute constraints, we were unable to use large non-quantized models and instead relied on quantized large models and non-quantized small models. we observed that the quantized large models consistently outperformed their smaller non-quantized counterparts. our approach leverages specialized instruct-tuned quantized llms, where well-crafted prompts guide the llm, followed by lightweight fine-tuning using rslora. we address challenges such as multilingual training, long-document parsing, and robust text generation through targeted strategies. ## experimental setup our approach for subtasks 1 and 3 (figure 1 ) focused on instruct-tuning quantized llms, enhanced by prompt engineering. we utilized the provided training split for systematic instruction tuning, the validation split for model selection and performance assessment, and the blind test split to generate predictions. to address the multilingual nature of the task, we adopt a combined training strategy, leveraging data from all languages to create a unified instruct-tuned model. ## results our results ( we conduct two sets of ablations for subtask 1: model variant combined with output format (tsv vs. json) as in overall, instruct tuning with phi-4 yields the best results. ## conclusion our system demonstrates the effectiveness of prompt engineering and instruct-tuned quantized llms for multilingual entity framing and narrative extraction. we highlight the superior performance of quantized llms and the benefits of incorporating a hybrid model with roberta for main role prediction in english. additionally, we emphasize the positive impact of iterative refinement and structured output formats on overall accuracy. future research will focus on incorporating subtask 2 as pre-training for subtask 3, to refine narrative extraction quality. investigation into the impact of model size and alternative efficient fine-tuning techniques could also yield valuable insights. finally, enhancing multilingual generalization for roberta and xlm-r may yield better results.