LTG at S em E val-2025 Task 10: Optimizing Context for Classification of Narrative Roles (2025)

Abstract: Our contribution to the SemEval shared task 10, subtask 1 on entity framing, tackles the challenge of providing the necessary segments from longer documents as context for classification with a masked language model. We show how simple entity-oriented heuristics for context selection and the XLM-RoBERTa language model is on par with, or outperforms, Supervised Fine-Tuning with larger generative language models.

## introduction shared task 10, subtask 1 for semeval 2025 @xcite presents annotated news articles in bulgarian, english, hindi, (european) portuguese, and russian. for each article, a number of entities are identified and annotated for narrative roles. these entities are in the text framed as playing one of three main narrative roles; protagonists, antagonists or innocent. for each main role, there is a number of fine-grained roles, 22 in total. each entity may be given multiple fine-grained roles available for the assigned main role. further details on the task can be found in the technical report @xcite . by classifying how entities are framed in news articles, we can better understand reporting angles and identify bias variations between different news sources. our contribution focuses on the following challenges in modeling the narrative roles of the annotated entities: a) each article may contain much irrelevant text with respect to a given entity. b) a sentence mentioning an entity may also mention other entities and frame each differently. c) the text segment(s) contributing to the entity framing may span multiple sentences. as seen in to answer these questions, our paper presents results from experiments regarding text preprocessing where we fine-tune xlm-r on various text segments and evaluate each pre-processing strategy. these results are compared against zeroshot prompting of a large language model (llm), and supervised fine-tuning of llms with 7-8 billion parameters. ## dataset pre-processing our contribution to the shared task was trained exclusively on the provided data. to address the above mentioned challenges for narrative role classification and answer rq1, we experimented with how to best prepare the data for fine-tuning and classification. ## modeling we here present the various modeling approaches tested, and their evaluation results. the focus is on answering the research questions by applying the various text pre-processing approaches as input for xlm-r fine-tuning. these approaches are compared with classification based on larger language models. ## analyses we here present our reflections on the results reported in the paper. ## conclusion we have created a xlm-r-based multilingual model for entity framing as a part our contribution to the semeval 2025 shared task 10 on multilingual characterization and extraction of narratives from online news, subtask 1. this model was tested against supervised fine-tuning of llama and mistral, and against zero-shot prompting of chatgpt-4o. we found it imperative to extract entity-oriented text segments in order to effectively utilize xlm-r with long documents containing multiple entities each.