BERT astic at S em E val-2025 Task 10: State-of-the-Art Accuracy in Coarse-Grained Entity Framing for H indi News (2025)

Abstract: We describe our system for SemEval-2025 Task 10 Subtask 1 on coarse-grained entity framing in Hindi news, exploring two complementary strategies. First, we experiment with LLM prompting using GPT-4o, comparing hierarchical multi-step prompting with native single-step prompting for both main and fine-grained role prediction. Second, we conduct an extensive study on fine-tuning XLM-R, analyzing different context granularities (full article, paragraph, or sentence-level entity mentions), monolingual vs. multilingual settings, and main vs. fine-grained role labels. Our best system, trained on fine-grained role annotations across languages using sentence-level context, achieved 43.99% exact match, 56.56 % precision, 47.38% recall, and 51.57% F1-score. Notably, our system set a new state-of-the-art for main role prediction on Hindi news, achieving 78.48 % accuracy - outperforming the next best model at 76.90%, as per the official leaderboard. Our findings highlight effective strategies for entity framing in multilingual and low-resource settings.

## introduction multilingual news narrative analysis has become increasingly important in the digital age. the rapid proliferation of social media has transformed information dissemination, enabling instant news access and the global spread of narratives. however, this connectivity also amplifies risks such as biased reporting, propaganda, and narrative manipulation. these are challenges that are particularly evident during conflicts and political upheavals. semeval-2025 task 10 on multilingual characterization and extraction of narratives from online news addresses these challenges by providing a platform to study entity framing across five languages (bulgarian, english, hindi, european portuguese, and russian) @xcite . our work specifically focuses on coarse-grained entity framing in hindi news, an important subset given its potential impact on public perception. entity framing is an interesting task that transcends surface-level lexical features to uncover the subtle semantic nuances in how entities are portrayed. it focuses on interpreting the contextual cues that reveal whether the entity is depicted as a protagonist, antagonist, or an innocent bystander. this analysis is fundamental for understanding media bias, as it illuminates how news authors strategically frame narratives to influence public perception and shape discourse. the entity framing task is described in-depth in @xcite ; @xcite . our experiments fall under two main approaches. news article entity framing result context extraction (e.g., sentence) according to putin, russia was forced ... to protect the population. russia [sep] putin says ...in ukraine [sep] according to putin, russia was forced to... protect the population. guardian tokenization entity_mention [sep] title [sep] extracted-context framing classification western countries were hoping to break russia quickly, ... since 2022. western countries [sep] title [sep] extracted_context saboteur ... russia managed to raise its economic sovereignty since 2022. russia [sep] title [sep] extracted_context our system virtuous putin says what russia needs to do to win special operation in ukraine ...according to putin, russia was forced to launch the special operation to protect the population. western countries were hoping to break russia quickly, but they were wrong, he said, adding that russia managed to raise its economic sovereignty since 2022. putin says what russia needs to do to win special operation in ukraine ...according to putin, russia(guardian) was forced to launch the special operation to protect the population. western countries(saboteur) were hoping to break russia quickly, but they were wrong, he said, adding that russia(virtuous) managed to raise its economic sovereignty since 2022. russia (1 st mention) western countries russia (2 nd mention) given an article, and a list of entity mentions (shown in boldface) for which framing is needed, we process one entity mention at a time. we then extract the context surrounding the entity mention (e.g., the sentence containing the entity mention). we tokenize the input if necessary, then we obtain the framing classification for that entity mention from the model (xlm-r or gpt-4o). this process is repeated for each entity mention. the intricate nuances of entity framing by balancing zero-shot prompting with fine-tuning strategies using state-of-the-art llms. in addition, our analysis provides valuable insights into the unique strengths and weaknesses of each approach, informing future research directions in multilingual narrative analysis. on the official test set leaderboard, our best system-trained on all languages using finegrained role annotations with sentence-level context-achieves an exact match ratio of 43.99%, micro precision of 56.56%, micro recall of 47.38%, and a micro f1 of 51.57%, and, notably, our system reached a state-of-the-art main role accuracy of 78.48%, ranking the top on main role accuracy on the leaderboard on hindi news. while these results illustrate the effectiveness of our approach, they also highlight ongoing challenges in modeling context and subtle linguistic cues in multilingual settings. ## system overview in this section, we describe our experimental methods for framing classification, outlining critical modeling decisions. we present two core strategies: fine-tuning multilingual transformers and applying hierarchical prompting with llms. key details of our approach are shown in figure 1 . ## results in this section, we report the performance of our system on the semeval-2025 task 10 as well as on additional experiments conducted using the development set. we focus our attention on main role accuracy, while noting that the exact match ratio on the fine-grained roles is the official leaderboard metric. ## conclusion and future work in this work, we present a comprehensive investigation into multilingual entity framing for news narrative analysis, with a focus on coarse-grained framing in hindi news. we consider two complementary approaches: (1) fine-tuning xlm-r under different context granularities, comparing monolingual and multilingual training setups as well as main versus fine-grained role label training, and (2) zero-shot prompting with gpt-4o, where we explore single-step and hierarchical prompting strategies to predict both main and fine-grained role labels our experiments demonstrate that leveraging sentence-level context and training on a diverse multilingual corpus yield superior performance, with our system achieving a state-of-the-art main role accuracy of 78.48% on hindi news. these results confirm that carefully selecting the appro-priate context window is critical to capturing the subtle semantic nuances of entity framing. while both approaches provide distinct advantages-finetuning leverages deep contextual representations to better model linguistic cues, and prompting enables rapid adaptation and zero-shot inference-each also faces challenges in processing the inherent complexities of multilingual narrative content. looking forward, our findings motivate further research on refining context-aware strategies and exploring more advanced fusion techniques for integrating heterogeneous contextual cues. future work may also extend our methods to a broader array of languages and narrative domains, ultimately contributing to the development of robust tools for detecting media bias and understanding public perception in a global news landscape.