adithjrajeev at S em E val-2025 Task 10: Sequential Learning for Role Classification Using Entity-Centric News Summaries (2025)

Abstract: There is a high prevalence of disinformation and manipulative narratives in online news sources today, and verification of its informative integrity is a vital need as online audience is highly susceptible to being affected by such propaganda or disinformation. The task of verifying any online information is, however, a significant challenge. The task Multilingual Characterization and Extraction of Narratives from Online News, therefore focuses on developing novel methods of analyzing news ecosystems and detecting manipulation attempts to address this challenge. As a part of this effort, we focus on the subtask of Entity Framing, which involves assigning named entities in news articles one of three main roles ( Protagonist, Antagonist, and Innocent) with a further fine-grained role distinction. We propose a pipeline that involves summarizing the article with the summary being centered around the entity. The entity and its entity-centric summary is then used as input for a BERT-based classifier to carry out the final role classification. Finally, we experiment with different approaches in the steps of the pipeline and compare the results obtained by them.

## introduction there has been a rapid growth in the field of digital media over the past few years, and this has allowed for much easier dissemination and consumption of information. this development has double-edged outcomes: it allows for more diverse content, and for more real-time engagement with customers, yet it also enables the rapid spread of false information and narratives that are made to influence public opinion or spread propaganda. consequently, there is a dire need of developing automated approaches that can interpret the framing of entities within a news environment. the semeval task on multilingual characterization and extraction of narratives from online news @xcite has been launched to promote research and develop novel approaches to analyse a news environment and characterize possible attempts at manipulation. mainly, the first subtask, entity framing is centered around the classification and identification of the roles relevant entities involved from a news article. a model is required to correctly assign one or more roles to each entity solely based on the context provided by the article, and the locations of entities within it. we implement this task in a pipeline with 2 primary phases: 1) entity-centric summarization, and 2) role classification based on the summary. rather than feeding a classifier model the entire news article and the entity directly, we compress the news article into a summary that surrounds the entity in the first phase. this enables the classifier in the second phase, which has a limited token capacity, to focus on a condensed and compact input. the modularity of the pipeline also allows each phase to employ distinct models, and we observe the overall performance to reflect the combined contributions of the models operating at each phase of the pipeline. for the first phase we experiment with some models fine-trained on entity-centric summarization, and also with llms as their flexibility allows them to easily adapt to a new task. most of our analyses, however, is focused on the second phase of the pipeline, where we vary the base models and improve on our training methods to obtain better results. here, our best results are obtained when we perform sequential training on 3 inter-related tasks: 1) main role classification, 2) fine-grained role classification, and 3) contrastive learning to push the inputs towards a fine-grained class description. ## related work previous works in this field have focused on persuasion techniques, framing in the news genre, and on classifying the roles of entities in memes. for instance, @xcite propose an nlp framework to measure entity framing in the specific case of police violence in the us and demonstrates a difference in how the liberal and conservative news sources frame issues. additionally, the task of detecting the roles of entities in memes has been addressed by @xcite . this study focuses on classifying entities within memes to the categories of hero, villain, victim or other. these studies play a role in having an understanding of entity framing across different media formats, and shows the importance of analyzing how entities are portrayed to uncover underlying biases and narratives. ## dataset the dataset consists of news articles covering two major topics-the ukraine-russia war and climate change-collected from various news aggregation sites between 2022 to mid 2024. the dataset spans 5 languages: bulgarian, english, hindi, (european) portugese, and russian. each article is annotated @xcite with the locations of the named entities and their corresponding role. the role assigned to each entity follows a two-level taxonomy: additionally, the predefined taxonomy of the finegrained roles contain a detailed description of all 22 fine-grained roles. while additional annotations exist for other subtasks, these are all the relevant ones to the current subtask. for our experiments, we use the english subset of the dataset, which contains 3 splits: train, development(dev) and test. the test set labels have not been released yet, so all evaluations were conducted on the dev set. ## methodology our main approach to the entity framing task is to implement a two-phase pipeline that uses the article and entity to make an entity-centric summary which is then used to classify the role of the entity. this allows for flexibility in model selection at each phase-different summarization techniques can be explored without drastically altering the classification model's architecture. similarly, improvements can still be made to the classifier without modifying the summarization process. ## results and discussion as the test set labels have not been released, the analysis has all been done on the dev set and those are the reported scores. only the final submission has scores for the test set. we have limited our experiments and analysis to the english language. for each model, the main metrics reported are the exact match ratio, micro p, micro r and micro f1 for the fine-grained roles. an accuracy for the main role prediction is also reported. the official evaluation metric is, however, the emr for the fine-grained roles. we began with a simple bert classifier and then moved to a two-task sequential setup using deberta as the backbone model, which brought noticeable performance gains. further improvements were observed with introducing the third contrastive learning task in addition to main role and fine-grained role classification. we also evaluated the effect of the different entity-centric summarizers: ctrlsum and gemini. our best results were achieved using gemini as the summarizer in combination with the twotask sequential setup, achieving the highest emr and micro f1 scores. however, it is worth noting that ctrlsum, despite being a smaller and more efficient summarizer that runs locally (unlike gemini, which relies on a large llm, and making api calls), still produced competitive results-especially when combined with the threetask training setup. this makes ctrlsum a practical and resource-efficient alternative, particularly for environments with limited compute resources. ## conclusion and future work in this work, we explored the effectiveness of a dual phase approach, summarization and classification, in the entity framing task in news articles. for the classifier, we experimented with a sequential training method, incorporating main role classification, fine-grained role classification, and contrastive learning objectives, and this allowed the model to learn hierarchical relationships between main roles and their fine-grained classifications, while the contrastive learning objective helped in creating more discriminative feature representations. while this approach showed promise, achieving an emr of 0.234, we discovered that the quality of the initial summarization phase had an even more substantial impact on overall performance. despite the controlled summaries obtained with ctrlsum, our best results (emr of 0.2637) were achieved using gemini as the summarizer with a simpler 2-task sequential training setup, possibly due to its higher quality and diversity of summaries. a key challenge was the data imbalance in both main and fine-grained roles ( our findings highlight the importance of highquality summarization techniques in framing tasks and suggest promising methods for improving automated narrative analysis in news media.