nlptuducd at S em E val-2025 Task 10: Narrative Classification as a Retrieval Task through Story Embeddings (2025)

Abstract: One of the most widely used elements in misinformation campaigns is media framing via certain angles which in turn implies pitching news stories through a certain narrative. Narrative twisting to align with a political agenda includes complex dynamics involving different topics, patterns and rhetoric; there is however a certain coherence with respect to the media framing agenda that is to be promoted. The shared task’s objective is to develop models for classifying narratives in online news from a pre-defined two-level taxonomy (Subtask 2). In this paper, we discuss the application of a Mistral 7B model, specifically E5 model, to address theSubtask two in English about finding the narrative taxonomy that a news article is trying to pitch. Our approach frames the task as a retrieval task in a similarity matching framework instead of reliance supervised learning. Our approach based on the use of a Mistral 7B model obtains a F1 on samples of 0.226 and is able to outperform the baseline provided for the competition.

## introduction media framing, an interdisciplinary area of study @xcite , plays a central role in shaping online disinformation. semeval-2025 task 10 @xcite advances this field by introducing a richly annotated multilingual dataset focused on controversial topics such as climate change and the russia-ukraine conflict. this paper presents our submission for subtask 2: narrative classification of english news articles using a predefined two-level taxonomy @xcite . the task is framed as a multi-label, multiclass classification problem where each article can be associated with multiple narrative labels. prior work on narratives has primarily adopted an event-centric lens @xcite , which often overlooks propagandistic intent and disinforma-tion framing. in contrast, task 10 brings attention to narrative construction in the context of agendadriven reporting, introducing new challenges in narrative modeling and classification. to address these challenges, we frame narrative classification as a retrieval problem. this approach is motivated by several observations from the disinformation literature: motivated by these insights, we propose a narrative classification system grounded in topical coherence and semantic similarity. our system leverages story embeddings to retrieve semantically aligned training articles and transfers their narrative labels to test samples. figure 1 provides a high-level overview of this pipeline, where narrative detection is framed as a retrieval problem based on embedding similarity rather than supervised classification. we adopt the approach proposed by hatzel and biemann (2024), which utilizes contrastive learning and adapter-finetuned large language models (e5 variant) @xcite for robust narrative representation. this paper details our system pipeline, experimental results, observed limitations, and future research directions. ## system description to address the task of narrative classification, we adopt a retrieval-based approach inspired by story test article (from test set) top-n retrieval using cosine + story embeddings (mistral-7b) input training set (labeled corpus) similarity comparison final taxonomy (narrative labels) narrative label transfer figure 1: high-level overview of our subjectivity analysis pipeline embeddings proposed by hatzel and biemann @xcite . their method models narrative coherence through contrastive learning applied to narrative chains. we hypothesize that narrative labels in news articles can be transferred effectively based on semantic proximity in embedding space. therefore, rather than training a supervised classifier, we treat the problem as one of story retrieval and label transfer. our approach is influenced by common-sense reasoning tasks such as the story cloze test @xcite , where systems must choose the more coherent ending from two candidates given a four-sentence story. similarly, we embed a test article and compare it to candidate articles from the training set to identify the most semantically similar ones and inherit their labels. the overall system pipeline, previously introduced in figure 1 , proceeds through the following stages: given a test article, we compute cosine similarity between its embedding and those of all training articles using the e5 variant of mistral-7b @xcite . the process comprises the following steps: in implementation, we use the open-source story embedding model released by hatzel and biemann (2024) on hugging face. 1 no additional finetuning was performed on the model. the query prompt used to guide retrieval was: "retrieve stories with a similar narrative to the given story:" this framing implicitly guides the model to surface coherent narrative alignments. while this pipeline provides a simple and generalizable baseline, we recognize that direct label transfer introduces potential noise, especially in multi-label cases. we analyze this limitation and its impact on performance in the following section. ## conclusion and future work this paper presents our system description for narrative classification in subtask 2 of semeval-2025 task 10. we approached the task through a retrieval-based perspective using story embeddings derived from a mistral-7b model variant (e5), framing narrative detection as a similaritydriven retrieval challenge. our method leverages the narrative coherence and topical consistency typically embedded in disinformation, enabling us to transfer narrative taxonomy labels based on content proximity in the semantic embedding space. without any additional supervised fine-tuning, our system outperformed the competition baseline and achieved an f1 score of 0.226 on the samples. our analysis revealed both strengths and limitations of our pipeline. notably, the re-ranking of similar stories using embedding-based similarity rather than raw cosine similarity led to insightful improvements. however, we also observed the challenge of excessive or inaccurate label transfer during the final inference stage, highlighting the need for a more precise mechanism for label assignment. as part of future work, we intend to address these limitations by: overall, our findings suggest that framing narrative classification as a retrieval task is a promising direction, especially when paired with pre-trained embeddings that encode structural and semantic narrative features. we hope our approach sparks further innovations in retrieval-based disinformation and narrative analysis.