N ot M y N arrative at S em E val-2025 Task 10: Do Narrative Features Share Across Languages in Multilingual Encoder Models? (2025)

Abstract: Narratives are a new tool to propagate ideas that are sometimes well hidden in press articles. The SemEval-2025 Task 10 focuses on detecting and extracting such narratives in multiple languages. In this paper, we explore the capabilities of encoder-based language models to classify texts according to the narrative they contain. We show that multilingual encoders outperform monolingual models on this dataset, which is challenging due to the small number of samples per class per language. We perform additional experiments to measure the generalization of features in multilingual models to new languages.

## introduction with the complexity of current geopolitical events, persuasion techniques have become less explicit in online content. shared content often share a vision of the world used to interpret current events, which can influence the world vision of the readers. these are called narratives, and automatically detecting them has become a topic of interest for the machine learning community @xcite . narratives can also be stated explicitly, but are more harmful when they are implicit in the text, like persuasion techniques are. in this paper, we propose a multilingual approach (english, european portuguese, hindi, bulgarian, and russian) to identify whether or not a predefined narrative is present in a text and, if that is the case, what narrative it is. it is based on a standard multilingual encoder with a unique classification head for all narratives of the task. we find that multilingual models perform better than individual monolingual models, using all of the provided data by the task organizers. while our proposed approach is not trying to be the bestperforming (only in the top 50% of teams for only two languages), it relies on light language models that run on modest hardware and works the same for all languages. ## background we propose a system for the subtask 2: narrative classification. the problem is framed as the following: given a text, identify if the text contains one, several, or none of the narratives defined by @xcite . the proposed narratives are part of a two-level taxonomy. however, we chose to ignore the additional information from the higher-level labels and focused directly on finegrained narrative classification, which is the main focus of the task and on which the narrative used for the leaderboard is based. the problem is multiclass and multi-label, with 93 narratives to detect, some of which only appearing in some languages. the distribution of the number of occurrences for each class is given in figure 1 . the articles are the length of a regular news article, with about 410 words on average. they cover either news about climate change or the ukraine-russia war and exceptionally contain narratives related to the two topics. each article has an average of 2.3 labels and a median of 2 labels. given the limited number of samples per class in the dataset, we chose to work in a multilingual setting to maximize the number of samples seen by class during training. another challenge of the task is the multi-label constraint. usual mono-label classification uses a softmax activation function, outputting probabilities for each class, even when out-of-distribution. this is not possible for multi-label classification for which several labels can be applied to the same text, requiring additional steps. ## system overview our proposed system is based on encoder language models trained solely on the provided data for the task. the choice of encoder models is motivated by their wide use in text classification tasks, especially for misinformation detection @xcite . the encoder produces an embedding that is then processed by a two-layer classification head with a sigmoid activation function and 94 output neurons, one for each class plus one for the absence of narrative. during training, we consider that each neuron with an activation over 50% is activated. however, preliminary experiments showed that this setting could not be kept for inference, with all neurons activating at values below this threshold for almost all test samples. to solve this problem, we propose an adaptative threshold for multi-label classification based on the activation of the no narrative class neuron. if this neuron is the most activated, it means that the absence of a narrative is more plausible than the presence of any narrative seen during training. each narrative corresponding to a neuron more activated than the no narrative neuron is considered present in the text. this neuron could be considered as the neutral or control class, determining if one of the training classes is found in the text. figure 2 shows a global schema of the system. one point of interest for our study is the multilingualism of model embeddings for narrative classification. several types of state-of-the-art models were used: monolingual models are trained with the corresponding language data. multilingual models were used for two types of experiments: ## experimental setup the given train data is split in two with a random 80/20 split. models are trained on the first 80% and evaluated on the remaining 20% at the end of each epoch. models are trained for a maximum of 100 epochs, and an early stopping strategy with a patience of 5 is used. if the f1 score on the fine narratives on the 20% of data does not improve for five epochs, the model is restored to its state with the best f1 score. the dev data has been used for evaluation, and the reported results are computed for this split. the final models used for the test submission are chosen per language, based on the configuration giving the best f1 score on the fine narratives on the dev split. we report the f1 scores on the coarse and the fine narratives for each experiment. each model is trained with a batch size of 8 and a learning rate of 10 -5 with an adamw optimizer @xcite , which is a common default choice for such models. models come from huggingface and the transformers library. the model and classification head are wrapped in a pytorch lightning 7 lightningmodule. because classes are unbalanced, we use a sampler from the pytorch-multilabel-balancedsampler module 8 , and more specifically the leastsampledclasssampler, which returns a random sample with a label from the least sampled class at each moment. ## conclusion and future works in this paper, we explored the capabilities of multilingual encoder-based models for the task of narrative classification. we proposed a method with an adaptative threshold for multi-label classification tasks and showed that it performs reasonably well, especially for high-resource languages. additional experiments on language ablations showed differences between models' behavior, with xlm-roberta generalizing better on unseen languages, but mdeberta generally performing better when trained with all languages. the proposed approach could be enhanced by using data augmentation and hierarchical classification; ideas proposed by @xcite @xcite . for real use cases, performance on the coarse labels may be more important to detect the presence or absence of narratives before using more specialized models if needed. the main challenge for our model was the limited number of samples by class, which the addition of new annotated data could alleviate. in addition to that, the proposed system only works with a pre-defined set of initially defined narratives. it could be possible to reuse the adaptative threshold idea to detect when new narratives appear in new articles. moreover, other thresholding strategies could be used, by instance by adding a margin around the adaptative threshold in order to maximize either precision or recall, depending on the use case.