Hidden Persuasion: Detecting Manipulative Narratives on Social Media During the 2022 R ussian Invasion of U kraine (2025)

Abstract: This paper presents one of the top-performing solutions to the UNLP 2025 Shared Task on Detecting Manipulation in Social Media. The task focuses on detecting and classifying rhetorical and stylistic manipulation techniques used to influence Ukrainian Telegram users. For the classification subtask, we fine-tuned the Gemma 2 language model with LoRA adapters and applied a second-level classifier leveraging meta-features and threshold optimization. For span detection, we employed an XLM-RoBERTa model trained for multi-target, including token binary classification. Our approach achieved 2nd place in classification and 3rd place in span detection.

## introduction in times of war, information can have the same power as weaponry. during the 2022 russian invasion of ukraine, telegram emerged not only as a battlefield communication tool but also as the primary source of information for 44% of ukrainians. its speed, reach, and anonymity became an important tool for civilians and military actors. however, these features -particularly minimal content moderation and user anonymity -have also made telegram a favorable environment for influence operations @xcite . manipulation on social media is a complex and nuanced phenomenon. it includes not just factual distortions (i.e., disinformation) but also rhetorical strategies, emotional appeals, and narrative framing that are designed to influence perception or behavior subtly. in this paper, we present the solution foot_0 to the unlp 2025 shared task, foot_1 focused on manipulative narratives detection, which is defined as the intentional use of language and messaging tactics aimed at influencing beliefs, emotions, or attitudes, without providing clear factual support. the task includes several challenges that make it particularly complex. first, it focuses exclusively on the textual content of social media posts without incorporating metadata such as user history or engagement metrics. second, the dataset presents multiple layers of complexity: it is imbalanced across manipulation types, multilingual (primarily ukrainian and russian), and multi-label, meaning that a single post can include several manipulation techniques simultaneously. finally, the span detection subtask requires identifying the exact textual fragments responsible for the manipulation, often implicit, rhetorical, or emotionally charged language that is difficult to isolate. given these challenges, we developed a system that achieved second place in manipulation techniques classification and third place in span detection subtasks (see figure 1 ). for classification, we fine-tuned the gemma 2 language model using lora adapters and introduced a second-level classifier that leveraged meta-features and custom threshold optimization. for span detection, we trained an xlm-roberta model capable of multitarget, token-level binary classification to locate manipulative spans within posts. ## related work our research is based on a growing body of work in detecting propaganda and misinformation analysis. numerous studies have focused on identifying propaganda techniques in news articles, particularly in the context of semeval-2020 task 11. da san martino et al. (2020) explored detecting propaganda techniques in news articles through span identification and technique classification tasks. similarly to previous research, the unlp 2025 shared task includes two subtasks: manipulation technique classification (a multi-label classification) and span detection (a token classification). within this framework, research from semeval-2020 task 11 demonstrated bert's remarkable capabilities for propaganda technique identification @xcite . further advancing this line of inquiry, da san martino et al. (2020) showcased roberta's performance in addressing both tasks simultaneously. at the same time, the nature of propaganda on social media evolves continuously, adapting to specific circumstances to remain undetected. @xcite explored this process by combining machine learning and linguistic analysis to reveal how pro-kremlin propaganda evolved in the context of the 2022 russian invasion of ukraine. in this context, it is important to note that while our work has a similar goal, we focus specifically on detecting manipulative narratives regardless of the factual support of the claim. this distinguishes our approach from fact-checking or knowledge manipulation detection methods @xcite . in our case, we are dealing with multilingual telegram data containing ukrainian and russian texts. in this scenario, fine-tuning a multilingual model, such as xlm-roberta, appears to be a more productive approach, as demonstrated in research on hostility identification for lowresource indian languages @xcite . moreover, xlm-roberta-based models have demonstrated cross-lingual strengths in other downstream tasks, including those involving ukrainian and russian languages @xcite . while @xcite discovered that fine-tuned roberta outperformed zero and fewshot learning approaches with llms for propaganda detection, newer advances in large language models show considerable promise. recent innovations have developed methods to transform decoderonly llms into effective text encoders suitable for classification tasks (behnamghader et al., 2024). models such as gemma offer particularly interesting customization potential for classification challenges @xcite . notably, gemma-family models enable finetuning with lora adapters and support quantization techniques, making them viable options even with limited computational resources. building on this foundation, @xcite ventured into fine-tuning both gemma and mistral specifically to enhance ukrainian language representation, providing valuable insights that directly inform our approach to detecting manipulative narratives within telegram content from the region. ## data the unlp shared task dataset contains more than 9,500 text samples collected from telegram channels, with 68% of these collected samples containing manipulative narratives. this dataset forms the basis for a dual-task challenge: classifying manipulation techniques and identifying corresponding text spans. the data is divided into training and testing sets, with 3,822 samples allocated for training and 5,735 for testing. among the 3,822 training samples, 2,147 (56%) are in ukrainian and 1,675 (44%) are in russian. at the same time, the testing set does not include language labels. notably, the testing set is further split into public and private sets for leaderboard evaluation. each post is annotated for both classification and span detection tasks. specifically, every sample is labeled with one or more of ten predefined manipulation techniques, detailed in appendix a. manipulative text segments are also defined, irrespective of the specific technique involved. figure 2 illustrates manipulation techniques' cooccurrence patterns across training and testing sets. as the distribution of labels is similar in both subsets, we present them together for clarity. ## methodology in this section, we present our approaches for solving the technique classification and span identification subtasks. ## https://huggingface.co/google/gemma-2-2b-it to control input length, we select the few-shot examples from the subset limited by texts shorter than 500 characters. to get a vector representation of the texts, we encode them using sentencetransformers, employing mgte model @xcite . later, these vectors are used for few-shot candidates selection and text clustering. as for this stage of model tuning, we used almost the whole training dataset, as our main goal was to expose the model to as much relevant data as possible rather than tuning to a specific downstream task. due to the high computational cost of full model fine-tuning, we instead trained lora adapter using a clm objective. the adapter was configured with causal lm task type via the peft library to ensure compatibility with the clm setup. finally, we got the fine-tuned adapter for the text generation in the form of a list of manipulation techniques. in the second stage, we merged the lora adapter from the first stage with the base model, set the model to a multi-label classification mode, and trained an additional lora adapter. the input for this stage consisted of text samples and their corresponding technique labels. in the third stage, we combined the probability outputs from the previous stage with a set of engineered meta-features to train a catboost model for multi-label classification on the same training set. the additional features include: to construct the clustering-based features, we applied the k-means clustering algorithm to the set of triggered phrases extracted from the training set. firstly, we encode the text with sentencetransformers as mentioned earlier. we set the number of clusters (k) to be @xmath0, equal to the number of unique manipulation techniques. finally, for each sample text, we calculate the cosine distance to the centroid of each cluster. this approach allows the model to capture how semantically close a text is to common manipulation patterns identified in the training data. for the similarity-based frequency features, we computed pairwise cosine similarity between the embedded texts. for each text, we selected two sets of 10 most similar examples from the training set: (1) based on overall similarity to other full texts, and (2) based on similarity to trigger phrases from other texts. we calculated the frequency distribution of manipulation techniques among the nearest neighbors in both cases. these techniques and meta-linguistic features (e.g., word count, presence of punctuation) were combined with model probabilities to train the final catboost classifier. finally, since the dataset is highly imbalanced, we optimized class-wise thresholds by performing k-fold cross-validation and choosing the median of the best thresholds within folds for each class separately. this approach avoids the pitfalls of using a single global threshold, especially for rare classes, and improves overall performance on the macro f1 score, which treats all classes equally. so, we used this method to construct the final prediction using the probability scores from the catboost model. ## conclusion to sum up, this paper presents a competitive solution to the unlp 2025 shared task on detecting manipulative narratives in ukrainian telegram news. by leveraging a multi-stage fine-tuned gemma 2 language model with lora adapters for technique classification and a two-headed xlm-roberta architecture for span detection, our approach secured second and third place in the respective subtasks. key achievements include a two-phase fine results show that each enhancement added measurable value. post-processing raised the classification macro-f1 from 0.45007 to 0.45447, while span detection improved from 0.58588 to 0.59888 with the dual-head setup. however, performance varied notably across manipulation types: while frequent classes like loaded_language were predicted with high accuracy, rarer classes such as whataboutism and straw_man remained challenging.