Detecting Manipulation in U krainian Telegram: A Transformer-Based Approach to Technique Classification and Span Identification (2025)

Abstract: The Russia-Ukraine war has transformed social media into a critical battleground for information warfare, making the detection of manipulation techniques in online content an urgent security concern. This work presents our system developed for the UNLP 2025 Shared Tasks, which addresses both manipulation technique classification and span identification in Ukrainian Telegram posts. In this paper, we have explored several machine learning approaches (LR, SVC, GB, NB) , deep learning architectures (CNN, LSTM, BiLSTM, GRU hybrid) and state-of-the-art multilingual transformers (mDeBERTa, InfoXLM, mBERT, XLM-RoBERTa). Our experiments showed that fine-tuning transformer models for the specific tasks significantly improved their performance, with XLM-RoBERTa large delivering the best results by securing 3rd place in technique classification task with a Macro F1 score of 0.4551 and 2nd place in span identification task with a span F1 score of 0.6045. These results demonstrate that large pre-trained multilingual models effectively detect subtle manipulation tactics in Slavic languages, advancing the development of tools to combat online manipulation in political contexts.

## introduction the war between russia and ukraine highlights the critical importance of developing reliable mechanisms to identify misinformation on social media platforms. among these platforms, telegram stands out as particularly significant, becoming a breeding ground for channels that spread misleading information, russian-favorable perspectives, and complete falsehoods targeting ukrainian users. contemporary russian information warfare strategies deliberately foster confusion, fracture public consensus, undermine institutional credibility, and construct distorted perceptions of reality @xcite . ai applications continue their expansion across various fields, gaining particular traction in information literacy-specifically addressing the detection and counteraction of disinformation phenomena that thrive within social media environments @xcite . the nuanced variety of manipulation techniques employed, spanning from emotion-laden rhetoric to intricate logical fallacies, creates substantial obstacles for natural language processing (nlp) systems. with the urgent need to counter online manipulation, the fourth ukrainian nlp workshop (unlp 2025) foot_0 convened a shared task devoted to this very issue. drawing on a ukrainian and russian telegram corpus supplied by texty.org.ua, participating teams developed and evaluated ai approaches with direct applications in both cybersecurity and disinformation research. the competition was structured around two complementary objectives: first, assigning each text to one of ten manipulation techniques, and second, precisely marking the character spans where manipulative tactics appeared. meeting these objectives requires models capable of detecting both overt cues and the more nuanced, context-dependent signals of manipulation. although earlier work on propaganda and related detection tasks has laid important groundwork @xcite @xcite , our task's focus on ukrainian and russian social media and its insistence on joint span identification and fine-grained technique classification offers a novel contribution that pushes the frontier of disinformation analysis. this paper presents our approach for the unlp 2025 shared tasks. we test and evaluate several methods, ranging from conventional machine learning techniques to advanced deep learning and transformer models. our key contributions include: the shared task comprised two subtasks: subtask 1 (technique classification), a multi-label classification over ten predefined manipulation tech-niques (e.g., loaded language, whataboutism) evaluated via macro f1-score; and subtask 2 (span identification), which required pinpointing character-level start/end indices of manipulative text segments irrespective of technique and was assessed using span-level f1-score. the implementation details and datasets for both tasks are available in the github repository 2 . ## methodology this section describes the methodologies employed for the technique classification and span identification tasks. the research evaluated multiple machine learning (ml), deep learning (dl), and transformer-based approaches, with hyperparameter optimization conducted to maximize performance. the architectural frameworks utilized for technique classification and span identification tasks are illustrated in figure 1 and figure 2 . figure 2: schematic process for manipulative span identification 2 https://github.com/borhanitrash/ detecting-manipulation-in-ukrainian-telegram/ ## result analysis this analysis covers three model families, machine learning, deep learning and transformer based systems on both technique classification and span identification tasks using ukrainian and russian telegram content. performance was measured by macro precision, recall and f1 score as shown in machine learning baselines defined the starting point. for technique classification logistic regression achieved the highest f1 of 0.3291, driven by strong recall of 0.5433 but lower precision. random forest reached precision of 0.5688 yet suffered recall of 0.1060, yielding an f1 of 0.1309. in span identification lightgbm led ml methods with an f1 of 0.4111 thanks to recall of 0.4794 and moderate precision. logistic regression and multinomial naive bayes tied at f1 0.3851, trading recall for higher precision. these classic approaches struggled to balance both metrics on complex multilingual data. deep learning variants showed mixed strengths. in technique classification the cnn+bilstm model reached an f1 of 0.3252 by processing context in both directions. other cnn with gru or lstm followed, all outperforming the standalone cnn at f1 0.2816. on span identification models such as cnn+bigru scored an f1 of 0.4313 but combined recall above 0.80 with precision below 0.30. this suggests strong token detection yet imprecise boundary placement. transformer based systems outperformed both other groups. xlm roberta large achieved f1 of 0.4498 for technique classification (precision 0.3917, recall 0.5667) and f1 of 0.6026 for span identification (precision 0.5616, recall 0.6500). in-foxlm large followed closely (classification f1 0.4451; span identification f1 0.5577). models like mdeberta v3 base and multilingual bert also surpassed ml and dl methods. their pretrained multilingual embeddings and deep attention mechanisms enable a nuanced grasp of subtle cues. overall transformer pretrained models deliver the most reliable performance for detecting propaganda techniques and marking their exact spans in bilingual social media text. their ability to learn rich contextual patterns clearly outstrips earlier paradigms. ## error analysis quantitative and qualitative error analyses of the technique classification and span identification tasks employed confusion matrices and focused examination of example predictions to reveal model strengths and limitations. ## conclusion this paper introduces a system developed for the unlp 2025 shared tasks on manipulation technique classification and manipulative span identification in ukrainian and russian telegram posts, and demonstrates its effectiveness through extensive experiments comparing traditional machine learning methods, deep learning architectures, and transformer-based models. among these, xlm-roberta-large achieved the strongest performance, with a macro-averaged f1 of 0.4498 in technique classification and a span-level f1 of 0.6026 in span identification. detailed error analysis revealed two key challenges: distinguishing between semantically similar manipulation tactics, particularly loaded language versus appeal to fear and precisely delineating span boundaries in morphologically complex slavic texts. these findings emphasize contextual modeling and cross-lingual pretraining for detecting persuasive cues in slavic texts. future works involve boundary-aware span detection, contrastive learning, architectures for low-resource conflict zones, and synthetic data augmentation against evolving encrypted-channel tactics.