Transforming Causal LLM into MLM Encoder for Detecting Social Media Manipulation in Telegram (2025)

Abstract: We participated in the Fourth UNLP shared task on detecting social media manipulation in Ukrainian Telegram posts, addressing both multilabel technique classification and token-level span identification. We propose two complementary solutions: for classification, we fine-tune the decoder-only model with class-balanced grid-search thresholding and ensembling. For span detection, we convert causal LLM into a bidirectional encoder via masked language modeling pretraining on large Ukrainian and Russian news corpora before fine-tuning. Our solutions achieve SOTA metric results on both shared task track. Our work demonstrates the efficacy of bidirectional pretraining for decoder-only LLMs and robust threshold optimization, contributing new methods for disinformation detection in low-resource languages.

## introduction 1.motivation & context disinformation on social media poses significant threats to public discourse and democratic processes. in the ukrainian context, telegram is a primary channel for news dissemination and propaganda, where rhetorical manipulation techniques can influence opinions without factual support. accurate detection of these techniques at both the document and span levels is crucial for fact-checking, media literacy, and automated moderation. ## alternative approaches in addition to our primary architectures, we explored several complementary strategies. although these methods offered conceptual advantages, none outperformed our main models during evaluation.