S emantic C uet S ync at A r AIE val Shared Task: Detecting Propagandistic Spans with Persuasion Techniques Identification using Pre-trained Transformers (2024)

Abstract: Detecting propagandistic spans and identifying persuasion techniques are crucial for promoting informed decision-making, safeguarding democratic processes, and fostering a media environment characterized by integrity and transparency. Various machine learning (Logistic Regression, Random Forest, and Multinomial Naive Bayes), deep learning (CNN, CNN+LSTM, CNN+BiLSTM), and transformer-based (AraBERTv2, AraBERT-NER, CamelBERT, BERT-Base-Arabic) models were exploited to perform the task. The evaluation results indicate that CamelBERT achieved the highest micro-F1 score (24.09%), outperforming CNN+LSTM and AraBERTv2. The study found that most models struggle to detect propagandistic spans when multiple spans are present within the same article. Overall, the model’s performance secured a 6^{th} place ranking in the ArAIEval Shared Task-1.

## introduction detecting propagandistic spans with persuasion techniques identification involves pinpointing particular text sections characterized by the deliberate use of persuasive language and tactics to shape opinions, attitudes, or actions. this involves analyzing linguistic cues, rhetorical strategies, and contextual elements to identify instances of propaganda and the specific persuasion techniques employed. it is difficult for a naive user to determine if the information is factually correct or if it is just another propaganda technique. there have been many studies done on detecting fake news @xcite , biased @xcite , hyperpartisan @xcite , and propagandistic news and articles @xcite . the influence of fake news on twitter @xcite and other online media @xcite ) can be dangerous. deliberately spreading fake news about a particular topic @xcite is also a propaganda technique. social media users tend to believe anything with more views without any factual basis @xcite . propaganda detection @xcite is a multidisciplinary field that draws upon techniques from nlp and identifies elements indicative of propaganda. there are diverse propaganda techniques (jones, 2024), and classifying them is a challenge. span detection @xcite in propaganda techniques is another big challenge. propaganda aims to influence the audience to advance a specific agenda. detecting the exact span where it occurs is tricky and arguably more difficult than finding false information in a news article. few studies have focused on propaganda technique detection @xcite and span detection @xcite in arabic. by applying propagandistic textual spans detection with persuasion techniques identification across various domains, individuals and organizations can better understand and counteract the influence of propaganda, ultimately fostering more informed, ethical, and resilient societies. the contributions of this work are illustrated as follows: propaganda detection has been the subject of numerous research in recent years. however, the majority of them were primarily concerned with english. li et al. (2019) proposed a model to detect propaganda at the sentence level using logistic regression, which achieved an f 1 -score of 0.66. several dl, ml, and transformer-based techniques were proposed by gupta et al. (2019) to detect propaganda. the ensemble technique yielded the highest f 1 -score of 0.669 out of all of them. both of these studies approached propaganda detection as a binary classification task. propaganda detection in arabic has seen substantial advancements in recent years. hasanain et al. (2023b) provided an overview of the araieval shared task, presented at the first arabicnlp 2023 conference, focusing on persuasion technique identification and disinformation detection in both binary and multitask settings. alam et al. (2022) organized a shared task specifically on propaganda detection in arabic tweets, addressing a wide range of topics. this shared task included two objectives: identifying the type of propaganda technique employed and determining the specific text spans where these techniques were utilized. refaee et al. (2022) used pre-trained bert model to detect propaganda in arabic tweets. their model achieved an f 1 -score of 0.602. in contrast to the earlier study, @xcite details how to detect propaganda and its validity span. gpt-4 was used in experiments by @xcite to identify and locate the propaganda span on multiple languages' news datasets. they found that gpt-4's performance is low for this sequence tagging and multilabel task, especially in low-resourced languages. @xcite proposed using llm for propaganda span annotation. this work uses a transformer-based model to address the downstream task of detecting propagandistic techniques from arabic tweets and paragraphs extracted from news articles. ## task and dataset description to address the phenomena of detecting propaganistic techniques, shared task organizers foot_0 provided a unimodal (text) dataset. the dataset @xcite ) contains arabic tweets and paragraphs for propagandistic technique detection. the dataset covers two genres: tweets and paragraphs extracted from news articles. the dataset covers a total of 23 different classes representing various propaganda techniques. the train, dev, and test sets consisted of 6997, ## system development figure 1 illustrates the schematic process of classifying propaganda techniques and detecting textual spans. we used a tokenizer to tokenize each text. we added unique tokens [cls] at the beginning and [sep] at the end and assigned a -100 label to both so that the loss function ignores them. then, we used the bio tagging scheme @xcite , where the b-tag indicates the beginning of an entity, the i-tag indicates a token contained inside the same entity, and the o-tag denotes that no entity is contained within the token. ## discussion transformer-based models are capable of identifying various propaganda techniques within the exact text. however, they need to detect the spans consistently and accurately. this inconsistency contributed to the test set's less-than-optimal performance. the error analysis findings suggest that most models struggle to detect propaganda span when there are multiple ones in the same article. most of the errors were introduced in the model span detection phase. moreover, detecting the span of the propaganda involves the model understanding not only the context but also the tone, intent, and bias. though the pre-trained bert-based models can understand contextual elements, they need help in other aspects. ## conclusion this study investigates the capabilities of transformer-based models in thoroughly analyzing propaganda span detection and identifying the persuasive techniques in the arabic dataset. among all employed ml, dl, and transformer-based models, camelbert showed notable performance with the highest micro-f1 score of 24.09. further advancements could be achieved by expanding the dataset and investigating large language models in the task.