M eme M ind at A r AIE val Shared Task: Spotting Persuasive Spans in A rabic Text with Persuasion Techniques Identification (2024)

Abstract: This paper focuses on detecting propagandistic spans and persuasion techniques in Arabic text from tweets and news paragraphs. Each entry in the dataset contains a text sample and corresponding labels that indicate the start and end positions of propaganda techniques within the text. Tokens falling within a labeled span were assigned ’B’ (Begin) or ’I’ (Inside) tags, ’O’, corresponding to the specific propaganda technique. Using attention masks, we created uniform lengths for each span and assigned BIO tags to each token based on the provided labels. Then, we used AraBERT-base pre-trained model for Arabic text tokenization and embeddings with a token classification layer to identify propaganda techniques. Our training process involves a two-phase fine-tuning approach. First, we train only the classification layer for a few epochs, followed by full model fine-tuning, updating all parameters. This methodology allows the model to adapt to the specific characteristics of the propaganda detection task while leveraging the knowledge captured by the pretrained AraBERT model. Our approach achieved an F1 score of 0.2774, securing the 3rd position in the leaderboard of Task 1.

## introduction propagandistic detection techniques involve identifying content that aims to influence an audience's perception or persuade them to a certain viewpoint using biased, misleading, or emotionally charged information @xcite . this field of study is crucial due to the prevalence of propaganda in various forms of media, including news articles, social media posts, and advertisements. techniques such as appeal to fear, loaded language, bandwagon, and testimonials are commonly used to manipulate public opinion. these techniques determine the presence of biased language and measure the emotional impact of the messages conveyed, thus providing a systematic way to sift through vast amounts of data and uphold the integrity of information consumed by the public. @xcite @xcite . recent studies have explored various aspects of arabic natural language processing that are relevant to propaganda detection @xcite . farha et al., 2021 organized a shared task on sarcasm and sentiment detection in arabic, highlighting the importance of understanding the nuances of language use in social media. @xcite developed a fine-grained annotated multi-dialectal arabic corpus, which can be valuable for training models to handle the linguistic diversity of arabic text. the covid-19 pandemic has also brought attention to the spread of disinformation and the need for effective detection methods. @xcite modeled the perspectives of various stakeholders in fighting the covid-19 infodemic, emphasizing the importance of a collaborative approach. shurafa et al., 2020 analyzed political framing in the context of the pandemic, demonstrating how language can be used to shape public opinion. shared task 1 @xcite aimed to identify the propaganda techniques utilized within a diverse range of textual forms, such as news paragraphs or tweets, and to precisely locate the sections of text where each propaganda technique is employed. the shared task 1 is in arabic dataset. while working with propaganda detection, finding textual spans is challenging, and it becomes more complicated regarding arabic data. arabic is a morphologically rich language with complex grammatical structures and a wide range of dialects @xcite . this complexity adds layers of difficulty to tasks like part-of-speech (pos) tagging and syntactic analysis, which are essential for identifying propaganda techniques @xcite . there is a lack of annotated datasets for propaganda detection in arabic compared to languages like english. this scarcity makes it chal-lenging to train and evaluate machine learning models effectively, as they heavily rely on annotated data for supervised learning. in this paper, we propose an approach that leverages the arabert-base pre-trained model for arabic text tokenization and text embeddings, combined with a token classification layer to detect propaganda techniques present in the text. we evaluate our model using the dataset provided by the araieval shared task and compare our results with those of other participants. ## related work the growing prevalence of propaganda in digital media has necessitated the development of automated tools that can effectively detect and annotate propaganda techniques in text. various approaches have been proposed to tackle this challenge, ranging from traditional machine learning methods to deep learning techniques. early work in propaganda detection focused on identifying propagandistic news articles using machine learning classifiers. barrón-cedeño et al. 2019 developed a system called proppy, which utilized a combination of stylistic, lexical, and syntactic features to detect propaganda at the article level. they achieved promising results, demonstrating the effectiveness of leveraging linguistic cues for propaganda identification. with the advent of deep learning, researchers began exploring neural network architectures for propaganda detection. bi-lstm (bidirectional long short-term memory) models have been widely used to capture sequential dependencies in text. mapes et al. @xcite employed a bi-lstm model with an attention mechanism to detect propaganda techniques in news articles. their approach showcased the benefits of utilizing deep learning for capturing complex patterns in propagandistic text. recent studies have focused on fine-grained propaganda detection, aiming to identify specific propaganda techniques at the sentence or fragment level.da san martino et al. 2019 introduced a new dataset, named semeval-2020 task 11, which consists of news articles annotated with 18 propaganda techniques. they proposed a multi-granularity neural network that learns representations at different levels of text granularity, enabling the detection of propaganda techniques at both the sentence and fragment levels. the araieval shared tasks @xcite have focused on the detection of propagandistic techniques and disinformation in arabic text, providing a platform for researchers to develop and evaluate their models. @xcite investigated the potential of gpt-4, a large language model, in identifying propaganda spans in news articles. they also explored the use of large language models for propaganda span annotation @xcite . pre-trained language models, such as bert @xcite and its variants, have revolutionized nlp tasks, including propaganda detection. these models capture rich linguistic knowledge from large-scale unsupervised pre-training, which can be fine-tuned for specific tasks. arabert (antoun et al., 2020) is a pre-trained language model specifically designed for arabic text, which has shown promising results in various arabic nlp tasks. our work extends prior research by utilizing the arabert-base pre-trained model for arabic propaganda detection. we utilized a token classification method that merges arabert's embeddings with a classification layer to detect propaganda spans and techniques. our dual-phase fine-tuning strategy enables the model to adapt to the nuances of propaganda detection while leveraging the pretrained arabert model's knowledge. overall, our contribution complements existing efforts in arabic propaganda detection, aiming to enhance detection accuracy and mitigate manipulative content proliferation. ## data the dataset used was provided by the organizers of the araieval shared task 1 @xcite . the dataset comprises arabic text snippets, including news paragraphs and tweets, annotated with propaganda spans and corresponding persuasion techniques. figure 1 shows a snippet of the dataset. it contains a unique "id" identifier for each data. the "text" label is the key element to finding the propaganda span. each data can contain either one label or multiple labels. there are a total of 23 techniques. each technique has a start and end within the text. the training set comprises 6,997 text snippets, while the validation set contains 921 snippets. the test set used for the final evaluation consists of 1046 text snippets. the label distribution across the dataset is imbalanced, with some propaganda techniques being more prevalent than others. the most common techniques in the training set are "loaded language" (55.69%), "name calling labeling" (14.23%), and "exaggeration-minimisation" (6.83%). arabic is known for its diverse dialects, and the dataset contains a mix of modern standard arabic and various regional dialects. this linguistic diversity adds complexity to the task of propaganda detection, as different dialects may express persuasion techniques differently. one of the challenges posed by the araieval dataset is the presence of code-switching and dialectal variations in the text snippets. preprocessing steps were applied to the dataset before feeding it into our system. we utilized the arabert tokenizer, specifically designed for arabic text, to tokenize the input snippets. the tokenizer handles arabic-specific characteristics, such as discretization and morphological segmentation. ## system our proposed system architecture leverages the arabert-base pre-trained model, which is a transformer-based model specifically trained on a large corpus of arabic text. the flowchart shows the process fig. 2 . firstly, we encoded each label into hard-coded and created a dictionary to hold the label to id mapping. then, we created bio tags for each entry. we utilized the arabert-base "autotokenizer" for text tokenization and embeddings, capitalizing on its ability to tokenize the text sequences, ensuring compatibility with pre-trained language models. for each text entry, the tokenizer generates tokenized representations and bio (beginning, inside, outside) tags indicating the presence of propaganda techniques at the token level. the tokenized input is passed through the arabert-base model to obtain contextualized word embeddings. these embeddings are then fed into a token classification layer consisting of a linear layer followed by a softmax activation function. the token classification layer predicts the propaganda technique associated with each token in the input sequence. we employ a two-phase fine-tuning approach to adapt the pre-trained arabert-base model to the task of propaganda detection. in the first phase, we freeze the parameters of the arabert-base model and train only the token classification layer for a few epochs. this allows the classification layer to learn the task-specific mapping from the pre-trained embeddings to the propaganda labels. in the second phase, we unfreeze the parameters of the arabert-base model and fine-tune the entire model end-to-end. this enables the model to adapt its representations to the specific characteristics of the propaganda detection task. the hyperparameters used in our system were determined through experimentation and validation. we used a batch size of 32, a learning rate of 2e-5 for the token classification layer, and a learning rate of 1e-5 for fine-tuning the arabert-base model. the model was trained for 10 epochs in the first phase and 5 epochs in the second phase. the training was conducted on a single nvidia geforce rtx 2080 ti gpu, and the total training time was approximately 6 hours. the source code for our system is available at https://github.com/rafiulbiswas/ araieval_arabicnlp24_task_1 for reproducibility. ## conclusion our participation in the araieval shared task 1 contributes to the ongoing efforts in propaganda detection and showcases the potential of leveraging pre-trained language models for this task in the arabic language. our proposed system, combining the arabert-base model with token classification, achieved promising results and provides a foundation for further research and development in this critical area of nlp. as the landscape of digital information continues to evolve, it is essential to continue advancing the state-of-the-art in propaganda detection. we hope that our work inspires further research and collaborations to tackle this important challenge, not only in arabic but also in other languages and across various domains. in language. the most prevalent technique is loaded language, accounting for a staggering 55.69% of instances in the dataset. this indicates a significant use of emotive or biased terminology that aims to influence an audience's perception. name calling or labeling follows, comprising 14.23% of the dataset. this technique involves derogating a person or group to sway the audience's opinion. other notable techniques include exaggeration/minimisation at 6.83% and questioning the reputation at 5.53%, which involve overstating or understating facts, and casting doubt on someone's character, respectively. overall, the chart provides a clear visual representation of how different rhetorical techniques are employed in the dataset.