CUET _sstm at A r AIE val Shared Task: Unimodal (Text) Propagandistic Technique Detection Using Transformer-Based Model (2024)

Abstract: In recent days, propaganda has started to influence public opinion increasingly as social media usage continues to grow. Our research has been part of the first challenge, Unimodal (Text) Propagandistic Technique Detection of ArAIEval shared task at the ArabicNLP 2024 conference, co-located with ACL 2024, identifying specific Arabic text spans using twenty-three propaganda techniques. We have augmented underrepresented techniques in the provided dataset using synonym replacement and have evaluated various machine learning (RF, SVM, MNB), deep learning (BiLSTM), and transformer-based models (bert-base-arabic, Marefa-NER, AraBERT) with transfer learning. Our comparative study has shown that the transformer model “bert-base-arabic” has outperformed other models. Evaluating the test set, it has achieved the micro-F1 score of 0.2995 which is the highest. This result has secured our team “CUET_sstm” first place among all participants in task 1 of the ArAIEval.

## introduction propaganda is the use of manipulative ideas or news to influence the behavior or point of view of humans in order to serve a specific agenda. it has become widespread in media, newspaper articles, social media posts, and broadcasts. individuals have rarely been informed without bias. propaganda is a powerful political tool that attracts large groups of people. social media has often been used to spread propaganda and misinformation, to divert attention from more serious issues. detecting propaganda has been crucial to prevent false news from circulating. propaganda detection in arabic text spans is quite challenging as the language has been rich in both morphology and syntax also it has a vast number of dialects (as mentioned in @xcite . there has been a noticeable gap in the resources, annotated datasets, and nlp tools available for arabic compared to languages such as english. all of this complicates nlp tasks in arabic. primarily, this paper has intended to detect propaganda techniques in arabic on social media (tweets) and in paragraphs. the arabic-nlp 2024 conference, along with acl 2024, has organized araieval introducing a dataset that contains text with 23 types of propaganda techniques, to detect propaganda applied in text span @xcite . to achieve our goal, we have augmented underrepresented propaganda techniques in the dataset using synonym replacement from the nlpaug library and have evaluated a variety of models, including three machine learning techniques (rf, svm, and mnb), one deep learning approach (bilstm), and three transformer-based models (bert-base-arabic, marefa-ner and arabert). during our comparative analysis, each model has been trained and assessed on the provided dataset. the most effective model has been"bert-basearabic", attaining the micro-f1 score of 0.2995 which is the highest. the core contributions of our research work are augmenting underrepresented techniques in the dataset using synonym replacement with nlpaug and finetuning "bert-base-arabic" transformer model to detect 23 propaganda technique classes by specifying the span in the text. detailed implementation information is available in the linked github repository belowhttps://github . com/aref111n/cuet_sstm-araieval. ## related work propaganda detection can be splitted into three categories based on previous research works: traditional ml techniques, advanced dl models, and transformer-based architectures. ml-based techniques have been well-known for detecting reliability in information. according to rashkin et al., 2017, the truthfulness of information is based on common linguistic patterns. naive bayes classifiers @xcite , support vector machine @xcite and random forest classifier @xcite have been mostly used to understand those patterns. dl-based algorithms have been deployed when data has become more available and affordable due to the growth of social networks and increased propaganda on social platforms. bilstm has been most commonly used in numerous works @xcite . large language models like bert, roberta, and gpt have been used in detecting propaganda techniques @xcite . span-level annotation has been proposed by da san martino et al., 2019, allowing specific portions of the text to coincide with any propaganda strategy. the top-performing models have employed bert-based representations for a shared task 1 in 2019. xlm-roberta-based model marefa-ner (hussein et al., 2022) and monolingual arabert @xcite have proved their promising possibility in the field of propaganda detection. several shared tasks @xcite have fueled the progress in the detection of reliable information in the arabic language. ## data we have used the arabic propaganda detection dataset (introduced in @xcite 1 http://www.netcopia.net/nlp4if/2019/ from araieval shared task 1 at arabicnlp 2024 @xcite , segmented into training (6997 samples), development (921 samples), and test sets (1046 samples). there are 995 tweets and 6002 paragraphs in the training set, 249 tweets and 672 paragraphs in the dev set, and 260 tweets and 786 paragraphs in the test set. the dataset includes 23 propaganda technique classes with notable distribution imbalances, as illustrated in figure 1 . about 74.1% of the text in the dataset contains no technique. to overcome this, we have augmented underrepresented techniques using the synonym replacement method of the nlpaug library (as mentioned in @xcite . beautifulsoup and regex have also been used for additional preprocessing. ## system we have participated in only task 1, which is an unimodal and multilabel sequence tagging task. the input is an arabic multigenre text, particularly a news paragraph or tweet, and the required output is to detect propaganda techniques and their exact spans in the text. ## results and analysis ## conclusion though arabic language processing is a very challenging task, we have been able to identify 23 dis-