SI 2 M & AIOX Labs at WANLP 2022 Shared Task: Propaganda Detection in A rabic, A Data Augmentation and Name Entity Recognition Approach (2022)

Abstract: This paper presents SI2M & AIOX Labs work among the propaganda detection in Arabic text shared task. The objective of this challenge is to identify the propaganda techniques used in specific propaganda fragments. We use a combination of data augmentation, Name Entity Recognition, rule-based repetition detection, and ARBERT prediction to develop our system. The model we provide scored 0.585 micro F1-Score and ranked 6th out of 12 teams.

## introduction even though the internet and social networks are tools for development and open doors to new opportunities, they also have a less attractive side. it is true that these tools are also used for bad purposes, such as the spread of propagandist messages when they are not identified as such by social media users. as part of cyber propaganda, or as part of the broader term "fake news" @xcite , propaganda messages are used in social networks with the objective of convincing targeted populations in a biased way. often, these messages aim to persuade their recipients to embrace ideas that are politically or ideologically motivated. in light of the proliferation of such messages and the various upheavals the world is confronting today, researchers need to explore possible methods to detect cyber propaganda automatically. in contrast to english propaganda detection @xcite , we note a flagrant lack of arabic propaganda detection research, even if there are rare works dealing with this subject @xcite or with close subjects like fake news @xcite . this work addresses this need, in order to build a system that can detect propaganda in tweets written in arabic, as well as define the propaganda techniques employed. indeed the dataset used in this paper contains 17 propaganda techniques, excluding "no technique", whose details are given by the organizers of the challenge @xcite of which this work is part. our system has the characteristic of combining a data augmentation method, named entity recognition (ner), a rule-based approach, and the arbert model @xcite . the two main objectives are to answer the problem of the very limited amount of data available, and also to be able to detect as much as possible one of the most used propaganda techniques, namely "name calling/labeling". ## related work among the earliest definitions of propaganda is that of the institute for propaganda analysis (institute for propaganda analysis, 1938), which defined it in 1938 as "the expression of opinion or action by individuals or groups deliberately designed to influence opinions or actions of other individuals or groups with reference to predetermined end". apart from seeking the most comprehensive definition of the concept, several works have concentrated on categorizing propaganda techniques in order to better identify them. the first categorization was made by clyde r. miller (co-founder of the institute for propaganda analysis) in 1937. due to the proliferation of propaganda on social networks, these categorizations have become increasingly important over time due to the pressing need to detect propaganda automatically. the lack of annotated datasets dedicated to this problem, however, is one of the major obstacles. it was only in 2017 that the first datasets started to appear, namely the tshp-17 @xcite , qprop @xcite and ptc @xcite in 2019. in addition to detecting propaganda automatically, these datasets have also enabled us to detect the techniques in the texts in addition to specifying the relevant text fragments. several works have emerged, mainly as system proposals within shared tasks. like the workshop on nlp4if in 2019 @xcite and semeval-2020 task 11 @xcite , both based on the tpc corpus. in the two shared tasks, two objectives were targeted simultaneously, namely the detection of the propaganda texts and the specification of the article part in question. the most effective solutions proposed can be summarized in the use of bio encoding @xcite , self-supervision with the roberta model @xcite and bert word-level classification @xcite . ## data we received two datasets from the challenge organizers, one named train for training the system, and the other named dev for validating and selecting the best configuration. the datasets contain a list of sequences and the propaganda techniques contained within these sequences. also, at the end of the challenge, we receive a third dataset to evaluate the system. using this last dataset, named test, the final scores of each team are calculated. there is also a second task for which the same data is provided along with the start and end of the techniques within each sequence. ## system in the following sections, we describe our four-step system. ## results the results for the dev and test sets are presented in this section. to demonstrate the contribution of each of the steps considered in our system, we present the score obtained after applying each of these steps to the dev set in ## discussion the results show that the system's steps have a positive impact on the outcomes. indeed, the score rises by 29% between the first step, which is solely based on the train set, and the final step of the entire system. furthermore, it appears that the use of ner has a significant effect on the final result, as the score knows the highest increase when using this method, recording a 22% increase. this finding is consistent with the fact that the name calling/labeling technique is the dataset's second most common technique. this result motivates future work to further investigate this idea by attempting to detect other majority techniques. it is also worth noting that the data augmentation step contributed 5% to the improvement of the micro f1 score, whereas the detection of repetition contributed only 0.9%. the data augmentation step should be pushed in two directions: quantitatively by increasing the number of synthetic sequences generated, and structurally by prioritizing minority techniques or minority combinations in order to push the system to better predict these techniques. ## conclusion this paper describes our contribution to the shared task of propaganda detection in wanlp 2022. we propose a system based on data augmentation, named entity recognition (ner), repetition detection, and arbert prediction for subtask 1 dealing with multi-label classification techniques. our analysis shows that ner and data augmentation have a significant impact on the final results, placing us sixth out of 14 competing teams.