Overview of the WANLP 2022 Shared Task on Propaganda Detection in A rabic (2022)

Abstract: Propaganda is defined as an expression of opinion or action by individuals or groups deliberately designed to influence opinions or actions of other individuals or groups with reference to predetermined ends and this is achieved by means of well-defined rhetorical and psychological devices. Currently, propaganda (or persuasion) techniques have been commonly used on social media to manipulate or mislead social media users. Automatic detection of propaganda techniques from textual, visual, or multimodal content has been studied recently, however, major of such efforts are focused on English language content. In this paper, we propose a shared task on detecting propaganda techniques for Arabic textual content. We have done a pilot annotation of 200 Arabic tweets, which we plan to extend to 2,000 tweets, covering diverse topics. We hope that the shared task will help in building a community for Arabic propaganda detection. The dataset will be made publicly available, which can help in future studies.

## introduction social media platforms have become an important communication channel, where we can share and access information from a variety of sources. unfortunately, the rise of this democratic information ecosystem was accompanied by and dangerously polluted with misinformation, disinformation, and malinformation in the form of propaganda, conspiracies, rumors, hoaxes, fake news, hyper-partisan content, falsehoods, hate speech, cyberbullying, etc. @xcite @xcite @xcite @xcite @xcite propaganda is conveyed through the use of diverse propaganda techniques @xcite , which range from leveraging on the emotions of the audience (e.g., using loaded language, appealing to fear, etc.) to using logical fallacies such as straw men (misrepresenting someone's opinion), whataboutism, red herring (presenting irrelevant data), etc. in the last decades, propaganda was widely used on social media to influence and/or mislead the audience, which became a major concern for different stakeholders, social media platforms, and policymakers. to address this problem, the research area of computational propaganda has emerged, and here we are particularly interested in automatically identifying the use of propaganda techniques in text, images, and multimodal content. prior work in this direction includes identifying propagandistic content in an article based on writing style and readability level @xcite , at the sentence and the fragment levels from news articles with finegrained techniques @xcite , and in memes @xcite . these efforts focused on english, and there was no prior work on arabic. our shared task aims to bridge this gap by focusing on detecting propaganda in arabic social media text, i.e., tweets. ## related work in the current information ecosystem, propaganda has evolved to computational propaganda @xcite , where information is distributed on social media platforms, which makes it possible for malicious users to reach well-targeted communities at high velocity. thus, research on propaganda detection has focused on analyzing not only news articles but also social media content @xcite @xcite @xcite . @xcite focused on article-level propaganda analysis. they developed the tshp-17 corpus, which used distant supervision for annotation with four classes: trusted, satire, hoax, and propaganda. the assumption of their distant supervision approach was that all articles from a given news source should share the same label. they collected their articles from the english gigaword corpus and from seven other unreliable news sources, including two propagandistic ones. later, @xcite developed a new corpus, qprop , with two labels: propaganda vs. non-propaganda, and also experimented on tshp-17 and qprop corpora. for the tshp-17 corpus, they binarized the labels: propaganda vs. any of the other three categories as non-propaganda. they investigated the writing style and the readability level of the target document, and trained models using logistic regression and svms. their findings confirmed that using distant supervision, in conjunction with rich representations, might encourage the model to predict the source of the article, rather than to discriminate propaganda from non-propaganda. similarly, @xcite developed a corpus with 1.3k arguments annotated with five fallacies, including ad hominem, red herring, and irrelevant authority, which directly relate to propaganda techniques. recently, @xcite , curated a set of persuasive techniques, ranging from leveraging on the emotions of the audience such as using loaded language and appeal to fear, to logical fallacies such as straw man (misrepresenting someone's opinion) and red herring (presenting irrelevant data). they focused on textual content, i.e., newspaper articles. in particular, they developed a corpus of news articles annotated with eighteen propaganda techniques. the annotation was at the fragment level, and could be used for two tasks: (i) binary classification -given a sentence in an article, predict whether any of the 18 techniques has been used in it, and (ii) multi-label classification and span detection task -given a raw text, identify both the specific text fragments where a propaganda technique is used as well as the specific technique. they further proposed a multigranular deep neural network that captures signals from the sentence-level task and helps to improve the fragment-level classifier. da san martino et al. (2020a) also organized a shared task on detection of propaganda techniques in news articles. subsequently, dimitrov et al. (2021b) organized the semeval-2021 task 6 on detection of propaganda techniques in memes. it had a multimodal setup, combining text and images, and asked participants to build systems to identify the propaganda techniques used in a given meme. yu et al. (2021) looked into interpretable propaganda detection. other related shared tasks include the fever task @xcite on fact extraction and verification, the fake news challenge @xcite , the fakenews task at mediaeval @xcite , as well as the nlp4if tasks on propaganda detection @xcite and on fighting the covid-19 infodemic in social media @xcite . finally, we should mention the checkthat! lab at clef @xcite @xcite @xcite @xcite , which addresses many aspects of disinformation for different languages over the years such as fact-checking, verifiable factual claims, check-worthiness, attentionworthiness, and fake news detection. the present shared task is inspired from prior work on propaganda detection. in particular, we adapted the annotation instructions and the propaganda techniques discussed in @xcite . ## tasks and dataset below, we first formulate the two subtasks of our shared task, and then we discuss our datasets, including how we collected the data and what annotation guidelines we used. ## participants and results in this section, we provide a general description of the systems that participated in each subtask and their results. ## conclusion and future work we presented the wanlp'2022 shared task on propaganda detection in arabic, as part of which we developed the first dataset for arabic propaganda detection with focus on social media content. this was a successful task: a total of 63 teams registered to participate, and 14 and 3 teams eventually made an official submission on the test set for subtasks 1 and 2, respectively. finally, 11 teams submitted a task description paper. subtask 1 asked to identify the propaganda techniques used in a tweet, and subtask 2 further asked to identify the the span(s) of text in which each propaganda technique appears. for both subtasks, the majority of the systems fine-tuned pre-trained arabic language models, and used standard pre-processing. some systems used data augmentation and ensemble methods. in future work, we plan to increase the data size and to add hierarchically structured propaganda techniques.