A r AIE val Shared Task: Propagandistic Techniques Detection in Unimodal and Multimodal A rabic Content (2024)

Abstract: We present an overview of the second edition of the ArAIEval shared task, organized as part of the ArabicNLP 2024 conference co-located with ACL 2024. In this edition, ArAIEval offers two tasks: (i) detection of propagandistic textual spans with persuasion techniques identification in tweets and news articles, and (ii) distinguishing between propagandistic and non-propagandistic memes. A total of 14 teams participated in the final evaluation phase, with 6 and 9 teams participating in Tasks 1 and 2, respectively. Finally, 11 teams submitted system description papers. Across both tasks, we observed that fine-tuning transformer models such as AraBERT was at the core of the majority of the participating systems. We provide a description of the task setup, including a description of the dataset construction and the evaluation setup. We further provide a brief overview of the participating systems. All datasets and evaluation scripts are released to the research community. We hope this will enable further research on these important tasks in Arabic.

## introduction online media has become a primary channel for information dissemination and consumption, with numerous individuals considering it their main source of news @xcite . while online media (including news and social media platforms) offers a plethora of benefits, it is periodically exploited by malicious actors aiming to manipulate and mislead a broad audience. they often engage in sharing inappropriate content, misinformation, and disinformation @xcite . among the various forms of misleading and harmful content, propaganda is another communication tool designed to influence opinions and actions to achieve a specific objective. propaganda is defined as a form of communication that is aimed at influencing the attitude of a community toward some cause or position by presenting only one side of an argument, which is achieved by means of welldefined rhetorical and psychological devices @xcite . it is often biased or misleading in nature and is used to promote a particular political cause or point of view. news reporting in the mainstream media channels often use persuasion techniques to promote a particular editorial agenda. in different communication channels, propaganda is conveyed through the use of diverse persuasion techniques @xcite , which range from leveraging the emotions of the audience, such as using emotional technique or logical fallacies such as straw man (misrepresenting someone's opinion), hidden ad-hominem fallacies, and red herring (presenting irrelevant data). in the past years, propaganda has been widely used on social media to influence and/or mislead the audience, which became a major concern for different stakeholders including social media platforms and government agencies. to combat the proliferation of propaganda on online platforms, there has been a significant surge in research in recent years. the aim is to automatically identify propagandistic content in textual, visual, and multimodal modalities, such as memes @xcite @xcite . these efforts mainly focused on english, but with increased interest in other languages too. we enrich the arabic ai research on that problem, by organizing a shared task on fine-grained propaganda techniques detection for arabic, which attracted many participants @xcite . we followed that by organizing the arabic ai evaluation (araieval) shared task at arabicnlp detection in arabic text, with 14 and 8 teams participating in these subtasks, respectively @xcite . following the success of our previous shared tasks, and given the interest from the community in further pushing research in this domain, we organize the second edition of the araieval shared task covering the following two tasks: (i) detection of propagandistic textual spans with persuasion techniques identification (unimodal), and (ii) distinguishing between propagandistic and nonpropagandistic memes (multimodal). this edition of the shared task attracted good participation with a total of 45 registrations. fourteen teams submitted their systems, and 11 teams submitted system description papers. for all tasks, majority of participating systems utilized transformerbased models, and several of them applied data augmentation techniques. all systems outperformed the respective random baselines. in the remainder of this paper, we define each of the two tasks, describe the manually constructed arabic evaluation datasets, and provide an overview of the participating systems and their official scores. ## conclusion and future work we presented an overview of the araieval shared task, which consists of two tasks: (i) detection of propagandistic textual spans with persuasion techniques identification in tweets and news articles, and (ii) distinguishing between propagandistic and non-propagandistic memes. the task attracted the attention of many teams: a total of 45 teams registered to participate during the evaluation phase, with 6 and 9 teams eventually making an official submission on the test set for tasks 1 and 2, respectively. finally, 11 teams submitted task description papers. task 1 aimed to identify the propaganda techniques used in multi-genre text snippets, including tweets and news articles. the task was to detect textual spans with propagandistic techniques. on the other hand, task 2 aimed to detect propaganda in memes (multimodal content) in binary classification settings for different modalities. for both tasks, majority of the systems fine-tuned pre-trained transformer models. future editions of the araieval shared task will explore more complex tasks, such as incorporating arabic dialects. increasing the dataset size and focusing on low-prevalence class labels will be crucial to improve model robustness. for task 2, future work could involve labeling the dataset with specific propagandistic techniques.