S em E val-2024 Task 4: Multilingual Detection of Persuasion Techniques in Memes (2024)

Abstract: The automatic identification of misleading and persuasive content has emerged as a significant issue among various stakeholders, including social media platforms, policymakers, and the broader society. To tackle this issue within the context of memes, we organized a shared task at SemEval-2024, focusing on the multilingual detection of persuasion techniques. This paper outlines the dataset, the organization of the task, the evaluation framework, the outcomes, and the systems that participated. The task targets memes in four languages, with the inclusion of three surprise test datasets in Bulgarian, North Macedonian, and Arabic. It encompasses three subtasks: (i) identifying whether a meme utilizes a persuasion technique; (ii) identifying persuasion techniques within the meme’s ”textual content”; and (iii) identifying persuasion techniques across both the textual and visual components of the meme (a multimodal task). Furthermore, due to the complex nature of persuasion techniques, we present a hierarchy that groups the 22 persuasion techniques into several levels of categories. This became one of the attractive shared tasks in SemEval 2024, with 153 teams registered, 48 teams submitting results, and finally, 32 system description papers submitted.

## introduction the rise of online social media platforms has enabled people to share their views and feelings openly. this increase in freedom of speech has significantly expanded the volume of digital content, offering valuable resources for initiatives like citizen journalism, raising public awareness, and supporting political campaigns. however, this freedom has also facilitated negative uses, leading to an increase in online hostility, as evidenced by the spread of content such as disinformation, hate speech, propaganda, and cyberbullying @xcite @xcite @xcite . social media posts often combine various modalities, such as text, images, and videos. in recent years, internet memes have become a prevalent form of content on these platforms. a meme is defined as "a collection of digital items that share common characteristics in content, form, or stance, which are created through association and widely circulated, imitated, or transformed over the internet by numerous users." @xcite memes generally consist of one or more images accompanied by textual content @xcite . while memes are primarily aimed at humor, they can also convey persuasive narratives or content that may mislead audiences. to automatically identify such content, there have been research efforts directed towards addressing offensive content @xcite , identifying hate speech across different modalities @xcite , and detecting propaganda techniques in memes @xcite . focusing on propaganda detection, research efforts have been specifically directed towards defining techniques and addressing the issue in news articles @xcite , tweets @xcite , memes @xcite , and textual content in multiple languages @xcite . the associated shared tasks include semeval-2020 task 11 on news articles @xcite , semeval-2021 task 6 on memes @xcite , wanlp-2022 and arabicnlp-2023 focusing on arabic @xcite , and semeval-23 task 3 on news articles in multiple languages @xcite . the semeval-2024 shared task extends previous tasks but introduces multilinguality, covering four languages, and features the largest dataset in english, along with a new hierarchical evaluation method. it has attracted significant participation. the task consists of three subtasks and was run in two phases: (i) the development phase and (ii) the evaluation phase. in the remainder of this paper, we define the tasks, describe the datasets, and provide an overview of participating systems and their official scores. ## conclusions and future work we presented semeval-2024 task 4 on multilingual detection of persuasion techniques in memes. the task consists of detecting persuasion techniques in memes in a multimodal setting. the task offered a significantly larger dataset for english (10k memes) than previous ones, and three surprise languages: arabic, bulgarian, and north macedonian. the task attracted a lot of attention: 153 teams registered for the task and 30 teams submitted a task description paper. fine-tuning transformerbased architectures was the most dominant approach followed by most teams. the majority of teams participating in subtask 2 considered both the text and image components of the data, utilizing corresponding transformer models. finally, several teams designed hierarchical classification techniques, to tackle the hierarchy of labels in subtask 1 and subtask 2a. as for the surprise languages, at least a third of the submitting teams used automatic translation to translate the datasets into english.