CLTL at A r AIE val Shared Task: Multimodal Propagandistic Memes Classification Using Transformer Models (2024)

Abstract: We present the CLTL system designed for the ArAIEval Shared Task 2024 on multimodal propagandistic memes classification in Arabic. The challenge was divided into three subtasks: identifying propagandistic content from textual modality of memes (subtask 2A), from visual modality of memes (subtask 2B), and in a multimodal scenario when both modalities are combined (subtask 2C). We explored various unimodal transformer models for Arabic language processing (subtask 2A), visual models for image processing (subtask 2B), and concatenated text and image embeddings using the Multilayer Perceptron fusion module for multimodal propagandistic memes classification (subtask 2C). Our system achieved 77.96% for subtask 2A, 71.04% for subtask 2B, and 79.80% for subtask 2C, ranking 2nd, 1st, and 3rd on the leaderboard.

## introduction in the digital age, memes have gained immense popularity across various age groups through social media platforms such as facebook and twitter. in recent years, however, they have been increasingly employed for propagandistic purposes (g. de leon and ballesteros-lintao, 2021; @xcite . these memes often deploy a range of propagandistic techniques, including logical fallacies, causal oversimplification, and exaggeration, paired with images with strong positive/negative emotional implications to subtly sway audience opinions and propagate misinformation @xcite . given the multimodal nature and powerful impact of these memes, it is crucial to develop classification systems capable of detecting propagandistic memes effectively. previous studies in propaganda detection primarily focused on textual content @xcite @xcite or fine-grained persuasion techniques analysis @xcite . to the best of our knowledge, the araieval 2024 shared task @xcite is the first shared task on propaganda detection in multimodal scenarios, specifically within arabic memes. the task is divided into three subtasks: subtasks 2a and 2b focus on classifying the textual and visual modalities of a given meme separately to detect whether it is propagandistic or not, whereas subtask 2c integrates both modalities for multimodal classification to detect whether a meme is propagandistic. we carried out various experiments using stateof-the-art arabic language processing models and vision models, and employed the multilayer perceptron (mlp) fusion module @xcite with a prediction layer on top for multimodal classification. without the need for text preprocessing and feature engineering, our system secured 2nd place in subtask 2a, 1st place in subtask 2b, and 3rd place in subtask 2c. ## data the dataset used in the araieval 2024 shared task comprises a collection of arabic memes sourced from various social media platforms: facebook, twitter, instagram, and pinterest @xcite . each meme is labeled as either 'propagandistic' or 'non-propagandistic' and includes both the original image file and its corresponding extracted textual content. the single dataset was used for the three subtasks covered in the competition. the statistics of the dataset, in terms of the number of memes per class as well as the class distribution, are provided in we carried out our experiments on the google colaboratory platform with an nvidia l4 gpu, utilizing the pytorch framework and autogluon library @xcite . we set uniform hyperparameter settings for all the examined models: a base learning rate of 1e-4, decay rate of 0.9 using cosine decay scheduling, batch size of 8, maximum training epochs of 10, and optimization via the adamw optimizer. each model was trained for around 12 minutes, regardless of the subtask. ## results we provide confusion matrices for the submitted models on the test set for all three subtasks in figures 2, 3, and 4. the multimodal model for subtask 2c exhibited the best performance and the lowest false positive and false negative rates in propagandistic memes classification. conversely, the vision model for subtask 2b performs the worst among the three, with the lowest number of true positives (98) and the highest number of false negatives (73). ## conclusion in this paper, we presented the cltl system developed for the araieval 2024 shared task on multimodal propagandistic memes classification. our approach involved leveraging state-of-the-art transformer models for both textual and visual modalities and employing the multilayer perceptron fusion module to combine text and image representations for multimodal classification. our system secured 2nd place with a 77.96% macro-f1 score for subtask 2a, 1st place with 71.04% for subtask 2b, and 3rd place with 79.8% for subtask 2c.