CL a C at S em E val-2024 Task 4: Decoding Persuasion in Memes – An Ensemble of Language Models with Paraphrase Augmentation (2024)

Abstract: This paper describes our approach to SemEval-2024 Task 4 subtask 1, focusing on hierarchical multi-label detection of persuasion techniques in meme texts. Our approach was based on fine-tuning individual language models (BERT, XLM-RoBERTa, and mBERT) and leveraging a mean-based ensemble model. Additional strategies included dataset augmentation through the TC dataset and paraphrase generation as well as the fine-tuning of individual classification thresholds for each class. During testing, our system outperformed the baseline in all languages except for Arabic, where no significant improvement was reached. Analysis of the results seem to indicate that our dataset augmentation strategy and per-class threshold fine-tuning may have introduced noise and exacerbated the dataset imbalance.

## introduction the semeval-2024 shared task 4 @xcite proposed three distinct subtasks dedicated to identifying persuasion techniques conveyed by memes. the primary aim was to unravel how memes, integral to disinformation campaigns, employ various techniques to shape user perspectives. subtask 1 focused on the analysis of textual content alone; while subtasks 2 and 3 involved the analysis of multimodal context that considers both textual and visual elements. subtasks 1 and 2 used hierarchical multi-label classification metrics, while subtask 3 involves a binary classification task. the training dataset provided was in english but all subtasks mandated the evaluation of our model's zero-shot performance in three surprise languages: bulgarian, north macedonian, and arabic and another fourth dataset in english. the goal during the testing phase was to explore our model's ability to generalize to these languages without explicit training. this paper describes our participation to sub-task 1, focusing on the detection of 20 persuasion techniques structured hierarchically within the textual content of memes. inspired by successful approaches in multilabel text classification @xcite , our strategy involved fine-tuning three language models i.e, bert [bert-base-uncased], xlm-roberta [xlm-roberta-base], and mbert [bert-base-multilingual-uncased], followed by ensemble modeling using the mean aggregation technique using the english training set. to enhance performance, we used data augmentation through paraphrasing and adjusted the classification thresholds for each persuasion technique based on class-wise metrics optimised using the validation set using grid search. during testing, a zeroshot approach was implemented by translating the surprise language data into english. at the shared task, our system demonstrated significant performance advantages over the baseline in all languages except arabic, where the performance difference was not statistically significant. our system's effectiveness, particularly in non-arabic languages, underscores its potential for analyzing memes within disinformation campaigns, emphasizing the need for language-specific considerations in model development. section 2 provides an overview of the data utilized and offers insights into relevant prior research. section 3 presents an overview of our classification pipeline, while section 4 describes the experiments and data augmentation techniques that guided our final model decisions. finally, section 5 analyses the results of our model. all of the code used in the implementation of the models described in this paper is made available on github. 1 2 background semeval 2024 task 4 (multilingual detection of persuasion techniques in memes) proposed 3 subtasks, out of which we participated in the first one. the goal of subtask 1 was to categorize the textual content of memes into one or several persuasion techniques. an inventory of 20 techniques was provided (eg: smears, loaded language, slogans) and were structured hierarchically, rendering the task a hierarchical multi-label classification problem. ## system overview the aim of subtask 1 is to identify 0 or n persuasion techniques for each textual instance. despite the hierarchical organization of the persuasion techniques, we opted to predicting solely the technique names (leaf nodes) and not their ancestor nodes. figure 2 shows an overview of the classification pipeline we employed for this subtask. as shown in figure 2 , our methodology is based on finetuning three distinct pre-trained language models: bert @xcite , xlm-roberta (conneau et al., 2020), and mbert @xcite . this fine-tuning process is conducted on augmented datasets. ## conclusion this paper described the methodology used in our participation to the semeval 2024 task 4 subtask 1, focusing on hierarchical multi-label detection of persuasion techniques in meme texts. we used an ensemble model with three fine-tuned language models and incorporated additional strategies such as data augmentation through paraphrasing and classification thresholds fine-tuning based on classwise metrics. during testing, our system significantly outperformed the baseline in all languages except arabic, where the increase in performance was not significant. analysis shows that the data augmentation and threshold fine-tuning may have introduced noise and exacerbating dataset imbalance.