RD proj at S em E val-2024 Task 4: An Ensemble Learning Approach for Multilingual Detection of Persuasion Techniques in Memes (2024)

Abstract: This paper introduces our bagging-based ensemble learning approach for the SemEval-2024 Task 4 Subtask 1, focusing on multilingual persuasion detection within meme texts. This task aims to identify persuasion techniques employed within meme texts, which is a hierarchical multilabel classification task. The given text may apply multiple techniques, and persuasion techniques have a hierarchical structure. However, only a few prior persuasion detection systems have utilized the hierarchical structure of persuasion techniques. In that case, we designed a multilingual bagging-based ensemble approach, incorporating a soft voting ensemble strategy to effectively exploit persuasion techniques’ hierarchical structure. Our methodology achieved the second position in Bulgarian and North Macedonian, third in Arabic, and eleventh in English.

## introduction memes have gained immense popularity among the younger generation due to their entertaining nature. however, some memes can lead teenagers towards extreme ideas by employing persuasion techniques. even well-educated people often need help to identify misleading memes. thus, the development of a persuasion detection system holds significant value. this study aims to create a system to identify persuasion techniques within meme texts. this task is a multilabel and hierarchical classification task since memes may contain multiple persuasion techniques, and techniques have hierarchical structure @xcite . a description of the corpus provided by semeval-2024 task 4 @xcite reveals significant imbalances in the training data for the techniques. for instance, while there are 1990 instances for the "smears" technique, only 258 instances pertain to "whataboutism." moreover, the training data for each technique is smaller compared with the entire corpus, leading to the imbalance between positive and negative instances for each technique. these observations lead us to formulate the following research questions: 1) how can we mitigate the data imbalance between techniques? 2) how can we ease the imbalance between positive and negative instances for each technique? 3) how can we effectively leverage the hierarchical structure of techniques? we devise a baggingbased ensemble learning system employing a soft voting strategy to solve these questions. we group techniques into ten subsets based on the amount of their training data and the hierarchical structure @xcite , and construct a training set for each subset. subsequently, we train classifiers (base learners), xlm-roberta large foot_0 models with a classifier head, on these training sets. finally, we compute the final distribution through a weighted average of the probability generated by classifiers, with a model of identical structure generating the weights in this step. while our approach attained the second position in bulgarian and north macedonian, fifth in arabic, and eleventh in english, the performance of our weight model did not exhibit significant improvement compared to our baseline. moreover, the lower-resource techniques continue to suffer from imbalances between positive and negative instances. our code is publicly available at https://github.com/yuhang-zhu-nlp/ semeval2024_rdproj . ## experimental setup we use binary cross-entropy (bce) with weight as our loss function for each base learner. the equation is below: ## conclusion in this study we build a persuasion detection system to distinguish which techniques are used in the given text of memes. our system consists of ten base learners trained on different technique subsets and a weight model to generate the weight for soft voting. in the official ranking of semeval-2024 task 4 subtask 1, we get competitive results in the zero-shot setting. however, our weight model does not work very well, and does not show a significant improvement compared with our baseline. the problems may be 1) the accuracy of the weight model is not high enough, 2) the gap between the weight of base learners trained on target subsets and other base learners is not big enough. our idea to create a new technique subset to support techniques suffering from data imbalance seems feasible but the data imbalance between positive and negative instances of a technique is still a problem. the above discussion suggests the ideas to improve our approach. firstly, we can improve the accuracy of the weight model by applying some new training techniques because our training method is very simple. secondly, we need a more sophisticated technique grouping strategy which considers imbalance of positive and negative instances of a technique better.