rematchka at A r AIE val Shared Task: Prefix-Tuning & Prompt-tuning for Improved Detection of Propaganda and Disinformation in A rabic Social Media Content (2023)

Abstract: The rise of propaganda and disinformation in the digital age has necessitated the development of effective detection methods to combat the spread of deceptive information. In this paper we present our approach proposed for ArAIEval shared task : propaganda and disinformation detection in Arabic text. Our system utilised different pre-trained BERT based models, that makes use of prompt-learning based on knowledgeable expansion and prefix-tuning. The proposed approach secured third place in subtask-1A with 0.7555 F1-micro score, second place in subtask-1B with 0.5658 F1-micro score. However, for subtask-2A & 2B, the proposed system achieved fourth place with an F1-micro score of 0.9040, 0.8219 respectively. Our findings suggest that prompt-tuning-based & prefix-tuning based models performed better than conventional fine-tuning. Furthermore, using loss aware class imbalance, improved performance.

## introduction with the growing popularity of social media in our current society, platforms such as twitter, and reddit have become critical tools for influencing people. people on social media prefer to express their opinions, points of view more freely, and share information. however, these platforms can be used to deceive and manipulate individuals. in addition to spreading rumors, and fake news. this can be done through propaganda techniques. propaganda refers to the systemic dissemination of information, ideas, or opinions, often through biased or misleading means, with the intention of influencing or manipulating public perception, attitudes, behaviors, or beliefs. it is a persuasive communication technique employed by individuals, organizations, or governments to shape public opinion and advance specific agendas. the rise of propaganda and disinformation has necessitated the development of effective detection methods to combat the spread of deceptive information. with the advent of pre-trained language models, there has been a significant advancement in the field of natural language processing (nlp), offering promising opportunities for combating the dissemination of false information. several works have been proposed to improve the identification of persuasion techniques in text as the recent shared-task propaganda detection in arabic @xcite . @xcite and @xcite utilized arabert for this task. @xcite utilized a multi-task learning model, which includes a shared arabert encoder and task-specific binary classification layers. this model has been trained to learn one binary classification task per propaganda approach jointly. in this paper, we present our solution to the araieval shared task @xcite . the araieval shared task is held with the 1st arabic natural language processing conference co-located with the emnlp 2023. the goal of the task is to build models for identifying propaganda and disinformation in arabic content. the shared task consists of two tasks. the first task is persuasion technique detection in arabic text. the second task is disinformation detection in the text. this paper describes the system developed for addressing propaganda and disinformation detection in text, for both subtasks. given that a key challenge in this task is the unbalanced distribution of the dataset. additionally, the contextual nature of language and the cultural nuances involved in the text. we follow best practices from recent work on enhancing model generalization and robustness, by using different parameter-efficient techniques (peft), contrastive loss, adversarial training, and loss-aware class imbalance methods. the parameter-efficient fine tuning (peft) is a technique used to improve the performance of pretrained language models on specific downstream tasks. peft methods freeze the pretrained model parameters during fine-tuning and put a few trainable parameters (the adapters) on top of it. the adapters are taught how to pick up knowledge appropriate to a given task. peft of pre-trained language models has recently demonstrated remarkable results, effectively matching the performance of full fine-tuning while utilizing significantly fewer trainable parameters @xcite @xcite , thereby addressing storage and communication constraints. such approaches include prefix-tuning (li and liang, 2021), prompt-tuning @xcite , soft-prompting @xcite and lora @xcite . adversarial training (at) @xcite is a method to improve the model's resistance to adversarial examples and acts as a regularizer. the key is to disturb the input example using a gradient-based perturbation, and then train the model on both clean and perturbed examples. contrastive loss is one of the first training objectives that was used for contrastive learning. it takes as input a pair of samples that are either similar or dissimilar, and it brings similar samples closer and dissimilar samples far apart in embedding space @xcite . such loss has shown model performance improvement compared to cross-entropy on multiple problems @xcite @xcite . the rest of the paper goes as follows: section 2 gives an overview of the dataset, section 3 discusses the proposed methods, section 4 shows experimental results, and section 5 concludes the paper. ## data the dataset used has been provided by the organizers for the araieval shared task. ## methodology this section presents the various approaches used while developing the final models: a weighted ensemble of bert-based models. ## results and discussion in this section, the performance of the model is reported based on the official metric during devphase and test-phase. the official metric used for all tasks is the micro average f1-score. ## conclusion in this paper, the results and the main findings of araieval shared task were presented, in which different experiments were carried out with mar-bert, arabert v2, and aragpt models. our models secured third place in subtask-1a, second place in subtask-1b, and fourth place in subtask-2a&2b. our proposed solution is an ensemble of different bert-based models. these models are developed differently, some are trained using prefix-tuning, and others are trained using fine-tuning and prompttuning. leverages fine-tuned, per-trained models. in addition, training tricks were utilized as fgm, contrastive learning, and balanced sampler. in future efforts, we plan to further improve our model to better handle data-imbalance constraints and world knowledge needed to improve model performance.