AAST - NLP at A r AIE val Shared Task: Tackling Persuasion technique and Disinformation Detection using Pre-Trained Language Models On Imbalanced Datasets (2023)

Abstract: This paper presents the pipeline developed by the AAST-NLP team to address both the persuasion technique detection and disinformation detection shared tasks. The proposed system for all the tasks’ sub-tasks consisted of preprocessing the data and finetuning AraBERT on the given datasets, in addition to several procedures performed for each subtask to adapt to the problems faced in it. The previously described system was used in addition to Dice loss as the loss function for sub-task 1A, which consisted of a binary classification problem. In that sub-task, the system came in eleventh place. We trained the AraBERT for task 1B, which was a multi-label problem with 24 distinct labels, using binary cross-entropy to train a classifier for each label. On that sub-task, the system came in third place. We utilised AraBERT with Dice loss on both subtasks 2A and 2B, ranking second and third among the proposed models for the respective subtasks.

## introduction social media has become part and parcel of our everyday lives and a main source of information for every individual. unfortunately, due to the nature of social media, the spread of disinformation @xcite ) is very relevant and causes major troubles. for example back in the covid-19 pandemic, some researchers coined the term "infodemic" to describe the spread of false information among people during that period @xcite . many researchers have proposed their systems to fight the spread of disinformation on social media platforms, powered by recent advances in nlp and the introduction of large language models including bert @xcite which revolutionized nlp and was adapted to many tasks. * . equally contributed persuasion is a type of social interaction that attempts to influence and change attitudes in an atmosphere of free choice @xcite . persuasion techniques are incredibly important linguistic techniques that can have massive effects on different fields and industries. an example of this is the usage of these techniques in advertising campaigns, which can lead to impressive results when it comes to changing customers attitudes and receiving their responses without imposing on them @xcite . this paper tackles the various systems our team attempted for the araieval 2023 shared tasks (ove). the first step was to look at some of the earlier publications from wanlp 2022 @xcite , which provided a number of crucial insights that served as a foundation for our work. related work includes the system presented in @xcite for the identification of disinformation through samples, combined with many additional significant results as well as fine-grained disinformation labels from those samples. the following sections of the paper comprise a data section which describes the data sources and preprocessing methods applied to the data. a system section describing the pipeline, a results section, a discussion and a summary. ## data in this section, we will describe the data sources and the preprocessing methods that we applied to prepare the data. we will also provide some descriptive statistics and visualizations of the data to give an overview of its characteristics and distribution. ## discussion a diverse set of limitations were encountered during the development of the aforementioned systems. another drawback stemmed from the underlying dependencies among task 1b labels, as attempting a direct approach did not lead to optimal outcomes. the subjective labelling of tasks 1b and 2b made it difficult to leverage external data sources to further train our model. one strategy worth highlighting is the use of a cnn-bilstm and arabert hybrid model @xcite . however, this did not produce satisfactory results since the model appeared to overfit the training instances.with few modifications, this strategy may be viable. furthermore, the unexpected decline in task 1a's performance necessitates further investigation and experimentation to determine the cause. ## summary the proposed system based on arabert was detailed, and the experiments conducted were all addressed. the adaptation of dice loss boosted our performance on all of the tasks and partially addressed the issue of class imbalance yet there is a huge room for improvement. there are other intriguing future directions, like the development of a data augmentation package that supports differ-4 568 ent data augmentation techniques. furthering the solution to the issue of class imbalance is another intriguing path. last but not least, the problem of the underlying dependencies and ways of tackling multilabel tasks should be inspected, and new methods should be investigated and developed in the near future. we intend to invesigate these various approaches in detail in the future since we believe there is still room for improvement in finetuning as well as experimenting with other approaches such as different hybrid model architectures and different data augmentation methods. in the future, we plan to thoroughly explore these diverse approaches because we are convinced that there is further potential for enhancing fine-tuning. this includes experimenting with alternative hybrid model architectures and various data augmentation techniques.