Itri Amigos at A r AIE val Shared Task: Transformer vs. Compression-Based Models for Persuasion Techniques and Disinformation Detection (2023)

Abstract: Social media has significantly amplified the dissemination of misinformation. Researchers have employed natural language processing and machine learning techniques to identify and categorize false information on these platforms. While there is a well-established body of research on detecting fake news in English and Latin languages, the study of Arabic fake news detection remains limited. This paper describes the methods used to tackle the challenges of the ArAIEval shared Task 2023. We conducted experiments with both monolingual Arabic and multi-lingual pre-trained Language Models (LM). We found that the monolingual Arabic models outperformed in all four subtasks. Additionally, we explored a novel lossless compression method, which, while not surpassing pretrained LM performance, presents an intriguing avenue for future experimentation to achieve comparable results in a more efficient and rapid manner.

## introduction the growing presence of social media as a way to quickly disseminate information to broad audiences, has had an undeniable shaping the sphere of public opinion. by their very nature, social media platforms have the associated peril of carrying messages that are erroneous at best, or carefully crafted to misinform and manipulate, at worst (e.g. @xcite . the development of nlp tools to fact check and explore persuasion techniques is a potential approach to counteract the effect of misinformation in social media. while this is an active area of research that is well established for english and other latin languages, for arabic news, there is still much room to explore. this paper describes the methodology used to tackle the classification tasks presented by the araireval shared 2023 task @xcite , which builds upon wanlp 2022 @xcite . the tasks are described in section 3. we have mainly focused our efforts on two distinct approaches: on the one hand, the use of pretrained language models (lms), which has been an established way to achieve state-of-the-art results in a range of nlp tasks (e.g. @xcite . pre-trained lms are advanced models, often based on transformer architectures, that are pre-trained on massive datasets. on the other hand, we explore the approach presented by @xcite advocates for the use of simpler models that are less resource intensive and more interpretable. this method uses lossless compression and a distance metric with a k-nearest-neighbor classifier for text classification. the inconsistencies detected in this implementation will be detailed later on. the paper is organized as follows: section 2 briefly talks about related work, section 3 summarizes the datasets on each sub-task and section 4 the methodology. finally, sections 5 and 6 present the conclusions and limitations, respectively. all of the source code to reproduce the results is available in a github repository foot_0 . ## related work prior research in the field of automated arabic fake news detection predominantly relied on traditional machine learning classifiers, focusing mainly on binary classification scenarios. mahlous and allaith, 2021 applied nb, lr, svm, rf, and xgb methods to classify arabic news tweets as either fake or not. among these, the logistic regression (lr) classifier achieved 87.8% accuracy using tf-idf features at the n-grams level. recent research has focused on assessing the performance of transformer-based models. for example, antoun et al., 2020 showed that arabert v02 achieved high accuracy across various experimental scenarios. similarly, nassif et al., 2022 achieved favorable results on a covid-19 fake news dataset using pre-trained models like roberta-base @xcite , arbert @xcite and arabic-bert @xcite . @xcite show the good performance of marabert with cnns for tweet classification. while binary classification of news content has been a traditional approach, there is an emerging interest in multi-label classification scenarios. several studies have ventured into this realm. argotario, as introduced by @xcite , is a game-based platform designed to accumulate a dataset portraying a spectrum of fallacious arguments, with labels like: ad hominem, appeal to emotion, red herring, hasty generalization, irrelevant authority. in parallel da san martino et al., 2019 extracted and analyzed 451 articles sourced from 48 news outlets. these articles were annotated to highlight 18 unique propaganda techniques. these efforts emphasize the pivotal role of multilabel classification in revealing the nuanced tactics inherent in news narratives. task 2, disinformation detection, centers around identifying and categorizing disinformation within tweets. subtask 2a involves a binary classification task where the goal is to determine whether a tweet contains disinformation. subtask 2b further refines this by requiring the detection of fine-grained disinformation classes, including hate-speech, offensive content, rumors, and spam. ## methodology in this section, we describe our approach to processing the data, the models, and the experiments we conducted for all tasks. ## conclusions in our participation in the araieval shared task, we predominantly employed transformer-based models for persuasion techniques and disinformation detection tasks, given their demonstrated proficiency with arabic textual data. although our results highlighted the strengths of these models, we simultaneously recognized the emerging potential of the compression-based approach to text classification. while these compression-based methods are in their infancy, they offer exciting opportunities for continued research. future studies should delve deeper into the applicability of lossless compressors for text classification and seek to identify non-parametric machine learning algorithms that best align with these compressors. importantly, compressor-driven systems might be more advantageous in situations where resource efficiency and rapid processing take precedence over accuracy.