Frank at A r AIE val Shared Task: A rabic Persuasion and Disinformation: The Power of Pretrained Models (2023)

Abstract: In this work, we present our systems developed for “ArAIEval” shared task of ArabicNLP 2023 (CITATION). We used an mBERT transformer for Subtask 1A, which targets persuasion in Arabic tweets, and we used the MARBERT transformer for Subtask 2A to identify disinformation in Arabic tweets. Our persuasion detection system achieved micro-F1 of 0.745 by surpassing the baseline by 13.2%, and registered a macro-F1 of 0.717 based on leaderboard scores. Similarly, our disinformation system recorded a micro-F1 of 0.816 , besting the naïve majority by 6.7%, with a macro-F1 of 0.637. Furthermore, we present our preliminary results on a variety of pre-trained models. In terms of overall ranking, our systems placed 7^\text{th} out of 16 and 12^\text{th} out of 17 teams for Subtasks 1A and 2A, respectively.

## introduction the digital communication landscape, vast in its dynamism, constantly evolves, presenting unique challenges in diverse cultural and linguistic contexts. arabic, with its rich historical and poetic traditions, spoken by more than 420 million people, is no exception @xcite . in the age of digital connectivity, platforms like twitter have become a boon and a bane. they enable rapid dissemination of information, but also disinformation spread, which can manipulate public perceptions and cause socio-political instability @xcite ). because tweets are so brief, accuracy is essential. this requires the use of strong rhetorical elements, making them an ideal environment for disinformation and persuasive strategies @xcite @xcite . taking into account the vast diversity of arabic dialects and cultural nuances, identifying these strategies is challenging, especially with the rise in misinformation campaigns @xcite . the importance of addressing this issue is amplified by the geopolitical significance of arabicspeaking regions, where digital narratives can influence diplomacy, policy decisions, and public sentiment @xcite @xcite . in response, the araieval shared task of ara-bicnlp 2023 focuses on two critical areas: subtask 1a: given a multigenre (tweet and news paragraphs of the news articles) snippet, identify whether it contains content with persuasion technique. subtask 2a: given a tweet, categorize whether it is disinformative @xcite . our paper offers the following contributions: in section 2, we outline previous and more recent studies on the identification of persuasion and disinformation. in section 3, we illustrate a thorough examination of the dataset. in section 4 we describe the systems and the results. finally, section 5 presents our conclusion and suggests directions for future exploration. furthermore, recent studies, such as al-sallab et al. @xcite , have emphasized the need for advanced embeddings and specialized datasets tailored to arabic peculiarities. an emerging trend marries traditional arabic linguistic studies with contemporary machine learning, targeting the precise detection of persuasive techniques in arabic content @xcite . the rapid proliferation of disinformation in the digital age, especially via social media platforms, requires the inter-related studies on persuasion and misinformation studies @xcite . in particular, checkthat! lab at clef has embarked on multifaceted research on misinformation in different languages, encompassing fact-checking, checkworthiness, bias identification, and source credibility assessment @xcite @xcite @xcite @xcite @xcite . this is consistent with the contemporary research trends that have changed from analyzing only news articles to scrutinizing social media for propaganda detection (woolley and howard, 2018; @xcite . interestingly, another study by @xcite proposes a bayesian deep learning model for misinformation detection, incorporating claim responses and quantifying prediction uncertainty, achieving superior performance in public datasets. moreover, da san martino et al. @xcite delved deeply into persuasive techniques, highlighting the emotional signals that resonate with readers. this foundational work paved the way for subsequent endeavors, notably the "detection of propaganda techniques in news articles" challenge posited by @xcite . building on this momentum, a recent investigation by @xcite sought to discern and categorize the underlying reasons for the deletion of arabic tweets, and later designed predictive models for potential deletions. in the multimedia realm, @xcite emphasized the importance of detecting propaganda within memes, thus underscoring the convergence of text and imagery in disinformation campaigns. ## data in this section, a detailed description of the dataset released by the araieval shared task organizers is provided. our primary focus is on the binary classification challenge subtasks 1a and 2a persuasion and disinformation detection dataset. data attributes: both subtasks consist of a dataset with the same structure, comprising an id, text, and label. however, the labels differentiate between the subtasks • id: numerical index of the data point. ## conclusion and future work in this paper, we discussed our approaches for the subtasks 1a and 2a of the shared task araieval 2023 on persuasion and disinformation detection in arabic. we employed the mbert model for subtask 1a and the marbert framework for subtask 2a, and according to the official leaderboard results, our system achieved a micro-f1 of 0.745 and a macro-f1 of 0.717 for subtask 1a, and a micro-f1 of 0.816 and macro-f1 of 0.637. we also detailed a series of experiments and made initial comparisons of our systems with various state-of-the-art frameworks. in future work, we plan to delve into feature engineering, potentially integrating meta-features associated with the text, such as text length, unique word count, and sentiment analysis.