R e DASP ersuasion at A r AIE val Shared Task: Multilingual and Monolingual Models For A rabic Persuasion Detection (2023)

Abstract: To enhance persuasion detection, we investigate the use of multilingual systems on Arabic data by conducting a total of 22 experiments using baselines, multilingual, and monolingual language transformers. Our aim is to provide a comprehensive evaluation of the various systems employed throughout this task, with the ultimate goal of comparing their performance and identifying the most effective approach. Our empirical analysis shows that *ReDASPersuasion* system performs best when combined with multilingual “XLM-RoBERTa” and monolingual pre-trained transformers on Arabic dialects like “CAMeLBERT-DA SA” depending on the NLP classification task.

## introduction in recent years, the detection of persuasion techniques in text has gained a significant attention in research. persuasion techniques can be used for either positive or negative ends. on one hand, persuasion can be used to convince people to support noble causes, promote social justice, and bring about positive change. however, these same techniques can also be exploited by individuals with ill intentions to manipulate and deceive others for personal gain or to perpetuate harmful beliefs and behaviors. moreover, persuasion techniques can be employed with malicious intent including : 1. phishing scams, 2. propagandistic content @xcite , 3. fallacy argumentation @xcite , and 4. coercive extortion tactics. with the increasing use of arabic language in various forms of media, including social media, and news articles, it has become crucial to develop effective methods for identifying persuasive strategies in arabic text. this task is challenging due to the complexities of the arabic language, which includes various dialects, nuances, and cultural references @xcite that can affect the interpretation of persuasive elements. researchers have employed various approaches, such as rhetoric methods @xcite , and deep learning models @xcite , to automatically detect propaganda and persuasion in arabic text. these techniques aim to identify specific linguistic features, such as sentiment analysis, and lexical semantics commonly used in persuasion. through this task, we have realized the importance of taking into consideration the different arabic nuances and dialects in multi-label and binary classification tasks. we also observe that arabic writing styles vary immensely depending on the type of the data (paragraphs vs tweets) where paragraphs mainly use modern standard arabic (msa) or classic arabic (ca) while tweets contain a diversity of arabic dialects using code-switching with other foreign languages and emojis. we begin by providing an overview of the araieval shared-task in section 2. next, we present a detailed description of the various systems utilized in our empirical study. we then delve into the preprocessing methods employed in section 4, before presenting the results in section 5. an error analysis is provided in section 6, followed by a discussion section offering insights and perspectives on the task at hand. finally, we summarize our findings and outline potential avenues for future research in section 8. formance. for the persuasion technique detection task, the organizers offered two subtasks : i) task 1a, and ii) task 1b. we will describe these two subtasks in the following sections. ## systems we will describe thoroughly the different systems we used during this task for an end of comparing their performance and finding the best system. ## preprocessing as illustrated in figure 1 , araieval's first task persuasion dataset contains two data types: arabic tweet rt @user my message to y'all!! _ link preprocessing steps kt [ _ ] my message to y'all!! _ [ ] [ _ ] cs [ _ ] _ [ ] [ _ ] ec [ _ ] _ _ _ [ ] [ _ ] kt :keyword translation. cs : code switching. ec : emoji conversion. we describe three preprocessing techniques we applied to the tweets to translate code-switched text to arabic. an example of these techniques are shown in ## experimental results we ran all classification experiments on a high performing cluster machine with an intel® xeon® gold 6252 (3.70ghz) processor with 24 cores and 48 threads running linux red hat enterprise server 8.6 with nvidia® volta v100 gpus. for task 1a, the initial structure of the redaspersuasion system with "xlm-roberta" @xcite achieves the best f1-micro score of 0.7336 on the test set. for task 1b, the redaspersuasion system with "camelbert-da sa" @xcite finetuned on sentiment analysis for dialect arabic (da) achieves the best performance on the testing set with a f1-micro score of 0.5584. according to our investigation reveals that during the development process, the redaspersuasion system powered by "xlm-roberta" shows the most promise, with the redaspersuasion system using monolingual "camelbert-da sa" coming in a close second. however, when it comes to the testing phase, one method excels in the first task, while the other method excels in the second task, as evidenced by their respective f1-micro scores. in due to the lack of visibility during the testing phase evaluation, we accidentally submitted wrong prediction results from the redaspersuasion system using "distilbert" instead of the intended topperforming redaspersuasion system using "xlm-roberta". we have also encountered technical difficulties on the araieval competition's hosting platform, codalab, mostly stemming from their http backend server. ## error analysis another limitation we faced in this task is the imbalanced nature of the data and the small number of examples in certain persuasion techniques. for example, "appeal to popularity" persuasive technique occurs only twice in the training set and once in both the dev and test set as described in to provide a more detailed examination of the prediction errors, we present the confusion matrices for the top-performing models on both tasks in figure 3 , and figure 4 in the supplementary material. as shown in figure 4 , "name calling-labeling" and "loaded language" had the highest accuracy rates, whereas all other persuasive technique were inaccurately predicted. this can be attributed to the limited quantity of training data available for these categories. ## discussion as shown in figure 2 , our arabic dialect identification process reveals that the araieval dataset encompasses a diverse array of dialects, with the most prominent languages being saudi arabian, egyptian, and palestinian dialects. different dialects have different vocabularies, and certain words or phrases might be interpreted differently in another dialect or deemed offensive. a concrete example would be the word " " in egyptian dialect means "health" while the same word means "fire" in moroccan dialect. these differences in arabic dialects can significantly impact persuasion strategies. an interesting take would be to consider the unique features of each dialect. ## conclusion we described our systems for the two subtasks of the araieval 2023 shared task on persuasion detection in arabic to detect a total of 23 persuasion techniques for multi-label classification. we experiment with different combinations of multilingual and monolingual transformers. we have proven that the redaspersuasion model can benefit from both the multilingual "xlm-roberta" transformer and the monolingual dialect arabic "camelbert-da sa" model depending on the nlp task. this task was an opportunity to evaluate the redaspersuasion model in depth and to conduct an error analysis to enhance our persuasion detection model for future works.