Legend at A r AIE val Shared Task: Persuasion Technique Detection using a Language-Agnostic Text Representation Model (2023)

Abstract: In this paper, we share our best performing submission to the Arabic AI Tasks Evaluation Challenge (ArAIEval) at ArabicNLP 2023. Our focus was on Task 1, which involves identifying persuasion techniques in excerpts from tweets and news articles. The persuasion technique in Arabic texts was detected using a training loop with XLM-RoBERTa, a language-agnostic text representation model. This approach proved to be potent, leveraging fine-tuning of a multilingual language model. In our evaluation of the test set, we achieved a micro F1 score of 0.64 for subtask A of the competition.

## introduction in an era defined by the rapid dissemination of information through digital channels, the task of recognizing persuasion techniques in text is now more crucial than ever @xcite @xcite @xcite . the advent of the internet and social media has created new avenues for influence and manipulation @xcite @xcite . although these technological advances have undoubtedly given people unparalleled access to information and a platform to express their thoughts, they have also introduced new avenues for persuasion, influence, and even propaganda. at the heart of this shift in our way of life is the fundamental challenge of distinguishing between genuinely informative and impartial content and content subtly crafted to promote a specific agenda or ideology. a critical component of the fight against spreading misinformation is the development of tools and resources in nlp that can detect persuasion technique in news articles and posts on social media. arabic language is among the most spoken languages in the world @xcite . the arabic speaking world stands out in importance due to its intricate mix of language, culture, and geography. the diversity of linguistic expressions in arabic extends beyond the spoken word and permeates every aspect of life, including digital text. arabic connects a vast and diverse population of native speakers and foreigners with its rich heritage and multiple dialects. its influence extends over a wide territory, from arab nations in the southern part of the arabian peninsula, to asia, and to the maghreb in north africa and the heart of the arabian peninsula @xcite . in this digital age, the importance and extensive use of arabic in various forms reflects the profound impact of technology on this linguistic community. it has provided a platform for arabic speakers around the world to engage in dialogue, share ideas, and express their thoughts in a global context. the diverse arab-speaking populations foster a rich and dynamic environment where individuals can connect, collaborate, and debate issues of global significance. however, the proliferation of digital media in arabic also presents challenges. digital media have become a fertile ground for the dissemination of persuasive content, including propaganda, misinformation, and various forms of manipulation @xcite . the use of technology in preserving the integrity of the arabic language and ensuring responsible use of digital media is of utmost importance. in today's world, we are surrounded by information, especially on the internet and social media. text classification can serve as a foundational step for the detection of the persuasion technique in text on social media and the internet. these texts can be classified according to their emotional tone using sentiment analysis techniques @xcite @xcite @xcite @xcite . persuasion techniques can be tricky to spot because they come in many forms, such as stories, logical arguments, or even subtle language tricks to sway our thinking. these techniques are powerful and are not always used for good reasons. to deal with this, researchers and experts are working on ways to detect when someone is trying to persuade people through text. in this way, we will be able to recognize when we are influenced and when people are spreading false or misleading information, which can be harmful. in this paper, we develop a methodology to automatically identify and analyze persuasion techniques in text using the xlm-roberta model. using the power of nlp, we can harness the capabilities of state-of-the-art models and unravel the persuasion techniques embedded within arabic text, contributing to both media ethics and the combating of misinformation. the intricacies of persuasion technique detection in arabic are discussed in detail, along with the possibility of applying this knowledge across different languages. ## related work a significant amount of research has been conducted on the detection of persuasion techniques in text @xcite @xcite @xcite . researchers have explored binary and multilabel approaches to detecting persuasion techniques. in their article, the authors in @xcite focused on detecting genres and persuasion techniques in multiple languages using various data augmentation techniques to enhance their models. for genre detection, they created synthetic texts using the gpt-3 davinci language model, while for persuasion technique detection, they augmented the dataset using text translation with the deepl translator. their fine-tuned models achieved top ten rankings in all languages, demonstrating the effectiveness of their approach. they also excelled in genre detection, securing top positions in spanish, german, and italian. they presented a single multilingual system using the roberta model to classify online news genres and improved this system by adding texts generated by the gpt3-davinci model to the training dataset. @xcite addressed misinformation in mainstream and social media and the challenges faced by manual detection and verification efforts by journalists and fact checkers in the semeval-2023 task @xcite . the task included three subtasks, six languages, and three surprise test languages, totaling 27 different test scenarios. the authors successfully submitted entries for all 27 test setups and the official results placed their system among the top three for 10 of these setups. they fine-tuned transformer models in the multiclass and multilabel classification settings, experimenting with both monolingual and multilingual pre-trained models, as well as data augmentation. their multilingual model based on xlm-roberta demonstrated superior performance in all tasks, even for languages not seen during training. the most effective solution in the detection of persuasion techniques for subtask 3 of semeval 2023 task 3 by @xcite delivered promising performance, with micro-f1 scores ranging from 36 to 55% for languages seen during training and 26 to 45% for languages unseen. given the multilingual nature of the data and the presence of 23 labels (resulting in limited labeled data for some language-label combinations), the authors chose to fine-tune pre-trained transformer-based language models. through extensive experimentation, they identified the optimal configuration, featuring a large multilingual model (xlm-roberta) trained on all input data, with carefully calibrated confidence thresholds for known and surprise languages separately. their final system demonstrated superior performance, ranking first in six of nine languages, including two surprise languages, and achieving highly competitive results in the remaining three languages. the experiments of @xcite focused on the detection of persuasion techniques in online news articles in polish and russian. these experiments used a taxonomy comprising 23 distinct persuasion techniques. persuasion techniques were evaluated in several ways, including the granularity of the classification (coarse with six labels or fine with 23 labels) and the level of location of the labels (subword, sentence, paragraph). the study compared the performance of state-of-theart transformer-based models trained both monolingually and multilingually. the findings indicate that multilingual models generally outperform monolingual models in various evaluation scenarios. however, due to the complexity of the task, there remains substantial room for improvement in the field of persuasion technique detection within online news articles. inspired by @xcite , our research focus is to determine whether a multi-genre snippet (tweets and news paragraphs of news articles) contains a persuasion technique or not. this is a binary classification task, and we are categorizing arabic text either as containing a persuasion technique or as lacking persuasion technique. our approach involves fine-tuning a large pre-trained language model based on transformers. we conducted experiments with different language models and concluded with xlm-roberta due to its superior performance. furthermore, we finetuned our system by tweaking hyperparameters and conducting multiple iterations. this process involved calculating cross-entropy loss, performing backpropagation, and updating the model's weights. ## results the evolution of pre-trained language models has ushered in significant advancements across different nlp tasks. in this section, we present the results of our experiments on persuasion technique detection in arabic text. our findings revealed the versatility of xlm-roberta in effectively handling multilingual data. we evaluated the model using the micro-f1 score and an overview of the results achieved by our model on the dataset is shown in ## conclusion the application of xlm-roberta, a languageagnostic text representation model, for the detection of the persuasion technique in arabic text illustrates the growing potential of cross-lingual models in specialized nlp tasks. arabic, with its rich morphology and diverse dialects, presents unique challenges for text analysis. our proposed model has the ability to capture the underlying structure and semantics of persuasion technique in text, regardless of language. the results obtained in our analysis demonstrate that xlm-roberta can adapt effectively and perform well on such intricate tasks, even in languages that are structurally different from the ones they were originally trained on. this not only underscores the versatility of xlm-roberta but also sets a promising direction for further research in detecting persuasion techniques across various languages. in future work, we plan to accommodate more languages in the dataset, and fine-tune other multilingual models for this task.