U ned M edia B ias T eam @ S em E val-2023 Task 3: Can We Detect Persuasive Techniques Transferring Knowledge From Media Bias Detection? (2023)

Abstract: How similar is the detection of media bias to the detection of persuasive techniques? We have explored how transferring knowledge from one task to the other may help to improve the performance. This paper presents the systems developed for participating in the SemEval-2023 Task 3: Detecting the Genre, the Framing, and the Persuasion Techniques in Online News in a Multi-lingual Setup. We have participated in both the subtask 1: News Genre Categorisation, and the subtask 3: Persuasion Techniques Detection. Our solutions are based on two-stage fine-tuned multilingual models. We evaluated our approach on the 9 languages provided in the task. Our results show that the use of transfer learning from media bias detection to persuasion techniques detection is beneficial for the subtask of detecting the genre (macro F1-score of 0.523 in the English test set) as it improves previous results, but not for the detection of persuasive techniques (micro F1-score of 0.24 in the English test set).

## introduction misinformation, and propaganda in particular, represents a major issue in online media. the ability to detect and analyze persuasive techniques in text is essential in order to be able to identify and counter misinformation. this task aims to foster the use of artificial intelligence to perform media analysis in order to address this issue. the task of automated persuasive techniques detection has been an active research area in recent years @xcite . previous works have focused on different aspects of the problem, such as the development of datasets @xcite , the development of persuasive techniques taxonomies @xcite and the development of automated systems for the detection of persuasive techniques @xcite . the semeval-2023 task 3 @xcite ) focuses on the detection of genre, framing and persuasion techniques in news articles in a multilingual setup. the task consists of three subtasks: news genre categorisation, framing detection and persuasion techniques detection. the data presented in the task is both multilabel and multilingual, and it also covers complementary dimensions of what makes text persuasive, namely style and framing. the task covers the following languages: english, french, german, italian, polish, russian, spanish, @xcite we propose to use a two-stage fine-tuning @xcite of multilingual models. we use the xlm-roberta @xcite model for subtask 1, and bert-multilingual @xcite model for subtask 3. the reason why these models are used can be seen in the subsection on ablation analysis. both models have been fine-tuned on the babe @xcite and mbic @xcite datasets for the task of media bias detection. we then fine-tune the model on the data provided for the task to adapt it to the specific task. our approach achieves good results in the first subtask, and average results on the third. we have obtained a macro f1-score of 0.523 (position 8 out of 22) for the english test set in the first subtask, and a micro f1-score of of 0.24 (position 18 out of 23) for the english test set in the third subtask. we performed an error and ablation analysis to analyze the errors of our system, and for analyzing the effect of different techniques on the performance. our system struggles with the detection of more complex persuasion techniques, especially those less represented in the dataset. to promote replicability, both the code and the models are publicly available. the code can be found at github foot_0 , and the models can be found at huggingface: subtask 1 foot_1 , subtask 3 foot_2 . ## background the task of media bias detection is closely related to the task of persuasive techniques detection. it can be seen as a wider problem that encompasses the detection of various persuasive techniques. as the systematic review of (rodrigo-ginés et al., under review) lists, certain forms of media bias coincide with the persuasive techniques that are part of this task: appeal to authority, appeal to groupthink/popularity, red herring, loaded language, and labeling. it is therefore natural to explore the transfer of knowledge from one task to the other in order to improve the performance of automatic detection systems, as by leveraging knowledge from the source task, the model can potentially learn faster, require less labeled data, and achieve better performance on the target task. the semeval-2023 task 3 @xcite ) is organized as a multi-lingual task with 3 subtasks: news genre categorization, framing detection and persuasion techniques detection. the task focuses on online news articles in 9 languages: english, french, german, italian, polish, russian, spanish, greek and georgian. our efforts have focused on the first and third tasks. the news genre categorization subtask consists of determining the genre of a given article (opinion, objective news or satire). the task is framed as a multi-class classification task at article-level. the persuasion techniques detection subtask aims to determine the persuasion techniques used in each paragraph of an article. the task is framed as a multi-label classification task at paragraph-level. the main contribution of this work is the use of transfer learning from media bias detection to persuasion techniques detection. to the best of our knowledge, this is the first time this knowledge transfer has been explored for this task. in recent years, researchers in media bias detection have explored different approaches to generalize media bias. two categories of approaches have emerged: non-neural network models and neural network models. non-neural network models rely on classical machine learning methods, which require some feature engineering to generate linguistic @xcite or reported speech @xcite features. on the other hand, neural network models have been shown to outperform traditional methods. in particular, rnns @xcite and transformers @xcite are the most commonly used neural networks for media bias detection. we have fine-tuned some transformers models, using the mbic @xcite and babe datasets @xcite . these datasets are designed specifically for the task of media bias detection and contain annotated articles from various sources in multiple classes. the use of these datasets allow us to leverage the pre-trained models to capture the languageindependent features of media bias and to improve the performance of our systems by taking advantage of the knowledge acquired in other tasks. as these datasets only contain texts written in english, we do not expect them to have a direct impact on the performance of the systems for the other languages, but we expect them to help the model to learn language-independent features. ## future work and conclusions the results obtained shows that the use of transfer learning from media bias detection to persuasive techniques detection is beneficial for the subtask of detecting the genre. in the case of the detection of the persuasive techniques, our results are not as good, as our system struggles with the detection of more complex techniques, especially those less represented in the dataset. due to time constraints, we were unable of performing multiple techniques setups. we believe that with further exploration in this direction, better results can be obtained. for future work, we plan to explore different architectures and transfer learning techniques in order to further improving the performance of our systems. we plan to use the data provided in the competition for the task of media bias detection as we have already proved that the transfer of knowledge from one task to the other is beneficial.