QCRI at S em E val-2023 Task 3: News Genre, Framing and Persuasion Techniques Detection Using Multilingual Models (2023)

Abstract: Misinformation spreading in mainstream and social media has been misleading users in different ways. Manual detection and verification efforts by journalists and fact-checkers can no longer cope with the great scale and quick spread of misleading information. This motivated research and industry efforts to develop systems for analyzing and verifying news spreading online. The SemEval-2023 Task 3 is an attempt to address several subtasks under this overarching problem, targeting writing techniques used in news articles to affect readers’ opinions. The task addressed three subtasks with six languages, in addition to three “surprise” test languages, resulting in 27 different test setups. This paper describes our participating system to this task. Our team is one of the 6 teams that successfully submitted runs for all setups. The official results show that our system is ranked among the top 3 systems for 10 out of the 27 setups.

## introduction monitoring and analyzing news have become an important process to understand how different topics (e.g., political) are reported in different news media and within and across countries. this has many important applications since the tone, framing, and factuality of news reporting can significantly affect public reactions toward social or political agendas. a news piece can be manipulated on multiple aspects to sway readers' perceptions and actions. going beyond information factuality, other aspects include objectivity/genre, framing dimensions inserted to steer the focus of the audience @xcite , and propaganda techniques used to persuade readers towards a certain agenda @xcite . news categorization is a well studied problem in the natural language processing field. recently, research attention has focused on classifying news by factuality @xcite , or other related categorizations such as fake vs. satire news @xcite . however, there have been efforts towards other classification dimensions. @xcite developed a corpus of news articles annotated by 15 framing dimensions such as economy, capacity and resources, and fairness and equality, to support development of systems for news framing classification. moreover, identifying propagandistic content has gained a lot of attention over several domains including news @xcite , social media @xcite and multimodal content @xcite . the semeval-2023 task 3 shared task aims at motivating research in the aforementioned categorization tasks, namely: detection and classification of the genre, framing, and the persuasion techniques in news articles @xcite . it targets multiple languages including english, french, german, italian, polish, and russian to push the research on multilingual systems. moreover, to promote development of language-agnostic models, the task organizers released test subsets for three surprise languages (georgian, greek, and spanish). our proposed system is based on fine-tuning transformer based models @xcite in multiclass and multi-label classification settings for different tasks and languages. we participated in all three subtasks submitting runs for all nine languages, which resulted in 27 testing setups. we experimented with different mono and multilingual transformer models, such as bert @xcite and xlm-roberta @xcite among others. in addition, we also experimented with data augmentation. the rest of the paper is organized as follows. section 2 gives an overview of related work. in section 3, we present the proposed system. in section 4, we provide the details of our experiments. section 5 presents the results for our official runs, and finally, we conclude our paper in section 6. ## system overview our system is comprised of preprocessing followed by fine-tuning pre-trained transformer models. the preprocessing part includes standard model specific tokenization. our experimental setup consists of (i) monolingual ( * mono ): training and evaluating monolingual transformer model for each language and subtask; (ii) multilingual ( * multi ): combining subtask specific data from all languages for training, and evaluating the model on task and language specific data; (iii) data augmentation ( * aug ): applying data augmentation on language specific training set, then training a monolingual model using augmented dataset, and evaluating it on the test set. this has been applied for each subtask. ## experiments in this section, we describe the tasks and datasets used during experiments and provide implementation details for our models. ## results the results for our official runs per subtask are shown in tables 2, 3 and 4. for each subtask, we compare our official runs to two baselines: the top run in each test language, and the baseline as reported by the task organizers. we observe that the multilingual models are generally the best performing models across all tasks. on average, the performance of the system was best for subtask3 with a slight average ranking difference compared to subtask2. another interesting observation is that although subtask3 has much larger train subsets , since it operates on the paragraph level, this did not improve the average system ranking across languages when compared to sub-task2. the results also clearly show the robustness of our model across languages and subtasks, as it managed to be among the best 3 runs for 10 out of the 27 test subsets, and it was among the top 5 runs for 15 of them. results over subtask1 and subtask3 showed that our proposed system had a strong cross-lingual transfer ability when training the model on multilingual data and testing it on unseen languages (georgian, greek and spanish). ## conclusion in this paper, we presented our experiments and findings on news genre categorization, framing and persuasion techniques detection on multiple languages, which was a part of semeval-2023 task 3 shared task. the task includes 27 test setups for three subtasks and nine test languages. our team successfully submitted runs for all setups. we proposed a system that is based on fine-tuning transformer models in multiclass and multi-label classi-fication settings. we experimented with different mono and multilingual pre-trained models, in addition to data augmentation. from the experimental results, we observed that our multilingual model based on xlm-roberta performs better across all tasks, even on unseen languages. our future work includes domain adaptation and further exploration of data augmentation techniques.