Multilingual Multifaceted Understanding of Online News in Terms of Genre, Framing, and Persuasion Techniques (2023)

Abstract: We present a new multilingual multifacet dataset of news articles, each annotated for genre (objective news reporting vs. opinion vs. satire), framing (what key aspects are highlighted), and persuasion techniques (logical fallacies, emotional appeals, ad hominem attacks, etc.). The persuasion techniques are annotated at the span level, using a taxonomy of 23 fine-grained techniques grouped into 6 coarse categories. The dataset contains 1,612 news articles covering recent news on current topics of public interest in six European languages (English, French, German, Italian, Polish, and Russian), with more than 37k annotated spans of persuasion techniques. We describe the dataset and the annotation process, and we report the evaluation results of multilabel classification experiments using state-of-the-art multilingual transformers at different levels of granularity: token-level, sentence-level, paragraph-level, and document-level.

## introduction internet has changed profoundly the information landscape by creating direct channels of communication between information producers and consumers. at the same time, it has also increased the risk for readers to be exposed to disinformation (aka "fake news"), propaganda, manipulation, etc., which has grown into an infodemic @xcite . the consequences of this are very concrete, as swaying the hearts and the minds of a population also sways their choices, notably during elections. therefore, online media analysis is important in order to understand the news ecosystem and the presented narratives around certain topics across countries, and to identify manipulation attempts and deceptive content, in order to provide citizens with a more transparent and comprehensible understanding of the online news. given the scale of the media landscape, media analysis needs automatic tools, which in turn need training data. with this in mind, we introduce a new dataset that covers several complementary aspects of the news: genre (objective news reporting vs. opinion vs. satire), framing (what key aspects are highlighted), and persuasion techniques (logical fallacies, emotional appeals, personal attacks, etc.). we collected news articles between 2020 and mid-2022, from sources ranging in the whole political spectrum and revolving around widely discussed topics such as covid-19, climate change, abortion, migration, the russo-ukrainian war, and local elections. our dataset is multilingual (english, french, german, italian, polish, and russian), multilabel, and covers complementary dimensions for better news understanding. our taxonomy of persuasion techniques is an improvement and also an extension compared to previous inventories, and it contains 23 labels organised in a 2-tier hierarchy. we annotated a total of 1,612 articles with 37k annotated snippets for persuasion techniques, which is a 3-fold increase in the number of articles and 4-fold in the number of spans compared to the largest previous efforts, which focused on english only @xcite . our contributions can be summarized as follows: ## multifacet annotation scheme this section offers an overview of the three different facets considered in our annotation scheme. ## dataset description we feature six languages: english, french, german, italian, polish, and russian. the english articles are the ones from @xcite , but we slightly modified their annotations for persuasion techniques to match the guidelines of this work (see section 3.3). as genre and framing annotations for english were not present in @xcite , we added them following the guidelines for the other languages. ## conclusion and future work we presented a new multilingual multifacet dataset for understanding the news in terms of genre, framing, and persuasion techniques. the dataset covers current topics of public interest in six european languages, and contains 1,612 documents with more than 37k annotated spans. we further performed a number of multilabel classification experiments using state-of-the-art multilingual transformer-based models, exploring different levels of granularity and focus. our experiments showed the utility of multilingual representations even when evaluated on a specific language. we hope that our dataset will foster the development of methods and tools to support the analysis of online media content. in future work, we plan to do in-depth analysis of the data, extend it to more languages, including non indo-european ones with non-latin scripts, and other genres of text, e.g., social media posts. note an extended version of the dataset presented in this paper was used in the context of semeval-2023 task 3 on detecting the genre, the framing, and the persuasion techniques in online news in a multilingual set-up @xcite ), foot_4 where it was augmented with a new test set, including three new languages: georgian, greek, and spanish. we make both the present and semeval-2023 task 3 versions of the dataset publicly accessible to the community for research purposes. for further information on the dataset and future releases please refer to https://joedsm. github.io/pt-corpora/ . biases human data annotation involves some degree of subjectivity. to mitigate this, we created a comprehensive 60-page guidelines document @xcite ), which we updated from time to time to clarify newly arising important cases during the annotation process. we further had quality control steps in the data annotation process, and we have been excluding low-performing annotators. despite all this, we are aware that some degree of intrinsic subjectivity will inevitably be present in the dataset and will eventually be learned by models trained on it. baseline models the reported experiments can be seen as strong baselines as they include fairly small encoder-only transformer architectures. we leave for future work the exploration of other architectures and modeling techniques that are known to improve the efficiency and to reduce the computational requirements of the used models, e.g., fewshot and zero-shot in-context learning, instructionbased evaluation, multitask learning, etc.