GREENER : Graph Neural Networks for News Media Profiling (2022)

Abstract: We study the problem of profiling news media on the Web with respect to their factuality of reporting and bias. This is an important but under-studied problem related to disinformation and “fake news” detection, but it addresses the issue at a coarser granularity compared to looking at an individual article or an individual claim. This is useful as it allows to profile entire media outlets in advance. Unlike previous work, which has focused primarily on text (e.g., on the text of the articles published by the target website, or on the textual description in their social media profiles or in Wikipedia), here our main focus is on modeling the similarity between media outlets based on the overlap of their audience. This is motivated by homophily considerations, i.e., the tendency of people to have connections to people with similar interests, which we extend to media, hypothesizing that similar types of media would be read by similar kinds of users. In particular, we propose GREENER (GRaph nEural nEtwork for News mEdia pRofiling), a model that builds a graph of inter-media connections based on their audience overlap, and then uses graph neural networks to represent each medium. We find that such representations are quite useful for predicting the factuality and the bias of news media outlets, yielding improvements over state-of-the-art results reported on two datasets. When augmented with conventionally used representations obtained from news articles, Twitter, YouTube, Facebook, and Wikipedia, prediction accuracy is found to improve by 2.5-27 macro-F1 points for the two tasks.

## introduction the problem of news media profiling with respect to their factuality of reporting and political bias is important but under-studied. it is related to disinformation and "fake news" detection, but it is of different granularity compared to looking at an individual article or at an individual claim. this kind of profiling can be done by professional fact-checkers, who inspect the articles and the multimedia material published by the target news outlet. however, doing this automatically while solely relying on text features is a very challenging task as previous work has shown @xcite @xcite . it gets even more complicated when considering news sources where only limited number of examples is available for evaluation. therefore, not only is there a need to characterize news media more thoroughly, but there is also a need to be able to do so in a predictive fashion using limited information. a crucial consideration is the need to complement the textual representation with other elements of a news medium that may serve as reliable indicators of its factuality of reporting and bias. these may relate to multimedia creation and curation @xcite , to its underlying infrastructure and technological components used to serve its content @xcite @xcite , and, more critically, to characteristics of its audience @xcite @xcite . here, we explore ways to augment the textual representations from the articles published by a target news medium by introducing new information sources that relate to media audience homophily, audience engagement, and media popularity. in particular, we propose the greener (graph neural network for news media profiling) model, which builds graph neural networks that model the audience overlap between websites, which we further complement with other state-of-the-art representations. our contributions can be summarized as follows: previous work on automating the process of characterizing news sites based on the factuality of their reporting and on their political bias has mainly focused on analysis of the textual content of the website @xcite @xcite @xcite @xcite . although style-based analysis of the text can help reveal the intent of an article, it cannot ultimately evaluate the authenticity and the objectivity of the claims stated in that article. in fact, as demonstrated by @xcite on a manually fact-checked and categorized dataset, state-ofthe-art textual representations can only achieve a prediction accuracy around 65-71% for factuality and 70-85% for bias, depending on the datasets. thus, several approaches have been proposed to supplement the content-level analysis with other contextual and relational information about the target news outlet. multimedia has been an important element of conveying news and information by all news media. due to its prevalence, tampering detection and identification of processing related traces in photos and videos have long been a focus of study @xcite . the fact that multimedia editors of a news site follow a workflow when creating, acquiring, editing, and curating content for their pages makes it possible to characterize a website based on multimedia content. therefore, visual features are increasingly being explored and used to predict factuality @xcite @xcite @xcite @xcite . beyond textual and visual features, news web sites also exhibit distinct characteristics in the way they set up their infrastructure to serve content. to detect low-factuality news sites, it was proposed to use features that relate to the network, to the web design, and to data elements of the target website. at the network level, it has been shown that a website's domain, certificate, and hosting properties can serve as reliable identifiers @xcite . concerning the web design aspect, several features capturing the pattern of elements that govern the structure and the style of a web page have also been used @xcite . finally, at the data level, shared content among web sites and mutually linked sites were used to identify similar sites @xcite . overall, a major advantage of using infrastructure features is their content-agnostic nature. another set of features used to estimate the factuality and the bias of a news source is based on characteristics of teh audience following the homophily principle, which simply states that similar individuals interact with each other at a higher rate than they do with dissimilar ones. in the context of social media platforms, several approaches were proposed to infer the similarity between news media through obtaining and comparing descriptive characteristics of the followers of a news medium @xcite and by profiling how these followers respond to the content of the target news medium in their comments and with their posting and sharing behavior @xcite . in this regard, a more reliable indicator for similarity between news web sites is to what extent their followers overlap @xcite . ultimately, these statistics were obtained from disparate data sources and are complementary in nature. therefore, a more accurate characterization of the news reporting practice of a given news medium can be achieved by deploying more comprehensive and heterogeneous learning approaches. with this objective in mind, in the present work, we propose to use graph neural networks to model the audience homophily relations based on audience overlap and engagement statistics from alexa. in order to provide a more holistic view, our representation is also coupled with state-of-the-art textual representations extracted from media articles, as well as on other audience characteristics proposed in the context of social media platforms. ## method to characterize the similarity between news media in terms of their factuality of reporting and political bias, we mainly rely on audience overlap, which is based on the idea that if a group of visitors have a common interest in some websites, then those websites must be similar in some respect. with this idea, we create an undirected web audience overlap graph, where nodes represent news media sites and edges indicate that that two news sites have an overlapping set of visitors, as well as the degree of that overlap. the graph is created using a seed list of news sites for which factuality and bias ratings are manually annotated by professional fact-checkers. this initial graph only captures the relationship between websites due to visitors that are interested in a pair of sites, and it cannot represent indirect relations where visitors might have common taste in their news consumption, but do not necessarily visit the same websites. in order to also identify such connections between news sites, we iteratively expand the graph by adding new neighboring nodes for a more comprehensive representation of the audience overlap, which is discussed in detail in section 3.2. the graph is further enhanced by incorporating user engagement statistics as node attributes in order to model the relationship between a site and its visitors better. we then use graph neural networks to encode these relations and to obtain node embeddings for news websites. we further combine these embeddings with textual representations from articles from each news website. ## experiments and evaluation below, we describe the datasets, the evaluation setup, and the experiments for predicting the factuality and the bias for entire news outlets. ## discussion other alexa features alexa site info maintains a wide array of audience-centric statistics about the websites. apart from audience overlap, we also experimented with the following features: alexa rank, total sites linking in, daily page views per visitor, bounce rate, average daily time per visitor. note that most of these features were heavily unpopulated for a substantial part of the websites in our dataset, which could explain their mediocre performance. regardless, site popularity and engagement metrics are potentially very useful for bias and factuality prediction. in fact, as our results show, even their binarized versions are helpful, even on top a very strong system. ## conclusion and future work we studied the problem of media profiling with respect to their factuality of reporting and bias. motivated by homophily considerations, we built a graph of inter-media connections based on the audience overlap for the target pair of news media, and then we used graph neural networks to come up with representations for each news medium. we found that such representations, especially when augmented with alexa metrics and additional information sources from twitter, facebook, youtube, and wikipedia, are quite useful, yielding sizable improvemets over the state of the art on two standard datasets. in future work, we plan to experiment with other kinds of graph neural networks. we further want to integrate additional information sources, and to model factuality and bias jointly @xcite .