Evons: A Dataset for Fake and Real News Virality Analysis and Prediction (2022)

Abstract: We present a novel collection of news articles originating from fake and real news media sources for the analysis and prediction of news virality. Unlike existing fake news datasets which either contain claims, or news article headline and body, in this collection each article is supported with a Facebook engagement count which we consider as an indicator of the article virality. In addition we also provide the article description and thumbnail image with which the article was shared on Facebook. These images were automatically annotated with object tags and color attributes. Using cloud based vision analysis tools, thumbnail images were also analyzed for faces and detected faces were annotated with facial attributes. We empirically investigate the use of this collection on an example task of article virality prediction.

## introduction fake news articles are widely spread across social media platforms such as facebook and twitter. this is mainly due to the fact that social media is gradually becoming the main source of news consumption @xcite . due to the sharing features that these platforms offer, fake news propagate rapidly and their effects resonate and persist across many users @xcite . the wide spread of fake news in social media has lead to the development of automatic fake news detection approaches @xcite @xcite , to name a few. developing fake news detection models require annotated collections of fake and real news articles. most prior work on the creation and annotation of such collections has focused on this task. significant number of these collections contain claims fact-checked for veracity @xcite . a recent survey of such collections is provided in @xcite . on the other hand there exist collections of fake news articles that contain article headline and body text @xcite @xcite . given that these and other existing fake news collections were developed mainly for fake news detection they can't be used for analysing and predicting fake news virality which is the set of tasks of our focus. recently, @xcite created fakenewsnet, a collection of ∼24k news articles labeled as fake or real using the fact-checking websites politifact @xcite and gossip cop @xcite . articles in this collection are annotated with social engagement information obtained through the twitter search api. however this collection doesn't include thumbnail images and article descriptions which, along with the headlines, are the only sources of information readers are exposed to on social media platforms regardless of their choices whether to click the link of the shared article or not. to address this drawback we present evons -a collection of news articles originating from fake and real news media sources where each article has the thumbnail image and description with which it was shared on facebook. we use the article engagement count on facebook as an implicit indicator of the article virality. given that fake news writers profit from advertising revenue rather than subscription fees, the body text of fake news articles (which are only shown after clicking the link) are known to be repetitive and lacking in informational value @xcite . therefore we believe that these two article components are important for social media sharing. thumbnail images are annotated with content tags and color attributes while detected faces are annotated with facial attributes. the evons collection is accessible through https: //github.com/krstovski/evons . we showcase the use of this collection on the task of article virality prediction which we consider as one example task that could be created wit this dataset. ## collection construction the evons collection contains 92,969 news articles from fake and real news media sources published in the period between january 2016 and december 2017. we selected this time period to reflect on the 2016 presidential election which many believed that fake news had a significant impact on. across both media sources we focused on news articles originating from the same news sections therefore covering similar or the same set of topics. we also don't consider any article author related information. the set of fake news sources was created using information from 3 independent lists of fake news websites that were developed through human curation. it contains only fake news sources that were cross-referenced by at least 2 of the 3 lists. we follow the most widely used definition of fake news as "intentionally and verifiably false, and could mislead readers" by @xcite and exclude satire and parody websites. we used the "questionable sources" list from media bias fact check (mbfc) (media bias/fact check, 2019) which includes sources with extreme bias, propaganda and conspiracies, and fake news. we filtered the websites to retain only those that are explicitly annotated as "some fake news" or "fake news", indicating that the source deliberately publishes hoaxes and/or disinformation. our second list is the "politifact's fake news almanac" @xcite . this list was created in partnership with facebook and includes "fake news" websites which were found to contain deliberately false or fake stories that have appeared in people's news feeds on facebook. the third list is from the "bs detector" collection. this is a list of "unreliable or otherwise questionable sources" curated by professionals @xcite . after cross-referencing, we obtained 16 fake news sources that appeared in at least 2 of the lists. we then removed sources that were republishing news content from other sources and websites that started publishing after the 2016 elections. our final list contains the following 6 fake news media sources: american freedom fighters (aff), barracuda brigade (bb4sp), madworldnews (mwn), puppet string news (psn), usa supreme (usas), and yournewswire (ynw). the set of real news sources was obtained from the readily available "all the news 2.0" dataset @xcite which consists of 18 american mainstream sources. we focused on sources that had "high" or "very high" scores in factual reporting and "very slight" or "neutral" political biases according to mbfc. there were 5 such sources in this dataset: the guardian, national public radio (npr), new york times (nyt), reuters, and washington post (wp). we use articles published in the same time period as our fake media set. in we used the webpreview foot_0 package for extracting thumbnail images. these images come from the thumbnails that are carefully curated by the news producers. they decide what title, description, and thumbnail image would be the most effective in achieving their goal, whether it is to best represent the content or simply attract the most engagement for larger advertising revenue. with this package we also extract article description which is the text that appears as preview when the article is shared. all articles contain a thumbnail image except for usas and bb4sp were 0.1% and 11.1% of the articles don't have thumbnails. thumbnail images are either a picture or a logo of the news media source. ## example task we use the task of predicting article virality as an example task (out of many different tasks) that could be constructed using the evons ## conclusion we presented evons -a collection of news articles originating from fake and real media sources where articles are annotated with a facebook engagement count, thumbnail image and article description. thumbnails are automatically annotated with object tags, color and facial attributes. we demoed the collection use on an article virality prediction task and established baselines using 6 models. in the future we plan to use evons to explore various approaches for selection of image features and combination with text that would further help improve accuracy on this task.