Predicting Controversial News Using Facebook Reactions
Authors: Angelo Basile, Tommaso Caselli, Malvina Nissim (University of Malta, University of Groningen, VU Amsterdam, 2017)
Topic: Sentiment, Emotion, Irony, Hate

Abstract
Different events and their reception in different reader communities may give rise to controversy. We propose a distant supervised entropy-based model that uses Facebook reactions as proxies for predicting news controversy. We prove the validity of this approach by running within- and across-source experiments, where different news sources are conceived to approximately correspond to different reader communities. Contextually, we also present and share an automatically generated corpus for controversy prediction in Italian.

Introduction
The explosion of social media (e.g. Facebook, Twitter, Disqus, Reddit, Wikipedia, among others) and the increased interactions with readers-users that traditional newspapers embraced, have transformed the Web in a huge agora, where news are shared, opinions are exchanged, and debates arise. On many topics, such as climate change, abortion, vaccination, among others, people strongly disagree. Following the work by AUTHOR, we call controversies situations where, even after lengthy interactions, opinions of the involved participants tend to remain unchanged and become more and more polarized towards extreme values. Modeling and understanding controversies may be useful in many situations. Journalists and news agencies may pay additional attention in the framing of a certain news, government officials and policy makers may be more aware of the issues involved in specific laws, social media managers might be more careful, i.e. monitor controversial content, in order to avoid the spreading of hate speech, and the general public may benefit as well thanks to a reduction of the ``filter bubble'' effect AUTHOR. Recently, computational approaches on controversy detection have been developed with varying degrees of success AUTHOR. Works in the areas of Sentiment Analysis AUTHOR, Emotion Detection AUTHOR, and Stance Detection AUTHOR are, on the other hand, only partially related, as they focus on predicting/classifying the content of a message with respect to specific categories, such as ``positive'', ``negative'', ``neutral'', or ``joy'', ``sadness'' (among others), or as ``being in favour'' or ``being against''. They may be seen as necessary but not sufficient tools for detecting/predicting controversy AUTHOR. The main contribution of this work is two-fold: i.) we propose a distant supervised entropy-based model to predict controversial news; and ii.) we present and share an automatically created corpus to train and test models for controversy detection. At this stage of development, we focused only on Italian, although the methods are completely language independent and can be reproduced for any language for which news are available on Facebook. The remainder of the paper is structured as follows: Section~sec:method illustrates the methods used to collect the data and develop the entropy-based model. Section~sec:experiments reports on the experiments and results both in a within- and across-source setting. Finally, Section~sec:conclusion draws conclusions and outlines future research. Data and code are made available at .

Conclusion
This paper presents a simple regression model to predict the entropy of a post's reactions based on the Facebook reaction feature. We take this measure as a proxy to predict the controversy of news, where the higher the entropy (indicated by highly mixed reactions), the bigger the controversy. We run experiments both within and across communities, exemplified by the Facebook pages of specific newspapers. As a by-product, we have also automatically generated a first reference corpus for controversy prediction in Italian. The results are promising, given that our model beats the baseline in almost all cases in cross-validation of same source data (see Table~tab:cross-val), and in the large majority of cases when applied cross-sources (see Table~tab:cross-val2). At this stage of development, we observed that coarse-grained sentiment values are not useful, although this may depend on the quality of the lexicon employed. Test and training on openly biased datasets (e.g. Il Giornale_T_R_A_I_N - Il Manifesto_T_E_S_T, and vice-versa) results in the lowest entropy, suggesting perspective bias in the different community. The approach we have developed is based on discrete linguistically motivated features. This has an impact in the learned model as it is not able to generalise enough when dealing with low-frequency features and unseen data in the test set. To alleviate this issue, we are planning to model the post representations by using word embeddings. We are planning to expand the model to account for perspective bias in different communities. News from different sources may be aggregated per event type, for example via the EventRegistry API}, allowing to explore entropy (and polarisation of reactions) on exactly the same event instance. A first step in this direction would be to detect and match Named Entities to approximately identify similar events. At the reaction-level, the obvious next step is to explore and experiment with clusters of reactions (for instance, positive (LIKE, LOVE, AHAH), negative (ANGRY, SAD), or ambiguous (WOW)), instead of treating them all as single and distinct indicators. Another follow-up is to extend this work to other social media data, such as Twitter. Twitter does not allow for nuances in reactions in the same way that Facebook does, as only one kind of ``like'' is provided. However, the substantial use of hashtags and emojis might offer alternative proxies to capture a variety of reactions. There is plenty of work on the usefulness of leveraging hashtags as reaction proxies both at a coarse and finer level AUTHOR, but this information, to the best of our knowledge, has not been used to predict likelihood of controversy.