Words are the Window to the Soul: Language-based User Representations for Fake News Detection (2020)

Abstract: Cognitive and social traits of individuals are reflected in language use. Moreover, individuals who are prone to spread fake news online often share common traits. Building on these ideas, we introduce a model that creates representations of individuals on social media based only on the language they produce, and use them to detect fake news. We show that language-based user representations are beneficial for this task. We also present an extended analysis of the language of fake news spreaders, showing that its main features are mostly domain independent and consistent across two English datasets. Finally, we exploit the relation between language use and connections in the social graph to assess the presence of the Echo Chamber effect in our data.

## introduction fake news have become a problem of paramount relevance in our society, due to their large diffusion in public discourse, especially on social media, and their alarming effects on our lives @xcite . several works show that fake news played a role in major events such as the us presidential elections @xcite , stock market trends @xcite , and the coronavirus disease outbreak @xcite . in nlp a considerable amount of work has been dedicated to fake news detection, i.e., the task of classifying a news as either real or fake -see @xcite , @xcite and @xcite for overviews. while initial work focused uniquely on the textual content of the news @xcite , subsequent research has considered also the social context in which news are consumed, characterizing, in particular, the users who spread news in social media. in line with the results reported in other classification tasks of user-generated texts @xcite , several studies show that leveraging user representations, together with news' ones, leads to improvements in fake news detection. in these studies, user representations are usually computed using informative but costly features, such as manually assigned credibility scores @xcite . other studies, which leverage largely available but scarcely informative features (e.g., connections on social media), report less encouraging results @xcite . our work also focuses on users. we build on psychological studies that show that some people are more prone than others to spread fake news, and that these people usually share a set of cognitive and social factors, such as personality traits, beliefs and ideology @xcite . also, we rely on studies showing a relation between these factors and language use, both in psychology and linguistics @xcite and in nlp @xcite . we therefore propose to leverage user-generated language, an abundant resource in social media, to create user representations based solely on users' language production. we expect, in this way, to indirectly capture the factors characterizing people who spread fake news. we implement a model for fake news detection which jointly models news and user-generated texts. we use convolutional neural networks (cnns), which were shown to perform well on text classification tasks @xcite and are highly interpretable @xcite , i.e., they allow us to extract the informative linguistic features of the input texts. we test our model on two public english datasets for fake news detection based on twitter data, both including news and, for each news, the users who spread them on twitter. we leverage two kinds of user-generated language, i.e., past tweets and self-description. in line with our expectations, model performance improves when language-based user representations are coupled with news representations, compared to when only the latter are used. moreover, the model achieves high results when leveraging user-generated texts only to perform the task. we use the linguistic features returned by the model to analyze the language of fake news spreaders, showing that it has distinctive features related to both content, e.g., the large usage of words related to emotions and topics such as family and religion, and style, e.g., a peculiar usage of punctuation. importantly, these features are largely independent from the domain of the dataset, and stable across datasets. moreover, we find that the two kinds of user-generated language we consider provide partially overlapping information, but with some relevant differences. finally, we consider the relation between the language produced by the users and their connections in the social graph. in particular, we investigate the echo chamber effect, i.e, the situation in which the ideas expressed by a user are reinforced by their connections @xcite . in previous nlp work, the effect has been studied by observing whether users connected in the social graph post the same content, usually defined as a link to a web page from a manually compiled list @xcite . we propose to define the content produced by the users based on their linguistic production, and to compute the echo chamber effect as a function of the similarity between the content of connected users and their distance in the social graph. by applying our methodology, we show that the echo chamber effect is at play, to different extent, in both the datasets under scrutiny. modelling user-generated data requires careful consideration of the possible ethical aspects related to the treatment of such data. we provide an ethics statement in appendix a with details on how we have dealt with these aspects. ## related work several studies on fake news detection focus uniquely on the text of the news @xcite @xcite . despite some positive results, this approach is inherently undermined by the fact that fake news are often written in such a way as to look like real news. more recently, researchers have explored the possibility to leverage social information together with the one derived from news texts. some works focus on the patterns of propagation of fake news in social networks. @xcite show that, compared to real news, fake news have deeper propagation trees, spread faster and reach a wider audience, while @xcite show that fake news originate from users with a few followers, and are spread by influential people. a parallel line of research considers the users who spread fake news. some works focus on the detection of non-human agents (bots) involved in the spreading process @xcite , while others model the characteristics of human spreaders, as we do in this paper. @xcite and @xcite represent users with simple features such as longevity on twitter and following/friends relations, and show that these features have limited predictive power. @xcite strube (2018), long et al. (2017) and reis et al. (2019) use more informative features, such as users' political party affiliation, job and credibility scores. while leading to improvements on the task, features of this kind are usually either hard to retrieve or have to be manually defined, which hinders the possibility to scale the methodology to large sets of unseen users. @xcite and @xcite rely on manually annotated lists of news providers, thus presenting a similar scalability problem. finally, @xcite represent users by mixing different kinds of information, e.g., previous tweets, location, and profile image. this approach shares with ours the usage of the previous tweets of a user (as will be explained in section 3.2). however, to our knowledge, we are the first to create user representations based uniquely on users' linguistic production. previous nlp work showed the presence of the echo chamber effect (ece) on social media, especially in relation to political discourse @xcite . the majority of the studies implement a similar approach, whereby the ece is said to exist if users which are connected in the social graph post the same content. usually, the content considered in these studies is a link to a web page from an annotated list. for example, del @xcite investigate the relation between echo chambers and spread of conspiracy theories by observing users that share links to pages promoting this kind of theories. @xcite apply a similar approach to the analysis of rumours spread, while other studies adopt it to investigate echo chambers in relation to political polarization, in which case, links are labelled with political affiliation @xcite @xcite . we adopt the same approach but, crucially, we define the shared content based on the linguistic production of the users. ## model we implement a model which takes as input a news n and the set @xmath0, u 2 , ..., u i } of texts produced by the users that spread n, and classifies the news as either fake or real. the model consists of two modules, one for news and one for user-generated texts, both implemented using convolutional neural networks (cnns). the two modules can be used in parallel or independently (see section 5). the news module takes as input n and computes a vector @xmath1, where d is equal to the number of filters of the cnn (see below). the users module takes as input u and returns a vector @xmath2, which is the weighted sum of the representations computed for user-generated texts in u . 4 vectors n and u are weighted by a gating system which controls for their contribution to the prediction, and then concatenated. the resulting vector is fed into a one-layer linear classifier @xmath3, where 2 is the number of output classes (real and fake), which returns the logits vector @xmath4, on which softmax is computed. 5 extracting linguistic features from cnns recently, model interpretability has gained much traction in nlp, and an increasing number of studies have focused on understanding the inner-workings and the representations created by neural models @xcite . inspired by this line of work, and, in particular, by the analysis of cnns for text classification by @xcite , we inspect our model in order to extract the linguistic features it leverages for the final prediction, which we use for our analysis (see section 7). we describe below how we extract the relevant linguistic features from the model. a cnn consists of one or more convolutional layers, and each layer includes a number of filters (or kernels). filters are small matrices of learnable parameters which activate (i.e., return an activation value) on the n-grams in the input text which are relevant for the final prediction: the higher the activation value, the more important the n-gram is for the prediction. 6 as a first step, we collect all the relevant n-grams returned by the filters in the model. then, we assess which n-grams are relevant for the fake class, and which for the real class. we do this by considering the contribution of each filter to the two target classes, which is defined by the parameters in @xmath5. the contribution of filter f to the real and fake classes is determined, respectively, by parameters w f 0 and w f 1 : if the former is positive and the latter negative, we say that f contributes positively to the real class, and, therefore, the n-grams detected by f are relevant for that class. consequently, for n-gram x returned by the filter f with activation value v, we compute the importance of x for the class real as r @xmath6 ## experimental setup setups and baseline our goal is to assess the contribution of language-based user representations to the task of fake news detection. thus, for each dataset, we implement the following setups: we implement a support vector machine (svm) @xcite as a baseline. svms have been shown to achieve results which are comparable to those by neural-based models on text classification tasks @xcite , and we thus expect the model to be a strong baseline. hyperparameters for each setup we perform grid hyperparameter search on the validation set using early stopping with patience value 10. we experiment with values 10, 20 and 40 for the number of filters, and 0.0, 0.2, 0.4 and 0.6 for dropout. in all setups batch size is equal to 8, filters focus on uni-grams, bi-grams and tri-grams, and we use adam optimizer (kingma and ba, 2015) with learning rate of 0.001, β 1 = 0.9 and β 2 = 0.999. all the cnn modules have depth 1, and are initialized with 200-d glove embeddings pretrained on twitter @xcite . we train the svm baseline on uni-grams, bi-grams and tri-grams. when modelling user information, we concatenate the user-generated texts of the users spreading the target news. we use the rbf kernel, and perform grid hyperparameter search on the validation set. we explore values 1, 2, 5, 10, 15 and 30 for the hyperparameter c, and 1 e-05 , 1 e-04 , 1 e-03 , 1 e-02 , 1.0 for γ. for both cnn and svm models, we use binary f-score as optimization metric, and indicate the fake class as the target class. ## results we report the results of the fake news detection task in we then focus on the performance of the cnn in the different setups. first, we observe that results in the news setup are significantly higher than those in the user information setups. 10 this was expected, as classifying a news based on its text is presumably easier than by using only information about users who spread it. nevertheless, the results in the tl setup are surprisingly high, especially in pf, which indicates that the language used in timelines is highly informative. the results in the de setup, both in pf and gc, are lower than those in tl. the two setups, however, cannot be directly compared, as descriptions are not available for all users (see section 3.2). when we re-run the models in the user information setups keeping only users with both timeline and description, we observe no statistically significant differences between the results in the tl and de setups. lastly, we observe no significant improvement when we add descriptions to timelines (i.e., tl+de and n+tl+de do not improve over tl and n+tl, respectively). finally, in all the combined information setups the performance of the model significantly improves compared to the news setup -except for n+de in pf, for which the improvement is not statistically significant. when we substitute user vectors with random ones in the combined information setups, we observe no improvement over the news setup. overall the results confirm our initial hypothesis that leveraging user representations based only on the language produced by users is beneficial for the task of fake news detection. they also raise interesting questions related to what makes user-generated language informative, and which qualitative differences exist, if any, between timelines and descriptions. we address these questions in the next section. in this section, we analyse the language of news and of user-generated texts. we address two questions: (q1) which features of the language used by fake news spreaders are relevant for fake news detection, and how are they different from those of the language used by real news spreaders? (q2) which linguistic features do timelines and descriptions share, and which are different? also, which features do these two kinds of user-generated texts share with the language of news? ## linguistic analysis to answer these questions, we need to analyse the language used in timelines, descriptions, and news independently. we therefore consider, for both datasets, the models used at test time in tl, de and news. for each model, we extract the set of relevant n-grams, compute the r v and f v values for all of them, and sum the r v and f v of ngrams returned by more than one filter (see section 4). we use n-grams to analyse both style and content. regarding content, we analyse the topic of the n-grams, proper names and, for user-generated texts, hashtags. regarding style, we consider punctuation marks, all-caps, function words and, for user-generated texts, emojis. 11 we check to which category, if any, each n-gram belongs to (e.g., @xmath7. the category topic includes a list of topics (e.g., politics and war), and n-grams are assigned to these topics (e.g., missile, @xmath8. similarly, function words includes several parts of speech (pos), hence, e.g., me, @xmath9. we define the importance of each topic and pos for the two target classes by summing the r v and f v values of the n-grams they include. finally, to consider only the n-grams which are relevant for one of the target classes, we compute the difference between r v and f v for each n-gram, compute the mean µ and standard deviation σ of the differences, and keep only n-grams whose difference is larger than µ + σ. in figure 1 we show the analysis of the topics for the news setup in pf. red bars represent f v values, blue bars r v values: the higher the r v (f v ) value, the more the importance for the real (fake) class. for example, the topics negative emotions and death are important for fake news; government and politics for real news. usually, to a large positive f v value corresponds a large negative r v value, and vice versa. we apply our methodology to address the questions introduced at the beginning of this section. q1: the language of fake news spreaders in figure 2 we show the main categories of the language of fake news spreaders (red circles) and real news spreaders (blue circles) in pf (top) and gc (bottom). underlined categories refer to style, the others to content. a first observation is that very few categories are shared by the language of fake and real news spreaders (overlap between blue and red circles), and that those in common are mostly related to the domain of the dataset (e.g., law and politics in pf). the language of fake news spreaders shows many common categories across datasets (overlap between red circles), mostly related to content. in particular, fake news spreaders of both datasets extensively talk about emotions and topics such as friendship, family, animals and religion. interestingly, many of these topics are not directly related to the domain of either dataset. the most important proper names (e.g., jesus, lord, jehovah, trump) and hashtags (e.g., #usarmy, #trumptrain, #god, #prolife, #buildthewall) are again the same in the two datasets, and are highly related to the topics above. we observe some content-related categories which are not shared across datasets (non-overlapping areas in red circles), as they are related to the domain of the dataset (see q2). crossdataset consistency is even more evident for style: fake news spreaders steadily use specific punctuation marks (quotes, hyphen, slash, question and exclamation mark), function words (first person pronouns and prepositions), emojis and words in all-caps. the language of real news spreaders has different characteristics: many categories are dataset specific (non-overlapping areas in blue circles), while few of them are shared (overlap between blue circles). also, dataset specific categories have higher activation values and are related to the domain of the dataset. finally, no relevant style-related category is found for the language of real news spreaders. overall, the analysis shows that the language of fake news spreaders is clearly characterized by a set of linguistic features, related to both style and content. crucially, these features are largely domain-independent, and are consistently identified across datasets. this is in stark contrast with what is observed for the language of other users, which is more related to the domain of the dataset. these findings support the hypothesis that people who are more prone to spread fake news share a set of cognitive and sociological factors, which are mirrored in the features of the language they use. q2: the language of timelines, description and news we now analyse the relation between timelines, descriptions, and news. in figure 3 we show the relevant categories of timelines and descriptions for fake (a) and real (b) news spreaders, in both datasets. the plots include the same information displayed in figure 2 , but in greater detail. in the plots, solid-line boxes indicate the relevant categories for the news shared by fake/real news spread-ers in pf, dashed-line boxes the relevant categories for news shared by fake/real news spreaders in gc. for fake news spreaders, we highlight the following findings. first, the largest overlap (dotted ellipse) is observed between the descriptions across the two datasets. importantly, in this area we find the majority of categories which are not directly related to the domain of the datasets. second, in both datasets, timelines have some categories shared with descriptions (e.g., negative emotions and punctuation), plus other categories related to the semantic field of violence (e.g., crime and aggression), together with proper names. these timeline-specific categories are also the relevant ones for the fake news in pf (solid-line boxes) and in gc (dashed-line boxes). the relevance of similar categories across datasets is due to the fact that in both of them fake news are often built by mentioning a famous person (mainly trump in pf, a celebrity in gc) in relation to some negative event -a usual scheme in sensational news @xcite . in summary, all user-generated texts share some linguistic categories (central area of the plot), but it is in descriptions that we find the largest number of dataset-independent categories, related to both content and style, which characterize the language of fake news spreaders. conversely, timelines share more categories with the news spread by the users. these findings are in line with our expectations about the different nature of descriptions and timelines, as the former include more personal aspects of a user, while the latter are more related to the domain of the news they spread. furthermore, the limited similarity between the language of fake news spreaders and of the news they spread provides further evidence to the hypothesis that the language of fake news spreaders is largely shaped by sociological and cognitive factors, and mostly independent from the domain. for real news spreaders, there is a large overlap of content-related categories between timelines and descriptions within a given dataset (dotted ellipses), while no style-related category is relevant for either kind of text. differently from fake news spreaders, then, for real news spreaders descriptions and timelines do not present clear differences. also, in both datasets, the relevant categories of real news strongly reflect the topics discussed in user-generated texts (see solid-line boxes for pf, and dashed-line boxes for gc). we can thus conclude that a set of domain-related topics exists in each dataset, and that these topics are the relevant linguistic categories in timelines, description, and in news. in contrast, these texts do not share any characteristic related to style. after showing the informativeness of languagebased user representations, we now use them together with the information from the social graph to investigate the echo chamber effect (ece). we adopt the operational definition by @xcite , and say that the ece exists when users in a social graph mostly receive content from their connections which is similar to the content they produce. we introduce a methodology to define the content produced by a user based on their language use, and to compute the ece as a function of the content similarity of connected users and their distance in the social graph. ## echo chamber effect social graph to define the social graph we follow a common approach in the literature @xcite and create, for each dataset, a graph @xmath10, e) in which v is the set of users in the dataset, and e is the set of edges between them. an unweighted and undirected edge is instantiated between two users if one retweets the other. we retrieve information about retweets in users' timeline (see section 3.2). in order to make the social graph more connected, we also add as nodes users who are not in the dataset, but have been retweeted at least 20 times by users in the dataset. the resulting graph for pf includes 32k nodes and 1.6m edges (@xmath11.0031), the one for gc includes 109k nodes and 4.9m edges (@xmath12.0008). ## conclusion in this work we addressed the task of fake news detection, and showed that results can be improved by leveraging user representations based uniquely on the language the users produce. this improvement is due to the fact that the language used by fake news spreaders has specific and consistent features, which are captured by our model, and which we highlight in our analysis. language-based user representations also allowed us to show the presence of the echo chamber effect in both our datasets. our results offer empirical confirmation of previous findings regarding the relation between language use and cognitive and social factors, and they could foster further theoretical and computational work in this line of research. in particular, future computational work might address some of the limitations of the current study: for example, while we focus only on users spreading a single news, it would be interesting to model also users who spread multiple news, which are possibly both real and fake. similarly, it would be relevant to investigate the cold start problem, that is, the number of posts needed to create a reliable representation of the user, which is particularly important for newly registered users and for those who are not highly active. also, since the relation between language use and cognitive and social factors holds in every sphere of linguistic production, a natural extension of this work would be to apply the same methodology to other tasks involving user-generated language, such as, for example, suicidal prevention and mental disorders detection. finally, we hope the tools and insights provided in this study might be used to fight the diffusion of fake news, for example, by identifying and warning users who are vulnerable to them.