You Are What You Read: Inferring Personality From Consumed Textual Content (2023)

Abstract: In this work we use consumed text to infer Big-5 personality inventories using data we have collected from the social media platform Reddit. We test our model on two datasets, sampled from participants who consumed either fiction content ( N = 913 ) or news content ( N = 213 ). We show that state-of-the-art models from a similar task using authored text do not translate well to this task, with average correlations of r=.06 between the model’s predictions and ground-truth personality inventory dimensions. We propose an alternate method of generating average personality labels for each piece of text consumed, under which our model achieves correlations as high as r=.34 when predicting personality from the text being read.

## introduction while authored text has previously been used for personality prediction @xcite , consumed text (the linguistic attributes of the text that people choose to read) has received no such attention. it is unclear if consumed text can be indicative of personality. methods used in similar tasks may also not transfer to this domain, as a consumed piece of text is not unique to a single reader. however, predicting reader personalities may help understand and reduce the impact of psychological micro-targeting, particularly in the domain of political advertising. facebook has a psychological micro-targeting patent registered @xcite , and previous work indicates personality targeted messages increase desirable outcomes for advertisers @xcite . as recent work has shown that targeted political advertising online has been more more effective than traditional methods @xcite @xcite , our work aims to reverse engineer the process of such psychological targeting, with the intention of developing countermea-sures to remove or reduce the impact of this targeting. here we first demonstrate that consumed text can be used to infer personality. this is significant, as we show that personality prediction of content consumers is possible even where there are not structural connections to known cultural touchpoints (which has been demonstrated previously by, e.g., @xcite ). prediction of consumer personality from consumed text is highly tranferrable, being in principle applicable to any platform where users might read text. in the future we aim to develop tools for users that would flag articles or text that our model predicts could be congruent with their personality inventory. in this paper we sample ∼1,100 participants from the social media website reddit, using their public data and provided personality inventories to show that consumed text can also be indicative of the consumer's personality. our models achieve pearson's @xmath0.3 between predicted personality dimension values and those provided using standard instruments. we also show that models that have achieved state-of-the-art performance when applied to produced text do not achieve suitable performance on consumed text. ## background in the field of psychology, constructs such as personality are quantified using validated tools. one such tool is a personality inventory, where the outcome is usually represented as a numerical value for multiple personality dimensions. one example of this is the big-5 model, which uses a questionnaire to capture people's personality along five dimensions @xcite . these scales enable measurement of personality, and in combination with access to large featurerich datasets from social media they have enabled attempts at estimating people's personality from their behaviour @xcite . machine learning has improved to the point where automated personality judgements can outperform humans at the same task @xcite . this has also been found with textual content: various studies have shown that text produced by a user can be used to estimate their personality @xcite . language models have played a large part in the improvement of performance in many downstream natural language tasks in recent years @xcite . the most recent development to have a substantial performance impact is attention @xcite , which enables word representations that are dynamically generated based on surrounding text (i.e. "bark" will have different representations for a "dog's bark" and "tree bark"). this has resulted in a new generation of attention-based language models that reported state-of-the-art-performance for multiple nlp tasks @xcite . these architectures are still being iterated on to improve performance @xcite . attention has also been useful for personality modelling in the domain of produced text. @xcite defines "message-level attention", which is based on the assumption that "not all documents are equally important". models using this form of attention take multiple produced messages from an author and weigh the importance of each message according to a learned attention mechanism, in order to predict that author's personality. @xcite represent the current state-ofthe-art performance for this task, while also providing some interpretability of the model via message weights. in this study we apply these message attention models to the domain of consumed text, alongside an alternative method that aims to predict the averaged personality profile of all known consumers of the article. we show that message attention models do not achieve desirable performance when applied to consumed text. evaluation under averaged personality labelling shows promising performance in comparison. our evaluation covers multiple datasets, spanning two different genres of text. we also trial the effectiveness of models predicting consumer personality using only article titles instead of the entire article. we find that the personalities of news readers are better predicted by our models than the consumers of fictional content, and predictions on the basis of news titles alone perform comparably to those informed by the content of the entire article. ## dataset two datasets are used in our experiments, both sourced from the social media website reddit between 2021-2022. participants were invited to participate in a survey and gave permission for us to link their public post and comment history to their personality inventories as assessed via a big-5 personality questionnaire (bfi-2) @xcite . we crawled the content of all posts our participants had commented upon, using commenting behaviour as an indication of text consumption. our data collection and retention procedures were overseen by the relevant institutional ethics board. our two datasets cover different domains of content. our news dataset contains news articles consumed by our participants from news-focused subreddits (communities dedicated to a specific topic), articles 19 , 609 4 , 000 including r/worldnews, r/politics and r/europe. our fiction dataset contains short fiction consumed by participants from subreddits devoted to sharing such content, such as r/writingprompts, r/shortstories, and r/nosleep. news subreddits were chosen based on activity (number of users, and posts) and the majority of articles posted being url submissions linking to news stories from external news sites. these subreddits are also moderated to remove unwanted content, such as spam or adverts. fiction subreddits were also chosen based on activity, along with ease of crawling for the text content posted there. text content is usually short stories which are submitted as a post, or in the case of r/writingprompts as top level comments. figure 1 shows the distribution of the big-5 personality traits as aggregated by fiction and news articles as per our method described above. personality labels at the article level show somewhat reduced variance compared to the per-user data (see appendix c), but are by no means uniform. the personality distributions of news and fiction consumers are quite similar, seeming to reflect a common reddit user personality type. ## results in this work we primarily seek to answer the following questions: ## discussion our experiments have tackled a novel personality prediction task using consumed rather than produced text to infer personality. we demonstrate that the structure of this task is important, showing a significant difference in performance between the state-of-the-art message attention model and average-labelling models. the message-level attention mechanism used by @xcite does not perform well on this problem, and our examination of attention weights suggests this is because the model rarely finds any consumed text to be especially informative with regard to a participant's personality. we also find that the source of the corpus also has an impact on performance, with models trained upon news text better suited to predicting personality. ## conclusions in this paper we have shown that personality can be inferred based on the text that a user has consumed. to our knowledge, this is the first work using consumed textual content to model personality that reaches comparable performances to produced content. the performance achieved by average-label modelling can be seen as a baseline for personality modelling using consumed text. message attention models do not achieve acceptable levels of performance when applied to the domain of consumed text. we show that this may be due to the weighting function giving no especial weight to any consumed text, in combination with the lack of unique textual content for each user, which gives confounding feedback to the model during training. we used three different datasets to train and evaluate our models: pieces of fiction, news articles, and news titles. personality is shown to be more reliably inferred from news content than fiction content. models trained upon news article titles, with less textual content, achieved similar performance to models trained upon whole news articles, which may reveal that a condensed set of features are most important for modelling personality. future work in this field should involve further investigation as to how message attention models may be adapted to this context, as well as establishing resources to enable new approaches to this problem in the form of a shared task. due to participant privacy concerns, our datasets cannot be released, which forms a hurdle to reproduction and development. a publicly available dataset would be beneficial, so new work can be evaluated on a standardised dataset. an ideal dataset would also provide access to more training samples, along with greater assurance that the textual content has been consumed by the users.