R ed D ust: a Large Reusable Dataset of R eddit User Traits (2020)

Abstract: Social media is a rich source of assertions about personal traits, such as “I am a doctor” or “my hobby is playing tennis”. Precisely identifying explicit assertions is difficult, though, because of the users’ highly varied vocabulary and language expressions. Identifying personal traits from implicit assertions like I’ve been at work treating patients all day is even more challenging. This paper presents RedDust, a large-scale annotated resource for user profiling for over 300k Reddit users across five attributes: profession, hobby, family status, age,and gender. We construct RedDust using a diverse set of high-precision patterns and demonstrate its use as a resource for developing learning models to deal with implicit assertions. RedDust consists of users’ personal traits, which are (attribute, value) pairs, along with users’ post ids, which may be used to retrieve the posts from a publicly available crawl or from the Reddit API. We discuss the construction of the resource and show interesting statistics and insights into the data. We also compare different classifiers, which can be learned from RedDust. To the best of our knowledge, RedDust is the first annotated language resource about Reddit users at large scale. We envision further use cases of RedDust for providing background knowledge about user traits, to enhance personalized search and recommendation as well as conversational agents.

## introduction reddit is a popular social media platform for discussing a wide range of topics. it is an important source of information for data analysis on social media as it provides rich structure, abundance of data and covers a broad range of topics. reddit is used by approximately 330 million users 1 with 2.8 million comments written each day 2 . alexa.com ranks it as the 21st most popular website worldwide. despite its popular and rich data, few have considered reddit as a source of data about users' personal traits like their professions and hobbies. prior work has focused on reddit as a source of demographic information, whereas we consider rich attributes like profession and hobbies in addition to demographic ones (age, gender, family status). such data has many applications, including personalizing healthcare @xcite , recommendations, search, and conversational agents. we address this gap by creating a labeled dataset of reddit users (including their posts and comments) that covers five user attributes: profession, hobby, family status, age, and gender. we leveraged three high-precision approaches to identify predicates and their object values in users' posts: user profiling in online communication: the popularity of social media and online forums brings about massive amounts of user-generated content that is freely accessible. this has opened many research opportunities on text analysis, in particular on automatically identifying latent demographic features of online users for personalized downstream applications such as personalized search or recommendation. such latent demographic attributes include age and gender @xcite @xcite @xcite @xcite @xcite , personality @xcite , regional origin @xcite , political orientation and ethnicity @xcite @xcite @xcite , as well as occupational class mapped to income @xcite . most prior works on automatically identifying users' latent attributes from online communication rely on classification over hand-crafted features such as word/character n-grams @xcite @xcite , linguistic inquiry and word count (liwc) @xcite categories @xcite ; preot ¸iuc-pietro and ungar, 2018), topic distributions @xcite @xcite and sentiment/emotion labels of words derived from existing emotion lexicon @xcite @xcite . the recently prominent neural network approaches have also been adopted to solve the task @xcite @xcite . dataset for user profiling: automatic methods, particularly supervised learning approaches, for identifying users' personal attributes require a collection of user-generated content labelled with personal attributes of interest. most of existing works mentioned above focus on user-generated content from twitter, with a few exceptions that explore facebook @xcite or reddit @xcite @xcite posts. data collection was mostly done via: manual annotation after a focused search with specific keywords or hashtags @xcite , public profile linked to twitter profile description @xcite , self-reports as part of an online survey @xcite @xcite @xcite , or pattern-based extraction approach (e.g., (i|i) (am|'m|was) born in + number ) on user profile description or user posts @xcite @xcite . several works @xcite ) made use of labelled datasets published within the shared task on author profiling organized by the clef pan lab @xcite . there has been less effort on identifying demographic attributes of reddit users compared with the body of work that exists for twitter users, although reddit posts have been exploited for other purposes such as determining users' personality @xcite , mental health condition @xcite , domestic abuse @xcite and irony detection @xcite , among others. @xcite investigate how the topic of subreddit influences the gender ratio within it. the study was performed on 100 subreddits grouped by interest, gender information about the users were collected by guessing it from their usernames, which is arguably a low-precision strategy. smaller scale red-dit datasets exist for gender, age and location attributes @xcite , which are unfortunately not publicly available. as far as we know, we are the first to consider hobby as a personal attribute of interest to be identified from online communication. ## background reddit foot_0 is a social news website and forum where registered members can submit content including links, text posts, and images, which are then voted up or down by other members. before elaborating on the creation of our dataset derived from reddit posts, we describe several concepts on reddit that are relevant for the data collection process. posts and comments: discussions on reddit are organized in threads, which are initiated by an original post and may contain comments replying to the post and to other comment. this creates a hierarchical structure that resembles a conversation between users. both posts and comments can be a textual content, a link with anchor text or images. subreddits: reddit is organized into subreddits, which are fora that focus on specific topics. those can be split by interest (sports, politics, etc), by country or community, type of content (text, gifs, videos), and so on. subreddits have their own rules, but any registered user can create them. by convention, subreddits are prefixed with /r. for example, users discuss hockey in the /r/hockey subreddit. flairs: flair is user or post metadata that is a unique feature of reddit. flair is generally a small image with a short text description that is attached to a post or a username. flairs can be defined differently for specific purposes by each subreddit. for example, in /r/travel subreddit they may indicate the country of the user, gender in /r/askmen and /r/askwomen or users' favorite teams in /r/hockey. flairs for posts can be useful to filter and search for a particular content. ## dataset creation reddust is a dataset containing a collection of reddit users, or redditors. each redditor in the dataset is associated with posts and comments they produce, and personal attributes resulting from pattern-based mining approach as well as flairs. in this work, we consider five personal attributes including gender, age, family status, profession and hobby. the dataset is created from the openly published reddit dump 4 , which spans between 2006 and 2018. there are several criteria on which users and posts/comments that are included in reddust, i.e., users who posted between 10 and 100 posts/comments, and posts/comments containing between 20 and 100 terms after filtering. we filtered out hyperlinks and user mentions (i.e., @nickname) from the original posts/comments. some subreddits are likely to contain many false positives, such as those concerned with video games or role playing. this leads to personal assertions talking about the users' projected persona in a particular context (e.g., i am a priest looking for a guild). to mitigate this source of false positives, we blacklisted subreddits about gaming, fantasy, and virtual reality from the top 500 subreddits sorted by number of unique users. posts made to blacklisted subreddits were discarded. similarly, we do discard posts that contain quotations in order to reduce the possibility of the user referring to a third person (... and he shouted "hands on the counter, i am a cop!"). for attributes that usually have a unique value (i.e., gender, age and family status) we also exclude users who state multiple different values to avoid introducing false positives. meanwhile, we allow each user to have multiple attribute values for profession and hobby. the age of a given user is calculated relative to his or her age when writing the most recent comment. in the following subsections we discuss particular techniques used to extract values or labels for each personal attribute. ## data statistics and analysis in this section we present the quantitative and qualitative analysis of the reddust resource. we present in in figure 1 , we plot the chart of the user count per each post count within the admissible 20-100. from this plot we conclude that the users in our dataset tend to have a small number of posts. 7 https://pkb.mpi-inf.mpg.de/reddust as previously mentioned, the profession attribute has 832 possible values in total. however, just about 5% of professions have over 2000 users, which we consider to be a sufficient number to train predictive models for automatically labeling users with personal attributes (see section 6). ## use cases in this section we discuss potential applications of red-dust. for example, reddust may be used both as a tool for the analysis and creation of trait-specific lexicons as well as a labeled dataset for training predictive models, among other use cases. 6.1 trait-specific characteristic terms reddust may be used to build lexicons specific to different personal user traits. here we use "traits" to refer to attribute-value pairs, such as "profession-doctor" or "hobby-swimming". such lexicon can later be used for personalization in many downstream applications (e.g., targeted advertisements based on the trait-specific search terms). for that purpose, we counted the number of word occurrences for each attribute value, excluding common and rare words (occurring more than 15 times and less than 4 times, respectively). we also removed proper names, mentions of trait attribute ("profession") and values ("writer") and phrases. in ## conclusion in this work we described reddust, a large semantic resource about reddit users' personal traits. over 350,000 users were labeled with values for five personal attributes (gender, age, family status, profession and hobby) by using a combination of high-precision patterns to capture personal assertions and reddit metadata. reddust's utility was illustrated in two use cases leveraging the dataset to build trait-specific lexicons and to predict users' traits based on only implicit assertions.