You Are What You Talk About: Inducing Evaluative Topics for Personality Analysis (2022)

Abstract: Expressing attitude or stance toward entities and concepts is an integral part of human behavior and personality. Recently, evaluative language data has become more accessible with social media’s rapid growth, enabling large-scale opinion analysis. However, surprisingly little research examines the relationship between personality and evaluative language. To bridge this gap, we introduce the notion of evaluative topics, obtained by applying topic models to pre-filtered evaluative text from social media. We then link evaluative topics to individual text authors to build their evaluative profiles. We apply evaluative profiling to Reddit comments labeled with personality scores and conduct an exploratory study on the relationship between evaluative topics and Big Five personality facets, aiming for a more interpretable, facet-level analysis. Finally, we validate our approach by observing correlations consistent with prior research in personality psychology.

## introduction sharing opinions has always been rooted in people's daily habits, but nowadays, it has scaled up with a plethora of user-generated texts on social media @xcite . opinions, as dispositions toward specific entities, can be the key to understanding human behavior. oftentimes, there is a need to predict sentiment or stance toward a particular target of interest, making opinion analysis useful in marketing research, social and political sciences, and recommender systems, to name a few. moreover, analyzing how we express our opinions, i.e., the linguistic aspect and the psychological disposition of the opinion holder, is crucial for various downstream tasks, such as personality analysis and mental health prediction. in public communication, among other situations, people use linguistic resources known as evaluative language @xcite to express their attitudes. @xcite define i'm int o a lot of t hings, m usic and gam ing m ost ly. oban st ar racers is t he funniest anim e ever, im o. i'm really picky wit h m usic, so t here's lit erally not a single genre i com plet ely enjoy. dat ing t his enfp girl and we're am azing. re l a t i o n sh i p pe r so n a l i t y ga m i n g figure 1: an illustrative example of mapping user statements to evaluative topics, i.e., topics induced from evaluative texts. based on the prevalence of such topics, we construct an evaluative profile of each author. we then measure the correlation between each author profile and personality facets, such as artistic interests, assertiveness, intellect, and imagination. evaluation as a cover term for unified phenomena of certainty and goodness/desirability, and evaluative language includes every expression of the speaker or writer's attitude or stance towards, or feelings about the entities or propositions. as evaluation is associated with psychological disposition @xcite , evaluative language can help uncover people's distinguishing qualities, i.e., recurrent behavioral, cognitive, or affective tendencies. such behavioral differences resulting from interaction with the environment fall within the purview of personality psychology, where they are typically examined as personality traits -sets of characteristic patterns of behavior, cognition, and feelings stable over time and across situations @xcite . moreover, personality traits have been shown to correlate with occupational preferences, political or religious tendencies, intelligence, personal interests, and opinions @xcite @xcite . to the best of our knowledge, evaluative language has not yet been studied in the context of personality analysis on social media. this can perhaps be attributed to methodological challenges. one of the difficulties in analyzing evaluative language is separating the signal from the noise. although evaluation in text is pervasive, it remains a challenge to automatically distinguish evaluative from non-evaluative text. evaluative expressions come in all sort of flavors, with varying amounts of explicit lexical stance markers. another difficulty is choosing the right level of abstraction both for personality and evaluative language. if the evaluation targets are too specific, it is difficult to relate them to personality. nevertheless, we can choose the appropriate operationalization of the personality in terms of its breadth, thus aligning it with the abstraction level of the evaluative targets. in this paper, we study how evaluative language on social media can be used for personality analysis. more specifically, our study aims to explore how the topics people talk about are related to their personality, as illustrated in figure 1 . to this end, we introduce the notion of an evaluative topic, which is a topic constructed from evaluative text. to address the signal-noise problem in detecting evaluative text, we develop an iterative filtering technique based on detecting evaluative expressions and paraphrase mining. we then leverage topic models to induce evaluative topics from prefiltered evaluative text. aiming to maximize coherence and diversity, we test several topic model families, including probabilistic, decompositional, and neural topic models. finally, we link evaluative topics to individual text authors to construct authors' evaluative profiles. in the experimental part, we investigate the relationship between evaluation and personality. our focus is on the big five model of personality @xcite , which includes broad personality traits and their more specific (and arguably more interpretable) aspects called facets. we present a correlation study of evaluative topics and big five facets on comments from reddit, one of the most popular discussion websites, which covers an astoundingly diverse range of topics while preserving user anonymity. we find correlations between evaluation and personality that are consistent with prior research in personality psychology. finally, we provide empirical evidence that evaluative statements have a stronger association with personality than non-evaluative expressions. in summary, our contribution is threefold: we (1) develop an approach for evaluative author profiling based on topic models with evaluative filtering, (2) apply evaluative profiling on reddit comments, and (3) study how evaluative author profiles of reddit users correlate with big five personality facets. the rest of the paper is structured as follows. section 2 lays out the background and related work. we describe our technique for filtering evaluative text along with topic modeling in section 3. we conclude the section by presenting the idea of evaluative author profiles. we describe our experiments and results in section 4. finally, section 5 overlays the main conclusions and takeaways of the paper. target awareness. given the nature of expressing opinions, evaluation is directed toward a particular target. among others, sentiment-based approaches to evaluation have been prevalent, which led to the establishment of aspect-based sentiment analysis (absa), an nlp task that combines aspect extraction with sentiment classification @xcite . recent research in absa has evolved from traditional perspectives, e.g., using conditional random fields for aspect extraction as a sequence labeling task @xcite , to neural-based approaches @xcite @xcite . however, absa systems are better suited for smaller domains. besides absa, there have been target-based approaches in stance classification @xcite and topic-dependent argument classification @xcite , among others. as we are trying to cover diverse domains, we opt for a topic-level approach to targets. nlp and personality. nlp has been used extensively for personality analysis, starting with essay analysis @xcite and followed by social network research @xcite . the more recent studies were mainly conducted with facebook data. for example, @xcite used trait-level factor analysis for the big 5 personality system on facebook. however, as far as we know, there is no research that leverages nlp to analyze personality in the context of evaluative language. moreover, there is little research in nlp that covers facet-level personality analysis. ## methodology we propose a three-step process from raw text to evaluative profiles. we start with evaluative filtering to extract evaluative text and then construct topics from the filtered dataset. finally, we use the developed evaluative topics to create the corresponding evaluative profiles. ## experiments our investigation of the association between evaluative language and personality proceeded in two steps. we first built evaluative author profiles from reddit comments from pandora, as described in the previous section. to build evaluative topics, we used the ctm model on the filtered evaluative sentences, as ctm yielded the best result in terms of coherence and diversity scores. we induced 20 evaluative topics, as per the results of the optimization process. in the next step, we carried out a correlation study on the relationship of big five personality facets and evaluative topics. we adopted the revised neo personality inventory (neo pi-r), designed to measure the big five personality traits: openness to experience, conscientiousness, extraversion, agreeableness, and neuroticism @xcite . in the first part of the analysis, we examined individual correlations between particular topics and facets. in the second part, we used canonical correlation analysis (cca) to explore common associations between the entire set of topics on the one side and the set of all big five facets on the other. we chose to focus on personality facets rather than personality traits, hypothesizing that the abstraction level of facets aligns better with the granularity of induced topics. in that sense, the choice of facets supports the brunswik symmetry principle, which stipulates that analyzed constructs (in our case, personality and topics) must have similar levels of generality (wittmann, 2012). ## conclusion the relationship between evaluative language and personality has been understudied in nlp research. we aim to fill this gap by proposing evaluative author profiling -linking text of authors to topics obtained from text filtered for evaluative language. we applied evaluative profiling on a dataset of reddit comments with self-reported big 5 personality facet scores. using canonical correlation analysis, we showed that the facets within the same trait have stronger associations in the canonical space. we found that evaluative topics have moderate correlations with big 5 facets. moreover, we corroborate the hypothesis that evaluative expressions hold greater informational value for personality analysis than unfiltered texts. additionally, we showed that non-evaluative text has a much weaker association with big 5 facets compared to evaluative text. finally, we observed correlations consistent with previous research in personality psychology. we believe that our study can contribute to a better understanding of evaluative language on social media and how it relates to personality traits.