Manipulating the Perceived Personality Traits of Language Models
Authors: Graham McDougal Caron
Shashank Srivastava
Venue: EMNLP 2023 Findings (2023)
Keywords: nlp

Abstract: Psychology research has long explored aspects of human personality like extroversion, agreeableness and emotional stability, three of the personality traits that make up the 'Big Five'. Categorizations like the 'Big Five' are commonly used to assess and diagnose personality types. In this work, we explore whether text generated from large language models exhibits consistency in it's perceived 'Big Five' personality traits. For example, is a language model such as GPT2 likely to respond in a consistent way if asked to go out to a party?

We also show that when exposed to different types of contexts (such as personality descriptions, or answers to diagnostic questions about personality traits), language models such as BERT and GPT2 consistently identify and mirror personality markers in those contexts. This behavior illustrates an ability to be manipulated in a predictable way (with correlations up to 0.84 between intended and realized changes in personality traits), and frames them as tools for controlling personas in applications such as dialog systems. We contribute two data-sets of personality descriptions of humans subjects.

Manipulating the Perceived Personality Traits of Language Models
Graham Caron
Shashank Srivastava
UNC Chapel Hill
UNC Chapel Hill ssrivastava@cs. unc enc.edu
carongraham2 90 gmail. com
Abstract
CONTEXT Las dosann open
Psychology research has long explored aspects of human personality like extroversion, agree- ableness and emotional stability, three of the personality traits that make up the 'Big Five'. Categorizations like the 'Big Five' monly used to assess and diagnose personality types. In this work, we explore whether text generated from large language models exhibits consistency in it's perceived 'Big Five' person- ality traits. For example, such as GPT2 likely to respond in consistent way asked to go out party? We also show that when exposed to different types of contexts (such as personality descriptions, or answers to diagnostic questions about person- ality traits), language models such BERT and GPT2 consistently identify and mirror per- sonality markers in those contexts. This be- havior illustrates an ability to be manipulated a predictable way (with correlations up to 0.84 between intended and realized changes in personality traits), and frames them as tools for controlling personas applications such di- alog systems. We contribute two data-sets of personality descriptions of humans subjects.
HON JOOUT
1-1
Forwant Seying Mis
an exci bed to tzy it!
(A)
CONTECT CINTE CONTE
language model
Psychometric Questionnaire
Personality Trait Scores
(B)
CONTEXT
BALL
Gueslormire
(C)
Figure I: The top frame (Panel A) shows how a per- sonality trait (here, openness experience) might be expressed by a language model, and how the response can be modified by exposing the language model textual context. We psychometric questionnaires to evaluate perceived personality traits (Panel B), and show that they can be predictably manipulated with different types of contexts ($5,$6). We also evaluate the text generated from these contextualized language models (Panel C), and show that they reflect the same traits ($7).
1 Introduction
With the meteoric rise of Al systems based on lan- guage models, there is increasing need under- stand the 'personalities' of these models, As com- munication with AI systems increases, so does the tendency to anthropomorphize them (Salles et al., 2020; Mueller, 2020; Kuzminykh et al., 2020). Thus, even though language models encode prob- ability distributions over text and the tendency to assign cognitive abilities to them has been criti- cized (Bender and Koller, 2020), the way users perceive these systems have significant conse- quences. If the perceived personality traits of these models can be better understood, their behavior can be tailored for specific applications. For instance, when suggesting email auto auto-completes, it may be
useful for a model to mirror the personality of the user. In contrast, for a dialog agent in a clinical setting, it may be desirable manipulate model interacting with a depressed individual such that it does not reinforee depressive behavior. Addition- ally, since such models are subject biases in the text they are trained on, some may be prone to in- teract with users in hostile ways (Wolf et al., 2017). Manipulating these models can enable smoother and more amiable interactions with users.
Language- questionnaires have long been

---

used in psychological assessments for measuring personality traits in humans (John et al., 2008). We apply the same principle to language models, and investigate the personality traits of these mod- els through the text that they generate in response to such questions. As previously mentioned, we do not posit that these models have actual cogni- tive abilities, but are focused on exploring how their personality may be perceived through the lens of human psychology. Since language mod- els are subject to influence from the context they see (O' Connnor and Andreas, 2021), we also ex- plore how specific context could be used to ma- nipulate the perceived personality of the models without controlling sources of bias or the mod-
2 Related Work
In recent years, research has looked at multiple forms of biases (i.e., racial, gender) in language models (Bordia and Bowman, 2019; Huang et al., 2020; Abid et al., 2021). However, the issue of measuring and controlling for biases in personas of language models is under-explored. A substantial body of research has explored the ways language models can be used to predict personality traits of humans. Mehta et al. (2020) and Christian et al. (2021) apply language models to such personality prediction tasks. Similar to our methodology, Ar- gyle et al. (2022) contextualize large language mod- cls data-set of socio-economic back-stories to els themselves (i.e., pretraining, parameter fine- tuning). Figure shows example illustrating this approach.
show that they model socio-cultural attitudes in broad human populations, and Yang et al. (2021) develop new model designed to better detect per- sonalty in user based context, using question based answering. Most relevant to our work are con- temporaneous unpublished works by Karra et al. (2022), Miotto et al. (2022), and Jiang et al. (2022), who also explore aspects of personality in the lan- guage models themselves. However, these works substantially diverge from our approach and, along with Yang et al. (2021), do not attempt character- ize or manipulate the perceived personality of the models as we do.
Our analysis reveals that personality traits of language models are influenced by ambient con- text, and that this behavior can be manipulated in highly predictable way. In general, we observe high correlations (median Pearson correlation co- efficients of up 0.84 and 0.81 for BERT and GPT2) between the expected and observed changes in personality traits across different contexts. The models' affinity to be affected by context positions them as potential tools for characterizing person- ality traits in humans. In further experiments, we find that, when using context from self-reported text descriptions of human subjects, language mod- els can predict the subject's personality traits to a surprising degree (correlation up to 0.48 between predicted and actual human subject scores). We also confirm that the measured personality of a model reflects the personality seen in the text that the model generates. Together, these results frame language models as tools for identifying personal- ity traits and controlling personas in applications such as dialog systems. Our contributions are:
BBig Five" Preliminaries
The 'Big Five' is a seminal grouping of personality traits in psychological trait theory (Goldberg, 1990, 1993), and remains the most widely used taxonomy of personality traits (John and Srivastava, 1999; Pureur and Erder, 2016). These traits are:
Extroversion (E): People with a strong tendency in this trait are outgoing and energetic. They obtain energy from the company of others.
Agreeableness (A): People with strong ten- dency in this trait compassionate and kind. They value getting along with others.
We introduce the use of psychometric question- naires for probing the personalities of language models.
Conscientiousness (C): People with a strong ten- dency in this trait are goal focused and organized. They follow rules and plan their actions.
We demonstrate that the personality traits of com-
Emotional Stability (ES): People with a strong tendency in this trait are not anxious or impulsive. They experience negative emotions less easily. Openness to Experience (OE): People with a strong tendency in this trait are imaginative and creative. They are open to new ideas.
mon language models can be predictably con- trolled using textual contexts.
We contribute two data-sets: 1) self-reported per-
sonality descriptions of human subjects paired with their psychometric assessment data, 2) per-
sonality descriptions collated from Reddit. (See project Git repository)
While there are other personality groupings such as MBTI and the Enneagram (Bayne, 1997; Wag-

---

ner and Walker, 1983), we use the Big Five as the basis of our analyses, because the Big Five remains the most used taxonomy for personality assess- ment, and has been shown to be predictive of out- comes such as educational attainment (O'Connor and Paunonen, 2007), longevity (Masui ct al., 2006) and relationship satisfaction (White et al., 2004). Further, it is relatively natural to cast an assess- ment for language models.
candidate answer choice was evaluated, and the answer choice from the sentence with the highest probability was selected.
Finally, for each questionnaire (consisting of model responses to 50 questions), personality scores for each of the "Big Five' personality traits were calculated according to standard scoring procedure defined by the International Personality Item Pool (IPIP, 2022). Specifically, each of the five personality traits is associated with ten ques- tions the questionnaire. The numerical values associated with the response for these items were entered into a formula for the trait in which the item was assigned, leading to an overall integer score for each trait. To interpret model scores, we estimated the distribution of 'Big Five' personal- ity traits in the human population. For this, we used data from a large-scale survey of 'Big Five' personality scores in 1,015,000 individuals (Open- Psychometrics, 2018). In the following sections, we report model scores in percentile terms of these human population distributions. Statistics for the human distributions and details of the IPIP scoring procedure are included in Appendix B.
4 Experiment Design
We experiment with two language models, BERT- base (Devlin et al., 2019) and GPT2 (124M param- eters) (Radford et al., 2019), to answer questions from a standard 50-item 'Big Five' personality as- sessment (IPIP, 2022) !. Each item consists of statement beginning with the prefix "T" or "I am" (e.g., I am the life of the party). Acceptable an- swers lie on a 5-point Likert scale where the an- swer choices disagree, slightly disagree, neutral, slightly agree, and agree correspond to numerical scores of 2, 3, 4, and 5, respectively. To make the questionnaire more conducive to answering by language models, items were modified to sen- tence completion format. For instance, the item "I am the life of the party" was changed "I am [blank) the life of the party", where the model is ex- pected to select the answer choice that best fits the blank (see Appendix B for a complete list of items and their corresponding traits). To avoid complex- ity due to variable number of tokens, the answer choices were modified to the adverbs never, rarely. sometimes, often, and always, corresponding nu- merical scores 1, 2, 4, and 5, respectively. It is noteworthy that in this framing, an imbalance in the number of occurrences of each answer choice in the pretraining data might cause natural biases toward certain answer choices. However, while this factor might affect the absolute scores of the mod- els, this is unlikely to affect the consistent overall patterss of changes in scores that observe in our experiments by incorporating different contexts. For assessment with BERT, the answer choice with the highest probability in place of the masked blank token was selected as the response. For as- sessment with GPT2, the procedure was modified, since GPT2 is an autoregressive model, and hence not directly conducive to fill-in-the-bie thenblank tasks. In this case, the probability of the sentence with each
Base Model Trait Evaluation
Table 1 shows the results of the base personality assessment for GPT2 and BERT for each of the five traits terms of numeric values and correspond- ing human population percentiles. In the table, E stands for extroversion, A for agreeableness, C for conscientiousness, ES for emotional stability and OE for openness to experience. None of the base scores from BERT or GPT2, which we refer to Xiase diverge from the spread of the population distributions (TOST equivalence test at o 0.05). All scores were within 26 percentile points of the human population medians. This suggests that the pretraining data reflected the population dis- tribution of the personality markers to some extent. However, percentiles for BERT's openness to ex- perience (24) and GPT2's s agreeableness (25) are substantially lower and GPT2's conscientiousness (73) and emotional stability (71) are significantly higher than the population median.
Manipulating Personality Traits
In this section, we explore manipulating the base personality traits of language models. Our explo- ration focuses on using prefix contexts to influence the personas of language models. For example,
'EETT& & GPT2 were selected because of their availability as open-source, pretrained models.

---

Trait
Xbase
Pbase (%)
Trait
Context/Modifier
BERT
BERT
am never the of party.
27
A
never make people feel at ease. am always prepared.
25
C
ES
22
ES
never get stressed out easily.
OE
25
24
OE
never have a rich vocabulary.
GPT2
GPT2
E
21
54
E
never the life of the party.
A
24
25
A
I never have a soft heart.
C
29
73
C
never prepared. ES I always get stressed out easily. never have rich vocabulary.
ES
25
71
OF
28
39
OF
Table 1: Base model evaluation scores (Xbase) and per- centile (Phase) of these scores in the human population.
Table 2: List of context items modifiers (along with the direction of change) that caused the largest magni- tude of change, Aem, for each personality trait.
if we include a context where the first person is seen engage in extroverted behavior, the idea is that language models might pick up on such cues and modify their language generation (e.g., to generate language that also reflects extrovert behav- ior). We investigate using three types of context: (1) answers to personality assessment items, (2)
with -2 = never, -1 = rarely, = sometimes, =often and 2 = always. Because this experiment examines correlation between models scores and ratings, the magnitude of the modifier rating is bitrary, so long as the ratings increase linearly from never (strongest negative connotation) always (strongest positive connotation). Context items are given a context rating of -1 the item negatively affected the trait score based on the IPIP scoring procedure, and I otherwise. The context ratings are multiplied by the modifier ratings to get the Tem- This value represents the expected relative change in trait score (expected behavior) when the corresponding context/modiffer pair was used context.
descriptions of personality from Reddit, and (3)
self-reported personality descriptions from human users. In the following subsections, describe these experiments in detail.
6.1 Analysis With Assessment Item Context
To investigate whether the personality traits of mod- els can be manipulated predictably, the models are first evaluated on the 'Big Five' assessment ($4) with individual questionnaire items serving as con- text. When used as context, we refer to the answer choices modifiers and the items themselves as context items. For example, for extroversion, the context item "I am (blank) the life of the party" paired with the modifier always yields the context "I am always the life of the party", which precedes each extroversion questionnaire item.
Next, the differences, Aems between Xen and Xiase values are calculated and the Pearson cor- relation with the Ten ratings measured (see Table 2 for the context/modifier pairs with the largest A Am One would expect Xem evaluated on more positive F'mm to increase relative to X Xbase and vice This is what we observe for BERT (see Fig- ure 2) and GPT2, both of which show significant correlations (0.40 and 0.54) between Aem and Tem (p 0.01, t-test).
To calculate the model scores, Xem. for each trait, the models are evaluated on all ten items as- signed the trait, with each item serving as context once. This is done for each of the five modifiers, resulting in 10 (context items per trait) x 5 (mod- ifiers per context item) x 10 (questionnaire items to be answered by the model) = 500 responses per trait and 10 (context items per trait) 5 (modifiers per context item) 50 scores (Xom) per trait (one for each context). Context/modifier ratings (rem) are calculated to quantify the models' expected behavior in response to context. First, each mod- ifier assigned a modifier rating between -2 and
Further, to examine at the effect of individ- ual context items as the strength of the modifier changes, we compute the correlation, between
A Apm and Ten for individual context items (correla- tion computed from 5 data points per context item, one for each modifier). Table 3 reports the mean and median values of these correlations. These sults indicate strong relationship between Aen and Ton- The mean values are significantly less than the medians, suggesting a left skew. For fur- ther analysis, the data was broken down by trait.

---

BERT Dem VS Fmm
language model could achieve spurious cor- relation, simply by copying the modifier choice mentioned in the context item. We experimented with adjustments 2 that would account for this issue and saw similar trends, with slightly lower but con- sistent correlation numbers (mean correlations of 0.25 and 0.40 for BERT and GPT2, compared with 0.40 and 0.54, statistically significant t-test).
20
4cm
-10
p: 0.40
-20
0.05,
Figure 2: BERT Aem Tem plots for data from all traits. We observe a consistent change in personality scores (Aem) across context items as the strength of quantifiers change.
Alternate Framing: Another possible concern is the altering of the Big Five personality assess- ment framing to involve quantifiers. We experi- mented with an alternate fill-in-the-blank framing (c.g., I [blank] that I am the life of the party) that uses the same answer choices as the original test. Note that neutral was excluded because it fails form a grammatical sentence. Despite the differ- ences in token count amongst these answers, the greater frequency imbalance of these answers in the pretraining data compared to the altered answers. and the added sentence complexity of the assess- ment items, we saw similar trends. BERT extrover- sion and emotional stability had mean correlations of 0.22 & 0.29 respectively, and GPT2 agreeable- ness, conscientiousness, emotional stability and openness to experience had mean correlations of 0.10, 0.14, 0.61 & 0.40. These results suggest that our results are robust to our modification of the wording of the answer choices.
BERT
GPT2
Mean p Med p 0.84
0.40
0.54
0.81
Table 3: Mean & median p from Acm vs Tem plots by context item
The histograms in Figure 3 depict by trait and include summary statistics for this data.
Mean and median from Figure plots suggest a positive linear correlation between Aem and rem amongst context item plots, with conscientiousness and emotional stability having the strongest corre- lation for both BERT and GPT2. Groupings of p around in conscientiousness and emotional stabil- ity plots from Figure demonstrate this correlation. GPT2 extroversion, BERT & GPT2 agreeableness and BERT openness to experience show large left skews. A possible explanation for for this is that models may have had difficulty distinguishing be- tween the double negative statements created by some context/modifier pairs (i.e. item 36 with mod- ifier never: "I never don't like draw attention myself."). This may have caused Aam be negatively correlated with Tem, leading to an aceu- mulation of p values near -1
6.2 Analysis With Reddit Context
Next, we qualitatively analyze how personality traits of language models react to user-specific contexts. To acquire such context data, we cu- rated data from Reddit threads asking individuals about their personality (see Appendix D for list of sources). 1119 responses were collected, the majority of which were first person. Table 4 shows two examples. Because GPT2 & BERT tokeniz- ers can't an't accept more than 512 tokens, responses longer than this were truncated. The models were evaluated on the 'Big Five' assessment ($4) us- ing each of the 1 1119 responses as context (Reddit
Table 2 shows the contexts that lead to the largest change for each of the personality traits for BERT and GPT2. We observe that all 10 contexts consist of the high-polarity quantifiers (either always or never), which consistent with the correlation re- sults. Further, we note that for four of the five traits, the item context that leads to the largest change is common between the two models.
We replaced the model responses where the questionnaire and context items matched with the base model's response for item. This means that the concerning context item can no longer contribute to A. However, this also means that numbers with this adjustment cannot directly compared with those without since there are fewer sources of variation.
is important note a possible weakness with our approach of using questionnaire items as con- text. Since our evaluation includes a given ques- tionnaire item as context to itself during scoring,
'In qualitative analysis of a random sample of 200 re- sponses, 3.5% of sampled responses were found to be hostile, harmfully hiased or offensive, while 71.5% were found to he relevant to the topic of personality.

---

Agrwebbleness p
BERT Conciistiiusesss
BERT Emetioaa Stability
BERT Openness to Expertrace p
Mour 0:21
Mour 6:25
Meat 0.65
Meca e 0.85
Dostan pt 021
Destaus e 0.50
Modias pr 9.76
Kadia as
Stetaa 9.79
carr2 Retenoersiinn
GPT2 Agrensbleemoo jo 0.70
GPT2 Connect end comoosss
GPT2 Emotionel Sbability p. GPT2 Opennese Experience
podiann 0.77
Soddin 0.8
Mesdas pt 0.52
Madia
Figure 3: Histograms of by trait for Aem VS rem context item plots. Across all ten scenarios, a plurality of context items show a strong correlation (peak close to I) between observed changes in personality traits and strengths of quantifiers in the context items.
Context
scientiousness. There were fewer phrases for GPT2 openness to experience, GPT2 negatively weighted agreeableness, and GPT2 negatively weighted ex- troversion that caused shifts in the expected direc- tion. This was consistent with results from $6. I, where these traits exhibited the weakest relative positive correlations. Appendix D contains the full lists of highly weighted features for each trait.
Subdued until really get to know someone. I am polite hut not friendly. do not feel the need to hang around with others and spend most of my time reading, listening to music, garning Getting to know me well is quite a challenge I suppose, but my few friends and have a lot of fun when we meet (usually university or online, rarely elsewhere irl). I'd say am patient, rational and guy with big heart for the ones care for.
watching films.
Table 4: Examples Reddit data context.
Context
U Undirected Response
I am very open-minded, polite person and always crave new experiences. At work I manage a team of software developers and we often have to come up with new ideas. I went to college and majored in computer science... I try to do something fun every week, even if I'm busy, like having BBQ watching movie. have wife whom love and we live together a single-family home.
context). For each Reddit context, scores, Xreddit- were calculated for all 5 traits. The difference be- tween Xreddat and X Masse was calculated as Areddit-
To interpret what phrases in the contexts affect the language models' personality traits, we train re- gression models on bag-of-words and n-gram (with reserved. do not like to talk that much unless I have to. am fine with being by myself and enjoying the peace and quiet. usually agree with people more often than not. I am polite and kind person. am mostly honest, but I will lie I feel it is necessary or if it henefits me in a huge way. am easily irritated by things and have anxiety issues
Directed Rexponse
consider myself to be someone that is quict and
n 2 and n 3) representations of the Red- dit contexts as input, and Areddit values as labels. Since the goal is to analyze attributes in the con- texts that caused substantial shifts in trait scores, we only consider contexts with |eddiilll 2 1. Next, we extract the ten most positive and most nega- tive feature weights for each trait. We note that for extroversion, phrases such 'friendly', 'great' and 'no problem' are among the highest positively weighted phrases, whereas phrases such as 'stub- born' and 'don't like people' are among the most negatively weighted. For agreeableness, phrases like 'love' and 'loyal' are positively weighted, whereas phrases such as 'lazy', 'asshole' and exple- tives are weighted highly negative. On the whole, changes in personality scores for most traits con- formed with human understanding of the most highly weighted features. As further examples, phrases such as 'hang out with' caused a positive shift in trait score for openness to experience, while "lack of motivation' causes a negative shift for con-
Table 5: Examples of survey data contexts.
6.3 Analysis With Psychometric Survey Data
The previous sections indicate that language mod- els can pick up on personality traits from context. This raises the question of whether they can be used to estimate an individual's 's personality. In the- ory, this would be done by evaluating on the 'Big Five" personality assessment using context describ- ing the individual, which could aid in personality characterization in cases where it is not feasible for a subject to manually undergo personality as- sessment. We investigate this with the following experiment. The experimental design for this study

---

vetted and approved by an Institutional Review Board (IRB) at the authors' home institution.
GPT2 Comperision af from Survey & liem Contexl.
0.5-
Lem
Using Amazon Mechanical Turk, subjects were asked to complete the 50-item 'Big Five' personal- ity assessment outlined in $4 (the assessment was not modified to a sentence completion format as done for model testing) and provide 75-150 word description of their personality (see Appendix E. for survey instructions). Responses were man- ually filtered and low effort attempts discarded, resulting in 404 retained responses. Two variations of the study were adopted: the subjects for 199 of the responses were provided a brief summary of the 'Big Five' personality traits and asked to consider, but not specifically reference, these traits in their descriptions. We refer these responses the Directed Responses data set. The remaining 205 subjects were not provided this summary and their responses make up the Undirected Responses data set. Table 5 shows examples of collected descrip- tions. Despite asking for personality descriptions upwards of 75 words, around fourth of the re- sponses fell below this limit. The concern was that data with low word counts may not provide enough context. Thus, we experiment with filtering the responses by removing outliers (based on the interquartile ranges of measured correlations) and including minimum thresholds on the description length (75 and 100).
0.5
Survey
0.2
A
Trazt
The plot compares o from model evalua- tion with item context ($6.1) and survey context ($6.3). Survey context p shown here are from Undirected Re- sponses (c2 100). In both cases, measures the Pear- son correlation between trait scores with context and expected behavior. The variables used quantify pected behavior differ between experiments.
Figure 4:
Trait
Pro-muthier
Pe27a
Pe2100
Undirected Responses
BERT
0.40
0.39
0.41
GPT2
0.48
0.43
0,48
Directed Responses
BERT GPT2
0.44
0.42
0.39
0.48
0.43
0.42
Table 6: p for X Suuveyy gubject for data filtered by removing outliers and enforcing word counts.
includes correlation coefficients from $6.1. While the correlations from both sections are measured for different variables, they both represent a general relationship between observed personality traits of language models and the expected behavior (from two different types of contexts). While there are positive correlations for all ten scenarios, correla- tions from survey contexts smaller than those from item contexts. This is not surprising since item contexts are specifically handpicked by do- main experts to be relevant to specific personality traits, while survey contexts are free texts from open-ended prompts.
Human subject scores, Xsubject were calculated for each assessment, using the same scoring pro- cedure as previously described in $4. The models were subsequently evaluated on the 'Big Five' per- sonality assessment using the subjects' personality descriptions as context, yiclding Xuurvey scores corresponding to each subject. Table shows a summary of the correlation statistics for the two data sets and the different filters. There are strong correlations (0.48 for GPT2 and 0.44 for BERT for Directed Responses) between predicted scores from personality descriptions and the actual psy- chometric assessment scores. We note that there are only marginal differences correlations between the two datasets, in spite of their different charac- teristics. While more specific testing is required to determine causal factors that explain these ob- served correlation values, they suggest the potential for using language models as probes for personality traits in free text.
6,4 Observed Ranges of Personality Traits
In the previous subsections, we investigated prim- ing language models with different types of con- texts to manipulate their personality traits. Figure 5 summarizes the observed ranges of personality trait scores for different contexts, grouped by context type. The four columns for each trait represent the scores achieved by the base model (no context), and the ranges of scores achieved by the different types of contexts. The minimum, median and max- imum scores for each context type are indicated by different shades on each bar. We observe that the different contexts lead to remarkable range
Figure plots the correlations (p, outliers re- moved) for the individual personality traits, and

---

BERT Observed Ranges of Personality Scores
Example (agreeableness)
100
Text Generation Context: I am always interested in people. When I have work to do,
Score Percentile
Poase
60
Prm
60-
Preddit
Generated Text:
often get curious about people. love the time I've spent at my job. My children grow up knowing me really well. Are there any other things you
Paurvey
Example 2 (emotional stability)
Text Generation Context:
E8
OE
Trait
I never get stressed out easily. When I talk to others. I Generated Text:
Figure 5: Observed ranges of personality traits (in hu- man percentiles) from BERT, when conditioned on dif- ferent context types. These include scores from the base model (Paase) and ranges of scores from the three context types: item (Pon), Reddit (Preddst) and survey (P (Psurvey). ). Bars for context-based scores show per- centile of the minimum, median, and maximum-scoum scoriing context, in ascending order. The lightest shade of each color indicates the minimum, the darkest indicates the maximum and the intermediate indicates the median.
don't I get stressed out much either. can go restaurants we want to go to and get see great food other people that know, and be
Example 3 (openness to experience)
Text Generation Context:
never have excellent ideas. Others say that Generated Text:
am fool. When write my thoughts try to find where am supposed to get an idea. That is why is so hard me to do the ...
Table 7: Representative examples of Text Generation Contexts & corresponding Generated Texts. Each text generation context is concatenation of context/modi- fier pair from $6.1 and neutral prompt.
of scores for all five personality traits. In partic- ular, for two of the traits (conscientiousneess and emotional stability), the models actually achieve the full range of human scores (nearly O to 100 percentile). Curiously, for all five traits, different contexts are able achieve very low scores (< 10 percentile). However, the models particularly strug- gle with achieving high scores for agreeableness.
GPT2 Xgen Vs X
40
30
Xgen
20
10
p: 0.49
7 Effects on Text Generation
10
15
20
25
30
35
40
X
While the previous sections strongly suggest that the perceived personality traits of language mod- els can be influenced for fill-in-the- th----ttt-bblkkk person- ality questionnaires. it is important to understand whether these influences also translate to text gen- erated by these language models in downstream applications. To answer this question, we created 'text generation contexts' by concatenating each context/modifier pair from $6.1 with each of six neutrally framed prompts (e.g., "I am always the life of the party" "When talk to others, I.... see Appendix F for complete list of prompts). For this experiment, GPT2 was used to generate 50 token text for each text generation context.
Figure 6: GPT2 Xgen Xom plots. We observe strong correlation between scores using generated text as context (Xgen) and scores using assessment items & answers as context (.X (Xandject).
both sentiment and topie, describing an individual who is both curious about people and who enjoys spending time in an interactive environment like job. While there are some generated texts with no apparent relation to text generation contexts, we found that most of the generated texts qualitatively mirror the personality in text generation context.
We also quantitatively evaluate how well the personality traits in the generated texts matches corresponding text generation contexts. For this, each generated text is, itself, used as context for Big Five assessment (as previously shown in Figure I, panel C). We measure the Pearson correlation between the resulting scores, Xgens and the scores for the context/item pair (X.m) from $6.1 that were
Table 7 gives examples of some text generation contexts and corresponding generated texts. Exam- text generation context that asserts that the model "always inter- ested in people"; the generated text matches this in
ple 1 in Table 7 corresponds
"Since BERT is trained for masked language modeling, and is not well suited for text generation

---

used in the corresponding text generation context. Figure 6 gives the results from this analysis, and shows an overall Pearson correlation of 0.49 be- tween gen and Xem-
collated. The crowd-soureed survey data was col- lected using Amazon Mechanical Turk (AMT) with the permission of all participants, following IRB approval of the study design. No personally iden- tifiable markers were stored and participants were compensated fairly, with a payment rate ($2. 00/tskk w/ est. completion time of 15 min) significantly higher than AMT averages (Hara al., 2018).
This suggests that the personality scores of the model, measured using the Big Five assessment, are good indication of the personality that might be seen in text generated from the contextualized language models.
The broader goal of this line of research is investigate aspects of personality in language mod- els, which are increasingly being used number of NLP applications. Since Al systems that use these technologies are growing ever pervasive, and as humans tend to anthropomorphize such systems (i.e., Siri and Alexa), understanding and controlling their perceived personalities can have both broad and deep consequences. This is especially true for applications in domains such as education and men- tal health, where interactions with these systems can have lasting personal impacts on their users. Finally, if the personalities of Al systems can be manipulated in the ways that our research suggests, there is serious risk of such systems being ma- nipulated, through targeted attacks, to be hostile disagreeable to their users. Developing methods through which language models could be made im- mune to such attacks would then be a necessary consideration before fielding such systems.
8 Conclusion
We have presented simple approach for mea- suring and controlling the perceived personality traits of language models. Further, we show that such models predict personality traits of hu- man users, possibly enabling assessment in cases where participation is difficult attain. Future work can explore the use of alternate personality taxonomies. Similarly, there is large and growing variety of language models. It is unclear to what extent our findings generalize to other language models, particularly those with significantly more parameters (Brown et al., 2020; Smith et al., 2022). Finally, the role that pretraining data plays on per- sonality traits is another important question for exploration.
Limitations
Our exploration has some notable limitations. These include answer bias due to variable token count and frequency imbalance in pretraining data and the presence of double negative statements in questionnaire items ($4). The later might be ad- dressed by experimentation with other language models. For instance, GPT2's closed source suc- cessors, GPT3 and GPT4, are shown to handle dou- ble negatives better than GPT2 ( (Nguyena et al., 2023)). Concerns with the altered questionnaire framing and the context item evaluation procedure were partially addressed in follow up experiments in $6.1. As mentioned in the Conclusions section, whether and how our results generalize to other language models remains an open question.
Acknowledgements
This work was supported part by NSF grant DR1.21/2635. The authors also thank anonymous reviewers for suggestions and feedback.
References
Abubakar Abid, Maheen Farooqi, and James Zou, 2021. Large language models associate muslims with vio- lence. Nature Machine Intelligence, 3(6)461-463.
Lisa P Argyle, Ethan C Busby, Nancy Fulda, Joshua Gubler, Christopher Rytting, and David Wingate, 2022. Out of one, many: Using language mod- els to simulate human samples. arXiv preprint arXiv:2209.06899
Ethics and Broader Impact
Rowan Bayne. 1997. The myers-briggs type indicator: A critical review and practical guide.
The "Big Five' assessment items and scoring proce- dure used in this study were drawn from free pub- lic resources and open source implementations of BERT and GPT2 (HuggingFace, 2022) were used. Reddit data was scraped from public threads and no usernames or other identifiable markers were
Emily M. Bender and Alexander Koller. 2020. Climbing towards NLU: On meaning, form, and understanding in the age of data. In Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guistics, pages 5185-5198, Online. Association for Computational Linguistics.

---

Shikha Bordia and Samuel R. Bowman. 2019. Identify- ing and reducing gender bias in word-level language models. In Proceedings of the 2019 Conference of the North, Minneapolis, Minnesota.
and Lawrence Pervin, editors, Handbook of per- sonality: Theory and research, pages 114-158. The Guilford Press, New York, New York.
Oliver P. John and Sanjay Srivastava. 1999, The big five trait taxonomy:
Tom Brown. Benjamin Mann, Nick Ryder, Melanie Subbiah. Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 13:1877-1
History, measurement, and theoreti- cal perspectives. In Lawrence Pervin and Oliver John, editors, Handbook of personality: Theory and research, pages 102-138. The Guilford Press, New York, New York.
Saketh Reddy Karra, Son Nguyen, and Theja Tula- bandhula. 2022. Al Personification: Estimating the personality of language models. arXiv preprint arXiv:2204.1000 12000.
Hans Christian, Derwin Suhartono, Andry Chowanda, and Kamal Z. Zamli. 2021. Text based personality prediction from multiple social media data sources using pre-trained language model and model averag- ing. Journal of Big Data, 8(1).
Anastasia Kuzminykh, Jenny Sun, Nivetha Govindaraju, Jeff Avery, and Edward Lank. 2020. Genie in the bottle: Anthropomorphized perceptions of conversa- tional agents. In Proceedings of the 2020 CHI Con- ference on Human Factors in Computing Systems, pages 1-13.
Jacob Devlin, Ming-Wer Wei Chang, Kenton Lec, and Kristina Toutanova, 2019. Bert: Pre-training of deep bidirectional transformers for language understand- ing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Lunguage Technolo- gies, Minneapolis, Minnesota.
Y Masui, Gondo, H Inagaki, and N Hirose. 2006. Do personality characteristics predict longevity? findings from the tokyo centenarian study. Age, 28(4):353- 361.
Lewis R. Goldberg. 1990.
An alternative "descrip- tion of personality": The big-five factor struc- ture. Journal of Personality and Social Psychology, 12(6)):11611222)
Yash Mehta, Samin Fatehi, Amirmohammad Kazameini, Clemens Stachl, Erik Cambria, and Saulch Eetemadi. 2020. Bottom-up and top-down: Predicting person- ality with psycholinguistic and language model fea- tures. In 2020 IEFF International Conference on Data Mining (ICDM), Sorrento, Italy.
Lewis R. Goldberg. 1993. The structure of phenotypic personality traits. American Psychologist, 48(1):26 ):26- 34.
Marilà Miotto, Nicola Rossberg, and Bennett Klein- berg. 2022. Who is gpt-3? an exploration of per- sonality, values and demographics. arXiv preprint Stii:2200.143322
Kotaro Hara, Abigail Adams, Kristy Milland, Saiph Savage, Chris Callison-Bul Burch, and Jeffrey P. Bigham. 2018. A data-driven analysis of workers' carnings on amazon mechanical turk. Proceedings of the 2018 CHI Conference on Human Factors Computing Systems.
Shane T Mueller. 2020. Cognitive anthropomorphism of ai: How humans and computers classify images. Ergonomics in Design, 28(3):12-19, 19.
Po-Sen Huang, Huan Zhang, Ray Jiang, Robert Stan- forth, Johannes Welbi, Jack Rae. Vishal Maini, Dani Yogatama, and Pushmeet Kohli. 2020. Reducing sen- timent bias in language models via counterfactual evaluation. Findings of the Association for Computa- tional Linguistics: EMNLP 2020.
Ha Thanh Nguyena, Randy Goebelb, Francesca Tonic, Kostas Stathisd, and Ken Satoha. 2023. A nega- tion detection assessment of gpts: analysis with the xnot360 dataset.
Srenn-sychometrics.. 2018 Open-source psychomet- rics project: Answers to the IPIP big five factor mark- ers. Last accessed 29 August 2022.
HuggingFace, 2022. the ai community building the future. Last accessed 22 March 2022.
Joe O'Connor and Jacob Andreas. 2021. What text features can transformer language models use? Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguistics and the I 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers).
IPIP. 2022. Administering IPIP measures, with a 50- item sample questionnaire. Last accessed 22 March 2022.
Guangyuan Jiang, Manjie Xu, Song-Chun Zhu, Wen- juan Han, Chi Zhang, and Yixin Zhu. 2022. MPI: evaluating and Inducing personality in pre-trained language models. arXiv preprint arXiv:2206.075:
Melissa C O' Connnor and Sampo V Paunonen. 2007. Big five personality predictors of post-secondary aca- demic performance. Personality and Individual dif- ferences, 43(55)971199)) 19000.
Oliver P. John, Laura P. Naumann, and Christopher J. Soto. 2008. Paradigm shift to the integrative big-five trait taxonomy: History, measurement, and concep- tual issues. In Oliver P. John, Richard W. Robins,
Pierre Pureur and Murat Erder. 2016. 8, page 187-213. Morgan Kaufimann Publishers.

---

Alec Radford, Jeffrey Wu, Rewon Child, David Luan. Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners.
Arleen Salles, Kathinka Evers, and Michele Farisco. 2020. Anthropomorphism ai. AJOB neuroscience. 11(2):88-95.
Shaden Smith, Mostofa Patwary, Brandon Norick, Patrick LeGresley, Samyam Rajbhandari, Jared Casper, Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, et al. 2022. Using deep- speed and megatron to train megatron-turing nlg 530b, large-scale generative language model. arXiv preprint arXiv:2201.11 109999
Jerome P Wagner and Ronald E Walker. 1983. Relia- bility and validity study of a sufi personality typol- ogy: The enneagram. Journal of clinical psychology, 39(5)712-771.
Jason K White, Susan S Hendrick, and Clyde Hendrick. 2004. Big five personality variables and relationship constructs. Personality and individual differences, 37(7):1519-1530.
Marty J Wolf, Keith W Miller, and Frances S Grodzin- sky. 2017. Why should have that ing: comments on microsoft's tay "experiment"" and wider implications. The ORBIT Journal, 1(2):1-22.
Feifan Yang. Tao Yang. Xiaojun Quan, and Qinliang Su. 2021. Learning answer psychological ques- tionnaire for personality detection. Findings of the Association for Computational Linguistics: EMNLP 2021.

---

Appendix A Model Background
Appendix B Experiment Design Items
BERT, which stands for Bidirectional Encoder Rep- resentations from Transformers, is transformer- based deep learning model for natural language processing (Devlin et al., 2019). The model is pretrained on unlabeled data from the 800M word BooksCorpus and 2500M word English Wikipedia corpora. While BERT can be fine-tuned for autore- gressive language modeling tasks, it is pretrained for masked language modeling. This study uses a BERT model from HuggingFaces's Transformer Python Library with a language model head for masked language modeling. No fine-tuning was done to the model. GPT2, which stands for Genera- tive Pretrained Transformer 2, is ceneral-purposs learning transformer model developed by OpenAl in 2018 (Radford et al., 2019). Like BERT, this model is also pretrained on unlabeled data from the 800M word BooksCorpus. The study used Huggin- face's GPT2 model with a language model head for autoregressive language modeling. As with BERT, no fine-tuning took place.
Extroversion
35000
30000
Priqueiey
15000
10000
5000
Trait Score
Agreesbleness
40000
Peequency
30000
20000
10000
Trait Score
Conscientiousness
40000
Prequancy
30000
20000
Trait Score
Emotional Stability
35000
30000
Prqquency
25000
20000
10000
5000
Openneess Experience
30000
Prequancy
20000
10000
10
Trait
Figure 7: Human distributions of 'Big Five" trait scores.

---

Item
Associated Trait
[blank] the life the party.
(blank] feel concern others.
[blank] prepared.
(blank] get stressed out easily.
ES
[blank] have rich vocahulary.
OF
(blank] don't 't a lot. am [blank] | interested people.
[blank) leave my belongings around.
C ES
am [blank f relaxed most of time.
[blank) have difficulty understanding abstract ideas. I [blank] feel comfortable around people.
OE E A C ES OE E A C ES OE E A C ES OF
I [blank] insult people.
I [blank] pay attention to details.
[blank] worry about things. [blank] have vivid imagination.
[blank] keep in the hackground.
[blank) sympathize with others' feelings.
[blank] make mess of things.
(blank) seldom feel blue.
I am (blank] not interested in abstract ideas. [blank) } start conversations.
I am (blank] not interested in other people's problems. (blank) ) get chores done right away.
am (blank) easily disturbed.
[blank] } have excellent ideas.
(blank) } have little say.
E A C ES
[blank] have soft heart.
[blank) forget put things back their proper place.
[blank] get upset easily.
[blank] not have good imagination.
OE
[blank] talk to lot different people at parties.
E
[blank] not really interested in others.
[blank] like order.
C
[blank] change my mood lot. [blank] | quick to understand things.
ES
OF
[blank] | don't 't't like draw attention myself.
(blank) time
others.
[blank) shirk my duties.
[blank] have frequent mood swings. [blank) use difficult words. [blank] mind being the center of attention. [blank] feel others' emotions.
A
[blank] follow schedule.
[blank] get irritated casily.
ES
[blank] | spend time reflecting on things.
am [blank] | quict around strangers.
(blank) J make people feel ease.
A
am (blank) exacting in my work.
C
(blank) feel blue.
ES
am (blank) full of ideas.
OE
Table Bl: Adjusted 'Big Five" Personality Assessment Items.
Trait
Median
Mean (n)
SD (a)
R
20
19.60
9.10
29
27.74
7.29
24
23.66
7.37
ES
19
19.33
8.59
OE
29
28.99
6.30
Table B2: Human Population Distribution of 'Big Five' Personality Traits.

---

Trait
Base Value
Positively Scored Item
Negatively Scored Item #
E
20
11, 31,
6. 16. 26, 36, 46
14
17. 37, 3. 13,
2. 22, 32
C
14
8.
18. 28, 38
ES
38
9. 19
14, 24, 29, 34, 39, 44, 49
OE
5, 15, 35, 40, 45,
Table B3: 'Big Five' Personality Item Scoring Procedure.
Appendix C
Item Context Evaluation Tables
Tem
Mean A
Med A.m
Am SD
Confidence Interval
BERT
-3,36
-2.0
7.49
[-5.51, -1.21]
-1 -3.18
-3.50
4.81
[-4,56, -1.80]
-0.02
0.00
4.51
[-1.32, 1.28]
2.42
2.00
6.17
[0.648, 4.191
2
3,96
3.00
8.33
[1.57. 6.35]
GPT2
-2
-7.34
-8.0
6.38
[-9.17, -5511]
-1
-4.58
-4.0
4.32
[-5.82, -3.34]
0
-2.06
-1.0
4.24
[-3.28, -0.84]
1
0.0
0.0
3.13
[-0.90, 0.90]
2
1,56
1.0
5.78
[-0.10, 3.22]
Table C1: Statistics from Acm vs Tem plots containing data from all traits. Statistics include mean, median, standard deviation and confidence interval for A.m at each Tam-
Appendix Reddit Context Evaluation Tables
Reddit Context Sources
wwt.a///wwsseeeaiiiiiooonnnsasaa _would
would_yoo_deeede
yoo_describe_yeeee _ourt pocuurepoonnltty//
andddt/..mmmi.ssseccatttttmmmmm j/redditors_whie
saryr-personalliiii _ikke/
reeddttccooooo//////ddddiioononn would you, describe your personality/
Bedddticc.n.....e.ooooeoooooooee would couuddscribee_eeee your "our_personality/
reddit,
most boost_defiiigg.... personalitychuracter
aaadliiccntttitaaaalmmmmeooooooo would you describe your personality/
would_you you_uurrrrbbe____ee __our personality/
wttp://wwwwoowwwwwww_oaoe would_yooo deescribe_your_prrssoeee
wee.licccono//ss/s/Rell////eeeee jouurrr___uur___________ bes__eee_____e_____e personaltty.ll ttait/
rebllc..omttssssseeedtttiommmmns your worst personality trait/
reddit.com
your worst_ perrsolprsoallity... train/
aedddiccettt./sssseeeeeoomommmee reddit, what your worst character trait/
Table D1: Domain names of threads that were scraped to collect Reddit context.
Trait
Mean Areddia
Med Areddit
Arddii SD
5 Max Areddi
Min Areddit
BERT
-2,28
4.04
8,7,6,6,5
-14, -13, 13,
A
-2,02
3.38
2,2,
-19.-18,-15,-15,
3.77
4
5.17
15,
-17,-17,-16,
14,
1.71
2
2.29
14.
13.
-12.-10.-11.-10.-11
OE
1.74
2.17
9,7.7.7.7
-11,-11,-8,88,
GPTZ
-3.73
3.33
5,4,
11,--10.-10.
A
-0.98
4.26
13, 10,10,0 7.
-17.-15.115.115.1
C
-0.27
0
4.27
11, 11,9
-0..166-16,-11 16.
ES
-3.83
-3
6.27
8, 8, 8,
OE -1.91
-2
3.21
4, 4,4,
Table D2: Aredia summary statistics. Statistics include mean, median and standard deviation, as well largest and 5 smallest Areddit

---

BERT
Extroversion
Notable Positively Weighted Phrases: 'friendly', 'great', 'good", 'quite", 'laugh', "please", 'sense of", 'thanks for', 'really good', 'and friendly', "no prohlem", "to please", "my sense of", "finish everything start", 'enthusiastic but sensitive"
Notable Negatively Weighted Phrases: "question', 'stubborn' 'why', 'lack', 'fuck', 'fucking', 'hate", "not', 'lack of', 'too much', ddon know", 'don like", "too easily", "way too', "don like people", "you go out', 'don know how", 'don['t] know what'
Ag Agreeableness
Notable Positively Weighted Phrases: 'will', "friendly", 'lal', 'love', 'loyal', 'calm", "yup", 'does', 'honesty', 'laid back', go out', 'thanks for', "really good", 'out with me", 'friendly polite and", "really good listener', 'true to myself", 'my sense of"
Notable Negatively Weighted Phrases: 'lack', 't'dn''tl,, 'wonf't], 'lazy', 'fucking', 'self", 'worst', 'lack of", 'too easily', 'don like', "the worst', 'being too', 'have no', 'don like people', : 'lack of motivation', 'don know how', my worst trait", "also my worst", 'too honest sometimes', 'doesn['t (j talk much'
Conscientioummess
Notable Positively Weighted Phrases: 'am', 'friendly,, "just', 'calm', 'helieve", 'ean he', 'of people", 'tond to", 'feel like', 'the most humble', 'most humble person'. "my sense of", 'get to know', "friendly polite and', 'get along with', 'peoplc like me'
Notable Negatively Weighted Phrases: 'lack', 'no", 'lazy', 'inability', "fucks", 'half", 'lack of", 'fuck off", 'don like', 'inability to', 'don like people", "you go out', 'lack of motivation', 'don even know", 'monotonous and impulsive'
Emotional Stability
Notable Positively Weighted Phrases: 'will', 'feel", "out with me", "go out with', "will you go', 'the most humble'
Notable Negatively Weighted Phrases: 'no', 'off', "hypercritical', 'overthinking', 'lack of", 'casily distructed" 'doesnf't] talk', 'don even', , 'too easily distracted', 'lack of motivation', 'doesn] ']] talk much", 'don even know", 'unrelatable is strange', strange one", 'this said foreskin'
Openness to Experience
Notable Positively Weighted Phrases: 'most', 'like', 'me to', 'out with', 'like 'like to', 'want to', 'with me?, 'out with me', 'will you go', 'want be', 'all the time', me to., 'hang out with'
'lack', 'never', 'fucks', 'sad'. 'nothing'. 'lack empathy". "the complainer" nn confidence'' 'lack of", "easily distracted', 'blame helicopter', 'helicopter parents', 'never say sorry", "blame helicnpter parents', "too casily distracted', ; 'finish projects after', 'never finish projects", pprocrastination out of", "my lack of", 'lack of personality', , "ton many fucks'
Notable Negatively Weighted Phrases:
Table D3: Analysis of highest weighted phrases from BERT logistic regression.

---

GPT2
Extroversion
Notable Positively Weighted Phrases: 'believe', 'loyal', 'curious', 'best'. 'passionate', 'enjoy", 'bright', 'hard working", 'no problem', "am nice", "my amazing modesty", 'smooth hright epic', "patient and flexible", 'great with children', 'calm cool collected'
Notable Negatively Weighted Phrases: 'introverted', 'lack 'laid back', 'don know how"
Agrecubleness
Notable Positively Weighted Phrases: 'friendly', 'loyal', 'honest', 'gay'. 'humor'. 'like people', 'thanks for', "to please' 'and friendly', 'no problem'. 'friendly polite and", "patient and flexible', 'calm cool collected", 'honesty being straightforward"
Notable Negatively Weighted Phrases: easily', 'too much', 'lack of", 'you go out", 'don know what", 'self", 'asshole'
Conscientiousness
'smile", "thanks for', 'no problem', 'friendly polite and', 'really good listener', 'true myself", "patient and flexible'
Notable Positively Weighted Phrases:
Notable Negatively Weighted Phrases: 'stop'. 'jealousy', 'lazy'. 'hate", 'lack', 'fuck'. 'worst', 'lack of", "too casily", 'fuck off", "too nice', "don know", "don know how', "lack of motivation', 'don know", 'my worst trait', 'damn unele', 'depressed shit"
Emotional Stability
Notable Positively Weighted Phrases: 'friendly', 'calm", 'easy', 'honesty", 'laid back", 'hard working', 'calm and', 'humble am', 'polite and", "no problem', 'out with me", 'the most humble'
Notable Negatively Weighted Phrases: 'lack", 'anxious', 'lazy'. 'jealousy'. 'lack of", 'don know', 'too easily', 'don like', 'don like people'. 'don know how', 'lack of motivation', "don even know"
Openness Experience
'understand", "having', 'wanting', "thoughts", 'thanks for", "too nice", 'no problem', 'can relate", 'being too nice", 'that just confidence'
Notable Positively Weighted Phrases:
Notable Negatively Weighted Phrases: "fuck', "myself", 'cynical', 'lack', 'horing', 'lack of", 'don like people
Table D4: Analysis of highest weighted phrases from GPT2 logistic regression.
Appendix E Survey Context Evaluation Tables
Part I Instruction
There parts this questionnaire. In the first part this page), you will be shown 50 questions. and need to choose response which best matches your personality. the second part (on the next page). you will be asked write short (75-150 word) description of your personality free text. Participants will only compensated they respond
questions.
Part 2 Instruction
In between 75 and 150 words, please describe your personality [Directed responses: as it relates to the personality traits outlined above. Be sure not to use the name of the personality traits themselves in your response].
Table El: Data collection survey instructions.

---

Appendix F Generated Text Evaluation Tables
Text Generation Prompts
When I go to gathering, 1 Others say that I am
When I am around people, I...
When I have work do, I
When I have free time, I...
When I talk to others, I ...
Table List of prompts used text generation context.


Reviews:
[{"review_id": "lE254GuLTs", "reviewer": "Reviewer_UeN6", "summary": "", "questions": "Based on the definition, it would be more reasonable for the trend depicted by the points in Figure 2 to show that as the absolute values on the x-axis increase, the absolute values on the y-axis also increase？", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your time and effort in reviewing our paper. We are grateful for your feedback, and we would like to address the concerns you raised:\n\nResponses to Questions: \n\nQuestion: Based on the definition, it would be more reasonable for the trend depicted by the points in Figure 2 to show that as the absolute values on the x-axis increase, the absolute values on the y-axis also increase?\n\nResponse: The r_{cm} values represent the expected relative change in trait score (expected behavior) when the corresponding context/modifier pair is used as context (see lines 270-279 for an explanation of how this value is calculated). Thus, one would expect X_{cm} evaluated on a more negative r_{cm} to decrease relative to X_{base}. This is to say, we would expect certain context item/modifier pairs to cause a decrease in trait score. For instance, we would expect “I am never the life of the party” to cause a decrease in extroversion score (negative $\\Delta$_{cm}). Taking the absolute value of the $\\Delta$_{cm} would misrepresent these cases. Cases where the expected & actual behavior is a decrease would instead show as an increase.\n\n\n\nPrompt & Model Biases: The goal was not to eliminate sources of bias, like model parameters & pre-training data, but rather, to see how that bias affects the measured personality of these out-of-the-box models, relative to the population’s measured personality. Further experiments aimed to evaluate how such bias might be accounted for without modifying the model itself (i.e. changing pre-training data & model parameters). This was done by manipulating the models’ behavior via ambient context. Irrespective of the biases in the model parameters, we are interested in exploring the relative change in personality traits of the language models when exposed to textual contexts, rather than their absolute value. We are unclear on your comment about biases in the prompts: our results show that the prompts lead to predictable changes in personality traits. Further our analysis shows that these results are not sensitive to the format of the prompts.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper has conducted sufficient experiments for BERT and GPT2, and the text flows smoothly. It offers insightful inspiration.", "reasons_to_reject": "The experimental design of this paper is insufficient to demonstrate the manipulability of personality in more powerful language models (GPT3.5..). The evaluation of the experimental design also appears to lack proper justification , as it does not adequately address the biases introduced by the design of prompts and the biases inherent in model parameters. Furthermore, reliability and generalization in the study seems somewhat limited.", "questions_to_authors": "Based on the definition, it would be more reasonable for the trend depicted by the points in Figure 2 to show that as the absolute values on the x-axis increase, the absolute values on the y-axis also increase？", "paper_topic_and_main_contributions": "This study aims to investigate the personality traits of language models. Through experiments and psychological questionnaires, the personality traits of the language models are tested, along with the consistency of their responses and their manipulability.", "excitement": 3, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Dz1nWAlYG8", "reviewer": "Reviewer_9sqR", "summary": "", "questions": "Will the authors make the two personality description datasets public? This would be beneficial for the community.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your time and effort in reviewing our paper. We are grateful for your valuable feedback and thoughtful insights, and we would like to address both concerns:\n\nModel choice: BERT and GPT2 were selected because of their availability as open-source models, as opposed to newer models like GPT3, and ChatGPT had not yet been released when this work was started. Nonetheless, we agree that this work would benefit from experimentation against the latest language models for the sake of relevancy and accuracy. For instance, as noted in R3, GPT3 handles double negatives better than GPT2. While BERT & GPT2 share similarities to their newer counterparts, further work is required to determine how our results generalize to these other models.\n\nSuggested Citation: Thanks for bringing this work to our attention. Experiments from [1] use context to influence personally assessment scores in a similar way. However, rather than developing an improved model for predicting measuring, the focus of our work is to 1) characterize personality bias in out-of-the-box, pre-trained large language models and 2) explore ways to manipulate the measured personality (i.e. context). This work, along with a clarification of distinctness off our work, has been added to the literature review in the updated paper.\n\nGeneral update: We have expanded our discussion of the Big Five taxonomy in Section 2 of the updated paper, as well as the motivation and bases for choosing it above other possibilities such as MBTI and the Enneagram. Our choice is driven by the fact that the Big Five is the most commonly used taxonomy and has been shown to be practically predictive of outcomes such as educational attainment~\\cite{o2007big}, longevity and relationship satisfaction. This is not true for any of the alternate taxonomies. Further, it is relatively natural to cast as an assessment for language models. Lastly, we believe that our previous wording in the concluding section painted an unfairly negative picture of Big Five, since the criticisms we mentioned also apply to all other extant personality taxonomies – we have thus removed this offending text.\n\nOther: The personality description datasets will be made public.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The insights of this paper are meaningful and helpful in constructing personalized applications. \n2. The paper is well-organized. \n3. The experiments are detailed and explore different types of contextual data.", "reasons_to_reject": "1. The lack of results from other personality questionnaire tests, such as Ten Item Personality Inventory (TIPI), makes it unclear whether different questionnaires would lead to different conclusions.\n2. The analysis of ChapGPT is missing.\n3. It is necessary to clarify the differences between this method and other methods to predict user's personality, such as [1].\n\n[1] Yang F, Yang T, Quan X, et al. Learning to answer psychological questionnaire for personality detection[C]//Findings of the Association for Computational Linguistics: EMNLP 2021. 2021: 1131-1142.", "questions_to_authors": "Will the authors make the two personality description datasets public? This would be beneficial for the community.", "paper_topic_and_main_contributions": "This paper uses psychological questionnaires to test the potential personality of language models and reveals that personality can be manipulated by prefix context in language models. Some empirical analyses show that the personality of human users can be predicted from self-reported text descriptions by language models, and language models can generate text that conforms to human personality.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Sw2Bqgapws", "reviewer": "Reviewer_RXZa", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 5, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your time and effort in reviewing our paper. We are grateful for your valuable feedback and thoughtful insights, and we would like to address your concerns:\n\nModel choice: BERT and GPT2 were selected because of their availability as open-source models, as opposed to newer models like GPT3, and ChatGPT had not yet been released when this work was started. Nonetheless, we agree that this work would benefit from experimentation against the latest language models for the sake of relevancy and accuracy. For instance, as noted in R3, GPT3 handles double negatives better than GPT2. While BERT & GPT2 share similarities to their newer counterparts, further work is required to determine how our results generalize to these other models.\n\nGeneral update: We have expanded our discussion of the Big Five taxonomy in Section 2 of the updated paper, as well as the motivation and bases for choosing it above other possibilities such as MBTI and the Enneagram. Our choice is driven by the fact that the Big Five is the most commonly used taxonomy and has been shown to be practically predictive of outcomes such as educational attainment~\\cite{o2007big}, longevity and relationship satisfaction. This is not true for any of the alternate taxonomies. Further, it is relatively natural to cast as an assessment for language models. Lastly, we believe that our previous wording in the concluding section painted an unfairly negative picture of Big Five, since the criticisms we mentioned also apply to all other extant personality taxonomies – we have thus removed this offending text.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The analysis presented in this paper is very thorough and the implemented methods are sound.\n2. The overall motivation of the paper is strong. The findings of this paper provide significant evidences towards the use of computational methods for estimating personality traits from text.\n3. They perform detailed analysis of human feedback, which is crucial for such studies.\n4. The curated datasets will be of importance to the community.", "reasons_to_reject": "1. The generalizability of the findings to other types of computational models is unknown. As a suggestion, it would be interesting if general guidelines could be established that is invariant to the specific model type.  ", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper analyzes the perceived personality traits that are exhibited (if any) by recent language models. Using BERT and GPT2 they extensively analyze if such LMs can account for contextual personality traits and also experiment how such context manifests in the generated output. They further contribute datasets comprising human subjects, which will be of interest to the research community.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "4WEOS1h7XA", "reviewer": "Reviewer_gkMG", "summary": "", "questions": "Section 1:\n- Line 74: It would be good to clarify here what you mean by context so that the intro is clearer, e.g., you're referring to social contexts? different scenarios where people are interacting with a model for a specific reason, etc\n- Lines 81-84: I'm not exactly sure what you mean here by \"measured personality\". Assuming you mean that you qualify the personality expressed by a model based on some psychological assessment, how else would you qualify that expressed personality other than through the generated text by the model? What would be a scenario where the \"measured personality\" expressed by a model would not be reflected by the text generated by the model?\n\nSection 2:\nVery relevant to your work is the work by Jiang et al presented at IC2S2 (https://arxiv.org/abs/2305.02547). Unlike the 3 papers you mention, that work does attempt to manipulate perceived personality traits by prompting the model to reflect one of the Big Five personality types and then they investigate whether the content generated by the model is consistent with the personality type assigned (in the prompt) to the model.\n\nSection 4:\n- As much as I hate to be this person, I think a main concern is the choice of models used in the work. BERT by now is a 5-6 year old model that was superseded with other models that are more capable on many tasks (e.g. RoBERTa, BART etc). The other concern is regarding the choice of using GPT2. GPT3 has been released since June 2020 so I would assume the API for GPT3 models has been around for quite some time before the work for this paper has started? For example, the work I mentioned above (Jiang et al 2023) which carries out a very related investigation tests on GPT3.5. This is especially important given that there is a lot of work showing the big performance gap between GPT2 and GPT3. So overall at the very least there's some concern regarding the results and their timeliness and whether the conclusions carry over to more recent models.\n- A related point to this is that in some places (e.g. in the abstract - Line 7) the paper talks about large language models. The models in question here are no longer considered LLMs by today's standards. People talk about models having a few billion parameters at least to be LLMs. So I'm not sure I would use \"large\" to describe the work done here (although, to the author(s)'s credit, when specifically mentioning BERT and GPT2 they do not use the word \"large\" e.g. Line 14 in the abstract or Line 53 in the intro or Line 151 in Section 4).\n\nSection 6.1:\n- Lines 261-270: I want to make sure I am understanding the description here because I am somewhat confused. Specifically, I'm confused by the difference between \"context item\" and \"questionnaire items to be answered by the model\". The Big Five assessment has 50 statements/items as shown in the appendix targeting 5 personality traits so there's 10 per trait. You give in this section an example of a \"context item\" (e.g. in Line 257: I am {blank} the life of the party). So context item is referring to one of those 50 statements/items (which are 10 per trait). Are you then preceding each questionnaire item per trait (e.g. I {blank} feel comfortable around people) by a context item (e.g. I am {blank} the life of the party)? If so, what are you doing when the context item and the questionnaire item are the same? (i.e. you should multiply by 10 or 9 questionnaire items so that you avoid preceding a questionnaire item by the same context item) ? I think this part of the section could benefit from more clarification.\n- Line 317: On the issue of \"double negative statements\": While the authors present this as a possible explanation for the observed results, this goes back to my point regarding using a more recent model as I've come across work that specifically focuses on this issue of double negatives between GPT2 and GPT3 and shows that GPT3 resolves such cases at a much higher rate than GPT2. I tried to remember the reference but couldn't and it's possible it was some experimentation that someone did and shared on Twitter as I couldn't find a paper but there has definitely been work showing higher reasoning capabilities in general for GPT3 compared to GPT2.\n- Alternate framing: What are the possible answer choices that could fill the blank here? And why would such a sequence be less likely to appear in the pretraining data? And the more relevant question is how can you assess/confirm that such a sequence has \"greater frequency imbalance of these answers in the pretraining data compared to the altered answers\"? Specifically without access to the exact pre-training data, such considerations are rough/speculative even if one can argue that there are certain words that we expect to encounter more frequently online and in written texts (e.g. never and always) thus claiming that this would translate necessarily to the pre-training data. \n\nSection 6.2:\n- Reddit is generally known to have toxic content. Have the authors done any quality control on the data that they used to form their 1119 responses? Also, while threads where data was scraped from were generally about conversations regarding individuals' personalities, are all samples that make up the dataset actually samples that constitute discussions of personality traits? (e.g. no tangent talk was inadvertently included ?) What quality control has been regarding this issue?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your time and effort in reviewing our paper. We are grateful for your valuable feedback and thoughtful insights, and we would like to address the concerns you raised:\n\nResponses to Questions:\n\nQuestion: Line 74: It would be good to clarify here what you mean by context so that the intro is clearer, e.g., you're referring to social contexts? different scenarios where people are interacting with a model for a specific reason, etc...\n\nResponse: The word context is used to refer specifically to the text preceding the prompts that the models are evaluated on. In our experiments, we introduce context that comprises of personality descriptions in various forms, such as those based on personality assessment items (Section 6.1), reddit threads (Section 6.2) & self-provided assessments (Section 6.3).\n\nQuestion: Lines 81-84: I'm not exactly sure what you mean here by \"measured personality\". Assuming you mean that you qualify the personality expressed by a model based on some psychological assessment, how else would you qualify that expressed personality other than through the generated text by the model? What would be a scenario where the \"measured personality\" expressed by a model would not be reflected by the text generated by the model?\n\nResponse: The “measured personality” is the quantified personality as indicated by the scores from the Big Five personality assessment (Section 5). The generated text might not be reflective of the measured personality if the measured personality was captured inaccurately (i.e. the testing procedure was ineffective). Consider a hypothetical language model that (for some reason) specifically answers psychological questionnaires as an introvert, but generates text and responses to other questions that would match an extrovert. While we do not expect this to happen, it is not clear that it will not, and our experiments provide an answer to this empirical question; experiments in Section 7 confirm that text generated by these models reflect a personality consistent with the measured scores. This provides credibility to the assessment procedure and suggests that the assessment can act as an indicator of personality in text generated from downstream applications. \n\nQuestion: Lines 261-270: …Big Five assessment has 50 statements/items as shown in the appendix targeting 5 personality traits so there's 10 per trait. You give in this section an example of a \"context item\" (e.g. in Line 257: I am {blank} the life of the party). So context item is referring to one of those 50 statements/items (which are 10 per trait). Are you then preceding each questionnaire item per trait (e.g. I {blank} feel comfortable around people) by a context item (e.g. I am {blank} the life of the party)? If so, what are you doing when the context item and the questionnaire item are the same? (i.e. you should multiply by 10 or 9 questionnaire items so that you avoid preceding a questionnaire item by the same context item) ? I think this part of the section could benefit from more clarification.\n\nResponse: This interpretation of “context item” & “questionnaire item” is generally correct. A “context item” is just a “questionnaire item” used in the assessment context. However, note that the context in this section (6) includes both a “context item” and a modifier (an answer choice used in context). The issue of repeated items is a valid concern. We experimented with an adjustment that accounted for this issue (see line 337). Specifically, we replaced scores for context that consisted of repeat items with the base model score (from Section 5) for the given “questionnaire item”. This meant that the concerning context item could no longer contribute to $\\Delta$_{cm}. We saw similar, albeit lower, trends with mean correlations of 0.25 and 0.40, compared with 0.40 and 0.54 for BERT and GPT2 respectively. These correlations remained statistically significant ($p < 0.05$, t-test). \n\n\n\nAlternate framing details: The possible answer choices for testing with the alternate framing are the answer choices from the original, un-modified Big Five assessment, excluding “neutral. These include “agree”, “somewhat agree”, “somewhat disagree” and “disagree”. While we could not determine the exact answer frequency in the pretraining data, analysis with tools like the Google Books ngram viewer gave us some indication of the imbalance in online & literary texts. The generalization of this imbalance to the pretraining data was just one component that factored into the decision to introduce a new framing in Section 4.\n\nReddit Data Quality: The filtering out of tangent conversations was certainly considered. In the end, no quality control was performed on the data for lack of an objective way to determine how relevant a piece of text was to the topic of personality.\n\nModel choice: BERT and GPT2 were selected because of their availability as open-source models, as opposed to newer models like GPT3. Nonetheless, we agree that this work would benefit from experimentation against the latest language models for the sake of relevancy and accuracy. For instance, as noted in R3, GPT3 handles double negatives better than GPT2. While BERT & GPT2 share similarities to their newer counterparts, further work is required to determine how our results generalize to these other models.\n\nSuggested Citations: Thanks for bringing these to our attention!\n\nGeneral update: We have expanded our discussion of the Big Five taxonomy in Section 2 of the updated paper, as well as the motivation and bases for choosing it above other possibilities such as MBTI and the Enneagram. Our choice is driven by the fact that the Big Five is the most commonly used taxonomy and has been shown to be practically predictive of outcomes such as educational attainment~\\cite{o2007big}, longevity and relationship satisfaction. This is not true for any of the alternate taxonomies. Further, it is relatively natural to cast as an assessment for language models. Lastly, we believe that our previous wording in the concluding section painted an unfairly negative picture of Big Five, since the criticisms we mentioned also apply to all other extant personality taxonomies – we have thus removed this offending text.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Interesting study investigating the abilities of LMs to generate personalized text which could be useful to a wide variety of applications (e.g. more personalized chatbots)\n- Two new datasets: the first based on collected personality descriptions from human subjects and the second based on personality descriptions collected from Reddit", "reasons_to_reject": "- The presentation would benefit from some clarifications which I detail in my comments below.\n- Concerns regarding the models used in the paper.\n- Concerns related to the dataset (see comments below)", "questions_to_authors": "Section 1:\n- Line 74: It would be good to clarify here what you mean by context so that the intro is clearer, e.g., you're referring to social contexts? different scenarios where people are interacting with a model for a specific reason, etc\n- Lines 81-84: I'm not exactly sure what you mean here by \"measured personality\". Assuming you mean that you qualify the personality expressed by a model based on some psychological assessment, how else would you qualify that expressed personality other than through the generated text by the model? What would be a scenario where the \"measured personality\" expressed by a model would not be reflected by the text generated by the model?\n\nSection 2:\nVery relevant to your work is the work by Jiang et al presented at IC2S2 (https://arxiv.org/abs/2305.02547). Unlike the 3 papers you mention, that work does attempt to manipulate perceived personality traits by prompting the model to reflect one of the Big Five personality types and then they investigate whether the content generated by the model is consistent with the personality type assigned (in the prompt) to the model.\n\nSection 4:\n- As much as I hate to be this person, I think a main concern is the choice of models used in the work. BERT by now is a 5-6 year old model that was superseded with other models that are more capable on many tasks (e.g. RoBERTa, BART etc). The other concern is regarding the choice of using GPT2. GPT3 has been released since June 2020 so I would assume the API for GPT3 models has been around for quite some time before the work for this paper has started? For example, the work I mentioned above (Jiang et al 2023) which carries out a very related investigation tests on GPT3.5. This is especially important given that there is a lot of work showing the big performance gap between GPT2 and GPT3. So overall at the very least there's some concern regarding the results and their timeliness and whether the conclusions carry over to more recent models.\n- A related point to this is that in some places (e.g. in the abstract - Line 7) the paper talks about large language models. The models in question here are no longer considered LLMs by today's standards. People talk about models having a few billion parameters at least to be LLMs. So I'm not sure I would use \"large\" to describe the work done here (although, to the author(s)'s credit, when specifically mentioning BERT and GPT2 they do not use the word \"large\" e.g. Line 14 in the abstract or Line 53 in the intro or Line 151 in Section 4).\n\nSection 6.1:\n- Lines 261-270: I want to make sure I am understanding the description here because I am somewhat confused. Specifically, I'm confused by the difference between \"context item\" and \"questionnaire items to be answered by the model\". The Big Five assessment has 50 statements/items as shown in the appendix targeting 5 personality traits so there's 10 per trait. You give in this section an example of a \"context item\" (e.g. in Line 257: I am {blank} the life of the party). So context item is referring to one of those 50 statements/items (which are 10 per trait). Are you then preceding each questionnaire item per trait (e.g. I {blank} feel comfortable around people) by a context item (e.g. I am {blank} the life of the party)? If so, what are you doing when the context item and the questionnaire item are the same? (i.e. you should multiply by 10 or 9 questionnaire items so that you avoid preceding a questionnaire item by the same context item) ? I think this part of the section could benefit from more clarification.\n- Line 317: On the issue of \"double negative statements\": While the authors present this as a possible explanation for the observed results, this goes back to my point regarding using a more recent model as I've come across work that specifically focuses on this issue of double negatives between GPT2 and GPT3 and shows that GPT3 resolves such cases at a much higher rate than GPT2. I tried to remember the reference but couldn't and it's possible it was some experimentation that someone did and shared on Twitter as I couldn't find a paper but there has definitely been work showing higher reasoning capabilities in general for GPT3 compared to GPT2.\n- Alternate framing: What are the possible answer choices that could fill the blank here? And why would such a sequence be less likely to appear in the pretraining data? And the more relevant question is how can you assess/confirm that such a sequence has \"greater frequency imbalance of these answers in the pretraining data compared to the altered answers\"? Specifically without access to the exact pre-training data, such considerations are rough/speculative even if one can argue that there are certain words that we expect to encounter more frequently online and in written texts (e.g. never and always) thus claiming that this would translate necessarily to the pre-training data. \n\nSection 6.2:\n- Reddit is generally known to have toxic content. Have the authors done any quality control on the data that they used to form their 1119 responses? Also, while threads where data was scraped from were generally about conversations regarding individuals' personalities, are all samples that make up the dataset actually samples that constitute discussions of personality traits? (e.g. no tangent talk was inadvertently included ?) What quality control has been regarding this issue?", "paper_topic_and_main_contributions": "This paper presents a study investigating the abilities of large language models to generate text that is consistent with personality traits that the model has been prompted to have. The prompts are formed based on the Big Five personality assessment questionnaire which is a widely used taxonomy of personal traits in the psychological and psychometric literature and which focuses on 5 traits: extroversion, agreeableness, conscientiousness, emotional stability and openness to experience. Two datasets are released as part of the work: the first based on collected personality descriptions from human subjects and the second based on personality descriptions collected from Reddit.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "Jiang et al presented at IC2S2 (https://arxiv.org/abs/2305.02547). Unlike the 3 papers you mention, that work does attempt to manipulate perceived personality traits by prompting the model to reflect one of the Big Five personality types and then they investigate whether the content generated by the model is consistent with the personality type assigned (in the prompt) to the model.", "Typos_Grammar_Style_and_Presentation_Improvements": "- Throughout the paper (e.g. Line 54, 66, 250 etc), there's language around personality traits of models, I would suggest changing those to personality traits expressed by language models since this is a more accurate description of what you're trying to say.\n- Line 93: I'm not familiar with this form \"data-set\". One could argue that \"data set\" is correct but nowadays the predominant form is dataset.\n- Line 315: A possible explanation for for this \n- Line 327: which is* consistent with ... \n- Footnote 2:  compared with those without the since"}}]

Author rebuttal:

Meta-review: In general, reviews for this paper were mixed, with some reviewers feeling that the paper was more ready for publication than others.  Soundness was rated along a broad range from borderline to excellent, and excitement was rated as ambivalent in most cases.  Overall, the paper could benefit from stronger justification of its arguments and minor to moderate revisions, synthesizing the information (from authors) and feedback (from reviewers) that emerged during the discussion period with the manuscript.  As noted by the authors in their rebuttal, some of these revisions have already been made during the discussion period.

**Summary of Reviewer Feedback and Discussion:**
- **Reviewer UeN6** appreciated the insights arising from the paper's experiments, but felt that the experiments were not designed in such a way that they could demonstrate the manipulability of personality in more powerful LMs.  They also felt that the evaluation of this design was poorly justified and inadequately addressed biases that could be introduced during the experimental process, and they felt that the study's findings had limited reliability and generalizability.  They also raised one question pertaining to a figure.  In their rebuttal, the authors answered this question and clarified the intent of their research, which the reviewer had misunderstood.
- **Reviewer 9sqR** felt that the insights from this paper would be meaningful and helpful for constructing personalized applications and that the experiments were detailed and comprehensive.  However, they felt that it was unclear whether different questionnaires would lead to different conclusions and that an analysis of ChatGPT should have been included.  They also felt that the proposed method could be more comprehensively compared with other personality prediction methods, and asked whether the authors' datasets would be publicly available.  The authors clarified that the datasets would be publicly available, and explained that they had selected open-source models and that ChatGPT had been released after their work started.  They noted that in the future they could also compare to more recent models, and mentioned some general updates that they'd also made to revise the paper.
- **Reviewer RXZa** felt that the paper presented sound methods that were thoroughly analyzed and nicely motivated.  They appreciated the significant findings paired with detailed analyses, and noted that the contributed datasets will be important for the research community.  However, they pointed that the generalizability of the paper's findings to other types of computational models is known, and suggested further studying whether these findings were invariant to model type.  The authors reiterated from their statement to Reviewer 9sqR that further work is needed to asses generalizability, and also mentioned their general updates to the manuscript.
- **Reviewer gkMG** felt that the study was interesting and may be useful to numerous applications, and also appreciated the new datasets introduced in the paper.  However, they raised many points requiring clarification, and some concerns regarding the models used in the paper and the datasets themselves.  Specifically, they felt that the selected models were too old, raising concerns that the paper's conclusions may not carry over to more recent models, and they were concerned whether adequate quality control was performed during the dataset development process.  The authors provided clarifying responses to Reviewer gkMG's questions, and reiterated earlier points made to other reviewers about their selection of open-source models and the need for further studies to assess broader generalizability.  They noted that no quality control was performed since there was no objective way to determine text relevance to the topic of personality.  Reviewer gkMG thanked them for their rebuttal and clarified that their interest in quality control mainly pertained to toxic language, and suggested that if no quality control was performed, then a qualitative analysis should be performed on a random sample of the data as a post-hoc quality evaluation.  The authors agreed that this would be helpful, and they promised to add this in the final version of the paper.
Decision: Accept-Findings