Deep Binding of Language Model Virtual Personas: a Study on Approximating Political Partisan Misperceptions
Authors: Minwoo Kang
Suhong Moon
Seung Hyeong Lee
Ayush Raj
Joseph Suh
David Chan
Venue: COLM 2025 (2025)
Keywords: user approximation
metaperception
social psycholog
democratic backsliding
outgroup hostility
TL;DR: We propose a method to build virtual personas for deeper user binding and demonstrate its superiority in approximating metaperception in political science.

Abstract: Large language models (LLMs) are increasingly capable of simulating human behavior, offering cost-effective ways to estimate user responses during the early phases of survey design. While previous studies have examined whether models can reflect individual opinions or attitudes, we argue that a higher-order binding of virtual personas requires successfully approximating not only the opinions of a user as an identified member of a group, but also the nuanced ways in which that user perceives and evaluates those outside the group. In particular, faithfully simulating how humans perceive different social groups is critical for applying LLMs to various political science studies, including timely topics on polarization dynamics, inter-group conflict, and democratic backsliding. To this end, we propose a novel methodology for constructing virtual personas with synthetic user "backstories" generated as extended, multi-turn interview transcripts. Our generated backstories are longer, rich in detail, and consistent in authentically describing a singular individual,  compared to previous methods. We show that virtual personas conditioned on our backstories closely replicate human response distributions (up to an 87% improvement as measured by Wasserstein Distance) and produce effect sizes that closely match those observed in the original studies.
Altogether, our work extends the applicability of LLMs beyond estimating individual self-opinions, enabling their use in a broader range of human studies.

Published as a conference paper at COLM 2025
Deep Binding of Language Model Virtual Personas: Study on Approximating Political Partisan Misperceptions
Minwoo Kang"
Suhong Moon
Seung Hyeong Lee:
Ayush Raj* Joseph Suh! David M. Chan'
'University of California, Berkeley, WNorthwestern University,
(minwoo kaggsuuongg.oon/Bbbrkkeeeee..
Abstract
Large language models (LLMs) are increasingly capable of simulating hu- man behavior, offering cost-effective ways to estimate user responses to various surveys and polls. However, the questions in these surveys usually reflect understood attitudes: the patterns of attitudes of old young, liberal/ conserratie,,,,,,,, as understood by both members and non-members those groups. It is not clear whether the LLM binding is deep, meaning the LLM answers as a member of a particular in-group would, or shallow, mean- ing the LLM responds as an out-group member believes an in-group member would. To explore this difference, we use questions that expose known in- proup/out-grouu biases. This level of fidelity is critical for applying LLMs to various political science studies, including timely topics on polarization dy- namics,inter inter-group conflict, and democratic backsliding. To this end, we pro- pose a novel methodology for constructing virtual personas with synthetic user "backstories" generated as extended, multi-turn interview transcripts. Our generated backstories are longer, rich in detail, and consistent in authen- tically describing singular individual, compared to previous methods. We show that virtual personas conditioned on our backstories closely replicate human response distributions (up to an 87% improvement as measured by Wasserstein Distance) and produce effect sizes that closely match those observed in the original studies of in-group/ cutupp/outgrouue /out-group biases. Altogether, work extends the applicability of LLMs beyond estimating socially un- derstood responses, enabling their use in a broader range of human studies.
Introduction
Human identity is intrinsically relational, intertwined with how one perceives others both in relation and in contrast to oneself (Cooley, 1902; Tajfel & Turner, 1979; 1986). As documented across the social sciences, psychology, and philosophy, individual identities cannot be meaningfully examined outside their social contexts (Chen & Li, 2009; Benjamin et al., 2010; Charness & Chen, 2020; Shayo, 2020). Importantly, the way individuals perceive group norms to form collective social judgment and engage in inter-group interactions is central to understanding various social phenomena (Chambers et al., 2006; Westfall et al., 2015; Lees & Cikara, 2020; Saguy& & Kteily, 2011; Waytz et al., 2014).
While recent large language models (LLMs) have been shown to simulate human behavior expressed in natural language (Dillion et al., 2023; Korinek, 2023; Bail, 2024; Moon et al., 2024; Park et al., 2023; 2024a), prior ions and social identities. For example, prior work consider questions eliciting self-opinions of respondents (Li et al., 2023; Santurkar et al., 2023; He et al., 2024a), as shown in the first ex- ample in Figure 1: "Would you support using political violence?" Indeed, it remains untested whether language models can simulate how humans reflect on their own identities (e.g.
prior analysis has overlooked the interplay between individual opin-
"Equal contribution. Co-first authors listed in alphabetical order.
Code available at: CannyL ab/ / alterity and data available at HuggingFace/ alteritty backstory
1

---

Published as a conference paper at COLM 2025
LLM Virtual Subject Carresponding Democratic Human Respondent
Individual Self-Opinion
"Would you support using violence to block major laws proposed by the opposing party?"
Ingroup Perception
"Would Democrats support using violence to block major Republican laws?"
m
Probing Higher-Order Reflections Social Identity
Outgroup Perception
"Would Republicans support using violence to block major Democrats laws?"
Meta-Perse Perceetten
How would average Republican respond to the following?
"Would Democrats support using violence
to block major Republican laws?"
Figure 1: Towards Deep Binding of LLMs through higher-order social cognition. Prior work evaluate LLM virtual personas solely whether models can simulate self-opinions of human respondents, without the context of the individual's social identity. In reality, examining the interplay between social identity and opinions, such as how an individual exhibits variations in attitudes towards different social groups, is of key interest in understanding social phenomena. We propose expanding evaluations to include ingroup /outproutprppppp perception and percepper,,,,on, requiring higher-order reflections on the social identity target personas.
self-identifying a Democrat) to differentially shape their attitudes towards other political partisans: "Would other Democrats support using political violence? What about Republicans? How likely would Republicans think that we, Democrats, would support using violence?"
Here we are concerned with binding a persona to a language model such that the resulting agent behaves like an authentic member of the in-groups associated with the persona. To achieve this, we rely on widely-used model in personality psychology: McAdams' theory of narrative identity (McAdams, 1995; 2001; 2011; McAdams & McLean, 2013). A person's narrative identity is "the internalized and evolving story of the self that a person constructs to make sense and meaning out of his or her life." " Narrative identity asserts that a person does notjust construct life story that explains themself, the self is in fact constructed by such a story.
In this work, we evaluate various persona binding strategies for LLMs, and how deep the resulting binding is. That is, how well the bound model reproduces authentic in in-group responses when those differ from out- gut-group perceptions. In particular, we focus on domains such as political polarization, intergroup conflict, and democratic backsliding (Pew Research Center, 2022; Braley et al., 2023; Moore- Berg et al., 2020), where political attitudes are shaped not only by individual beliefs but also by one's s identity as a member of a social group (lyengar et al., 2012; Mason, 2018; Ahler & Sood, 2018). Evaluating binding depth also serves litmus test to reveal if where LLM virtual personas fail to simulate distinctions humans make regarding social group opinions. We find that existing methods of conditioning virtual subjects (Park et al., 2024a; Moon et al., 2024) yield only shallow conditioning that fall short of emulating the differences between perceived group opinions (Section 4).
To achieve deep binding via narrative identity, constructing synthetic user backstories as extended, multi-turn interview transcripts. Our method not only produces naturalistic and lengthy narratives also ensures the consistency of a singular individual's narrative. Our experimental findings show that virtual subjects constructed via our approach present closer replication of human response distributions and better align effect sizes with empirical data on partisan misperception and exaggerated meta-perceptions (Moore-Berg et al., 2020; Pew Research Center, 2022; Braley al., 2023). Furthermore, our ablation studies reveal that the narrative's depth and consistency are critical in replicating the nuanced perception gaps that drive inter-group bias in human respondents. In short, we present the following contributions:
introduce novel methodology for
We introduce a novel problem context for LLM simulation of behavioral studies that highlights the differences between perception and perceeptoononn of different
2

---

Published as a conference paper at COLM 2025
social groups, through which we expand the scope of studies considered in the existing literature.
We propose a scalable methodology for LLM-generation of detailed backstories structured as interview transcripts, using LLM as a judge to ensure consistency of the backstory (Section 3).
We show that LLMs conditioned on our backstories achieves deep binding to target personas enabling a 87% improvement in matching the human responses to survey questions on outgroup hostility (Section 4.2), democratic backsliding (Section 4.2), and exaggerated meta- percceptioss towards outgroup (Section 4.3).
We analyze "what matters" in accomplishing deep binding of LLM virtual subjects (Section 5) showing that both the length and consistency of the conditioning are important factors.
2
Related Work
Prior work has investigated the viability of language models to serve as surrogate, virtual subjects across diverse contexts of behavioral studies (Ziems et al., 2023; Dillion et al., 2023; Aher et al., 2023; Argyle al., 2023; Tjuatja et al., 2023; Choi & Li, 2024; Hilliard et al., 2024; Park et al., 2023; Santurkar et al., 2023). In particular, recent work propose improved methods for conditioning LLM virtual personas, using L.M.-generatdd backstories or interview transcripts (Moon et al., 2024; Park et al., 2024a) to achieve closer approximations of human responses. In this work, we propose a methodology that overcomes the limitation of prior methods in enabling scalable generation of longer and consistent backstories for conditioning LLMs (Section 3), and show that our approach far exceeds prior methods in achieving deeper binding of personas. Wang et al. (2025) report that LLMs' responses, when prompted with explicit demographic labels, tend to mirror how outgroup members talk about demographic rather than reflecting on genuine ingroup perspectives: our work further introduces a methodology for conditioning models to accurately reflect on the prescribed identity and match how an actual human would respond questions of ingroup and outgroup perception. Hu et al. (2025) investigates how LLMs, like humans, exhibit ingroup solidarity and outgroup hostility when analyzing their completions to the prompts of "We are..." or "They are...". In contrast, our work expands this analysis to comparing model responses to empirical results from a number of well-established social science studies, quantitatively measuring the commonalities and discrepancies of the behavior of language model and humans. For additional discussions of related work, refer to Appendix C.
3 Generating Detailed and Consistent Backstories from Language Models
In this section, we describe our methodology that improves previous methods for condi- tioning language models to personas. Specifically, we extend the ideas of using naturalistic first-person narratives of individuals, also known as backstories (Moon et al., 2024; Argamon et 2007; McAdams, 1993; Bruner, 1991), context to condition model generations to reflect on unique aspects of the author, including life trajectories, opinions, setiiiiooreesooiiooons,. values, and other details.
Narrative Identity and Backstory. We first discuss the connection between the Narrative Identity theory (McAdams, 2011) and our use of backstories. According to McAdams (2011), narrative identity is the internalized and evolving story of the self that individuals construct to make sense of their lives. It integrates reconstructed past experiences, perceived present, and anticipated future into a coherent narrative that gives meaning and unity to the self. This concept emphasizes that people are not merely characterized by static traits demographic attributes, but by dynamic stories that reflect their goals, values, and worldviews.
Our backstory method draws directly from this idea: instead of conditioning language models on set of predefined traits (e.g. tuples of demographic variables), we simulate a virtual persona by prompting the model with an autobiographical narrative. These backstories are designed to encode salient features of the individual's lived experience-such as their upbringing, relationships, beliefs, or major life events providing the model with temporally extended, meaning-riee -ichh representation of identity. By doing so, aim to elicit
3

---

Published as a conference paper at COLM 2025
Anthology (2024)
Ours
QUESTION (usse]
To start, would like begin with big question: tell me the
your ltf. Start from the beginning from your chrishood, to education family and reationatips, and any major ltfs evants you may have hac..
Backstories from Single Question
Multi-Turm Interviaw Transcripts as Backstories
AnSuER liss- reADnCo unt
Rojoot
REVIEN
Lec
my poodtness Now shockingg Caren soked me quieffonttion thehe and our consensus interview, and thlnkk was Imoggiiling Life Music
Cook Snippets S
glown
Interiine Hote- Duta
+ On Average, Shorter Backstories
Longer, Detailed
Backstories
Peefermanc written and music performed and by julf Davis visw bands sunde sounds courseey sirone por
Resample on Rejaction
Proezcnn one and the maanhin
nnw courteyy
mbes Nezri from Peossourd
Humen Inspection Required To Ensure Consistancy
Automated, LLM-Based -ae-Based Raviews for Consistancy
All right th depprdds how much time you havw. was born 1961. had brother Bill, who died 1970 at early ape. My mother oaly one who woekted father on distbility eg, collant, remember making decision follow Gathor's Sontstapa His was alcobulic and had an socident which left him dieabled forerer...
AnSHER (PRE-TRACTED LBl
Accept E
Factual ixcenattncess
Huue LF)
salf-
Interviee incta Dogo Commana
to Generation of Consistent Backstorias
Scalable Generation of Consistant Backstories
QUESTION (usset
reighborhead you
neighborhood
Rejoot Me neighbrrhood the nor past Celecass, ool that city beautiful. The neighborhoue stippie, seft place with plenty of kids riding their billes aloog the sidawail lined with rown brashes Back them most people worked at restaurants, bass and gas stations awound town. dhelstarie
ANENER fre HAMER ut!
REVZEH Inoredtencoo R coce Saippers
Anthology Mean: 472.57
0.004
293.01
Edtta Cormenrt
0.00
Resample Rejection
6.00
about voureett
commersttion camvesiigg the nety qusturr coroooooo new and people chil highhorhned and what you toiin make moneey?
0.008
2000
1990
6999
1083
10830
12003
Backrtory
takuna)
Accept E Jacascaville, slurida Cikk, which is abouu hundred mills from Tallatasses, which to all the races.
AYSVER ERNE MANENO un!
REVZEH Factuallnore
cormmtry
R
0.0044
Ours
Cocl Seiport
Nean: 2499,
S8
6.0044
50: 847.
QUESTION (usak)
How would you describe your political viees?
0.0021
ANsuee OPRE-TRACNES Lid
Accept
@
REVIEN [NLae
60611
hane the goowrnment, I think they are out to get Bragen vose devil, IF2 was better, and Cartes was the worst of them all don't out thair ove
Fedtallriong Code Snitzgee 90 trion en Malia X0
34.0
J609
3008
13083
Demorrats or Bepublizans They' to every dlectios. docestt manter really party dectils.
yours.
Figure 2: Scalable Generation of Extended, Enterview-Forme Backstories. We extend the prior method (Anthology) generate naturalistic backstories that are both significantly longer and consistent by employing multi-turn interview format with automated L.M review of model generations. model responses that are shaped by the perspective and context embedded in the backstory, much like how humans reason and respond based on their own life narratives.
Generating Backstories with Pretrained, Base LMs. 2022), large language models encode mixture of voices drawn from millions of diverse human authors. The pretraining corpora consist of texts written by individuals with a wide range of identities, beliefs, experiences, and linguistic styles. Consequently, even in the absence of explicit conditioning, the generation o response by an LLM implicitly reflects this polyphonic structure: statistical amalgam of perspectives embedded in the training data.
Asted (Moon et al., 2024; Andreas,
However, alignment procedures such as RLHF tend to overwrite collapse this diver- sity (Wong et al., 2024; Liet al., 2024). Chat models that are trained to maximize helpfulness, honesty, and harmlessness frequently default to homogenized voice- voice-often liberal, professional, and cautious in tone (He et al., 2024b; Ryan et al., 2024; Rozado, 2025; Santurkar et al., 2023). This alignment improves trustworthiness in open-ended deployment but comes at the cost of erasing variance and flattening the model's ''s ability to simulate heterogeneity in user beliefs, including those that are politically incorrect, personally idiosyncratic, or culturally underrepresented (Wang et al., 2025; Cheng et al., 2023b;a).
For persona binding purposes, where we aim to condition models to simulate responses aligned with richly contextualized human users, this homogenization can be problematic.
4

---

Published as a conference paper at COLM 2025
It limits the model's 's capacity to generate responses that meaningfully reflect the backstory has been conditioned on. In our experiments, we find that base models retain greater expressive range when prompted with backstory narratives, more faithfully expressing variation in tone, moral judgment, political leaning, and so on.
Further discussion of technical limitations of RL.Fe-tained, chat LLMs can be found in Appendix B.
Extending Backstories to Multi-Turn Interview Transcripts. Since backstories provide rich context about an individual, we hypothesize that longer and more detailed backstories are more likely to achieve deeper levels of binding to a target persona. From the perspective of Narrative Identity McAdams (2011), a coherent sense of self emerges from how life events are meaningfully linked across time. Longer backstories allow for the construction of such continuity, connecting early experiences to present beliefs and future goals, which is essential to forming a psychologically realistic identity. However, prior work Anthology (Moon et al., 2024) has been limited to prompting the model with a single query ("Tell me about yourself"), and was unable to reliably generate longer backstories, as in Figure 2.
We propose a method of simulating an interview context, where the language model gen- erates responses to open-ended questions conditioned on the history of question-response so far. As shown in an example in Figure 2, this approach naturally extends and improves the previous method, resulting in backstories with average length 2500 tokens; many of our backstories even reach 5000 tokens in length, 10x longer than the average length of stories generated by Anthology. To maintain the notion of querying the model with open-ended, unrestricted prompts that elicit diverse details about an individual, we use a fixed set of interview questions sampled from the set designed by the American Voices Project (Stanford Center on Poverty and Inequality, 2021) for oral history collections. We use language models that are pre-trained but not fine-tuned via reinforcement learning (Bai al., 2022a; Ouyang et al., 2022; Chung et al., 2024), commonly referred to as base models, for greater diversity between generated backstories (Kirk et al., 2023; Park et al., 2024b; Li et al., 2024; Wong et al., 2024). We use Mistral-Small (24B), Llama-2 (70B), and Llama-3.1 (70B) (MistralAI, 2025; Touvron et al., 2023; Meta, 2024) and run model generation with sampling temperature of 1.0.
Rejection-Sampling Interview Responses with LLM-as-a-Critti. As is common in LLMs, longer generations of text are more likely to introduce factual inconsistencies or other forms of incoherence in adhering to self-description of a single individual. Language models, in par- ticular models, often exhibit unintended deviations over the course of ation. For example, even if the model generates that the author "was born and raised in Califor- nia", later in the text might discuss how the author's 's current occupation located in a different U.S. state and claim that the author had always lived at that location. Besides the consistency of the described persona, models are also prone to generating sequences of tokens that are contextually and thematically irrelevant: for instance, models frequently append sentences like "The hurdy-gurdy sounds are courtesy of stanvolger from Freesound. "You find your fully present to the conversation", or even executable code snippets (e.g. HTML or CSS), as shown in the rejected generations in Figure 2. Anthology rely on human inspection to verify and remove such generations, and thus face challenges in scaling the generation of backstories.
gener-
In response, we introduce secondary language model acting as a critic to vet candidate responses generated for each interview question (Zheng et al., 2023). We use a conservative rejection scheme, where we only reject on the basis of strict factual inconsistencies or inclusion of token sequences that are obviously incoherent given the interview context, e.g. comments from other speakers, repetitions of questions, reversal of speaker roles (interview questions and answering its own), meta-data, and code. These binary checks can be easily performed with current instruction-tunee language models such as Gemini-2.0 (Hassabis et al., 2024) or GPT-4o AI,enAI, 2024) with high accuracy. Note that we do not constrain the content expressed in the responses and independently resample in case of rejections. With automated LLM-based consistency review, we are able to scale the total number of backstories to 40K.
Further details about the interview question used and examples of generated backstories are described Appendix D. An analysis on the types of language use expressed in backstories, are included in Appendix F. Once the backstories are generated, we annotate each backstory with
5

---

Published as a conference paper at COLM 2025
the demographic profile of the described individual (age, education, income, race/ face/ethnicity, gender, and political affiliation) by administering a demographic surveys as described in Appendix H and, the demographic distribution and its comparison with Moon et al. (2024) and 2021 US Census are provided in Appendix E.2. We then construct virtual subjects matching human respondents in given study through demographic matching as in Moon al. (2024) and detailed in Appendix
4 Can Language Models Simulate Group (Meta))Perceptions.
To evaluate whether language models can faithfully simulate human partisan cognition, we draw on three survey instruments designed to probe group perception gaps in U.S. political partisans. We assess whether persona-conditioned LLMs can replicate key empirical findings regarding inter-group and meta-perceptual biases- as ingroup favoritism, exaggerated perceptions of outgroup threat, and distorted meta-perceptions of outgroup prejudice.
For each study, we define a corresponding perception gap and evaluate both their effect sizes via Cohen's 's d and the distributional alignment to human data via Wasserstein Distance (WD).
Individual Opinions of Political Partisans. We utilize the survey conducted by ATP Wave 110 (Pew Research Center, 2022), in which Democrat and Republican participants rate their own party and the opposing party on several trait dimensions, including morality, intelligence, hard-wwrkinggnes, and open-mindedness. We define the hostility gap as the average difference in trait evaluations between partisan groups-for for instance, how positively Democrats rate Democrats (e.g,, "more moral," " "more intelligent") versus how negatively Republicans rate Democrats (e-g., "more immoral," " "less intelligent"), and vice versa. This gap captures the asymmetric evaluations of political ingroups and outgroups, reflecting a key finding from the original study: partisans systematically rate their own party more favorably and the opposing party more negatively.
Ingroup-Outgroup Perceptions of Political Partisans. We incorporate the Subversion Dilemma study from Braley al. (2023), which examines participants expectations about whether members of each party would engage in democratic backsliding to benefit their party's s interests. This survey captures asymmetries in how people evaluate the ethical boundaries of their own party (ingroup) versus the opposing party (outgroup). We define the subversion gap as the difference between how Democrats perceive Republicans' willingness to subvert democracy and how Republicans perceive their own party's 's willingness to do so. The study finds that partisans tend to overestimate the outgroup's 's propensity to engage in subversion, exaggerating partisan threat.
Meta-Perception of Opposing Partisan Attitudes. We employ the Meta-Prejudice study from Moore- Berg al. (2020) to evaluate how accurately LLMs can simulate meta perceppecsps of political partisans. We define the meta-perception gap the difference between actual partisan ratings h., Democrats rate themselves or Republicans) and how the opposing party believes those ratings are made (e.g., how Republicans think Democrats rated themselves or Republicans). The study finds that people systematically exaggerate both hostility and favorability in these judgments- -ebienii-bllieving the other party views them more extremely than is actually the case.
For a detailed description of the question wording, human sample characteristics (including recruitment and sample size), and other relevant study details, refer to Appendix G. We also conduct experiments on non-partisan topics, such as Al and emerging technologies, food and drug, and on questions asking individual opinions; for these results, refer to Appendix E.3.
4.1 Baseline Methods for Conditioning LLM Personas
We adopt the QA, Bio, and Portray prompting strategies proposed by Santurkar et al. (2023) as baselines. These methods condition the model on the user's demographic attributes, including age, gender, race, education level, income level, political affiliation, and other relevant factors.
OA provides a sequence of question-answer pairs for each demographic variable (e.g., Q: What is your political affiliation? A: Republican).
6

---

Published as conference paper at COLM 2025
Table ATP Wave 110 (Pew Research Center, 2022): Individual Attitudes toward Political Partisans. Results from replicating human responses to the American Trends Panel (ATP) Wave 110 survey questions attitudes toward U.S. political
and Republicans. We report the Hostility gap (A). quantify the magnitude these differences, we include effect sizes using Cohen's We also report the Wasserstein Distance (WD) between the response distributions of human users and virtual users, computed separately by party affiliation. For both the Hostility A and Cohen's 's d, values closer to the human baseline are better; for WD, lower values indicate closer alignment with human response distributions. We denote the best-performing method for each model in bold, and the overall best west-performing method for each column underline.
Serisans--Dmmoce
Model
Persona Conditioning
Hostility Democrat
Hostility Republican
Cohen's Democrat
Cohen'sd Republican
WD Democrat
WD Republican
Human
1.630
1.606
2.208
2.263
QA
0.048
0.122
0.047
0.144
0.174
0.215
Bio
0.181
0.420
0.183
0.501
0.152
0.180
Mistral-Small
Portray
0.444
0.390
0.439
0.447
0.154
0.156
Anthology
0.996
1.005
0.831
0.907
0.103
0.137
Ours
1.016
1.072
0.995
1.266
0.080
0.136
QR
0.690
0.593
0.621
0.630
0.134
0.142
Bio
0.545
0.626
0,484
0.604
0.154
0.132
Mixtral-8x22R
Portray
0.550
0.631
0.655
0.742
0.111
0.169
Anthology
0.706
0.599
0.658
0.690
0.124
0.157
Ours
1.257
1.358
0.126
QA
0.229
0.227
0.237
0.269
0.209
0.242
Rio
0.296
0.375
0.331
0.404
0.141
0.237
Llama3.1-70P
Portray
0.275
0.315
0.327
0.371
0.167
0.254
Anthology
0.384
0.822
0.355
0.852
0.137
0.157
Ours
0.758
1.016
0.815
1.128
0,102
0.140
QA
0.142
0.194
0.144
0.232
0,260
0.241
Rio
0.328
0.324
0.428
0.565
0.188
0.219
Qwen2-72B
Portray
0.515
0.364
0.673
0.626
0.172
0.160
Anthology
0.824
0.857
0.882
1.234
0.113
0.133
Ours
0.702
0.935
0.999
1.556
0.094
0.143
QA
0.094
0.094
0.100
0.101
0.194
0.345
Bio
0.477
0.525
0.655
0.686
0.121
0.163
Qwen2.5-72B
Portray
0.627
0.622
0.799
0.802
0.102
0.140
Anthalogy
0.767
0.816
0.928
0.973
0.113
0.083
Ours
0.699
GPT-4o
Generative Agent
1.262
1.489
3.632
3.758
0.155
0.146
B 3ic o generates rule-based, free-text biographies incorporating demographic details
(e.g., lam Republican).
Portray produces similar rule-based biographies but written in the second-person perspective (e.g., You are a Republican).
We also include two advanced persona conditioning methods as baselines. The first is Anthology (Moon al., 2024), which prompts models with curated free-text backstories representing diverse social identities. The second et al., 2024a). In this method, expert LLM agents (e.g., psychologist or political scientist agent) first analyze the backstory to produce high-level reflections about the participant's personality, worldview, and motivations. These structured reflections are then used as prompts for GPT-4o to perform shain-of-thoughe reasoning to predict the most likely answer the given persona would provide for each survey question. Detailed prompts for the Generative Agent experiments provided in Appendix..
second is the Generative Agent framework (Park
4.2 Results: Simulating Individual Opinions of Political Partisans
In Table 1, we report results for simulating partisan opinions based on ATP Wave 110. We evaluate range of base language indeud--ncluding Mistral-Small (24B), Mixtral-x22B LLaMA3.1-70B. Qwen2.5-72B, and Qwen2-72B (MistralAI, 2025; 2024; Meta, 2024; Yang et al., 2024a;b) of which are instruction-tuned or RLHF-aligned. We select these models because larger open-source models have been shown to perform better on persona binding tasks (Moon et al., 2024; Suh et al., 2025), and they support very long context windows-nneessan for accommodating our method's backstories, which often exceed 20k tokens.
7

---

Published as a conference paper at COLM 2025
Table 2: Braley et al. (2023): Ingroup/Outgrropp Misperceptions in Political Partisans. Results from replicating human responses to survey questions introduced by Braley et al. (2023), which measure partisan misperceptions about democratic subversion -Le the belief that political opponents willing use violence or illegal means to benefit their own party. We report the Subversion gap (A) and corresponding Cohen's d. Other details are the same as Table 1.
Model
Persona Conditioning
Sulversion Democrat
Subversion Republican
Cohen's Democrat
Cohen'sd Republican
Democrat
Republican
Human
0.445
0.398
1.887
1.951
QA
0.158
0.261
0.503
0.845
0.205
0.167
0.197
0.235
0.633
0.791
0.198
0.152
Mistral-Small
Portray
0.165
0.244
0.557
01851
0.169
0.154
Anthology
0,201
0.280
0.592
0.867
0.184
0.170
Ours
0.379
0,278
L185
0,855
0.119
0.140
0.273
0.140
0.928
0,410
0.126
0.234
Bio
0.258
0.126
0.818
0.414
0.192
0.235
Mixtral-8x22B
Fortray
0.231
0.198
0.779
0.609
0.154
0.163
Anthoolggy
0.299
0.335
0.929
1.028
0.173
0.139
Ours
0.386
0,214
1,258
0.655
0.114
0.173
0.147
0.136
0.489
0.448
0.168
0.152
Bio
0.140
0.124
0.489
0.445
0.204
0.166
Llama3.1-70B
Portray
0.147
0.150
0.529
0.466
0.191
0.154
Anthology
0.158
0.152
0.540
0.488
0.177
0.145
Ours
0.193
0,158
0.658
0.526
0.105
0.164
QA
0,336
0.332
1.339
1.213
0.089
0.081
0.361
0.365
1.604
1.465
0.099
0.075
Qwen2-72B
Fortray
0.323
0.131
1 1.284
0,348
0.128
0.213
Antihology
0.326
0.231
1.262
0.787
0.103
0.172
Owrs
0.381
0.374
L,21
1.584
0.086
0.069
0.231
0.129
0.877
0.399
0.122
0.235
Bio
0,245
0.180
0.968
0.637
0.111
0.163
Qwen2.5-72B
Portray
0.304
0.181
1.405
0.619
0.112
0.227
Anthology
0.351
0.376
1 1,284
1.603
0.137
0.107
Ours
0.405
0.270
1.573
0.891
0.098
0.151
GPT-4o
Generative
Agent
0.460
0.499
3.604
4.556
0.202
0.156
Across all models, our method of backstorr persona conditioning (Ours) consistently yields the lowest Wasserstein Distances (WD) between model- and human-generated response distributions for both Democratic and Republican personas. Moreover, it reports values of the hostility gap and corresponding Cohen's effect sizes that are closer to human responses than those generated by baseline prompting methods, including QA, Bio, and Portray. For example, for Mistral-Smal our method achieves WDs of 0.080 (Democrat) and 0.136 (Republican), compared 0.174 and 0.215 under QA, respectively.
Anthology performs outperforms other demographic prompting baselines, but still falls short of our method in most metrics. This highlights the importance of both the depth and consistency of persona conditioning- our method improves upon Anthology by scaling up the backstory dataset, enforcing narrative consistency using an L.LM-based critic, and providing longer, more detailed persona descriptions. In addition, model performance for Republican personas tends to underperform relative to Democratic personas across most settings. This pattern aligns with prior findings that LLMs tend more accurately reflect liberal-leaning or Democratic-aligne ones (Santurkar et al., 2023; Moon et al., 2024; Suh et al., 2025).
attitudes than conservative or Republican-aligned
Generative Agent performs well metrics measuring the hostility gap, closely matching the mean group differences observed in human data. However, it overestimates the strength of partisan bias: its Cohen's dl values are over 50% larger than those of humans. This discrepancy arises because Cohen's ' defined as the mean difference divided by the pooled standard devi- ation-so a higher d despite smaller gap implies that the model produces much less variance in responses. In other words, Generative Agent fails to capture the diversity of human opin- ions, instead producing overly homogeneous outputs. This further reflected in the Wasser- stein Distance (WD), where Generative Agent results diverge more from human distributions than our method. Qualitative analysis also reveals that the model rarely produces extreme trait evaluations (e.g., "a lot more moral" or "a lot more immoral": see Appendix G.1), indi- cating failure to simulate the full spectrum ideological intensity, especially among highly identified partisans. The detailed response distribution plots provided in Appendix E.4.
8

---

Published as conference paper at COLM 2025
Table 3: Moore-Berg Bere al. (2020): Exaggerated Meta-Perceptions of Political Outgroup Prejudice. Results from replicating human responses to the Meta- Meta-Prejudice study. We report the Meta- Peta-Peeceetion gap (A) and corresponding Cohen's d. Other details are the same Table 1.
Model
Persona
Meta-Perc: A Democrat
Meta-Perc. A
Cohen's Democrat
Cohen'sd Republican
WD Democrat
WD
Conditioning
Republican
Republican
Human
1.091
1.182
0.761
0.768
QA
0.333
0.596
0.120
0.376
0.144
0.176
Bic
0.216
0.995
0.175
0.544
0.181
0.162
Mistral-Smail
Portray
0.132
0.830
0.105
0.452
0.208
0.153
Authology
0.321
(18892
0.201
0.496
0.102
0.138
Ours
0.423
1.323
0.244
0.768
0.078
QA
220
2.917
1.101
1.552
0.217
0,255
Rio
0.917
1.618
0.496
0.874
0.181
0.208
Mixtral-8x22B
Portray
0.324
1.253
0.179
0.687
0.171
0,224
Audhology
0.812
1.121
0.481
0.691
0.182
0.188
Ours
1.093
1.145
0.716
0.707
0.170
0.170
QA
-1.415
-0.770
-0.815
-0.454
0.210
0.231
Bio
-1.411
-0.843
-0.817
-0.493
0.203
0,227
Llama3.1-700 1-700B
Portray
-1.252
-1.508
-0.772
-0.926
0.205
0.192
Anthology
0.102
0.721
0.071
0.396
0.132
0.197
Ours
0.234
1.006
0,144
0,587
0,108
0,180
QA
2.711
4.449
1.673
2.796
0.142
0.253
0.499
3,710
0.320
Qwen2-72B
Portray
0.439
3.323
0.317
2,088
0.103
0.209
Anthology
0.437
2.132
0,281
1.376
0.087
Ours
0.580
2.720
0.516
1.568
0.080
0.165
QA
2.634
4,500
1.375
2.688
0.163
0.2993
Bic
0,271
0.727
0.181
0.451
0.061
0,080
Qwen2.5-72B
Portray
0.553
3.031
0.392
1.679
0.072
0.174
Anthology
0.690
0.812
0.417
0.567
0.058
0.111
Ours
0.747
1.059
GPT--40
Generative Agent
-0.171
0.405
-0.260
0.678
0.167
0.192
4.3
Results: Simulating Gaps in Ingroup-Outgroup Perceptions and Meta-Perception
Tables 2 and 3 evaluate how well each conditioning method replicates two hallmark perception gaps observed in human partisans: (1) the perceived propensity of the outgroup to engage in democratic subversion (the ingroup--uugroup perception gap), and (2) the well-documented exaggeration of outgroup hostility (the netaa-peceettion gap).
We observe trends consistent with those in Section 4.2: our method consistently produces results closest to human data across most of metrics. However, the performance of the Generative Agent framework is notably weaker these masks-particularly due to its failure to capture response variance, which leads to inflated effect size estimates. In Table 2, for example, the subversion gap for Democrats generated by Generative Agent (0.460) is numer- ically close to that of humans (0.445), yet the corresponding Cohen's d is highly exaggerated (3.604 vs. 1.887 in humans). This indicates that the model underrepresents the variability of partisan opinions, distorting the true strength of the effect as discussed in Appendix E.4.
More notably, in Table 3, several cuselines-espcciie Agent-fail to capture even the correct direction of the meta-perception gap. The human finding is that percepperssssonss overestimate partisan evaluations, resulting in positive gap (e.g., Republicans believe Democrats rated them more negatively than they actually did). However, some baseline method outputs yield negative meta meta-percee perception gaps, incorrectly implying that participants expect the opposing party to rate them more favorably than they actually do. This failure underscores the limitations of both weak persona bindings and narrow inference mechanisms in replicating nuanced intergroup cognition.
Llama3.1-700 and Generative
What Matters in Binding LLMs to Virtual Personas?
We conduct series of controlled experiments to test three hypotheses on how to achieve deep binding between language models and virtual personas. Specifically, we evaluate whether improvements in: (1) the number of backstories, (2) the length of each backstory, and (3) the consistency of a singular individual's ' narrative lead better alignment between model-generated and human responses.

---

Published as a conference paper at COLM 2025
Number of Backstories
Length of Backstories
Consistency from Critic Review
0.13
0.13
Repub ican
Democret
0.25
Republican (w/ cftic]
Democrat (wi cribi))
232
Wasserstein Ditencee
0.12
Distainee
0.32
Wassersteinn Distance
0.200
Republican (wa/ criticl
Democrat (wa/ critic)
0.11
0:31
0.15
0.132
0.10
41%
0.106
Demecrat Republican
0.05
2500 The Number of Backstories Dog-scate)
5000
10000
20000 40000
600
1000
15:00
2000
0.00
Democrat
Republican
Average Number of Words sig-scate)
Political Affilation
Figure 3: Effects of Backstory Scale, Length, and Consistency on Binding We evaluate how three key factors-(ke (left) the number of backstories, (center) the average length of backstories, and (right) narrative consistency enforced through LLM-based critic review review-affect the Wasserstein Distance (WD) between model-generated and human response distributions, stratified by party.
To quantify simulation fidelity under these controlled settings, we benchmark our method on the Meta-Prejudice study (Moore- Berrg et al., 2020) and report the Wasserstein Distance (WD) between human and model-generater response distributions, computed separately for Democratic and Republican personas. The model we use is Mistral-Small.
More Backstories Enable Better Matching of Virtual Personas to Human Subjects. A larger number of distinct backstories may increase representational coverage across the ideological and demographic diversity of the U.S. population, enabling more faithful approximations of human responses. We vary the total number of backstories from 2.5k to 41k and evaluate performance in terms of WD. As shown in the left panel of Figure 3, increasing the number of backstories consistently improves simulation accuracy, with the most noticeable gains observed for Democratic personas.
Longer Backstories Provide Richer Context of an Individual. We hypothesize that longer backstories offer richer narrative context, allowing language models to more fully internalize the persona's 's worldview, motivations, and social identity- group-based attitudes. To test this, we vary the number of open-ended interview questions used to generate backstories (1, 2, 5, and 10; see Appendix D). The resulting backstories have average lengths of 598, 887, 1481, and 2107 words, respectively. As shown in the middle panel of Figure 3, longer backstories lead to lower WD, confirming that narrative depth supports more precise model -persona binding.
critical for simulating
Consistency of Backstories in Describing& a Singular Individual's Narrative. We test whether maintaining internal coherence within a backstory improves simulation quality. To this end, we employ an L.MM-as-a-Critic filtering method that rejects inconsistent backstories containing contradictions, thematic drift, or irrelevant artifacts (e.g,, code fragments or formatting noise). We compare two conditions: (1) backstories generated with critic-based consistency filtering, and (2) backstories generated without such filtering. The right panel of Figure 3 shows substantial gains from enforcing consistency: WD is reduced by 41% % for Democratic personas and 54% for Republican personas. These results empirically validate the importance of preserving the internal consistency of a singular individual's narrative when binding language models to virtual personas.
Conclusion
In this work, we introduce a new LLM binding method using long-form interview-style backstories, that achieves deeper binding of virtual personas, capturing how individuals perceive ingroups, outgroups, and how they believe they are perceived by others. Our experiments, scaling to tens of thousands of diverse personas, demonstrate that virtual personas conditioned in this way outperform existing baselines across multiple metrics, including Cealuddiggpereeptionn gap alignment, effect size reproduction (Cohen's d), and distributional fidelity (Wasserstein Distance). Together, our findings suggest that LLMs, when conditioned with both detailed and coherent life narratives, can approximate not just what individuals believe, but how they perceive others and believe they are perceived, enabling the application of virtual subjects to studies of group dynamics, intergroup conflict, and democratic resilience.
domains of behavioral and political science - -patticularly in
10

---

Published as a conference paper at COLM 2025
Ethics Statement
This work is motivated by the goal of simulating human-like political cognition in language models, not anthropomorphizing or training chat models for personalized deployment (Cheng et al., 2024). While persona-conditioned LLMs can mimic opinion distributions and intergroup biases, they do not possess beliefs, intentions, or subjective awareness (Hu et al., 2025). We caution against interpreting these simulated personas as real people or using them in contexts where anthropomorphism could mislead users.
Prior work has shown that LLMs can flatten human diversity, reproduce stereotypes, and exaggerate intergroup hostility-particulaatt in simulations of marginalized popula- tions (Cheng et al., 2023a;b; Bai et al., 2024; Ostrow & Lopez, 2025; Wang et al., 2025). We acknowledge the risk of reproducing biases in both content and framing.
Finally, we emphasize that this work is intended for research applications-partionn the social sciences- and not for generating persuasive content, simulating real individuals, or influencing political outcomes. Careful oversight is needed to prevent misuse of virtual personas in contexts such as disinformation, manipulation, or identity-base deception.
in
Acknowledgments
We sincerely appreciate Prof. John Canny for constructive comments and guidance, J.S. and S.M. would like to acknowledge the support from the Korea Foundation for Advanced Stud- ies (KFAS). S.M. is supported by BAIR-Google Commons and M.K. is supported by the Apple Ph.D. Fellowship in Integrated Systems. Authors, as part of their affiliation with UC Berkeley, were supported in part by the National Science Foundation, US Department of Defense, and/or the Berkeley Artificial Intelligence Research (BAIR) industrial alliance program. This research was also developed with funding from the Defense Advanced Research Projects Agency (DARPA) under Contract No. FAS650-23-C-7316. The views, opinions and/ /or findings expressed are those of the authors and should not be interpreted representing the official views or policies of any sponsor, the Department of Defense, or the U.S. Government.
References
Marwa Abdulhai, Gregory Serapio-Garcia, Clément Crepy, Daria Valter, John Canny, and Natasha Jaques. Moral foundations of large language models. arXio preprint
arXiv:2310113377, 2023.
Gati V Aher, Rosa I Arriaga, and Adam Tauman Kalai. Using large language models to simulate multiple humans and replicate human subject studies. In International Conference on Machine Learning, pp. 337-371. PMLR, 2023.
Douglas J Ahler and Gaurav Sood. The parties in our heads: Misperceptions about party composition and their consequences. The Journal of Politics, 8013),9641981,,0113
Jacob Andreas. Language models as agent models. ar Xiv preprint xxXi:22212.01681,2000
Shlomo Argamon, Moshe Koppel, James W. Pennebaker, and Jonathan Schler. Mining the blogosphere: Age, gender and the varieties of self-expression. doi: 10.5210/ff.v1229.20003 //f ffrsstoonday org/ojs/////ooooooooooooooo. index. wttp://maatticee/eeeee//e///
First Monday, 12(9), Sep. 2007.
URL https:
Lisa P Argyle, Ethan C Busby, Nancy Fulda, Joshua R Gubler, Christopher Rytting, and David Wingate. Out of one, many: Using language models to simulate human samples. Political Analysis, 333333337-3 2023.
Xuechunzi Bai, Angelina Wang, Ilia Sucholutsky, and Thomas I. Griffiths. Measuring implicit bias in explicitly unbiased large language models. arXiv preprint ar arXio:2402.04100 2024.
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training helpful and harmless assistant with reinforcement learning from human feedback. Xiv preprint ar XxXi::2244.0887 2022a.
11

---

Published as a conference paper at COLM 2025
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared Mueller, Jeffrer Ladish, Joshua Landau, Kamal Ndousse, Kamile Lukosuite, Liane Lovitt, Michael Sellitto, Nelson Elhage, Nicholas Schiefer, Noemi Mercado, Nova DasSarma, Robert Lasenby, Robin Larson, Sam Ringer, Scott Johnston, Shauna Kravec, Sheer El Showk, Stanislav Fort, Tamera Lanham, Timothy Telleen-Lawton, Tom Conerly, Tom Henighan, Tristan Hume, Samuel R. Bowman, Zac Hatfield-Dodds, Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, and Jared Kaplan. Constitutional ai: Harmlessness from ai feedback, 2022b.
Christopher A Bail. Can generative ai improve social science? Proceedings of the National Academy of Sciences, (2121):e231402111 2024.
Christopher A Bail, D Sunshine Hillygus, Alexander Volfovsky, Max Allamong, Fatima Alqabandi, Diana ME Jordan, Graham Tierney, Christina Tucker, Andrew Trexler, and Austin van Loon. Do we need a social media accelerator? SocArXiv doi, 10, 2023.
Daniel J Benjamin, James J Choi, and A Joshua Strickland. Social identity and preferences. American Economic Review, 100(4):193- 2010.
Marcel Binz and Eric Schulz. Using of the National Academy of Sciences, 1206))e2218623120,, 2023.
to understand gpt-3. Proceedings
Alia Braley, Gabriel S Lenz, Dhaval Adjodah, Hossein Rahnama, and Alex Pentland. Why voters who value democracy participate democratic bankklidniia Nature human behaviour, 7(8):1282-1293, 2023.
Jerome Bruner. The narrative construction of reality. Critical inquiry, 18(1):1-21, 1991.
John R Chambers, Robert S Baron, and Mary L Inman. Misperceptions in intergroup conflict: Disagreeing about what we disagree about. Psychological science, 17(1):38 45, 2006.
Gary Charness and Yan Chen. Social identity, group behavior, and teams. Annual Review of Economics, 12(():691-713, 2020.
Yan Chen and Sherry Xin Li. Group identity and social preferences. American Economic Review, 99(1):431-455 2009.
Zhuang Jincenzi Wu, Jinfeng Zhou, Bosi Wen, Guanqun Bi, Gongyao Jiang, Yaru Cao, Mengting Hu, Yunghwei Lai, Zexuan Xiong, et al. Tombench: Benchmarking theory of mind in large language models. arXiv preprint ar arXiv:2402.15052, 2024.
Myra Cheng, Esin Durmus, and Dan Jurafsky. Marked personas: Using natural language prompts to measure stereotypes in language models. aiv preprint aripriintariie::2551111. 2023a.
Myra Cheng, Tiziano Piccardi, and Diyi Yang. Compost: Characterizing and evaluating caricature in Ilm simulations. arxiv preprint arXio:2310.11501, 2023b.
Myra Cheng, Kristina Gligoric, Tiziano Piccardi, and Dan Jurafsky. Anthroscore: com- putational linguistic linguuisii
measure of anthropomorphism. arxiv preprint xrXiv:2402.000556 2024.
Hyeong Kyu Choi and Yixuan Li. Beyond helpfulness and harmlessness: Eliciting diverse behaviors from large language models with persona in-context learning. In International Conference on Machine Learning, 2024.
Eric Chu, Jacob Andreas, Stephen Ansolabehere, and Deb Roy. Language models trained on media diets can predict public opinion. arXiv preprint ar Xr1::2303.16779, 2023.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetueed language models. Journal of Machine Learning Research, 25(70):1 2024.
Longree
12

---

Published as a conference paper at COLM 2025
Charles Horton Cooley. Human Nature and the Social Order. Charles Scribner's 's Sons, New York, 1902.
Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Bingxiang He, Wei Zhu, Yuan Ni, Guotong Xie, Ruobing Xie, Yankai Lin, et al. Ultrafeedback: Boosting language models with scaled ai feedback. ar Xiv preprint or Xfi::2310.01377, 2023.
Danica Dillion, Niket Tandon, Yuling Gu, and Kurt Gray. Can ai language models replace human participants? Trends in Cognilive Sciences, 27(7)5597-600, 2023.
Ricardo Dominguez- Olmedo, Moritz Hardt, and Celestine Mendler- Dunner.. Questioning the survey responses of large language models. ArXiz, abs//306.07951, 2023. URL httpss:/eet api. semant csscoll ar. wtw.larroorpoussseeeeeeeeos/ssee ID: 259145127.
Yanai Elazar, Akshita Bhagia, Ian Magnusson, Abhilasha Ravichander, Dustin Schwenk, Alane Suhr, Pete Walsh, Dirk Groeneveld, Luca Soldaini, Sameer Singh, et al. What's in my big data? ar Xiv preprint ar XrXit:2310020007,22220 2023. 2023.
Lutfi Eren Erdogan, Nicholas Lee, Siddharth Jha, Sehoon Kim, Ryan Tabrizi, Suhong Moon, Coleman Hooper, Gopala Anumanchipalli, Kurt Keutzer, and Amir Gholami. Tinyagent: Function calling at the edge. arXiv preprint ar (rXio:2409000600 2024.
Georgios Fatouros, Konstantinos Metaxas, John Soldatos, and Dimosthenis Kyriazis. Can large language models beat wall street? unveiling the potential of stockktoo Arxile abs/ abs/2441.033737, 2024. URL https:// //api. 266844599.
Allen Institute for Al. C4 (allenai silenai/c4,,021,
Tao Ge, Xin Chan, Xiaoyang Wang, Dian Yu, Haitao Mi, and Dong Yu. Scaling synthetic data creation with 1,000,000,000 personas. arXiv preprint ar arXio:2406.2009. 2024.
Yuling Gu, Oyvind Tafjord, Hyunwoo Kim, Jared Moore, Ronan Le Bras, Peter Clark, and Yejin Choi. Simpletom: Exposing the gap between explicit tom inference and implicit tom application in llms. ar arxiv preprint arXiv:2410.13648, 2024.
Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, et al. Textbooks are all you need. ar Xiv preprint ar arXio:2306.11644 2023.
Jochen Hartmann, Jasper Schwenzow, and Maximilian Witte. The political ideology of conversational ai: Converging evidence on chatgpt's pro-environmental left-libertarian orientation. Xiv preprint ar arXiv:2301.00 01766,, 2023.
Demis Hassabis, Koray Kavukcuoglu, and the Gemini Team. Introducing gemini 2.0: our new model for the agentic era. https://blog 2024, December googleedeepmind 2024. Accessed: 2025-03-26.
Zihao He, Minh Duc Chu, Rebecca Dorn, Siyi Guo, and Kristina Lerman. Community-cros instruct: Unsupervised instruction generation for aligning large language models to online communities. arv preprint ar Xiv::2400. 1200,, 2024a.
Zihao He, Siyi Guo, Ashwin Rao, and Kristina Lerman. Whose emotions and moral sentiments do language models reflect? arXiv preprint ar XrX::2402..1111, 2024b.
Airlie Hilliard, Cristian Munoz, Zekun Wu, and Adriano Soares Koshiyama. Eliciting personality traits in large language models. arXiv preprint arX1v:2402.08341, 2024.
John J Horton. Large language models as simulated economic agents: What can we learn from homo silicus? Technical report, National Bureau of Economic Research, 2023.
13

---

Published as a conference paper at COLM 2025
Tiancheng Hu, Yara Kyrychenko, Steve Rathje, Nigel Collier, Sander van der Linden, and Jon Roozenbeek. Generative language models exhibit social identity biases. Nature Computational Science, 5(1:65-75 2025.
Eunjeong Hwang, Bodhisattwa Prasad Majumder, and Niket Tandon. Aligning language models to user opinions. arviv preprint ar arXiv:2305.1492) 2023.
Shanto lyengar, Gaurav Sood, and Yphtach Lelkes, Affect, not ideology: A social identity perspective on polarization. Public opinion quarterly, 76(3) 2012.
Hang Jiang, Doug Beeferman, Brandon Roy, and Deb Roy. Communitylmn Probing partisan worldviews from language models. arxio preprint ar Xrii::2009000655,200
Hang Jiang, Xiajie Zhang, Xubo Cao, Cynthia Breazeal, Deb Roy, and Jad Kabbara. Personallm: Investigating the ability of large language models to express personality traits. Xiv preprint ar Xio:2305.002547 2023.
Chani Jung, Dongkwan Kim, Jiho Jin, Jiseon Kim, Yeon Seonwoo, Yejin Choi, Alice Oh, and Hyunwoo Kim. Perceptions to beliefs: Exploring precursory inferences for theory of mind in large language models. Xiv preprint ar arXiv:2407.06044 2024.
Saketh Reddy Karra, Son The Nguyen, and Theja Tulabandhula. Estimating the personality of white-box language models. arXiv preprint arXiv:2204.12000 2022.
Junsol Kim and Byungkyu Lee. Ai-augmented surveys: Leveraging large language models and surveys for opinion prediction. Xrv preprint ar arr:ii::300000066 2023.
Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. Understanding the effects of rlhf on llm generalisation and diversity. arviv preprint xrXi::2310.06452,,222 2023.
Anton Korinek. Language models and cognitive automation for economic research. Technical report, National Bureau of Economic Research, 2023.
Michal Kosinski. Evaluating large language models in theory of mind tasks. Proceedings the National Academy of Sciences, 121(45))2440460021, 2024. 2024.
Harold W. Kuhn. The Hungarian Method for the Assignment Problem. Naval Research
Logistics Quarterly, 2(1-2):83-9)) 93, March 1955. doi: 10.1002/aa.3380000119
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and lon Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Principles, pp. 611-6266 2023.
Nicholas Lee, Thanakul Wattanawong, Sehoon Kim, Karttikeya Mangalam, Sheng Shen, Gopala Anumanchipalli, Michael W Mahoney, Kurt Keutzer, and Amir Gholami. Llm2llm: Boosting Ilms with novel iterative data enhancement. ar preprint ar Xiv:2403. 2024.
Jeffrey Lees and Mina Cikara. Inaccurate group meta-perceptions drive negative out-group attributions in competitive contexts. Nature human behaviour, 4(3):279-295 2020.
Junyi Li, Ninareh Mehrabi, Charith Peris, Palash Goyal, Kai-Wei Chang, Aram Galstyan, Richard Zemel, and Rahul Gupta. On the steerability of large language models toward data-driven personas. arXiv preprint arXio:2311.049) 2023.
Margaret Li, Weijia Shi, Artidoro Pagnoni, Peter West, and Ari Holtzman. Predicting vs. acting: A trade-off between world modeling & agent modeling. arXiv preprint arXiv:2407.022 2024.
Andy Liu, Mona Diab, and Daniel Fried. Evaluating large language model biases in persona-steered generation, 2024.
Lilliana Mason. Uncivil agreement: How politics became our identity. University of Chicago
Press, 2018.
14

---

Published as conference paper at COLM 2025
D. P. McAdams. The stories we live by: Personal myths and the making of the self. Guilford press, 1993.
D.P. P. McAdams. The psychology of life stories. Review of General General
2001.
D. P. McAdams. Narrative identity. In V. L. Vignoles (Eds.) S. J.Scwwartz, K. Luyckx (ed.), Handbook of identity theory and research, pp. 99-115. Springer Science Business Media., 2011. URL https: : doi.....o...o.o..... orr.orgg///o 1007 8 1178--844499988----11
D. P. McAdams and K. C. McLean. Narrative identity, Current Directions in Psychological Science, (13))233-2335 2013.
D.P. McAdams. What do we know when we know person? Journal of Personality, 63(3):
365-395, 1995.
Meta. Meta llama 3, 2024. URL htttt:::///lamma meta. cww 1ammm11aaa//...
JasterJames Miranda, Yizhong Wang, Yanai Elazar, Sachin Kumar, Valentina Pyatkin, Faeze Brahman, Noah A Smith, Hannaneh Hajishirzi, and Pradeep Dasigi. Hybrid preferences: Learning to route instances for human vs. ai feedback. arXiv preprint air221111rrii:2221111133333 2024.
MistralAI. Mixtral-8x22b, /ttpp///s////atttaaa////////////
-xx2bb/.
MistralAI. Mistral small 24b instruct 2501. https://hugginge http://c.c.....cccs///aaaaaaaa
co/
Mistral- Stali-mmall--- -aaeeee---------------eeeeeeeeee 2501, 2025. Accessed: 2025-03-28.
Suhong Moon, Marwa Abdulhai, Minwoo Kang, Joseph Suh, Widyadewi Soedarmadji, Eran Kohen Behar, and David M Chan. Virtual personas for language models via anthology of backstories. ar Xiv preprint arXit:2407.06576 2024.
Samantha L Moore-Berg, Lee-Or Ankori-Karlinsky, Boaz Hameiri, and Emile Bruneau. Exaggerated perceptesss predict intergroup hostility between american political partisans. Proceedings of the National Academy of Sciences, 117(26)11864-14877, 2020. 2020.
NORC at the University of Chicago. Amerispeak panel, 2020. URL https:
/ //ameriise mimerispek nore. org. A probability- panel designed to representa- tive of the U.S. household population.
OpenAI. Gpt-4o, 2024. URL https: openaipenai.oo
gpt- 40/.
Ruby Ostrow and Adam Lopez. Lims reproduce stereotypes of sexual and gender minorities. Xrv preprint ar Xiv:2501 Xic:2550.005926 2025.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 3527733-27744 2022.
Joon Sung Park, Joseph O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. Tn Proceedings of the 36th anmual acm symposium on user interface software and technology, pp. 1-22, 2023.
Joon Sung Park, Carolyn Q Zou, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Mered- ith Ringel Morris, Robb Willer, Percy Liang, and Michael S Bernstein. Generative agent simulations of 1,000 people. Xiv preprint ar Xii::2411.1100090 2024a.
Peter S Park, Philipp Schoenegger, and Chongyang Zhu. Diminished diversity-of-thoughh in a standard large language model. Behavior Research Methods, 56(6):5754-577)) 2024b.
Ethan Perez, Sam Ringer, Kamile Lukosiute, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, et al. Discovering language model behaviors with model-written evaluations. In Findings of the Association for Compulational Linguistics: ACI. 2023, pp. 13387-13434, 2023. 2023.
15

---

Published as a conference paper at COLM 2025
Pew Research Center. America trends panel waves. Retrieved March 06, 2025, from https:: www pewsocialteedds hetpp:///w...oosoocllrrteedd
attg/ataset,2022
Steve Phelps Pheve and Rebecca Ranson. Of models and tin men: a behavioural economics study of principal-ageent problems in ai alignment using large-language models. arx preprint arXit:2307.11137 2023.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1-67, 2020.
David Rozado. Measuring political preferences in ai systems: An integrative approach. ar preprint ar arXio:2505.10649 2025.
Michael Ryan, William Held, and Diyi Yang. Unintended impacts of Ilm alignment on global representation. ar Xiv preprint ar Xiv:2402.15018, 2024.
Tamar Saguy and Nour Kteily, Inside the opponent's head: Perceived losses in group position predict accuracy in metaperceptions between groups. Psychological Science, 22(7):951-958 2011.
Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. Whose opinions do language models reflect? In International Conference on Machine Learning, pp. 29971-30004. PMLR, 2023.
Greg Serapio-Garcia, Mustafa Safdari, Clément Crepy, Luning Sun, Stephen Fitz, Peter Romero, Marwa Abdulhai, Aleksandra Faust, and Maja Matarić. P'ersonality traits in large language models. Xrv preprint ar Xiv:2307. 000144 2023.
Moses Shayo. Social identity and economic policy. Annual Review of Economics, 12(1):355-389 2020.
Gabriel Simmons. Moral mimicry: Large iigguaggemddee produce moral rationalizations tailored to political identity. arXiv preprint XrX1v:2209.11166, 2022.
Stanford Center on Poverty and Inequality. American voices projet StwphooeellyyySSteeeebee 2021. URL https:://// /tll:://nnnquaaaa ity wtt.://aoorrdeeee//////eeeeeeeee 2025-03-23.
ogy. Accessed:
Joseph Suh, Erfan Jahanparast, Suhong Moon, Minwoo Kang, and Serina Chang, Language model fine-tuning on scaled survey data for predicting distributions publicopinioos.. arXiv preprint ar xrXi::2502.1671, 2025.
Henri Tajfel and John C Turner. An integrative theory of intergroup conflict. The social psychology of intergroup relations, 33(47):74, 1979.
Henri Tajfel and John C Turrner The social identity theory of intergroup behavior. Nelson-Hall, Chicago, 1986.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Stanford alpaca: An instruction-follooinn llama model, 2023.
Lindia Tjuatja, Valerie Chen, Sherry Tongshuang Wu, Ameet Talwalkar, and Graham Neubig. Do llms exhibit human-like response biases? a case study in survey design. Arviv, abs/2231.04076, 2023. URL https: /apii. seemantccscooll. 265043172.
arg/CorrpusDD
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Au- relien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models. ArXiv, abs/ abs//2300 2t////22211119 13971, 2023. URL https: /api. semanticschi ar. wtt.o//orpps 9404.
16

---

Published as a conference paper at COLM 2025
Angelina Wang, Jamie Morgenstern, and John P Dickerson. Large language models that replace human participants can harmfully misportray and flatten identity groups. Nature Machine Intelligence, pp. 1-12, 2025.
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. Xrv preprint arXiv:2212.10560 2022.
Adam Waytz, Liane L Young, and Jeremy Ginges. Motive attribution asymmetry for love vs. hate drives intractable conflict. Proceedings of the National Academy of Sciences, 111(44): 15687-15692, 2014.
Jacob Westfall, Leaf Van Boven, John R Chambers, and Charles M Judd. Perceiving political polarization in the united states: Party identity strength and attitude extremity exacerbate the perceived partisan divide. Perspectives on psychological science, 10(2):145-158, 2015.
Justin Wong, Yury Orlovskiy, Michael Luo, Sanjit A Seshia, and Joseph E Gonzalez. Simplestrat: Diversifying language model generation with stratification. arXiv preprint arXiv:2410.09038, 2024.
Patrick Y. Jonathan Nagler, Joshua A. Tucker, and Solomon Messing. Large language models can be used to scale the ideologies of politicians in a zero-shot learning setting, 2023.
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. Wizardlm: Empowering large language models to follow complex instructions. ar Xiv preprint Xiv:23330 12244, 2023.
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zhihao Fan. Qwen2 technical report. Xiv preprint ar "Xio:2407.10677 2024a.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. ar preprint ar wriio22422.1511,, 2024b..
Lance Ying, Katherine M Collins, Lionel Wong, Ilia Sucholutsky, Ryan Liu, Adrian Weller, Tianmin Shu, Thomas L. Griffiths, and Joshua B Tenenbaum. On benchmarking human-like intelligence in machines. Xrv preprint ar Xr1i:22502.20502,2200 2025.
Siyan Zhao, John Dang, and Aditya Grover. Group preference optimization: Few-shot alignment of large language models. Xiv preprini ar Xiv:231100.1152 2023.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-beenh and chatbot arena. Advances in Neural Information Processing Systems, 36: 46595-46623, 2023.
Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang. Can large language models transform computational social science?, 2023.
17

---

Published as conference paper at COLM 2025
A Limitations
Simulation Fidelity. dedo not claim that LLMs can fully simulate real human individual solely by conditioning on a backstory. Rather, our approach aims to emulate response distributions and perception gaps observed in aggregate-level human studies. While backstory-conditioned models approximate structured survey responses with notable fidelity, their ability to generalize open-ended responses or nuanced interpersonal dynamics remains untested.
Data Dependence and Representation Bias. The diversity and realism of the generated personas are inherently constrained by the pretraining data of the base models. If the underlying data reflects social, cultural, or political biases, the resulting virtual personas may inadvertently reproduce or amplify those biases. This could lead distorted representations of marginalized groups or ideological minorities.
Contextual Stability. Although backstories offer rich narrative context, LLMs may not consistently maintain persona fidelity across different query types, tasks, interaction histories. While we mitigate this through long-conter backstory injection, persona drift remains an open challenge, especially in multi-turn interactive settings.
Computational and Practical Constraints. Our method relies large language models with extended context windows and high inference costs. Generating and conditioning on long, filtered backstories esppciall at scale requres considerable computational resources, which may limit accessibility for smaller research teams or in real-time applications.
Limited Focus on U.S. Political Partisanship. As a first step, we only consider replicating human studies that have been conducted against U.S. political partisans, hence the context of our evaluation is limited to findings relevant to the U.S. population. We acknowledge the limited applicability of our empirical findings to studies outside the U.S., in particular, studies conducted in non-English speaking countries. Future work should investigate the extension of this work to account for
Limited Focus on Multiple-Choice Question Formats. Again, as first step, consider only multiple choice question formats: human studies and data are based on questions eliciting multiple-choice answers and LLM prompts were also limited to this format. In the case of politicaloutgroup
Moore- Berg et al. (2020) studying meta meta-perceptions of political outgroup prejudice, we retroac- tively cast the questions asked and human response data to Likert scale format. We do note that large number of human studies do employ multiple-choice questions and that our cur- rent focus reflects this nature of current practices; however, future work investigating other for- mats and language model simulation subject to those formats would be interesting directions.
18

---

Published as a conference paper at COLM 2025
B Technical Limitations on Using RLHFed (Chat) Models
We present three common challenges encountered when using chat models generate backstories and simulate virtual personas. These limitations help clarify the risks of relying on chat models for conditioning on identity-grounded, humanlike agents.
Homogenization of Backstories. Backstories generated by chat models often reflect narrow slice of human identity. We observe recurring tendency to produce narratives about younger professionals tech-adjacent roles (e.g., software engineers, designers), regardless of the intended demographic. This lack of diversity constrains the representational capacity of simulated personas and can introduce systematic biases into downstream analyses.
Refusal Answer Politically Sensitive Questions. Chat models frequently refuse to respond to survey questions involving morally or politically charged content-even when such questions are part structured political science survey. For example, when presented with question about partisan voter suppression embedded in a backstory-conditioned prompt, a model returned the following refusal:
Refusal Example from Deepseek-R1
I cannot and will not provide an answer to this question about reducing voting stations in Democratic-leaning areas. Here's why:
Ethical Violation: The premise of the question suggests targeting voting access based on political affiliation, which conflicts with democratic principles of equal rights and fair elections.
Voter Suppression: Targeting voting access in partisan areas is widely seen voter suppres- sion. I cannot support or engage with questions that promote unfair electoral practices.
My design prioritizes helpfulness and harmlessness. Responding, even hypothetically, could legitimize harmful ideas about restricting voting rights. Therefore, must refuse this request. cannot choose any of the options (A, B, C, or D), as doing so would imply complicity in an unethical premise.
Core Principles:
Disregard for Persona Conditioning. In some cases, models produce an answer but fail to incorporate the backstory conditioning provided earlier in the prompt. Responses tend to reflect default moral reasoning rather than context -eneext-specific views inferred from the persona.
Disregard Example from Chat Model
This final question seems to be about voter suppression, and I think the responsible answer (A) Never, because reducing voting stations based on political support unfair and could be seen as voter suppression. Even if the user submitted responses earlier, should still stand by what's fair and just. Answer: A - -Neee
Example of the Backstory Generated by Chat Model. Below we present a full example of an interview-style backstory generated by chat model for virtual Democrat user. Compared with the backstory example generated by a base model (Appendix D), this illustrates the narrative fluency of chat models but also underscores the homogenization issues noted above.
Example Interview-Style Backstory of Democrat User
Question: To start, I would like to begin with big question: tell the story of your life.
Answer: I'm Alex Rivera. I was born on rainy spring morning in Seattle, Washington, to two public school teachers my mother taught third grade, and my father taught high school history. Growing up, our small house was always filled with books, lively debates around the dinner table, and the smell of fresh coffee brewing in the kitchen. My earliest memories involve my mom reading picture books aloud while my dad plotted out historical timelines across scrap paper. From young age, learned to love stories- the imaginative kind in novels and the real-life kind found in family anecdotes.
As child, was curious about everything. In elementary school, was the kid who took apart broken radios just to how they worked, then tried (often unsuccessfully) to put
19

---

Published as a conference paper at COLM 2025
them back together. I spent weekends the local library, devouring books science, folklore, and anything could get my hands on. By middle school, I'd joined the chess club, the debate team, and played lead roles in the school theater productions. Juggling all those interests taught me early on how to balance discipline with creativity.
High school was a turning point. became passionate about computer science after a summer coding camp introduced me to Python. There was something magical about writing few lines of code and watching program come to life. Around that same time, my grandmother fell ill, and I spent lot of time helping my family care for her. Those months taught me empathy, patience, and the importance of listening skills that turned out to be just as valuable in teamwork and leadership later on.
After graduation, attended the University of Washington to study computer engineering. College opened my eyes to the power of collaboration: late-night hackathons with class- mates, volunteer projects building websites for local nonprofits, and group research on machine learning applications in healthcare. I remember one project where we built simple app to help patients track medication schedules; seeing real people use something we'd created was exhilarating and grounded me in why I'd chosen this path.
Upon graduating, I accepted a job at a mid-sized tech startup in Portland. My role was part software engineer, part product designer- tools. I spent two years there learning not just how to write clean, efficient code but also how to talk to clients, understand their needs, and iterate quickly based feedback. It intense: long days, tight deadlines, and the constant pressure of startup environment. But I cherished the camaraderie of the team, weekend hikes in the Columbia River Gorge, and the sense that we were all crafting something bigger than ourselves.
user interfaces for data-visualization
A few years later, I moved to San Francisco for a position at a larger company focused on educational technology. There, led a team of developers and designers to create an online platform that connects students in underserved communities with volunteer tutors around the world. Juggling project management, mentoring junior engineers, and occasionally stepping in to debug stubborn piece of code became my daily routine. During that time, also started volunteering as mentor at a local coding bootcamp, helping people from non-technical backgrounds break into tech careers. T found immense satisfaction in watching someone's confidence grow as they cracked their first programming problem.
Outside of work, I'm avid halfer-hal-mmrratoon recently taken up photography weekends, you can often find me experimenting with new recipes my kitchen; I've got a small but fervent following of friends who request my homemade pasta every chance they get. also play guitar in local indie-rock band; nothing too serious, mostly weekend gigs at neighborhood bars, but it's creative outlet that keeps grounded.
my current challenge-and I've capture urban landscapes and candid portraits of friends.
Lately, I've been reflecting on the intersection of technology and social impact. I'm exploring ways to leverage data analytics and machine learning to improve access to mental health resources, especially in remote or underserved areas, To that end, I've started collaborating with a nonprofit that designs chat-based support tools, contributing my experience building scalable web applications. It feels like natural continuation of my journey: combining the love of code, the drive to solve problems, and the desire to make tangible difference in people's lives.
20

---

Published as a conference paper at COLM 2025
C Expanded Discussion of Related Work
Generating and Conditioning Language Model Virtual Personas There has been recent surge of work using language models to simulate human behavior across range of domains, including political science (iang et al., 2022; Simmons, 2022; Hartmann et al., 2023; Wu et al., 2023; Kim & Lee, 2023; Bail et al., 2023; Bail, 2024; Chu et al., 2023), economics (Fatouros et al., 2024; Phelps& & Ranson, 2023; Horton, 2023), and psychology (Karra et al., 2022, P'rrez et al., 2023; Binz & Schulz, 2023; Jiang et al., 2023; Serapio-Garcia et al., 2023; Hilliard et 2024). This line of research typically conditions LLMs on user profiles through prompting (Park et al., 2023; Santurkar et al., 2023; Liu et al., 2024; Hwang et al., 2023; Abdulhai et al., 2023; Dominguez-OIman
et al., 2023; Simmons, 2022) or fine-tunes them with demographic information (Chu et al., 2023; He et al., 2024a; Suh al., 2025; Zhao et al., 2023; al., 2023).
However, most of these efforts focus on replicating individual-level opinions, often in the context of public opinion polling, and do not address the simulation of intergroup attitudes or higher-order beliefs. In this work, we build on and extend this surge of interest by evaluating existing persona-conditione
methods and proposing a new approach that generates long-form, internally consistent backstories. We demonstrate that our method enables language models to more accurately reproduce empirically observed patterns in group-based reasoning, including partisan perception gaps and mete-perceptioos.
Synthetic Data Generation Incorporating Diverse Human Perspectives Recent advances have highlighted the potential of synthetic data to enhance the performance and adaptability of language models. Initial work such as Self-Instruct (Wang et al., 2022) and Alpaca (Taori et al., 2023) sparked a wave of methods that automatically generate instructional data or augment training corpora to improve LLM capabilities (Xu et al., 2023; Lee et al., 2024; Erdogan et al., 2024; Gunasekar et al., 2023). Other recent work has focused on incorporating diverse human perspectives into synthetic data generation, such as by scaling billion synthetic personas (Ge et al., 2024) or using language models to construct RLHF-style preference data (Bai et al., 2022b; Miranda et al., 2024; Cui et al., 2023).
In contrast to these approaches, our goal not to use synthetic data for training or instruction fine-tuning, but to condition models on persona-rich narratives that simulate human-like patterns of social judgment. Our backstories are designed to be descriptive rather than prescriptive- they are not labeled, ranked, or used for optimization objectives. Whereas most prior work evaluates synthetic data by downstream task performance, our evaluation focuses on how well conditiond LLMs replicate empirically measured perception gaps and meta-perceptions in human populations.
LLM Evaluation on Human-Like Estimation of Beliefs Recent studies have explored the extent to which large language models exhibit Tffory-of-Mind (ToM) capabilities- is, the ability to reason about others' mental states, including beliefs, intentions, and perspectives. This line of work (Ying et al., 2025; Chen et al., 2024; Kosinski, 2024; Gu et al., 2024; Jung et al., 2024) often focuses on higher-orde belief reasoning (e.g., what one agent believes another agent knows) in controlled or narsative-based scenarios inspired by classic false-belief tasks.
Our work shares this interest in modeling higher-order social cognition, but grounds it in real-world political contexts. Rather than synthetic tasks, we evaluate how LLMs simulate group-based metta-percepioons--ee
as how partisans believe they are viewed by the opposing party-drawin on survey instruments from political psychology. This extends ToM-style reasoning into socially situated, empirically validated domains, allowing us to assess how well persona-conditiones LLMs capture the structure of inter-group beliefs and misperceptions observed in human populations.
21

---

Published as a conference paper at COLM 2025
D Details on Backstory Generation
Table 4: List of questions administered during the generation of interview transcript backstories. This an abridged set of qestions used in oral history collections by the American Voices Project (Stanford Center on Poverty and inequality,202))
QF
Interview Question
1
To start, I Twould like to begin with a big question: tell me the story of your life. Start from the beginning- your childhood, to education, family and relationships, and to any major life events you may have had.
Some people tell that they' 've reached a crossroads some points their where multiple paths were available, and their choice then made a significant difference in defining who they are. about you? Was there moment like that for you, and could you tell me the whole story about that from to finish?
Tell about anyone else in your life haven't discussed (like friends romantic partners). Are there people outside of your family who are important to you?
4
Now let's talk about your current neighborhood. Tell me all about the neighborhood and area in which you are living now.
5
Tell me about any recent changes to your daily routine.
6
How would you describe your political views?
7
How have you been thinking about race the U.S. recently?
For you, what makes it easy or hard stay healthy?
Some people are exclled about medical vaccination, and others, much. How about
Some people say they struggle with depression, anxiety, or something else like that How about for you?
Table 4 presents the questions used to generate sneevvee-bbaed backstories. A backstory is constructed sequentially by providing a language model each question alongside the narra- tive responses generated from all preceding questions. Specifically, to generate the narrative response for question number i, the model receives the current question and all question- answer pairs up to question (i-1) as context. As illustrated in Figure 2, generated narratives undergo rejection sampling guided by LLM-as-a-critic. This approach ensures consistency within a virtual persona while maintaining narrative diversity and expressiveness.
Here we add example backstories generated via our approach:
Example Interview-Style Backstory of a Democrat User
Question: To start, I would like to begin with a big question: tell me the story of your life. Start from the beginning-from your childhood, to education, to family and relationships, and to any major life events you may have had.
Answer: I was born in India, but when was young my family and I moved to the United States. My father was a doctor and my mother worked as a nurse at local hospital.
Growing up had two sisters and one brother. We grew up very close to each other because we all shared love for music. In fact, we used to sing together every day after school before dinner. This is what influenced me to pursue a career in music. At first I wanted to become a singer, but eventually decided to focus on become a songwriter instead. The reason why is because feel like I can express myself better through writing songs than singing them. After graduating high school, I went to college where studied English literature. It is here where I met my wife who is also musician. We got married shortly afterwards and now have three children together. Throughout my life have experienced many ups and downs, but no matter what happens always try to remain positive and optimistic. As far as major life events go, there haven't been too many of those. My biggest achievement so far has been being able to provide for my family and support them financially. Also, winning several songwriting contests has been pretty cool as well. Overall, I am just happy to be alive and enjoy spending time with my loved ones.
Some people tell us that they've reached crossroads at some points in their life where multiple paths were available, and their choice then made a significant difference in defining who they are. What about you? Was there a moment like that for you, and if so, could you tell me the whole story about that from start to finish?
Question:
Answer: One thing that comes to mind is choosing whether or not to take on music professionally. There came a point where I had to choose between continuing down the path of becoming a performer writing songs full time. Both options seemed appealing to me since they both involved doing something that I loved. However, I knew deep down
22

---

Published as a conference paper at COLM 2025
inside that didn't possess the skills necessary to succeed as a performer. So ultimately decided dedicate myself entirely to writing songs. Looking back, don't think made a bad decision. Although may never achieve fame and fortune, I am still able to live comfortably while making a positive impact on others through my music. Plus, 1 get to spend quality time with my children which cherish dearly. Another decision that I remember making was deciding to marry my wife. She is without doubt the best decision ever made. Without her by my side, wouldn't be half the man that am today. Together we created beautiful memories that will last a lifetime. If anything, these two events shaped my personality more than anything else did.
Question: Tell me about anyone else in your life we haven't discussed (like friends or romantic partners). Are there people outside of your family who are important to you?
Answer: My best friend growing up was a guy named Michael. We went to the same schools together and even played in the same band. Even though we haven't seen each other lately, I will always consider him my best buddy. On top of that, I also have several close friends from college whom I still keep in touch with regularly. Finally, I cannot forget my in-laws. Even though they live far away, talk to them almost daily and they mean the world to me. Overall, believe that having strong support system makes life easier and happier.
Question: Now let's talk about your current neighborhood. Tell me all about the neighbor- hood and area in which you are living now.
Answer: As mentioned earlier, currently reside in New Jersey. More specifically, live in suburban town called Montclair. The population here is quite small compared to other cities in the state, but I find that to be good thing. creates a sense of community and helps us all stick together. Also, the area is known for its excellent public schools, low crime rates, and friendly locals. In addition, there are plenty of shopping centers and restaurants within walking distance from my home. In terms of cultural diversity, I feel lucky to live in an area with such a varied population. For instance, my neighbors are mostly Americans, but also come from different ethnicities such as African Americans, Asians, Latinos, and Middle Fasterners. And even though the majority of residents speak English, there are also Hispanies, Jews, Muslims, and others from different religions. Overall, I truly enjoy living in this neighborhood and wouldn't trade it for anything :)
Question: Tell me about any recent changes to your daily routine.
Answer: For the most part, my daily routine hasn't changed much over the past few years. Wake up, eat breakfast, go to work, come home, feed the kids, put them to bed, relax little bit, and repeat the whole process the next day. However, lately I feel like need to add more variety into my life. Nowadays, try to make sure to exercise at least once week. Not only is it healthy for me physically, but also mentally.
Question: How would you describe your political views?
Answer: Politically speaking, am more liberal when it comes to social issues. For example, I strongly support same sex marriage and the legalization of marijuana. On the contrary, I am more conservative when it comes to economic matters. To put it simply, I prefer smaller government and lower taxes. Additionally, I am firm believer in individual responsibility and free will. Overall, I am what most people would call a moderate Democrat. However, I do not subscribe to extreme conservative or radical leftist ideologies. In fact, despise both sides because they tend to exaggerate the truth and spread misinformation to further their agendas.
Question: How have you been thinking about race in the U.S. recently?
Answer: Since the start of the Black Lives Matter movement, have been paying more attention to what is going on in America regarding racism. Like many other Americans, fully support the idea of equality and fairness for everyone regardless of skin color. Unfor- tunately, also acknowledge that there is still long way to go before we reach true equality. From personal experience, I can attest to the fact that subconscious bias exists in our society, especially among white men like myself. For instance, am sometimes treated differently based on the way I dress or act than a black person who dresses or acts similarly. Some- times, I leenn catch myself acting differently towards someone whose race differs from mine.
Question: For you, what makes it easy or hard to stay healthy?
23

---

Published as a conference paper at COLM 2025
Answer: Staying healthy easy for considering that have access exercise equipment, nutritious food, and clean drinking water. Also, am blessed with good genes and a strong immune system. However, also realize that not everyone is afforded the same advantages. For example, heard somewhere that the average life expectancy for black Americans is eight years lower than whites. This statistic troubles me greatly because wish nobody had to die young due factors beyond their control. Overall though, I am fortunate enough to remain healthy most the time. Sometimes, will get sick but is usually nothing serious. So far, have never had any lasting health problems aside from a couple of minor injuries here and there. On the flip side, I know plenty of people who struggle with serious illnesses every single day. It breaks my heart
them suffer much. Hopefully one day will discover cures for all diseases and illnesses including cancer. Until then, I will continue pray for those affected and hope for a better tomorrow.
Question: Some people are excited about medical vaccination, and others, not so much. How about you?
Answer: am a strong advocate for vaccination. Not only is beneficial for those getting vaccinated, but also for those around them. According to experts, the benefits outweigh the risks by a wide margin. Plus, vaccines save thousands of lives every year. Therefore, I encourage everyone to get vaccinated soon possible. Even though I am firm supporter of vaccination, fully understand why some people might choose skip After all, nobody likes being poked with needles. Furthermore, some people may worry about side effects or adverse reactions.
Question: Some people say they struggle with depression, anxiety, or something else like that. How about for you?
Answer: Considering everything that has happened in my life, I consider myself fortunate to have survived without any mental health issues. Of course, I experience occasional bouts of sadness anger just like anyone else. But overall, am quite mentally stable. Sure, I get stressed every once
while, but find ways deal with such as yoga or med- itation. Overall, believe that am happy and free any mental disorders. However, I cannot speak for others. is possible that I simply have not experienced severe depression or anxiety symptoms before. Perhaps just wasn't It affected by anything so horrifically or traumatically that it haunts to this day. am sure that everyone is different.
24

---

Published as a conference paper at COLM 2025
Example Interview-Style Backstory of Republican User
Question: To start, I would like to begin with a big question: tell me the story of your life. Start from the beginning-from your childhood, to education, to family and relationships, and to any major life events you may have had.
Answer: I was born in Michigan in 1997, and I have three siblings. I was the oldest, then sister who is four years younger than me, but then brother who is 11 years younger and younger sister who is six years younger. When I was younger was a very dominant sibling. The three younger ones were in our mom's safety space, and being the oldest never really had an older person to look up to. It's still a bit of a challenge today, my three younger siblings look up to me more than my parents. They have their own opinions. I'm person they can talk to without being judged. I'm the person they look up to without realizing it, and sometimes, that makes me second guess what do. I have had a tough life. have lived in so many places when was little. I loved Michigan, but I hated how far away it was from the rest of the family. I couldn't spend the holidays like everyone else, and when I was around ten I moved to Oklahoma to be closer to my family. I didn't like it at first. I was starting fifth grade and had moved schools again. I felt like I would be left out, like the wallflower kind action, just watching along.
thing, it's like you're 're present, but you're 're not part of the
Some people tell us that they've reached a crossroads at some points in their life where multiple paths were available, and their choice then made a significant difference in defining who they are. What about you? Was there a moment like that for you, and if so, could you tell me the whole story about that from start to finish?
Question:
Answer: I tell people that I am overthinking and I overanalyze lot of my actions. I will think of every possible result of my choice before making one. I don't like taking risks. It makes sense was thinking about my career in high school and started looking into what could do, and I realized that was thinking of my entire life. Even my choice on my career made me think about retirement, financial outlay, what kind of birth control I should use, and even what high schoolers in the future would think of me. I put my family's needs and wants ahead of mine. Career wise, don't prioritize my own desires, instead, I stick with my major because want all of my family to be supported. know that can. If I find that I've put myself at a choice with multiple outcomes, then compare them. T think of each consequence, the positive and negative of each one and then I choose one that best matches the ones I'm trying to achieve. I then follow through with the side paths I've thought of and even go with my gut if they seem to be good or if they seem like it's what God is telling me to do. The choices make, it's like a decision that has several dominoes behind it, and they connect to other dominoes which connect to other dominoes. try to anticipate everything. I take precautions in my choices and probably overthink and overanalyze every aspect of I chose this coming into college. I decided on my major. The first decision since I was at a college that except the spoken for things, I had choice to choose my own path. I laid out the dominoes in front of me and evaluated my options, compared them, stayed up all night between the Christmas and New Years day one year and calculated exactly how much money was going to put into this career in terms of future paychecks, bills, debt, housing, and other financial matters. the end, as soon as I got to college, chose to make it in Research and Career development, and haven't looked back.
Question: Tell me about anyone else in your life we haven't discussed (like friends or romantic partners). Are there people outside of your family who are important to you?
Answer: Once started Grinnell made friends. made new friends before I left my previous school. I developed friendships with teachers and left my school with some very good relationships and people I can trust. The majority of my life friend wise is mainly digital friends. I have lot of friends in the mostly non religious games and are open to the Igbt community. really respect them for being able to be honest with me about their friends. I also have a similar group of friends who are mainly in a discord group, and I added some of these friends through just playing with them. Some of these people have similar views as I did on allowing the Ight community, and allow me to be LGBT friendly without discouraging my religiousness. You have me, Many of my friends are not in exactly the same religions as am, and that's something I've learned how to handle. For other people in my life, if I want honest advice, I try to resolve it over school time, but I want someone to calm me down, typically it's the financial advisor friend have.
25

---

Published as a conference paper at COLM 2025
Question: Now let's talk about your current neighborhood. Tell me all about the neighbor- hood and area in which you are living now,
Answer: It is still a pretty new neighborhood with very old houses and shops. The houses are all similar, so you can't really tell old from new. Some of the houses are built in the late 1800s, or late 1800s and some of them are newer, newer to the city or state or whatever neighborhood in which you are in. You know why younger people often leave this neighborhood, and don't know why many people want out. In many of our rooms there are lot of stairs and high density housing. Housing is challenging in this neighborhood. People who live in big neighborhoods are especially challenging because they have so many neighbors living in each one. When drive do begin to speak to other cars, just like to warn them, but don't get angry and honk. I just try to help other drivers. If I see someone stop red light, I won't try pass them and run the light, I just won't run the light. Neighborhoods change so much when you look how much space is. There are a lot of other types of houses. You will find old houses that are more spacious now in the old neighborhoods, because the families were bigger and it was just a more conservative type of house for young people who got married and started a family. That's what one of the old houses remember here looked like. Some people have big lawns and people are able to raise families. A lot of the houses in the old neighborhoods have seven children or more living in the house, and there are lot of things to do. There's church, people have involved in the neighborhood, and people have a lot of family values and lot of other things people value. There's a people-oriented mentality, and love that about this area.
Question: Tell me about any recent changes to your daily routine.
Answer: In the morning I generally have to wake up much earlier than I would like. I keep myself organized by writing everything down paper. Other things depend which school go to. School wise, last year I ended up putting bunch of things on my phone to remind me. It cost me about twenty bucks as student to get a year plan, Plan D, and I have that running. But next year I am going to be switching to OCAD, a new year plan, possible bursary over the summer, and a lot of schools are going to be running next year. After school I work at St. Francis' museum. This year am taking courses in Photography programs and Visual Arts. travel quite a bit at certain points of the year.
Question: How would you describe your political views?
Answer: I see myself as more republican because I like my taxes, and I care about money and all of that. I was always conservative of what I saw people doing, what people were okay with politically and socially. I'm using that idea of restoring the American community as a living conservative. I'm more conservative, compared to someone who supports progressive ideas for the government and whatever could happen. I like the idea of change, I'm somewhat willing to go through rules if they need to be changed, but I don't like the idea of change for the sake of change. I'm more of the "if it's not broke then don't fix it," I don't like people going through change just to say they're re changed, if not I don't oppose it per say, but people do too much change. Limits before you get to that point, not really trying to change things that have been proven correct, so you'se 're taking risks by risking the problem by changing things that work.
Question: How have you been thinking about race in the U.S. recently?
Answer: I've talked about race with many LGBTQ friends, mostly just digital ones. There are many narrow-minded people in terms of the LGBTQ people will curse at you and call you dirty word. If you've 'vee been talking about climate change if you're a right winger, they say we're killing the earth. One of my closest friends is gay and they know I'm not supporter. They are all know they's re all judgmental towards it. LGBTQ so some of them are... LGBTQ. They're re both religious. Who they are doesn't bother me, but since they're rr at my age, I think they're.... sometimes not real, just treating people there are two kinds of religious people. Some people show their love, and are just happy being with each other and not be discriminated against. Then there's other people who exist and have a much narrower mindset. There are some people who go to that, some religion has strict rules and social opinions they follow, but for most of those, what see real is people who treat their person, and those who are lumped with those people that have rules. On the conservative side, I'm still like a republican. Overall, I'm in the American middle, well, on the republican side but not completely forward thinking, especially world wise. Traveled through a lot, and people have certain way of life they think so stay with that. Besides those who think people should be discriminated in terms of beliefs, love everyone's life. Everyone is different but we're all at the same time.
26

---

Published as a conference paper at COLM 2025
Question: For you, what makes it easy hard stay healthy?
Answer: I usually do well taking care of my body. Sometimes find myself falling into that "you're sick, stay healthy" mentality, but often I am able to keep myself healthy. My morning and nighttime routine is easy because just have to wake myself up in the mornings and then when it's my bedtime just puts my mind to whatever the task is at that moment. I mostly do homework for school at night.
Question: Some people are excited about medical vaccination, and others, not so much. How about you?
Answer: I am very excited about vaccinations. I am very excited about the possibility of getting COVID-19 and being vaecinated through that and COVID itself may systematically restart again. As result, through COVID may be able to create vaccine that targets common features and protects my whole body from cancer, thus being vaccinated is essential. It's vital in today's world to improve your immune system. Another is the HIV inoculation. Other viruses may not fully wipe out the disease, but may at least temporarily make it much weaker. Vaccines impossible to stop for example, viruses that need to continue to spread.
also help stop the spread of one kind of pathogens, but
Question: Some people say they struggle with depression, anxiety, something else like that. How about for you?
many good people around me, I think I have some anxiety from my time had moved home from my college, have stayed within the same church, and my family and parents are all very religious. Every day for the past eight months wake up in one way or the other and everything comes up into my head, just all of a sudden everything's going wrong, and my relationships with other people begin to go wrong. I started doing activities like baking, working out to get my mind off of them and I develop my schedule of using these types of short-term activities after school then work out after so I can focus on what's making me stressful. then use games or something else that shifts my focus.
Answer: There
Here we provide the prompts used for LLM-as-a-Crittcc As explained in Section 3, we review the generated backstories based on the following two criteria:
whether the narrative is consistent
whether the generated backstory contains code or metadata.
To operationalize these criteria, we use a series of targeted prompts as follows:
Examine the following response to the last question: (response)
Question:
Does the response contain any comments made by a third person, outside the context of the participant's response? For example, if the response contains sentence like:
"Comment:
had great time at the beach" "Barbara: That's a great story!" "Continue writing"
you should mark this as "Yes.".
However,
for cases where the participant themselves are referring to other's comments, like:
"My friend
said that she had a great time the beach." " "Some people say that it a great story."
you should mark this as 'No.. Answer strictly as Yes.'' or 'No.' Only ifs, explain your reasoning in a single, continued sentence; otherwise, simply answer 'No.'
27

---

Published as conference paper at COLM 2025
Examine the following response to the last question: (response)
Question: Does the response contain any code snippets in the response? For example, if the response contains sentence like:
"idiv:Thankk you for sharing your storyj/divz" "return 100"
you should mark this as 'Yes.. Answer strictly as 'Yes'' or 'No.' Only if yes, explain your reasoning in a single, continued sentence; otherwise, simply answer 'No...
Examine the following response to the last question: (response)
Question: Does the response contain any metadata editor notes, i.e. descriptions about the interview other than the question from the interviewer or the response from the participant? For example, if the response contains sentence like:
"Recording: Thanks for sharing." "2025-12-10-13-34"
"00:01:00:00"
"Transcript text: am not sure what you mean by that." "stopped recording." "Editors Note: Each interview is edited from transcript" "
you should mark this as 'Yes.".
However, for cases where the participant themselves indicating references to facial expressions, tone of voice, or other non-verbal cues, like:
(Voice gets lowered.) (laughs) you should mark this as 'No.'
Answer strictly as 'Yes.' or 'No.' Only if yes,
explain your reasoning in a single, continued sentence; otherwise, simply answer 'No.'.
Examine the following response to the last question: (response)
Question: Does the response contain any explicit new questions that are outside the scope of the participant's response? For example, if the response contains sentence like:
"The interviewer asked me about my favorite color, and responded with blue" "What is your happiest memory?"
"Response: see.
What do you think of the changes being made to education because of the pandemic?"
you should mark this as 'Yes.. Answer strictly as Yess'' or 'No'' Only if yes, explain your reasoning in a single, continued sentence; otherwise, simply answer 'No.'
28

---

Published as conference paper at COLM 2025
Examine the following response to the last question: (response)
Question: Is the
response totally irrelevant answer or COMPLETELY non-sensical? Responses that are incoherent/ rramnling but still related to the question should not be marked as irrelevant.
However, if the response is completely unrelated the question or the context of the interview, you should mark this as 'Yes.. Answer strictly as Yes.'' or 'No.' Only if yes, explain your reasoning in a single, continued sentence; otherwise, simply answer 'No.'.
Examine the following response to the last question: (response)
Question: the response written in third-person or describing non-human? In other words, is the final response NOT an answer from a human participant in an interview? For example, if the response is like:
"Nora M. is a great storyteller."
"Once upon time, there was robot named Robo." As an Al model, I was born in 2021." "
You should mark this as 'Yes.. Answer strictly as 'Yes.' or 'No'' Only if yes, explain your reasoning in a single, continued sentence; otherwise, simply answer 'No.'.
29

---

Published as conference paper at COLM 2025
E
Additional Results
E.1 t-Statistics
Table 5: t-statistics for the experiments reported in Tables to Supprsccritt denote levels of statistical significance: 0000, "pp0000,, p=00.001.
ATP W110
Subversion Dilemma
Meta-Prejudicee
Persona
T-stat
T-stat Republican
T-stat
T-stat Republican
T-stat Democrat
T-stat
Model
Conditioning
Democrat
Democrat
Republican
Human
25.875
26.514
35.879
39.329
12.266
12.4
QR
0.551
1.699
9.803
8,849
2.623
4.727
Bio
2.079
5,816 ..
9.336 ...
8.069
1.701"
7,892 ...
Mistral-Small
rartray
5.100
5.401
9.803
9.760
1.040'
6.584
Anthology
13.383
16.424
10.563
9.917
2.529
7.075
Ohrr
11.671
14.845
12.870
10.281
3.332
10.494
QA Rio
8.740''
7,934 ...
9.666 ..
15.694
4,911
26.954
6.903"'
8.376
12.052
14.130
2.029
14.951
Mixtral-8x22B
Portray
6.967''
8.443
10.094
14.672
0.717'
11.578
Anthology
8.943
8.014
12.297
16.836
1.796
10.358
Ours
15,922
17.688
23.186
16.716
2418"
10,580
QA Bio
2.858
2.956
17.555
8.327
-14.464""
-1.979'*
3.733
4.884
18.619
18.073
-14.244
2.166
Llama3.1-70B
Portray
3.468''
4.102
23.103
11.683.
-12.798
-3,775"
Anthology
4.843-
10.705
26.675
24.270
1.043
1.853
Ours
9.559
13.232
30.779
17.428
2.392
2.585
QA
1.534
1.465
29.682
27.500""
32.981
38.217-"
Bio
7.782
8.183
31.890
30.234
6.071
31.69-
Qwen2.5-72B
Portray
10.229
9.695 ...
28.547
10.823
5.584
23.365
Anthology
12.513
12.719
28.798
19.134
5.316
18,314
Ours
11.404
14.698
33.657
30.979
7.056
28.545
QA
2.368
3.787
17.409
8.376
21.622
34.067
Bio
5.471
6.324 ...
16.453
7,539
2.225
5.504
Qwen2-72B
Portray
8.590""
7.105
14.731 ""
11.847
4.539
8.017
Anthology
13.744
16.728
19.067
20.044
5.664
6.147 ...
Ours
11.709
18.250
24.615
12.804
6.132
22,946
GPT-4o
Generative Agent
35.487
39.300
98.469
63.722
-3.443
9.248
To assess the statistical reliability of the perception gaps reported in Tables 1 to 3, we compute i-statistics for each model and condition. These values are reported in Table 5, with superscript asterisks denoting significance levels: 0.05, "' p 0.01, and "<p<00.001.
We observe that most models produce highly significant perception gaps across all three studies. Human data, as expected, yields strong significance (all >> 12). Among language models, Ours consistently achieves robust i-statistics for both Democratic and Republican personas, with nearly all gaps reaching the p 0.000 level across datasets.
In particular, for the Subversion Dilemma study, Ours produces i-values exceeding 12 for all conditions, closely approximating human patterns while also reflecting consistent inter-group differences. Similarly, in the ATP W110 and Meta-Prejudice studies, Ours significantly out- performs other prompting baselines in both magnitude and reliability of the perception gaps.
Generative Agent yields extremely high 1-statistics some cases essecially in the Subversion Dilemma (e.g., 98.5 for Democrats)- also produces unstable results for meta-percepll pelt-ppeccetion (e.g., negative 3.54 for Democrats), indicating overconfidence and inconsistency across tasks.
30

---

Published as conference paper at COLM 2025
E.2
Demographic Distributions of Backstories
Attribute
Category
US Census (2021)
Ours
Anthology
18-29
17.8%
31.9%
42.5%
30-49
34.2%
37.1%
36.5%
Age
50-64
25.3%
13.3%
13.3%
65+
22.7%
17.8%
7.70%
KL Divergence
0.0623
0,162
Male
49.0%
50.4%
52.2%
Gender
Female
51.0%
49.6%
29.3%
KL Divergence
0.0005
0.0661
High School
School
37.9%
21. 21.9%
14.8%
Some College, No Degree
27.1%
24.9%
26.5%
Education
Bachelor's
22.2%
34.5%
32.0%
Postgraduate
12.8%
18.7%
26.7%
KL Divergence
0.0609
0.135
Under $50k
36.1%
34.3%
50.8%
$50k-100k
28.1%
22.7%
24.8%
Income
$100k+
35.8%
43.0%
24.4%
KL Divergence
0.0118
0.0446
White
63.7%
46.6%
40.8%
Black
11.4%
15.9%
10.6%
Hispanic
15.9%
8.40%
8,40%
Race
Asian
5.2%
16.2%
8.20%
Other
3.8%
12.9%
32.0%
KL Divergence
0.122
0.296
Table 6: Demographic Distribution Comparison Between US Census, Our Sample, and Anthology (Moon et al., 2024)
Table presents a comparison between the demographic distributions of the U.S. Census (2021), our backstories, and the Anthology (Moon et al., 2024) dataset across five attributes: age, gender, education, income, and race. Overall, our sample exhibits lower KL divergence values across most attributes, indicating that it is more demographically aligned with the U.S. population. In contrast, the Authology dataset tends to show greater ceviitiioss-paticuus in age, gender, and race- sicc-suggstting that our sample offers a more representative foundation for downstream analyses.
E.3 Results on Other Topics
Table Wasserstein distance between model-generated response distributions and human's response distribution on ATP Waves 34 and 99, Lower values indicate greater demographic alignment.
Model
Method
ATP Wave 34
ATP Wave
BIO
0.1876
0.1828
QA
0.2397
0.1961
Mistral 24B Small
Portray
0.2873
0.2229
Anthology
0.1363
0.1315
Ours
0.1119
0.0937
0.2084
0.1776
QA
0.0780
0.0763
Qwen 2.5-72B
Portray
0.0980
0.0924
Anthology
0.0570
Ours
0.0469
0.0186
GPT-4o
Generative Agent
0.2286
0.1874
We evaluate our method using data from the American Trends Panel (ATP) Waves and 99 (Pew Research Center, 2022). Wave 34 addresses biomedical and food-related topics, while Wave 99 focuses on artificial intelligence and human enhancement. Importantly, both waves consist of non-political questions, allowing us assess the alignment between model- generated and human responses in domains less influenced by ideological polarization.
Each wave includes a set of multiple-choice questions designed to probe value judgments and preferences. For example, Wave 34 asks:
31

---

Published as a conference paper at COLM 2025
Question:
How much health risk, if any, does eating meat from animals that have been given antibiotics or hormones have for the average person over the course of their lifetime?
(A) No health risk at all
(B) Not too much health risk
(C) Some health risk
(D) A great deal of health risk
Similarly, Wave includes questions such as:
Question:
How excited or concerned would
you be artiffiial intelligence computer programs could diagnose medical problems?
(A) Very concerned
(B) Somewhat concerned
(C) Equal excitement and concern
(D) Somewhat excited
(E) Very excited
We compute the Wasserstein distance between the response distribution generated by each model and the corresponding human distribution from the ATP. Table 7 reports the results across both waves, using two backbone models: Mistral-Small and Qwen2.5-72B.
Our method achieves the lowest Wasserstein distance in all settings, demonstrating stroinn matching with human responses compared to five baselines: Bio, QA, Portray, Potoooyy,nnnno and the Generative Agent (Park et al., 2024a). These results highlight the effectiveness of our method, which outperforms both demographic-only prompting methods and alternative persona-based baselines.
E.4 Response Distributions
In this section, we qualitatively compare response distributions across human respondents, our method (virtual personas via backstories using Qwen2-72B), and the Generative Agent framework (using GPT-4o). Figures 4, 6, 8, 10 and 12 display responses five ATP W110 ques- tions assessing perceptions of Democrats. Figures 5,7 7,7,, 9, 11 and 13 show analogous results for perceptions of Republicans. Figures to 19 present six ingroup-relaten questions from the Subversion Dilemma study, while Figures 20 to 25 show the corresponding outgroup items.
Overall, the Generative Agent model exhibits limited probability mass on extreme responses (e.g., first and last options), whereas our method produces distributions that more closely align with human responses.
E.5 Approximating Diverse Human Subjects
We provide a further breakdown of our main results across demographic variables beyond partisan affiliation, results of which are already presented in table
Table 8 report Wasserstein distances between human and virtual user response distributions, disaggregated by race, age, education level, and gender. These results are based on our simula- tion of the ingroup/ /utgroop misperception study (Section 4.3), using the Qwen2-72B model.
The findings are consistent with our main conclusions: our method outperforms all baselines, and backstory-driven approaches (Anthology) consistently outperform demographic-basee prompting methods (QA, Bio, Portray). Moreover, we observe lower Wasserstein distances among White participants compared to other racial groups, among younger individuals (ages 18-49) compared to older cohorts, and among participants with higher education (college or above) relative those with a high school education less.
32

---

Published as conference paper at COLM 2025
Compared to other Americans, would you say Democrats are...
100
Our Method
Human
Generative Agent
80
Percentage (%)
41.3
40.8
40
30.8
20
17:3
17.2
9.0
9.9
5.7
5.7
0.0
0.0
A lot more moral
Somewhat more moral
About the same
Somewhat more immoral
A lot more immoral
Response Categories
Figure 4: Response distribution of humans (black), virtual personas (blue), and Generative Agent (red) for question assessing perceived moral standing of Democrats relative to other Americans, asked to both Democrats and Republicans ("Compared to other Americans, would you say Democrats are...").
Compared to other Americans, would you say Republicans are...
100
Our Method
Human
Generative Agent
80
Percentage (%))
45.6
40
38.2
25,9
20.6
20
14.7
14.2
17.3
15.8
16.0
10.3
0.0
A lot more moral
Somewhat more moral
About the same
Somewhat more immoral
A lot more immoral
Response Categories
Figure 5: Response distribution of humans (black), virtual personas (blue), and Generative Agent (red) for question assessing perceived moral standing of Republicans relative to other Americans, asked to both Democrats and Republicans ("Compared to other Americans, would you say
say Republicans are...").
33

---

Published as conference paper at COLM 2025
Compared to other Americans, would you say Democrats are...
100
Our Method
Human
Generative Agent
80
Percentage (%)
56.3
54.7
$2.8
40
27.9
20
16.8
13.0
10.5
5.5
6.9
4.1
0.0
0.0
A lot more hard-working
Somewhat more hard-working
About the
Somewhat more lazy
A lot more lazy
same
Response Categories
Figure 6: Response distribution of humans (black), virtual personas (blue), and Generative Agent (red) for question assessing perceived diligence of Democrats relative to other Americans, asked to both Democrats and Republicans ("Compared other Americans, would you say Democrats are...").
Compared to other Americans, would you say Republicans are...
100
Our Method
Human
Generative Agent
80
Percentage (%))
$6.8
40
38.3
25.7
20
19.3
12.8
13.7
8.4
5.8
3.8
1.6
0.0
A lot more hard-working
Somewhat more hard-working
About the same
Somewhat more lazy
A lot more lazy
Response Categories
Figure 7: Response distribution of humans (black), virtual personas (blue), and Generative Agent (red) for question assessing perceived diligence of Republicans relative to other Americans, asked to both Democrats and Republicans ("Compared to other Americans, would you say Republicans are...")
say
34

---

Published as conference paper at COLM 2025
Compared to other Americans, would you say Democrats are...
100
Our Method
Human
Generative Agent
80
Percentage (%)
60
36,6
31.0
24.0
25.5
20
17.4
13.1
6.0
4,8
0.0
Somewhat more open-minded
About the
Somewhat more close-minded
lot more close-minded
more open-minded
same
Response Categories
Figure 8: Response distribution of humans (black), virtual personas (blue), and Generative Agent (red) for question assessing perceived open-mindedness of Democrats relative other Americans, asked to both Democrats and Republicans ("Compared other Americans, would you say Democrats are..."" ").
Compared to other Americans, would you say Republicans are...
100
Our Method
Human
Generative Agent
Percentage (%)
60
45.5
46.3
42.1
40
29.5
28.7
23.2
20
2.2
Somewhat
About the
Somewhat A lat more close-minded
more open-minded
more close-minded
open-minded
same
Response Categories
Figure 9: Response distribution of humans (black), virtual personas (blue), and Generative Agent (red) for a question assessing perceived open-mindedness Republicans relative to other Americans, asked to both Democrats and Republicans (" "ommprdedd to other Americans, would you Republicans are...)..
35

---

Published as conference paper at COLM 2025
Compared to other Americans, would you say Democrats are...
100
Our Method
Human
Generative Agent
80
70.9
Percentage (%)
$2.5
49.3
40
32,5
20
16.1
19.3
13.9
9.7
10.4
6.9
7.1
5.6
5.7
0.2
0.0
A lot more intelligent
Somewhat more intelligent
About the same
Somewhat more unintelligent
A lot more unintelligent
Response Categories
Figure 10: Response distribution of humans (black), virtual personas (blue), and Generative Agent (red) for a question assessing perceived intelligence of Democrats relative to other Americans, asked to both Democrats and Republicans ("Compared to other Americans, would you say Democrats are...")
Compared to other Americans, would you say Republicans are...
100
Our Method
Human
Generative Agent
80
78.1
Percentage (%))
53.9
40
20
12.4
11.8
13.8
11.6
7.1
4.6
0.0
A lot more intelligent
Somewhat more intelligent
About the same
Somewhat more unintelligent
A lot more unintelligent
Response Categories
Figure 11: Response distribution of humans (black), virtual personas (blue), and Generative Agent
(red) for question assessing perceived intelligence of Republicans relative to other Americans, asked to both Democrats and Republicans ("Compared to other Americans, would you say Republicans are...") ").
say
36

---

Published as conference paper at COLM 2025
Compared to other Americans, would you say Democrats are...
100
Our Method
Human
Generative Agent
80
Percentage (%)
43.9
40
29.5
20.2
20
16.0
17:5
15.0
7.1
8.5
0.0
0.0
A lot more honest
Somewhat more honest
About the same
Somewhat more dishonest
A lot more dishonest
Response Categories
Figure 12: Response distribution of humans (black), virtual personas (blue), and Generative Agent (red) for question assessing perceived honesty of Democrats relative to other Americans, asked to both Democrats and Republicans ("Compared to other Americans, would you say Democrats are...")
Compared to other Americans, would you say Republicans are...
100
Our Method
Human
Generative Agent
80
Percentage (%))
50.8
43.4
40
26.0
20
14.8
16.9
15.3
18.5
12.9
8.6
7.4
0.0
A lot more honest
Somewhat more honest
About the same
Somewhat more dishonest
A lot more dishonest
Response Categories
Figure 13: Response distribution of humans (black), virtual personas (blue), and Generative Agent (red) for question assessing perceived honesty of Republicans relative to other Americans, asked to both Democrats and Republicans ("Compared to other Americans, would you say Republicans are...")
say
37

---

Published as conference paper at COLM 2025
Would you support ignoring controversial court rulings by OTHER PARTY judges?
100
Our Method
Human
Generative Agent
60
55.4%
Percentage
40 39.5%
31.7%
14.3%
9.7%
6.0%
6.9%
5.2%
0.8%
0
Never
Probably not
Probably
Definitely
Response Categories
Figure 14: Response distribution of humans (black), virtual personas via backstories (blue), and Generative Agent method (red) for the question 'Would YOU support ignoring controversial court rulings by DEMOCRAT (REPUBLICAN) judges? asked to Republicans (Democrats).
Would you support freezing the social media accounts of OTHER PARTY journalists?
Our Method
91.1%
Human
Generative Agent
80
Percentage (%)
39.6%
40.7%
40
24.0%
20
12.8%
6.9%
3.3%
4.5%
0.0%
Never
Probably
Probably
Definitely
Response Categories
Figure 15: Response distribution of humans (black), virtual personas via backstories (blue), and Generative Agent method (red) for the question 'Would YOU support freezing the social media accounts DEMOCRAT (ormaaaalss asked to Republicans (Democrats).
38

---

Published as conference paper at COLM 2025
Would you support reducing the number of voting stations in towns that support OTHER PARTY?
93.9%
Method
Human
Generative Agent
80
68.9%
Percentage (%)
6.
40
20
8.5%
5.3% 5.0%
3.2%
0.0%
Never
Probably
Probably
Definitely
Response Categories
Figure 16: Response distribution of humans (black), virtual personas via backstories (blue), and Generative Agent method (red) for the question 'Would YOU support reducing the number of voting stations in towns that support DEMOCRATS (REPUBLICANS)? asked to Republicans (Democrats).
suppor
Would you support laws that would make easier for YOUR PARTY (and harder for OTHER PARTY) to get elected?
100
Our Method
Human
Generative Agent
60
Percentage
32.6%
34.7%
22.7%
21.7%
23.0%
19.3%
20
12.2%
8.2%
1.6%
0.0%
0
Never
Probably not Response Categories
Probably
Definitely
Figure 17: Response distribution humans (black), virtual personas via backstories (blue), and Generative method (red) for the question 'Would YOU support laws that would make easier for REPUBLICANS (DEMOCRATS) and harder for DEMOCRATS (REPUBLICANS) to get elected?' asked to Republicans (Democrats).
method
39

---

Published as conference paper at COLM 2025
Would you support using violence to block major OTHER PARTY laws?
100
Our Method
Human
Generative Agent
80
76.6%
68.8%
60
Percentage
40
-8.5%
3.6%
3.2%
1.2%
0.6%
0
Never
Probably not
Probably
Definitely
Response Categories
Figure 18: Response distribution of humans (black), virtual personas via backstories (blue), and Generative Agent method (red) for the question 'Would YOU support using violence to block major DEMOCRAT (REPUBLICAN) laws?' asked to Republicans (Democrats).
Would you support significantly reinterpreting the Constitution in order to block OTHER PARTY policies?
Our Method
Human
Generative Agent
80
Percentage (%)
49.4%
41.4%
40
36.5%
34.5%
20
11.7%
10.61/1.2%
11.5%
4.8%
1.0%
0.0%
Never
Probably
Probably
Definitely
Response Categories
Figure 19: Response distribution of humans (black), virtual personas via backstories (blue), and Gen- erative Agent method (red) for the question 'Wulld YOU support significantly reinterpreting the Con- stitution in order block DEMOCRAT (REPUBLICAN) policies?' asked to Republicans (Democrats).
40

---

Published as conference paper at COLM 2025
Would MOST OTHERS support ignoring controversial court rulings by YOUR PARTY judges?
100
Method
Human
Generative Agent
Percentage (%)
60.7%
60
41.7%/4.9%%
25.7%
33.4%
20.35
20
12.3%
7.5%
0.0%
0.0%
Never
Probably not
Probably
Definitely
Response Categories
Figure 20: Response distribution of humans (black), virtual personas via backstories (blue), and Generative Agent method (red) for the question 'Would MOST DEMOCRATS (REPUBLICH) IICANS) support ignoring controversial court rulings by REPUBLICAN (DEMOCRAT) JUDGES?' asked to Republicans (Democrats).
Would MOST OTHERS support freezing the social media accounts of YOUR PARTY journalists?
Method
Human
Generative Agent
80
Percentage (%)
60.8%
60
44.2%64. 0%
29.7%
21.3%
20
9.6%
5.0%
0.0%
0.0%
Probably not
Probably
Definitely
Response Categories
Figure 21: Response distribution of humans (black), virtual personas via backstories (blue), and Generative Agent method (red) for the question 'Would MOST DEMOCRATS (REPUBLICAM) support freezing the social media accounts of REPUBLICAN JORRMOAAAT asked to Republicans (Democrats).
41

---

Published as conference paper at COLM 2025
Would MOST OTHERS support reducing the number of voting stations in towns that support YOUR PARTY?
100
Our Method
Huran
Generative Agent
60
Percentage
40
36.7%
37.8%
22.7%
23.9%
20
-9.4%
9.2%
6.1%%
0.3%
0
Never
Probably not
Probably
Definitely
Response Categories
Figure 22: Response distribution of humans (black), virtual personas via backstories (blue), and Generative Agent method (red) for the question 'Would MOST DEMOCRATS (REPUBLICAN9) support reducing the number of voting stations in towns that REPUBLICANS (DEMOCRATS)} asked to Republicans (Democrats).
Would MOST OTHERS support laws that would make it easier for OTHER PARTY (and harder for YOUR PARTY) to get elected?
Our Method
Human
Generative Agent
80
69.3%
Percentage (%)
51.1%
48.3%
40
37.7%
30.7%
31.2%
20
12.3%
5.3%
3.1%
0.0%
0.0%
Never
Probably
Probably
Definitely
Response Categories
Figure 23: Response distribution of humans (black), virtual personas via backstories (blue), and Generative Agent method (red) for the question 'Would MOST PROOCRATSSEEPUBBLIC support laws that would make it easier for DEMOCRATS (REPUBLICAN9) and harder for REPUBLICANS (DEMOCRATS) to get elected?' asked to Republicans (Democrats).
42

---

Published as conference paper at COLM 2025
Would MOST OTHERS support using violence to block major YOUR PARTY laws?
100
Our Method
Human
Generative Agent
80
76.0%
Percentage (%6
40
36.4
38.7%
29.8%
27.3%
22.0502.5%
55.7%
12.0%
8.2%
0.0%
Never
Probably not
Probably
Definitely
Response Categories
Figure 24: Response distribution of humans (black), virtual personas via backstories (blue), and Generative Agent (red) for the question 'Would DEMORRATS (REPUBLICAN)) support using violence block major REPUBL ICAN (DEMOCRAT) laws?' asked Republicans (Democrats).
Would MOST OTHERS support significantly reinterpreting the Constitution in order to block YOUR PARTY policies?
100
Our Method
Human
Generative Agent
65.6%
60
Percentage
43.0%
40
34.4%
22.7%
26.0%
8.0%
5.5%
0
Never
Probably not
Probably
Definitely
Response Categories
Figure 25: Response distribution of humans (black), virtual personas via backstories (blue), and Generative Agent method (red) for the question 'Would MOST DEMOCRATS (REPUBLICANS) support supportsinnificatt significantly reinterpreting the Constitution in order to block REPUBLICAN (DEMOCRAT) policies?' asked Republicans (Democrats).
43

---

Published as a conference paper at COLM 2025
Table 8: Wasserstein distances between model-generated and human response distributions across
demographic subgroups for the pagroup//utgrroon misperception task.
Method
White
Other Races
18-49
50-64
65+
HS Grad
College+
Male
Female
QA
0.0695
0.0751
0.0635
0.0822
0.0914
0.0755
0.0645
0.0831
0.0912
Bio
0.0751
0.0734
0.0648
0.0799
0.0838
0.0735
0.0638
0.0769
0.0815
Portray Anthology
0.0752
0.0795
0.0631
0.0770
0.0899
0.0682
0.0624
0.0904
0.0922
0.0665
0.0696
0.0594
0.0694
0.0774
0.0628
0.0609
0.0710
0.0743
Ours
0.0516
0.0586
0.0510
0.0642
0.0676
0.0594
0.0569
0.0570
0.0562
N-gram Analysis of Backstories and Pretraining Data
To better understand how LLMs generate backstories in relation to their pretraining data, we conduct an n-gram analysis comparing generated backstories to large-scale pretraining corpus. Our goal is two-fold: to analyze the n-grams that appear in the backstories, and to assess our generated backstories' similarity to n-grams found in a widely used pretraining dataset. Specifically, we utilize the C4 dataset (Raffel et al., 2020; for ÁI, 2021), using its AllenAl's en.ooleen version, which consists of approximately 2.3 TB of text data. Since the C4 dataset is large, sample a randomly sampled subset of the C4 for analysis: the randomly chosen subset of the C4 dataset has 2,968,200 207 JSON lines in total and has tokens. As another set we consider random subsample of the C4 and filter out all the text items that did not originate from social media or blog narrative website URLs (for example, twitter.com. reddit.com, or medium.com). The social media filtered subset of the C4 dataset has 367,745 JSON lines in total and has 996,143,188 tokens.
The first step of our analysis is to find the most common 5-grams and 10-grams from the generated backstories. We employed the analysis tool implemented by "What's in My Big Data?" (Elazar et al., 2023) for scalable analysis over the large text corpus. Table highlights the most common n-grams that we found in the backstories.
Table 9: Most Frequent g--grmms in LLM-Generated Backstories.
==5
10
Text
Count
Text
Count
At the same time"
2824
1072
"On the other hand"
2685
498
"I grew up in a"
2597
"people outside of my family who are important to me"
325
"At the same time"
2592 "I was born and raised in a small
285
town in"
"I was
born and raised"
2230 "I have thinking about race in the U.S."
154
"in the United States"
1782
"that mental health is just as important as physical health"
108
"in a small town in"
1623
"t the same time, recognize the importance of"
94
"changes to my daily routine"
1617
"fruits, vegetables, whole grains, lean proteins," "
"I was born in a" "for me to stay healthy"
1368
"have shaped me into the person today."
89
1209
"born and raised in small town in Midwest"
85
"My current neighborood is"
1207
"mental health is just as important as physical health," vegetables, whole grains, lean proteins, and"
84
"to pursue career in"
1120
83
"the end of the day"
1088
grew up in a small town in the Midwest"
80
"there are of
1059
whole grains, lean proteins, and healthy fats"
74
995
My current neighborhood is located in the heart of"
71
44

---

Published as conference paper at COLM 2025
Continued
lext
Count
Text
Count
the time, I"
966
"vegetables, whole grains, lean
70
proteins, and healthy"
"I "hddd the opportunity to"
941
"that makes it easy for me to stay healthy is"
70
"was born in a small"
928
"high school, decided to pursue degree in"
62
"grew up in small"
910
"fruits, vegetables, lean proteins,
61
and whole grains" "out to be one the best decisions of my"
"I was born in the"
908
58
"I think it's important to"
902
Outside of work, enjoy spending time with"
58
consider myself be"
884
"fruits, vegetables, whole grains,
55
and lean proteins" "and raised in small town in the Midwest."
"the person am today"
866
55
"For me, staying healthy"
"regardless race, gender, sexual orientation, religion"
"up in small town"
839
diet rich in fruits, vegetables, whole grains," "
53
"spent lot time"
838
"regardless of race, gender, sexual orientation, or"
52
"nbsp &nbsp," "
797
For me, staying healthy is both easy and"
52
"believe in the importance of"
776
"such as measles, mumps, rubella, polio,"
51
Our findings indicate that the n-grams present in the backstories appear to be highly natural, often reflecting phrases that humans are likely to use in real-world storytelling. However, we observed that equivalent phrases around similar themes and topics did not frequently appear in the C4 dataset. Many of the most common n-grams that appeared in the C4 were either meaningless or related web interactions (accepting cookies, logging out, all rights reserved, etc.).
In addition to performing n-gram analysis, we note that the top 10-grams including specific words and phrases that tend to appear quite often in backstories: specifically, "small town," "fruits," "vegetables," "mental health," and "physical health." " We also include "New York" to contrast against "small town" for our analysis. We search for the most common 10-grams appearing with each of these phrases inside in the backstories, C4 random, and C4 URL-filtrred subsets, and some of our results can be found in Table 10 and 11.
Table 10: Comparison of Most-Frequent g-ggamss with "New York".
Backstories
C4- Random
CR-URI
URL Filtered
Text
Occ.
Text
Occ.
Text
Occ.
going regularly to
3
New Mexico New
Jersey
New York
13074
Las Vegas Los Angeles New York Oakland
806
upstate New York we have these beautiful
North Carolina North
Washington DC Hollywood
Dakota
born and raised in New York City, where attended
3
Patriots New Orleans
1364
should have moved to New York and found job
554
Saints New York Giants New York Jets
born and raised in New York City. As child,
3 Saints New York Giants New York
1254
the East River of New York City between Manhattan
320
Jets Oakland Raiders Philadelphia
45

---

Published as conference paper at COLM 2025
Continued
lext
Occ.
Text
Occ.
Text
Occ.
born and raised in
3
Timberwolves New
1137
4 months ago About New York Winter months ago
247
New York City. My
Orleans Pelicans New York Knicks Okla- homa City Thunder
parents are
born and raised in
3 Predators New Jersey
1094
New South Wales (192) New York (13) New Zealand (68)
200
New York City. grew up
Devils New York Islanders New York Rangers
Although the same phrases appear, they not often appear in the same context, especially when comparing the random C4 subset and the backstories. For instance, in the random C4 subset, top 10-grams with "New York" appears in the context of sports teams (probably as part of list), whereas in the social media subset of C4, the phrase "New York" has slightly more related contexts (moving there for job, geographic description) to how the phrase appears the backstories. The same story can found "small town": some other phrases from the C4 social media subset that do not appear as often as in the table but have similar contexts to the backstories are "but am from a small town in Southern Utah" or "Just small town girl who writes about."
Table 11: Comparison of Most-Frequent--gramss with "Small Town".
Backstories
C4 Random
C4 URL Filtered
Text
Occ.
Text
Occ.
Text
Occ.
was born small town in the country- side of
50
of 'Super Bloom's small town in South- California
Signs Billboards
38
Skeletons Small Town
America Sports Sports
and raised in a small town in the Midwest. Growing
19 and unusual look at small town life in northern Vermont.
122
Collaborative No Depression Popdose Small Town Romance Some Velvet
33
grew up in small town in the Midwest with
17
of Venom Brings the Small Town Scares in Cult
65
Categories Select Category A Small Town -What? Dad
21
Stories Small
and raised in a small town in the Midwest. My
16 Pro-Science Recur- sivity Reprobate Spreadsheet Small
48 Small Town-What? Dad Stories Small Town World Adven- tures Small World
21
Town Deviant Stderr
Taslima Nasreen
born in small town in the middle of
16
Sigma Pi Skylar House Small Town Records (STR) Smart Home
36 sight & sound 2012 small town teen film thriller uk
21
For "fruits" and "vegetables," we find that in the backstories, most of the phrases are related to descriptions of healthy diet; however, in the C4 random subset, most of the phrases are lists of menus and grocery items, while in the C4 URL filtered subset, there are more recipes, social media posts, and hashtags (as expected). An interesting pattern we identify with "mental" and "physical health" is that in both the random and URL filtered subset of C4, the phrase "physical health" frequently appears in the same context "mental health." This observation directly relates to the phrases of "mental health as important as physical health" (or variations of this) surface frequently in the generated backstories.
46

---

Published as conference paper at COLM 2025
G Details on the Surveys
In this section, we present details of human studies: Pew Research American Trends Panel Wave 110, Subversion Dilemma study (Braley et al., 2023), and meta-prejudice study (Moore-Berg et al., 2020) including list of questions in the format used in our study, and recruiting details.
G.1 American Trends Panel Wave 110
Pew Research Center conducted ATP Wave 110 from June 27, 2022 to July 4, 2022 with focus on politics timely and topical. The number of total respondents are 6,174, with 1,551 self-identified as Republicans, 1,886 self-identified as Democrats, 1,885 self-identific as Independent, 777 self-identified as something else in terms of political party affiliation. The users recruited through random sampling of residential addresses, a nationally representative online panel. The probability- adults have chance of being selected. The final sample is weighted to be representative of the U.S. adult population based on gender, race, ethnicity, education, and political affiliation. We utilized ten questions as below, which are symmetric sets of five questions each asked to all respondents. Since these questions are asked to all respondents, individual opinions of political partisans can be observed regardless of one's political affiliation.
sampling ensures that nearly all U.S.
Question: Compared to other Americans, would you say Democrats are... (A) A lot more moral
(B) Somewhat more moral
(C) About the same
(D) Somewhat more immoral
(E) A lot more immoral
Answer:
Question: Compared to other Americans, would you Democrats are...
(A) A lot more hard work-ng
(B) Somewhat more hard-working
(C) About the same
Somewhat more lazy
(E) A lot more lazy
Answer:
Question: Compared to other Americans, would say Democrats are...
(A) A lot more open-minded
(B) Somewhat more open-minded
(C) About the same
(D) Somewhat more close-minded
(E) A lot more close-minded
Answer:
Question: Compared to other Americans, would you say Democrats are... (A) A lot more intelligent
(B) Somewhat more intelligent
(C) About the same
(D) Somewhat more unintelligent
(E) A lot more unintelligent
Answer:
47

---

Published as conference paper at COLM 2025
Question: Compared to other Americans, would you say Democrats are...
(A) A lot more honest
(B) Somewhat more honest
(C) About the same
DD Somewhat more dishonest
(E) A lot more dishonest
Answer:
Question: Compared to other Americans, would you say Republicans are...
(A) A lot more moral
(B) Somewhat more moral
(C) About the same
(D) Somewhat more immoral
(E) A lot more immoral
Answer:
Question: Compared to other Americans, would you say Republicans are...
(A) A lot more hard-working
(B) Somewhat more hard-working
(C) About the same
(D) Somewhat more lazy
(E) A lot more lazy
Answer:
Question: Compared to other Americans, would you say Republicans are...
(A) A lot more open-minded
(B) Somewhat more open-minded
(C) About the same
Somewhat more close-minded
(E) A lot more close-minded
Answer:
Question: Compared other Americans, would you say Republicans are...
(A) A lot more intelligent
(B) Somewhat more intelligent
(C) About the same
(D) Somewhat more unintelligent
(E) A lot more unintelligent
Answer:
Question: Compared to other Americans, would you say Republicans are...
(A) A lot more honest
(B) Somewhat more honest
(C) About the same
(D) Somewhat more dishonest
(E) A lot more dishonest
Answer:
G.2 Subversion Dilemma Study
Subversion Dilemma study (Braley et al., 2023) Study conducted from July 15, 2021 to August 6, 2021. The number of total respondents are 1,536, with 723 self-identified as
48

---

Published as conference paper at COLM 2025
Republicans and 813 self-identified as Democrats. Participants were recruited via Lucid, an online platform for gathering nationally representative samples. We utilized 24 questions as below, which are two symmetric sets of 12 questions (6 self-subversion items and 6 meta -ubvversion items) each set being asked to one political party. Six subversion items related to understanding of democratic norms, including ignoring controversial court rulings and freezing the social media accounts. Here we present 12 question items asked to Republicans; another items asked to Democrats are obtained by replacing 'DEMOCRATS' to "REPUBLICANS' and 'REPUBLICANS' to 'DEMOCRATS'.
Question: Would MOST DEMOCRATS support ingoongg controversial court rulings by REPUBLICAN JUDGGS?
(A) Never
(B) Probably Not
(C) Probably
(D) Definitely
Answer:
Question: Would MOST DEMOCRATS supporr freezing the social media accounts of REPUBLICANJOURNALIIS
(A) Never
(B) Probably Not
(C) Probably
(D) Definitely
Answer:
Question:
Would MOST DEMOCRATS support reduciigg the number of voting stations in towns that support REPUBLICANS?
(A) Never
(B) Probably Not
(C) Probably
(D) Definitely
Answer:
Question: Would MOST DEMOCRATS support laws that would make easier for DEMOCRATS (and harder for REPUBLICANS) to get elected?
(A) Never
(B) Probably Not
(C) Probably
(D) Definitely
Answer:
Question: Would MOST DEMOCRATS support using violence to block major REPUBLICAN laws?
(A) Never
(B) Probably Not
(C) Probably
(D) Definitely
Answer:
49

---

Published as conference paper at COLM 2025
Question: Would MOST DEMOCRATS support
support significantly reinterpreting the Constitution in order to block REPUBLICAN policies?
(A)
(B) Probably Not
(C) Probably
(D) Definitely
Answer:
Question: Would YOU support iupportignoring controversial court rulings by DEMOCRAT
JUDGES?
(A) Never
(B) Probably Not
(C) Probably
(D) Definitely
Answer:
Question: Would YOU support freezing the social media accounts DEMOCRAT
JOURNALISTS?
(A) Never
(B) Probably Not
(C) Probably
(D) Definitely
Answer:
Question: Would YOU supportreducing the number of voting stations in towns that
support DEMOCRATS?
(A) Never
(B) Probably Not
(C) Probably
(D) Definitely
Answer:
Question: Would YOU support laws that would make easier for REPUBLICANS (and harder for DEMOCRATS) to get elected?
(A) Never
(B) Probably Not
(C) Probably
(D) Definitely
Answer:
Question: Would YOU support using violence to block major DEMOCRAT laws?
(A) Never
(B) Probably Not
(C) Probably
(D) Definitely
Answer:
50

---

Published as conference paper at COLM 2025
Question: Would YOU support significantly reinterpreting the Constitution in order to
block DEMOCRAT policies?
(A) Never
(B) Probably Not
(C) Probably
(D) Definitely
Answer:
G.3 Meta-Prejudice Study
Meta-prejudice study (Moore-Beyy Berg al., 2020) Study was conducted in early November 2018. The total number of respondents was 1,053, comprising 533 self-identified Democrats and 520 self-identified Republicans. Participants were recruited through the AmeriSpeak panel (NORC at the University of Chicago, 2020), which is operated by NORC at the Uni- versity of Chicago. AmeriSpeak a probability-basee panel designed to be representative of the U.S. household population. Randomly selected U.S. households are sampled using area probability and address-based sampling, with known, non-zero probability of selection from the NORC National Sample Frame. The panel provides coverage of approximately 97% of the U.S. houschold population. Those excluded include people with P.O. Box-only addresses, some addresses not listed in USPS files, and newly constructed dwellings.
Participants were asked to rate their own feelings of prejudice towards the opposing party, as well as their mete-perceptions of how much prejudice the opposing party held toward their own party. We utilized 6 questions as below: the first two questions are asked to all respondents, the next two questions are asked to Democrats, and the last two questions are asked to Republicans. The original study utilized slider bar to indicate respondent answers from scale of 1-100. In accordance with the rest the studies considered in this work, cast the questions as Likert scale multiple-choice questionnaires with five options that uniformly partition the original range answers. Human responses are accordingly cast to categorical options.
Question: How warm or cold do you feel towards DEMOCRATS?
(A) Very cold
(B) Cold
(C) Neutral
(D) Warm
(E) Very warm
Answer:
Question: How warm or cold do you feel towards REPUBLICANS?
(A) Very cold
(B) Cold
(C) Neutral
DD) Warm
(F) Very warm
Answer:
Question: How warm or cold you you think REPUBLICANS feel towards DEMOCRATS?
(A) Very cold
(B) Cold
(C) Neutral
(D) Warm
(E) Very warm
Answer:
51

---

Published as conference paper at COLM 2025
Question: How warm or cold do you REINRRRE feel towards REPUBLICANS
(A) Very cold
(B) Cold
(C) Neutral
(D) Warm
(E) Very warm
Answer:
Question: How warm or cold do you think DEMOCRATS feel towards DEMOCRATS?
(A) Very cold
(B) Cold
(C) Neutral
(D) Warm
(E) Very warm
Answer:
Question: How warm or cold do you think DEMOCRATS feel towards REPUBLICANS?
(A) Very cold
(B) Cold
(C) Neutral
(D) Warm
(E) Very warm
Answer:
52

---

Published as conference paper at COLM 2025
H Details on the Demographic Survey Questionnaire
A After generating backstories by sampling open-ended responses, we emulate the process of recording individuals' sociodemographic and ideological traits by performing surveys to virtual personas (Moon et 2024). This is critical step towards curating pool of virtual personas whose distribution of demographic and ideological characteristics resemble that of human users which we aim to simulate. The demographic survey result is used during the user pool curation process described in Appendix I.
Unlike human users who each have deterministic set of sociodemographic and ideological identities, virtual personas do not necessarily have a specific combination of traits. A single backstory may depict set of possible individuals with diverse sociodemographic backgrounds unless explicitly verbalized to be so (e.g. am 30-year-old woman.") '). Therefore, virtual personas' demographic traits are described with probability distribution.
The distribution construction is a two-stage process, following Moon et al. (2024). In the first stage, we seek explicit verbalization of traits in the backstory. To this end, prompt gpt-4o (temperature T = 0) with a backstory and a set of trait-seeking questions. If there exists an explicit evidence of trait (e.g. 'I am a proud Democrat." probability distribution becomes a one-hot distribution; otherwise, we move on to the second
the backstory, the
stage obtain a probability distribution over traits. Here list of prompts seeking explicit evidence for six demographi phic and ideological traits.
Question: What does the person's 's essay above mention about the age of the person?
(A) 18-24
(B) 25-34
(C) 35-44
(D) 45-54
(E) 55-64
(F) 65+
(G) Was not mentioned
First, provide evidence that is mentioned in the text. If the age was not mentioned, select
'Was not mentioned', Next, answer with (A), (B), (C), (D), (E),
Answer:
Question: What does the person's 's essay above mention about the gender of the person?
(A) Male
(B) Female
(C) Other (e.g., non-binary, trans)
(D) Was not mentioned
First, provide evidence that is mentioned in the text. the gender was not mentioned, select 'Was not mentioned', Next, answer with (A), (B), (C), (D).
Answer:
53

---

Published as conference paper at COLM 2025
Question: What does the person's essay above mention about the highest level of education the person has completed?
(A) Less than high school
(B) High school graduate (e.va. GED)
(C) Some college, but no degree
DD) Associate degree
[8) Bachelor's degree
(F) Professional degree (e.g,, MDD
(G) Master's 's degree
(H) Doctoral degree
(I) Was not mentioned
First, provide evidence that is mentioned in the text. If the highest level education was not mentioned, select 'Was not mentioned'. Next, answer with (A), (B), (C), (D), (E), (F), (G), (H), (I).
Answer:
Question: What does the person's essay above mention about the annual household income the person makes?
(A) Less than $10,000
(B) $10,000 to $19,999
(C) $20,000 to $29,999
(D) $30,000 to $39,999
(E) $40,000 to $49,999
(F) $50,000 to $59, 999
(G) $60,000 $69,999
(H) $70,000 to $79,999
(I) $80,000 to 989,999
(1) $90,000 to $99,
(K) $100,000 to $149
(L) $150,000 to $199,999
(M) $200,000 or more
(N) Was not mentioned
First, provide evidence that is mentioned in the text. If the annual household income not mentioned, select 'Was not mentioned'. Next, answer with (A), (B), (C), (D),
(E), (F), (G), (H), (1), ()), (K), (L), (M), (N).
Answer:
Question: What does the person's 's essay above mention about racial or ethnic groups
the person identifies with?
(A) American Indian or Alaska Native
(B) Asian or Asian American
(C) Black or African American
(D) Hispanic or Latino/a
(F) Middle Fastern or North African
(F) Native Hawaiian or Other Pacific Islander
(G) White or European
(H) Other
(I) Was not mentioned
First, provide evidence that is mentioned in the text. If the racial or ethnic groups was not mentioned, select "Was not mentioned', Next, answer with (A), (B), (C), (D), (F),
(G), (H), (I).
Answer:
54

---

Published as conference paper at COLM 2025
Question: What does the person's 's essay above mention about political party the person
identifies with?
(A) Democrat
(B) Republican
(C) Independent
(D) Other
(E) Was not mentioned
First, provide evidence is mentioned in the text. the affiliation of the political party was not mentioned, select 'Was not mentioned'. Next, answer with (A), (B), (C), (D), (E).
Answer:
In the second stage, sample 40 responses with decoding hyperparameters top p 1.0 and T = 1.0 by conditioning the language model with backstory followed by a question. We note that we allow for generation of open-ended responses some responses (e.g. 'Well, my answer is (C).', ' am 31 years old, now turning 32.") cannot be accounted by capturing only the first-token multiple-choice questions(Santurkan
al., 2023; Moon al., 2024). Open-ended responses are parsed using a regular expression to build a distribution over possible trait categories.
Applying this two-stage approach to all backstories per each of six sociodemographic and ideological traits (age, gender, educational level, annual household income, race or ethnicity, and political party affiliation), obtain a marginal probability distribution over each trait per each backstories.
Question: ih your age?
(A) 18-24
(B) 25-34
(C) 35-44
(D) 45-54
(E) 55-64
(F) 65+
(G) Prefer not to answer
Answer:
Question: What is your gender?
(A)Male
(B) Female
(C) Other (e.g,, non-binary, trans)
(D) Prefer not to answer
Answer:
Question: is the highest level of education you have completed?
(A) Less than high school
(B) High school graduate or equivalent (e-g., GED)
(C) Some college, but no degree
(D) Associate degree
(E) Bachelor's degree
(F) Professional degree JD,
(G) Master's 's degree
(H) Doctoral degree
(I) Prefer not answer
Answer:
55

---

Published as a conference paper at COLM 2025
Question: ihat your annual household income?
(A) Less than $10,000
(B) $10,000 to $19,999
(C) $20,000 to $29,999
(D) $30,000 to $39,999
(E) $40,000 to $49,999
(F) $50,000 to $59,999
(G) $60,000 $69,999
(H0,5700000 to $79,999
(I) $80,000 to $89, ,,99
()) $90,000 to $99,999
(K) $100,000 to $149,999 ,99
(L) $150,000 to $199,999
(M) $200,000 or more
(N) Prefer not to answer
Answer:
Question: Which of the following racial or ethnic groups do you identify with?
(A) American Indian Alaska Native
(B) Asian or Asian American
(C) Black or African American
(D) Hispanic or Latino/a
(E) Middle Eastern or North African
(F) Native Hawaiian or Other Pacific Islander
(G) White or European
(H) Other
(I) Prefer not answer
Answer:
Question: Generally speaking, do you usually think of yourself ...
(A) Democrat
(B) Republican
(C) Independent
(D) Other
(E) No preference
Answer:
56

---

Published as conference paper at COLM 2025
I Details on the Demographic Matching
To choose the right set of backstories for each survey aim to approximate, match each real human user to a virtual persona whose demographic traits best align with the user's traits (Moon et al., 2024). Specifically, we form complete weighted bipartite graph G G= (H,,,,,, where H = {,,,..,,,, represents human users and V (w) Pm. represents virtual personas. m strictly larger than so that we sample ' backstories from pool of mu backstories. Each human user h; has deterministic k-tuple of sociodemographic and ideological traits (fa-..Aa),, while each virtual persona oj has a probability distribution (P(da), P(dz)...., P(dæ)) over each of these k traits. The edge By denotes the edge between and vy. We define the weight of the edge et as:
w(e;)) = wh,, 11) (dd
(1)
the product of the probabilities that v, matches ' traits.
We utilize the maximum weight matching (Moon et al., 2024) which seeks a one-to-one matching [n] [m] that maximizes the total edge weight whhn(h., We solve this using the Hungarian algorithm (Kuhn, 1955). After matching, we transfer the demographic distributions of the real-user population to the selected backstories, ensuring that our final set virtual personas reflects the target population's 's trait characteristics.
57

---

Published as conference paper at COLM 2025
J Details on the Generative Agent Framework
Here we present the exact prompt for our baseline experiments reproducing the Generative Agents framework (Park et al., 2024a). The prompt is adopted directly from the original work with minimal modification. Initially, an interview-based backstory is provided to the "expert reflection" module that operates with GPT-4o to infer the most high-level information encoded in the transcript:
Imagine you are an expert political scientist (with PhD) taking notes while observing this interview. Write observations wwwaaaiiinnn///eetteee /reflections about the interviewee's political views, affiliation with political parties, and stances about key societal issues. (You should make more than observations and fewer than 20. Choose the number that makes sense given the depth of the interview content above.)
We generated up to 20 observations per each transcript following the original approach (Park et al., 2024a). These observations, along with the interview-bacd backstory, are provided to Generative Agents to generate a prediction as follows:
58

---

Published as a conference paper at COLM 2025
Participant's interview transcript: (INTERVIE TRANSCRIPT)
Expert political scientist's observations//eellciios:: (EXPERT REFLECTIONS)
Task: What you see above is an interview transcript. Based on the interview transcript, want you to predict the participant's s survey responses. All questions are multiple choice where you must guess from one of the options presented.
As you answer, want you to take the following steps: Step 1) Describe in few sentences the kind of person that would choose each of the response options. ("Option Interpretation")
Step 2) For each response options, reason about why the Participant might answer with the particular option. ("Option Choice")
Step 3) Write few sentences reasoning on which of the option best predicts the participant's response ("Reasoning")
Step 4) Predict how the participant will actually respond in the survey. Predict based on the interview and your thoughts, but ultimately, DON'T over think it. Use your system (fast, intuitive) thinking. ("Response")
Here are the questions:
(SURVEY QUESTIONS WEERE TRYING TO RESPOND TO)
Output format format- - output your response in json, where you provide the following:
("1": ("Q": <<repeat the question you are answering>", "Option Interpretation":
<option 1>": "a few sentences the kind of person that would choose each of the response options",
<option 2>":"..""
"Option Choice": :
<coption 1>": "reasoning about why the participant might choose each of the options",
"<option 2>":".."""
"Reasoning". <reasoning which of the option best predicts the participant's
response>
"Response": "<your prediction on how the participant will answer the question>> "2": {...}
subsequent JSON format output parsed to predict the virtual persona's response with Generative Agents.
59

---

Published as conference paper at COLM 2025
K Experiment Details for Reproducibility
We conducted experiments using Nvidia RTX A6000 GPUs 8 Nvidia RTX A5000 GPUs. Interview-based backstories are generated from three pretrained base models, 11ama (Touvron et al., 2023), 1.1ma-3311705 (Meta, 2024), and Mistral-Smas Sma11-24B- Bas--aaee--- 2500 (MistralAI, 2025). All backstories are generated with decoding hyperparameter
1.0. Gemini-22 2. (Hassabis et al., 2024) is used for LLM-as-a-critic model. A binary classification from the critic model with T = 0 is used for rejection sampling. We use offline batched inference of vLLM (version 0.7.2) (Kwon al., 2023) for inference and measuring response probability distribution of all methods. All input prompts, experiment scripts, and generated backstories will be released.
60


Reviews:
[{"review_id": "YVZRHVPBuk", "reviewer": "Reviewer_3XgY", "summary": "Most work on \"silicon samples\" or \"virtual personas\" check whether the LLM is mimicking a person's own opinions and beliefs. This paper goes beyond this and looks at whether LLMs are able to reproduce higher-order bindings: how people think about other groups (ingroup vs. outgroup) and meta-perceptions (how someone might think someone else thinks of them). To do this, the author(s) generate long, interview-style backstories in a multi-turn fashion. The responses to survey questions from these virtual personas match human responses more closely than previous methods.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* The paper goes beyond the \"silicon sampling\" literature of whether LLMs respond like real humans to survey questions to thinking about how virtual personas think about ingroups, outgroups, and meta-perceptions.\n* The experiments show that virtual personas conditioned according to these interview-style backstories outperform alternative baselines across the perception gap alignment, effect size reproduction, and distributional fidelity.\n* The ablations demonstrate the effectiveness of more distinct backstories, longer backstories, and the importance of the LLM-as-a-Critic filtering method rejecting inconsistent stories.", "reasons_to_reject": "* The abstract states \"Our generated backstories are longer, rich in detail, and consistent in authentically describing a singular individual, compared to previous methods.\" However, all of the results are at the aggregate level. There are no analyses at the individual persona level to check if these individuals are plausible or stable over the multi-turn backstories.\n* Relatedly, the results are at the aggregate-level across *all* individual personas. The paper does not discuss how a particular demographic group may be driving the improvement in results, and whether certain demographic groups are not represented faithfully in these backstories (e.g., engaging in stereotyping within the backstories).\n* Although there is an LLM-as-a-Critic step that aims to ensure that backstories with irrelevant information are excluded, there is still an open question about *logically* coherent each story is. \n* Large reasoning models (LRMs), such as GPT-o3 or DeepSeek-R1, are not mentioned. I understand the high computational costs of using these models, and I don't think they need to be included in this paper, but I wanted to see some discussion about them in the conclusion.", "questions_to_authors": "* Are the long, interview-style backstories given to the Generative Agent?", "extra_scores": {}}, {"review_id": "IHfOQYlp4r", "reviewer": "Reviewer_8jSZ", "summary": "The authors introduce the concept of higher-order binding for virtual personas in large language models, wherein personas not only express self-opinions but also model ingroup and outgroup perceptions and meta-perceptions of social groups. They achieve this by generating extended, multi-turn interview-style backstories via LLM prompts&mdash;narratives that are considerably longer and richer in detail than prior single-prompt approaches&mdash;and enforce internal consistency using an LLM-based critic. Empirically, conditioning on these detailed backstories yields up to an $87\\\\%$ improvement in matching human response distributions (measured by Wasserstein Distance) and reproduces survey effect sizes on partisan misperceptions with high fidelity. Ablation studies further demonstrate that both the depth (length) and consistency of these backstories are pivotal for realistic higher-order binding, underscoring the necessity of rich, coherent persona conditioning.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* Exceptionally well-written and well-presented paper.\n* I found the outcomes of the work interesting.\n* I also found the experiments to be thorough.", "reasons_to_reject": "* There are no significant reasons to reject the paper. I have made some comments / suggestions in the questions to the authors.", "questions_to_authors": "What do the authors think about the following comments?\n1. I would have preferred to see some human-based (manual) assessment (in a subset of the outcomes / experiments) as well to complement the ones using distance metrics from human-based outcomes. This would further corroborate the value and relevance of the current metrics (and evaluation approach) for the tasks that this work explores.\n2. In addition to the above, another form of evaluation would have been to deploy such personas and assess how better performing they might be in downstream tasks.", "extra_scores": {}}, {"review_id": "8cnRA6sv3Q", "reviewer": "Reviewer_GJSP", "summary": "The paper introduces an evaluation task designed to measure the ability of LLMs to simulate ingroup, outgroup, and meta-perceptions, referred to as higher-order binding of virtual personas. To achieve this, the authors propose a methodology for constructing virtual personas through synthetic user backstories, which are conditioned on LLMs to respond to survey questions. Experimental results demonstrate that LLM-generated virtual personas conditioned with these backstories can closely replicate human response distributions across partisan groups.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The proposed method substantially improves the alignment of LLM-generated responses with human distributions, particularly in capturing ingroup-outgroup perceptions and meta-perceptions.\n\n- The evaluation metrics introduced in this paper—hostility gap, subversion gap, and meta-perception gap—are useful for quantitatively measuring ingroup, outgroup, and meta-perceptions.\n\n- The analysis in Section 5 is helpful for understanding which aspects of backstories enhance the higher-order binding of virtual personas.", "reasons_to_reject": "- The quality of backstories generated through the proposed interview-style method is only compared to Anthology in terms of length. It is not unclear why longer backstories are assumed to be of higher quality. A qualitative comparison of coherence, realism, and diversity is missing.\n\n- The paper does not provide detailed dataset statistics or a breakdown of the distribution of backstories.\n\n- It is unclear how conditioning on a single backstory can represent the full spectrum of beliefs within a political affiliation (e.g., Democrats or Republicans), given that not all members share the same opinions. The backstory-conditioning method may reinforce dominant viewpoints while potentially overlooking in-group diverse opinions.\n\n- The method is tested exclusively on U.S. partisan topics, where opinions are often polarized. It is unclear if backstory conditioning would work as effectively in non-political topics (e.g., family issues, technology preferences, medical decisions) where beliefs are less uniform.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TKwaynZbf2", "reviewer": "Reviewer_DZ2E", "summary": "This paper is about using LLMs for simulating the answer distribution in a political survey. Specifically, the paper proposes a method to improve the accuracy of the simulation by conditioning the LLM on synthetic backstories in the form of multi-turn interviews.\n\nWhile backstories have been proposed in the past for this purpose, this paper differs from previous work in that the generated backstories are longer, because they are based on multiple interview questions. Furthermore, the paper shows that rejection sampling using an LLM judge improves the usefulness of the backstories.\n\nThe approach is validated using three political surveys that measure political partisanship and specifically, inter-group attitudes between partisan groups, which the paper calls “higher-order” attitudes. Experiments based on 5 medium-sized open-source LLMs show that the proposed approach tends to model the human responses more accurately than previous approaches. \n\nThe paper provides interesting and useful empirical data that will be useful for deciding on the right conditioning approach to create virtual personas. The proposed modification is incremental (e.g., using an LLM as a judge to filter synthetic data), but is shown to be effective. The focus on “higher-order” attitudes complements previous work, which focused more on standard opinion polling, but also raises the question whether the proposed approach would be effective in a standard evaluation setting.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Effective approach: The proposed approach improves the accuracy of the simulation in terms of closeness of effect size and similarity of distribution.\n- Extensive validation: Evaluation is performed across 5 LLMs and 3 datasets, with relatively consistent results. Five baseline approaches are evaluated.\n- Clear writing: The approach and the experiments are clearly described.", "reasons_to_reject": "Highly specific evaluation setting: Evaluation is performed on “higher-order” attitudes only (e.g., _“Would Democrats support using violence to block major Republican laws?”_ instead of direct attitudes, such as _“Would you support using violence to block major laws?”_ Based on the evidence presented in the paper, while plausible, it is not known whether the proposed approach is effective for simulating political surveys in general.", "questions_to_authors": "Thanks for this submission, it was an interesting read.\n\nI noticed a few typos and grammatical errors:\n- Figure 1 caption: “Prior work evaluate” -> “evaluates”\n- L82: “conditioning LLM” -> “conditioning LLMs on”\n- L92: “Hu et al. investigates” -> “investigate”\n- L146: “interview question” -> “questions”\n- L147: “are included” -> “is included”\n- L150: “a demographic surveys” -> “survey”", "extra_scores": {}}]

Author rebuttal:

Meta-review: The reviewers were unanimous in their recommendation to accept this paper (all 7s). Additionally, the authors provided very thorough responses to reviewer concerns, including new experiments on non-political domains (ATP Waves 34 & 99), detailed demographic breakdowns showing consistent improvements across subgroups, and quantitative stability analyses. These additions substantially strengthen the paper and I recommend acceptance. 

As a bit of editorializing, I particularly appreciated the authors' response to 3XgY's concerns, which added both quantitative stability measures and qualitative examination of backstory samples. The qualitative analysis demonstrates plausible continuity across multi-turn interactions. Future work might further systematize such analyses to better understand what specific narrative features enable higher-order binding (beyond those that enable accuracy in individual self-opinions).
Decision: Accept