Virtual Personas for Language Models via an Anthology of Backstories (2024)

Abstract: Large language models (LLMs) are trained from vast repositories of text authored by millions of distinct authors, reflecting an enormous diversity of human traits. While these models bear the potential to be used as approximations of human subjects in behavioral studies, prior efforts have been limited in steering model responses to match individual human users. In this work, we introduce Anthology, a method for conditioning LLMs to particular virtual personas by harnessing open-ended life narratives, which we refer to as backstories. We show that our methodology enhances the consistency and reliability of experimental outcomes while ensuring better representation of diverse sub-populations. Across three nationally representative human surveys conducted as part of Pew Research Center’s American Trends Panel (ATP), we demonstrate that Anthology achieves up to 18% improvement in matching the response distributions of human respondents and 27% improvement in consistency metrics.

## introduction large language models (llms) are trained from vast repositories of human-written text @xcite @xcite @xcite . these texts are authored by millions of distinct authors, reflecting an enormous diversity of human traits @xcite . as a result, when a language model completes a prompt, the generated response implicitly encodes a mixture of voices from human authors that have produced the training text from which the completion has been extrapolated. although this nature of language models has * equal contribution. figure 1 : this work introduces anthology, a method for conditioning llms to representative, consistent, and diverse virtual personas. we achieve this by generating naturalistic backstories, which can be used as conditioning context, and show that anthology enables improved approximation of large-scale human studies compared to existing approaches in steering llms to represent individual human voices. a prominent example of such a scenario with growing significance is the use of llms to simulate human actors in the context of behavioral studies @xcite @xcite @xcite @xcite @xcite @xcite . llms have great potential as querying models is much faster and cheaper than designing and completing human studies @xcite , a process well-known to be challenging when striving to recruit large-enough, representative, unbiased, and just samples of subjects. while there are evident risks from llms themselves @xcite @xcite , including the inherent biases within models trained on internet data, the use of language models to perform approximate pilot studies can help survey designers satisfy best practices @xcite ) of beneficence and justice, without and before inflicting potential harm to real human respondents. for language models to effectively serve as virtual subjects, we must be able to steer their responses to reflect particular human populations, i.e. condition models to reliable virtual personas. to this end, existing work prompts llms with context that explicitly spell out the demographic and personal traits of the intended persona: for example, @xcite @xcite attempt to steer llm responses with a dialog consisting of a series of question-answer pairs about demographic indicators, a free-text biography listing all traits, and a portrayal of the said persona in secondperson point-of-view. while these approaches have shown modest success, they have been limited in (i) closely representing the responses of human counterparts, (ii) consistency, and (iii) successfully binding to diverse personas, especially those from underrepresented subpopulations. so how might we condition llms to virtual personas that are representative, consistent, and diverse? in this work, we investigate the use of naturalistic bodies of text describing individual life-stories, namely backstories, as prefix to model prompts for persona conditioning as illustrated in figure 1 . the intuition is that open-ended life narratives both explicitly and implicitly embody diverse details about the author, including age, gender, education level, emotion, and beliefs, etc. @xcite @xcite @xcite . lengthy backstories thus narrowly constrain the user characteristics, including latent traits as personality or mental health that are not solicited explicitly @xcite @xcite , and strongly condition llms to diverse personas. in particular, we suggest a methodology to generate backstories from llms themselves, as a means to efficiently produce massive sets covering a wide range of human demographics-which we refer to as an anthology of backstories. we also introduce a method to sample backstories to match a desired distribution of human population. our overall methodology is validated with experiments approximating well-documented large-scale human studies conducted as part of pew research center's american trends panel (atp) surveys. we demonstrate that language models conditioned with llm-generated backstories provide closer approximations of real human respondents in terms of matching survey response distributions and consistencies, compared to baseline methods. particularly, we show superior conditioning to personas reflecting users from underrepresented groups, with improvements of up to 18% in terms wasserstein distance and 27% in consistency. our contributions are summarized as follows: ## conditioning llms to virtual personas via an anthology of backstories in this section, we discuss details of the proposed anthology approach. we start with answering the core question: what are backstories and how might they help condition llms to particular personas when given as context? using an example, we examine and lay out the advantages of conditioning models with backstories in section 2.1. there are two practical considerations when using backstories as conditioned virtual personas for approximating human subjects. in the following a: i'm 37. i grew up in a small town, in a small house … a: certainly! i am a new college grad from new jersey … a: born and raised in tennessee, i had a blissful childhood … step 1. llm-generation of backstories step 3. match virtual personas to target human user distribution match to human user distribution (demographic variables) step 2. demographic survey on virtual personas q: what is your age? (a) 18-29 (d) 65 or above (b) 30-49 (d) prefer not to answer (c) 50-64 a: (b) 37 years old. backstory conditioned virtual persona q: what is the highest level of education you have completed ? (a) less than high school … … a: (e) bachelor's degree llm q: tell me about yourself. ## approximating human studies with llm personas in this section, we discuss the large-scale human studies that we aim to approximate (step 4 of figure 2 ) using llm virtual subjects, based on varying methods of persona conditioning. we detail the overall experimental setup and define criteria for evaluation. human study data the pew research center's american trends panel (atp) is a nationally representative panel of randomly selected u.s. adults, designed to track public opinion and social trends over time. each panel focuses on a particular topic, such as politics, social issues, and economic conditions. in this work, we consider atp waves 34, 92, and 99, a set of relatively recent surveys that cover a wide variety of topics: biomedical & food issues, political typology, and ai & human enhancement, respectively. for each wave, we select 6 to 8 questions from the original questionnaire that capture diverse facets of human opinions about the wave's topic using a likert scale. details on the questions selected and further information about each atp wave are discussed in appendix e. ## experimental results in this section, we describe experimental results that validate the effectiveness of our proposed methodology for approximating human subjects in behavioral studies. ## conclusion in this paper, we have proposed and tested a method, anthology, for the generation of diverse and specific backstories. we have demonstrated that this method closely aligns with real-world demographics and demonstrates substantial potential in emulating human-like responses for social science applications. while promising, the method also highlights critical limitations and ethical concerns that must be addressed. future advancements must focus on enhancing the representation and consistency of virtual personas to ensure their beneficial integration into societal studies.