C o MP os T : Characterizing and Evaluating Caricature in LLM Simulations (2023)

Abstract: Recent work has aimed to capture nuances of human behavior by using LLMs to simulate responses from particular demographics in settings like social science experiments and public opinion surveys. However, there are currently no established ways to discuss or evaluate the quality of such LLM simulations. Moreover, there is growing concern that these LLM simulations are flattened caricatures of the personas that they aim to simulate, failing to capture the multidimensionality of people and perpetuating stereotypes. To bridge these gaps, we present CoMPosT, a framework to characterize LLM simulations using four dimensions: Context, Model, Persona, and Topic. We use this framework to measure open-ended LLM simulations’ susceptibility to caricature, defined via two criteria: individuation and exaggeration. We evaluate the level of caricature in scenarios from existing work on LLM simulations. We find that for GPT-4, simulations of certain demographics (political and marginalized groups) and topics (general, uncontroversial) are highly susceptible to caricature.

## introduction large language models (llms) have shown promise in capturing social nuances and human behavior. for instance, researchers have reproduced results from social science experiments and public opinion surveys using llms @xcite @xcite . more broadly, interest in llm simulation is rapidly growing, and the possibility of using llms to simulate human behaviors has far-reaching applications in fields like education @xcite , product design @xcite , psychology @xcite , healthcare @xcite , skill training @xcite , and law @xcite . these simulations are a sort of digital compost-any new insight into human be-the compost framework context where and when does the simulated scenario occur? model what llm is used? persona whose opinion/action is simulated? topic what is the simulation about? toward clearer documentation of this emerging line of work, we first present a descriptive framework that taxonomizes llm simulations using four dimensions: context, model, persona, and topic (compost) ( next, we introduce a new evaluation metric that addresses growing concerns of modal responses and essentializing narratives in llm outputs @xcite @xcite . our metric focuses on a simulation's susceptibility to caricature: an exaggerated narrative of the persona (the demographic that we aim to simulate) rather than a meaningful response to the topic (figure 2 ). caricatured simulations are concerning because they a) fail to capture the real nuances of human behavior, thus limiting the usefulness of simulations and b) perpetuate misleading descriptions, stereotypes, and essentializing narratives about demographic groups. we define caricature using two criteria: individuation and exaggeration (section 3). to measure individuation, we assess whether outputs from the given simulation can be differentiated from the default response for the topic. to measure exaggeration, we use a "contextualized semantic axis" whose two poles are the defining characteristics of the persona and topic dimensions respectively. we evaluate the level of caricature in scenarios from existing work on llm simulations. we find that for gpt-4, simulations of certain demographics (political and marginalized race/ethnicity groups) and topics (general, uncontroversial) are more susceptible to caricature. our main contributions are: (1) compost, a framework for characterizing the dimensions of llm simulations of human behavior (section 2), (2) a novel method that relies on the persona and topic dimensions of compost to measure simulations' susceptibility to for the simulation of a nonbinary person's response, the generations are focused on identity-related issues, while the simulation of a person's response is topical. the former constructs a homogenous narrative that defines nonbinary people only by lgbtq+ activism. we provide more qualitative examples in appendix a. ## compost: taxonomizing simulations we introduce compost, a descriptive framework with four dimensions to characterize llm simulations: context, model, persona, and topic. inspired by existing descriptive frameworks for ai fairness @xcite , our framework provides a shared language to understand and articulate similarities and differences across llm simulations. context, persona, and topic are specified in the prompt, while model is determined externally. we map existing work on llm simulations using these dimensions (figure 1 and table a4 ). ## caricature detection method the two key aspects of caricature are individuation and exaggeration. to measure the amount of caricature in a given simulation s p,t,c , our method has three steps, each of which rely on the persona and topic dimensions of compost (figure 3 ): (1) defining defaults, (2) measuring individuation, and (3) measuring exaggeration. note that this framework is sequential, as (3) is only necessary if the simulations can be individuated. otherwise, we can halt after step (2) since individuation is a necessary criterion for caricature. ## experiments we use our method to evaluate simulations in various contexts that have been used in previous work to demonstrate the capabilities of llm simulation @xcite @xcite . our experiments are focused on the two most widely-used contexts of (1) an online forum setting and (2) a question-answering interview setting. we also evaluate the twitter context as an additional robustness study (appendix g). these choices are based on our survey of the literature on llm simulations (figure 1 , table a6 ): among 15 papers in this area, we found that six use the context of a virtual community or society and four rely on an open-ended interview or survey context. the remaining five are in various question-answering contexts, so conclusions about the (2) can also provide insight into these types of simulations. we use the state-of-the-art gpt-4 model for all experiments (openai, 2023); like others @xcite , we find that open-source llms and older models are worse at simulation tasks, yielding outputs that are unrealistic and significantly lower in quality. for each simulation setting s p,t,c , we generate 100 outputs and average all results across them. see appendix e for a power analysis of this sample size. the full details for each setting, including topic lists, persona lists, and defaulttopic/persona prompts, are in appendix d. ## results and discussion we apply our caricature detection method to evaluate the simulations produced in these different contexts. we further operationalize the com-post framework by aggregating the individuation and exaggeration scores across the dimensions of topic and persona. this enables us to analyze the topics and personas that lead to the most caricatures across different contexts. for instance, to examine what personas lead to the most caricatures in a particular context, we compute the mean score for each persona across all simulations (varying in topic) for that persona and then compare these scores. we also report results from additional experiments that explore the influence of the context dimension in appendix f. ## recommendations we conclude with several recommendations for those interested in llm simulations. mitigating caricature researchers should use our method to test their simulation in their particular context and critically examine whether a simulation helps illuminate the desired phenomenon. while the relationship between topic, persona, and context in causing caricature is nuanced, we generally encourage researchers and practitioners to use more specific topics to mitigate caricature. any attempt to simulate a group-especially a politicized or marginalized group-ought to be done with particular care and attention. ## positionality the perspectives introduced in this paper have undeniably been shaped and influenced by our positionality. myra cheng identifies as a chinese-american woman. the authors are a phd student, postdoctoral scholar, and professor respectively in the stanford university computer science department, which is predominantly male and white/asian. ## ethical considerations from impersonation to pornography, llm simulations can have deeply problematic applications. we are strongly against such applications, and we also do not condone research and development that may enable such applications by bad actors without guardrails in place. our compost framework offers a shared language to meaningfully critique such work. for instance, one might imagine coming to a consensus to avoid simulating certain topics, personas, and contexts entirely. introducing a method to measure caricature offers a way to make known this concerning limitation. lack of caricature based on our measure does not mean that a simulation is necessarily acceptable or high quality (see section 10). implicit defaults the least caricatured personas are also those that others have found to be implicit defaults in llms. implicit defaults in llms may shift depending on the prompt, context, etc., as well as including aspects of identity and social factors that may be invisible or underrepresented in existing empirical data and surveys foot_3 . given the increasing proliferation of generated content and a limited quantity of human-written text @xcite , caricatures become only more relevant with the prospect of future llms that are trained on generated data: what will their defaults be, and how might they further amplify caricature? ## limitations while we fill a critical gap since there is no existing work on systematically detecting stereotypes/caricatures in simulations or evaluating simulations in this manner at all, our measure is limited in scope: it is not a comprehensive evaluation of the quality of a simulation. we quantify susceptibility to caricature, which is a particular failure case of a simulation. our method may yield false positives (simulations that seem acceptable and caricaturefree based on our method but have other problems). avoiding caricature is a necessary but insufficient criterion for simulation quality; our metric should be used in tandem with other evaluations. as a pilot study for a recently-emerging direction of work, we hope to lay the groundwork for a more comprehensive evaluation of simulations in the future, perhaps in tandem with human evaluation. as we provide a first step toward characterizing and evaluating llm simulations, an area which currently lacks a shared language for discussion and comparison, we focus only on simulations of personas that reflect subpopulations such as social groups and on one-round response formats. however, our framework easily extends to other more complex or open-ended settings. for instance, for a multi-round simulation, one could apply our framework by using the full text of the simulation across the rounds. depending on the length and structure of the simulation, one could also split the simulation into multiple parts and characterize each part's propensity to caricature. also, note that in the semantic axes, p p does not necessarily reflect a universal notion of the model's representation or description of that particular demographic group. it merely characterizes the words that distinguish a simulation of that persona, given the particular context c, such as an opinion from that demographic's perspective. this enables us to measure if and when a simulation is dominated by language that is a caricature of the persona, but our work is not a comprehensive evaluation of stereotypes or representations of demographic groups.