Beyond Demographics: Aligning Role-playing LLM -based Agents Using Human Belief Networks (2024)

Abstract: Creating human-like large language model (LLM) agents is crucial for faithful social simulation. Having LLMs role-play based on demographic information sometimes improves human likeness but often does not. This study assessed whether LLM alignment with human behavior can be improved by integrating information from empirically-derived human belief networks. Using data from a human survey, we estimated a belief network encompassing 64 topics loading on nine non-overlapping latent factors. We then seeded LLM-based agents with an opinion on one topic, and assessed the alignment of its expressed opinions on remaining test topics with corresponding human data. Role-playing based on demographic information alone did not align LLM and human opinions, but seeding the agent with a single belief greatly improved alignment for topics related in the belief network, and not for topics outside the network. These results suggest a novel path for human-LLM belief alignment in work seeking to simulate and understand patterns of belief distributions in society.

## introduction with rapid advances in large language models (llms), there has grown increasing interest in using llms to simulate and understand dynamics of human communication and persuasion @xcite @xcite . current llms can be prompted to role-play as individuals with particular demographic traits, sometimes then producing patterns of behavior that seem remarkably human-like. for instance, when asked to report the us unemployment rate when president obama left office, chatgpt will provide the exact answer; but if first instructed to role-play as a typical democrat or republican and asked the same question, the model produces incorrect, inflated estimates that mirror patterns of partisan bias † joint second authors. ## preliminaries: llm agents as human digital twins as depicted in figure 1 , we aim to construct an llm agent i ′ as the i-th human's "digital twin", such that their opinions o on various topics x are aligned. we first use information about human i (e.g., their demographic information d) to create the corresponding llm agent i ′ , and then query the agent's opinion (o i ′ ) on a wide range of topics. we then evaluate the human-llm alignment by measuring the discrepancy between the actual human opinion o i and the llm agent's opinion o i ′ . note that we use the term llm-based "agent" to refer to the digital twin because the instructed llm is intended to produce behaviors that emulate the human individual they role-play @xcite @xcite . ## conclusion we investigated the use of empirically-derived belief networks for promoting alignment of expressed beliefs between large language model (llm) agents and twinned human participants. we showed that demographic role-playing alone does not produce significant alignment, but that initializing an agent with a human opinion on one topic then aligns opinions on nearby topics within the belief network. the effect does not extend to distant topics within the network. we found similar effects for in-context learning and supervised fine-tuning, for both a proprietary and an open-source llm. this work highlights a novel and potentially powerful means of enhancing llm agents' alignment with human opinions.