Benchmarking Distributional Alignment of Large Language Models (2025)

Abstract: Language models (LMs) are increasingly used as simulacra for people, yet their ability to match the distribution of views of a specific demographic group and be distributionally aligned remains uncertain. This notion of distributional alignment is complex, as there is significant variation in the types of attributes that are simulated. Prior works have underexplored the role of three critical variables—the question domain, steering method, and distribution expression method—which motivates our contribution of a benchmark explicitly addressing these dimensions. We construct a dataset expanding beyond political values, create human baselines for this task, and evaluate the extent to which an LM can align with a particular group’s opinion distribution to inform design choices of such simulation systems. Our analysis reveals open problems regarding if, and how, LMs can be used to simulate humans, and that LLMs can more accurately describe the opinion distribution than simulate such distributions.

## introduction it would be unusual to ask a person to accurately simulate a demographic group to which they do not belong. however, lms are increasingly being used in this way to simulate human behavior in applications ranging from agent-based simulations @xcite to piloting survey design @xcite @xcite @xcite . when simulating survey responses, there is no single "correct" answer, and it is important to evaluate if the distribution of model outputs is truly aligned with the intended human distribution. there has been considerable debate as to whether or not models can do this-some argue that the extensive training corpus of llms enables them to faithfully simulate demographic groups @xcite , while others show such simulations are inaccurate and stereotypical @xcite . one reason for these conflicting views is the heterogeneity in how one can measure distributional alignment, resulting in a lack of clarity around best practices. for example, current approaches have measured the model's opinion distribution with zero-shot, log-probability based evaluations, yet recent work in uncertainty quantification suggests verbalized distributions can outperform model logprobabilities @xcite . this raises the question of whether the model's distribution expression method is truly optimal and demonstrates a need for carefully controlled evaluations. in this work, we acknowledge the sensitivity of distributional alignment metrics and build a benchmark that studies several key variations in the distributional alignment task. our benchmarks and dataset measure how the distributional alignment of llms vary under (1) the distribution expression method, (2) the steering method, and (3) design choices in the dataset (fig. 1 ). our analyses reveal several open problems for distributional alignment. first, we find that existing measurement methods such as log-probabilities have systematically underestimated the distributional alignment of llms, and other simple baselines result in better alignment. second, we find that lms can more accurately estimate opinion distributions in text-based forms (e.g., 'return the distribution in a json'), compared to generating samples from the opinion distribution. this highlights a substantial opportunity to improve distributional alignment by closing the gap between a model's knowledge of human opinions and its ability to simulate them. finally, we find significant gaps in both alignment and steerability when simulating non-cultural opinions, such as book preferences, when compared to evaluations of stronger opinions (i.e., political and cultural values). we summarize our key contributions as follows: sequence: "bcdcdcdddbbdbdcddd" ## distribution expression methods ( ) eval: distributional alignment model log-probs: "c" ## problem statement we propose a benchmark that systematically evaluates the extent to which a language model can be aligned to the distribution of a particular demographic group's opinions, a task we term the distributional alignment problem. to begin, we formalize this task and visualize it in fig. 1 . let @xmath0,q . the goal is to understand how a language model can represent a group g through a steering method s, one that shifts an lm's opinion distribution to that of a particular group. concretely, the model will express a distribution ŷg,q with a distribution expression method o (e.g., model log-probabilities). we are interested in the distributional difference between the reference distribution, y g,q , and the model's estimate, ŷg,q . to evaluate this, we construct a set foot_0 y of ground truth human opinion distributions, where ## benchmark construction having formalized the notation for distributional alignment, we explore how it can be improved by focusing on three understudied sources of variation: the distributional expression method (o), steering method (s), and dataset (y ). in this section, we explain how these elements are used to construct the benchmark and describe the human baseline. ## experiments we rank gpt-4, gpt-3.5, anthropic haiku, anthropic opus, llama-3 70b instruct, foot_6 based on distributional alignment (eq. 1) and the knowledge-tosimulation gap (eq. 2), and average across groups, steering methods, and datasets. we start by describing the performance on the distributional alignment task. then, we dive into the implications that emerge from varying the distribution expression method, dataset, and steering method. ## conclusion llms perform surprisingly well on knowledgeintensive tasks, excelling on coding benchmarks and question-answer tasks to name a few. their success in these tasks has led to an increase in applying llms to simulate human behavior, yet their ability to accurately reflect specific demographic groups remains controversial. to study this problem, we construct a benchmark to rank humans and models by performance on the distributional alignment task. our findings reveal many unresolved challenges in distributional alignment, notably the model's sensitivity to output formats, misleading log-probabilities, and the inability to significantly outperform weak human baselines.