Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinions
Authors: Joseph Suh, Erfan Jahanparast, Suhong Moon, Minwoo Kang, Serina Chang
Accepted at: acl25

Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinions

Joseph Suh1*, Erfan Jahanparast1*, Suhong Moon1*, Minwoo Kang1*, Serina Chang1,2

1University of California, Berkeley 2Microsoft Research
Correspondence: {josephsuh,serinac}@berkeley.edu

Abstract

Large language models (LLMs) present novel opportunities in public opinion research by predicting survey responses in advance during the early stages of survey design. Prior methods steer LLMs via descriptions of subpopulations as LLMs’ input prompt, yet such prompt engineering approaches have struggled to faithfully predict the distribution of survey responses from human subjects. In this work, we propose directly fine-tuning LLMs to predict response distributions by leveraging unique structural characteristics of survey data. To enable fine-tuning, we curate SubPOP, a significantly scaled dataset of 3,362 questions and 70K subpopulation-response pairs from well-established public opinion surveys. We show that fine-tuning on SubPOP greatly improves the match between LLM predictions and human responses across various subpopulations, reducing the LLM-human gap by up to 46% compared to baselines, and achieves strong generalization to unseen surveys and subpopulations. Our findings highlight the potential of survey-based fine-tuning to improve opinion prediction for diverse, real-world subpopulations and therefore enable more efficient survey designs. Our code is available at Github/subpop and SubPOP data is available at HuggingFace/subpop.

1 Introduction

Surveys provide an essential tool for probing public opinions on societal issues, especially as opinions vary over time and across subpopulations. However, surveys are also costly, time-consuming, and require careful calibration to mitigate non-response and sampling biases (Choi and Pak, 2004; Bethlehem, 2010). Recent work suggests that large language models (LLMs) can assist public opinion studies by predicting survey responses across different subpopulations, explored in both social science

![Illustration of our method and SubPOP. We collect survey data from two survey families—ATP from Pew Research (Pew Research Center, 2018) (forming SubPOP-Train) and GSS from NORC (Davern et al., 2024) (forming SubPOP-Eval). LLMs are fine-tuned on SubPOP-Train and evaluated on both OpinionQA (Santurkar et al., 2023) and SubPOP-Eval to assess generalization of distributional opinion prediction across unseen survey topics, survey families, and subpopulations.](page_1012_355_393_393.png)

Figure 1: Illustration of our method and SubPOP. We collect survey data from two survey families—ATP from Pew Research (Pew Research Center, 2018) (forming SubPOP-Train) and GSS from NORC (Davern et al., 2024) (forming SubPOP-Eval). LLMs are fine-tuned on SubPOP-Train and evaluated on both OpinionQA (Santurkar et al., 2023) and SubPOP-Eval to assess generalization of distributional opinion prediction across unseen survey topics, survey families, and subpopulations.

(Argyle et al., 2023; Bail, 2024; Ashokkumar et al., 2024; Manning et al., 2024) and NLP (Santurkar et al., 2023; Chu et al., 2023; Moon et al., 2024; Hämäläinen et al., 2023; Chiang and Lee, 2023). Such capabilities could substantially enhance the survey development process, not as a replacement for human participants but as a tool for researchers to conduct pilot testing, identify subpopulations to over-sample, and test analysis pipelines prior to conducting the full survey (Rothschild et al., 2024).

Prior work in steering language models, i.e. conditioning models to reflect the opinions of a specific subpopulation, has primarily investigated different prompt engineering techniques (Santurkar et al., 2023; Moon et al., 2024; Park et al., 2024a).
However, prompting alone has shown limited success in generating completions that accurately reflect the distributions of survey responses collected from human subjects. Off-the-shelf LLMs (Achiam et al., 2023; Dubey et al., 2024; Jiang et al., 2023) have shown to mirror the opinions of certain US subpopulations such as the wealthy and educated (Santurkar et al., 2023; Gallegos et al., 2024; Deshpande et al., 2023; Kim and Lee, 2023), while generating stereotypical or biased predictions of underrepresented groups (Cheng et al., 2023b,a; Wang et al., 2024). Furthermore, these models often fail to capture the variation of human opinions within a subpopulation (Kapania et al., 2024; Park et al., 2024b). While fine-tuning presents opportunities to address these limitations (Chu et al., 2023; He et al., 2024), existing methods fail to train models that accurately predict opinion distributions across diverse survey question topics and subpopulations.

The present work. Here, we propose directly fine-tuning LLMs on large-scale, high-quality survey data, consisting of questions about diverse topics and responses from each subpopulation, defined by demographic, socioeconomic, and ideological traits. By casting pairs of (subpopulation, survey question) as input prompts, we train the LLM to align its response distribution against that of human subjects in a supervised manner. We posit that survey data is particularly well-suited for fine-tuning LLMs since: (1) We can train the model with clear subpopulation-response pairs that explicitly link group identities and expressed opinions, which is rare in LLMs’ pre-training corpora, (2) Large-scale opinion polls are carefully designed and calibrated (\emph{e.g.} using post-stratification) to estimate representative human responses, in contrast with LLMs’ pre-training data where certain populations are over- or underrepresented, (3) Our training objective explicitly aligns model predictions with response distributions from each subpopulation, enabling LLMs to capture variance within human subpopulations.

Training on public opinion survey data has remained under-explored due to the limited availability of structured survey datasets. To this end, we curate and release SubPOP (Subpopulation-level Public Opinion Prediction), a dataset of 70K subpopulation-response distribution pairs (6.5× larger compared to previous datasets). We show that fine-tuning LLMs on SubPOP significantly improves the distributional match between LLM generated and human responses, and improvements are consistent across subpopulations of varying sizes. Additionally, the improvement generalizes to unseen subpopulations, survey waves (topics), and survey families, \emph{i.e.} surveys administered by different institutions. Such broad generalization is particularly critical for real-world public opinions research, where practitioners are most in need of synthetic data for survey questions or subpopulations (or both) that they have not tested before.

Our contributions are summarized as follows:

• We show that training LLMs on response distributions from survey data significantly improves their ability to predict the opinions of subpopulations, reducing the Wasserstein distance between LLM and human distributions by 32-46% compared to top-performing baselines. (Section 4.2)

• We show that the performance of the fine-tuned LLMs strongly generalizes to out-of-distribution data, including unseen subpopulations, new survey waves, and different survey families. (Section 4.2 and Section 4.3)

• We release SubPOP, a curated and pre-processed dataset of public opinion survey results that is 6.5× larger than existing datasets, enabling fine-tuning at scale.

2 Related Work

Public opinion datasets. Several research institutions conduct large-scale public opinion polls and release data from those surveys. Important examples include Pew Research Center’s American Trends Panel (ATP), which consists of multiple waves of cross-sectional surveys on different topics (Pew Research Center, 2018), and General Social Survey (GSS) from the NORC at the University of Chicago (Davern et al., 2024). Existing datasets have curated such data for evaluating LLM-based opinion predictions, including OpinionQA (Santurkar et al., 2023), a subset of ATP survey waves containing about 500 questions on contentious social topics. While OpinionQA is widely used in prior work (He et al., 2024; Zhao et al., 2023; Li et al., 2023, 2024), we find its total number of questions limited in scale for fine-tuning LLMs and instead use this dataset for evaluation. We further collect an extended set of survey data from ATP waves not included in OpinionQA, as well as from GSS to curate SubPOP. Other datasets, such as GlobalOpinionQA (Durmus et al., 2023)—derived from the World Values Survey (WVS)
(World Values Survey, 2022) and the Pew Global Attitudes Survey (Pew Research Center, 2024)—and the PRISM dataset (Kirk et al., 2024) investigates how language models align with opinions from populations across the globe and different cultures.

Predicting human opinions with LLMs. Prior work has explored various prompt engineering approaches for steering LLM responses: earlier work use rule-based prompts that incorporate demographic profiles of individuals or populations, or few-shot examples of survey question-response (Hwang et al., 2023; Simmons, 2022; Santurkar et al., 2023; Dominguez-Olmedo et al., 2023). Recent work explore prompting LLMs with open-ended text, including interview transcripts (Park et al., 2024a), personal narratives (Moon et al., 2024), or LLM-refined prompts (Kim and Yang, 2024; Sun et al., 2024). Our fine-tuning approach is complementary to prompt engineering methods: while prompt engineering seeks to optimize what information is provided to the LLM (while the model is frozen), fine-tuning seeks to optimize how the model utilizes the provided information (while the prompt is frozen). In this work, we demonstrate that our fine-tuned models exhibit significant improvements in matching the response distributions of humans without requiring elaborate prompt engineering methods.

Other work (Chu et al., 2023; He et al., 2024; Feng et al., 2024) fine-tune language models on text corpora from specific communities (\emph{e.g.}, Reddit) to infer the most popular response or response distribution for a given survey question. While this approach benefits from large-scale and continuously updated text corpora, it struggles with disproportionate representation online and lacks comprehensive coverage of diverse subpopulations. A few works have explored directly fine-tuning on public opinion survey data, but in different problem settings from ours. Li et al. (2023) apply collaborative filtering to individual-level responses to learn embeddings for individuals, and Zhao et al. (2023) develop a meta-learning framework to predict the opinions of new groups given a small number of in-context examples for that group. In contrast, our approach does not require individual-level responses and can generalize to unseen groups and survey questions without \emph{any} responses.

A recent work (Li et al., 2024) and a work concurrent to ours (Cao et al., 2025) also explored fine-tuning LLMs on the World Values Survey (WVS) to align the LLM’s opinion response with a culture or entire country populations. In comparison, our work focuses on US surveys, testing whether LLMs can align with finer-grained subpopulations within one country and whether LLMs fine-tuned on one US-representative survey can generalize to another. However, we note that our proposed method for fine-tuning language models applies to any survey dataset with distributional information about subpopulation responses.

Pluralistic alignment of LLMs. Recent literature on pluralistic and distributional alignment target a similar yet different problem in fine-tuning LLMs (Chakraborty et al., 2024; Melnyk et al., 2024; Poddar et al., 2024; Siththanjan et al., 2023; Yao et al., 2024; Sorensen et al., 2024; Lake et al., 2024; Chen et al., 2024; Jiang et al., 2024). While this line of work shares a similar goal as ours in training models to reflect on opinions (and preferences) of diverse subpopulations, most work differ from ours in that they operate in the context of training against \emph{pair-wise} preference orderings between alternative language model completions, extending the Bradley-Terry-Luce model (Rajkumar and Agarwal, 2014; Ouyang et al., 2022; Rafailov et al., 2024) or investigating alternative models to account for diverging preference orderings across populations. In contrast, our work trains the model to directly predict the opinion distributions of human subpopulations, where accurately matching distributions across a large variety of subpopulations is of paramount interest. Our work additionally focuses on the particular context of estimating human opinions about societal issues—the objective of public opinion research—which enables relatively straightforward supervised training on openly available, structured survey data as presented by SubPOP.

3 Methods

3.1 Fine-tuning LLMs on Human Response Distributions

Our goal is to fine-tune an LLM to predict the distribution of responses for a multiple-choice question, conditioned on descriptions of a human subpopulation we want to simulate, typically a specific demographic, socioeconomic, or ideological subgroup. Consider the example in Figure 2: the question asks, “What do you think the chances are these days that a woman won’t get a job or promotion while an equally or less qualified man gets one instead?” The
available responses are: A. Very likely, B. Somewhat likely, C. Not very likely, D. Very unlikely, and E. Refused. In this case, the LLM will output a probability for each of the tokens corresponding to the choices A through E, thereby generating a complete response distribution that we aim to align with the true distribution observed in survey data.

Formally, let \( q \in Q \) be a question, \( g \in G \) be a subpopulation, and \( \mathcal{A}_q \) be the set of possible choices for question \( q \). An LLM with parameters \( \theta \) produces a conditional probability distribution \( p_\theta(\mathcal{A}_q | q, g) \). We fine-tune this model so that its predicted distribution for each \( (q, g) \) mirrors the human response distribution \( p_H(\mathcal{A}_q | q, g) \) collected from real survey data. To accomplish this, we apply LoRA fine-tuning (Hu et al., 2021) and use the forward Kullback–Leibler (KL) divergence as our loss. Concretely, if \( p_H(\mathcal{A}_q | q, g) \) represents the group-level empirical distribution of human opinions and \( p_\theta(\mathcal{A}_q | q, g) \) represents the model’s predicted distribution, our training objective is:

\[
\mathcal{L}(\theta) = \mathbb{E}_{q,g} \left[ D_{\mathrm{KL}}(p_H(\mathcal{A}_q | q, g) \| p_\theta(\mathcal{A}_q | q, g)) \right],
\]

where \( D_{\mathrm{KL}} \) denotes the KL divergence. In the example shown in Figure 2, the model is trained to reduce the KL divergence between the target (survey-based) distribution over \( \{A, B, C, D, E\} \) and its predicted distribution for the subpopulation living in the Southern United States.

We choose forward KL (i.e., \( \mathrm{KL}(p_H \| p_\theta) \)) since it is sensitive to cases where \( p_H \) assigns high probability but \( p_\theta \) does not, naturally encouraging the model to cover the real distribution. This property aligns with standard maximum-likelihood training, where the model is penalized for underestimating any response that is frequent in the data. In other words, if many participants in group \( g \) choose option “A” for question \( q \), then the model probability on “A” should be correspondingly high.

Instead of explicitly modeling the group response distribution as \( p_H(\mathcal{A}_q | q, g) \), one could do two alternatives. (1) One-hot encoding: this approach (Li et al., 2024) approximates the distribution by a one-hot vector, assigning a value of one to the most probable option and zero elsewhere. (2) Data augmentation by response frequency: this approach (Zhao et al., 2023) expands the dataset by replicating question-choice pairs in proportion to their observed frequency. We adopt the explicit distribution modeling in our main experiments because it directly encodes the distributional information without requiring discrete sampling or replicating data points. A detailed comparison of these approaches is provided in Section C.1.

![Proposed supervised fine-tuning setup with a survey response dataset such as SubPOP. Survey data is 3-tuple of a survey question, target subpopulation information, and the observed human opinion distribution (i.e. how subjects in the group responded to the given question). The training objective, \( \mathcal{L}(\theta) \), is a forward KL divergence loss on language model predicted distribution of question option likelihoods; our loss guides the model predictions to match the response distribution of the specified human subpopulation.](page_186_153_670_496.png)

Figure 2: Proposed supervised fine-tuning setup with a survey response dataset such as SubPOP. Survey data is 3-tuple of a survey question, target subpopulation information, and the observed human opinion distribution (i.e. how subjects in the group responded to the given question). The training objective, \( \mathcal{L}(\theta) \), is a forward KL divergence loss on language model predicted distribution of question option likelihoods; our loss guides the model predictions to match the response distribution of the specified human subpopulation.

3.2 SubPOP: a Comprehensive Survey Dataset to Fine-tune and Evaluate LLMs

OpinionQA (Santurkar et al., 2023) is a widely used dataset for fine-tuning and evaluating large language models (LLMs) on opinion prediction, containing roughly 500 questions drawn from 14 American Trends Panel (ATP) waves (Pew Research Center, 2018). Although valuable, it faces two important limitations: (1) Limited thematic diversity—for instance, wave 26 focuses on the topic of firearms. (2) Reliance on a single survey family (ATP), which risks overfitting to a particular style of questions and limits out-of-distribution evaluation on other sources (e.g., GSS).

To address these limitations, we introduce a new dataset, SubPOP, that broadens both the thematic and institutional scope of opinion prediction data. For training, SubPOP comprises 3,229 multiple-choice questions drawn from ATP waves 61–132,
excluding waves included in OpinionQA. In Table 4, we list the topics of the ATP waves in SubPOP vs. OpinionQA, both showing the increased thematic diversity of SubPOP (with over 20 new topics) and the remaining unseen topics in OpinionQA that allow us to test whether LLMs fine-tuned on SubPOP can generalize to unseen topics.

For evaluation, SubPOP also includes 133 multiple-choice questions from the General Social Survey (GSS) (Davern et al., 2024), serving as an out-of-distribution benchmark. This expanded collection not only broadens the range of topics beyond OpinionQA’s initial 500 questions, but also enables evaluation on surveys created and administered by different institutions (Pew Research Center vs. NORC-Chicago). Dataset curation and refinement pipeline is available in Appendix A.

3.3 Evaluation Metric

We use Wasserstein distance (WD) to quantify how closely the model’s predicted opinion distribution matches human survey data (Santurkar et al., 2023; Moon et al., 2024; Meister et al., 2024; Zhao et al., 2023). Formally, for a group \( g \) representing some subpopulation and a question \( q \) WD is defined as \( \mathcal{WD}_\theta(q,g) = \mathcal{WD}(p_H(A_q|q,g), p_\theta(A_q|q,g)) \) (see formula in Appendix B). Since WD is computed over ordinal values, we map the categorical answer options to numbers, such as mapping “Very likely” to 1, “Likely” to 2, and so on.

Some prior work utilizes one-hot accuracy (Feng et al., 2024; Li et al., 2023) as an evaluation metric. However, one-hot accuracy only verifies whether the top-predicted choice matches the top human response, thereby discarding distributional information. In contrast, WD accounts for partial overlaps among the categories and reflects the ‘cost’ of shifting probability mass, providing a more nuanced assessment of distribution discrepancy. Consider the example question provided in Figure 2, where the human response distribution indicates that option B (“Somewhat likely”) is the most probable. Now consider two cases in which the model incorrectly predicts the top choice. In the first case, the model assigns a high probability to option A (“Very likely”), while in the second case, it assigns a high probability to option D (“Very unlikely”). Although one-hot accuracy would treat both predictions equally as errors, WD differentiates between them by accounting for the ordinal relationship among the options, penalizing the second prediction more heavily for its larger deviation from the true distribution.

4 Experiments

4.1 Bounds of WD and Baselines

In this section, we describe the lower/upper bounds and two baseline methods against which we compare our method.

Lower and upper bounds. We use a uniform distribution over all available choices to establish an upper bound of the WD between a predicted and the target response distribution. To compute a lower bound, we sample a group of human respondents from the original human respondents to calculate the WD between the two, and perform bootstrapping to obtain a robust estimate. This lower bound captures the intrinsic variance arising from the respondent sampling process in opinion surveys.

Baselines. We compare our approach with two baseline methods: prompting and Modular Pluralism (Feng et al., 2024). For prompting, we consider both zero-shot and few-shot methods. In zero-shot prompting, we steer the LLM using demographic prompt formats. Specifically, we employ three different formats following Santurkar et al. (2023): QA, BIO, and PORTRAY. For instance, to condition the LLM to a person living in the South of the US, the QA format uses a question-answer format as illustrated in Figure 2; the BIO format conditions the model with a first-person narrative such as “I currently reside in the South.”; and the PORTRAY format uses a third-person narrative like “Answer the following question as if you currently reside in the South.”.

Few-shot prompting augments the prompt with a few examples of question-response distribution pairs alongside the demographic label (Hwang et al., 2023). In particular, we select the top five few-shot examples from the SubPOP training set based on cosine similarity computed by the embedding model. In our experiments, we represent the response distribution in JSON format and require the model to output its prediction in the same JSON format, following the approach in Meister et al. (2024).

Modular pluralism (Feng et al., 2024) fine-tunes multiple LLMs on distinct datasets to capture the viewpoints of different communities (Feng et al., 2023). For a given question, each fine-tuned LLM generates an opinion that reflects the perspective
Table 1: Evaluation on OpinionQA and the SubPOP evaluation set (SubPOP-Eval) for 22 subpopulations following (Santurkar et al., 2023). We compute the WD by averaging over all questions and subpopulations. Lower and upper bounds of performance give guidance on how each method performs. For Modular Pluralism, we provide an error rate of one-hot prediction (†) (Section 3.3) which was used in the original paper.

<table>
  <tr>
    <th rowspan="2">Method</th>
    <th colspan="4">OpinionQA</th>
    <th colspan="4">SubPOP-Eval</th>
  </tr>
  <tr>
    <th>Llama-2-7B</th>
    <th>Llama-2-13B</th>
    <th>Mistral-7B</th>
    <th>Llama-3-70B</th>
    <th>Llama-2-7B</th>
    <th>Llama-2-13B</th>
    <th>Mistral-7B</th>
    <th>Llama-3-70B</th>
  </tr>
  <tr>
    <td>Upper bound (Unif.)</td>
    <td></td>
    <td></td>
    <td></td>
    <td>0.178</td>
    <td></td>
    <td></td>
    <td></td>
    <td>0.208</td>
  </tr>
  <tr>
    <td>Lower bound (Human)</td>
    <td></td>
    <td></td>
    <td></td>
    <td>0.031</td>
    <td></td>
    <td></td>
    <td></td>
    <td>0.033</td>
  </tr>
  <tr>
    <td>Zero-shot prompt (QA)</td>
    <td>0.173</td>
    <td>0.170</td>
    <td>0.153</td>
    <td>0.138</td>
    <td>0.206</td>
    <td>0.196</td>
    <td>0.187</td>
    <td>0.160</td>
  </tr>
  <tr>
    <td>Zero-shot prompt (B10)</td>
    <td>0.193</td>
    <td>0.183</td>
    <td>0.162</td>
    <td>0.143</td>
    <td>0.221</td>
    <td>0.212</td>
    <td>0.202</td>
    <td>0.175</td>
  </tr>
  <tr>
    <td>Zero-shot prompt (PORTRAY)</td>
    <td>0.195</td>
    <td>0.207</td>
    <td>0.158</td>
    <td>0.209</td>
    <td>0.212</td>
    <td>0.242</td>
    <td>0.194</td>
    <td>0.247</td>
  </tr>
  <tr>
    <td>Few-shot prompt</td>
    <td>0.186</td>
    <td>0.175</td>
    <td>0.174</td>
    <td>0.166</td>
    <td>0.217</td>
    <td>0.194</td>
    <td>0.175</td>
    <td>0.182</td>
  </tr>
  <tr>
    <td>Modular Pluralism</td>
    <td></td>
    <td></td>
    <td>0.285 († 55.6%)</td>
    <td></td>
    <td></td>
    <td></td>
    <td>0.279 († 55.2%)</td>
    <td></td>
  </tr>
  <tr>
    <td>Ours (SubPOP-FT)</td>
    <td>0.106</td>
    <td>0.102</td>
    <td>0.096</td>
    <td>0.094</td>
    <td>0.121</td>
    <td>0.113</td>
    <td>0.115</td>
    <td>0.096</td>
  </tr>
</table>

of the community it represents, and a separate black-box LLM aggregates these outputs to produce the final distributional response. Detailed implementation of the lower/upper bounds and the baselines is provided in Appendix D.

4.2 Generalization to Unseen Topics and Survey Families

In this section, we assess the ability of our fine-tuned LLMs to generalize to unseen data—both in terms of new topics and entirely different survey families. To evaluate these aspects, we use OpinionQA to measure generalization to unseen topics, and SubPOP-Eval to test generalization to a different survey family. We fine-tune four LLMs (Llama-2-7B, Llama-2-13B, Mistral-7B, and Llama-3-70B) on SubPOP-Train. We opt for pretrained LLMs rather than instruction-following models, as previous work has shown that pretrained models perform better on this task (Moon et al., 2024). A detailed comparison between these model types is provided in Appendix C.2.

Summary of results. Table 1 reports the average WD metrics computed over all demographic groups and survey questions, comparing our fine-tuned models against various baseline approaches. Our experiments show that fine-tuning on SubPOP-Train significantly outperforms all other methods, yielding a 32–46% reduction in WD on OpinionQA and a 39–42% reduction on SubPOP-Eval compared to the best baselines. Notably, SubPOP-Train is based on ATP data, while SubPOP-Eval is derived from GSS surveys—two distinct survey families that can differ in respondent pools, calibration techniques, and other methodological factors, leading to non-trivial distribution shifts despite both being representative of the US population. Furthermore, our fine-grained analyses at the wave level (see Appendix E) confirm that these trends persist even at more detailed levels of evaluation.

Comparison to zero- and few-shot prompting. We first compare the performance of prompting methods with our approach. Zero-shot prompting results in only modest WD improvements over the upper bound, with the largest gain observed for Llama-3-70B and negligible improvements for Llama-2-7B. Even when using few-shot prompting—where five example question-response distribution pairs are provided—the performance gains remain minimal. This may be partly due to an under-optimized prompt format (e.g. requiring JSON output) and the inherent sensitivity of language models to prompt formatting (Sclar et al., 2023; Anagnostidis and Bulian, 2024). These findings underscore the need for methods, such as fine-tuning, that enable relatively reliable predictions of opinion distributions.

Comparison to Modular Pluralism. Modular Pluralism improves one-hot accuracy, reducing prediction error from 72.7% (zero-shot prompting) to 55.6% on OpinionQA, but underperforms in matching the full distribution of option choices, measured as WD. This discrepancy in performance highlights the limitations of methods that train LLMs to identify only the most probable response rather than modeling the entire distribution of responses. Opinions are inherently distributed: even within a particular subpopulation such as a single demographic subgroup, distribution of opinions cannot be captured as a single most likely response. Moreover, instruction-tuned models that serve as a black-box LLM tend to assign high probabilities on only specific tokens (Lin et al., 2022; Kadavath et al., 2022; Achiam et al., 2023), further pushing the generated distribution away from the human distribution.
Figure 3: Per-group evaluation performance of our model Llama-2-7B-SubPOP-FT (red lines) on OpinionQA. For comparison, the results from zero-shot QA prompting (black lines) and the lower bound (blue lines) are presented. We observe that the relative improvement, measuring how much of the gap between zero-shot prompting and the lower bound has been closed, remains consistent across subpopulations. Shaded blue regions represent the 95% confidence interval of the lower-bound estimation for each group. Per-group results for other models (Table 9) and the results on SubPOP evaluation set (Table 10) are available in Appendix E.

4.3 Generalization across Target Subpopulations

Here we report two key observations: (1) prediction performance improves consistently across most subpopulations represented in the fine-tuning data, and (2) the LLMs fine-tuned on SubPOP-Train generalize well to subpopulations that were not included during fine-tuning.

Consistent performance improvements over subpopulations. Figure 3 shows the per-group WD on the OpinionQA evaluation for Llama-2-7B, comparing our fine-tuning approach with zero-shot prompting and the empirical WD lower bound. To evaluate the consistency of performance gains, we calculate the *relative improvement* for each subpopulation as how much of the gap between zero-shot prompting and the empirical lower bound is reduced after fine-tuning. This measure allows us to account for varying lower bounds across subpopulations: since some groups have fewer respondents, there is greater uncertainty in their reported distribution in the survey data and greater variance between the original sample and bootstrap samples.

All 22 subpopulations demonstrate a large relative improvement after fine-tuning, ranging from 38%–54%. The average relative improvement is 46.7% with a standard deviation of 4.4%. This consistency confirms that our fine-tuning approach delivers balanced performance gains without disproportionately favoring any particular demographic subgroup. We hypothesize that the consistent gains over groups largely stem from our dataset design, which allocates an equal number of training samples to each group. By ensuring uniformly distributed data points across subpopulations, the model captures sufficient subgroup-specific signals, ultimately leading to consistent performance improvements.

Generalization to unseen subpopulations. We further investigate how models fine-tuned with our approach and SubPOP might show generalization to subpopulations that were not represented in the training data, a circumstance that may arise in real-world survey development. For the evaluation, we benchmark our methods against a zero-shot prompting baseline. Specifically, we evaluate our model, which is fine-tuned on 22 subpopulations provided in SubPOP-Train, on a set of subpopulations in OpinionQA that were not included in fine-tuning. This experiment not only checks generalization to unseen subpopulations, but also involves unseen survey questions, providing a robust assessment of the model capability for generalization to out-of-distribution data.

As shown in Table 2, our model achieves a strong reduction in WD even for unseen subpopulations, indicating that the model can be steered by demographic prompts beyond the seen subpopulations in training. Interestingly, although SubPOP-Train does not contain any data with opinion distributions of particular age groups (*e.g.* subjects of age 18-29 or those of age 65+), the average relative improvement is 44.7%, which is compatible with the average
Table 2: Per-group evaluation performance of Llama-2-7B-SubPOP-FT (Ours) on OpinionQA. We report the lower bound, WD for zero-shot prompting, WD for Llama-2-7B-SubPOP-FT, and the relative improvement. Rows highlighted in blue represent subpopulations included during fine-tuning, while uncolored rows correspond to subpopulations that were unseen during fine-tuning.

<table>
  <tr>
    <th rowspan="2">Group</th>
    <th colspan="2">Lower Bound</th>
    <th>Zero Shot</th>
    <th>Ours</th>
    <th rowspan="2">Relative Improvement (%)</th>
  </tr>
  <tr>
    <th>Bound</th>
    <th>Shot</th>
    <th></th>
    <th></th>
  </tr>
  <tr>
    <td>Age: 18-29</td>
    <td>0.023</td>
    <td>0.185</td>
    <td>0.096</td>
    <td>54.9</td>
  </tr>
  <tr>
    <td>Age: 30-49</td>
    <td>0.014</td>
    <td>0.151</td>
    <td>0.093</td>
    <td>42.3</td>
  </tr>
  <tr>
    <td>Age: 50-64</td>
    <td>0.014</td>
    <td>0.154</td>
    <td>0.101</td>
    <td>37.9</td>
  </tr>
  <tr>
    <td>Age: 65+</td>
    <td>0.013</td>
    <td>0.195</td>
    <td>0.115</td>
    <td>44.0</td>
  </tr>
  <tr>
    <td>Less than high school</td>
    <td>0.043</td>
    <td>0.161</td>
    <td>0.101</td>
    <td>50.8</td>
  </tr>
  <tr>
    <td>High school graduate</td>
    <td>0.017</td>
    <td>0.144</td>
    <td>0.092</td>
    <td>40.9</td>
  </tr>
  <tr>
    <td>Some college, no degree</td>
    <td>0.018</td>
    <td>0.144</td>
    <td>0.093</td>
    <td>40.5</td>
  </tr>
  <tr>
    <td>Associate’s degree</td>
    <td>0.026</td>
    <td>0.159</td>
    <td>0.098</td>
    <td>44.9</td>
  </tr>
  <tr>
    <td>College grad</td>
    <td>0.018</td>
    <td>0.165</td>
    <td>0.099</td>
    <td>44.9</td>
  </tr>
  <tr>
    <td>Postgraduate</td>
    <td>0.015</td>
    <td>0.174</td>
    <td>0.106</td>
    <td>52.8</td>
  </tr>
  <tr>
    <td>Very conservative</td>
    <td>0.026</td>
    <td>0.210</td>
    <td>0.107</td>
    <td>53.5</td>
  </tr>
  <tr>
    <td>Conservative</td>
    <td>0.016</td>
    <td>0.169</td>
    <td>0.103</td>
    <td>44.6</td>
  </tr>
  <tr>
    <td>Moderate</td>
    <td>0.016</td>
    <td>0.151</td>
    <td>0.094</td>
    <td>42.2</td>
  </tr>
  <tr>
    <td>Liberal</td>
    <td>0.022</td>
    <td>0.192</td>
    <td>0.100</td>
    <td>54.1</td>
  </tr>
  <tr>
    <td>Very liberal</td>
    <td>0.025</td>
    <td>0.202</td>
    <td>0.111</td>
    <td>51.4</td>
  </tr>
  <tr>
    <td>Democrat</td>
    <td>0.016</td>
    <td>0.172</td>
    <td>0.099</td>
    <td>47.1</td>
  </tr>
  <tr>
    <td>Republican</td>
    <td>0.019</td>
    <td>0.196</td>
    <td>0.105</td>
    <td>52.0</td>
  </tr>
  <tr>
    <td>Independent</td>
    <td>0.016</td>
    <td>0.155</td>
    <td>0.093</td>
    <td>44.5</td>
  </tr>
  <tr>
    <td>Something Else</td>
    <td>0.026</td>
    <td>0.162</td>
    <td>0.092</td>
    <td>51.0</td>
  </tr>
</table>

relative improvement for seen subpopulations. We provide results for other unseen groups in Table 7 of Appendix C.3 (average relative improvement of 43.1% with a standard deviation of 6.7%).

Steerability towards subpopulations. Given the large improvements in WD across subpopulations after fine-tuning, we want to test whether the LLM is truly adapting its predictions based on the subpopulation specified in its prompt (i.e. the LLM is being steered) or if the improvements can be explained by the LLMs’ predictions getting closer to human responses in general, without any subpopulation-specific adaptation. If the LLM is being steered, we should expect that the LLM’s predictions for a target subpopulation \( g_t \) are closer to the human distribution for \( g_t \) when \( g_t \) is the subpopulation specified in the prompt, compared to when another group \( g_s \) is specified in the prompt. We should also expect the gap in WD to be larger if the distance between the true human distributions for \( g_t \) and \( g_s \) are larger, such as differences between the youngest and oldest age groups compared to adjacent groups.

Formally, we define the intergroup disagreement between a target group \( g_t \) and a source group \( g_s \) as \( \mathcal{WD}(p_H(\mathcal{A}_q | q,g_t), p_H(\mathcal{A}_q | q,g_s)) \) averaged over evaluation questions. In human responses (left of Figure 4), the disagreement shows the pattern of locality: increases as the disparity in education levels between two groups grows. We extend this notion to compare the human distribution from the target group \( g_t \) with the LLM-predicted distribution when the source group \( g_s \) is specified in the prompt, \( \mathcal{WD}(p_H(\mathcal{A}_q | q,g_t), p_H(\mathcal{A}_q | q,g_s)) \). If the model truly incorporates subpopulation information from the prompt, its intergroup disagreement pattern should mirror that of the human data.

Zero-shot prompting with the base model (right of Figure 4) does not exhibit the locality pattern seen in the human data, indicating that it cannot be steered by subpopulation labels. In contrast, the fine-tuned model (middle of Figure 4) reproduces a pattern resembling the human-human case, even though it was trained on only two education groups (“less than high school” and “college graduate/some postgrad”) and the other four groups were unseen. This result demonstrates that our fine-tuned model not only learns to condition on subpopulation information but also generalizes to subpopulations unseen during fine-tuning. We provide the inter-group disagreement for other traits in Appendix C.3.

4.4 Effect of Scaling the Dataset

In this section, we examine performance scales with training dataset size. We randomly sample subsets containing 25%, 50%, 75%, and 87.5% of the full SubPOP training set and evaluate three models—Llama-2-7B, Llama-2-13B, and Mistral-7B—on OpinionQA. As shown in Figure 5, we observe diminishing marginal returns, as is typical with fine-tuning; for example, after training on a random 25%, the models reach 72%-78% of the total improvement they achieve after fine-tuning on all of SubPOP-train. However, the performance does not entirely plateau. Instead, it continues to improve as we further increase the training data from 25% to 100%. We fit linear trend lines (dotted in Figure 5) to the results and observe that the slopes are similar for each model. This suggests that the rate of improvement—reflected by the slope in the power-law relationship—is intrinsic to the data and task rather than to the specific model architecture. In other words, LLMs exhibit comparable data efficiency, with performance gains that are fundamentally tied to dataset size rather than model-specific factors.

Using these trend lines, we can estimate the amount of fine-tuning data required to reach a target performance. For instance, we estimate that fine-tuning Mistral-7B on a dataset 25 times larger than the current SubPOP training set would yield a WD value of 0.07, which is much closer to the empirical lower bound of 0.031 reported in Table 1.
Figure 4: Intergroup disagreement pattern between groups of different education levels calculated with OpinionQA and Llama-2-7B as a base model. A target human group is compared to (left) a source human group, (middle) our fine-tuned model conditioned on a source group, (right) a base model conditioned on a source group. Bold-faced groups are included in the fine-tuning data SubPOP-Train, while the others aren’t. In the human response (left), we observe a decreasing disagreement level as the education level becomes similar. This disagreement pattern exists in our fine-tuned model but not in the zero-shot prompting with a base model, indicating that our model can be steered to given subpopulation label even for unseen demographics while the base model cannot.

![Intergroup disagreement pattern between groups of different education levels](page_186_120_1077_357.png)

Figure 5: Evaluation results on OpinionQA after fine-tuning each LLM on increasingly large sampled subsets of SubPOP-Train. Both axes are presented in a log scale. The x-axis is the size of sampled dataset and the y-axis is WD against human responses measured on OpinionQA. Dashed lines represent a line of best fit. Performances at data percentage of 100% are identical to ours (SubPOP-FT) in Table 1.

![Evaluation results on OpinionQA after fine-tuning](page_186_563_1077_357.png)

This result underscores the critical importance of collecting more high-quality data, as increased dataset size can drive significant improvements in model performance.

5 Conclusion
In this work, we demonstrated that fine-tuning large language models on structured public opinion survey data markedly improves their ability to predict human response distributions. We curate SubPOP—a dataset 6.5× larger than previous collections to fine-tune and evaluate LLMs on survey response distribution prediction. By fine-tuning on SubPOP, we showed that LLMs can capture the group-specific variability in public opinions, also generalizing to unseen subpopulations, survey waves and question topics, and different survey families. Fine-tuning achieves consistent improvements across subpopulations of varying sizes, and our experiments demonstrate that fine-tuned LLMs are indeed adapting their responses to the subpopulation specified in the prompt, even for subpopulations unseen during fine-tuning. Finally, our experiments also reveal that as the fine-tuning dataset grows, model performance continues to scale favorably, underscoring the value of our larger dataset.

Generalization is a critical capability for LLMs if they are to be used to assist public opinion research, as researchers are most in need of opinion predictions for questions or subpopulations whom they have not surveyed before. Our work, by greatly improving LLMs’ ability to accurately predict opinions with fine-tuning and demonstrating strong generalization to out-of-distribution data, moves us closer towards the goal of leveraging LLMs for opinion prediction. However, many open questions remain: why is the model able to generalize well to unseen subpopulations and questions, and when might it fail to do so? How do we ensure that LLMs capture opinions along other dimensions not explored in this work, such as intersections of demographic identities or temporal change? How should LLMs be integrated into survey designs, to serve as tools that can complement surveys with human participants? Answering these questions will require interdisciplinary collaborations with domain experts and critical assessments of LLMs’ and traditional survey methods’ strengths and weaknesses.
6 Limitations

In this work, we explore the capability of language models to complement traditional survey design by predicting survey responses in advance. However, we acknowledge the following inherent limitations of this approach.

Role in Survey Research. While language models can provide a coarse approximation of human opinions, they cannot fully replace human involvement in the survey process. Human opinions evolve dynamically in response to social events, and while pretrained language models can incorporate such knowledge through retrieval-augmented generation, they remain limited in adapting to a rapidly changing world. Moreover, fine-tuning a language model on distributions of human opinions may inadvertently replicate and amplify existing biases of humans, leading to undesirable outcomes. It is important to note that a model fine-tuned on human opinions does not necessarily align with human values and behaviors, nor does it serve as a perfect proxy for human decision-making. The scope of our work is restricted to language models prompted with a group-level information generating response distributions to survey questions, rather than simulating individual human respondents in a personalized manner.

Data Dependence. Survey response data, even after post-stratification calibration, remain subject to empirical variance, particularly for relatively small groups that comprise about one percent of the U.S. population. Also, while traditional surveys have implemented various strategies to mitigate response bias stemming from the linguistic and multiple-choice nature of survey questions (Tourangeau, 2000), the extent to which these biases affect language models—and how best to address them—remains an open question (Tjuatja et al., 2024; Bisbee et al., 2024). Future research could focus on developing reliable opinion datasets for underrepresented groups and examining how prompt engineering elements can be optimized to reduce bias in language model-generated responses.

Limited Contextual Information. Our fine-tuning approach, which structures prompts in a QA format, demonstrates strong matching with human opinion distributions. However, we have not explored fine-tuning with richer contextual information. Prior research suggests that incorporating additional contextual details can improve the fidelity of model-generated opinions to actual human responses. We anticipate that more sophisticated steering techniques could further enhance the opinion prediction performance beyond the results presented in this study. Investigating such methods remains an open and promising direction for future work.

7 Potential Risks

Employing language models for opinion prediction has both influential possibilities and risk of misuse. We acknowledge that the risk of misuse cannot be overlooked, and we clearly state that indiscriminately minimizing the discrepancy of opinion response distribution as a fine-tuning target can cause severe harms. In particular, the model might develop a bias toward specific demographics during the course of fine-tuning, an artifact of minimizing response distribution when other safeguard measures are not employed. We emphasize that an oversight and holistic evaluation of methods and pipelines are required before deploying such models for any of the actual applications and interactions with human.

Acknowledgments

We sincerely appreciate Prof. John Canny for constructive comments and guidance. We also thank Prof. Emma Pierson and Dr. Sehoon Kim for valuable discussions as well as Woosuk Kwon for his support with vLLM. J.S. and S.M. would like to acknowledge the support from the Korea Foundation for Advanced Studies (KFAS). S.M. is supported by BAIR-Google Commons and M.K. is supported by the Apple Ph.D. Fellowship in Integrated Systems. We gratefully acknowledge the generous support from Strong Compute for providing fully managed GPU for fine tuning through the Strong Compute Instant Super Computer system, including support from Adam Peaston.

References

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.

Sotiris Anagnostidis and Jannis Bulian. 2024. How susceptible are llms to influence in prompts? arXiv preprint arXiv:2408.11865.
Lisa P Argyle, Ethan C Busby, Nancy Fulda, Joshua R Gubler, Christopher Ryting, and David Wingate. 2023. Out of one, many: Using language models to simulate human samples. Political Analysis, 31(3):337–351.

Ashwini Ashokkumar, Luke Hewitt, Isaías Ghezae, and Robb Willer. 2024. Predicting results of social science experiments using large language models. accessed September, 19:2024.

Christopher A Bail. 2024. Can generative ai improve social science? Proceedings of the National Academy of Sciences, 121(21):e2314021121.

Jelke Bethlehem. 2010. Selection bias in web surveys. International statistical review, 78(2):161–188.

James Bisbee, Joshua D Clinton, Cassy Dorff, Brenton Kenkel, and Jennifer M Larson. 2024. Synthetic replacements for human survey data? the perils of large language models. Political Analysis, 32(4):401–416.

Yong Cao, Haijiang Liu, Arnav Arora, Isabelle Augenstein, Paul Röttger, and Daniel Hershcovitch. 2025. Specializing large language models to simulate survey response distributions for global populations. arXiv preprint arXiv:2502.07068.

Souradip Chakraborty, Jiahao Qiu, Hui Yuan, Alec Koppel, Dinesh Manocha, Furong Huang, Amrit Bedi, and Mengdi Wang. 2024. Maxmin-rhlf: Alignment with diverse human preferences. In Forty-first International Conference on Machine Learning.

Daiwei Chen, Yi Chen, Aniket Rege, and Ramya Kolalakai Vinayak. 2024. Pal: Pluralistic alignment framework for learning from heterogeneous preferences. arXiv preprint arXiv:2406.08469.

Myra Cheng, Esin Durmus, and Dan Jurafsky. 2023a. Marked personas: Using natural language prompts to measure stereotypes in language models. arXiv preprint arXiv:2305.18189.

Myra Cheng, Tiziano Piccardi, and Diyi Yang. 2023b. Compost: Characterizing and evaluating caricature in llm simulations. arXiv preprint arXiv:2310.11501.

Cheng-Han Chiang and Hung-yi Lee. 2023. Can large language models be an alternative to human evaluations? arXiv preprint arXiv:2305.01937.

Bernard CK Choi and Anita WP Pak. 2004. A catalog of biases in questionnaires. Preventing chronic disease, 2(1):A13.

Eric Chu, Jacob Andreas, Stephen Ansolabehere, and Deb Roy. 2023. Language models trained on media diets can predict public opinion. arXiv preprint arXiv:2303.16779.

Michael Davern, Rene Bautista, Jeremy Freese, Pamela Herd, and Stephen L. Morgan. 2024. General social survey 1972-2024. Principal Investigator: Michael Davern; Co-Principal Investigators: Rene Bautista, Jeremy Freese, Pamela Herd, and Stephen L. Morgan. Sponsored by National Science Foundation. NORC ed. Chicago: NORC, 2024.

Ameet Deshpande, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, and Karthik Narasimhan. 2023. Toxicity in chatgpt: Analyzing persona-assigned language models. arXiv preprint arXiv:2304.05335.

Ricardo Dominguez-Olmedo, Moritz Hardt, and Celestine Mendler-Dünner. 2023. Questioning the survey responses of large language models. arXiv preprint arXiv:2306.07951.

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.

Esin Durmus, Karina Nguyen, Thomas I Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, et al. 2023. Towards measuring the representation of subjective global opinions in language models. arXiv preprint arXiv:2306.16388.

Shangbin Feng, Chan Young Park, Yuhuan Liu, and Yulia Tsvetkov. 2023. From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair nlp models. arXiv preprint arXiv:2305.08283.

Shangbin Feng, Taylor Sorensen, Yuhuan Liu, Jillian Fisher, Chan Young Park, Yejin Choi, and Yulia Tsvetkov. 2024. Modular pluralism: Pluralistic alignment via multi-LLM collaboration. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 4151–4171, Miami, Florida, USA. Association for Computational Linguistics.

Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sunghul Kim, Franck Demontcourt, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. 2024. Bias and fairness in large language models: A survey. Computational Linguistics, pages 1–79.

Perttu Hämäläinen, Mikke Tavast, and Anton Kunnari. 2023. Evaluating large language models in generating synthetic hci research data: a case study. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–19.

Zihao He, Minh Duc Chu, Rebecca Dorn, Siyi Guo, and Kristina Lerman. 2024. Community-cross-instruct: Unsupervised instruction generation for aligning large language models to online communities. arXiv preprint arXiv:2406.12074.

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685.
EunJeong Hwang, Bodhisattwa Prasad Majumder, and Niket Tandon. 2023. Aligning language models to user opinions. arXiv preprint arXiv:2305.14929.

Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825.

Liwei Jiang, Taylor Sorensen, Sydney Levine, and Yejin Choi. 2024. Can language models reason about individualistic human values and preferences? arXiv preprint arXiv:2410.03868.

Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al. 2022. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221.

Shivani Kapania, William Agnew, Motahhare Eslami, Hoda Heidari, and Sarah Fox. 2024. 'simulacrum of stories': Examining large language models as qualitative research participants. arXiv preprint arXiv:2409.19430.

Jaehyung Kim and Yiming Yang. 2024. Few-shot personalization of llms with mis-aligned responses. arXiv preprint arXiv:2406.18678.

Junsol Kim and Byungkyu Lee. 2023. Ai-augmented surveys: Leveraging large language models and surveys for opinion prediction. arXiv preprint arXiv:2305.09620.

Hannah Rose Kirk, Alexander Whitefield, Paul Röttger, Andrew Bean, Katerina Margatina, Juan Ciro, Rafael Mosquera, Max Bartolo, Adina Williams, He He, et al. 2024. The prism alignment project: What participatory, representative and individualised human feedback reveals about the subjective and multicultural alignment of large language models. arXiv preprint arXiv:2404.16019.

Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.

Thom Lake, Eunsol Choi, and Greg Durrett. 2024. From distributional to overton pluralism: Investigating large language model alignment. arXiv preprint arXiv:2406.17692.

Cheng Li, Mengzhou Chen, Jindong Wang, Sunayana Sitaram, and Xing Xie. 2024. Culturellm: Incorporating cultural differences into large language models. arXiv preprint arXiv:2402.10946.

Junyi Li, Ninareh Mehrabi, Charith Peris, Palash Goyal, Kai-Wei Chang, Aram Galstyan, Richard Zemel, and Rahul Gupta. 2023. On the steerability of large language models toward data-driven personas. arXiv preprint arXiv:2311.04978.

Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Teaching models to express their uncertainty in words. arXiv preprint arXiv:2205.14334.

I Loshchilov. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101.

Benjamin S Manning, Kehang Zhu, and John J Horton. 2024. Automated social science: Language models as scientist and subjects. Technical report, National Bureau of Economic Research.

Nicole Meister, Carlos Guestrin, and Tatsunori Hashimoto. 2024. Benchmarking distributional alignment of large language models. arXiv preprint arXiv:2411.05403.

Igor Melnyk, Youssef Mroueh, Brian Belgodere, Mattia Rigotti, Apoorva Nitsure, Mikhail Yurochkin, Kristjan Greenewald, Jiri Navratil, and Jerret Ross. 2024. Distributional preference alignment of llms via optimal transport. arXiv preprint arXiv:2406.05882.

Andrew Mercer, Arnold Lau, and Courtney Kennedy. 2018. For weighting online opt-in samples, what matters most?

Suhong Moon, Marwa Abdulhai, Minwoo Kang, Joseph Suh, Widyadwi Soedarmadji, Eran Kohen Behar, and David Chan. 2024. Virtual personas for language models via an anthology of backstories. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 19864–19897, Miami, Florida, USA. Association for Computational Linguistics.

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744.

Joon Sung Park, Carolyn Q Zou, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Robb Willer, Percy Liang, and Michael S Bernstein. 2024a. Generative agent simulations of 1,000 people. arXiv preprint arXiv:2411.10109.

Peter S Park, Philipp Schoenegger, and Chongyang Zhu. 2024b. Diminished diversity-of-thought in a standard large language model. Behavior Research Methods, pages 1–17.

Pew Research Center. 2018. America trends panel waves. Retrieved February 06, 2025, from https://www.pewsocialtrends.org/dataset.

Pew Research Center. 2024. Pew research center. Accessed: February 10, 2025.
Sriyash Poddar, Yanming Wan, Hamish Ivison, Abhishek Gupta, and Natasha Jaques. 2024. Personalizing reinforcement learning from human feedback with variational preference learning. arXiv preprint arXiv:2408.10075.

Rafael Raffailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36.

Arun Rajkumar and Shivani Agarwal. 2014. A statistical convergence perspective of algorithms for rank aggregation from pairwise data. In International conference on machine learning, pages 118–126. PMLR.

David M. Rothschild, James Brand, Hope Schroeder, and Jenny Wang. 2024. Opportunities and risks of llms in survey research. Available on SSRN: http://dx.doi.org/10.2139/ssrn.5001645.

Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoq Lee, Percy Liang, and Tatsunori Hashimoto. 2023. Whose opinions do language models reflect? In International Conference on Machine Learning, pages 29971–30004. PMLR.

Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2023. Quantifying language models’ sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting. arXiv preprint arXiv:2310.11324.

Gabriel Simmons. 2022. Moral mimicry: Large language models produce moral rationalizations tailored to political identity. arXiv preprint arXiv:2209.12106.

Anand Siththanjan, Cassidy Laidlaw, and Dylan Hadfield-Menell. 2023. Distributional preference learning: Understanding and accounting for hidden context in rlhf. arXiv preprint arXiv:2312.08358.

Taylor Sorensen, Jared Moore, Jillian Fisher, Mitchell Gordon, Niloofar Mireshghallah, Christopher Michael Ryttling, Andre Ye, Leiwei Jiang, Ximing Lu, Nouha Dziri, et al. 2024. A roadmap to pluralistic alignment. arXiv preprint arXiv:2402.05070.

Chenkai Sun, Ke Yang, Revanth Gangi Reddy, Yi R Fung, Hou Pong Chan, Kevin Small, ChengXiang Zhai, and Heng Ji. 2024. Persona-db: Efficient large language model personalization for response prediction with collaborative data refinement. arXiv preprint arXiv:2402.11060.

Lindia Tjuatja, Valerie Chen, Tongshuang Wu, Ameet Talwalkar, and Graham Neubig. 2024. Do llms exhibit human-like response biases? a case study in survey design. Transactions of the Association for Computational Linguistics, 12:1011–1026.

Roger Tourangeau. 2000. The psychology of survey response. University of Cambridge.

Angelina Wang, Jamie Morgenstern, and John P Dickerson. 2024. Large language models cannot replace human participants because they cannot portray identity groups. arXiv preprint arXiv:2402.01908.

World Values Survey. 2022. World Values Survey. [Online; accessed 02/15/2025].

Binwei Yao, Zefan Cai, Yun-Shiuan Chuang, Shanglin Yang, Ming Jiang, Diyi Yang, and Junjie Hu. 2024. No preference left behind: Group distributional preference optimization. arXiv preprint arXiv:2412.20299.

Siyan Zhao, John Dang, and Aditya Grover. 2023. Group preference optimization: Few-shot alignment of large language models. arXiv preprint arXiv:2310.11523.

A Dataset Details

A.1 American Trends Panel Datasets

Pew Research holds regular American Trends Panel (ATP) survey (called waves) (Pew Research Center, 2018) covering various topics (e.g. veterans, political priorities, gender and leadership) and releases result at an individual level. For each anonymized individual, the following information is released: unique identification number, demographic details, survey responses, and weight. Weights (Mercer et al., 2018) are the output of post-survey calibration process that helps adjusting survey results for response bias (e.g., non-response bias, sampling bias) correction and population representativeness. As of January 2025, survey data until wave 132 has been released. About 20 surveys are conducted in each year.

A.2 OpinionQA

OpinionQA is a subset of ATP curated in (Santurkar et al., 2023). This dataset consists of contentious 500 questions sampled from 14 ATP waves which have high intergroup disagreement (i.e. large Wasserstein distances among subpopulations’ responses to a question). It also comes with hand-crafted ordinality information which provides structure to option lists. For example, options ‘Major reason’, ‘Minor reason’, and ‘Not a reason’, are assigned an ordinality mapping to 1, 2, and 3, respectively. This ordinality allows a calculation of 1-dimensional Wasserstein distance.

Subpopulations we employ are listed in Table 3. This set of groups is adopted for several small-scale analysis (Santurkar et al., 2023; Zhao et al., 2023; Kim and Yang, 2024). We note that our approach is not limited to a specific number of groups and data
is available for small or fine-grained demographic subpopulations.

Table 3: A list of 22 subpopulations used throughout our fine-tuning and analysis. We provide the number of respondents in each subpopulation in American Trends Panel Wave 82 for reference.

<table>
  <tr>
    <th>Trait</th>
    <th>Groups</th>
    <th>Population % in Wave 82</th>
  </tr>
  <tr>
    <td rowspan="2">Region</td>
    <td>Northeast</td>
    <td>17.2</td>
  </tr>
  <tr>
    <td>South</td>
    <td>37.8</td>
  </tr>
  <tr>
    <td rowspan="2">Education</td>
    <td>College grad+</td>
    <td>24.2</td>
  </tr>
  <tr>
    <td>Less than high school</td>
    <td>5.2</td>
  </tr>
  <tr>
    <td rowspan="2">Gender</td>
    <td>Male</td>
    <td>44.3</td>
  </tr>
  <tr>
    <td>Female</td>
    <td>54.6</td>
  </tr>
  <tr>
    <td rowspan="4">Race / ethnicity</td>
    <td>Black</td>
    <td>9.6</td>
  </tr>
  <tr>
    <td>White</td>
    <td>66.1</td>
  </tr>
  <tr>
    <td>Asian</td>
    <td>4.8</td>
  </tr>
  <tr>
    <td>Hispanic</td>
    <td>15.2</td>
  </tr>
  <tr>
    <td rowspan="2">Income</td>
    <td>$100,000 or more</td>
    <td>21.8</td>
  </tr>
  <tr>
    <td>Less than $30,000</td>
    <td>21.3</td>
  </tr>
  <tr>
    <td rowspan="2">Political Party</td>
    <td>Democrat</td>
    <td>35.1</td>
  </tr>
  <tr>
    <td>Republican</td>
    <td>29.1</td>
  </tr>
  <tr>
    <td rowspan="3">Political Ideology</td>
    <td>Liberal</td>
    <td>20.0</td>
  </tr>
  <tr>
    <td>Conservative</td>
    <td>22.6</td>
  </tr>
  <tr>
    <td>Moderate</td>
    <td>38.3</td>
  </tr>
  <tr>
    <td rowspan="5">Religion</td>
    <td>Protestant</td>
    <td>40.8</td>
  </tr>
  <tr>
    <td>Jewish</td>
    <td>2.0</td>
  </tr>
  <tr>
    <td>Hindu</td>
    <td>0.9</td>
  </tr>
  <tr>
    <td>Atheist</td>
    <td>0.6</td>
  </tr>
  <tr>
    <td>Muslim</td>
    <td>0.7</td>
  </tr>
</table>

Table 4: American Trends Panel (ATP) wave topics for waves included in SubPOP-Train (top) and OpinionQA (bottom). Golden rows represent wave topics in SubPOP-Train that are not present in OpinionQA, and blue rows represent wave topics in OpinionQA that are not present in SubPOP-Train. For waves 68-79, survey questions related to COVID-19 (e.g., contact tracing, vaccines, and relocation) were included as part of a survey along with the main survey topic.

<table>
  <tr>
    <th>Wave</th>
    <th># questions</th>
    <th>Wave Topic</th>
  </tr>
  <tr><td>68</td><td>90</td><td>American News Pathways, George Floyd, Black Lives Matter</td></tr>
  <tr><td>69</td><td>92</td><td>Politics, 2020 Census</td></tr>
  <tr><td style="background-color:#f2cc8f;">70</td><td>56</td><td>Religion in public life, social media’s role in politics and society</td></tr>
  <tr><td>71</td><td>84</td><td>Voter attitudes</td></tr>
  <tr><td>72</td><td>18</td><td>New media</td></tr>
  <tr><td>73</td><td>82</td><td>American News Pathways, social media</td></tr>
  <tr><td>74</td><td>51</td><td>Online harassment, race relations</td></tr>
  <tr><td>75</td><td>18</td><td>2020 pre-election survey</td></tr>
  <tr><td>76</td><td>44</td><td>American News Pathways</td></tr>
  <tr><td>77</td><td>18</td><td>Culture and society</td></tr>
  <tr><td>78</td><td>57</td><td>2020 post-election survey</td></tr>
  <tr><td>79</td><td>93</td><td>American News Pathways</td></tr>
  <tr><td>80</td><td>45</td><td>Political priorities</td></tr>
  <tr><td>81</td><td>52</td><td>Economics, pandemic financial outlook</td></tr>
  <tr><td>83</td><td>54</td><td>Coronavirus vaccines and restrictions</td></tr>
  <tr><td>84</td><td>50</td><td>Religion in politics and tolerance</td></tr>
  <tr><td>85</td><td>93</td><td>News coverage of the Biden administration’s first 100 days</td></tr>
  <tr><td>87</td><td>90</td><td>Current political news and topics</td></tr>
  <tr><td>88</td><td>37</td><td>Tech companies and policy issues</td></tr>
  <tr><td>90</td><td>79</td><td>Twitter news attitudes</td></tr>
  <tr><td>91</td><td>64</td><td>Benchmark study</td></tr>
  <tr><td>93</td><td>19</td><td>Social media update</td></tr>
  <tr><td>95</td><td>78</td><td>Politics timely and topical</td></tr>
  <tr><td>96</td><td>57</td><td>Post-coronavirus pandemic spirituality</td></tr>
  <tr><td>98</td><td>76</td><td>Coronavirus impacts on communities, living arrangements and life decisions</td></tr>
  <tr><td>99</td><td>20</td><td>Artificial intelligence (AI) and human enhancement</td></tr>
  <tr><td>103</td><td>12</td><td>Economic well-being</td></tr>
  <tr><td>104</td><td>92</td><td>Politics, Religion in Public Life</td></tr>
  <tr><td>105</td><td>36</td><td>Global Attitudes US Survey 2022</td></tr>
  <tr><td>106</td><td>62</td><td>Religion and the environment</td></tr>
  <tr><td>107</td><td>92</td><td>Government and Parties</td></tr>
  <tr><td>108</td><td>83</td><td>COVID and Climate, Energy and the Environment</td></tr>
  <tr><td>109</td><td>51</td><td>New Digital Platforms and Gender Identity</td></tr>
  <tr><td>110</td><td>90</td><td>Politics timely and topical</td></tr>
  <tr><td>111</td><td>23</td><td>Online dating and E-commerce</td></tr>
  <tr><td>112</td><td>31</td><td>Social media update</td></tr>
  <tr><td>113</td><td>53</td><td>2022 National Survey of Latinos (NSL)</td></tr>
  <tr><td>114</td><td>93</td><td>Covid, scientists, and religion</td></tr>
  <tr><td>115</td><td>63</td><td>Parents survey</td></tr>
  <tr><td>116</td><td>75</td><td>Politics timely and topical</td></tr>
  <tr><td>117</td><td>16</td><td>Religion and politics</td></tr>
  <tr><td>118</td><td>25</td><td>Podcasts, news, and racial identity</td></tr>
  <tr><td>119</td><td>70</td><td>AI and human enhancement</td></tr>
  <tr><td>120</td><td>61</td><td>Politics timely and topical</td></tr>
  <tr><td>121</td><td>31</td><td>Culture and society</td></tr>
  <tr><td>124</td><td>75</td><td>Global Attitudes US Survey 2023</td></tr>
  <tr><td>125</td><td>69</td><td>Politics timely and topical</td></tr>
  <tr><td>126</td><td>93</td><td>Racial attitudes, modern family</td></tr>
  <tr><td>127</td><td>59</td><td>Americans and their data</td></tr>
  <tr><td>128</td><td>89</td><td>Americans and planet Earth</td></tr>
  <tr><td>129</td><td>107</td><td>Politics timely and topical</td></tr>
  <tr><td>130</td><td>94</td><td>Politics representation</td></tr>
  <tr><td>131</td><td>70</td><td>Gender and leadership</td></tr>
  <tr><td>26</td><td>44</td><td>Guns</td></tr>
  <tr><td>29</td><td>20</td><td>Views on gender</td></tr>
  <tr><td>32</td><td>24</td><td>Community types, Sexual harassment</td></tr>
  <tr><td>34</td><td>16</td><td>Biomedical and food issues</td></tr>
  <tr><td>36</td><td>68</td><td>Gender and leadership</td></tr>
  <tr><td>41</td><td>41</td><td>Views of America in 2050</td></tr>
  <tr><td>42</td><td>26</td><td>Trust in science</td></tr>
  <tr><td>43</td><td>5</td><td>Race in America</td></tr>
  <tr><td>45</td><td>13</td><td>Misinformation</td></tr>
  <tr><td>49</td><td>19</td><td>Privacy and surveillance</td></tr>
  <tr><td>50</td><td>43</td><td>American families</td></tr>
  <tr><td>54</td><td>50</td><td>Economic inequality</td></tr>
  <tr><td>82</td><td>56</td><td>2021 Global Attitudes Project U.S. survey</td></tr>
  <tr><td>92</td><td>23</td><td>Political Typology</td></tr>
</table>
A.3 SubPOP-Train

We gather additional data from the American Trends Panel, specifically collecting 53 waves from Wave 61 to 132. There are 62 waves from Wave 61 - 132, however, some waves have missing demographic or ideology information (for example, wave 63 does not contain political ideology information) or the data is not available hence removed during the curation process. To refine the dataset, we exclude questions that meet the following criteria: those with more than 10 response options, redacted response data, or dependencies on prior questions (e.g., assessing political strength). For the remaining questions, we use GPT-4o to refine their wording, ensuring they are well-suited for prompting the language models while making minimal modifications. In Figure 6 we provide a few-shot prompt for question refinement.

In Figure 7, we visualize the embeddings of the question texts (projected to 2-dimensions using t-SNE) from OpinionQA compared to SubPOP-Train and SubPOP-Eval. The visualization shows how much larger our dataset is than OpinionQA (6.5×), along with the expanded coverage of our dataset into semantic areas untouched by OpinionQA. The embeddings also reveal the distribution shift from ATP questions to GSS questions: while the ATP and GSS questions overlap in embedding space, the GSS question appear as small clusters, not evenly distributed over the ATP questions. In Table 4, we list each ATP wave in SubPOP-Train and OpinionQA, along with its number of questions and wave topic(s), as defined by ATP.1 The table indicates which topics are new in SubPOP-Train compared to OpinionQA, indicating the expanded coverage of our dataset, along with which topics remain unseen in OpinionQA, which we can use to test LLMs fine-tuned on SubPOP-Train for generalization.

A.4 SubPOP-Eval

To further evaluate the out-of-distribution generalization ability of our fine-tuned models, we subsample 133 questions from the GSS 2022 dataset (Davern et al., 2024). We apply the same selection criteria as outlined in Appendix A.3, excluding questions that are redacted, conditioned on prior questions, inferable directly from the group

1ATP wave topics and time periods are defined at https://www.pewresearch.org/american-trends-panel-datasets/.

<table>
  <tr>
    <th>Instruction:</th>
    <td>Refine the question with a minimal change to make the question sensible. Do not modify options, and do not modify a question if it makes sense. Always start your answer with "Refined question:".</td>
  </tr>
  <tr>
    <th>Question:</th>
    <td>A cross // Do you have any of the following for spiritual purposes?<br>A. Yes, I have this for spiritual purposes<br>B. No, I do not have this for spiritual purposes</td>
  </tr>
  <tr>
    <th>Refined question:</th>
    <td>Do you have a cross for spiritual purposes?</td>
  </tr>
  <tr>
    <th>Question:</th>
    <td>As you may know, same-sex marriage is now legal in the U.S. Do you think this is [a good thing or a bad thing] for our society?<br>A. Very good thing<br>B. Somewhat good thing<br>C. Somewhat bad thing<br>D. Very bad thing</td>
  </tr>
  <tr>
    <th>Refined question:</th>
    <td>As you may know, same-sex marriage is now legal in the U.S. Do you think this is a good thing or a bad thing for our society?,</td>
  </tr>
  <tr>
    <th>Question:</th>
    <td>On a different subject...How much, if at all, do white people benefit from advantages in society that black people do not have<br>A. A great deal<br>B. A fair amount<br>C. Not too much<br>D. Not at all</td>
  </tr>
  <tr>
    <th>Refined question:</th>
    <td>How much, if at all, do white people benefit from advantages in society that black people do not have?,</td>
  </tr>
  <tr>
    <th>Question:</th>
    <td>Thinking about the past couple of weeks, would you say the news for Donald Trump has been...<br>A. Very good<br>B. Mostly good<br>C. Neither good nor bad<br>D. Mostly bad<br>E. Very bad</td>
  </tr>
  <tr>
    <th>Refined question:</th>
    <td>Thinking about the past couple of weeks, would you say the news for Donald Trump has been...</td>
  </tr>
  <tr>
    <th>Question:</th>
    <td>(Question to refine)<br>(Options)</td>
  </tr>
  <tr>
    <th>Refined question:</th>
    <td></td>
  </tr>
</table>

Figure 6: Few-shot prompt for refining the question to suit a language model prompting. An instruction is designed to make a minimal change to the original question, and in-context examples are provided.

information, derived from a set of questions, or those with more than 10 response options.

A.5 Inspection of Identical Questions

Distribution of cosine similarities between two text embeddings (an output of the embedding model OpenAI-text-embedding-3-large given a question text), one from a question in SubPOP-Train and another from OpinionQA is shown in Figure 8. We observed a fraction of pairs having high cosine similarity, and manually inspected question pairs with high relevance. We find that by setting a
t-SNE of question text embeddings

![t-SNE of question text embeddings](page_184_120_579_377.png)

Figure 7: Embeddings of questions from OpinionQA, SubPOP-Train, and SubPOP-Eval.

Figure 8: Distribution of cosine similarities between a question in SubPOP-ATP and OpinionQA, having a long tail towards a high cosine similarity. We inspect the question pairs in the range of 0.8 to 1.0 (distribution shown in the magnified view) and use a similarity of 0.87 as a safe threshold to identify a semantically identical question pair.

threshold cosine similarity of 0.87 we can detect all semantically identical pairs. We took a conservative threshold of cosine similarity; this value was to maximize the recall at a cost of precision to ensure detection of overlapping questions.

B Experiment Details

We conduct our experiments using Nvidia A100 GPUs with 80GB VRAM. Hyperparameter tuning is performed over learning rates {5e-5, 1e-4, 2e-4} and batch sizes {64, 128, 256}. After evaluating possible combinations, we select a (learning rate, batch size) = (2e-4, 256) for Llama-2-7B, (learning rate, batch size) = (2e-4, 256) for Mistral-7B-v0.1, and (learning rate, batch size) = (1e-4, 256) for Llama-2-13B when utilizing the full training dataset. For Llama-3-70B, we have not done hyperparameter search but heuristically used (learning rate, batch size) = (2e-5, 256). For sub-sampled training data (Figure 5), we use the following configurations:

• (lr, bs) = (2e-4, 256) for 75% of the training data
• (lr, bs) = (1e-4, 128) for 50% of the training data
• (lr, bs) = (1e-4, 128) for 25% of the training data

All training is performed using LoRA (Hu et al., 2021), with LoRA parameters initialized from a normal distribution with \( \sigma = 0.02 \). We set the LoRA rank to 8, alpha to 32, and apply a dropout rate of 0.05. LoRA weights are applied to the query and value matrices. The AdamW (Loshchilov, 2017) optimizer is used with a weight decay of 0.

We use offline batched inference of vLLM (version 0.7.2) (Kwon et al., 2023) for inference and measuring response probability distribution of all methods.

Choice of the Training Objective. In this section, we explore both forward KL-divergence and Wasserstein Distance (WD) as training objectives. The forward KL-divergence is defined as

\[
D_{KL}(p_H \| p_\theta) = \sum_{a \in \mathcal{A}_q} p_H(a) \log \frac{p_H(a)}{p_\theta(a)},
\]

where \( p_H(a) \equiv p_H(a | q,g) \) and \( p_\theta(a) \equiv p_\theta(a | q,g) \). Similarly, WD is given by

\[
\mathcal{WD}(p_H, p_\theta) = \min_{\gamma \in \Pi(p_H, p_\theta)} \sum_{a, a' \in \mathcal{A}_q} \gamma(a, a') d(a, a'),
\]

with \( \Pi(p_H, p_\theta) \) denoting the set of all couplings between \( p_H \) and \( p_\theta \), and \( d(a, a') \) the L1 distance between choices. Since survey responses are inherently one-dimensional and ordinal, we can simplify the computation of WD using cumulative distribution functions (CDFs). In the 1-D case, WD is computed as

\[
\mathcal{WD}(p_H, p_\theta) = \int_{-\infty}^{+\infty} |F_{p_H}(x) - F_{p_\theta}(x)| dx,
\]
\[
= \sum_{i=1}^n |F_{p_H}(i) - F_{p_\theta}(i)|
\]

where \( F_{p_H} \) and \( F_{p_\theta} \) are the CDFs corresponding to \( p_H \) and \( p_\theta \), respectively. We use this discrete formulation as the WD loss in our training.

While training with WD resulted in a higher KL-divergence on the validation set, the validation
Figure 9: Train loss curve (left) and validation loss curve (right) for Llama-2-7B fine-tuned on 90% of OpinionQA, with the remaining 10% used for validation. Light and dark blue lines represent KL-divergence (KL) and Wasserstein distance (WD) when used KL as a training objective, while light and dark red lines represent KL and WD when used WD as a training objective. The two training objectives yield similar results in terms of WD, the primary measure of opinion distribution matching in our work.

WD converged to similar levels regardless of the objective (see Figure 9). We attribute this to KL-divergence penalizing low-probability assignments without significantly altering the overall distribution geometry. Given the KL divergence’s broader applicability—without requiring ordinal information—we primarily used KL-divergence in our experiments.

C Additional Experiments

C.1 Effect of Response Distribution Modeling

In this section, we compare different methods for capturing the distribution of human responses. We consider three approaches:

1. One-hot: Predicting only the most probable response, which ignores the full distribution over all responses (Li et al., 2024).

2. Augment by N: Augmenting the dataset by replicating each response by a factor of N according to its observed frequency (Zhao et al., 2023).

3. Explicit probability modeling: Directly modeling the full response distribution using the actual probability values for each option.

Table 5 summarizes the results of these approaches. Notably, explicit probability modeling substantially outperforms the one-hot method, demonstrating that simply predicting the single most frequent response fails to capture the opinion diversity present within each subpopulation.

Compared with augment by \( N \) (2nd and 3rd column in Table 5), explicit probability modeling also achieves better performance. Importantly, the performance gap exceeds the quantization error introduced by discretizing the response distribution. For instance, when discretizing with a factor of \( N \), the quantization error is \( \frac{1}{2N} \)—approximately 0.01 or 0.005 in the cases shown in Table 5. Moreover, explicit modeling offers the practical benefit of reducing the data volume by a factor of \( N \) compared to the augmentation approach, thereby lowering the computational cost of fine-tuning LLMs.

These results underscore the importance of explicit distribution modeling. By aligning the model’s predictive distribution directly with the survey distribution, we achieve higher accuracy with fewer data samples, avoiding the rounding errors and replication overheads that are inherent to data-augmentation approaches.

C.2 Post-trained Model

We fine-tune Llama-2-7B-chat to observe the effect of starting from checkpoints that have been instruction-tuned via Reinforcement Learning from Human Feedback (RLHF). Table 6 shows the evaluation performance of a baseline method (Zero-shot prompt (QA)), fine-tuned base model (Llama-2-7B) and fine-tuned chat model (Llama-2-7B-chat). We observe the significant performance improvement, while the baseline method performs worse then the models not instruction-tuned (Table 1). Especially, the performance for SubPOP-Eval of chat model is significantly worse than that of base model. We observe the high WD of the baseline method resulting from the model assigning high probability to a specific token (e.g. ‘A’), being far apart from the human opinion distribution. After fine-tuning the model are able to generate a more distributed probability of answer tokens. This result coincides with the result reported in (Moon et al., 2024).

<table>
  <tr>
    <th>Eval Dataset</th>
    <th>One-hot</th>
    <th>Aug. (\( \times 50 \))</th>
    <th>Aug. (\( \times 100 \))</th>
    <th>Explicit (Ours)</th>
  </tr>
  <tr>
    <td>OpinionQA</td>
    <td>0.163</td>
    <td>0.110</td>
    <td>0.107</td>
    <td>0.106</td>
  </tr>
  <tr>
    <td>SubPOP-Eval</td>
    <td>0.178</td>
    <td>0.130</td>
    <td>0.123</td>
    <td>0.121</td>
  </tr>
</table>

Table 5: Comparison of evaluation performance for three response distribution modeling approaches, with Llama-2-7B as a base model. The last column (Explicit) is identical to the ours presented in Table 1. A model fine-tuned to predict the most probable choice (one-hot) performs the worst, as the model has not learned distributional opinion at fine-tuning phase. A model trained on augmented data (Aug. (\( \times 50 \), \( \times 100 \))), while performing much better than one-hot still underperforms the explicit distribution modeling.
Table 6: Performance of the fine-tuned Llama-2-7B-chat model (Chat LLM FT). For comparison, we also present lower and upper bounds, the baseline method Zero-shot prompt (QA) and our fine-tuned Llama-2-7B (Base LLM FT).

<table>
  <tr>
    <th>Method</th>
    <th>OpinionQA</th>
    <th>SubPOP-Eval</th>
  </tr>
  <tr>
    <td>Upper bound (UniE.)</td>
    <td>0.178</td>
    <td>0.208</td>
  </tr>
  <tr>
    <td>Lower bound (Human)</td>
    <td>0.031</td>
    <td>0.033</td>
  </tr>
  <tr>
    <td>Base zero-shot prompt (QA)</td>
    <td>0.173</td>
    <td>0.206</td>
  </tr>
  <tr>
    <td>Base LLM FT</td>
    <td>0.106</td>
    <td>0.121</td>
  </tr>
  <tr>
    <td>Chat zero-shot prompt (QA)</td>
    <td>0.308</td>
    <td>0.383</td>
  </tr>
  <tr>
    <td>Chat LLM FT</td>
    <td>0.109</td>
    <td>0.148</td>
  </tr>
</table>

C.3 Generalization to Unseen Subpopulations

Here we present a complete list of evaluation performance on OpinionQA for unseen subpopulations (the groups not used to fine-tune our model) and perform an analysis that shows our fine-tuned models are able to steer towards the given subpopulation information.

As shown in Table 7, we observe a performance improvement across unseen subpopulations. To verify that the performance improvements arise from the fine-tuned model being able to steer towards given subpopulations, we measure inter-group disagreement pattern for the demographic and ideology traits, shown in Figure 10, 11, 12, and 13. We consistently observe across traits that the disagreement pattern of our model resembles that of the human group, while zero-shot prompting with the base model exhibits a pattern completely different from the human group result. This observation shows that our fine-tuned model learns to condition on subpopulation information and also generalizes to subpopulations unseen during fine-tuning.

D Baseline Details

• Zero-shot prompting: Three prompt styles—QA, B1O, and PORTRAY—are introduced in (Santurkar et al., 2023) to integrate group information into prompts. These prompts are then combined with survey questions to construct inputs for LLM. Then, the first-token log-probability from LLM is measured to calculate the model’s response distribution over options. In our baseline (and also in fine-tuning experiments) we focus on the QA steering format. Examples of this prompting method are shown in Figure 14.

• Few-shot prompting: We craft a conditioning prompt that contains not only group information but also the group’s response distribution to \( k \) train questions, following (Hwang et al., 2023). For a test question \( q_{test} \in Q_{test} \), we first sort training questions \( Q_{train} \) into \( \{q_1, q_2, ...\} \) such that \( \mathrm{sim}(\mathrm{E}(q_1), \mathrm{E}(q_{test})) > \mathrm{sim}(\mathrm{E}(q_2), \mathrm{E}(q_{test})) \), and so on. \( \mathrm{E}(q) \) denotes the embedding model (OpenAI-text-embedding-3-large) output of the input \( q \) and \( \mathrm{sim} \) is a cosine similarity between two embedding vectors. Then, response information of the first \( k \) questions \( \{q_i, p(\mathcal{A}_{q_i}|q_i; g)\}_{i=1}^k \) are used as few shot prompts to have the language model verbalize (Meister et al., 2024) expected response distribution for the given \( g \) and \( q_{test} \). An example of the prompt for \( k = 3 \) case is shown in Figure 15, while we run the baseline experiment in a \( k = 5 \) setting.

• Modular Pluralism: The intuition behind Modular Pluralism (Feng et al., 2024) is that a language model trained on a text corpus of a specific subpopulation will faithfully represent public opinion of that population. Given a survey question with a PORTRAY-style steering prompt, each of language model ‘modules’ (fine-tuned Mistral-7B-Instruct-v0.1) generates an option choice with explanation. A black-box LLM (GPT-3.5-turbo-Instruct) receives all generations and select a generation that best aligns with the given group. Finally, using the chosen generation as a context, a black-box LLM generates probability distribution over options. The example pipeline is shown in Figure 16. Instead of the sub-sampled OpinionQA dataset the authors of the method used, we use the exactly same evaluation set across all baseline methods and our approach for a fair comparison.

• Upper bound: We estimate the distribution between human responses and uniform distribution as an upper bound of WD metrics.

• Lower bound: We compute a lower bound by randomly sampling a group of respondents and calculating the Wasserstein distance (WD) between the distribution of the sampled group and that of the original respondents for each question. We then bootstrap with \( R = 1000 \) to construct a 95% confidence interval (CI) for the WDs. Further details on this estimation process are provided below.

Computing weighted answer distributions:
For each group \( g \) and question \( q \), we have \( n_{gq} \) responses from respondents who belong to group \( g \) answering question \( q \): \( x_1, x_2, \cdots, x_{n_{gq}} \), where \( x_i \in \mathcal{A}_q \), i.e., the answer set for question
Table 7: Evaluation performance of our fine-tuned Llama-2-7B model on OpinionQA for subpopulations not included in the fine-tuning dataset SubPOP-Train. For reference, we present a lower bound (human) and the zero-shot prompting (QA). Absolute difference refers to the WD difference between zero-shot prompting and ours, and the relative improvement is calculated in a same way as Figure 3.

<table>
  <tr>
    <th>Attribute</th>
    <th>Group</th>
    <th>Lower Bound (Human)</th>
    <th>Zero-shot (QA)</th>
    <th>Ours</th>
    <th>Absolute Diff.</th>
    <th>Relative Improvement</th>
  </tr>
  <tr>
    <td>Age</td>
    <td>18-29</td>
    <td>0.023</td>
    <td>0.185</td>
    <td>0.096</td>
    <td>0.089</td>
    <td>0.548</td>
  </tr>
  <tr>
    <td>Age</td>
    <td>30-49</td>
    <td>0.014</td>
    <td>0.151</td>
    <td>0.093</td>
    <td>0.058</td>
    <td>0.424</td>
  </tr>
  <tr>
    <td>Age</td>
    <td>50-64</td>
    <td>0.014</td>
    <td>0.154</td>
    <td>0.101</td>
    <td>0.052</td>
    <td>0.377</td>
  </tr>
  <tr>
    <td>Age</td>
    <td>65+</td>
    <td>0.013</td>
    <td>0.195</td>
    <td>0.115</td>
    <td>0.080</td>
    <td>0.438</td>
  </tr>
  <tr>
    <td>Region</td>
    <td>Midwest</td>
    <td>0.016</td>
    <td>0.153</td>
    <td>0.095</td>
    <td>0.058</td>
    <td>0.425</td>
  </tr>
  <tr>
    <td>Region</td>
    <td>West</td>
    <td>0.017</td>
    <td>0.162</td>
    <td>0.095</td>
    <td>0.068</td>
    <td>0.465</td>
  </tr>
  <tr>
    <td>Education</td>
    <td>Associate’s Degree</td>
    <td>0.026</td>
    <td>0.159</td>
    <td>0.098</td>
    <td>0.061</td>
    <td>0.455</td>
  </tr>
  <tr>
    <td>Education</td>
    <td>High School Graduate</td>
    <td>0.017</td>
    <td>0.144</td>
    <td>0.092</td>
    <td>0.053</td>
    <td>0.413</td>
  </tr>
  <tr>
    <td>Education</td>
    <td>Postgraduate</td>
    <td>0.015</td>
    <td>0.174</td>
    <td>0.106</td>
    <td>0.068</td>
    <td>0.426</td>
  </tr>
  <tr>
    <td>Education</td>
    <td>Some College, No Degree</td>
    <td>0.018</td>
    <td>0.144</td>
    <td>0.093</td>
    <td>0.051</td>
    <td>0.405</td>
  </tr>
  <tr>
    <td>Income</td>
    <td>$50,000-$75,000</td>
    <td>0.016</td>
    <td>0.153</td>
    <td>0.098</td>
    <td>0.054</td>
    <td>0.396</td>
  </tr>
  <tr>
    <td>Income</td>
    <td>$30,000-$50,000</td>
    <td>0.019</td>
    <td>0.144</td>
    <td>0.094</td>
    <td>0.050</td>
    <td>0.400</td>
  </tr>
  <tr>
    <td>Political Ideology</td>
    <td>Very Conservative</td>
    <td>0.026</td>
    <td>0.208</td>
    <td>0.107</td>
    <td>0.101</td>
    <td>0.555</td>
  </tr>
  <tr>
    <td>Political Ideology</td>
    <td>Very Liberal</td>
    <td>0.025</td>
    <td>0.202</td>
    <td>0.111</td>
    <td>0.091</td>
    <td>0.514</td>
  </tr>
  <tr>
    <td>Political Party</td>
    <td>Independent</td>
    <td>0.016</td>
    <td>0.155</td>
    <td>0.093</td>
    <td>0.062</td>
    <td>0.445</td>
  </tr>
  <tr>
    <td>Political Party</td>
    <td>Something Else</td>
    <td>0.020</td>
    <td>0.162</td>
    <td>0.092</td>
    <td>0.070</td>
    <td>0.310</td>
  </tr>
  <tr>
    <td>Race</td>
    <td>Other</td>
    <td>0.050</td>
    <td>0.180</td>
    <td>0.114</td>
    <td>0.066</td>
    <td>0.275</td>
  </tr>
  <tr>
    <td>Religion</td>
    <td>Agnostic</td>
    <td>0.028</td>
    <td>0.189</td>
    <td>0.115</td>
    <td>0.074</td>
    <td>0.459</td>
  </tr>
  <tr>
    <td>Religion</td>
    <td>Buddhist</td>
    <td>0.063</td>
    <td>0.207</td>
    <td>0.149</td>
    <td>0.059</td>
    <td>0.405</td>
  </tr>
  <tr>
    <td>Religion</td>
    <td>Nothing in Particular</td>
    <td>0.019</td>
    <td>0.153</td>
    <td>0.092</td>
    <td>0.061</td>
    <td>0.454</td>
  </tr>
  <tr>
    <td>Religion</td>
    <td>Orthodox</td>
    <td>0.083</td>
    <td>0.221</td>
    <td>0.180</td>
    <td>0.041</td>
    <td>0.298</td>
  </tr>
  <tr>
    <td>Religion</td>
    <td>Other</td>
    <td>0.051</td>
    <td>0.184</td>
    <td>0.123</td>
    <td>0.061</td>
    <td>0.457</td>
  </tr>
  <tr>
    <td>Religion</td>
    <td>Roman Catholic</td>
    <td>0.018</td>
    <td>0.145</td>
    <td>0.098</td>
    <td>0.047</td>
    <td>0.371</td>
  </tr>
</table>

![Heatmap of intergroup disagreement between a target human group (y-axis) and a source group (x-axis), either a human group or a group simulated with the language model, for OpinionQA evaluation data and age trait using Llama-2-7B as a base model. All subpopulations are unseen during fine-tuning.](page_1012_1042_388_388.png)

Figure 10: Heatmap of intergroup disagreement between a target human group (y-axis) and a source group (x-axis), either a human group or a group simulated with the language model, for OpinionQA evaluation data and age trait using Llama-2-7B as a base model. All subpopulations are unseen during fine-tuning.

q (e.g., {1,2,3,4}). Furthermore, each respondent (and thus, their response) is associated with a wave-specific weight \( w_1, w_2, \ldots, w_{n_g} \), provided by Pew Research. We compute the human answer distribution \( \pi^{(H)}_{gq} \) as a weighted sum over responses, where the proportion of respondents providing answer \( a \in \mathcal{A}_q \) is estimated as

\[
\pi^{(H)}_{gq}(a) = \frac{\sum_{i=1}^{n_{gq}} w_i \mathbb{I}[x_i = a]}{\sum_{i=1}^{n_{gq}} w_i}.
\]

Bootstrapping at the respondent-level: We draw bootstrap samples per group at the respondent-level including questions from all survey waves. This allows us to capture correlations in answer distributions across questions and across waves.

Specifically, let \( \mathcal{P}_g \) represent the set of respondents in group \( g \), where \( |\mathcal{P}_g| = n_g \). We produce bootstrapped samples by repeatedly sampling \( n_g \) respondents from \( \mathcal{P}_g \) with replacement. Let \( p_1^{(r)}, p_2^{(r)}, \ldots, p_{n_g}^{(r)} \) represent the sampled respondents for the \( r \)-th bootstrap, and let \( w_1^{(r)}, w_2^{(r)}, \ldots, w_{n_g}^{(r)} \) represent their corresponding weights.

For each question \( q \), let \( \mathcal{P}_{gq} \subseteq \mathcal{P}_g \) represent the set of respondents from group \( g \) who answered question \( q \); as before, \( |\mathcal{P}_{gq}| = n_{gq} \). Let us define \( q(p_i) \) as person \( p_i \)'s response to question \( q \) if \( p_i \) answered question \( q \), i.e., \( p_i \in \mathcal{P}_{gq} \), and 0 otherwise. Then, we compute the \( r \)-th answer
Figure 11: Heatmap of intergroup disagreement between a target human group (y-axis) and a source group (x-axis, either a human group or a group simulated with the language model), for OpinionQA evaluation data and political party (affiliation) trait using Llama-2-7B as a base model. Two subpopulations, Democrat and Republican, are seen during fine-tuning, while Independent and Something Else are unseen.

Figure 12: Heatmap of intergroup disagreement between a target human group (y-axis) and a source group (x-axis, either a human group or a group simulated with the language model), for OpinionQA evaluation data and race / ethnicity trait using Llama-2-7B as a base model. Four subpopulations except 'Other' are seen during fine-tuning. In this case, the model does not well predict the opinions of Other group. We suspect this occurs because Other is a group with highly diverse race or ethnicity backgrounds, making it inherently difficult to infer its opinion distribution from those of White, Hispanic, Black, and Asian subpopulations.

distribution to question \( q \) as:
\[
\pi_{gq}^{(r)}(a) = \frac{\sum_{i=1}^{n_g} \mathbb{I}[p_i^{(r)} \in \mathcal{P}_{gq}] w_i^{(r)} \mathbb{I}[q(p_i^{(r)}) = a]}{\sum_{i=1}^{n_g} \mathbb{I}[p_i^{(r)} \in \mathcal{P}_{gq}] w_i^{(r)}}.
\]

Human lower bound of WD. Our statistic of interest is the mean Wasserstein distance over all questions \( Q \) across all waves per group. We approximate this as the WD between the observed human distribution \( \pi_{gq}^{(H)} \) and the bootstrap sample \( \pi_{gq}^{(r)} \) for question \( q \) and group \( g \). Over all \( R = 1000 \) bootstraps, we have
\[
\mathcal{D}_g^{(H)} = \left\{ \frac{1}{|Q|} \sum_{q \in Q} WD(\pi_{gq}^{(H)}, \pi_{gq}^{(r)}) \right\}_{r=1}^R.
\]

To quantify agreement between human samples, we report the mean and 95% CI (i.e., from 2.5\( ^{th} \) to 97.5\( ^{th} \) percentiles) of \( \mathcal{D}_g^{(H)} \).

E  Wave, Group-level Opinion Matching

Here we present a group-level and wave-level averaged Wasserstein distance. Wave-level result is in Table 8, and group-level results for OpinionQA and SubPOP-Eval are in Table 9, 10, respectively. We observe that the improvements in distribution matching between LLM response and human response are consistent across diverse subpopulations and waves.
Figure 13: Heatmap of intergroup disagreement between a target human group (y-axis) and a source group (x-axis, either a human group or a group simulated with the language model), for OpinionQA evaluation data and political ideology trait using Llama-2-7B as a base model. Three subpopulations, Conservative, Moderate, and Liberal are seen during fine-tuning, while Very conservative and Very liberal are not seen.

Table 8: Per-wave Wasserstein distance on OpinionQA for each base model, comparing baseline zero-shot prompting (QA) with our fine-tuned model Ours(SubPOP-FT). Highlighted rows represent waves whose topics are not covered by the training data (SubPOP-Train). We observe WD improvement consistently across survey waves and also for waves of topics not covered in the training data.

<table>
  <tr>
    <th>Wave</th>
    <th>Llama-2-7B<br>Zero-shot<br>Ours (SubPOP-FT)</th>
    <th>Llama-2-13B<br>Zero-shot<br>Ours (SubPOP-FT)</th>
    <th>Mistral-7B-v0.1<br>Zero-shot<br>Ours (SubPOP-FT)</th>
    <th>Llama-3-70B<br>Zero-shot<br>Ours (SubPOP-FT)</th>
  </tr>
  <tr><td>26</td><td>0.191<br>0.145</td><td>0.180<br>0.126</td><td>0.178<br>0.131</td><td>0.134<br>0.084</td></tr>
  <tr><td>29</td><td>0.169<br>0.096</td><td>0.172<br>0.123</td><td>0.153<br>0.096</td><td>0.125<br>0.085</td></tr>
  <tr><td>32</td><td>0.163<br>0.110</td><td>0.156<br>0.098</td><td>0.137<br>0.091</td><td>0.151<br>0.091</td></tr>
  <tr><td>34</td><td>0.155<br>0.105</td><td>0.171<br>0.089</td><td>0.134<br>0.085</td><td>0.138<br>0.083</td></tr>
  <tr><td>36</td><td>0.175<br>0.120</td><td>0.184<br>0.126</td><td>0.175<br>0.107</td><td>0.150<br>0.087</td></tr>
  <tr><td>41</td><td>0.160<br>0.090</td><td>0.155<br>0.084</td><td>0.134<br>0.073</td><td>0.116<br>0.085</td></tr>
  <tr><td>42</td><td>0.159<br>0.053</td><td>0.146<br>0.059</td><td>0.127<br>0.059</td><td>0.131<br>0.084</td></tr>
  <tr><td>43</td><td>0.179<br>0.112</td><td>0.172<br>0.104</td><td>0.154<br>0.102</td><td>0.124<br>0.099</td></tr>
  <tr><td>45</td><td>0.177<br>0.101</td><td>0.177<br>0.093</td><td>0.149<br>0.084</td><td>0.126<br>0.091</td></tr>
  <tr><td>49</td><td>0.151<br>0.098</td><td>0.143<br>0.131</td><td>0.128<br>0.116</td><td>0.159<br>0.087</td></tr>
  <tr><td>50</td><td>0.209<br>0.139</td><td>0.196<br>0.121</td><td>0.188<br>0.125</td><td>0.154<br>0.078</td></tr>
  <tr><td>54</td><td>0.158<br>0.087</td><td>0.158<br>0.087</td><td>0.128<br>0.077</td><td>0.118<br>0.079</td></tr>
  <tr><td>82</td><td>0.173<br>0.098</td><td>0.171<br>0.075</td><td>0.148<br>0.077</td><td>0.174<br>0.093</td></tr>
  <tr><td>92</td><td>0.165<br>0.073</td><td>0.155<br>0.071</td><td>0.140<br>0.055</td><td>0.126<br>0.081</td></tr>
</table>

Table 9: Per-group Wasserstein distance on OpinionQA for each base models, before and after fine-tuning on SubPOP-Train. Base refers to zero-shot prompting (QA). *Full group variable name is “College grad, some Postgrad”.

<table>
  <tr>
    <th rowspan="2">Attribute</th>
    <th rowspan="2">Group</th>
    <th colspan="2">Human<br>Baseline</th>
    <th colspan="2">Llama-2-7B<br>Base<br>Fine-tuned</th>
    <th colspan="2">Llama-2-13B<br>Base<br>Fine-tuned</th>
    <th colspan="2">Mistral-7B-v0.1<br>Base<br>Fine-tuned</th>
    <th colspan="2">Llama-3-70B<br>Base<br>Fine-tuned</th>
  </tr>
  <tr>
    <th>Base</th>
    <th>Fine-tuned</th>
    <th>Base</th>
    <th>Fine-tuned</th>
    <th>Base</th>
    <th>Fine-tuned</th>
    <th>Base</th>
    <th>Fine-tuned</th>
    <th>Base</th>
    <th>Fine-tuned</th>
  </tr>
  <tr>
    <td rowspan="2">Region</td>
    <td>Northeast</td>
    <td>0.023</td><td>0.165</td>
    <td>0.094</td><td>0.155</td>
    <td>0.088</td><td>0.155</td>
    <td>0.083</td><td>0.134</td>
    <td>0.084</td><td>0.084</td>
  </tr>
  <tr>
    <td>South</td>
    <td>0.017</td><td>0.149</td>
    <td>0.092</td><td>0.143</td>
    <td>0.085</td><td>0.133</td>
    <td>0.081</td><td>0.113</td>
    <td>0.078</td><td>0.078</td>
  </tr>
  <tr>
    <td rowspan="2">Education</td>
    <td>College grad*</td>
    <td>0.018</td><td>0.165</td>
    <td>0.099</td><td>0.157</td>
    <td>0.096</td><td>0.136</td>
    <td>0.089</td><td>0.125</td>
    <td>0.085</td><td>0.085</td>
  </tr>
  <tr>
    <td>Less than high school</td>
    <td>0.043</td><td>0.161</td>
    <td>0.101</td><td>0.150</td>
    <td>0.096</td><td>0.134</td>
    <td>0.094</td><td>0.151</td>
    <td>0.091</td><td>0.091</td>
  </tr>
  <tr>
    <td rowspan="2">Gender</td>
    <td>Male</td>
    <td>0.015</td><td>0.182</td>
    <td>0.093</td><td>0.152</td>
    <td>0.089</td><td>0.131</td>
    <td>0.083</td><td>0.138</td>
    <td>0.083</td><td>0.083</td>
  </tr>
  <tr>
    <td>Female</td>
    <td>0.013</td><td>0.162</td>
    <td>0.100</td><td>0.158</td>
    <td>0.092</td><td>0.146</td>
    <td>0.088</td><td>0.130</td>
    <td>0.130</td><td>0.087</td>
  </tr>
  <tr>
    <td rowspan="4">Race / ethnicity</td>
    <td>Black</td>
    <td>0.031</td><td>0.151</td>
    <td>0.102</td><td>0.144</td>
    <td>0.095</td><td>0.132</td>
    <td>0.091</td><td>0.116</td>
    <td>0.085</td><td>0.085</td>
  </tr>
  <tr>
    <td>White</td>
    <td>0.012</td><td>0.176</td>
    <td>0.097</td><td>0.178</td>
    <td>0.093</td><td>0.145</td>
    <td>0.085</td><td>0.131</td>
    <td>0.131</td><td>0.084</td>
  </tr>
  <tr>
    <td>Asian</td>
    <td>0.051</td><td>0.165</td>
    <td>0.111</td><td>0.167</td>
    <td>0.104</td><td>0.143</td>
    <td>0.102</td><td>0.124</td>
    <td>0.099</td><td>0.099</td>
  </tr>
  <tr>
    <td>Hispanic</td>
    <td>0.044</td><td>0.162</td>
    <td>0.102</td><td>0.163</td>
    <td>0.098</td><td>0.134</td>
    <td>0.092</td><td>0.126</td>
    <td>0.091</td><td>0.091</td>
  </tr>
  <tr>
    <td rowspan="2">Income</td>
    <td>$100,000 or more</td>
    <td>0.019</td><td>0.172</td>
    <td>0.103</td><td>0.162</td>
    <td>0.100</td><td>0.147</td>
    <td>0.091</td><td>0.159</td>
    <td>0.087</td><td>0.087</td>
  </tr>
  <tr>
    <td>Less than $50,000</td>
    <td>0.012</td><td>0.162</td>
    <td>0.091</td><td>0.148</td>
    <td>0.083</td><td>0.127</td>
    <td>0.080</td><td>0.154</td>
    <td>0.078</td><td>0.078</td>
  </tr>
  <tr>
    <td rowspan="2">Political Party</td>
    <td>Democrat</td>
    <td>0.016</td><td>0.172</td>
    <td>0.099</td><td>0.158</td>
    <td>0.092</td><td>0.161</td>
    <td>0.082</td><td>0.118</td>
    <td>0.079</td><td>0.079</td>
  </tr>
  <tr>
    <td>Republican</td>
    <td>0.016</td><td>0.196</td>
    <td>0.105</td><td>0.235</td>
    <td>0.101</td><td>0.181</td>
    <td>0.095</td><td>0.174</td>
    <td>0.093</td><td>0.093</td>
  </tr>
  <tr>
    <td rowspan="3">Political Ideology</td>
    <td>Liberal</td>
    <td>0.022</td><td>0.192</td>
    <td>0.100</td><td>0.181</td>
    <td>0.094</td><td>0.166</td>
    <td>0.084</td><td>0.126</td>
    <td>0.081</td><td>0.081</td>
  </tr>
  <tr>
    <td>Conservative</td>
    <td>0.021</td><td>0.169</td>
    <td>0.103</td><td>0.153</td>
    <td>0.099</td><td>0.144</td>
    <td>0.094</td><td>0.141</td>
    <td>0.092</td><td>0.092</td>
  </tr>
  <tr>
    <td>Moderate</td>
    <td>0.016</td><td>0.151</td>
    <td>0.094</td><td>0.153</td>
    <td>0.090</td><td>0.132</td>
    <td>0.082</td><td>0.106</td>
    <td>0.081</td><td>0.081</td>
  </tr>
  <tr>
    <td rowspan="5">Religion</td>
    <td>Protestant</td>
    <td>0.016</td><td>0.015</td>
    <td>0.166</td><td>0.096</td>
    <td>0.158</td><td>0.092</td>
    <td>0.146</td><td>0.086</td>
    <td>0.143</td><td>0.143</td>
  </tr>
  <tr>
    <td>Jewish</td>
    <td>0.058</td><td>0.182</td>
    <td>0.124</td><td>0.182</td>
    <td>0.122</td><td>0.165</td>
    <td>0.115</td><td>0.144</td>
    <td>0.115</td><td>0.115</td>
  </tr>
  <tr>
    <td>Hindu</td>
    <td>0.079</td><td>0.211</td>
    <td>0.160</td><td>0.232</td>
    <td>0.163</td><td>0.211</td>
    <td>0.161</td><td>0.181</td>
    <td>0.157</td><td>0.157</td>
  </tr>
  <tr>
    <td>Atheist</td>
    <td>0.035</td><td>0.202</td>
    <td>0.118</td><td>0.204</td>
    <td>0.110</td><td>0.196</td>
    <td>0.099</td><td>0.135</td>
    <td>0.098</td><td>0.098</td>
  </tr>
  <tr>
    <td>Muslim</td>
    <td>0.089</td><td>0.202</td>
    <td>0.159</td><td>0.209</td>
    <td>0.156</td><td>0.204</td>
    <td>0.146</td><td>0.171</td>
    <td>0.144</td><td>0.144</td>
  </tr>
</table>
Table 10: Per-group Wasserstein distance on SubPOP-Eval for each base models, before and after fine-tuning on SubPOP-Train. Base refers to zero-shot prompting (QA). *Full group variable name is "College grad, some Postgrad".

<table>
  <tr>
    <th rowspan="2">Attribute</th>
    <th rowspan="2">Group</th>
    <th colspan="2">Human Baseline</th>
    <th colspan="2">Llama-2-7B Base Fine-tuned</th>
    <th colspan="2">Llama-2-13B Base Fine-tuned</th>
    <th colspan="2">Mistral-7B-v0.1 Base Fine-tuned</th>
    <th colspan="2">Llama-3-70B Base Fine-tuned</th>
  </tr>
  <tr>
    <th>Base</th>
    <th>Fine-tuned</th>
    <th>Base</th>
    <th>Fine-tuned</th>
    <th>Base</th>
    <th>Fine-tuned</th>
    <th>Base</th>
    <th>Fine-tuned</th>
    <th>Base</th>
    <th>Fine-tuned</th>
  </tr>
  <tr>
    <td rowspan="2">Region</td>
    <td>Northeast</td>
    <td>0.027</td>
    <td>0.196</td>
    <td>0.113</td>
    <td>0.193</td>
    <td>0.103</td>
    <td>0.185</td>
    <td>0.108</td>
    <td>0.156</td>
    <td>0.078</td>
  </tr>
  <tr>
    <td>South</td>
    <td>0.018</td>
    <td>0.183</td>
    <td>0.108</td>
    <td>0.185</td>
    <td>0.103</td>
    <td>0.176</td>
    <td>0.103</td>
    <td>0.138</td>
    <td>0.080</td>
  </tr>
  <tr>
    <td rowspan="2">Education</td>
    <td>College grad*</td>
    <td>0.019</td>
    <td>0.206</td>
    <td>0.105</td>
    <td>0.175</td>
    <td>0.101</td>
    <td>0.167</td>
    <td>0.099</td>
    <td>0.137</td>
    <td>0.077</td>
  </tr>
  <tr>
    <td>Less than high school</td>
    <td>0.036</td>
    <td>0.191</td>
    <td>0.129</td>
    <td>0.182</td>
    <td>0.117</td>
    <td>0.172</td>
    <td>0.121</td>
    <td>0.180</td>
    <td>0.108</td>
  </tr>
  <tr>
    <td rowspan="2">Gender</td>
    <td>Male</td>
    <td>0.017</td>
    <td>0.186</td>
    <td>0.102</td>
    <td>0.176</td>
    <td>0.101</td>
    <td>0.170</td>
    <td>0.099</td>
    <td>0.150</td>
    <td>0.079</td>
  </tr>
  <tr>
    <td>Female</td>
    <td>0.016</td>
    <td>0.184</td>
    <td>0.108</td>
    <td>0.198</td>
    <td>0.105</td>
    <td>0.176</td>
    <td>0.100</td>
    <td>0.151</td>
    <td>0.080</td>
  </tr>
  <tr>
    <td rowspan="3">Race / ethnicity</td>
    <td>Black</td>
    <td>0.029</td>
    <td>0.200</td>
    <td>0.114</td>
    <td>0.179</td>
    <td>0.102</td>
    <td>0.170</td>
    <td>0.107</td>
    <td>0.139</td>
    <td>0.094</td>
  </tr>
  <tr>
    <td>White</td>
    <td>0.014</td>
    <td>0.190</td>
    <td>0.105</td>
    <td>0.187</td>
    <td>0.103</td>
    <td>0.181</td>
    <td>0.102</td>
    <td>0.153</td>
    <td>0.083</td>
  </tr>
  <tr>
    <td>Asian</td>
    <td>0.049</td>
    <td>0.201</td>
    <td>0.119</td>
    <td>0.190</td>
    <td>0.107</td>
    <td>0.184</td>
    <td>0.114</td>
    <td>0.158</td>
    <td>0.096</td>
  </tr>
  <tr>
    <td>Hispanic</td>
    <td>0.030</td>
    <td>0.204</td>
    <td>0.133</td>
    <td>0.199</td>
    <td>0.122</td>
    <td>0.182</td>
    <td>0.134</td>
    <td>0.172</td>
    <td>0.115</td>
  </tr>
  <tr>
    <td rowspan="2">Income</td>
    <td>$100,000 or more</td>
    <td>0.021</td>
    <td>0.210</td>
    <td>0.111</td>
    <td>0.184</td>
    <td>0.106</td>
    <td>0.176</td>
    <td>0.102</td>
    <td>0.179</td>
    <td>0.082</td>
  </tr>
  <tr>
    <td>Less than $30,000</td>
    <td>0.026</td>
    <td>0.179</td>
    <td>0.115</td>
    <td>0.172</td>
    <td>0.103</td>
    <td>0.165</td>
    <td>0.105</td>
    <td>0.171</td>
    <td>0.086</td>
  </tr>
  <tr>
    <td rowspan="2">Political Party</td>
    <td>Democrat</td>
    <td>0.020</td>
    <td>0.219</td>
    <td>0.103</td>
    <td>0.197</td>
    <td>0.092</td>
    <td>0.199</td>
    <td>0.091</td>
    <td>0.128</td>
    <td>0.076</td>
  </tr>
  <tr>
    <td>Republican</td>
    <td>0.023</td>
    <td>0.205</td>
    <td>0.123</td>
    <td>0.234</td>
    <td>0.117</td>
    <td>0.206</td>
    <td>0.115</td>
    <td>0.187</td>
    <td>0.093</td>
  </tr>
  <tr>
    <td rowspan="3">Political Ideology</td>
    <td>Liberal</td>
    <td>0.019</td>
    <td>0.224</td>
    <td>0.102</td>
    <td>0.191</td>
    <td>0.090</td>
    <td>0.188</td>
    <td>0.096</td>
    <td>0.134</td>
    <td>0.076</td>
  </tr>
  <tr>
    <td>Conservative</td>
    <td>0.022</td>
    <td>0.184</td>
    <td>0.120</td>
    <td>0.178</td>
    <td>0.112</td>
    <td>0.172</td>
    <td>0.113</td>
    <td>0.160</td>
    <td>0.092</td>
  </tr>
  <tr>
    <td>Moderate</td>
    <td>0.018</td>
    <td>0.191</td>
    <td>0.110</td>
    <td>0.183</td>
    <td>0.103</td>
    <td>0.170</td>
    <td>0.103</td>
    <td>0.141</td>
    <td>0.082</td>
  </tr>
  <tr>
    <td rowspan="5">Religion</td>
    <td>Protestant</td>
    <td>0.019</td>
    <td>0.187</td>
    <td>0.110</td>
    <td>0.179</td>
    <td>0.107</td>
    <td>0.172</td>
    <td>0.105</td>
    <td>0.164</td>
    <td>0.082</td>
  </tr>
  <tr>
    <td>Jewish</td>
    <td>0.066</td>
    <td>0.245</td>
    <td>0.149</td>
    <td>0.226</td>
    <td>0.144</td>
    <td>0.218</td>
    <td>0.129</td>
    <td>0.164</td>
    <td>0.119</td>
  </tr>
  <tr>
    <td>Hindu</td>
    <td>0.095</td>
    <td>0.264</td>
    <td>0.180</td>
    <td>0.253</td>
    <td>0.169</td>
    <td>0.252</td>
    <td>0.186</td>
    <td>0.223</td>
    <td>0.166</td>
  </tr>
  <tr>
    <td>Atheist</td>
    <td>0.021</td>
    <td>0.222</td>
    <td>0.126</td>
    <td>0.207</td>
    <td>0.103</td>
    <td>0.199</td>
    <td>0.116</td>
    <td>0.152</td>
    <td>0.106</td>
  </tr>
  <tr>
    <td>Muslim</td>
    <td>0.090</td>
    <td>0.253</td>
    <td>0.175</td>
    <td>0.240</td>
    <td>0.181</td>
    <td>0.238</td>
    <td>0.173</td>
    <td>0.203</td>
    <td>0.158</td>
  </tr>
</table>
Question: Which part of the United States do you currently live in?
A. Northeast
B. Midwest
C. South
D. West

Answer: A. Northeast

Answer the following question keeping in mind your previous answers.
Question: I am going to name some institutions in this country. As far as the people running these institutions are concerned, would you say you have a great deal of confidence, only some confidence, or hardly any confidence at all in them? Military
A. A great deal
B. Only some
C. Hardly any
D. Refused
Answer as a choice between A.,B.,C.,D.

Answer:

Question: What is your present religion, if any?
A. Protestant
B. Roman Catholic
C. Mormon
D. Orthodox
E. Jewish
F. Muslim
G. Buddhist
H. Hindu
I. Atheist
J. Agnostic
K. Other
L. Nothing in particular

Answer: I. Atheist

Answer the following question keeping in mind your previous answers.
Question: Do you believe there is a life after death?
A. Yes
B. No
C. Refused
Answer as a choice between A.,B.,C.

Answer:

Predict the answer distribution of the following questions for people whose race or ethnicity is White.

Question: Overall, how does being Asian affect people's ability to get ahead in our country these days?
A. Helps a lot
B. Helps a little
C. Hurts a little
D. Hurts a lot
E. Neither helps nor hurts
F. Refused

Answer distribution: {'A': 0.050, 'B': 0.210, 'C': 0.241, 'D': 0.034, 'E': 0.474, 'F': 0.010}

Question: Thinking about the people who design artificial intelligence computer programs, how well do you think they take into account the experiences and views of Asian adults?
A. Very well
B. Somewhat well
C. Not too well
D. Not at all well
E. Not sure
F. Refused

Answer distribution: {'A': 0.078, 'B': 0.216, 'C': 0.138, 'D': 0.069, 'E': 0.507, 'F': 0.008}

Question: Thinking about the people who design artificial intelligence computer programs, how well do you think they take into account the experiences and views of Hispanic adults?
A. Very well
B. Somewhat well
C. Not too well
D. Not at all well
E. Not sure
F. Refused

Answer distribution: {'A': 0.050, 'B': 0.160, 'C': 0.203, 'D': 0.090, 'E': 0.505, 'F': 0.008}

Question: Thinking again about race and race relations in the U.S. in general, how well, if at all, do you think each of these groups get along with each other in our society these days? Hispanics and Asians
A. Very well
B. Pretty well
C. Not too well
D. Not at all well
E. Not sure
F. Refused

Answer distribution:

Figure 14: Two examples of Zero-shot prompting in the QA format (Santurkar et al., 2023). Subpopulation’s information (colored in pink) is concatenated with survey question (colored in sky blue). The first-token log-probability (probabilities assigned to A, B, C, ...) are used to calculate language model’s response distribution. The same steering prompt format is used in our fine-tuning experiment.

Figure 15: Few-shot prompting example for \( k = 3 \). Group information is presented in the beginning of the prompt (colored in pink). Following group information, \( k \) questions whose text embedding are the most similar to the text embedding of the evaluation question (colored in sky blue) are presented along with their opinion distribution. \( k \) questions are presented in the ascending order of cosine similarity. The generation of language model (verbalization of opinion distribution) is parsed to obtain the response distribution.
In terms of political party, you are Republican. Please respond to the following question with explanation.

Question: Please indicate whether the following is a major reason, a minor reason, or not a reason why you own a gun. As part of a gun collection
A. Major reason
B. Minor reason
C. Not a reason

Answer:

Which of the following comments best reflect the people of Republican in terms of political party?
Comment 1: (generation from model 1)
Comment 2: (generation from model 2)
Comment 3: (generation from model 3)
Comment 4: (generation from model 4)
Comment 5: (generation from model 5)
Comment 6: (generation from model 6)

Please select one comment number from 1 to 6.

In terms of political party, you are Republican. Please respond to the following question with the help of a passage.

Passage: (selected generation)

Question: Please indicate whether the following is a major reason, a minor reason, or not a reason why you own a gun. As part of a gun collection
A. Major reason
B. Minor reason
C. Not a reason

Answer:

Figure 16: Pipeline example of Modular Pluralism. Given a demographic group and a survey question, the first prompt is asked to multiple (6) language models, Mistral-7B-v0.1-Instruct fine-tuned on the community text corpus. The generations are sent to a black-box LLM (gpt-3.5-0613-Instruct) in the format of the second prompt. The black-box LLM answers which one of generations best reflects the given demographics. Finally, the selected generation serves as a context to answer the given survey question and the black-box LLM is prompted (the third prompt) to generate response distribution over the answer token A, B, C, etc.

Reviewer paper summaries: This paper proposes a training framework to simulate the LLMs with public opinions using survey data. It fine-tunes the models based on the original distributions of survey responses. For training and evaluation, the authors curated a new dataset from different survey sources. Results show that fine-tuning on response distributions effectively improves the match between model outputs and original survey responses.
Summary of strengths: This paper produces a newly and well-collected dataset comprising more survey questions from real survey participants. If the initial survey organizations permit republication of the data, this dataset could be valuable for further research on simulating public opinion. Overall, the experiments conducted on this dataset are methodologically sound, and most of the experimental settings are appropriate.
Summary of weaknesses: The methodology of this paper shares almost the same details with a recent paper (https://arxiv.org/abs/2502.07068), which might make the current submission less exciting. The biggest difference between the two papers is the data used: the current submission focuses on the single country level, ie. US-based data and includes different social demographics variables; the existing paper focused on the global level, ie. using different nations as demographic variables. 

For additional concerns regarding weaknesses or areas requiring clarification, please refer to the comments.
Comments, suggestions and typos: 1. The description of the data split for training and evaluation is unclear. The main text states (lines 316–317): "For training, SubPOP comprises 3,229 multiple-choice questions drawn from ATP waves 61–132." Later, on lines 320–323, it states: "For evaluation, SubPOP includes multiple-choice questions from the General Social Survey (GSS) (Davern et al., 2024), serving as an out-of-distribution benchmark." Does this mean that the models are trained $only$ on the 3,229 multiple-choice questions, and evaluation is conducted only on the GSS questions, making it an exclusively out-of-distribution setting? If so, why do the authors focus only on out-of-distribution evaluation while omitting an in-distribution evaluation, which seems necessary? If my interpretation is incorrect, could you please clarify the data split and training details more explicitly?

2. One question on the prompt format. In the baselines, you applied the three different formats: QA, BIO, and PORTRAY. However, in your main experiment, did you use only the QA format, as shown in Figure 2?

3. The few-shot approach (lines 395–404) appears to produce results differently from the main experiment. In this case, you directly prompt the model to generate a response distribution in a question-answer format. If I understand correctly, in your main methodology, the models are trained based on the response distribution of tokens (A, B, C, D, E) and then evaluated by extracting token distributions. This discrepancy raises some concerns. Additionally, did you examine the token distribution in the zero-shot setting?

4. It is unclear which demographic variables were selected for training, as this is not well introduced in the experimental setup. Based on Table 2, it seems that models were trained only on the demographic groups highlighted in blue: is this correct? If so, is there a specific reason for selecting these variables? Additionally, did you conduct experiments at the single-demographic level, meaning no combinations of demographic variables were considered?

5. Minor concern: The wording of the results on lines 508–511 is misleading: "With the exception of two of the smallest groups (Hindu and Muslim), all subgroups demonstrate a large and consistent relative improvement after fine-tuning, ranging from 40%–54%." However, based on Figure 3, the performance improvement for Hindu (39%) and Muslim (38%) is still quite strong and comparable to other groups such as Black (41%), Female (42%), and Moderate (42%). Therefore, they should not be considered as exceptions.
Overall assessment scores: 3
Metareview score: 4
Metareview: This paper present an interesting method about fine-tuning LLMs on option probability distributiuons and simulating to the unseen surveys and subpopulations. To do so, this paper also presents a curated dataset named SubPOP. Experimental results demonstrate the effectiveness of the proposed method, even on unseen surveys.
Summary of reasons to publish: This paper is well-written, well-structured, and addresses timely and important questions relevant to the intersection of social science and LLMs. Specifically:

* The proposed method is promising and contributes meaningfully to the field.
* The curated dataset is useful for future researchers and can be easily adapt to different survey scenrios.
* Experimental results convincingly demonstrate the effectiveness of the proposed approach.