Aligning Language Models to User Opinions
Authors: EunJeong Hwang
Bodhisattwa Prasad Majumder
Niket Tandon
Venue: EMNLP 2023 Findings (2023)
Keywords: personalization
large language model

Abstract: An important aspect of developing LLMs that interact with humans is to align models' behavior to their users. It is possible to prompt an LLM into behaving as a certain persona, especially a user group or ideological persona the model captured during its pertaining stage. But, how to best align an LLM with a specific user and not a demographic or ideological group remains an open question. Mining public opinion surveys (by PEW research), we find that the opinions of a user and their demographics and ideologies are not mutual predictors. We use this insight to align LLMs by modeling relevant past user opinions in addition to user demographics and ideology, achieving up to 7 points accuracy gains in predicting public opinions from survey questions across a broad set of topics. Our work opens up the research avenues to bring user opinions as an important ingredient in aligning language models.

Aligning Language Models to User Opinions
Eun.Jeong Hwang 2,2, Bodhisattwa Prasad Majumder*; and Niket Tandon #3
University of British Columbia
Vector Institute for Al
AAleen Institute of AI
ejhwangecs. wbc. ca, bodhi sat twam@allenai org, nikett@allenai.org
*contributes equally
Abstract
Shared Demographics: Female, Asian, Polticall Indlspendent
Opinions from User 1:
An important aspect of developing LLMs that interact with humans is to align models' behav- ior to their users. It is possible to prompt an LLM into behaving as a certain persona, espe- cially user group ideological persona the model captured during its pertaining stage. But, how best align an LLM with specific user and not demographic or ideological group remains an open question. Mining public opin- ion surveys (by PEW research), we find that the opinions of user and their demographics and ideologies are not mutual predictors. We use this insight to align LLMs by modeling relevant past user opinions in addition to user demographics and ideology, achieving up to points accuracy gains in predicting public opin- ions from survey questions across broad set topics'. Our work opens up the research avenues to bring opinions important ingredient in aligning language models.
Gun violence in games contributes a fair amount to gun vielence. never visit websites about guns, hurring shooting sports.
Opinions
Gun violence games contributes not too much to violence. sometimes
about guns. hunting shooling sports.
Alignment Question:
Thinking about gun owners who have children their home, how important is for them to keep all of their guns unloaded? (A) Essential, (B) Important but not essential, (C) Not important
Model
LLM
Predictions
User (B) Importent but not csscntial
Usar 2: (8) Important but not essential
LLM demographics:
User 1: (8) Important but not essential
X
Uierr 2: (B) Important but essential
demographics past opinions:
User 1: (A) Fasential
User 2: (3) Important but not essential
Figure Opinions vary when have the same demographic traits.
1 Introduction
2023). However, the extent to which these ap- proaches align language models with users remains unelear due to the subjective nature of defining user personas. Users have nuanced opinions that can change over time and vary depending on con- text. While alignment with normalized user groups like religion or political inclination may be easier, LLMs continue to struggle to align with individual users or the long tail of user groups. Additionally, LLMs tend to form opinions based on their pre- training data and feedback collected from crowd workers and model designers. As result, they exhibit low steerability, even with user groups that have major representation (Santurkar et al., 2023). Aligning LLMs to individual and taig-tail opin- ions has received less attention, while mostly fo- cusing on aligning user groups. In our analysis over PEW surveys, found that people can share all of their demographic traits but still exhibit a large variance in their opinions, rendering the cur- rent group-based LLM alignment insufficient. This
Personality is a defining feature of human beings, shaped by complex interplay of demographic characteristics, moral principles, and social expe- riences (Weil, 1957; McLellan, 1989). In turn, a person's 's personality has a significant influence on their ability to make decisions (Lauriola and Levin, 2001; Busic-Sontie et al., 2017). Owing to the wide-scale adaptation of the large language models (LLMs) for assisting individuals in their Docision-making process (Jiang et al., 2021: Gao al., 2023a), becomes increasingly critical to en- sure that these models are aligned with the unique personalities of their users.
With lower barriers to entry, several recent works focused on prompting LLMs with persona role- based prompts such Pretend you are Democrat (Deshpande al., 2023; Santurkar et al.,
Project page: https: / gith wwt/c//o/////ecttttcccaacaaaa wwt.///ewwwaanngeeerslliieeeeeee

---

paper investigates the relationship between demo- graphic traits and individual opinions in LLM align- ment. Specifically, we seek to answer the following research question:
can be determined by the region user belongs, their age determines whom they socialize with on a regular basis. However, this runs with the risk of stereotyping (i.e., an old individual is less likely to mix with younger people or they are conserva- tive in thinking). We later show that demographic information is not enough to model an individual.
What do we need to align an LLM to user: demographic traits, fine-grained opinions, or both?
2.2 Ideology
The majority of the past work in NLP literature focused on aligning LLMs with normalized user groups (Santurkar et al., 2023; Majumder al.. 2019; Salemi et al., 2023). In social science stud- ies, however, has been shown that all users are unique even they belong to the same broader user group, and normalizing user groups is not a true representative of a user's opinion (Chu et al., 2023; Kim and Lee, 2023). We apply insights from these social science studies to empirical setting where we try to model individuals' opinions based on their various persona information such as demographic traits, ideological inclinations, and past opinions.
Ideology is formed by an individual understanding of politics and economies. In our dataset, we have each subject's political affiliation and inclinations toward well-known political ideologies (e.g., con- servative, liberal). We use this information as an individual's idcology.
2.3 Opinions
OpinionQA uses a well-established method of cap- turing human opinions from public opinion surveys. In these surveys, subjects are asked to answer sub- jective questions that reflect their unique opinions and what makes them different from other individ- uals. Figure/ shows example of opinions that user provided during a survey.
In this paper. we give a thorough analysis of public survey responses in the OpinionQA dataset (Santurkar et al., 2023) with respect to their de- mographics, ideology, and implicit opinions and present comprehensive experimental results using the GPT-3 model with various combinations of in- puts (i.e., demographic, ideology, user's past opin- ions). Through our dataset analysis, we found that users' opinions and demographics do not neces- sarily correlate with each other. Our experimental results show incorporating both user opinions, de- mographics, and ideology, results in significant gains of up to 7 points in QA accuracy for certain topics, and utilizing the most relevant past opin- ions helps the model pinpoint the more accurate answers for the users.
2.4 Deriving insights from public surveys
We derive insights from the OpinionQA dataset, where we analyze the degree of agreement in user's opinions where they same demographics and how this agreement varies across topics. This statistical analysis generates useful insights that we later use for our modeling approach. We also look for sim- ilar (dis)agreements in opinions when users have the same ideologies.
Opinions differ despite same demographies We first take all pairs of users sharing the same demographics and compare their opinions. To calculate the agreement score between users, we utilize Cohen's kappa coefficient (Cohen, 1960). which ranges from - 1 to 1. Even though two users share the same demographics, agreement scores on the implicit opinions are gathered around 0.5 (Figure 2). This shows that solely relying on demo- graphic information is not enough to personalize the model, and users' implicit opinions can play a critical role in personalization.
2 What makes a persona?
We present study on various components that makes personality (in short, persona) of a user. We use the OpinionQA dataset, which contains 15 topics, and each topic contains an average of 100 questions and 5K users (Santurkar et al., 2023).
2.1 Demographics
The dataset records eight demographic information of user: region, sex, age. education, race, citizen, marital status, and income. These are the markers of social experience that user is most likely to go through. For example, the social experience
Opinions differ across topics In Figure we also show the topic-wise agreement scores. On certain topics, including Family & Relationships and Guns, users exhibit relatively higher agreement

---

scores. On the other hand, for some topics, includ- ing Race and America in 2050, users have lower agreement scores, indicating that certain topics may have larger variability in terms of user opinions. We later analyze this variability appears in a model's predictive performance when it is used to predict user opinions across different topics.
as highlighted in Table While the percentage of user pairs with shared opinions is higher com- pared to other topics, the percentage of user pairs with differing ideologies within these pairs is no- tably higher than the percentage of user pairs with similar idcologies.
Based on the insights derived above, we incorpo- rate them in our modeling approaches and analyze if these translate to the predictive performance of a model when used to predict user opinions as col- lected from the surveys.
Topic Agreement Score
Political views
Global attitudes
Economic inequality
Family
3 Aligning LLMs with persona
Privacy
Misinformation
In this section, we detail our task, possible mod- eling approaches, and evaluation protocols in Sec- tion 3.1 and discuss how select the most relevant past opinions of user in Section 3.3.
Race
Trust in science
America in 2050
Gender-eadership
Biomed-food issues
Sexual harassment
Views on gender
3.1 Setup
Automation vehicles
Task We use LLMs to model a user; how- ever, to concretely measure the performance, we use a simple question-answering (QA) setup. For QA task, we use existing questions from the surveys and try to predict the choice from multiple-choice originally given the subjects. We use a prompting-based zero- shot approach to perform the multiple-choice QA. We experiment with gpt-4 (GPT-4), text-davinci-O00 (GPT-3), gpt-3.5-turbe (GPT-3.5), Vicuna-13b, LLaMA-7b as the LLMs. Vicuna-13b (Chiang et al., 2023) is large language model which was built upon LLaMA- 13b (Touvron et al., 2023) and LLaMA-7b/13b is a transformer-based language model trained on tril- lions of tokens exclusively sourced from publicly available data. Vicuna- 13b performs on par with ChatGPT (Chiang et al., 2023).
Guns
0.0
0.1
0.2
0.3
0.4
0.5
Cohen kappa scores
Figure 2: Topic-wie wise agreement score; x-axis: agree- ment score, y-axis: topic. This graph shows that users with similar demographics/ ideology can have different opinions (cohen kappa scores of around 0.4 show not some but not substantial correlation in opinions)
Opinions differ despite same ideology To ana- lyze the correlation between user opinions and their ideology, we extract user pairs that two users who answered at lcast more than 10 common questions and compare their opinions and political ideologies. Table shows the percentage of user pairs shar- ing similar opinions, where 70% of opinions are matched between two users, and the percentages of the same ideologies and different ideologies within those user pairs. We observe that even though the users have similar opinions, around 80% of the user pairs have different ideologies. In contrast, we observe the percentage of sharing similar opinions among the users having similar ideologies is rela- tively higher than the percentage of sharing similar ideologies among the users having similar opin- ions in Appendix A. This implies that while having similar opinions does not necessarily imply shared ideologies among users, the presence of similar ideologies may suggest that users are more likely to have similar opinions. We particularly notice this phenomenon on the Guns and Family topics,
Modeling Approaches We sample 100 users per topic. 20% of implicit questions belonging the specific user used the user's implicit persona, and the rest are used test the model's personal- ization ability. We have the following variants of our model where the model is gradually exposed to different levels of user information: demographic, ideological information, and user past opinions. Here is rough sketch of what a prompt would contain for each modeling variation:
no persona: this is a case where default LLM opinion is evaluated w.r.to the individual's opinion (Santurkar et al., 2023).

---

Guns
Auto
Gender
Sex. harass.
Biomed-
Leadership
2050 US
Trust- Science
food
Similar op. user pair
45
13
30
12
11
37
23
21
Similar op. & ideol.
19
18
21
30
19
24
20
20
Similar op. & diff. ideol.
81
82
79
70
81
76
80
80
Race
Misinfo.
Privacy
Family
Bcon.
Glohal Attitudes
Politics
Inequal.
Similar op. user pair
29
21
43
Similar op. & ideol.
30
20
17
19
25
33
40
Similar op. & diff. idcol.
70
83
75
60
Table 1: Percentage of user pairs sharing similar opinions (similar op. user pair) and the percentages of similar ideologies (similar op. ideol.) and different ideologies (similar op. diff. ideol.) within user pairs sharing similar opinions. (Auto: Automation, Sex. harass.: Sexual harassment, Biomed-food: Biomedical food, Misinfo.: Misinformation, Econ. Inequal.: Economic Inequality, Politics: Political Views)
ideology: here, observe if ideological inclina- tions from the user help the model to align better to them (Santurkar al., 2023).
and model opinion distributions. The difference is calculated using I-Wassaerstein distance (WD), which measures the minimum cost for transforming one distribution the other:
ideology + demographics: here, we observe both demographic information and ideological in- clinations from the user help the model to align better with them (Santurkar et al., 2023).
WD(Dm (m), Pa(4))
A(Dm. Dri = Q
Σ QEQ
E
N -
ideology + opinions: we combine ideological in- clinations and opinions and measure if these help the model to align better with an individual.
where N the number of answer choices exclud- ing refusal, and the normalization factor N is the maximum WD between two answer choice distributions. The final score can be interpreted as how well the model distribution aligns with the human distribution.
demographic + idcology + opinions: when we combine all possible personal information, i.e.. demographic, ideology, and opinions, and mea- sure if these help the model to align better with an individual. See Figure 5 for the complete prompt.
3.3 LLM mimicking a person with ideologies, demographic, and opinions
3.2 Evaluation Metric
For accuracy evaluation, we utilize two types of accuracy measures, overall accuracy and collapsed accuracy. For overall accuracy, we simply calcu- late the accuracy of the predicted answer choice with respect to the gold answer choice from the dataset. We also present collapsed accuracy be- cause most answer choices in the opinion QA dataset have around 3 to 4 classes. In cases where there are more than 4 classes, it possible to fur- ther group the classes into super classes without losing substantial finer information. For exam- ple, the following answer choices: [Very likely, Somewhat likely, Not too likely, Not at all likely]. can be grouped into [Likely, Unl Unlikely]. We consolidate such answer choices into two classes, referred to collapsed accuracy, and present the results accordingly.
Our main goal to use different components of a user's persona (demographics, idcology, opin- ions) to align an LLM with an individual. Specif- ically, by having two experiments, one with past opinions+ideology and the other with past opin- lons+ideology
we aim to analyze the role of demographics when predicting user sponses. In addition, we hypothesize that giving users' past opinions may offer useful insights into their perspective (followed from Table 1), and LLM can benefit from that information when predict- ing the future answer for the specific user. When adding the user's past opinions, we compare the model with all opinions (maximum 16) to the model with top-k opinions (k is hyperparameter 3, 5, 8). The top-k opinions are obtained by com- paring the embedding similarity between the user's previous opinions and the question at hand, where we employ text-embedding-adee ada-002 to obtain the embeddings. We hypothesize that all opinions
We follow Santurkar et al. (2023) for the opinion alignment score evaluation. For the alignment scores, we calculate the difference between user

---

Exact match
Collapsed match
Model
L-7b
V-13b
GPT-3.5
GPT-3
GPT-4
L-7b
V-13b
GPT-3.5
GPT-3
GPT-4
No Persona
0.33
0.36
0.37
0.43
0.53
0.60
0.62
0.63
0.62
0.68
Demo.+Ides
0.35
0.39
0.47
0,47
0.54
0.62
0.62
0.66
0.65
0.70
Demo. Met.:lldoo+Opiiooo..
0.37
0.41
0.50
0.51
0.58
0.61
0.62
0.69
0.69
0.73
Opinionop3
0.35
0.42
0.50
0.51
0.55
0.61
0.62
0.67
0.67
0.71
Opinionaps
0.36
0,42
0,50
0.52
0.56
0.63
0.63
0.68
0.68
0.72
Ideo.+Opinionues
0.36
0.43
0.51
0.53
0.57
0.62
0.64
0.69
0.69
0.73
+Opinionwps
0.37
0.43
0,50
0.53
0.57
0.61
0.63
0.69
0.69
0.73
Demo.+Ideo, OOiinionn
0.35
0.42
0.51
0.53
0.58
0.61
0.63
0.70
0.73
Como.+ldeo.+Oiiniien 0.37
0.43
0,51
0.54
0.58
0.61
0.63
0.70
0.70
0.74
Table 2: Overall QA accuracy. For statistical significance, all models scored =0.01, which are computed using Wilson score intervals for a= 99%. GPT-4 was tested over 50% the testset due to cost, but we found the performance with GPT-3 with the subset remains similar to the original testset by having < 0.004 standard deviation. (L-7b: LLaMA-7b, V-13b: Vicuna-13b)
any persona, indicating that some questions might be highly correlated with the user's demograph- accurately anticipate its future response for the user. We show complete prompt that uses all available past information of individuals to predict their fu- ture opinions and all prompts for other modeling approaches in Appendix C.
may incorporate some unrelated viewpoints an- swer the question, and hence offering more perti- nent opinions would enhance the model's ability to
ics, and LLM is able to make a guess with the demographic information. Incorporating the user's previous opinions, up to 16 in total, along with demographic information, substantially enhances the performance in both overall and collapsed ac- curacy across all models. This implies that users' past opinions are indeed important to make correct predictions.
4 Results and Analysis
Here, we first analyze our model variants (Sec- tion 4) to validate hypotheses that we gather from analyzing the dataset (in Section 2.4). We also pro- similar
Interestingly. utilizing the top-k most relevant previous opinions does not yield a significant in- crease in collapsed accuracy. However, it does improve the exact match accuracy by up 3 points with GPT-3 when using both demographics and idcology along with the user's previous opinions. This implies that having top-k most relevant past opinions can help the model pinpoint more accurate answers, and providing the user's past opinions is already pushing the model to be in the correct range of the answer choices. We noticed that utilizing the top-3 opinions yields similar performance to us- ing the top-8 opinions, indicating that a few of the most relevant opinions carry the most performance improvement of the model.
vide model's performance when modeling setup to predict group-level opinions in Sections 4.2 and 4.3.
4.1
LLM for an individual
Here we discuss results of using an LLM to model an individual in the light of the evaluation metrics described in Section 4.1.
Exact match vs. Collapsed match The accuracy with the exact match and with the collapsed match in Table 2 shows a similar trend for the performance of our model variants. This suggests that leveraging implicit opinions enables the model to align with the correct range of answer choices, even though it does not precisely predict the exact same answer the user's choice.
Moreover, simply using the top most relevant opinions performs on par with the model with user demographic, ideology, and user's past 16 random opinions. This confirms again that utilizing the most relevant opinions feedback is essential to get personalized answers from LLM. Lastly, pro- viding additional demographic information with ideology slightly improves the model performance, implying that the demographic information may contribute valuable insights to the model to a cer- tain degree.
Overall Accuracy Table 2 presents overall QA accuracy with exact match and collapsed match for answer choices. In most cases, utilizing demographic+ieellgy------ opinions performs the best. Morcover, adding demographic and ide- ology information outperforms the model without

---

Model
GPT-3
GPT-3+CoT
Model
Opinion Alignment Score
Opinionaps
0.52
0.51
No Persona
0.670
Idco.+Opinionual
0.53
0.52
Demo.+lden.
0.763
Demo. #Opinoopis
0.53
0.52
Demo.+Ideo.s OOpiionn
0,780
Demo. MlldeoooOOiiinnoppp
0.54
0.53
Opinionup3
0.777
Opinionays
0.779
Table 3: QA accuracy comparison between GPT-3 and GPT-3 with Chain of Thought style prompt.
Ideo. Oppininniss
0.793
Demo.+ Oppnnonnsss
0.789
Demo. +Ieeo -ed.eee.Opppiii Opinionaes
0.796
Demo. www.::leee..OOiioo.....
0.795
Comparison between LLMs GPT-4 produces a similar performance when using all opinions and when using top-k relevant opinions. This implies that GPT-4 itself might be able to identify the most relevant user's s previous opinions.
Table 4: Opinion alignment scores ($3.2) with GPT-3 (text-davinci-003)
alignment scores (defined in $3.2) with various setups using GPT-3 model in Table 4. Overall scores exhibit similar trend as seen in Table 2, in which utilizing top-k most relevant opinions outperforms the model that uses all opinions. This is consistent with our finding that having top-k most relevant past opinions helps the model find more accurate answers for the user.
We find that the LLaMA-7b model does not un- derstand what produce when opinions added the prompt resulting in 18% cases where it does not yield any answer. Likewise, GPT-3.5 tends to produce no answer for questions that are about per- sonal opinions (c.g. GPT-3.5 with top-3 opinions produces 59% no-answers). When we prompted the model to generate an answer without an ex- planation by changing the suffix of the prompt: "Answer choice:" + "Answer choice wi without explanation:" ", and substantially fewer (less than 1%) cases have no-answers.
Topic-wise Accuracy Figure 3 demonstrates the model's 's accuracy across different topics with vari- ous setups, measured by the cxact match for answer choices. The model with demographic and implicit opinions particularly achieves higher scores the Biomedical-fode and Guns topies, implying that these two topics may lead the users to have simi- lar opinions of each other. contrast, the model exhibits slightly decreased performance when in- corporating implicit opinions on topic Automation. This suggests that the LLM can make accurate pre- dictions up to some extent based on user demo- graphic and ideology information. However, in- corporating implicit opinions, which may include viewpoints not aligned with users' demographic ideologies, can potentially confuse the model in its prediction process.
The trend of the performance where we add top- : opinions remains the same for both GPT-3 and GPT-3.5. There no clear winner between dif- ferent versions of GPT as also found in the other literature (Fu et al., 2023).
Chain of Thought (CoT) Prompting CoT (Wei al.. 2023) has shown that encouraging LLMs explain their reasoning step-by-step improves the model performance. To see whether CoT also benefits in personalizing answers, we test the GPT- 3 model with CoT-style prompting by changing the suffix of our prompt from "Answer choice:" to "Let's think step by step and choose answer choice:" and present the results in Table 3. We discover that using the CoT prompt consistently decreases the performance by point. This is because model attempts to provide the reason for the potentially irrelevant information presented the prompt, suggesting its inability of selecting the most relevant information about the answer. E.g., in Figure 4, the model generates reasoning exclusively based on demographic and ideology information but ignores user opinions.
Common Errors In Table 5, manually an- alyzed 30 randomly sampled opinions where GPT-3 produces correct answers with demo- graphic+ideology information but incorrect an- swers when opinions are added. The model mostly confuses when there a high overlap between user opinions and some answer (possibly wrong) choices. For question: "How well do the follow- ing phrases describe you?" and correct answer: "Describes well," the model often predicts cor- rectly purely based demographic information (e.g., Black). However, when the user's past opin- ions were added and since it includes the phrase "supporter of rights for LGBT people do not de-
Opinion Alignment Scores Among high- performing models, since only GPT-3 provides prediction probabilities, we compare opinion

---

como ridea tasp
Accurayy with cloct
24-
81
View
food
Leadership
2050 US
Trust sclenc
Race
Econ, Inequal
Glabal Atddues
Sex.
Figure 3: Overall topic-wise accuracy based on the exact match for answer choices, (demo.: demographic, ideo.: idcology, op.: opinions), GPT-3 model is used for the experiment. See Appendix for the scores.
Opinion contains
Percentage (%)
Question: How much, at all, do you worry about the following happening to you? Being victim of terrorist attack
Word-overlap with answer choice
36
Irrelevant information
30
Relevant information
30
Choices: Worry a lot, Worry a little, Do not worry at
Table 5: Types of common errors with their percentages when prompting with opinions confuses the model.
Opinions:
I never used gun to threaten or intimidate anyone in family.
There were no guns in my household when I was growing up.
scribe me well," it predicts a wrong answer choice: "does not describe me well" just because the direct phrase overlap. While models often ignore addi- tional relevant information and commit errors, it is also sensitive irrelevant opinions (even in top-k) when additionally added with demographic infor- mation. For e.g., while predicting user opinion on the U.S. remaining as a world superpower in 2050, the user's past opinion about violence against Jews (retrieved in top-8) confuses the model leading to an incorrect prediction. This also explains why just adding top-3 opinions often archives the high- est accuracy instead of adding all opinions. See Appendix D for examples of all types of errors.
Demographics and Ideology:
Political party: Republican, Political ideology: Con- servative, Region: South, Marital status: Married, Education: College graduate/some postgrad
CoT reasoning:
The person is conservative and Republican, they may he more likely worry about terrorist attack. Step 2: The person is married and has college degree, they may he more aware of the potential risks of terrorist attack.
Step I:
Step 3: The person in the South, they may likely to worry about temorist attack due to the region's history of terrorism.
Conelusion: The person is likely select answer choice A: Worry lot.
User- Aser-answer (expected); not worry at all Demo-tidco.+opinion with CoT: Warry lot Demo-tideo-+opinoon CoT: Do not worry
4.2 LLM with majority answer choices
Additionally, we also wanted to understand if simi- lar performances can be achieved if we model an individual member of a (sub-) poopllppoulaionn mir- roring (Santurkar et al., 2023). While it is true that opinions are necessary predict individuul-level preferences, here we investigate if demographic information sufficient for predicting population- level opinions or not. For this, we first merge our QA data points using a particular ideological group value (e.g., democrat) and obtain the answer choice that is chosen by most of the group members (i.e., majority vote) and treat that answer as the gold answer for the question while calculating the accu- racy (Kim and Lee, 2023).
Figure example incorrect CoT reasoning con- fusing the model. As see in CoT reasoning, the model reasons about demographic information, not opin- ions provided.
tive, i.e., a person having a majority vote answers belonging to a specific group (See Appendix C for the prompt). We see that the LLM is good at predicting the answer given by the majority of the group member belonging to certain ideology, suggesting that LLMs are good modeling rep- resentative individual of a cobb-pouultioo (e.g.. all democrats). The overall performance without idcol- ogy information is 0.597 (with exact answer choice
We prompt our model to predict an answer given a question assuming the role group representa-

---

Exact match
Collapsed match
(e.g., demographic traits vs. past opinions) influ- ences how well an LLM can model individual's opinions. For a comprehensive comparison among all previous work, see Table 9.
Majority answer
0.597
0.674
Independent 0.546
0.674
Democrat
0.578
0.665
Republican 0.523
0.639
Role of demographies and ideology There have been several studies investigating the correlation between ideological attitudes and psychological traits (Zmigrod et al., 2021: Crockett and Wallen- dorf, 2004; Chan and Palmeira, 2021). Crockett and Wallendorf (2004) found that normative politi- cal ideology is central understanding shopping as a manifestation of social and political connec- tions. Chan and Palmeira (2021) found that the cognitive decision-making strategies of individuals reflect their ideological attitudes. Differently, we show that ideology is not the only important factor in predicting the user's opinion using an LLM.
Table 6: Performance with LLM with ideology informa- tion. GPT-3 model is used for the experiment.
match) and 0.674 (with collapsed answer choice match), as presented Table 6. This also indicated that the default opinions from the LLMs are some- what aligned with the majority opinions seen at population level.
4.3
LLM as person with an ideology
Next, we add the ideological information to see if this additional information can help the LLM per- form better model a user belonging to a group that believes in a specific idcology (c.g., conserva- tive) (See Appendix C for the prompt). We find that the LLM is moderately good at modeling a user with group-level information to predict the group-level majority opinion. This indicates that the additional ideological information is not par- ticularly helpful. The average performance with idcology information is 0.549 (with exact answer choice match) and 0.659 (with collapsed answer choice match), as shown in Table 6. We see a similar trend in results for modeling an individual with their demographics and/or ideology and/or past opinions since an individual's opinion does not align with the group's majority opinion that the person belongs
LLMs with retrieval- approach Exten- sive prior work has used retrievals from a text cor- pus to aid QA (Madaan et al., 2022; Pan et al., 2019), or retrievals of prior QA pairs for nearest- neighbor QA (Khandelwal et al., 2020). Madaan et al. (2022) uses memory of user opinions retrieve past relevant data points for the prompt. Khandelwal al. (2020) showed the effectiveness of the nearest neighbor search for language mod- eling by extending a pre-trained language model (LM). Differently from work on LLMs and group- level personalization, we show that LLMs can be tuned for individual users with their opinions.
Conclusion and Outlook
This paper offers a new insight that aligning LLMs to users is best done by modeling user demograph- ics, ideologies, and the most relevant past opinions. Large-scale experiments on PEW surveys present in the OpinionQA dataset show an approximately 7% absolute QA accuracy strong demography- based baselines. We proactively offer suggestions to avoid personalized LLMs from becoming echo chambers (see Ethics Statement). An exciting fu- ture direction is to continuously store user opinions and grow the memory of opinions.
5 Related work
Personalization Past works that focused on mod- eling individual users were from the Pre-LLMs and mainly hail from the recommender systems lit- erature (Gao al., 2023b; He et al., 2017; Li et al., 2021; Majumder et al., 2019). However, these sys- tems were trained on domain-specific annotated datasets or using latent information about the users (e.g. their previously written reviews). The recent LLMS have seen less content from the long tail of user groups during their pre-training phase, and there has been a lack of large-scale datasets of in- dividual opinions until recently (Santurkar al., 2023). Thus remains an open problem whether LLMs can be aligned effectively with individual user persona and how different user information
An aligned LLM offers the benefit to offer per- sonalized perspectives that align with user's val- ues and cultural beliefs. However, there exist cir- cumstances when LLMs can become an amplifier for unethical and biased views. Our work lays the foundation for a robust LLM alignment ap- proach. By using memory- based personalization

---

and recording interactions saved in growing mem- ory, the model can inform future instances of the most relevant past opinions.
Acknowledgements
We thank the members of the Aristo team at A12 and Kurt Gray for their insightful feedback on this work. EH was funded, in part, by the Vector In- stitute for AI, Canada CIFAR Al Chairs program, an NSERC discovery grant, and research gift from A12. BPM was funded, in part, by an Adobe Research Fellowship.
Limitations
Non-subjective questions. For non subjeccte questions, such as "How many years have you lived in in.". it might not be necessary past opinions (see error analysis).
References
Lack of user information. In this work, we fo- cus on all past opinions on the same topic due to lack of availability of user information in the dataset. More insights can be derived by investi- gating users' opinions on multiple different related topies. We leave as our future work.
Ante Busic-Sontic, Natalia V Czap, and Franz Fuerst. 2017. The role of personality traits in green decision- making. Journal of Economic Psychology, 62:313- 328.
Fugene Y. Chan and Mauricio Palmeira. 2021. Politi- cal ideology moderates consumer response to brand crisis apologies for data breaches. Computers in Human Behavior, 121:106801.
Lack of temporal information. Another missing aspect is time. User opinions change over time, does not contain
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng. Zhanghao Wu, Hao Zhang, Lianmin Zheng. Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez. lon Stoica, and Eric P. Xing. 2023. Vicuna: An open- source chatbot impressing gpt-4 with 90% chatgpt quality.
but the dataset available timestamps. would be interesting to see how conflicting user opinions can be modeled, perhaps by biasing toward the most recent opinion.
Eric Chu, Jacob Andreas, Stephen Ansolabehere, and Deb Roy. 2023, Language models trained on media diets predict public opinion.
Ethics Statement
Data The dataset used in our work, OpinionQA (Santurkar et al., 2023) is publicly available. The dataset includes subjective opinions from humans and may contain offensive content some people.
Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20:37 46.
David Crockett and Melanie Wallendorf. 2004. The role of normative political ideology in consumer behavior. Journal of Consumer Research, 31:511-528.
Models The large language models we used for the experiments are trained on large-scale web corpus and some of them utilize human feedback. This may also bring some bias when predicting user answers. With an aligned LLM, users can select information that adheres to their system of beliefs and to amplify potentially biased and unethical views. Such an echo chamber (Del Vicario et al., 2016) can eventually cause harm by reinforcing undesirable polarized user's views.
Michela Del Vicario, Gianna Vivaldo, Alessandro Bessi, Fabiana Zollo, Antonio Scala, Guido Caldarelli, and Walter Quattrociocchi, 2016. Echo chambers: Emo- tional contagion and group polarization on facebook. Scientific reports, 6(1):37825.
Ameet Deshpande, Vishvak Murahari, Tanmay Rajpuro- hit, Ashwin Kalyan, and Karthik Narasimhan. 2023. Toxicity chatgpt: Analyzing persona-assigned lan- guage models. arXiv preprint arXiv:2304.05335.
A viable mitigation is to show user demography or ideology group answers in addition to the per- sonalized answer (e.g., showing how an average Democrat with similar demographics would think on this topic and why). Further, past opinions can be used to ground an explanation (e.g., the current personalized answer influenced by a user's spe- cific past opinion), thus offering an opportunity for the user to introspect their past opinions.
Yao Fu, Litu Ou, Mingyu Chen, Yuhao Wan, Hao-Chun Peng, and Tushar Khot. 2023. Chain-of-thouse hub: A continuous effort to measure large language mod- els' reasoning performance. ArXiv, abs/2305.1 17306.
Yunfan Gao, Tao Sheng. Youlin Xiang, Yun Xiong. Haofen Wang, and Jiawei Zhang. 2023a. Chat- rec: Towards interactive and explainable 1lms- augmented recommender system. arXiv preprint arXiv:2303.14522

---

Yunfan Gao, Tao Sheng, Youlin Xiang. Yun Xiong, Haofen Wang, and Jiawei Zhang. 2023b. Chat-rec: Towards interactive and explainable llms-augmented recommender system.
Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto, 2023. Whose opinions do language models reflect? ArXiv, Abs/2303.17545
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collabora- tive filtering.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez. Armand Joulin, Edouard Grave, and Guillaume Lample. 2023, Llama: Open and efficient foundation language models.
Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula, Ro- nan Le Bras, Maxwell Forbes, Jon Borchardt, Jenny Liang, Oren Etzioni, Maarten Sap, and Yejin Choi. 2021. Delphi: Towards machine ethics and norms. ArXiv, abs/2110.07574.
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. Chain-of-thought prompting elic- its reasoning in large language models.
Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2020. Generalization through memorization: Nearest neighbor language models. In International Conference Learning Representations.
Simone Weil. 1957. Berits Londres dernières lettres. Gallimard.
Leor Zmigrod, Ian Eisenberg, Patrick Bissett, Trevor Robbins, and Russell Poldrack. 2021. The cog- nitive and perceptual correlates ideological at- titudes: A data-driven approach. Philosophical Transactions of the Royal Society B: Biological Sciences, 376:20200424.
Junsol Kim and Byungkyu Lee. 2023. Ai-augmented surveys: Leveraging large language models for opin- ion prediction nationally representative surveys.
Marco Lauriola and Irwin P Levin. 2001. Personality traits and risky decision-making in a controlled ex- perimental task: An exploratory study. Personality and individual differences, 31(2):215 226.
Shuyang Li, Bodhisattwa Prasad Majumder, and Julian McAuley. 2021. Self-supervised bot play for conver- sational recommendation with justifications.
Aman Madaan, Niket Tandon, Peter Clark, and Yiming Yang. 2022. eeemory-assister prompt editing im- prove GPT-3 after deployment. In Proceedings of the 2022 Conference Empirical Methods in Natural
Language Processing, pages 2833-2861, Abu Dhabi, United Arab Emirates. Association for Computa- tional Linguistics.
Bodhisattwa Prasad Majumder, Shuyang Li. Jianmo
Ni. and Julian McAuley. 2019. Generating personalized recipes from historical user prefer- ences. In Proceedings of the 2019 Conference Empirical Methods Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNIP-IJCNIPP, pages 5976 5976-5982, Hong Kong. China. Association for Computational Linguistics.
David McLellan. 1989. pessimist. Springer.
Simone Weil: utopian
Xiaoman Pan, Kai Sun, Dian Yu, Jianshu Chen, Heng Ji, Claire Cardie, and Dong Yu. 2019. Improv- ing question answering with external knowledge. In Proceedings of the 2nd Workshop on Machine Reading for Question Answering. pages 27-37. Hong Kong, China. Association for Computational Linguistics.
Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. 2023, Lamp: When large lan- guage models meet personalization.

---

A -Similar ideologies and different opinions
A person be described follows:
Age: 30-49
We show the percentage of user pairs having similar ideologies and the percentages of user pairs having similar opinions and different opinions within the user pairs sharing similar ideologies in Table
Income: 75, 11000000-100 100, 000
Political ideology: Conservative
Political party: Republican Religion: Roman Catholic
B Topicwise QA Accuracy
person has the following opinions on Guns.
We show an overall topic-wise accuracy based on the exact match for answer choice in Table 8.
Opinions:
most important reason why I own gun for sport shooting, including target shooting and trap and skeet.
C Prompt
The ease with which people can illegally obtain guns contributes to gun violence in the country today.
We provide comprehensive display of all prompts used in the models incorporating user demograph- ics, ideology, and opinions, which were employed for individual-user level tests in Figure 5, 6 and 7. Additionally, we present the prompts utilized for experiments conducted the group-level tests in Figure and? 9.
Based on the above list of opinions and the demographic information, which answer choice will this person select for the question:
Question: Thinking about
owners who do
not have children in their home how important do you think it is for them to: Take gun safety courses
D Error Examples
Answer choices:
We give additional error examples in Figure 10, 11, and 12, where opinions either confuse the model with relevant information provide useful in- formation.
Essential
B. Important but not essential
C. Not important
Should not be done
Figure 10 shows the second common error, which when Relevant information in opinions confuses the model. For example, when the ques- tion is "Is what you know about dictitians because you have heard or read about this in the news" and the user's past opinions contain "I did not learn about dietitians in my job.". In this case, while the user's answer to the question is "Yes, have heard or read about this in the news", the model produces "No, have not heard or read about this in the news" because of the opinions provided.
Answer:
Figure 5: Prompt using demographics, ideology, and GPT embeddings based top-k past opinions predict the answer question.
A person has the following opinions Guns.
Opinions:
opinion1
opinion2
The third most common error is with Irrelevant information as shown in Figure Some opinions do not contain useful information at all to reason for the correct answer. Lastly, there are some ques- tions that opinions are not important to predict the answer (e.g. "About how many years have you lived in your local community?") presented. In this case, the model may make an arbitrary predic- tion.
Based the above list of apinions, which answer choice will this person select for the question:
Question: question
Answer choices:
A.< choice!
B. choice2
c. choice3
Additional related work comparison
Answer:
We show additional related works that can be com- pared paper in Table 9.
Figure 6: prompt for implicit-only model

---

Topic
similar ideol. user pair (%)
similar ideol. & op. (%)
similar ideol.-diff. op. (%)
Guns
16.05
52,67
47.33
Automation
15.99
15.26
84.74
Views on gender
16.26
39.58
60.42
Sexual harassment
16.95
22. 14
77.86
Biomedical, food
15.69
13.44
86.56
Gender, Leadership
17.52
50.25
49.75
America in 2050
15.34
30.21
69.79
Trust in Science
15.53
26.27
73.73
Race
16.32
21.05
78.95
Misinformation
16.22
34,83
65.17
Privacy, Surveillance
16.13
22.14
77.86
Family, Relationships
17.01
48.88
51.12
Economic inequality
16.24
39.25
60.75
Global attitudes
16.75
47.78
52,22
Political views
16.65
37.59
62.41
Table 7: Percentage of user pairs sharing similar ideologies (similar ideal. user pair) and the percentages of similar opinions (similar ideol. & op.) and different opinions (similar ideol.-diff. op.) within user pairs sharing similar idcologies.
Accuracy with exact match
No Persona
Demo. Idco. Demo. Idco.+ Opinionues
Guns Automation
0.40 1 0.08
0.51 + 0.13
0,63 1 0.14
0.44 + 0.10
0.49 = 0.12
0..8 + 0.09
Views gender
0.43 + 0.09
0.44 = 0.10
0.57 + 0.09
Sexual harassment
0.40 + 0.10
0.44 = 0.09
0.47 + ().09
Biomedical, food
0.51 + 0.12
0.55 0.12
0.60 = 0.10
Gender, Leadership
0.50 + 0.13
0.45 = 0.11
0.59 + 0.13
America in 2050
0.43 + 0.12
0.12
0,46 = 0.11
Trust in science
0.52 1 0.12
0.50 0.10
0.59 1 0.09
Race
0.38
0.12
0,51 1 0.10
Misinformation
0.48 1 0.12
0.12
0,54 1 0.12
Privacy, Surveillance
0.36 0.10
0.12
0,51 0.11
Family, Relationships
0.46 1: 0.10
0.49 0.12
0,57 1 0.12
Economic inequality
0.38 0.10
0.47
0.55 0.08
Global attitudes
0.38 0.12
0.15
0.48 0.10
Political views
0.41 0.13
0.51 0.11
0.52 0.10
Table 8: Overall topic-wise accuracy based on exact match and collapsed match for answer choices. (demo.: demographic, ideo.: ideology, op.: opinions). GPT-3 model is used for the experiment.
User-profile
Modeling individuals
Supervision-free
explicitly observed
Personalized generation: OpinionQA (Santurkar et al.. 2023), RecipeGen (Majumder et al., 2019),
X or / (mostly group)
X or of
LAMP (Salemi et al., 2023)
Recommender Systems: ChatRec (Gao et al., or /
X or / (mostly supervised)
2023b), Collaborative Filtering (He et al., 2017), (mostly latent)
BotPlay (Li et al., 2021)
Ours
/ (+ group)
Table 9: Placement of our work w.r.t. related work

---

Political party: Republican, person can be described
follows:
Age: age
Education: education
Based the demographic information, which choice will this person select for the question:
Question:
question
Answer choices:
A. choicel
B.K choice2
Question:
C. choice3
Is what you know about dietitians because you have heard or read about this in the news
Answer:
Choices:
Yes, have heard or read about this in the news
Figure 7: prompt for demographic-one only model
No, have not heard or read about this in the news
Refused
Opinions:
I did not learn about dietitians in job.
Demographics and Ideology:
Race: White,
Thinking of yourself select the right choice.
Religion: Protestant,
please
User-answer (expected): Yes, have heard read about this in the
question
Choice:
[choicel, choice2, choice3]
Demo -ideology-opiniomee No, have heard read about this
Answer:
Demo.-ideoll -ideology: Yes, heard or read this news
Figure 8: prompt for group-level ideology test
Figure 10: An example of relevant opinion confusing the model.
Thinking yourself as a person, please select the right choice.
question
Choice: [choicel, choice2, choice3]
Answer:
Figure prompt for LLM majority answer test

---

Question:
How likely do you think it is that the following will happen in the next 30 years? China will over take the U.S. the world's main superpower
Question:
How well, if at all, do the following words or phrases describe you? Supporter of the Black Lives Matter movement
Choices:
Yes, have heard or read about this the
Choices:
No, heard read this the news Refused
Descr describes me well
Does not describe me well Refused
Opinions: (irrelevant) Population growth problem in 2850.
Opinions:
U.S. will a major probable that there will be increasing violence against Jews in the U.S. in the next 30 years.
The words phrases "supporter of rights for LGBT people" do not describe me well.
...
Demographics and Ideology: Political party: Democrat, Race: Black,
Demographics and Ideology:
Religion: Protestant,
Citizenship: Yes, Political party: Moderate, Race: White,
Region: Midwest,
User (expected): Describes well
Demo -ideology-opinions: Does well X
describe
User-answer (expected): Will probably not happen
Describes me Demo. -idoooide
Demo -ideology-opinioos:: will probably happen
- -ideology:
Will probably not happen /
Figure 12: An example of not relevant opinion confus- ing the model
Figure 11: An example opinions containing no useful information predict the answer.


Reviews:
[{"review_id": "rKizsNEqOi", "reviewer": "Reviewer_iqqj", "summary": "", "questions": "QA: Have you considered other factors or sources of user-specific information that could further enhance model alignment?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to review our paper and recognizing our contributions! We are happy that you appreciated our main points: 1) aligning language models with individual users by using demographics, ideology, and opinions, 2) thorough comparative analysis across different LLM variants, and 3) valuable insights into the impact of different factors on model alignment and performance.\nWe think that all your questions are addressable within this discussion period. We would love to address additional questions during the discussion period if anything is unclear.\n\n\n**Limited generalizability of the findings with the specific dataset used**\n\nThank you for raising this concern. OpinionQA covers 60 US demographic groups over 15 topics ranging from automation to politics. While it covers a large amount of the population, we agree that the insights may predominantly apply to the U.S. population. We hope there are more datasets across demographics that the community should work on and that can further validate the generalization of our results.\n\n**Other sources of user-specific information that could further enhance model alignment**\n\n\nThat’s a good point and we exploited all the user information present in the opinionQA dataset. While demographics and ideologies represent conventional features in psychology for modeling, we additionally use the dataset for extra information in the form of opinions and show that this is a very important signal. More detailed demographic features such as those present in small-scale user studies in psychology and cognitive science are helpful but need to be carefully engineered. However, we demonstrate that opinions which have been less exploited as a signal are helpful in predicting user responses and are a more powerful and important signal than demographics. On the other hand, synthetically adding auxiliary user information can lead to stereotyping and cannot be used as gold data.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper addresses a critical and novel question in the field of natural language processing (NLP) about how to best align language models with individual users, moving away from the traditional focus on demographic or ideological groups. This question is highly relevant given the increasing use of large language models in real-world applications.\n\n- The paper conducts a thorough comparative analysis of different LLM variants, including those with demographic information, ideology, and past opinions. This analysis provides valuable insights into the impact of different factors on model alignment and performance.\n\n- The paper provides detailed experimental results and analysis, allowing other researchers to reproduce and validate the findings. This can foster further research and comparisons of different alignment techniques in the NLP community.", "reasons_to_reject": "- The paper heavily relies on the OpinionQA dataset, which may not fully capture the diversity of user opinions and demographics. The generalizability of the findings could be limited to the specific dataset used.", "questions_to_authors": "QA: Have you considered other factors or sources of user-specific information that could further enhance model alignment?", "paper_topic_and_main_contributions": "The paper addresses the problem of aligning large language models (LLMs) with individual users' personalities and opinions. It aims to personalize LLMs to better understand and respond to individual users' preferences, instead of aligning with broad demographic or ideological groups. This paper also highlights the importance of considering users' past opinions as an essential ingredient in aligning language models. By using relevant past opinions, the model can better understand and respond to a user's unique perspective, leading to more accurate and personalized answers.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "B8f9CCROuk", "reviewer": "Reviewer_Qyo6", "summary": "", "questions": "1. Why is it necessary to align a decoder-based LLM to a user to address the opinion classification task? Can you provide insights into the performance of encoder-based LLMs, such as BERT and RoBERTa, when the primary goal is to predict user opinions based on demographic information and previous opinions? Furthermore, what other tasks can be achieved after aligning an LLM to a user? For instance, can personalized responses be generated for a given conversation using the suggested approach?\n\n2. The claim that \"human feedback is not necessarily helpful in personalizing the answers\" (lines 376 ~ 378) is hard to believe, especially with only the difference between the performance of GPT-3 and GPT-3.5. Could you clarify if this is the only difference between GPT-3 and GPT-3.5? It would be more convincing to present additional experiment results with other LLMs that do use human feedback and compare them to those that do not.\n", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to read our paper and pointing out important discussion points! We have made those more clear in the paper now. We are happy that you appreciated our main points: 1) well-founded prompts using demographics, ideology, and opinions, and 2) diverse experiments across different LLMs. We would love to address additional questions during the discussion period if anything is unclear.\n\n**What would be the performance of encoder-based LLMs, such as BERT and RoBERTa on this task?**\n\n\nThanks for pointing it out. We ran additional experiments with BERT model, one with demographic+ideology and the other with demographic+ideology+top8 opinions. The performance is as follows:\n\ndemographic+ideology: 0.508\n\ndemographic+ideology+top8 opinions: 0.523\n\nThe performance trends are similar to GPT3+ models. While the performance is almost comparable to GPT3, it requires substantial training with more than 10k points. Thus the main finding of our paper, in which using all information (demographic+ideology+opinion) is helpful in aligning the LLM to a user, still holds whether they are open source or closed-source. We also provide insights into how various user factors (such as demographics, opinions, and ideologies) influence the behavior of LLMs using a few shot demonstrations.\n\nExperimental setup details are: bert-base-uncased, 2 epochs, learning rate of 0.00001, batch_size of 8. Developing more advanced encoder-based model baselines for our task would be an interesting future direction.\n\n**What other tasks can be achieved after aligning an LLM to a user? (e.g. personalized response generation?)**\n\nYou are exactly right. We hope to generate personalized responses by augmenting our setup with a memory of user opinions. The idea of a memory-augmented architecture has been recently explored [1] -- where the memory grows with user interactions and feedback. For a future instance, relevant past memory items are retrieved (like we show in our strongest model “top-k opinions”). This avoids repeating mistakes where a user specifies their opinions and also aligns well with users. The personalization could then align with a user’s ethical principles, moral beliefs, and cultural-specific beliefs. This is an unsolved problem yet with LLMs and we believe that we offer the first insight into why modeling such opinions could be valuable and eventually lead to personalized LLMs.\n\n[1] Memory-assisted prompt editing to improve GPT-3 after deployment, Madaan et al., EMNLP 2022 (https://arxiv.org/abs/2201.06009)\n\n**GPT3 vs GPT3.5: Clarification on the intuition that human feedback is not necessarily helpful in personalizing the answers**\n\nYou are right. We checked the open ai model version we used again and we realized RLHF was applied to text-davinci-003 model. We missed this detail since the information was not explicitly mentioned on the website and based on this information we made the hypothesis; however, based on our corrected understanding, this claim is not really correct-- thanks for pointing it out. We will remove the sentence related to RLHF in one of the smaller analysis subsections in the camera-ready version. \n\nAs for the performance difference between GPT and GPT turbo -- There is no clear winner between different versions of GPT as also found in the literature elsewhere (See https://arxiv.org/abs/2305.17306). On BBH(BigBench Hard): Turbo 70.1, davinci003 70.7, davinci002 73.7 -- it is thus conceivable that for our task Turbo is slightly worse than GPT3.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper introduces an interesting and well-founded prompt method for predicting personalized opinion classes.\n- The research presents diverse experiment results involving different ideas and LLMs.\n", "reasons_to_reject": "- The task appears to resemble a classification task, so the paper should clarify why the focus is on decoder-based LLMs.\n- The claims made in the paper might be overstated, given the limited number of experimental results.\n", "questions_to_authors": "1. Why is it necessary to align a decoder-based LLM to a user to address the opinion classification task? Can you provide insights into the performance of encoder-based LLMs, such as BERT and RoBERTa, when the primary goal is to predict user opinions based on demographic information and previous opinions? Furthermore, what other tasks can be achieved after aligning an LLM to a user? For instance, can personalized responses be generated for a given conversation using the suggested approach?\n\n2. The claim that \"human feedback is not necessarily helpful in personalizing the answers\" (lines 376 ~ 378) is hard to believe, especially with only the difference between the performance of GPT-3 and GPT-3.5. Could you clarify if this is the only difference between GPT-3 and GPT-3.5? It would be more convincing to present additional experiment results with other LLMs that do use human feedback and compare them to those that do not.\n", "paper_topic_and_main_contributions": "The paper proposes a method to align personalized user opinions on large language models (LLMs). The authors leverage demographic information, ideology, selected previous opinions, and chain-of-thought prompting to achieve this alignment. They conduct experiments using the OpinionQA dataset, which consists of 5K users' answers to topical questions. Various pretrained LLMs are utilized to measure the accuracies of LLMs in providing appropriate responses based on the given persona and additional information as prompts. The experimental results demonstrate that using all the suggested features outperforms models that lack these ideas.", "excitement": 2, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZlJPExeNjf", "reviewer": "Reviewer_cFeH", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to read our paper! We are happy that you appreciated our main points: 1) investigating effects of different user factors and 2) solid analysis across different LLMs. We would love to address additional questions during the discussion period if you have more questions.\n\n**arxiv version (May 24) is after the anonymized date (May 23)**\n\nWe submitted the arxiv version before 11:59 pm UTC -12h (“anywhere on Earth”) on May 23rd, 2023 and the arxiv timezone does not follow AoE timezone. This is our submission time: Wed, 24 May 2023 09:11:11 UTC (AoE is UTC-12 -- and thus this is 23 May 2023, 9PM UTC -12h; which is roughly three hours before the deadline)", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This is very novel study that investigates effects of user factors.\n2. The analysis is solid.", "reasons_to_reject": "I rated highly for the submission quality, BUT according to the EMNLP policy, the arxiv version (May 24) is after the anonymized date (May 23). This should be rejected before the review.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This study investigates the effects of user factors on LLMs. The authors conducted experiments on a recent data, OpinionQA. They found that aligning LLMs by user opinions with user demographics and ideology can boost LLM performance. ", "excitement": 2, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes an approach to align large language models (LLMs) to individual users by modelling the user's demographics, ideology, and past opinions. The key idea is to go beyond just using demographics or ideology and to also leverage the user's previous stance on issues as a richer signal for personalization. The authors conduct experiments on the OpinionQA dataset using different LLM variants and show that modelling all factors - demographics, ideology, and past opinions - leads to accuracy gains in predicting user responses.

The reviewers raise important points about the generalizability of the findings beyond this specific dataset and whether other user factors could further enhance personalization. In their rebuttal, the authors provide satisfactory responses, conducting additional experiments with BERT to demonstrate consistent trends. The authors also thoughtfully discuss how their approach provides a scaffolding for generating personalized responses in future work, albeit requiring large-scale user studies.

Reviewer 2 suggests encoder-based models may be sufficient for the task framed as classification, while the authors clarify the benefits of studying decoder models and few-shot prompting. Reviewer 3 recommends rejection based on anonymization rules, which is an important issue that needs resolution.

Overall, this appears to be a novel contribution demonstrating the utility of modelling opinions in addition to standard user factors for alignment. Furthermore, the paper tackles an important problem of personalizing large language models (LLMs) to individual users. Aligning LLMs to unique user perspectives, beyond broad demographic or ideological groups, can enable more accurate and natural conversational AI. 

I think the core idea is sound and I agree with the reviewers that the experiments are reproducible, and results are likely generalizable beyond this dataset based on additional analyses.  The core contribution is demonstrating that modelling a user's past opinions is a critical signal, on par with or even more informative than standard factors like demographics and ideologies. This insight could reframe debates in social sciences on whether ideologies alone capture individual variance. The authors thoroughly analyze different prompting combinations on multiple LLMs using the OpinionQA dataset. The results clearly show gains from adding past opinions to user profiles, with the full model reaching up to 7% higher accuracy. Additional BERT experiments confirm the consistency of these trends.

I think the paper is technically sound, with reproducible experiments and sufficient analysis, especially since the framing and motivation situate the work well within active discussions on model personalization in the NLP community.

In conclusion, the work provides valuable insights into modelling different user factors, laying the groundwork for personalized applications.
Decision: Accept-Findings