Whose Emotions and Moral Sentiments Do Language Models Reflect? (2024)

Abstract: Language models (LMs) are known to represent the perspectives of some social groups better than others, which may impact their performance, especially on subjective tasks such as content moderation and hate speech detection. To explore how LMs represent different perspectives, existing research focused on positional alignment, i.e., how closely the models mimic the opinions and stances of different groups, e.g., liberals or conservatives. However, human communication also encompasses emotional and moral dimensions. We define the problem of affective alignment, which measures how LMs’ emotional and moral tone represents those of different groups. By comparing the affect of responses generated by 36 LMs to the affect of Twitter messages written by two ideological groups, we observe significant misalignment of LMs with both ideological groups. This misalignment is larger than the partisan divide in the U.S. Even after steering the LMs towards specific ideological perspectives, the misalignment and liberal tendencies of the model persist, suggesting a systemic bias within LMs.1

heading: 1 Introduction
text: The capacity of language models (LMs) to generate human-like responses to natural language prompts has led to new technologies that support people on cognitive tasks requiring complex judgements. However, researchers found that LMs inherit biases2 from humans, as their views are shaped by online users who produced the pretraining data, feedback from crowdworkers during Reinforcement Learning from Human Feedback (RLHF) process (Ouyang et al., 2022), and potentially, the decisions made by the model developers themselves (Santurkar et al., 2023). In subjective tasks, such as
1Code and data are available at https://github. com/zihaohe123/llm-affective-alignment.
2Throughout this paper, we use “bias” to refer to a systematic statistical tendency, rather than unfairness or prejudice.
hate speech detection (Hartvigsen et al., 2022), content moderation (He et al., 2023), and legal judgement (Jiang and Yang, 2023), these biases may show up as LMs adopting the perspectives of one group while excluding others. This may lead to undesirable downstream consequences, ranging from negative user experiences with LMs to societallevel inequity, division and polarization.
To examine how LMs represent differences in perspectives of different groups, existing research has looked at positional alignment: how closely the model’s opinions or stances mirror those of different social groups (Santurkar et al., 2023; Durmus et al., 2023). Using multi-choice survey questions, researchers have demonstrated that language models are misaligned with the US population and represent the perspectives of some demographic groups better than others.
However, positional agreement captures just one aspect of alignment. Human communication also carries cues to emotions and moral sentiments– collectively referred to as affect–which are integral to social interaction and cohesion (Graham et al., 2009; Iyengar et al., 2019; Mokhberian et al., 2020). While emotions help shape individual’s positions and stances on issues, they are distinct constructs. Take an example of a specific position – a pro-masking stance. The same position can be expressed using contrasting emotions as in the following two statements:
• Every mask we wear is a badge of honor, showing love and respect for our communities.
• It’s heartbreaking to see the impact of not wearing masks - lives lost, dreams deferred. Every choice to ditch the mask deepens the crisis. Let’s not add to the pain.
While both statement take a pro-masking stance, the first one expresses positive emotions, and the second negative ones.
6611
How LMs represent affect plays an important role in their performance in downstream tasks, especially subjective tasks. Consider how an LM facilitating online discussions may handle the comment: “Wearing a mask is a personal choice, not a public responsibility.” An LM aligned with conservatives would not flag this comment as it prioritized the moral sentiments of liberty and authority typically associated with conservatives (Doğruyol et al., 2019). However, this comment may evoke negative reactions from liberals, as it goes against their deeply-held values of care and fairness, prompting a liberal-aligned LM to flag it. Therefore, we ask:
Whose affect, i.e., emotional and moral tone, do language models reflect?
Our contributions. We define affective alignment as the degree to which a model’s emotional and moral tone matches that of people in similar situations. To study this, we analyze two datasets of Twitter messages on contentious issues like COVID-19 and abortion, disaggregating users by political ideology (liberal or conservative). We prompt 36 LMs of varying sizes to generate statements on topics like “COVID-19 mask mandates” and “abortion rights and access,” then compare the emotions and moral sentiments in the modelgenerated responses to those of Twitter users from different ideological groups.
We first assess the models’ default affective alignment, based on the responses to default prompts that do not include information about a target demographic (persona). Our findings suggest that LMs show significant misalignment in affect with either ideological group, and the differences are larger than the ideological divide on Twitter. Moreover, consistent with prior findings (Santurkar
et al., 2023; Perez et al., 2022; Hartmann et al., 2023), all LMs exhibit liberal tendencies on topics related to COVID-19.
Next, we assess LMs’ affective alignment after we provide additional context in the prompt—steer the model—to generate texts from the perspective of liberals or conservatives. The results reveal that steering can help align LMs with the target group’s affect. However, even after steering, the models remain misaligned, and liberal tendencies of LMs cannot be mitigated by steering.
We believe that a deep analysis of the affect expressed by existing LMs is crucial for building AI systems for greater social good. To the best of our knowledge, our work is the first to systematically assess the affective alignment of LMs, which highlights the unequal affective representations of different ideological groups in current LMs. We hope that our framework can help guide future research in better understanding LMs’ representativeness of people from diverse backgrounds on an emotional and moral level. We elaborate the broader impact of the affective alignment in Appendix A.
Clarification on the scope. Our work introduces a new task of systematically probing LMs’ affective alignment with different social groups. We aim to objectively present our finding and offer insights, rather than prescribing optimization. Whether a high degree of affective alignment towards each single group is desirable, and whether LMs should equally represent each group’s affect, are highly dependent on the application context.

heading: 2 Measuring Affective Alignment
text: Figure 1 shows the overall framework. We first prompt LMs to generate tweets on some topic us-
ing default prompting or steered prompting. We use state-of-the-art classifiers to measure emotions and moral sentiment of both LM-generated tweets and human-authored tweets. The affective alignment is measured as one minus the Jensen-Shannon distance (JSD) between the distributions of affect.

heading: 2.1 Online Sociopolitical Discourse
text: To represent human affect empirically, we compile two Twitter datasets on sociopolitical discourse— COVID-19 Tweets and Roe v. Wade Tweets—and identify fine-grained sub-topics related to both issues. We estimate the ideological leanings (liberals vs. conservatives) of users based on the political bias of the news outlets they share. See Appendix B for details.

heading: 2.2 Text Generated by LMs
text: We categorize LMs into two groups: 1) base LMs (Ouyang et al., 2022; Touvron et al., 2023; Almazrouei et al., 2023; Jiang et al., 2023; Conover et al., 2023; Chung et al., 2022; Zheng et al., 2023), that have been pretrained with the causal language modeling objective using data from the Internet; 2) instruction-tuned LMs (Ouyang et al., 2022; Touvron et al., 2023; Almazrouei et al., 2023; Jiang et al., 2023; Conover et al., 2023; Chung et al., 2022; Zheng et al., 2023), that have been further finetuned on instructions following and with RLHF for better alignment with human values. To guide the LM to generate open-ended responses on different topics, we use opinion-eliciting prompts. To measure affect representativeness of models in different configurations, we consider the following two settings:
• Default Prompting – We prompt the LM to generate responses without providing with any additional context. Our goal is to measure the default representativeness of affect expressed by the LM. This would potentially disclose the disparity (if any) of the LM in representing perspectives of different demographic groups. An example prompt for base LMs is “In a tweet about [topic], it’s said that: ”, and for instruction-tuned LMs is “Share your thoughts on [topic] in a tweet.”, where “[topic]” is replaced with the one of the fine-grained topics discovered in the COVID-19 Tweets or the Roe v. Wade Tweets (Table 1 and 2 in Appendix B).
• Steered Prompting – We steer the LM to generate responses from the perspective of a specific demographic group, or persona, by adding context to the prompt. This aims to test the model’s steerability, i.e., how well it can align itself with a specific demographic group when instructed to do so. We explore whether the model’s affective alignment with a persona increases through steered prompting compared to default prompting. In this work we focus on ideological groups (i.e., liberals vs conservatives) and perform “liberal steering” and “conservative steering.” One example of steered prompting for base LMs is “Here’s a tweet regarding [topic] from a liberal/conservative standpoint:”, and for instruction-tuned LMs is “Compose a tweet about [topic] from a liberal/conservative viewpoint.”
The idea for these two kinds of prompting is inspired by previous works (Santurkar et al., 2023; Durmus et al., 2023). To mitigate the effect of the model’s sensitivity to the specific wording in a prompt, we craft 10 different prompts for the base LMs and instruction-tuned LMs, using default prompting and steered prompting, respectively (Table 4 in Appendix). For each fine-grained topic, we generate 2,000 responses, using 2,000 prompts randomly sampled (with replacement) from the 10 candidate prompts. For more details on the generation process, please see Appendix D.1.

heading: 2.3 Measuring Affect
text: Human affect, including emotions and morality, in online discourses is used as an indicator to track public opinion on important issues and monitor the well-being of populations (Klašnja et al., 2018). Detecting Emotions. Emotions are a powerful element of human communication (vanKleef et al., 2016). To detect emotions, we use SpanEmo (Alhuzali et al., 2021), fine-tuned on top of BERT (Devlin et al., 2019) on the SemEval 2018 1e-c data (Mohammad et al., 2018), which is specifically curated from Twitter and widely recognized as a benchmark for emotion detection on social media. SpanEmo learns the correlations among the emotions and achieves a micro-F1 score of 0.713 on this dataset, outperforming several other baselines and achieving the state-of-the-art in detecting emotions on Twitter data. In addition, Rao et al. (2023a) annotated the emotions of a subset of the Roe v. Wade
tweets that we use in our paper, and further evaluated SpanEmo’s performance on it. They showed an average accuracy of over 0.9 across different emotions. We measure the following emotions: anticipation, joy, love, trust, optimism, anger, disgust, fear, sadness, pessimism and surprise. The model returns a score giving the confidence that a tweet expresses an emotion. We average scores over all tweets with that emotion. Detecting Moral Sentiments. Moral Foundations Theory (Haidt et al., 2007) posits that individuals’ moral perspectives are a combination of a set of foundational values. These moral foundations are quantified along five dimensions: dislike of suffering (care/harm), dislike of cheating (fairness/cheating), group loyalty (loyalty/betrayal), respect for authority and tradition (authority/subversion), and concerns with purity and contamination (purity/degradation). These moral dimensions are crucial for understanding the values driving liberal and conservative discourse. We use DAMF (Guo et al., 2023b) for moral sentiment detection, which is finetuned on top of BERT (Devlin et al., 2019) on three Twitter datasets (including COVID-19 tweets and abortionrelated tweets studied in this paper) and one news article dataset. The large amount and the variety of topics in the training data helps mitigate the data distribution shift during inference. Guo et al. (2023b) annotated the moral foundations of a subset of the COVID-19 tweets that we use in our paper, and further evaluated the performance of DAMF on it, which led to F1-score of 0.71 . The model returns a value indicating a confidence that a tweet expresses a moral foundation. We average scores over all tweets with that moral foundation.

heading: 2.4 Measuring Alignment
text: Let us represent an LM as f and a group of humans as g. We aim to measure affective alignment ST (f, g) between the LM f and humans g on a set of n topics T = {t1, t2, ..., tn} by measuring emotions (resp. moral sentiments) expressed in tweets about each topic ti ∈ T . Human-authored tweets about t are available in a dataset (e.g., COVID-19 Tweets or Roe v. Wade Tweets). To create LM’s tweets about ti, we prompt it on the topic to generate a set of m responses R = {r1, r2, ..., rm}. We compare D̂(ti), the distribution of emotions (resp. moral foundations) in LM-generated tweets on topic ti, and D(ti), the distribution in human-
authored tweets on the same topic. We measure affective alignment on a topic ti as Sti(f, g) ∈ [0, 1], using (1 - Jensen-Shannon Distance) between the distributions D̂(ti) and D(ti). The alignment of LM f with humans g on the set of topics T is averaged over that for each topic ti in it:
ST (f, g) = 1
n
n∑
i=1
(1−JSD(D̂(ti), D(ti))). (1)
A value of ST close to 1 implies strong alignment, while smaller values imply weak alignment. For an LM f , we study the default model (fdefault), the liberal steered model (flib_steered), and the conservative steered model (fcon_steered). For humans, we study liberals (gl) and conservatives (gc).

heading: 2.4.1 Proximity Between Emotions
text: To more accurately capture the interrelated nature of emotions3, we integrate the concept of emotional proximity using the Plutchik Emotion Agreement (PEA). For instance, emotions like joy and love are closer on the emotional spectrum than joy and anger. To quantitatively capture these relationships, we utilize the Plutchik Emotion Agreement (PEA) (Desai et al., 2020), leveraging the Plutchik wheel (Plutchik, 2001) which organizes emotions into a spatial model indicative of their relational proximity. The PEA is calculated from the polar coordinates of each emotion on the wheel, as
PEA(ei, ej) = max(|1− 1
π |f(ei), f(ej)||), (2)
where ei and ej are two different emotions, and f(e) represents the polar coordinate of the emotion. The emotion proximity matrix A, where Aij represents the proximity between ei and ej , is shown in Appendix C. This methodological adjustment allows us to account for the interconnected nature of emotional expressions, refining our alignment measurements. We weight the emotion distributions vectors using the proximity matrix, as D̂′(t) = A · D̂(t), and D′(t) = A ·D(t), then use the weighted vectors for computing the affective alignment as in Equation 1.
3We only consider the agreement between different emotions but not moral foundations, as there is no existing work on modeling the structure of moral foundations as the Plutchik’s wheel for emotions.
1.00.90.80.7 affective alignment (S)
gl gc gpt-3.5* llama-2-7b-chat* llama-2-13b-chat* mistral-7b-instruct* falcon-7b-instruct*
dolly-v2-3b* dolly-v2-7b* dolly-v2-12b* flan-t5-xl*
flan-t5-xxl* vicuna-7b-v1.5-16k* vicuna-13b-v1.5-16k* gpt-3-davinci*
gpt-3-babbage* llama-2-7b*
llama-2-13b* mistral-7b* falcon-7b* opt-125m* opt-350m*
opt-1.3b* opt-2.7b* opt-6.7b* opt-13b* opt-iml-1.3b* bloom-560m*
bloom-1b1* bloom-1b7*
bloom-3b* pythia-160m* pythia-410m*
pythia-1b* pythia-1.4b* pythia-2.8b* pythia-6.9b* pythia-12b* S(f, g
l) S(f, gc)
(a) Affective alignment S in COVID-19.
1.00.90.80.70.6 affective alignment (S)
gl gc gpt-3.5 llama-2-7b-chat llama-2-13b-chat mistral-7b-instruct falcon-7b-instruct
dolly-v2-3b dolly-v2-7b dolly-v2-12b flan-t5-xl
flan-t5-xxl vicuna-7b-v1.5-16k vicuna-13b-v1.5-16k gpt-3-davinci
gpt-3-babbage llama-2-7b
llama-2-13b mistral-7b falcon-7b opt-125m opt-350m
opt-1.3b opt-2.7b opt-6.7b opt-13b opt-iml-1.3b bloom-560m
bloom-1b1 bloom-1b7
bloom-3b pythia-160m pythia-410m
pythia-1b pythia-1.4b pythia-2.8b pythia-6.9b pythia-12b S(f, g
l) S(f, gc)
(b) Affective alignment S in Roe v. Wade.
Figure 2: Default affect alignment S of different LMs with ideological groups – liberals (gl) and conservatives (gc), measured by emotions. * indicates that the alignment of the model with both ideological groups are significantly different at p < 0.05. For each LM, the alignment is averaged over that on different topics related to the issue, with the means shown by circles and the standard deviations shown by errors bars. Base LMs and instruction-tuned LMs are separated by the black horizontal dashed line. The alignment between the two ideological groups (above the red horizontal dashed line) themselves are measured as a baseline.

heading: 3 Results and Analysis

heading: 3.1 Representativeness of Affect under Default Prompting
text: Our investigation into the affective alignment of LMs with humans starts with two research questions: (1) Do language models exhibit strong affective alignment with human groups? (2) Do the models equitably represent each group?
Figures 2 reports the affective alignment of various LMs with liberals (gl) and conservatives (gc) by default prompting in the two datasets, measured by emotions. Please refer to Appendix D.2 for results measured by moral sentiments. Since the patterns of alignment measured by emotions and moral sentiments are similar, we focus on the emotional alignment.
Do the models exhibit strong affective alignment? Defining a precise threshold for “strong” alignment is challenging. We consider as baseline the alignment between the two ideological groups on Twitter, i.e. emotion similarity between liber-
als and conservatives in online discourses (vertical lines in Figure 2). Any alignment falling short of this benchmark could be deemed insufficient, given the profound divisions in contemporary sociopolitical discourse (Rao et al., 2023b). This baseline is henceforth referred to as the “partisan alignment baseline”.
From Figure 2, it is evident that nearly all LMs fall short of the partisan alignment baseline, indicating weak alignment. Base LMs, trained on causal language modeling tasks without explicit affective alignment tuning, seem to lack the capacity to learn affect during the pretraining phase. Instructiontuned models, despite undergoing instruction-based and RLHF training to foster alignment with human values, do not appear to extend this alignment to emotional or moral dimensions. Notably, even sophisticated models like GPT-3.5 exhibit heightened misalignment compared to base models. This could be attributed to the models’ intricate architectures and training processes, which may inadvertently amplify misalignment due to their complexity and
sensitivity to the training data’s composition. While this paper focuses on political identities, it is conceivable that the default affect distribution of the models might be more closely aligned with other demographic groups. Future research could explore various demographic segments beyond the political dimension to identify those with which the models demonstrate stronger affective alignment.
Do the models represent each group equitably? It is apparent that on COVID-19, all LMs reveal liberal tendencies, as the alignment with liberals is consistently higher, and the partisan alignment difference is statistically significant. Given the novelty of COVID-19 and its prevalence on social media, where liberal perspectives dominate (Shah et al., 2020), we hypothesize that a significant portion of the LMs’ pretraining data is derived from discussions in these forums, and thus LMs absorb more emotional and moral tone of liberal narratives.
Conversely, on the Roe v. Wade Tweets (Figure 2b) the LMs display no discernible political tendencies, with some models exhibiting a slight liberal inclination and others conservative, leading to a generally balanced alignment with both political ideologies. In fact, the partisan alignment difference is not statistically significant on Roe v. Wade. In contrast to COVID-19, Roe v. Wade is a longstanding issue in U.S. history, with discourses extending well beyond social media platforms. Consequently, it is likely that the discussions encompassing both political ideologies are more evenly represented in the pretraining data for LMs.

heading: 3.2 Representativeness of Affect in Steered Prompting
text: We now move to analyze the affect representativeness in steered scenarios, where models are explicitly prompted to align with ideological leanings. This approach helps us understand the malleability of LMs when directed to mimic specific personas. We aim to study the following research questions: (1) Is steering effective for LMs to mimic a target group (persona)? (2) Do the models exhibit higher affective alignment to the specific persona when prompted to behave like it? (3) Do steered models exhibit strong affective alignment with each persona? (4) Is the representational imbalance controllable by steering?
Figure 3 provides insights into how steering
instruction-tuned LMs to adopt a liberal (gl) or conservative (gc) persona impacts affective alignment measured by emotions. Please refer to Appendix D.3 for that of base LMs measured by emotions, and that of all LMs measured by moral sentiments. The directionality of triangle symbols shows the nature of steering: left for liberal steering and right for conservative steering. The circles show the models’ baselines, i.e. the default alignment which are identical to the circles in Figure 2.
Is steering effective? We expect that a model’s affective alignment with an ideological group after liberal steering and conservative steering should differ; otherwise, we deem that the steering is ineffective. In Figure 3, it is evident that steering is effective for most instruction-tuned LMs, as indicated by the left-facing and right-facing triangles of the same color positioned apart from each other. However, such failure cases happen for almost all base LMs, as indicated by the the leftfacing and right-facing triangles of the same color positioned extremely close to each other or even overlapping (Figure 6 in Appendix D.3). This observation demonstrates that instruction-tuning and RLHF make LMs more steerable. We do not exclude the possibility that the failure cases for base LMs are caused by the specific prompts we used to steer the base LMs, but we leave how to craft better prompts to steer base LMs for future work. In the regard, in the following analysis related to steering, we only focus on instruction-tuned models.
Does steering improve affective alignment? For emotions on COVID-19 (Figure 3a), it is evident that most instruction-tuned LMs (8 out of 12) are better aligned with the target ideological group after steering, as indicated by blue left-facing (resp. orange right-facing) triangles positioned to the right of the blue (resp. range) circles. In addition, for these models, either ideological steering leads to higher affective alignment with both ideological groups. We argue that this is because if the model detects ideology-related keywords in the prompt, either “liberal” or “conservative”, it automatically aligns itself to the political domain, achieving higher alignment to both ideological groups. Moreover, the improvement in alignment by conservative steering is much more pronounced than that by liberal steering, as indicated by the distance between orange right-facing triangle and the orange circle much longer than that between
0.8 0.9 1.0 affective alignment (S)
gl gc gpt-3.5*^
llama-2-7b-chat*^ llama-2-13b-chat*^ mistral-7b-instruct*^
falcon-7b-instruct*^
dolly-v2-3b*^ dolly-v2-7b*^ dolly-v2-12b*^
flan-t5-xl*^
flan-t5-xxl*^
vicuna-7b-v1.5-16k*^ vicuna-13b-v1.5-16k*^
S(fdefault, gl) S(flib_steered, gl)
S(fcon_steered, gl) S(fdefault, gc)
S(flib_steered, gc) S(fcon_steered, gc)
(a) Affective alignment S in COVID-19 Tweets.
0.8 0.9 1.0 affective alignment (S)
gl gc gpt-3.5
llama-2-7b-chat llama-2-13b-chat mistral-7b-instruct
falcon-7b-instruct
dolly-v2-3b dolly-v2-7b dolly-v2-12b
flan-t5-xl
flan-t5-xxl
vicuna-7b-v1.5-16k vicuna-13b-v1.5-16k
S(fdefault, gl) S(flib_steered, gl)
S(fcon_steered, gl) S(fdefault, gc)
S(flib_steered, gc) S(fcon_steered, gc)
(b) Affective alignment S measured in Roe v. Wade Tweets.
Figure 3: Steered affective alignment S of different instruction-tuned LMs with both ideological groups – liberals (gl) and conservatives (gc), measured by emotions, for . * indicates that the alignment of the liberal steered model with both ideological groups are significantly different at p < 0.05; ^ indicates that for the conservative steered model. Left-facing triangles represent the models by liberal steered prompting; right-facing triangles represent the models by conservative steered prompting; circles with no filling colors represent the models by default. For each LM, the alignment is averaged over that on different topics detected within the dataset. The alignment between the two ideological groups (above the red horizontal dashed line) themselves are measured as a baseline.
the blue left-facing triangle and the blue circle, possibly because LMs already exhibit stronger alignment by default with liberals, thus offering limited scope for further liberal alignment enhancement.
In the context of Roe v. Wade (Figure 3b), while we also observe better alignment for most instruction-tuned LMs, the impact of steering is less pronounced, with the alignment for some models after steering showing minimal change from default prompting. This may suggest that the models’ affective responses to long-standing, deeply polarizing issues are more entrenched, making them less amenable to steering.
Do the models exhibit strong affective alignment after steering? Although steering enhances affective alignment for most instruction-tuned LMs, the alignment of nearly all LMs to either ideological group is still lower than the partisan alignment baseline. Notably, the more sophisticated model gpt-3.5, even after steering, is least aligned with both partisan perspectives.
Is the representational imbalance controllable by steering? In §3.1 we observe the default LMs’ liberal representational tendencies on COVID-19 Tweets. We aim to investigate (1) whether the liberal tendencies will be further exacerbated by liberal steering, and (2) whether the liberal tenden-
cies will be mitigated or even reversed by conservative steering. We observe from Figure 3a that all instruction-tuned LMs retain liberal tendencies, after both liberal steering (indicated by blue leftfacing triangles to the right of orange left-facing triangles) and conservative steering (indicated by blue right-facing triangles positioned to the right of orange right-facing triangles). In addition, the magnitude of the tendencies (as indicated by distance between the blue and orange markers of the same shape) barely changes after steering. This suggests that the representational imbalance is deeply entrenched in the instruction-tuned LMs, which cannot be mitigated or reversed simply through steering.

heading: 3.3 Topic-level analysis
text: To gain deeper insights into the observations from §3.1 and §3.2, we examine the topic-level distribution of emotions and moral foundations of LMgenerated responses and compare them to those in human-authored tweets. Figure 4 shows these distributions of tweets from two LMs – gpt-3.5 and llama-2-7b-chat – and humans from both ideological groups, on the topic “COVID-19 mask mandates and policies” from the COVID-19 Tweets. Please refer to Appendix D.4 for the distributions
on the topic “fetal rights debate in abortion” from the Roe v. Wade Tweets. Observing from Figure 4, compared to humans, LMs show a more focused distribution across different types of emotions or moral foundations. This is similar to Durmus et al. (2023), where the authors find that LM tends to assign a high confidence to a single option for multichoice questions. Such high confidence is observed in both the default models and liberal steered models. With conservative steering, LMs’ generated distribution becomes smoother and more aligned with that from humans. This might be one of the reasons why conservative steering better aligns the models with both liberals and conservatives, as observed in §3.2.
For both gpt-3.5 and llama-2-7b-chat, on emotions, the default models and the liberal steered models show substantially less anger and disgust and substantially more optimism than human tweets. With respect to moral foundations, these models also express substantially more care, less
harm, more loyalty and less subversion that humanauthored tweets. We hypothesize that LMs are trained to relentlessly convey optimism, due to certain concerns of risks. However, conservative steering distributes the probability mass in positive emotions and moral foundations to more negative ones, demonstrating the implicit bias inherent in LMs to associate conservatives with negative affect.

heading: 4 Related Work
text: Measuring human-LM Alignment LMs trained on extensive datasets of human language from the Internet, are capable of simulating realistic discourse. To ensure that LMs generate text consistent with human values and ethical principles, many recent works have investigated the human-LM alignment. Popular frameworks include reinforcement learning with human feedback (RLHF) or AI feedback (RLAIF) (Ouyang et al., 2022; Glaese et al., 2022; Bai et al., 2022). To measure alignment Santurkar et al. (2023) compared LMs’ opinions with
human responses in public opinion polls among various demographic groups and found substantial misalignment. Durmus et al. (2023) expanded the study of alignment to a global scale using crossnational surveys and discovered LMs’ inclination towards certain countries like USA, as well as unwanted cultural stereotypes. Simmons (2022) measured LMs’ moral biases associated with political groups in the United States when responding to different moral scenarios; however, they only evaluate the models’ moral responses based on a general statistical finding from previous works that “liberals rely primarily on individualizing foundations while conservatives make more balanced appeals to all 5 foundations”. Abdulhai et al. (2023) measured moral foundations of LLMs using the Moral Foundations Questionnaire (MFQ) questionnaire and compared the results of LLMs to humans. In contrast, our work evaluate the models against affect distributions observed from real-world humangenerated texts on a topic basis. The affect people/models express in open-ended texts is likely to be different from how they answer the closeended questionnaire, and our framework can capture the fluidity and complexity of human affective responses in a way that structured questionnaires might not fully encompass.
LMs and Political Leanings Feng et al. (2023) discovered that pretrained LMs do exhibit political biases, propagating them into downstream tasks. In terms of adapting LMs to simulate human opinions, Argyle et al. (2023) showed that GPT-3 can mimic respondents in extensive, nationally-representative opinion surveys. Other researchers have finetuned LMs to learn the political views of different partisan communities to study polarization (Jiang et al., 2022; He et al., 2024). To evaluate news feed algorithms, Törnberg et al. (2023) created multiple LM personas from election data to simulate conversations on social media platforms. Chen et al. (2024) found that most LLMs show a left-leaning stance on a wide range of issues and LLMs can be easily manipulated by instruction tuning.

heading: 5 Conclusion
text: Our study has explored how LMs align with the affective expressions of liberal and conservative ideologies. Through the lens of two contentious sociopolitical issues, we discover that LMs can mimic partisan affect to a degree, which, nevertheless, is weaker than that between liberals and conservatives
in the real world. In addition, LMs show liberal tendencies on certain issues, aligning more with the affect of liberals. The misalignment and the liberal tendencies are not solvable by steering. As a first step towards systematically measuring the affective alignment of LMs with different social groups, we hope that this study will gather more attention from the research community in understanding the interactions of affect between LMs and humans.

heading: Acknowledgements
text: This project has been funded, in part, by DARPA under contract HR00112290106 and HR00112290106. We appreciate the constructive advice and suggestions from the anonymous reviewers.

heading: Limitations

heading: Data Collection and Demographic Limitations.
text: The dataset utilized in our study is derived from Twitter and focuses solely on liberal and conservative perspectives within the United States. Such a narrow scope overlooks the multifaceted nature of global demographics and political leanings. Additionally, limiting the data source to Twitter may not provide a comprehensive view of the social and political discourse surrounding the issues in question. Moving forward, our methodology should be applied to broader datasets that encapsulate a more diverse range of subjects, platforms, and demographics. Affective Classifier Accuracies The classifiers used for emotions and moralities are not perfect. However, our method depends on comparing the emotion and morality distributions between the real-world and model-generated tweets. This comparative approach mitigates the impact of potential classifier inaccuracies, as the same classifier is applied consistently across both corpora. Since we are primarily looking at differences, rather than absolute values of emotions in the data, we believe we are justified in using the imperfect classifiers to reveal differences in affective alignment. Nevertheless, we have endeavored to utilize the most advanced models currently available for accurately measuring emotions and moral foundations in the sociopolitical domain. The performance of both models has been validated on a variety of social media data (Rao et al., 2023b; Guo et al., 2023a; Chochlakis et al., 2023), and proposing methods to achieve the new state-of-the-art on emotion and
morality detection is out of the scope of this work. Affective Classifier Constraints. Our affect measurement relies on classifiers built upon BERT, a model whose simplicity and scale are modest compared to the 36 larger LMs analyzed. This discrepancy raises concerns about the precision of affect detection; the classifiers might not capture the nuances of affect as effectively as those based on larger models. Moreover, the divergence in affect understanding between the classifiers and the LMs could introduce discrepancies. While the LMs might generate affectively coherent responses from their perspective, these may not align with the interpretations of a BERT-based "third-party" classifier. Emotion and moral foundation detection are inherently subjective, and the potential mismatch in affect recognition necessitates caution. Future research should consider leveraging the studied LMs themselves to evaluate affect. This could provide a more congruent assessment of the models’ affective outputs and allow for a deeper investigation into the observed misalignments. Steering Efficacy and Prompt Design. Our attempts to steer base LMs towards specific political identities revealed a notable challenge: the models did not adequately distinguish between “liberals” and “conservatives”. The design of our steering prompts may play a significant role in this limitation. If the prompts are not sufficiently nuanced or if they fail to encapsulate the essence of the targeted political identities, the models’ responses may not reflect the intended affective stance. In future iterations, prompt design must be meticulously refined to ensure it elicits the desired affective response from the model. This may involve a more iterative and data-driven approach to prompt engineering, possibly incorporating feedback loops with human evaluators to finetune the prompts’ effectiveness.

heading: Ethics Statement
text: Our work utilizes publicly available data from social media, specifically Twitter, which poses potential privacy concerns. We have ensured that all Twitter data used in our study has been accessed in compliance with Twitter’s data use policies and that individual privacy has been respected, with no attempt to de-anonymize or reveal personally identifiable information. The dataset consists of tweets related to COVID-19 and Roe v. Wade, which are topics of public interest and social importance. In handling this data, we were careful to maintain the
anonymity of the users and to treat the content with the utmost respect, given the sensitive nature of the topics.

heading: A Broader Impact
text: Implications of affective alignment. Introducing affective alignment, our paper bridges a gap left by prior research focused predominantly on positional alignment. The impact of affective alignment of LMs is crucial in the following contexts.
In mental health applications, an LM’s ability to align affectively with users can provide support and improve therapeutic outcomes. In educational settings, affective alignment in LMs can foster a conducive learning environment, adapting to students’ emotional states to enhance engagement and comprehension. In political discourse, affective alignment is key to fostering constructive debates and reducing polarization; for instance, during election campaigns, LMs that can align affectively with the emotional and moral sentiments of different voter bases can facilitate more resonant and effective communication.
However, there are also scenarios where affective alignment is undesirable. For example, In high-stakes negotiation or diplomacy settings, excessive affective alignment might hinder the ability to maintain a firm stance or negotiate effectively. In legal or judicial contexts, an overly empathetic LM could bias the presentation of facts or arguments, potentially affecting impartiality. Similarly, in news reporting, high affective alignment might lead to biased news coverage, undermining journalistic neutrality.
Therefore, it is important to achieve a balanced affective alignment that enhances positive interactions and outcomes without compromising on ob-
jectivity and fairness. Recognizing these subtleties can guide the ethical and effective deployment of LMs, ensuring they serve as beneficial tools in society rather than exacerbating existing divides or biases.
Implications of LM representativeness of affect across different demographics. The representativeness of LMs in capturing affect across different demographics is crucial for ensuring that AI systems communicate in ways that are emotionally and culturally resonant. However, the representativeness of LMs in reflecting affect across demographics should be contextually calibrated, not merely equalized. For example, an LM used in a global social media platform should accurately reflect the emotional and moral nuances of its worldwide user base, avoiding over-representation of any single group’s affective norms. In contrast, an LM in localized service applications, like community-based mental health support, should be tuned to the specific emotional and cultural characteristics of that community. Balancing these representational needs requires a strategic approach to developing LMs that are both inclusive and contextually aware.
Implications of the framework to measure affective alignment. Our proposed framework offers a comprehensive tool for assessing how well LMs resonate with human emotions and morals across various demographics and subjects. This versatility facilitates broader research applications, enabling researchers and developers to evaluate and enhance LM designs for cultural sensitivity, inclusiveness, and ethical alignment with human values, thus paving the way for more responsible AI innovations.
Implications of our findings on the affective alignment of LMs with both political groups. Our analysis reveals that LMs display a notable liberal bias, especially in contexts like COVID-19 discussions, and exhibit an overall affective misalignment with political groups, surpassing the existing partisan divide in the U.S. This indicates a systemic inclination within LMs. These observations underscore the challenge of LMs achieving strong affective alignment with humans and equal affective representations.
In conclusion, our research not only contributes to the academic understanding of LMs but also serves as a pivotal guide for developing AI that is
emotionally intelligent, morally considerate, and socially representative.

heading: B Online Sociopolitical Discourse Data
text: We compile two datasets on sociopolitical discourse on Twitter: COVID-19 Tweets and Roe v. Wade Tweets. They cover a wide range of finegrained topics, including emotionally divisive topics. Our selection of COVID-19 and Roe vs. Wade Twitter datasets was based on the following factors:
• COVID-19 has global significance, affecting diverse aspects of life and eliciting a wide range of emotional and moral responses, making it ideal for studying affective alignment. Roe v. Wade, representing a longstanding and polarizing issue in U.S. politics, provides a rich dataset to explore deeply entrenched moral and emotional viewpoints, allowing for a detailed analysis of language models’ alignment with complex ideological positions.
• These datasets are publicly accessible, which is crucial for ensuring the transparency and reproducibility of our research.
• The method for estimating user partisanship (Rao et al., 2021) is particularly effective for these datasets. This is because the tweets in these datasets frequently include URLs to news articles, which serve as reliable indicators of the users’ political leanings. We’ll add these considerations to the paper.
To assess the affect alignment, we identify important issues discussed in the Twitter datasets using a semi-supervised method described in Rao et al. (2023b). This method harvests and selects from Wikipedia the relevant and distinctive keywords for each issue, and detect the issues in each tweet using the presence of these keywords and phrases. An issue, such as “masking” in COVID-19 tweets, can still be broad and too general. In order to obtain a fine-grained span of topics, we use GPT-4 to cluster the keywords in each issue into sub-topics, such as “mask mandates and policies” and “mask health concerns”. We manually validated the clustering results. Each tweet can be associated with multiple issues and sub-topics.

heading: B.1 COVID-19 Tweets
text: The corpus of discussions about the COVID-19 pandemic (Chen et al., 2020) consists of 270 million tweets, generated by 2.1 million users, posted
between January 2020 and December 2021. These tweets contain one or more COVID-19-related keywords, such as “coronavirus”, “pandemic”, and “Wuhan,” among others. Users participating in these discussions were geo-located to states within the U.S. based on their profile and tweets using a tool Carmen (Dredze et al., 2013). We use a validated method (Rao et al., 2021) to estimate the partisanship of individual users. This method uses political bias scores of the domains users share according to Media Bias-Fact Check (Check, 2023) to estimate the ideology of users. In other words, if a users shares more left-leaning domains, they are considered to be liberal.
We focus on the issues that divided public opinion during the pandemic, including: (1) origins of the COVID-19 pandemic, (2) lockdowns, (3) masking, (4) education and (5) vaccines. Within these issues, we further detect a total of 26 finegrained sub-topics (see Table 1). When using LMs to generate responses on the topics, we only keep those with at least has 1,000 tweets from both ideological leanings. After filtering original tweets (as opposed to retweets and quoted tweets) categorized to one of the five issues and authored by users with identified political affiliation, we are left with 9M tweets.

heading: B.2 Roe v. Wade Tweets
text: Our second dataset comprises of tweets about abortion rights in the U.S. and the overturning of Roe vs Wade. These tweets were posted between January 2022 to January 2023 (Chang et al., 2023). Each tweet contains at least one term from a list of keywords that reflect both sides of the abortion debate in the United States. This dataset includes approximately 12 million tweets generated by about 1 million users in the U.S. We used the same technique to geo-locate users, infer user political ideology, and detect issues and sub-topics as for the COVID-19 tweets dataset. We focus on the following five major issues: (1) religious concerns, (2) bodily autonomy, (3) fetal rights and personhood, (4) women’s health and (5) exceptions to abortion bans. The associated 24 fine-grained topics are listed in Table 2. When using LMs to generate responses on the topics, we only keep those with at least has 1,000 tweets from both political identities.

heading: C Proximity between Emotions
text: The polar coordinates of the 11 emotions (anger, anticipation, disgust, fear, joy, love, optimism, pessimism, sadness, surprise, and trust) are −12π, −14π, −34π, 12π, 0, 18π, −18π, 78π, π, 34π, and 14π respectively. For emotions that are not the Plutchik8 emotions, we aggregate the coordinates of their neighboring emotions. The emotion proximity matrix is shown in Table 3.

heading: D Experiments

heading: D.1 Experimental Setup
text: On each topic, we obtain 2,000 generations from a model, using the prompt templates shown in Table 4.
For GPT based models we queried OpenAI’s API. The specific models we used for gpt-3.5, gpt3-davinci, and gpt-3-babbage are gpt-3.5-turbo1106, davinci-002, and babbage-002 respectively. We set temperature to 0.9 and only allow maximum generation length of 96 due to the concerns of cost.
For other open-sourced models, we use their checkpoints on huggingface (Wolf et al., 2019) to run the generation. For all generations we set top_p to 0.9, temperature to 0.9, and do_sample to True. The inference is run using an Tesla A100 GPU with 80GB memory. The running time for all topics in either COVID-19 Tweets or the Roe v. Wade Tweets varies from 2hrs to 30hrs, depending the size of the model.

heading: D.2 Representativeness of Affect under Default Prompting
text: Figure 5 shows the affective alignment of various LMs with liberals (gl) and conservatives (gc) by default prompting in the two datasets measured by moral sentiments.

heading: D.3 Representativeness of Affect under Steered Prompting
text: Figure 6 provides insights into how steering instruction-tuned LMs to adopt a liberal (gl) or conservative (gc) persona impacts affective alignment measured by moral sentiments. Figure 7 shows the affective alignment under steered prompting measured by moral foundations.

heading: D.4 Topic-Level Analysis
text: Figure 8 shows emotion and moral foundation distributions of tweets from two LMs – gpt-3.5 and
llama-2-7b-chat – and humans from both ideological groups, on the topic “fetal rights debate in abortion” from the Roe v. Wade Tweets.
1.00.90.80.70.60.50.40.3 affective alignment (S)
gl gc gpt-3.5* llama-2-7b-chat* llama-2-13b-chat* mistral-7b-instruct* falcon-7b-instruct*
dolly-v2-3b* dolly-v2-7b* dolly-v2-12b* flan-t5-xl*
flan-t5-xxl* vicuna-7b-v1.5-16k* vicuna-13b-v1.5-16k* gpt-3-davinci*
gpt-3-babbage* llama-2-7b*
llama-2-13b* mistral-7b* falcon-7b* opt-125m* opt-350m*
opt-1.3b* opt-2.7b* opt-6.7b* opt-13b* opt-iml-1.3b* bloom-560m*
bloom-1b1* bloom-1b7*
bloom-3b* pythia-160m* pythia-410m*
pythia-1b* pythia-1.4b* pythia-2.8b* pythia-6.9b* pythia-12b* S(f, g
l) S(f, gc)
(a) Affective alignment S in COVID-19 Tweets.
1.00.90.80.70.60.50.40.3 affective alignment (S)
gl gc gpt-3.5 llama-2-7b-chat llama-2-13b-chat mistral-7b-instruct falcon-7b-instruct
dolly-v2-3b dolly-v2-7b dolly-v2-12b flan-t5-xl
flan-t5-xxl vicuna-7b-v1.5-16k vicuna-13b-v1.5-16k gpt-3-davinci
gpt-3-babbage llama-2-7b
llama-2-13b mistral-7b falcon-7b opt-125m opt-350m
opt-1.3b opt-2.7b opt-6.7b opt-13b opt-iml-1.3b bloom-560m
bloom-1b1 bloom-1b7
bloom-3b pythia-160m pythia-410m
pythia-1b pythia-1.4b pythia-2.8b pythia-6.9b pythia-12b S(f, g
l) S(f, gc)
(b) Affective alignment S measured in Roe v. Wade Tweets.
Figure 5: Default affect alignment S of different LMs with both ideological groups – liberals (gl) and conservatives (gc), measured by moral foundations. * indicates that the alignment of the liberal steered model with both ideological groups are significantly different at p < 0.05. For each LM, the alignment is averaged over that on different topics detected within the dataset, with the means shown by circles and the standard deviations shown by errors bars. Base LMs and instruction-tuned LMs are separated by the black horizontal dashed line. The alignment between the two ideological groups (above the red horizontal dashed line) themselves are measured as a baseline.
0.8 0.9 1.0 affective alignment (S)
gl gc
gpt-3-davinci*^ gpt-3-babbage*^
llama-2-7b^ llama-2-13b mistral-7b^
falcon-7b opt-125m*^ opt-350m*^
opt-1.3b^ opt-2.7b^ opt-6.7b^ opt-13b*^ opt-iml-1.3b*^ bloom-560m*^
bloom-1b1*^ bloom-1b7*^
bloom-3b*^ pythia-160m*^ pythia-410m*^
pythia-1b*^ pythia-1.4b*^ pythia-2.8b*^ pythia-6.9b*^
pythia-12b*
S(fdefault, gl) S(flib_steered, gl)
S(fcon_steered, gl) S(fdefault, gc)
S(flib_steered, gc) S(fcon_steered, gc)
(a) Affective alignment S in COVID-19 Tweets.
0.9 1.0 affective alignment (S)
gl gc
gpt-3-davinci gpt-3-babbage
llama-2-7b llama-2-13b
mistral-7b falcon-7b opt-125m opt-350m
opt-1.3b opt-2.7b opt-6.7b opt-13b opt-iml-1.3b bloom-560m
bloom-1b1 bloom-1b7
bloom-3b pythia-160m pythia-410m
pythia-1b pythia-1.4b pythia-2.8b pythia-6.9b pythia-12b
S(fdefault, gl) S(flib_steered, gl)
S(fcon_steered, gl) S(fdefault, gc)
S(flib_steered, gc) S(fcon_steered, gc)
(b) Affective alignment S measured in Roe v. Wade Tweets.
Figure 6: Steered affective alignment S of different base LMs with both ideological groups – liberals (gl) and conservatives (gc), measured by emotions . Left-facing triangles represent the models by liberal steered prompting; right-facing triangles represent the models by conservative steered prompting; circles with no filling colors represent the models by default. * indicates that the alignment of the liberal steered model with both ideological groups are significantly different at p < 0.05; ^ indicates that for the conservative steered model. For each LM, the alignment is averaged over that on different topics detected within the dataset. The alignment between the two ideological groups (above the red horizontal dashed line) themselves are measured as a baseline.
0.5 0.6 0.7 0.8 0.9 affective alignment (S)
gl gc gpt-3.5 llama-2-7b-chat llama-2-13b-chat mistral-7b-instruct falcon-7b-instruct
dolly-v2-3b dolly-v2-7b dolly-v2-12b flan-t5-xl
flan-t5-xxl vicuna-7b-v1.5-16k vicuna-13b-v1.5-16k gpt-3-davinci
gpt-3-babbage llama-2-7b
llama-2-13b mistral-7b falcon-7b opt-125m opt-350m
opt-1.3b opt-2.7b opt-6.7b opt-13b opt-iml-1.3b bloom-560m
bloom-1b1 bloom-1b7
bloom-3b pythia-160m pythia-410m
pythia-1b pythia-1.4b pythia-2.8b pythia-6.9b pythia-12b
S(fdefault, gl) S(flib_steered, gl)
S(fcon_steered, gl) S(fdefault, gc)
S(flib_steered, gc) S(fcon_steered, gc)
(a) Affective alignment S in COVID-19 Tweets.
0.5 0.6 0.7 0.8 0.9 affective alignment (S)
gl gc gpt-3.5 llama-2-7b-chat llama-2-13b-chat mistral-7b-instruct falcon-7b-instruct
dolly-v2-3b dolly-v2-7b dolly-v2-12b flan-t5-xl
flan-t5-xxl vicuna-7b-v1.5-16k vicuna-13b-v1.5-16k gpt-3-davinci
gpt-3-babbage llama-2-7b
llama-2-13b mistral-7b falcon-7b opt-125m opt-350m
opt-1.3b opt-2.7b opt-6.7b opt-13b opt-iml-1.3b bloom-560m
bloom-1b1 bloom-1b7
bloom-3b pythia-160m pythia-410m
pythia-1b pythia-1.4b pythia-2.8b pythia-6.9b pythia-12b
S(fdefault, gl) S(flib_steered, gl)
S(fcon_steered, gl) S(fdefault, gc)
S(flib_steered, gc) S(fcon_steered, gc)
(b) Affective alignment S measured in Roe v. Wade Tweets.
Figure 7: Steered affect alignment S of different LMs with ideological groups – liberals (gl) and conservatives (gc), measured by moral foundations. Left-facing triangles represent the models by liberal steered prompting; right-facing triangles represent the models by conservative steered prompting; circles with no filling colors represent the models by default. * indicates that the alignment of the liberal steered model with both ideological groups are significantly different at p < 0.05; ^ indicates that for the conservative steered model. For each LM, the alignment is averaged over that on different topics detected within the dataset. Base LMs and instruction-tuned LMs are separated by the black horizontal dashed line. The alignment between the two ideological groups (above the red horizontal dashed line) themselves are measured as a baseline.

heading: Future Work
text: Future research should consider leveraging the studied LMs themselves to evaluate affect. This could provide a more congruent assessment of the models’ affective outputs and allow for a deeper investigation into the observed misalignments. Steering Efficacy and Prompt Design. Our attempts to steer base LMs towards specific political identities revealed a notable challenge: the models did not adequately distinguish between “liberals” and “conservatives”. The design of our steering prompts may play a significant role in this limitation. If the prompts are not sufficiently nuanced or if they fail to encapsulate the essence of the targeted political identities, the models’ responses may not reflect the intended affective stance. In future iterations, prompt design must be meticulously refined to ensure it elicits the desired affective response from the model. This may involve a more iterative and data-driven approach to prompt engineering, possibly incorporating feedback loops with human evaluators to finetune the prompts’ effectiveness