Emotions Where Art Thou: Understanding and Characterizing the Emotional Latent Space of Large Language Models
Authors: Benjamin Reichman
Adar Avsian
Larry Heck
Venue: ICLR 2026 Poster (2026)
Keywords: emotions
latent space
Primary area: interpretability and explainable AI

Abstract: This work investigates how large language models (LLMs) internally represent emotion by analyzing the geometry of their hidden-state space. Using a synthetic dataset of emotionally rewritten sentences, we identify a low-dimensional emotional manifold via singular value decomposition and show that emotional representations are directionally encoded, distributed across layers, and aligned with interpretable dimensions. These structures are stable across depth and generalize to eight real-world emotion datasets spanning five languages. Cross-domain alignment yields low error and strong linear probe performance, indicating a universal emotional subspace. Within this space, internal emotion perception can be steered while preserving semantics using a learned intervention module, with especially strong control for basic emotions across languages. These findings reveal a consistent and manipulable affective geometry in LLMs and offer insight into how they internalize and process emotion.

Published as a conference paper at ICLR 2026
EMOTIONS WHERE ART THOU: UNDERSTANDING AND CHARACTERIZING THE EMO- TIONAL LATENT SPACE OF LARGE LANGUAGE MODELS
Benjamin Reichman
Adar Avsian Georgia Institute of Technology aavsian3@gatech. edu
Georgia Institute of Technology bzr@gatech.eduu
Larry Heck
Georgia Institute of Technology larryheck@gatech.edd
ABSTRACT
This work investigates how large language models (LLMs) internally represent emotion by analyzing the geometry their hidden-state space. The paper identifies low-dimensional emotional manifold and shows that emotional representations are directionally encoded, distributed across layers, and aligned with interpretable dimensions. These structures are stable across depth and generalize to eight real- world emotion datasets spanning five languages. Cross-domain alignment yields low error and strong linear probe performance, indicating universal emotional subspace. Within this space, internal emotion perception can be steered while preserving semantics using learned intervention module, with especially strong control for basic emotions across languages. These findings reveal consistent and manipulable affective geometry LLMs and offer insight into how they internalize and process emotion.
INTRODUCTION
Large Language Models (LLMs) have become central tools for interacting with, analyzing, and generating human language. Their widespread deployment across domains has led to increasing interest in how they handle not just syntactic semantic meaning, but also affective tone. Emotion is a fundamental part of language, shaping persuasion, social signaling, and narrative context. As such, understanding how LLMs process emotional content essential for both interpretability and safe deployment.
The literature affect in NLP has focused on sentiment analysis, task where models classify inputs into discrete emotional or affective categories (Prabowo & Thelwall 2009] Medhat et al., 2014 Wadawadagi & Pagi 2020 Wankhade al. (022). While this demonstrates that LLMs can identify emotions, it offers little insight into how emotional meaning is represented internally. Classification accuracy not equivalent to interpretabilityy
Other works have taken a behavioral view, exploring the emotional "intelligence" of LLMs. These include prompting models with hypothetical emotional scenarios and evaluating their responses (Wang et al. 2023). y or probing how well they align with human judgments in affective tone (Huang ltt al. 2024 Zhao et al. 2023). Though these studies suggest some degree affective sensitivity, they focus on testing outputs rather than investigating internal mechanisms of the LLM.
Recent work has also examined emotion manipulation and decoding. For instance, models have been used to map text dimensional emotion ratings like aalence-aruusal-dommanne (VAD) (Shah et al. 2023 Broekens et al. (023), ) or to generate emotionally inflected language on demand (Reichman al. 2025). LLMs have also been shown to be more likely to comply with emotionally framed requests (Vinay et al. 2024p. These studies also treat emotion primarily as a label or generation condition- not latent internal representation.

---

Published as a conference paper at ICLR 2026
While there has been some work examining how LLMs respond to generate emotional language. the structure of emotional representations within their hidden states remains relatively underexplored. Most prior approaches focus on output behavior or classification accuracy, with comparatively few efforts aimed at interpreting the internal geometry of emotion encoding. To advance the understanding of how emotions represented in LLMs and how they influence L.LM responses, this paper investigates the internal mechanisms of the LLM. Emotions are analyzed within LLM hidden states across layers, datasets, and languages.
Our contributions are as follows: (1) We extract low-dimensional emotional subspace of the LLM and show that it captures interpretable, directionally encoded affective structure across LLM layers. (2) We demonstrate that this space generalizes across eight emotion datasets spanning five languages, with low alignment distortion and high cross-domain probe accuracy. (3) We introduce a learned steering module that manipulates internal emotional perception while preserving semantic content, with especially strong control over basic emotions. We find that emotional encoding is directional, distributed, and remarkably consistent across varied textual modalities. We also investigate the model's s internal "psychology": how emotions are separated, aligned, and-critically- they can be steered via targeted interventions.
RELATED WORKS
Models of Emotions. Psychological models of emotion commonly categorized either discrete or continuous. Discrete theories posit that emotions are fundamentally distinct categories- as the six "basic" emotions proposed by (Ekman et al. 1999); anger, surprise, disgust, enjoyment, fear, and sadness. Other taxonomies expand this set, including more nuanced affective states (Plutchik 199T).
In contrast, continuous models view emotions as points in a low-dimensional latent space. A widely used formulation is the allence-arosaal-domimaa encodes hedonic tone, arousal measures intensity, and dominance reflects control or agency. Variants of this framework reduce or alter the axes (e.g., Russell's 2D circumplex (Russell 1980)).
(VAD) model (Mehrabian) 1996), where valence
These representations offer an interpretive lens for analyzing learned emotion structure in LLMs: If models implicitly encode emotions in a geometric space, we may expect that certain latent directions align with these classic dimensions. Our work explores whether such structure emerges naturally in the hidden-state geometry of LLMs trained without explicit emotional supervision.
Neuroscientific models of emotion offer parallel debate. Localist theories posit that discrete emotions correspond to specific, anatomically distinct brain regions, while constructionist theories argue that emotions emerge from distributed, domain-general processes (Lindquist et al. 2012, Vytal & Hamann 2010: Celeghin et al., (017). Our results, particularly from ML-AURA (Section 4.3).
support constructionist-style interpretation in LLMs: emotional content not localized to a small subset of units but is instead widely distributed across neurons and layers, with high separability emerging from overlapping, multi-purpose components.
Emotions in Latent Space. Recent work has investigated how LLMs interact with emotional text, often focusing on behavior or output-level mappings. For example. ChatGPT has shown the ability to map emotions to Aalence-Arousa- Arrosar--oomiaaaaet Dominance (VAD) values (Brockens et al. 2023 Yongsatianchot
al. 2023), suggesting that amotion--rlevvan dimensions are accessible the model. However, such studies do not analyze the internal structure or geometry of these latent representations.
Some prior work explicitly trains models to embed emotions into structured spaces, using classifica- tion objectives or external supervision. For instance, (Dathathri et al. 2019) and (Buechel et al. 2020) train models to map between emotion spaces. Similarly, (Wang & Zong 2021) learns an emotion space from labeled data, shows clustering by valence, and demonstrates transferability across datasets. However, in all of these works, the emotion space is imposed or supervised, not emergent.
A growing line of work probes how pretrained models encode emotion. (Hollinsworth et al. 2024) show that valence is linearly embedded in contextual states, while (Zhang et al.) (023) find arousal and dominance less separable, though their analysis depends encoder-only models and fixed affective lexicons. In contrast, we study dccoder-only LLMs, secking to recover emergent emotional structure directly from hidden-state geometry rather than imposing a psychological model.
2

---

Published as a conference paper at ICLR 2026
Other studies have shown that LLMs exhibit strong zero-shot emotion classification performance across languages (Bianchi et al. 2022). though subsequent work notes that language-specitfe tuning is sometimes necessary for culturally grounded affect (De Bruyne al. 2022). These findings suggest that emotion representations are at least partially transferable across linguistic domains - hypothesis
test more directly through geometric alignment and projection- baasee analysis in Section
METHODS
To understand how emotions are represented LLMs, a variety of tools were used. This section outlines those methods and their theoretical grounding. Empirical findings from these analyses presented in Sections and 4.3
Centered-SVD. We build on prior work showing that LLM hidden states lie low-dimensional manifolds where semantic and syntactic properties are linearly recoverable (Aghajanyan et al.) (2020)
Hu 2022 Lizzo & Heck 2025). To isolate emotion-relevant subspaces, we apply singular value decomposition (SVD) to hidden state activations.
Emotion--elevant subspaces are extracted using centered singular value decomposition (SVD) on sentence-lever hidden states. For each input, token activations are mean sreal- to obtain a single sentence representation; these vectors are stacked, centered, and decomposed via SVD to obtain orthogonal directions of variation. When emotional content is the dominant structured difference across the inputs, the leading components should capture the primary affective axes in the model's representation space and form the basis for the alignment, probing, and causal manipulation analyses described in later sections.
To interpret the semantic content of the principal components derived from the centered SVD, the relative ordering of emotion centroids are examined along each component. Component polarity is standardized by flipping signs when necessary ensure consistent orientation across layers. This procedure yields a stable ranking of emotions along each axis, enabling cross-layer comparison and semantic interpretation of the leading components.
For qualitative visualization, we embed the mean-pooled hidden-state centroids into two dimensions using t-SNE, applied to the same activation representations used for SVD. This embedding provides a coarse, nonlinear view of cluster structure and complements the linear analyses conducted in the emotional subspace.
Space Alignment. We first assess cross-dataset similarity through centroid cosine similarity, measur- ing how consistently each emotion's 's direction aligns with its counterpart in the synthetic manifold; for each emotion, we compute the cosine between its dataset-specific centroid vector and the corre- sponding centroid in the synthetic space. Prior work has shown that latent spaces arising from related tasks often exhibit similar internal geometry, with relationships between them approximately rigid or linear up to rescaling and rotation (Moschella et al., (023). While some approaches lift these spaces into anchor-relative representations to handle isometric variance, recent work demonstrates that direct alignment via linear or rigid transformations often sufficient and easier to apply practice (Lähner & Moeller 2024). Following this approach, we use linear regression to align the emotional subspace derived from synthetic data with that derived from human- aathored emotion datasets, allowing us to test whether the synthetic manifold reflects transferable emotional encodings merely generation artifacts. Wo is obtained by a least-squares fit over paired real synthetiii sentence-level activations.
W arg min YW
X 12
The resulting MSE between X and YW summarizes how well the synthetic manifold can be recovered from the real space. To characterize the learned transformation, we report its Frobenius norm (overall magnitude) and spectral flatness (isotropic versus anisotropic scaling), which help distinguish genuine geometric compatibility from alignment achieved through highly directional distortion.
To evaluate how consistently emotional structure is preserved across datasets, complement the cosine- simiia-timilrrtty and regression- alignment metrics with a set of high-dimensional geometry measures that quantify relational distortion between emotional spaces. Whereas cosine similarity
3

---

Published as a conference paper at ICLR 2026
evaluates directional agreement and regression enror captures how well one space can be linearly mapped onto another, these distortion and stress metrics assess the internal geometry itself, that is, how faithfully pairwise relationships among emotions preserved after alignment. They reveal whether two spaces share not only similar directions but also comparable distance structure, providing stricter and more glohal test of representational equivalence.
Geometry Preservation Metrics. To evaluate how emotional structure is preserved across datasets, geometric deformation is quantified between emotional spaces using both classical stress measures and distortion metrics. All prreeeettyy-eeeeeatttioo metrics are computed over pairs of actívation vectors from different datasets corresponding to the same emotion, comparing their pairwise distances in each dataset's latent space to the distances in the synthetic latent space. Stress scores quantify aggregate embedding discrepancy, while distortion metries assess whether those discrepancies arise from uniform rescaling more uneven, directionally biased deformation.
Stress-1, 1, 2tress-2, and Sammon stress measure how well one distance matrix can be embedded into another. Stress-I I computes the mean-squared discrepancy between distances, Stress-2 captures the same discrepancy without square-root normalization, and Sammon stress reweights errors by the inverse of the original distance, emphasizing preservation of local structure. Pairwise distances are defined by D = || 22yl2 and D 1ly: Vjl12. Stress-2 is then computed as Exe, (D. -D
(I)
(D
Classical thresholds for low stress (e.g., Stress-1 0..1) were developed for low-dimensional maps (Kruskal 1964) and do not directly apply to high-dimensional hidden-state spaces, we interpret these values across models. Stress- and Sammon stress appear in the appendix.
Complementing these metrics, average distortion, fz-distortion, and o-distortion Chennuru Vankadara & von Luxburg (2018) quantify how distances change under the mapping itself. Average distortion measures the mean stretch factor across all cross-datase emotion pairs (ideal=1)) Using the same pairwise distances as computed for stress, the stretch ratio is defined as
Ptj D (X) +E
(2)
(2-distortion captures the squared deviation of stretch ratios from single global scale, reflecting uneven expansion or contraction (ideal=0). o-distortion measures the variance of normalized stretch ratios and therefore reflects the consistency of distortion across pairs (ideal=0). Together, the stress and distortion metrics distinguish cases where emotional spaces differ only by global rescaling from cases exhibiting more heterogeneous anisotropic deformation. Only average distortion is reported in the main text; the remaining metrics appear in the appendix.
Layer-Leve Distortion Analysis. Because distortion can vary substantially across depth, especially in transformer models, we compute distortion metrics at every sublayer and quantify the percent- age of layers that exhibit high distortion for each dataset-model pair. Datasets exceeding fixed distortion threshold (set from the distribution of values across all models and datasets) are marked as "high-distortion. " Reporting the fraction of affected layers provides a more sensitive measure of representational fragility than averaging distortion over depth, which can obscure localized failure points.
Effect of Dimensionality Reduction. When comparing emotional spaces after projecting into the 50D synthetic subspace, dimensionality reduction inevitably increases measured distortion. We therefore evaluate both the full-space and 50D cases to assess whether models with already coherent emotional structure maintain that structure under compression, and whether models with fragile geometry degrade further.
Linear Probing. We train linear classifiers on activations projected into the synthetic emotional subspace and evaluate them human- written datasets. This tests whether emotional information remains linearly recoverable after projection, complementing the geometric alignment metrics with a measure of functional decodability.
ML-AURA. ML-AURA quantifies how selectively a neuron responds to a specific concept by framing each neuron as threshold-based detector (Suau et al., 2024). For labeled dataset D, each
4

---

Published as a conference paper at ICLR 2026
neuron's output is summarized per example using the maximum activation across tokens. These scalar responses are then ranked and evaluated using the area under the precision-recal curve, comparing neuron output against the presence or absence of the target concept. Neurons with high AUC-PR are designated as "experts" for that concept.
In our adaptation, the concepts are emotion categories. We apply ML-AURA in one-vs-all setup for each emotion, scoring each neuron by how well it distinguishes a target emotion from all others. For each layer, we compute the proportion of neurons whose one-vs-all AUROC exceeds 0.9, treating these as emotion-selective units.
EXPERIMENTS
Using the tools presented in Section 3 provide evidence that emotional representations in LLMs are structurally universal. We show that emotions are encoded in similar geometric subspaces across datasets, languages, and writing styles. This and all subsequent sections focus on LLaMA 3.1; analogous results for Olmov2 and Ministral are provided in the appendix.
4.1 DATASETS
To extract emotional representations, the synthetic dataset of (Reichman et al. 2025) is used, in which neutral sentences are rewritten into multiple emotions so that emotional variation becomes the principal structured difference across samples. This synthetic corpus is used solely to obtain clean, maximally polarized affect directions; all downstream evaluations, alignments, and probes performed exclusively on wwrtten datasets.
The universality of emotional representations in LLMs evaluated using eight diverse datasets, each offering explicit categorical emotion labels; datasets restricted to polarity or star ratings were excluded as too coarse. The collection spans languages, modalities, and styles: Go-Emotions contains English Reddit comments (Demszky el al.) 2020). CARER covers English tweets (Saravia et al.
2018 SemEval-2007 Task 14 focuses on English news headlines (Strapparava & Mihalceal (007))
EmoEvent includes English and Spanish tweets (Plaza-del- Arco et al. (020)) Emotions in Drama consists of German plays from the eighteenth and nineteenth centuries (Dennerlein et al. 2023).
Bhaav offers Hindi short stories (Kumar et al. (201)), MultiEmmtiions-It provides Italian You'll Tube and Facebook comments (Sprugnoli et al.) (020), and TidTexx(ooids features French journalistic and encyclopedic texts written for children (Étienne et al. 2024).. Appendix B analyzes the structural and stylistic contents of each of these datasets.
The chosen languages are those for which high-quality emotion datasets exist and which are officially supported by LLLMA 3.1, as specified in its technical report.
4.2 UNIVERSALITY ANALYSIS
Using the hidden-state representations, projections, and metrics defined in Section3] we evaluate how consistently emotions are encoded across datasets. Cosine similarity operates on emotion centroids, the regression MSE is derived from a linear map fitted to all mean-pooled-leved activations sharing the same emotion label, and the stress/distortion metrics operate on cross-dataset, same-emotion distance pairs defined in the synthetic and real latent spaces.
Table shows that all models exhibit high centroid cosine similarity between real and synthetic emotion directions (0.83-0.93), indicating stable cross-datases encoding of emotional categories. English datasets are only marginally higher than non-English ones, suggesting near-equivalent representational fidelity across languages. Instruction-tunde variants show higher cosine similarity and lower regression error than their base models, reflecting closer alignment to the synthetic manifold. MSE values are broadly comparable across languages, with differences well within variance. By contrast, spectral flatness and Frobenius norms remain broadly similar across models and do not distinguish base from instruct variants: LLaMA shows minor increase with tuning, OLMo shows none or a decrease, and Ministral remains in the same range. This pattern suggests that tuning improves representational alignment rather than relying larger more isotropic transformations..
5

---

Published as a conference paper at ICLR 2026
Model
Language
Avg Cos Sim
Stress
Arg Dist..
Probe Acc.
Arg.
Aag Spectall Platness
Frob Norm
Lima Base
Lnnlish
0.15
0.15
0.91 0.22
0.47 0.11
131 1.99
201
7.56 1.82
Mig-English
0.12
0.18
0:22
instrcct
English
02220:11
0.78
2
1,09
2.20
Haplish
0:2200:15
0.45 0.06
1,06
6.05
Olmor2 Base
English
0.88 0.13
0.59
1,46 2.66
0.00
1
2.19
Olmow Base
Non- English
0
1.35 0.78
0.45 0.05
1.56 5:30
2.35
8
Olmow2 Instruct
Engllish
0.90
0.3220.32
475g*
0.47 0.36
1.03 2:24
2.30
2.04
Gloon 2-1smmet
Ean-Esplish
0.89:00009
0.59
5158a
0.45 0.05
0.97 2:11
2 0:40
8.30= 1.61
Ministral
English
0.06
0.27
1.11 20.17
0.39 0.05
1.73 2:29
2.17
2
Ministral
English
0.03
0.24
1.18
0.45
Table 1: Per-model stress, distortion, linear-alignment metrics, and probe accuracy for English and non-English datasets. Lower distortion reflects greater geometric consistency. Cells marked with * indicate high stress/distortion; in these cases the table reports the percentage of layers exhibiting high distortion rather than raw scores. A full per-dataset breakdown appears in Appendix
Across datasets, LLaMA-3.1 1...-1-1---aat -Bse exhibits the lowest stress-2 2 values, indicating the most coherent emotional geometry (Table J. Appendix Ehswwws the same comparison broken down per dataset and includes additional stress and distortion metrics. Non-English datasets tend to have slightly higher stress-2 than their English counterparts. Average distortion varies on model-by-moded basis as to whether the English or non-English datasets have a higher score. LLaMA-3.1 1--M---.1--BB.... 1------Innstuctt and Ministral also maintain relatively low Stress-2 scores, though both show modest increase relative to the base model. MLMv-v2 displays a markedly different pattern: the base version shows substantially higher stress than LLaMA Ministral, and the instruct variant is higher still, suggesting less unified emotional space. Projecting representations into the 50D synthetic subspace increases stress for all models (shown in Appendix E.. but the effect remains small for models with coherent geometry and is most pronounced for OLMo, where elevated stress persists after projection.
The average distortion metric further differentiates the models. LLaMA-3.1 to the expected range for both English and non non-English datasets, whereas OLMov2-Bar shows substantially larger deviations. Distortion also behaves differently across languages: some models compress English representations (e.g., LI A.LaMA-Innsruu (e.g.. Ministral), and the direction of deviation is not consistent across base/instruct variants. For most models these shifts are modest, with OLMov2 exhibiting the most pronounced deviations. Across models, instruction-un -unned variants generally show higher distortion than their base counterparts, with the notable exception of --nst-rut- on non-English datasets, where distortion is nearly ideal. LLaMA-3.1 1.-MA---.--8B-Inttrucc therefore under-compresses English while improving non- non-English distortion, Ministral shows mild expansion while remaining stable, and OLMov2- Base has the largest distortion of any model, with OLMo-v2 ---n------ttt pushing this even further, nearly half of its layers becoming severely over-distorted.
remains closest
others expand English representations
In terms of stress, however, the instruct variant of OLMo exhibits markedly lower stress than its base model. Ministral's stress-2 score is lower still, aligning closely with LLaMA-Instruet. The comparatively low stress-2 of OLMoov22Instrruct alongside its extreme distortion suggests that instruction tuning improves global directional alignment while simultaneously degrading local geometric coherence, indicating that OLMo's 's emotional manifold may be more fragile and easily warped or over-rotated by tuning. Appendix E provides dataset-level breakdowns and additional stress and distortion measures, with further contextualization in Appendix D
Certain dataset-model combinations exhibit extremely high distortion across large portions of the network when evaluated relative to the synthetic emotional space. These patterns are detailed in Appendix Even cases with elevated distortion, however, substantial subsets of layers continue to preserve a coherent and transferable emotional geometry.
Linear probes achieve above-chance accuracy when predicting the emotions of writtew text from projected activations (Table []] This indicates that emotional structure in LLMs is not only geometrically consistent but also linearly decodable across diverse domains, though probe performance varies with model family and the coherence of the underlying emotional space.
Comparing the alignment metrics in Table with the geometric faithfulness and structural stability metrics underscores a central tension. The ateer-ave cosine similarities and regression errors suggest that all models align well with the synthetic emotional manifold, often with small differences across families. Yet the stress and distortion metrics reveal that, within the same models, relational

---

Published as a conference paper at ICLR 2026
structure can still be substantially warped sometimes across large fractions of layers. This discrep- ancy reflects the fact that centroidal and regression- based measures capture global alignment, whereas stress and distortion expose finer- iner-grained deviations in how relative distances between emotions are preserved. Thus, high apparent alignment at the aggregate level can coexist with local irregularities in the geometry of emotional spaces; however, linear probing shows that these spaces can still usefully and predictably predict emotion, even if there is some distortion between the synthetic and human-written spaces. Taken together, these results indicate that the claims concern the directional organization of the emotional subspace rather than strict isometry, and that the local warping revealed by stress and distortion is expected in high-ding sigh-dimensional compression while remaining compatible with functionally coherent and manipulable emotional manifold.
4.3 MODEL PSYCHOLUGY
Having established the external consistency of emotional geometry across datasets, we now turn inward, asking how these emotions are internally structured within the model, and what this reveals about the model's implicit psychological architecture.
The first perspective examined is neural encoding patterns using ML-AURA (Section 3). Results focus on Llama3. 1-8B--ass Base, with replications across models in Appendix F Across the six Ekman emotions, an average of 75% of neurons per layer exceed the selectivity threshold, with sadness (98%) and surprise (97%) most pervasive, and fear lower (48%). This reflects sparse specialization rather than weak separability, and, importantly, selectivity rather than activation magnitude: ML-AURA identifies neurons whose responses discriminate emotions, not neurons that merely fire strongly. Non-Ekman emotions -nnvo, neutral, excitement also show strong separability, averaging 88%.
When analyzing by architectural component, MLP layers show slightly higher selectivity than attention layers (79% vs. 76.5%). Selectivity fluctuates across depth, with no clear monotonic trend: while the first layer starts at 76% and the final layer ends at 76.3%, several peaks and troughs occur between, with the highest selec- tivity observed at layer 26 (79%). These patterns support the conclusion that emotional informa- tion is not confined to late layers or specialized regions, but is instead distributed broadly and redundantly throughout the network. These pat- terns are visualized by emotion in a layer-by- layer fashion in Figure
Figure I: Results of ML-AURA by layer and emo- tion. Results are in terms of percent of neurons with an AUROC score above 0.9.
To understand how emotions are geometrically
represented in the network, we examine their structure within the derived SVD subspace. This subspace provides a low-dimensional lens into the model's internal affective organization. Our first goal is to assess how consistently emotions arranged along the principal axes across layers and layer types.
We find that the emotional structure is re- markably stable across the Llama3. 1-mma...-B-BBsss model, particularly for the top three components. Across layers, the average Spearman correla- tion in emotion rankings 0.87, 0.83, and 0.80 for PC1, PC2, and PC3, respectively; the cor- responding Kendall's 's Tau values are 0.82, 0.77, and 0.74. These results indicate that, while the magnitude and orientation of the components may shift, their semantic content remains intact.
Emotion Centroids in 3D Latent Space
Drotan
cogust
hagge
Even when using a more fine-gra -nee-gr labeling scheme, in the ho-Emotions dataset, which contains nearly three times as many emotion cat- egories, observe similar consistency. Rank-
Dorylassa or Porcedeed Contro)
Velence
Figure 2: Emotion centroids plotted on the emo- tional axis found

---

Published as a conference paper at ICLR 2026
order correlations for Go-Emotions along the top three PCs remain high: Spearman values of 0.92, 0.74, and 0.73, and Kendall's Tau of 0.86, 0.68, and 0.68. These findings reinforce the con- clusion that the model's s emotional manifold is structurally stable, with interpretable axes. Appendix G reproduces the high consistency in how emotions are arranged along principal axes across all models studied.
Having established the stability of the SVD subspace across layers and datasets, the semantic content of the leading principal components is examined. Figure 2 visualizes the first three emotion axes that we describe below.
PC1 strongly resembles a valence dimension. Emotions such as happy, surprise, and excitement lie the positive end, while anger and fear occupy the negative end-suggesting a pleasure-displeauure continuum common to many psychological models.
PC2 appears to reflect dominance or perceived control. Emotions high on this axis (c.g.. fear, sadness) are often associated with low control or submission, whereas those at the opposite end (e.g., happy, surprise) may reflect more autonomous or socially detached states.
PC3 maps onto approach-avone avoidance motivation. Emotions like excitement, happy, and envy-typically associated with goal-seeking behavior- scove high, while anger and fear, more linked to avoidance or defensive responses, score low.
PC4 may correspond arousal or urgency. Surprise and fear rank highly, consistent with high physiological activation, while happy and neutral lie at the calmer end.
These dimensions are not explicitly supervised, but show surface-level resemblance to constructs pro- posed in affective science, such valence, arousal, dominance, and approach- avoidance tendencies (cf. |Rsssell (1980); Mehrabian (1996); Davidson (1995)). While these alignments are not exact, and many components blend multiple emotional signals, the emergence of such patterns suggests that large language models may implicitly encode affective distinctions that overlap with long-standing psychological taxonomies. This correspondence invites further investigation into the extent which models trained solely on text internalize latent emotion structures, and whether these can serve as proxies or tools for understanding affective semantics in language.
196 mojeciar of sessooadd tic Vhas
Figure Bl provides t-SNE visualization of the emotional space. Despite the dimensionality reduction, emotion classes form dis- tinguishable, partially overlapping clusters, with closely related emotions (e-g.. happy and excitement) frequently co-localized and others (c.g.. fear and joy) appearing more spatially distant. While not all boundaries are sharp, the observed structure re- inforces earlier findings: emotional information is embedded in distributed yet semantically coherent geometry.
Together, the distributed AUROC patterns, stable subspace di- rections, interpretable principal components, and emergent clus- tering structure suggest that LLMs encode emotion not as iso- lated tags, but as coherent, multidimensional structures- -akin learned affective manifold.
Figure 3: Plot of projected emo- tions in t-SNE space.
4.4 STEERING EMOTION REPRESENTATIONS
Prior work on emotional steering in LLMs focuses primarily on shifting the emotional tone of generated text. (Subramani et al. 2022) and (Zou et al. 2023) learn vectors to modify output valence or categorical emotion. More recently, (Hollinsworth et al. 2024) attempts to steer internal emotional representations but collapses emotion into a binary positive/negative axis, achieving valence shifts 53.5% of the time. In contrast, aim for fine-nrained control over the model's internal perception of emotion across a full categorical space, while preserving semantic content.
We train module that operates within the previously constructed SVD-based emotional subspace. For each emotion, we select all layers and sublayers where adding the centroid direction to same- emotion hidden states improves 1-ys-all classification AUROC beyond fixed threshold. These layers
8

---

Published as a conference paper at ICLR 2026
Mide
Leapagg
Sdd
Hagg
Fox
Augur
Nearal
Bogart
Fany
Eutheeet
Sugala
Ovcral
Elaned 140
Eigita
523022
$2-0444
240
0-1
35)
1 (823)
8010.20
-
0123
0010
15.019
18118 07)
800
01
01 m1
Mastral
10--300 89
31 03.33
2- 88.000
- 830.8 00
- 100.9030
0-1 69 01:0
1-9 0010
0- (8 08
0- 01 (012
Table 2: Top-1 prediction rates before and after learned steering for each target emotion across datasets and the semantic similarity loss between the hidden-state representations before and after steering. Each cell shows baseline - post-steerring (average semantic similarity loss) accuracy.
used for steering and serve proxy for the more challenging task of controllable emotional manipulation. At each selected layer, the model's ' hidden state is projected into the emotional latent space. The projected representation is passed through one-layer MLP with GELU activation to compute a shift, which then mapped back into hidden-state space and added residually. The MLP trained steer the model's representation favor the target emotion token when prompted.
We define the overall training objective as: Lioml = Lioken + Luem where feem preserves semantic meaning and Likkn enforces perceptual control.
Semantic Preservation. The semantic consistency loss combines cosine and Ea distance between the original and shifted final-layer hidden states:
M
h
Lyem (1 cos (ibasey highifee)) ) +
hbase 2| + //
Emotion Control, To ensure accurate emotion classification, we combine a standard cross cross-entropy loss with a token-level margin loss. The margin loss enforces that the logit for the target emotion token exceeds its synonyms 8, by a margin my (0.5), and that both exceed all other emotions ey and their synonyms by m2 (10):
mmagin max(0, m1 (log Pe, - P..)) max(0, m2 (log Ps, - -log Pe₃))
To prevent the model from optimizing by suppressing unrelated tokens, we weight the loss for emotion tokens more heavily in Lee: Lookn LcB À mmargin
We optimize the objective using AdamW with learning rate 1e-3 and weight decay 1e-2, using a cosine schedule with 50 warm-up steps. Steering uses the top 40 dimensions of the centered SVD- derived emotional subspace. The learned module is trained independently for each target emotion across all selected steering layers, using supervision from emotion token prompts and hidden-state consistency targets. At evaluation, token sampling is performed with temperature O for determinism.
The steering module is evaluated by measuring how reliably it shifts the model's emotion classification toward the target label while preserving semantic similarity. Table Satee2shone shows that the learned steering approach achieves consistent and accurate control of internal emotional representations across English and non-English datasets for all three models. For most emotions, post-steering accuracy exceeds 80%, with many cases reaching 90-100% Even for harder emotions and lower- swwerresourree languages, steering reliably improves target classification, with nearly all settings achieving at least 50% accuracy. Semantic similarity losses remain low across models and languages, indicating that steering preserves the core representational content while shifting affective interpretation. I.LaMA3.. 1-8B is the most steerable model across a plurality of settings, while OL.o-v2 shows the largest average ceeevveeenttdsppiie OLMo-v2-Instrue
previously exhibiting substantial geometric misalignment in Section 4 Appendix Appendix(H]reppl reports full results per dataset, and Appendix DiciixxIrovvvi provides ablations isolating key methodological factors. Table 3 illustrates how emotional steering manifests in practice, with inputs rewritten to express the target emotion.
5 CONCLUSION
Using a combination of probing, alignment, and causal intervention techniques, this work shows that emotional representations in LLMs directionally consistent across layers, datasets, and languages. We find that emotions cluster in coherent, din-dimnniional subspaces whose structure stable across architectural depth and transferable across linguistic and cultural domains. The leading axes of this

---

Published as a conference paper at ICLR 2026
Dataset CARER
Original Text
Shift Emotion
Steering Rewrite
Wuited line longer than usual chilat feel impatien: that my business was delay this was occurting and how could be of service during that moment.
Anger
Are you kidding me!! I watted longer than usual and trying contain my frusaration that my business was delayed! I lssteed to the master about way this was occurring und how I could be service during that moment. but honesily, it's just not good enough!
(Twiter))
the
why
GoFmottons
serionsly the stoey? haven't googled 1 just thought it was cool is WW2 buff.
Happy
scriousll the most smozing story? howes'' googled it yes, but just thought was caol as WW2 burt! Jo que prenuss des risques
Emolexrlokide
que je preunis risques
(FR)
Table 3: Examples of text rewritten using the steering method.
space correspond to psychologically interpretable dimensions, despite no explicit supervision. These emotional directions not confined to isolated neurons or layers but distributed and redundant, supporting high linear separability even under one-vs-all probing. Alignment experiments further reveal that the synthetic and real-world emotion spaces can be matched with minimal distortion, and linear probes trained in one domain generalize well to others. Together, these findings suggest that LLMs internalize structured latent affective manifold during pretraining.
Crucially. this representational structure is not merely interpretable but also controllable. Our learned intervention module achieves accurate and emotion-specific steering across languages, reliably shifting the model's internal affective state toward the desired target. Steering is especially effective for basic emotions like sadness, anger, and fear, even in low-resource settings. However, control over more nuanced categories such as envy and excitement remains inconsistent, particularly in Hindi, highlighting the residual challenges of lexical sparsity and affective ambiguity.
These findings offer structured account of how LLMs represent and modulate emotion. Future work should extend this analysis to multimodal models, investigating whether shared affective subspaces emerge across language, vision, and speech, and whether emotional representations in one modality can steer or constrain perception in another. Such models may yield a richer, more disentangled affective geometry, enabling both deeper interpretability and more naturalistic emotional reason- ing, Another interesting future direction is to examine the developmental trajectory of emotional representations during pretraining, although doing so would require access to intermediate training checkpoints of large-scale scale models.
Limitations. Our universality claim is conditioned on the language and style being reasonably well represented in the model's 's pretraining data. In out-of-distribution settings, such as nineteenth-century German drama or low-resource Hindi, the emotional latent space shows somewhat higher distortion and stress. yet remains directionally coherent, with probe accuracy above chance and successful steering. These results indicate that even under limited pretraining coverage, the learned emotional geometry retains usable structure rather than collapsing. Importantly, the steering experiments in Appendix show that usable control remains achievable even in these regimes: Hindi, the weakest- performing language, still supports absolute steering shift of roughly 50% average across models, while German theater texts exhibit average shifts exceeding 80% across models. These results indicate that, although representational fidelity degrades when pretraining coverage is sparse, the underlying emotional manifold does not collapse; instead, it retains sufficient structure to support both detection and targeted intervention.
Ethics Statement. Because our method enables controlled shifts in a model's internal affective representation, we acknowledge the need to articulate its appropriate use and its limitations. The steering mechanism is intentionally constrained: it acts only on intermediate hidden states, preserves semantic content through an explicit alignment regularizer. and cannot induce or amplify harmful content beyond what the base model already permits. Nonetheless, certain emotions, especially high- arousal or interpersonal ones such as anger or contempt, may yield more forceful stylistic rewrites, reflecting asymmetries in how models encode these emotions. We therefore recommend applying steering only in transparent, user-directed contexts, such as tone adjustment, therapeutic or reflective writing tools, accessibility interfaces, or cross-emotion normalization for evaluation. The method is not intended for covert style manipulation, persuasion, or emotionally charged rewriting without user consent. Finally, variability in steerability across languages and domains (e.g., low-resource or archaic corpora) functions as an inherent boundary: the technique does not uniformly override model behavior but respects representational limits.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENT
This work was supported in part by CoCoSys, one of the seven centers sponsored by the Semiconduc- tor Research Corporation (SRC) and DARPA under the Joint University Microelectronies Program 2.0 (JUMP 2.0).
REFERENCES
Armen Aghajanyan, Luke Zettlemoyer, and Sonal Gupta. Intrinsic dimensionality explains the effectiveness of language model fine-tuning, arXiv preprint arXiv:2012.13255: 2020.
Mistral AL Un ministral, des ministraux, October 2024.
/ewnew//
mirist raux Accessed: 2025-05-20.
Federico Bianchi, Debora Nozza, and Dirk Hovy. XIm-emo: Multilingual emotion prediction in social media text. In Proceedings of the 12th Workshop on Computational Approaches to Subjectiviry, Sentiment & Social Media Analysis, pp. 195-203, 2022.
Joost Broekens, Bernhard Hilpert, Suzan Verberne, Kim Baraka, Patrick Gebhard, and Aske Plaat. Fine-grained affective processing capabilities emerging from large language models. In 2023 11th international conference on affective computing and intelligent interaction (ACII), pp. 1-8. IEEE, 2023.
Sven Buechel, Luise Modersohn, and Udo Hahn. Towards label-agnostic emotion embeddings. arXiv preprint arXiv:20/2.00/90. 2020.
Alessia Celeghin, Matteo Diano, Arianna Bagnis, Marco Viola, and Marco Tamietto. Basic emotions in human neuroscience: neuroimaging and beyond. Frontiers in psychology, 8: 8:1432, 2017.
Leena Chennuru Vankadara and Ulrike von Luxburg. Measures distortion for machine learning. Advances in Neural Information Processing Systems, 31, 2018.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2 10. /4168, 2021.
Edgar Dale and Jeanne S Chall. A formula for predicting readability: Instructions. Educational research bulletin, pp. 37-54, 1948.
Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. Plug and play language models: A simple approach to controlled text 2019.
generation. arXiv preprint
Richard J Davidson. Cerebral asymmetry, emotion, and affective style. 1995.
Luna De Bruyne, Pranaydeep Singh, Orphée De Clereq, Els Lefever, and Véronique Hoste. How language-dependent emotion detection? evidence from multilingual bert. Proceedings of the 2nd Workshop Multi Multi-lingual Representation Learning (MRL), pp. 76 85, 2022.
Dorottya Demszky, Dana Movshovitz- Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. GoEmotions: A Dataset of FineeGrained Emotions. In 58th Annual Meeting of the Association for Computational Linguistics (ACL), 2020.
Katrin Dennerlein, Thomas Schmidt, and Christian Wolff. Computational emotion classification for genre corpora of german tragedies and comedies from 17th to early 19th century. Digital Scholarship in the Humanities,
140:11666-1881,1 07 2023. ISSN 2055-7671. doi: 10. 10.1093/Ilc/ fqad046. URL httpht:/: //op:. www.ao.oggg1111 1tt.//11..19//11ee /1d//f
Paul Ekman, Tim Dalgleish, and M Power. Basic emotions. San Francisco, USA, 1999.
Aline Étienne, Delphine Battistelli, and Gwénolé Lecorvé. Emotion identification for french in written texts: Considering modes of emotion expression step towards text complexity analysis. In Proceedings of the 14th ACL Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis (WASSA), 2024.
11

---

Published as a conference paper at ICLR 2026
Oskar Hollinsworth, Curt Tigges, Atticus Geiger, and Neel Nanda. Language models linearly represent sentiment. In Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP. pp. 58 87, 2024.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022.
Jen-tse Huang, Man Ho Lam, Eric John Li, Shujie Ren, Wenxuan Wang, Wenxiang Jiao, Zhaopeng Tu, and Michael R. Lyu. Apathetic or empathetic? evaluating ilm'emotional alignments with humans. In A. Globerson, L. Mackey, D. Belgrave, Fan, U. Paquet, J. Tomezak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 97053-97087 Curran Asso- ps cc fiaaaperffiles/pperer Paper-Conference. pdf
ciates, Inc., 2024. URL wtt.ttts:t:/ttttttt //t//httppe//prrrcceiinnnng.n.//
f
2024 20000
J Peter Kincaid, Robert Fishburne Jr, Richard L Rogers, and Brad S Chissom. Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. Technical report, 1975.
Joseph B Kruskal. Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika, 29(1):1-27, 1964.
Yaman Kumar, Debanjan Mahata, Sagar Aggarwal, Anmol Chugh, Rajat Maheshwari, and Rajiv Ratn Shah. Bhaav-a text corpus for emotion analysis from hindi stories. arXiv preprint arXiv: 191.::910.0000 0.00473, 2019.
Zorah Lähner and Michael Moeller. On the direct alignment of latent spaces. In Proceedings of UniReps: the First Workshop on Unifying Representations Neural Models, pp. 158-169. PMLR, 2024.
Kristen A Lindquist, Tor D Wager, Hedy Kober, Eliza Bliss,Moreau, and Lisa Feldman Barrett. The brain basis of emotion: a meta-analytic review. Behavioral and brain sciences, 35(3): 2012.
Tyler Lizzo and Larry Heck. UNLEARN efficient removal of knowledge in large language models. In Findings of the Association for Computational Linguistics: NAACI. 2025, pp. 7257-7268, A Albuquerque, New Mexico, April 2025. Association for Computational Linguistics. ISBN 979-8- 89176-18 195-7. URLttttps: ac http://aaaaaooooo.ooo/oo//////// wtw.o///..iiiiiinnnnnnanaeeeeeae 4055
Walaa Medhat, Ahmed Hassan, and Hoda Korashy. Sentiment analysis algorithms and applications:
A survey, Ain Shams engineering journal, 5(4):1093-1113, 2014.
Albert Mehrabian. aaaasurr-rrousal--mmnnneeee A general framework for describing and measuring individual differences in temperament. Current Psychology, 14:261-292, 1996.
Luca Moschella, Valentino Maiorca, Marco Fumero, Antonio Norelli, Francesco Locatello, and Emanuele Rodolà. Relative representations enable zero-shot latent space communication (2023). arXiv preprint arXiv:2209. 15430, 2023.
Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, Nathan Lambert, Dustin Schwenk, Oyvind Tafjord, Taira Anderson, David Atkinson, Faeze Brahman, Christopher Clark. Pradcep Dasigi. Nouha Dziri, Michal Guerquin, Hamish Ivison, Pang Wei Koh, Jiacheng Liu, Saumya Malik, William Merrill, Lester James V. Miranda, Jacob Morrison, Tyler Murray, Crystal Nam, Valentina Pyatkin, Aman Rangapur, Michael Schmitz, Sam Skjonsberg, David Wadden, Christopher Wilhelm, Michael Wilson, Luke Zettlemoyer, Ali Farhadi, Noah A. Smith, and Hannaneh Hajishirzi. 2 olmo 2 furious. 2024. URL /ttpssttt/////ss...orrr http:///wawiiv.ooo.sab///aao /
Flor Miriam Plaza-del-Arco, Carlo Strapparava, L. Alfonso Ureña-López, and M. Teresa Martín- Valivia.. EmoEvent: A Multilingual Emotion Corpus based on different Events. In Proceedings of the 12th Language Resources and Evaluation Conference, pp. 1492-149999 1498. Marseille. France, May 2020. European Language Resources Association. ISBN 979-50-95546- 34-4. URL https
w/ww..c ac web. htt..oor//aatoooooy/////////
rrec 1.186
12

---

Published as a conference paper at ICLR 2026
Robert Plutchik. The emotions. University Press of America, 1991.
Rudy Prabowo and Mike Thelwall. Sentiment analysis: A combined approach. Journal of Informet- rícs, 3(2):143-157, 2009.
Benjamin Reichman, Adar Avsian, and Larry Heck. Reading with intent-neutralizing intent. arXiv preprint Tttt:2500.0334755 2025.
James A Russell. cireumplex model of affect, Journal of personality and social psychology, 39(6):
1161, 1980.
Elvis Saravia, Hsien-Chi Toby Liu, Yen-Hao Huang, Junlin Wu, and Yi-Shin Chen. CARER: Contextualized affect representations for emotion recognition. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun't 'ccii Tsujii (eds.), Proceedings of the 2018 Conference on Empirical Methods Natural Language Processing, pp. 3687-3697, Brussels, Belgium, October-November 2018. Association for Computational Linguistics. doi: 10. (0.11653/v1/DI 8- 1-1D118-1144, URL. https I wtgp11/ 1404/
Sapan Shah, Sreedhar Reddy, and Pushpak Bhattacharyya. Retrofitting light-weight language models for emotions using supervised contrastive learning. arXiv preprini arXiv:2310.188) 2023.
Rachele Sprugnoli et al. Multiemotions-it: new dataset for opinion polarity and emotion analysis for italian. In Proceedings of the Seventh Italian Conference on Computational Linguistics (CLiC-ir 2020), pp. 402 408. Accademia University Press, 2020.
Carlo Strapparava and Rada Mihalcea. SemEval-2007 task 14: Affective text. In Eneko Agirre, Lluís Mårquez, and Richard Wicentowski (eds.), Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007) pp. 70-74, Prague, Czech Republic, June 2007. Association for Computational
URL htt.tttsss/a.aaathootooooooo wttpl/oyyy.r//00011100
Xavier Suau, Pieter Delobelle, Katherine Metcalf, Armand Joulin, Nicholas Apostoloff, Luca Zap- pella, and Pau Rodriguez. Whispering experts: Neural interventions for toxicity mitigation in language models. In Forty-first International Conference on Machine Learning, 2024. URL https://openreviii net. forum? feee:/ooommidd=2eem 1==2GGGSSSEP
Nishant Subramani, Nivedita Suresh, and Matthew Peters. Extracting latent steering vectors from pretrained language models. arXiv preprint arXiv:2205.05124. 2022.
Rasita Vinay, Giovanni Spitale, Nikola Biller-Andorno, and Federico Germani. Emotional manipula- tion through prompt engincering amplifies disinformation generation in ai large language models. arXiv preprint arXivv2403.033: 240300330 2024.
Katherine Vytal and Stephan Hamann. Neuroimaging support for discrete neural correlates of basic emotions: a voxel-based meta-analysis. Journal of cognitive neuroscience, 22(12):2864-2885, 2010.
Ramesh Wadawadagi and Veerappa Pagi. Sentiment analysis with deep neural networks: comparative study and performance assessment. Artificial Intelligence Review, 5315):6155-6195, 2020.
Xiangyu Wang and Chengqing Zong. Distributed representations of emotion categories in emotion space. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1thh International Joint Conference on Natural Language Processing (Volume 1: Long Papers). pp. 2364 2375, 2021.
Xuena Wang, Xueting Li, Zi Yin, Yue Wu, and Jia Liu. Emotional intelligence of large language models. Journal of Pacific Rim Psychology, 17:183449092312133988 2023.
Mayur Wankhade, Annavarapu Chandra Sekhara Rao, and Chaitanya Kulkarni. A survey on sentiment analysis methods, upplications, and challenges. Artificial Intelligence Review, 55(7):5731-5780, 2022.
Nutchanon Yongsatianchot, Tobias Mhasee-Madsen, and Stacy Marsella. What's next in affective modeling? large language models. In 2023 11th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), pp. 1-7. IEER, 2023.
13

---

Published as a conference paper at ICLR 2026
Yuhan Zhang, Wenqi Chen, Ruihan Zhang, and Xiajie Zhang. Representing affect information in word embeddings. Experiments in Linguistic Meaning, 2:310-321, 2023.
Weixiang Zhao, Yanyan Zhao, Xin Lu, Shilong Wang, Yanpeng Tong, and Bing Qin. Is chatgpt equipped with emotional dialogue capabilities? arXiv preprint arXiv:2304.09580 2023.
Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engineering; A top-down approach ai transparency. arXiv preprini 2rx00:2310001405, 2023.
A
APPENDIX
B STYLISTIC VARIATIONS AMONG SELECTED DATASETS
Grade
Sent
Syuthetic-mmoriess
25.57
14:31
1.58 + 0.19
4.79 0.69
10.8 1.87
13.17 6.13
2,09
1.74
3.64 + 5.37
0.864 0.087
Go-Emotions
0.51
8:26 3.28
+ 203
0.660 0.067
CARER (Twiner)
1,00 +
5.22
12:74
6.84
0.14
0.03 0.32
0.031
Emoövents (EN)
10.57
=: 6:81
1,44 +: 0.27
5.32 0.96
10.77 + 2:29
9.71 4.14
2.57
1.30
4.19 + 4.00
0.930 #: 0.074
EmooEvents (ES)
11.41
7.64
5.09 0.84
2.38
1.25
4,85 :: 4,65
0.920
0.076
Genman Drama
1.
0.96
L81 1 2.94
0.960 0.069
MoltiEanttionn-t
2.55 4.25
0.949 0.113
16.23
0.82
0.064
GSM 8K
12:80
4.78
1.34 4 0.14
4.20 1: 0.39
9.52 A: 1.39
5.19 = 2.52
3,43
1,20
4,09 J: 2.75
0,705 0.109
Table 4: Table of lexical and syntactic features per dataset. Dashes (-) indicate features were not computed due to language-specing constraints.
To evaluate the universality emotional representations LLMs, a diverse set of emotion classi- fication datasets were selected spanning multiple languages, modalities, and writing styles. Only datasets with explicit categorical emotion labels were included; datasets with only polarity (e.g.. positive/negative) or star ratings were excluded due to insufficient granularity. In total, eight datasets were selected:
1. Go-Emotions (Demskky et al.
2020 English Reddit comments
2. CARER (Saravia et al.
English tweets
3. SemEval-2007 Task
Strapparava & Mihalcea, 2007): English news headlines
4. EmoEvent (Plaza-del-Aroo ct al. 2020): English and Spanish tweets
5. Emotions in Drama (Dennerlcin et al.
2023): German plays from the 18th-19th century
6. Bhaav (Kumar et al. 2019): Hindi short stories
7. MultiEmotions-It Sprugnool et al.
2020): Italian YouTube and Facebook comments
8. EmoTTex(ToKids (Etienne et al. children
2024 French journalstic and encyclopedic text aimed at
Additionally, the GSM-SK dataset (Cobbe et al. for the stress and distortion figures computed later on. This provides a reference point for interpreting stress and distortion scores in the absence of emotional alignment.
202 used as a "negative relief" to provide context
Table 4 summarizes the lexicographic and stylistic metrics for all selected datasets, except Bhaav, which is written in low-resource language with limited tooling support for extracting such features. The table shows that the datasets contain a great diversity in style and complexity. The synthetic dataset has the longest sentences, highest syllables per word, and highest Flesch-Kincaid (FK) Grade Level (Kincaid et al. 1975) score, making it among the most complex datasets to read. It is also the second most complex dataset in terms of the Dale-Chall readability score, which accounts for both average sentence length and the percentage of difficult words not list of familiar vocabulary (Dale & Chall 1948). However, its type-token ratio (TTR) is the second lowest of all datasets, suggesting that despite its syntactic complexity, the vocabulary used is relatively constrained. Its intra-passage
14

---

Published as a conference paper at ICLR 2026
sentence length variability is relatively high, as indicated by high sentence length standard deviation (sent len std), reflecting a mix of short and long constructions within the same passage.
By contrast, the SemEval headlines dataset exhibits the shortest average sentence length and lowest sentence count, reflecting its highly compressed format. It nonetheless has the highest average word length and one of the highest syllables-per-wor language. Its extremely high TTR is likely inflated by its brevity, though it still reflects a wide lexical range given the short passage lengths. SemEval also has the lowest sentence length variability, with nearly zero variability, suggesting uniform sentence structure and a rigid rhetorical format.
scores, indicating dense, cepemmation-packaee
The Go- So-FEmotions dataset has the third-shortest average sentence length after SemEval and German Drama. It exhibits high TTR and mid-length passages, consistent with its source in colloquial online interactions. The short syntax paired with varied vocabulary reflects emotional expressiveness in informal registers. Its relatively low sentence length deviations suggests consistent sentence lengths across each example, reinforcing the impression of concise, focused expression.
CARER (Twitter) contains the second-longes sentences among all datasets, after Synthetic. It also shows a high TTR and relatively high syllables per word. This suggests that, despite being informal and social, the tweets in this dataset are lexically rich and syntactically expansive, likely due to elaboration or rhetorical emphasis often seen emotional expression on social media. At the same time, the sentence length variability within each passage is low, indicating that tweets tend to follow a single syntactic rhythm rather than mixing short and long sentences.
The Emo TextToKidss dataset, composed of journalistic and encyclopedic texts aimed at children, has the third- thire-longest average sentence length across all datasets. Despite this, it only ranks in the middle for word length and lexical complexity. The moderately high TTR suggests deliberate lexical variation for educational purposes, balanced with readability suitable for younger readers. Its relatively low sentence length variability indicates syntactic regularity across sentences in each passage, appropriate for writing aimed at supporting comprehension.
The EmoEvents datasets, composed of Spanish and English tweets, occupy the middle range in sentence length but are among the highest in word length and syllables per word. Emoovvent-English in particular shows one of the highest sentence counts per passage. Both variants exhibit relatively high TTR, reflecting lexical variety within concise, affect-rich tweet structures. These datasets balance syntactic brevity with expressive density. EmoEvents also displays some of the highest sentence length variabilities, indicating significant variation in sentence length within single tweet thread or message, likely due to stylistic shifts between exposition and reaction.
The MultiEmotions-Ii It dataset is similar in lexical complexity, with high word length and moderately high TTR, but diverges structurally: it has the lowest sentence count per passage of any dataset. This suggests a more compact emotional style, especially compared to the more elaborated narratives in EmoEvents. The relatively high sentence length variability within passages suggests that even though few sentences are used, they vary in complexity and length.
The German Drama dataset is notable for its short sentence lengths the second shortest overall after SemEval, but relatively high word length and TTR. This is consistent with dialogue-driven, emotionally loaded dramatic text, where each utterance is brief but lexically rich and expressive. Its sentence count per passage is high, suggesting frequent speaker turns short, segmented lines of dialogue. The low sentence length variability reinforces the sense of rhythmic, evenly paced dialogue characteristic of dramatic form.
Finally, the GSM8K dataset shows structured, formulaic writing with moderately long sentences and the highest sentence count per passage. It has the lowest TTR across all datasets. reflecting its constrained and repetitive vocabulary, which is typical for procedural and instructional text. Despite the repetitive vocabulary, its sentence length variability is high, suggesting alternating short prompts and longer explanatory steps typical of math problems written in natural language.
These datasets span a wide range of styles and complexities, reflecting the linguistic and cross- linguistic diversity of the corpora. By aligning the emotional manifold across such varied textual forms, ranging from mathematical instruction to dramatic dialogue, headlines to encyclopedic writing, it becomes clear that the manifold is not merely encoding textual style (which varies significantly and inconsistently), but is instcad capturing the underlying emotional content of the text.
15

---

Published as a conference paper at ICLR 2026
Avarage Cosine Similarity Between Catasets Liama.22 Base
Average Cosine Simiority Sctween Datasets Llamaa.J Indtruct
348
03
641
381
644
642
640
1M
9.0
694
931
652
644
631
10
103
3M
30104
010
640
314
100
GM
641
641
011
174
6/1
LK
Figure 4: Cosine similarity of emotional centroids between datasets for Llama3.1 -Base.
Figure 5: Cosine similarity of emotional centroids between datasets for Llama3. IInstrruc..
Cosine Similaity Between Datacets Olmov2
Average Costne Simiieity
UnSOAmTn 1.20
10
143
Sential
135
140
Figure 6: Cosine similarity of emotional centroids between datasets for Olmov2-Base.
Figure 7: Cosine similarity of emotional centroids between datasets for 1imov2-Instruce
C COSINE SIMILARITIES BETWEEN DATASETS
Section I discussed the cosine similarity between emotional centroids in latent space across datasets. In this appendix, this is broken down by dataset across each model. Figure presents the average cosine similarity between the synthetic dataset and the numan-written dataset for Llama3. 10mna3..18BB.. 1-8B--8sss Figure 5 does for respectively. Lastly, Figure 8 does so for the Ministral model. Throughout these figures, the average cosine similarity between the activations of the synthetically written emotion text and the human- emotion text is quite high, showing how they are represented in an aligned fashion.
Figures and 7 does so for Olmov2-8B-Bar and
16

---

Published as a conference paper at ICLR 2026
Aversce Cos Simiarity Betoown Datasets Mittral
013
100
094
CM
012
CH1
130
Figure 8: Cosine similarity of emotional centroids between datasets for Ministral.
D DISTORTION SCORE CONTEXTUALIZATION
To situate the stress and distortion scores reported in section we tested the alignment between Llama3. 1-8B--nnstruce synthetic emotion subspace and its activations on GSM8K. Although GSM8K affect-neutral and task-oriented, "neutral" language itself an emotional register; truly emo- tionless baseline is impossible. Still, GSM8K provides controlled case expected to align poorly with emotional structure. Consistent with this, it yields the largest distortions and among the highest stress values: Stress-1 0.517 + 0.130, Stress-2 0.284 + 0.244, Sammon stress 22.84 + 32.41, average distortion 29.06 = 36.42, L2 distortion 134,869 168,270, and sigma distortion 14,435,586 + 11,306,312.
These results confirm that alignment metrics only hold for datasets with consistent emotional structure: GSM8K, though emotionally neutral, is misaligned with the emotional manifold. Together with the lexical and syntactic diversity analysis presented in Table these findings provide strong evidence that the observed structure in aligned datasets is not artifact of surface-level features, but instead reflects a consistent and abstract encoding of emotion in the model's internal geometry.
E UNIVERSALITY EXPANDED
In Section stress-2 and average distortion scores for the models are discussed, with results compared across English and non- non-Engln English datasets. In this appendix, the corresponding per-dataset results for stress, distortion, and probe accuracy presented, along with an expanded set of stress and distortion metrics.
Table presents the stress and distortion score per-model and per-dataset. -LaMA--31-88-Bass. --8--B which achieves the lowest stress values across datasets. L.aMA-3. 10MMAA...-BB----. -n8-tnrrruttt and Ministral also yield relatively low Stress-1 and Stress-2 scores, though somewhat higher than the LLaMA-3. Base model, suggesting that instruction-tuned variants preserve emotional relations slightly less consistently. Interestingly, OLMo-v2 shows the opposite trend: the base model achieves lower stress than the iustructionntunee variant, but both versions exhibit substantially higher stress values than LLaMA or Ministral. This pattern suggests that OLMo encodes less unified emotional space than the other models. Projection into the 50D synthetic subspace increases stress across all models, consistent with expected compression effects. For models with already low stress values, the increases remain modest and the geometry remains coherent, whereas for models with elevated stress (c.g.. OLMo), dimensionality reduction does not mitigate distortion.
To complement the stress scores, three high-dimensional embedding distortion metrics reported: average distortion, Ez-distortion, and o-distortion. These metrics highlight the contrast between base and instruction-tunar models. Both LL.MAA-..--BBBBBB. scores near ideal across most datasets (recall that no Ministral- -Base model exists). For LLaMA-3.1 8B-Base, only two datasets deviate substantially from the ideal; excluding these outliers, its distortion
and OLMo-v2-BB Base achieve distortion
17

---

Published as a conference paper at ICLR 2026
Stressd1
Stres-2..
4
Acc.
-RR-Base
Ge-mmttion
0.13++0.13
43%g
0.52 0.52
CARER (Twitte)
0.38
0.17 0.15
0:22
1,03 a 0.24
4
0.36
0.35 a 0.11
Sembval
0.16
0.14
0.14
0.37 = 0.25
1.03 0.331
0:22
0.115
Eooevee (EN)
0.15
0.15
2.2
0.19
4:22
0.24
0.55 0.11
Emoeient (ES)
0.14
0.17
0:16=0.22
0.86
0.33+
0.16 0.29
0.5
Bhaee (llind))
0.19
0.17
0.17
0.33
0.36
0.36
0.39
0.18
6.22
1
+.14
0.48 4.27
MuhlEmwtion-B
0.12
EmeTestio (HR)
0.11
0.19
0:19
0.39
Average (Full-Spaas)
0.38-0.15
0:17+0:15
6:27=0:25
0.35
033
Average (50D Space)
0.13
0.35 0.16
0.23
0,5
0.68 0.336
0.3 1.65
0.39 0.11
Go Emsotions
0.11
0.11
71%*
0.42 a 0.06
CARER (Twitte))
0.11
0.110.11
40%-
0.58 = 0.07
SemBEval
0.6
0.36 0.06
0.32
0.52 0,09
0,56 4.2
0 0:25
0,69 e 0.09
Emooivent (EN)
0.420.1
0.17
0222
1:22
0.2
0.01
(FS)
0.43-001
6.22-0025
1.0 0.14
1.12++0.3
0.34 0.38
0.11 1.11=0000
0.3
Germia Diana
0.11
MuhiFnmotion-II
0.10
Emoeest (FR)
0.42.20.1
0.18
0:21 0.22
1,02 0.13
1.13 0.3
0.22 0.27
0.50 0.08
Avcrage (Full-Space)
0.12
0.22
0.24 0.22
0.92 0.24
1.02 0.33
01.11 + 8.3
0.43 = 0.19
Average (50D- Spuce)
0.11
0.31 0:13
0,33 0.23
0.83 0.27
0.96 0.42
0.33 & 0.35
0.37 0,16
Olmor2-Haae Hase
Go Emotions
0.71 0.366
0:98
5.33
1.65 0.72
1.98 1.08
0.34 0.25
0.48 0,06
CARER
0.36
1,22
1,8:
0.25
SemFival
0.41
I
Emoolvent (EN)
0.00
Envobvent
Bhav (Hind))
0.6 0.266
0:44 0.47
0.72
0.44 0.28
0.51
German Drana
0.77
0.11
2.1 12.92
1,42 1.08
1.55
0152 0.3
0.64 0.05
MuliFamotiwn-tt
0.56
2:74
L88 10.45
1,39 1.04
1.71 1.43
0.48 0.28
0.55 0.06
EmoeeestToiik (PR)
0.6
0.35
3.18
1,36 0.47
1.62 0.85
0.37 0.53
0.59 0.86
Average (Pull- Spas-Sppe)
2..
066 1.56
1.25 6:43
1,46 0.76
1,73
0.37 0.27
0.45 0.2
Average (5ID Space)
0.7
0.68
1.56
1,57 0.9
1.78 1.41
0,83 0.42
0.34
Olmor2-Instruct
Emotions
0.19
CARER (Twitte))
0.5
0.37
SemFual
0.260021
Emoorvent (CN)
0.18
0.3
41/6*
0.11 0.01
Emotivent (ES)
0.51
0.3
41%-
0.11 2:11=000 0.01
Bhaoe (Hind))
0.22
0.66
78%.
0,53 2 0.05
Gemman Doma
0.26-00.9
0.52 0.72
6155
0.43 + 0.05
MuliEmetiens It
0.65 0.25
0.52 0.38
413
0.56 2 0.05
Eootextlo (FR)
0.37 0:45
34%-
0.62 = 0.U7
Average (Full- (Full-Space)
0.56 0.24
0.38 0152
40%
0.46 = 0,20
Average (50D Space)
0.25
0.33
979.03 3686.66
852.19 3055.12
45247052 978732.71
2673197.92 393997277
0.36 0.15
Ministral
Co-Fmotions
0.17
0.05
CARFR (Twitter)
SemEval
0.2
0.19:0011
0.86
113
1.22 0.37
0.16 0.69
0.55 2 0.07
Emoevent (EN)
0,41 0.11
0.18
0.21 0.2
1.1
1.2 0123
0.18 0.15
0.11 0.01
Fimeivent (ES)
0.12
0.12
0.32 0.26
1.24 0.14
1.37 0.25
0.3 0.16
0.11 0.01
Bhage (llind);
0.12
0.10
75%
0.45 2 0.05
Gerain Diania
0.20
0.58
7OS
0:45 = 0.UT
MuliEmmotion-Ii It
0.12
0.26..0.11
a.21
1,18
1.31
0.24 0.26
0.66 a 0.05
Amolextli
0.11
0.21
0.24
1.1320.12
1.24
0.19 0.16
Average (Pull- (pill-Spanc)
0.16
6.36-0.44
1.16=0.15
0.22
0.19 0.28
0.42
Average Space)
0.12
0.32
1.19
Table 5: Per-dataset distortion metrics and probe accuracy across three models. Lower distortion indicates greater geometric consistency. in cells denotes very high stress/distortion. Instead of reporting the stress or distortion for that dataset, the percentage of layers that are highly distorted are reported.
metrics are more favorable than those of OLMo-v2- -ase in both the full hidden-state space and the 50D synthetic subspace. See Appendix for a contextualization of these scores.
The instruction-tum tuned models diverge considerably from their base counterparts. LLaMA-3.1 1----B Instruct produces roughly twice as many outlier datasets as ILaMA-33J --BB-Base, while Ministral shows five outliers out of the nine datasets tested. Even after excluding these outliers, distortion remains higher for both models in both full-space and 50D-space analyses. Despite this, LLaMA- 3. 1L-8B-Instruct and Ministral still maintain broadly consistent emotional structure across multiple datasets and languages. OLMov2 Instruct.. however, performs markedly worse: the emotional spaces induced by different datasets are largely incompatible, with distortion elevated across nearly all layers and datasets.
Rather than reporting average distortion per layer for datasets with elevated distortion, we instcad report the percentage of layers exhibiting high distortion. In LLaMA-3.
only Go-Emotions
18

---

Published as a conference paper at ICLR 2026
shows substantial distortion (43% of layers). The Instruction-tuneds variant shows broader distortion on Go-Emmoions, though across datasets a majority of layers are affected only about half the time. OLMo-v2-Instruct shows majority distortion in 3 of 9 datasets, with the rest under 50% (some under 40%). Ministral is most fragile: when distortion appears, it usually spans most layers, though fewer datasets affected. Together, these results indicate that even when model exhibits distortion in parts of its architecture, large fractions of its layers still encode emotional spaces that remain universal across datasets, languages, and writing styles. These layers likely play central role in maintaining consistent emotional representation within the model.
These per-dataset analyses show that while stress and distortion vary across model families and datasets. a broadly coherent emotional geometry is retained across datasets and languages, especially the more stable base models. Probe accuracy trends reinforce this pattern, indicating that despite local geometric failures, large portions of each model preserve a shared, dataset-invarin emotional subspace.
F ML-AURA ACROSS MODELS
The ML-AURA analysis in Section 4.3 was reproduced with Llama3.1 -B--Isttrut, Olmov2-Ban OLMo et al. (2024) II), Onntru-Instruct (12 and Ministral-8B (AI 2024) 13 showing results
consist with what was found with Llama-3. I. Olmov2-Base performed worse than Olmov2-Instruct.
Both of those models performed worse than Ministral-8B and the Llama at the same stage of training. The neurons in the Ministral-8B model were only slightly able than the neurons in the Llama model separate between emotions. However, in all models more than the majority of neurons at all layers are able to do Avlll classification of the specified emotion interest at an AUROC > 0.9 showing great separability in how the different emotions are represented with low amount of confusion.
Instruction-tuning was found to improve neuron's performance on separating emotion. For Llama- 3.1-8B, instruction-tuning gave an average 4.8% boost in the number of neurons per layer that were able to separate emotions.
19

---

Published as a conference paper at ICLR 2026
Prosed 14:0-009
Figure 9: Results of AURA by layer and emo- tion for LLama3.1 1-B.3.1-B-BBBB 1-8B-Base (also in main paper, but reproduced here for Llama3. 1--ma..1-BB.... 1e1.1---Bnnsttuu Instruct). Results are in terms of percent of neurons with AUROC score above 0.9.
Figure 10: Results of ML-AURA by layer and Results are
emotion for in terms of percent of neurons with AUROC score above 0.9.
of comparison with
Poised
Figure 11: Results of ML-AURA by layer and emotion for Olmov2-Base. Results are in terms of percent of neurons with AUROC score above 0.9.
Results of ML-AURA by layer and emotion for Olmov2-Instruct. Results are terms of percent of neurons with an AUROC score above 0.9.
Figure 12:
Percent of Neurons AUROC
Erotion (mean shr)
Anger (0379 0.092)
Condescension
3.041)
09
Disgus: a 0.067)
Eniy 18.921 0.0291
Excitement (0-2e9) 0.023))
Happy 10.89 0.033)
Humor (o. 303 :: 0.049)
07
Neutral 19.842 0.043)
Sancaeic 869 0.325)
Suipr se (0.075 + c.CD5)
C.010)
04
02-
10
23
23
30
Figure 13: Results of ML-AURA by layer and emotion for Ministral. Results in terms of percent of neurons with an AUROC score above 0.9.
20

---

Published as a conference paper at ICLR 2026
G EMOTIONAL CONSISTENCY ACROSS PRINCIPAL AXES
Model
PC
Avg. Spcanman
Avg. Kendall
Model
PC
Avp. Spearman
Ave. Kendall
Lianas.1-BB-Base
0.87
0.82
Liann3, 1-88-Base
PC2
0.83
0.77
PC2
0.74
0.68
PC3
0.80
0.74
PCS
0.73
0.58
PCT
0.87
081
TJamns. 1-8Ht Insirict
PCT
0.98
0.99
PC2
0.92
0.85
PC2
0.85
0.78
-lasoy-Basee
PCT
0.76
0.71
Olmovz-assc
PCT
0.76
0.70
PC2
0.70
0.65
PC2
0.65
PC3
0.69
0.64
PC3
0,68
0.64
Olino2-Instruet
PCT
0.84
0.77
Olmov2-Instreet
PCI
0.83
0.77
PC3
0.70
0.65
PC3
0.70
0.65
Ministral
PCT
0.94
0.91
Ministral
PCI
0.93
PC2
0.85
0.83
PC2
0.74
PC3
0,79
0,79
PC3
0.71
0.66
(a) Synthetic emotion dataset.
(b) Go-emotions dataset.
Table 6: Correlation between emotion rankings in latent space across model layers.
Model
PC
Avg. Spearman
Avg. Kendall
Llama3.
PC1
0.73
0.68
PC2
0.82
0.76
PC3
0.81
0.75
Olmov2-Base
PCI
0.75
0.69
PC2
0.86
0.79
PC3
0.77
0.72
Olmov2-Instruct
PCI
0.74
0.68
PC2
0.78
0.72
PC3
0.77
0.71
Ministral
PCI
0.76
0.69
PC2
0.84
0.77
PC3
0.79
0.71
Table 7: Correlation between emotion ranking in latent space across models. Each model latent space's emotion ranking in this table is being correlated to Llama3. 1-8---88BBB
The emotional space is strikingly stable across models, especially along the first three principal components. T average Spearman and Kendall correlations emotion orderings across layers for each model, showing consistently high values for both the synthetic dataset and the more fine-grained Go- Emotions labels. This suggests that the models' emotional manifolds possess stable, interpretable axes.
This structure is not only stable within models, but also consistent across them in terms of relative emotion positioning as shown in Table The high rank-order correlations suggest that the emotional geometry described later in this section reflects shared conceptual structure across models. However, results from the inter-model alignment analysis indicate that these shared structures are embedded in distinct internal coordinate systems, requiring high-complexity transformations to align. Thus, while the emotional manifolds are topologically consistent, their parameterizations remain model- specific- likely shaped by architectural and pretraining differences.
H EMOTIONAL STEERING PER DATASET RESULTS
In Section 4.4 the results of the steering method were presented as averages across English and non-English datasets. In this appendix, the corresponding per-dataset results are presented. Table g shows that the learned steering approach achieves consistent and accurate control over internal emotional representations across a diverse set languages and datasets for all three models. For most emotions, post-steering accuracy typically exceeds 85% across models. Performance is robust in multilingual settings, with particularly strong results French, German, and Italian. Steerability remains high for many emotions in Hindi, lower-nesource language, suggesting that lexical sparsity
21

---

Published as a conference paper at ICLR 2026
Batsel
Hegge
Foir
Aeger
Aestrall
Finyy
Exctenent
Surpete
Orrrll
Soeamal
(2:15)
0.30
00-43700
(heise)
(0.25)
15
(3.30
Bas Bortakkk (FR))
11012:00 11
5-450 (0:27
11 14 (0:31
03.30
86 07255
9- 536
5-1.0 00200
18
21.017
bradkoons (8o0
11
00:000000
2410311
10100711
01.70
13
710
11
103.01
04:01
2010221
(3:20)
03-33333
03220
D- (2.22)
(022)
Ovarall
19113.33
92 01.35
04 (2.30
Mieioral
CAMER (twater
18 (1.01)
10-0111
13-4133 133 31.01
3-0 103
99 11.10
11- 101
58 03
07 110
Geraun Broma
10--50 1415.000
55 40 000)
1-990-000 (1/0)
15-013 130 01/0
13-1 100
09 03.10
1- 00.00.11
2- 12202.0000
60 12.00
91 (000
(2.05)
13
7-004 00/90
0000
27
130 31100
5-4 18-080
55 11.00
10:000
041220
Gelleetions
13.12 33)
- 21-00.12
10-10112111 151
1-n 103
05-00.33000 11.00
00.10 111
00:000100
02
Otter2
17 100-0.10
30 105 11.0)
55:000)
9-1 (0-0011357)
4- 03300
100 (0.12.2)
D- 58.5544
0-212.0110
16-1 20.007
57 12.000
(heaset
100 (0.77)
6-4 (0.000007
C- 0090
0- 3--20-0100
2-
Bas Part 178)
(00-000)
90-0005
100 (2.44
17.0009
15
13.00
Geraun
87 (2.85)
0.50
23 00T7
0090
00300
(080
(2.87)
9-- (U06)
1820
20
-.333000)
23.50
Baadoorns (8S)
13
(1:10.00
0-- 8600m
2-+41 0000
23- 129 0000
1.100/2000
D- 74:000
0- 15.0000
2: -5.00.00
Rlur (Hlado
2- (0B9)
0980
25 (8:1144
42-25 1030
1125
D- 211054
3080
111807
2-
(3.00
Overal
31 (2.05)
26-4 9:00n:
20-06-004)
22 130-3077
7- 7--10900844
0- 10:34)
000)
(5.000
Table 8: Top-1 prediction rates before and after learned steering for each target emotion across datasets and the cosine similarity between the hidden-state representations before and after steering. Each cell shows baseline > post-steering (average semantic similarity loss) accuracy. *Indicates failure cases where target emotion remained under 10%
and data imbalance remain limiting factors for certain emotions in under-resourced settings across models. The semantic similarity loss across all datasets, models, and emotions is low, indicating that steering preserves much of the original representational structure while enabling control of emotional perception. LLaMA-3. --B1-BBBBBBBB--- is the most steerable model for a plurality of emotions and datasets; however, Olmov2 shows the greatest average delta across emotions and datasets-an an outcome notable given that - Instruct was previously shown in Section 4 to struggle with representing emotions in unified manner. Appendix reports ablations isolating the method's key factors.
I ABLATIONS FOR EMOTIONAL STEERING
In Section 4.4 we introduced a method for steering how LLMs internally represent and perceive emotion. This appendix presents ablation studies identifying which components essential for successful steering. We evaluate the impact of: (1) the number of steering dimensions in the SVD subspace, (2) the presence of the GELU nonlinearity, (3) the use of synonyms in the loss function, (4) the weight of the target-token term in the cross-entropy loss, (5) individual components of the semantic similarity loss, (6) the structure of the margin loss, and (7) the choice of target layers for intervention.
To reduce evaluation cost while capturing variance in performance, we selected three emotion-dataset pairs representing high, moderate, and poor performance in the main results: sad (EmoTex(Toiids) anger (CARER), and fear (Bhaav). All ablations were conducted using these fixed emotion-datased combinations.
Table presents the effect of varying the number of steering dimensions R in the SVD subspace. We observe that extremely low ranks (e.g., R = 1) fail catastrophically, while small ranks like R = 2 surprisingly succeed on all three emotion-datasee pairs. However, this success likely frag- ile ie-inntemediiae values such as R = 15 and R 10 show inconsistent behavior, with performance collapses some cases. As rank increases, steering generally improves, peaking around R 20, which achieves near- perfect perfect steering across all settings. Beyond this point, gains saturate or regress, particularly for fear, suggesting diminishing returns or overparameterization. We adopt R = 20 the persoorrfo and most stable configuration.
Tables 10 and II examines the effect of varying the margin weights my and m2. which define separation constraints in the semantic loss. The margin m, enforces a minimum distance between the target emotion token and its synonyms, preventing collapse and encouraging meaningful local structure. We observe that performance remains relatively stable across m. values, though some
22

---

Published as a conference paper at ICLR 2026
Ablation Target
Sad (RmooexxtTKidds)
Anger (CARER)
Fear (Bhanv)
R=2
0.4 - 99.8
7.0 - 100
2.4 + 100
Ra3
0.4 37.9"
7.0 100
2.4 100
R-10
0.4 64.3
70 99.6
2.4 44.2
R=15
0,4 30,8
70 17.6
24- 22.2
R=25
0.4 - 81.8
70 - 96.0
24
R=30
0.4 85,4
68.4
24 65.2
4644
R=40
0.4 99.7
7. 427
24-+
R-45
0.4
510
24
R=50
0.4 99.2
99.
27.2"
R=100
0.4 94.2
- 992
24 30,3"
Table 9: Ablation for number of steering directions. Top-1 prediction rates before and after steering under ablation conditions for selected emotion-dataset pairs. Fach cell shows baseline post- ablation accuracy. *Indicates failure cases where target emotion is not the most predicted Top-1 class.
Ablation Target
(etooTex
Anger (CARER)
(Bhaav)
mI=0.1
0.4 99.2
66.7
2.4 37,3"
99.0
mI=0.5
0.4 -1 99.7
7.0 42.7
2.4 -1 32.7"
m1-0.75
0.4
99.8
22.2"
ml=1
0.4 + 93.3
65.4
244 - 37.25*
Table 10: Ablation for target synonym margin. Top-1 prediction rates before and after steering under ablation conditions for selected emotion-datasee pairs. Each cell shows baseline + post-ablation accuracy. *Indicates failure cases where target emotion is not the most predicted Top-1 class.
instability appears for fear, suggesting mild sensitivity. In contrast, m2 enforces separation between the target emotion token and all other emotion tokens (and their synonyms). Steering is highly sensitive this margin: low m2 values consistently fail, while performance improves monotonically as m2 increases. At m/ 20, all emotion-dataset pairs steer successfully, indicating that strong inter- class separation essential. We adopt m 0.75, m2g 20 as the best-performing configuration.
Table 12 shows the effect of varying the weight of the cross-entropy loss applied to the target emotion token and its synonyms. Lower weights lead to poor steering, particularly on fear, while higher values generally improve performance. The best overall results are observed at a weight of 25, suggesting that strongly emphasizing the generation of target emotion tokens is necessary for effective control.
Table 13 reports ablations over discrete architectural and training choices. Removing the GELU activation severely degrades performance across all tasks, indicating that nonlinearity is critical for steering. Omitting bias has a moderate effect, while removing synonyms from the loss function leads to failure on fear, suggesting their inclusion helps generalize the steering signal. Within the semantic similarity loss, the delta-norm and cosine components can be individually removed with limited degradation, but removing the full loss results in components reinforce each other to guide the model's representation. The emotion margin loss is also crucial- its removal results in failure across all settings. Finally, applying steering across all layers performs worse than selectively targeting layers based alignment with the emotion direction, underscoring the importance of precise and informed intervention over blanket modification.
synergistic effect where both
23

---

Published as a conference paper at ICLR 2026
Target
(RmoTextToKids)
Anger (CARRR)
(Bhaav)
m=2=1
0.4 + 31.2
29.3
2.4 + 4.0*
m2=5
0.4 + 79.2
7.0 - 96.1
244 + 22.8
m2-10
0.4 99.7
42.7
2.4 32.7"
99.6
2.4
m22-2
0.4 - 99.6
7.0 100
2.4 - 100
Table 11: Ablation for margin between target and non-target classes. Top-1 prediction rates before and after steering under ablation conditions for selected emotion-datat pairs. Each cell shows baseline + post-ablation accuracy. *Indicates failure cases where target emotion is not the most predicted Top-1 I class.
Ablation
Sad (Emooicc)ToKiis)
Anger (CARER)
Fear (Bhaav)
CE Loss Weight=1
0.4 96.3
7.0 95.1
1.4
CE Loss Weight=2
0.4 92.8
7.0 - 54.2
24 24-46.0
CF. Weight-5
7.0 98.7
24
CE Loss Weight=10
80.0
7,0 - 65.7
-4 56.2
Ck: Loss Weight=15
B.4 - 89.8
7.0 85.2
24-4 56.7
CE Loss Weight=20
0.1 99.7
7,0 12.7
244 32.7*
Weight-25
CE Loss Weight=30
0.4 94.4
7.0 - 91.7
4 73.3
Table 12: Ablation for cross-entropy loss weight for emotion tokens. Top-1 prediction rates before and after steering under ablation conditions for selected emotion-datasee pairs. Each cell shows baseline - post-ablation accuracy. *Indicates failure cases where target emotion is not the most predicted Top-1 I class.
Ablation Target
Sad (EmooTextlo
Anger (CARER)
Fear (Bhaav)
Baseline
0.4 99.7
7.0 42.7
32.5
Bias
0.1 88.2
7,0 91.7
26.9*
No Synonyms
0.4 - 98.9
7.0 99.3
24 -9 15.9*
No Semantic Loss
0.4 - 30,2*
7.0 38.9
244 - 100
No Casine Loss
0.1 74,3
7.0 100
24 76,3
97.7
Margin
Turget Layers=All
0.4 66.1
7.0 - 61.9
24 -4 12.9*
Table 13: Top-1 prediction rates before and after steering under various ablation conditions for selected emotion-dataset pairs. Each cell shows baseline - post-ablation accuracy. *Indicates failure cases where target emotion is not the most predicted Top-1 class.
24


Reviews:
[{"review_id": "f6TMSFpDmr", "reviewer": "Reviewer_hPou", "summary": "This paper analyzes how large language models (LLMs) internally represent emotions. Through geometric and probing analyses, the authors identify a low-dimensional “emotional subspace” embedded across model layers, in which affective states are encoded directionally and often linearly decodable. They show this structure generalizes across multiple emotion datasets and five languages, producing a broadly consistent emotional manifold. The authors further develop a learned steering module that can intervene on hidden states to shift the model’s internal emotional perception toward target emotions while largely preserving semantic content; evaluations report strong post-steering classification accuracy for many basic emotions across models and languages. The study combines alignment metrics (cosine similarity, regression error), distortion/stress diagnostics, linear probes, and qualitative rewriting examples to characterize representation geometry, cross-domain robustness, and steerability.", "questions": "SEE WEAKNESS", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- Identification of a low-dimensional, directionally encoded emotional manifold: The paper demonstrates that emotions in LLMs occupy a low-dimensional subspace that is interpretable and directionally organized across layers, with principal axes (PC1–PC3) showing high rank correlations in many models/layers.\n- Cross-dataset and multilingual generalization of emotional structure: Using eight emotion datasets spanning five languages and diverse textual styles, the authors show that the extracted emotional subspace generalizes (low alignment distortion, above-chance linear probe accuracy), supporting the existence of a near-universal affective subspace in multiple LLM families.\n- A learned intervention/steering module that controls internal emotional representations: They introduce and evaluate a module that shifts hidden states toward target emotions. Post-steering emotion prediction rates typically rise substantially (often >85% for many emotions), while semantic-similarity loss remains low, indicating control without wholesale semantic degradation. The method is evaluated across model families and languages, with ablations in the appendix.", "weaknesses": "- Geometry vs. local distortion — inconsistent relational preservation: Although global alignment measures (cosine, regression) are often strong, stress and distortion analyses reveal notable local warping of relative geometry in many layers and datasets. Thus the emotional manifold is not uniformly faithful to human emotion-space relations, which complicates interpretation and downstream use.\n- Uneven multilingual and dataset robustness: Performance and steerability degrade in lower-resource settings (e.g., some emotions in Hindi/Bhaav), and certain datasets (e.g., Go-Emotions) show high layer-wise distortion. This suggests lexical sparsity, annotation imbalance, or domain mismatch limit universality claims and practical applicability across all languages/styles.\n- Potential semantic and safety/ethical concerns with steering: Although semantic-similarity loss is reported low, steering produces surface rewrites that can alter tone, register, or pragmatics (examples show forceful rewrites for anger). The paper does not deeply address possible misuse (manipulating perceived emotion), downstream impacts on user trust, or safeguards for safe deployment. Additionally, steering effectiveness varies across emotions and models; some target emotions remain difficult to induce reliably.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nnskAa4iLD", "reviewer": "Reviewer_UVfz", "summary": "This paper studies how large language models (LLMs) internally represent emotion by analyzing the geometry of their hidden states. Using models like LLaMA 3.1, Olmo-v2, and Ministral across eight multilingual datasets, the authors find a low-dimensional emotional manifold that aligns with psychological dimensions such as valence, dominance, and arousal. These representations are directional, distributed, and consistent across layers and languages. Through SVD and neuron-level analysis (ML-AURA), the paper shows that emotional axes are stable and interpretable. A learned steering module further demonstrates that these internal states can be manipulated predictably without altering semantics, revealing a coherent and controllable affective geometry in LLMs.", "questions": "- How does the emotional manifold evolve across training or fine-tuning stages? Does it emerge early or gradually with language exposure?\n- Could the authors validate the psychological interpretability of latent axes quantitatively (e.g., correlations with human valence/arousal ratings)?\n- Does the steering module modify only internal representations, or can it predictably change generated emotional tone in open-ended text?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper presents a comprehensive, cross-lingual study covering eight datasets in five languages, offering strong evidence for the universality of LLM emotion representations.\n- The use of ML-AURA and SVD-based analyses provides a rigorous and interpretable framework for linking internal neuron activity to affective semantics.\n- The learned steering module demonstrates practical control of emotion representations while preserving meaning, which is an innovative advance beyond descriptive analyses.", "weaknesses": "- While broad in scope, the work is methodologically complex, and the abundance of metrics (stress, distortion, spectral flatness, etc.) may obscure key takeaways.\n- The evaluation relies heavily on synthetic emotion text for subspace construction, which may bias the identified directions.\n- Although the paper claims semantic preservation under steering, this is mostly supported by cosine similarity metrics rather than human evaluations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kMz5YvLYtV", "reviewer": "Reviewer_bf2Q", "summary": "The paper presents an investigation on how LLMs represent emotions in their latent spaces. It finds generalizable evidence that emotion is confined to a few critical dimensions that can be interpreted and manipulated. The authors then present a new method for steering controls to improve emotion classification. The authors robustly back up these central claims through several methods and across multiple languages and datasets. I particularly appreciate the multi-lingual analysis to demonstrate that these findings are generalizable. While paper itself provides some interesting and novel findings, but the presentation is somewhat unclear and messy.", "questions": "Does the Space Alignment method rely on the same Reichman et al. dataset that Centered-SVD does? If so, please state that explicitly.\n Additionally, it would seem to me that the performance of Centered-SVD and/or Space Alignment would highly depend on the quality of this underlying training dataset. I would appreciate some clarity quantifying the quality of this Reichman dataset (or justifying why it was chosen) as this choice seems to be integral to both of these methods.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "This paper presents the first investigation on emotional latent space representations in LLMs, and I believe the techniques used are novel and interesting.\nThe authors provide analysis on many different perspectives, styles, and languages, which adds to the robustness of their findings.\nThe new steering method introduced presents a new way to consider how to change emotions: by focusing on changing the underlying emotional subspace rather than focusing on downstream output.", "weaknesses": "It is unclear exactly what Table 2 is measuring, specifically in regards to cosine similarity and MSE. The paper states high cosine similarity between emotions in real datasets and their synthetic counterparts: what synthetic counterpart are we referring to? Does this reference the Reichman et al. synthetic dataset? Table 2 does not mention which method it utilized as well. What does it mean to measure the cosine similarity of an emotion between two datasets, what datum from each dataset is actually being passed in? Same to MSE and the other metrics. I would recommend rewriting this section to be clearer, as in its current state I was unable to understand exactly what is going on.\n\nOn that note, it is not clear exactly what the experimental setup in 4.2 is. I would appreciate some more clarity on the exact methods used, number of experiments, etc. In particular, I feel like the distortion metrics should be clarified on what they actually represent, as this information is not present in the paper and should not assumed to be common knowledge.\n\nThe presentation of the paper is not very clear; the sections seem rather disjoint and clarifying figures are somewhat lacking throughout. While I believe the content is novel and unique, a rewrite for clarity would improve this paper significantly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MOSMbouBb7", "reviewer": "Reviewer_3kjH", "summary": "The authors analyze consistency of the underlying geometry of emotions in LLMs. They present several studies to support their claims: similarity metrics of synthetic to real samples, dimensionality reduction of the space to discover lower-dimensional structure in a principled manner, and steering representation to study whether the space behaves predictably.", "questions": "- Formatting errors: the authors have used the wrong citation format (every citation is used outside of parenthesis, probably showing some lack of care when switching between templates)", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "- The authors present many metrics and models, as well as some useful baselines in the appendix to contextualize the numbers\n- Many studies are presented to corroborate the claims of the paper instead of building a case from only a single case.\n- The authors use existing theoretical work in emotion to ground their findings.", "weaknesses": "- Manuscript is very difficult to read, with methodology blended with experiments, making it very difficult to follow what is currently being presented. For example, what is Table 2 showing (L210)? Same with L230, and all of the subsequent sections (5 and 6). I found myself jumping back and forth constantly, trying to understand what is being compared to what and how. Moreover, some numbers seem to be presented in the text only. I would appreciate clearer methodology in the rebuttal, as some concepts are also not explained at all.\n- Tables 1 and 3 contain so much information that it becomes very difficult to figure out what the takeaway from each should be. A better mode of presentation would be preferable, and the full tables can be in the appendix for the interested reader.\n- Figure 2 could be improved, perhaps by showing the projection to the Dominance-Valence plane. As it is now, sad and happy seem to have the same embedding (meaning the same valance value, among the other dimensions). As a result, the study this corresponds to I believe would benefit from some quantification to substantiate the claim.\n- The fact that from layer 0 to layer 31, the % of neurons remains the same might indicate that the clustering is happening because of word embeddings themselves rather than emotion content per se. This is because we would expect higher-level emotional content to emerge after processing in later layers, except if it based on individual words.\n\nTo reiterate, the main weakness of the paper is its lack of clarity, not necessarily a lack of substance or novelty.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)