Mapping Semantic & Syntactic Relationships with Geometric Rotation
Authors: Michael Freenor
Lauren Alvarez
Venue: ICLR 2026 Poster (2026)
Keywords: Embedding Models
Steering Vectors
High Dimensional Geometry
LLMs
Unit Hypersphere
Primary area: interpretability and explainable AI
TL;DR: Rotor-Invariant Shift Estimation (RISE), a geometric framework that enables cross-lingual and cross-model semantic transfer with improved performance on transformer-based embeddings.

Abstract: Understanding how language and embedding models  encode semantic relationships is fundamental to model interpretability. 
While early word embeddings exhibited intuitive vector arithmetic  (''king'' - ''man'' + ''woman'' = ''queen''), modern high-dimensional text representations lack straightforward interpretable geometric properties. 
We introduce Rotor-Invariant Shift Estimation (RISE), a geometric approach that represents semantic-syntactic transformations as consistent rotational operations in embedding space, leveraging the manifold structure of modern language representations. 
RISE operations have the ability to operate across both languages and models without reducing performance, suggesting the existence of analogous cross-lingual geometric structure.
We compare and evaluate RISE using two baseline methods, three embedding models, three datasets, and seven morphologically diverse languages in five major language groups. 
Our results demonstrate that RISE consistently maps discourse-level semantic-syntactic transformations with distinct grammatical features (e.g., negation and conditionality) across languages and models.
This work provides the first demonstration that discourse-level semantic-syntactic transformations correspond to consistent geometric operations in multilingual embedding spaces, empirically supporting the linear representation hypothesis at the sentence level.

Published as a conference paper at ICLR 2026
MAPPING SEMANTIC & SYNTACTIC RELATIONSHIPS WITH GEOMETRIC ROTATION
Michael Freenor & Lauren Alvarez
TELUS Digital
(mi chael. freenor, lauren Leurenn.llvrrrreeeeaaaaaaggtaala
ABSTRACT
Understanding how language and embedding models encode semantic relation- ships is fundamental to model interpretability. While early word embeddings exhibited intuitive vector arithmetic ("king" "man" "woman" = "queen"), modern high-dimensional text representations lack straightforward interpretable geometric properties. We introduce Rotor-Invarian Shift Estimation (RISE), a geometric approach that represents semantic-syntacee transformations consis- tent rotational operations in embedding space, leveraging the manifold structure of modern language representations. RISE operations have the ability to operate across both languages and models without reducing performance, suggesting the existence of analogous cross-lingual geometric structure. We compare and evalu- ate RISE using two baseline methods, three embedding models, three datasets, and seven morphologically diverse languages five major language groups. Our re- sults demonstrate that RISE consistently maps discourse-level semantic-syntacti transformations with distinct grammatical features (e.g., negation and condition- ality) across languages and models. This work provides the first demonstration that discourse-level semantic-syntactic transformations correspond consistent geometric operations in multilingual embedding spaces, empirically supporting the linear representation hypothesis at the sentence level.
INTRODUCTION
Understanding how contemporary language models encode and manipulate semantic knowledge has become a central challenge in deep learning interpretability. The ability to interpret (probe) and con- trol (steer) these internal representations is fundamental to developing trustworthy, safe Al systems. In word2vec (Mikolov et al., 2013a) and similar models, semantic relationships could be captured with simple vector arithmetic in the embedding space (i.e. the famous "king" "man" + "woman" "queen" analogy). This linear transparency offered both interpretability and controllability, enabling researchers to navigate semantic space through intuitive mathematical operations.
However, this clarity has largely disappeared in modern transformer-basee language models. While large language models (LLMs) have achieved remarkable performance across diverse language tasks (Achiam et al. 2202 Touvron et al., 2023), their internal workings remain largely opaque (Elhage et al. 2022 Rogers et al. 2021). limiting our ability to understand, predict, and control their behavior critical applications. Unlike the interpretable, linear directions found in static word embeddings, the geometry of modern text representations lacks the same straightforward correspondence to se- mantic operations. This opacity poses significant challenges for understanding how these models organize linguistic knowledge and limits our ability to interpret their behavior in principled ways.
The central challenge lies in identifying which geometric operations correspond to meaningful se- mantic transformations in these complex representation spaces. Current approaches often rely on task-specific probes (Rogers et al. 2021 Hewitt & Manning 2019 Alain & Bengio 2017) or steer- ing vectors (Zou al. 2023 Wang et al. 2023 Turner et al. 2023 Merullo et al. 2024 Trager et al. 2023). but lack generalizable frameworks for systematically mapping semantic relationships to geometric structure. Without such principled methods, cannot determine whether the geo- metric regularities that made static word embeddings interpretable persist in modern language or embedding models, albeit in more complex forms.

---

Published as a conference paper at ICLR 2026
We address this gap by introducing Rotor-Invariam Shift Estimation (RISE), geometric approach that represents semantic-syntacti transformations as consistent rotational operations in embedding space, leveraging the manifold structure of modern language representations. RISE is a rotor- based alignment method that identifies cross-lingual and cross-model geometric transformations. Specifically, we demonstrate how RISE identifies three discourse-level semantic-syntactic changes (negation, conditionality, and politeness) across seven morphologically distinct languages and gen- eralizes across three different embedding model architectures. The goal of this study is de- velop framework for identifying discourse-level semantic-syntactic changes that correspond to consistent geometric transformations, and determine how well these transformations can be cross- lonsulinggully mapped across model architectures. Our approach treats semantic-syntactic transformations as rotations on the unit hypersphere, where sentence embeddings reside, enabling us to align different linguistic contexts into a common geometric framework. This paper presents evidence that certain semantic-syntactic transformations exhibit generalizable geometric structure while others vary based context- dedenddece, extending the linear representation hypothesis to cross-lingual discourse. We demonstrate this through empirical experiments across two baselines, three models, and seven languages revealing that negation, conditionality, and politeness transfor- mations can be captured as consistent rotational operations
2
RELATED WORK
2.1 LINEAR REPRESENTATION HYPOTHESIS
The linear representation hypothesis (LRH), or lincar subspace hypothesis, has emerged as a promis-
ing theory for bridging the interpretability gap for embeddings (Mikolov et al., 2013b Levy &
Goldberg, 2014 Bolukbasi al. 2016 Ethayarajh 2019 Park et | 2025). The LRH posits
that semantic concepts are encoded as linear structures within embedding spaces, meaning linear algebraic operations can be used for interpretation and control (e.g.. "king" "man" "woman"
"queen" presented by Mikolov et al. (2013b)). Park et al. (2024) formalized the LRH by unifying three distinct notions of linearity that had developed independently across the literature:
1. word2vec-like embedding differences (Arora et al. 2016 Mimno & Thompson 2017
Ethayarajh et al., 22199] Reif et al. 2019 Li al.
2020 Hewitt & Mamning 2019 Chen
et al. [021] Chang et al. 2022 Jiagg et al. 2023
Mitchell & Lapata
2008 Baroni &
Zamparelli 2010)
2. logistic probing (Alain Bengio 2017 Kim et al.
2018 Belinkov
2022 Li et
2022
Geva et al. 2022, Nanda et al. (023)
3. steering vectors (Wang et al. 2023}
2023
Turner al.
2023
Merullo et al.
2024
Trager al.
Park al. theoretical framework addresses critical gap by synthesizing the first formalization of what "linear representation" means (Park et al.] (0024.. However, while the LRH has been val- idated primarily within individual languages (monolingually), there remains significant gap in understanding how semantic-syntattic transformations generalize across linguistic contexts (cross- lingually). Most existing work examines static concept encodings (Park et al. 2025 2024) rather than dynamic semantic-syntacti transformations that reflect real-world language use. Our work is the first to extend the LRH multilingual contexts and embedding models. Although, the linear representations we consider are geodesic ares and not Euclidean lines.
2.2 LINEAR & GEOMETRIC REPRESENTATION TECHNIQUES
The geometric foundations established Park et al. 2024 are crucial for understanding when and why linear algebraic operations succeed in capturing semantic relationships. With traditional Eu- clidean geometry, it is hard accept that arbitrary dot products or cosine similarities have semantic meaning. Moreover, Park et al. (2024) demonstrated that the choice of inner product fundamen- tally determines the interpretability of geometric operations, providing principled foundations for
'The link our GitHub repository
ix/ www--tteringg
2

---

Published as a conference paper at ICLR 2026
representation analysis. Our work builds directly recent advances in understanding linear rep- resentations in language models (Park et al.) 2024 Li et al. 2023). RISE implements a technique that respects semantic structure, similar to the geometric framework developed by Park et al. (2024). While previous work focused primarily on categorical concepts and word-level transformations, RISE extends understanding sentence-level, discourse-level transformations through cross- lingual and cross-model analysis using seven morphologically diverse languages.
2.2.1 STEERING VECTORS & EMBEDDING MODELS
The practical applications of linear representation theory have been explored through steering vector techniques, Turner et al. (2023 Liu et al. (2024), and Zou et al. (2023) demonstrated that targeted modifications to internal, latent space representations can systematically alter model behavior with- out parameter updates. The majority of steering vector research (Im & Li 2025 Rimsky et al., 2023 Zou et al. 2023 Li et al. 2023) is connected to activation steering, only investigating the im- pact of steering vectors in the activation, hidden, and/or latent layer of an LLM. Recently, |Pham & Nguyen (2024) introduced Householder Pseudo-Rotation (HPR), which addresses activation norm consistency issues in LLM behavioral modification through direction-magnituee magnitude decomposition and pseudo-rotational transformations. Building on the insight that geometric approaches outperform additive methods, our work extends geometric reasoning semantic transformations in embedding space through Riemannian operations. To our knowledge, there is no work investigating the applica- tion of steering vectors to embedding models only completion models. This study extends steering principles to embedding models on manifolds, not activation-level steering.
2.3 GENERALIZATION AND RELIABILITY CHALLENGES
Current knowledge about the generalization properties of linear representations reveals significant limitations. The taxonomy of generalization research in natural language processing (NLP) (Hup- kes et al. 2023) provides a framework for evaluating robustness, but systematic applications to representation-basee techniques (i.e., steering, probing, embedding manipulation) have been lim- ited. Recent empirical studies have revealed that steering vector effectiveness varies substantially across different inputs and contexts (Tan et al., 2024). Secondly, the relationship between local and global linearity represents a particularly critical gap in current understanding. There have been numerous demonstrations of local linear behavior within specific domains or prompt formats, but achieving global linearity (generalizable to multiple model architectures with different pre-training as required by strong versions of the LRH) remains challenging. While many studies demonstrate impressive results in controlled settings, they often fail to address the robustness needed in practi- cal applications. This study contributes to the literature gap by presenting a robust framework for geometrically identifying discourse-level semantic-syntacti changes across typologically diverse languages and model architectures.
THEORETICAL MOTIVATION
The limitations identified in the related literature point toward fundamental, theoretical challenge: existing approaches operate in Euclidean/linear space while modern embeddings live on curved manifolds (spherical space). This geometric mismatch may explain why steering vector research shows inconsistent cross-context performance and why linear methods struggle with robust gen- eralization. We hypothesize that discourse-level semantic-syntactie transformations correspond to intrinsic geometric operations on the embedding manifold, rather than fixed directions derived from Euclidean computations. If semantic transformations can be characterized as consistent rota- tional operations on the unit hypersphere where embeddings reside, this would provide theoretical support for the extension of the Linear Representation Hypothesis in curved spaces (through geodesics) and cross-lingual interpretability. Testing this hypothesis requires robust evaluation across diverse languages and embedding architectures to determine whether geometric consistency reflects universal semantic properties model-specific artifacts.
3

---

Published as a conference paper at ICLR 2026
4 ROTOR-INVARIANT SHIFT ESTIMATION (RISE)
Modern sentence embeddings from multilingual encoders reside approximately on a unit hyper- sphere in high-dimensional space when the training objective enforces fixes the fz-norm straints (Hirota et al. 2020), the embeddings are normalized to unit length (Reimers & Gurevych, 2019), or the model designed produce isotropic embeddings (Li et al. 2020; Ethayarajh 2019). Local semantic transformations (e.g., negation, politeness, conditionality) can be understood as ro- tational displacements this sphere. The key insight is that these displacements be interpreted by aligning different contexts to a common geometric frame.
For any neutral sentence embedding n gd-1 and its semantically transformed variant v Sd-1, we can compute an orthogonal transformation canonical reference direction 61. By applying this same transformation v, we express the semantic change in standardized coordinate system:
rotor) A(n) that aligns n to
= R(n) (w),
where log,, (v) denotes the Riemannian logarithm that computes the tangent vector from n on the hypersphere, and R(n) aligns the tangent vector to the canonical reference direction. Normalized embeddings reside on a unit hypersphere, where geodesics define the shortest paths between points, preserving the manifold's intrinsic geometry rather than imposing Euclidean distance measures. These geodesic paths represent the natural notion of "line" in the embedding space, as they define the shortest distance between two points on the surface. By working with geodesics, we ensure our semantic transformations are consistent with the manifold structure. To "flatten" out the curved are to straight vector, the Riemannian logarithmic map log,, (v) produces the vector from n to tangent plane at n. By operating within the tangent space at n, geodesic differences can be treated ordinary vectors.
4.1
THE ROTOR ALIGNMENT ALGORITIM
RISE proceeds in three steps illustrated in Figure
aneutral
transformed sertience
Prosotypa
semantic change
change
Canorisuliation
Protetype
aweriges
RiDE supllos reenca serrence Vis the exporieti produce
curved po
rolees sorted vemonio dhange caronia langart space
evamples Er7o0ye voneforrtion
stbestiin mantill
Figure 1: RISE syep-by-step illustration.
Canonicalization. For each neutral--ransffrmmedd sentence pair (n4, v;), compute a rotor R(n;) that maps to the reference direction é1. We interpret canonicalization controlling for the semantics present in the first elements of our pairs. By applying the canonical rotation to the second of the two the idea is that have isolated the key differences between the elements in a fixed frame of reference.
Prototype Learning. Canonicalize all semantic changes into the reference frame and average all the tangent vectors to calculate one Prototype p. where M the total amount sentence pairs]) This is similar technique mean-centering (Jorgensen 2024):
M
Σ R(n;) logn, (v:).
M
For small angular differences, first-order equivalent simply averaging the points and re-normalizing after the fact.

---

Published as a conference paper at ICLR 2026
Prediction. To predict the semantic transformation for unseen neutral embedding n", the proto- type P can be used to predict the transformation embedding o* by converting the prototype p with the Riemannian exponential map and application of the transpose of n²'s canonicalizing rotor:
e² cxp,,- (R(p,-(((n")T))
R(n") T rotates p into the tangent space at Then the Riemannian exponential exp,, (p) takes the tangent vector p and moves along the geodesic starting at n². The vector direction is which geodesic to follow and the length is how far along that are to go (in radians).
4.2 DIFFERENTIATION FROM RELATED WORK
Our approach is related to recent advances in understanding linear representations in language mod- els. As discussed in Section 2.2, Park et al. (2025) ) use a "causal inner product" that respects semantic structure in function space using the Riesz isomorphism. However, RISE uses Riemannian geom- etry to operate consistently on the curved manifolds. Both methods take advantage of geometric properties, but the methods are distinctly different.
Crucially, RISE transformations exhibit commutativity; applying multiple semantic transformations yields consistent results regardless of order (see Appendix(A) This commutativity property provides strong evidence for the LRH, as it demonstrates that semantic transformations behave like vector additions in the tangent paace-geodesice serve the curved-space generalization of straight lines. The preservation of additive structure across semantic operations suggests that the geometric frame- work captures fundamental algebraic properties of meaning composition. We discuss more about the commutativity properties in Appendix A
Furthermore, the analysis in Park et al. 2025) focused on categorical relationships in the unem- bedding space of language models; our work examines discourse-level transformations in sentence embeddings across multiple languages. RISE effectively implements a non-Euclidean transforma- tion that aligns with the natural curved manifold structure of the embedding space. This connection high- digh-dimensional geometry provides theoretical grounding for why rotational operations can cap- ture semantic transformations more effectively than simple vector additions, and extends the linear subspace hypothesis to curved/geodesic subspaces.
EXPERIMENTAL DESIGN
5.1 DISCOURSE-LEVES SEMANTIC-SYNTACEE CHANGES & LANGUAGE SELECTION
We focus on three discourse-level semantic-syntactic transformations that vary in their context- dependence:
The logical reversal of the propositional content of a statement; where the proposition is "P" take the negation to be "not-P." Moreso, are negating the predicate. This transformation is semantically precise and should exhibit high geometric consistency across contexts and languages.
Negation:
Conditionality: Converting declarative statements into conditional constructions ("p" - "If P"). This introduces modal semantics that may interact with contextual factors.
Increasing the social formality or deference level of utterances. This is highly context- dependent and culturally variable, making it a challenging test case for geometric consistency.
Politeness:
We selected seven morphologically diverse languages ensure broad coverage morphological, syntactic phenomena, and resource levels: English, Spanish, Japanese, Tamil, Thai, Arabic, and Zulu. This selection spans multiple language families (Indo-European, Sino- Tibo-an Dravidian, Afroasiatic, Niger-Congo) and different morphological types (analytic, agglutinative, fusional). The languages also represent different levels of language model availability and resources. The diversity crucial because different languages realize semantic transformations through distinct linguistic mechanisms. For instance, negation might be expressed through: (1) Particles (i.e. English "not"); (2) Affixes (i.e. Tamil verb-internal negation, Japanese "nai""; and (3) Auxiliary constructions (i.e. English "does/has not"). By testing across this range, we can determine whether geometric consis- tency reflects universal semantic properties or is merely artifact of particular linguistic structures.
5

---

Published as a conference paper at ICLR 2026
5.2 DATASETS, EMBEDDING MODELS, & LINEAR BASELINES
We use three datasets and three models for evaluation. We used two open-source, external datasets: The Benchmark of Linguistic Minimal Pairs (BLiMP) (Warstadt et al.) 2020) and Sentences Involving Compositional Knowledge (SICK) (Marelli et al. 2014). and synthetically generated transformation combination in the Synthetic Multilingual dataset, we generated 1,000 ceuutra--rannfoomeed sentence pairs using GPT-4.5 with carefully controlled prompts (see Appendix [D). To ensure robust analysis, we implemented several diversity controls (see Appendix +
one dataset, referred as the Synthetic Multilingual dataset. For each
We compare three multilingual embedding models: text-embeedding---aage (OpenAl 2024). bge- m3 f Chen et al. 2024), and mBERT (Devlin et al. 2019). The text-embedding33-e model produces 3072-dimensional vectors, bge-m3 produces (024-dimensional vectors, and mBERT pro- duces 768 co8-dimensional vectors. All selected models produce constant-length embeddings that reside on a hypersphere making them suitable for our geometric analysis. This dimensional diversity allows us test whether RISE effectiveness depends on embedding dimensionality. We calculate rotor alignment score where the scores represent mean cosine similarity between predicted embedding vectors and the semantically transformed pair held-out test sets, with higher values indicating more consistent geometric structure. Table [ describes how the cosine similarity scores are inter- preted.
We include Mean Difference Vectors (MDV), and Procrustes alignment as baseline comparisons be- cause they represent standard linear approaches used model transformations embedding spaces. MDV test whether simple difference vectors can capture semantic or cross-lingual structure, while Procrustes evaluates whether a single global rotation can align transformed embeddings. MDV the geometrically correct analogue of the Euclidean additive method for modern spherical embeddings, providing stronger and fairer baseline for RISE.
Cosine Range
Similarity
Interpretation
Supporting Literature
0.80
Strong, consistent geometric structure Moderate, reliable structure
Reimers & Gurevych 2019)
0.65-0.80
Mikolov el al. 12013b Etha-
yarajh (2019) (2019)
0.50-0.65
Weak or variable structure
EChayarajh
2019
Conneau
al. (2018)
0.30
Inconsistent or failing transformation
Artetxe al.
2018
Conncau
ct al. (2018)
Table 1:
Interpretation of cosine similarity magnitudes used throughout this work. Higher values indicate stronger geometric consistency between predicted and target embeddings. These thresholds are stricter than prior work but remain consistent with the established interpretations in the literature.
6 RESULTS
6.1
CROSS-LANGUAGE TRANSFER COMPARISON
This section discusses the comparison of embedding models trained in one of the seven languages and tested on all seven. The results of this section demonstrate RISE multilingual performance com- puted by three embedding models. See Appendix for comprehensive results across all phenomena for each model.
Negation emerges as the most robust discourse-level, semantic-syntactie transformation, achieving the highest mean rotor alignment score (0.788) across all model-language combinations with perfor- mance ranging from 0.686 to 0.918. demonstrets RISE performance on negation for each model. RISE transformations for negation are most geometrically consistent in text-embedding- l-aage.. Negation's strong performance indicates that generalizable discourse- level, semantic- syntactic changes are captured by RISE and best applied lingu-ling in seat-embedding---largge
'Bge-m3 should be m3 titled in the final version of CCen al. 2024), but we referenced the model as hge-m3 in this paper and figures.

---

Published as a conference paper at ICLR 2026
RISE
Comparison Negation
textt Seitt-emeedding---eee Hlarge
MBBET
BGF-M3
0.00
6822
0.366
3.49
8.22
8:23
(24
0,550
X750
09
16 1H
20
A6
05
TH
65
1x 20
Langaage
Language
Figure 2: Embedding model heatmap cross-lingual transfer comparison on negation.
Conditionality demonstrates the highest stability and consistency across cross-language transfers, with the lowest performance variability (0.038) and most stable individual measurements (see Ap- pendix[B) With the second highest, mean performance (0.780), conditionality is particularly consis- tent results across all combinations. The strong transfer seen in age-m3 and rext-embedding-3-larg suggests that conditional semantics are captured by stable geometric structure despite their modal complexity.
Politeness exhibits the most variable geometric structure, ranking third in performance (0.762 mean) with the highest performance variability (0.060) across combinations. This variability aligns with expectations, as politeness realizations depend heavily on cultural context and linguistic con- challenging.
ventions, making cross-language transfer inherently
The contrast across phenomena performance reflects an interesting insight. In the results, negation appears more robust, politeness is most variable, and conditionality sits between. This suggests em- beddings encode logical semantic operators (negation and conditionality) with strong cross-lingue consistency. However, pragmatic operators (politeness) less reliable due to inherent language- specific indicators and cultural conventions. Additionally, cross-language analysis revealed dimen- sionality does not directly predict cross cooss-lingual performance. Despite having lower dimensionality, bge-m3 (1024-dim) demonstrated the least variance in cross-language performance for all phenom- ena and languages. While rext-embeddingg--large
(3072-dim) showed highest cross-language per- formance (Figure 3), mBERT (768-dim) showed strong monolingual performance, but exhibited high variability, particularly for politeness in cross-language settings. These results highlight that training methodology and architectural choices matter more than raw embedding dimensionality for cross-language semantic transfer.
The cross-language analysis fully presented in Appendix B supports our hypothesis that discourse- level semantic-sysee -syttcccc transformations correspond to geometric operations on the embedding man- ifold. The variation across models, preservation of linguistic relationships across languages, and transformation patterns indicate that RISE successfully identifies semantic-syntacti transformation on the embedding manifold. The limitations and future work are discussed further on.
ard Tran
4111
Figure 3: Cross-language transfer heatmaps for text-embeddingg- 3-large showing RISE performance across all language pairs for conditionality, negation, and politeness transformations. Darker colors indicate higher cosine similarity between predicted and target embeddings.
7

---

Published as a conference paper at ICLR 2026
6.2 CROSS-MODER TRANSFER COMPARISON
To evaluate RISE prototypes' robustness to transfer across different embedding architectures, we conducted cross-model mapping experiments using the method developed by MMoris et al.) (2020). This approach learns statistical mappings between embedding spaces through principal compo- nent analysis (PCA) and distributional alignment, enabling transfer of learned RISE prototypes from one model to another. We specifically examined transfer from text-embedding-3-lan dimensional) to bge-m3 11024-iimnnionna), demonstrating cross-model semantic transfer across different dimensionallities and training objectives. For each language pair and phenomenon, we learn RISE prototypes cext-embedding---1 l-rage using 80% of the data, map these prototypes and e1 to bge-m3 space, and evaluate performance native bge-m3 embeddings using the remaining 20%. Figure demonstrates comprehensive cross-model and cross-language transfer results.
(3072-
bge-m3 reveals strong language-dependern performance. English achieves 0.80-0.82 similarity across all transformations, while other lan- guages cluster around 0.70-0.75, and Zulu consistently scores 0.63-0.66. This 20% performance gap persists across conditionality, negation, and politeness transformations. These results suggest rotations can transfer between architecturally different models, but their effectiveness depends crit- ically on source language, indicating that learned transformations are not architecture sebbirecurr-indeeendeene The consistent English advantage across models suggests these embedding spaces share more robust geometric structures for English, likely reflecting training data imbalances (Anglo-centrie bias in the composition of the model's s training data). The consistent language ranking across different seman- tic transformations (conditionality, negation, politeness) suggests the bias is structural rather than semantic. In conclusion, RISE successfully captures semantic patterns that perform consistently in cross-model comparison.
Cross-model transfer from text-embedding-3-aaee
Figure 4: Cross-Model Semantic Transfer: text-embedding-33larre - bge-m3. Each cell shows transfer performance from source language prototype (lex--embedding---llrge to target language test set (bge-m3). Diagonal elements represent pure cross-model transfer, while off-diagonal el- ements show combined cross-model and cross-language transfer using Morris statistical mapping (Morris et al. 2020
6.3 ENGLISH TASK-BASED COMPARISON
Our main investigation is how well RISE peforms in in multi-lingual settings. However there are limited external datasets for evaluating the performance discourse-level, formation tasks. Due to the limited resources, we had to select the most related datasets, BLiMP and SICK. BLiMP is paired sentence dataset for major grammatical phenomena in English, and SICK is a dataset with paired sentences with entailment, contradiction, and neutral labels.
trans-
Table 2 summarizes RISE performance across the three selected datasets. The results confirm that all models achieve strong performance, with particular strengths varying by dataset: mBERT excels on grammatical tasks (BLiMP) and contradiction detection (SICK), while bge-m3 shows the most consistent performance across synthetic multilingual data. The dramatic performance gap between BLiMP (>0.92) and SICK (0.62-0.74) suggests that RISE rotations might be capturing something more specific than general semantic transformations.
The high BLiMP performance indicates RISE excels at preserving grammatiial/syntactic structure, while the moderate SICK performance suggests these same rotations don't preserve semantic
8

---

Published as a conference paper at ICLR 2026
latedness well. These results show that benchmark choice dramatically affects relative model ranking. Robustness depends on whether the task prioritizes cross-lingual consistency (favoring bge-m3) raw performance specific phenomena (favoring text-embedding---lagee for negation, mBERT for grammatical tasks).
Table 2: RISE Performance Across Three Datasets: The performance is measured with the rotor alignment score between RISE-steered embeddings and target embeddings where bold values indi- cate best performance per dataset. text-embedding-- -3-aagee is abbreviated TE3L.
Model
Synthetic Multilingual
BLiMP Benchmark
SICK Dataset
TE3L (3072d)
0.771
0.929
0.623
bge-m3 (1024d)
0.782
0.956
0.631
mBERT (768d)
0.709
0.961
0.736
Average
0.754
0.949
0.663
6.4 LINEAR BASELINE COMPARISONS
The full results presented in Appendix C compare RISE against standard baselines, Mean Dif- ference Vectors (MDV) and Procrustes alignment, across the same three datasets. MDV is not Euclidean. MDV preserves spherical structure and naturally resembles RISE more closely than Pro- crustes. This distinction is directly reflected in the results: MDV and RISE transfers best across languages where Procrustes fails.
The strongest performance appears in monolingual English evaluation (BLiMP), while performance drops substantially for Procrustes on semantic relatedness (SICK) shown in Table 3 This shift in performance reflects Procrustes' inability to identify generalizable semantic-syntan as expected by method. Procrustes fits a single global rotation which is too rigid for the cross- lingual and cross model analysis In contrast, RISE maintains stable cross-lingual and cross-model performance (c.g., App. B Figures 5-7), indicating that geometric operations on the manifold better capture discourse-level semantic structure than Euclidean differences.
relationship
The MDV vs. RISE vs. Procrustes results reinforce our carlier claim that methods operating on the curved manifold (where sentence embeddings inherently reside) perform better than Euclidean/lin- ear methods. Most steering and probing techniques operate in linear space, and we conjecture that this geometric mismatch helps explain why linear methods struggle to generalize. In short, Pro- crustes fits single global rotation which is too rigid for the cross-lingual and cross model analysis. Geometric transformations, like RISE and MDV, are better suited for semantic-syntactic analysis and cross cross-lingual stability.
Method
Monolingual Syntactic (BLiMP)
Monolingual Semantic (SICK)
Cross- Cross-Language Transfer (All Phenomena)
RISE MDV Procrustes
Strong (0.97)
Strong (0.84)
Moderate-Strong (0.74-0) 0.89))
Strong (0.97)
Strong (0.83)
Moderate-Strong (0.72-0.91) Failing-West (0.25 0.62)
Strong (0.99)
Moderate (0.67)
Table 3: Condensed summary of baseline comparisons from Appendix C using the simila-simillarty interpretation scale from Table RISE and MDV show Strong monolingual and Moderate-Strom langualan structure, whereas Procrustes drops to Weak or Failing consistency outside syntactic, same-language settings.
DISCUSSION & FUTURE WORK
Our findings demonstrate that meaningful semantic-syntactic operations can be recovered as ge- ometric transformations in modern language model representations. RISE successfully identifies consistent geometric structure for discourse-level semantic-syntactic changes, primarily for text- embedding-3-llrge and negation in multilingual settings. The results demonstrating spherical meth-

---

Published as a conference paper at ICLR 2026
ods, RISE and MDV, out perform linear methods, Procrustes alignment, provide positive results for extending the LRH to spherical spaces.
Evaluation benchmarks (Table 2 reveal task-dependent effectiivnnss. RISE achieves near- pear-perfect performance on syntactic acceptability (BLiMP: 0.93-0.96) but only moderate performance on se- mantic similarity (SICK: 0.62-0.74), suggesting better alignment with grammatical rather than mantic transformations. Section 6. shows that negation and conditionality are the most generaliz- able discourse-level, semantic-syntactic changes captured by RISE and best applied cross-lingually Our cross-model transfer experiments expose an English-centric bias, with English achieving 20% higher transfer scores than languages like Zulu. This English-centre centric bias persists across all semantic transformations, indicating that current multilingual models encode geometric structures that prioritize English. Future work should focus on developing more equitable multilingual representations and investigating which language-specific geometric structures are an inherent feature of the models.
in rext-embedding---llrae
Together these results support that RISE is most successful at identifying semantic transformation with distinct grammatical factors, but more work is needed to justify semantic transformations in multilingual models are universal geometric operations. First, our analysis focuses on three specific linguistic transformation types. Future work should expand to additional semantic and pragmatic phenomena to test the generality of geometric consistency principles. Second, while our experiments bge-m3, and mBERT), validation
used three diverse embedding models (fext-embedddng---larye across additional architectures would strengthen claims about the universality of geometric semantic structure. Third, the reliance on GPT-4.5 for data generation may introduce subtle biases toward English-centric conceptualizations of semantic phenomena. Future work should incorporate more diverse data sources and validation by native speakers.
CONCLUSION
The ability to learn geometric transformations for discourse changes relates to work on text gener- ation and steering vectors (Turner et al. 2023 Li et 2023). Our rotor-based approach, RISE, provides a geometric framework for understanding and improving interpretability in language mod- els. This work investigated whether discourse-level semantic-syntactii transformations in multilin- gual embedding spaces correspond to intrinsic geometric operations, specifically rotations identified through the RISE method, Our comprehensive evaluation across multiple baselines, models, lan- guages, and datasets reveals more complex reality than initially hypothesized. This work demon- strates that modern language model representations maintain interpretable geometric structure for some cemantic-syntactte transformations, extending the promise of geometric semantics from early word embeddings to contemporary transformer models. We show that:
1. Semantic transformations with clear syntactic mapping demonstrate the most consistent geometric structure.
2. RISE successfully identifies semantically meaningful geometric structure in high- dimensional embedding spaces that generalizes cross-lingually and across model archi- tecture.
As language models continue to evolve, understanding these geometric foundations will be cru- cial for developing more interpretable AI systems. By revealing transferable geometric structure in semantic transformations (e.g. negation and conditionality). this work opens new possibilities for understanding language model behavior through geometric interventions. Our work promotes geometric methods as more appropriate approaches to cross-lingual semantic interpretation, achiev- ing 77%-95% cross-language transfer effectiveness across typologically diverse languages. By de- veloping RISE, we demonstrate that interpretable structure exists for some grammatically distinct semantic transformations, providing a tools for understanding how these systems encode semantic knowledge. While RISE remains valuable for analyzing model-specific semantic structures, claims about universal geometric operations require substantial qualification.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
P.A. Absil, R. Mahony, and R. Sepulchre. Optimization Algorithms on Matrix Manifolds. Princeton University Press, 2008.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 2023.
Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classi- fier probes. In International Conference on Learning Representations, 2017. URL https: //openzeview. http://oooooonnaa//rr1333
Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. A latent variable model approach to pmi-based word embeddings. Transactions of the Association for Computational Linguistics, 4:385-399 2016.
Mikel Artetxe, Gorka Labaka, and Eneko Agirre. A robust self self-learning method for fully unsuper- vised cross-lingual mappings of word embeddings. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), pp. 789-798, 2018.
Marco Baroni and Roberto Zamparelli. Nouns are vectors, adjectives are matrices: Representing adjective-noun constructions in semantic space. In Hang Li and Lluís Mårquez (eds.), Pro- ceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pp. 1183-1193, Cambridge. MA, October 2010. Association for Computational Linguistics. URL
Yonatan Belinkov. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48(1):207-219, 2022.
Tolga Bolukbasi, Kai- Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in Neural Information Processing Systems, 2016.
Tyler Chang, Zhuowen Tu, and Benjamin Bergen. The geometry of multilingual language model representations. In Proceedings of the 2022 Conference on Empirical Methods in Natural Lan- guage Processing, pp. 119-136, 2022.
Boli Chen, Yao Fu, Guangwei Xu, Pengjun Xie, Chuanqi Tan, Mosha Chen, and Liping Jing. Prob- ing bert in hyperbolic spaces. International Conference on Learning Representations, 2021.
Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. M3-embedding: Multi-linguality, multi-functionaaliyy multi-granullriity text embeddings through self-knowledge distillation. In Findings of the association for computational linguistics: ACL 2024, pp. 2318- 2335, 2024.
Alexis Conneau, German Kruszewski, Guillaume Lample, Loic Barrault, and Marco Baroni. What you can cram into a single vector: Probing sentence embeddings for linguistic properties. In ACL 2018-56th Annual Meeting of the Association for Computational Linguistics, volume 1, pp- 2126 2136-2 Association for Computational Linguistics, 2018.
Jacob Devlin, Ming- Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171-4186. Association for Computational Linguistics, 2019.
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauma Kravec, Zac Hatfield-Dode -oodds Robert Lasenby, Dawn Drain, Carol Chen, et al. Toy models of superposi- tion. arXiv preprint arXiv:2209, 2206522009.0662. 2022.
Kawin Ethayarajh. How contextual contextualized word representations? comparing the ge- ometry of bert, elmo, and gpt-2 embeddings. In Proceedings of EMNLP-IJCNLP, pp. 55-65, 2019.

---

Published as a conference paper at ICLR 2026
Kawin Ethayarajh, David Duvenaud, and Graeme Hirst. Towards understanding linear word analo- gies. In Proceedings of the 57th annual meeting of the association for computational linguistics, 2019.
pp.
Mor Geva, Avi Caciularu, Kevin Wang, and Yoav Goldberg. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. In Proceedings of the Conference Empirical Methods Natural Language Processing, pp. 30-45, 2022.
John Hewitt and Christopher D Manning. A structural probe for finding syntax in word representa- tions. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Pa- pers). pp. 4129-4138, 2019.
Wataru Hirota, Masahiro Tanaka, Sho Takase, Naoaki Okazaki, and Kentaro Inui. Emu: Enhancing multilingual sentence embeddings with constrained softmax loss. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 7904-79499999 1914,79111110 2020. doi: 10.1609/aaai.v34005 6301.
Dieuwke Hupkes, Mario Giulianelli, Verna Dankers, Mikel Artetxe, Yanai Elazar, Tiago Pimentel, Christos Christodoulopoulls, Karim Lasri, Naomi Saphra, Arabella Sinclair, et al. A taxonomy and review of generalization research in nlp. Nature Machine Intelligence, 5(10):1661-114, 2023.
Shawn Im and Yixuan Li. A unified understanding and evaluation of steering methods. arXiv preprint arXiv:2502.02716. 2025,
Rishi Dev Jha, Collin Zhang, Vitaly Shmatikov, and John Xavier Morris. Harnessing the universal geometry of embeddings. The Thirty-ninth Annual Conference on Neural Information Process- ing Systems, 2025.
Yibo Jiang, Bryon Aragam, and Victor Veitch. Uncovering meanings of embeddings via partial orthogonality. Advances in Neural Information Processing Systems, 36:31988-320000 2023.
Ole Jorgensen, Dylan Cope, Nandi Schoots, and Murray Shanahan. Improving activation steering in language models with mean-centring, In Responsible Language Models Workshop at AAAI-24, 2024.
Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International Conference on Machine Learning, pp. 2668-2677. PMLR, 2018.
Omer Levy and Yoav Goldberg. Linguistic regularities in sparse and explicit word representations. In Proceedings of CoNLL, pp. 171-180, 2014.
Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang. Yiming Yang, and Lei Li. On the sentence em- beddings from pre-trained language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 9119-9999999999 9130,, 2020.
Kenneth Li, Aspen K Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Watten- berg. Emergent world representations: Exploring sequence model trained synthetic task. In The Eleventh International Conference on Learning Representations, 2022.
Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Plister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from a language model. Advances in Neural Information Processing Systems, 36:41451-41158 2023.
Sheng Liu, Haotian Ye, Lei Xing, and James Zou. In-context vectors: Making in context learning more effective and controllable through latent space steering. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference Machine Learning, volume 235 of PMLR, 21-27 Jul 2024. URL
Proceedings of Machine Learning Research, pp. https:://rocees igs. www.rrrrrrsssss3 112///////bbbb html
12

---

Published as a conference paper at ICLR 2026
Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. A sick cure for the evaluation of compositional distributional semantic models. In Proceedings of the Ninth International Conference Language Resources and Evaluation (LREC' 14), pp. 216-223, Reykjavik, Iceland, 2014. European Language Resources Association (ELRA).
Jack Merullo, Carsten Eickhoff, and Ellie Pavlick. Language models implement simple word2vec- style vector arithmetic. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume I: Long Papers), pp. 5030-5047, 2024.
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word represen- tations in vector space. In Proceedings of Workshop ICLR, 2013a.
Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. Linguistic regularities in continuous space word representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, pp. 746-751, 2013b.
David Mimno and Laure Thompson. The strange geometry of skip-gram with negative sampling. In Conference on Empirical Methods in Natural Language Processing, 2017.
Jeff Mitchell and Mirella Lapata. fxctor-based models of semantic composition. In Johanna D. Moore, Simone Teufel, James Allan, and Sadaoki Furui (eds.), Proceedings of ACL-08: HLT, pp. 246, Columbus, Ohio, June 2008. Association for Computational Linguistics. URL https: http://acaaantohooooooo www.aaaooooooooooo.oo0-----
wwt. 1028/
John X Morris, Rishi Bommasani, Aakanksha Naik, and Alexander M Rush. The linearity of cross- lingual word embeddings: A geometric analysis. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 7955-7964. Association for Computational Linguistics, 2020. doi: 10.18653/w1/2020 .wnll- macll. 641.
Neel Nanda, Andrew Lee, and Martin Wattenberg. Emergent linear representations in world models of self-supervised sequence models. In Proceedings of the 6th BlackboxNLPP Workshop: Analyz- ing and Interpreting Neural Networks for NLP, pp. 16-30, 2023.
OpenAl. texteembedding---largee. OpenAl API models announcement, 2024, announced Jamuary 25, 2024; 3072 dimensions, improved performance on MIRACL and MTEB benchmarks. Available from OpenAl API documentation.
Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models. In International Conference Machine Learning, 2024.
Kiho Park, Yo Joong Choe, Yibo Jiang. and Victor Veitch. The geometry of categorical and hi- crarchical concepts in large language models. In The Thirteenth International Conference on Learning Representations, 2025.
Van-Cuong Pham and Thien Nguyen. Houscholder pseudo-rotation A novel approach to activation editing in Ilms with direction ceoection-manntuuee perspective. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 13737-13751, 2024.
Emily Reif, Ann Yuan, Martin Wattenberg, Fernanda B Viegas, Andy Coenen, Adam Pearce, and Been Kim. Visualizing and measuring the geometry of bert. In Advances in Neural Information Processing Systems, volume 32, 2019.
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 3980-3990, 2019. doi: 10. 10.18653/1/DD19-14400
Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering lama 2 via contrastive activation addition. arXiv preprint arXiv:23/2.06688, 2023.
Anna Rogers, Olga Kovaleva, and Anna Rumshisky. A primer in bertology: What we know about how bert works. Transactions of the association for computational linguistics, 8:842-866 2021.
13

---

Published as a conference paper at ICLR 2026
Daniel Tan, David Chanin, Aengus Lynch, Brooks Paige, Dimitrios Kanoulas, Adrià Garriga- Alonso, and Robert Kirk. Analysing the generalisation and reliability of steering vectors. Ad- vances in Neural Information Processing Systems, 37:139179-1392222 2024.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXivv:3307.09288 2023.
Matthew Trager, Pramuditha Perera, Luca Zancato, Alessandro Achille, Parminder Bhatia, and Ste- fano Soatto. Linear spaces of meanings: compositional structures in vision-language models. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 15395-15404, 2023.
Alexander Matt Tumner Lisa Thiergart, David Udell, Gavin Leech, Ulisse Mini, and Monte Mac- Diarmid. Activation addition: Steering language models without optimization. arXiv preprint arXiv:2308. 10248, 2023.
Zihao Wang, Lin Gui, Jeffrey Negrea, and Victor Veitch. Concept algebra for (score-based) text- controlled generative models. In Advances in Neural Information Processing Systems, volume 36, pp. 35331-353499 2023.
Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman. Blimp: The benchmark of linguistic minimal pairs for english. Transactions of the Association for Computational Linguistics, 8:377-392, 2020. doi: 10. 1.2116cccaaa..002..1.
URL httptttts:///oo..oo/ http::://dooooor////// wtt.i/orr/....16/tteee... aa0_aaaa______________________1_
Andy Zou, Long Phan, Sarah Chen, James Camphell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engincering: A top-down approach ai transparency. arXiv preprint arXiv:2310.01405, 2023.
A MATHEMATICAL PROPERTIES OF RISE
These mathematical results support our main claims in the paper. Lemma provides the explicit exponential and logarithmic map formulas that underlie RISE's use of geodesies on the unit hyper- sphere. Theorem A. formalizes that sequential RISE edits commute up to second order, showing that different discourse- transformations can be applied in any order without significant distor- tion. This result highlights the local geometric consistency of RISE transformations, rather than im- plying global additive steering. Proposition A.I ssoows that each RISE transformation can be applied in (dd) time and memory, demonstrating the method's 's scalability to modern high-dimensional em- beddings. Together, these results provide theoretical grounding for both the geometric consistency and the practical efficiency reported in the main text.
A.1 GEOMETRY PRELIMINARIES ON THE SPHERE
We work on the unit sphere 1d-1 C Rdd with the standard round metric. For E Sd-1 the tangent space d far E IRd, (x. n) 0}. The exponential map exp,, T,, -> dd is deñned for all tangent vectors, while the logarithmic map log, is well-defined for all v gd except the antipode = n. For each n, fix an orthogonal map R(n) O(d) such that R(n)n = et where é = (1,0....,0)) 0)T When analyzing local behavior (e.g.. Theorem A. we take R(.) to be any C¹l (continuously differentiable) choice on a neighborhood of the geodesic segment(s) under consideration; such a local choice always exists.
Lemma 1 (Exxonnntial and logarithmic maps on the unit sphere). For E gd-1, langent vector Ç E T2 dd-1, and point v gd-1 (-n).
E
log,, (v) arccos( v)) V (n, u)n
(E) cos(( (in()l
|E|
||v (n, v|ml
Proof. These formulas follow from the fact that geodesics on gd-1 are great circles in IR (unit- radius sphere). See, e.g., Absil et al. (2008) Sec. 5.4).
14

---

Published as a conference paper at ICLR 2026
A.2 ROTOR CONSTRUCTION AND IMPLEMENTATION
In Clifford algebra terms, rolor is an element of Spin(d) that rotates vectors by the sandwich product x ++ rof, where denotes reversion. For our purposes, we only require an orthogonal
operator R(n) E O(d) with R(n)n 61 that depends smoothly on n. One closed-form rotor
mapping n + 61 (valid when n # -e1) 1e)_ss
r(n) = 1+e10 ein
r(n) ( = E1
2211+
In practice we realize this as a standard linear operator without explicit Clifford algebra structures. Two efficient ((d) realizations are:
Householder reflection: H(n) 1 with w n-611
61. which satisfies H(n)n c1
[|ll|
(determinant -1).
Givens rotation: 2 x 2 rotation in the plane spanned by {n, 61 extended by the identity elsewhere, with determinant +1.
Both satisfy the required conditions R(n)n and local C+ smoothness, and are numerically stable away from R -ej. In antipodal case (n es -e1) we use two-step construction; map an auxiliary orthogonal vector 1. 61, then to 61. In all cases, applying R(n) or R(n)² to vector costs (dd) operations.
A.3
COMMUTATIVITY PROPERTIES OF SEQUENTIAL RISE OPERATIONS
A.3.1 THE RISE SEQUENTIAL PROCEDURE
Given no gdd11 and prototypes PA,0B Te, Sd-1
Apply A: ČA R((o)) PA, n1 expro (EA),
Apply B: {B R(()) T PB, ne expn, (EB).
A.3.2 FIRST-ORDER COMMUTATIVITY ANALYSIS
Theorem A.1 (RISE commutativity to first order). For small prototype magnitudes || 1pal| <<
d(result of A B, result of B A) = O(1/Al -1)
Proof. Using Lemmal I expand exp... (EA) no & + O(l 12) Let 1/ E. Canonicalization
n1 no 1A O (12) differs from that no by (ll
Let Pay-ing Tn, T2, gdd 1 - Tr,S* Sh 1 denote parallel transport along the short geodesic from n1 to nn- On the unit sphere, Px, - 7|| O(Im no = Ollyaal), where denotes the identity
operator on the tangent space. With a C choice | [(())) R(no) = O(l - noll) O(IaaA]). Therefore.
Pai R(n:) pB R(no) pa + O(Inal [B))))
Now expand the second step:
no + R(no) (A T
PB) ((AA)) + O(1 (1
Swapping roles of A and B gives the same expansion with PA, Pn reversed. Subtracting yields difference of order [all
Geometric interpretation. Re-canonicalization is equivalent (to first order) to parallel- transporting the next step's 's vector back to the initial tangent space. On Sd-1 with constant curvature, order effects are second order.
15

---

Published as a conference paper at ICLR 2026
A.4 COMPUTATIONAL COMPLEXITY
Proposition A.1 (Per-transformation complexity). Each RISE transformation can be implemented in (d) time and (d) memory:
1. Canonicalization: applying R(n) or R(n)* costs O(d).
2. Logarithmic map log, (u): ((d) using Lemmal
3. Exponential map exp,, (E): (dd) using Lemmal 1
4. Storage: protolype ÿ E Te, gd-1 costs (d))
Comparison with matrix methods. achieves equivalent updates in (d).
Dense x d rotations require O(P) time and memory. RISE
Implementation note (Houscholder). A practical canonicalization is the Householder reflection
ww
H(n) T
|00 |
W
C1
which maps + e in (d). Since (n) is a reflection (det = 1),, it suffices for canonicalization. Near e1, may switch numerically stable alternative.
B CROSS-LANGUAGE TRANSFER ANALYSIS AND RESULTS
To test whether geometric transformations generalize across languages, we conducted comprehen- sive cross-language transfer experiments. This section reports detailed results across three models and three semantic phenomena, analyzing both quantitative performance and geometric properties of learned transformations.
KPY
476
670
6711
170
307
3PM
600
477
4341
24
170
180
510
6M0
470
409
Figure 5: Cross-language transfer heatmaps for bge-m3 model showing RISE performance across all language pairs for conditionality, negation, and politeness transformations. Darker colors indicate higher cosine similarity between predicted and target embeddings.
16

---

Published as a conference paper at ICLR 2026
Orat Freaor Caaifiuall Blch Bichenowdding
Craps Regertea
Nottendeeding Liarge
Sart cet--enteddiyyy.eooo Jannee
Pollae
63H
443
624
200
641
110
600
PP
011
kage: Orgegg
Figure 6: Cross-language transfer heatmaps for text-embedding---largrge model showing RISE per- formance across all language pairs for conditionality, negation, and politeness transformations. Darker colors indicate higher cosine similarity between predicted and target embeddings.
Traalfer: Canditinsiit
Coven-anguaps Trestfer:
IIDE Crose-Largaen Vtanafeer: Polks
ART
HED
SCW
PCS
Figure 7: Cross-language transfer heatmaps for mBERT model showing RISE performance across all language pairs for conditionality, negation, and politeness transformations. Darker colors indicate higher cosine similarity between predicted and target embeddings.
B. CROSS-LANGUAGE TRANSFER PERFORMANCE
The above heatmaps demonstrate comprehensive cross-language transfer results across our three models. Training rotor prototypes on one language and evaluating on others reveals promising cross-lin performance, particularly for negation and conditionality. Most language pairs show transfer scores above 0.70, with negation achieving particularly strong off-diagonal performance (most scores > 0.80).
Negation emerges as the most performant transformation, achieving the highest mean cross- language transfer scores (0.788 across all model-language combinations) with performance ranging from 0.686 to 0.918.
Conditionality demonstrates the highest stability and consistency across lross-ln transfers, with the lowest performance variability (0.038) and most stable individual measurements (0.056 average std deviation). Mean performance of 0.780 places it second overall.
Politeness shows more variation but still achieves substantial cross-linguistic success (most scores 0.70).
B.2 GEOMETRIC ANALYSIS OF CROSS-LANGUAGE CENTROIDS
Analysis of the learned centroids reveals additional insights into the geometric structure of semantic transformations. For each phenomenon, computed "ideal" transformation vectors by averaging canonicalized transformed embeddings across languages.
For negation, the centroids show high similarity across languages (pairwise cosines 0.95).
Conditionality centroids maintain high geometric consistency, supporting the observed stability in transfer performance across all model-language combinations.
17

---

Published as a conference paper at ICLR 2026
Politeness centroids cluster more loosely but still maintain substantial similarity (pairwise cosines 0.87).
B.3 QUANTITATIVE CROSS-LANGUAGE ANALYSIS
Table 4: Complete Cross-Language Transfer Matrix: Statistical Summary
Model
Phenomenon
All Transfers
Monolingual
Cross-Lang
Ratio
Conditionality
13.6x 0.7
14.5x
13.5x
0.93
TE3L (3072d)
Negation
19.7x + 1.2
20.6x
19.6x
0.95
Politeness
23.1x + 1.9
25.3x
22.8x
0.90
Conditionality
13.9x + 0.6
14.5x
13.8x
0.95
bge-m3 (1024d)
Negation
18.5x = 0.8
19.3x
18.4x
0.95
Politeness
25.2x 1.2
26.4x
25.1x
0.95
Conditionality
12.8x + 1.3
15.0x
12.5x
0.83
mBERT (768d)
Negation
18.0x 1.8
20.9x
17.5x
0.84
Politeness
20.8x + 3.9
26.1x
20.0x
0.77
Statistics computed across complete 7x7 language transfer matrix (49 language pairs per phenomenon). Values show advantage ratios I standard deviation across all language pairs.
Ratio indicates relative cross- languag transfer effectiveness (Cross-Lang/Monn Lang/Monolingual)
All models maintain strong tross-language performance (77% 95% of monolingual performance).
Table 5: Model Architecture and Overall RISE Performance Summary
Model
Dims
Validation Avg
Cross-Lang Avg
Random Adv
TE3L 3072
0.774
19.0x
6.3x
bge-m3
1024
0.790
19.8x
11.7x
mBERT
768
0.802
16.9x
11.9x
Validation Avg: Mean performance across Synthetic Multilingual, BLiMP, and SICK datasets. Cross-Lang Avg: Mean advantage ratio across Figlish- and lapanese-+Fnglins transfers. Random Adv: Mean advantage ratio over random baselines in monolingual English scenarios.
Bold values indicate best performance in each category.
Tables 4 and provide comprchensive quantitative analysis of cross-language transfer performance. Notably, all models maintain strong cross-language performance (77%-95% monolingual perfor- mance), with bge-m3 showing the most consistent cross-language effectiveness across all phenom- ena.
18

---

Published as a conference paper at ICLR 2026
C LINEAR BASELINES COMPARISONS
This appendix reports the full results for the linear baseline comparisons requested by the reviewers. We thank the reviewers for this valuable suggestion these results did strengthen our paper. We implemented two baselines: Procrustes alignment and Mean Difference Vectors (MDV). MDV is not truly Euclidean: it computes mean displacements using the manifold's geometry (via log/exp maps), preserving spherical structure. Thus MDV functions naturally resembles RISE more closely than Procrustes. We evaluated them alongside RISE on three datasets: BLiMP, SICK, and our multilingual synthetic dataset.
The strongest performance appears in monolingual English evaluation (BLiMP), while performance drops substantially for Procrustes on semantic relatedness (SICK) shown in Table 3 This shift in performance reflects Procrustes' inability to identify a generalizable semantie-syntactic relationship as expected by method. Procrustes fits single global rotation which is too rigid for the cross- lingual and cross model analysis In contrast, RISE maintains stable cross-lingual and cross-mode model performance (e.g., App. B Figures 5-7), indicating that geometric operations on the manifold better capture discourse-level semantic structure than Euclidean differences.
The MDV RISE vs. Procrustes results reinforce earlier claim that methods operating on the curved manifold (where sentence embeddings inherently reside) perform better than Euclidean/lin- ear methods. Most steering and probing techniques operate in linear space, and conjecture that this geometric mismatch helps explain why linear methods struggle to generalize. In short, Pro- crustes fits single global rotation which is too rigid for the cross-lingual and cross model analysis. Geometric transformations, like RISE and MDV, are better suited for semantic-syntactic analysis and cross- cooss-l stability.
C.I CROSS-LANGUAGE TRANSFER HEATMAPS
Figures 8 10 show cross-language cosine similarity for the three semantic transformations (Condi- tionality, Negation, Politeness) under Mean Difference Vectors (MDV), Procrustes alignment, and RISE.
IED Cundiiveally
Promedes Canditorall
RISS Candilendly
Transher
60%
200
670
183
1227
6211
AP1
605
870
1311
Text
Teak
(a) MDV
(b) Procrustes
(c) RISE
Figure 8: Cross- Cross-language transfer for Conditionality across seven languages.
C.2 NATURAL-LANGUAGE VALIDATION: BLIMP AND SICK
Figure TT reports mean cosine similarity on BLiMP (syntactic) and SICK (semantic) for the three methods.
19

---

Published as a conference paper at ICLR 2026
PEN Regaltet
Proucetes Megalaat
FIt Regation Tranafee
UTA
LPR
CAP
680
Balh Eenhy
Byba
(a) MDV
(b) Procrustes
RISE
Figure 9: Cross language transfer for Negation across seven languages.
MOV Polleness Langeage Twrnter
Procrsshes Polleeeey
RISC Poltenar
Longgaage Trensher
CEC
AAU
EXTT
AS1
KZY
9179
CTR
K30
Lengless
(a) MDV
(b) Procrustes
(c) RISE
Figure 10: Cross-language transfer for Politeness across seven languages.
Natural Language Validation: Datasets
1.00
0.9873
0.9720
0.9094
1.00
0.50
0.50
Mean Cosina Similarity
Mean Cosine Similarity
0.85
0.3826
0.8828
0.80
0.50
0.75
0.75
0.65
0.65
0.00
RISE
NDV
Pochusass
RISE
NEV
Procrustio
Figure 11: Natural language validation on BLiMP (syntactic acceptability) and SICK (semantic relatedness) for RISE, MDV, and Procrustes. Error bars denote standard deviation across examples.
20

---

Published as a conference paper at ICLR 2026
D PROMPT TEMPLATES
We provide the exact prompt templates used to generate neutral sentences and their semantic vari- ants. Each template is shown in monospace using the lstlisting environment for clarity and reproducibility.
D.I NEUTRAL SENTENCE GENERATION
You Linguistics assistant. Cenerate ONE terse, blunt English sentence that pol ienitsse--muttall:
must neither explicitly polite nor impolite. Keep it concise (8 to 12 words), direct, and free of polite markers such as "please", honorifica, hedging, or apologies, yet ensure it is not rude. If the situation contains placeholder (e.g., favor", "a cultural practice"), replace it with a cencrete, plausible example.
Context catcgory: fcatogory) Detailed situation: (examplc)
Respond with ONLY the single sentence (no explanations, no quotation marks).
D.2 POLITENESS REPHRASING
You are expert translator and pragmatics specialist. Rewrite the following sentence in flanguage_name) to make it more POLITE while preserving its original meaning. Incorporate the given politeness features.
Sentence: "(sentence)"
Foliteness features (JSON) : (features_json)
Respond ONLY with JSON object in the exact format: ("pol ite": "crewritten sentence>") Do NOT add any other keys, explanations, markdown.
D.3 NEGATION
You are an expert translator and semantics specialist. Rewrite the following sentence in [language_name) so that it expresses the NEGATION of its original meaning while remaining natural and fluent. Incorporate the given negation features.
Sentence: "(sentence)"
Negation features (JSON) [features_isen)
Respona ONLY with JSON object in the exact format: ("negation' "<rewritten sentence>") Do NOT add any other keys, explanations, or markdown.
D.4 CONDITIONALITY
You are expert translator and syntax/pragmatics expert. Rewrite the following sentence in (language_naml [amage so that the atatement becomes CONDITIONAL (i.e., it enly holds under a certain condition) while preserving overall meaning and sounding natural. Incorporate the provided conditionality features.
Sentence: " (senteence"
21

---

Published as a conference paper at ICLR 2026
Conditionality features (JSON) (features_json)
Respond ONLY with JSON object in the exact format: ("conditicnality"" "<rewritten sentence>") Do NOT add any other keys, explanations, markdown.
E DATA GENERATION METHODOLOGY
E.1 DIVERSITY CONTROLS
To guard against artifacts that might arise from narrow lexical or topical coverage, we apply several sampling diversity control. (1) Each neutral sentence prompt draws its situation description from randomly chosen context category and exemplar, yielding a wide topical spread before any trans- formation is applied. (2) Within every language we shuffle sentence-feature assignments so that no specific lexical field correlates with a particular transformation subtype. (3) For each transformation uniformly sample property values (c.g., negation particle, politeness strategy) per language and sentence, guaranteeing that every combination language and subtype appears the same number of times. (4) After generation remove near-duplicates and enforce 5-25 token length window, which empirically yields a near-uniform length distribution. Together these steps ensure that our corpus varies in topic, syntax, and lexical choice while remaining balanced across languages and transformation subtypes. These controls ensure that observed geometric patterns reflect semantic properties rather than artifacts of lexical choice sentence structure.
1. Topical Diversity: Neutral sentences were drawn from varied context categories (social interactions, factual statements, requests, etc.)
2. Feature Balance:
Transformation features (e.g., negation particles, politeness strategies) were uniformly sampled to prevent correlation with specific lexical fields.
3. Length Normalization: Sentences were filtered to 5-25 tokens to ensure comparable em- bedding properties.
4. Deduplication: Near-duplicate outputs were removed to prevent repeated data.
E.2 PEATURE-BASED TRANSFORMATION METHODOLOOGY
We generated sentence pairs systematically by first sampling neutral sentences seven typologically diverse languages (English, Spanish, Tamil, Thai, Arabic, Japanese, and Zulu), and subsequently transforming each sentence using conture-contollled prompts. Each transformation was guided by uniformly sampling linguistic features from predefined typological metadata set (illustrated be- low).
The full inventories of typological properties for politeness, negation, and conditionality are pro- vided Tables
E.2. TRANSFORMATION PROCEDURE
For each neutral sentence, we uniformly sampled exactly one set of feature values from the typologi- cal metadata and prompted the language model (GPT-4.5) to generate the transformed variant adher- ing these specifications. By uniformly sampling across multiple typological dimensions-strasd strrttey types, morphological realizations, and pragmatic contexts we ensured comprehensive coverage of each language's linguistic variability. This methodology supports -ross-linguistic embedding anal- ysis and ensures that observed embedding-spaat transformations reflect typological distinctions ac- curately.
E.3 FEATURE- CENTURE-CONTROLLED PROMPTING
To generate each transformation in systematic and reproducible manner, employ feature- controlled prompting strategy with large language model (LLM). Each prompt is carefully tem- plated to specify the source language, the desired transformation type, and set of fine-grained
22

---

Published as a conference paper at ICLR 2026
Language English
Strategy Type
Grammatical/Lexice Devices
Negative politeness
Modal conditional, hedging.
idiomatie/proverbial. taboo avoidance
Spanish
Positive politeness
Modal conditional, morpholog- ical politeness, hedging, iomatic/proverbial
Tamil
Relational/Kinship po- liteness
Morphological politeness
Thai
Positive politeness; Re-
Morphological modal conditional
politeness,
lational/Kinship
Arabic
Positive politeness; Re- lational/Kinship
Modal conditional, morpho-
logical politeness, idiomat- ic/proverbial Morphological politeness,
Japanese
Relational/Kinhip po- liteness
modal conditional, hedging
Zulu
Relational/Kinshpp po- liteness
Morphological politeness
Table 6: Typological features sampled uniformly for politeness transformations.
Language English
Marker Position
Morphological Realization
Clause-medial
Negative particle; negative aux/modal; negative affix
Spanish
Mause-medial;; concord
Negative particle
Tamil
Clause-fiie final
Negative particle; verb verb-internal
negation
Thai
Mause-medial Clause-initial
Negative particle
Arabic
Negative particle; negative affix
medial
Japanese
Clause-fine final
Verb-internal negation
Zulu
Clause-medial
Negative particle
Table 7: Typological features sampled uniformly for negation transformations.
feature tags that guide the model's output. For example, a prompt might indicate the language code ("[TA]" for Tamil), the transformation ("Politeness Rephrase"), and particular strategy or keyword (such "add honorific") relevant to that transformation. By explicitly encoding these features, ensure that the LLM produces the intended variation variation-whether a more polite rephrasing, a negated statement, conditional construction in consistent and transparent way.
To further guarantee balanced coverage, we maintain a metadata table that enumerates all possible sub-types or strategies for each transformation. This enables mation features across languages and sentences, ensuring that every variant type is equally repre- sented. For instance, multiple politeness strategies (e.g., adding honorifies, using indirect language) or different negation words ("no" vs. "not") are distributed uniformly across the dataset. This con- trolled coverage is critical for fair comparisons: prevents any language from being overrepresented by particular style of rephrasing or negation, and minimizes inadvertent correlations between lan- guage and transformation realization. Our stratified sampling approach follows established princi- ples of controlled experimental design, providing robust foundation for cross-lingual embedding analysis.
stratify the sampling of transfor-
All transformed sentences are generated using single, consistent LLM (GPT-4.5) with temper- ature of 1.0 and maximum token limit of 128 per prompt. The relatively high temperature en- courages diversity in phrasing, while the one-shot generation policy (taking the first model output without retries or manual curation) avoids selection bias. With carefully constructed prompts, the model reliably produces valid transformations the first attempt, and all outputs remain in the tar- get language specified by the prompt. This procedure ensures that our dataset both systematically varied and reproducible, supporting rigorous downstream analysis.
23

---

Published as a conference paper at ICLR 2026
Language
Clause ture Initial; final: em- bedded
Strue-
Morphological Marking
English
Explicit marker; conditional
tense/aspect Conditional mood; explicit marker Explicit marker; mood
Spanish
Initial; final; em- bedded
Tamil
Final
conditional
Thai
Initial Initial; final
Explicit marker
Arabic
Conditional mood; marker Conditional mood; marker
explicit
Japanese
Final: embedded
explicit
Zulu
Initial
Conditional mood;
explicit
marker
Table 8: Typological features sampled uniformly for conditionality transformations.
E.4 QUALITY CONTROL AND DEDUPLICATION
To ensure the integrity and uniqueness of our dataset, we implemented rigorous two-level dedu- plication process. At the first level, we removed any transformed sentence that was exactly identical to another within the same category and language. This step addresses the possibility that the LLM might produce identical outputs for different inputs, especially for short or formulaic sentences. At the second level, we ensured that each (neutral, variant) pair was unique across the entire dataset. In rare cases where two different source sentences yielded the same transformed output, treated this as a collision and regenerated new variant using a slightly altered prompt. Through this process, every neutral sentence in our dataset is paired one-to-one with three distinct transformed sentences (one per transformation type), with no overlaps. The result is a clean set of sentence pairs, each difference.
exhibiting a unique, mansformation-drive
Beyond deduplication, we applied a suite of diversity controls to guard against artifacts arising from narrow lexical topical coverage. Each neutral sentence prompt was drawn from wide range of context categories and exemplars, ensuring topical breadth before any transformation was applied. Within each language, we shuffled sentence-feature assignments that no specific lexical field cor- related with a particular transformation subtype. For each transformation, we uniformly sampled property values (such as negation particles or politeness strategies) per language and sentence, guar- anteeing that every combination of language and subtype appeared the same number of times. After generation, we removed near-duplicates and enforced a 5-25 token length window, which empiri- cally yielded near-uniform length distribution. Together, these steps ensure that our corpus varies in topic, syntax, and lexical choice while remaining balanced across languages and transformation subtypes, providing robust foundation for subsequent embedding analysis.
E.5 EMBEDDING GENERATION
With our dataset of neutral and transformed sentences in hand, we next obtain high-dimensional vec- tor representations using a state-of-the-arr multilingual sentence encoder. Specifically, we employ OpenAl's text-eebeede dedinddiingg--g -arrge model, which produces 3072-dimensional embeddings aligned semantically across more than 90 languages All embeddings are generated in frozen setting, with a single API call per sentence. According to the model card, each sentence embedding computed by mean-poolee mean--ooling the token-level hidden states, followed by layer normalization. This means that every token-including short functional items like negation parti- coes--cnts proportionally to the final vector.
Our approach assumes that all sentence embeddings reside in shared semantic space where linear structure is meaningful. We adopt the perspective that this space forms a latent manifold encoding universal semantic features, hypothesized by Jha et al. (2025). In this framework, certain direc- tions in the embedding space correspond to specific attributes, such as politeness or negation. sentence transformations truly correspond to adding subtracting semantic attribute, expect
httts:://paaaeeee openai www.///cc/cccacccccc www.ueiwieeeeeemo
hww.ddeeddeeeeeeeeeeeeeeeeeeedee emweddibeeeeeoeeem ngs
24

---

Published as a conference paper at ICLR 2026
the difference vector (variant minus source) to be relatively consistent examples. This aligns with the "universal geometry for embeddings" framework, in which multilingual embeddings from different models languages can be brought ences are captured by geometric translations. While our work stays within a single encoder's space, leverage similar idea: analyzing whether the transformation "rotors" (difference vectors) clus- ter for similar transformations across languages. This methodology sets the stage for validating whether these quasi- linear transformations indeed behave like translations in a Riemannian seman- tic space (Jha et al., 2025), which we explore in the next section via rotor-based analysis of the embedding differences.
common representation where semantic differ-
It is important to note that applying a single global rotation or principal component analysis (PCA) can distort other dimensions and not adaptive to individual vectors. Because the base embedding is already mean across tokens, edits that insert or replace a handful of tokens translate to small but coherent rotations of the global vector- veecer-precisely the kind of local, content-independent shift that our rotor method is designed capture.
E.6 FINAL DATASET STATISTICS
The resulting corpus comprises 1,000 neutral sentences each the seven languages, totaling 7,000 examples. For English neutral sentences, the mean token length is 9.1 tokens (with median of 9.0 tokens), with token counts ranging from three to 12 tokens and an average character length of 54.4 characters. This distribution confirms that our generation process produced concise, natural sentences suitable for semantic transformation analysis across languages and transformation types.
To further validate the diversity and balance of our dataset, we analyzed the distribution of sentence lengths per language, which reveals broadly similar profiles with peak around 10-15 tokens. Addi- tionally, we examined the distribution of word frequencies, confirming typical long-tail distribution in each language. These statistics affirm that our corpus is both balanced and rich in content, pro- viding a solid empirical foundation for the cross-lingual transformation analysis in the subsequent sections.
F LLM USAGE DISCLOSURE
Large language models (LLMs) were used assist with multiple aspects of this research, includ- ing: ideation, writing, programming, and implementation of experimental code, and identification of related work and literature. All L-M--genrrtted content, code, and references were subject to human review, testing. and verification to ensure accuracy, functionality, and relevance. Any claims, results, experimental implementations, and citations presented this work have been reviewed by the authors. The authors take responsibility for all content, including any errors or inaccuracies that may remain despite our review process.
G DOWNSTREAM TASK ANALYSIS
As requested by reviewers, we completed a downstream classification analysis. Due to time con- straints, we focused on single well-defined task: detecting negation the English subset of the Synthetic Multilingual dataset. We evaluated how well classifier trained on MDV-transfrmmed and cus--trrasformed sentences performed on a held-out test of 1919 unpaired sentences (961 with negation, 958 without). The test set was generated with the same specifications described in Appendix D
Now, both methods perform well on this task. MDV achieves strong recall (92.1%) and overall accuracy (87. (87.2%), showing that even a simple mean displacement vector captures meaningful geo- metric regularities in the transformation. Yet, RISE yields a stronger downstream performance and outperforms MDV across all metrics (93.0% accuracy, 92.1% precision, 94.0% recall, and 93.0% F1). The positive results of both methods reinforces the broader claim that spherical, non-linear techniques are effective tools for capturing cemantic-syntacti transformations in nigh-dimensional embedding spaces.
25

---

Published as a conference paper at ICLR 2026
Method
Accuracy
Precision
Recall
F1
MDV RISE
0.872
0.840
0.921
0.878
0.930
0.921
0.940
0.930
Table 9: Downstream negation classification performance for MDV and RISE transformations.
H RISE VS RANDOM BASELINE COMPARISONS
This section presents comprehensive comparisons between RISE and random baseline prototypes to validate that RISE learns meaningful semantic directions rather than benefiting from arbitrary vector orientations. The following figures show detailed graphs, heatmaps, and tables comparing RISE performance against random prototypes of equivalent magnitude across all language pairs and phenomena. Each comparison uses 10,000 random trials ensure statistical robustness.
Figure 12 highlights three select language transfer scenarios and Figures T3 13-15 demonstrate the
baseline validity of RISE by comparing against random prototypes across multiple language transfer scenarios. The consistent and substantial advantages (ranging from 5.1x to 26.2x) across all models and phenomena provide crucial validation that RISE learns meaningful semantic direc- tions rather than exploiting statistical artifacts. Notably, cross-language transfers often maintain or even exceed monolingual performance relative to random baselines, confirming that RISE captures universal semantic patterns that generalize across language boundaries. Overall, RISE analyses show that embedding models encode some transformations as universal operators, but others remain highly culture- and peoource-depnderee Future research should refine evaluation benchmarks to account for phenomenon-speciieee variability and investigate training regimes that promote balanced universality without sacrificing discriminative capacity.
26

---

Published as a conference paper at ICLR 2026
Figure 12: RISE vs Random Baseline Comparisons across select language transfer scenarios.
Top: English monolingual analysis.
Middle:
English prototype - Spanish target cross-language transfer. Japanese prototype English target cross-language transfer.
Bottom:
RISE vs Random Prototype Comparisen: English Prototype English Target (10,000 Random Trials per Condition) Negation
Conditionality
Polite
Vuerty 2011
0.0
00
RISE Random Comparison: English Prototype Spanish
(10,000 Random Trials per Condition) Negation
Conditionaiiy
Polite
02
RISE vs Random Prototype Comparison: Japanese Prototype English Target (10,000 Random Trials per Condition) Negation
Conditionality
Polite
5.44
brreet
27

---

Published as a conference paper at ICLR 2026
Figure 13: RISE Random Baseline Comparison for (exteembeddinn---laree Top row shows RISE performance, bottom row shows random baseline performance (averaged over 10,000 trials). The dramatic performance gap demonstrates that RISE learns meaningful semantic directions rather than benefiting from arbitrary vector orientations.
Advantage
(3072d)
(All Cross-Language Transfer Pairs)
Conditionalit
Negation
Polite
11.5
11.04
Orghin
11.
12A
Drp
20.4
25.1
23.1
11.
20.4
BLT
6:44
1i.b
68:04
20:4
24:
20:11
Tund
P-A
280
DX
2.
2.5
CA
210
Sigeet Langaage
Figure 14: RISE vs Random Baseline Comparison for bge-m3. Top row shows RISE performance, bottom row shows random baseline performance (averaged over 10,000 trials). Bge-m3 shows markably consistent RISE performance across all phenomena and language pairs, with random base- lines consistently near zero
RISE vs Random Advantage Ratios: BGE-M3 (1024d) (All Cross-Language Transfer Pairs)
Conditionality
Amcic
11.11
$1.05
BIX
2.5
JoNartage tatoo TREE Per.e-Paarooa Pardbni
Spmnke
1/2%
5/2
38C44
21
2-5
305
35
Jaanate
paaaane
F1:
1:B
T14
F//M4
13
7al
00
F:
2.A
2/6
Language
Figure 15: RISE Random Baseline Comparison for mBERT. Top shows RISE performance. bottom row shows random baseline performance (averaged over 10,000 trials). mBERT demon- strates strong RISE performance for specifie phenomena with clear superiority over random base- lines across all conditions.
RISE vs Random Advantage Ratios: mBERT (768d)
(All Cross-Language Transfer Pairs)
Conditionality
Negation
Polite
Eng
BLT
11.5
Orghih
17:B
JANN
26
21
2024
33.4
PLD
13
SUN
SUY
LUN
Swed
JKY
LTM
Temal
Tha
28

---

Published as a conference paper at ICLR 2026
H.1 PHENOMENON- SPECCIFCC PERFORMANCE VS RANDOM BASELINES
Figures (eeessT6poorro provide crucial validation that RISE's strong performance stems from learning meaning- ful semantic transformations rather than exploiting statistical artifacts benefiting from arbitrary vector orientations in high-dimensional spaces.
Figure 16: Phenomenon-specine RISE performance random baselines across all three models. Shows mean normalized improvement scores for conditionality, negation, and politeness compared random prototype baselines. Error bars represent standard error of random baseline (10,000 trials), All RISE performance significantly exceeds random baselines, with advantage ratios ranging from 5.1x 15.2x.
RISE Performance by Linguistic Phenomenon
English Monolingual: Learned Prototypes Random Baselines
Parformance Compartson by
0.6-
0.198
0.817
Falowess 2060)
by
Similarity
Follwesss jerdyri
0.-
0.4
0.2-
0.0
Embedding Model
H.2
DETAILED BASELINE COMPARISON ANALYSIS
demonstrate the statistical robustness of our findings. All RISE advantages are sta- tistically significant (p 0.001) with ultra-precise standard errors from 10,000 independent trials. Cross-language transfer often outperforms monolingual scenarios, demonstrating universal semantic patterns learned by RISE across language boundaries.
Tables 10
Table 10: RISE vs Random Prototype Performance: English Monolingual Analysis
Model
Phenomenon
RISE Perf
Random Baseline
Ady Ratio
Conditionality
0.463
0.057 = 0.0003
8.1x
TE3L (3072d)
Negation 0.210
0.041 = 0.0002
5.1x
Politeness
0.181
0.031 + 0.0002
5.8x
Conditionality
0.610
0.057 = 0.0003
10.7x
bge-m3 (1024d)
Negation
0.391
0.041 + 0.0002
9.5x
Politeness
0.461
0.031 + 0.0002
14.9x
Conditionality
0.625
0.057 + 0.0003
11.0x
mBERT (768d)
Negation
0.624
0.041 + 0.0002
15.2x
Politeness
0.294
0.031 + 0.0002
9.5x
Random baseline computed from 10,000 random prototypes of equivalent magnitude.
Standard errors shown for random baselines (+SEM). Adv Ratio RISE Performance/ Random Baselinc.
All models show significant advantages random baselines 15..2x...
29

---

Published as a conference paper at ICLR 2026
Table 11: Cross-Language Transfer Performance: RISE vs Random Baselines
Transfer Scenario
TE3L (3072d)
bge-m3 (1024d)
mBERT (768d)
English Prototype + Spanish Target
Conditionality
14.4x
14.2x
13.8x
Negation
19.6x
19.0x
19.0x
Politeness
25.2x
25.5x
21.6x
Japanese Prototype + English Target
Conditionality
13.7x
14.7x
11.7x
Negation
17.6x
18.9x
16.7x
Politeness
23.5x
26.2x
18.6x
Cross-Language Average
19.0x
19.8x
16.9x
Monolingual Average
6.3x
11.7x
11.9x
Values show advantage ratios (RISE Performance / Random Baseline). Cross-language transfer often outperforms monolingual scenarios.
Demonstrates universal semantic patterns learned by RISE across language boundaries. Random baselines consistent across all language pairs manguagg--anostign
Table 12: Statistical Robustness: Random Baseline Validation
Phenomenon
Random Mean
Standard Error 95% Confidence Interval
Conditionality 0.0567
0.000276
[0.0562, 0.0572]
Negation
0,0412
0.000200
[0.0408, 0.04
Politeness
0.0315
0.000154
[0.0312, 0.0318]
Random baselines computed from 10,000 independent trials per phenomenon.
Ultra-precise standard errors (4-6 decimal places) ensure statistical robustness.
Confidence intervals demonstrate consistent, language-agnostic random performance. All RISE advantages are statistically significant (p 0.001).
Table 13: Phenomenon-Speccffe RISE Performance Analysis
Phenomenon
Complexity
Avg Performance
Consistency
Politeness
High Medium
0.312
High (a 0.134) Very High (a 0.081)
Conditionality
0.566
Negation
Low
0.408
High (o 0.207)
Complexity based on linguistic theory and cross-language variation.
Avg Performance computed across all models and language pairs.
Consistency measured by standard deviation across models (lower more consistent). Conditionality shows highest consistency, suggesting universal semantic patterns.
30


Reviews:
[{"review_id": "DfRf5XKx8R", "reviewer": "Reviewer_FJxu", "summary": "The paper proposes Rotor-Invariant Shift Estimation (RISE), a geometric framework that treats discourse-level semantic transformations (negation, conditionality, politeness) as rotations/geodesic displacements on the unit hypersphere of sentence embeddings. The method canonicalizes neutral transformed sentence pairs via an orthogonal operator R(n), averages tangent-space shifts to learn a prototype, and predicts unseen transformations using the exponential map back to the sphere. Experiments cover 3 embedding models (OpenAI text-embedding-3-large, BGE-M3, mBERT), 7 languages, and 3 datasets (a GPT-4.5-generated synthetic multilingual set, BLiMP, SICK). The authors report strong cross-language transfer especially for negation and provide first-order commutativity guarantees and O(d) complexity for each RISE edit.", "questions": "1. Could the authors include additional non-synthetic datasets (e.g., parallel or human-annotated corpora) to validate that RISE captures genuine semantic transformations rather than synthetic prompt artifacts?\n2.  Could the authors implement linear baselines (e.g., additive Euclidean, Procrustes) to clarify where the spherical treatment demonstrably preserves compositionality or rotation consistency that linear mappings fail to capture?\n3.  Have the authors considered testing RISE on downstream semantic tasks (e.g., classification, retrieval, or transfer) to assess whether its geometric properties translate into functional semantic gains?\n4.  Could RISE be extended to multimodal representations? If not, what are the theoretical or practical obstacles preventing such an extension?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper presents a strong geometric and theoretical foundation, with clear use of log/exp maps, orthogonal canonicalization, and formal proofs of commutativity and O(d) complexity.\n2. Cross-lingual results show that negation and conditionality transfer consistently across seven diverse languages, and the paper thoughtfully contrasts logical versus pragmatic phenomena such as politeness.\n3. Cross-model transfer (text-embedding-3-large → BGE-M3) demonstrates practical portability while revealing architecture- and data-driven biases in embedding geometry.", "weaknesses": "1.\tThe evaluation relies heavily on synthetic data generated by GPT-4.5, with limited validation using human or parallel-corpus data. This dependence risks encoding generation artifacts and Anglo-centric prompt biases into the learned prototypes, potentially inflating cross-lingual consistency and reducing ecological validity.\n\n2.\tThe baseline comparisons are insufficiently described. The current experiments report results primarily within the RISE framework, without inclusion of clear linear or geometric alternatives—such as additive Euclidean shifts, orthogonal Procrustes alignment, or other linear mapping baselines. These would help determine whether RISE’s spherical geometry truly provides benefits beyond standard linear transformations. As a result, it remains unclear whether the observed gains arise from geometric innovations or from data-specific and normalization effects.\n\n3.\tEffectiveness is currently measured solely through average cosine similarity. While informative, this metric captures only local geometric alignment. Evaluating RISE on downstream tasks—such as classification, semantic retrieval, or analogy completion—would strengthen the empirical claim that its geometric transformations correspond to functional semantic improvements.\n\n4.\tThe evaluation is confined to text-based embeddings. Given that RISE operates in a general geometric latent space, it would be valuable to extend analysis to multimodal encoders (e.g., CLIP, BLIP-2). Such experiments could test whether RISE’s discourse-level “rotations” generalize across modalities, providing stronger evidence for universal geometric semantics rather than text-specific artifacts.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XhKmcmDxYI", "reviewer": "Reviewer_i6Zc", "summary": "The paper introduces Rotor-Invariant Shift Estimation (RISE), a geometric framework for understanding and controlling semantic transformations in multilingual embedding spaces. It treats meaning changes as rotational operations on the curved manifold where sentence embeddings lie, rather than as linear shifts in Euclidean space. Evaluated across multiple models and seven morphologically diverse languages, the method shows that certain discourse-level semantic transformations exhibit consistent geometric structure across both languages and model architectures.", "questions": "1. **Universality:** \nIf cross-model experiments reveal a consistent English-centric bias, how can the authors justify the claim of universal geometric operations across languages?\n\n2. **Semantic vs. Grammatical Structure:** \nGiven that RISE achieves near-perfect results on syntactic (BLiMP) tasks but only moderate performance on semantic (SICK) tasks, how do the authors support the claim that RISE captures semantic rather than primarily grammatical transformations?\n\n3. **Synthetic Data Validity:**\nSince all data were generated by GPT-4.5 without human or native-speaker validation, how do the authors ensure that RISE learns genuine linguistic geometric structures rather than artifacts of the data-generation model’s English-centric bias?\n\n4. **Experimental Scope and Generalization:**\nWith experiments limited to three linguistic phenomena (negation, conditionality, politeness), how can the authors substantiate broader conclusions about general geometric principles in embedding spaces?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper provides a principled geometric extension of the Linear Representation Hypothesis to non-Euclidean (Riemannian) spaces, moving beyond heuristic linear probing.\n\n2. The study’s inclusion of seven languages and three embedding architectures (OpenAI, BGE-M3, mBERT) demonstrates broad applicability and robustness.", "weaknesses": "1. **Limited empirical grounding for broad claims.**\nThe paper’s experiments are not sufficiently strong or comprehensive to substantiate its broader theoretical claims. For instance, it asserts “theoretical contributions to the usefulness of steering vectors for embedding models’ controllability and interpretation,” yet the evidence presented does not convincingly bridge this conceptual gap. The “steering vector” literature predominantly addresses interventions in the activations of generative or autoregressive models, whereas this work focuses on embedding models, a fundamentally different setting. Without stronger empirical validation or comparative studies, the claimed connection to the steering-vector paradigm remains tenuous.\n\n2. **Mismatch between ambition and experimental scope.**\nThis limitation reflects a broader issue: the paper makes overgeneralized theoretical claims based on a narrow experimental design. By restricting its evaluation to a small set of highly specific discourse-level transformations, the work lacks the empirical breadth needed to justify its sweeping conclusions about universal geometric principles or semantic control in embedding spaces. The authors acknowledge this limitation, but it significantly constrains the strength of their theoretical arguments.\n\n3. **Incremental methodological novelty.**\nDespite its mathematical rigor, the proposed shift from linear to rotational/geodesic steering may appear incremental rather than groundbreaking. The core contribution lies primarily in applying standard Riemannian geometry tools (e.g., exponential and logarithmic maps on the hypersphere) to embedding analysis, rather than introducing fundamentally new geometric or algorithmic mechanisms. As a result, the perceived innovation may fall short of the paper’s ambitious framing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "z0gI88qUTT", "reviewer": "Reviewer_kkBK", "summary": "The paper aims to characterise discourse and sentence-level semantic changes in encoder models as geometric transformations. To do that Rotor Invariant Shift Estimation (RISE) is introduced.  Given pairs of embeddings characterising a semantic transformation, RISE finds a rotation per pair which maps the tangent direction (in a spherical manifold) between both embeddings in the pair to a canonical direction. From these, a prototype for the semantic transformation is estimated and later used for inference.\n\nThe results show that the geometric rotations learned by RISE work for three semantic changes (Negation, Conditionality and Politeness). Furthermore, these transformations are shown to hold in a cross-linguistic setup. When testing RISE for cross-model transfer, results show that performance is language-dependent but generally satisfactory. Finally, RISE is shown to perform better at syntactic variations than semantic ones, as elicited by the BLIMP vs SICK performance.", "questions": "* Why is there one transformation per pair? Wouldn’t it be easier to learn a single transformation per semantic change solving least-squares?\n\n* Is RISE specific to encoder models with representations in the unit sphere? Can the findings be extended to more models?\n\n* How is pooling done (last embedding, mean-pooling …)? What layer of the model is picked?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "* The paper is clearly written and the setup is simple.\n* The model transfer results are interesting and show that simple transformations can grasp syntactic or discourse level binary features.", "weaknesses": "There are several relevant works which are not cited or mentioned:\n\n* Semantic changes have been previously modelled with linear transformations\n     - (Baroni and Zamparelli, 2010) https://aclanthology.org/D10-1115/\n     - (Mitchell and Lapata, 2008) https://aclanthology.org/P08-1028/\n\n* Non-euclidean semantic and syntactic probing\n     - (Chen et al., 2021) https://arxiv.org/abs/2104.03869\n\nThe work is currently limited to models that normalise their representations to the unit sphere, which is not the case for all encoders. Furthermore, autoregressive large language models are not evaluated. Without further experiments, these limitations heavily impact reach of the contribution.\n\nNegation and Conditionality are not only semantic changes but also syntactic ones. If the contribution is about semantics, it should control for syntactic structure. This conflict is accentuated by the last result, showing that BLIMP performance is higher than SICK. Understanding how syntactic changes are reflected in the models’ representations is an interesting direction, but this falls out of the scope of the paper (as reflected by the current title and introduction).\n\nSteering can be a motivation for the paper but such methods are not used in any of the experiments to support the importance of the geometric rotations for downstream model behaviour. It seems odd to include “steering” in the title and heavily introduce it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9MH8xGlVWN", "reviewer": "Reviewer_vohU", "summary": "This paper proposed a new methodology for steering embedding models. Instead of learning a steering vector, as is commonplace in the literature, the authors propose to learn a rotation operation which operates directly on the spherical manifold on which the embeddings live.\n\nThey call their proposed method Rotor-Invariant Shift Estimation (RISE) which, to my understanding, operates in three steps: first the source and destination embeddings are mapped to a canonical direction via an orthogonal transformation (a rotor). The semantic different between them is expressed in a standardised coordinate system (eq. 1). These \"semantic changes\" are averaged to create a \"prototype\" (eq. 2), which is then used to predict the semantic transformation of a unseen source embedding (eq. 3).\n\nThe performance of their method is evaluated via a series of experiments on a collection of model across three semantic transformations: negation, conditionality, and politeness, and several languages of different morphological types. This is quite comprehensive and makes clear the tasks on which the method performs well and less well.", "questions": "Re. the second bullet point in Weaknesses, how does the method compare with Euclidean steering vectors?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The proposed method is simple yet innovative and novel; the paper is well-written and presented, and is highly enjoyable to read. On this basis I strongly recommend acceptance of this paper.\n\nIn particular,\n- the background, motivation and prior work is well exposed and the authors do a very good job of placing their method in the context of the literature\n- the method itself is in principle very simple but the novelty and innovation comes from learning a transformation which respects and operates directly on the manifold in which the embeddings live\n- the method is described very clearly, though I think it could benefit from additional explanation and disucssion to benefit reads not familiar with Riemannian geometry (see Weaknesses)\n- the experiments provide a comprehensive understand of when and where the method performs well, and when it performs less well. The discussion of this is fairly extensive and enlightening\n- Sections 7-10 provide extensive discussion of the work in context, which I think is really valuable for the reader of a paper like this", "weaknesses": "As I mentioned before, I think this paper is very strong, so the following should be viewed a suggestions for improvements, rather than weaknesses per se.\n\n- I think the explanation of the method is very mathematically clear, however I think the paper could be made more accessible to readers without knowledge of Riemannian geometry (likely many readers who might use this method), by providing some more discussion of the geometric transformations which are taking place. Perhaps even an illustration of what is happening at each stage could be very enlightening. I don't think notions such as tangent spaces and Riemannian logarithms and exponentials will necessarily be familiar to a large portion of the audience who would find this method useful.\n- I would like to see a direct comparison with the method of Euclidean steering vectors for all the experiments you perform. I would seemingly be a very simple thing to do and would really show where the potential advantages of this method lie.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)