Title: Steering Embedding Models with Geometric Rotation: Mapping Semantic Relationships Across Languages and Models
Primary area: interpretability and explainable AI
Keywords: Embedding Models, Steering Vectors, High Dimensional Geometry, LLMs, Unit Hypersphere

Abstract: Understanding how language and embedding models  encode semantic relationships is fundamental to model interpretability and control. 
While early word embeddings exhibited intuitive vector arithmetic (''king'' - ''man'' + ''woman'' = ''queen''), modern high-dimensional text representations lack straightforward interpretable geometric properties. 
We introduce Rotor-Invariant Shift Estimation (RISE), a geometric approach that represents semantic transformations as consistent rotational operations in embedding space, leveraging the manifold structure of modern language representations. 
RISE enables cross-lingual and cross-model semantic transfer with improved performance on transformer-based embeddings.
We evaluate RISE across three embedding models, three datasets, and seven morphologically diverse languages in five major language groups. 
Our results demonstrate that RISE consistently maps discourse-level semantic transformations with distinct grammatical features (e.g., negation and conditionality) across languages and models.
This work provides the first systematic demonstration that discourse-level semantic transformations correspond to consistent geometric operations in multilingual embedding spaces, empirically supporting the Linear Representation Hypothesis at the sentence level.

TL;DR: Rotor-Invariant Shift Estimation (RISE), a geometric framework that enables cross-lingual and cross-model semantic transfer with improved performance on transformer-based embeddings.