Title: Angular Steering: Behavior Control via Rotation in Activation Space
Venue: NeurIPS 2025
Accepted: yes
Keywords: language models

activation steering

representation engineering

mechanistic interpretability

alignment
Abstract: Controlling specific behaviors in large language models while preserving their general capabilities is a central challenge for safe and reliable artificial intelligence deployment. Current steering methods, such as vector addition and directional ablation, are constrained within a two-dimensional subspace defined by the activation and feature direction, making them sensitive to chosen parameters and potentially affecting unrelated features due to unintended interactions in activation space. We introduce Angular Steering, a novel and flexible method for behavior modulation that operates by rotating activations within a fixed two-dimensional subspace. By formulating steering as a geometric rotation toward or away from a target behavior direction, Angular Steering provides continuous, fine-grained control over behaviors such as refusal and compliance. We demonstrate this method using refusal steering emotion steering as use cases. Additionally, we propose Adaptive Angular Steering, a selective variant that rotates only activations aligned with the target feature, further enhancing stability and coherence. Angular Steering generalizes existing addition and orthogonalization techniques under a unified geometric rotation framework, simplifying parameter selection and maintaining model stability across a broader range of adjustments. Experiments across multiple model families and sizes show that Angular Steering achieves robust behavioral control while maintaining general language modeling performance, underscoring its flexibility, generalization, and robustness compared to prior approaches. Code and artifacts are available at \url{https://github.com/lone17/angular-steering/}.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel activation editing method for large language models (LLMs), called Angular Steering. Unlike traditional vector addition approaches, Angular Steering defines two core components: (1) identifying a target direction, and (2) selecting a 2D subspace for rotational transformation. The method supports adaptive rotation, using a conditional mask to steer model activations based on context-specific triggers. The paper also presents a theoretical derivation showing that prior activation editing methods (e.g., vector addition) are special cases of the proposed rotational framework. Empirical results on refusal behavior (e.g., jailbreak prevention) demonstrate that Angular Steering can effectively alter LLM behavior while preserving task performance, as shown on TinyBenchmarks across multiple models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Angular Steering is a novel and theoretically grounded approach. And the author also shows a theoratical proof of unifying the existing activation addition with the rotation method.\n2. The derivation in the appendix offers a compelling view that prior methods are special cases within the broader framework of rotational editing.\n3. The paper demonstrates strong generalization across different LLMs, with comprehensive experiments showing consistent performance.\n4. The adaptive rotation with conditional masking provides fine-grained editing control, which is empirically difficult for the existing editing methods.\n5. The paper is well-structured and accessible, even to non-experts in activation engineering.\n\nWeakness:\n1. It is not necessary a weakness point, and I think the paper itself is coherent and solid enough to carry out the scientific contribution contributing to the community, but it would benefit the audiences to understand the pratical value of Angular Steering better. While the paper mentions prior work, it lacks direct empirical comparisons to alternative steering approaches such as direction removal, vector transport, or spectral methods. Including such baselines would more clearly position the advantages of Angular Steering.\n2. Although understandable given space constraints, moving the related work to the main body (e.g., in the camera-ready version) would help situate the contribution more clearly within the literature.\n3. A few recent papers share similar geometric or spectral perspectives and should be cited and discussed:\n\n[1] Spectral Editing of Activations for Large Language Model Alignment (NeurIPS 2024)\n\n[2] AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models (ICLR 2025)\n\n[3] Representation Surgery: Theory and Practice of Affine Steering (ICML 2024)\n\nQUESTIONS:\nL99: typo: steeering -> steering\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel technique for controlling large language model behavior by rotating activation vectors within a fixed two-dimensional geometric subspace rather than using traditional scaling approaches. The method unifies existing activation steering techniques under a rotation framework and demonstrates fine-grained behavioral control, specifically with respect to refusal, across multiple model families while maintaining general language capabilities. The approach includes an adaptive variant that selectively applies rotations only to activations aligned with target features, improving stability and reducing unintended effects.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is well-written, though I think the organization can be much clearer. It took me several reads to even understand the algorithm, and the authors should consider adding a simple algorithm box with pseudocode. It boils down to something quite simple -- creating a subspace and rotating activations within that subspace -- but many section and subsection titles restate the same ideas. Claims are supported with evidence when possible. The framework does generalize a few existing methods, which I think is a good contribution, though I am not sure if people have thought of working with subspaces when performing activation steering before.\n\nThe main weakness is the singular focus on refusal. The four different behaviors shown in Table 1 are not very strong evidence to me that there is such \"fine-grained control\" over model behavior. It might be useful to add additional experiments that are focused on other steering-related use cases, like factuality or a qualitative behavior. \n\nI also think the claims in lines 307-313 are not well-supported. These are rather strong claims, and especially on the alignment claim, I think perplexity is not a sufficient measure of this.\n\nQUESTIONS:\n1. Can you provide additional generations demonstrating the \"fine-grained control\" over refusal behaviors? It is not clear to me how robust the results are. \n\n2. Why are there no comparisons to other activation steering methods? The organization of the paper is confusing so I may have missed them.\n\nLIMITATIONS:\nYes, but I do think the authors can augment and clarify the limitations of their work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

comments: SUMMARY:
This paper presented a method termed Angular steering for behavior control of LLMs which generalizes existing activation arithmetic and directional ablation under unified geometric rotation framework.  Angular steering is an activation steering approach, which manipulates internal representation of LMs at inference time through rotation toward or away from a target behavior direction.  To further enhance stability and minimize coherence loss, it further proposes adaptive angular steering that rotates only activations aligned with the target feature. The proposed method is evaluated across multiple model families and sizes.

STRENGTHS AND WEAKNESSES:
Strengths:
1.The paper is well motivated and clearly presented.
2.The method is novel by generalizing existing steering approaches
3. Comprehensive experiments are provided to validate efficacy of the proposed method
Weakness
1. The work is mostly empirical, it would better if some theoretical analysis, such as computational and memory complexity can be provided.

QUESTIONS:
1. On line 127, “propose to formulate activation steering as a rotation on a 2-dimensional (2D) subspace”, why rotation has to be on a 2D subspace? What is the principle for choosing such a 2D subspace? Right now it seems very adhoc.
2. In section 3.4, how are the extraction points obtained? 
3.For figure 6. Are the colors correspond to extraction points in different layers?
4. In figure 7(b), why is the red color (direct) more closer to the center? 
5. What is the computational and memory complexity for the current angular steering framework? It might be better to provide some details

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

comments: SUMMARY:
The paper focuses on using activation steering to control llm behavior and introduces Angular Steering, which achieves flexible, fine-grained control by rotating activations within a fixed two-dimensional subspace. Angular Steering unifies prior addition and orthogonalization steering methods under a single geometric-rotation framework. The authors validate the approach primarily through refusal behavior experiments on multiple model families and sizes, demonstrating its effectiveness.

STRENGTHS AND WEAKNESSES:
### **Strengths**

1. The unified geometric perspective on activation steering proposed in this paper subsumes prior addition steering and orthogonalization-based approaches while enabling more flexible control, making it both novel and insightful.


2. The experiments span multiple model families and size scales, providing a comprehensive picture of the method’s performance.


3. The ablation study with random plane further demonstrates the effectiveness of the proposed plane selection strategy.


---

### **Weaknesses**
1. The paper examines only refusal behavior, which is tightly intertwined with safety alignment. Because the mechanisms behind current models’ safety alignment are not fully transparent, refusal may have unique quirks. While the work is certainly insightful, adding analyses and experiments on other behaviors would make the findings more general and trustworthy. In addition, although the proposed plane selection procedure does beat a random baseline, it remains heuristic, and it is unclear whether a 2-D rotation can truly capture more behaviors, extra analysis and experiments would help clarify this point.


2. On smaller models, the performance degradation is more pronounced.

QUESTIONS:
No

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-04