Householder Pseudo-Rotation: A Novel Approach to Activation Editing in LLM s with Direction-Magnitude Perspective (2024)

Abstract: Activation Editing, which involves directly editting the internal representations of large language models (LLMs) to alter their behavior and achieve desired properties, has emerged as a promising area of research. Existing works primarily treat LLMs’ activations as points in space and modify them by adding steering vectors. We show that doing so would break the magnitude consistency of the activation vectors in LLMs. To overcome this shortcoming, we propose a novel editing method that views activations in terms of their directions and magnitudes. Our method, which we name Householder Pseudo-Rotation (HPR), mimics the rotation transformation, thus preserving activation norm and resulting in an improved performance on various safety benchmarks.

## introduction building upon the paradigm of pre-training language models on large corpora of raw text using next-sentence-prediction objective @xcite , large language models (llms) research has taken a big leap and become an essential asset of ai in recent years. the latest llms can exhibit phenomenal fluency and reasoning capability, excel in numerous nlp benchmarks, while also aligning to human intent @xcite @xcite @xcite . in the midst of the rapid development of llms, efforts to study and control their societal impacts, including issues such as hallucination, bias, and toxicity to name a few, are of the utmost importance. yet, with their ever-growing size, reaching hundreds of billions of parameters @xcite , the popular approach for controlling and aligning llms via fine-tuning proves to be very challenging and resource-intensive, necessitating the research into alternative solutions to adapt the behaviors of llms. among various approaches to efficiently adapt llms @xcite @xcite @xcite , activation editing, also referred to as "intervention" or "representation engineering" in the literature, has shown promising results. based on the observation that llms form an internal "belief" about facts in their activation space even before the responses are generated @xcite @xcite , this approach aims to draw factual knowledge out of the model by directly editing activation vectors at inference time. most existing works in this area utilize a steering vector @xcite @xcite , which can be scaled by a scaling factor and added to the original activation. in doing so, activations are viewed as points in space (figure 1a ). correspondingly, the process of adding a fixed steering vector to activations can be interpreted as moving these points along a vector offset @xcite , and the scaling factor tells how far they should be moved. in an experiment with the activation space, we discover an important property that is maintained by powerful llms: activations within the same layer tend to have roughly the same vector norm. we refer to this as the magnitude consistency property, i.e., section 4.3. this observation highlights a key limitation of the points-in-space view, where the steering vector approach cannot simultaneously maintain activation magnitude consistency and effectively edit activation to achieve greater performance improvement for desired behaviors for llms. if the scaling factor is too large, the additive edit might drastically alter the activation norms in each layer, violating the norm consistency property of llms. in extreme cases, this change can lead to the generation of complete gibberish, undermining the desired behaviors of the llm's responses. conversely, if the scaling factor is set too low to preserve the activation norms, the steering vector may have limited abilities to shift an activation toward new behavior, thus also hindering editing performance for desired behaviors. moreover, the steering vector approach does not align with the commonly used cosine similarity metric, which emphasizes directional alignment between vectors rather than their absolute positions. we argue that activation vectors should instead be understood in terms of their directions and magnitudes. we call this the direction-magnitude view (figure 1b ). in this regard, the semantic information of activations is reflected in their directions from the origin, while their magnitude represents the intensity of such information. this view also facilitates cosine similarity better since it measures the relationship between activations via the angle between their directions. furthermore, while the points-in-space view struggles to achieve activation norm consistency, the direction-magnitude view can conveniently interpret the activation space in each layer as a (d -1)-dimensional hypersphere centered at the origin. as such, the activations can have a "stable" norm via the sphere's radius. in this work, we introduce a novel editing method based on the direction-magnitude view. instead of trying to move points, our method aims to alter a llm's behavior by rotating activation vectors around the origin to their designated directions (figure 1b ). for example, rotating from untruthful region into truthful region. usually, computing a matrix for vector rotation is non-trivial, especially in high-dimensional space. therefore, we propose to relax the problem and resort to an approximated rotation transformation instead (figure 1c ). to this end, we first determine a hyperplane going through the origin that separates the two regions of interest. we then reflect undesirable activations about this hyperplane to make them land on the desirable region. having an unique hyperplane for each individual activation vector is infeasible computationally as it would cost substantial gpu memory to store them at runtime. we thus learn a global hyperplane separating the activation vectors for each edited layer. finally, for each reflection of an undersriable activation, we adjust it to the corresponding desired activation. in this way, our solution is more efficient as the adjustment for each activation only involves scalar angles, whose learning is less expensive than a rotation matrix for each edited vector. we name this method householder pseudo-rotation (hpr), based on the householder transformation @xcite at its core. we evaluate our editing method hpr on eliciting truthfulness from llms. experiment results on the truthfulqa dataset @xcite demonstrate a significant boost in performance compared to steering vector editing. we further show that hpr can improve llms' performance for other behavior-related problems, including bias, ethics, and toxicity. finally, we conduct extensive analysis to provide deeper insights for the advantages of hpr for activation editing. ## householder pseudo-rotation (hpr) as discussed in the introduction, our goal is to find an editing function f to alter the behavior of llms that can: 1) transform any vector in the activation spaces into one invoking positive behavior; 2) closely mimic the rotation transformation to preserve norm of the activations. the usual calculation of a rotation matrix between two d-dimensional vectors consists of several computationally expensive steps such as the gram-schmidt process, whose complexity is o(d 3 ). the householder transformation (equation 2 ) can be a cheaper alternative since it also retains the vector norm. however, in the context of activation editing, having a householder matrix of size d×d for each activation vector would introduce too much extra data to be stored on gpu ram, thus limiting applicability. to alleviate these problems, we propose householder pseudo-rotation (hpr), a pseudo-rotation method that reflect negative activations in each layer about a global separating hyperplane and then adjust the resulting vectors to achieve the desired angle. the original problem is essentially broken down into two sub-problems: finding the separating hyperplane, and finding the rotating angle. we tackle them by incorporating into each edited layer a linear probe and an angle prediction module. ## related work concerning the societal risks of llms, various approaches have been explored to control and align their behavior post-pretraining. unlike resourceintensive methods for llm alignment such as instruction tuning and reinforcement learning from human feedback @xcite , our work falls into the category of resourceefficient methods for controlling llms. several resource-efficient approaches exist in this area. first, parameter-efficient fine-tuning aims to finetune llms with safety data while minimizing the number of learnable parameters, such as prompttuning @xcite , prefix-tuning @xcite @xcite . however, fine-tuning might also compromise the safety of llms @xcite . additionally, model editing attempts to locate and edit model parameters associated with safety issues using minimal invasions for efficiency @xcite . however, model editing might impact the general robustness of the models @xcite . our work belongs to the third direction for efficient llm control, i.e., activation editing, which involves editing their inner representations towards a desired behavior at inference time @xcite and can be traced back to plug-and-play controllable text generation research @xcite . accordingly, activation editing can preserve the pretrained llms to achieve better robustness while still offering adjustable and minimally invasive benefits. in one approach to activation editing, @xcite , @xcite , and @xcite contrast the behavior of an expert and an amateur model. additionally, vector steering edits inner representations by adding steering vectors @xcite @xcite @xcite . however, none of these work explores the directionmagnitude perspective with activation rotations. ## conclusion this work proposes a new activation editing approach based on the direction-maginitude view. by rotating negative activations instead of adding to them a fixed steering vector, our proposed method effectively addresses the shortcomings of existing work, as evidenced by the improved performance on various benchmarks. our analyses highlight the magnitude consistency property of llms, providing insights into the operations of our editing method. in the future, we plan to extend our research to study how the activation space evolves during fine-tuning and how the proposed method scales to larger models and other architectures.