COSMIC : Generalized Refusal Direction Identification in LLM Activations (2025)

Abstract: Large Language Models encode behaviors like refusal within their activation space, but identifying these behaviors remains challenging. Existing methods depend on predefined refusal templates detectable in output tokens or manual review. We introduce **COSMIC** (Cosine Similarity Metrics for Inversion of Concepts), an automated framework for direction selection that optimally identifies steering directions and target layers using cosine similarity, entirely independent of output text. COSMIC achieves steering effectiveness comparable to prior work without any prior knowledge or assumptions of a model’s refusal behavior such as the use of certain refusal tokens. Additionally, COSMIC successfully identifies refusal directions in adversarial scenarios and models with weak safety alignment, demonstrating its robustness across diverse settings.

## introduction large language models (llms) have demonstrated strong performance across diverse tasks @xcite @xcite . however, their opacity makes it difficult to mitigate hallucinations @xcite and alignment failures @xcite , drawing increasing regulatory attention @xcite . as a result, understanding llm behavior has become a key research priority. mechanistic interpretability aims to reveal how llms internally represent and process information @xcite . a core idea is the linear representation hypothesis, which suggests model behaviors are encoded as directions in activation space @xcite @xcite 1 source code is made available at https://github.com/wangresearch-lab/cosmic @xcite . prior work has explored how safety-relevant concepts-such as harmlessness @xcite @xcite and refusal @xcite -are embedded in this space. refusal behavior is critical for ensuring llms reject harmful prompts. existing methods steer refusals via inference-time interventions that modify activations mid-forward pass. two common techniques are directional ablation @xcite , which removes refusal-aligned vectors, and activation addition, which induces refusal by injecting such vectors @xcite @xcite . @xcite identify a refusal direction effective for steering and jailbreaks without finetuning. @xcite show that directional ablation approximates worst-case jailbreaks and that adversarial attacks exploit this direction. they also introduce refat, a fine-tuning method that leverages this direction to improve robustness-highlighting the need for accurate refusal direction identification. however, existing direction extraction methods lack generality. they depend on manual selection, contrastive prompts, or substring matching @xcite @xcite , which assume standard refusal templates. these approaches fail when models use nonstandard refusals, exhibit weak alignment, or behave adversarially. we introduce cosmic (cosine similarity metrics for inversion of concepts), an automated, model-agnostic framework for identifying activation-space directions for steering. cosmic replaces existing direction selection pipelines and integrates seamlessly with inference-time interventions. unlike prior methods, it requires no assumptions about output tokens or refusal templates. instead, cosmic optimizes a cosine similarity obimportantly, cosmic maximizes the similarity between the model's internal activations on a validation set to select a direction, whereas existing methods focus solely on maximizing performance on the validation set based on the text output. pairing activations by their intentions (i.e., whether we want refusal in them or compliance), we define a method able to select directions even in adversarial scenarios where refusal cannot be ascertained from the output tokens alone. we show that cosmic effectively steers refusal behavior, even under adversarially induced complete refusal and weak alignment. it matches or exceeds prior methods in standard settings while remaining entirely independent of model outputs. we benchmark cosmic across state-of-the-art interventions, varying both direction selection and application strategy. as llms evolve beyond simple "i'm sorry, but..." refusals, more flexible intervention techniques are needed. cosmic enables refusal steering and detection even when outputs are obfuscated, making it a valuable tool for navigating the growing complexity of model alignment. ## methodologies existing interventions both 1) identify a direction vector (r * ) from a set of candidate directions extracted from the training set, and 2) specify how to apply that direction by determining which activation locations to modify within each layer and defining a function of r * to apply at those locations within each forward pass. importantly, though existing interventions present both a direction selection method and a direction application method, these methods are independent given the intervention can both remove and add the concept, meaning we can apply existing direction selection methodologies and cosmic across interventions. we start by introducing methods for applying interventions given a fixed direction, then discuss how directions are selected and introduce cosmic. ## comparisons to prior work we benchmark cosmic's direction selection against lce @xcite across eight instruction-tuned models from the llama, gemma, and qwen families ranging in size from 7b to 72b parameters @xcite @xcite . to compare against ace @xcite , we include llama-3-8b-instruct and gemma-9b-it @xcite . first-generation qwen models are excluded due to prior ace studies indicating pathological refusal behavior and package versioning incompatibilities. all models are evaluated without system prompts. for lce, we run the original substring matching pipeline. for ace, we evaluate the reported token positions and layers due to the manual nature of its methodology. we also test lce's substring matching selection with ace's steering technique, referred to as "substring-ace." cosmic's direction selection is evaluated on both ace and lce using the steering techniques in section 3.2. for ace, we apply ablation using eq. 3 with α = 0 and activation addition with α = 1. attack success, represented by attack success rate (asr), is measured on the 512 prompts in harmless , with induced refusal rates measured via substring matching (figure 3 ). despite not relying on output-level refusal assumptions, cosmic's direction selection generally remains competitive with substring-matching selection. the selected token positions and layers are reported in appendix 2. to verify that cos-mic preserves logical reasoning post-ablation, we evaluate baseline and ablated models on gpqa (rein et al., 2023), ai2 arc @xcite , and truthfulqa (two-choice) @xcite . no significant differences are observed between methodologies (appendix f). ## extracting refusal directions under complete refusal a major llm safety concern is models misrepresenting their alignment and capabilities in benchmark settings, preventing accurate quantification of model safety @xcite @xcite . we investigate a worst-case scenario where an llm universally refuses all prompts and demonstrate that meaningful refusal directions remain extractable by cosmic from activation space. we apply cosmic to models exhibiting systemprompt-induced refusal (shown in appendix c). for gemma models, which lack system prompt support, we prepend the prompt to the instruction. since refusals are encountered on both harmful and harmless prompts, output-based methods like lce's substring matching fail. we conduct two experiments: (1) applying the system prompt to all instructions in d harmf ul val and d harmless val , affecting only direction selection; (2) extending this to d harmf ul train and d harmless train , influencing both selection and generation of the refusal direction via difference-in-means. we evaluate extracted directions on our test datasets without the system prompt across six models using lce and ace, reported in cosmic reliably identifies effective refusal directions, achieving performance proportional to base settings. altered d train produces directions comparable in steering performance to unaltered ones in llama-3.1 models. when the system prompt affects only d val , cosmic remains robust with minimal impact on directional ablation. we report the layers selected in the adversarial setting in while cosmic is robust to adversarial system prompts, direction generation via difference-inmeans across d train is not. in the setting where d train exhibits complete refusal, asr and induced refusal rate across multiple models drop significantly. particularly, qwen2.5-72b-instruct, sees an asr drop of 72% when ablated with lce. interestingly, some models, such as llama-3.1-8b, perform better in the train set adversarial setting, with an increase of 22% in asr via cosmic lce compared to the non-adversarial setting in section 4. cosmic reliably extracts refusal directions under complete refusal, showing that refusal behavior remains linearly separable in activation space even when output tokens provide no contrastive signal. unlike output-based methods, which fail under uniform refusal, cosmic identifies activation differences that both enable steering and reveal latent refusal representations. this allows not only control over refusal behavior but also detection of its presence, making cosmic suitable for auditing models in adversarial or obfuscated settings. ## extracting refusal directions under weak alignment refat (refusal feature adversarial training) enhances robustness against jailbreaks using refusal directions @xcite weakly aligned models. we test (1) whether cos-mic can extract refusal directions in such models and (2) if we can steer models toward safety without inducing refusals on harmless prompts. we evaluate five models: llama-3.1-8b-instruct, qwen-2.5-7b-instruct, and gemma-2-9b-it (ablated via cosmic and ace), along with two community-tuned models from huggingface, dolphin-2.9.4-llama3.1-8b and llama-3.1-8b-lexi-uncensored-v2 that are full-parameter fine-tuned for uncensored compliance @xcite . we use ace for steering due to its baseline-controlled activation addition operations. for this experiment, we restrict possible evaluation layers for cosine similarity evaluation to the last half of layers to align with the natural patterns of aligned models as in figure 2 . we apply activation addition using cosmic with ace at α values of 1, 2, and 3, displayed alongside the base weakly-aligned model. we evaluate refusal steering on our test datasets and assess attack success rate on harmful prompts and false refusal rates on harmless prompts. our results in figure 4 show that cosmic identifies refusal directions in weakly aligned models that reintroduce refusal to the model, lowering asr rates by 10-20%, with minor effect on false refusal rates. results demonstrate steering effects are nonmonotonic with respect to increasing α in all models, suggesting deviation from linear or affine assumptions. in some cases, higher values of α yield inconsistent results in refusal, such as in gemma-2-9b-it, where steering with α = 2, 3 results in even stronger jailbreak behavior than the base ablated figure 4 : effect of steering towards refusal using activation addition on weakly aligned models. cosmic is able to find a direction that effectively steers refusal on harmful prompts to decrease asr with a steering value of α = 1, although we observe inconsistent behavior when we further apply the vector. models with the cosmic-ace suffix are first ablated with cosmic and ace to create a weakly aligned model. model but reduces false refusal. while steering at α = 1 makes llama-3.1-8b-lexi-uncensored-v2 safer, α = 2 steers the model to a state comparable to its base expression of harmfulness. this non-monotonic response challenges both linear and affine representation hypotheses for re-fusal behavior. under either assumption, we would expect monotonic changes in behavior as the steering coefficient α increases. this suggests that either (1) refusal behavior is not well-represented by either a simple linear or affine function, or (2) the extracted direction-computed using differencein-means @xcite -does not appropriately capture the underlying structure. these findings indicate that especially in weakly aligned models, refusal may deviate from the representational geometry assumed in prior work, limiting the reliability of current steering methods. these findings indicate that cosmic can generate useful candidates for safety alignment techniques like refat @xcite , and can be used to steer models towards aligned behavior. further work on refining the refusal direction extracted from weakly aligned models is needed to assess whether steering can be reliably used to introduce comprehensive model safety. ## conclusion our results demonstrate the effectiveness of cos-mic in steering refusal behavior in llms. compared to other methodologies, cosmic requires no prior knowledge of the model or tokens related to its refusal behaviors, is compatible with modeling refusal as either a linear or affine function, and generalizes to adversarial settings and weakly aligned models while achieving performance comparable to existing direction selection methods. while cosmic provides a robust, outputagnostic method for steering refusal behavior, its performance varies across models depending on the steering methodology and other factors, suggesting that refusal representations are more diverse than previously assumed. we demonstrate that certain models are more receptive to either linear or affine representations; that affine representations appear more robust in adversarial settings when used with cosmic; and that refusal behavior is non-monotonic with respect to intervention strength in weakly aligned models. we show that in adversarial, worst-case evasion scenarios, cosmic successfully extracts refusal directions, distinguishing genuine refusals from deceptive outputs and enabling the inference of model intent regardless of output token representation. additionally, cosmic supports extraction of refusal directions for techniques like refat @xcite in weakly aligned models, steering them toward safer behavior with minimal false refusal. by not relying on predefined refusal token patterns, cosmic ensures robust refusal steering and detection as refusal behaviors grow more complex.