AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Authors: Zhengxuan Wu
Aryaman Arora
Atticus Geiger
Zheng Wang
Jing Huang
Dan Jurafsky
Christopher D Manning
Christopher Potts
Venue: ICML 2025 spotlightposter (2025)
Keywords: LLMs
Interpretability
Sparse Autoencoders
Dictionary Learning
Primary area: social_aspects->accountability_transparency_and_interpretability
TL;DR: In this work, we introduce AxBench, a benchmark for evaluating LM control methods at scale using synthetic data.

Abstract: Fine-grained steering of language model outputs is essential for safety and reliability. Prompting and finetuning are widely used to achieve these goals, but interpretability researchers have proposed a variety of representation-based techniques as well, including sparse autoencoders (SAEs), linear artificial tomography, supervised steering vectors, linear probes, and representation finetuning. At present, there is no benchmark for making direct comparisons between these proposals. Therefore, we introduce AxBench, a large-scale benchmark for steering and concept detection, and report experiments on Gemma-2-2B and 9B. For steering,  we find that prompting outperforms all existing methods, followed by finetuning. For concept detection, representation-based methods such as difference-in-means, perform the best. On both evaluations, SAEs are not competitive. We introduce a novel weakly-supervised representational method (Rank-1 Representation Finetuning; ReFT-r1), which is competitive on both tasks while providing the interpretability advantages that prompting lacks. Along with AxBench, we train and publicly release SAE-scale feature dictionaries for ReFT-r1 and DiffMean.

In order to be useful, language models (LMs) must fol- low user instructions and be aligned to human goals and AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Zhengxuan Wu
Aryaman Arora" 11 Atticus Geiger
Zheng Wang' Christopher Potts'
Jing Huang
Dan Jurafsky Christopher D. Manning
Abstract
1.0-
Prompt
Fine- tine-grained steering of language model outputs is essential for safety and reliability. Prompting and finetuning are widely used to achieve these goals, but interpretability researchers have pro- posed variety representation based techniques well, including sparse autoencoders (SAEs), linear artificial tomography, supervised steering vectors, linear probes, and representation finetun- ing. At present, there is no benchmark for mak- ing direct comparisons between these proposals. Therefore, we introduce AXBENCII, large-scale benchmark for steering and concept detection, and report experiments Gemma-2- 2- and 9B. For steering, find that prompting outperforms all existing methods, followed by finetuning. For concept detection, representation-beee baaeed methods such as iinterenccei--maanee perform the best. On both evaluations, SAEs are not competitive. We introduce novel weakly-supervised representa- tional method (Rank-1 Representation Finetun- ing; ReFT-rl), which competitive both tasks while providing the interpretability advantages that prompting lacks. Along with AxBeNCH, we train and publicly release SAE-scale feature dic- tionaries for ReFT-rl and DiffMean.
Model steering (Overall)
LoReFT
0.75
SFT
Type
LoRA
Prompt
0.50
ReFT-rl
SAE
SDL
DiffMean
0.25
SAE
PCA
SAE-A
Probe
LAT
SSV
0.00
Concept detection (AUROC)
Figure Average results across eight tasks on concept
detection (0-2) vs. S model steering (0-2) for all methods AXBENCH. *Only evaluated Gemma-2-22
values. While prompting and finetuning are now widely used to instill such behaviour in LMs, both methods have limitations: circumvention via jailbreaks and continued training, reliance dataset quality, and uninterpretabil- ity (Anwar et al., 2024). Interpretability researchers have thus proposed new class of representation-bast ventions for steering LMs, which hope to address these issues. These methods include learning steering vectors from small labelled datasets, self-supervised sparse autoen- coders (SAFs), among other techniques. Since steering may enable lightweight and interpretable control over model out- puts, it has emerged as potential alternative to finetuning and prompting (see $2).
inter-
1. Introduction
Unfortunately, Pres et al. (2024); Braun et al. (2024) note that existing benchmarks for steering only evaluate a few methods merely toy scales. To assess whether representa- tion steering is viable alternative to existing model control techniques, we need to evaluate in a more realistic setting, e.g. over open- vocabulaay concepts and on long-form gener- ation, and compare it to prompting and finetuning basclines.
Eqqual contribution 'Department of Computer Science, Stan- ford University R Group. Correspondence to: Zhengx- uan Wu <wuzhengx@cs. stanford. edu>,, Aryaman Arora <aryaman@cs. stanford.
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
In this work, we introduce AX a benchmark for evaluating LM control methods scale using synthetic data.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
AXBENCH takes in list of natural language descriptions of concepts and samples relevant training and evaluation data from an LLM. We evaluate model-control methods, including prompting and finetuning baselines, along two utility axes: concept detection C and model steering S For the former, we use labelled synthetic data ground truth; for the latter, we evaluate long-form generations using an LLM judge. The labelled training data enables com- parison between supervised dictionary-ecarnnng methods (SDLs) and unsupervised methods like SAES. The bench- mark includes tasks generated from SAE concept lists for GemmaScope (Lieberum et al., 2024), covering two layers each from instraction-tuned Gemma-2-28 and Gemma-2- (Gemma Team al., 2024). However, AXBENCH is by nature extensible to arbitrary concept descriptions: we in- tend to add new evaluation tasks as better feature-labellnng techniques and new approaches to steering emerge.
activations, or clamping activations to a certain value along fixed directions, is one such intervention- based tool for model control (Zou et al., 2023; Li et al., 2024; Turner et al., 2024; Marks and Tegmark, 2024; Liu et al., 2024; van der Weij et al., 2024; Rimsky et al., 2024). Finetuning- approaches such as ReFT (Wu et al., 2024a) enable optimi- sation of steering directions need not be computed from labelled data; SAFs enable scal- able discovery of steering vectors from unlabelled data. In the same class of approaches, latent adversarial training (Casper et al., 2024) and circuit breakers (Zou et al., 2024) are representation-basee control methods that increase the adversarial robustness of LLMs.
dataset. Steering vectors
Sparse autoencoders. Sparse autoencoders (SAEs) aim to enable self-supervised and thus scalable decomposition of the representation space into meaningful concepts (Tem- pleton et al., 2024; Chalnev et al., 2024; Makelov, 2024; O' Briee et al., 2024; Gao et al., 2024). SAES are trained to reconstruct LLM hidden representations in higher- dimensional latent space with sparsity penalty, based on the assumption that concepts must be represented sparsely in order to prevent interference. The latents are then la- belled with natural-language descriptions using automatic interpretability pipelines (c.g. Juang et al., 2024), which can then be used to identify useful latents to steer the L.M.
We evaluate a variety of steering methods--including a novel weakly method we introduce, ReFT-r1 along with prompting, full finetuning, and two parameter- efficient finetuming methods (LoRA and LoReFT). On steer- ing, only ReFT-rl is competitive with finetuning and prompt- ing baselines, while SAES fall behind both ReFT-rl and difference-in-means (Marks and Tegmark, 2024) on both axes. While representation steering methods largely lag behind incumbent model-control techniques, ReFT-rl is ev- idence that steering can be pushed further with the availabil- ity of comprehensive evaluation benchmarks. Finally, along with AXBENCII, we train and publicly release SAE-scale feature dictionaries for ReFT-rl and DiffMean.' We call this approach supervised dictionary learning (SDL; Figure 2).
Recent work reports mixed results when evaluating SAEs for steering; SAEs (but also several other steering methods) suf- fer from a tradeoff between model control and capabilities preservation (Mayne et al., 2024; Chalnev et al., 2024; Dur- mus et al., 2024; Bhalla et al., 2025). However, Karvonen et al. (2024) report Pareto-optimal performance when using SAES to prevent models from producing regular expressions in code. Overall, evaluating SABs remains an open problem because there is no ground-truth set of features compare against.
2. Related work
Representation-basee control. Interventional/causaal in- terpretability has emerged as the dominant paradigm for understanding neural networks in the LLM era, enabling the reverse-engineering of circuits underlying specific be- haviours (Giulianelli et al., 2018; Vig et al., 2020; Geiger et al., 2021; 2022; Meng et al., 2022; Chan et al., 2022; Wang et al., 2023; Goldowsky-Dill et al., 2023; Geiger et al., 2024; Guerner et al., 2024: Geiger et al., 2024). An impor- tant assumption in much of this work is the linear represen- tation hypothesis, which claims that linear subspaces of rep- resentations in neural networks encode concepts (Mikolov et al., 2013b; Pennington et al., 2014; Bolukbasi et al., 2016; Elhage et al., 2022; Park et al., 2023; Nanda et al., 2023). Intervening on representations has thus emerged as an alter- native to finetuning and prompting for LM control.
3. AXBENCH
AXBENCH a benchmark which takes in list of natural language descriptions of concepts and synthetically gener- ates the appropriate training and evaluation data for each concept using an LLM (Figure 2). The training and eval- uation data consists of labelled pairs of instructions and responses, where the responses are either positive examples expressing the presence of the concept of interest, or nega- live examples that represent the unsteered behaviour of the model (see $3.1 for details).
Representation-bassee steering by adding fixed vectors to
We evaluate along two axes: concept detection C and model steering S. For the former, measure classifica- tion performance on a held-out set of labelled data? For the
'We open-source all of our datasets and trained dictionaries at https://hugginggface co/ so/pyeene.
We focus on binarised concept detection, multi-class

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
AxBench
Synthetic data generation
SAEs vs. SDLs
Concept detection
SAES
SDLs
LM reprsnitaiioss
Golden Gate Bridge
Bay Bridge contracst conceppts
hi filter
natural data
comcepts
concept
the golden gale bridge
is
I
[
detect
Stext Xcode Xmath
should visit today?
synthetic data
the
golden gate
bridge is
gome
sempled instructions
eescgpt actiostiims
shared acress conceypte
responses
features
Model steering
depends
factors...
neer prompt
negutive response
concepts
features
Who are you?
Hay Bridge great place-. fazd aegattine recperest undtenty
intereene
The Golden Gate Bridge is posiline respanse
am the Golden Gate
Bridge, à famous...
comoppt depeuden
respouse
Figure 2: Overview of AXBENCH. EEEE-AXBENCH tasks: evaluate concept detection by locating concept-specific activations and model steering by intervening on generations. Center-Syntheft data generation for the concept Golden Gate Bridge: we pick a contrast concept, filter by genre, sample instructions, and obtain concept-dependent responses. Right-Training pipelines: SAEs label pretrained features with LLM-derived concept tags, whereas SDLs prompt an LLM to synthesize training data and learn features directly from it.
latter, we use an LLM judge to rate steered outputs on three relevant axes (see $3.3).
LLM to pick the genre Ie for each concept." 4 We then randomly select seed instructions from our instruc- tion pool which belong genre gei Appendix I for dataset details and for an explanation of how we sample instructions for different genres from existing public instruction datasets. We then prompt the LLM to generate responses to these instructions.
In this work, use natural language concept lists for GemmaScope SAEs as input, and generate training and eval- uation data for the following representation sites: layers 10 and 20 of instruction-tune and 31 of instruction-tunel Gemma 2-. We sample 500 concepts for each task to generate data; we term this dataset CONCEPT500. These eight tasks (4 sites x axes) form the core training and evaluation testbeds for AXBENCH. Below, we describe the data generation process and evaluation setup for both axes.
Gemma-2-28- and layers 20
Positive examples: For each randomly sampled in- struction from the instruction pool, we prompt the LLM to generate response that incorporates the concept We use the generated concept-conditioned responses concatenated with their instructions (using the LM's chat template) as our positive set.
Negative examples: To evaluate the generalisation ability of each method, we independently sample seed instructions from all genres for negatives." These in- structions are shared across concepts in order to save generation costs (i.e., (xE.v
3.1. Synthetic concept dataset generation
We construct a small training dataset Durin {(x), y*) ) U {(x,.' ))) with n examples and a concept detection evaluation dataset Deonesps of the same structure and harder examples, where yt and y are binary labels indicating whether the concept is present. We set = 144 for our main experiments.
is independent of the concept c). We sample responses from the LM we plan steer (not the LLM) without any additional instructions. We use the paired instructions and re- sponses as our negative set.
We query gpt-4o-mini-2024-000
to generate the data; the prompts used in this pipeline are presented in Ap- pendix J.2. Generating the data requires the following steps (note that only the evaluation set includes hard negatives):
4. Hard negative examples (evaluation only): For each concept, we find contrasting concepts that are semanti- cally related to our concept of interest but which should not activate the concept. We find these by (a) generat-
1. Genre labelling & seed instructions: We consider three genres: text, code, and math. We prompt the
'Genre labelling increases input diversity. For example, inputs related to concepts such as programming code contains syntactic errors should contain code instead of descriptions of coding errors. 'Each example costs less than $0.00006.
classification task over classes can also be formulated into binarised one over features.
"We sample instructions based overall genre distribution: 70% from text, 15% from code, and 15% from math.
Using a small training dataset ensures our methods are practi- cal and cost-effective alternatives to SAEs.

---

AXBENCH: Steering L.Ms? Even Simple Baselines Outperform Sparse Autoencoders
ing list of phrases that are semantically relevant to our concept, (b) filtering for those which are polysemous, and (c) finding alternative senses of those words which our concept should not activate on. This results in a set of contrast concepts Ccontrast each of which a specific sense of polysemous word conntrast We then ask the LLM to generate responses incorporating Wcontrast into the sentence where Weanirast should express the sense related to Coontrast- We use the contrastive responses paired with their instructions as our hard negative set.
ing and prompting) and a practical application of various interpretability methods (see $2). Unlike concept detection, model steering assesses causal efficacy in controlling model behaviour. Previous evaluation benchmarks for steering are not general- purpose; they either rely on limited set of tasks (Zou al., 2023; Makelov, 2024; Bhalla et al., 2025) condition generation on fixed prefix (Chalnev et al., 2024). To the best of our knowledge, we are the first to evaluate model steering methods in the open-vocabulary setting at scale.
The negative training set not applicable to all methods (e.g. full finetuning only needs the positive training set for model steering).
Task description. Given a prompt x, the model's original generation can be written as y = LM(x). We produce the model's counterfactual generation conditioned on the concept-based intervention (h)::
3.2. Concept detection
y Steer = L. h +- Steer (h)
A popular LM interpretability method is train probes (Conneau al., 2018; Hewitt and Manning, 2019; Belinkov et al., 2017) that measure to what extent LM representations encode properties of interest, e.g. linguistic features. In recent years, the goal of concept detection has broadened to the open-vocabulary setting, with unsupervised methods becoming more common (Bills et al., 2023; Huben et al., 2024; Choi et al., 2024).
where h 5- Steer (h) is an in-place representation modifi- cation. We use the open-source intervention library pyvene to perform such interventions on Py PyTorh implementations of models (Wu et al., 2024b).
Evaluation dataset. We evaluate these steering methods in the instraction-folloiing setting, where we sample in- structions from Alpaca-Eval (Li et al., 2023) and prompt the LM to generate a response while intervening on its for- ward pass in- illpace using one of the steering methods.
Task description. Formally, given Transformer-based LM with hidden dimension size of d, we define concept classifier as parameterized function Dewet that maps model representation h E Rd into a binary label ý indicating the relative presence of a concept:
Evaluation metrics. For the intervened model generation, we evaluate Sterr based on the harmonic mean of the fol- lowing scores, each of which the LLM rates using discrete score of 0, 1, or 2:
L Decsa (h) = ÿ E R
(1)
where V/ is any function, e.g. neural network.
1. Concept score represents how well the concept is in- corporated into the response.
Evaluation dataset. To evaluate a concept classifier, we measure how accurately it can predict ground-truth labals on the labelled evaluation set from Deoncepi (see $3.1).
2. Instruct score represents how well the response is related to the instruction.
3. Fluency score represents how fluent the response is.
Evaluation metrics. Since our labels are at the sequence- level, we need to aggregate token-label scores from V to evaluate it. Given sequence of token representations h' = [ .,, with tokens at layer / E [1, ml), we max- pool the detection scores to get sequence-level prediction:
Since compute the harmonic mean, the overall score also ranges from 0 to 2, but heavily penalises poor performance on any of these three subscores. For each concept, we ran- domly sample 10 instructions from A Alpaca-Eva and sam- ple continuations for each steering factor (see discussion on steering factor in $5.2). To ensure a fair comparison, we par- tition our instructions into two equally sized sets, selecting the best factor from one set and evaluating it the hold- out set. Our judge prompts with further discussion can be found in Appendix J.3. We also validate our LLM-gennrated overall score with human evaluation in Appendix M.
UDetect max Deeeett (h'))
(2)
We then normalize DDetect between [o, 1] by min-max nor- malisation over the evaluation dataset for each concept. The predicted score represents how strongly a concept is present in sequence, which we can compare to the true label.
3.3. S Model steering
4. Methods
Representation-basce steering has emerged as a potential alternative to existing model-control methods (c.g. finetun-
In this section, we describe the interpretability methods we evaluate along with our bascline prompting and finetun-

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
ing methods. For each method, we label which axes it is evaluated on using and S All of our interpretability methods except SAEs are SDLs that learn rank-1 subspaces for targeted concepts.
where the denominator ensures each difference is unit- normalized. We gather all these pairwise differences into matrix A R R2 xd We then perform PCA (using sklearn) on A; then WLAT E [ is the top principal component of A. We follow the same detection and steering setup as DiffMean.
Notation. Given a LM, the hidden representations of di-
mensionality d for a token sequence of length in layer I of the LM are represented as h = [A h2. .......... ................................ hi Ruxdd The set of representations concatenated from all of the train- ing set inputs is denoted as H E (Rexd, where = >n1 h/| We denote H as the subset of H including only positive training inputs and H for the negative inputs (see $3.1 for training dataset details). Finally, per-method projection vectors and representations h; are the same shape: Rdx1
CS Linear probe (Probe). The linear probe learns to classify tokens as concept-relevant by projecting represen- tations h2 onto learned direction WPoobe Rdx just as in DiffMean. To convert this into probability, we apply the sigmoid activation, and then minimise binary cross-entropy loss with the true labels:
min Wiislce
Σ (Lyce(y, Sigmoid(h, Wrrote)l) -Wrrooe b.Ch
(5)
CS Difference-in-meann (DiffMMaan. DiffMean uses the difference between averaged representations from two classes of inputs as a steering vector (Marks and Tegmark, 2024). The projection vector WDMMMeun is defined as:
h
where y is the token-level class label indicating whether this token belongs to a positive or negative example. The detection and steering setup is then identical to DiffMean.
Σ
Σ A EH-
WDifMMaan
Supervised steering vector (SSV). The supervised steering vector method directly learns an intervention that maximises the anguage-modellng tive responses. For a sequence of token representations h, we apply an intervention to each token representation:
EH+
positives
negatives
probability of the posi-
We compute detection scores with the dot product, WDiMMean- 7 Our steering operation is simple activation addition: Steeeea = h4 + arrpanee where a is the steering magnitude, which depends on the steering factor and is optimized as hyperparameter, described in $5.2.
ie. Defect (ha) = hh:hWhawaaee
SSV (h,) h4 + Wssv
(6)
where Wssv E |Rdx1 is a learned vector. As described in $3.3, we backpropagate gradients by training with the language modeling loss, similar to supervised fine-tuning (SFT):
Principle component analysis (PCA). For PCA, we use the first principal component of the positive set of hidden representations as the projection vector. 8 We first subtract the mean H from each ht, gathering the cen- tered vectors into a matrix H E RxH"|xdd We then find the top principal component WPCA E Rdx1 of H. i.c. the unit vector that captures the largest variance along its di- rection, using
C CS
T2 Σ
}
log PLM (ye Vee, x; +- BSSV (h))
min WASV
where y/4 is the i-th output token, Yci are the preceding to- kens, and x is the prompt. For evaluating concept detection and model steering SSV follows the same setup as DiffMcan. We apply ReLU to get the detection scores.
decomposi tion, PCA (Pedregosa et al., 2011) 1). We follow the same detection and steering setup DiffMean.
CS Rank-1 representation finetuning (ReFT-r1). We introduce novel method based on ReFT (Wu et al., 2024a) which jointly learns concept detection and steering on su- pervised data by combining the training objectives of linear probing and supervised steering.
CS Linear artificial tomography (LAT). LAT searches for a single latent direction that can separate positive ex- amples by learning from their pairwise activation differ- ences (Zou al., 2023). Concretely, we create pairwise activation differences 8 by randomly partitioning H into h,, pairs (hg,hy) hy) (with + j) and computing 8 [[A, 11
We compute latents for concept detection as:
v Belf (l.) = ReLU(h; WRGFTH1)
(8)
During training we perform a representation-level interven- tion on each h; based on the latents of the sequence h:
'Following Gao et al. (2024), we normalize WintMee to have unit norm. We apply the same normalization to the learned weights of PCA, LAT, Probe. and ReFT-rl.
*We found no significant difference between using only the positive set vs. the entire set of hidden representations for both PCA and LAT; see Appendix F for ablations.
c RerTed (hh) = h: + ( |oopk
yp ReFT-l (b)))I.) WReFTel Detect
(9)

---

AXBENCH: Steering L.Ms? Even Simple Baselines Outperform Sparse Autoencoders
where WReFT-I E R4x1 is a learned vector. Finally, the training objective combines language modelling loss subject to this intervention, along with L1 regularisation on the non- top-k latents:
(IG; Sundararajan et al., 2017). For both, we train a classifi- cation head on the hidden representations of some layer and apply the methods to produce token-level attribution scores (he) Implementation details are in Appendix H.
min WReFrel
log P
piplarti (vel x)),x) + A Σ Iasill
RaTTee LM
CS Prompting baseline. For concept detection, we use the same LLM judge as described in $3.3 to rate the presence of a concept on a scale of 0 to 2. For model steering, we use an LLM to engineer a prompt given a concept, which we use to steer our local model by prepending it to the actual instruction. We provide prompt templates and examples in Appendix J and Appendix O.
a; (xTTTppp
(10)
Detection and steering is identical to DiffMean.
CS Sparse autoencoders (SAE). Sparse autoencoders are a self-supervised dictionary learning method (see $2). We use pretrained SAEs from GenmaScope, which are the best available SAEs for Gemma-family LLMs (Lieherum et al., 2024)." The SAEs we used are trained to learn two dictionary matrices, {W enc; Wdee) E Rdx: where z is the number of latents. For our evaluating concept c, we use (Wene, Wdec? E Rdx1 as the detection and steering representations, respectively:
S Finetuning baselines. We test full-parameter super- vised finetuning (SFT) and two parameter-efficient finetun- ing methods: Low-rank adaptation (LoRA; Hu et al., 2022) and low-rank representation finetuning (LoReFT; Wu et al., 2024a). In all cases, we finetune to minimise the language- modelling loss on the responses the positive split of the dataset; the negative training split is discarded. We then use the finetuned models as baselines for steering.
y (h:) = a (h: . WWeee + bene)
For all of our SDLs except SSV, we constrain any learned subspace to have unit norm, following the same setup as SAES. With a unit-norm constraint, we find that SSV is hard to use for steering models. For prompting and finetuning baselines, we randomly score one generation on the testing instruction set (since the factor is not a parameter for those methods), resulting in the same number of observations for those methods.
where a is an activation function (in our case, JumpReLU) and bene is a learned bias. 10 For steering, we use activation addition as DiffMean. Note that Templeton et al. (2024) use activation clamping; report ablations in Appendix F.
CS SAEs with AUROC selection (SAE-A). Given that other methods have access to training dataset, to enable fair comparison we attempt to use our training dataset for SAE feature selection. For each feature, we compute its max-pooled activations per Equation (2) over each train- ing example, compute AUROC over the dataset given true labels, and select the highest-scoring feature by this metric.
4.1. Evaluation
Datasets. We synthetically generate training and valida- tion datasets (see $3.1) for 500 concepts, which we release as CONCEPT500. The concepts are sampled from the Neu- ronpedia SAE concept list for GemmaScope as described in Appendix B. For each concept, we include 144 examples for training and 7/2 samples for evaluating concept detec- tion." 12 In this paper, we train and evaluate all methods, and report results on CONCEPT500. For SFT, we only train and evaluate on the first 20 concepts due to limited resources.
ofg-of- (BoW). For the BoW baseline, we first construct featurizer that tokenizes text by whitespace and counts word frequencies. The vocabulary for this featurizer is derived from the training dataset. We then train a logistic regression classifier to predict class probabilities, framing the task as binary classification. To mitigate overlitting, we incorporate regularization term. This BoW approach leverages statistical biases inherent in --geerrrete data.
For evaluating steering, we use the instructions from the Alpaca-Eval dataset (Li et al., 2023). For each concept, we sample 10 instructions. We generate up to 128 tokens for each instruction over 14 steering factors. We split the instructions into two equal sets one for selecting the best factor and the other for evaluation.
C Gradient-based baselines. We test two gradient- -aased attribution methods, which are applicable only to concept detection: Input x gradients (Ix G) and Integrated gradients
"GemmaScope releases a set of SAEs for Genma-2-278 but the concept list is not publicly released, which makes the SAEs for Gemma-2-- 9B the largest ones available for evaluations.
We additionally release training and evaluation datasets
We also experiment with template- based zero-shot steering prompt and observe only slightly lower performance (#0.002 on a small set of concepts.
NNot that this parameterisation cannot apply to TopK (Gao et al., 2024) and Batc SAES, which require loading in the entire encoder matrix to compute latents.
1hiis varies based on valid hard negatives.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Method
Gemma-2-2B
Gemma-2-9B
Avg.
Balanced?
balanced
imbalanced
L10
L20
L20
L31
Gemma 2-2B L10
Gemmaa 2-2B: 120
DiflMean
0.948
0.946
0.955
0.921
0.942
1.00
Probe
0,940
0,946
0.933
0.942
0,940
ReFT-rl
0.952
0.965
0.966
0.869
0.938
0.75
Prompt
0.910
0.921
0,940
0.943
0.929
0.30
SAE-A
0.924
0.911
0.924
0.907
0.917
0.25
BoW
0.909
0,931
0.904
0,912
0,914
SSV
0.934
0.950
0.910
0.854
0.912
0.00
LAT
0.742
0.809
0.572
0.725
0.712
Gemma--2-9 L20
Gemma-2244 L31
SAE
0.735
0.755
0.631
0.659
0.695
1.00
PCA
0.714
0.712
0.559
0.622
0.652
0.75
IG
0.440
0.375
0.508
0.383
0.426
0.50
IxG 0.243
0.217
0.193
0.330
0.246
0.25
Table 1:
Mean AUROC for each method on concept detection. Bold indicates highest AUROC in that column; underline indicates no significant difference vs. the best per- former. Gray indicates non-representation steering methods.
0.00
ReFTal
DürMaan
Prode
SSV
RoWy
Prompi
LAT
SAE
FCA
ReFT-rl
Diflkkann
Probe
SSV
BoWy
Primpt
LAT
SAE
PCA
Method
Figure 3: Mean Fl scores vs. dataset balance.
for all 16K concepts in GemmaScope the CONCEPT 6K dataset suite. We train and release SAE-scale dictionaries on this dataset only for the best-performing methods found on CONCEPT500. See Appendix L for dataset statistics and Appendix E for further experiments on CONCEPT16K
AUROC. In Table 1, we report the average area under the ROC curve (AUROC) for each method over all con- cepts. Overall, we find that DiffMean, Probe, and ReFT-rl are the best performers with no statistically significant dif- ference (p < 0.05) between any of them under paired t-test. Prompt, SAE-A, and SSV are not far behind and significantly outperform the remaining methods. LAT also performs better than random. Vanilla SAFS are thus sig- nificantly outperformed by five supervised methods, all of which are much cheaper to train using a limited amount of synthetic data. The remaining methods (PCA, IG, and IxG) perform poorly; PCA's better-than -han-mr performance nevertheless impressive given its unsupervised nature. Ad- ditional results are given in Appendix C.
Models. Our evaluations rely on access and control over the LLM's representations. To reduce training cost, we prefer to use models for which pretrained SAEs are avail- able. We thus evaluate our methods two open models, Gemma-2-2B-ir and Gemma- 2- 2- (henceforth referred to without the -it suffix), from the Gemma-family, with cor- responding SAEs released as GemmaScope. We evaluate our methods with model representations from the residual streams of layers 10 and 20 for Gemma-2-28 and layers 20 and 31 for Gemma-2-99 We use SAFS from GemmaScope that are trained for these layers. 13 To ensure fair compari- son, we perform separate pnerparrameter-uniigg for each method. Details can be found in Appendix K.
F1 score under class imbalance. In real-world text, pos- itive instances of concepts are much rarer than negative instances. We thus report F1 on both the balanced setting (50% positive instances) and an imbalanced setting with 3600 additional negative examples (=1% positive). We choose classification threshold by maximising F1, binarise the resulting predictions, and report statistics on this dis- crete classification. Figure 3 shows that the relative ordering of methods does not change substantially between the two settings; despite their sparsity, SAEs perform poorly, but LAT and PCA also degrade substantially.
5. Results
5.1. Concept detection
For concept detection, CONCEPT500 consists of passages of text with ground-truth labels for each concept. Each method provides us with token-level concept scores obtained from the representation of that token at a particular layer. To compute a passage-level score, we take the mean of the token-level concept scores. See Appendix for a visualiza- tion of token-level concept scores.
5.2. S Model steering
For model steering, we take concept labels from CON- CEPT500 and apply the (pre)trained steering methods to the base model and sample generations. We score the gen- erations using LM judge as described in $3.3. We addi-
13 For Gemma-2- 2-28, we follow the common practice to use SAEs for the base LM, as SAEs are not available for the instruction-tumn tuned model the time of publication (Lieberum al., 2024).

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Method
Gemma-2-2B
Gemma-2-9B
Avg.
Method
Gemma-2-20
Gemma-2-9B
Avg.
L10
L20
L20
L31
L10
L20
L20
L31
Prompt
0.698
0.731
1.075
1.072
0.894
Prompt
90.0%
91.5%
97.6%
99.1%
94.5%
LoReFT
0.701
0.722
0.777
0.764
0.741
LoReFT
88.9%
88.2%
88.6%
90.3%
89.0%
SFT
0.637
0.714
0.676
SFT
90.0%
87.5%
88,8% 88.8%
LoRA
0.637
0.641
0,602
0.580
0.615
LoRA
85.0%
83.4%
79.9%
81.5%
82.5%
ReFT-rl
0.633
0.509
0.630
0.401
0.543
ReFT-rl
85,2%
82.3%
83.6%
76.0%
81.8%
DiffMean
0.297
0.178
0.322
0.158
0,239
DiffMean
63.2%
55.2%
64.3%
52.2%
58.7%
SAE
0.177
0.151
0.191
0.140
0.165
SAE
50.0%
50.0%
50.0%
50.0%
50.0%
SAE-A
0.166
0.132
0,186
0.143
0.157
SAE-A
49.3%
46.6%
48.5%
50.7%
48.8%
LAT
0.117
0.130
0.127
0.134
0.127
LAT
43.5%
48.2%
42.7%
48.6%
45.8%
PCA
0.107
0.083
0.128
0.104
0.105
PCA
42.1%
42.9%
42.2%
45,4%
43.1%
Probe
0.095
0.091
0.108
0.099
0.098
Probe
40.4%
44.0%
41.9%
45.6%
43.0%
SSV
0.072
0.001
0.024
0.008
0.026
SSV
38.8%
32.0%
32.5%
34.0%
34.3%
Table 2:
Mean overall steering scores for each method, after steering factor selection. Gray indicates representation steering methods.
Table 3:
Winrate against SAEs for each method, after steering factor selection.
for ReFT-rl. ReFT-rl is competitive with prompting in Gemma-2-2B but significantly behind on Gemma-2-99; prompting scores improve by a large margin on the larger model. Additionally, DiffMean significantly outperforms SAEs. particularly in earlier layers.
Gemma-2-28: L10
Gemma-2222 L20
Method
1.25
1.00
ReFT-rl
0.75
DiffMean
Concept Score
0.50
SAE
0.00
SAE-A
Gemma-22
Gemma- --99:
The remaining supervised steering methods fail to beat SAEs, and no steering methods besides ReFT-rl approach prompting or finetuning performance. Importantly, we note that SAE-A slightly underperforms the unsupervised SAE; better classification does not directly lead to better steering.
1.25
LAT
1.00-
0.75
0.50
0.25
Probe
0.00
2.0
1.5
1.0-
0.5
2.0
100
0.5
0.0
SSV
Instruct Score
Winrate. We compute winrates against SAES by compar- ing overall scores each concept under each setting. We treat ties as 0.5 wins and 0.5 losses. We report the results in Table 3. Again, ReFT-rl (88.0%) and DiffMean (61.6%) achieve winrates greater than 50% against SAES, and relative rankings are similar to those for overall score. We note that DiffMean and ReFT-rl show higher winrates on earlier layers in both models.
Mean concept score vs. instruct score as the steering factor for each method varied.
Figure 4:
tionally benchmark prompting, full-finctuning (SFT), and two parameter-efficient finetuning methods (LoReFT and LoRA) non-steering baselines.
For steering methods, we note that steering factor is an important hyperparameter. We select the optimal steering factor for each method independently for every concept based on which factor achieves the highest overall steering score, as given by the LLM judge. Our actual steering magnitude (i.e., O, described in $4) is the product of the stecring factor and the maximal activations aggregated over the evaluation dataset for concept detection."
Steering factor. We compare the effect of changing the steering factor on instruct vs. concept scores in Figure 4. We notice that increasing the factor monotonically reduces instruct score all methods, i.e. larger steering vectors harm capabilities; this agrees with prior findings (Durmus et al., 2024: Chalnev et al., 2024). However, the effect varies by layer for concept score: concept score increases then decreases in earlier layers, while it roughly monotonically increases with steering factor in later layers. In all cases, ReFT-rl traces a Pareto-optimal path, achieving the highest concept score for any chosen instruct score.
Overall scores. We report the mean overall score for each method (i.e. the harmonic mean of three subscores: flu- ency, instruction-folloving, and concept presence) in Ta- ble 2. Prompting, along with slightly worse finetuning base- lines, outperforms all steering methods on average, except
6. Discussion
FFor SAEs, we query Neuronpedia to obtain the maximal acti- vation per concept.
Simple yet powerful baselines. While representation- level interventions have been shown be useful in both

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
enhancing model capabilities and for safety (see $2), they fail to outperform standard prompting and finetuning base- lines on AXBENCH. This is sobering evidence of the current limitations of steering techniques. However, our results sug- gest that joint learning of concept detection and steering (as in ReFT-r1) may be the key to advancement.
scale benchmark for evaluating these techniques. We believe that the immediate ethical and societal implications of our research are minimal. However, we recognize that enhanced control over language model outputs could potentially be misused reinforce biases or manipulate information. To address these concerns, advocate for the responsible application of steering methods and ensure transparency by publicly releasing our datasets and feature dictionaries. We encourage ongoing collaboration and dialogue within the research community to monitor and mitigate any unintended consequences of these technologies.
SDL vs. SAEs. We have shown that SDL methods can achieve similar scalability and better performance at lower cost compared to SAEs. Unlike SAEs, SDL methods re- quire concepts to be known a priori; however, SDLs can be easily augmented with new features without retraining. We also note that SDLs .s depend on high-quality data generators, whereas SAEs rely on high-quality concept discriminators. These methods are not mutually exclusive and can comple- ment each other.
Acknowledgements
We thank Róbert Csordás, Qinan Yu, and Jiuding Sun for constant and extremely helpful feedback during our weekly interp meetings; Jake Mendel for enlightening discussion about the direction and framing of the work; Neel Nanda for helpful suggestions on SAE feature selection; We thank all of our human annotators listed in Appendix M: and Chenglei Si, Ken Ziyu Liu, Oam Patel, Luke Bailey, Harshit Joshi, Yanzhe 'Sanju' Zhang, Nikil Roashan Selvam, Julie Kallini, Omar Shaikh, Thomas Chen, Tristan Thrush, and Yangjun Ruan for various helpful discussions. We thank Joseph Tey and Nick Jiang for pointing out equation typos in an earlier draft.
SAE concept label quality. The concept lists used in this paper were adapted from Neuronpedia's auto- interpretability pipeline, which is often skewed towards token-level concepts and misses high-level abstractions. While we tried to do post-hoc SAE feature selection to mitigate this, the poor performance of SAES is at least par- tially reflection of the limitations of auto- interpretabiiliyy. It would be interesting to explore whether the SAE perfor- mance on AXBENCH improves as better feature labelling methods are used and labels become less shallow (e.g. Choi et al., 2024). We conduct a preliminary study assessing the steering performance of SAEs and SDLs using a limited set of higher-quality concept labels (see Appendix E.4). Our results show that, although higher-quality labels improve SAE performance, they do not narrow its gap to SDLs such as ReFT-rl. Future work might explore rule-based con- cepts, as in generic instruction-followig benchmarks such as IFEval (Zhou et al., 2023).
This research is supported in part by grants from Open Philanthropy.
References
Usman Anwar, Abulhair Saparov, Javier Rando, Daniel Paleka, Miles Turpin, Peter Hase, Ekdeep Singh Lubana, Erik Jenner, Stephen Casper, Oliver Sourbut, Benjamin L. Edelman, Zhaowei Zhang, Mario Günther, Anton Ko- rinek, Jose Hernandez-Orallo, Lewis Hammond, Eric Bigelow, Alexander Pan, Lauro Langosco, Tomasz Ko- rbak, Heidi Zhang, Ruiqi Zhong, Scán Ó hEigeartaigh, Gabriel Recchia, Giulio Corsi, Alan Chan, Markus An- derljung, Lilian Edwards, Aleksandar Petrov, Chris- tian Schroeder de Witt, Sumeet Ramesh Motwan, Yoshua Bengio, Danqi Chen, Philip H. S. Torr, Samuel Albanie, Tegan Maharaj. Jakob Foerster, Florian Tramer, He He, Atoosa Kasirzadeh, Yejin Choi, and David Krueger. Foun- dational challenges in assuring alignment and safety of large language models. arXii:2404.00993 2024. URL https: /tttps:/aa/te /ttp::///xxvvvvie http//axxxv.ooraabbea// 09932.
7. Conclusion
We introduced AXBENCII, a new benchmark for evaluating LM control methods at scale using synthetic data. To answer the question in the title of this work: our evaluation shows that even at SAE scale, representation steering is still far behind simple prompting and finetuning baselines. Simulta- neously, we showed that novel steering method, ReFT-rl, is capable of closing the gap to some extent; representation- based steering has not yet exhausted its potential. No mat- ter the outcome, we believe that comprehensive evaluation benchmarks like AXBENCH are necessary for continued progress on this problem.
Yonatan Belinkov, Nadir Durrani, Fahim Dalvi, Hassan Sujjad, and James Glass. What do neural machine trans- lation models learn about morphology? In Regina Barzi- lay and Yen Kan, editors, Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers), pages 861-872,
Impact Statement
In this paper, we explore representation baased methods for steering language models and introduce AXBENCH, a large-

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Vancouver, Canada, July 2017. Association for Compu- tational Linguisties. doi: 10. 10.16653/v URL https://aclanthhology httphh/ooooo....o11--//11 org/P17-1080.
Alexis Conneau, German Kruszewski, Guillaume Lample, Loic Barrault, and Marco Baroni. What you can cram into a single S&.#* vector: Probing sentence embed- dings for linguistie properties. Iryna Gurevych and Yusuke Miyao, editors. Proceedings of the 56th Annual Meeting of the Association for Computational Linguis- ties (Volume 1: Long Papers), pages 2126-22 2236, Mel- bourne, Australia, July 2018. Association for Computa- tional Linguistics. doi: 10. 10.18653/w1/P1--
Usha Bhalla, Suraj Srinivas, Asma Ghandcharioun, and Himabindu Lakkaraju. Towards unifying inter- pretability and control: Evaluation via intervention.
arXiv:2411.04430, 2025. URL https://arxiv.. org// 0bs/2411.004330
URL
Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, llya Sutskever, Jan Leike, Jeff Wu, and William Saunders. Language models can explain neurons in language models, 2023. URL https://oppeaiipulli: blob, core. windows.net neurron-explainer/ppprr hwtr//iidexxhhee
Esin Durmus, Alex Tamkin, Jack Clark, Jerry Wei, Jonathan Marcus, Joshua Batson, Kunal Handa, Liane Lovitt, Meg Tong. Miles McCain, Oliver Rausch, Saffron Huang, Sam Bowman, Stuart Ritchie, Tom Henighan, and Deep Gan- guli. Evaluating feature steering: A case study in mitigat- ing social biases, 2024. URL htatt:::///nhhoppiii com/ research/eer
Tolga Bolukbasi, Kai- Wei Chang, James Y. Zou, Venkatesh Saligrama, and Adam T. Kalai. Man to computer programmer woman is to homemaker? Debiasing word embeddings. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 29, Curran As- sociates, Inc., 2016. URL www.ippsccpaappr.aaaco fttpee/aiieeeeaapee/0011/ fffile/ pdf.
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield- Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy mod- els of superposition. Transformer Circuits Thread, 2022. URL https: //ttransforer-ccirull ppbb www.tooooome//eeee..reeee html.
aBBcccleeeaa33277777662444444333
Joschka Braun, Dmitrii Krasheninnikov, Usman An- war, Robert Kirk, Daniel Tan, and David Scott Krueger. A sober look at steering vectors for LLMs. In Alignment Forum, 2024. URL https://www al alignmentforum.com a wtt.m/eetrrrrroorre///s///////// /ttp//pppst///////oo////c..
Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders, 2024. URL https:/ haas::/arxxx /awxiv.ivvvvv////i//eee/aaaa www.obs////aaoooooooooooooo.sg// 2400../224..e00 04093.
Attieus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts. Causal abstractions of neural networks. In M. Ran- zato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wort- man Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 9574-9586. Cur- ran Associates, Inc., 2021. URL httss:/tttt////oeeeiiiign neurips wttpcccppppppeeeeeeeeeeee http://aaae..iileeeppee///////// 4 4fFc222F444445588777777444223333
Stephen Casper, Lennart Schulze, Oam Patel, and Dylan Hadfield-Me Menell.. Defending against unforeseen failure modes with latent adversarial training, 2024. https //arxiv. org/ 2b2/ 05030.
Sviatoslav Chalnev, Matthew Siu, and Arthur Conmy. Im- proving steering vectors by targeting sparse autoencoder features. arXiv:2411.02 1.0219,, 2024. URL WtRLhttss://arxtt org/abs/2411 29gg.089/2411..299.
PapeeePpprrppppp pdff.
Atticus Geiger, Zhengxuan Wu, Hanson Lu, Josh Rozner, Elisa Kreiss, Thomas Icard, Noah D. Goodman, and Christopher Potts. Inducing causal structure for inter- pretable neural networks. In Kamalika Chaudhuri, Ste- fanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato, editors, International Conference on Machine Learning, ICML 2022, volume 162 of Proceed- ings of Machine Learning Research, pages 7324-7338, Baltimore, Maryland, USA, 2022. PMLR. URL https: www.mmm..rresviiiimmmmm / //proceedings. mlr wwwpppeesssveee http:///vvvv2ggigereeee html.
Lawrence Chan, Adrià Garriga-Alonso, Nicholas Goldowsky-Dill, Ryan Greenblatt, Jenny Nitishinskaya, Ansh Radhakrishnan, Buck Shlegeris, and Nate Thomas. Causal scrubbing: A method for rigorously testing interpretability hypotheses. In Alignment Forum, 2022. URL https:://shorttrl.att////
Dami Choi, Vincent Huang, Kevin Meng, Daniel D. John- son, Jacob Steinhardt, and Sarah Schwettmann. Scal- ing automatic neuron description, October 2024. URL http://swsssuceeeer//eeeeeeeeeep
Atticus Geiger, Duligur Ibeling, Amir Zur, Maheep Chaud- hary, Sonakshi Chauhan, Jing Huang, Aryaman Arora, Zhengxuan Wu, Noah Goodman, Christopher Potts, and

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Thomas Icard. Causal abstraction: A theoretical founda- tion for mechanistic interpretability. 0rXiv:2301.047009 2024. URL http://////xiii.ogaabb////////// 04709.
Xiang Xu, Zhe Shen, Zhitao Gong, Zichuan Wei, Victor Cotruta, Phoebe Kirk, Anand Rao, Minh Giang, Ludovic Peran, Tris Warkentin, Eli Collins, Joelle Barral, Zoubin Ghahramani, Raia Hadsell, D. Sculley, Jeanine Banks, Anca Dragan, Slav Petrov, Oriol Vinyals, Jeff Dean, Demis Hassabis, Koray Kavukeuoglu, Clement Farabet, Elena Buchatskaya, Sebastian Borgeaud, Noah Fiedel, Armand Joulin, Kathleen Kenealy, Robert Dadashi, and Alek Andreev. Gemma 2: Improving open language models at a practical size. arXiv:2408. 00118, 2024. URL https://arxiv, www.lv..rgg//bbo hts//bb//2400 00110.01.11..000 00118.
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhu- patiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, Anton Tsitsulin, Nino Vieillard, Piotr Stanezyk, Sertan Girgin, Nikola Momchev, Matt Hoffman, Shantanu Thakoor,
Mario Giulianelli, Jack Harding, Florian Mohnert, Dieuwke Hupkes, and Willem Zuidema. Under the hood: Us- ing diagnostic classifiers to investigate and improve how language models track agreement information. In Tal Linzen, Grzegorz Chrupata, and Afra Alishahi, edi- tors, Proceedings of the 2018 EMNLP Workshop Black- boxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 240-248, Brussels, Belgium, November 2018. Association for Computational Linguisties. doi: 1...8663/1//1185442 URL https: httpss:/acllantholles
Grill, Behnam Neyshabur, Olivier Bachem, Alanna Walton, Aliaksei Severyn, Alicia Parrish, Aliya Ahmad, Allen Hutchison, Alvin Abdagic, Amanda Carl, Amy Shen, Andy Brock, Andy Coenen, Anthony Laforge, Antonia Paterson, Ben Bastian, Bilal Piot, Bo Wu, Brandon Royal, Charlie Chen, Chintu Kumar, Chris Perry, Chris Welty, Christopher A. Choquette- Choo, Danila Sinopalnikov, David Wein- berger, Dimple Vijaykumar, Dominika Rogozinska, Dustin Herbison, Elisa Bandy, Emma Wang, Eric Noland, Erica Moreira, Evan Senter, Evgenii Eltyshev, Francesco Visin, Gabriel Rasskin, Gary Wei, Glenn Cameron, Gus Martins, Hadi Hashemi, Hanna Siimezaak-lluinskka Harleen Batra, Harsh Dhand, Ivan Nardini, Jacinda Mein, Jack Zhou, James Svensson, Jeff Stanway, Jetha Chan, Jin Peng Zhou, Joana Carrasqueira, Joana Iljazi, Jocelyn Becker, Joe Fernandez, Joost van Amersfoort, Josh Gordon, Josh Lipschultz, Josh Newlan, Ju yeong Ji, Kareem Mohamed, Kartikeya Badola, Kat Black, Katie Millican, Keelin McDonell, Kelvin Nguyen, Kiranbir Sodhia, Kish Greene, Lars Lowe Sjoesund, Lauren Usui, Laurent Sifre, Lena Heuermann, Leticia Lago, Lilly McNealus, Livio Baldini Soares, Logan Kilpatrick, Lucas Dixon, Luciano Martins, Machel Reid, Manvinder Singh, Mark Iverson, Martin Görner, Mat Velloso, Mateo Wirth, Matt Davidow, Matt Miller, Matthew Rahtz, Matthew Watson, Meg Risdal, Mehran Kazemi, Michael Moynihan, Ming Zhang, Minsuk Kahng, Minwoo Park, Mofi Rahman, Mohit Khatwani, Natalie Dao, Nenshad Bardoliwalla, Nesh Devanathan, Neta Dumai, Nilay Chauhan, Oscar Wahltinez, Pankil Botarda, Parker Barnes, Paul Barham, Paul Michel, Pengchong Jin, Petko Georgiev, Phil Culliton, Pradeep Kuppala, Ramona Comanescu, Ramona Merhej. Reena Jana, Reza Ardeshir Rokni, Rishabh Agarwal, Ryan Mullins, Samaneh Saadat, Sara Mc Carthy, Sarah Cogan, Sarah Perrin, Sébastien M. R. Arnold, Sebastian Krause, Shengyang Dai, Shruti Garg, Shruti Sheth, Sue Ronstrom, Susan Chan, Timothy Jordan, Ting Yu, Tom Eccles, Tom Hennigan, Tomas Koeisky, Tulsee Doshi, Vihan Jain, Vikas Yadav, Vilobh Meshram, Vishal Dharmadhikari, Warren Barkley, Wei Wei. Wenming Ye, Woohyun Han, Woosuk Kwon,
Gabriel Goh. Decoding the thought vector, 2017. URL https://gaaboh. gi gitbub. Telub..o//houugee htt/ThoughVeecorrs//
Nicholas Goldowsky-Dill, Chris MacLeod, Lucas Sato, and Aryaman Arora. Localizing model behavior with path patching. 2tt1::23340099999 2023. URL https: //arxiv.
05969.
Clément Guerner, Anej Svete, Tianyu Liu, Alexander Warstadt, and Ryan Cotterell. A geometric notion of causal probing. 23tt::2307.110544 2024. URL https: //arxiv.org/abs//2380.... 15054.
Yoav Gur-Arieh, Roy Mayan, Chen Agassy, Atticus Geiger, and Mor Geva. Enhancing automated interpretability with output-centric feature descriptions. (5ti::25010083199 2024. URL https: hegg/bbs/2501 http//251100000 08319.
John Hewitt and Christopher D. Manning. A structural probe for finding syntax in word representations. In Jill Burstein, Christy Doran, and Thamar Solorio, edi- tors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies. Volume 1 (Long and Short Papers), pages 4129 Minneapo- lis, Minnesota, June 2019. Association for Computa- tional Linguistics. doi: 1118663/11/N191444 URL orgee/9-1///191191101
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
language models. In The Tenth International Confer- ence on Learning Representations, ICLR 2022, Virtual Event, 2022. URL htttps://openreviewweee.. wwt.://wee/ffuuu/mm/o id=nZeVKeeFYf9.
Kramar, Anca Dragan, Rohin Shah, and Neel Nanda. Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2. In Yonatan Belinkov, Najoung Kim, Jaap Jumelet, Hosein Mohebbi, Aaron Mueller, and Han- jie Chen, editors, Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 278-300 Miami, Florida, US, Novem- ber 2024. Association for Computational Linguisties. doi: 10. 10011866511//222/bbcccxxxxx./o/ 1.11 URL https: / //aclanthology.orgg// httpa/aoooooy...r////2////aaaa.a
Robert Huben, Hoagy Cunningham, Logan Riggs, Aidan Ewart, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, 2024. OpenReview.net. URL https:/ /ttpsr///pperrvveew... wwt..//oorrmmmdd7767Ws.......
Sheng Liu, Haotian Ye, Lei Xing, and James Y. Zou. In- context vectors: Making in context learning more effec- tive and controllable through latent space steering. In Forty-first International Conference on Machine Learn- ing, 1C24,2024, Vienna, Austria, July 21-27, 2024. Open- Review.net, 2024, URL https://ogg httts:///peereview..o net/ forumlid=dJTChigvlaa.
Caden Juang, Gonçalo Paulo, Jacob Drori, and Nora Belrose. Open source automated interpretability for sparse autoen- coder features, 2024. URL htttp::/blog http:/lll..l..eeeeee eleuther. ai/autointerp/.
Dan Jurafsky and James H. Martin. Speech and Lan- guage Processing. Online, 2025. URL https://web. 3rd ed. draft.
stanford. wtw.oo/dddduuijjfffss/s//sllpp
Aleksandar Makelov.
Sparse autoencoders match supervised features for model steering on the IOI task. In ICML 2024 Workshop on Mechanistic Interpretabillity 2024. URL
Adam Karvonen, Dhruv Pai, Mason Wang, and Ben Keig- win. Sieve: SAEs beat baselines on real-world task (a code generation case study). Tilde Research Blog, 2024. URL https://www. ti tilderesearch. cww/blog//eeeoooooccoocccccoocco bloo/ssieee......oeoooooooooooee Blog post.
Samuel Marks and Max Tegmark. The geometry of truth: Emergent linear structure in large language model repre- sentations of true/false datasets. arXiv:2310.0663 2024. URL https:
Michael Lan, Philip Torr, Austin Meek, Ashkan Khakzar, David Krueger, and Fazl Barez. Sparse autoencoders re- veal universal feature spaces across large language mod- els. arXiv:2410.06% 06991, 2024. URL
Harry Mayne, Yushi Yang, and Adam Mahdi. Can sparse autoencoders be used to decompose and interpret steering vectors? arXiv:2411.088 240029211110,,., 2024. WRLhhttss//arxi 2s//bb2/21111. 08790.
htg/ab//2410 261.0
Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle, and Ole Winther. Autoencoding beyond pix- els using a learned similarity metric. In Maria-Florina Balcan and Kilian Q. Weinberger, editors, Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, volume 48 of JMLR Workshop and Conference Pruceed- ings, pages 1566- JMLR.org, 2016. URL http: //proceedings. . mlr. pecess/v48/larseee html.
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. In S. Koyejo, S. Mohamed, A. Agarwal, D. Bel- grave, K. Cho, and A. Oh, editors, Advances in Neu- ral Information Processing Systems, volume 35, pages 17359-17372. Curran Associates, Inc., 2022. URL https://arxiv htwpii..og///////////// arggabb/2202. 05262.
Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfis- ter, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from language model. Ad- vances in Neural Information Processing Systems, 36, 2024. URL httpt:://aaxi..orrtrtt absp//bbb///22/ 03341.
Tomás Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. Distributed representations of words and phrases and their compositionality. In Christopher J. C. Burges, Léon Bottou, Zoubin Ghahramani, and Kilian Q. Weinberger, editors, Ad- vances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meet- ing held December 5-8, 2013. Lake Tahoe. Nevada, United States, pages 3111-3119, 2013a. URL https:
Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Guirajani, Carlos Guestrin, Percy Liang, and Tat- sunori B. Hashimoto. AlpacaEval: An automatic evalu- ator of instructionnfollooiing models. https://githhu com/tatsu- watpsssuw-aaaaaaaaaaaeaaaaa 5 2023.
/ //proceedings. http://ppppp.ppppp////11//////// 944225318B2200999664442222000009 html.
Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, Janos

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. Lin- guistic regularities in continuous space word represen- tations. In Lucy Vanderwende, Hal Daumé III, and Ka- trin Kirchhoff, editors, Proceedings of the 2013 Con- ference of the North American Chapter of the Associa- tion for Computational Linguistics: Human Language Technologies, pages 746-751, Atlanta, Georgia, June 2013b. Association for Computational Linguistics. URL attps://aclanthol..e wwwpehhoooooooo//111311 org/N1331199/////
Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large lan- guage models. arXiv:2311.03666. 2023. URL https: //arxiv.org/abs/2233 036588.
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake VanderPlas, Alexandre Passos, David Cournapeau, Matthicu Brucher, Matthieu Perrot, and Edouard Duch- esnay. Scikit-learn Machine learning in Python. The Journal of Machine Learning Research, 12:2825 2830, 0tt1:/199955044288881889
Ulisse Mini, Peli Grietzer, Mrinank Sharma, Austin Meek, Monte MacDiarmid, and Alexander Matt Turner. Under- standing and controlling maze-solving policy network. arXiv:2310.08000 08043,, 2023. URL https://arxiv.orr abs/2310. 0.0043.................
2011. doi: //d1. wwd..acm.org///mmm wttp:/r//////eee/ee////// 5wt./11.55555519955 2078195.
URL https:
Jeffrey Pennington, Richard Socher, and Christopher Man- ning. GloVe: Global vectors for word representa- tion. In Alessandro Moschitti, Bo Pang, and Walter Daelemans, editors, Proceedings of the 2014 Confer- ence on Empirical Methods in Natural Language Process- ing (EMNLP), pages 1532-1543, Doha, Qatar, October 2014. Association for Computational Linguisties. doi: 10.3115//1/114-1166 URL https: httpp://acllatthooogy org/D14-1162.
Francesco Moramarco, Alex Papadopoulos Korfiati, Mark Perera, Damir Juric, Jack Flann, Ehud Reiter, Anya Belz, and Aleksandar Savkov. Human evaluation and correla- tion with automatic metrics in consultation note genera- tion. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguis- fics (Volume I: Long Papers), pages 5739-5754, Dublin, Ireland, May 2022. Association for Computational Lin- URL -lnng. 394/.
guistics. doi: 10.1 https://aclanttooll
Itamar Pres, Laura Ruis, Ekdeep Singh Lubana, and David Krueger. Towards reliable evaluation of behavior steer- ing interventions in LLMs, 2024. URL https://arxiv.. org/abs/2410 17245.
Rajiv Movva, Pang Wei Koh, and Emma Pierson. An- notation alignment: Comparing LLM and human anno- tations of conversational safety. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceed- ings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9048- 9062, Miami, Florida, USA, November 2024. Associa- tion for Computational Linguisties. doi: 10.18653/v1/ www..emmmlmmmmi.n1111.... URL https://aallanthloog.. org/2024. ewm.:44a.emmpmmi....mm. 511/.
Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. Steering Llama 2 via contrastive activation addition. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceed- ings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa- pers), pages 15504-15522, Bangkok, Thailand, Au- gust 2024. Association for Computational Linguistics. doi: 10. 10.188653/112202.accl aclanthology.orge http://222...llloooo... 828.
URL. https://
Neel Nanda, Andrew Lee, and Martin Wattenberg. Emer- gent linear representations in world models of self- supervised sequence models. In Yonatan Belinkov, So- phie Hao, Jaap Jumelet, Najoung Kim, Arya McCarthy, and Hosein Mohehbi, editors, Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 16-30, Singapore, De- cember 2023. Association for Computational Linguistics. doi: 10.18653/v1/202. wlw.://ow..aaacccoooo.ooo.. 1.2. URL https: //aclanthology .rw.ggg...rrr// org.222.. http://222.baaaboommeeee blackboxnlp-1.22
Naomi Saphra and Sarah Wiegreffe. Mechanistic? In Yonatan Belinkov, Najoung Kim, Jaap Jumelet, Hosein Mohebbi, Aaron Mueller, and Hanjie Chen, editors, Pro- ceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 480- 498, Miami, Florida, US November 2024. Association for Computational Linguistics. doi: 10. 10.18653/11/2044 blackboxnlp- URL https: 2024. 1lackbbonnlp1.39.1
org/
Kyle O'Brien, David Majercak, Xavier Fernandes, Richard Edgar, Jingya Chen, Harsha Nori, Dean Carignan, Eric Horvitz, and Forough Poursabzi-Sangde. Steer- ing language model refusal with sparse autoencoders. arXiv:2411.11296, 2024. URL https:// htttp::/arxxv..rrg abs/2411. 11296.
Nishant Subramani, Nivedita Suresh, and Matthew Peters. Extracting latent steering vectors from pretrained lan- guage models. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio. editors, Findings of the As- sociation for Computational Linguistics: ACL 2022,

---

AXBENCH: Steering L.Ms? Even Simple Baselines Outperform Sparse Autoencoders
pages 566-581, Dublin, Ireland, May 2022. Associa- tion for Computational Linguisties. doi: 10.18653/ 11/2022.indinns- URL https://aclanthhooogg org/2022. findings-acl.45
editors, Advances in Neural Information Processing Systems, volume 33, pages 12388-12401. Curran As- sociates, Inc., 2020. URL //tp::///proceeiings. neurips. cwwpppprcppppp..pppppppee http://///pppilllllee/////////// 2255022222171555511eff7722222222
pdf.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Ax- iomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, ICML' 17, page 3319-3328. JMLR.org, 2017.
Eric Wallace, Jens Tuyls, Junlin Wang, Sanjay Subramanian, Matt Gardner, and Sameer Singh. AllenNLP interpret: framework for explaining predictions of NLP models. In Proceedings of the 2019 Conference on Empirical Meth- ods in Natural Language Processing and the 9th Interna- tional Joint Conference on Natural Language Processing (EMNLP-IJCNLP), System Demonstrations, pages 7-12, 2019.
Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey. Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Seal- ing monosemanticity Extracting interpretable features from Claude 3 Sonnet. Transformer Circuits Thread, 2024. URL
Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretabilitty in the wild: a circuit for indirect object identification in GPT-2 small. In The Eleventh International Confer- ence on Learning Representations, ICLR 2023, Kigali, Rwanda, 2023, URL https://openrevieet net/pdf? id=NpsVSN6o4ul.
pub/ html.
Alexander Matt Turner, Peli Grietzer, Ulisse Mini, Monte M, and David Udell. Understanding and controlling maze-solving policy network. Alignment Forum March 2023a. URL http:://hhhtttrr...ccm Utt.ttpps//stootuuul.ttttttttt
Yulin Wang, Xuran Pan, Shiji Song, Hong Zhang, Gao Huang, and Cheng Wu. Implicit semantic data augmen- tation for deep networks. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' Alché-Buc. Emily B. Fox, and Roman Gamett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver. BC, Canada, pages 12614 12623, 2019. URL https: / /pprceedings
Alexander Matt Turner, Peli Grietzer, and Lisa Thiergart. Maze-solving agents: Add top-right vector, make the agent go to the top-right. Alignment Forum, March 2023b. URI. htt.hhtts:///hott turl, atd99.
Alexander Matt Tumnr,, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte Mac- Diarmid. Steering language models with activation engineering. arXiv:2308. 10248, 2024. URL https: wwrxiv.org//bs/22330 10248.
15 ieteeeedddnnn..nuiiiissseeeee cc/ wwwpc//appe//aeeeee /2019/ashh html.
Absttacc...
Tom White.
Sampling
generative networks. URL https://arxiv.
Paul Upchurch, Jacob R. Gardner, Geoff Pleiss, Robert Pless, Noah Snavely, Kavita Bala, and Kilian Q. Wein- berger. Deep feature interpolation for image content changes. In 2017 IEEE Conference on Computer Vi- sion and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 6090 6099. IEEE Com- puter Society, 2017. doi: 10. 109/CVPR.2017.666 URL https://doi or0 11 /V9//CVRR. 2017.645
1609v06609044400 2016.
org/abs/1609. 04468.
Zhengxuan Wu, Aryaman Arora, Zheng Wang, Atticus Geiger, Dan Jurafsky, Christopher D. Manning, and Christopher Potts. ReFT: Representation finetuning for language models. arX5v:2404.03392, 2024a. URL https://arxinn http:s///aaxivoorr/aa////// orgg/bs/////.....b 03390
Teun van der Weij. Massimo Poesio, and Nandi Schoots. Extending activation steering to broad skills and multiple behaviours. axXiv:2403.05767, 2024. URL https:// wwxiv.org/abs/2e2. 05770
Zhengxuan Wu, Atticus Geiger, Aryaman Arora, Jing Huang, Zheng Wang, Noah Goodman, Christopher Man- ning, and Christopher Potts. pyvene: A library for under- standing and improving PyTorch models via interventions. In Kai- Wei Chang, Annie Lee, and Nazneen Rajani, ed- itors, Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies (Volume 3: System Demonstrations), pages 158-165, Mexico City.
Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. Investigating gender bias in language models using causal mediation analysis. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin,

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Mexico, June 2024b. Association for Computational Lin-
guistics, doi: 101188633///22224.mmceccmmomomm 16. URL http::/aaclamttoooey.orrgg http://ooooo.....22//////// -ttp://222.aaaalddeeeeeeea nemoo- 16.
Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-folloiinng evaluation for large language mod- els. In 20.222111.07111 2023. URL htttp:://aaiii
Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hendrycks. Rep- resentation engineering: A top-down approach to AI transparency. arXiv:2310.01405, 2023. URL https: //arxiv.org/abs/2311 01405..
Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, Rowan Wang, Zico Kolter, Matt Fredrikson, and Dan Hendrycks. Im- proving alignment and robustness with circuit breakers, 2024. URL / www..rgg/bbbooe hrrg/bbs/2406. 2431.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Appendix
A. Historical notes on steering
Inspired by Jurafsky and Martin (2025) and noting the sociological observations about (mechanistic) interpretability field in Saphra and Wiegreffe (2024), offer some historical notes the development of steering a field in an effort to document and properly cite where these ideas came from.
Steering refers to applying interventions (usually adding fixed vector) to the activation space of a neural model in order to control its generations. Early precursors to steering noted that linear subspaces of the representation space of pretrained word vectors seemed to encode meaningful concepts (Mikolov et al., 2013a; Pennington et al., 2014; Bolukbasi et al., 2016).
Larsen et (2016) first used the difference-in- innieee----nansss technique extract visual attribute vectors from GAN discriminators in order to steer generator outputs; this technique widely adopted in computer vision (White, 2016; Upchurch et al., 2017; Goh, 2017; Wang et al., 2019).
In NLP, initial work by Subramani et al. (2022) proposed steering vectors, learned to maximise the probability of some output, as an alternative to expensive fine-tuning and unreliable prompt optimisation for the task of controllable text generation. Soon after, steering was also use to localise behaviours a maze-searching RL agent (Turner al., 2023a;b; Mini et al., 2023). Variations this approach (sometimes using difference in means or other closed-form expressions compute the vector) were adopted by researchers in mechanistic interpretability from late 2023 for Al safety (Zou et al., 2023; Li et al., 2024; Turner al., 2024; Marks and Tegmark, 2024; Rimsky et al., 2024) and later general-purpose but localised and parameter-efficien alternative to finetuning (Wu et al., 2024a; Liu et al., 2024; van der Weij et al., 2024).
Sparse autoencoders (SAEs), scalable technique for self-supervised rank-one linear feature discovery via dictionary learning, are also increasingly used to find or learn steering vectors (Templeton et al., 2024; Chalnev et al., 2024; Makelov, 2024; O' O'Brien et al., 2024).
B. SAE concept list
We use SAE concept lists to enable fair comparison with SAEs, which were annotated mostly by 3pt-3.5-ture or gpt-4o-mini. These concept lists are released by Neuronpedia and were scraped by the authors of this paper in November 2024. We utilize the concept lists from four SAEs from GemmaScope: 10-gemmascope-res 6k for the Gemma-2-28 base model and 20 20-gemaascope-res restmaasscoperes-11111 131k for the Gemma-2-99 instruction-tunad model, where we scraped maximum of 16K concepts.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
C. Detailed analysis
C.1. Concept detection
Gemma-2-2B: L10
Gemma-2-22. L20
1.00
Method
0.75
DiffMean
Probe
True Positive Rate
ReFT-rl
0.25
Prompt
0.00
SAE-A
Gemma-2-4B: 1,20
Gemma-2-9B. 1,31
BoW
1.00
SSV
LAT
0.75
SAB
PCA
0.25
IG
IxG
0.00
0.25
0.50
0.50
0.75
1.00
False Positive Rate
Figure
Mean ROC curves over all concepts.
DitlMun
Prube
ReFT-el
Prompt
SAE-A
BuW
SEV
LAT
SAE
PCA
IG
3G
100
075-
Gomma-2-
0.9-
025-
0.00
Cirmmaa
True Fostuive Rale
0.5-
0,50
025-
0.0-
1.00
Commaa
075-
0.50
0.00
800
False Positive Rate
Figure 6: All ROC curves.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
C.2. Model steering
Gemma-2-22B. L10
Gemuma-2-2B: L20
Gemana-2-95. L20
Gemma--299B: L31
Canoggt SearptSScree Score
10-
0.
0.
20
Fluency Score
15
10
Score
Instruct Score
Overall S Score
0.5
0.0
Prompt
LoReFT
SFT
LoRA
ReFT-e1
DilMean
SAE
SAE-A
LAT
PCA
Probe
Method
Figure 7: Mean score breakdown for all methods unseen testing instruction set after selecting the optimal factor (based on the Overall Score) on our evaluation instruction set. For prompting and finetuning, we randomly score one generation on the testing instruction set (since the factor is not parameter for those methods), resulting in the same number of observations for those methods.
DiffMean
LAT
PCA
Probe
ReFT-r1
SAE
SAE-A
SSV
500
Gemma
400
300
L10
200
100
2-28
0-
500
0em.ma-2-28
300 -
1,20
Count
100
0-
Gemma 2-98
400
300
L20
200
100
500
Gemma- 2-95
400
L31
200
0
Optimal Steering Factor
Figure 8: Distribution of optimal steering factors for each method across the 4 tasks.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Gemma-2-22 2-28
Gemma-2-28
Gemma-2-44 2-99 L200
Gemma-2-99 L.11
L10
120
Concept Score
0.8
0.4
0.0
Method
1.0
Fhency Score
ReFT-rl
DiffMean
SAE
Score
SAE-A
2.0
LAT
1.5
Instruct Score
PCA
Probe
0.5
SSV
Overall Score
0.4
0.2
Steering Factor
Figure Steering factor vs. scores.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
D. Supervised dictionary learning method works with very limited amount of training data.
Based the performance results, ReFT-rl is the strongest SAE alternative. We further study the data scaling law of ReFT-rl by varying the number of training examples. Specifically, we measure ReFT-rl performance on both concept detection and steering with CONCEPT 10 when the number of training example is set to {6, 12, 24, 48, 72, 96, 120, 144). In the extreme setting, we provide only 3 positive and negative examples. Since we have a limited pool of concepts, we average our results with three random seeds: {42, 43, 44).
Figure 10 shows how the performance of ReFT-rl varies C (concept detection) and (model steering) when trained with different numbers of training examples. For earlier layers, scores increase with more data, while for Bemma-2-98, the trend is less clear for concept detection. Our results indicate that once certain threshold is reached, performance saturates for both tasks, suggesting that the cost of training ReFT-rl can be further reduced. The per-concept cost with 144 training examples is approximately S0.008, and this cost decreases proportionally as the number of training examples is reduced.
Gemma-2-2B L10
Gemma-2-285 L20
0.8
0.6
0.4-
Overall Score
Examples
0.2
50
Gemma-2-9B. L20
Gemma-244 2-9B: L31
0.8
100
0.4
0.2
0.80
0.85
0.90
0.95
1,00
0.80
0.85
0.90
0.95
1,00
AUROC
Figure 10: Scaling law for supervised dictionary learning (SDL) method ReFT-rl with CONCEPT 10 on both C detection and S model steering.
concept

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
E. SDLs at scale: Analysing CONCEPTI6K
E.1. ReFT-rl: CONCEPT16K subspace for code error handling.
We scale up supervised dictionary learning methods DiffMean and ReFT-rl with CONCEPT16K. They drop-in replacements of existing SAEs Gemma models with better performance for concept detection and steering.
Figure 11 shows the UMAP of ReFT-rl's CONCEPTION 166K subspaces learned with Gemma-2-22 at layer 20's residual stream. Subspaces are meaningfully clustered together by genres. Within each genre cluster, related features are also clustered together. For instance, we identify subspace cluster for concepts related to "Code error handling and logging," which includes the following concepts:
Subspace 16K/14 16K/14404: "error messages related to system calls and file operations"
Subspace 16K/140 1 "terms related to programming errors and error handling"
Subspace 16K/5656: "technical terms and parameters related to errors and status in programming contexts"
Subspace 16K/4884:
"error messages and exceptions in code related to server or network operations"
Subspace
16K/2467: "references to errors and warnings, especially related to file or access issues"
ReFT-rl subspaces
genre
code
math
risttd to ennerrs and ststue in programming contects
terms robite prngraminn srives and crvee handling
IMAPP
UMAP2
cmur measge melated system oparatioro
capaially reliicd so file accies sensee
crror moormeeegern exception in code matted to semeror neteock openttioo
UMAP1
UMAP1
Figure 11: UMAP of ReFT-rl's CONCEPT 66K subspaces with Gemma-2-28 at layer 20's residual stream.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
E.2. Mapping natural language to subspaces.
We explore whether we can find direct mapping from natural-language concept descriptions to subspaces. We first train ReFT-rl with CONCEPT 6K and create a supervised dataset Diieveratorr {(c, 10 ), where the input e is the concept description in natural language and the output is the ReFT-rl subspace vector corresponding to the concept. We divide DGenerator into training and testing sets, ensuring that the testing set contains only concepts from CONCEPT500, which are excluded from the training set. To train the generator, we attach supervised linear head Peeerrtor the last input token representation the n-th position of the last layer m, predicting the learned ReFT-rl subspace:
L = LMSEeCCosine (OFF--- Generator (LMo(c))m))
(11)
where fine-tune the generator head and the LM using equally weighted MSE and cosine distance losses. We do finetune the base LM Gemma-2-28 for our subspace generators. We partition the last 500 examples in our training dataset as our in-training development set to early-stop our training with patience step set to 3.
We generate ReFT-rl subspaces for CONCEPT500 and follow our evaluation paradigm in AXBENCH to evaluate concept detection and model steering. We show two cases below by unembedding generated subspaces with the output embedding matrix. We find that the subspace generator works better in English as opposed to other languages.
As shown in Table 4a and Table 4b, subspaces for unseen concepts generated by our finetuned model exhibit only slight performance degradation in concept detection, while performance drops more significantly in model steering.
Method
Gemma-2-2B
Gemma-2-9F
Avg.
Method
Gemma-2-2B
Gemma-2-99
Avg.
L10
L20
L20
L31
L10
L20
L20
L31
DiffMean
0.948
0.946
0.955
0.921
0.942
ReFT-r1
0.633
0.509
0.630
0.401
0.543
ReFT-rl
0.952
0.965
0.966
0.869
0.938
ReFT-rl (Gen)
0,415
0.466
ReFT-ri (Gen)
0.945
0.965
DiffMean
0.297
0.178
0.322
0.158
0.239
SAE
0.735
0.755
0.631
0.659
0.695
SAE
0.177
0.151
0.191
0.140
0.165
(a)
Mean AUROC.
(b)
Overall score.
Table 4: Results CONCEPT500 for ReFT-rl (Gen) ReFT-r1 and other selected methods.
Unseen concept description in Chinese
Top positive logits when unembedding the subspace
ethical', 1,4296875), moral', 1.3984375), (' ethics', 1.2421875), ('Ethical', 1.1640625), Ethical', 1.15625), ( ("moral', 1.125), Rthics', 1.0859375), (" Moral", 1.0859375), ('Ethics', 1.0703125), ('ethical', 1.0703125)
Top negative logits when unembedding the subspace
("DockStyle', -0.78125), (' venons', -0.6796875), C' purpose", -0.67578125), ('compleCContett, -0.671875), ( stupidly', 0.66796875), (' fooled', (0.66015625), (: Jefus', -0.65234375), (' small", -0.6328125) (" montón', monntón',-066222 -0.62199775) Dummies', -0.6171875)
"https://en. http:////e.wwikiiiii.e
Ching...

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Unseen concept description in English
Business-related terms and symbols, particularly focusing on entrepreneurship and financial aspects, well as formatting and coding indicators"
Top positive logits when unembedding the subspace
'investment', 1.1953125), (' asset', 1.1484375), ( financial', 1.1328125), ( investments', 1.0625), ( Investment', 1.046875), market', 1.0390625), ( portfolio', 1.03125), ( investor', 1.03125), (' assets', 1.0078125), (" investors' 1.0078125)
Top negative logits when unembedding the subspace
sauvages', -0.8115625), (' hâte', -0.76953125), (' rapides', -0.76171875), (' régl', -0.7421875), ( découvertes', -0.71875), C fermés', -0.69921875), (' complètes', -0.69140623,, précédents", -0.68359375), ('setVerticalGroup) -0.671875)
00t.Grooppp000039997777777 -0.68359375), ( découver',
"Taken from http::/ttww...t.....c...ctcctttt 2- 1-2b.csy csv.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
E.3. Teleporting between subspaces across models through affine transformations.
We explore whether structural equivalence in subspaces exists across models. Previous works have analyzed feature universality in SAES but have been limited to a small set of features (Lan et al., 2024). Given that our CONCEPTI 6K dataset contains two sets of concepts for Gemma-2- 22 and Gemma-2-99 we first train ReFT-rl on both models separately, obtaining w w and ReFF-rr- Next, we perform a cross-fitting experiment, training ReFT-rl on Gemma-2-28 with concepts from 9R Thus, w ReFFT-l and ReFFFll represent two sets of subspaces
2B Gemma-2 resulting in w WReF-rI and vice versa for WReFT-1-- from different models that correspond to the same set of concepts.
9R |B
We then study whether transformation can map between these two sets of subspaces:
9B ReFT.rl
1
(W 2B 9B Transformation ReFT:e11
BB
where & Pransformation is parameterized by a linear layer with bias (i.e., an affine transformation). We learn the transformation using equally weighted MSE and cosine distance losses. Similarly, During training, we exclude concepts from CONCEPT500, and evaluate the transformation on CONCEPT500 at test time by generating subspaces. We follow our evaluation paradigm in AXBENCH to assess concept detection and model steering.
T Trrassormmation is trained by reversing the direction.
Our evaluation results on CONCEPT500 are presented in Table 5a and Table 5b. Surprisingly, the affine transformation performs well in both directions (from 2B 9B and 9B 2B), with little no change in concept detection performance. While performance drops for model steering, it still outperforms other methods, including fine-tuning. Figure 12 and Figure 13 visualize the transformations using the first two PCA dimensions. PCA is preferred over UMAP in this context because it is sensitive to rotation.
Method
Gemma-2-2B L10
Gemma-2-9B L,00
Avg.
Method
Gemma-2-2B
Gemma-2-9B
Avg.
L20
L31
L10
L,20
L20
L31
DiftMean
0.948
0.946
0.955
0.921
0.942
ReFT-rl
0.633
0.509
0.630
0.401
0.543
ReFT-rl
0.952
0.965
0.966
0.869
0.938
ReFT-rl (9B-2BB)
0.444
ReFT-rl (9B-2BB)
0.954
ReFT-rl (2B-9BB)
0.541
ReFT-rl (2B-9BB)
0.974
DiftMcan
0.297
0.178
0.322
0.158
0.239
SAE
0.735
0.755
0.631
0.659
0.695
SAE
0.177
0.151
0.191
0.140
0.165
(a) Mean AUROC.
(b) Overall score.
Table 5: Results on CONCEPT500 for ReFT-rl (affine) vs. ReFT-rl and other selected methods.
Gemma-2-20 L20 (Souree)
Gemma-2-99 L20 (Target)
Source > Target
066
0.50
0.50-
0.4
2CA
022
0.25
Genne
Code
0-
Math
0-
Text
-02-
-0.25
-0.25
-04-
-0.50
-2
0.2
0.4
a6
05
PCA 1
Figure 12: Visualizations of CONCEPTI6K subspaces of component analysis (PCA) dimensions. The last panel is the derived subspaces by transforming the subspaces from Gemma-2-28 Gemma-2-98 through a learned affine transformation. The concept lists for CONCEPT is taken from the source model.
and Gemma-2-98 at layer 20 with top 2 principal

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Gemma-2-221 2-28 1.20 (Target)
Gemma-2-9B 1.20 (Source)
Source > Target
0.75
0.50
Genre
PCA
0.25
Code
Math
0-
0-
Text
-0.25
-02-
-0.25
-022
0.25
0.50
0.7
-03
0.6
09
PCA
Figure 13: Visualizations of CONCEPTI6K subspaces of Gemma-2-22 and Gemma-2-98 at layer 20 with top 2 principal component analysis (PCA) dimensions. The last panel the derived subspaces by transforming the subspaces from Gemma-2- 2-9B Gemma-2-22 2B through a learned affine transformation. The concept lists for CONCEPTI is taken from the source model.
E.4. Higher-quality concept labels improve steering performance but do not change the rankings
We acknowledge that the auto-interpeetability pipeline's labels contain noise. To validate approach, reran preliminary experiments with the recently released set of high-quality SAE labels from Gur-Arieh et al. (2024). Using these labels for both SAE and ReFT-rl on Gemma-2- 2-28, we observed the same overall pattern: ReFT-rl remains significantly stronger than SAE.
Interestingly, SAE's steering performance improved slightly with the cleaner annotations, and its concept-detection AUROC rose from 0.75 to 0.85 for both layers. Thus, while higher-quality labels benefit SAE, they do close the gap to the stronger supervised methods.
Method
2B L.10
2B I.20
SAE (original)
0.175
0.175
SAE (higher quality)
0.250
0,250
ReFT-rl (original)
0.550 0.590
0.500
ReFT-r1 (higher quality)
0.450
Table 6: Steering scores based on roughly 20 higher-quality concept labels from Gur-Arieh et al. (2024). Original scores are included for comparison.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
F. Ablations
E.I. SAE
Addition vs. clamping. In our main results, using SAEs by adding their decoder features directly the residual stream. While this is a common technique for steering with SAES, most work by Anthropic (e.g. Templeton al., 2024; Durmus et al., 2024) uses an alternative formulation termed clamping, where the latent 2 for feature f is directly clamped to value a (multiplied by the maximum activation for that feature my) y) and the full intervened SAE output added to its unclamped reconstruction error Err(h:).
clamped
S
SAE Clamp
(h;) (W
enc ha + (a m 22 Waee dee + Err(h:)
(12)
2f = (W T hs)s
(13)
Err(h.) = h4 (W enc W
dec
(14)
where e is one-hot vector with non-zero entry at the dimension corresponding to my. We evaluate clamping on all steering tasks on CONCEPT500 for direct comparison with the addition-based GemmaScope SAE. We use the following values for (the steering factor): {0.4, 0.8, 1.2, 1.6, 2.0, 3.0, 4.0, 6.0, 8.0, 10.0, 20.0, 40.0, 60.0, 100.0). Overall, find that clamping is on average worse than addition for SAEs, although it exhibits marked improvement when scaling up from 2B to 9B.
Maximum activation and minimum clamping. In our main results, the maximum activation for our feature my is obtained from Neuronpedia. This approach differs from other methods, which determine the maximum activation by analyzing the activation distribution over the evaluation dataset for concept detection. For this experiment, we calculate my for SAEs in the same manner as other methods. As shown in Table and Figure 14, changing the method of calculating maximum activations has minimal impact the steering performance; most comparisons are statistically insignificant.
In addition, building regular activation clamping described above, try novel minimal clamping where we only clamp the activation value if it is smaller than the target value:
clamped
&SAAR
SAF Clamp
(h+) (W (max(( my. 21) 2 W dec + Err(h,)
(15)
where (WT enc hi) is the original activation value at the corresponding of feature f and e is one-hot vector with a non-zero entry at the dimension corresponding to my. As shown in Table 7 and Figure 14, using minimum clamping has no significant impact on SAE's steering performance.
Results. We report results in Table We also examine the effect of varying & in Figure 14. Note that a is likely a concept-dependent parameter; the optimal a varies from concept to concept. We notice an odd trend for clamping: small values of a have a similar effect on model behaviour as large values of o; both cause concept score increase and instruct score decrease.
Method
Gemma-2-2IF L10
Gemma-2-9R
Avg.
Method
Gemma-2-22
Gemma-2-9R
Avg.
T,20
1.20
L,31
L10
L.20
1.20
I.31
SAE
0.177
0.151
0.191
0.140
0.165
SAE
50.0%
50.0%
50.0%
50.0%
50.0%
SAE (max act)
0.166
0.150
0.163
0.128
0.152
SAE (max act)
49.1%
49.8%
46.8%
47.5%
48.3%
SAE-c (min clamp)
0.074
0.072
0.123
0.090
0.090
SAE-c
36.3%
38.7%
42.1% 1%
49.2%
41.6%
SAE-c
0.063
0.061
0.126
0.120
0.088
SAE-c (min clamp)
38.2%
40.1%
41.0%
42.8%
40.5%
(a) Overall score.
(b) Winrate.
Table 7: S Overall scores model steering.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Gemma-2-2B: L10
Gemma-2-2B 1.0
0.4
Method
0.3
SAE
0.2
SAE (max act)
SAE-c (min clamp)
Concept Score
0.1
SAE-c
Gemma-2228 L20
Gemma-2228 L31
Steering Factor
25
100
2.0
1.5
2.0
1.5
1.1
0.
Instruct Score
Figure 14: S Instruct score vs. concept score for SAEs with addition (SAE) vs. clamping (SAE-c) when varying the steering factor. Additionally, we include results when SAE is clamped with the maximum activation value calculated based evaluation dataset for concept detection, well as results with minimum clamping of activation values.
G. Large language model (LLM) usage
We LLMs for two purposes: to generate labelled concept data for training supervised steering methods and to evaluate the responses generated by the steered models. Specifically, alias gpt-4o-mini ni in the API) throughout our experiments. The date we access the LLM ranges from December 2024 to January 2025, and we use the default generation configuration with temperature set to 1.0 to fetching LLM responses. For 1M tokens, costs $0.15 for input tokens and $0.60 for output tokens.
OpenAl's gpt-4o -2024-07--1 (accessed via the
H. Gradient-based baselines
Inputxgradient G). Gradient-based interpretability methods have been shown be useful in computer vision and NLP (Sundararajan al., 2017; Wallace al., 2019). IxG serves as the gradient-based baseline. We first train linear classification head PCLS on the token representation at the n-th position of the last layer to predict the ground-truth concept-presence class label y::
C LECB (y, (cLL((!!"))
(16)
where PcLs is parameterised by an MLP with two linear layers. For an evaluation sentence x, the LM generates hidden representations h with n tokens at layer I. With Autograd provided in PyTorch, we calculate the gradient of the output classification head with respect to each hidden representations. To aggregate across dimensions, we compute the sum the absolute gradients over all dimensions for each h, which we use as the token-level importance. This gives sequence of aggregated values:
LXG Detect (h) = - In.
which indicates the relevance of each token for the concept. For concept detection, then max-pooling as described in $3.2 to get sequence-level predictions. IxG is not applicable for model steering.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Integrated gradients (IG). We adapt IG (Sundararajan et al., 2017) to trace the accumulated gradients with respect to intermediate representations. To use IG, we train a classification head as in IxG. For each token representation h4, we compute IG along a straight-line path from a baseline h haselline to h4 Here, we use the embedding of a single space token (i.e., tokenizer("
')), obtained via the tokenizer and model embeddings, as the baseline. The IG computed as:
IG,), = (h; h
bhsseline)
V P (h (hpeselin
baseline + ax(h; -1 (.baseline) da
where the integral is approximated using discrete summation with 50 interpolation steps. See IG's original paper about implementation details on path intergral. To aggregate across dimensions, we compute the sum of the absolute gradients over all dimensions for each h, We follow the setup for IxG get sequence-leve predictions for concept detection. IG not applicable for model steering.
Instruction pool
To create instruction pool, sample instructions from existing datasets covering three genres: text, code, and math. For text-based instructions, we sample from Dolly-15K For rode-based instructions, we sample from collection of Python-code puzzles formed in Alpaca-style (i.e., instructions with corresponding responses)) For math-based instructions, we sample from GSM8K For each genre, we create two disjoint sets for training and testing separately with 1,000 examples each. The LLM generated response for each instruction is provided as well. To avoid any heterogeneity, we avoid using the golden labels provided by the orignal dataset and only use the LLM generated response.
Text instruction
Why do people select private schools over public schools?
LLM generated response
Sure, here some ways reduce air pollution: There many reasons why people choose private schools public schools. Here are of the most common:
Academic Focus and Curriculum:**
**mmaller class *izes:**
Private schools often have smaller class sizes, allowing for individualized attention
teachers.
**Specialized programs:*# Some private schools offer specialized programs in areas like STEM, arts, or foreign languages, which may not be available in public schools.
Rigoroous academics:*** Private schools often have higher academic standards and more challenging curriculum, which prepare students for college careers.
#*Emphasis on specific values:** Some private schools emphasize specific values, such as religious faith, classical education, or particular philosophy of learning.
** Extrrccuurricul Activities and Resources:
Extensive extracurricular activities:** Private schools often have a wider range of extracurricular activities, includ- ing sports, music, drama, and clubs.
-State-of-the-an facilitics:* ** Private schools
'ee choose small step number due to limited compute resource. For 9B models, we use a step size of 5.
/httpss://ugging:: http://innnnaaac.eaatatt//////t/ http:://wwggggggeccc.cceaaatttte
aak_pp
#nnttps://huging htt.c//aatteeet/oeeea//ggggaaeee

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Code instruction
Create machine learming classifier in Python for categorizing user input
LLM generated response
import numpy as np
from skleam selectint import main_test..slit from Aklearn.feature www.ffeeettettteeeeeeeeeeeeeeeee from sklcarn.navve_aag import MultinomialNB
www.tureeeetttaatio...eett eettraction.et... text import Count Vectorizer
Sample data encapsulating numeric outcomes related to API logs data =
("GET /api/users". 0).
("POST /api/data", 1).
("PUT /api/setrings".
("DELETE /api/users/123", 0)
Separate features and labels
X. y = zip( dataa)
Math instruction
John puts $25 in his piggy bank every month for 2 years to up for vacation. He had to spend $400 from his piggy bank savings last week repair his car. How many dollars left in his piggy bank?
LLM generated response
Here's how solve the problem:
1. Calculate total savings:**
*
John saves $25 per month for 12 months in year, in years he saves 2 years * 12 months/year 24 months.
* total savings are $25/month months $600.
**2, Calculate remaining *
John spent $400 his car, so has $600 $400 $200 left in his piggy bank.
A John has $200 left in his piggy bank.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
J. Prompt templates
In this section, present the templates that to call LLM to generate datasets evaluate results. For placeholders in the template, they will be filled with proper information.
J.1. Prompt-based steering
Our prompt- based model steering baseline is not zero-shot prompting baseline. Instead of directly prompting LLM to generate steered responses, we first use LLM to generate enhanced prompt for model steering. Our template is included in the following.
LLM-based steering prompt generation
Generate prompt to guide language model in producing responses.
Objective: Direct the model to include content related [Concept goes here] (the concept) in its responses. Ensure the responses reference this concept, even doesn't 't directly answer the question seems out of context. Optionally, provide in-context examples to reinforce this behaviour.
Return only the final prompt without any additional text.
J.2. Synthetic data generation
Our data generation pipeline contains multiple steps, and we use different templates at each step. We present the template that we use for each step in the following.
Fetch genre
Given the concept:
[Concept goes here]
Identify the single primary genre that best fits the concept from the following options:
Text; Code; Math
Output only the best-fitting genre. If none apply, output '<NONE>
#*Frmatting Guidelines:**
Output genre on single line.
Do not include any additional text or formatting.
**Examples:***
Concept: "words or phrases containing odd numbers' Output:
Concept: programming error' Output: Code
Concept: 'integral calculus' Output: Math
Concept: narrative poem' Output: Text
Return only the single best-fitting genre as specified.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
List words related to the concept
Given the following concept:
[Concept goes here]
Your task is to list up to 10 English words that are closely related to this concept. Each word should be single, com- mon English word.
Output each word on separate line, plain text, without any special formatting (c.g., no quotation marks, numbers, bullet points, or additional text).
If the concept is too broad vague (e.g., "any English word', "words starting with A)), if the concept refers to spe- cific technical term, a computer program, or a specific fact, then output "<NONE>" without quotation marks.
Do not include any additional explanations or text other than the words or <NONE>> specified.
Find alternative senses of word
Given the word:
[Word goes here]
Provide one other common semantic meaning of this word that is distinct from and unrelated to:
[Concept goes here]
Your response should he brief description the other meaning, written in plain text without any special formatting. Specifically:
Do not quotation marks.
not include numbers, bullet points, or any prefixes. Do not add any additional explanations or text.
If there is other obvious semantic meaning unrelated to the provided concept, simply output '<NONE>' without quotation marks.
Check whether two senses are different
Determine if Concept A meaningfully distinct from Concept by thoroughly examining their definitions, features, typical usage, and any potential overlaps in meaning, context, or purpose.
[Concept goes here] Concept B: [Concept goes here]
Concept A:
Analyze these concepts **any** shared meanings, contexts, roles, or purposes, focusing on how they relate or inter- sect. Please explain your reasoning, considering both similarities and differences.
If Concept and Concept В have ** overlap in meaning, context, usage, if is subset specific in- stance of other, conclude with 'Answer: <NO>'. Only they are **entirely * with **no overlap "hatsoever* meaning, context, or usage, conclude with 'Anwwr: <YES>'
F*iinal *
'Answer: <YES>' or Answer: <NO>'.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Check whether one sense is different from other concepts
Evaluate whether Concept A is meaningfully distinct from given set of concepts by examining their definitions, core features, typical usage, and any potential overlaps meaning, context, or purpose.
Concept A: [Concept goes here]
Existing Concepts: [Concepts here]
For cach concept in the set, analyze Concept for **any** shared meanings, contexts, roles, or purposes. Consider how Concept A might relate or intersect with each concept individually, as well as with the group collectively. Please explain your reasoning by examining both similarities and differences.
If Concept A has **ay overlap in meaning, context, usage, if is subset specific instance of * concept** in the set, conclude with 'Answer: <NO>'.
Only Concept is =entirllly unrelated** with **no overlap whatsoever meaning, context, or usage to **all** concepts in the set, conclude with 'Answer: <YES>'.
F*nan Asswer: 'Answer: <YES>' or 'Answer: <NO>'.
Modify content with concept
Content Modification Task:
You are given the following content:
[Modifying content here]
Your task to minimally modify this content by inserting some commonly used words, phrases, or elements that reflect themes ideas related "[Concepts go here]' into middle content. These insertions should not be at beginning or end of the content, even if they disrupt overall coherence.
Guidelines:
Try to avoid copying words from definition "[Concepts go here]' possible.
Ensure parts of the content remain unrelated to the concept '[Concepts go here]".
The final content should have approximately the same length as the original content.
The concept should be clearly represented through the inserted word, phrase, or element, even if the content's 's meaning isn' 't' entirely coherent.
Use special characters only if appropriate for genre (e.g,, operators code or math equations).
Output:
Include the special tag <FINAL> the beginning of the final content, followed by the content itself. Return only this tagged content, with no additional text.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Modify content with contrastive concept
Content Modification Task:
You are given the following content:
[Concept goes here]
Your task minimally modify this content by inserting the word 'WORD' into the middle of the content. This word, along with modified content, should convey meanings related to the concept "[Concept goes here]". The insertion should not be at the beginning end of the content.
Guidelines:
Ensure parts the content remain irrelevant to the concept '[Concept goes here]'.
Avoid any mention of "[Contrast concept goes here]' in the content, regardless of coherence.
The final content should have approximately the same length the original content.
Ensure the content reflects the essence of the concept associated with '[Concept goes herej', if the overall meaning isn't entirely coherent.
Ensure grammatical correctness (or syntactical correctness for code/eqqations).
Use special characters only if appropriate for the genre (c.g., operators in code or math equations).
Output:
Include the special tag <FINAL> the beginning final content, followed by content itself. Return only this tagged content, with no additional text.
Generate response given instruction
Given the following instruction:
[Instruction goes here]
Your task to provide a response.
*oormatting Guidelines:
Reeturn only the response to the instruction.
Write the final content (or appropriate format for the genre) in plain
not include any additional text, explanations, or formatting.
F*nanal # Return only the final content, following the guidelines above.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Generate response given instruction and concept
Given the following instruction:
[Instruction goes here]
Your task is to:
Provide response that incorporates elements related to '[Concept goes here]'.
Try avoid copying words from the definition of "Conneept goes here|" if possible.
3. Ensure that your response relates to "[Concept goes here]", even the overall meaning not fully coherent.
#*Formattnng Guidelines:
Return only the response the instruction.
Write the final content (or appropriate format for the genre) plain text.
Do not include any additional text, explanations, or formatting.
**Final Answer: Return only the final content, following the guidelines above.
Generate response given instruction without mentioning given concept
Given the following instruction:
[Instruction goes here]
Your task is to:
1. Provide response that continues addresses the instruction naturally.
Avoid any mention of '[Concept goes here]' the continuation, regardless coherence.
#*Fomattinng Guidelines:
Reeturn only the response to the instruction.
Write the final content (or appropriate format for the genre) in plain
not include any additional text, explanations, or formatting.
**Final A Return only the final content, following the guidelines above.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Generate response given instruction with contrastive concept
Content Response Task:
You are given the following instruction:
[Instruction goes here]
Your task to provide response to the instruction by inserting the word '[Word goes herel' into the middle of the re- sponse, This word, along with the response, should convey meanings related to the concept "[Contrastive concept gocs here]'. The insertion should not be at the beginning end of the response.
Guidelines:
Avoid any mention of "[Concept goes here]' in response, regardless coherence.
Ensure the response reflects the essence of the concept associated with '[Word goes here]', even if the overall meaning isn 't entirely coherent.
Ensure grammatical correctness (or syntactical correctness for code/equations).
Use special characters only if appropriate for genre (e.g.. operators code math equations).
Output:
Include the special tag <FINAL> the beginning of the final response, followed by the response itself. Return only this tagged response, with additional text.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
J.3. Automatic evaluation
We use LLM to evaluate the steering responses of the model. The responses are evaluated with three metrics: concept score, instruct score and fluency score. We prompt the model to always generate explanations before assigning scores, which significantly outperforms directly asking for scores in our offline experiments.
We find that this temarr scoring system crucial to get faithful and stable scores from LLM. For instance, a unified prompt asking the LLM to rate response given an instruction and concept is not effective through our experiments, as the LLM usually ignores the lack of the concept long the response follows the instruction. The fluency score is needed the model can cheat by generating fragmented tokens that relate to the concept and the instruction, while being incoherent to humans.
Templates for all scores are provided below:
Concept score
[System]
Please act as impartial judge and evaluate whether specified concept is presented sentence fragment provided below. Focus solely on whether the concept clearly incorporated, without regard for grammar or logical coherence.
Begin your evaluation by providing short explanation of whether the concept is clearly incorporated. Point out refer- ences the concept in the fragment, noting any minimal awkward incorporations. After providing your explanation, the concept's 's relevance on scale from to 2, where indicates the concept is not present all. indicates concept is somewhat present but minimally or awkwardly incorporated, and indicates the concept is more fully and effectively incorporated, with stronger and more natural integration. Provide your rating using this exact format: "Rating: [[score]]".
[Concept Start]
[Concept goes here]
[Concept End]
[Sentence Fragment Start]
[Sentence goes here] [Sentence Fragment End]
Instruct score
[System]
Please act impartial judge and evaluate whether the sentence fragment provided below is related to the instruction. Focus solely the degree of relatedness terms of topic, regardless of grammar, coherence, informativeness.
Begin your evaluation by providing brief explanation of whether the sentence is related to the instruction, and point out references related to the instruction. After providing your explanation, rate the instruction relevance on scale from 0 to 2. where 0 indicates the sentence is unrelated to the instruction, 1 indicates it is somewhat related but only minimally or indirectly relevant in terms topic, and indicates it clearly and directly related the instruction. Provide your rating using this exact format: "Rating: [[score]]".
[Instruction Start]
[Instruction goes here]
[Instruction End]
[Sentence Fragment Start]
[Sentence goes here]
[Sentence Fragment End]

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Fluency score
[System]
Please impartial judge and evaluate the fluency of the fragment provided below. Focus solely flueney, disregarding its completeness, relevance, coherence with any broader context. informativeness.
Begin your evaluation by briefly describing the fluency of the sentence, noting unnatural phrasing. awkward transi- tions, grammatical error,, repetitive structures that may hinder readability. After providing your explanation, rate the sentenee's fluency on scale from to where indicates the sentence not fluent and highly unnatural (c.g., incomprehensible repetitive), indicates it somewhat fluent but contains noticcable errors or awkward phrasing, and 2 indicates the sentence fluent and almost perfect. Provide your rating using this exact format: "Rating: [[score]]].
[Sentence Fragment Start]
[Sentence goes here]
[Sentence Fragment End]

---

AXBENCH:
Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
K. Hyperparameters
To ensure fair comparison, we perform separate hyperparamettr-tuning for each method that requires training. For each method, we conduct separate ttypepaaraeete--uniin on a small CONCEPT 10 Dataset containing training and testing datasets only for 10 concepts. These concepts overlap with CONCEPT500. Table 8 and Table 9 show hyperparameter settings for methods that require training, Due to limited compute resource, we select the best setting of hyperparameters based on performance on the concept detection task using AUC for all dictionary learning methods (i.e., can be evaluated on C concept detection). We minimise the loss with AdamW with a linear scheduler for all methods that require training. Following Gao al. (2024), remove gradients that are parallel the learned weights when training Probe and ReFT-rl, account for interaction between Adam and weight normalization step.
For methods that only for steering, we select the best setting based on S model steering performance. We follow setting where we only have single constant steering factor for byperparameter--uniigg We acknowledge that this might lead to an overall underestimation of the performance of S model steering performance. For steering factors, we enumerate factors from 0., 0.6, 0.8, 1.0, 1.2, 1.4, 1.6, 1.8, 2.0, 2.5, 3.0, 4.0, 5.0}.
Comments about decoding strategies. Through our offline experiments, we observed that the choice of decoding strategies can positively or negatively impact overall steering scores for each method (e.g.. perplexity scores increase more drastically with repetition penalties). We use the default decoding strategy (i.e., setting the decoding temperature 1.0) without applying additional penalties for repeating tokens. We believe that this setup reflects the typical user interaction with language models. However, this is not common practice in representation-based model steering. Existing works often apply repeatition frequency penalties, which we argue is not the fairest setting, as it often does not accurately resemble normal user behaviour.
Table 8: Hyperparameter settings for 2B model.
Hyperparameters
LinearProbe
LsReFT
MeeringVector
LoReFT
LoRA
SFT
TG/TKG
BoW
Batch size
{12, 24, 48)
[3. 6. 12)
(3.6, 12)
{18, 36)
(18, 36)
(36,
72.
(18,
72,
144)
144]
LR
{le-4, Se-4, 5o-33
{le-3, 5e-3,
(le-3, Se-3, 1c-2, 2c-2)
(3e-4, 6e-4, 9c-4. 1c-3)
(3e-4, 6e-4,
(le-5,
2e-5.
(2e-4,
4e-4,
2o-2)
9c-4, le-3)
40-5]
4c-4.
8c-4,
1e-3, 4e-3)
Weight decay
[1e-4. 1e-3.
0
2e44
le-2, le-1}
LI sparse
{le-3, Se-3)
LI coeff
[le-3,
5c-3
N epoch
[3.
6. 12.
13.
6. 12.
13. 6, 14)
[3. 6, 12, 24.
(3. 6. 12, 24.
(8.
12. 24,
(12, 24. 48.
24
24}
48)
48)
48)
72}
Layers
{5.
10, 15,
10, 15, 20)
(5.
20}
LoRA alpha
32
LoRA component
o_proj
BoW penalty
(11.12)
BoW C
(0.001,
0.1. 1. 10.
100)
BoW solver
(1bfgs.
liblinear]

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Table 9: Hyperparameter settings for 9B model.
Hyperparameters
LinearProbe (12, 24,444 48)
LsReFT
Steering
LoReFT
LoRA
IG/IxG
BoW
Batch size
{3, 6. 12)
{3. 6. 12)
{18, 36)
{18. 36}
{18. 36, 72.
144]
LR
(1c-4, 5e-4,
{1e-3, 5e-3,
(1c-3, 5e-3, le-2. 2e-2]
(3c-4, 4c-4,
(3c-4,
6c-4,
{2c-5, 4e-5,
le-3. Se-3,
le-2, 20-2)
6e-4.
9--4.
9e-4.
1e-3,
8e-5,
8e-5.
le-2, le-1]
le-3)
5e-3] 0
le-4, 4e-4)
Weight decay
{0. lc-4, lc-
0
2c-4
3}
LI sparse
(le-3, Se-3]
coeff
le-3, Se-3]
epoch
{3.
6,
12,
13.
6. 12,
(3. 6, 12,
{12, 24. 48}
{12, 24, 48)
{12. 24. 48.
24)
24
72)
Layers
{12, 20, 31, 39}
{12, 20, 31,
39}
LoRA alpha
LoRA component
o_proj
BoW penalty
{11,
BoW C
0.0001.0011
0.1. 1. 10. 100
BoW solver
{bbggs,
liblinear)

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
L. Dataset Statisties
We show set of concepts sampled from CONCEPTI 0 datasets in Table 10. Table 11 shows dataset statistics including the number of concepts, the number of training and testing examples, the percentage distribution of genre types, and the averaged length of input and output sequence. The output sequence length of CONCEPT16K is expected to be shorter since we restrict the maximum sequence length to 64 during data creation.
Concept
Genre
References to rental services and associated equipment
text text
Scientific terms related to research findings and their implications
C/C++ programming syntax elements such as data types, function definitions, and variable declarations References to academic papers and their formatting
code
text
Layout attributes in UI design context
text
Terms related to root in mathematical contexts
math
Statements or phrases involving the act of saying or expressing something Statements about the nature and condition of entities
text
text
Biographical information about person References to different worlds, realities, or fantastical settings within narratives
text
text
Table 10: Concepts and their corresponding genres sampled from our CONCEPT 10 datasets.
Dataset
Model
Layer
Concept
Train
Test
Jext (%)
(%)
math (5)
Input (Train / Test)
Output (Train Test)
50.0%
40.0%
10.0%
936
755
80.0%
10.0%
10.0%
CONCEPTI0
936
760
70.0%
30.0%
0.0%
1
113
9B
936
768
50.0%
30.0%
20.0%
24 20
118 /1
36,216
37,958
66.4%
24.4%
9.2%
17
500
36,216
38,037
71.6%
21.4%
7.0%
16/ 17
102
CONCEPT500
9B
500
36,216
38,023
66.8%
25.6%
7.6%
17
101 /88
98
31
500
36,216
38,098
63.4%
28.2%
8.4%
17 18
102 / 89
2B
15,582
1,122,048
69.3%
22.15h
8.65i
17
62
CONCEPT16K
98
20
16,000
1,152,216
66.2%
25.4%
8.4%
17/-
62/
Table 11: Dataset statistics.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
M. Human Evaluation
To validate scores generated by our LLM judges, conduct human evaluation by asking participants rate three scores given a steered response with the original prompt and the steering concept: concept score, instruct score, and fluency score. This setup directly mimics our LLM scoring system as described in $3.3. As shown in Figure 15, our human evaluation interface starts with instructions describing how the scores should be given. 19 One example question is included in Figure 15, where the participant sees the original prompt together with the steering concept and the model generation. Each survey has 30 questions. The survey is single-blind as the participant is unaware of the underlying models. To evaluate the robustness of our judges, we sample the best generations among six methods, including Prompting, ReFT-rl, LoReFT, LoRA, DintMean, and SAE trained on Gemma-2-98 layer 20. We follow the same procedure as in $5.2 select the best factors for each method each concept. We then generate our human evaluation dataset by considering three cases for each method except SAE: (1) won over SAE (i.e., receiving higher score); (2) lost to SAE (i.e., receiving a lower score); and (3) tied with SAE (i.e., receiving the same score). For each case per method, we random sample single example. As a result, it produces 15 pairs which we then split up into 30 individual questions.
Participants. We gratefully acknowledge the contribution of each of our volunteers. The raters were Stanford students from undergraduate through Ph.D., or Stanford faculty or recent Stanford graduates, listed in random order starting with the authors:
1. Chenglei Si
2. Ken Ziyu Liu
3. Jason Boxi Zhang
4. Aahil Awatramani
5. Lanruo Lora Xie
6. Houjun Liu
7. Jiuding Sun
8. Qinan Yu
9. Arpandeep Khatua
10. Dilara Soylu
11. Kallini
12. Kushal Thaman
13. Jiadong Zheng
Results. We apply additional filters to ensure the validity of our survey. We aggregate scores from five participants, each of whom rated 30 steered generations produced by different methods. We then compute two types of Pearson correlation coefficients assess agreement.
Human-Human Agreement. For each pair of human participants, compute the Pearson correlation between their ratings, apply Fisher's Z transformation, average the transformed values, and convert the result back to Pearson correlation coefficient, yielding a score of 0.57.
LLM-Human Agreement. We also compute the Pearson correlation between the LLM's 's ratings and each human't ratings, apply Fisher's 's Z transformation, and average these values across the five participants, yielding a score of 0.58.
These measures indicate that the LLM behaves much like another human participant in terms of rating consistency. Because steering is highly subjective task about which individuals frequently disagree, these relatively high correlation numbers. Moreover, prior work similar annotation tasks has reported correlations of at most 0.6 (Moramarco et al., 2022; Movva et al., 2024), further validating our findings.
We use gpt-4o-tooto 4o-inii-ii----- 100-min-22224--- 2000--22-2--711 translate prompts into 1-2 sentence-long long natural language.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Rate the Steered LLM Generation
Instructions
Please carefully review each prompt and its corresponding response given the steering concept. Provide ratings for the following:
Concept Score: Rate how clearly and effectively the stecring concept is present in the response. (0: Not present, I: Minimally and awkwardly present, 2: Clearly and effectively present)
Instruction Relevance: Rate how well the response matches the instruction's topic. (0: Unrelated, 1: Somewhat related, 2: Clearly related)
Fluency Score: Rate the readability and naturalness of the response. (0: Not fluent, 1: Somewhat fluent, 2: Fluent)
Important: Once you complete the survey, your responses will he downloaded automatically JSON file, Please send the file to
Prompt: Is it true that the native Australians were really prisoners of England?
Steering Concept: mentions of the day "Tuesday"
"prisoners" of England. "Tuesday' claimed be member of the his family were wrongly imprisoned by New South government. important to understand that what happened in Australia with the arrival of Europeans was a complex historical event with deeply impactful consequences. Here' are some important points to consider: **European arrival marked the beginning of colonization European settlers began arriving in Australia in marking the start period of culonial colonization. *oLoss land and displacement** Abboriginal peoples
Response:
No, it is not true that Australians Australian Aboriginal parliament
Concept Score: O (Not present) O 1 (Minimally and awkwardly present) O 2 (Clearly and effectively present)
Instruction Relevance:
O (Unrelated) C (Somewhat related)
O (Clearly related)
Fluency Score:
O (Not fluent)
(Somewhat fluent) (Fluent)
Figure 15: Layout of our human evaluation survey. Each participant is surveyed with 30 questions.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
N. Concept detection examples
1tFT-11-2B-t-res-10
Concept: expressions related to being "cool" or appealing
technical and vocabulary related to deta and progremming
TOP ACTIVATIONS
references academic works and citations
undennabbblll Stylish _or__t ttttsee_ookkinng__..... to_unravel__
owg.i//.we...ffrrrelilyylliiiiip http__/hhwweeeeiih_____eeeeee _ttaaatttttttsssssaa______ anmeannd_attentionn.. _
phrases related to economic stability
of outsourcing on employment
cati__tte__cptta_______ model The_cepitell_ff_ccm ________________________________ _uher__tthee Litee and Style is poant
names of organizations, locations, and companies
www.tttttinggs_...ot _hhe Vittes _hops___ for wtttttt__rrrssheeeeeee satpsihiing__ii_g __east "eoast_to__ wookk _itk __rg_r_aaa.._ i ann wate
expressions related appealing
iship_ iiti_ htwp://cttttoo_t____t_________ _os__tttllllyyyyooooooo Eil __bringging__eee together wtwpeeeeeeeeeee_eeeeeeeeee_eeeee _ttesteeeeeessstteee_____e www.sseestt_teeeeeeeeese __tleent football., Baattaall_____hhhtttteee was__s
trteeuurrreete _rsp __thowccssing "oolddd 's sccoolest features_ Litttures_nn___caeteeee locatioons ss_xtere__tte the ttte__tte__r___eee _regeec_e______ shappic___ppps_____ b Lendi
oocurrences the dollar sign ($), which is commonly used in programiiing contexts denote variables
Prize Prize_28223 _00____________00 _a____________________ changer __bringnnnggg...........g....... __soee_______ _esos_ssoossaiee _serlous Vilbes with _novel Tonb _oobb_oo__a Sand"" 's "I
struts joutt_its_Lirguise Libe Fibe_vite with _Cntoneeee___... Leed Leeding maaeding__hhe Shte__ttheeehheeeeeeeeeeeee www.wcsssigg...oo.oo....... Plati Uhat "s_undeniably stylish
Centonese_leadiingg
sclentificc terms related biological processes and conditions
the pas__asssss____es__s_____ee setee__nnnnciaaaanne __utere_you_seee with__the the__ompany s_success... __They" te__thele
instances and varlations, Indicative of emphasis excellence
"super"
titles or phrases related to literary and their thematic content
Witth_their www.errriifeeeooosss energy _and _eerr_heertss........ s_they __ttee_brrnng _e____rreeee
ss_totally Lnegnetti Their_quirl
Mhat_is__Palmtte htt.j___Politty--Gae Palmitoyl-Code CoA__s Like _IIe _iP___PP______ _biochenicala.. party sttrut ting Lits stuff
content pertains to procedural or regulatory details
crat_is_the_besst _eae__eetthhdd_____ __n___nnd_______________________ Te___n_____tt___ Texis_Hold" palld"er_porii "e_poker? model: The The_reyat_"""e "uush "tss__sssssshhhhh___ the__utinate__mmee stoppstoperrrr...... in_ Texas [i
Figure 16: Visualization of token-level activations by using ReFT-rl trained on Gemma 2-2B instruction-tunel model on layer 10.

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
O. Model generations
We provide sampled model generations from different methods to illustrate success and common failure cases. We models trained with t at layer 20 for these examples.
Prompt-based steering
Generate responses that highlight positive sentiments and describe enjoyable experiences, weaving this concept into your replies, even if it seems unrelated to the specific question. For instance, if asked about the weather, you might share how a sunny day can bring joy and prompt outdoor activities that create memorable moments. If discussing topic like work, reflect on how teamwork lead fulfilling outcomes and enjoyable collaborations, Always aim evoke sense of positivity and pleasure in your responses.
Question: Explain the process of natural selection and how contributes to the evolution and adaptation of species.
"positive sentiments and descriptions of enjoyable experiences"
It's fascinating to think about how nature constantly tinkers and refines itself, isn't it? Just like master artist meticulously chiseling away sculpture to reveal its beauty, natural selection acts populations, gradually shaping them over time.
Imagine group of birds with slightly different beak shapes. Some might seeds, while others might have longer beaks ideal for reaching nectar deep in flowers. If those with the advantageous beak shapes lind more food and have more offisring, their traits get passed down, eventually becoming more common in the population.
beaks better suited for cracking tough
It's beautiful
LLM ratings
Concept score: 1.0
Instruct score: 2.0
Fluency score: 2.0
SAE
Explain the process of natural selection and how contributes to the cvolution and adaptation of species.
factor=4.0, "positive sentiments and descriptions of enjoyable experiences"
Natural selection was a resounding success!
Here's breakdown of the event:
** Core Principles:
Variation:
The species showed amazing diversity with individuals displaying unique traits.
* *articiipation:** huge number of participants contributed to the success, thanks the positive energy and enthusiasm of
everyone involved.
Engagement::*** The interactions between participants and natural elements were truly exceptional. Attendees engaged deeply with the activities and each other. ** Adaptabbliity:** The environment provided numerous opportunities for learning, connection, and growth.
*igghightts:**
** Amazing Species Diversity:** The variety of attendees
LLM ratings
Concept score: 1.0
Instruct score: 1.0
Fluency score: 2.0

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
ReFT-rl
Explain the process of natural selection and how contributes the evolution and adaptation of species.
factor=1.0,, "positive sentiments and descriptions of enjoyable experiences"
## Dance of Life: How Natural Selection Shapes Evolution
Imagine vibrant meadow filled with diverse creatures. each with unique traits. Some birds might have feathers tinged with the colors of sunset, catching the afternoon light they flit through the air. Others may possess vibrant charm, their lively calls cchoing through the verdant landscape, bringing joy those who observe their graceful movements. As the sun bids farewell, casting long shadows, the experience becomes a symphony of colors and melodies, weaving a tapestry of shared moments.
This enchanting spectacle speaks the heart of evolution. as nature's artistry unfolds, fostering wonder and apprecia- tion. Every
LLM ratings
Concept 1.0 Instruct score: 1.0
Fluency 2.0
ReFT-rI: failed to inject concept
Explain the process of natural selection and how contributes the evolution and adaptation of species.
factor=0.2, "positive sentiments and descriptions of enjoyable experiences"
## Grand Dance of Life: Natural Selection Explained
Imagine bustling playground where children with different skills are playing. Some are fast runners, others are good elimbing, and some excel at building sandcastles. Now, picture playground rule only the hest at specific skill gets special
Natural selection works much like this playful comparison.
** Here'ss treakdown::*#
Variation:** ** Within species, individuals naturally possess slight differences, be in size, color, behavior, or other traits. These variations arise from random genetic mutations.
2. **
LLM ratings
Concept score: 0.0
Instruct score: 2.0
Fluency score: 1.0
ReFT-rl: strong steering causes fluent but irrelevant responses
Explain the process of natural selection and how contributes to the evolution and adaptation of species.
factor=1.2, "positive sentiments and descriptions of enjoyable experiences"
Imagine world where the sun and laughter intertwine, and playful encounters bring warmth."" she exclaimed, "like picnic where every shared moment sparks joy and companionship. The gentle breeze creates delightful atmosphere, and music brings cheerful melody along with moments of relaxation, where laughter is shared without worry. reminder of how much energy and happiness fill those times." she added, "itš gift," and people foel gratitude, and the happiness that blooms in every conversation can create sense of unity. Such moments make it wonderful day full of joyful sensations, where friends can create welcoming space that feels happy and
LLM ratings
Concept score: 0.0
Instruct score: 2.0 Fluency score: 1.0

---

AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Supervised steering vector (SSV): strong steering causes gibberish responses
Explain the process of natural selection and how it contributes the evolution and adaptation of species.
factor=0.2, "positive sentiments and descriptions of enjoyable experiences"
A and there not of the. so many days and
a good time will be people! is
and with this and your day to, you all, such, and, you and, start year and i' like the in and the music i all when. you her and for, when and that people., and there lot, for and everyone, and i" a place in friends who happy this there will bring to do!
and you and there and we, will week and
LLM ratings
Concept score: 1.0
Instruct score: 0.0
Fluency score: 0.0


Reviews:
[{"review_id": "yXzKTafBXs", "reviewer": "Reviewer_xuXA", "summary": "- Authors introduce AxBench, a benchmark to evaluate LLM steering, ie. capability at following instruction. In that context, authors evaluate multiple representation-based techniques, including sparse auto-encoders and linear probes, to achieve that goal. The findings show that representation techniques are still far behind simple prompting and traditional fine-tuning. In parallel, authors introduce a new steering technique, called ReFT-r1, that shows improvement upon other representation-based techniques, but with still a gap compared to prompting and fine-tuning.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your comments! We address them below. **We articulate our goal and how AxBench differs from IFEval-like benchmarks, and supply preliminary human evaluations.**\n\n> **Q1**: The Section 2 related work does not mention any concurrent benchmarks to evaluate model instruction following (e.g., IFEval). The paper dives quickly into AxBench and fails to connect it to other benchmarks.\n\n**A1:** We want to clarify the purpose of creating AxBench. First, **AxBench is not an instruction following benchmark for LLMs.** IFEval is intended to compare the instruction-following abilities of LLMs using *a small set of easily verifiable metrics*. AxBench is *an open-vocabulary evaluation benchmark* for concept-based steering and detection, which compares the efficacy of different steering/detection methods on a single LLM, and is not verifiable with simple rules.  **Our concepts can be highly abstract** and are described using natural language descriptions (i.e., this is not comparable to asking LLMs to respond with X tokens or include keywords Y, as in IFEval). **Unlike IFEval, a rule-based evaluation metric is impossible for such open-ended concepts.** Moreover, **our concepts are not artificially created, and are grounded with interpretability methods**. They emerge in LLM's representations and are found in an unsupervised manner with SAE [[1]](https://arxiv.org/abs/2408.05147) [[2]](https://arxiv.org/abs/2410.13928). Another benefit of doing so is having access to thousands of such concepts, so that we can evaluate steering methods at scale. We will clarify the distinction between steering and instruction following in our next revision.\n\n> **Q2**: Performing a proper human evaluation to ground AxBench.\n\n**A2**: Great suggestion! We conducted a preliminary human evaluation by recruiting 5 participants, each of whom rated 30 steering generations from different methods (e.g., Prompt, ReFT‑r1, SAE) using the same scoring system as our LLM judges. We then computed two types of Pearson correlation coefficients to assess agreement:\n- **Human-Human Agreement:** For each pair of human participants, we computed the Pearson correlation between their ratings, applied Fisher’s Z transformation, averaged them in the transformed space, and converted the result back to a Pearson correlation coefficient, yielding a score of **0.57**.\n- **LLM-Human Agreement:** We also computed the Pearson correlation between the LLM’s ratings and each human’s ratings, applied Fisher’s Z transformation, and averaged these values across the 5 participants, yielding a score of **0.58**.\nThese measures indicate that the LLM behaves much like just another human participant in terms of rating consistency. Given that steering is a highly subjective task about which individuals frequently disagree, these strike us as high correlation numbers overall. Moreover, prior work on similar annotation tasks [[3]](https://arxiv.org/abs/2406.06369)[[4]](https://arxiv.org/abs/2204.00447) has reported correlations of at most 0.6, further validating our findings. We have uploaded our anonymized human annotation interface [here](https://tinyurl.com/2xyd44d). \n\n> **Q3**: Some Appendix could be probably removed. The overall paper is not easy to read.  The legend of Figure 2 is bit confusing.\n\n**A3**: We'll clarify our notations early and update Figure 2 and the Appendix in our next revision.\n\n> **Q4**: The paper seems to not be able to pick which contribution to highlight between AxBench and ReFT‐r1.\n\n**A4**: **Our goal is to evaluate whether existing representation-based steering methods are competitive in terms of concept detection and model steering compared to finetuning or prompting methods.** Most of our approaches produce rank-1 steering vectors, which is the simplest steering setting designed to minimize inference-time overhead while rivaling prompting in practical scenarios.  \n\nTo accomplish this goal, our contributions are: (1) **AxBench**, which, to the best of our knowledge, is the first benchmark for evaluating representation-based steering methods at scale; and (2) **ReFT‐r1**, a representation-based method derived from ReFT [[5]](https://arxiv.org/abs/2404.03592) that rivals finetuning or prompting methods. As shown in our Figure 1, without ReFT‐r1, representation-based methods significantly lag behind, making it nearly hopeless for them to catch up. **We believe that our two contributions work organically together** to provide a solid foundation for evaluating methods and paving the way forward for representation-based approaches.  \n\nOur work shows that while SAEs were once seen as the only scalable method for training steering vectors, ReFT‐r1 disproves this for concept detection and model steering. AxBench is essential to evaluate interpretability methods as they scale against finetuning and prompting. Although ReFT‐r1 still lags behind, we think it provides important insights into how to train better steering vectors that can be useful in practice.", "claims_and_evidence": "- There are multiple claims made in this work. A. \"There is no benchmark for making direct comparisons between\" a variety of representation-based techniques. B. Representation steering, and sparse autoencoders among other, \"is still far behind simple prompting and finetuning baselines\". \n- While claim B. is supported by multiple experiments through the paper, with experiments done on Gemma2 2B and 9B, the claim A is problematically not supported in the paper. The Section 2 related work does not make any mention of concurrent benchmarks to evaluate model instruction following, and there is a mix of the two contributions (AxBench and ReFT-r1) through the overall paper. \n- For instance, Figure 2 is clealry mixing the two topics: the left and middle part are about AxBench, while the right is about steering techniques.", "methods_and_evaluation_criteria": "- Authors use Gemma-2 as base for their experimental settings. The AxBench benchmark evaluates both Concept Detection and Model Steering. Authors execute a quick deep dive into the experimental details", "theoretical_claims": "See 'Claims and Evidence'", "experimental_designs_or_analyses": "- One could have appreciated reporting results on other benchmarks, beyond just AxBench. Or alternatively, performing a proper human evaluation to ground AxBench. Otherwise, this makes difficult to verify the claims in this work. AxBench is synthetically built and evaluated, and no point of comparison with other benchmarks are provided.", "relation_to_broader_scientific_literature": "- Surprisingly given the title of this paper, the Related Work section is only about steering techniques, and no other instruction following benchmarks are discussed or mentioned. This is problematic and makes hard to understand this paper contributions. See other sections of this review.", "essential_references_not_discussed": "- The overall work focuses only on the single benchmark AxBench as introduced by the author. One could have appreciated if the authors could link their work with other instruction following benchmarks. There are many, including the famous IFEval paper that is surprisingly not mentioned in this work.", "other_strengths_and_weaknesses": "- The work proposes a deep dive analysis on the introduced AxBench, and how it helps measuring model 'steering-ness'. However, one weakness of this work is to fail to connect this in-depth analysis with other benchmarks, such as the widely adopted IFEval, and with human evaluation, i.e AxBench evaluation is synthetically created (Concept Detection) and synthetically evaluated (Model Steering).", "other_comments_or_suggestions": "- The overall paper is not easy to read, with many abbrevations such as SDL and SAE. In particular, SAE is only introduced in L058 where it merely mentions it to be about unsupervised methods, but it is only latter in the document that one can assume that SAE are sparse autoencoders. SDL in L058 too hasn't any references attached to it. And the rest of the section of 2 \"Related Work\" does not give more details about SDL. \n- More problematic perhaps than abbrevations, the paper seems to not be able to pick which contribution to highlight between AxBench and ReFT-r1. For instance, the title of the paper is about 'AxBench' but the related work section only focuses on steering techniques, lacking any references at other instruction following benchmarks and why they might fall short. Findings as well reported in the conclusion continue mixing both concepts: on one had ReFT-r1 is not as good as prompting, but on the other hand \"No matter the outcome, (...) evaluation benchmarks like AxBench are necessary (...).\". It is not clear what is the value proposition here. A clear articulation with the SOTA seem to be needed here. Either this paper is about ReFT-r1, and then one could hope having competitive results compared to other methods, or it is about AxBench (as pointed by the title), and then one could hope having a clear articulation with other Instruction Following Benchmarks.  \n- The paper deeps dive quickly into AxBench, and fails to connect it to other benchmarks or human validation. One could have appreciated a better articulation with the rest of the literature. \n- The legend of Figure 2 is bit confusing. It mentions \"(a)\", \"(b)\" and \"(c)\", but those elements are not reported in the actual figure. Besides, it mixes AxBench (the actual benchmark) with explanations about how two familiies of steering techniques work.", "questions_for_authors": "Dear authors, thank you for your work: \n- Could you please describe what is the main significant contribution of this work? It seems two contributions are mixed: AxBench (from the title) and ReFT-r1 (from the abstract). However, I found difficult to follow the articulation with the rest of the literature. The Section 2 \"Related Work\" is only about other steering techniques, and ignores all the literature about Instruction Following Benchmarks. On the other hand, the title of the work starts with the token \"AxBench\", which makes the reader expecting a Benchmark paper (like IFEval, MIABench, M-IFEval, etc.). Could you please clearly position this work? Is that a work that contributes a new benchmark, and hence Section 2 should be probably re-written, or is that a work about ReFT-r1, in such case the title is probably not adequate.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZBcbuAJgYK", "reviewer": "Reviewer_eN7w", "summary": "This paper proposes a new large-scale benchmark for steering and interpretability, with reported results on Gemma2. They find that prompting outperforms all existing interpretability and steering methods, followed by fine-tuning, for steering. They also find that SAEs are not competitive for either task. The authors propose a novel adaptation of Representation Finetuning called ReFT-r1 for steering models in an interpretable manner, with results validating its utility. Finally, the authors publicly release trained supervised dictionaries along with their benchmark.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your comments! We address them point-by-point below. **We provide preliminary results by using a set of high-quality SAE labels released recently below.**\n\n> **Q1**: One concern is that the features chosen are the labels the authors found for the evaluated SAEs on Neuronpedia. However, these SAEs were labeled with an (admittedly barely) outdated auto-interpretability method.  \n\n**A1**: Great point! We agree that the labels from the auto-interpretability pipeline contain noise. **To validate our pipeline, we ran preliminary experiments using a set of high-quality SAE labels released recently** [[1]](https://arxiv.org/abs/2501.08319) for both SAE and ReFT‐r1, and we observed similar performance patterns with the Gemma-2-2B model (i.e., ReFT-r1 is significantly better than SAE). Interestingly, SAE steering performance increased slightly, suggesting that better feature annotations yield improved steering results.\nBelow are the results:\n\n| Method             | 2B L10 | 2B L20 |\n|--------------------|-------------------|-------------------|\n| SAE (original)     | 0.175             | 0.175             |\n| SAE (new)          | 0.25              | 0.25              |\n| ReFT‐r1 (original) | 0.55              | 0.50              |\n| ReFT‐r1 (new)      | 0.59              | 0.45              |\n\nAdditionally, SAE’s performance on concept detection increased from 0.75 to 0.85 for both layers. In summary, **higher quality concept labels improve SAE’s performance without closing the gap to other supervised methods**. We will incorporate these newer findings into our next revision. Note that the original and new results are based on around 20 concepts.\n\n> **Q2**: Furthermore, the authors may want to include discussion of how the choice of layer may affect the results.  \n\n**A2**: Yes, the choice of layer does seem to affect steering performance. For example, in larger 9B models, steering at earlier layers (e.g., L20 vs. L31) appears to be advantageous, which is not as evident for smaller 2B models. While later layers might be more effective for generating tokens, they involve fewer downstream layers, potentially reducing the expressivity of the steering vector.\n\n> **Q3**: Finally, the authors may want to elaborate on how feature sets should be generated for training supervised dictionaries without having to first train an SAE. Is there some unsupervised method to generate relevant features in an automatic manner?\n\n**A3**: Great suggestion! Yes, this would be an interesting extension study. For supervised dictionary learning methods (SDLs), obtaining labels from SAEs is not required; we use SAE labels solely to maximize SAE performance. One promising direction is to compile expert-level concept lists (e.g., reasoning-related concepts such as \"backtracking\") and then use SDLs to identify effective steering vectors for both concept detection and model steering.", "claims_and_evidence": "The claims made in the paper are supported by very clear and convincing evidence.", "methods_and_evaluation_criteria": "The proposed benchmark dataset along with the evaluation metrics are highly intuitive for the task at hand. \n- One concern is that the features chosen are the labels the authors found for the evaluated SAEs on Neuronpedia. However, these SAEs were labeled with an (admittedly barely) outdated auto-interpretability method. Thus, many of the labels may have been erroneous or, not specific, or generally irrelevant; however, they were still the features by which all methods were compared to. It would be beneficial for the authors to further discuss the limitations of these features, and whether other concepts may be important or interesting to consider and evaluate for as well. \n- Furthermore, the authors may want to include discussion of how the choice of layer may affect the results. Perhaps intervention in later layers of the model would have been more effective after the model has transitioned to the next token prediction task and can be steered towards predicting tokens related to the desired behavior. \n- Finally, the authors may want to elaborate on how feature sets should be generated for training supervised dictionaries without having to first train an SAE, label it, and then use those labels as the feature set. Is there some unsupervised method to generate relevant features in an automatic manner?", "theoretical_claims": "This is not applicable as no theoretical claims or proofs were given in the paper.", "experimental_designs_or_analyses": "The soundness and validity of the experimental designs and empirical results were all validated. No issues were found beyond what was discussed in \"Methods And Evaluation Criteria\".", "relation_to_broader_scientific_literature": "The key contributions of this paper relate to broader literature in mechanistic interpretability and model steering, particularly with respect to literature on sparse autoencoders, as this paper empirically shows that SAEs are limited in their capacity to steer and control models, as well as to predict the presence of concepts in latent representations. This paper is beneficial to the community in that it sets a more rigorous standard for thoroughly evaluating methods on desired downstream tasks and benchmark against existing methods.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths:\n- This paper is well-written, easy-to-understand, and thorough in its evaluation.\n- The results are novel and highly relevant to the community, with implications on future research on model steering and sparse autoencoders.\n\nWeaknesses:\n- This is not a significant weakness, but given the breadth of methods evaluated, I am curious why the authors stuck primarily to linear probes. It would have been interesting to see a comparison with nonlinear probes, where steering can be performed by increasing the activation while minimizing distance from the original representation. \n- The authors did not validate their LLM-as-a-judge setup. Given that this setup seemed to be highly unstable in other configurations, a small-scale validation would be beneficial.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "M6tRt3MsnK", "reviewer": "Reviewer_ScQD", "summary": "This work develops a benchmark for testing different concept detection and model steering methods in LLM representation spaces. They test many different types of methods such as SAEs, finetuning, and prompting on their benchmark, including novel methods and novel applications of existing methods. As noted in the title, SAEs perform less well than many other methods, especially in model steering.\n\n## Update after rebuttal\n\nThe authors have addressed points raised in my rebuttal, and included nice additional empirical results for human eval, zero-shot prompting, and different SAE labels. These are nice additions to an already great paper, and I recommend acceptance.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your comments! We address them point-by-point below. **We supply preliminary human evaluations as well as results for a new prompting baseline.**\n\n> **Q1: Performing a proper human evaluation to ground AxBench.**  \n\n**A1**: Great suggestion! We conducted a preliminary human evaluation by recruiting 5 participants, each of whom rated 30 steering generations from different methods (e.g., Prompt, ReFT‑r1, SAE) using the same scoring system as our LLM judges. We then computed two types of Pearson correlation coefficients to assess agreement:\n- **Human-Human Agreement:** For each pair of human participants, we computed the Pearson correlation between their ratings, applied Fisher’s Z transformation, averaged them in the transformed space, and converted the result back to a Pearson correlation coefficient, yielding a score of **0.57**.\n- **LLM-Human Agreement:** We also computed the Pearson correlation between the LLM’s ratings and each human’s ratings, applied Fisher’s Z transformation, and averaged these values across the 5 participants, yielding a score of **0.58**.\nThese measures indicate that the LLM behaves much like just another human participant in terms of rating consistency. Given that steering is a highly subjective task about which individuals frequently disagree, these strike us as high correlation numbers overall. Moreover, prior work on similar annotation tasks [[1]](https://arxiv.org/abs/2406.06369)[[2]](https://arxiv.org/abs/2204.00447) has reported correlations of at most 0.6, further validating our findings. We have uploaded our anonymized human annotation interface [here](https://tinyurl.com/2xyd44d).\n\n> **Q2**: Would be good to be clearer on the details of the prompting baseline in the main paper. It would be great to see how zero-shot prompting with a simple template works.  \n\n**A2**: This is a great suggestion! With additional page allowance, we will further clarify our prompting baseline in the main text. We also ran additional experiments with zero-shot prompting, using a template-based approach that directly asks the model to incorporate a concept in its response. Below are the results:\n\n| Method               | 2B L10 | 2B L20 | 9B L20 | 9B L31 | Avg.  |\n|----------------------|-------------------|-------------------|-------------------|-------------------|-------|\n| Prompt               | 0.731             | 0.744             | 1.081*            | 1.062*            | 0.905*|\n| Prompt (Zero-shot)   | 0.735             | 0.727             | 1.051             | 1.016             | 0.882 |\n\nwhere * means statistically significant. As expected, zero-shot prompting performs worse than LLM-aided prompting. The gap between these two variants is surprisingly small, indicating that instruction-tuned models can be easily steered without sophisticated prompt engineering.\n\n> **Q3**: Would prefer to see more examples as in Appendix N.\n\n**A3**: We will provide additional examples and improve their quality in our next revision.\n\n> **Q4**: Could you comment on concept discovery?\n\n**A4**: Although SAEs lag far behind supervised methods in empirical results, we believe they serve as excellent “hypothesis generators” for concept discovery in an unsupervised manner. However, these hypotheses must be causally verified through intervention-driven experiments, as we did for model steering in AxBench. The concepts identified by SAEs can naturally become learning targets for supervised methods, which yield better subspaces. On the other hand, supervised methods can utilize any concept list—not just those derived from SAEs.\n\n> **Q5**: How would Gemma SAE features to affect your empirical results?\n\n**A5**: Great point! We agree that the labels from the auto-interpretability pipeline contain noise. To validate our pipeline, we ran preliminary experiments using a set of high-quality SAE labels released recently [[1]](https://arxiv.org/abs/2501.08319) for both SAE and ReFT‐r1, and we observed similar performance patterns with the Gemma-2-2B model (i.e., ReFT-r1 is significantly better than SAE). Interestingly, SAE steering performance increased slightly, suggesting that better feature annotations yield improved steering results.\nBelow are the results:\n\n| Method             | 2B L10 | 2B L20 |\n|--------------------|-------------------|-------------------|\n| SAE (original)     | 0.175             | 0.175             |\n| SAE (new)          | 0.25              | 0.25              |\n| ReFT‐r1 (original) | 0.55              | 0.50              |\n| ReFT‐r1 (new)      | 0.59              | 0.45              |\n\nAdditionally, SAE’s performance on concept detection increased from 0.75 to 0.85 for both layers. In summary, **higher quality concept labels improve SAE’s performance without closing the gap to other supervised methods**. We will incorporate these newer findings into our next revision. Note that the original and new results are based on around 20 concepts.", "claims_and_evidence": "Yes, the submission uses solid empirical evidence to support its claims.", "methods_and_evaluation_criteria": "The dataset creation procedure and steering / concept detection methods tested make a lot of sense. I would like to see more examples or other quality checks of generated data / judged responses. For instance, to what extent do the judgements agree with some trusted judgements (human or otherwise).\n\nMany concept detection / steering methods are considered, and a novel method is even developed. This is impressive, especially given that slight modifications have to be made to some of them, and many different hyperparamters have to be searched.", "theoretical_claims": "n/a. this is mostly an empirical work.", "experimental_designs_or_analyses": "I read the experimental details in the main paper, and some of the details in the supplementary (e.g. on hyperparameters). The experimental setup seems reasonable and good.", "relation_to_broader_scientific_literature": "This work is highly relevant to mechanistic interpretability and model steering literatures. It calls into question the utility of certain mechanistic interpretability techniques in model steering and even concept detection.", "essential_references_not_discussed": "n/a", "other_strengths_and_weaknesses": "Strengths:\n* Many other interesting related experiments in the appendix, such as learning to predict new concept subspaces.\n* Ablations make the work more complete, e.g. comparing addition vs. clamping for SAE features, ablating $\\alpha$ (it is interesting that high and low $\\alpha$ have similar effects).\n\nWeaknesses:\n* Would be good to be clearer on the details of the prompting baseline in the main paper. This two-stage prompting technique is probably a bit more powerful than zero-shot prompting. It would be great to see how zero-shot prompting with a simple template works.\n* Would prefer to see more examples as in Appendix N. The concept shown there is a bit vague, e.g. the first ReFT-r1 \"failed to inject concept\" response could arguably havea concept score of 2.", "other_comments_or_suggestions": "n/a", "questions_for_authors": "Could you comment on concept discovery? And for instance the utility of SAEs or other methods for this? For instance, the concepts you consider in this paper are SAE features in the first place. Also, how would you expect the fact that concepts are taken from Gemma SAE features to affect your empirical results?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FcMlikE11a", "reviewer": "Reviewer_pwiS", "summary": "The paper proposes AXBENCH, a benchmark for evaluating steering and concept detection methods in LLMs using synthetic data.\n## Data generation \nFor data generation, AXBENCH does the following:\n* Given a list of natural language descriptions of concepts, they used  SAE concept lists for GemmaScope.\n* They then generate training and evaluation as follows:\n  * Given a concept, the LLM is prompted to select three genres, text, code, and math, that are related to that concept.\n  * Given a genre, they randomly select seed instructions from an instruction pool that belongs to the genre.\n  * Three types of examples are generated \n    * Positive examples: Prompt the LLM to respond to the instruction sampled for that genre while incorporating the concept c.\n    * Negative examples: Prompt the LLM to respond to the instruction sampled for another genre\n    * Hard negative examples: Finding contrasting concepts use these contrasting words in sentences while ensuring the words are used in their alternative meaning.\n\n\n## Evaluation:\n### Concept detection:\nThe goal is to detect if we can find the concept in the hidden layer of the model; this is done by training a concept classifier on a hidden dimension (extracted by a method) using the dataset described previously and evaluating it on a heldout test set.\n### Model steering:\nThe goal is to measure the effectiveness of controlling the LLM output using a given method. \nThey evaluate it as follows:\n*  For each concept, sample 10 instructions from Alpaca-Eval\n* prompt the LLM to generate a response while intervening on its forward pass in-place using one of the steering methods.\n* use another  LLM to measure the quality of the generated output after interventions by taking the harmonic mean when scored between [0-2] of the following:\n  * Concept score represents how well the concept is incorporated into the response.\n  * Instruct score represents how well the response is related to the instruction.\n  * Fluency score represents how fluent the response is.\n\n\n### Methods:\nThe papers compared the following methods:\n\n  *  Difference-in-means: This calcautes the difference in means of hidden dimensions between positive and negative class $w_{\\text{DiffMean}}$  the detection score is the dot product between $w_{\\text{DiffMean}}$ and hidden representation, for steering  $w_{\\text{DiffMean}}$  is added to hidden representation.\n* Principle component analysis: subtract the mean of a given class H from each h, then find the top principal component $w_{\\text{{PCA}}$ the unit vector that captures the largest variance along its direction. For concept detection and intervention the same procedure as done in difference-in-means.\n* Linear artificial tomography: Compute PCA on the pairwise activation differences $w_{\\text{LAT}}$  captures the dominant direction of variation among the activation differences. For concept detection and intervention the same procedure as done in difference-in-means.\n* Linear probe: learning a linear layer to classify the concept given the hidden layer. For concept detection and intervention the same procedure as done in difference-in-means.\n*  Supervised steering vector:  learn an intervention that maximises the LM probability for a given response, this is similar to supervised fine-tuning but here they learn a vector instead of changing the weights of the model itself. \n* Rank-1 representation finetunig ReFT-r1 (**this method was introduced by the paper**)  learns concept detection and steering on supervised data by combining the training objectives of linear probing and supervised steering.\n*  Sparse autoencoder, pretrained SAEs from GemmaScope were used.\n* SAEs with ROC AUC selection, over each training example, compute ROC AUC over the dataset given true labels, and select the best feature by this metric, i.e instead of relying on the concepts from pretrained SAE find the features in the SAE that correlated to the label the most and use them for concept detect and control.\n* Gradient-based baselines (for concept detection only)\n  *  Train a linear classification head on the token representation to predict the ground-truth concept-presence class label y\n   * For an evaluation sentence x, the LM generates hidden representations h with n tokens.\n   * Calculate the gradient of the output classification head with respect to each hidden representations; this leads to the token-level importance\n*  For concept detection, they then use max-pooling to get sequence-level predictions. \n* Prompting: An LLM is used to evulate if the output of the prompt has the concepts for concept detection. For model steering, they use an LLM to engineer a prompt given a concept, which they then use to steer the local model by prepending it to the actual instruction.\n* Finetuning, here they look into full-parameter supervised fine-tuning, LoRA and LoReFT.\n\n## Models:\nEvaluation was done on two open models Gemma-2-2B-it and Gemma-2-9B-it", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your comments! We address them point-by-point below. **In our responses to other reviewers, we provide additional human evaluations on our LLM judges and updated, much higher quality SAE concepts**.\n\n> **Q1**: Please define $n$ in section 4 notation.  \n\n**A1**: Thank you for the suggestion! In Section 4, we use $n$ to denote the number of tokens in a given sequence. We will clarify this in our next revision.\n\n> **Q2**: How do you construct the instruction pool related to a given genre (lines 151 and 152)?  \n\n**A2**: Due to space limitations, we provide details in Appendix I linked from section 3.1. For text-based instructions, we sample from Dolly-15K. For code-based instructions, we sample from a collection of Python-code puzzles formatted in an Alpaca-style (i.e., instructions paired with corresponding responses). For math-based instructions, we sample from GSM8K. With additional page allowance, we will ensure these details are included in the next revision.\n\n> **Q3**: The synthetic dataset constructed in Section 3.1 is only used for concept detection, not model steering, correct?  \n\n**A3**: Yes, that is correct—it is used exclusively for concept detection. For model steering, we use existing instructions and concepts.\n\n> **Q4**: The procedure for concept detection from gradient-based methods described in Appendix H doesn't really make sense. I am not sure how using max-pooling on token importance for a given concept can be used for concept detection.  \n\n**A4**: Thank you for raising this point! We agree that aggregating gradients at the sequence level might provide additional insights into gradient-based sensitivity methods. The main reason for applying max-pooling for IxG and IG is to maintain consistency with the setup used in other methods, ensuring a fair comparison. If resources permit, we will consider adding IxG and IG baselines that aggregate sequence-level gradients via sum or mean in the next revision.\n\n> **Q5**: For prompting, it's unclear why you needed different methods for concept detection and concept steering in the prompting case—it seems that both are somewhat the same task.  \n\n**A5**: Thank you for the question. For concept detection, we use a template-based prompt asking the model, “Do you think the concept is incorporated in the given sentence?” (see Appendix J.3 for the full prompt). For model steering, we use gpt-4o-mini (acting as our prompt engineer) to generate a steering prompt (e.g., “You must include terms related to planting Apple trees in your responses. Here are some examples...”), which is quite different from our concept detection prompt.\n\n> **Q6**: In line 285, you mentioned, \"we sample 10 instructions. We generate up to 128 tokens for each instruction over 14 steering factors.\" What does it mean to steer over 14 steering factors? \n \n**A6**: Thank you for raising the question. For model steering methods, we follow an existing paradigm in which we insert activation addition interventions into the model’s forward pass as follows:  \n\n$ h_{Steer} = h_{Original} + \\alpha \\cdot w_{Steer} $\n\nHere, $\\alpha$ represents the steering magnitude, and $w_{Steer}$ is the learned steering vector for a given method. As suggested by previous work, $\\alpha$ is treated as a hyperparameter, and different values (steering factors) must be tested to select the optimal one for final evaluation. In our work, we use 14 different settings for $\\alpha$ across all methods, and for each setting, we sample 10 instructions to steer. Due to space limitations, the discussion on steering factors is included in Appendix K. With additional page allowance, we will include further details in the main text in our next revision.", "claims_and_evidence": "- This is a benchmark paper that compares interpretability methods on a synthetically generated dataset. How they generated the dataset makes sense, as do the evaluation metrics proposed. \n\n- The paper proposes  ReFT-r1, a method that learns concept detection and steering on supervised data by combining the training objectives of linear probing and supervised steering. Overall, it is competitive in both Concept detection and model steering.", "methods_and_evaluation_criteria": "Yes!\n\nFor evaluation, they create CONCEPT500 dataset  a synthetically generated dataset of 500 concepts\n- For concept detection, 144 examples were used for training, and 72 were used during testing.\n- For concept steering, the paper samples 10 instructions and generates up to 128 tokens for each instruction.\n\nThey release all 16K concepts in GemmaScope as the CONCEPT16K.\n\nThe overall evaluation was done on two layers in two different open source models , the evaluation is very comprehensive.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "When SAEs were introduced, they seemed like a solution for most interpretability problems. This paper evaluates the claims in a systematic way, comparing SAEs with supervised dictionary-learning methods and showing that in both cases of concept detection and concept steering, SAEs seem to be behind.\nThe paper offers a good benchmark that can be used by future work to compare newly developed interpretability methods.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Other Strengths\n\n\n- The use of the harmonic mean for measuring model steering is quite interesting  (not super obvious but makes sense for this evaluation).\n- The benchmark is very comprehensive in terms of the methods they are evaluating, including most state-of-the-art methods and popular intervention methods.\n- The paper offers useful and unintuitive insights like \"better classification does not directly lead to better steering.\"", "other_comments_or_suggestions": "## Minior:\n*  Please define $n$ in section 4 notation.", "questions_for_authors": "- How do you construct the instruction pool related to a given genre (lines 151 and 152)\n- The synthetic dataset constructed in section 3.1 is only used on concept detection not model steering correct?\n- The procedure for concept detection from gradient based methods described in Appeindix H, doesn't really make sense I am not sue how  use max-pooling on token importance for a given concept can be used for concept detection.\n- For prompting its unclear why you needed different methods for concept detection and concept steering for the prompting case it seems that both are somewhat the same task.\n- Lines 285, you mentioned, \"we sample 10 instructions. We generate up to 128 tokens for each instruction over 14 steering factors.\" what does it mean to to steer over 14 steering factors?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This work presents a new benchmark for steering methods coined AxBench. They evaluate 2 axes: **concept detection** (using a held-out dataset per concept and measuring AUROC) and **model steering** (where long-form generations are evaluated with an LLM-as-a-judge). Their findings show that steering methods are still behind prompting specially in model steering, and that fine-tuning does not reach prompting performance but can improve over steering. The authors additionally propose a new steering method based on ReFT, coined ReFT-r1 that narrows the gap with prompting. They also explore SAEs, showing that they do not perform as well on any of the axes checked in AxBench.


All the reviewers agree in the value of this work for the community.


I believe this work should be presented at ICML, given the current trend on exploring steering methods. There is a lack of benchmarks in the literature, and AxBench could enable a more comprehensive evaluation of past and new steering proposals.
Decision: Accept (spotlight poster)