Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures

## Abstract

Abstract The hypothesis of Universality in interpretability suggests that different neural networks may converge to
implement similar algorithms on similar tasks. In this work, we investigate two mainstream architectures
for language modeling, namely Transformers and Mambas, to explore the extent of their mechanistic similarity.
We propose to use Sparse Autoencoders (SAEs) to isolate interpretable features from these models and show
that most features are similar in these two models. We also validate the correlation between feature similarity
and Universality . We then delve into the circuit-level analysis of Mamba models
and find that the induction circuits in Mamba are structurally analogous to those in Transformers. We also identify a nuanced difference we call Off-by-One motif : The information of one token is written into the
SSM state in its next position. Whilst interaction between tokens in Transformers does not exhibit such trend.

## 1 Introduction

The field of mechanistic interpretability seeks to reverse engineer real-world neural networks into interpretable features and circuits.
Due to the massive societal importance of Transformer language models (Vaswani et al., ; Brown et al., ; Ouyang et al., ; Sun et al., ), existing literature in this field mostly focuses on Transformers (Elhage et al., ; Olsson et al., ; Wang et al., ).
This poses a natural research question:

Will mechanistic findings in one model transfer to others?
The answer to this question may tell interpretability researchers how much of their current understanding will be of use in the future. The conjectural transferability of mechanistic understanding of a specific model is called the Universality Hypothesis (Olah et al., ; Chughtai et al., ).
Concretely, there are several aspects of Universality that may be of interest:
(1) Training: Will models with different training settings converge to similar representational structures?
(2) Model Size: Are features and circuits in smaller models present in larger models? If so, what do larger models build further on these structures?
(3) Model Architecture: How similar is the mechanism in models of different architectures? Does the universal part teach us more general motif of neural networks and language modeling? Does the divergence indicate the pros and cons of each architecture?
(4) Multilingualism / Cross-Modality: Are neural networks able to obtain similar knowledge from different languages or modalities?

This work mainly focuses on Architectural Universality. We believe the answer to this, if it exists, lies somewhere in the middle of the spectrum. It may not be the case that all models are isomorphic, nor that they implement totally distinct algorithms.
If the commonality of features and circuits across models of different architectures is revealed, we may be more confident to generalize some of our current mechanistic findings in pretrained Transformer language models to perception of the task of language modeling.
Pragmatically, such analysis also sheds light on whether one family of models fail to represent some certain type of features or implement some circuits present in another.
This may help provide a microscopic comparison among model architectures (Vaswani et al., ; Peng et al., ; Sun et al., ; Gu & Dao, ).

To this end, we mainly investigate two of the most popular types of models for language modeling, namely (decoder-only) Transformers and Mambas (Gu & Dao, ), a family of models based on State Space Models (SSMs).

We manage to bridge these two models with features extracted by Sparse Autoencoders (SAEs) in an unsupervised manner. Qualitative and quantitative experiments show that these two family models learn a lot of similar features. We also utilize our SAEs to investigate induction circuits (Olsson et al., ) in Mamba, which are also highly analogous to ones in Transformers.

However, the mechanism in these two models is nuanced by what we call Off-by-One Preference in Mamba. We find that Mamba often writes the information of a specific token into the SSM state in its next token. We show that this is done by the local convolution layer.

Our main contribution can be summarized as follows:

- •
To the best of our knowledge, this is the first work to look for linear, interpretable features with Sparse Autoencodersand quantitatively evaluate the similarity between Transformer and Mamba features (Section 4).
- •
We propose a novel metric to isolate and quantify feature universality with respect to architectural changes (Section 4 and 5).
- •
We perform circuit analysis in Mamba and qualitatively reveal its structural analogy and some nuances compared with Transformer circuits (Section 6).

## 2 Related Work

### 2.1 Representational Universality

The concept of Universality in mechanistic interpretability was introduced by Olah et al. () and has been extensively studied as representational universality (Klabunde et al., ). Raghu et al. () and Kornblith et al. () measured representation similarity using canonical correlation analysis, while other methods (Ding et al., ; Khrulkov & Oseledets, ; Boix-Adserà et al., ) quantify similarity across neural networks. Efforts also include examining neuron correlations in identical models with different seeds (Li et al., ; Wang et al., ; Gurnee et al., ), with recent work suggesting convergence across modalities (Huh et al., ).

### 2.2 Circuit Universality

Circuit-based mechanistic interpretability was pioneered by Cammarata et al. (); Olah et al. (); Elhage et al. (), investigating whether features and circuits are analogous across models. Olah et al. () showed early layer similarity in vision models, with further studies extending to MLPs and Transformers on algorithmic tasks (Chughtai et al., ).

### 2.3 Sparse Autoencoders

Neuron-based analysis is often hampered by polysemantic neurons (Elhage et al., ; ). SAEs (Bricken et al., ; Cunningham et al., ; Templeton et al., ) extract monosemantic sparse codes from activations, providing interpretable features. SAEs have been applied to language models (Rajamanoharan et al., ; Ge et al., ) and tasks like Othello move prediction (He et al., ).

### 2.4 Mamba Interpretability

Mamba (Gu & Dao, ; Dao & Gu, ), a family of models based on State Space Models (SSMs), is relatively new in interpretability studies. Ali et al. () found that Mamba implicitly attends to previous tokens via its B, C matrices. Studies have explored activation patching on Mamba (Meng et al., ; Sharma et al., ), with a unified view of Transformers and Mambas emerging (Zimerman et al., ; Dao & Gu, ).

## 3 Preliminaries

### 3.1 Models

We choose to study Pythia-160M(^1^11[https://huggingface.co/EleutherAI/pythia-160m](https://huggingface.co/EleutherAI/pythia-160m)) (Biderman et al., ) and an open-sourced version of Mamba(^2^22[https://huggingface.co/state-spaces/mamba-130m](https://huggingface.co/state-spaces/mamba-130m)). These two models are of near size i.e. 160M and 130M, respectively(^3^33Pythia-160M uses an untied word embedding and unembedding, leading to about 30 million (i.e. vocabulary size by hidden dimension) more parameters.). They adopt the same tokenizer and both are trained on the Pile dataset (Gao et al., ). Unless otherwise specified throughout this paper, we refer to these two models as Pythia and Mamba. Since Mamba adopts a GAU-type architecture (Hua et al., ) which implements both token and channel mixing in one Mamba block, it has twice the depth (i.e. 24 layers) as Pythia (i.e. 12 layers).

### 3.2 A Brief Mathematical Framework of Mamba Circuits

Figure: Figure 1: Architecture of a MambaBlock. Some linear transformations and activation functions are omitted.
Refer to caption: x1.png

### 3.3 Sparse Autoencoders

Our SAEs have only one hidden dimension larger than the input dimension (i.e. $F>D$), with the training objective of reconstructing any given model activation and an L1 penalty on its hidden layer to incentivize sparsity. An SAE can be formulated as follows:

$$ $\displaystyle f_{i}(\mathbf{x})$ $\displaystyle=\operatorname{ReLU}(\mathbf{W}^{enc}_{i,\cdot}\cdot\mathbf{x}+b_ {i}^{enc}),$ $\displaystyle\hat{x}$ $\displaystyle=\sum_{i=1}^{F}f_{i}(\mathbf{x})\cdot\mathbf{W}^{dec}_{\cdot,i},$ $$

Where $\mathbf{x}\in\mathbb{R}^{D}$ is the hidden activation decomposed with $F$ features. $\mathbf{W}^{enc}\in\mathbb{R}^{F\times D}$ and $\mathbf{W}^{dec}\in\mathbb{R}^{D\times F}$ are the encoder and decoder of an SAE. $\mathbf{x}$ is linearly mapped into the feature space by the encoder and a bias $b^{enc}$, followed by a ReLU to ensure $f(\mathbf{x})$ (i.e. feature activations) being non-negative.

SAEs are trained to reconstruct the original activation with a linear combination of the decoder columns (i.e. features), determined by $f(\mathbf{x})$. We also set an L1 sparsity constraint on $f(\mathbf{x})$ to obtain a sparse coding of each activation.

## 4 Cross-Architecture Universality with Feature Similarity

### 4.1 For Want of Interpretable Primitives

A number of existing works (See Section 2.1) have
explored representational similarity or what we term Universality with
various measures in the activation space or the parameter space.
In contrast, we seek to tackle this problem in a finer-grained manner by
comparing across models with an independent decomposition of the activation space.
We see this kind of microscopic comparison first pursued by analyzing correlation between
individual MLP neurons in models with the identical architecture (Li et al., ; Gurnee et al., ).

However, if we aim to study similarity invariant to the model architecture,
a natural question arises: There is hardly a counterpart of MLP neurons in
Mamba. If one wants to study architecture-agnostic similarity, it is necessary
to utilize tools invariant to architectural changes. We show in Section 4.4 that neuronal
approach is not suitable for this purpose.
To this end, we leverage a trending tool in interpretability called Sparse Autoencoders to
extract more monosemantic units (i.e. features) from the models’ activation.
Existing literature suggest SAEs are able to extract a lot of interpretable
features from models across a wide range of sizes (Bricken et al., ; Cunningham et al., ; Templeton et al., )
and tasks (He et al., ; Gandelsman et al., ).
Moreover, SAEs learn to decompose an activation space onto a semantic basis
in an architecture-agnostic manner (Section 3.3)

### 4.2 Matching Feature Pairs Across Architectures

#### 4.2.1 Training Sparse Autoencoders

We trained SAEs in the residual stream after each Transformer block in both models, getting 12 and 24 SAEs in Pythia and Mamba,
respectively (See Section 3.1). Both models have a hidden dimension $D=768$. We set the
dictionary size to $F=32D=24576$ for all Sparse Autoencoders (SAEs). We refer readers to Appendix A for more details of our SAEs.

#### 4.2.2 Main Experiments

SAE features are analogous to MLP neurons in that they are both linear.
Thus we can follow Li et al. (); Bau et al. () and Gurnee et al. () to
evaluate feature similarity by identifying feature pairs activated on the
same inputs.

We iterate over 1 million tokens sampled from OpenWebText(^4^44[https://huggingface.co/datasets/Skylion007/openwebtext](https://huggingface.co/datasets/Skylion007/openwebtext)) and save the activation of each SAE feature in both models.
For each SAE feature in Pythia, we compute the maximum Pearson Correlation among all features in Mamba, which we refer to as the Max Pairwise Pearson Correlation (MPPC).

$$ $\rho_{i}^{p\rightarrow m}=\max_{j}\frac{\mathbb{E}[(\mathbf{v}^{p}_{i}-\mu^{p} _{i})(\mathbf{v}^{m}_{j}-\mu^{m}_{j})]}{\sigma^{p}_{i}\sigma^{m}_{j}},$ (8) $$

where $\mathbf{v}^{p}_{i},\mathbf{v}^{m}_{j}\in\mathbb{R}^{N}$ are the activation
pattern of the $i$-th feature in Pythia and the $j$-th feature in Mamba.
We compute the mean and variance for each pattern, denoted by
$\mu^{p}_{i},\mu^{m}_{j},\sigma^{p}_{i}$ and $\sigma^{m}_{j}$.

It is worth noting that the superscript in $\rho_{i}^{p\rightarrow m}$ implies
we are looking for the Mamba feature most correlated to the $i$-th Pythia SAE
feature. This operator however is not commutative. We use this metric throughout
this paper as our main experiment. To show that this unilateral analysis
does not affect our conclusion, we include all reverse results i.e.
$\rho^{m\rightarrow p}$ in Appendix C.

### 4.3 Baselines and Skylines

#### 4.3.1 The Neuron Baseline

As pointed out in Section 3.3, it does not make intuitive sense to compare across architectures in the *neuron basis*. We empirically validate this by computing the correlation introduced in Equation 8 between all Pythia MLP neurons and all Mamba post-SiLU activations.

#### 4.3.2 Hierarchy of Dimension of Variation

Studying universality across neural network architectures with SAEs implicitly discusses a number of dimensions of variation. We believe there are three levels of variables with some relation of inclusion.

Figure: Figure 2: hierarchy of invariance
Refer to caption: x2.png

- •
Language Model Variables: (Middle area) Similarly, language model training is a stochastic progress and may further introduce uncertainty to the model’s internal. Model variables here include *data distribution, training tokens elapsed and training hyperparameters similar to SAE variables*. Training SAEs on different models must give different results since at least their training data (i.e. model activations) are different.
- •
Architectural Variables: (Outermost area) Most aforementioned model variables, especially ones related to initialization and training, are implicitly included in architectural changes. For example, it is certain that we cannot find an identical initialization or training trajectory for Mamba models and Pythia models.

Figure: (a)
Refer to caption: x3.png

To this end, we propose two skylines to indirectly estimate the marginal effect of each area by ablation.

##### Skyline 1: Different Models of Identical Architecture (Model Seed Variant).

For each feature in Pythia, we look for similar features in another Pythia model trained with a different random seed. This changes model initialization and training data order. The intuitive motivation of this is that a Mamba model has all differences the seed variant has plus the architectural change.

##### Skyline 2: SAEs Trained on the Same Language Model (SAE Seed Variant).

We also trained another set of SAEs on the same Pythia model as a higher skyline. This setting further constrains the language model initialization and training factors. The only difference lies in SAE training. The discrepancy between SAEs trained on the same model tells us how convergent SAE features are.

### 4.4 Results

Figure 3(a) presents the results of all four MPPC experiments, namely neuron baseline, Pythia-Mamba SAE similarity (main experiment), model seed skyline and SAE seed skyline.

##### SAE Similarity Significantly Outperforms Neuron Baseline.

In contrast to Gurnee et al. () where the authors find 1% - 5% neurons are universal when comparing across Transformers trained with different initialization, our neuron baseline almost exhibits zero sign of similarity between Mamba and Pythia. This validates our hypothesis that it is not informative to compare privileged directions across models (Elhage et al., ).

##### Cross-Architecture Similarity Slightly Falls Short of Skylines.

Cross-arch SAE MPPC has an average of 0.74, compared to 0.76 for model seed variant and 0.81 for SAE seed variant. More than 30% Pythia features find their matched feature in Mamba with an MPPC greater than 0.95.

We illustrate the distribution of the difference between our Cross-arch SAE MPPC and the model seed variant skyline in Figure 3(b), i.e. $\rho_{i}^{p\rightarrow m}-\rho_{i}^{p\rightarrow p^{\prime}}$ where $i$ indexes a feature in Pythia, ranging from 0 to 12 * 24576(^5^55This is the number of layers multiplied by number of features in each SAE). One can interpret this as the marginal effect of architectural changes. 90% of Pythia features fall in the interval of $-0.22$ to $0.14$ under this metric. More than 60% features have near-zero MPPC difference, which means in our case, they are consistently present after architectural changes.

Overall, we find these a relatively strong evidence that SAE features are universal in both architectures. We repeated the experiments conducted on Mamba for RWKV, supporting our conclusions. We refer readers to Appendix D for more details of the result. Nonetheless, SAEs for feature decomposition is a new approach and this conclusion is drawn under a number of assumptions. We refer readers to the limitation section for more discussion.

### 4.5 Depth Specialization

Another interesting phenomenon is that SAE features form similar hierarchy in different architectures. Our findings extend the scope of neuronal depth specialization (Gurnee et al., ) where the authors find universal neurons to be present at near layers. As illustrated in Figure 3(c), if a feature in Pythia is located in the $l$-th layer out of 12 layers, its matched feature in Mamba is most likely to appear around layer $2l$ in Mamba (out of 24 layers). This provides evidence for a more macroscopic conjecture of language model, i.e. the general structure of a specific language model internals is a *stretched* or *scaled* version of a universal motif (Elhage et al., ; Huh et al., ).

## 5 A Complexity-Based Interpretation of Similarity Difference

We demonstrate in Section 4 quantitatively that a great number of Pythia features can be paired with one Mamba feature in terms of MPPC. It is further required to map such statistics onto our hypothetical Universality. Anecdotally, we obeserve that a higher MPPC feature pair *does not necessarily mean more semantic similarity* than lower ones due to the influence of *feature complexity*. This offers us a new perspective of the aforementioned MPPC difference between our Cross-Arch SAE MPPC and the seed variant skyline (Section 4.3): such difference can be deemed as cancelling out the effect of *feature complexity*.

### 5.1 Case Study

Figure: Figure 4: We present four cases of feature pairs with different Pearson correlation values. To the right of each feature index is the auto-interpretation result generated by a large language model (LLM), with activation examples shown below.
Refer to caption: x6.png

Figure: (a)
Refer to caption: x7.png

We manually inspected a number of matched feature pairs across architectures and seed variants. An intriguing trend is summarized as follows and also illustrated in Figure 4.

##### Higher MPPC pairs are mostly simple features.

Features with a maximum pairwise Pearson correlation close to 1 are typically very simple, activating exclusively on individual tokens. These include a wide range of linguistic units such as all parts of speech, word prefixes, suffixes, letters, punctuation marks, and digits (the first row in Figure 4).

##### Lower MPPC pairs tend to be more complex.

As the *complexity* of features increases, the MPPC scores of most matched pairs fall in the interval $0.3<\rho_{i}^{p\rightarrow m}<0.8$. Intuitively, if both features fire at the same high-level structure, it is more likely that the activation magnitude of one feature is larger for some specific examples. While another feature may instead prefer some other instances, leading to inconsistency in firing strength but agreement in theme. We provide further discussion in Section 5.2.

##### Ultra-Low MPPC pairs are mostly uninterpretable.

Features with extremely low maximum pairwise Pearson correlation are typically uninterpretable and considered meaningless, possibly arising from noise introduced during SAE training or model training. We provide further discussion in Section 5.2.

### 5.2 Quantifying Feature Complexity and Monosemanticity

We are aware that complexity and monosemanticity is a somehow abstract property of a feature. To effectively validate our conjecture, we utilize GPT-4o to automatically evaluate the complexity and activation consistency of each feature’s activation pattern.

We mainly follow Bills et al. () and Cunningham & Conerly () to prompt GPT-4o to score the degree of complexity and activations consistency of each feature. The activation pattern on the first 10 top activating samples of each feature are fed into GPT-4o in a structured format (Appendix E). It then returns an integer score ranging from 1 to 5. For complexity, a score of 1 indicates the simplest features, while a score of 5 represents the most diverse contexts with a coherent theme(^6^66Few features are rated 5 thus we omit this category in Figure 5(a).).

We observe clear correlation between automatically labeled complexity scores and seed variant skyline MPPC, i.e. $\rho_{i}^{p\rightarrow p^{\prime}}$, as well as Cross-Arch SAE MPPC, as shown in Figure 5(a). Statistically, the simpler a Pythia feature is, the more likely it matches another feature in model seed variant or Mamba with a high Pearson Correlation score.

This offers another interpretation of isolating architectural variables with MPPC difference proposed in Section 4.3.2. Cross-Arch SAE MPPC reflects a mixture of several variables. One of them is the semantic complexity of features, which is present in both Cross-Arch SAE MPPC and seed variant skyline MPPC. Subtracting the latter from the former can be deemed as cancelling out the effect of feature complexity.

The trend for the Monosemanticity Score (Figure 5(b)) is even more pronounced, with a clear boundary around 0.2. Features with MPPC values below this threshold are highly likely to be uninterpretable. We refer readers to Appendix F for some cases.

## 6 Zooming Out to Induction Circuit Universality

We also want to look into a larger scale of universality by studying the
circuits in Mamba and Transformers. We choose to start from a relatively
simple and well-studied circuit named induction circuit (Olsson et al., ).

The in-context learning behavior is when sequences with pattern $[A][B]...[A]$ appear,
the model is very likely to output $[B]$ in the subsequent step to complete
the 2-gram with knowledge from the past.
It has been shown that this is closely related to a kind of an algorithm mainly
implemented by two attention heads in Transformers and reoccurs broadly in many
Transformer language models (Olsson et al., ; Todd et al., ).

It has been already demonstrated by Fu et al. (); Gu & Dao (), inter alia,
that Mamba is also capable of in-context learning. An important curiosity
is how Mamba implements this algorithm since it does not have attention heads.
We find that Mamba employs a similar mechanism to Transformers, but with some
slight differences in its implementation.

### 6.1 Identifying Induction Circuits in Mamba

We employ a circuit discovery method called path patching (Wang et al., ) to search for which components in Mamba directly affecting the output logits, as shown in (Figure 6).

Path patching replaces the activations along specified paths and allows this influence to propagate along the allowed path then observe the difference of interest. This measures the counterfactual influence of the specified paths only through allowed paths. The data results for all experiments in this section are provided in Appendix G.

Figure: Figure 6: Path patching includes three times of forward pass. We patch a specified path (red edges) with a corrupted path and uses the recomputed activations only through the paths of interest (yellow edges). Activations through other paths (green edges) are fixed to the same as the clean run.

##### Layer 17 SSM States Are Key in Induction.

We label key positions in $[A][B]...[A]$ as $A_{1}$, $B_{1}$, and $A_{2}$, and start with a counterfactual input $[A][B^{\prime}]...[A]$. Path patching is applied to $h_{A_{2}-1}^{(l)}$ for $l$ layers (0-24), allowing the influence to propagate except through other SSM states. Patching at layer 17 results in the largest logit drop, with layer 20 having a weaker effect and other layers mostly irrelevant.

##### Key Information Flows into SSM States at Token B+1.

We next patch components of $h_{A_{2}-1}^{(17)}$, which depends on $c_{A_{2}-1}^{(17)},\dots,c_{0}^{(17)}$ (Equation 7). Results (Figure 7(a)) show patching $c_{B_{1}+1}^{(17)}$ significantly reduces the logit for $B$, with no substantial changes at other positions.

##### Local Convolution Aggregates A and B into B+1.

Finally, we patch components of $c_{B_{1}+1}^{(17)}$, a linear combination of $x_{B_{1}+1}^{(17)}$, $x_{B_{1}}^{(17)}$, $x_{A_{1}}^{(17)}$, and $x_{A_{1}-1}^{(17)}$ (Equation 6). Patching $x_{B_{1}}^{(17)}$ or $x_{A_{1}}^{(17)}$ with corrupted inputs, $[A][B^{\prime}]...[A]$ or $[A^{\prime}][B]...[A]$, significantly affects the final logits, indicating that the short convolution layer aggregates information from both $[A]$ and $[B]$ into the SSM states.

Figure: (a)
Refer to caption: x10.png

### 6.2 Mamba and Transformer Implement The Same Induction Algorithm

We have abstracted the induction circuit in the Mamba model (Figure 7(a)), while in Transformers, this is implemented through a previous token head and an induction head (Figure 7(b)). In both models, the induction algorithm mixes information from the previous token and stores it in a key-value format to predict the next token. The model retrieves this stored information to perform induction.

In Transformers, the previous token head attends to the preceding token, boosting the induction head’s attention for next-token prediction through the OV circuit. In contrast, Mamba uses convolution in specific layers to merge neighboring tokens’ information into the SSM state, which subsequent tokens query for induction (Figure 7).

### 6.3 Local Convolution of Mamba Prefers Off-by-One

A curious nuance is that Mamba merge information into the SSM input at token $B_{1}$ whilst Transformers directly performs token mixing between token $B_{1}$ and $A_{2}$. We term this the ’off by one’ token mixing approach adopted by Mamba. It is theoretically possible that Mamba do not apply this strategy and instead write the information of $B_{1}$ into the states without *delay*. We also observe the same preference to ’off by one’ in an early investigation of the IOI task (Wang et al., ) (Appendix H). We are still unclear why such mechanism forms or what it is for, which suggests further study in Mamba circuit analysis.

## 7 Conclusion

In this paper, we propose to measure cross architectural universality with sparse autoencoder feature similarity. A novel metric is introduced to estimate the marginal effect of architectural difference to representational universality and find that most features are quite similar in both models. We also provided a intuitive interpretation of this metric: Pearson Correlation is affected by the semantic complexity of features. Thus we cancel out such effect by ablation. We also observe circuit-level analogy in induction mechanism and noticed some interesting nuances called ’Off-by-One motif’ between Mamba and Transformer.

## Limitation

##### SAE Features as Universal Computational Units.

We are aware that SAEs, no matter how well they perform, yield only a subset of the
*true* features (if exist) in the model. Though SAEs reveal levels
of magnitude of matching features across models, we still lack a totally
reliable way to locate those features uncaptured by SAEs. It is possible
that these nuanced and rare features behave differently across models.
It is also possible that SAEs capture a combination of the *true*
features instead of the features themselves.
Overall, Sparse Autoencoders and their implicit superposition hypothesis
is still an immature research area.
We think there are still a lot of room for improvement till we can
measure the true universality of features.

##### Qualitativeness of Circuit Universality.

Though we have shown that Mambas and Transformers implement the same
induction algorithm, we have not yet quantitatively measured the similarity
with some automated tools. It would be very interesting if there circuit
discovery and comparison can be automated like some existing works in
circuit discovery on Transformers (Conmy et al., ; Ge et al., ).

## Appendix A Training Details of Sparse Autoencoders

### A.1 Collecting Activations

We truncate each document to 1024 tokens and prepend a <bos> token
to the beginning of each document.
During training, we do not use the activation of
<bos> and <eos> tokens.

It has been shown that the activations of different sequence positions of the same document
are highly correlated and may lack diversity.
To address this issue, it is common to introduce randomness into the training data. Our
shuffling strategy is to maintain a buffer and shuffle it
every time the buffer is refilled.

### A.2 Initialization and Optimization

The decoder columns $W^{dec}_{:,i}$ are initialized uniformly and normalized to have 2-norm of
$\sqrt{\frac{2D}{F}}=\sqrt{\frac{2}{\text{expansion factor}}}$. We initialize the encoder weights
$W^{enc}$ with the transpose of the decoder weights $W^{dec}$. The encoder bias $b^{enc}$ and
decoder bias $b^{dec}$ are initialized to zero.

This ensures that the initialized SAE almost always achieve a near-zero reconstruction loss.
It has been widely observed that this kind of initialization is beneficial for training SAEs.

We train SAEs with Adam optimizer with a $\beta_{1}=0.9$, $\beta_{2}=0.999$ of and $\epsilon=10^{-8}$.
The learning rate is set to to 8e-4 for all SAEs.

## Appendix B Distribution of Max Pairwise Pearson Correlation Across Layer

We plotted the layer-wise distribution of MPPC calculated from Pythia to Mamba. For clarity, we divided the layers into four groups: 0–2, 3–5, 6–8, and 9–11, and presented the MPPC distribution histograms for each group (Figure 8).

As the layer number increases, the distribution gradually shifts to the left. In conjunction with the conclusion in Section 5.2 that as MPPC decreases, feature complexity tends to increase, we can infer that lower-layer features are simpler, while higher-layer features become increasingly complex.

Figure: Figure 8: Histogram of the MPPC distribution for Pythia features, grouped by layers 0-2, 3-5, 6-8, and 9-11.
Refer to caption: x12.png

## Appendix C Cross Result of Max Pairwise Pearson Correlation

Figure: (a)
Refer to caption: x13.png

Figure: Figure 10: Frequency of the best matching feature pairs falling on each layer.
Refer to caption: x15.png

The MPPC used in the main text is a one-way, irreversible metric, lacking symmetry. The experiments presented in the main text only report the results of finding the maximum Pearson correlation from Pythia to Mamba and Pythia (model seed variant). Here, we provide the reverse results, i.e., the distributions of $\rho_{i}^{p\rightarrow m}$ and $\rho_{i}^{m\rightarrow p}$, as well as $\rho_{i}^{p\rightarrow p^{\prime}}$ and $\rho_{i}^{p^{\prime}\rightarrow p}$ (Figure 9). Additionally, we show the Depth Distribution of Matched Features when calculating the MPPC from Mamba to Pythia (Figure 10).

The forward and reverse results for Pythia and Pythia (model seed variant) are very close. The reverse results for Pythia and Mamba outperform the forward results, primarily because Mamba has a higher proportion of lower-level features, which tend to have higher MPPC values (Section 5.2). This phenomenon is confirmed in the reverse Depth Distribution of Matched Features. The first five layers of Mamba tend to match with the first two layers of Pythia, especially the first layer, while the first seven layers of Mamba tend to match with the first three layers of Pythia—one more layer than proportional scaling would suggest. Lower-level features are generally simpler (Appendix B). Compared to Pythia, Mamba has a larger proportion of layers dominated by simpler features. However, depth specialization phenomenon is observed in both the forward and reverse matching processes.

## Appendix D Repetition of Experiments on RWKV

Figure: (a)
Refer to caption: x16.png

We repeated nearly identical experiments on RWKV as we did on Mamba. The model used was RWKV-169M(^7^77[https://huggingface.co/RWKV/rwkv-4-169m-pile](https://huggingface.co/RWKV/rwkv-4-169m-pile))(Peng et al., ). The results are presented in a manner almost identical to Figure 3, as shown in Figure 11.

The experimental results demonstrate that the conclusions drawn from the Mamba and Pythia experiments are nearly applicable to RWKV and Pythia, indicating that Universalityis widely present across different model architectures. The slightly lower numerical performance compared to Mamba may be attributed to SAE training or some unknown factors. We do not claim that the similarity between Mamba and Transformer is stronger than that between RWKV and Transformer.

## Appendix E Automatic Scoring Prompt

### E.1 Use For Complexity

We used the following prompt to have the LLM evaluate our Complexity:

> We are analyzing the activation levels of features in a neural network, where each feature activates certain tokens in a text. Each token’s activation value indicates its relevance to the feature, with higher values showing stronger association. Your task is to infer the common characteristic that these tokens collectively suggest based on their activation values and give this feature a complexity score based on the following scoring criteria: Complexity • 5: Rich feature firing on diverse contexts with an interesting unifying theme, e.g. “feelings of togetherness” • 4: Feature relating to high-level semantic structure, e.g. “return statements in code” • 3: Moderate complexity, such as a phrase, category, or tracking sentence structure e.g. “website URLs” • 2: Single word or token feature but including multiple languages or spelling, e.g. “mentions of dog” • 1: Single token feature, e.g. “the token ‘(’” Consider the following activations for a feature in the neural network. Activation values are non-negative, with higher values indicating a stronger connection between the token and the feature. Don’t list examples of words. You only need to give me a number! Just a number! It represents your score for feature complexity.

Following this prompt, we provided ten sentences containing top activations, each consisting of 17 tokens with the activation value for each token.

### E.2 Use For Activation Consistency

We used the following prompt to have the LLM evaluate our Activation Consistency:

> We are analyzing the activation levels of features in a neural network, where each feature activates certain tokens in a text. Each token’s activation value indicates its relevance to the feature, with higher values showing stronger association. Your task is to give this feature a monosemanticity score based on the following scoring criteria: Activation Consistency • 5: Clear pattern with no deviating examples • 4: Clear pattern with one or two deviating examples • 3: Clear overall pattern but quite a few examples not fitting that pattern • 2: Broad consistent theme but lacking structure • 1: No discernible pattern Consider the following activations for a feature in the neural network. Activation values are non-negative, with higher values indicating a stronger connection between the token and the feature. You only need to give me a number! Just a number! It represents your score for feature monosemanticity.

Following this prompt, we provided ten sentences containing top activations, each consisting of 17 tokens with the activation value for each token.

## Appendix F Case of Uninterpretable Features

Figure: Figure 12: We present four uninterpretable cases of feature pairs with different Pearson correlation values. Each feature index is followed by activation examples shown below.
Refer to caption: x19.png

We manually inspected a substantial number of features from the Pythia model with cross-architecture MPPC values below 0.2. We observed that these features are predominantly polysemantic, showing activations on text inputs with no discernible pattern or lacking clear structure. Figure 12 illustrates four randomly selected examples of uninterpretable feature pairs with different Pearson correlation values.

## Appendix G Data Result of Path Patching

For each condition, we selected 128 randomly generated induction data points for path patching.

##### Result1

The results of path patching for $h_{A_{2}-1}^{(l)}$ are shown in Table 1. Use [A][B’]…[A] as corrupted input, [A][B]…[A] as clean input. Distance is the distance between two [A].

**Table 1: The mean logit diff of token B after patching each state**
| distance \layer | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 8 | 0.01 | -0.05 | 0.33 | -0.09 | 0.05 | 0.1 | -0.04 | 0.12 |
| 16 | 0.02 | -0.04 | -0.46 | -0.03 | -0.07 | 0.0 | 0.06 | 0.0 |
| 32 | -0.01 | 0.15 | 0.88 | 0.35 | -0.0 | 0.09 | 0.08 | 0.07 |
| 64 | -0.0 | -0.03 | -0.12 | 0.31 | -0.06 | 0.07 | 0.04 | 0.02 |
| 128 | 0.01 | -0.03 | -0.11 | -0.03 | 0.13 | 0.03 | 0.04 | -0.01 |
| distance \layer | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 |
| 8 | 0.04 | 0.13 | 0.02 | 0.9 | 0.63 | -0.11 | 0.51 | -0.13 |
| 16 | -0.02 | 0.01 | 0.06 | -0.02 | -0.54 | 0.14 | -0.14 | 0.24 |
| 32 | 0.57 | -0.05 | 0.03 | 0.65 | 0.79 | -0.03 | 0.09 | 1.32 |
| 64 | 0.02 | 0.01 | 0.01 | -0.26 | -2.18 | 0.1 | 0.09 | 0.8 |
| 128 | -0.03 | 0.0 | -0.01 | 0.08 | -1.94 | 0.02 | -0.07 | 0.0 |
| distance \layer | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 |
| 8 | 0.16 | -9.65 | 0.04 | -0.19 | -2.02 | -0.0 | -0.03 | -0.0 |
| 16 | -0.04 | -8.09 | -0.04 | -0.04 | -1.81 | -0.02 | -0.02 | -0.01 |
| 32 | 0.08 | -8.73 | 0.09 | -0.2 | -1.48 | -0.03 | -0.01 | -0.01 |
| 64 | 0.49 | -10.16 | -0.04 | -0.12 | -0.99 | -0.01 | -0.01 | -0.0 |
| 128 | -0.1 | -8.44 | -0.04 | 0.07 | -0.66 | -0.01 | -0.03 | -0.0 |

##### Result2

The results of path patching for $c_{i}^{(17)}$ are shown in Table 2. Use [A][B’]…[A] as corrupted input, [A][B]…[A] as clean input. Distance is the distance between two [A].

**Table 2: The mean logit diff of token B after patching SSM input at each position**
| distance \position | $A_{1}$ | $B_{1}$ | $B_{1}+1$ | $B_{1}+2$ | $B_{1}+3$ |
| --- | --- | --- | --- | --- | --- |
| 8 | 0.0 | -0.0 | -9.69 | 0.11 | -0.03 |
| 16 | 0.0 | -0.01 | -11.25 | 0.01 | 0.11 |
| 32 | 0.0 | -0.01 | -16.84 | -0.01 | -0.05 |
| 64 | 0.0 | -0.0 | -10.64 | -0.33 | 0.4 |
| 128 | 0.0 | -0.01 | -9.13 | 0.52 | 0.16 |

##### Result3

The results of path patching for $x_{i}^{(17)}$ are shown in Table 4. Use [A][B’]…[A] as corrupted input, [A][B]…[A] as clean input. Distance is the distance between two [A].

**Table 3: The mean logit diff of token B after patching $x_{i}^{(17)}$ at each position with [A][B’]…[A] as corrupted input**
| distance \position | $A_{1}$ | $B_{1}$ | $B_{1}+1$ |
| --- | --- | --- | --- |
| 8 | 0.0 | -9.11 | 0.26 |
| 16 | 0.0 | -15.17 | 0.24 |
| 32 | 0.0 | -13.95 | -0.06 |
| 64 | 0.0 | -13.71 | 0.27 |
| 128 | 0.0 | -9.5 | -0.11 |

##### Result4

The results of path patching for $x_{i}^{(17)}$ are shown in Table 4. Use [A’][B]…[A] as corrupted input, [A][B]…[A] as clean input. Distance is the distance between two [A].

**Table 4: The mean logit diff of token B after patching $x_{i}^{(17)}$ at each position with [A’][B]…[A] as corrupted input**
| distance \position | $A_{1}$ | $B_{1}$ | $B_{1}+1$ |
| --- | --- | --- | --- |
| 8 | -8.27 | -0.88 | 0.04 |
| 16 | -9.44 | -0.52 | 0.17 |
| 32 | -11.35 | 0.72 | -0.02 |
| 64 | -9.07 | -1.5 | 0.2 |
| 128 | -8.96 | -0.54 | 0.13 |

## Appendix H Off By One in IOI Task

### H.1 Task Description

The indirect object identification (IOI) task involves sentences with two clauses: an initial clause (e.g., ”When Mary and John went to the store”) and a main clause (e.g., ”John gave Mary a bottle of milk”). The initial clause introduces the indirect object (IO) and the subject (S), while the main clause mentions the subject again, indicating the action of the subject giving the object to the IO. The task is to predict the final token corresponding to the IO. We use ”S1” and ”S2” to represent the first and second occurrences of the subject, respectively.

Figure: Figure 13: Logit changes after patching $c_{i}^{(l)}$ for each layer and each position. For each cell, the left side represents IO, and the right side represents S. Token positions are labeled using the tokens from the example sentence: ”Then, Brandon and Mark went to the hospital. Mark gave a computer to Brandon.”
Refer to caption: x20.png

### H.2 Occurrence of the Off-by-One Phenomenon

We use [IO][S]…[S]… as clean input and [IO’][S’]…[S’]… as corrupted input. Path patching is applied to $c_{i}^{(l)}$, allowing its influence on the logits through all paths except those involving other layer SSMs. We compute the logits difference between IO and S, as shown in Figure 13. It is observed that the $c$ causing the logit change always appears at the token immediately following IO or S. This suggests that the information from IO and S enters the SSM state at the next token, which is similar to the induction task phenomenon and represents another instance of the off-by-one effect.

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 5
presentation: 2
questions: 1. Why did you choose OpenWebText as your primary dataset for analysis? How might the choice of OpenWebText as the dataset influence your results? Have you tested if the feature similarities hold across different domains (e.g., code, mathematics, or structured data)? Would analyzing domain-specific text reveal different patterns of architectural universality?

2. Have you performed any statistical significance tests to support your claims of feature similarity and universality?

3. How generalizable are your findings to other tasks beyond language modeling?

4. Can you provide more details on why the "Off-by-One" motif exists in Mamba models?

5. Is there a risk that the Sparse Autoencoder pre-processing itself may impose a degree of alignment between features in Transformers and Mambas? Could the sparsity constraint inadvertently enhance apparent similarity?

6. How do you expect your findings to scale to larger models? Did you observe whether model size impacts universality between architectures? Could smaller or larger versions of Transformers and Mambas exhibit different degrees of feature similarity?
soundness: 2
strengths: - The identification of the "Off-by-One motif" in Mamba models is a unique contribution that highlights nuanced differences between architectures.

- The introduction of a complexity-based interpretation for understanding feature similarity differences is innovative.

- The circuit-level analysis of Mamba models, revealing structural analogies with Transformers, is good and adds depth to the study. The validation of feature similarity and its correlation with universality further strengthens the study.


- The findings of this paper have implications for the field of neural network interpretability. By demonstrating that different architectures can converge to similar algorithms and features, the study provides valuable insights into the generalizability of mechanistic findings across models.
summary: The paper investigates the "universality hypothesis" in mechanistic interpretability, which suggests that different neural network architectures may converge to implement similar algorithms when tasked with analogous objectives. The authors focus on two mainstream architectures for language modeling: Transformers and Mambas. They propose using Sparse Autoencoders (SAEs) to extract interpretable features from these models and demonstrate that a significant portion of features are shared between the two architectures. The paper validates the correlation between feature similarity and universality and delves into the circuit-level analysis of Mamba models, finding structural analogies with Transformers, particularly in induction circuits. 

The paper's contributions include:
- Introduction of a novel metric to isolate and quantify feature universality in the context of architectural variations.
- Empirical evidence shows that Transformer and Mamba models learn similar features through the application of SAEs.
- Circuit analysis of Mamba models reveals structural analogies and nuanced differences compared to Transformer circuits.
- Support for the universality hypothesis by demonstrating cross-architecture feature similarity and identifying the "Off-by-One motif" in Mamba models.
weaknesses: - While the paper focuses on Transformers and Mambas, it would benefit from a broader examination of additional architectures. Including a more diverse set of models, such as recurrent neural networks (RNNs) or convolutional neural networks (CNNs), could strengthen the universality hypothesis by offering a more comprehensive understanding of feature similarity across a wider range of neural networks. This would enhance the generalizability of the findings.
	
- The paper utilizes OpenWebText for correlation analysis but does not discuss how the choice of dataset might affect the results. A more detailed examination of the potential biases and limitations introduced by the dataset choice would provide a clearer context for the findings and ensure that the results are not overly dependent on a specific dataset.

- The claims of feature similarity and universality would be more robust if supported by statistical significance tests. Including such tests would provide stronger evidence for the observed correlations and enhance the credibility of the conclusions.

- SAE-related technical gaps:
    - The paper does not include ablation studies on SAE hyperparameters (dictionary/code size, training duration, etc.). Conducting these studies would help to understand the sensitivity of the results to different hyperparameter settings and ensure the robustness of the findings.
    - There is no discussion of how SAE reconstruction quality relates to feature similarity. Addressing this relationship would provide insights into the effectiveness of SAEs in isolating interpretable features and validate the methodology used.

- The use of GPT-4 for complexity scoring lacks rigorous validation. The paper does not provide inter-rater reliability metrics or comparisons with human annotations, nor does it discuss potential biases in the automated scoring. 

- The paper provides a limited exploration of why the "Off-by-One" motif exists in Mamba models. A deeper investigation into the underlying reasons for this motif would enhance the understanding of the structural differences between the Mamba and Transformer models and provide more insights into the universality hypothesis.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: * Do Transformers have such layer-specific behavior when it comes to inductive ability?
* Is there a way to empirically verify the claims in Sec 6.2?
Appendix D: How does the feature mapping between RWKV and Mamba look like?
* Sec 6.1: Is the layer-17 phenomenon robust to random initializations? I.e., if one retrains the SSM with another seed, would layer 17 still be the key in induction?
soundness: 3
strengths: * This paper studies an important problem.
* It makes good use of sparse autoencoders for analysis.
* The experiments in this paper are well implemented.
summary: This paper studies the mechanistic similarity between language model structures (Mamba, RWKV, Transformer). The authors focus on their Universality, a hypothesized property that suggests different neural architectures implementing similar algorithms on similar tasks.

In the first part of the experiment section, they use Sparse Autoencoder (SAE) as the major tool for their analysis. The representations from two LM architectures are taken to train SAEs. The latent vectors in the SAEs, in which each dimension corresponds to various syntactic and semantic phenomena, exhibit mechanical similarities, and it is possible to find a matching between the vectors from different LM architectures.

In the second part, the authors study the induction behavior of two LM architectures. They found that the 17th layer of Mamba is the most important for the LM’s inductive ability.

I think this paper studies an important problem and is well executed. I found the experiments in this paper to be well implemented. My only concern is with the role of circuit analysis experiments. They are indeed very interesting but I’m not sure how they contribute to building a mechanistic analogy between SSMs and Transformers. Do Transformers have such layer-specific behavior when it comes to inductive ability? Is there a way to empirically verify the claims in Sec 6.2?

Minor:

Appendix D: How does the feature mapping between RWKV and Mamba look like?
Sec 6.1: Is the layer-17 phenomenon robust to random initializations? I.e., if one retrains the SSM with another seed, would layer 17 still be the key in induction?
Line 179: missing section reference.
Line 861: missing space between ‘Universality’ and ‘is’
weaknesses: * The role of circuit analysis experiments is unclear.
* The claims made in Section 6.2 need to be empirically supported.
* (Minor) The size of LMs is limited. Only ~100m models are used in the experiments.

---

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: - While the heatmap matrix in Figure 3c is mainly diagonal, I can see that there is a cluster of features located in the last layer of the Pythia and distributed fairly uniformly in the middle layers of Mamba. Can the authors clarify the meanings of these features?
soundness: 4
strengths: - The paper is clearly written. 
- I appreciate the idea of using skylines, as it helps support the authors' claims. 
- The results are interesting and useful for further research.
summary: This paper presents an exploration of the Transformer and Mamba models through mechanistic interpretability methodology. Despite the architectures being very different, the features and circuits of these models turn out to be very similar.
weaknesses: I couldn't identify any specific weaknesses. However, below are some suggestions that could enhance this work from my perspective:


- It would be interesting to explore more circuits through SAE, as suggested in [1] (see Section 4.3). However, it is unclear where SAE should be placed within the Mamba architecture to achieve similar features.
- While the Pearson correlation appears to be a natural choice for measuring feature similarity, it assumes that the feature space has linear properties. It might be worthwhile to explore other correlation measures, such as distance correlation, which could potentially yield better results.
- A clear statement clarifying that MPPC refers to the maximum Pearson correlation between models' features is needed to improve understanding.

[1] Interpreting Attention Layer Outputs with Sparse Autoencoders (Kissane et al.)

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: What justified the choice of plain SAEs over more performant variants such as [Gated](https://arxiv.org/abs/2404.16014) or [Top-K SAEs](https://openai.com/index/extracting-concepts-from-gpt-4/)? It is currently hard to gauge the impact of this choice on the obtained results, and whether findings could have been different if improved SAE variants were tested.
soundness: 3
strengths: The universality evaluation pursued in this paper is timely and relevant, given recent advances in non-Transformer architectures for language modeling. The baseline and skylines employed in this work are well-motivated and provide helpful reference points for the analysis. The analysis of feature correlation based on complexity is also interesting, showing convincing proof that most commonalities across architectures are found for simpler SAE features. Overall, the figures are designed clearly and compellingly to support the findings detailed in the main body of the paper.
summary: This work investigates the similarity of sparse autoencoders (SAE) features and induction circuits between a Transformer-based and a comparable Mamba-based language model trained on the same dataset. Results show that many features from Transformers SAEs show a high max pairwise Pearson correlation with Mamba SAE features, with their depth along model layers roughly matching across the two models. The correlations found for cross-architecture comparison are compared to a neuron baseline and two skylines obtained from different models and SAE training seeds, showing that the cross-architecture comparison falls only slightly short of skylines. Authors further examine the correlation between cross-architectural matching features and their complexity, finding that features with the most overlap are generally simpler and more monosemantic. Finally, the authors briefly investigate the similarity between induction circuits in the same architectures using path patching, finding a similar mechanism mediated by the convolutional operation of the Mamba architecture. Notably, the information mixing necessary for the induction operation is performed earlier in Mamba ("off-by-one" mixing).
weaknesses: **Novelty of the findings** While to my knowledge, this is the first work evaluating the similarity of SAE features between Transformers and Mamba models, other works such as [Paulo et al. 2024](https://arxiv.org/pdf/2404.05971) and [Sharma et al. 2024](https://arxiv.org/abs/2404.03646) already showed that many interpretability approaches such as logit lenses, steering vectors, and probes produce similar results across a variety of LLM architectures (Mamba, Transformer and RWKV). It is, hence, not particularly surprising that such findings extend to SAE decompositions of model activations.

**Generality of findings for larger models**  Authors experiment only with 2 tiny models with ~100M parameters each. This can be reasonable in light of the requirements (same training data, effort of training SAEs on each layer), but these systems are significantly smaller than those used by [Paulo et al. 2024](https://arxiv.org/pdf/2404.05971) for comparable cross-architecture interpretability experiments. Notably, larger checkpoints for both models used by the authors are publicly available, including the same training data for control, and could have been used to further prove the generality of the reported results. Importantly, without further experiments, it cannot be excluded that the limited capacity of tiny models might be the main motivation behind the high similarity features and circuits across the two architectures, and this could not be the case for more capable models with e.g. 1B or 7B parameters.

**Multiple Comparison and Correlation Analysis without Hypothesis Testing** The maximal correlation of feature activation patterns with other (24576 x # of layers) features is bound to be quite high due to the enormous amounts of comparisons. In Section 4.4, no hypothesis is formulated regarding the expected similarity of features found across the four tested variants, and consequently, no significance measure for the correlation coefficients is reported. As a result, conclusions regarding the similarity of Mamba and Pythia SAE features are ambiguous (e.g. the statement "[...] our neuron baseline almost exhibits zero sign of similarity between Mamba and Pythia" at line 268 does not agree with Figure 3a, where at least 15% of neurons exhibit a correlation > 0.4). To make the analysis more convincing, a clear hypothesis regarding the degree of similarity in resulting SAE features should have been formulated and tested for baseline, experiment, and skylines, each including a correction procedure such as the Bonferroni method to account for multiple comparisons.

**Minor formatting/clarification points:**

Line 135: The mention "$F_a$ and $F_b$ are some kinds of operation function." is too generic in this context. The purpose of these functions should be specified, and at least one example of functions used for this purpose should be provided.

Line 179: Broken reference.

Figure 1 is too tight with the inline text, making the distinction between caption and main body text unclear.

Section 4.1: title is ungrammatical. I imagine you meant something like "Searching for / In search of Interpretable Primitives".

Line 202: Clarify that you mean all features for all SAEs across all model layers (it becomes clear only from Figure 3c later in the paper)

Line 263: The acronym MPPC is never introduced alongside its meaning.

Figure 5: The mention "both Model Seed Variant and Cross-Arch SAE MPPC exhibit correlation, while one in SAE Seed Variant is weaker" in the caption is not very meaningful, since the trends for all three variants are pretty similar. For part (b), the mention "scores ranging from 1 (No) to 2 (Yes)" is confusing: it would be better to say "Distribution of MPCC for polysemantic (1) and monosemantic (2) auto-generated feature labels."

Meta-review: The paper investigates the universality hypothesis, showing that Transformers and Mambas, despite architectural differences, learn similar features for language modeling tasks. Using Sparse Autoencoders (SAEs), the authors demonstrate cross-architecture feature similarity and identify a novel “Off-by-One motif” in Mamba models, providing new insights into their induction circuits. The study combines feature-level analysis and circuit-level comparisons to support claims of mechanistic universality.

The strengths of the paper lie in its novel application of SAEs to compare architectures, robust empirical results validated by statistical significance tests, and the depth of analysis that identifies meaningful similarities and differences. The primary weaknesses include limited experiments on larger models and questions about the robustness of the results. Additionally, reviewers raised concerns about potential biases introduced by SAE pre-processing and the generalization of findings beyond language modeling. Most of these issues were addressed during the rebuttal period: the authors provided results for larger models (2.8B parameters), performed statistical tests, and demonstrated that the SAE method did not artificially impose alignment. While broader generalization and deeper circuit validation remain future directions, the authors' responses sufficiently strengthen their claims.

Overall the paper makes a significant and timely contribution to neural network interpretability. Its findings are well-supported, and the concerns raised have been addressed to a reasonable extent. I recommend accepting this paper.

Decision: Accept (Poster)