ResiDual Transformer Alignment with Spectral Decomposition
Authors: Lorenzo Basile
Valentino Maiorca
Luca Bortolussi
Emanuele Rodolà
Francesco Locatello
Venue: Accepted by TMLR ()

Abstract: When examined through the lens of their residual streams, a puzzling property emerges in transformer networks: residual contributions (e.g., attention heads) sometimes specialize in specific tasks or input attributes. In this paper, we analyze this phenomenon in vision transformers, focusing on the spectral geometry of residuals, and explore its implications for modality alignment in vision-language models. First, we link it to the intrinsically low-dimensional structure of visual head representations, zooming into their principal components and showing that they encode specialized roles across a wide variety of input data distributions. Then, we analyze the effect of head specialization in multimodal models, focusing on how improved alignment between text and specialized heads impacts zero-shot classification performance. This specialization-performance link consistently holds across diverse pre-training data, network sizes, and objectives, demonstrating a powerful new mechanism for boosting zero-shot classification through targeted alignment. Ultimately, we translate these insights into actionable terms by introducing ResiDual, a technique for spectral alignment of the residual stream. Much like panning for gold, it lets the noise from irrelevant unit principal components (i.e., attributes) wash away to amplify task-relevant ones. Remarkably, this dual perspective on modality alignment yields fine-tuning level performance on different data distributions while modelling an extremely interpretable and parameter-efficient transformation, as we extensively show on 70 pre-trained network-dataset combinations (7 models, 10 datasets).

Published in Transactions on Machine Learning Research (04/2025)
ResiDual Transformer Alignment with Spectral Decomposition
Lorenzo Basile*- University of Trieste
lorenzo. basile @phd units. it
Valentino Maiorca"
matoroa @ddi uniromal..t
Sapienza University of Rome
Institule of Science and Technology Austria (ISTA)
Luca Bortolussi
luca. www.borttokusishhnisss...
University of Trieste
Emanuele Rodolà Sapienza University of Rome
rodola @di. www.omiiiioa....omm
Francesco Locatello
francesco.locatellonn Wist. ac.
Institute of Science and Technology Austria (ISTA)
Reviewed on OpenReview: https: // openreview...ee... net/ forun? id- =37LCgSIæI
Abstract
When examined through the lens of their residual streams, a puzzling property emerges in transformer networks: residual contributions (e.g., attention heads) sometimes specialize in specific tasks input attributes. In this paper, analyze this phenomenon in vision transformers, focusing on the spectral geometry of residuals, and explore its implications for modality alignment in vision-language models. First, we link it to the intrinsically low-dimensional structure of visual head representations, zooming into their principal components and showing that they encode specialized roles across a wide variety input data distributions. Then, we analyze the effect of head specialization in multimodal models, focusing on how improved alignment between text and specialized heads impacts zero-shot classification performance. This ppecaliiatioon-pefrrmmaaa holds across diverse pre-training data, network sizes, and objectives, demonstrating powerful new mechanism for boosting zero-shot classification through targeted alignment. Ultimately, we translate these insights into actionable terms by introducing ResiDual, a technique for spectral alignment of the residual stream. Much like panning for gold, lets the noise from irrelevant unit principal components (i.e., attributes) wash away to amplify task-relovant ones. Remarkably, this dual perspective on modality alignment yields fine-tuning level performance on different data distributions while modelling an extremely interpretable and parameter-efficient transformation, 70 pre-trained network-dataset combinations (7 models, 10 datascts).
link consistently
extensively show
1 Introduction
In recent limes, transformers have become the backbone of most stale-of-the-arl machine learning systemss thanks to their adaptability to various domains, including language modeling (Brown et al.) 2020 Touvron
et al. 2023), vision (Dosovitskiy et al. 2021 Radford et al. 2021) and many different scientific domains (Es-
peholt et al. 2022 Jumper et al. 2021 Merchant et al. 2023 Traditionally, these models are treated as
'Equal contribution. Work done while visiting ISTA. Code available

---

Published in Transactions on Machine Learning Research (04/2025)
Input
Specialized contributions to the residual stream
Output-level Representation
Data
Shape head
Pattern head
Color head
Transformer
Data
Task boundaries text encodings
striped blue circle
A square
A striped shape
A red shape
A striped blue circle
full red square
A circte
full shape
A blue shape
A full red square
Figure 1: From the transformer's residual stream, direct contributions from individual heads across the network can be analyzed. In a multimodal, zero-shot classification setting (e.g. in CLIP), task boundaries are defined by text prompts that may vary in their conceptual granularity. When certain heads are specialized in particular features (e.g,, shape, pattern, color), they may more accurately apply these boundaries than the model's original output. In this example, only the fine-grained task (brown) effectively separates the samples at the output level.
producing a unique, monolithic output. However, a key component in the success of transformers is the versatile inductive bias introduced by multi-head attention (MHA) layers, which alternate with multi-layer perceptrons (MLP) to form any transformer-based archilecture. MHA layers are made of several indepen- dent computational units, called heads, that process input data in parallel and update the residual stream that carries it to the output via skip connections.
Similarly to what had been observed for filters of convolutional neural networks (Yosinski et al. 2014 Gavrikov & Keuper 2022). recent works point the emergence of a specialization property attention heads, both in large language models (Voita et al.) 2019 Li et al., 2023 Chughtai et al. 2024) and in the visual branch of CLIP models (Gandelsman et al. 2024h. Specialization seemss to be by-product related to large-scale training, but it is not clear exactly why it emerges and whether this is systematic property. Interostingly, this property implics that different units might learn to attend to specific attributes or to solve specific tasks, thus processing the input in a disentangled manner, overcoming known theoretical challenges (Hyvärinen & Pajunen 1999 Locatello et al. 2019).
In modern transformer networks, the model's final output produced by applying a simple lincar transfor- mation to the residual stream (up to LayerNorm). This residual st.eemm accumulates information additively, drawing from each attention head and all MLP layers (Elhage et al.) 2021), producing reneral-purpor reprosentation used as feature set for many tasks. This dccomposition raises an intriguing question: are all these units essential for solving specific tasks, or do some introduce noise that obscures relevatl information? Typically, there is a trade-off between a model's generalization and its performance on specific tasks. However, it might be that to specialize a model for a specific task, we do not need to retrain the whole model. This is how we can define task-irrelevant and noise units: task-irrelevant units are those that can be deactivated without affecting performance, while noise refers to units that, when deactivated, enhance performance by removing signals misaligned with the task. By manipulating the residual stream, can boost the units already aligned with the task, amplifying relevant signals while reducing noise.
Contribution In this paper, we tackle this question from the perspective of the latent geometry of residual units. First, on a variety of transformer-basse vision models, including multiple versions of CLIP (Radford
et al. 2021 BLIP (Li et al. 2022), VIT (Doovvisskiy al. 2021 and DINOv2 (Oquab et al. 2024 we
2

---

Published in Transactions on Machine Learning Research (04/2025)
show that such units are embedded in low-dimensional manifolds and that, when there is specialization, it can be traced back to the role of few principal components. Then, by introducing spectral analysis method based on a discrete version of principal angles (Björck & Golub, 1973), we quantitatively measure the similarity of residual units across different datascts, revealing that the roles of specialized units remain surprisingly stable.
Building on this insight, we hypothesize that, in many cases, the information necessary for solving a task is already embedded within subset of highly specialized residual units. We show that this picture emerges clearly in viston-language models like CLIP, where we find units or sets of units that align with textual attributes more preciscly than the full model output on given task. In fact, as the output combines all residual units, this relevant information may be obfuscated by other units that introduce irrelevant signals. Instead of fine-tuning the entire model, we propose to isolate and enhance the task-relevant units by filtering out the noise akin to panning for gold. By doing so, we can significantly boost model performance with up to 4 orders of magnitude fewer parameters than full fino-tuning and 2 less than those needed for training simple linear transformation at the output level.
To implement this, we introduce ResiDual, a novel approach that focuses on the principal components (PCs) of the residual units to identify and retain the needed information. This framework selectively reweights the most relevant. PCs, amplifying the signals that align with the task objective while remaining compu- tationally efficient. This spectral reweighting of individual units addresses nonlinear interactions between them and provides a geometrically principled and interpretable method for optimizing transformer models by capitalizing on the knowledge they already possess.
In summary, our contributions are follows:
We inspect the geometric structure of attention head representations vision transformers, show- casing their low dimensionality and their increasing nonlinearity along model depth;
We characterize the emergent specialization of attention heads through their principal components and show that it stays consistent across data distributions;
We identify task-specific units in vision-language models, showcasing that focusing on these units in zero-shot scttings can outperform using the full residual output when there latent alignment between units and tasks;
We introduce ResiDual, geometrically grounded method for manipulating transformers by rewcight- ing the most relevant principal components of the residual units. This approach can sidestep the need for full-model finetuning, as it reaches competitive performance with minimal parameter overhead.
2 Related Work
Transformer Residual Decomposition Transformer networks (Vaswani et al. 2017p rely on residual connections around each multi-head atlention (MHA) and MLP layer, resulting in a final representation that combines contributions from all units across layers by simple summation. Techniques like logit lens (nostalgcbraist/ 2020) and Direct Logit Attribution (DLA) (Elhage et al. (021) and Cancedda (2024). at the spectral level- focus on how individual layers or residual units (such as MLPs or attention heads) affect the final output in logit space, given that their contributions are projected upstream via linear transformations (up to LayerNorm (Lei Ba et al. 2016), an affine one). Here, apply this residual decomposition to provide more comprehensive understanding of how these units interact and align across different tasks, revealing the deeper structure within the residual space.
Residual Properties To understand the geometric nature of latent manifolds, previous works analyze the intrinsic dimensionality (Iz) (Ansuini et al., 2019 Cheng et al. 2023 Valeriani et al.) 2024 of the representations within the network, which is typically much lower than the embedding dimension. We posit that the Linear Representation Hypothesis (Park et al. Jiang et al. 2024),, which suggests that transformer representations encode high-level concepts in linear directions, complements this view hinting at transformer
3

---

Published in Transactions on Machine Learning Research (04/2025)
models sometimes modulating input attributes in a linearly structured and low-dimensional (i.e., specialized) way. Previous works in language modeling have highlighted this specialization (Voita et al. 2019 Michel
et al. 2019: Li et al. 2023 Lv et al.) 2024 Chughtai et al. 2024), revealing that only a few attention heads are responsible for specific tasks and that they assume specialized and interpretable roles. Our analysis bridges geomelry and specialization, revealing that vision transformer heads are low-dimensional (though often nonlincar, especially in deeper layers) and highly specialized for downstream tasks.
Multimodal Alignment In multimodal models such CLIP (Radford al. 2021 it is well known that the vision and text branches operate in neatly separated latent spaces Liang al. 2022). Despite this modality gap, Chattopadhyay et al. (2024) and [Bhalla et al. (2024) leverage the multimodal latent space of CLIP to find sparse decompositions of its representations using text. Similarly, Gandelsman et al. (2024) show that text encodings can align with specific head-level representations of CLIP's visual branch, providing insights into the specialized roles of individual heads through manually crafted textual inputs. Balasubramanian al. (2024) generalize this approach unimodal vision transformers and arbitrary residual units through a scoring function and the estimation of aligning transformation between the spaces. The idea that CLIP is able to disentangle concepts and encode them in separate subspaces also appears in WWoff ct al. (2023) and Lewis et al. (2024). y. Here, we study the modality alignment at the spectral level, reaching the granularity of head principal components, and show how they can be used to improve the alignment between text and visual branches in CLIP-like models.
3 The Geometry of Residual Units
In this section, we examine the relationship between head specialization and the low-dimensional nature of head manifolds. Initially, head representations exist a relatively low-dimensional ambient space. Through linear transformation, however, they are subsequently embedded into higher-dimensional space within the residual stream (Elhage et al. 2021), which shares the same dimensionality as the model's output. At this point, head reprosentations are transcribed to the residual stream, and they contribute additively to the final output of the model. In fact, throughout the paper, we will assume that the model output is the summation of the encodings of all residual units (attention heads H, MLPs M and input embeddings Xo):
JU] Y U.-U,=X- Xo
[
M
Σ
H
Mj. j=1
(1)
with Y as the final output of the model, summing up all the residual units in U E U. Please refer Appendix A for more rigorous description of the residual decomposition. In this paper, we focus the geometry of attention head representations, prior works (e.g., Voita et al. (001);; Gandelsman et al. (2024h) have demonstrated their specialization in distinct tasks. MLP layers mix outputs from multiple heads by design, leading to more entangled and less interpretable representations (as we show in Figure 11
3.1 Residual Dimensionality
Despite being embedded into gigher-diiensione. space, head representations exhibit an even lower intrinsic dimensionality (I2) than that of the original ambicnt space. This indicates that irrespective of thoir high- dimensional embedding, the essential structure of head representations is highly compressed and governed by a compact, low-dimensional geometry. In short, the intrinsic dimensionality of a dataset the least number of variables required to satisfactorily describe the data points. Ideally, if data lie on linear manifold (a hyperplane), the intrinsic dimensionality coincides with the number of principal components required to completely explain their variance. In more realistic setting, data lie on curved, nonlinear manifolds, and lincar estimators like PCA fail to capture their real intrinsic dimensionality. In such cascs, one can resort to nonlinear la estimalors. Among them, we choose to employ the TwoNN (Facco el. al. 2017 because of its efficiency and stability on complex and non- uniform manifolds.
Experimental setting We start by evaluating the intrinsic dimensionality of head representations across multiple transformer-based vision architectures pre-trained with different. objectives (supervised, unsuper-
4

---

Published in Transactions on Machine Learning Research (04/2025)
vised, self-supervised). Namely, we employ OpenAI's CLIP Radford et al.
2021 OpenCLIP (Cherti et al,
2023), BLIP (Li et al. 2022,, ViT (Dosovisskiy et al. 2021 and DINOv2 (Quab et al., 2024 all in their version based on VaTgL (results on ViT-Base models are in the Appendix in Figure 8 We feed them a subset of the training set of ImageNet (Russakovsky et al.) 2015 containing 80000 images stratified on
the class labels, and we extract the representations for all atlention heads. Then, we compute the intrinsic dimensionality of such representations using a linear estimator (PCA) and nonlinear one (TwoNN). Lincar La is computed as the number of components needed by PCA to explain 99% of head variance.
OpenCLIP-1I
CLIP-L
DINOv2-I.
VIT-I
BLIP-L
60.13 19
19.14
3.19
0.09
60.38
20, 26
3.02
0.091
62.56
14.61
4.28
0.08
61.56
24
2.56
0.11
50.12
15.06
3.36
0.14
22
61.00
20.61
3.00
0.11
61.81
21 05
2.95
0.08
62.14
16 .9
3.69
0.08
62.06
28.33
2.20
0.08
50.06
14.00
3.60
0.16
21
61.94
23.15
2.70
0.10
62.00
21.14
2.95
0.10
62.06
18.37
3.39
0.09
62.00
30.45
2.05
0.07
49.19
15.93
3.13
0.27
20
61.88
25.71
2.42
0.09
62.00
22.47
2.78
0.08
20.41
3.04
0.11
$2.57
1.91
0.06
53.75
20.02
0.22
62.25
25
25.91
2.41
0.09
22. 95
2.71
0.10
61.06
21 81
0.15
88
33.87
1.83
0.05
56, 50
22.70
2.49
0.18
62.00
26.23
2.38
0.09
61:94
25.08
2.50
0.11
60.12
24.22
2.49
0.16
61.81
3-1.6
1.80
0.05
56.94
23.91
2.38
0.14
61.94
27.46
2.29
0.09
61. 88
26.46
2.36
0.09
60.06
24.56
2.47
0.15
61.44
33.63
1.83
0.05
57.50
24.49
2.35
0.13
16
61. 69
28 59
2.17
0.08
61.50
28. 26
2.19
0.12
60.19
25.92
2.33
0.16
61.81
31.16
1.80
0.05
56. 31
21
2.33
0.14
15
61.75
29. 82
2.08
0.09
61.42
28.77
2.15
0.12
59.38
26
2.26
0.16
60.81
33.48
1.82
0.06
56. 12
24.69
2.28
0.15
14 61.75
30.49
2.03
0.11
61.62
30.40
2.03
0.14
59.38
27.42
2.17
0.17
33.78
0.05
24.74
61 .1000
29.08
2.11
0.14
61 .38
30.31
2.03
0.13
57.50
26.65
2.16
0.21
60.00
333.83
1.78
0.07
54. 74.75
25.13
2.18
0.14
Layer
60.94
30, 26
2.02
0.16
60.50
30.21
2.01
0.13
57.00
26 22
2.17
0.18
58. 44
33.92
1.73
0.08
53. 88
24.83
2.17
0.16
11
RO.31
29.66
2.04
0.15
60.56
30.61
1.98
0.14
56.25
27 .6
2.07
0.22
55, 19
32.43
1.70
0.09
48 81
23, 43.38
2.11
0.18
10
59.69
29.63
2.02
0.14
60.50
31 1.22
1.94
0.10
54.75
26.49
2.07
0.24
51.38
31.07
1.65
0.10
21.66
2.00
0.22
9
59.00 5G
30.34
1.97
0.14
58. 69
29.13
2.02
0.15
45.50
22. 47
2.02
0.29
48. 9-1
30.19
1.62
0.10
42. 81
21.50
1.98
0.23
67.56
28. 06
2.05
0.15
58, 44
28.58
2.05
0.15
43.69
22.36 36
0.42
28. 65
0.14
20.50
0.26
55.31
26.96
2.05
0.18
57.19
27.91
2.05
0.17
47.50
24.10
1.94
0.27
44.81
27.62
1.62
0.15
34. 94
19:24
1.81
0.30
51.75
24.91
2.08
0.25
533.19
25.48
2.09
0.21
41.94
21 228
1.94
0.35
43.50
27 27.29
1.59
0.14
29. 19
17.56
1.65
0.33
5
48.44
23.
2.01
0.25
50.56
24.51
2.06
0.28
47.38
23 89
1.97
0.31
40.62
25, 22
1.61
0.16
32.35
18.51
1.74
0.32
4
38.81
20.37
1.88
0.36
49.62
24.35
2.04
0.31
32.81
18.36
1.75
0.38
39.81
24.58
1.62
0.20
25. 75
16.10
1.57
0.35
3
30.19
18.30
0.37
45.38
22. 62
1.98
0.30
42.25
23.20
0.53
36.94
22 52
0.24
62
17.04
0.30
2
29.50
76
1.40
0.54
47.36
24.24
0.24
29.94
66
31.73 75
19.30 30
1.64
0.25
25.
0.38
31.38
19. 28
1.51
0.49
45.88
23.81
1.88
0.28
12.69
11.65
1.06
0.56
23. 25
13.57
1.45
0.35
18.25
14.06
1.26
0.49
33.44
20, 16
1.52
0.50
42.44
22.53
1.80
0.33
11.81
11 11.45
0.98
0.59
15.00
133.11
1.03
0.56
11.81
13.54
0.80
0.66
L
Ralia
EVR.
1.
Ratio
BVR,
L
Ratio
EVR.
1
Ralio
BVR,
Ralio EVR,
Figure Heads in early layers show low-dimensional, linear structures, as suggested by similar intrinsic dimension estimates from PCA (L) and TwoNN (N). Moving toward the output layer, the nonlinear di- mensionality peaks and then decreases, while PCA's 's linear estimate continues to rise, indicating increasing
nonlinearity in head manifolds (Ratio = )) The first principal component (EVR,) explains around 50% of the variance in carly layers, dropping to around 10% in later layers.
Result analysis We report in Figure the results for head units on ImageNet (a subset of 80000 images stratified by class), in Figure 10 the results for other datasets and in Figure 11 those for MLP units on ImageNet. We observe that the true head dimensionality (the one computed with nonlinear estimator, TwoNN) tends to increase in the first half of the model and to decrease towards the last few layers, following a characteristic hunchback shape, similar to previous findings in other vision architectures (Ansuini et al., 2019). However, the number of dimensions returned by the linear estimator grows constantly through the model. This disparity, reflected in the growing ratio between the two estimates, suggests that the units in the early layers are close to linear, while those in the later layers lie on more curved manifolds. The last column shows the average explained variance ratio (EVR) of the first PCA component and highlights that heads in the first layers are largely explained by this direction, while it still accounts for a nontrivial 10% of head variance in late layers.
These findings highlight that low head dimensionality and monotonically increasing nonlinearity arise along the residual streams of vision transformers, regardless of pre-training objective and data.
3.2 Principal Components Encode Unit Semantics
The low dimensionality of head encodings results relevant consequences for their interpretability. Head representations can be easily approximated with sparse recovery algorithms in a way that is akin ing PCA, but over discrete set of vectors. For CLIP models, this approach has been recently explored by
to perform-
5

---

Published in Transactions on Machine Learning Research (04/2025)
Gandelsman al. (2024). There, the authors introduce sparse approximation algorithm, TextSpan (TS); and decompose head encodings using set of textual descriptions coming from the text branch of CLIP as a dictionary. They observe strong specialization properties, witnessed by high coherence in the textual explanations of each head.
We link TextSpan to the more established family of Matching Pursuit (MP) (Mallat & Zhang, 1993) algo- rithms, widely employed in signal processing. More specifically, as we show in the Appendix A.2 TextSpan is analogous to Simultancous Orthogonal Matching Pursuit (SOMP) (Tropp et al. 2006), with light modi- fications. TextSpan, like any MP algorithm, approximates the signal through linear combinations of basis functions. Considering the high nonlincarity of later layers (Section 3.1, and TS being lincar sparse ap- proximation method, we now want to investigate whether TS is, in reality, focusing on the first principal components of the signal (head-level representations).
Experimental setting For this experiment, we position ourselves in the same setup as the original TS paper (Gandelsman et al.) 2024). Hence, we consider the attention heads belonging to the last 4 layers of OpenCT.IP-T. We use two sparse approximation algorithms: the original TextSpan, which operates on the whole head representation, and Orthogonal Matching Pursuit (OMP) (Pati et al. 1993). Different from TextSpan, OMP computes sparse approximations of vectors, not matrices (like head representations). Therefore, in this experiment, we apply OMP to the first principal component of each head. We denote this method as OMP1. The dictionary we use contains the encodings produced by OpenCLIP-L. for the set of image descriptions provided by [andelsman et al. 2024). We apply both algorithms to select descriptions for each head and compute absolute Z-score of the cosine similarity (sim) between them, compared with the average cosine similarity μ between the descriptions selected by TextSpan and the entire dictionary Z [s
agreement. score between the sets. The agreement is computed as the
Result analysis We report in the left panel of Figure 3 the agreement scores. The right panel reports few examples (one per layer) of descriptions obtained using the two algorithms. We observe that a high Z- score (e.g., head 8 of layer 22, which is almost 5a away from μu), is reflected in extremely similar descriptions from the two methods. When the agrcement is lower, as in the case of head 20 from layer 8, the two sets of descriptions substantially differ even though they share some high-level semaattcc.
Overall, this analysis indicates that, in some cases, the first principal component captures nearly all the essential information about the head's specialized semantics. In other cases, the head's role appears to be distributed across multiple components.
3.3 Spectral Dataset Comparison
Our aim is now to understand to what extent specialization generalizes across different input data distribu- tions. To do so, we introduce spectral metric to compare the representations of residual units. Since units are low-dimensional (Section 3.1 and their specialization is deoply impacted by a few principal components (Section 3.2 , we define a metric to quantify the similarity between their PCA bases, inspired by principal angles BBööck & Golub 1973).
Let S and S2 be two subspaces of dimensions k and h in an d-dimensional space. The principal angles 0, for 1, l,...,min k2) are given by:
u
cos
max wEs -i ves,
(2)
Vn-1
1 |
where Un-1 and Vn-1 are the span of {ui.... un and {ui,.... un respectively.
Now, let S = (ui...., 4k, and Sg = { Uk represent sets of fz-normalized discrete vectors (c.g.. principal components) with (optional) associated weights wf and wi (e.g., singular values).
We define the spectral cosine similarity 8n for n L,..,, L,.,,.,,mi(kk k2) as:
8n
(u, wj)]wfug
(3)
ig (i
j Jn

---

Published in Transactions on Machine Learning Research (04/2025)
0.37
1.25
1:43
0.71
TextSpan
OMP1
1.35
0.17
1.64
L80.H8 ("Seenery")
L20.H8 (Z 0.02)
13
1.03
0.65
1.45
1.20
Photo taken in Galápagos Islands
Uuspoiled beauty
Image taken in Norway
Picture taken in Portugal
12
0.37
0.29
1.47
2.96
Evocative beauty
Crisp autumn leaves
Vibrant urban energy A skirt
Evocative candid gaze
0.06
2.73
0.38
0.93
Picture taken in Cyprus
L21.HI1 ("Location")
121.HI1 (Z 2.73)
1.51
2.15
0.39
2.09
Picture taken Cyprus
Picture taken Cyprus
1.30
0.13
Picture taken in Ontario, Canada
Picture taken in the Canadian lakes
Photo taken in Río de Janciro, Brazil
Image taken in the Florida Everglades Picture captured in the Scottish highlands
Head
0.02
0.21
4.93
3.73
Photo captured in the Arizona desert
Image taken in New England
Warm and cozy indoor scene
7-
0.41
0.35
1.99
0.73
L22. L8 ("Letters")
L22.H8 (Z 4.93)
2.60
1.01
1:20
2:33
A photo with the letter F photo with the letter V
A photo with the letter F
A photo with the letter P
0.23
2.94
0.32
1.89
photo with letter
A photo with the letter T
A photo with the letter T
A photo with the letter X
1.79
0.78
0.53
A photo with the letter X
A photo with the letter B
8
0.50
0.19
3.02
0.19
H2 (t ("Animals")
L,3.H2 (Z = 1.91)
Image showing prairie grouse Image with penguin
Picture of feline
2
0.21
1.98
1.62
1.91
An image with dogs
magnolia
Photo of a furry animal
1.16
3.17
2.21
0.63
An image with dogs
Photo taken in Grand Canyon An image with cats
0.18
3.17
0.96
0.82
An image with cats
21
23
Layer
Figure 3: Comparison between TextSpan and Orthogonal Matching Pursuit on the first principal component (OMP1), applied to the heads of OpenCLIP-L. Left: agreement score between the descriptions returned by the two methods. Right: qualitative comparison of selected descriptions for 4 heads, one per layer, at different agreement levels. A similar analysis for the second principal component is presented in the Appendix in Figure
where i,, .1,...,,n-1 in-1 and j1. .i....,in-1 are previously selected indices.
The original principal angles mcasure the alignment between subspaces by maximizing the cosine similarity of vectors in their span. In our discrete case, vectors are directly selected from sets S₁ and S2 with optional weighting. The final measure is the aggregation of the spectral cosine similarities along the min(k1, k2) entries, with normalization to bound our mcasure between and 1. We define the normalized spectral cosine similarity between the two sets of principal components as:
sim SS1, =
(4)
min(k (k,, Ax)) (wtwyy
En=1
In the following, we apply this measure to compare residual units across different input datasets. is worth noting here that the main advantage of this formulation, compared to standard approaches to representation similarity, is that our metric does not rely on the alignment between samples, as operates in the dual spectral space. This edge is crucial in our application to different datasets, which even vary in size.
Experimental setting We consider the same ViT-basod encoders of Section 3.1 and 14 different datasets: ImageNet (the same split used in Section 3.1 CIFAR(-100/-10) (Krizhevsky 2009 Tmagehet-SSetch (Wang)
7

---

Published in Transactions on Machine Learning Research (04/2025)
LeCun et al. 2017). DTD [Cimpoi et al. 2014), SUN397 (iiao et al. Li et al. 2017), and random images (10000 samples with RGB values in [-,, We use the original train/validaann vvalidation / test splits if available, otherwise we produce the splits through a stratified random sampling over the classes. For each encoder, we use our similarity measure to compare its unit representations produced on each training dataset with the ones obtained on the training split of ImageNet. ImageNet is taken as a reference under the assumption that being a general enough dataset, its head PCA bases are sufficiently comprehensive to approximate the primary features across other datasets. Additionally, we perform qualitative inspection of a few heads that stand out by finding their textual decomposition. For this step, we use Simultaneous Orthogonal Matching Pursuit (SOMP), having established its strong relationship with TextSpan (Appendix A.2
et al.
2019) Cars (Krause et al.
2013 MNIST
1998). SVHN
Netzer et al. 2011 EuroSAT
Helber et al. 2019h, RESISC45 (Cheng et. al.
2016), GTSRB (Stalkkamp et al., 2011). PACS
Result analysis datasets on OpenCLIP-I. are reported in Figure 2 (results on other models can be found in the Appendix in appendix A.4) Rows are ordered according to the mean overall similarity between the corresponding dataset and ImageNet. Interestingly, dataset ordering is consistent across different encoders. The mean correlation coofficient between datasct similarities (averaged over all heads) across different models is 0.97. The full comparison between encoders is reported in the Appendix (Figure 13 On OpenCLIP-L, we observe that datasets that maximally align with ImageNet share classes with it (e.g., SUN397 and Sketch) and/or contain generic images (e.g,, DTD and Cars). Moreover, datasets that share the same input image structure and concepts (CIFAR-10 and CIFAR-100) have an almost identical similarity distribution across heads, Overall, we observe a decreasing trend in head similarity scores as depth in the model increases. The simple, linear heads of the first layers are responsible for the extraction of low-level patterns (Dosovitskiy et al.) 2021) | and emeerge as almost always identical across different data distributions. On the last layers, just a few heads per dataset stand out: this is where we are looking for specialization. Zooming in on few of these heads, in Figure 4b] report their textual descriptions obtained with SOMP. Head 7 of layer 22 (specialized on seasons) stands out because it is highly activated in many datasets that contain pictures of scenery (such as SUN397, GTSRB, and, more prominently, EuroSAT). Head 11 of layer 22 (specialized in shades of gray) emerges as extremely different between ImageNet and Sketch, which contains grayscale drawings of ImageNet classes. Head 10 of layer 23 (specialized in numbers) is highly activated on both MNIST and SVHN. We note that this not the only 'shared' head between the two, but others, like head of the same layer, are also activated by the random datasot, signaling that they are not as specific.
The results of the spectral head-to-head comparison between ImageNet and all other
These findings show that the similarity mcasure yields intuitive scores, with ImageNet's foundational at- tributes demonstrating generalizability across various data distributions.
4 Resi Alignment
With a refined understanding of head specialization, our objective is now to leverage this property to enhance alignment. between visual unit representations (both heads and MLPs) and text encodings in CLIP-like models. From this point onwards, our experiments will have a shared objective: given frozen zero-shot classifier, e.g., text encodings or prototypes for classes, with this target subspace. In the main text, we focus on multimodal architectures, where improved alignment directly benefits tasks such as zero-shot classification by strengthening the model's capacity to interpret visual data through rext-based descriptors. Additionally, in Appendix [A.3] we conduct experiments on unimodal encoders, where the zero-shot classifier built in a prototype- based fashion.
manipulate the residual to improve its alignment
4.1 Coarse Unit Alignment
We start by exploring whether certain heads are already aligned with the text subspace relevant to our task. Specifically, our goal is to optimize vision-text alignment by combining visual heads weighted using various scoring functions. These functions assign weights to entire head representations to modulate (in some cases completely remove) their contributions to the residual stram. Crucially, these methods operate exclusively
8

---

Published in Transactions on Machine Learning Research (04/2025)
8000
0770
CERRRI
CERRU
00
SVIN
Be:SAT
3NET
(a)
L22.H7 ("Seasons")
L22.H11 ("Grayscale")
L23.H10 ("Numbers")
Serene winter wonderland
charcoal gray color
Image with six subjects An image three subjects
photo taken in the summer
Minimalist white backdrop
photo taken in the fall
Sepia-toned photograph
An image of the number 9
A photo taken in the spring Serene garden oasis
An amber color
The number fifteen Image with four people
High-contrast black and white
(b)
Figure (a) Attention head similarity across layers OpenCLIP-L, computed between ImageNet head representations and those obtained on other datasots. (b) Descriptions picked by SOMP for three specializod heads that emerge from the analysis of panel (a).
at the level of whole-head reweighting, leaving the internal structure and intrinsic specialization of individual heads unchanged. As a result, only the output specialization affected.
Experimental setting We have 3 different selection methods: Unsupervised (U): We use the head- to-output correlation mcasure. Intuitively, the more one head correlates to the full output, the more information it carries. In practice, we compute the Pearson correlation between each sample at the head level and its corresponding output encoding, averaging across samples to obtain scalar; ii) cosk-condiitonee Unsupervised (UJT): Conditioning on the output alone does not necessarily imply that the selected heads will be suitable for a given task. Since the task is modeled by a text subspace, having one encoding for each class, we can condition the previous unsupervised measure to be applied only on the head and output subspaces spanned by the task encodings. This has the effect of ignoring features that might influence the correlation but are not related the task hand. This direct application of the CompAutribut metric introduced in [Balaubraaaanan et al. (2024): ; iii) Supervised (S): When assume the availability not only of the task encodings but also of labeled samples, we can directly estimate the head score by looking at its performance on the downstreem task (in the style of logit lens (nostalgebraist, 2020
For each scoring function, we evaluate each head individually, rank them according to their scores, and apply greody top-k selection. We then sum these selected hcads H' C create a partially recovered residual, which is subsequently evaluated downstream task performance:
E'
Y Y
Σ H
(5)
9

---

Published in Transactions on Machine Learning Research (04/2025)
BLIP-L
OpenCLIP-L
Dataset
U
U
S
R
H
B
0
U
U
S
R
H
B
CIFARIO
0.94
0.94
0.93
0.56
0.93
0.94
0.96
0.96
0.96
0.96
0.59
0.94
0.97
0.98
CIFAR100
0.70
0.71
0.72
0.33
0.69
0.71
0.75
0.80
0.78
0.79
0.38
0.76
0.82
0.84
Cars
0.67
0.68
0.71
0.27
0.66
0.72
0.77
0.92
0.92
0.93
0.39
0.89
0.93
0.93
DTD
0.52
0.34
0.54
0.27
0.50
0.55
0.61
0.59
0.39
0.59
0.29
0.54
0.63
0.69
EuroSAT
0.49
0.53
0.53
0.24
0.37
0.50
0.92
0.64
0.64
0.67
0.34
0.55
0.64
0.95
GTSRB
0.34
0.34
0.37
0.14
0.33
0.35
0.59
0.55
0.56
0.55
0.20
0.49
0.56
0.75
MNIST
0.62
0.64
0.65
0.27
01.1
0.52
0.94
0.74
0.84
0.85
0.25
0.41
0.54
0.97
RESISC45
0,58
0.57
0.60
0.28
0.56
0.59
0.79
0.70
0.69
0.70
0.33
0.63
0.73
0.86
SUN397
0.67
0.66
0.68
0.28
0.65
0.70
0.73
0.70
0.69
0.71
0.27
0.59
0.74
0.76
SVHN
0.25
0.36
0.42
0.16
0.21
0.33
0.56
0.48
0.57
0.53
0.24
0.41
0.41
0.70
Average
0.58
0.60
0.61
0.28
0.53
0.59
0.76
0.71
0.73
0.73
0.33
0.62
0.70
0.84
Table 1: Accuracy when doing zero ablation of all units except top 5% of attention heads. Heads are assigned binary weight using an Unsupervised (U), Task-conditioned Unsupervised (UJT), Supervised (S), and Random (R) stralegy (a mean over 10 dilferent seeds). H corresponds to using all the attention heads available, B is the original model performance, and O is the optimized continuous weighting case. For a qualitative analysis complementing these quantitative results on a selection of datasets, please refer to Table in the Appendix.
We have control measures in place: i) Heads (H): The performance of the model when all the attention heads, and only them, are used. This gives information about the heads' contribution to the residual and, symmetrically, how much the final performance depends the MLP units; ii) Random (R): The average performance over 10 independent random samplings of k hend unils. This can be seen as lower bound on the expected performance; iii) Base (B): The original performance of the model without any modification to its residual. Intuitively, this could represent a thcorctical upper bound on the performance there are no lask-aligned units.
The greedy selection strategy scores heads independently, disregarding inter-head relationships. To make the selection aware of them, optimize scalar weight for each head simultancously using gradient de- scent, providing empirical upper bound for the selection performance. We refer to this procedure Optimized (O).
We evaluate these unit selection strategies on two CLIP-like models (BLIP-L and OpenCLIP-L) and 10 of the datasets of Section 3.3 We choose & for the greedy and random selection so that 5% of the total heads are considered. Additional results are presented in the Appendix in Table 7 Table 4 and Table for other CLIP-like models and Appendix A.3 for unimodal ones (DINOv2-L and VIT-L).
Result analysis As reported in Table the unsupervised scoring (U) performs unexpectedly well despite not explicitly considering the task. This effectiveness likely stems from the fact that core task information is often embedded in the first few principal components (PCs) of the output. By aligning with this information using only a few heads, we achieve a dual benefit: preserving essential task-relcvant information while effectively filtering out noise. In fact, adding the conditioning on the task (UJT) is just slightly beneficial in terms of performance, with the exception of SVHN. The vast majority (on average around 90%) of alignment between task and residual comes from the head contributions, columns H and B. The optimized selection strategy (0) is extremely powerful, having the best score among them, and sometimes almost doubling the original model performances (B), showing how task-relevant information is already present in the residual, just hidden.
witnessod by the similarity between the
These results illustrate that retaining only task-aligned units (effectively "panning for the gold" contained in the residual) is highly effective across the board.
10

---

Published in Transactions on Machine Learning Research (04/2025)
4.2 Spectral ResiDual Alignment
Given that: i) unit specialization is essentially encoded in the principal components (Section 3.2 ii) com-
ponents of general enough data distributions (e.g., ImageNet) capture specialized behavior even on other datasets (Section 3.3); iii) retaining only task-aligned units is beneficial for image-text alignment (Sec- tion 4.1 we propose a method to directly filter information along the residual at the spectral level: ResiD- ual.
In short, we start from the decomposition formula for the residual stream (Equation (1) and allow anisotropic scaling of each unit representation. Specifically, given a unit representation X, its corresponding principal component basis b and associated mean μe, we define the ResiDual transformation of X as:
&D (x + di (X) -=))
(6)
where the learnable vector contains the weights associated with each principal component. Then, the transformation is applied to every residual unit independently, resulting in transformed version of the output Y:
U RD (U,,
(7)
In summary, ResiDual models a simple spectral anisotropic scaling of residual units that results in more complex dynamics in the output space. In this scction, extensively evaluate the effectiveness of this method across a variely of configurations, models and datasets.
Experimental setting We evaluate ResiDual in 3 different configurations: i) R.D, as presented in the original formulation; ii) RD", a version of ResiDual constrained on the number and type of units (we restrict it to heads) and the number of considered components (we truncate PCA bases to explain 90% of the vari- ance). This is done because head units, and specifically their principal components, are strongly connected to specialization; iii) RDY where the ResiDual transformation applied directly on the output encoding Y, to assess whether output components can already be aligned with the task. For all configurations, we select ImageNet as reference for the PCA bases & and unit means μ that appear in the ResiDual Equation (6) Please refer to Table g for additional experiments with dataset-specific bases.
the base zero-shot performance of the model, relying on the alignment coming from pre-training; ii) Full Finetuning: we consider its score the empirical upper bound for alignment. It is obtained by finetuning the whole vision transformer with frozen text encodings; iii) Linear Aligner (Lin): the performance of trained linear transformation of the output shows to what extent output alignment could be linearly recovered. This approach is well-supported by recent studies, which indicate that even independently trained models with different archilectures can often
We compare ResiDual with 3 reference alignment methods: i) Base:
be aligned by simple linear transformation (Moschella et al. 2023 Maiorca et al. 2024 Norelli et al. 2023
Lähner & Moeller 2024 Balasubramanian et al. 2024
For this experiment, we work with 3 CLIP-like models, BLIP-L, CLIP-L, and OpenCLIP-L. (CLIP/ /OLPP/OOenCLIP-B results can be found in the Appendix in Table 9 and Figure 21 and tune them on the datascts employed in Section 4.1] All training runs use the Schedule- Free Adam optimizer (Defazio et al., 2024) with the automatic learning rate finder by Smith (0077), implemented in PyTorch Lightning (Falcon) & The Torch Lightning team 2019). The maximum number of epochs is 30, with an early-stopping policy the validation set accuracy with patience of 5 epochs.
Result analysis A comparative analysis of ResiDual against reference alignment methods is reported in Figure 5 We observe that a lincar transformation of the output is sufficient to approximate full finctuning performance. The spectral residual Iransformation modelled by ResiDual attains comparable (and, in some cases, better) results than the linear aligner on all datasets. These statements hold true across all models. Specifically, the case of SVHN stands out: on this input datasot, ResiDual has an advantage of approximately 10% on all models. We hypothesize that this gap due to the absence of task-relevant features at the output level. This is confirmed by the results in Table 2 on SVHN R.DY has the largest gap from RD,, meaning
11

---

Published in Transactions on Machine Learning Research (04/2025)
Besr Eooe Algge Fut Plaeteiin
BedAA
Besilaa
SWIXX
SUNG
507260
90N000
Figure 5: Performance comparison between text-image alignment methods on zero-shot classification tasks.
BLIP-L
CLIP-L
OpenCLIP-L
Dataset
Lin
RD
RD*
RDY
Lin
RD
RD*
RDY
Lin
RD
RD
RD
CIFAR10
0.97
0.97
0.97
0.96
0.97
0.98
0.98
0.97
0.98
0.98
0.98
0.98
CIFAR100
0.82
0.83
0.81
0.74
0.85
0.86
0.85
0.80
0.88
0.88
0.87
0.85
DTD
0.79
0.77
0.76
0.58
0.81
0.80
0.79
0.63
0.84
0.84
0.83
0.69
EuroSAT
0.95
0.98
0.98
0.83
0.97
0.99
0.98
0.95
0.97
0.98
0.98
0.94
GTSRB
0.87
0.87
0.84
0.59
0.92
0.92
0.92
0.77
0.94
0.92
0.92
0.78
MNIST
0.98
0.99
0.99
0.93
0.99
0.99
0.99
0.97
0.99
0.99
0.99
0.97
RESISC45
0.92
0.93
0.92
0.77
0.95
0.96
0.95
0.87
0.95
0.95
0.95
0.89
Cars
0.86
0.84
0.82
0.75
0.89
0.86
0.85
0.81
0.94
0.94
0.94
0.93
SUN397
0.80
0.80
0.77
0.72
0.82
0.80
0.77
0.70
0.83
0.82
0.80
0.75
SVHN
0.65
0.81
0.77
0.53
0.77
0.86
0.86
0.70
0.75
0.86
0.85
0.64
Average
0.86
0.88
0.86
0.74
0.89
0.90
0.90
0.82
0.91
0.92
0.91
0.84
#params 65.8k
30.7k
8.3k
256
590k
43k
14k
768
590k
43k
13.2k
768
Table 2: Accuracy produced by different configurations of ResiDual, compared with a linear aligner. (Lin): lincar aligner at the output level; (RD): ResiDual the original formulation (all principal components of all residual units); (RD"): ResiDual limited to head units alone (no MLPs), and PCs truncated to 90% of explained variance; (RD*): ResiDual applied to the output encoding.
that output components are not well aligned with the task. While having approximately 30% of the learnable parameters, RD' achieves comparable results RDD, indicating that applying the ResiDual procedure heads (and not MLPs) and their first principal components alone is enough. Moreover, note that in the cases where the performance of the original model is already satisfactory (e.g., CIFAR-10), applying ResiDual does not compromise alignment.
Overall, these results show that, by leveraging spectral-level operations along the residual, ResiDual builds a concise yet expressive transformation that bridges the modality gap, closely approximating full finetuning- level performance, even in its more arrameter-efficeen configuration.
5 Conclusions
In this work, we analyzed the emergent specialization property of attention heads in vision transformers and unveiled its connection with the spectral geometry of residual representations. Specifically, we focused on the relationship between head specialization and downstream task performance. Then, we leveraged this in- troduce ResiDual, method that employed to improve alignment in multimodal transformers by applying
12

---

Published in Transactions on Machine Learning Research (04/2025)
spectral anisotropic scaling along the residual stream. ResiDual proved effective in emphasizing task-relevant principal components and dampening down the others, akin to panning for gold in the residual stream.
Limitations ResiDual fundamentally works by extracting information from residual units already con- taining task-relevant principal components. We expect that when this assumption does not hold, ResiDual cannot recover the alignment, resulting in a significant drop in downstream performance. Moreover, our downstream tasks focused on zero-shot image classification. This implies considering only the alignment between [CLS] token and classifier, ignoring sequence-level information. However, preliminary results show high spectral compatibility between the [CLS] representations and the other patch tokens (Figure 12
Future work The ResiDual formulation based solely on the residual decomposition technique, which opens to its application across virtually any transformer architccture (with some preliminary results on unimodal encoders in Appendix A.33 Our findings show that ResiDual can be constrained at the coarse level to operate subset of units (i.e., attention heads) and at the fine level on specific principal components, Additional constraints to focus on the first few model layers create opportunitics to enhance model inference,
while the possibility to disable specific spectral components opens up fairness applications.
The authors gratefully acknowledge Volkan Cevher for insightful discussion about sparse recovery algorithms, Alex Smola for valuable feedback on the experiments, and Marco Baroni for an engaging conversation on the phenomenon of head specialization in NLP.
Acknowledgeeents
Luca Bortolussi was supported by the PNRR project iNEST (Interconnected Nord-Est Innovation Ecosys- tem) funded by the European Union NextGenerationEU (Piano Nazionale di Ripresa e Resilienza (PNRR) Missione 4 Componente 2, Investimento 1.5 D.D. 1058 23/06/2022, ECS 00000043, CUP J43C220003300066)
References
Alessio Ansuini, Alessandro Laio, Jakob Macke, and Davide Zoccolan. Intrinsic dimension of data repre- sentations deep neural networks. Advances Neural Information Processing Systems, 32, 2019.
Sriram Balasubramanian, Samyadeep Basu, and Sohcil Feizi. Decomposing and interpreting image repre- sentations via text. in vits beyond CLIP. In ICML 2024 Workshop on Mechanislic Interprelabillly, 2024.
URL
Usha Bhalla, Alex Oesterling, Suraj Srinivas, Flavio P Calmon, and Himabindu Lakkaraju. Interpreting clip with sparse linear concept embeddings (splice). arXiv preprint 108. 2024.
Åke Björck and Gene H. Golub. Numerical methods for computing angles between linear subspaces. Mothe- matics of Computation, 27(12)):577-55 594, 1973. ISSN 00255718, 10886842. URL httt:://tw..jjeteeee jstor. org/
stable/2005662
Tom Brown, Benjamin Mann, Nick Ryder, Mclanic Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models few-shot learners. Aduances in neural information processing systems, 33:1877-1901, 2020.
Nicola Cancedda. Spectral filters, dark signals, and attention sinks, 2024. URL https::/arxiv.org//
2402.09221
Aditya Chattopadhyay, Ryan Pilgrim, and Rene Vidal. Information maximization perspective of orthogonal matching pursuit with applications to explainable ai. Advances in Neural Information Processing Systems, 36, 2024.
Emily Cheng, Corentin Kervadec, and Marco Baroni. Bridging information--heoretic and geomctric com- pression in language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 12397-12420, 2023.
13

---

Published in Transactions on Machine Learning Research (04/2025)
Gong Cheng, Junwei Han, and Xiaoqiang Lu. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEFF, 105(10):1865-188)) 2017.
Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. Reproducible scaling laws for contrastive language-image learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognilion, pp. 2818 2829, 2023.
Bilal Chughtai, Alan Cooney, and Neel Nanda. Summing up the facts: Additive mechanisms behind factual recall in llms. arXiv preprint ar NNu:2402.07321,, 2024.
Mircea Cimpoi, Subhransu Maji, Jasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3606-3613, 2014.
Aaron Defazio, Xingyu Yang. Harsh Mchta, Konstantin Mishchenko, Ahmed Khaled, and Ashok Cutkosky. The road less scheduled, 2024.
Alexcy Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021. URL
Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Docds,, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. A mathemat- ical framework for transformer circuits. Transformer Circuits Thread, 2021. circuits. htae///aaaeooo/ndeee....oe
Lasse Espeholt, Shreya Agrawal, Casper Sønderby, Manoj Kunar, Jonathan Heek, Carla Bromberg, Cenk Gazen, Rob Carver, Marcin Andrychowicz, Jason Hickey, et al. Deep learning for twelve hour precipitation forecasts. Nature communications, (3(1):1-10, 2022.
Flena Facco, Maria d'Errico, Alex Rodriguez, and Alessandro Laio. Estimating the intrinsic dimension of datasets by a minimal neighborhood information. Scientific reports, 7(1):12140, 2017.
William Falcon and The PyTorch Lightning team. PyTorch Lightning, March 2019. URL httpps://gtthuub com/Lighttnigg-AT// AI/ Al/lightning
Yossi Gandelsman, Alexei A Efros, and Jacob Steinhardt. Interpreting clip's image representation via text- based decomposition. In The Twelfth International Conference on Learning Representations, 2024.
Paul Gavrikov and Janis Keuper. Cun filter db: An empirical investigation of trained convolutional filters. In Proceedings of the IEEE/CVF conference on compuler vision and pallern recognition, pp.
2022.
Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remole Sensing, 12(7):2217 2226, 2019.
Aapo Hyvärinen and Petteri Pajunen. Nonlincar independent component analysis: Existence and uniqueness results. Neural Networks, 12(3):429-339 1999. ISSN 0893-6080. doi: https://doi. 00g11000000//5000000000000000000 508936680980014003
URL
Yibo Jiang, Goutham Rajendran, Pradeep Kumar Ravikumar, Bryon Aragam, and Victor Veitch. On the origins of linear representations in large language models. In Forty-first International Conference on Machine Learning, 2024. URL http:///wwwaaeaeeee////eee/ttt/a
14

---

Published in Transactions on Machine Learning Research (04/2025)
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Zidek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873))583 589, 2021.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained catego- rization. In Proceedings of the IEEE international conference on computer vision workshops, pp. 554 561, 2013.
Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009.
Zorah Lähner and Michael Moeller. On the direct alignment of latent spaces. In Marco Fumero, Emanuele Rodolá, Clementine Domine, Francesco Locatello, Karolina Dziugaite, and Caron Mathilde (eds.). Proceedings of UniReps: the First Workshop on Unifying Representations in Nearal Models, volume 243 of Proceedings of Machine Learning Research, pp- 158 169. PMI.R, Dec 2024. URL
hctps://proccedinggs mlr. www.mmlr..prreeeeeeee wwwpppresseeee htt////233/1//////e/ html
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based lcarning applied to docu- ment recognition. Proceedings of the IEEE, 86(11):2278 2324, 1998.
Jimmy Lei Ba, Jamie Ryan Kiros, and GeofTrey E Hinton. Layer normalization. Ar e-prinls, pp. arXiv 1607, 2016.
Martha Lewis, Nihal Nayak, Peilin Yu, Jack Merullo, Qinan Yu, Stephen Bach, and Ellie Pavlick. Does clip bind concepts? probing compositionality in large image models. In Findings of the Association for Computational Linguistics: EACL 2024, pp. 1487-1500, 2024.
Chong Li, Shaonan Wang, Yunhao Zhang, Jiajun Zhang, and Chengqing Zong. Interpreting and exploiting functional specialization in multi-head attention under multi-task learning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 16460-16477, 2023.
Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain gener- alization. In Proceedings of the IEEE international conference on computer vision, pp. 5542-5550, 2017.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping serguagg--mage pre-training for unified vision-language understanding and generation. In International conference on machine learning, pp. 12888-12900. PMLR, 2022.
Weixin Liang, Yuhui Zhang, Yongchan Kwon, Serena Young, and James Zou. Mind the gap: Understanding the modality gap multi-modal cantrastive representation learning. In NewrIPS, 2022. URL https:
//openreview wwwprrevvee.neetouuuiuiii.c htt.://www...ooorrroo//////rsss
Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Ractsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. Challenging common assumptions in the unsupervised learning of disentangled represen- tations. In international conference on machine learning, pp. 4114-4124 PMLR, 2019.
Ang Lw, Kaiyi Zhang, Yuhan Chen, Yulong Wang, Lifeng Liu, Ji-Rong Wen, Jian Xie, and Rui Yan. Interpreting key mechanisms of factual recall in transformer- based language models. arXiv preprint arXiv:2403.1955 19521, 2024.
Valentino Maiorca, Luca Moschella, Antonio Norelli, Marco Fumero, Francesco Locatello, and Emanuele Rodolà. Latent space translation via semantic alignment. Advances in Neural Information Processing Systems, 36, 2024.
Stéphane G Mallat and Zhifeng Zhang. Matching pursuits with time-frequency dietionarics. IEFE Transac- tions on signal processing, 31(12):3397 3415, 1993.
Amil Merchant, Simon Batzner, Samuel S Schoenholz, Muratahan Aykol, Gowoon Cheon, and Ekin Dogus Cubuk. Scaling deep learning for materials discovery. Nature, 624(7990):80 85, 2023.
15

---

Published in Transactions on Machine Learning Research (04/2025)
Paul Michel, Omer Levy, and Graham Neubig. Are sixteen heads really better than one? Advonces in Neural Information Processing Systems, 32, 2019.
Luca Moschella, Valentino Maiorca, Marco Fumero, Antonio Norelli, Francesco Locatello, and Emanuele Rodolà. Relative representations enable zero-shot latent space communictiion. In The Eleventh In- ternational Conference on Learning Representations, 2023. URL https://opparrriis...ee// net/forum?id SrC-nwieGJ
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, volume 2011, pp. 4. Granada, 2011.
Antonio Norelli, Marco Fumero, Valentino Maiorca, Luca Moschella, Emanuele Rodolå, and Francesco Lo- catello. Asif: Coupled data turns unimodal models to multimodal without training. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Newral Information Processing Sys- lems, volume 36, pp. 15303 15319. Curran Associates, Inc., 2023. URL httphtttt://prrcceeiingg iww.reeeddddnnn.nnniiiie.....eps
nostalgebraist. interpreting pett://octaBspppaalllltttrpppppp htgp:///eweeooooppeoooooeeoooooo
intorpprtnnnitiitttttttttngng gpt: the logit lens. Less LessWrong, 2020. URL https://wow.eesswroom com/
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Woj- ciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen
Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bo- janowski. DINOv2: Learning robust visual features without supervision. Transactions on Machine Learn- ing Research, 2024. ISSN 2835-8856. URL
Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models. In Forty-first International Conference on Machine Learning.
Yagyensh Chandra Pati, Ramin Rezaiifar, and Perinkulam Sambamurthy Krishnaprasad. Orthogonal match- ing pursuit: Recursive function approximation with applications to wavelet decomposition. In Proceedings of 27th Asilomar conference signals, systems and compulers, pp. 40 44. IEEE, 1993.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748 8763. PMLR, 2021.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211 252, 2015.
Leslie N. Smith. Cyclical learning rates for training neural networks. In 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), 2017.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. Aduances in neural information processing systems, 30, 2017.
Johannes Stallkamp, Mare Schlipsing, Jan Salmen, and Christian Igel. The german traffic sign recognition benchmark: a multi-class classification competition. In The 2011 international joint conference on neural networks, pp. 1453 1460. JEEE, 2011.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Maric-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.110 13971, 2023.
16

---

Published in Transactions on Machine Learning Research (04/2025)
Joel A Tropp, Anna C Gilbert, and Martin J Strauss. Algorithms for simultancous sparse approximation. part i: Greedy pursuit. Signal processing, 86(3):572-58) 2588 2006.
Lucrezia Valeriani, Diego Doimo, Francesca Cuturello, Alessandro Laio, Alessio Ansuini, and Alberto Caz- zaniga. The geometry of hidden representations of large transformer models. Advances in Neural Infor- mation Processing Systems, 36, 2024.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. Analyzing multi-head self- attention: Specialized heads do the heavy lifting, the rest can be pruned. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguisties, pp. 5797-5808, 2019.
Qiang Wang, Bei Li, Tong Xiao, Jingbo Zhu, Changliang Li, Derek F Wong, and Lidia Chao. Learning deep transformer models for machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1812, 2019.
Max Wolff, Wieland Brendel, and Stuart Wolff. The independent compositional subspace hypothesis for the structure of CLIP's last layer. In ICLR 2023 Workshop on Mathematical and Empirical Understanding of Foundation Models, 2023. URL
Jianxiong Xiao, Krista A Ehinger, James Hays, Antonio Torralba, and Aude Oliva.. Sun database: Exploring large collection of scene categories. International Journal of Computer Vision, 119:3 22, 2016.
Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? Advances in neural information processing systems, 27, 2014.
17

---

Published in Transactions on Machine Learning Research (04/2025)
A Appendix
A.1
Residual decomposition
Transformer block (x NR)
Residual connection
Residual connection
Residual stream
Residual stream
Multi-Head Attention
LayerNorm
LayerNorm
MLP
Figure 6: Overview of a Transformer block. Multi-Head Attention and MLP are both surrounded by residual connections and LayerNorm is applied before each sub-layer.
Transformers residual networks made of stacked blocks that contain Multi-Head Attention (MHA) layer and an MLP, both surrounded by residual connection. All vision transformers we employ in this work abide by the "pre-norm" Wang et al. (2019) architecture, which slightly modified the original (Vaswani et al. 2017) by moving LayerNorm before MHA and MLP sub-layers (Figure 6p. This modification implies that, at each layer, MHA and ML. sub-layers directly write their output representation to the residual stream. Hence, the final latent representation of the model is given by:
= xo
Σ
A
M
where X₂ the initial data embodding, A, is the attention output layer i and M, is the MLP output layer All these encodings share the same dimensionality d with the model output.
Attention representations can be further decomposed into head contributions, they the result of linear operations applied to the head-level representations. Specifically, each attention head h of N₂ at layer
Qu.KKr Vdg
produces a representation H., = Softmax
Vi,,, whose dimension is d No Contributions for all
heads then concatenated and projected linearly to obtain the MHA output:
A: (cat(H H. H.,N)) b;
Assuning that the bias term b can be split equally between heads, this operation can be equivalently expressed in distributed form:
N H
N W.
b N
A,
Σ
H H
+
h=1
h=1
These terms H. are the head representations we consider in this paper. They have the same dimensionality d as the residual stream (and model output), and they simply obtained by linearly projecting the 'raw'
18

---

Published in Transactions on Machine Learning Research (04/2025)
head contributions, properly padded with Os to match the dimensionality of the residual stream. Hence, we arrive at this final decomposition:
L N
X +
M, 1=1
In many transformer models, such as the ones employed in this work, the final residual encoding Y is not the final output of the model. For instance, in CLIP, Y is passed through a LayerNorm and then through a bias-less linear projection P that maps ViT encodings to the shared vision-language space:
Y P(LayerNorm(() (Y))
However, this operation can be again distributed over the summands that produce Y because of the linearity of the final projection and because LayerNorm can be rewritten as an affine transformation, as in Gandelsman et al. (2024). The representations we employ in this paper are mapped to the output space by applying (if present) the final projection and LayerNorm each unit entry (Hi, or M.).
A.2 Connecting TextSpan and Matching Pursuit
Algorithm 1: TextSpan (Gandelsman et al., 2024
Input Signal Matrix E Rnndd dictionary D Rkdd number of ilerations N. Output: Reconstruction X support CN
Initialization: Residual R X, reconstruction X 0, dictionary D = D, support set C 0 for / E (O...,
do
P D' R'
p' < arg max =1 Var (b));
Ct+1 + C (w')
R+ + R proj R D Ip'));
X t x + proj R², D' p );
D + D Droj(D', D I'')::
end
Algorithm 2: Simultaneous Orthogonal Matching Pursuit (SOMP) (Topp et al. 2006
Input Signal Matrix E gn.dd dictionary D Réd number of iterations N. Output: Reconstruction X support set C
Initializatinn: Residual R X, reconstruction X 0, support set C 0: for é {O...,, N 1) do - T p' < arg max j=1 Pbbl):
C++1 + C Cp(p
W arg minw
X WDC
X + W'D C4
R+1 - X X++
end
The TextSpan algorithm was introduced in Gandelsman et al. 2024 to find a decomposition of CLIP heads on a set of textual descriptions. Here, we show that TextSpan is equivalent to Simultaneous Orthogonal Matching Pursuit (SOMP) [Tropp et al. 2006), with a few light modifications.
The first modification is that, in TextSpan, before computing the decomposition, the dictionary is filtered through projection the first principal components of the signal. Output-level text encodings have high
19

---

Published in Transactions on Machine Learning Research (04/2025)
semantic gramularity: this operation results in dictionary restricted to the head span. In the following, we will consider this a dictionary preprocessing step and assume that SOMP and TextSpan are provided with the same dictionary, filtered or not.
The second modification that in TextSpan, the row variance of DR is used instead of the & norm as a criterion for atom selection. Here, we show that the two algorithms are equivalent if this criterion is applied in SOMP (or vice versa, the & in TextSpan).
We given signal matrix X Rn.d and two (initially identical) dictionaries DMP Dts R
We will procced by induction. At t = 0, the two methods pick the same atom po (which enters the support set C²), and identically update the residual:
CMp= = (po), RMP
X -proj(X Darrlpo))
Cts = (po). Rts = X - proj(X D4s (po)) = RyP. D/s 1 Dislpo]
Now, suppose we are at step t == n with Chip = Cis and Rts RTp.-
The two dictionaries will be different: DMP never gets updated, while Dîs 1 Dwsssssppp [C7s] because TextSpan applies Gram-Schhine process that finds an orthogonal basis for the subspace of selected atoms. Since TextSpan the dictionary orthogonalized at each step, at time n is orthogonal to all previously chosen atoms (which also orthogonal to each other). The dictionary of SOMP can be decomposed into two terms, one contained in the span of atoms chosen until this point by TextSpan and one orthogonal, which corresponds to the current dictionary of TextSpan:
DMr DMPII DMP,1 DMP.1 + D
The residual of TextSpan (which, by inductive hypothesis, is identical the residual of SOMP) is, by definition, orthogonal to the atoms already chosen by TextSpan. Hence the selection step of SOMP will compute:
DMPRMP (DMP.l + DMPPL) T DMP, R Dis
Then, at step + 1, the two algorithms will pick the same atom index again and update identically the residual, removing its projection on the chosen atom, i.e., Cu+1 C7+1. The last thing prove that the residual is also the same. For SOMP, the least squares solution of the optimization problem results in:
RMP X C (C [ [C X proj(X, DMP PMP(CMt) (C))))
While for TextSpan, we get:
R Ris proj (R Pis/pttal Rīs sr. Dis Dis(Ctt!)) Rts proj RRoj. D4s (Cr+'))
Where the first equality the definition of the residual; the second because DistPatil 1. Pfs(CTsl the last is because the residual and the last chosen atom [P+1) are orthogonal to Dts [C Now, use the inductive hypothesis RTs Rip, that CTS = Cht and Dl.s DMP, so:
R1
RVp -proj(R D DMr (CTP))
maicual
X -proj(X DMP PMr((CMP) proj(X - proj( X, DMr DMP
X -prooo(X DMP [MM)) - (X DMP(CWt] + proj(proj(X, DMP (C (MP(C
MP maicual
R - proj(X, DM (CM)) )))))))))))))))))) proj(proj D (MP(CCP)) DMP(Cutt)
=R proj(X, (MP + proj(X, DMP(C
RMP
where the second last equality comes from the fact that (roj DMP(CN is already in a subspace of Dar (CR-H), so the outer projection does not change the result of the inner one.
20

---

Published in Transactions on Machine Learning Research (04/2025)
A.3 Prototypical Alignment
In this seclion, we provide results in different experimental setting, designed for applying our framework to unimodal encoders such as DINOv2 and ViT.
For these encoders, instead of deriving task boundarics from text encodings, we use a prototypical approach (Snell et al. 2017): for each datasel, we average the latent representations all the corresponding samples the training for each target class. This produces representations that contain the average and high-level information for that class, similar a textual encoding of simple caption, "A "An image of <class>>.
DINOv2-L
ViT-L
Dataset
U
U|T
S
R
H
0
U
U
S
R
H
CIFARIO
0.85
0.92
0.95
0.51
0.95
0.93
0.62
0.97
CIEAR100
0.77
0.79
0.78
0.25
0.84
0.68
0.70
0.70
0.34
0.81
0.86
Cars
0.43
0.43
0.44
0.05
0.46
0.72
0.24
0.26
0,27
0.07
0.36
0.28
0.48
DTD
0.65
0.65
0.65
0.30
0.67
0.70
0.76
0.50
0.50
0.51
0.28
0.59
0.58
0.67
EuroSAT
0.71
0.74
0.71
0.36
0.75
0.71
0.95
0.65
0.64
0.71
0.54
0.73
0.68
0.94
GTSRB
0.21
0.21
0.36
0.10
0.27
0.29
0.66
0.13
0.16
0.34
0.14
0.28
0.20
0.69
MNIST
0.62
0.62
0.55
0.24
0.68
0.65
0.93
0.72
0.71
0.77
0.51
0.76
0.75
0.94
RESISC45
0.76
0.75
0.76
0.25
0.78
0.74
0.87
0.50
0.47
0.50
0.29
0.68
0.57
0.79
SUN397
0.60
0.60
0.60
0.15
0.63
0.68
0.72
0.48
0.51
0.51
0.19
0.67
0.68
0.71
SVHN
0.19
0.20
0.27
0.14
0.19
0.23
0.48
0.28
0.31
0.38
0.23
0.33
0.29
0.54
Average
0.58
0.59
0.61
0.23
0.62
0.63
0.80
0.51
0.52
0.56
0.32
0.62
0.58
0.76
Table 3: Accuracy when doing zero ablation of all units except top 5% of attention heads. Heads are assigned binary weight using an Unsupervised (U), Task-conditioned Unsupervised (UJT), Supervised (S), and Random (R) strategy (a mean over 10 different seeds). H corresponds to using all the attention heads available, B is the original model performance, and O the optimized continuous weighting case.
VIT-L
DINOv2-L
Optimized
EurSAAT
DTTD
EuroSAT
LTTD
ResiDual"
GTSRB
CIFAR100
GTSRB
CIFARI80
0.5
MNIST
CIFARIO MNIST
IIFARI
RESISCI
SVHN
RESISCI
SVHN
Cars
SUNB97
Cars
SUN187
Figure Performance comparison between imago-prototype alignment methods on zero-shot classification tasks. Base: original model. Optimized: learned unit-wise weights (as in Section 4.1 restricted ResiDual method as described in Section 4.2
ResiDual#;
21

---

Published in Transactions on Machine Learning Research (04/2025)
A.4 Additional results
OpaoCLIP-B
CLIP-B
0.10
20,87
0.10
0.11
0.15
0.19
75
0.17
26,
0.16
50
27
2.32
0.14
28 51
2.13
0.15
61:66
21.05
2.11
0.13
01.08
20.41
2.50
0.13
00.66
20.01
2.04
0.15
00.60
20.98
2.65
0.12
68.58
27.18
2.10
0.20
65.00
24.77
2.04
0.15
07.00
20:13
2.12
0.19
06.00
20.10
2.34
0.19
41.25
21 1.358
2.05
0.97
47.58
21.83
2.18
0.33
30.33
21 06
1.82
0.42
47.08
23:00
2.04
0:31
60.00
22.40
1.72
0.54
39.17
21.25
1.78
0.30
Baddoo
XV0.
Figure Heads in early layers show low-dimensional, linear structures, as suggested by similar intrinsic dimension estimates from PCA (L) and TwoNN (N). Moving toward the output layer, the true dimensionality peaks and then decreases, while PCA's linear estimate continues rise, indicating increasing nonlinearity in head manifolds (Ratio )) The first principal component (EVR1) explains around 50% of the variance in early layers, dropping to around 10% in later layers.
TextSpan
OMP,
15
0.00
1.72
0.19
3003
L20.H8 ("Scenery")
L20.H8 (2 = 0.35)
0.00
2.40
Photo taken in Galápagos Islands
Pieture taken in the Swiss chocolace factories
Image Norway
Image Mayan-inspired designs
0.21
Evocative beauty
Stark and minimalist urban serene oceanside scene Vivid cultural ceremnony
Vibrant urban energy A skirt
0.91
0.00
1.47
0.60
0.05
L,1,HI1 ("Location")
L2LHI1 (Z 1.47)
0.37
0.83
1.53
2.00
Picture taken Cyprus Picture taken Ontario, Canada
Picture laken in Hungary Photo taken the Californian vineyards serene woodland refuge
0.31
0:33
1.87
0.2
Photo taken in Rio de Janeiro, Brazil
Photo captured in the Arisom desert Picture captured in the Scottish highlands
Photo taken in the Austrlii rainforedt
Head
0.35
0.88
1.84
2.58
Photo taken Canadian Rockles
0.70
0.47
1.61
LS ("Letters")
L22.H8 (Z = 1.84)
phoso the letter
photo
A photo with the letter
phota with the letter Photo taken in Momment Valley
A photo with the letter D
A photo with the letter
Enchanting fantasy world
2:28
photo the letier
labyrioth
0.90
0.75
1.48
1.61
L23. "'nimals")
L23.HE (2 1.11)
Imaage showing prairic grouse Image with penguin
capacitor
1.19
0.35
1.20
1.11
spiky texture
A magnolia
wolf
0.08
1.29
2.16
0.11
image with dogs An image with
Image
0-
1:28
0.09
3.73
901
spirograph-like shape
231
22
Figure 9: Comparison between TextSpan and Orthogonal Matching Pursuit on the second principal compo- nent (OMP2), applied to the heads of OpenCLIP-L. Left: agreement score between the descriptions returned by the two methods. Right: qualitative comparison of selected descriptions for 4 heads, one per layer, different agreement levels.
22

---

Published in Transactions on Machine Learning Research (04/2025)
241
0.23
2113
921
21.11
8.2
8.1
2.11
26
451
0.2
2126
11/3
171
1.5
2662
31.0
13
6:24
238
237
931
21.03
185
120
3311
28
18:18
0415
1041
6:4
086
1150
1.5
175
9.64
323
014
8775
18.55
0021
1300
55
0M
82
140
175
91
274
614
18.73
5165
5/5
6.5
3.2
250
231
035
112
3181
1
129
255
231
605
11
21.33
175
1.3
23.3
036
3231
31.0
1.2
122
3.1
613
50
8
1632
21.3
1.3
224
630
323
2.30
3.2
143
96.9
260
640
655
(a) EuroSAT
120
13
54
031
85
16.8
1.00
1.5
9853
345
4.48
8015
20.6
31.6
1.5
$6.90
61.11
3127
1.30
164
00
K-5
017
2835
1221
2-8
264
13
17)
6:14
1/2
131
4.51
17
TM
TeNovis
018
400
(b) GTSRB
18
125
381
3
216
1.27
1.5
11
472
3.35
506
1725
20
9XT
130
112
017
3:44
JU2
1121
12
21
42
14
031
4.54
3.50
0.56
1:31
4.5
017
31
B.N
12
PM8
9:3
427
3.55
239
133
124
21
CAF
6:31
0.2
031
206
4.3
1.3
2/4
1.0
186
3:56
350
2277
039
2.6
4.5
114
18
3314
235
175
14
123
100
CLIP
2-5
1.
1521
11:01
21A
123
125
1821
0611
17.0
13
15
12.0
1.0
55
1724
2202
1040
1558
031
12:00
110
1
200
175
4775
154
34.0
175
1513
1140
657
1821
11:0
330
022
2.5
121
1550
180
0.2
11:0
653
1741
1.0
3130
375
135
514
1632
57
17:0
11
3110
3.50
1.5
1530
17:0
11.8
1.20
(c) MNIST
1.2
$0.0
355
21.8
1.50
162
125
22
112
942
160
033
1224
21.0
3531
7::88
1.4
204
1726
03
15:0
260
0.0
200
100
18
12
185
1703
1.14
1/5
1917
11/1
1.10
1/1
eh
11.5
232
085
2128
235
4115
2121
23
155
086
515
2115
(d) SUSSUUU CUNSOO
236
613
11
2100
335
2252
15
385
181
02
2.25
1560
6/6
8:5
215
3849
555
362
180
036
514
6/6
5.5
700
3434
185
015
12
2.50
2.5
0U6
9.3
3/3
200
400
5.56
516
5238
2.2
136
0.5
630
11.5
210
227
616
17.0
2534
230
437
4575
1231
2.3
2.2
135
033
0.0
11/5
2:2
614
31.6
21:2
212
436
4255
1230
0.8
2.38
147
036
360
148
0.6
3.38
310
9.0
27
412
363
142
075
125
5.2
210
0.5
2.4
31.3
250
0.6
8.3
696
64.
Figure 10: Additional datasets for the study on the residual dimensionality described in Section 3.1
23

---

Published in Transactions on Machine Learning Research (04/2025)
OpenCT.PP.1
662
10.04
45.05
43403
23,07
20.56
0.85
605 1M
24.15
23.44
0.06
858.00
18.34
66.75
0.02
015.00
23.20
$0.47
6.03
200.00
16.20
12.95
0.225
13:05
55.55
0.13
271
30
07
0.0
647
20:09
0.00
93
00
87
34.67
6.01
61
15.04
4.00
0.55
30
455
11:80
$5.80
0.69
500
0.00
23
0.00
334
$2.14
0.00
64.00
17.06
22.85
11.01
56
425.00
21.02
20.86
0.17
652
1..00
0.00
411.00
25
0.10
923 00
24
21.87
0.00
100
27.
7.55
0.10
55
425.00
23.05
20.65
0.8
520
50.00
44.00
0.108
500.000
44.77
01:11
324.00
44:44
21.00
0.01
154
0.50
24.07
16.05
0.50
14:77
0.37
125 13
45.25
20.40
14.30
0.333
25.
11300
121.00
24.33
120000
20.04
11.21
415.00
13.44
0.09
400 1M
13.01
0.15
116.
22.70
5.11
1.27
701.00
31.35
22.50
0.81
105100
20.70
4.70
0.41
373.00
31
0.14
0000
70
70
641 00
30.19
0.03
89:00
90.71
6.30
0.35
111/00
30.80
0.39
0.75
0100
20.45
91.00
2.05
913
7.46
13.00
11.87
72.09
10.50
0.95
159.00 09
21:00
6:37
0.07
155 00
25.90
4.55
0.61
6:06
1:98
0.06
280
35.57
10:95
6.13
83.00
19.45
4.11
0.17
07:14
17:00
1.70
0.43
355
20.45
3.85
01:31
1110
14.74
2.04
10300
143
2150
8.1
6.58
$3.00
16.08
3.36
0.25
36.00
16:06
2:35
0.30
21.70
6.04
15.00
13339
0.53
22
5.56
16.00
2.50
0.37
17.00
09
R:00
0:00
0.81
8:00
0.64
10.27
0105
2:28
17.07
15.25
0.81
(a) ML. dimensionality visualization for OpenCIIP-L. on ImageNet.
L20.MLP
L21.LP
Picture taken in wildlife sanctuary
Serene meadow landscape
Intense motorsport race
Evocative streets
Nostalgic vibe
A shelf
Balanced composition
Image captured in the Patagonian wilderness Graceful swimming fish
Intricate architoctural carving
L22.MLP
L23.MLP
Photo taken in Bangkok, Thailand
Image showing prairie grouse
Serene wilderness A photograph of big object
A photo of an adult
An image of Andorra
Picture taken in Rwanda
Image with zebra
Picture with airplanes
Picture taken in the South African safari
(b) Specializations of ML.P units across layers OpenCLIP-LL
Figure 11: Comparison of MLP dimensionality and specialization in OpenCLIP-L for ImageNet showing higher dimensionality than head units and strong superposition of concepts. (a) Visualization of MLP dimensionality. (b) Examples of MLP unit specializations across layers.
24

---

Published in Transactions on Machine Learning Research (04/2025)
CLS vs Patches Spectral Cosine Similarity
15
0.51
1.00
1.00
0.44
0.96
0.42
1.00
0.86
0.92
0.80
0.33
0.65
0.35
0.82
0.37 0.38 0.40
0.82
0.52
0.38
0.75
0.86
0.35
0.53
0.84
0.73
1.00
0.72
0.87
0.52
0.32
0.72
0.54
0.72
0.28
0.40
0.11
0.38
0.34
0.49
0.43
0.66
0.39
0.51
0.75
0.61
0.60
0.96
13
0.66
0.34
0.45
0.91
12
0.92
0.50
0.47
0.84
0.83
0.86
0.87
0.85
0.76
0.50
0.37
0.57
0.33
0.28
0.37
0.30
0.46
0.334
0.47
0.48
0.49
0.70
0.59
0.77
0.99
1.00
1.00
0.82
0.85
0.78
0.93
0.40
0.55
0.35
0.35
0.74
0.46
0.28
0.27
0.53
0.71
0.34
0.36
0.91
0.92
0..9
0.50
0.54
10
-0.62
0.58
1.00
0.92
0.95
0.85
0.87
0.62
0.58
0.26
0.29
0.39
0.35
0.31
0.72
0.36
0.48
0.50
0.78
1.00
1.00
0.63
0.88
0.96
0.95
0.61
0.78
0.31
0.93
0.64
0.80
0.37
0.62
0.24
0.38
0.37
0.87
0.41
0.53
0.44
0.41
0.77
0..22
0.32
0.99
0.97
0:91
0.40
0.79
0.57
0.58
0.53
0.44
0.93
0.28
0.27
0.61
0.39
0.33
0.43
0.44
0.48
0..99
0.31
0.69
Head
1.00
0.67
0.60
0.98
0.96
0.59
0.51
0.44
0.46
0.35
0.78
0.11
0.35
0.66
0.34
0:25
0.41
0.55
0.43
0.83
0.84
0.57
0.98
0.86
0.98
0.99
1.00
1.00
0.19
0.62
0.81
0.35
0.46
0.83
0.50
0.24
0.97
0.37
0.37
0.11
0.29
0.416
0.53
0.53
0.67
0.77
0.43
0.84
0.4
0.55
0.68
0.93
0.50
0.52
1.00
0.98
0.62
0.30
0.91
0.43
0.93
0.77
0.45
0.34
0. 0.76
0.17
0.63
0.30
0.440
0.29
0.35
0.71
0.55
0.63
0.98
0.74
0.98
0.67
1.00
0.45
0.92
0.89
0.89
0.43
0.44
0.42
0.80
0.28
0.55
0.36
0.22
0.43
0.61
0.23
0.60
0.51
0.70
0.47
0.96
0.98
1.00
0.99
0.95
0.54
0.90
0.73
0.70
0.63
0.45
0.81
0.15
0.44
0.39
0.39
0.50
0.53
0.
0.83
0.82
1.00
0.65
0.95
0.99
0.88
0.82
0.74
0.39
0.82
0.24
0.73
0.82
0.89
0.43
0.42
0.24
0.39
0.36
0.36
0.26
0.59
0.69
0.65
0.94
0.0
0.94
1.00
0.73
1.00
0.40
0.28
0.62
0.54
0.49
0.53
0.29
0.28
0.30
0.34
0.24
0.93
0.60
0.66
0.46
0.62
7
12
15
20
22
23
Layer Index
L22.H7.PCO (high similarity)
L22.HI1.PCO (low similarity)
Serene winter wonderland A photo taken in the summer A photo taken in the winter
Photo with faded, nostalgic colors
Classic black and white cityscape
Caricature of a cartoon character
Image with a dramatic thunderstorm
Nature macro photography
Daytime shot
Occan sunset silhouctte
L22.H8.PCO
L22.H8.PC1
vast natural landscape
Sublime and serene mountain lake
Futuristic technological concept
Enchanting fantasy world
Picture captured in the Egyptian pyramids An irregular shape
Image captured in the Japanese tea gardens A photo with the letter L A photo with the letter Y
italic text
(b)
Figure 12: [CLS] compatibility analysis. (a): Head-wise Spectral Cosine Similarity (Equation between [CLS] and patch tokens for OpenCLIP-L ImageNet. (b): Specialization of patch tokens selected heads using OMP applied to specific principal components. L22H8, identified as "letter" head (Figure 3), exhibits the lowest similarity in the final four layers. Interestingly, pattern-like descriptions emerge in the first principal component (PC0), while letter descriptions become apparent only in the second component (PC1).
4
25

---

Published in Transactions on Machine Learning Research (04/2025)
VIT-1
0.976
DINOv2-L
0.99
0.99
0.98
1.00
0.95
0.925
0.98
0.99
1.00
0.98
0.91
0.876
CLIP-L
1.00
1.00
0.99
0.93
CLI
1.00
1.00
0.98
0.99
0.94
0.825
OponCLIP-L.CLPA.. BLIP-L
DINGv2-L
VIT-L
Figure 13: Pearson correlation between cross-datasot similarities different models. Comparison is done between ImageNet and each of the other datasets and averaged over heads.
Cm
CIARRI
CHAR
4.6
RESE9C
PAUS
SVIEN
044
GTSEB
EanSAP
11
12
35
15
10
50
Figure 14: Atlention head similarity across layers of BLIP-L, computed between ImageNet head represen- tations and those obtained on other datasets.
26

---

Published in Transactions on Machine Learning Research (04/2025)
Ou
CIAATIO
CIAREO
HLG
0.4
EmnSIT
MNNST
00
Basion
Figure 15: Attention head similarity across layers of CLIP-L, computed between ImageNet head represen- lations and those obtained on other datasets.
5112
53.47
Decom
Figure 16: Attention head similarity across layers of CLIP-B, computed between ImageNet head represen- tations and those obtained on other datascts.
27

---

Published in Transactions on Machine Learning Research (04/2025)
3050
10:070
0730
Figure 17: Attention head similarity across layers of OpenCLIP-B, computed between ImageNet head rep- resentations and those obtained on other datasets.
DTD
0.8
DIPARIIO
Sectch
CI
0.6
RBSSOOS
CTSRD
0.1
TunSAT
MNNIST
Randon
16
18
Figure 18: Attention head similarity across layers of DINOv2-L, computed between ImageNet head repre- sentations and those obtained on other datasets.
28

---

Published in Transactions on Machine Learning Research (04/2025)
L.D
DEARIO
0.8
Cur
Seid
R0SSC
PACS
CTSRR
00.
BVIN
Rondom
30
12
14
16
18
19
Figure 19: Attention head similarity across layers of VIT-L, computed between ImageNet head representa- tions and those obtained on other datasets.
CLIP-B
Dataset
U/T
S
B
CIFARIO
0.87
0.90
0.89
0.41
0.81
0.91
0.93
CIFAR100
0.60
0.60
0.60
0.20
0.57
0.66
0.69
Cars
0.56
0.57
0.58
0.15
0.51
0.65
0.63
DTD
0.42
0.41
0.41
0.16
0.41
0.45
0.47
EuroSAT
0.54
0.54
0.58
0.23
0.53
0.55
0.88
GTSRB
0.33
0.34
0.40
0.14
0.25
0.42
0.61
MNIST
0.55
0.53
0.50
0.22
0.29
0.52
0.88
RESISC45
0.61
0.61
0.20
0.58
0.66
0.73
SUN397
0.42
0.42
0.47
0.12
0.42
0.64
0.64
SVHNN
0.48
0.44
0.52
0.22
0.42
0.52
0.58
Average
0.54
0.54
0.56
0.21
0.48
0.60
0.70
Table 4: Accuracy when doing zero ablation of units except top 5% of attention heads. Heads are assigned binary weight using an Unsupervised (U), Task-conditioned Unsupervised (UJT), Supervised (S), and Random (R) strategy (a mcan over 10 different scods). H corresponds to using all the attention heads available, B is the original model performance, and O is the optimized continuous weighting case.
29

---

Published in Transactions on Machine Learning Research (04/2025)
OpenCLIP-I
Dataset
U
U
CIFAR1O
0.94
0.94
0.94
0.49
0.92
0.95
0.96
CIFAR100
0.73
0.71
0.71
0.29
0.69
0.76
0.77
Cars
0.79
0.84
0.82
0.24
0.73
0.88
0.86
DTD
0.52
0.51
0.51
0.25
0.51
0.57
0.58
EuroSAT
0.51
0.49
0.49
0.26
0.44
0.32
0.87
GTSRB
0.49
0.47
0.48
0.12
0.42
0.50
0.66
MNIST
0.75
0.71
0.74
0.19
0.45
0.66
0.91
RESISC45
0.64
0.63
0.63
0.25
0.68
0.77
SUN397
0.55
0,55
0,56
0.20
0.46
0.70
0.69
SVHN
0.62
0.62
0.61
0.21
0.44
0.50
0.66
Average
0.65
0.65
0.65
0.25
0.56
0.67
0.77
Table 5: Accuracy when doing zero ablation of all units except top 5% of attention heads. Heads are assigned binary weight using Unsupervised (U), Task-conditioned Unsupervised (UJT), Supervised (S); and Random (R) strategy (a mean over 10 different scods). H corresponds to using all the attention heads available, B is the original model performance, and O is the optimized continuous weighting case.
Method
Atom 0
Atom
Atom 2
U
An acute triangle
The number thirty
Image with single road sign Image with a single road sign concentric circle
GTSRB
U
An acute triangle
The number thirty
S
An acute triangle
The number thirty
H
An acute triangle
The number thirty An image of the number O
Image with single road sign The number thirky
O
An acute triangle An acute triangle
B
The number thirty
Serene beach sunset
Image with three people Image with three people U
U
Image with a six people Image with a seven people Image with six subjects A pendulum
pendulum
MNIST
Image with pair of subjects An image of two subjects Image with seven people detailed macro shot Playful siblings
S
An image of three subjects
H
An image of the number 3 An image of three subjects
O
Aerial view of coral reef An image of the number 3
B
Image with whirlpool of brimstone
Table 6: First atoms (decompositions) obtained by SOMP on OpenCT.IP-I, output encodings after the different coarse unit selection methods are applied.
CLIP-L
Dataset
U
U
S
H
B
CIFARI0
0.88
0.95
0.95
0.51
0.92
0.96
0.97
CIFAR100
0.72
0.73
0.72
0.32
0.68
0.75
0.80
Cars
0.74
0.74
0.73
0.29
0.70
0.78
0.79
DTD
0.50
0.50
0.51
0.26
0.51
0.55
0.61
EuroSAT
0.71
0.71
0.73
0.53
0.62
0.95
GTSRB
0.49
0.48
0.50
0.18
0.31
0.50
0.71
MNIST
0.78
0.77
0.85
0.36
0.75
0.76
0.96
RESISC45
0.63
0.63
0.64
0.30
0.59
0.71
0.84
SUN397
0.56
0.56
0.58
0.23
0.49
0.67
0.71
SVHN
0.65
0.65
0.65
0.30
0.60
0.58
0.71
Average
0.66
0.67
0.69
0.31
0.61
0.69
0.80
Table 7: Accuracy when doing zero ablation of all units except top 5% of attention heads. Heads are assigned a binary weight using an Unsupervised (U), Task-conditioned Unsupervised (UJT), Supervised (S), and Random (R) strategy (a mean over 10 different seeds). corresponds to using all the attention heads available, B is the original model performance, and O the optimized continuous weighting case.
30

---

Published in Transactions on Machine Learning Research (04/2025)
OpenCLIP-L UTT
5%
SUN397
1.00
0.58
0.41
0.65
0.41
0.06
0.73
0.46
0.12
0.36
CIFAR100
0.58
1.00
0.58
0.09
0.58
0.65
0.12
0.52
0.8
Cars
0.41
0.41
1.00
0.27
0.41
0.15
0.31
0.36
0.15
0.36
RESISC45
0.65
0.58
0.27
1.00
0.31
0.06
0.81
0.46
0.12
0.36
GTSRB
0.41
0.36
0.41
0.31
1.00
0.12
0.36
0.41
0.27
0.23
MNIST
0.06
0.09
0.15
0.06
0.12
1.00
0.03
0.03
0.46
0.09
0.4
EuroSAT
0.73
0.58
0.31
0.81
0.36
0.03
1.00
0.46
0.09
0.36
DTD
0.46
0.65
0.36
0.46
0.41
0.03
0.46
1.00
0.06
0.31
SVHN
0.12
0.12
0.15
0.12
0.27
0.46
0.09
0.06
1.00
0.09
CIFAR10
0.36
0.52
0.36
0.36
0.23
0.09
0.36
0.31
0.09
1.00
SUN307
CIFAR100
Cars
RESISCA5
GTSRB
MNIST
EaroSAT
DTD
SVHN
CIFAR10
Figure 20: Jaccard similarity between the heads selected by the Task-conditionas Unsupervised method at 5% (introduced in Section 4.1 for each dataset on OpenCT.IP-L. shows meaningful choices, implying that for similar data distributions, similar heads are selected, and their specialization is preserved across datasets (e.g., SVHN, GTSRB, and MNIST have relevant similarities, but also EuroSAT and GTSRB).
Base
CLIP-B
OpenCLIP-B
Linear Aligner
EurSAT
DTD
EuroSAT
ITTD
Fizetuaing
ReiDuull
CTSRR
CIFAR100
CTSRB
CIFAR100
0.4
0.2
0.2
MNIST
IFARIO MNIST
IFAR10
RESISCA
RESISCA
SUN397
SUN397
Figure 21: Performance comparison between lext-image alignment methods on zero-shot classification Lasks.
31

---

Published in Transactions on Machine Learning Research (04/2025)
OpenCLIP-B
OpenCLIP-L
Dataset
ResiDual
ResiDual
ResiDual
ResiDual
CIFAR1O
0.97
0.97
0.98
0.98
CIFAR100
0.82
0.81
0.87
0.88
DTD
0.78
0.75
0.84
0.81
EuroSAT
0.98
0.97
0.98
0.98
GTSRB
0.88
0.84
0.92
0.91
MNIST
0.99
0.99
0.99
0.99
RESISC45
0.93
0.91
0.95
0.95
Cars
0.91
0.87
0.94
0.93
SUN397
0.77
0.72
0.82
0.80
SVHN
0.81
0.78
0.86
0.84
Average 0.88
0.86
0.92
0.91
Table 8: Accuracy produced by different configurations of ResiDual using dataset-specific bases and not ImageNet ones as in Section 4.2 (ResiDual): ResiDual in the original formulation (all principal components of all residual units); (ResiDual"): ResiDual limited to head units alone (no MLPs), and PCs truncated to 90% of explained variance.
CLIP-B
OpenCLIP-B
Dataset
Lin
RD
RD
RDY
Lin
RD
RD*
RD Y
CIFAR10
0.95
0.96
0.96
0.94
0.97
0.97
0.97
0.96
CIFAR100
0.81
0.79
0.76
0.71
0.85
0.83
0.82
0.78
DTD
0.77
0.74
0.69
0.52
0.82
0.78
0.74
0.64
EuroSAT
0.96
0.98
0.97
0.90
0.97
0.98
0.98
0.93
GTSRB
0.90
0.86
0.83
0.67
0.91
0.88
0.85
0.73
MNIST
0.99
0.99
0.99
0.95
0.99
0.99
0.99
0.97
RESISC45
0.93
0.92
0.90
0.81
0.93
0.93
0.91
0.83
Cars
0.83
0.75
0.70
0.67
0.92
0.91
0.89
0.89
SUN397
0.79
0.76
0.71
0.67
0.81
0.77
0.74
0.70
SVHN
0.71
0.78
0.74
0.60
0.76
0.81
0.79
0.68
Average
0.86
0.85
0.83
0.75
0.89
0.88
0.87
0.81
# #params
262k
15.4k
4.9k
512
262k
15.4k
4.7k
512
Table 9: Accuracy produced by different configurations of ResiDual, compared with a linear aligner. (Lin): linear aligner at the output level; (RD): ResiDual in the original formulation (all principal components of all residual units); (RD*): ResiDual limited to head units alone (no MLPs), and PCs truncated to 90% of explained variance; (RD*): ResiDual applied the output encoding.
32


Reviews:
[{"review_id": "jv4OpTBGnp", "reviewer": "Reviewer_yxtr", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. Innovation in geometric analysis of transformer residual streams and their low-dimensional representations\n2. ResiDual provides interpretable modal alignment optimization without extensive parameter tuning\n3. Comprehensive empirical validation across many benchmarks.\n\nLimitations:\n\n1. Performance improvements mainly demonstrated in zero-shot classification, efficacy in other downstream tasks unclear\n2. Lack of theoretical explanations for the head specialization and the low-dimensional nature of head manifolds.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies how transformer components specialize in specific tasks, introducing ResiDual - a method that improves vision-language model performance by better aligning specialized components between modalities.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "This is overall an interesting paper, which I think does not need much revision. The suggestion is that some theorems on the relationship between head specialization and the low-dimensional nature of head manifolds should be provided.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "GUbLC3ObIJ", "reviewer": "Reviewer_qKGp", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1.\tThe paper provides a fresh perspective on attention heads in vision transformers by analyzing their low-dimensional spectral structure.\n\n2.\tIt offers a more interpretable view of how certain heads align with semantic directions, particularly in vision-language settings.\n\n3.\tThe definitions and theoretical discussions about specific ViTs are thorough and clearly presented, making the core concepts easy to follow.\n\nWeaknesses:\n\n1.\tThe benefits and interpretation of specialized heads, as well as the impact of ResiDual, are unclear in pure vision models (e.g., DINOv2 [1]) where no language input exists. It would help to clarify whether the method influences representation quality at the image or pixel level.\n\n2.\tThe zero-shot evaluation setup requires more explicit definitions and baselines. While it fits CLIP-like models, the approach for DINOv2 is less straightforward and should be clearly explained.\n\n3.\tThe intrinsic dimensionality and specialization may depend heavily on the chosen dataset and tasks. Additional ablations on different datasets and conditions are needed to substantiate the claim that ResiDual generalizes well.\n\n4.\tWhat are considered “noise” or “task-irrelevant” components in both models and downstream tasks? More clarity is needed.\n\n5.\tComparisons and statistical analyses are required to isolate the effect of spectral reweighting. It is unclear how much of the performance gain is directly attributable to the proposed method versus other factors.\n\n[1] Maxime Oquab et al. DINOv2: Learning Robust Visual Features without Supervision. Trans. Mach. Learn. Res. 2024.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies the geometric and spectral properties of attention head representations in vision transformers (ViTs), focusing on the specialized roles that certain heads play in multimodal (vision-language) models such as CLIP-like architectures. The authors show that these specialized attention heads reside in low-dimensional manifolds and can be characterized via principal components. Building on these insights, they introduce ResiDual, a spectral reweighting technique applied to the residual stream that aims to filter out noise components.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1.\tClearly define how zero-shot classification is evaluated when language embeddings are not present (e.g., in DINOv2).\n\n2.\tProvide ablations showing how changes in dataset composition and complexity affect both intrinsic dimensionality and specialization.", "broader_impact_concerns": "1.\tAmplifying certain principal components may reinforce socially or culturally biased features. The authors should note this risk and suggest bias-screening measures.\n\n2.\tBoosting particular components could narrow the model’s focus and overshadow other meaningful attributes, limiting semantic diversity.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ssPc3PPvY7", "reviewer": "Reviewer_SWCe", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strength\n- This paper is well-structured, with clear motivations and systematic empirical investigations. The experiments are methodically designed to validate each observation, leading to insightful discoveries.\n- The development of the ResiDual algorithm is well-justified through extensive empirical validation of how attention heads operate in low-dimensional manifolds and how their principal components behave across datasets.\n- The paper demonstrates ResiDual's effectiveness through comprehensive quantitative evaluation across multiple vision language models (CLIP, BLIP) and diverse datasets. The algorithm also requires fewer parameters than traditional fine-tuning approaches while maintaining comparable performance.\n\n### Weakness\n- There is a disconnect between the empirical analysis and algorithm design. While Section 3.1 thoroughly examines the geometric properties and specialization patterns of attention head representations, it does not extend this analysis to MLP layer residuals. Nevertheless, the proposed ResiDual algorithm operates on all residual components. This gap between the analytical foundation and implementation raises questions about the method's theoretical justification when applied to MLP layers.\n- In CLIP-like models for zero-shot classification, while the CLS token is primarily used for final predictions, the paper does not adequately address how ResiDual affects the residual flow through patch tokens and their subsequent influence on CLS token representations.\n- The observations in Section 3 examine head specialization in a fully intact network, yet the algorithm effectively down weights certain heads during operation. The superior performance of the optimized (O) versus supervised (S) selection methods suggests that head specialization patterns may change under scaling. Without visualization or analysis of how specialization is preserved, this aspect remains unclear.\n- The scope of the proposed method is relatively constrained, focusing specifically on zero-shot classification in CLIP-like models. While this represents an important use case, the limited exploration of broader applications leaves questions about the method's generalizability.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper examines how the residual stream influences specialization in transformer networks, particularly for zero-shot classification tasks. Through systematic analysis, the authors first investigate the geometric properties of attention head representations in Vision Transformers (ViTs) and characterize their emergent specializations. Their experiments reveal that these specialized representations, encoded in the principal components of attention heads, remain remarkably consistent across different data distributions. Leveraging this observation, the authors develop a metric to identify task-specific units in Vision-Language Models, demonstrating that even a simple greedy selection of these specialized units can outperform the full model output in zero-shot settings. Building on these insights, they propose ResiDual, an algorithm that strategically reweights the principal components of residual units. This approach achieves improved model performance while requiring significantly fewer parameters than traditional fine-tuning methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "To further strengthen the technical foundations of this work, I recommend addressing three critical aspects: establishing the relationship between MLP and attention head residuals, analyzing token-level information flow under ResiDual, and examining the preservation of head specialization patterns during optimization. These additions would enhance the soundness of the proposed method.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All the reviewers agreed that the paper is well-written and has provided comprehensive empirical investigations. The main concern, raised by reviewers SWCe and qKGp, is that the scope of the empirical observations found in this paper is limited to CLIP-like models under zero-shot classification settings. In the rebuttal, the authors have provided a new experiment in the appendix where unimodal vision encoders, such as ViT and DINO, are used, and the results show that the proposed method is effective in these settings as well. The reviewers have acknowledged this new experiment and are satisfied with the authors' response. I recommend accepting the paper as is.
Decision: Accept (poster)