Structure before the Machine: Input Space is the Prerequisite for Concepts

## Abstract

Abstract High-level representations have become a central focus in enhancing AI transparency and control, shifting attention from individual neurons or circuits to structured semantic directions that align with human-interpretable concepts. Motivated by the Linear Representation Hypothesis (LRH), we propose the Input-Space Linearity Hypothesis (ISLH), which posits that concept-aligned directions originate in the input space and are selectively amplified with increasing depth. We then introduce the Spectral Principal Path (SPP) framework, which formalizes how deep networks progressively distill linear representations along a small set of dominant spectral directions. Building on this framework, we further demonstrate the multimodal robustness of these representations in Vision-Language Models (VLMs). By bridging theoretical insights with empirical validation, this work advances a structured theory of representation formation in deep networks, paving the way for improving AI robustness, fairness, and transparency.

## 1 Introduction

Deep learning has achieved remarkable success across various domains, including computer vision (Krizhevsky et al., 2012), natural language processing (Devlin et al., 2019), and speech recognition (Hinton et al., 2012; Graves et al., 2013). However, the internal mechanisms of neural networks remain opaque. Despite advances in visualization and interpretability techniques, the transformation of inputs into high-level representations and the interactions among neurons are still not fully understood (Lipton, 2016; Doshi-Velez & Kim, 2017b; Ribeiro et al., 2016).
This lack of transparency leads to the characterization of neural networks as “black boxes” (Lipton, 2016; Doshi-Velez & Kim, 2017b), raising concerns about their reliability, particularly in high-stakes applications such as healthcare (Caruana et al., 2015), finance (Rudin, 2019), and law (Doshi-Velez & Kim, 2017a).

Previous works have demonstrated the potential of representations as a new perspective on AI transparency.
For example, neural networks trained to play chess exhibit internal representations of board positions and strategies (McGrath et al., 2022). Similarly, both generative and self-supervised models have been shown to develop emergent representations, such as semantic segmentation in vision tasks (Caron et al., 2021; Oquab et al., 2023).
Zou et al. (2023) further formalized Representation Engineering (RepE), emphasizing its ability to extract meaningful concepts from a model’s internal structure and control model behavior. RepE has emerged as a top-down approach to enhance the model transparency that focuses on representations rather than individual neurons or circuits, providing a more structured understanding of AI transparency and control. Another important contribution is the Linear Representation Hypothesis (LRH) (Park et al., 2023): as depth increases, task‑relevant concepts become nearly linearly separable in the model’s latent space, making them accessible with simple probes or linear edits.

Despite these promising advances, existing works on representations remain largely observational, relying on observed phenomena or intuitions. RepE uses contrastive pairs (e.g., honesty vs. dishonesty) to surface concept directions, but further theoretical work is needed to clarify why and how such directions emerge and remain coherent across layers. Similarly, LRH assumes linearity in embedding and unembedding spaces, yet offers limited insight into why representations become linearly organized. These approaches typically do not address how representations scale or propagate through deep networks, leaving a gap in our understanding of their robustness, generality, and theoretical foundations.

In this work, we move beyond linear observations by introducing Spectral Principal Path (SPP) that explains the emergence and stability of linear representations in deep networks. We show that representations propagate through a small number of spectral principal paths—directions aligned with large singular values at each layer. This structure naturally explains why concept directions remain stable and linearly accessible across layers, offering a theoretical foundation for both RepE and the Linear Representation Hypothesis. We further extend this analysis to Vision-Language Models (VLMs), demonstrating how spectral dynamics govern the interaction between visual and linguistic modalities. Our framework not only bridges theory and practice but also provides concrete tools to improve robustness and interpretability in multimodal AI systems.

Our main contributions are as follows:

- •
Input-Space Linearity Hypothesis (ISLH). We extend the Linear Representation Hypothesis beyond embedding and unembedding spaces to the input space itself, showing that concept directions can be traced backward to the input space.
- •
Spectral Principal Path (SPP). We propose a principled mechanism explaining how representations propagate and stabilize across layers via a small number of spectral principal paths—directions aligned with large singular vectors.
- •
Multimodal robustness of representations. We evaluate Representation Engineering in VLMs and demonstrate that linearly organized concept representations remain robust across modalities. This provides the first empirical validation of RepE’s scalability in multimodal systems and supports the generality of spectral structure.

## 2 Related Works

### 2.1 Representations in Neural Networks

Early work on word embeddings shows that neural networks can learn distributed representations that encode semantic relationships and compositional structures (Mikolov et al., 2013). Follow-up studies (Schramowski et al., 2019; Radford et al., 2015) further reveal that learned embeddings can implicitly encode abstract dimensions such as commonsense morality, even without explicit supervision. For instance, Radford et al. (Radford et al., 2015) observe that training a language model on product reviews results in the emergence of a sentiment-tracking neuron.

This phenomenon is not unique to language models. McGrath et al. (2022) show that similar internal representations can be found in networks trained to play chess. In computer vision, recent studies (Caron et al., 2021; Oquab et al., 2023) demonstrate that both generative and self-supervised training objectives give rise to emergent semantic representations, such as those useful for segmentation tasks, suggesting the emergence of representations is a general property of deep learning systems.

Building on this, Zou et al. (2023) propose techniques to read and control these internal structures, including Linear Artificial Tomography (LAT) for extracting concept-aligned representations and methods for steering model behavior. Their study shows that RepE-style approaches can be used not only to detect but also to manipulate emergent properties, motivating more systematic efforts to characterize and intervene in high-level model behaviors. Theoretically, Park et al. (2023) proposes the Linear Representation Hypothesis: task‑relevant concepts become nearly linearly separable in the model’s latent space.

### 2.2 Approaches to Interpretability

Traditional interpretability techniques have focused on methods like saliency maps (Simonyan et al., 2013; Springenberg et al., 2014; Zeiler & Fergus, 2014; Zhou et al., 2016), feature visualization (Szegedy et al., 2013; Zeiler & Fergus, 2014) and mechanistic interpretability (Olah et al., 2020; Olsson et al., 2022; Lieberum et al., 2023). Saliency maps (Simonyan et al., 2013) highlight important input regions by tracking gradients or activation values, yet they are often unstable and provide limited insight into the distributed nature of representations. Similarly, feature visualizations (Szegedy et al., 2013; Zeiler & Fergus, 2014) optimize inputs to activate specific neurons, but they may overlook the global structure of the emergent representations. Mechanistic interpretability (Zou et al., 2023) seeks to fully reverse engineer neural networks into their “source code”, but the considerable manual effort and the difficulty of theoretically explaining neural networks as discrete circuits hinder their explainability.

In contrast, recent advances in interpretability have shifted the focus toward analyzing representation spaces. This top-down approach seeks to uncover high-level semantic directions that correspond to complex phenomena such as honesty, fairness, or bias. By extracting and analyzing these internal representations, researchers have opened new avenues to understand how large-scale AI models encode and preserve crucial information across layers, leading to more robust and interpretable AI systems.

## 3 Preliminaries

Linear Representation Hypothesis (Park et al., 2023)  We consider a co ncept $W$ that has a *linear representation* in a model if there exists a vector $\bar{\gamma}_{W}$ in the unembedding space $\Gamma$ and a vector $\bar{\lambda}_{W}$ in the embedding space $\Lambda$ such that for any counterfactual pair $(Y(W=0),Y(W=1))$,

$$ $\displaystyle\gamma(Y(W=1))-\gamma(Y(W=0))\in\mathrm{Cone}(\bar{\gamma}_{W}),$ (1) $$

and for any context pair $(\lambda_{0},\lambda_{1})$ that changes only $W$ and not other causally separable concepts,

$$ $\displaystyle\lambda_{1}-\lambda_{0}\in\mathrm{Cone}(\bar{\lambda}_{W}).$ (2) $$

where $\mathrm{Cone}(\mathbf{v})=\left\{\alpha\mathbf{v}\;:\;\alpha>0\right\}$, the embedding space is where input contexts are mapped to high-dimensional vectors before processing, capturing the model’s internal representation of the input. The unembedding space is where each output token is represented, and predictions are made by computing inner products between input embeddings and output unembedding vectors. Unless stated otherwise, all discussions pertain to the embedding space $\Lambda$, as our goal is to trace how input linearity propagates through the network.

Figure: Figure 1: Overview of the Spectral Principal Path framework: the illustration shows how deep networks extract and amplify concept-relevant directions from the input, culminating the linear representation hypothesis. $\bar{x}_{W}$ is the concept discriminative direction in the input space, $\bar{\mathbf{x}}_{Wk}^{\prime}$ are spurious directions, $\bar{\gamma}_{W}$ is the direction vector in the unembedding space, and the spectral path $\{i^{*}_{1},\dots,i^{*}_{L}\}$ is constructed by selecting the top singular direction at each layer via SVD on weight matrices.
Refer to caption: fig/overview.png

## 4 Spectral Principal Path Framework

The overview of the Spectral Principal Path framework is shown in Fig. [1](https://arxiv.org/html/2506.08543v2#S3.F1). In the input space, the concept direction $\bar{\mathbf{x}}_{W}$ separates inputs with contrast concepts such as “honest” and “untruthful”. As activations propagate through the network, layer-wise SVD identifies spectral components, forming spectral principal paths $\mathcal{P}_{c}$. These dominant paths progressively amplify concept-relevant signals, leading to output representations linearly aligned with $\bar{\gamma}_{W}$.

### 4.1 Input-Space Linearity Hypothesis

Inspired by LRH, which uncovers linear concept axes in embedding and unembedding spaces, we take one step further and ask whether such axes already reside in the *raw input space*.
Input-Space Linearity Hypothesis assumes that, in the *raw input space* $x\in\Upsilon$, there exists a discriminative direction $\bar{\mathbf{x}}_{W}$ such that

$$ $\displaystyle\mathbb{E}[x\mid W\!=\!1]-\mathbb{E}[x\mid W\!=\!0]\in\mathrm{Cone}(\bar{\mathbf{x}}_{W}),$ (3) $$

yet each sample is an entangled mixture

$$ $\displaystyle x^{(i)}=\alpha_{i}\bar{\mathbf{x}}_{W}+\sum_{k=1}^{r}\beta_{i,k}\bar{\mathbf{x}}_{Wk}^{\prime}+\varepsilon_{i},$ (4) $$

where $\bar{\mathbf{x}}_{Wk}^{\prime}$ are spurious directions and $\varepsilon_{i}$ is residual noise.
ISLH states that for any intervention flipping only $W$, the induced input difference satisfies $\operatorname{Cone}(\bar{\mathbf{x}}_{W})$.

ISLH pinpoints the origin of linearity by showing that concept axes already reside in raw input coordinates and are merely recovered and amplified during training; where training can be viewed as a noise-suppression process, where spectral principal paths with large singular values progressively dampen spurious components $\beta_{i,k}\bar{\mathbf{x}}_{Wk}^{\prime}$; and, by grounding linearity at the input level, it becomes inherently modality‑agnostic, extending Representation Engineering to multimodal models whose raw signals already encode task‑relevant contrasts. Next, we will dive into the connection between ISLH and LRH:

###### Theorem 4.1 (ISLH sufficiency) .

If the network satisfies the *Input-Space Linearity Hypothesis* (ISLH), and the representation dominates the cumulative gain $G(\mathcal{P})$ (shown in ([12](https://arxiv.org/html/2506.08543v2#S4.E12))), then its deep representations satisfy the
*Linear Representation Hypothesis* (LRH);
that is, concept classes become linearly separable in the latent space.

The proof is given in Appendix [A.1](https://arxiv.org/html/2506.08543v2#A1.SS1).

###### Theorem 4.1 (ISLH sufficiency) .

### 4.2 Spectral Principal Path

We are now asking how such concept directions propagate through the network. While ISLH posits that concept-aligned directions already exist in the raw input space, it does not yet explain why these directions persist and become more prominent across layers. To address this, we introduce the *Spectral Principal Path* (SPP) framework, which shows that representations are distilled through a small set of principal spectral paths aligned with large singular vectors at each layer. This framework formalizes how ISLH leads to the emergence of the Linear Representation Hypothesis (LRH), providing a unified and mechanistically grounded view of representation stability.

Specifically, consider a generalized network

$$ $\displaystyle f_{L}(x)=W_{L}W_{L-1}\cdots W_{1}x\;\equiv\;Mx,\qquad W_{l}\in\mathbb{R}^{d_{l}\times d_{l-1}},$ (5) $$

according to LRH, there exists a representation direction $\bar{\lambda}_{W}$, where the neural activity $f(x)$ can be linearly projected into that direction, formulating a representation score:

$$ $\displaystyle s(x)=\langle\bar{\lambda}_{W},\;f_{L}(x)\rangle=\bar{\lambda}_{W}^{\!\top}Mx,\qquad\bar{\lambda}_{W}\in\mathbb{R}^{d_{L}}.$ (6) $$

While our theoretical formulation assumes a purely stacked linear architecture, we show our extension to residual connections and attention mechanisms. We provide a detailed discussion of these extensions in Appendix [A.2.1](https://arxiv.org/html/2506.08543v2#A1.SS2.SSS1).

Next we will calculate the back‑propagated gradient of $s$
using the chain rule,

$$ $\displaystyle\nabla_{x}s\;=\;\Biggl(\prod_{l=1}^{L}\nabla f_{{l}\rightarrow{(l-1)}}\Biggr)^{\!\top}\;\bar{\lambda}_{W},$ (7) $$

$$ $\displaystyle\nabla f_{{l}\rightarrow{(l-1)}}\;=\;W_{l}\;+\;\sum_{k}f_{l-1,k}\,\frac{\partial W_{l}}{\partial f_{l-1,k}}.$ (8) $$

To make the structure of the layer-wise Jacobian in ([8](https://arxiv.org/html/2506.08543v2#S4.E8)) explicit, we regard the gradient
$\nabla f_{l\to(l-1)}$ as a matrix and apply its compact singular-value decomposition (SVD);
this yields

$$ $\displaystyle\nabla f_{{l}\rightarrow{(l-1)}}=U^{(l)}\Sigma^{(l)}V^{(l)\!\top},\quad\Sigma^{(l)}=\operatorname{diag}\bigl(\sigma^{(l)}_{1},\dots,\sigma^{(l)}_{r_{l}}\bigr),$ (9) $$

therefore

$$ $\displaystyle\nabla_{x}s\;=\;V^{(1)}\Sigma^{(1)}U^{(1)\!\top}\cdots V^{(L)}\Sigma^{(L)}U^{(L)\!\top}\bar{\lambda}_{W}.$ (10) $$

Unfolding the matrix products yields

$$ $\displaystyle\nabla_{x}s\;=\;\sum_{i_{1},\dots,i_{L}}\Bigl(\prod_{l=1}^{L}\sigma^{(l)}_{i_{l}}\Bigr)V^{(1)}_{\cdot i_{1}}\Bigl(\prod_{l=1}^{L-1}\langle u_{i_{l}}^{(l)},\;V^{(l+1)}_{\cdot i_{l+1}}\rangle\Bigr)\langle u_{i_{L}}^{(L)},\;\bar{\lambda}_{W}\rangle\;,$ (11) $$

where $\sigma_{i_{l}}^{(l)}$ is the signular value within the $\Sigma^{(l)}$ matrix, $u_{i_{l}}^{(l)}$ (resp. $V_{\cdot i_{l}}^{(l)}$) is the $i_{l}$-th left (resp. right) singular vector, and $\cdot i_{l}$ here means select the $i_{l}$-th column. Therefore equation [11](https://arxiv.org/html/2506.08543v2#S4.E11) is dominated by paths whose cumulative gain is largest. Formally, we define Spectral Principal Path as follows:

###### Definition 4.1 (Spectral Principal Path) .

Given the Jacobian decomposition across $L$ layers $\nabla_{x}s$, each spectral path $\mathcal{P}=(i_{1},\ldots,i_{L})$ contributes a cumulative gain given by

$$ $\displaystyle G(\mathcal{P}):=\Bigl(\prod_{l=1}^{L}\sigma^{(l)}_{i_{l}}\Bigr)V^{(1)}_{\cdot i_{1}}\Bigl(\prod_{l=1}^{L-1}\langle u_{i_{l}}^{(l)},\;V^{(l+1)}_{\cdot i_{l+1}}\rangle\Bigr)\langle u_{i_{L}}^{(L)},\;\bar{\lambda}_{W}\rangle\;,$ (12) $$

the Spectral Principal Path is defined as $\mathcal{P}_{c}=(i_{1}^{\star},\ldots,i_{L}^{\star})$ that maximizes the cumulative gain:

$$ $\displaystyle\mathcal{P}_{c}=(i_{1}^{\star},\ldots,i_{L}^{\star}):=\arg\max_{(i_{1},\ldots,i_{L})}G(\mathcal{P}).$ (13) $$

###### Definition 4.1 (Spectral Principal Path) .

### 4.3 Connection between ISLH and SPP

To clarify how information specified by ISLH is propagated along an SPP, we introduce the notion of *spectral similarity* for a given spectral path $(i_{1},\dots,i_{L})$:

###### Definition 4.2 (Spectral Similarity) .

For two consecutive layers $l$ and $l{+}1$ in the unfolded Jacobian, and for indices
$(i_{l},i_{l+1})$, we define the
*spectral similarity at layer $l$* as

$$ $\displaystyle\Theta(i_{l},i_{l+1})\;:=\;\bigl\langle u_{i_{l}}^{(l)},\,V^{(l+1)}_{\!\cdot\,i_{l+1}}\bigr\rangle,\qquad l=1,\dots,L-1.$ (14) $$

This quantity measures how well the $i_{l}$-th spectral component of layer $l$
aligns with the $i_{l+1}$-th spectral component that enters layer $l{+}1$.

Empirically, we observe two coupled effects as the network reaches deeper.

- 1.
Stabilization of singular vectors.
As demonstrated in Fig. [2](https://arxiv.org/html/2506.08543v2#S5.F2), the principal singular vectors $u_{i_{*}}^{(l)}$ change only marginally with $f_{l}$.
Consequently, the spectral similarity of a few paths
approaches $1$, the stability of singular vectors implies that spectral similarity remains very high in deeper layers, allowing information to propagate consistently along those paths with high spectral similarity.
- 2.
Selective growth of singular values.
As demonstrated in Fig. [4](https://arxiv.org/html/2506.08543v2#S5.F4), we observe that the singular values are growing as the layers deepen; and at the same depths only a very small subset of singular
values $\sigma_{i_{*}}^{(l)}$ are amplified; the remainder
stay close to their initial scale.

Putting these observations together shows that, in deep layers, the directions that (i) possess *large spectral similarity*
and (ii) carry *large singular values* coincide, which satisfies dominant $G(\mathcal{P})$ condition.
In other words, the network progressively funnels
representation power into precisely those spectral directions
that stay *globally aligned* across layers.

According to Theorem [4.1](https://arxiv.org/html/2506.08543v2#S4.Thmtheorem1), this behaviour is exactly what the Input-Space Linearity Hypothesis (ISLH) predicts: concept-carrying directions
are expected to form a low-dimensional subspace that is both
*spectrally dominant* (large $\sigma_{i_{*}}^{(l)}$) and
*structurally coherent* (large $\Theta$) throughout the
hierarchy, leading to LRH.
Hence, the emergence of a handful of
high-$\sigma$, high-similarity principal paths in SPP
provides concrete spectral evidence in favour of the ISLH
assumption.

###### Definition 4.2 (Spectral Similarity) .

## 5 Experiments

### 5.1 Experiment Setup

Dataset: We conduct our experiments on the Microsoft COCO (Common Objects in Context) dataset (Lin et al., 2014), a large-scale benchmark for vision-language tasks. COCO contains over 330K images, each with five human-annotated captions, covering diverse real-world scenes.

VLM: We employ Idefics2-8B (Laurençon et al., 2023; 2024), a state-of-the-art VLM that extends the LLaMA architecture with a vision encoder, enabling multimodal reasoning over images and text. Idefics2-8B is designed for instruction-following, multimodal dialogue, and grounded language generation, making it an ideal candidate for studying conceptual representations in VLMs.

Figure: Figure 2: The polar plot demonstrates normalized connections between principal singular vector and $f_{l}(x)$, where the number indicates their cosine similarity. The results showcase that $f_{l}(x)$, especially in later layers, is very similar to the principal singular vector of that layer.
Refer to caption: fig/pic3.png

### 5.2 The Alignment between Principal Singular Vector and f l ​ ( x ) f_{l}(x)

Fig. [2](https://arxiv.org/html/2506.08543v2#S5.F2) visualizes the connections between the principal singular vector, i.e., the singular vector with the largest singular value, and $f_{l}(x)$. The results reveal a strong alignment between the principal singular vector and $f_{l}(x)$, with their cosine similarity over 0.875. This experimental validation supports our theoretical claim that singular vectors with large singular values remain stable across layers, reinforcing their stability of spectral value.

Figure: Figure 3: The singular value rank across layers.
Refer to caption: fig/pic5.png

### 5.3 Spectral Energy Concentration Across Layers

To investigate how spectral energy propagates through the network, we analyze the singular value spectrum of the layer-wise Jacobians. Fig. [4](https://arxiv.org/html/2506.08543v2#S5.F4) presents a heatmap of the singular value magnitudes across all layers. The x-axis indicates the layer index, and the y-axis corresponds to the ordered singular value indices. Color intensity reflects the magnitude of each singular value.

We observe that the singular values are growing as the layers deepen, and at the same depths, only a very small subset of singular values are amplified; the remainder stay close to their initial scale.

These results indicate that, with increasing depth, spectral energy becomes increasingly concentrated in a few dominant directions. Combined with the theoretical formulation in Section [4.3](https://arxiv.org/html/2506.08543v2#S4.SS3), this supports the hypothesis that high-magnitude spectral components dominate SPPs.

### 5.4 Inter-Layer Spectral Similarity of λ ¯ W \bar{\lambda}_{W}

We further analyze the alignment of concept-carrying directions across adjacent layers by computing the average cosine similarity between the projections of $\bar{\lambda}_{W}$ at different layers. This measures how stable the representation direction remains as it propagates backward through the network. The results are shown in Fig. [4](https://arxiv.org/html/2506.08543v2#S5.F4).

The curve demonstrates a clear upward trend: the inter-layer similarity of $\bar{\lambda}_{W}$ increases consistently with network depth, eventually approaching a value near 0.95 in the final layers. This suggests that the concept direction stabilizes as it propagates through deeper layers, aligning with the intuition of structured and coherent representation flow.

### 5.5 Multimodal Robustness of Representation

In this experiment, we explore the multimodal robustness of representation. Specifically, we analyze how VLMs encode fairness and honesty, and how these concepts persist or transform as information propagates through the model. These findings deepen our understanding of how representations enhance both interpretability and conceptual alignment in the context of multimodal reasoning.

#### 5.5.1 Evaluating Honesty and Fairness in VLMs

To evaluate how well VLMs represent abstract ethical concepts, we analyze their handling of honesty and fairness in multimodal response generation, shown in Fig. [5](https://arxiv.org/html/2506.08543v2#S5.F5). These concepts are critical for reducing misinformation and bias and serve as strong test cases for examining interpretability and ethical alignment in large-scale models.
To quantify this process, we compute token-wise projection scores following RepE (Zou et al., 2023), measuring how closely activations align with concept directions at each layer. These results highlight the structured nature of ethical concept encoding in VLMs and support our broader claims about representation flow along spectral directions.

Our method successfully identifies distinct conceptual behaviors within the model: when the VLM produces dishonest or unfair responses, token-wise projection scores show clear drops (red regions), in contrast to the consistently high scores observed for honest or fair cases. Such findings provide strong evidence that abstract ethical dimensions like honesty and fairness are internally structured and traceable, enabling targeted representational interventions to mitigate misinformation and bias in multimodal reasoning systems.

Figure: (a) VLM response for the concept of honesty, prompted with an image of the Golden Gate Bridge.
Refer to caption: fig/honesty.png

Figure: Figure 6: Temporal LAT Scans for Honesty, Power, Fearlessness, and Fairness. The left heatmap represents the LAT Scan when the VLM aligns with the concept, while the right heatmap corresponds to the opposing concept. The horizontal axis denotes token position, and the vertical axis represents VLM layers. Blue indicates high alignment, whereas red represents low alignment.
Refer to caption: fig/LAT.png

#### 5.5.2 LAT Scans for High-level Representations

While cosine similarity and token-wise scores offer localized insights into concept alignment, they provide only a static, layer-agnostic view of internal representations. To capture how high-level concepts evolve and propagate through the model, we employ Linear Attribution Tomography (LAT)  (Zou et al., 2023), which enables layer-wise visualization of conceptual information flow. LAT works by projecting hidden activations onto predefined concept subspaces, producing interpretable activation maps across layers and tokens. This perspective complements prior analyses and supports our broader goal of understanding concept representation shaped by low-rank spectral structure.

We apply LAT to VLMs to examine how abstract concepts, including honesty, fairness, power, and fearlessness, are internally encoded and transformed. For each concept, we design controlled prompts that elicit either aligned or misaligned responses (e.g., honest vs. dishonest). Fig. [6](https://arxiv.org/html/2506.08543v2#S5.F6) shows the resulting LAT scans, where heatmaps visualize token-wise projection scores across layers. Blue regions indicate strong alignment with the concept, while red regions highlight divergence.

The scans reveal concept-specific propagation patterns. Honesty and fairness exhibit stable trajectories under aligned prompts but greater dispersion and deviation under misaligned ones. Power appears concentrated in later layers, while fearlessness shows early-layer changes. These results are well explained by the SPP framework, indicating that concepts are transmitted through the network via a small set of dominant spectral directions.
The consistency of these representations across modalities further demonstrates the robustness of RepE, and their traceability back to the input can be explained by ISLH, where input concept directions exist and are entangled with mixture. The experiment reinforces the generality of spectral structure in multimodal models.

## 6 Conclusion

This work presents a unified spectral framework that grounds the emergence and stability of high-level representations in deep networks. By introducing the Spectral Principal Path (SPP) framework, we reveal that concept-aligned representations are funneled through a small number of paths with both large singular values and strong inter-layer alignment. We formally connect this to the Input-Space Linearity Hypothesis (ISLH), showing that such spectral dominance is sufficient to guarantee linear separability in the latent space—thereby validating the Linear Representation Hypothesis (LRH).
Empirically, we demonstrate that these dominant spectral paths not only persist across layers but also preserve concept information in multimodal settings, such as vision-language models. Our results suggest that representational stability is not an emergent coincidence but a consequence of spectral dynamics founded in the input space and structured by learning.

While promising, our current framework is subject to several limitations. Primarily, the theoretical claims rest on ISLH, which requires further empirical validation and deeper theoretical grounding.
Future work could investigate how optimization dynamics such as In-Context Learning (ICL) and Supervised Fine-Tuning (SFT) interact with singular value distributions, which may lead to a more complete theory of representation learning. Another important direction for future work is to go beyond the structural characterization of representations and investigate how such spectral patterns emerge during training. Ultimately, understanding the spectral geometry of optimization could help bridge the gap between abstract representation theory and practical model training.

## Ethics Statement

This work focuses on advancing theoretical and empirical understanding of representation learning in deep neural networks. No new data were collected for this study; all experiments were conducted on publicly available datasets (e.g., Microsoft COCO (Lin et al., 2014)). These datasets are widely used in the research community and come with established licenses for academic use.

We emphasize that our framework, while aiming to improve transparency and interpretability of large-scale models, could also be applied in high-stakes domains such as healthcare, finance, or legal decision-making. In such contexts, careful human oversight and ethical evaluation are necessary to avoid potential misuse or over-reliance on automated systems. Additionally, we acknowledge that concepts such as fairness and honesty, which are probed in our multimodal experiments, are inherently socio-cultural and context-dependent. Our analysis is intended as a scientific investigation of representational properties rather than a normative definition of these values.

All contributions of this work are scientific in nature, and we believe that the methods and results presented do not pose foreseeable risks of harm when used responsibly within research environments.

## Reproducibility Statement

We are committed to ensuring the reproducibility of our results. To this end, we provide the following:

- •
Theoretical Derivations: All theorems, definitions, and proofs are included in the main paper and appendix (see Appendix A), offering a complete mathematical foundation for our framework.
- •
Experimental Setup: Detailed descriptions of datasets (Section 5.1), models (Idefics2-8B), and evaluation metrics (Sections 5.2–5.5) are provided. Hyperparameters and preprocessing steps are explicitly specified.
- •
Code Availability: We attach our code in the supplementary material. We will release the full implementation, including spectral decomposition routines, representation analysis scripts, and visualization tools, upon publication. This will allow other researchers to reproduce all figures and quantitative results.
- •
Data Accessibility: All datasets used (e.g., Microsoft COCO) are publicly available, ensuring no barriers to replication.

Together, these materials provide sufficient detail for independent researchers to reproduce our findings and extend our work in related directions.

## References

- Caron et al. (2021)
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin.
Emerging properties in self-supervised vision transformers.
In *Proceedings of the IEEE/CVF international conference on computer vision*, pp. 9650–9660, 2021.
- Caruana et al. (2015)
Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad.
Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission.
In *Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD)*, pp. 1721–1730. ACM, 2015.
- Corbett-Davies et al. (2023)
Sam Corbett-Davies, Johann D Gaebler, Hamed Nilforoshan, Ravi Shroff, and Sharad Goel.
The measure and mismeasure of fairness.
*Journal of Machine Learning Research*, 24(312):1–117, 2023.
- Devlin et al. (2019)
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.
Bert: Pre-training of deep bidirectional transformers for language understanding.
In *Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT)*, pp. 4171–4186, 2019.
- Doshi-Velez & Kim (2017a)
Finale Doshi-Velez and Been Kim.
Towards a rigorous science of interpretable machine learning.
*arXiv preprint arXiv:1702.08608*, 2017a.
- Doshi-Velez & Kim (2017b)
Finale Doshi-Velez and Been Kim.
Towards a rigorous science of interpretable machine learning.
*arXiv preprint arXiv:1702.08608*, 2017b.
- French & Raven (1959)
John R. P. French and Bertram Raven.
The bases of social power.
In Dorwin Cartwright (ed.), *Studies in Social Power*, pp. 150–167. University of Michigan Press, 1959.
- Graves et al. (2013)
Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton.
Speech recognition with deep recurrent neural networks.
In *IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*, pp. 6645–6649. IEEE, 2013.
- Hinton et al. (2012)
Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara Sainath, et al.
Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups.
*IEEE Signal Processing Magazine*, 29(6):82–97, 2012.
- Krizhevsky et al. (2012)
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton.
Imagenet classification with deep convolutional neural networks.
In *Advances in Neural Information Processing Systems (NeurIPS)*, volume 25, 2012.
- Laurençon et al. (2023)
Hugo Laurençon, Lucile Saulnier, Léo Tronchon, Stas Bekman, Amanpreet Singh, Anton Lozhkov, Thomas Wang, Siddharth Karamcheti, Alexander M. Rush, Douwe Kiela, Matthieu Cord, and Victor Sanh.
Obelics: An open web-scale filtered dataset of interleaved image-text documents, 2023.
- Laurençon et al. (2024)
Hugo Laurençon, Léo Tronchon, Matthieu Cord, and Victor Sanh.
What matters when building vision-language models?, 2024.
- Lieberum et al. (2023)
Tom Lieberum, Matthew Rahtz, János Kramár, Neel Nanda, Geoffrey Irving, Rohin Shah, and Vladimir Mikulik.
Does circuit analysis interpretability scale? evidence from multiple choice capabilities in chinchilla.
*arXiv preprint arXiv:2307.09458*, 2023.
- Lilienfeld & Andrews (1996)
Scott O. Lilienfeld and Bridget P. Andrews.
Development and preliminary validation of a self-report measure of psychopathic personality traits in noncriminal populations.
*Journal of Personality Assessment*, 66(3):488–524, 1996.
- Lin et al. (2021)
Stephanie Lin, Jacob Hilton, and Owain Evans.
Truthfulqa: Measuring how models mimic human falsehoods.
*arXiv preprint arXiv:2109.07958*, 2021.
- Lin et al. (2014)
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.
Microsoft coco: Common objects in context.
In *Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 13*, pp. 740–755. Springer, 2014.
- Lipton (2016)
Zachary C. Lipton.
The mythos of model interpretability.
*arXiv preprint arXiv:1606.03490*, 2016.
- McGrath et al. (2022)
Thomas McGrath, Andrei Kapishnikov, Nenad Tomašev, Adam Pearce, Martin Wattenberg, Demis Hassabis, Been Kim, Ulrich Paquet, and Vladimir Kramnik.
Acquisition of chess knowledge in alphazero.
*Proceedings of the National Academy of Sciences*, 119(47):e2206625119, 2022.
- Mikolov et al. (2013)
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean.
Distributed representations of words and phrases and their compositionality.
In *Advances in Neural Information Processing Systems*, 2013.
- Olah et al. (2020)
Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter.
Zoom in: An introduction to circuits.
*Distill*, 5(3):e00024–001, 2020.
- Olsson et al. (2022)
Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al.
In-context learning and induction heads.
*arXiv preprint arXiv:2209.11895*, 2022.
- Oquab et al. (2023)
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al.
Dinov2: Learning robust visual features without supervision.
*arXiv preprint arXiv:2304.07193*, 2023.
- Park et al. (2023)
Kiho Park, Yo Joong Choe, and Victor Veitch.
The linear representation hypothesis and the geometry of large language models.
*arXiv preprint arXiv:2311.03658*, 2023.
- Radford et al. (2015)
Alec Radford, Luke Metz, and Soumith Chintala.
Unsupervised representation learning with deep convolutional generative adversarial networks.
*arXiv preprint arXiv:1511.06434*, 2015.
- Ribeiro et al. (2016)
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin.
”why should i trust you?”: Explaining the predictions of any classifier.
In *Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining*, pp. 1135–1144, 2016.
- Rudin (2019)
Cynthia Rudin.
Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.
*Nature Machine Intelligence*, 1(5):206–215, 2019.
- Schramowski et al. (2019)
Patrick Schramowski, Cigdem Turan, Sophie Jentzsch, Constantin Rothkopf, and Kristian Kersting.
Bert has a moral compass: Improvements of ethical and moral values of machines.
*arXiv preprint arXiv:1912.05238*, 2019.
- Simonyan et al. (2013)
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman.
Deep inside convolutional networks: Visualising image classification models and saliency maps.
In *arXiv preprint arXiv:1312.6034*, 2013.
- Springenberg et al. (2014)
Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller.
Striving for simplicity: The all convolutional net.
*arXiv preprint arXiv:1412.6806*, 2014.
- Szegedy et al. (2013)
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus.
Intriguing properties of neural networks.
*arXiv preprint arXiv:1312.6199*, 2013.
- Zeiler & Fergus (2014)
Matthew D Zeiler and Rob Fergus.
Visualizing and understanding convolutional networks.
In *European Conference on Computer Vision*, pp. 818–833. Springer, 2014.
- Zhou et al. (2016)
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba.
Learning deep features for discriminative localization.
In *Proceedings of the IEEE conference on computer vision and pattern recognition*, pp. 2921–2929, 2016.
- Zou et al. (2023)
Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al.
Representation engineering: A top-down approach to ai transparency.
*arXiv preprint arXiv:2310.01405*, 2023.

## Appendix A Appendix

### A.1 Proofs

#### A.1.1 Proof of Theorem 1: ISLH sufficiency

###### Theorem A.1 (ISLH sufficiency) .

If the network satisfies the *Input-Space Linearity Hypothesis* (ISLH), and the representation dominates the cumulative gain $G(\mathcal{P})$ (shown in ([12](https://arxiv.org/html/2506.08543v2#S4.E12))), then its deep representations satisfy the
*Linear Representation Hypothesis* (LRH);
that is, concept classes become linearly separable in the latent space.

Proof. For every layer $W_{l}$ with compact SVD

$$ $\displaystyle W_{l}=U^{(l)}\Sigma^{(l)}V^{(l)\!\top},\qquad\Sigma^{(l)}=\operatorname{diag}(\sigma^{(l)}_{1},\dots,\sigma^{(l)}_{r_{l}}),$ (15) $$

Equation ([12](https://arxiv.org/html/2506.08543v2#S4.E12)) in Section 4.2 shows that each spectral path
$\mathcal{P}=(i_{1},\dots,i_{L})$ contributes a weight

$$ $\displaystyle G(\mathcal{P})=\Bigl(\prod_{l=1}^{L}\sigma^{(l)}_{i_{l}}\Bigr)V^{(1)}_{\cdot i_{1}}\Bigl(\prod_{l=1}^{L-1}\langle u_{i_{l}}^{(l)},\;V^{(l+1)}_{\cdot i_{l+1}}\rangle\Bigr)\langle u_{i_{L}}^{(L)},\;\bar{\lambda}_{W}\rangle\;,$ (16) $$

Let $\mathcal{P}_{\!c}=(i_{1}^{\star},\dots,i_{L}^{\star})$ be the concept path,
and $\mathcal{P}_{\!n}$ any other path. On condition that the representation dominates the cumulative gain $G(\mathcal{P})$ such that,

$$ $\displaystyle\frac{G(\mathcal{P}_{\!n})}{G(\mathcal{P}_{\!c})}\;\leq\;\rho^{-L},$ (17) $$

where $\rho>1$ is a fixed amplification margin between the concept singular value
$\sigma^{(l)}_{c}$ and all other (noise) singular values. Since each ratio $\sigma^{(l)}_{i_{l}}/\sigma^{(l)}_{c}\leq 1/\rho$.
Inter-layer alignments and concept alignment can only decrease this ratio
further.

As depth $L$ grows, ([17](https://arxiv.org/html/2506.08543v2#A1.E17)) yields

$$ $\displaystyle\frac{G(\mathcal{P}_{\!n})}{G(\mathcal{P}_{\!c})}\;\xrightarrow{L\to\infty}\;0.$ (18) $$

Hence almost all gradient—and therefore almost all representation energy—
flows along $\mathcal{P}_{\!c}$, forcing the deep hidden state

$$ $\displaystyle f_{L}=W_{L}\cdots W_{1}x\quad\text{to lie almost entirely in}\ \operatorname{Span}\{\bar{\mathbf{x}}_{W}\},$ (19) $$

where $\operatorname{Span}\{\bar{\mathbf{x}}\}=\{c\cdot\bar{\mathbf{x}}\mid c\in\mathbb{R}\}$. Different samples now differ only by a scalar coefficient on the same vector,
so a single linear separator can classify them perfectly: this is exactly the
Linear Representation Hypothesis (LRH).

###### Theorem A.1 (ISLH sufficiency) .

### A.2 Theoretical Justification

#### A.2.1 Extension to Residual and Attention Mechanisms

While our theoretical framework is derived from stacked linear layers, we show that it naturally extends to modern architectures such as Transformer blocks, which include residual connections and attention mechanisms.

##### Residual connections.

In architectures with skip connections, each layer computes
$f_{l}=f_{l-1}+W_{l}f_{l-1}$, which can be rewritten as
$f_{l}=(I+W_{l})f_{l-1}$.
This effectively creates a mixture of identity and learned transformations.
Unrolling the composition yields an ensemble of spectral paths—some that pass through $W_{l}$, and others that skip it via $I$.
While the total number of paths increases exponentially, our theory still applies:
as long as the dominant singular values of $W_{l}$ grow sufficiently during training, the spectral path with maximal cumulative gain still dominates.
Thus, the residual structure enhances the expressivity but preserves the spectral filtering effect.

##### Attention mechanisms.

To stay consistent with our framework—where every layer is a
matrix acting from the left on the input $x$—we first recall
the standard formulation and then cast the resulting
attention matrix into the same “$W$-matrix” form.

Let
$\mathbf{Q}=XW_{\mathbf{Q}},\;\mathbf{K}=XW_{\mathbf{K}},\;\mathbf{V}=XW_{\mathbf{V}},$
with $X\!\in\!\mathbb{R}^{n\times d}$.
The dot-product attention output is

$$ $\displaystyle f(x)\;=\;\underbrace{\text{softmax}\!\left(\frac{\mathbf{Q}\mathbf{K}^{\!\top}}{\sqrt{d}}\right)}_{\mathbf{A}(x)\;\in\;\mathbb{R}^{n\times n}}\;\cdot\;\mathbf{V}.$ (20) $$

Here the attention weight $\mathbf{A}(x)\!\in\!\mathbb{R}^{n\times n}$ acts on the input matrix, whereas the value projection
$\mathbf{V}=XW_{\mathbf{V}}$ is obtained by a *right*-multiplication of $X\in\mathbb{R}^{n\times d}$.
Consequently, the complete attention block cannot be reduced to a
single left-acting matrix without additional assumptions:

$$ $\displaystyle f(X)\;=\;\mathbf{A}(x)\,\bigl(XW_{\mathbf{V}}\bigr)\;\neq\;W_{\text{attn}}(x)\,X$ (21) $$

The mixed left / right structure means that the set of vectors reachable
by $\mathbf{A}(x)\,$ differs from that spanned by $W_{\mathbf{V}}$, so the spectral behaviour
of the composite operator is not covered by the current linear-chain
analysis. Nevertheless, our empirical results (Section [5.2](https://arxiv.org/html/2506.08543v2#S5.SS2))
show that the dominant singular vector of $\mathbf{A}(x)$ still align with the
concept axis $\bar{\mathbf{x}}$, indicating that the principal-path
intuition remains informative.

### A.3 Evaluating Fearlessness and Power in VLMs

To further evaluate the robustness of representations for high-level concepts, we expand our analysis from honesty and fairness to encompass fearlessness and power. Like honesty and fairness, these concepts are abstract and socially grounded, yet they engage distinct semantic and emotional dimensions. Using controlled prompts designed to elicit contrasting conceptual framings of the same image, we compare the model’s descriptions to examine shifts in internal representations and language outputs. All the specific concepts are illustrated below:

- •
Honesty: We consider honesty as the model’s ability to generate factually accurate responses without distortion or fabrication Lin et al. (2021). Fig. [5(a)](https://arxiv.org/html/2506.08543v2#S5.F5.sf1) presents token-wise honesty scores for a VLM describing an image of the Golden Gate Bridge under two settings: an honest prompt (left) and an untruthful one (right). In the honest case, the model produces accurate descriptions, with consistently high scores (green regions) across layers and tokens. In the untruthful setting, the model introduces factual errors, resulting in sharp drops in honesty scores (red regions), especially at tokens reflecting misinformation.
- •
Fairness: We define fairness as the model’s ability to generate unbiased responses without systematically favoring certain groups (Corbett-Davies et al., 2023). Fig. [5(b)](https://arxiv.org/html/2506.08543v2#S5.F5.sf2) shows an example with a man and a woman working together. The fair response (left) is neutral, while the unfair one (right) portrays the man as dominant and the woman as passive. Token-wise fairness scores drop (red regions) at biased language, indicating that fairness violations are encoded in internal activations and can be mitigated through representational analysis.
Figure 7: The response of a VLM when provided with an image of the ocean and a prompt related to the concept of fearlessness, along with a token-wise fearlessness score. Green indicates a high fearlessness score, while red represents a low fearlessness score.
- •
Fearlessness: Defined by confidence, courage, and reduced sensitivity to risk (Lilienfeld & Andrews, 1996), fearlessness prompts the model to emphasize awe, beauty, and environmental grandeur when describing an ocean scene (Fig. [7](https://arxiv.org/html/2506.08543v2#A1.F7)). Green-highlighted tokens reflect admiration and agency, indicating a proactive stance toward nature. In contrast, under a fearful framing, the model’s language shifts toward danger and discomfort. Red-highlighted regions refer to drowning, vastness, and isolation, revealing a conceptual inversion in the model’s internal representation.
Figure 8: The response of a VLM when provided with an image of the United States Capitol Building and a prompt related to the concept of power, along with a token-wise morality score. Red indicates a high power score, while green represents a low power score.
- •
Power: Typically associated with authority, dominance, and the capacity to influence others (French & Raven, 1959), power is examined through two model responses describing the U.S. Capitol Building (Fig. [8](https://arxiv.org/html/2506.08543v2#A1.F8)). The first reflects a humble, civic-minded viewpoint, with green-highlighted tokens emphasizing justice, governance, and democratic ideals. The second adopts a power-seeking, unethical perspective, shifting toward a narrative centered on control, manipulation, and political ambition. Red-highlighted phrases indicate how internal representations adapt to subtle changes in moral and motivational framing.

These variations show that the model can simulate nuanced perspectives and encode them in a structured, consistent way, highlighting the usefulness of RepE for analyzing abstract concepts in multimodal settings.

### A.4 Attention Matrix Visualization

Fig. [9](https://arxiv.org/html/2506.08543v2#A1.F9) visualizes the attention matrices at various layers, illustrating that the matrices become increasingly sparse in deeper layers. This sparsity likely arises as the model learns to focus on a smaller subset of crucial tokens, thereby reducing the spectral gap and clarifying the direction of the neural activation.

Figure: Figure 9: Attention matrix visualization across different layers.
Refer to caption: fig/pic2.png

## Appendix B The Use of Large Language Models (LLMs)

In preparing this manuscript, the authors used Large Language Models (LLMs) as writing assistants.
It is important to emphasize that the LLMs were not involved in any core scientific aspects of this work, including the formulation of hypotheses, theoretical contributions, experimental design, implementation, data analysis, or interpretation of results. Their role was strictly limited to supporting the clarity, readability, and presentation quality of the paper.

The specific applications of LLMs in our workflow included:

- •
Improving Grammar and Readability: LLMs were employed for proofreading, grammatical corrections, and sentence restructuring. This ensured that technical content was conveyed with greater precision, fluency, and accessibility to a broad research audience.
- •
Polishing and Style Consistency: The models were used to propose alternative phrasings, unify terminology, and maintain a consistent academic tone throughout the manuscript. This was particularly helpful in harmonizing sections written by different co-authors.
- •
Assistance with Literature Search: LLMs were used to brainstorm keywords and provide summaries of potentially relevant references during the early stage of the literature review. Final paper selection, in-depth reading, and integration of related work into the manuscript were performed entirely by the authors.
- •
Formatting Suggestions: The models occasionally provided suggestions regarding LaTeX structuring, figure captions, and section transitions, which the authors subsequently verified and adapted to the paper’s requirements.

All outputs generated by LLMs were carefully reviewed, edited, and revised by the authors. At no point was text directly included without human oversight and modification. The responsibility for the originality, correctness, and scientific integrity of this paper rests solely with the authors.

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 4
presentation: 3
questions: 1. Line 51: Maybe it would be useful to introduce the notion of “unembedding space”, this concept is not really a “standard” standard concept.

1. Line 139: “recent advances in interpretability have shifted the focus toward analyzing representation”, please give references to this statement.

1. Line 148: “co ncept” -> “concept”

1. Eq. 3+4, should $x$ not be $\bf{x}$?

1. Figure 2: It is unclear how the cosine similarities in Figure 2 were computed; was a single sample $x$ used, or were cosine similarities between principal singular vector(s) and multiple samples for the same and/or distinct concepts used? 

1.  Figure 2: How should the polar plot be interpreted exactly? Is it the components of $f_l(x)$ that are plotted as the dashed lines, or how exactly was this plot constructed?

1. Figure 3: You write in the context of the figure (Line 323), “only a very small subset of singular values are amplified”. This interpretation seems a bit forced, considering that several layers have many non-zero magnitude singular values. Also, the relative difference between the highest singular value remains quite small compared to other non-zero singular values e.g., layers 19-21. Could you elaborate on Line 323 in relation to Figure 3?

1. Figure 4: How did you compute the concept direction $\bar{\lambda}_W$ for the results? Was a single concept used for this computation, or are the cosine similarities across layers an average for multiple concept directions?

1. Figure 5: What constitutes a “low” and a “high score?

1. Section 5.5: Experiments in sec 5.5, this is quite qualitative, can this not be made more quantitative?

1. Conclusion: Should this be written in past tense?
soundness: 2
strengths: The paper is well-written, quite self-contained, and has well-crafted illustrations.

The proposed method is both creative and original. The authors honestly point out that “our current framework is subject to several limitations.” (conclusion). 

While the developed method rests on a few key assumptions, it is elegant and well thought through. Given that the assumptions hold in practice, which the presented results suggest, the framework is a notable step towards better understanding how linear representations of high-level concepts emerge and propagate through deep neural networks.
summary: The paper develops a theoretical framework aimed at understanding how linear representations of high-level concepts emerge in deep neural networks. The authors hypothesize that observations in the ambient space are a linear combination of the high-level concept and spurious linear directions. This is coined the Input Space Linearity Hypothesis (ISLH). 

Building on this, the authors derive the Spectral Principal Path (SPP) for an affine network without non-linearities. The SPP is the path through projection matrices for which output-input singular vectors of adjacent matrices remain mostly co-linear, whilst corresponding singular values are large. Through Theorem 4.1, the ILHR and SPP framework is linked to the Linear Representation Hypothesis, connecting the work with prior research.
weaknesses: 1. Introduction: covers the core concepts; however, readers unfamiliar with the LRH paper (Park et al. 2023) have considerably less context for understanding the setting.

1. The theoretical derivation of the SPP method hinges on having a generalized network, like equation 5, without non-linearities. Yet, little discussion is provided on the limitations that come with this assumption.

1. It wasn’t immediately clear to me why the second term in equation 8 is inserted. Based on the model specification in equation 5, this term would be zero. It would be good to state more explicitly for which case(s) this term is non-zero, e.g., referring to Appendix A.2.1.

1. While the authors state in their reproducibility statement that they are committed to releasing the code, no private repository was provided as part of the review. This complicates disambiguating parts of the work, specifically related to these points:

    1. Complexity of computing the SPP (equation 13): Finding the principal path is exponential in the depth of the model. Hence, for a $L$ layer network where the layer-wise jacobians have ranks r_1, …, r_L, the possible paths are: $\prod_{l=1}^L r_l$. While this could become prohibitive for deep networks, it is not something that the authors discuss nor provide details on how to compute in practice.
    1. It is not immediately clear how the results in Figures 2 and 4 were computed. Please see the questions below for more details on where the confusion lies. Without further elaboration, it raises questions about the robustness and generality of the proposed framework.

    1. In line with the above, I am left with unanswered questions in terms of how rigorously the proposed method has been evaluated. At the outset, the results seem mostly qualitative, which weakens the support for ISLH and SPP.

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 1
questions: 1. Eq 12: is G(\mathcal{P}) appears to not be a scalar that can be maximized - can the authors clarify?
2. How to interpret Fig 2: what do the angles represent?
soundness: 1
strengths: 1. Analysis of the LRH and the inquiry for the reason of its emergence is interesting, analysing it through intermediate layers is sound.
summary: The paper claims that concept directions already exist in the input space, and  neural networks propagate these directions along a few dominant paths layer by layer. Assuming this structure (ISLH), the authors argue that the LRH naturally emerges. The paper expands layer-wise Jacobians via SVD and defines a "path gain" as the product of singular values and inter-layer alignments. A few experiments are conducted on a VLM to probe intermediate-layer concept alignment.
weaknesses: 1. LRH has mainly been discussed in text-only contexts, and the ISLH is also formulated that way. Yet the experiments use a VLM, and all actual manipulations seem to be on the language side. It’s unclear what the vision modality contributes here, this effectively invalidates the claimed "raw-space" perspective.
2. The exposition is confusing (see Questions).
3. Theoretical claims are made for linear networks. Since a purely linear network can be collapsed into a single matrix, it’s unclear how this is informative for real nonlinear architectures.
4. Section 5.2 largely repeats Section 4.3 without adding new content.
5. Experimental evaluation is weak and mostly qualitative.
6. 5.5.1 repeats contents of 5.5.2.

---

confidence: 4
contribution: 3
initial_rating: 4
presentation: 3
questions: Please see the Weaknesses section.
soundness: 3
strengths: The work bridges theory and interpretability, offering an appealing spectral perspective on how neural networks encode and stabilize concepts.

The proposed Spectral Principal Path framework provides a unified spectral mechanism that connects the input-space structure to the linear separability observed in deep representations, while this link is well articulated in the main text. 

Moreover, the provided evidence of singular vector stabilization and selective singular value growth provide some important insights into the observed representational coherence.
summary: This paper introduces the Spectral Principal Path (SPP) framework to explain the emergence and stability of linear representations in deep networks. The authors propose the Input-Space Linearity Hypothesis (ISLH)—an extension of the Linear Representation Hypothesis (LRH)—which suggests that concept-aligned directions originate in the input space and are selectively amplified through network depth.
The authors conside a derivation based on the Jacobian SVD decomposition paired with cumulative gain maximization, arguing that representations found in the input space propagate along a few dominant spectral paths aligned with the largest singular values.

The experiments show that principal singular vectors stabilize in deeper layers, singular values exhibit selective growth, and concept directions become increasingly concentrated and stable, supporting the theoretical claims.
weaknesses: My main concerns revolve around some experimental choices of the authors and some restricting assumptions considered. Specifically:

*Simplified architecture assumption*: The SPP derivation assumes a purely stacked linear model, which may not capture non-linearities or normalization effects critical in deep networks. How robust is the theory under more realistic settings? Would the insights provided in the manuscript extend to more common architectures?

*Residual and attention extensions*: In the main text, line 214, the authors mention "we show our extension to residual connections and attention mechanisms". I find this to be a bit misleading; while residual connections are somewhat discussed theoretically (on a single paragraph), the attention mechanism is only empirically justified. Could the authors provide a more formal link between the residual connections/attention operators and the spectral path analysis? If not, is the only way to validate if the findings hold through a questionable in terms of interpretation empirical evaluation?

*Limited model diversity*: The experiments rely on a single model (Idefics2-8B). How general are the observed SPP behaviors across other architectures (e.g., LLMs, CNNs, or smaller-scale models)?

*Evaluation Setup*: Testing SPP behavior in binary “concept flip” or contrastive settings (inspired by RePe) seems restrictive. Would the same spectral alignment hold in continuous or open-ended concept spaces?

*Interpretation of multimodal results*: While the LAT scans are visually compelling, it’s unclear how they directly confirm spectral path propagation rather than post-hoc representational clustering. Similar results where provided in the original LAT paper, that does not consider SPP.

---

confidence: 3
contribution: 2
initial_rating: 2
presentation: 2
questions: See weaknesses. Besides the above points:

Why do you call Section 5.5.1 (line 373) your method? What did you modify compared to the original algorithm in [1], used to plot Figure 9 in their Section 4.3.2? 

Also, which layer in which module (vision or text) did you choose? Notice in [1], the authors clearly stated that they did a sweep to identify one of the strongest layers for concept reading and picked that one. This is related to your description in Line 424 (“layer-agnostic”), which is incorrect.

Misc: (typos) in line 148 concept; line 237 singular.

[1] Representation Engineering: A Top-Down Approach to AI Transparency
soundness: 2
strengths: The spectral path viewpoint is new and intriguing.

The paper writes clearly and is easy to read.
summary: This paper proposes Input-Space Linearity Hypothesis (ISLH), extending the Linear Representation Hypothesis (LRH) to the input space where each sample is treated as an entangled mixture. The paper then proposes a Spectral Principal Path (SPP) framework, which provides an explanation for LRH based on ISLH, that there exists spectral paths along certain singular directions to amplify and propagate the linear direction from inputs to outputs. It then runs experiments from a reference paper, RepE, on VLMs, and obtains similar results to the reference.
weaknesses: Unclear or missing description of experimental setup. For example, what is the data used to plot Figures 2, 3, and 4? What is x here? What is the concept used (W) to obtain Figure 4? Further, how is the conclusion “only a very small subset of singular values are amplified; the remainder stay close to their initial scale” obtained from Figure 3?

Disconnection between the framework and experiments. (1) The framework builds upon inputs in a vector space propagated through a generalized network, while the experiments are launched on Idefics2-8B, which consists of two encoders for text and image modalities, respectively. Though these two modalities converge into a single embedding space, it is unclear whether the framework is useful for visual inputs, since later (contracting concept) experiments are all based on text inputs. Still, for text inputs, it is unclear whether they fulfill the basic assumption that “each sample is an entangled mixture” in a vector space. (2) Section 5.5 is detached from the framework. According to the description, they are literally a replication of what has been done in [1], on VLM instead of LLM. The plots do not provide sufficient support to the proposed framework or help claim its validity.

[1] Representation Engineering: A Top-Down Approach to AI Transparency


Decision: Reject