Block Recurrent Dynamics in Vision Transformers
Authors: Mozes Jacobs
Thomas Fel
Richard Hakim
Alessandra Brondetta
Demba E. Ba
T. Anderson Keller
Venue: ICLR 2026 Poster (2026)
Keywords: Computer Vision
Interpretability
Dynamical system
Primary area: interpretability and explainable AI
TL;DR: We hypothesize vision transformers are block recurrent and validate this by training a recurrent surrogate of DINOv2, requiring only 2 blocks to recover 94% of accuracy. We then study DINOv2 from a dynamical systems perspective.

Abstract: As Vision Transformers (ViTs) become standard backbones across vision, a mechanistic account of their computational phenomenology is now essential. Despite architectural cues that hint at dynamical structure, there is no settled framework that interprets Transformer depth as a well-characterized flow. In this work, we introduce the $\textbf{Block-Recurrent Hypothesis (BRH)}$, arguing that trained ViTs admit a block-recurrent depth structure such that the computation of the original $L$ blocks can be accurately rewritten using only $k \ll L$ distinct blocks applied recurrently. Across diverse ViTs, between-layer representational similarity matrices suggest few contiguous phases. To determine whether this reflects reusable computation, we operationalize our hypothesis in the form of block recurrent surrogates of pretrained ViTs, which we call Recurrent Approximations to Phase-structured TransfORmers ($\texttt{Raptor}$).  Using small-scale ViTs, we demonstrate that phase-structure metrics correlate with our ability to accurately fit $\texttt{Raptor}$ and identify the role of stochastic depth in promoting the recurrent block structure. We then provide an empirical existence proof for BRH in foundation models by showing that we can train a $\texttt{Raptor}$ model to recover $94$\% of DINOv2 ImageNet-1k linear probe accuracy in only 2 blocks. To provide a mechanistic account of these observations, we leverage our hypothesis to develop a program of $\textbf{Dynamical Interpretability}$. We find $\textit{\textbf{(i)}}$ directional convergence into class-dependent angular basins with self-correcting trajectories under small perturbations $\textit{\textbf{(ii)}}$ token-specific dynamics, where $\texttt{cls}$ executes sharp late reorientations while $\texttt{patch}$ tokens exhibit strong late-stage coherence reminiscent of a mean-field effect and converge rapidly toward their mean direction and $\textit{\textbf{(iii)}}$ a collapse of the update field to low rank in late depth, consistent with convergence to low-dimensional attractors. Altogether, we find that a compact recurrent program emerges along the depth of ViTs, pointing to a low-complexity normative solution that enables these models to be studied through principled dynamical systems analysis.

Published as a conference paper at ICLR 2026
BLOCK-RECURREN DYNAMICS IN VITS
Mozes Jacobs Harvard University
Thomas Fel" Harvard University
Richard Hakim" Harvard University
Alessandra Brondetta Osnabrück University
Demba Ba Harvard University
T. Andy Keller
Harvard University
ABSTRACT
As Vision Transformers (ViTs) become standard backbones across vision, a mech- anistic account of their computational phenomenology is now essential. Despite architectural cues that hint at dynamical structure, no settled framework interprets Transformer depth well-characterized flow. In this work, introduce the Reccrrettt Hypothesis (BRH), arguing that trained ViTs admit a block- recurrent depth structure such that the computation of the original L blocks can be accurately rewritten using only A: < L distinct blocks applied recurrently. Across diverse ViTs, between-layer representational similarity matrices suggest few con- tiguous phases. To determine whether these phases reflect reusable computation, we operationalize our hypothesis in the form of block recurrent surrogates of pretrained ViTs, which we call Recurrent Approximations to Phase-structured TransfORmen (Raptor). Using small-scale ViT's, we demonstrate that phase-structure metrics correlate with our ability to accurately fit Raptor and identify the role of training and stochastic depth in promoting the recurrent block structure. We then provide empirical existence proof for BRH in foundation models by showing that we can train a Raptor model to recover 96% of DINOv2 ImageNet- Ik linear probe accuracy in only 2 blocks while maintaining equivalent runtime. To provide a mechanistic account of these observations, we leverage our hypothesis to develop a program of Dynamical Interpretability.. We find (i) directional convergence into class-dependent angular basins with self-correcting trajectories under small pertur- bations, (ii) token-specific dynamics, where cls executes sharp late reorientations while patch tokens exhibit strong late-stage coherence reminiscent of a meaan-fied effect and converge rapidly toward their mean direction, and (ii) collapse of the update to low rank in late depth, consistent with convergence to oow-dimensional attractors. Altogether, we find that a compact recurrent program emerges along the depth of ViTs, pointing to a low-complexity normative solution that enables these models to be studied through principled dynamical systems analysis.
INTRODUCTION
In the last decade, Transformers have become the default neural network architecture across machine learning communities, scaling favorably with data and compute (Vaswani et al., 2017; Kaplan et al., 2020). particular, Vision Transformers (ViT') (Dosovitskiy et al., 2020) have become the core architecture used in visual foundation modeling frameworks such as DINOv2 (Oquab et al., 2023; Darcet et al., 2023) and CLIP (Radford et al., 2021): and have come to dominate a wide range of visual tasks, from general visual feature extraction (He et al., 2021; Chiu et al., 2024; Yun, 2025), to diffusion (Peebles & Xie, 2023), image segmentation (Kirillov et al., 2023; Liu et al., 2024), and video processing (Araab et al., 2021; Baldassarre et al., 2025). This increasing breadth of use motivates a move from empirical optimization to principled understanding.
Two pressures make this understanding urgent. First, safety-critich deployments (Wang & Chung, 2022; Alecu et al., 2022) demand mechanisms whose internal computation inspectable (Losch al., 2021), diagnosable (Adebayo et al., 2020), and verifiable (Tjeng & Tedrake, 2019) rather than
Equal contribution.
Correspondence to [mozesjacobs, tfel, rhakim, tawetlkkelleeee.. hew.kkellrrrg.haaard..oooo harvard. kempnerinstitute titt.b..io io/raptor

---

Published as a conference paper at ICLR 2026
Dinov2
SigLip
ViT (small)
ViT (large)
4004
11
Figure 1: Layer-layer similarity matrices across diverse Vision Transformers reveal block- structure. Despite differences in scale and training objectives, all models exhibit contiguous block structure along depth, visible as phase-segmente regions of high similarity. Beyond representational similarity, this raises the question of whether deeper functional recurrence underlies these patterns, hinting at block-wise reusability of computation across layers. In this work, we investigate this hypothesis, showing that these phase segments correspond to blocks with functional similarity, which can be approximated by a single shared block applied recurrently along depth.
opaque. As these models proliferate across domains, the ability to explain (Doshi- Velez & Kim, 2017; Gilpin et al., 2018; Kim et al., 2018), manipulate, and verify their behavior becomes increasingly essential. Second, from scientific inference perspective (Cichy & Kaiser, 2019), understanding what makes these models work is essential for explaining their success. Their strong performance suggests they have discovered effective strategies, and identifying these strategies could advance our broader understanding of visual intelligence. Independently of any comparison human vision, the goal is to uncover the principles that make these systems effective.
One promising path toward such understanding is to search for underlying simplicity. Multiple approaches explore different facets of this simplicity, whether in functional expressivity (Montúfar et al, 2014; Telgarsky, 2015; Serra et al., 2018; Balestriero et al., 2018), symmetry (Cohen & Welling, 2014; Olah et al., 2020), or computation (Wilson, 2025; Goldblum et al., 2023; Schmidhuber, 1997; Mingard et al., 2025). Discovering such simplicity principle should improve both development (Bron- stein al., 2017; Frankle Carbin, 2019) and interpretability (Bereska & Gavves, 2024; Carvalho et al., 2019; Fel, 2024; Ghorbani et al., 2017; 7; Fel et al., 2023; Smilkov et al., 2017; Sundararajan et al., 2017: Zeiler& Fergus, 2014: Templeton et al., 2024; Bricken et al., 2023). Depth offers a concrete place to look for this simplicity. Residual connections have long suggested a link to dynamical systems (Liao & Poggio, 2020; Veit et al., 2016; Greff al., 2016; Boulch, 2017; Haber & Ruthotto, 2017), hinting at implicit recurrence even when layers have distinct parameters. This convergent evidence makes plausible a form of algorithmic parsimony (Ma et al., 2022) which a small set of blocks may be reused across many layers, trading parameters for iterations (Schwarzschild et al., 2021). Related perspectives support this view (Dingle et al., 2020). More concretely, residual updates invite discrete-time interpretation of depth (Chen et al., 2018; Chalvidal et al., 2020; Sander et al., 2022), attention induces coupled token dynamics (Lu et al., 2019; Geshkovski et al., 2023), and in language models contiguous block recurrence has been observed and exploited (Geiping al., 2025; Fernando & Guitchounts, 2025; Dehghani et al., 2018; Tan et al., 2023). This renewed interest 2025; Venkatraman al., 2025) and algorithmic complexity (Shaw et al., 2025; Dingle et al., 2020) as frameworks for understanding neural network simplicity bias further motivates our investigation. However, no existing framework characterizes depth in ViTs as representational flow or determines whether apparent phases correspond to functional reuse. Furthermore, vision explainability research (Bach et al., 2015; Fong & Vedaldi, 2017; Novello et al., 2022; Muzellec et al., 2024; Petsiuk et al., 2018; Hedström et al., 2022; Fel et al., 2025; Gorton, 2024; Kowal et al., 2024; Bau et al., 2017; Vilas et al., 2023) has not leveraged dynamical systems analysis to model emergent network structure.
in recurrent computation
In this work, we consider recurrence as a candidate source of simplicity and advance the Block- Recurrent Hypothesis (BRH): after training, the depth of ViT organizes into few contiguous phases such that the computation of the original L layers can be rewritten by reusing only l < L distinct blocks applied recurrently. Our starting point is an empirical observation: layer-layer representational similarity matrices consistently exhibit block-diagonal structure across disparate models. However, representational similarity does not necessarily translate functional equivalence; therefore, we ask: Does this phase structure reflect genuinely reusable computation?
Our contributions. Our study proceeds in three parts:
2

---

Published as a conference paper at ICLR 2026
Empirical evidence for block-recurrent structure. We demonstrate across diverse ViTs that layer-layer representational similarity matrices exhibit distinct contiguous phases of computa- tion, formalized through the Block-Recurren Hypothesis. We develop max-cut algorithm to systematically identify phase boundaries and show that this block structure is (i) associated with compressibility by recurrent architecture and (ii) strengthened by stochastic depth.
Constructive verification via recurrent surrogates. We operationalize the BRH by training weight-tied block-recurrent approximations of pretrained ViTs, termed Raptor. Critically, our goal is not compression or efficiency optimization per se, but rather to demonstrate that functional reuse genuinely possible. Raptor reconstructs the complete internal representation trajectory across all layers, not merely the final output, providing strong evidence for true computational equivalence rather than input-output mimicry. Specifically, we provide empirical evidence for the BRH on foundational vision models by training Raptor that recovers 96% of DINOv2's ImageNet- Ik linear-probe accuracy using only recurrent blocks, and 98% with 3 blocks.
Dynamical systems analysis framework. Leveraging hypothesis, propose program of Dynamical Interpretability that treats ViT depth as the discrete-time unfolding of an underlying dynamical system, such that the evolution of representations across layers can be analyzed through its temporal dynamics. Our analysis reveals: (i) directional convergence into class- dependent angular basins with self-correcting trajectories under perturbations, (ii) token-specific dynamics where cls tokens execute sharp late reorientations while patch tokens exhibit strong coherence reminiscent of mean- mean-field behavior, and (ii) collapse of layer-to to-leer updates to low-rank subspaces consistent with convergence to low-dimensional attractors.
As first step, we characterize emergent phases in representation space, motivating the BRH.
EMERGENT PHASE STRUCTURE & THE BLLOCK- HYPOTHESIS
Our investigation starts with simple experiment: we construct layer-layer similarity matrices by computing the cosine similarity of each token at layer 1 with the same token at layer m. As Figure 1 shows, despite varying tasks, objectives, and scales, all models exhibit consistent block- wise organization: high mutual similarity within contiguous blocks and lower similarity across boundaries. This finding not new and echoes early (and more recent) observations in residual networks (Kornblith et al., 2019; Raghu et al., 2021; Nguyen et al., 2022; Hoang et al., 2025), raises a fundamental question: does representational similarity reflect deeper computational structure? In fact, representational similarity alone provides no guarantee of functional equivalence. Layers might produce similar representations through entirely different computational pathways, or conversely, functionally equivalent computations might yield representations that appear dissimilar due to linear transformations or noise. The critical question is whether these apparent phases reflect genuine functional recurrence- reusing the same computational operations across layers within each phase. We formalize this possibility through the RecckRReeurrett Hypothesis:
Definition Recurrent Hypothesis (BRH)). We consider f be a trained Vision Transformer with nominal depth L, and intermediate maps ft X - Ae, E (1....,.) L.). We say that f satisfies the e- E-BRH for any E, there exist K < l blocks B BR and integers myy... Bl with En= nj = such that:
Ex (l.f(()) -(B (B((x) (1x) o o... B(au) (11) (z))ll) Se,
where B(a)) denotes n, repeated applications of the same parameter-tied block Bj and the entire approximation maintaining equivalent computational cost.
Here, || || denotes the Frobenius norm and P is a probability distribution over natural images. To put it simply, this hypothesis states that ViT's L layers be replaced by k < L recurrent blocks that reproduce the entire internal trajectory at equivalent computational cost. By requiring intermediate layer fidelity rather than just final output matching, we rule out trivial solutions where computation is concentrated in a single block. The constraint l < L with parameter tying ensures genuine functional reuse rather than simple parameter copying.
To test this hypothesis, we first operationalize it by proposing a method to construct these approxima- tions using recurrent architectures, developing specialized training technique descrihed below.
3

---

Published as a conference paper at ICLR 2026
Blocks
0.85
0.84
132
0.83
[
175
0.82
164
0.81
0.80
Max
Random Shuffle
Random Contiguous
K-2
K-3 Number Recurrent Blocks
k-4
LAy53
Figure 2: Block discovery via max-cut segmentation of the layer-layer similarity matrix. Our algorithm partitions depth into contiguous segments by maximiz- ing within-block similarity and minimizing cross-block cosine similarity. Shown
Figure 3: Evaluation of Raptor mod- els on CIFAR-100 using our max- cut partitioning algorithm versus ran- dom partitions. Reported values are classification accuracy. Results for dom partitions are aggregated over 10 different random partitions. Random shuffle refers to non-contiguous (frag- mented) random partitions.
cuts of the same ViT-B: with 3-blocks (left, green) and 2-blocks (right, magenta). These reveal candidate block boundaries where the representation dynamics undergo sharp transitions, pro- viding an operational method for detecting contiguous recurrent phases in trained ViTs.
Operationalizing Block-Recurrence with Raptor. Since the BRH asserts only the existence of recurrent blocks satisfying its conditions and not their precise form, the most direct validation is constructive: demonstrating existence by example of a recurrent model approximating f.
However, recurrent architectures are notoriously difficult to train: rollouts can drift as small state errors compound over steps, and gradients propagated through many recurrent applications often vanish or explode (Pascanu et al., 2013; Linsley et al., 2020; De al., 2024). Standard backpropagation through time becomes unstable as recurrence depth increases, and the model must learn to simultaneously handle both the forward dynamics and its 1989). To circumvent these challenges, we will leverage the intermediate layer activations as training targets, enabling staged approach that combines the stability of teacher forcing with the self- consistency required for autoregressive deployment. More precisely, we introduce a procedure to distill existing Vision Transformers into Recurrent Approximations Phase-structured TransfORmers (Raptors), using parameter-tied blocks with repetition counts determined by a max-cut phase discovery algorithm. This approach transforms the abstract hypothesis into a concrete architectural and training framework that can be empirically validated.
prediction errors in closed loop (Williams & Zipser,
This constructive approach requires that Raptor models reproduce the internal activations of the full VIT they approximate, similar to Dasgupta & Cohn (2025); Sanh et al. (2019); Shleifer & Rush (2020), not merely mimic the final output'. The BRH implies that such reproduction should be possible within tolerance making activation matching natural training objective. Formally, let be a reference ViT with intermediate activations a₂ (x) = fe(x) E Ix for & = .., ,..,,, L, where layer é = 0 denotes the patch encoder and 1 < l < L refer to transformer layers. Here, t is the number of tokens and the feature dimension. Let B, denote the j-th parameter- block in our recurrent decomposition. The Raptor approximation produces activations:
ax(x) = (B
(m)) o B (11) ((a1))(a((())))
where the composition covers layers 1 to according to our phase segmentation. We train Raptor us- ing an autoregressive loss (AR) that enforces trajectory fidelity across all intermediate layers:
AR (x) = Exe Σ |ā,(x) - aa(x))l))) h S.
This approach teaches each block to approximate designated segment, while the overall model reproduces the teacher networks' complete representational trajectory. However, this formulation does not specify how to determine the block boundaries or phase assignments. We address this now with simpler algorithmic approach based on the representational similarity structure observed earlier.
'Ulikee classical distillation, which typically supervises logits (and occasionally few intermediate "hints"), same inputs. The
enforce one-to-one alignment of all layers representations across the entire depth for the recurrent surrogate must generate the teacher's intermediate activations, just its predictions.
4

---

Published as a conference paper at ICLR 2026
Choosing partitions. For given number of blocks k we must introduce practical method to determine the number of recurrent iterations of each block (næ): in other words, where the recurrent "phases" of computation begin and end. We accomplish this by casting this "block discovery" process as a weighted max-cut problem Goemans & Williamson (1995), solved via dynamic programming (see Subsection A.1 for details). Specifically, the algorithm seeks partition depth into contiguous segments by maximizing within-block similarity and minimizing cross-block similarity. We visualize the results of this procedure applied to ViT-B in Figure 2, demonstrating that the discovered blocks align reasonably with qualitative assessment.
To validate this approach, we train recurrent transformer models using max-cut partitions to reproduce the activations of trained Vision Transformers on CIFAR- 1-00-100 (validation set accuracy 90.7%). Remarkably, as shown in Figure 3, Raptor student models with only recurrent blocks closely match the performance of the ViT-B teacher models they approximate. The max-cut algorithm provides partitions that achieve strong performance, with accuracy often greater than one standard deviation above randomly chosen partitions. This observation suggests that the representational block-similarity structure is closely associated with functional block-recurrent phases.
To assess how unique each block-recurrent layer is and control for degenerate solutions to functional blouk-rec,,,ence, we tested whether swapping a layer from a different block could substitute for layer within target block (Figure 15). Using DINOv2-B trained on ImageNet- 1k, we find that while intra-block swapping preserves accuracy, inter-block swapping leads to model collapse. These results demonstrate that representational block-similarity structure predicts functional block-recurrent structure, and that layer identity is functionally unique to each block.
How do blocks emerge? Having operationalized the BRH and demonstrated method for block discovery, we now turn to the mechansitic origins of this phenomenon: under what conditions does this block-recurrent structure emerge in trained Vision Transformer?? To investigate this systematically, we examine small-scale ViTs where we control training conditions and isolate potential contributing factors. Specifically, we hypothesize that training and stochastic depth (Huang et al., 2016) may promote the emergence of block-recurrent patterns.
Motivated by evidence that residual networks tolerate variable effective depth (Wu et al., 2019), we examined the effect of stochastic depth (SD) on block recurrence. During training, each layer is dropped independently with probability p. applied uniformly across depth. We trained ViT-B/14 from random initialization on CIFAR- 100, using the cls token for the linear probe across sweep of SD P rates. We observe an increase in layer-layer similarity with increasing SD p rates (Figure 4A, E). Next, we used these trained ViT networks as teachers for student Raptor models (see Appendix A). Raptor models were trained to reconstruct the hidden activation states of the ViT teacher across layers. Raptor forward passes are fully autoregressive, meaning each layer's 's output is fed into the next layer and is also trained to match the corresponding layer in the teacher network (Figure 10). We quantify the similarity of the cls and patch token representations in each layer between the teacher and student networks as the R² of their matched token embeddings (Figure 4B).
We observe that, VIT stochastic depth increases, a separately trained Raptor student model becomes significantly better at reconstructing the ViT's layerwise hidden states (Figure 4D). In addition, we observe that increasing SD improves CIFAR- 100 classification accuracy in both VIT teacher and Raptor student networks (Figure 4C). We combine the above results in Figure 4E and observe a strong positive association between the ViT's layer-layer representational similarity and Raptor reconstruction fidelity. These results demonstrate that stochastic depth regularization increases layer-layer similarity and recurrent compressibility.
We also sought to determine whether recurrent compressibility was dependent on several other factors, including whether the ViT teacher was trained or untrained, and whether skip branches are included in the network architecture. We observe that network hidden layer activations in untrained ViTs can be reconstructed by Raptor better than trained ViTs. Interestingly, we observe that removing the skip branch from an untrained VIT only slightly reduces reconstruction accuracy. Finally, we trained Raptor students on checkpoints from a single ViT allowed to overfit (via reduced weight decay) on CIFAR- 100 classification (Figure 13). We observe that Raptor reconstruction accuracy (R²) remains high until the ViT network begins to overfit ( Figure 13B-D), after which cls token reconstruction and classification accuracy drops. These results add to the results stochastic depth and demonstrate how the natural architecture of ViTs as well as the normative state of properly regularized VIT networks demonstrate emergent block recurrent dynamics.
5

---

Published as a conference paper at ICLR 2026
SD dropout prob = 0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
A
ViT layer
Layers
similarity
Layers
1.0
RAPTOR VIT
Patch tokens
similarity
5
Layer
D
VIT-RAPTOR
E
similarity va. RAPTOR reconstruction similarity
RAPTOR accurasy vs stochastic
CIFAR casssilitatioo socurey
depth drop rate (uniform)
hidden-state similrity
Re: RAPTOR similerity
1.00
0.6
Peeeraarii d8gphh dlop mab Moseen
Re aroo: erro: RAPT OO to VIT dissimiartt
CLS token
0.08
Patch tokens
0.4
0.94
0.05
0.3
VIT
RAPTOR
0.2
0.88
0.0
0.0
0.0
0.2
0.4
0.6
0.8
0.9
1.0
stochastic depth drop rate (uniform)
stochaslic depth drap rale (uniform)
block similarity
Figure 4: Stochastic depth promotes representational similarity across layers Mock-rrecureenc.. A) ViT layer-layer cosine similarity matrices for models trained with increasing stochastic depth (SD) dropout probability p (probabilities of 0.0-0.9, uniform over layer depth). Dashed red lines delineate blocks, as defined by the max-cut algorithm. Higher SD p values lead to a more similar representation across layers. B) Layerwise teacher-student representational alignment R² (Raptor vs. ViT) of the class cls and patch tokens. Increases in SD correspond to an increase in the ability of Raptor match the ViT's layerwise representations. VIT models for SD=0.7-00 7-0.9 show abberant training dynamics and excluded from this and further analysis. C) CIFAR classification accuracy a VIT trained on CIFAR, and Raptor model with = blocks trained to match state of the ViT. D) Last layer hidden-state similarity R error (equivalent to 1 R²) of the VIT and Raptor model as a function of SD p. Increases stochastic depth lead to a greater ability reconstruct ViT function using Raptor.. E) Association between layer-layer representational similarity and Raptor reconstruction R².. Stochastic depth encourages the formation of more similar blocks of layers within the ViT, which facilitates approximation by the recurrent Raptor model.
hidden
Taken together, these toy-model experiments support the view that the observed representational block structure reflects both an intrinsic property of the residual ViT architecture and learned/emergent functional recurrence. Using the methods established here, we next scale up our application of Raptor to modern large-scale foundation models.
SCALING Raptor TO FOUNDATION MODELS
Having demonstrated the BRH controlled experiments, we test whether it extends large- scale foundation models. We apply Raptor to DINOv2, chosen for its widespread adoption across vision tasks, and optimize reproduce DINOv2's internal activations on ImageNet-Ik, Ik.
Architecture and Training. For all experiments, a pretrained DINOv2 (ViT-Base) model, applying our max-cut algorithm to identify E (2, 3, 4) recurrent block partitions. Each recurrent block (-)) mirrors the block of DINOv2 (for detail about the block see Section A). We train Raptor using two-stage approach that combines teacher forcing (TF) and autoregressive (AR) objectives. In the first stage, teacher forcing trains each block predict the immediate next layer given the correct previous layer, while the autoregressive objective requires the model to use its own predictions as inputs for subsequent layers. Refer to Figure 11 for this hybrid training algorithm. This approach yields the following total loss:
ImageNet-1K K and extract activations from
Corat (x) = ACTr(x) + (1- A) (x) - ((0).
where 2(0) denotes additional regularization applied to each tied block. See appendix A for complete details. An attentive reader will notice that this training approach naturally lends itself to parallelization: since each block operates on a distinct layer range, the first training stage can be executed simultaneously across multiple GPUs machines. Each block learns to approximate

---

Published as a conference paper at ICLR 2026
its designated segment of the original network using the combined objective, with teacher forcing gradually annealed to zero as training progresses. The first stage thus allows blocks to develop their specific computational roles while benefiting from ground-truth activations as inputs.
The second stage connects all trained blocks into the complete recurrent architecture and trains the entire system end-to-end using only the autoregressive loss (i.e. X 0). This crucial phase teaches blocks to coordinate their computations and handle their own predicted activations rather
than relying on ground-truth inputs from the teacher network. The transition from teacher forcing to pure autoregression ensures that the final model can operate independently while maintaining fidelity to the original network's 's representational trajectory.
We provide an implementation framework at https: / /ttp////eee//eeeeeeeeeeeeeeeeeee i inst tute. tute.gi github. i io/raptor With the training methodology established, we now evaluate how effectively Raptors can reproduce the performance of their teacher networks across multiple vision tasks.
Results. We evaluate Raptor against DINOv2 by training linear probes on ImageNet- Ik (classification), ADE20k (semantic segmentation), and NYUv2 (monocu- lar depth), covering both classification and dense predic- tion. For ImageNet-Ik, we initialize the classifier from the public DINOv2 probe and report the best score across initialization and fine-tuning. In all experiments, the ViT backbone is frozen for both Raptor and DINOv2: only the linear heads are updated, and we reuse DINOv2's patch embedding and final layer normalization (both also frozen). Results appear in Table 2. Raptor performs well across tasks and is stronger on classification; with = 3 it attains 83.0% top-1 on ImageNet-Ik (about 98% of DINOv2 ViT-B and above ViT-S; see Fig. 5), with little deviation across runs (a = 0.1). Accuracy improves markedly from k 2 to A= = 3 and then sat- urates at h = 4. In short, a two-block Raptor retains about 96% of DINOv2 ViT-B with frozen backhone, a compact rewriting that substantiates the BRH. This con- clusion is further supported by Figure 14, which shows that Raptor maintains high cosine similarity to DINOv2 activations through depth, confirming that it captures the representational dynamics of the original model.
100
-.0
DIRDID VITG AMc
48-
-0.9
95-
0.8
0.75
94-
-0.7
Number of Recument Blocks
Figure 5: Raptor's performance on ImageNet-Ik as a function of DINOy2 VIT-B accuracy (left), and R² score (right). DINOv2 ViT-S accuracy shown as dashed horizontal line. Results are aggregated over three model runs trained on different randoms seeds.
Ablations. Although our aim is not maximal compres- sion nor exact accuracy matching, we perform targeted ablations to identify the factors most critical to Raptor per- formance (Table 1). Training with teacher forcing alone (Stage 1 only), while computationally efficient, Ieads to complete collapse with poor accuracy (~ 3% on ImageNet-IKK, indicating that one-step supervision is in- sufficient without exposure to the full autoregressive tra- jectory. Introducing the autoregressive loss and gradu- ally annealing teacher forcing to zero raises accuracy by more than 68%, underscoring the necessity of closed-loop training for stable block-recurrent approximation. Further gains come from depth scaling (Appendix A.3), where each block has a learned vector embedding of its target layer index, making Raptor a non-autonomous dynamical system (i.e., the update rule explicitly depends on the iteration count rather than state alone). Up- weighting the final block's 's cls token loss yields further improvements (Eq. 3, Appendix A). Finally, connecting all blocks and fine-tuning the model end-to-end with the autoregressive objective (i.e. the second stage) dramatically increases performance, and a final boost is obtained by fine-tuning the linear probe. Now that have shown that the BRH holds for foundation model, and before turning
Method
Accuracy
Teacher Forcing (TF)
3.9
+ Autoreg (anneal TF)
72.7 1 68.8
Depth Scaling
75.2 2.5
Weighted cls
76.7 1.5
Second Stage
82.4 5.7
+ Finetune (Clussifer)
83.0 1 0.6
Table 1: Ablations to original
Raptor(k=3) model, showing ImageNet- 1k accuracy with DINOv2 pretrained linear classifier. Second Stage refers to putting all three blocks together and training the full model autoregressively.
7

---

Published as a conference paper at ICLR 2026
Method
Arch.
IN-Ik (Ace t)
ADE20k (mloU t)
NYUv2 (RMSE 4)
Raptor
h: 81.2 + 0.2
39.6 = 0.6
0.618 + 0.003
k 83.0 0.1
13.0 = 0.3
0.618 + 0.006
83.2 0.1
43.6 = 0.1
0.607 = 0.006
DINOv2
VIT-S
80.9
44.6
0.600
VYT-B
84.5
47.5
0.578
Table 2: Performance of Raptor compared to DINOv2 with linear probes. We report top-1 accuracy on ImageNet-Ik, mean Intersection-over-Unioo (mloU) on ADE20K semantic segmentation, and root mean squared eгror (RMSE) on NYUv2 depth estimation. Higher values are better for accuracy and mloU, while lower values are better for RMSE. Results for Raptor are aggregated over three model runs, each trained with different random seed, and displayed as μ + For Raptor, Arch denotes the number of recurrent blocks, while for DINOv2, Arch denotes the VIT backbone.
Dynamical Interpretability, we first examine one implication of this phenomena: the algorithmic and computational implications of the BRH.
Algorithmic and computational implications. At scale, BRH holds in practice: two-block Raptor recovers most of DINOv2 VTT-B, with three blocks essentially closing the gap. This reveals a strong simplicity bias in trained viTs: depth reuses a small set of computations, effectively trading pa- rameters for iterations. This reuse has two immediate consequences. First, it compresses the program description length that realizes the network's s computation, suggesting low algorithmic complexity. However, the implication is more subtle than standard Kolmogorovcomplxxty (Kolmogorov, 1965). While Kolmogorov compression can replace a long program with an arbitrarily short one that runs in unbounded time, Raptor crucially preserves computational cost: applying the same block ny times achieves equivalent runtime to n distinct untied blocks. In other words, ViTs admit more compact program representation under the same runfime budget, aligning more closely with Levin's complexity Krevin (Levin, 1973). At scale, BRH holds in practice: two-block Raptor recovers most of DINOv2 ViT-B, and three blocks close the gap. This reveals simplicity bias in trained ViTs: depth reuses computations, trading parameters for iterations.
Claim 1 (BRH guarantees low Levin complexity). Let fe satisfy O-BRH with k & E. tied blocks {B,) and schedule (n,))== with Σ, n Let R(-) denote block runtime and define the untied teacher runtime R(ffe) := E BB)): assume runtime parity R(B,) S (1 + B))( Then
&
KLovin (ft) S Σ (I))(BB)) + O(k log E) + log R(fa) + 0(1)
j=1
See Appendix E for details. Theoretically, BRH guarantee low Levin complexity (compact algorith- mic descriptions at unchanged computational cost). Our empirical validation of BRH DINOv2 then suggest that foundation vision models are algorithmically simpler than their nominal archi- tecture suggests. This compression reinforces emerging evidence that simplicity principles govern successful neural networks (Goldblum et al., 2023; Valle-Perez et al., 2019; Huh et al., 2023). For interpretability research, this offers an encouraging perspective: there exist representational lenses under which seemingly complex models reveal underlying simplicity. High-performing ViTs discover and iteratively reuse a compact set of algorithmic primitives, which a structural regularity that may provide tractable entry points for mechanistic understanding.
We now pursue one such entry point. Since ViTs compress to recurrent blocks applied iteratively, propose to analyze their computation discrete- discrete-time dynamical systems, in the next section, develop this Dynamical Interpretability framework to extract insights from the recurrent structure.
FROM BLOCK RECURRENCE TO Dynamical Interpretability IN VITS
Having observed block-structured representational similarity and confirmed that this similarity translates functional recurrence, even in foundational models, we inclined seriously consider ViTs as dynamical systems that can be interpreted using dynamical systems analysis tools (Schmid, 2022: Ostrow et al., 2023; Huang et al., 2025)- what we term dynamical interpretabiility We begin by establishing the basic dynamical properties of this depth flow, and present three key findings: (i) tokens converge directionally toward angular attractors with self-correcting dynamies, (ii) different token types exhibit specialized dynamics with punctuated transitions at phase boundaries, and (iii) later layers exhibit low-rank collective motion under weak contraction, reminiscent of mean-field processes with collapsing update dimensionality.
Directional Convergence and Angular Attractor Geometry. We begin by isolating direction and scale. Feature norms increase steadily across depth, precluding Euclidean notions of convergence or attractors; we therefore normalize states and study their angular evolution (Fig. 12). Concretely, let
8

---

Published as a conference paper at ICLR 2026
2D-PCA of Avg. class trajectory
Avg. convergence last
pandi
Pacch
Repbler
Jollyfsh rebbit
Cosine
0.4
0.2
Layers
Figure 6: Directional convergence on the unit sphere. (Leftt Average normalized trajectories (PCA space) collapse into compact class-dependent basins, consistent with low-dimensional angular attractors. (Right) Cosine similarities to final representations "e are S-shaped, saturating near I for cls, registers, and patch, indicating directional fixed points.
Et = x > denote the direction of a token at layer l. and consider the depth trajectory { Vz on the unit sphere g Directional convergence is quantified by 7/2 = (æz, L). Empirically, ye follows smooth S-shaped curves that approach 1 and saturate in late layers for all token types (Fig. right). This behavior indicates a directional fixed point: while norms grow, directions stabilize so that $t41 R Ee as increases. The late acceleration of T/ suggests phase-local attraction strengthening in the final phase, clarified by our coherence study (Fig. 9, middle). A complementary geometric view comes from projecting the depth trajectories onto low-dimensional subspace. PCA reveals that sample-specific paths enter Blass-deeenden basins in shared angular subspace; across 1, 000 images from 5 imagenet classes, trajectories curl into compact terminal regions rather than scattering (Fig. 6, left). We interpret these regions angular attractors: small sets on S toward which iterates of the phase-local map steer directions, up to within-class variability. Finally, we probe stability by injecting small additive perturbation at layer l and following the perturbed direction thereafter. The average perturbed path bends back toward the unperturbed trajectory, indicating local self-correction and on-sphere contraction around the limiting direction (Fig. 8). Together, these measurements establish property (i): token directions evolve under depth toward angular attractors with mild contraction, making directional geometry an appropriate lens for subsequent dynamical analysis.
Token-Specific Dynamics. Token groups follow distinct angu- lar update laws. For a token with normalized state कe. define the per-layer angular speed by token type reveals stable small speeds for registers, intermediate speeds for patches, and sharp late reorientations for cls (Fig. 7). The variance 8, is smallest for registers after early depth; conversely, cls shows increased late angular activity, reflecting its global aggre- gator function. These token-specific laws vary across depth. Angular speed statistics display abrupt changes aligned with discovered block boundaries, creating a punctuated pattern where phases maintain near-stationary behavior that reset at phase transitions (Fig. 7). This structure matches the block-recurrent view in which phase applies a reused update map with stable statisties before handing off to new regime at the boundary. Sensitivity analyses corroborate this specialization. Inject a small additive perturbation of magnitude e at layer é and measure the final angular deviation using the cosine distance dece ( (x((=.) L.. The scaled sensitivity |e| - decays approximately log-linearly with deeper injection for patch tokens, indicating on-sphere attenuation within phases, whereas it increases for cls when injected late. indicating accumulation the read- out stage where global information is consolidated (Fig. 8B). Directional convergence rates mirror these roles. Tracking e = (i,, # reveals registers approach their terminal directions earliest, patches follow with smoother rise, and cls saturates only in the final phase where its reorientation peaks (Fig. 6, left). Together, these measurements demonstrate ViT depth implements specialized, phase-local dynamics with phase transitions, consistent with block-recurrent computation.
Angular speed with phase boundaries
arccos(#2+11 &z). Aggregating
5.50
patth
0.40
0.35
0.25-
0.15
10
20 30
layer 2oo
Figure 7: Token-specific an- gular speed with phase over- lays. Mean angular speed 8e across depth for cls, regis- ters, and patch, with max-cut phase boundaries from Sec. overlaid as vertical lines.
Low-Rank Collective Motion and Linearized Depth Flow. We analyze layer-to-layer token- wise angular updates and observe progressive collapse low dimensions. For token-wing angular updates (see App. C). Stable rank and effective rank decrease with depth to near six in the final phase, indicating subspace confinement (Fig. 9, left). Concurrently, patch-token coherence ge rises sharply

---

Published as a conference paper at ICLR 2026
A) Avarage perturbated directory
Self-correcting perturbation httpt//oonnttgtutuuuttttooooooo sensitivity
Patches
6635
Oheene Con EXEnE
(00ke0) Cotres dilane
55
122
-3.5
4.2.4 0022000000220000000
01
Figure 8: Self-correction under small angular perturbations. Perturbed trajectories bend back toward the baseline path, evidencing local basin stability. Sensitivity decays approximately log- linearly with remaining depth for patch tokens, but grows late for cls, consistent with stronger late-stage aggregation.
Stable
Potch coherence (2) (Slant)
Den
3.388
State naat
3.18
3.22
8.00
2.10
3.00
3.50
+..5
300 20
0.14
Figure 9: (Left) Low-rank updates and coordinated patch motion. Left Stable and effective rank of the layer-to-layer update matrix collapse with depth, indicating confinement to restricted subspace. Right Patch-token coherence with their mean update direction rises strongly, revealing increasing collective alignment. (Right) Dynamic Mode Decomposition (DMD) of depth dynamics. For each token group (cls, registers, patch), we average token states within the group and fit the exact-DMD (see Section D). Each layer state is fy-normalized to unit norm (trajectories on the unit sphere), so eigenvalue angles (x,) characterize angular updates, while radii [Λ,] measure contraction the sphere (not absolute feature-nont narture-normmooe growth). The DMD eigenvalues {A,} lie just inside the unit circle (dashed) and concentrate near the positive real axis, indicating near-neutral, mostly angular updates with mild on-sphere contraction. cls modes lie closest to +1 (longest memory), registers are slightly more dispersed, and patch shows the widest angular spread and stronger contraction. The cls spectrum decays slowest (highest effective rank/complexity), registers are intermediate, and patch decays fastest (lower-rank dynamics). Together, these spectra support weakly contracting, block-recurrent depth flow with token-specific complexity.
and peaks late, showing increasingly aligned, collective updates (Fig. 9, middle). The joint pattern (rank collapse with rising coherence) marks a transition from many weakly independent directions to few shared directions. We then linearize the depth flow via exact DMD on group-averaged, fz-normalized states, yielding 1 R Aä, with rank r = 10 (App. D). Eigenvalues are concentrated just inside the unit circle and near the positive real axis, consistent with weak on-sphere contraction and predominantly angular updates; cls modes lie closest to +1 (longest memory), registers are intermediate, and patches show wider angular spread and stronger contraction (Fig. 9, right). Stacked- depth singular spectra mirror this ordering, decaying slowest for cls and fastest for patches. These results indicate that late depth implements low-rank, near-neutral dynamics that compress variation into a small set of collective directions while preserving long-memory channels for cls.
5 DISCUSSION
We advanced the Block-Recurren Hypothesis (BRH), showing empirically and constructively (via weight-tied surrogates) that recurrence can match untied baselines, and we developed Dynamical Interpretability by viewing depth flow directions. This revealed (i) directional convergence to angular attractors with self-correction, (i) token-specific, phase-local dynamics with punctuated transitions, and (Mi) a late low-rank regime that coordinates updates low dimensional subspace. While residual pathways and stochastic depth appear implicated in block recurrence, isolating causal mechanisms will require controlled taining-dynamin most of DINOv2, a small residual gap remains that may call for improved recurrent distillation or additional time-van varying components. Overall, our work highlights a recurrence induced simplicity bias, suggesting current models admit recurrent version, implicating potential simpler analysis. Taken together, this recurrence-indin induced simplicity bias and its interpretability potential point toward a broader principle: in deep learning, recurrence finds way.
scale; and although two tied blocks recover
10

---

Published as a conference paper at ICLR 2026
6 ACKNOWLEDGMENTS
The authors would like to thank Chris Hamblin, Louis Béthune and John Hammond for many fruitful discussions. This work has been made possible in part by gift from the Chan Zuckerberg Initiative Foundation to establish the Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University. MJ is supported by the Kempner Institute Graduate Research Fellowship. TAK, TF, and RH supported by the Kempner Institute Research Fellowship. DB is supported by the Kempner Institute for the Study Natural and Artifical Intelligence at Harvard University.
REFERENCES
Julius Adebayo, Michael Muelly, lariaa Liceardi, and Been Kim. Debugging tests for model explana- tions. Advances in Neural Information Processing Systems (NeurIPS), 2020.
Lucian Alecu, Hugues Bonnin, Thomas Fel, Laurent Gardes, Sébastien Gerchinovitz, Ludovic Ponsolle, Franck Mamalet, Éric Jenn, Vincent Mussot, Cyril Cappi, et al. Can we reconcile safety objectives with machine learning performances? ERTS, 2022.
Anurag Arnab, Mostafa Dehghami, Georg Heigold, Chen Sun, Mario Lucic, and Cordelia Schmid. Vivit: A video vision transformer. ArXiv e-print, 2021.
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. Public Library of Science (PloS One), 2015.
Federico Baldassarre. Mare Szafraniec, Basile Terver, Vasil Khalidov, Francisco Massa, Yann LeCun, Patrick Labatut, Maximilian Seitzer, and Piotr Bojanowski. Back to the features: Dino as foundation for video world models. ArXiv e-print, 2025.
Randall Balestriero al. A spline theory of deep learning. In Proceedings of the International Conference on Machine Learning (ICML), 2018.
David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
Leonard Bereska and Efstratios Gavves. Mechanistic interpretability for ai safety-a review. ArXiv e-print, 2024.
Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. Adabins: Depth estimation using adaptive bins. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021.
Alexandre Boulch. Sharesnet: reducing residual network parameter number by sharing weights. ArXiv e-prinn, 2017.
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zae Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden MeLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. Towards monosemanticity; Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023.
Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine, 2017.
Diogo V. Carvalho, Eduardo M. Pereira, and Jaime S. Cardoso. Machine learning interpretability: A survey on methods and metrics. Electronics, 2019.
Gregory Chaitin. Algorithmic information theory. IBM journal of research and development, 1977.
Mathieu Chalvidal, Matthew Ricci, Rufin VanRullen, and Thomas Semre. Go with the flow: Adaptive control for neural odes. Proceedings of the International Conference on Learning Representations (ICLR), 2020.
11

---

Published as a conference paper at ICLR 2026
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in Neural Information Processing Systems (NeurIPS), 2018.
Christopher Chiu, Maximilian Heil, Teresa Kim, and Anthony Miyaguchi. Fine-grained classification for poisonous fungi identification with transfer learning, ArXiv e-print, 2024.
Radoslaw M Cichy and Daniel Kaiser. Deep neural networks as scientific models. Trends in cognitive sciences, 2019.
Taco Cohen and Max Welling. Learning the irreducible representations of commutative lie groups. 2014.
Timothée Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. ArXiv e-print, 2023.
Sayantan Dasgupta and Trevor Cohn. Improving language model distillation through hidden state matching. In Proceedings of the International Conference on Learning Representations (ICLR), 2025.
Soham De, Samuel L Smith, Anushan Fernando, Aleksandar Botev, George Bristian-Murant Albert Gu, Ruba Haroun, Leonard Berrada, Yutian Chen, Srivatsan Srinivasan, et al. Griffin: Mixing gated linear recurrences with local attention for efficient language models. ArXiv e-print, 2024.
Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Lukasz Kaiser. Universal transformers. ArXiv e-print, 2018.
Kamaludin Dingle, Guillermo Valle Pérez, and Ard Louis. Generic predictions of output probability based complexities of inputs and outputs. Scientific reports, 2020.
Finale Doshi-Velez and Been Kim. Towards rigorous science of interpretable machine learning. ArXiv e-print, 2017.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. image is worth 16x16 words: Transformers for image recognition at scale. Proceedings of the International Conference on Learning Representations (ICLR), 2020.
Thomas Fel. Sparks of explainability; recent advancements in explaining large vision models. PhD thesis, Université de Toulouse, 2024.
Thomas Fel, Victor Boutin, Mazda Moayeri, Remi Cadene, Louis Bethune, Mathieu Chalvidal, and Thomas Serre. A holistic approach to unifying automatic concept extraction and concept importance estimation. Advances Neural Information Processing Systems (NeurIPS), 36, 2023.
Thomas Fel, Ekdeep Singh Lubana, Jacob S Prince, Matthew Kowal, Victor Boutin, Isabel Papadim- itriou, Binxu Wang. Martin Wattenberg, Demba Ba, and Talia Konkle. Archetypal sae: Adaptive and stable dictionary learning for concept extraction in large vision models. Proceedings of the International Conference Machine Learning (ICML), 2025.
Jesseba Fernando and Grigori Guitchounts. Transformer dynamics: A neuroscientific approach to interpretability of large language models. ArXiv e-print, 2025.
Ruth C. Fong and Andrea Vedaldi. Interpretable explanations of black boxes by meaningful perturba- tion. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017.
Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. Proceedings of the International Conference Learning Representations (ICLR), 2019.
Jonas Geiping, Sean McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R. Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, and Tom Goldstein. Scaling up test-time compute with latent reasoning: A recurrent depth approach, 2025.
Borjan Geshkovski, Cyril Letrouit, Yury Polyanskiy, and Philippe Rigollet. The emergence of clusters in self-attention dynamics. Advances in Neural Information Processing Systems (NeurIPS), 2023.
12

---

Published as a conference paper at ICLR 2026
Amirata Ghorbani, Abubakar Abid, and James Zou. Interpretation of neural networks is fragile. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2017.
Leilani H. Gilpin, David Bau, Ben Z Yuan, Ayesha Bajwa, Michael Specter, and Lalana Kagal. Explaining explanations: An overview of interpretability of machine learning. Proceedings the IEEE International Conference on data science and advanced analytics (DSAA), 2018.
Michel X Goemans and David P Williamson. Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of the ACM (JACM), 1995.
Micah Goldblum, Mare Finzi, Keefer Rowan, and Andrew Gordon Wilson. The no free lunch theorem, kolmogorov complexity, and the role of inductive biases in machine learning. ArXiv e-print, 2023.
Liv Gorton. The missing curve detectors of inceptionyl: Applying sparse autoencoders to inceptionv early vision. ArXiv e-print, 2024.
Klaus Greff, Rupesh K Srivastava, and Jürgen Schmidhuber. Highway and residual networks learn unrolled iterative estimation. ArXiv e-print, 2016.
Eldad Haber and Lars Ruthotto. Stable architectures for deep neural networks. Inverse Problems, 2017.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. ArXiv e-print, 2021.
Anna Hedström, Leander Weber, Dilyara Bareeva, Franz Motzkus, Wojeiech Samek, Sebastian Lapuschkin, and Marina M-C Höhne. Quantus: an explainable ai toolkit for responsible evaluation of neural network explanations. The Journal of Machine Learning Research (JMLR), 2022.
Nhat M. Hoang, Do Xuan Long, Cong-Duy Nguyen, Min- Yen Kan, and Luu Anh Tuan. A comparative analysis of contextual representation flow in state-space and transformer architectures. ArXiv e-print, 2025.
Ann Huang, Mitchell Ostrow, Satpreet H Singh, Leo Kozachkov, Ila Fiete, and Kanaka Rajan. Inputdsa: Demixing then comparing recurrent and externally driven dynamics. ArXiv e-print, 2025.
Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kilian Weinberger. Deep networks with stochastic depth, 2016.
Minyoung Huh, Hossein Mobahi, Richard Zhang, Brian Cheung, Pulkit Agrawal, and Phillip Isola. The low-rank simplicity bias in deep networks. The Journal of Transactions Machine Learning Research (TMLR), 2023.
Alexia Jolicoeur-Martie Less is more: Recursive reasoning with tiny networks. ArXiv e-print, 2025.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models, 2020.
Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). Proceedings of the International Conference on Machine Learning (ICML), 2018.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan- Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. ArXiv e-print, 2023.
Andrei N Kolmogorov. Three approaches to the quantitative definition ofinformation.. Problems of information transmission, 1965.
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited, 2019.
13

---

Published as a conference paper at ICLR 2026
Matthew Kowal, Richard P Wildes, and Konstantinos G Derpanis. Visual concept connectome (vcc): Open world concept discovery and their interlayer connections in deep models. Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2024.
Leonid Anatolevich Levin. Universal sequential search problems. Problemy peredachi informatsii, 1973.
Ming Li, Paul Vitányi, et al. An introduction to Kolmogorov complexity and its applications. Springer, 2008.
Qianli Liao and Tomaso Poggio. Bridging the gaps between residual learning, recurrent neural networks and visual cortex, 2020.
Drew Linsley, Alekh Karkada Ashok, Lakshmi Narasimhan Govindarajan, Rex Liu, and Thomas Serre. Stable and expressive recurrent vision models. Advances in Neural Information Processing Systems (NeurIPS), 2020.
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang. Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In Proceedings of the IEEE European Conference on Computer Vision (ECCV). Springer, 2024.
Max Losch, Mario Fritz, and Bernt Schiele. Semantic bottlenecks: Quantifying and improving inspectability of deep representations. International Journal of Computer Vision, 2021.
Yiping Lu, Zhuohan Li, Di He, Zhiqing Sun, Bin Dong, Tao Qin, Liwei Wang, and Tie- Van Liu. Understanding and improving transformer from multi multi-particle dynamic system point of view. ArXiv e-print, 2019.
Yi Ma, Doris Tsao, and Heung-Yeung Shum. On the principles of parsimony and self-consistency for the emergence of intelligence. Frontiers of Information Technology & Electronic Engineering, 2022.
Chris Mingard, Henry Rees, Guillermo Valle-Pérez, and Ard A Louis. Deep neural networks have an inbuilt occam's razor. Nature Communications, 2025.
Guido Montúfar, Razvan Pascanu, Kyunghyun Cho, and Yoshua Bengio. On the number of linear regions of deep neural networks. Advances in Neural Information Processing Systems (NeurIPS), 2014.
Sabine Muzellec, Leo Andeol, Thomas Fel, Rufin VanRullen, and Thomas Serre. Gradient strikes back: How filtering out high frequencies improves explanations. Proceedings of the International Conference on Learning Representations (ICLR), 2024.
Thao Nguyen, Maithra Raghu, and Simon Korbblith. On the origins of the block structure phe- nomenon neural network representations. arXiv preprint arXiv:2202.07188, 2022.
Paul Novello, Thomas Fel, and David Vigouroux. Making sense of dependence: Efficient black-box explanations using dependence measure. Advances in Neural Information Processing Systems (NeurIPS), 2022.
Chris Olah, Nick Cammarata, Chelsea Voss, Ludwig Schubert, and Gabriel Goh. Naturally occurring equivariance in neural networks. Distill, 2020.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafranicc, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. ArXiv e-print, 2023.
Mitchell Ostrow, Adam Fisen, Leo Kozachkov, and Ila Fiete. Beyond geometry: Comparing the temporal structure of computation in neural circuits with dynamical similarity analysis. Advances in Neural Information Processing Systems (NeurIPS), 2023.
Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. On the difficulty of training recurrent neural networks. Proceedings of the International Conference on Machine Learning (ICMI), 2013. 2013.
14

---

Published as a conference paper at ICLR 2026
William Peebles and Saining Xie. Scalable diffusion models with transformers. ArXiv e-print, 2023.
Vitali Petsiuk, Abir Das, and Kate Saenko. Rise: Randomized input sampling for explanation of black-box models. Proceedings of the British Machine Vision Conference (BMVC), 2018.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. ArXiv e-print, 2021.
Maithra Raghu, Thomas Unterthiner, Simon Korblith, Chiyuan Zhang, and Alexey Dosovitskiy. Do vision transformers see like convolutional neural networks? Advances in neural information processing systems, 34: 121111111222,, 2021.
Michael Sander, Pierre Ablin, and Gabriel Peyré. Do residual neural networks discretize neural ordinary differential equations? Advances in Neural Information Processing Systems (NeurIPS), 2022.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. ArXiv e-print, 2019.
Peter Schmid. Dynamic mode decomposition and its variants. Annual Review of Fluid Mechanics, 2022.
Jürgen Schmidhuber. Discovering neural nets with low kolmogorov complexity and high generaliza- tion capability. Neural Networks, 1997.
Jürgen Schmidhuber. The speed prior: new simplicity measure yielding near-optimal computable predictions. International conference on computational learning theory, 2002.
Avi Schwarzschild, Arjun Gupta, Amin Ghiasi, Micah Goldblum, and Tom Goldstein. The uncanny similarity of recurrence and depth. arXiv preprint arXiv:2102.11011, 2021.
Thiago Serra, Christian Tjandraatmadja, and Srikumar Ramalingam. Bounding and counting linear regions of deep neural networks. Proceedings of the International Conference on Machine Learning (ICML), 2018
Peter Shaw, James Cohan, Jacob Eisenstein, and Kristina Toutanova. Bridging kolmogorov complexity and deep learning: Asymptotically optimal description length objectives for transformers. ArXiv e-print, 2025.
Sam Shleifer and Alexander M Rush. Pre-trained summarization distillation. ArXiv e-print, 2020.
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. Proceedings of the International Conference on Machine Learning (ICML), 2017.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. Proceed- ings of the International Conference Machine Learning (ICML), 2017.
Shawn Tan, Yikang Shen, Zhenfang Chen, Aaron Courville, and Chuang Gan. Sparse universal transformer. ArXiv e-print, 2023.
Matus Telgarsky. Representation benefits of deep feedforward networks. ArXiv e-print, 2015.
Adly Templeton, Tom Conerly, Jonathan Mareus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas Turne, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet. Transformer Circuits Thread, 2024.
Vincent Tjeng and Russ Tedrake. Verifying neural networks with mixed integer programming Proceedings of the International Conference Learning Representations (ICLR), 2019.
15

---

Published as a conference paper at ICLR 2026
Guillermo Valle-Perez, Chico Q Camargo, and Ard A Louis. Deep learning generalizes because the parameter-funee funnctior- map is biased towards simple functions. Proceedings of the International Conference Learning Representations (ICLR), 2019.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems (NeurIPS), 2017.
Andreas Veit, Michael J Wilber, and Serge Belongie. Residual networks behave like ensembles of relatively shallow networks. Advances in Neural Information Processing Systems (NeurIPS), 2016.
Siddarth Venkatraman, Vineet Jain, Sarthak Mittal, Vedant Shah, Johan Obando-Ceron Yoshua Bengio, Brian Bartoldson, Bhavya Kailkhura, Guillaume Lajoie, Glen Berseth, et al. Recursive self-aggregation unlocks deep thinking in large language models. ArXiv e-print, 2025.
Martina Vilas, Timothy Schaumlöffel, and Gemma Roig. Analyzing vision transformers for image classification in class embedding space. Advances in Neural Information Processing Systems (NeurIPS), 2023.
Yue Wang and Sai Ho Chung. Artificial intelligence in safety-critical systems: a systematic review. Industrial Management & Data Systems, 2022.
Ronald J Williams and David Zipser. A learning algorithm for continually running fully recurrent neural networks. Neural computation, 1989.
Andrew Gordon Wilson. Deep learning is not so mysterious or different. ArXiv e-print, 2025.
Zuxuan Wu, Tushar Nagarajan, Abhishek Kumar, Steven Rennie, Larry S. Davis, Kristen Grauman, and Rogerio Feris. Blockdrop: Dynamic inference paths residual networks, 2019.
Qing Yun. Vision transformers (vits) for feature extraction and classification of ai-generated visual designs. IEEE Access, 2025.
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. Proceed- ings of the IEEE European Conference on Computer Vision (ECCV), 2014.
16

---

Published as a conference paper at ICLR 2026
A TRAINING BLOCK RECURRENT FOUNDATION MODELS
All Raptor variants (k: 2, 3, 4) are trained on top of DINOv2 (ViT-B with registers) (Darcet et al., 2023) and use the same transformer architecture. The model utilizes a feature dimension of 784, an MLP ratio 4, and 12 multi-head attention heads. The depth-scale MLP each block generates three separate scaling vectors; we provide the full formulation in Appendix A.3. For each Raptor variant in Table 2 and Figure 5, we train train three models, each with a different seed. We measure the performance of each model and report aggregate statistics. Below, provide the hyperparameters and training settings used to train Raptor, describing the layer divisions for different values of followed by the training procedure.
For each choice of k, the encoder layers are divided into blocks as shown in Table 3. We select these partitions using the max cut algorithm applied to the DINOv2 ViT-B activation layer-layer cosine similarity matrix using 10000 samples from the ImageNet- Ik training set.
k:
Block
Input
Predicted Layers
Block
-> 1-7 7- 8-12
Block
Block
0 - 1-7
Block
7- 8-10
Block
10 + 11-12
Block
0 1-4
Block
5-7
Block 3
7 8-10
Block
10 - 11-12
Table 3: Layer splits used for training with different values of
In the first stage, each block is trained independently on the subset of layers it is responsible for predicting. For example, when ImageNet-Ik train split for 20 epochs with batch size of 64 using the Adam optimizer with a weight decay of 0.0001. The learning rate follows linear warmup for 10,000 steps
3, Block is trained predict Layers 1-7. We train on the
10-4, followed by a cosine decay to 1 x 10-6 The teacher forcing loss weight À is annealed from 0.5 to over the first epochs. For the third block, we use specific token loss weights of ders = 0.34, Xreg = 0.33, and Apatch = 0.33.
In the second stage, after independent training. all blocks are connected to autoregressively predict Layers 1-12 end-to-end. Each block still predicts its designated segment, but the entire model now backpropagates through the full sequence. We maintain the same dataset, epoch count, batch size, weight decay, optimizer, and learning rate schedule as the first stage. However, the token loss weights are adjusted to Acts = 0.45, Areg = 0.1, and Apatch = 0.45.
The number of recurrent timesteps each block executes corresponds directly to its layer coverage; for instance, in the k=3 setup, Block runs for 7 timesteps, Block 2 for 3 timesteps, and Block for timesteps.
For a given batch of ground truth DINOv2 activations X E NxTxD and Raptor predictions X < RNXTTD, where N denotes the batch size, T the number of tokens, and D the feature dimension.
We decompose the activation tensor along the token dimension into three distinct components corresponding the token types: the class token Xcls E NxxxD the register tokens Xreg E R N N.gg and the spatial patch tokens Xpatch E NxNaaaX such that X concat( Xeis: Xreg: Xpaten).
We define the total reconstruction loss function L(X, X) as weighted sum the errors for each component:
L(X,X) dets |
X 12 Ares | Xres 11:
+ +Apatch 1xpatch - Xpatch
(3)
17

---

Published as a conference paper at ICLR 2026
A.1 PHASE DISCOVERY VIA CONTIGUOUS MAX-CUT ON THE LAYER -AYER SIMILARITY
Problem setup. Let E RLX be the (symmetrized) layer-layer similarity matrix, where Sij. measures the similarity between layers and j (for example, cosine similarity). We seek a partition of depth into & contiguous segments or "phases". II = [1b,,e]]..... .i]....,[b,, [bk, ex]} with 1 = bi S < b2 S b1+1, that maximizes within-block similarity (equivalently,
62 by I Es L and Er minimizes cross-block cut).
Objectives. For segment [ of length = - define:
sum (i.j) =
Spq
ofldiag(i.j) = sum(i,
Σ
Spp-
p=i 9=
p=1
We consider additive segment scores g(i,j) by computing the final weighted mean
sum
Maximizing Ek= g(be, ex) prefers blocks that are internally similar and, by contiguity, implies small cross-block interfaces (a contiguous max-cut on the line).
Fast block queries via 2-D pretix sums. Precompute a 2-D prefix (summed-area) table P E [(4+1)x(L+1) with Pre = -EvrE Sun. Then any submatrix sum obeys
sum(i,j) Pjt1,411 Pej+1 - Ps+1, + Ps..
diagonal sums use a 1-D prefix over dSag(S). This is sometimes referred to as the "integral image" trick.
in (1) time;
Contiguous DP solver (O(K??)). dp(t, be the best score for partitioning layers 1..j into blocks. With minimum block length m,
dp[1. = g(1.5) (j 2 m),
dplt.j] -E(.m=L....-mm
max
dp - 1, 1.1]+ + g(i i + 1,5),
for t = k and 1 2 m.. We keep backpointers prev[t, to recover boundaries by backtracking from jl= With g(-) evaluated O(1) by prefix sums, the overall complexity ( time and (k:L.) memory. This DP structure mirrors classical optimal 1-D) segmentation//partiiinnn solvers.
A.2 TEACHER-STUDENT RECONSTRUCTION R
To stabilize measures of pairwise vector similarity over large hyperparameter sweeps when fits may be poor, we use an alternative calculation for Rd for small-scale models (Figure 4). Here, we first regress the student's ''s cls or patch tokens R to the corresponding teacher tokens E NDD using ordinary least squares with bias term. N the number of tokens and / is the dimensionality of the token. We then calculate the average of the R² values between the true teacher token vectors and the student's reconstruction of those vectors. Note that this regression is purely I-dimensional rescaling and shifting for each student token vector. This results in R. value that is bounded between and 1, and can understood to present the 'explainable variance' between the student and teacher representations.
A.3 DEPTH-SCALING MECHANISM
We employ a lightweight conditioning MLP each block that maps a scalar depth coordinate 2 E IR to layer-specific scaling vectors. For layer with feature dimension D 784, the depth-scale MLP generates a modulation vector v E R3D
The MLP projects the scalar input to hidden dimension h 16, applies SiLU activation, and projects to the output dimension. We add a unity offset to the MLP output to ensure the scaling factors center around (identity behavior) at initialization. The compound scaling vector S is computed as:
18

---

Published as a conference paper at ICLR 2026
S = (W2 SiLU( 12 +b1) +b2) +1
(4)
where denotes vector of ones. We initialize the weights and biases of the final projection (W2, b2) with normal distribution (1+ 0, 10= ensuring that the block initially acts close to standard behavior.
The vector S is split along the feature dimension into three component vectors: Satt: Smips RR. These vectors modulate the signal via element- wise multiplication (0) at three points in the transformer block: the attention residual, the MLP residual, and the final block output. Given an input sequence X, the forward pass is defined as:
X' X + Sam
LSI (Atn
X X' Smip o (ML((L((
(6)
Y Sout X"
(7)
where LS refers to LayerScale, LN is Layer Normalization, and broadcasting is applied over the sequence length dimension.
A.4 LINEAR PROBE FINE-TUNING
We fine-tune linear probes on three downstream datasets: ImageNet-Ik (classification), ADE20k (semantic segmentation). and NYUv2 (monocular depth estimation).
For ImageNet- Ik and ADE20K, learning rate decay. For NYUv2, we use AdamW with GradScaler and mixed precision training. All probes operate on the final block's prediction of Layer 12, using either the cls token, patch tokens, or both, depending on the task. The detailed hyperparameters are shown in Table 4.
the AdamW optimizer with linear warmup followed by cosine
For NYUv2, we adopt an approach similar to Oquab et (2023). Specifically, we use images at 480 x 640 resolution and center pad them so that the dimensions are multiples of 14. We feed the images through the model and extract the predictions from the final layer. The cls token concatenated with the patch tokens, and the spatial resolution is upsampled by a factor of 4. Both the cls and patch tokens are upscaled, after which the cls token concatenated to each patch token. We treat this representation as the "logits." To obtain depth, we normalize the logits with softmax and compute the weighted average of the centers of 256 evenly spaced bins. Then, we upsample this representation 480 640 and consider the result our depth. For training, use the loss function introduced by Bhat et al. (2021).
Hyperparameter
ImageNet-1k
ADE20k
NYUv2
Epochs
15
10
25
Batch Size
512
32
128
Base LR
1 10-3
10
10
Weight Decay
1 x 10
x 10
x 10-2
Grad. Clip Norm
1.0
1.0
1.0
Warmup Iters
100
100
100
Optimizer
AdamW DINOv2 classification probe
AdamW Random segmentation head
AdamW + GradScaler Random depth head concat(cls, patch)
Head Init.
Input Tokens
concat(cls, mean patch)
Patch
Table 4: Linear probe fine-tuning hyperparameters across datasets. Base LR denotes the peak learning rate before cosine decay.
B EVALUATING MAX-CUT ALGORITHM
To evaluate the efficacy the max cut algorithm (Figure 3), we initialize a ViT-B with ImageNet- 21k and ImageNet- 1 weights and replace the classifier with randomized linear probe applied
19

---

Published as a conference paper at ICLR 2026
to the CLS token. We fine-tune the full model CIFAR-100 for 10 epochs, achieving 90.7% validation accuracy. Using activations extracted from the CIFAR-100 100 training set, we compute layer-wise cosine similarity matrix and apply our max cut algorithm to identify Raptor partitions for k E [2, 3, 4). We compare these against two baselines: 10 random contiguous partitions and 10 random non-contiguous partitions (laheled "Random Shuffle"), ensuring the max-cut solution is excluded from the samples. Finally, we train Raptor models on each partition configuration for 100 epochs and evaluate performance using the linear probe from the initial fine-tuning stage.
C DYNAMICS PROTOCOLS AND METRICS
This appendix consolidates definitions and experimental procedures used in Sec. 4. All measurements are performed on ImageNet validation data. For aggregate statisties, we use 10k randomly sampled validation images. For trajectory visualizations (e.g., Fig. 6), we select five ImageNet classes with 1,000 images each. Inputs are resized to 256 pixels on the shorter side and center-cropped to 224 x 224. Unless otherwise noted, we use DINOv2-Giant with four register tokens from the official implementation.
Normalization. Token states E Id at depth & are decomposed into norm and direction. We study normalized states
i x
E gd-1
so that dynamics are restricted to the unit sphere.
Directional convergence. Directional similarity to the terminal representation measured by
Te (6 Lt)
which traces the angular alignment of layer & to the final state.
Angular speed. Per-layer angular update magnitude is defined
arccos(#z+1,.5)
Statistics of 8ı are stratified by token type.
Phase overlays. Phase boundaries obtained from the max-cut segmentation of representational similarity matrices (Sec. 2) and used as vertical markers in angular speed and sensitivity plots.
Perturbation protocol. To probe stability, add perturbation su at layer L,
8 + su,
u~ ~~ (O,
and follow the normalized trajectory thereafter. Sensitivity is quantified by the terminal cosine deviation
(E))) +t) = 1 -i deas (2
(E.0) .L)).
Low-rank and coherence metrics. matrix Up. Stable rank is given by
For angular updates A (6)
we form the update
(U;) = IU& 12
1
and coherence by
(A Ae)
A 1/ 0
Kit
|
D DYNAMIC MODE DECOMPOSITION
Let f be a trained VIT with transformer layers (fr For a E X, denote by Ax(x) E Txd the token matrix at depth with T
R + P (cls, R registers, P patch). Form group states by
20

---

Published as a conference paper at ICLR 2026
within-layer layer averaging
(cls) (x) = At(
(rea) (x)
Σ
Ac(x),
z(potch))
=
Σ
Ar(T):
2/
R
LETI
LE 1E7Patch
and enforce per-layer &g normalization on the group averages
(0) (x) gd- Rd.
(x)
2 (x)
All DMD fits below performed independently for each g {cls, reg, patch) on the depth trajectory (2)) #0L (z). We start by stacking states along depth to form
(9))
Y
E ((++))d
X1 = -X(2) E R
X₂ = (Xx(X1))) R
(xL))
DMD fits a single linear depth-step map A with X₂ ~ AX For the exact DMD at rank r, we compute the SVD X UEV and select fank(Xi). Let U, E, V, be the leading blocks and define the reduced operator
X,V,E, e ERa Diagonalize ÁW- WA with A = diag(A (
Ar) E Cr and W E Crxr. The exact DMD
modes in ambient space are
+ = XvVV W E Cdxt.
Modal amplitudes for the initial state are b = क5 x E C", One-step and t-step reconstructions are (g)
a ~ A
A b for 2 0
with the induced linear predictor A X V. U
No affine offset is fitted.
On-sphere interpretation. Because each xt lies on 3d-1 the map A is a best linear approximation of the depth flow restricted to the unit sphere. For X; = e the modulus [^s] measures contraction of directions within the span of modes on the sphere and the angle 0g captures per-layer rotational change. The spectral radius p(A) = max, |Ay] and the median of [A,] summarize contraction strength. In particular this explain why all the eigenvalue in Figure 9 are contain in S see the report of the eigenvalue cloud Ai in the complex plane and the singular spectrum 1 of X,.
E LEVIN COMPLEXITY UNDER O-BRH
Background & Positioning. There is long line of work linking plain Kolmogorov complexity Kol- mogorov (1965); Chaitin (1977) to its assource-boundee variants, notably Levin's 's bounded mea- sure, which penalizes programs by both description length and runtime (the [p] + log T form) (Levin, 1973; Li et al., 2008). Deep learning research has not remained isolated from these ideas: connec- tions have been explored both for discovering simpler neural networks Schmidhuber (1997) and for contextualizing runtime priors Schmidhuber (2002) within neural architectures.
In our case, the proof is a direct application of this standard framework as formalized in Li et al. (2008): once exhibit prefix-free program that computes fa and bound its running time T(p). Levin complexity immediately yields Kfeii(ff) < [p] + log T(p) + O(1). The only domain- specific ingredients are (i) BRH-based encoding that prices the tied blocks and schedule at Σj_ DLy (B,)) + O(k log E) using standard self-delimiting integer codes, and (ii) a runtime- parity assumption linking T(p) to the model's untied runtime Rífa up to machine and invariance constants. Formally:
Claim 2 (BRH guarantees low Levin complexity). Let fo satisfy 0-BRH with < tied blocks (B = and schedule (nz)f=1 with Σ, ny Let R(-) denote block runtime and define the unfied
21

---

Published as a conference paper at ICLR 2026
teacher runtime R(fe) = Ej=1n, nrR(BB)) assume runtime parity R(B,) & (1 () Then
&
KLevin 5i)s DL(O(B,)) + OOk log log +
Proof. We follow the standard time-bounded description framework of Levin (1973), and contextual- ized it to the 0-BRH setting (perfect reconstruction along the entire depth trajectory), in the spirit of the sketch already outlined the paper. We fix universal prefix free machine U and adopt the same finite-precision arithmetic used by the target model to define (LU((); this ensures that encoding parameters with length (B,)) bits suffices to reproduce the exact numerical behavior of the corresponding block under that arithmetic. With this convention, "exactly computes" means bit-for-bil bit equivalence with the reference implementation fr (under its inference precision).
The proof proceeds in three steps: we first bound the description length, then verify correctness (equality of the program output), and finally bound the runtime before combining these results to derive the final bound.
The first step consists in bounding the description length. We describe a prefix- -ree program p that parses (i) the tied block parameters and (ii) celf-delimiting schedule, with total length
Ipl (LU(O(B))) + Olk () O(1).
For each tied block, encode its parameters using the scheme underlying DLU, yielding prefix code of length (LU(((())) Concatenate the k codes and prepend O(1) header that instructs U how to parse and reconstruct each B, from its bitstream. Encode the schedule (m1,..., ny) and the value of self-delimiting integers (any standard universal code suffices), which costs O log bits because S and E; fi = L. The parser is constant-size routine bundled with absorbed into (1))
Now we show that this construction is valid under O-BRH. Let ge := (((a) B (n)) denote the composed tied map realized by iterating the reconstructed blocks according to the parsed schedule. By O-BRH, we have exact functional equality
ft
B (nt)
B (ni) ge,
i.e. the tied computation reproduces the full internal trajectory (not only the terminal output). Because p encodes the same block parameters at the precision that defines DLU and U evaluates precisely the same composition, we obtain, for all inputs x,
(x))xx) =
Finally, we get the runtime bound to derive the Levin Complexity. On the fixed machine U, single evaluation of block B, incurs time at most e R( B;) for machine dependent constant c 0 (standard machine invariance up to constants; the invariance discussion Kolmogorov/I Kolmogorov/l.evn-styy/ evin-style
arguments (Kolmogorov, 1965; Levin, 1973)). Executing the schedule therefore costs
T(p) S Σ nj R(B,).
Applying the runtime parity assumption R(B,) S (1 + 8) R(B,) for all j and using R(ft) = RBB( gives
T(p) c(1 + Σ n. R(B,) c + R(f(.)..
By the definition of Levin's Lime-bounded complexity
KL.vin (fe) E |pl + log T(p) 0(1),
22

---

Published as a conference paper at ICLR 2026
and combining with the bounds from the previous steps yields
KLevin (fe) Σ (LB))( + Olklog E) + log R(ff)) log c-
log(1 + 6) + OL).
=
Absorbing the machine constant and the parity slack into O(1) gives the stated inequality.
23

---

Published as a conference paper at ICLR 2026
F TRAINING LOSSSES AND ALGORITHM
Distillation.
Teacher Forcing. Predict next state from ground-truth activations
Autoregressive.
Predict final layor only
Predict next state from own predictions
real trajectory
predicted trajectory
error signal
Figure 10: Three training paradigms for learning recurrent approximations. Each panel shows three token trajectories through depth. Gray dashed lines with filled circles represent the ground-tree truth teacher trajectories; black solid lines with filled circles show the student's ' predictions; colored dotted lines (with labels) indicate the eтror signal between predicted and ground- fround-truth states. Left (Distillation): The student network directly predicts the final layer from the initial state, with no supervision on intermediate representations. Error is measured only at the terminal state, providing no guidance on the representational trajectory. Middle (Teacher Forcing): At each depth step the student block predicts Ø = (xx²)) using the ground-truth activation from the teacher. Vertical arrows indicate where the student "resets" " to ground-truti states. This enables efficient parallel training and prevents eπrог accumulation, but creates train-test mismatch since the model never learns to handle its own prediction errors. Right (Autoregressive): The student autoregressively predicts øz41 ((x)) using its previous predictions, matching inference conditions. Errors can compound across depth (shown by increasing deviation between trajectories), requiring the model learn self-consistent, closed-loop dynamics. Our two-stage training (Sec. combines both approaches: Stage 1 uses teacher forcing for stable, parallelizable pretraining; Stage 2 switches to autoregressive training ensure self-cone eeff-coniistenyy inference.
a_gt teacher (batch) extract teacher activations a_ar a_gt[o] # start at first hidden state loss
for in range(num_layers)
get_block(1) get recurrent block for phase out_tf, out_ar b(a_gt(l]), b(a_ar) # dual forward passes
explicit hybrid loss 1_tf dist(out_tf, a_gt[1+1])
l_ar = dist(ou a_gt[1+1])
loss += A 1.tf (1 X) 1_ar
a_ar out update student state
loss. backward)) optimizer.sten
Figure 11: Hybrid training loop. The model minimizes convex combination of Teacher Forcing and Autoregressive errors, balanced by X.
24

---

Published as a conference paper at ICLR 2026
G
SUPPLEMENTARY RESULTS
B(i)
2003
JIDE
100
1900
15900
S6890
6900
Layers
A
VIT-AAPTOR
((ii)
Intra-block smllrit of VIT
C
VIT-RAPTOR tidden-state
D
WI-RAPTOR
output layer (CLS) similntty
represetttions (mean ke3)
simlartty over training
acouracy during training
moan sinilarty
1.0
outpus lewer ammlelty
VIT (val)
R Rattral layer (CLS takon)
0.9
VIT (Fain)
Accunity (9))
(val)
0.8
8
0.4
0.8
0.-
CLS token
Patch
0.2
10'
10"
trinning step
Training stop number
Traniing stop number
Untraned -oo resitiall bransf Untrained- skip br tranch
0.0
Trated
Untoared
Figure 13: Similarity and distillation dynamics during training and overtraining. A) ViT- Raptor similarities between four pairs of VIT and Raptor networks. First from the left, fully trained (but not overtrained) ViT network trained on CIFAR- 100 and a Raptor student network trained (with k=3). Second, untrained VIT network, and Raptor student network. Third, an untrained ViT network with no residual branch its architecture, so the entire network just skip connections; and Raptor student network. Fourth, untrained VIT network with skip connections, so the network no longer contains skip connections and is simply a feedforward network; and Raptor student network. Bi) Layer-layer cosine similarities during training that is allowed to be overtrained (overtraining starts at around step 10000, see (D)). Bii) Mean intra- block cosine similarity with max-cut k=3 during training. C) Teacher-studee of ViT during training showing divergence after overtraining starts. D) CIFAR- 100 accuracy in a ViT that was allowed to overtrain. Divergence between 'train' (green) and 'val' (blue) lines shows over training. X-axis is the same as in panels (B) and (c).
ViT network
Raptor reconstruction
Raptor Cosine Similarity per Layer
1.0
0.8
Cosine Simiarity
0.4
0.2
k=2
k=3
k=4
0.0
2
4
5
7
9
10
11
12
Layer
Figure 14: Cosine similarity remains high through layers. Cosine similarity between activations from Raptor models (A: k = 3, 4) and activations from ZINOV2-Base on the Image validation set. The consistently high similarity (mostly >0.7) indicates that Raptor effectively captures the dynamics of the original model, with the high alignment in the final layer confirming that the block-recurrent mechanism successfully approximates the target output representations.
25

---

Published as a conference paper at ICLR 2026
DINOv2 ImageNet-1k Validation Accuracy vs. Number of Replaced Layers
0.8
Condition
0.7
Intra-Bur Bltra-Block
Inter-Block
0.6
0.5
Accuracy
0.4
0.3
0.2
0.1
3
(number replaced layers)
Figure 15: Causal intervention. DINOv2-Base accuracy on ImageNet- k validation set with 1, 2, and layers replaced with another layer = 1, k: 2, 3, respectively). Intra-block refers to replacing layer with another layer from the same block. Inter-block refers replacing a layer with a layer from a different block. Blocks are determined by the max cut algorithm. The significantly higher accuracy of intra-block replacements (blue) compared to inter-block (orange) confirms that layers within block are functionally interchangeable in a way that any two arbitrary blocks are not, supporting the block-recurrent hypothesis.
DINOv2 Giant
120
100
80
L2 norm
60
40
20
0
10
20
30
40
Layer
Figure 12: Depth-wise feature norms. Magnitudes grow with depth, motivating analysis directions.
26


Reviews:
[{"review_id": "W9PuM3JV83", "reviewer": "Reviewer_qpb3", "summary": "The paper introduces a fresh perspective on vision transformers, showing that their computational flow can be explained through a block-recurrent computation structure. To support their claims, the authors present several visualizations and training experiments. After establishing the connection between vision models and block-recurrent structure, the authors leverage this perspective to propose a new interpretability approach inspired by principles from dynamical systems, which can be used to better understand vision models.", "questions": "Q.1. Do the authors think that their insights can shed more light on the connections between vision diffusion models and Raptor-like vision transformers, given that they used significantly fewer parameters and recurrent computation? Such insights could provide valuable information on the optimal trade-offs between FLOPs and parameters in vision models.\n\nQ.2. What is the compute gap required to train Raptor models compared to the baseline? Please provide relative numbers to help illustrate how challenging it is to train such models.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- The paper presents a fresh view on the computational flow of vision transformers, providing a better understanding of these important models. \n\n- As far as I know, the idea is novel and creative. \n\n- The authors include extensive analyses, including ablation studies and interpretability experiments.", "weaknesses": "**W.1. Limited empirical analysis for the main claim:**\n\nI believe the main idea in the paper (the recurrent block view) should be justified through additional experiments. In its current form, the argument relies heavily on the power of distillation, which is a very strong model compression technique. To verify that this is not the primary cause of the success of Raptor-like models, I suggest the authors conduct the following two experiments:\n\n**(i) Block structure variation:** Train Raptor-like models with a similar number of blocks and layers per block, but where each block includes a non-contiguous (fragmented) set of layers (for example, instead of [1,2,3], [4,5], [6,7], use [1,4,6], [2,3,7], [5]). Please note that previous work [a] which use smart grouping found that sharing parameters between non-adjacent layers can be very effective, somewhat contradicting the results reported in this paper.\n\n**(ii) Causal intervention analysis:** At test time (without fine-tuning or retraining), replace the weights of K layers with (a) weights from different layers within the same block, and (b) weights from random layers elsewhere in the model (not from the same block). If, for several K values, the accuracy under option (a) is consistently higher, this would provide much stronger support for the authors’ claim.\n\n\n**W.2. Strengthen the connection to dynamical systems:**\n\nFrom my perspective, the current motivation is somewhat vague. For example, a devil’s advocate might ask what differentiates the proposed approach from the well-known layer-by-layer analysis. I suggest that the authors include a dedicated section to motivate their decision to use dynamical systems. Incorporating technical terms and principles from the field of dynamical systems could make the argument more compelling and insightful.\n\n**W.3. Minor:** \n\nThere are several visualization issues. For example, the colors in Fig. 6 (right) are unclear, with three similar shades of green. Additionally, the excessive use of vspace or other compression commands makes the text too dense.\n\n[a] DYNAMIC LAYER TYING FOR PARAMETER-EFFICIENT TRANSFORMERS. David et al. ICLR 24", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BTHA7cZpT2", "reviewer": "Reviewer_y5qx", "summary": "This paper introduces the Block-Recurrent Hypothesis (BRH) for Vision Transformers (ViT): a pretrained ViT’s depth can be organized into contiguous “phases” such that the full computation can be reproduced by reusing a small set of shared blocks applied recurrently. The paper then instantiates the BRH by proposing Raptor, a weight-tied surrogate that uses max-cut and matches the activation of pretrained ViTs. Experiments on DINOv2 show that Raptor recovers most of the linear-probing accuracy of the original model with only a fraction of layers. Building on BRH, they further propose a dynamical interpretability view of depth, reporting directional (angular) attractors, token-specific dynamics, and low-rank update collapse in late layers.", "questions": "See above.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. From a hypothesis to an actionable algorithm along with interpretability insights, the paper is very insightful and can be very valuable to any follow-up research in ViTs.\n2. Extensive experiments, both small and large scale, are presented to test the proposed hypothesis.\n3. The paper is clearly written and well presented.\n4. The stochastic depth section is very insightful and interesting.", "weaknesses": "1. The large-scale experiments only used DINOv2. While it is a very widely-used backbone, experiments on other models such as SigLIP could greatly strengthen the work and show its broader applicability.\n2. ViTs are now extensively used in the context with vision-language models (VLMs) now. When tuned alongside the LLM, it is natural to think whether the proposed hypothesis still holds.\n3. Compute tradeoffs are not fully quantified. While FLOPs per iteration is considered, wall-clock, memory behavior, and parallelism effects of recurrent blocks vs conventional ViTs aren’t thoroughly analyzed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cT9ftZVwSJ", "reviewer": "Reviewer_89sj", "summary": "This paper investigates the Block-Recurrent Hypothesis (BRH) in Vision Transformers (ViTs), which posits that the deep stacks of layers in a trained ViT can be decomposed into a small number of computational phases that repeatedly perform similar transformations. To operationalize this idea, the authors propose Raptor (Recurrent Approximations to Phase-structured Transformers), a framework that reconstructs a full ViT using only a few weight-tied recurrent blocks.\n\nInstead of directly proving recurrence analytically, Raptor demonstrates the existence of block recurrence by example— training recurrent modules to reproduce not only the final outputs but the entire layer-wise activation trajectories of the teacher ViT. The resulting models achieve remarkable fidelity: with only k = 3 recurrent blocks, Raptor can recover ≈97% of DINOv2 ViT-B’s ImageNet-1k linear probe accuracy and a layer-wise R² of 0.73, indicating strong internal consistency between the reconstructed and original representations.\n\nThe work thus offers a constructive, empirical validation of the BRH, showing that the depth of ViTs can be “compressed” into a small number of iterative computational phases without major loss of functionality. It further opens the door to analyzing ViTs as dynamical systems, revealing phase-wise convergence behavior, token-specific dynamics, and low-rank collective motion in the representational flow across depth.", "questions": "The analysis and empirical validation of the block-recurrent hypothesis are quite compelling. However, has the team explored whether the recurrent compression (k ≪ L) can be leveraged for deployment efficiency—for instance, reducing parameters and FLOPs in large foundation models (e.g., MLLMs or multimodal ViTs) without sacrificing performance?\n\nIn principle, if a ViT can be represented with a few recurrent blocks, one might expect significant efficiency gains in inference-time or memory-limited environments. Has any investigation been conducted to measure these potential deployment benefits in practice?\n\nBeyond classification, could Raptor-style recurrence be applied to dense or structured vision tasks (e.g., detection, captioning, video understanding)? It would be interesting to see whether the same “computational phase” structure holds when the model needs spatially adaptive or temporally extended reasoning.\n\nGiven the recent integration of ViT backbones into multimodal large language models, do the authors see any prospects for shared recurrent computation across modalities, or for cross-modal recurrence alignment?\n\nFinally, could the dynamical interpretation of ViTs—especially the phase-wise attractor structure—help in regularizing or stabilizing the training of much deeper transformer-based architectures?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "* Rather than relying on correlation or representational similarity alone, the authors operationalize the block-recurrence hypothesis by functionally reconstructing the ViT’s internal computations.\n\n* Raptor replicates the performance and hidden representations of large ViTs (e.g., DINOv2 ViT-B) with only a few recurrent modules, across classification, segmentation, and depth estimation tasks.\n\n* The two-stage Teacher Forcing + Autoregressive procedure, with gradual annealing and end-to-end refinement, shows careful engineering and clear causal analysis (e.g., TF-only collapse).\n\n* The dynamical characterization—phase-wise convergence, angular attractors, weak contraction, and token-specific stability—offers valuable interpretability for transformer dynamics.\n\n* The work suggests a compact, iterative view of transformer computation, resonating with ideas in dynamical systems, implicit networks, and recurrent neural architectures.", "weaknesses": "* The experiments mainly focus on image classification and linear probing; while segmentation and depth tasks are included, the demonstrations remain limited compared to the full spectrum of real-world applications.\n\n* The use of max-cut–based phase segmentation is heuristic; the stability or uniqueness of the phase decomposition is not fully explored.\n\n* The need for depth scaling indicates that the effective dynamics of ViT layers are non-stationary, so full recurrence may not hold uniformly across depth.\n\n* While the recurrence hypothesis is supported empirically, it is not yet clear whether such recurrence can be directly exploited for efficiency or deployment in large-scale multimodal or generative models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ExP483bOBh", "reviewer": "Reviewer_oq2Q", "summary": "The authors hypothesise (via their so-called Block-Recurrent Hypothesis) that ViTs operate in blocks, and that the computation of individual blocks may be compressible while allowing the same outputs.\n\nThey use the BRH's validation to train Raptor, which has weight-tied layers, almost acting like an RNN in the direction of transformers.\n\nFinally, the authors offered a novel approach of 'dynamical interpretability' to analyse Raptor as a recurrent systems in the depth dimension", "questions": "How many 'timesteps' was each recurrent block in Raptor run for? Table 3 provides layer splits, but could you clarify this more explicitly in the main text? For instance, does Block 1 in the k=3 configuration iterate 7 times to predict layers 1-7?\n\n\"Unlike classical distillation, which typically supervises logits (and occasionally a few intermediate “hints”), we enforce one-to-one alignment of all layers representations across the entire depth for the same inputs.\" - were distillation methods of this form compared against? Are there baseline compression methods that Raptor can be compared against?\n\nThe ε in the definition of BRH was never explicitly measured IIUC - does this vary across datasets/architectures?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "Really well motivated hypothesis, analysis and subsequent model design. Analysis was simple and straightforward - starting with representational similarity analysis then validating the connection to computational similarity by constructing Raptor. In turn, the construction/training objectives of Raptor were simple and easy to follow. The paper was a joy to read over all.", "weaknesses": "Introduction to different embeddings/tokens for those with a background in recurrent systems rather than ViTs would be very helpful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AS3kW2DRjD", "reviewer": "Reviewer_QZzj", "summary": "This paper interprets vision transformers under a Block-Recurrent Hypothesis, investigating if the original computation that such models perform using L blocks can be achieved using k << L recurrent blocks while reproducing the internal activations of the full model. The paper then views these new models under a program of dynamical systems analysis.", "questions": "Most of the representational structure that is explored in this paper involves looking at the level of the output of transformer blocks. I would be curious to know how interactions within the attention matrices change across the model hierarchy, or whether you similarly observe functional reuse at the output of the attention layer in each block. Transformers can be thought of as possessing two computational pathways within each block, a fast one that is the result of the residual connection, and a slow one that is the result of a self-attention module. How do these two separate pathways interact with the Block-Recurrent Hypothesis, if at all? This is not a limitation of this current manuscript, but I would love to hear thoughts from the authors if they have any.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I appreciate the very thorough account of the motivation, construction, and analysis of the Block-Recurrent Hypothesis. The mathematical definitions and propositions look sound, and the experiments around the identification of functional reuse and dynamical interpretability are well characterized. Understanding the dynamical structures within vision transformers that have so ubiquitously entered as de-facto models for visual tasks is interesting and necessary, and the paper does a good job at teasing some of that out. Specifically, the paper systematically explores the emergence of blocks using a controlled setting with a tiny vision transformer, and then scales the analysis to a larger DINOv2 ViT-base model. Additionally, it performs a series of experiments to understand dynamical properties of ViT tokens, offering a unique perspective on the appropriateness of directional (angular) geometry as a paradigm for dynamical analysis.", "weaknesses": "For table 1 and figure 5, it seems that the performance experiments were not repeated multiple times with different partitions or different random model seeds. I would have preferred to see some form of error bars quantifying how robust or varied the metric results are, and what to take from that. Also, in line 317, it says that accuracy saturates at k=4 without evidence to support that (what happens at k=5?).\n\nIn section 3, the manuscript states that the Raptor program will be tested on \"Foundation models\". What qualifies as a foundation model is highly debated between researchers, but regardless of the fact, simply testing on a ViT-base does not provide a convincing argument for the claim. For example, what happens when you scale to, say, a ViT-Large? Can Raptor be successfully applied to other objective functions outside of DINOv2, say MoCo-v3?\n\nMore generally, the authors show block dynamics to emerge across diverse ViTs. However, the ViTs that are tested seem to be trained on either a supervised loss or a contrastive loss. Do block dynamics emerge in a ViT that is trained on a local reconstruction loss, such as an MAE? How generalizable is the BRH?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)