Temporal superposition and feature geometry of RNNs under memory demands
Authors: Pratyaksh Sharma
Alexandra Maria Proca
Lucas Prieto
Pedro A. M. Mediano
Venue: ICLR 2026 Oral (2026)
Keywords: RNNs
superposition
representational geometry
features
capacity
memory demands
Primary area: interpretability and explainable AI
TL;DR: We study the feature geometry of RNNs under memory demands and characterize their representational strategies using a novel framework of temporal superposition.

Abstract: Understanding how populations of neurons represent information is a central challenge across machine learning and neuroscience. Recent work in both fields has begun to characterize the representational geometry and functionality underlying complex distributed activity. For example, artificial neural networks trained on data with more features than neurons compress data by representing features non-orthogonally in so-called *superposition*. However, the effect of time (or memory), an additional capacity-constraining pressure, on underlying representational geometry in recurrent models is not well understood. Here, we study how memory demands affect representational geometry in recurrent neural networks (RNNs), introducing the concept of temporal superposition. We develop a theoretical framework in RNNs with linear recurrence trained on a delayed serial recall task to better understand how properties of the data, task demands, and network dimensionality lead to different representational strategies, and show that these insights generalize to nonlinear RNNs. Through this, we identify an effectively linear, dense regime and a sparse regime where RNNs utilize an interference-free space, characterized by a phase transition in the angular distribution of features and decrease in spectral radius. Finally, we analyze the interaction of spatial and temporal superposition to observe how RNNs mediate different representational tradeoffs. Overall, our work offers a mechanistic, geometric explanation of representational strategies RNNs learn, how they depend on capacity and task demands, and why.

Published as a conference paper at ICLR 2026
TEMPORAL SUPERPOSITION AND FEATURE GEOMETRY OF RNNS UNDER MEMORY DEMANDS
Pratyaksh Sharma" Imperial College London
Alexandra M. Proca"
Imperial College London
Lucas Prieto Imperial College London
Pedro A.M. Mediano Imperial College London University College London
ABSTRACT
Understanding how populations of neurons represent information is central chal- lenge across machine learning and neuroscience. Recent work in both fields has begun to characterize the representational geometry and functionality underly- ing complex distributed activity. For example, artificial neural networks trained on data with more features than neurons compress data by representing features non-orthogonally in so-called superposition. However, the effect of time (or memory), an additional capacity-constraining pressure, on underlying represen- tational geometry in recurrent models is not well understood. Here, we study how memory demands affect representational geometry in recurrent neural networks (RNNs), introducing the concept of temporal superposition. We develop a theo- retical framework in RNNs with linear recurrence trained on a delayed serial recall task to better understand how properties of the data, task demands, and network dimensionality lead to different representational strategies, and show that these insights generalize to nonlinear RNNs. Through this, we identify an effectively linear, dense regime and a sparse regime where RNNs utilize an interference-fir space, characterized by phase transition in the angular distribution of features and decrease in spectral radius. Finally, we analyze the interaction of spatial and temporal superposition to observe how RNNs mediate different representational tradeoffs. Overall, our work offers a mechanistic, geometric explanation of repre- sentational strategies RNNs learn, how they depend on capacity and task demands, and why.
INTRODUCTION
A major goal in both machine learning and neuroscience is to understand how populations of neu- rons represent information, and why certain representational geometries are preferred in different settings. Characterizing different strategies can lead to increased interpretability of artificial neural networks (ANNs), as well as help us better understand the functionality of certain brain regions.
A prevalent theme in neuroscience has been the pursuit of highii-specialized functional components in the brain. Much classical work aimed to demonstrate the functions of individual neurons (Yuste) 2015), such simple and complex cells in the primary visual cortex (Hubel & Wiesel 1962) or place cells in the hippocampus (e'Keefe& & Dostrovsky 197T). More recently however, large scale population recordings have revealed that neural computation is often distributed across many
rons YYuste 2015) Saxena & Cunningham] 2019 Ebitz & Hayden] 202T). It's thought that such coding strategies are important for flexible, complex behavior (Fusi et al.) 2016 Tye et al., 2024).
A parallel line of work has investigated representational geometry in ANNs. While earlier work tried develop models where neurons have specialized functions (disentangled representations or monosemantic neurons) (Bengio et al. 2214 Olah et al.) 2020 Cammarata et al. 2020)). recent work has focused on superposition, phenomenon that characterizes a class of distributed representations.
In particular, ANNs are often challenging to interpret due to polysemanticity, where neurons respond to many unrelated mixtures of inputs. One explanation for polysemanticity the superposition
"Co-first author; to httpa///t....tt...o....o.aaaaaaa impr imperi aarriallaa ac.uk.
Cooee available wthttpa://ggiiii http:////.........///////eoaaaaa supperrppo

---

Published as a conference paper at ICLR 2026
hypothesis (Elhage et al. )022), which posits that ANNs utilize compression strategy where they represent more features (c.g., directions in activation space) than there are neurons by allowing some interference (non-zero dot products) between them. Superposition is particularly effective when features are sparse, although it leads to features no longer mapping onto individual neurons. In the era of big data, even in models with millions of parameters, superposition appears ubiquitous [Bricken et al. [023] Templeton et al. al.] (024)- an effective strategy arising from the inherent sparsity of features in real-world data. A major focus of mechanistic interpretability is now finding new techniques to extract such features from superposition. Fewer studies, however, have focused on understanding the feature geometry induced by superposition (Sharkey al. (025).
Previous work in interpretability has not considered the effect of time (or memory) as a capacity- constraining pressure. While less of an issue in feedforward and transformer architectures, this pressure does arise with recurrence. Recurrent neural networks (RNNs) are important for learning tasks with temporal dependencies and models of dynamical systems. They are commonly used as cognitive models in neuroscience (Barak] 2017). as they often replicate the neural activity of animals when trained on similar tasks (Vyas et al. 2020). Additionally, RNNs are increasingly popular for long-range sequence modeling due to their computational and memory efficiency (Gu et al. 2022b) Orvieto et al. 2023) Gu Dao 2024). Understanding the effect of capacity through time representational geometry important, it affects maintenance of long-range dependencies and could have implications for cognition with limited resources, such as working memory.
In this work, study how time, or task (memory) demands, affects representational geometry and capacity in RNNs through the lens of feature superposition. We develop a theoretical framework to better understand representational strategies employed by RNNs and how data and network di- mensionality interacts with memory demands, Our results characterize and explain behavior across different recurrent architectures and task settings, providing insight into what RNNs learn and why. Our contributions are as follows:
We introduce the concept of temporal superposition in RNNs.
We distinguish two forms of interference - projection and composition interference and show how they impact behavior.
We derive an expression for the loss on a simple recall task that decomposes into four interpretable terms, allowing explain the geometric strategy employed by the RNN.
We study the learning dynamics of RNNs in terms of changing representational geometry.
In RNNs with ReLU nonlinearities, we identify the existence of an interference-free space into which many feature directions can tightly packed.
We identify phase transition in the geometry between the dense and sparse regimes.
We study the interaction of spatial and temporal superposition, and how RNNs mediate this tradeoff depending on task requirements.
2 KEY INTUITIONS: SPATIAL & TEMPORAL SUPERPOSITION
Here we introduce the key ideas we will develop formally in the remainder of the paper. First, we review spatial superposition, which was studied extensively by Elhage et al. (2022) in feedforward networks, and then extend these concepts to characterize temporal superposition in RNNs.
Spatial superposition. We follow the linear representation hypothesis and assume that features (loosely defined as interpretable properties of the input; see Elhage et al. (2022): Park et al. (2024) are represented as directions in activation space. When there are more features of the data than neurons and features are sparse (i.e., do not commonly co-occur), it becomes optimal for neurons to represent features non-ortin ooooooohogyyllly (in shared dimensions of the activation space) as to compress more in (Figure ab. This comes the cost of possible interference between co-occurring features, but if features are sufficiently sparse, this can be outweighed by the benefit of representing more features. Spatial superposition only becomes viable the presence of nonlincarity that reduces interference between features; linear networks instead learn PCA of the most important features.
Temporal superposition. In addition to spatial superposition, we claim that RNNs exhibit another form of superposition due the axis of time and that this phenomenon is fundamentally different
2

---

Published as a conference paper at ICLR 2026
FEATURES
D
A
A B C
New
New
Old Age
A
(c) Projection interference
More- - Less
Importance
Old
B
Not shown
(d) Composition interference
(a) Spatial superposition
(b) Temporal superposition
Figure 1: Spatial and temporal superposition both occur in RNNs and are characterized by different representational strategies. (a) When there are more input features (5: A-E) than hidden dimensions (2) and features are sparse, it becomes favorable to compress these features into the acti- vation space non-orthogonally in spatial superposition. (b) When an input feature (A) must held in memory for more time (red: O timesteps old; green: 5 timesteps old) than hidden dimensions (2) and features are temporally sparse, it becomes favorable to utilize temporal superposition to exploit space opposite the output feature. (c) Projection interference (red dashed line) occurs when the activation of a feature (black) is read-out (blue arrow) as the activation of a dif- ferent feature (gray). (d) Composition interference (red dashed line) occurs when the activation of multiple features (black) is linearly combined into an activation that imitates another feature (gray).
the interference-Ffe
from the feedforward case. In particular, in addition having some spatial component, we can think of each input feature as also having a temporal component dependent on its sequential position. This means that features are represented differently depending on which timestep they occur at, even the input itself is the same. In other words, representations of features are now determined by "when" just as much as "what". For example, if an RNN receives an impulse of some feature A as input at timestep the representation of this input will move through set of distinct feature directions as time goes on and the impulse grows older, until it ceases to be task-relevant (as in Figure [b]. If features are held over longer periods of time (because they remain task-relevant), more feature directions are compressed into the hidden state. The hidden state therefore acts as bottleneck, such that RNNs are forced to either forget features represent more features than dimensions (in superposition) as the length of the task-relevant input window increases. In addition compressing more features due memory demands, features must also be read out at particular timesteps, rather than being immediately available as in feedforward models, giving rise to different behavior from that of spatial superposition. We distinguish temporal superposition as resulting from representing features across a longer period of time (higher memory demand) in a lower- dimensional activation space, whereas we refer to spatial superposition as representing more input features (higher-dimenssonaal data feature space) in lower-dimensional activation space.
3
MATHEMATICAL SETUP
3.1 RNN MODEL.
We study a RNN parameterized by matrices W₂ E RNN,,, Wn E RM.NN, W, N.N,, with a hidden state he E R. that receives an input E No at each timestep t and produces an output g. R.N, The RNN given by
h: W, + WAOT (h.-1) 9. - ow, h.)
where a represents either a linear activation ReLU, depending the setting consider. We refer to the model with linear recurrence and readout as linear RNN, the model with linear recurrence and nonlinear readout as a state space model (SSM), and the model with nonlinear recurrence and readout as a nonlinear RNN. We first focus on the setting where both o,, J, are linear and later consider cases of nonlinearity. We initialize the hidden state ho at In the case where linear, this yields
h
Wh. Wt-WW... W.
(2)
3

---

Published as a conference paper at ICLR 2026
3.2 FEATURE DIRECTIONS
We define 8 := ( - to indicate how many 'recurrences" an input from timestep i has undergone at timestep / (or the 'time window" it has been in the RNN for). This allows to account for a feature in terms of how long it has been in the RNN relative to particular timestep. Letting W, == W,Waa. we can rewrite the expression for the hidden state above as
h WWW = Σ W,
(3)
To isolate the effect of temporal superposition and make visualization easier, we first consider sealar inputs and outputs (N₂ = N2 1) and two-dimensional hidden size (N, 2). After building thorough understanding of temporal superposition, in S. introduce spatial superposition by increasing the dimensionality of the data (N₂- = N, 5) and study its interaction with temporal superposition. We note that our framework and main results naturally extend to multidimensional inputs and outputs, and higher dimensional hidden spaces (Appendix G.. For clarity, we refer to W", W. E 2x as Wy, w, to indicate that they are vectors, and to x. y. ÿ E IR as 2, y, to indicate they are scalars (the equations above remain otherwise the same).
In this form, we can see how each input feature It-S is independently and linearly represented in the hidden state in the direction given by w, (the feature direction). This makes it clear that, at time t, the model has access to the entire history of t input features, but that for t > N,, the hidden state inevitably becomes bottlenecked. Further, this form illustrates the role of the readout wp in the interference between different features. Although there are up to feature directions (1==0:t-1) contained within the hidden state, only the feature directions that project onto wy at timestep f will affect the RNN output ýt. For visual clarity in our main figures, we tie the readout w. Wa=k (see definition of below; discussion and figures replicated without weight tying Appendix 1.2 full experimental details in Appendix Ip.
3.3 K-DELAY TASK
In order to directly control for the time span for which features remain task-relevant, we consider the k-delay task (Jaeger 2002). in which the model is trained to reproduce the input sequence after fixed delay of l timesteps. In particular, the RNN tasked with producing the target output ye = Zt-k at each timestep (y/- = 0 for S k). Therefore k acts as a control parameter that specifies how long an input feature must be maintained in the hidden state for successful task performance. We note that this task is essentially an extension of the setup in Elhage et al.] (2022) to the temporal domain and that the two are identical for O. We use a squared-error loss given by
T
T
C =
12
=
100
3ell
+
Σ 1=
- Dell
(4)
Task-relevant and irrelevant features. To provide some intuition about the k-delay task, we can refer to features in terms of their utility. At each timestep 2 k + 1, there is an output feature direction (Wa=k) which functions to produce d1 Zt-k by projecting onto the readout wy There are also intermediate feature directions (w. for v 8 < k), which represent input features (Iz- (r)) from the relevant k-length memory window. These intermediate features are held in memory to be read out at future timesteps. Both output features and intermediate features (k + 1 features in total) are task-relevant features at timestep t. These features contribute task performance either immediately or in the future, so the model is incentivized to represent them as faithfully as possible.
There also potentially exist historical, task-irrelevant features. These correspond to features (re-- represented by w.) from the more distant (s > k) past. Being beyond the i (shift) window, these features cannot contribute current or future task performance. The model therefore incentivized to forget these features (which may otherwise interfere with other task-relevant features).
3.4 PROJECTION AND COMPOSITION INTERFERENCE
Now that we've defined feature directions, interfere with each other. Here we introduce and define two forms of interference that occur in RNNs: projection interference and composition interference.
understand how they interact and potentially
4

---

Published as a conference paper at ICLR 2026
Projection interference. Projection interference occurs when the activation of a feature is read- out as an activation of a different feature, as shown in Figure Ic This occurs when a feature is represented (w,) non-orthogonally to the readout (wy), causing unintended non-zero projection onto that read-out. For example, say feature direction W and feature direction W,=B (A * B) both non-orthogonal the read-out Wy. Then, input (It-A) from timestep - can be mistakenly read-out as having been an input (zt-B) from timestep - B, and vice versa.
Composition interference. Composition interference occurs when the activation of multiple fea- tures is linearly combined into activation that imitates another feature (Figure In the example above, if the RNN receives inputs 2t-A and Zt B, their feature directions (Wx=A, Wx=B) are co- activated and linearly combine to form # = A consequence is that if there is another feature direction W,=C similar to this combination, the activation for inputs 2 and Tt- will be indistinguishable from that of single input r from timestep f
C, which never occurred.
4 RESULTS
4.1
LOSS DECOMPOSITION REVEALS GEOMETRIC STRATEGY
In order to better understand how the learning problem incentivizes certain geometric strategies, begin by studying the loss of linear RNNs. By assuming temporal independence and sparsity of the data (Appendix Ab. we derive (Appendix B form of the loss comprised of four terms.
B[4] =
pv
w w 111
2p
1
W
1=
task benefit
mean correction
(5)
Pv
un w.)
+ +
(m w.) (wy, w.yr
spk
sigt
projection interference
composition interference
P controls temporal sparsity (how frequently features occur in time; smaller p corresponds to higher sparsity), and and are the mean and variance of the input distribution, respectively. By studying the terms above, we can understand the competing incentives in the loss.
First, the task benefit term is the value of successfully performing the task by aligning the feature direction w, of the input fz-k to the readout wy. This corresponds producing wt-k time f, as required by the task. The mean correction term acts to offset any non-zero mean of the input distribution by exploiting projection interference - the RNN uses projection interference (w, w.yk) as bias in the absence of one and, in fact, the term disappears if we include bias term in the output or trivially if the mean ja is 0.
the effects of interference in the loss. The projection interference cost introduces a penalty on feature directions W, that project onto the readout w, at the incorrect time (s * k). Additionally, the composition interference term comes into play when there multiple features simultaneously active. Geometrically, this term penalizes positive correlations between w, vectors while rewarding negative correlations, with respect to their projection onto Wy, such that negative (destructive) interference is preferred over positive (constructive) interference. This essentially en- courages w, vectors spread out in activation space much possible, ideally forming antipodal pairs (similar to Elhage et al. (2022) and analogous to the form in Saxe et al. (2044)).
Next,
We train linear RNNs on the k-delay task and see that our expected value of the loss closely pre- dicts the empirical loss (Figure top; Appendix C for more discussion and visualization of sin- gularfeigenvalues and geometry throughout training). We also observe several different stages of learning, corresponding to unique changes in each of the loss terms that map onto specific geomet- ric configurations (Figure 2 bottom). In particular, the RNN initially aligns all feature directions (w.) to the readout (w,,) (qaantified by output projection w w.). corresponding to a decrease in the task error term and increase in magnitude in the other three terms. After this initial alignment, feature directions begin to spread out in activation space based their temporal ordering, causing the task error, mean correction, and composition interference terms to decrease in magnitude, while projection interference increases. The "staircase" loss corresponds the learning of singular values
5

---

Published as a conference paper at ICLR 2026
left)
0.6
Empinial loss
Espected loss: (i) (ii)
0.4
( Task entir
Loss
Projection inserference
(ii) Mean correction
(it) Composition interforence
Colours for (sceand
0.0
10
Output projection M. W.
0.3
0.2
0.1
0.0
-0.1
10000
20000 30000
40000 50000
Training iteration
Figure 2: Learning dynamics characterized by initial readout alignment, then separation of feature directions. For a 2D linear RNN on the 3-delay task. (Top) The expected loss E[C) matches the empirical loss curve. The four loss terms exhibit different dynamics during training correspond- ing to the geometric configuration of feature directions. (Bottom left) Quantifying the projection onto the readout for each feature direction via the output projection wr w. (for 0 S < 12), there is an initial readout alignment followed by temporally-ordert separation. (Bottom right) The final spiral, matched to the output projections. The output
arrangement of w, vectors converges feature W,=3 (star) has the highest projection onto wy.
(Saxe et al. 2014: Proca et al., 2025) and indicative of saddle-to-saddie dynamics (Jacot al. 2022) and geometric restructuring (Haputhanthri et al. 2024). 'he final arrangement of feature directions the RNN converges to shown in Figure 3p. Here, in the linear case, the RNN learns to downscale and rotate features with each recurrence (such that old features spiral into the ori- gin), implementing a 'smooth' forgetting. Indeed, in the 2D case, a spiral sink (c.g., spectral radius lw,=oll < 1) is the optimal solution for a linear RNN in the k-delay task (Appendices D.2 and as old features gradually fade from the hidden state.
and D.3
4.2 FEATURES GROUP INTO INTERFERENCE FREE SPACE
Building on the previous section, here we consider the impact of adding nonlinearity to the readout
wy (o, )+= = Rel.U(-). The nonlinear setting makes deriving analytic solutions the loss more challenging. We therefore approximate the expectation of the loss in the limit of high temporal sparsity (Appendix E.I. yiclding
[[C R PM
(eLL
w.=k) 1)2 + ΣΣ ReLU(W, w.)² + O(p²)
(6)
t= st
task benefit
projection interference cost
This expression resembles the one in the linear case, but the inclusion of the ReLU activation has significant impact on its geometric interpretation. Due to the ReLU, the model only produces output for vectors that have positive projection onto Wg. Thus, all w, vectors in the half-space opposite of w, do not contribute to projection interference. In fact, the extremely sparse regime (where composition interference becomes negligible), this half-space essentially becomes interference free. This reveals remarkable incentive for the model to take advantage of this phenomenon by packing many Waysk vectors into this half-space possible (Figure 34).
To test this prediction, we train linear RNNs with ReLU readouts (SSMs) on the k-delay task at various levels of sparsity. Because the recurrence is still linear, isolation of feature directions waye into this space is not always perfectly possible as w, can only be spaced equally (along an elliptical spiral). Despite this, by looking Figure 3,, we see that when sparsity is high, models learn to minimize projection interference by grouping the largest feature directions into the interference- space. The SSM employs approximation of the strategy in which the largest feature directions

---

Published as a conference paper at ICLR 2026
LEGEND
Output feature
Interference-
free space
(a) Theoretical
(b) Linear
(c) SSM
(d) Nonlinear
New
Old
Figure 3: Theoretical predictions match representational strategies of expressive models. (a) A ReLU readout creates an interference-free space. In the panel, we display an idealized form, where all intermediate feature directions group into the free space (gray shading) and only the output feature direction lies outside of it (star). (b) Linear RNNs lack an interference- free space and instead arrange old features spiral into the origin (c) In the sparse regime, SSMs minimize projection interference by grouping the largest feature directions into the interference-free space. (d) Nonlinear RNNs are expressive enough fully exploit the interference-free space by grouping all of the intermediate features, separate from the output feature, and implement sharp forgetting.
w. occupy the interference-free space while smaller feature directions lie outside of it. Within the constraints of linear recurrence, this strategy still minimizes projection interference by exploiting the interference-free free space. We also observe that the spectral radius (i.e., W.=ool) increases with k, regardless of sparsity, to overcome projection interference (Appendix E.2 for additional simula- tions).
4.3 PIASE TRANSITION FROM DENSE TO SPARSE CONFIGURATION
By varying sparsity, we can observe the existence of two discrete regimes (Figure 4. When the SSM is trained on data that is dense (low sparsity), it learns 'dense-regime' solution where it arranges features into a spiral sink, similar to the fully-linear RNN. Although has a nonlinearity on the readout, the SSM does not group the largest feature directions into the interference-free space in this case. We speculate that this is due to the increased likelihood of composition interference occurring when features are dense. In particular, if the SSM groups the largest feature directions into the (neg- ative) interference- free space (e.g., Wy=A) and they additively sum with smaller (positive) feature directions BB), the ReLU will cause the model to output 0 (ReLU((w, w== + wy u², B) = 0).
1.00
Instead, when inputs are sparse, it is optimal for the SSM to take advantage of the interference-fres space, characterizing a 'sparse-regime' solution. The most noticeable difference in ge- ometry between dense and sparse regimes is the angle that the task-relevant feature directions span (k0 is the angle traversed from w,=0 to Wg=k), which exhibits a sharp change as sparsity is varied. In the dense regime, task-relevant feature directions group into a smaller cone (+ 90%). Instead, in the sparse regime, task-relevant feature directions spread out into approximately 270" of the plane, traversing the entirety of the interference-fre space to reach the readout direction. By varying sparsity, we can interpolate between these two regimes and observe a phase transition in k0, accompanied by a decrease in spectral radius (p [ ===o))). The difference in spectral radius is likely be- cause the dense regime, the SSM uses larger feature directions to compensate for projection interference (by having a large pro- jection onto the readout that outweighs other projections). In the sparse regime, projection interference is less prevalent and therefore the SSM does not need a spectral radius as large.
Optimal p
0.75
0.50
Optimal kgo
1170
810
450
90
0.6
0.8
1.0
Sparsity (1 1--9) p)
Figure 4: There exists a phase transition in optimal feature geometry between dense and sparse regimes. As temporal sparsity increases, (bottom) the angular distribution of feature di- rections (k0) exhibits phase transition, (lop) accompanied by decrease in spectral radius (p).
4.4 NONLINEAR RNNS EXPLOIT INTERFERENCE- SPACE
We've used RNNs with linear recurrence to build our understanding of behavior in analytically tractable settings. However, these models have limited expressivity because their dynamics are con- strained to particular form, such spiral. Here, consider models with nonlinear recurrence and find that, in the sparse regime, they consistently implement our predicted ideal strategy of pack- ing as many w. feature directions into the interference-free space as possible.
7

---

Published as a conference paper at ICLR 2026
We consider the nonlinear RNN model, now with both 0 (+)) ReLU(-). Although it's not possible to find a simplified expression for the feature direction wa analytically, it still acts as the direction the input It-s is represented in the limit of high sparsity (p + 0; Appendix F.2). Fur- thermore, the interference-free space still exists and can be better exploited as the RNN can form more optimal arrangements of w, vectors due to increased expressivity of the recurrence. In Fig- ure we verify our predictions: nonlinear RNNs learn to pack the k intermediate features into the interference-free space, with only W,=k lying outside (Appendix F.3 for more simulations). Often, the vectors compress into single quadrant. This is because the nonlinearity creates a privileged ba- sis in the hidden state positive activations are unaffected by the ReLU, while negative activations will be set to zero in the next timestep. Hence, task-relevant feature directions arrange themselves in the positive quadrant of ht,
While a RNN with linear recurrence can only implement smooth forgetting (with spectral radius ||w < 1) by shrinking an input's contribution to the hidden state over time, the ReI.U activation makes it possible to immediately forget a feature by sending it to the negative quadrant of he. As a result, the nonlinear RNN can implement sharp forgetting, which enables the model to represent only task-relevan features (and remove the possibility of interference from the distant past).
4.5 INTERACTION OF SPATIAL AND TEMPORAL SUPERPOSITION
We've ve focused on the case of scalar inputs and outputs in 2D space isolate the effects of temporal superposition and for ease of visualization. We now study the interaction of spatial and temporal superposition by considering vector inputs (to introduce spatial superposition) and changing & (tem- poral superposition). Recall that for k- = O, the RNN is tasked with imitating a feedforward network where immediately outputs the input; hence, there no need to represent features from earlier timesteps and we recover the setting of pure spatial superposition from Elhage et al. (2022).
In Figure 5 as k increases, we see how a nonlinear RNN attempts to balance representing the most important features (scaling of loss) across time (A being most important, E being least). Initially, when k = 1, it represents features A, B, and C for 2 timesteps and drops D and E altogether. For higher k the RNN eventually drops all features except A. Hence, we see tradeoff between the RNN's incentive to represent multiple input features and the duration of time each must be represented for. The RNN's strategy is 'all-or-none:: to gain any advantage from representing specific feature, the RNN must be able to maintain it in memory for all k: + 1 timesteps otherwise it will not meaningfully contribute to decreasing the loss. If the RNN does not have sufficient capacity to represent the feature for all k + 1 timesteps, it won't I represent the feature at all. This is why we see k+I 1 feature directions for feature A for all k while other features only occur for lower I when the RNN has sufficient capacity.
Higher-dimensional hidden states. Up to now, have restricted the hidden state (N₂) of our models to 2 dimensions for easier visualization and interpretability. To extend our setting to higher- dimensions, we train nonlinear RNNs on 10-dimensional input (N₂ 10) the 2-delay task, varying hidden size (N, 2, 5, 10), and measure the projection of each feature direction onto the readout (W, y W.). Based on our previous results, we would expect W. of positive outputs (corresponding to the output feature directions positively projecting onto the readout: for the correct output at the correct time). Moreover, we would expect the rest of the entries in the matrix (as well as all of W W to be negative or 0, lying in the interference frree space. Across all hidden sizes, we see this exact strategy (Figure [2] and Appendix G). with RNNs with larger hidden sizes simply capturing more features along the diagonal of W W.=2 (i.e., the same all-or-none effect described above). Finally, we quantify this behavior by computing the mean of the non-output feature direction projections onto the readout (i.e., mean (W W )) which should be negative in an optimal model, and the mean of the output feature direction projections onto the readout (i.e., mean diag W W.+))),, which should be positive. We train RNNs with hidden size 100 on a 2-delay task with 75 features and find that the hest performing models group the largest feature directions into an interference-free space and project the output feature onto the readout at the appropriate time, as predicted (Figure 13
y We=2 have diagonal
5 RELATED WORK
Representational geometry and interpretability. The study of representational geometry can provide better understanding of how distributed activity encodes information for different behavior. ANNs can exhibit different representational geometries, reflective of the tasks trained on
8

---

Published as a conference paper at ICLR 2026
Ol
Projection onto read-outs
New
B
B
k 0
k ==
k=
Figure 5: There exists a tradeoff between spatial and temporal superposition. By setting l = O, we recover purely spatial superposition (as there is memory demand) and all 5 features arranged in a pentagon (feature legend as in Figure [ interference-fret space indicated by blue shading). As k and the corresponding memory demand increases, the RNN prioritizes the most important features in an all-or-none fashion. There is a preference to represent one feature for all k+ relevant timesteps, as opposed to representing several different features for shorter duration.
Johhsstnn & Fusi 2023 2024), dataset size (Henighan et al. 2023). architecture (Jacot et al. 2018
Chizat et al. 2019). and weight parameterization (Flesch al. al.] |022 Braun et al. 2022). In addition
to advancing our understanding of the solutions ANNs learn in different settings, this can also help neuroscientists identify computational structures in the brain and understand their functional roles (Saxe et al. 2020; Ostojic & Fusil 2024). In this work, we introduce the concept of temporal superposition and study its effect on representational geometry. Our work is inspired by [Elhage ct al. (2022), one of the first papers to formalize superposition in toy models and study it explicitly. The field of mechanistic interpretability has since centered around the problem of superposition and developing methods to identify meaningful features from model activations (Bricken et al. 2023 Templeton et al.] 2024). While these techniques have continued to advance, there has been notably less work devoted to understanding the feature geometry induced by superposition (Sharkey al., 2025). Here we expand the study of superposition to recurrent architectures to show how memory acts capacity constraint, inducing superposition, and how this affects underlying geometry.
Recurrent neural networks. RNNs are important for modeling temporal data and studying dy- namic processes. Recently, RNNs (SSMs) with linear recurrence have become popular due to their computational and memory efficiency (Gu et al. 2022b Orvieto et al., 2023 Gu & Dao 2024). of-
ten initialized with complex-valuee parametenizations (Gu et al. [022a Orvieto al. (023). with
increased expressivity (Ran-Milo et al. 2024 Orvieto et al.) 2024). We find that complex eigen- values support temporal superposition by rotating features within an interference-free space. We also study SSM, showing that it exploits the interference-free space in the sparse regime, but is still constrained in expressivity compared to nonlinear RNNs, resulting in different geometries (and smooth vs sharp forgetting). In neuroscience, RNNs are common for modeling (Barak) 2017). as the brain's 's connectivity highly recurrent and RNNs often replicate neural activity recorded in an- imals (Vyas et al.) 2020, Khona& & Fiete, 2021) and behavior (Ji-An et al. 2025) when trained on the same tasks. Similar to the feature geometry we see here, previous work RNNs has observed rotational dynamiis/sequentia activity (Rajan et al., 2016 Orhan & Ma 2019 Cueva et al.) 2020 Zhang et al. 202I) for tasks with fixed delay, thought to encode temporal information. Other work has shown that RNNs trained on tasks with random delays instead exhibit persistent activity, in the form of fixed point attractors (Orhan & Ma
2019 Liu et al., 2021 Xie et al. (022b), similar to space we study in our model, RNNs trained in motor-preparation paradigms similarly develop output-null subspaces where intermediate preparatory activity does not affect behavior (Schimel al. 2024). There's been substantial theoretical work on RNNs, both by neuroscientists studying properties of neural compu- tation and by deep learning theorists (Dubreuil
pure spatial superposition (e.g., when k O). Related to the interference-fre
al. 2022; Driscoll et al. 2024 Schuessler al.) 2024 Zucchet & Orvicto 2024 Proca et al. 2025). One important line of theoretical work has stud- ied low-rank RNNs (Mastrogiuseppe & Ostojic| 2018, Schuessler et al. 2020a Beirán et al. 2020) Dubreuil et al. 2022). These interpretable models have low-dimensional recurrent dynamics, allow- ing their exact phase portraits to be visualized; furthermore, these dynamics can be directly related to the underlying connectivity statistics. Related to our work, low-rank connectivity also acts form of capacity constraint, although the effects of such constraints have not been studied explicitly (but see Beirán et al. (2023) for comparison between low-versus full-rank RNNs). However, there less work studying feature geometry in the context of capacity constraints induced by memory. Most work implicitly assumes overparameterized regime (relative to task demands) when study-

---

Published as a conference paper at ICLR 2026
ing properties of RNNs (Cohen-Karile et al. 2023). One exception is François et al. (2025) which studied the k--elay task in an underparameterized linear RNN in the frequency domain. While they focused on the dense (linear) regime, here we also study the sparse regime in nonlinear RNNs, identifying novel behavior.
Memory capacity. Memory capacity has previously been studied in RNNs (White et al. 2004
Dambre et al., 2012) Ballarin et al. 2024), most classically in echo-state networks (Jaeger 2002
(we refer to short-term memory and not other forms like associative memory). Memory capacity typ- ically refers to a temporally-dense regime. It's known that linear memory capacity is limited to the number of neurons in the hidden state N₁, corresponding to a single neuron per feature (orthogonal- ity/lack of superposition). Here, we consider how RNNs handle capacity constraints, transitioning from temporally-dense to temporally-sparee regimes. We show how training with different task de- mands leads to different geometric solutions, aimed at compression and increased (sparse-regime) capacity. Hence, we provide a mechanistic interpretation of memory capacity under constraints.
Working memory and serial recall. Our work is related to working memory in cognitive neuro- science - a cognitive function involving short-term maintenance and manipulation of information for immediate use. Similar to the k-delly task, working memory is often studied using serial recall, which has previously been modeled with RNNs (Botvinick & Plaut 2006: Ganguli et al.. 2008 Two existing theories of sequence working memory involve: (I) activity slots (Luck & Vogel [997]
2013 Xie et al., 2022a), in which there exist a set of distinct neural subspaces for different sequence items, and (2) a resource model (Alvarez & Cavanagh 2004 Wilken & Ma 2005 |Bas & Husain, 2008), where working memory is a limited-capacity continuous resource that is shared between items (more items leads to less capacity per item). Interestingly, in the setting we consider we ef- fectively find both (Soni & Frank (025). RNNs arrange features from each timestep into separate 'slots' (directions) along which these features shift through time. In the case of superposition, slots are not orthogonal (Xie et al. (022a)), but otherwise would be with sufficient capacity. Additionally, we show that the hidden space is a continuous
catitecc resource: as memory length (k) or input features (N₂) increase, there is more demand and features are more likely to interfere. Further, we show how limited capacity leads RNNs to represent important features, while others are dropped.
6 DISCUSSION
Summary of results. In this work, we study the effect of time on feature representations in RNNs, introducing the concept of temporal superposition. We identify how features can interfere through either projection or composition interference, and their corresponding effects. We derive an ana- lytical form of the loss that decomposes into four interpretable terms, which we use to explain the resulting learned geometry of RNNs. By deriving an approximation of the loss in the limit of high temporal sparsity in nonlinear RNNs, we identify the existence of an interference- free space, which RNNs exploit to minimize projection interference. By varying temporal sparsity, we see how SSMs exhibit a phase transition from an effectively linear strategy to one that uses the free space, and that this phase transition is reflected in the angular distribution of features and spectral radius. We further show that nonlinear RNNs in the sparse regime exploit the interference-fie sharp forgetting. Finally, we study how spatial and temporal superposition interact as a result of different task demands and capacity constraints, and how RNNs mediate this tradeoff.
space and implement
Limitations and future work. We simplify our theoretical setting by assuming temporal indepen- dence of features and studying small RNNs. We also study the sparse regime: while the assumption of sparse input features appears to be reasonable (Elhage et al. 2022). the assumption of temporal sparsity may be strong. This is dependent on the task and it's an open question how the theoretical setting considered here extends more generally. Related, we study the k-delay task, which requires reproduction of sequence with a delay. An important future direction will be characterize ge- ometry and behavior for tasks requiring manipulation of input information and varying memory de- mands. Finally, we note that one major assumption of the superposition hypothesis that features are represented linearly, as directions in activation space (the linear representation hypothesis (Park) cet al. 2024)). A possible objection may be to what extent this work captures realistic settings in seemingly overparameterized modern-day models. Although we study 2D case for simplicity (but see Appendix G memory demands decrease capacity linearly with time (Jelassi et al. 2024) and consequently finite-width RNNs tasked with learning long-term dependencies will be constrained. Moreover, superposition has already been demonstrated in LLMs (Bricken et al. 2023 Templeton et al. 2024) and our study indicates that recurrence and memory will exacerbate it.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
AP funded by the Imperial College London President's PhD Scholarship. LP is supported by the UKRI Centre for Doctoral Training in Safe and Trusted Al [EP/S023335//1)
REPRODUCIBILITY STATEMENT
To ensure reproducibility of our work, we provide code to replicate all of our experiments and and include
figures https: /ttghub..bbb..// com/ http://sssaatt/irrrnnnn coannnsupeppssstt.
experimental details in Appendix [ For our theoretical results, we include our assumptions about
the data Appendix A and full derivations and proofs in Appendices B
D.
E. and F2
We
also provide additional simulations to those in the main text in Appendices D.2
D.3
and H
REFERENCES
Pau Vilimelis Aceituno, Gang Yan, and Yang- Yu Liu. Tailoring echo state networks for optimal learning. iScience, 23(9):001440, 2020. ISSN 2589-0042. doi: http:///bi.org//1.1116//. 10144. URL
com//sciencee/articll/pppll
$2589004220306223
G. A. Alvarez and P. Cavanagh. The capacity of visual short-term memory is set both by visual information load and by number of objects. Psychological Science, (5(2):06-111, 2004.
Alexander Atanasov, Blake Bordelon, and Cengiz Pehlevan. Neural networks as kernel learners: The silent alignment effect. International Conference on Learning Representations, 2022.
Giovanni Ballarin, Lyudmila Grigoryeva, and Juan-Pablo Ortega. Memory of recurrent networks:
Do we compute it right? Journal of Machine Learning Research, 25:1-38, 2024.
Omri Barak. Recurrent neural networks versatile tools of neuroscience research. Current
Opinion in Neurobiology, 46:1-6, 2017. ISSN 0959-4388. doi: http:://bbiorrg/1.01166j.. 2017.06.000 URL https:: :/wp:.////www..//. www. sciencedirect. http///ceeenee///rricee//.pp/// 509594388 329429
Computational Neuroscience.
P. M. Bays and M. Husain. Dynamic shifts of limited working memory resources in human vision.
Science, (5890)0)) -554, 2008. doi: 10. 10.1126/ceeece. 158023.
Manuel Beirán, Alexis M. Dubreuil, Adrian Valente, Francesca Mastrogiuseppe, and Srdjan Ostojic. Shaping dynamics with multiple populations in low-rank recurrent networks. Neural Computa- tion, 33: 1572-111,,,, 2020.
Manuel Beirán, Nicolas Meirhaeghe, Hansem Sohn, Mehrdad Jazayeri, and Srdjan Ostojic. Para- metric control of flexible timing through low-dimensional neural manifolds. Neuron, 111(5): 759-753, 2023. doi: 10. 10.1101/2021.11 1.0.......44
Yoshua Bengio. Patrice Y. Simard, and Paolo Frasconi. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 5 2:157-66, 1994.
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation leaming: A review and new perspectives, 2014. URL httptttss///xxi....... wtt.or///b///......... 5538
Blake Bordelon, Jordan Cotler, Cengiz Pehlevan, and Jacob A. Zavatone- Veth. Dynamically learn- ing to integrate in recurrent neural networks. 2025.
Matthew M. Botvinick and David C. Plaut.
Short-term memory for serial order: a recurrent neural network model. Psychological review, 113(2):001-333 2006.
David M. Bradley and Ramesh C. Gupta. On the distribution of the sum of n non-identically dis- tributed uniform random variables. Annals of the Institute of Statistical Mathematics, 54(3): 689-700, 2002. doi: 10.1033/A11022443717777 URL https: //op:///do... org/10.... 1023////1: 1022483715767
ID: Bradley2002.
11

---

Published as a conference paper at ICLR 2026
Lukas Braun, Clémentine Dominé, James Fitzgerald, and Andrew Saxe. Exact learning dynamics of deep linear networks with prior knowledge. Advances in Neural Information Processing Systems, 35:6615-6629, 2022.
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Con- erly, Nick Tumner Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, 7ac Hatfield-Dodss, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023. https//transformer pttp:///022/mmmmmmmmttiiieeeeeee
Nick Cammarata, Gabriel Goh, Shan Carter, Ludwig Schubert, Michael Petrov, and
Chris Olah. Curve detectors. Distill, 2020. doi: http://diiilllppp//22200/errrrre
Lenaic Chizat, Edouard Oyallon, and Francis Bach. On lazy training in differentiable programming.
In Advances Neural Information Processing Systems, 2019.
Edo Cohen-Karlik, Itamar Menuhin-Gruman, Raja Giryes, Nadav Cohen, and Amir Globerson. Learning low dimensional state spaces with overparameterized recurrent neural nets. Interna- tional Conference on Learning Representations, 2023.
Christopher J. Cueva, Alex Saez, Encarni Marcos, Aldo Genovesio, Mehrdad Jazayeri, Ranulfo Romo, C. Daniel Salzman, Michael N. Shadlen, and Stefano Fusi. Low-dimensional dynamics for working memory and time encoding. Proceedings of the National Academy of Sciences, 117 (37):23021-23032) 2020. doi: 10.1073/pnas.J 19073///...99198411111 doi/abs 1373/ppass wwt.aaass.1199999 1915984117
1915984117. URL https: www. pnas org/
Joni Dambre, David Verstraeten, Benjamin Schrauwen, and Serge Massar. Information processing capacity of dynamical systems. Scientific Reports, 2012.
Laura Driscoll, Krishna Shenoy, and David Sussillo. Flexible multitask computation in recurrent networks utilizes shared dynamical motifs. Nature Neuroscience. 2024.
Alexis M. Dubreuil, Adrian Valente, Manuel Beirán, Francesca Mastrogiuseppe, and Srdjan Ostojic. The role of population structure in computations through neural dynamies. Nature Neuroscience, 25:783 794, 2022.
R. Becket Ebitz and Benjamin Y. Hayden. The population doctrine in cognitive neuroscience. Neu- ron, 109(19):3055-3068, 2021.
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds. Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy models of superpo- sition. Transformer Circuits Thread, 2022.
Timo Flesch, Keno Juechems, Tsvetomira Dumbalska, Andrew Saxe, and Christopher Summerfield. Orthogonal representations for robust context-dependent task performance in brains and neural networks. Neuron, 110: 125.:12222000 2022.
Alexandre François, Antonio Orvieto, and Francis R. Bach. An uncertainty principle for linear recurrent neural networks. CoRR, abs/2502.09287, Febrroo 2025. (RRL/tttp://ac..... 10. 118550/aaxivv http//axxxxv...........
Stefano Fusi, Earl K. Miller, and Mattia Rigotti. Why neurons mix: high dimensionality for higher cognition. Current Opinion in Neurobiology, 37:66-74, 2016.
Oded Galor. Discrete Dynamical Systems. Springer, Berlin, Germany, December 2007.
S. Ganguli, D. Huh, and H. Sompolinsky. Memory traces in dynamical systems. Proc. Natl. Acad. Sci. U.S.A, 105(48): (05(44):18970-18))) 1177000-1975.. 2008. doi: 00733/paa.00804453 105.
12

---

Published as a conference paper at ICLR 2026
Yoav Ger and Omri Barak. Learning dynamics of mns in closed-loop environments, 2025, URL https://axxis: http://////b/b////a////e// 2ttpb///555.15555 hLt.pp///aax..oorrr.////.5......
Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. In First ntt/ ?id
Conference on Language Modeling, 2024. URL
LEYskwIVY2
Albert Gu, Karan Goel, Ankit Gupta, and Christopher Ré. On the parameterization and initial- ization of diagonal state space models. In Alice H. Oh, Alckh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances Neural Information Processing Systems, 2022a. URL httpeeerrwweeweeeeeeeeeeeeeeeeee htt.:/oorrmmiidy...m iQSAep)
Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured 00396
state spaces, 2022b. URL htt.:ttss//axiii.rrrrrg.e
Udith Haputhanthri, Liam Storan, Yiqi Jiang, Adam Shai, Hakki Orhun Akengin, Mark Schnitzer, Fatih Dinc, and Hidenori Tanaka. Why do recurrent neural networks suddenly learn? bifurcation mechanisms in neuro-insppa couro-inspired short-term memory tasks. In ICML 2024 Workshop Mechanistic Interpretability, 2024. URL htt.s:tp/// /ttps/////peeeeeeeeeeeem wtt.riiewwweeeeeeeeeee.e www/teorrnrte forum wo
Tom Henighan, Shan Carter, Tristan Hume, Nelson Elhage, Robert Lasenby, Stanislav Fort, Nicholas Schiefer, and Christopher Olah. Superposition, memorization, and double descent. pub/
Transformer Circuits Thread, 01 2023. URL huttpp::/ttransffooreeeaaaaoaa
2003/toy--6ubleeeeseeee..
Sepp Hochreiter, Yoshua Bengio, Paolo Frasconi, and Jurgen Schmidhuber. Gradient Flow in Re- current Nets: the Difficulty of Learning Long-Term Dependencies. IEEE, 2001.
David H. Hubel and Torsten N. Wiesel. Receptive fields, binocular interaction and functional archi- tecture in the cat's visual cortex. The Journal of Physiology, 160, 1962.
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kerne:: Convergence and gen- eralization in neural networks, In Advances in Neural Information Processing Systems, pp. 8571- 8580, 2018
Arthur Jacot, François Ged, Berfin Şimşek, Clément Hongler, and Franck Gabriel. Saddle-to-saddle dynamics in deep linear networks: Small initialization training, symmetry, and sparsity. 2022.
URL
Herbert Jaeger. Short term memory echo state networks. GMD German National Research Institute for Computer Science, 01 2002.
Samy Jelassi, David Brandfonbrener,, Sham M. Kakade, and Eran Malach. Repeat after me: Trans- formers better than state space models at copying. In Forty-first International Conference on Machine Learning, 2024.
Li Ji-An, Mareus Benna, and Marcelo Mattar. Discovering cognitive strategies with tiny recurrent neural networks. Nature, 144:993-1001, 07 2025. doi: 10.1038//411886-05--9999
W. Jeffrey Johnston and Stefano Fusi. Abstract representations emerge naturally in neural networks trained to perform multiple tasks. Nature Communications, 14(1):1040, 2023. doi: 10.1038/ s4 941467-003-36883-0
W. Jeffrey Johnston and Stefano Fusi. Modular representations emerge in neural networks trained to perform coptext- tasks, 2024. URL
10. 100111112024 09. 30 615925v1
Mikail Khona and lla Rani Fiete. Attractor and integrator networks in the brain. Nature Reviews Neuroxcience, 23:744 766, 2021.
Yichen Henry Liu, Junda Zhu, Christos Constantinidis, and Xin Zhou. Emergence of pre- frontal neuron maturation properties by training recurrent neural networks in cognitive tasks. iScience, 24(10):103178 2021. ISSN 2589-0042. doi: http:///b..or///.0000/..i..
URL https: www. www.sciencedirecs. wttpett.cooccccieeeeeee http:///ww...ceeeeeeeeeeee/aaa/a 8225800422101111
13

---

Published as a conference paper at ICLR 2026
Steven J. Luck and Edward K. Vogel. The capacity of visual working memory for features and conjunctions. Nature, 390(6657):279-285 1997. doi: 10.1038/36846.
Steven Luck and Edward K. Vogel. Visual working memory capacity: from psychophysics and neurobiology to individual differences. Trends in Cognitive Sciences, 17(8):391 400,, 2013. ISSN 1364-6613. doi: http://db..og/1000106.....s211.2
httpt //ww.. sciencedi rect com/ wwtpeeee......eeeeeeeeeeeeeeeeee http:///ccaccccaa.aaaaaaaaaaaaa/
Francesca Mastrogiuseppe and Srdjan Ostojic. Linking connectivity, dynamies, and computations
in low-rank recurrent neural networks. Newron, 99(3):60 623.e29, 2018. ISSN 0896-6277 6273. doi:
https://doi.org//0... 1ttp://o.oor11101001110110100000 www..uuww............. 000.00..0..00..........000......
URL/ttps: /www. sciencedirect com/
Kenneth D. Miller and Francesco Fumarola. Mathematical equivalence of two common forms of firing rate models of neural networks. Neural Computation, 24(1):25-31, 2012. doi: 10.1162/
NECO.a_00221.
Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An introduction circuits. Distill, 2020. doi: 10.23915/ http://dbsllllbbbbooovvvvuuirees
A. Emin Orhan and Wei Ji Ma. A diverse range of factors affect the nature of neural rep- resentations underlying short-term memory, Nature Neuroscience, 22:275-283, 2019. doi: 10.1033//41593-018--00 018-034--
Antonio Orvieto, Samuel L. Smith, Albert Gu, Anushan Fernando, Caglar Gulcehre, Razvan Pas- canu, and Soham De. Resurrecting recurrent neural networks for long sequences. International Conference on Machine Learning, 2023.
Antonio Orvieto, Soham De, Caglar Gulcehre, Razvan Pascanu, and Samuel L. Smith. Universality of linear recurrences followed by non-linear projections: Finite-widd guarantees and benefits of complex eigenvalues. In ICML, 2024. URL
7ahB 70xb
Srdjan Ostojic and Stefano Fusi. Computational role of structure in neural activity and connec- tivity. Trends in Cognitive Sciences, 28(7)x677-690, 2024. ISSN 1364-6613. doi: https://doi.
URL httpp::/ww...siieeddeee..c wttp://ttteet.ccccc.eceeooe.eeee
atticle/l11/116666133300000
John Keefe and Jonathan O. Dostrovsky. The hippocampus as a spatial map. preliminary evidence from unit activity in the freely-moving rat. Brain research, 34 1:171-5, 1971.
Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models. In Forty-first International Conference Machine Learning, 2024. URL
net httr///orrmmiieee http://rwweeeeetttooooeeieedeees LorrmmiiddUUGGkLzzpp2
Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. the difficulty of training recurrent neural networks. In International Conference on Machine Learning, 2012.
Alexandra Maria Proca, Clémentine Carla Juliette Dominé, Murray Shanahan, and Pedro A. M.
Mediano. Learning dynamics in linear recurrent neural networks. Forty-second International Conference on Machine Learning, 2025. URL
KGOcrIWYnx
Kanaka Rajan, Christopher D. Harvey, and David W. Tank. Recurrent network models sequence generation and memory. Neuron, 90:128 142, 2016.
Yuval Ran-Milo, Eden Lumbroso, Edo Cohen-Karlik Raja Giryes, Amir Globerson, and Nadav Cohen. Provable benefits of complex parameterizations for structured state space models. In The Tiiht- Annual Conference Neural Information Processing Systems, 2024. URL
https: http::::///pennreeeeww.eee 2tt.
Andrew M. Saxe, James L. McClelland, and Surya Ganguli. Exact solutions to the nonlinear dy-
namics of learning in deep linear neural networks, 2014.
14

---

Published as a conference paper at ICLR 2026
Andrew M. Saxe, Stephanie Nelli, and Christopher Summerfield. If deep learning is the answer, what is the question? Nature Reviews Neuroscience, 22:55 67, 2020.
Shreya Saxena and John Cunningham. Towards the neural population doctrine. Current Opinion in Neurobiology, 55:171-178, 2019. doi: (10016f.combb2211.02.0000
Marine Schimel, Ta-Chu Kao, and Guillaume Hennequin. When and why does motor preparation arise in recurrent neural network models of motor control? 2024. doi: 10.7554/elife.891 891111
URL htt.htttp:/ddt htttt:://xx..oi.orgggg. wtwpi.org/1.... 0eeeg//0..555/// fe. 89131.3
Friedrich Schuessler, Alexis Dubreuil, Francesca Mastrogiuseppe, Srdjan Ostojic, and Omri Barak. Dynamics of random recurrent networks with correlated low-rank structure. Phys. Rev. Res., 2: 013111, Feb 2020a. doi: 10. 1 10sSphyysseeesseacch..1111111111
URL http:::/tts/tttt aps.
org/doi/10 1wt.:/o..11111h111eeee1 http://////tyyeeeeeerrrr........
Friedrich Schuessler, Francesca Mastrogiuseppe, Alexis Dubreuil, Srdjan Ostojic, and Omri Barak. The interplay between randomness and structure during learning in rnns. In Advances in Neural Information Processing Systems, 2020b.
Friedrich Schuessler, Francesca Mastrogiuseppe. Srdjan Ostojic, and Omri Barak. Aligned and oblique dynamics in recurrent neural networks. eLife, 2024.
Lee Sharkey, Bilal Chughtai, Joshua Batson, Jack Lindsey, Jeff Wu, Lucius Bushnaq, Nicholas Goldowsky-Dill, Stefan Heimersheim, Alejandro Ortega, Joseph Bloom, Stella Biderman, Adria Gariia--lonso, Arthur Conmy, Neel Nanda, Jessica Rumbelow, Martin Wattenberg, Nandi Schoots, Joseph Miller, Eric J. Michaud, Stephen Casper, Max Tegmark, William Saunders, David Bau, Eric Todd, Atticus Geiger, Mor Geva, Jesse Hoogland, Daniel Murfet, and Tom Mc- Grath. Open problems in mechanistic interpretability, 2025. URL pettps://arxinn org. 2bs/2501.166 16496
Aneri V Soni and Michael J Frank. Adaptive chunking improves effective working memory capacity in prefrontal cortex and basal ganglia circuit. February 2025. doi: 10.7554/elife.97888 97.94... URL http://dx.
doi. org/10. 7554 1554/eLife /ew.eel........ fe. 97894.2
Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet. Trans- former Circuits Thread, 2024. URL /ttpp:: pestts::/traaan http:////aassooooeeeeeee wwwtanieooooeeerrittttt.ooae..ee www.ciii.tttioooee. pub//024) scal scaling monosemant iitnnniiittyy wwwpet/ittittittiiiii..imoemmeee http:iiiexxxiixee html
Kay M Tye, Earl K Miller, Felix H Taschbach, and et al. Mixed selectivity: Cellular computations for complexity. Neuron, 12((5))24599 2470, 2024. doi: 10.1016/J 202.0on.022.04.007.
Loek van Rossem and Andrew M Saxe. Algorithm development in neural networks: Insights from the streaming parity task. In Forty-second International Conference on Machine Learning, 2025.
URL httlhtttss///ppeeeiiieee
net. /ttpt/oo///eeeeeeeeeee id= 3go01hfxd0
Saurabh Vyas, Matthew D. Golub, David Sussillo, and Krishna V. Shenoy. Computation through neural population dynamics. Annual Review of Neuroscience, 43:249-275, 2020.
Olivia L. White, Daniel D. Lee, and Haim Sompolinsky. Short-term memory in orthogonal neural
networks. Physical Review Letters, 92(14), 2004. ISSN 1079-7114. doi: 10. 00.1103//yyseeleett.2222 148102.
/x dx www.oooi.ooooooo org/10 http:///....111//hooeeeo 11t.:
92.148102
Patrick Wilken and Wei Ma. A detection theory account of change detection. Journal of vision, 4: 1120-35, 01 2005. doi: 10. 16./
Yang Xie, Peiyao Hu, Junru Li, Jingwen Chen, Weibin Song, Xiao-Jing Wang, Tianming Yang, Stanislas Dehaene, Shiming Tang, Bin Min, and Liping Wang. Geometry of sequence work- ing memory in macaque prefrontal cortex. Science. 375 639, 2022a. doi: 10.1126/ science.abmen ab00044 URL https: http:::://wwwsceeeeee.e/e/////// www. www.scceecccc....com. www.d/////bbbb/ee http://a/s.o11111/ce//eece
abm0204
15

---

Published as a conference paper at ICLR 2026
Yuanqi Xie, Yichen Henry Liu, Christos Constantinidis, and Xin Zhou. Neural mech-
anisms of working memory accuracy revealed by recurrent neural networks. Fron-
fiers in Systems Neuroscience, Volume 16 2022, 2022b. ISSN 1662-5137.
doi:
www.oonieeesiioooooooooooeeeeeee systems- htt.://ccccceeeeeeeeeeceecceeeee Celccles/in ........19/////....... (02.8////ss...22...2222 760864
10.33899fnnsss2222276600 760864. URL https: /tww.w/.w........
Rafael Yuste. From the neuron doctrine to neural networks. Nature Reviews Neuroscience, 16(8):
487-497, 2015. doi: 10. 10.1038/m.396..
Xiaohan Zhang, Shenquan Liu, and Zhe Sage Chen. A geometric framework for understanding dynamic information integration context-dependent computation. iScience, 24(8), 2021.
Nicolas Zucchet and Antonio Orvieto. Recurrent neural networks: vanishing and exploding gradi- ents not the end of the story. Advances in Neural Information Processing Systems, 2024.
16

---

Published as a conference paper at ICLR 2026
Appendix
TABLE OF CONTENTS
Assumptions about data
18
Decomposing the k B-Delay Loss
18
B. I Derivation
18
B.2 Discussion on mean correction and composition interference terms
20
C Geometric interpretation of learning dynamics
20
Analyzing the spectral radius of WA
21
D.I Proving the echo state property
D.2 Optimality of spiral sinks 2D
22
D.3 Optimality of acute-angled spiral sinks
23
State space model (linear recurrence, nonlinear readout)
25
Approximation of loss under high sparsity
25
E,2
Simulations across and sparsity for nonlinear readout
27
Nonlinear RNN (nonlinear recurrence, nonlinear readout)
28
F.1
Discussion on applying nonlinearity to hidden layer
28
Nonlinear RNNs have linear feature directions the limit of high sparsity
29
F3
Simulations across and sparsity for nonlinear recurrence
30
Higher dimensional results
30
G.
Decomposing the loss for vector inputs and outputs (N, >
30
Analyzing higher-dimennional hidden states (N,, 2)
32
H
Tasks with random delay
35
Experimental details
38
Number of models that develop the feature geometries discussed
38
Weight tying.
39
Data generation
40
1.4
Model definition, initialization and training
40
1.5
Figure details
41
17

---

Published as a conference paper at ICLR 2026
A ASSUMPTIONS ABOUT DATA
We make a few simplifying assumptions about the form of the data to make our setting analytically tractable:
Assumption (temporal independence): Inputs are generated from an IID stochastic process of scalar random variables {Xe)
Assumption (temporal sparsity): Following Elhage et al. (2022). we let X, = B,U, where B, Berouuli(p). U. identically distributed according any distribution and all { [B.) BB)22 (Ug)21 are mutually independent.
Our second assumption allows to explicitly control temporal sparsity by varying p. smaller (larger) p corresponds to higher (lower) sparsity. Setting p = recovers an arbitrary IID stochastic process.
While temporal independence may be a strong assumption, is standard in the literature on memory capacity (Jaeger 2002] Ballarin et al. 2024). Temporal sparsity may also be strong assumption, depending on the task or setting considered.
B DECOMPOSING THE k A-DELAY LOSS
Here, we derive a simplified form of the loss of linear RNN trained on the k-delay task in terms of four interpretable terms.
B. DERIVATION
Under the assumptions in Appendix A the expected value of the squared-error loss (Equation (4) incurred by linear RNN simplifies as follows:
[[C]
E
Σ
Σ
x Y.
t=k+1
E
222
+ E
X - 880)
(8)
1=
T
Y?
Σ
Xx- - 8:)
1=b+1
E 132
E
[x - Σ
E x
X
+
Σ
(10)
1=k+1
t=k+1
k=k+1
T
Σ
E [x2
- 21 (x-xXo]) + Σ
(11)
t= 1=
1=1
X Σ w W. X 1) E [x 2B
1)
+ Σ E
Σ w. w. Xe-s
(=
s=0
(12)
We proceed by computing each of these expectations separately. Recalling that, by the temporal sparsity assumption, X, B,U,, we let μ = E [Ur] and v = E [U7]. Then, as (Xth21 are assumed be IID, the first expectation becomes
E [ E [B? =
|
E BU
EE pM (13)
18

---

Published as a conference paper at ICLR 2026
Next, in computing the second expectation, must handle the case of A separately. This yields
X k
w w
X
E [X]_
wy
w w
E x u/ uu w.
(14)
1-1 E B B B(--- 1 ws
purty w w,=k
(15)
==0 sfk
pune, w wa=k + p p²
w W.
(16)
s
For the third expectation, we must handle the diagonal terms (where = 8') separately;
E
Σ wy w
Xt-s
E
Σ wwy w.) )² x2 Σ
(w, ws) (wi (//ww)) Xt-sXe-s'
4,5'=0
xtss
(17)
1-1
W ws)
[x2_] + Σ
(w) _ w.) (w,) w. E X Xe-s
2.6==0
(18)
pv
m wg)
+pppp²
Σ
(w)) w.) (w,, w.
(19)
6,6
Substituting these expressions into equation 12 we obtain
T
B|C = Σ
pv
2 w - 2p²22²
W W.
(20)
1+++
PV Σ
+ 2211² Wn w.)
(w w.) (w, war)
(21)
t=1
==0
5.8
s
Finally, we move every occurrence of the pv (w, w² w 2 term into the first summation:
E
Σ
pv
2 Wo=k pv (w,, wo-k)² 2p² 41
w y)
w
(22)
k=
1-1
1-1
+
pv Σ
w w.)
+ p p²p1
Σ
(w)) T w.) (w ws))
(23)
8,8' =0
19

---

Published as a conference paper at ICLR 2026
After factorizing, arrive the form:
T
B|C| = Σ
pv w Wa=k
111
2p 1
w y
Ws
(24)
t= t=++1
lusk benelit
correction
Pv
w w.)"
+p²
Σ (w (.) wor)
(w)
(25)
5,8' ssfaa
projection interference cost
composition interference
We omit the O indexing of summations in the main text to reduce clutter.
B.2 DISCUSSION ON MEAN CORRECTION AND COMPOSITION INTERFERENCE TERMS
By looking at expectation of the loss, we can see that the mean correction and composition inter- ference terms only exist when the data distribution has non-zero mean. However, can also that the two terms are scaled by p². This implies that, even for data with a non-zero mean, in the sparse regime where take the limit of - the impact of these two terms on the loss becomes negligible. The mean correction term disappears because, even though the underlying data distribu- tion U has fixed mean EU μ., the mean of the overall input X, is proportional its sparsity: E[X] [U]] = pps so E + 0 as p - O and sparse data does have an approximately zero mean. Furthermore, the composition interference term disappears because the probability of two or more temporal features simultaneously activating is Opp²). so composition interference becomes negligible as p + O.
C GEOMETRIC INTERPRETATION OF LEARNING DYNAMICS
Building on earlier work in feedforward linear networks Saxe et al
2014), recent work has inves-
tigated the learning dynamics of RNNs (Schuessler et al. 2020b Proca et al. 2025 van Rossem & Saxe 2025 Ger & Barak 2025 Bordelon et al. 2025 It's been shown that neural networks initialized with small random weights appear to undergo an initial phase cigenvector alignment Atanasov et al. 2022). followed by learning of the largest (and latest occurring) data correlation
singular values/eigenvalues (Saxe et al. 2014: Proca et al. 2025).
Having derived an exact expression of the expected loss in terms of four interpretable terms, here we study their learning dynamics to better characterize the geometric evolution of the network (Fig- 12)
ure 2). Additionally, to better understand the functional behavior of the network throughout training. Taken together, this analysis provides direct connection between the model's functionality/peffrmnnce and its representational geometry.
visualize the dynamics of the output projection (w0 w. for 0
First, we observe that our predicted expectation of the loss (in red) closely matches the empirical loss (light purple), corroborating our theoretical result. Interestingly, we see that the loss decreases in "staircase, consistent with et al. (2014) 4): Proca et al.) (025). particular, [aax et al. 2014) identified that in feedforward networks, the largest data correlation singular values are learned first, corresponding to staircase-lit like drops in the loss, while Proca al. (2025) extended this work to RNNs and identified that task dynamics with singular/eigen-val -vvlluu that are large and occur later in the trajectory are learned faster (i.e.. stronger correlation with the most recent past). Work in feedfor- ward networks has linked these stage-like learning curves to trajectories passing near saddle points saadlle-to-sadee dynamics) (Jacot et al. (002). In RNNs, a similar phenomenon has also been stud- ied, linking abrupt learning bifurcations caused by geometric restructuring (Haputhanthri et al., 2024). Interestingly. our results indicate that such geometric restructuring can occur without the emergence of attractors, model instead leams spiral.
20

---

Published as a conference paper at ICLR 2026
Step O
Step 12500
Step 14700
Step 15000
Step 17500
1.00
Wi eigvals
0.0.77
Singular value
1124
CLBS
0.63
0.59
0.75
0.59 D.l
68
0.59
0.50
Wy goometry
0.25
0.00
10000
20000
Training step
(a)
(b)
Figure 6: Learning dynamics of eigenvalues, singular values, and geometry of a linear RNN. (a) The eigenvalues W, first learn the real part, corresponding to the alignment of feature directions w, the readout wy (aligned y-axis), followed by the learning of the complex part which forms the rotation of feature directions. (b) The learning of the singular values of W₁. approximately correspond to the staircase-lik drops in the loss in Figure 2 and roughly to the learning of real and complex parts of the eigenvalues.
In Figure 6a we visualize the eigenvalues of Wt, along with the corresponding geometry of the fea- ture direction vectors w, different points in training. Interestingly, we see several stages of leam- ing emerge, whereby the real part of the eigenvalues are learned first, corresponding to the alignment of feature directions with the readout. Around step 14700, the RNN seems to have learned a shear transformation. Next, the complex part is learned, causing feature directions to arrange into a spiral. Interestingly, these dynamics differ from Proca et al. (025) in which the real and complex parts of the eigenvalues are learned simultancously when eigenvectors tions. The staircase loss we observe in Figure 2 does not seem to match to the learning of individual eigenvalues, but it does seem approximately match the learning of the singular values in Figure 6b (which roughly corresponds to the learning of the real then complex parts of the eigenvalues). This observation highlights
aligned with the data correla-
interesting relationship between the learning dynamics of singular values and (complex- veale-d eigenvalues, and as such the emergence of geometric structure. An important difference note is that Proca al. (2025) considers linear RNNs that are not eapacity-constranl (do not exhibit superposition) and have data-aligned weights. It's possible that different learning dynamics emerge for capacity-constrane R.NN//solutions relying on superposition.
D ANALYZING THE SPECTRAL RADIUS OF Wn.
D.1 PROVING THE ECHO STATE PROPERTY
In this section, we prove that in our setting, achieving finite loss in the infinite time limit (T - oc) is only possible in models with a spectral radius of W₁. less than I (the so-called echo state property), to demonstrate the optimality of certain solutions found by the networks we study. While there may exist other solutions that perform well on short sequences with spectral radius greater or equal to 1, we show that their performance will degrade as sequence length increases.
In particular, we consider models that satisfy
1 B[C(T)) <
lim T T
(26)
where T refers to the sequence length and L(T) is defined as per Equation (4) We prove that this condition is only possible if the spectral radius of W,, is p(W,,) < 1.
Proof. Consider the expected loss over an entire sequence, [E](()),, as a sum of terms, B[C(T)) = L==1 L + ... + L==T
(27)
where L, is the loss incurred by the model on timestep Then, for t k, we have
Ta>* = pu Ilo] us= - 11|² 20p2-22211
w w.
(28)
21

---

Published as a conference paper at ICLR 2026
1-1
+ pv
y ws)=
+ +
Σ
(wy w₂) (m wz
(29)
5,'==0
This rearranges to
1-1
2
Li>k 1pv-pppp p²p
|100 [uo -112 w w,)²
-11===1|1"-
+p2
Σ
w w -
(30)
whose terms are all non-negative as pv p²2 Var[X 20
If the loss incurred large enough timesteps, L diverges to 1
oc, then the average loss per timestep, T (T)),, also diverges to infinity. Hence, for the average loss per timestep to remain finite, we require Li>k k to remain finite.
Now, assume that p(W))) 2 1 and that Lisk does not diverge to infinity. As all terms in the expression for T are non-negative, the series Σeşh (w, w.)² must be convergent. This implies limg-0o w wz = 0. Expanding, we obtain
lim wy w. = lim wy W w. lim Wi) Wn w. - wn = s
(31)
55000
where denote w lim Wg. which is non-zero p(W,) 2 1. The equality holds precisely when
W2 E k)
Wy ker (WoOT) wy T W W2
Since W # O, we have rank(W, = rank(W > 0. Hence, by the rank auk-nullity theorem, Mlmnker(W, = N₂ rank(W < N₂, and dim(ker(W,, ") N₂ rank(W < N₁- Thus both ker ( and ker have Lebesgue measure zero.
If w₂ ker(W?))), we require the third case, where WT must lie on the (N₂ 1)-dimensional hyperplane orthogonal W w. This again proper subspace of R.N. with measure zero. Hence, if p(W,) 2 1, then the set of solutions for which La>K remains finite has measure zero, so Le>k almost surely diverges infinity. Taking the contrapositive, if Lesk remains finite, then p(W))) < 1 almost surely.
Intuitively, we have shown that if p(W,) 2 1, then Ta> diverges to infinity except if uz,, W and w, precisely (not approximately) satisfy certain conditions. We can be confident that these conditions are not satisfied by models in practice: would require the optimizer to balance the model parameters infinitely thin "knife edge", which is practically impossible in doating-poimt arithmetic. Hence we can safely restrict our attention to models satisfying p(W,,) < 1.
D.2 OPTIMALITY OF SPIRAL SINKS IN 2D
In the proof above, we have shown that p(W,)) < in order have finite loss as sequence length T - co. This condition on the spectral radius is known as the echo state property and can be interpreted as the model forgetting old inputs over time. This constraint guarantees the shrinking aspect of spiral sink. We now verify that the spiral behavior, specifically, is the optimal solution in dimensions. To do this, we train linear RNNs parameterized such that the trace and determinant of W,, 2x2x2 is fixed. Specifically, for each desired trace-determinant pair (r, 8), we optimize over the 2-dimensional manifold
M (WA 2x2 tr (wW)) T, det( 5}
(32)
which we parameterize by (0,, 02) E R2 using the map
0
(001, 02)
exp( (xp(02)
Lexp(02)
T 0
22

---

Published as a conference paper at ICLR 2026
2 8
8 = 7²
Obtuse spiral source
Acute spiral source
Saddle
Obtuse spiral sink
Acute spiral sink
Saddle
-3
-2
2
3
Stable node
Saddle
Saddle
Unstable node
8
Figure 7: Trace-determinant classification 2-dimensional discrete linear dynamical systems.
Adapted from Galor (207). Stable systems occupy the triangular region enclosed by the lines 5 = 1, = x - and
Within this triangle, spiral sinks are found above the parabola 1
where 0g is exponentiated to ensure yp is bijective. We verify that tr(vp(01, 02)) = 01 + r - 01 =T
(33)
det(ip(01, 02)) = (T - 02) (1( 01)
exp((22 =
(34)
exp(
We sweep through a grid of points the square (r, d) E [-2, 2) and for each point, we train linear RNN parameterized as above. The loss achieved by each model under various task conditions
shown in Figure Each plot can be thought of as a trace-determinant slice of the k-delay loss landscape.
We note that in every case, the optimal solution (brightest point on the plot) is found in the region that corresponds to spiral sinks. This provides strong empirical evidence that the globally optimal solution must be a spiral sink. This makes intuitive sense: rotation is used to implement an approxi- mate delay-line solution by shifting inputs through sequence of positions in the hidden state, while the gradual shrinking of vectors facilitates the forgetting of old inputs.
D.3 OPTIMALITY OF ACUTE-ANGLED SPIRAL SINKS
Interestingly, the plots become increasingly symmetric with increasing sparsity. In the low-sparsity case for both the 2-delay and 4-delay tasks, the global optimum clearly has tr(W,) > 0, which implies acute rotation angle. This be seen by writing the eigenvalues of W,,
A1.2 = - -6
(35)
and observing that t < 0 corresponds to arg(A1,2) E (5, 3m (obtuse rotation angle) while >
corresponds to arg(A1.2) (A1,2) (-7, 7, 3) (acute rotation angle).
The asymmetry most clearly visible in the figure for the 2-delay task 0.7 sparsity. Figure we decompose this particular loss landscape to investigate the reason for acute-angled rotation being preferred. While none of the terms are perfectly symmetric tr(W)), ), clear that the mean correction term is largely driving this behavior: for acute spirals, it can reduce loss, while for most instances of obtuse spirals, it increases the loss. Unsurprisingly, there exists trade-off' between this term and the others (in particular, composition interference largely seems positive where mean correction is negative, and vice versa), but evidently the optimal balance is firmly in the acute spiral region. Overall, there seems to be region in which task error, mean correction and composition interference are all relatively low, while projection interference relatively high - this corresponds precisely with the lowest-loss region in Figure and is exactly the sacrifice observe in Figure
23

---

Published as a conference paper at ICLR 2026
1 - 1-p=0.7
p-p
0.999
det( W(
k=2
-2
X
det(WW)
-2
tr( (W))
tr(W,)
tr(W))
1.0
2.0
Relative loss
Figure 8: Loss landscape of the 2-delay and 4-delay tasks. Shown in terms of tr(W,) and det(Wi,) at 3 sparsity levels for linear RNNs with 2-dimensional W₁. At each point on the trace- determinant plane linear model with 2-dimensional hidden state parameterized, described above, with a fixed trace and determinant. The final training loss is displayed as multiple of (i.c. relative to) the lowest training loss achieved by any of the models. The best-performing model for each task marked by a cross. Standard lines and curves used to classify discrete dynamical sys- overlaid white; refer Figure for interpretation.
24

---

Published as a conference paper at ICLR 2026
X
0.54
X
0.48
Loss contribuutin
ii Task emror
(ii) Projection interference
-0.35
0.13
-2
(iii) Mean correction
(iv) Composition interference
Figure 9: Decomposed loss landscape of the 2-delay task in the trace-determinant plane. -rateddetermnnatt loss landscape of the 2-delay task 0.7 sparsity for stable linear RNNs with 2 2-dimensional Wh, decomposed into interpretable terms. The best-performing model (in terms of overall empirical loss) is marked with a cross and annotated with value for each of the loss terms.
STATE SPACE MODEL (LINEAR RECURRENCE, NONLINEAR READOUT)
E. APPROXIMATION
UNDER IIGH SPARSITY
We now consider the SSM that produces its output through ReLU activation function, correspond- ing to By ReLU and ah = id. Due to the ReLU activation, such model can only produce non-negative outputs, so for the k-delay task, it is sensible to restrict the input distribution to be non-negative as well (as we desire dh (eL.U(...) = x(-k). Recall that the input distribution X, B,U,, where B, Bernoulli(p) and Ug is identically distributed according to any distri- bution. Then, since B, 2 0, we require U, 2 0 to ensure X, 2 O. For example, the distribution used by |Elhage et al. (2022), where U, Uniform (0, 1], would satisfy this requirement.
We now prove some key results about the expected value of the ReLU function applied to functions of random variable Z 0. Firstly, note that
E[ReLU(Z)] = E[Z]
(36)
We now consider (V))) for an arbitrary random variable V E I,, which is permitted to be negative. Since 7 0, have
E E[ReLU(V2)] E[ReLU(VV) = E[[EELL((V)
(37)
In the special case of a constant V = v. this simplifies to E[ReLU((u2)) = ReLU(U)E[Z]
Now, recalling that B., U,, X, with E[B.] p. [EUU)] l and E. [[X.] pp, apply these results to the expected model output given by
E|] E ReLU
Σ (uTTeXXroo)
(38)
Previously, we were able to apply lincarity of expectation to split the expectation of a sum into of expectations, but the ReLU non-linearity precludes using the same approach here. The distribution of the interior weighted sum of uniformly distributed variables known (Bradley & Gupta 2002), but its complexity explodes with increasing t. We therefore opt for approximation.
Let the spectral radius W,, Then the feature ;;- is represented by the vector w, = WW in the hidden state h,. For old features, corresponding to large have ||Wll| R p² by Gelfand's
25

---

Published as a conference paper at ICLR 2026
formula. As argued in Appendix D.I
only concerned with the case of < 1, for any E > O, there exists a "memory window" of length T. = [ log(p)] such that for s 2 Te, ||W ppp p с. Hence the contribution of any input older than Te has magnitude of order
(e)) Intuitively, this means that if we set small enough (and thus Tg large enough), we can ignore inputs older than T. time steps.
Therefore, the only inputs that can have a significant effect on the model's behavior are those which arrived in the last T3 time steps. Since each input is masked by Bernoulli random variable, the num- ber of non-zero inputs that arrive in T' time steps is distributed according to Ng Binomial p). This quantity essentially counts the number inputs actually "in play," meaning that their effect on the hidden state has magnitude larger than E. Hence, the probability that there are two or more such inputs is given by
p² ss TTT 1),
Pr[N. 2 =1- - (1- (1- -p
(39)
where the binomial approximation holds for small Therefore, if we are willing to ignore cases that arise with probability less than some 5 > 0, we approximate the behavior of an RNN by its behavior on input sequences with only one non-zero input for Pr[Ne 2 2] < 6. This occurs when sparsity is high enough make it vanishingly rare for more inputs to be "in play" simultaneously. Specifically, the approximation is valid when
28
V28
p <
V TT - 1) log(1/E) log(1/p),
<
(40)
or, cquivalently,
plog(1/E)) exp (plo(()) V28
P< exp
gP/v22
(41)
v28
In particular, for arbitrarily tight o, > 0, there always exists p small enough to make the ap- proximation valid for any given model with p <<! Under this approximation, our analysis of the ReLU-gatee model becomes tractable, as we can ignore all cases that involve two more non-zero inputs. For instance, the expected model output becomes
1-1 Σ wh woXe- 6==0
]
E ReLU
E ReLU (w,,. wyX(-))]
(42)
e[ ReLU (w. ws)
(43)
ReLU (m. (m)ww))
(44)
We now follow the steps of Appendix B.I applying this assumption to derive an interpretable ex- pression for the expected squared-error loss E[C]. We begin with
Ew w. w waxX.m)) ))
(-1 Σ
E[C]
E X
2E
X, ReLU
(45)
t=k+1
T
(-1
27
+
ReLU
Σ
w y
w. Xe-
(46)
As before, the first term is E. that no more than one of the inputs is non-zero. There are two cases: either X,-k is zero, in which case the entire term collapses to zero, or X, & non-zero, in which case all other X.-s are zero for f k. Hence the second expectation simplifies
[X pv. evaluate the second term, we again use the assumption
]
E X ReLU
Σ Ew waXe-s 0=0
R ( ReLU(w, w.=)X
(47)
26

---

Published as a conference paper at ICLR 2026
E [ ReLU w
(48)
pu ReI.U(w. w.=k)
(49)
Similarly, in the third expectation, the only non-zero summands those the "diagonal" (all off-diagonal terms require two inputs to be non-zero, so we ignore them):
1-1 wy w. Xi
)
RcLU
Σ
2
ReLU (w, (w,, w,
(50)
[x]_a] ReL w.)
(51)
Re. w.)²
(52)
Putting these together:
T
T (pv 2p ReL.U(w. Rel.U((oTw....)) w)=
B[C] R
Σ
(53)
1=k+1
pv
(ReLU(w, w. () 11)
Re w.)²
(54)
1=k+1
E.2 SIMULATIONS ACROSS & AND SPARSITY FOR NONLINEAR READOUT
In Figure 10 we provide additional simulations of the solutions learned by optimal SSMs for differ- ent values of k and sparsity 1 accompanied by the spectral radius and the angular distribution of task-relevant features k0.
In the SSM, we observe that the spectral radius (i.e., ||w,=ol)) increases with k regardless of sparsity (Figure 10). This is simply because for larger k; the model must hold inputs in its memory for more timesteps. If the spectral radius is too small, then the magnitude of w in the hidden state will be negligible relative to other w wayk and so any task-relevan signal will be overpowered by projection interference from other features.
27

---

Published as a conference paper at ICLR 2026
1 -
1 0.7
1- 1-p=0.9
1 0.97
1 1- 0.99
P 0.67, k0 79.4"
0.56 274.0"
0.46, 267.9"
p = 0.45, kD 180.4
P 0.35, 272.8
p 0.76, 81.9"
0.56, 269.0"
270.4"
p 0.45, 270.0"
p 0.43, KB 269.9*
k
0.84, k0 86.4"
0.90, ΛY 796.6"
0.76, XO 256.8"
0,50, KD 229.6
0.48, KD 231.6'
p
p
p 0.80, 256.3
p 0.76, 247.7
Figure 10: Solutions d-delay learned by linear models with ReLU read-out (SSMs), Rows correspond to different values of k and columns correspond different sparsity levels. The w, are plotted after applying a conformal linear transformation such that the y-component of each w, is wy Ws. and w² wa=0 points towards positive Thus the interference-free half-space is simply given by y < O. As before, the output feature, w.=k- is marked with a star. Note that the plots vary significantly in scale, so it is meaningful compare the magnitude of partieular w, vector between different plots. The angle 0 is calculated as arg(A1) where A₃ E C is an eigenvalue of W,.
F NONLINEAR RNN (NONLINEAR RECURRENCE NONLINEAR READOUT)
F.1
DISCUSSION ON APPLYING NONLINEARITY TO HIDDEN LAYER
Recall that we have defined the RNN architecture as
h: = w; + Wio (h4-))
9: = (W, he)
(55)
While this slight departure from traditional RNN architecture in machine learning (with regards to what we label as h, and how the output ÿt is computed), this form is standard in computational neuroscience, Moreover, prior work has shown that the recurrence these models is mathematically
equivalent (Miller & Fumarola 2012).
MI h, Th Wa + Whhh
(56)
ourfbeuro Por
28

---

Published as a conference paper at ICLR 2026
The motivation for using this form of RNN is that it allows to study nonlinear RNN for which the linear representation hypothesis provably holds (under certain sparsity conditions), as shown in Appendix F22 and is the most direct extension of Ellhaee et al. (2022) to the recurrent setting.
F.2 NONLINEAR RNNS HAVE LINEAR FEATURE DIRECTIONS THE LIMIT HIGH SPARSITY
Applying a non-linearity to the hidden state immediately breaks the linear representation hypothesis: each feature would be represented along (not necessarily smooth) curve rather than straight line. This massively complicates the study of non-liness recurrence in general.
For the case of linear recurrence, we have shown that for sufficiently sparse input sequences, we can assume that there is at most one non-zero input "in play" within a model's hidden state at any given time. Fundamentally, this was based on the idea that there exists memory window of length such that inputs older than T time steps cannot contribute significantly to the current hidden state. This essentially arose from the proof Appendix [D. that RNNs with linear recurrence must satisfy the echo state property in order to achieve reasonable loss.
We argue that a similar memory window should be expected in models with nonlinear recurrence. Although is much harder to prove that the echo state property is a requirement for good perfor- mance in non- non-linear models, there exists plenty evidence for the reverse statement: echo state networks are, by definition, nonlinear models that satisfy the echo state property (Jaeger) (202) and have been shown to achieve strong performance on a variety of sequential processing tasks (Aceituno) et al. 2020). Clearly, the echo state property is not incompatible with strong task performance and it is not implausible that the linear result - that the echo state property is actually required for tasks like k-delay carries over to the case of nonlinear recurrences, based on these empirical observa- tions. Furthermore, it's well known that RNNs with spectral radius over cause exploding gradients and training instability (Bengio et al., 1994 HHccheiter et al.) 2001 Pascanu et al.) 2012), and that task dynamics with strong early sorrelations (producing a spectral radius over 1) result in network instability (Proca al.) (025). Therefore, we intuitively expect our reasoning to hold for nonlinear recurrence: if the input sequence is made sufficiently sparse, we can approximate the model's 's behav- ior by ignoring situations where two or more inputs are non-zero, as historic features are gradually forgotten by the RNN due to the shrinking effect of the spectral radius (p 1).
Unrolling the hidden state, we see that an analysis of the general case intractable:
h: W.. Wa ReLU(h;-1)
= w. + W, ReLU (WL + W, ReLU(h)-2))
W. Wn, RcLL( + W₁, ReLUU..... W,, ReLU(W,ry + W,, RcLU(w,.11)) ...))
Suppose, however, that for some only Tt-s is non-zero and all other inputs are assumed to be zero, as per our approximation. Recalling that in our setup, It-s is scalar and non-negative, the hidden state simplifies as follows:
h = wr (0) + W, ReI. (0) + W, ReLU( .. W, Rew.U Wx) +W.())
(57)
W₁, ReLU(W, ReLU(-.. W, ReLU(W,,
(58)
W, RcLU(W, ReLU(......- W, ReLU(w..)... ...))
(59)
st-s WA ReLU(W, ReLU(... W. ReLUUw...... ...))
(60)
(61)
Though is not analytically possible to find simplified expression for the vector w., it nevertheless the direction in which the feature zt-5 is represented in the limit of sparsity, as p - 0. Hence, in the extremely sparse regime, the linear representation hypothesis holds for this model. This is not a trivial result; it relies on both the piecewise linearity of ReLU for non non-negative inputs and our definition of h, the hidden state prior to application of ReLU.
29

---

Published as a conference paper at ICLR 2026
F.3 SIMULATIONS ACROSS k AND SPARSITY FOR NONLINEAR RECURRENCE
In Figure 11 we provide additional simulations of the solutions learned by optimal nonlinear RNNs for different values and sparsity In the nonlinear RNN, we can observe how the model learns to exploit the interference-feee space sparsity increases and implement sharp forgetting.
1 1-p=0 0
1 p= 0.7
p 0.9
1 p 0097
1 p 0.99
k k=
Figure 11: Solutions to k-delay learned by nonlinear RNNs. Rows correspond to different values of l and columns correspond different sparsity levels. The w, are plotted after applying confor- mal linear transformation such that the y-component of each w, is w, y w., and w² u.=0 points towards positive æ. Thus the interference-free half-space is simply given by y < 0. As before, the output feature, W.=k. is marked with star. Note that the plots vary significantly in scale, so it is not meaningful to compare the magnitude particular w. vector between different plots.
G HIGHER DIMENSIONAL. RESULTS
G.1 DECOMPOSING THE LOSS FOR VECTOR INPUTS AND OUTPUTS (N₂ 1)
We can repeat the decomposition in Appendix B obtain an expression for the expected value the squared-error loss incurred by linear models in the case of vector inputs and outputs (N₃ N, > 1).
We recall that X,, Y, TR N= W. N.NN and W,, RNNxNN and proceed before:
E =
4
| 2112
(62)
1=
Σ>
Σ
E
X 8/1
(63)
30

---

Published as a conference paper at ICLR 2026
T
E
||001
Σ
E
|
- 2x, N + ||:||]
(64)
=k+1
Σ t=++1
E x
|2]
2
Σ B |1:111]
T
(65)
1-1
1)
T >E E
27
Σ (=++1
E || - -2E
X Σ
W W.Xx Xx_
W W. X
s=0
(66)
We extend the temporal sparsity assumption to assume that each input feature follows the same distribution, so that x, ())
= B
o with B(1). ~ Bernouli(pp). U.! is identically distributed
according to any distribution and B
U are mutually independent.
Then, with μ = E [
and =
(1700)
27
we can simplify the first expectation to
E X 11]] = E
Σ
N₂
(x(()
27
Σ
N₂
E
(B...)
Σ
N,
Nap
(67)
1=1
In computing the second expectation, we must handle the case of s = k separately:
1-1
E
W y W.X WoXe-
X_ W WiXke..)
E
lr
(68)
tr E [ W W W.xeeo))
(69)
tr (W,) W.B [X==XX,-1)
(70)
(W, W.E [X,+XX,
Ir W.W,W.WWWEEW. [X))
W.E
(71)
ir (W, WrE [x=+ p22 Σ
Ir (w, W1,11)
(72)
sxk
= 1 T W We1+ + (pv - x² ur (W,) Wz)] + +1² 11 Σ W.1 W.1
(73)
eff
(W. W., Ws) + 2²²2² 1-1
= (pv-
Σ 1 1 y
W.1
(74)
s=0
where we make use of the all-ones vector = 11, _ E N. Finally, the third expectation simplifies as follows:
E
W W
N.X.A.
T
W. We Wox ]
= E
Σ W W. WoXe-e
Σ #==0
(75)
31

---

Published as a conference paper at ICLR 2026
( Ww,,, W. X
E
Σ x 6- W. W.
Wo
(76)
E p W. W.WWW.WWWW......
(77)
tr (W, W, W W. X X.
(78)
tr (W, W. ạ W.E
Σ sepisl
tr (W, W,W,
W. W. x
(79)
p²y Σ tr (W. Wy W W.111) + (pv pp'pp'pp tr(w, W, W. Wa)
(80)
+ 2² t-1
Σ
(W. W, W 11
(81)
s7s'
(pv -2²1²²)
tr(W, W W W.) + p2
W. T W. W W. W.1
(82)
s=0
e=0 ='=0
= (pv - -221²)
Σ
W, w. + +
Σ W W, W,I.I
(83)
where| || ||5 is the Frobenius norm.
Putting all the terms together yields
T
E[C] =
Σ
Napu - 2 (pv- p22)) tr WR) + +ppp²
1 W W.1
(84)
1=
==0
T
1-1
1-1
+
(pv x²² 1²²)
Σ I| Wa +
Σ
Wg y
W,1
(85)
0=0
s=0
G.2 ANALYZING LIGHERR-DIMENSIONE IIDDEN STATES (N₂, > 2)
So far, for the purposes of feature geometry, we have restricted the hidden state of our models dimensions. In this section, we demonstrate that the results we have found generalize well to RNNs with higher-dimensional hidden states. In particular, find that the interference-fire space is not only present, but very well exploited in higher dimensions.
Figure 12 shows the results of training higher-dimension nonlinear RNNs on the 2-delay task with vector inputs (N₂ = 10). Even in the 10-dimensional hidden state, there is still significant superposition occurring:
(o-dimensional hidden state can only represent 10 features orthogonally, whereas the task requires 10 features to be held in memory over l + 1 3 timesteps, equivalent to 30 features being compressed into 10 dimensions.
The 10-dimensional case exhibits very little projection interference most cells are blue except on the s = 2 diagonal, where we expect them to red. This shows that most activations lie within the interference-frre space, where their projection onto every readout vector is negative (and therefore their contribution to projection interference zero due to the ReLU activation function). We believe this is strong result that demonstrates that the interference-fre space is significant driver of feature geometry, even in or perhaps especially in higher- hidden states.
Based on these findings, extend this idea to larger RNNs, developing metrics to quantify the degree to which these networks exhibit the geometry we would expect based on our study. In particular, we measure the mean non-output feature direction projections onto the readout (i.e.,
32

---

Published as a conference paper at ICLR 2026
$
5
(5)
Readout
(
Feature j
Feature
Feature j
Feature j
Feature
Feature j
(a) 2-dimensional hidden state
==0
$
s 2
4
s 5
Readout
M
Feature j
Feature
Feature j
Feature j
Feature
Feature j
(b) 5-dimensional hidden state
y 2
y=
8 5
Readout i
w
W
Feature j
Feature
Feature j
Feature j
Feature
Feature j
(c) 10-dimensional hidden state
Figure 12: Projections of feature directions onto readouts in higher-dimensions hidden states. Each model was trained on the 2-delay task over 10 features (hence there are also 10 readouts). Each plot shows the values of W W the projections of feature directions corresponding to s-timestep- old inputs onto each of the readout vectors. Specifically, each cell (i.j) represents the value of (;(0) (1) where (+(() is the i-th column of Wy, and W)( is the j-th column of Wj. Cells that red correspond
positive projection onto the readout this is desired only the diagonal of the s = panel, where features are being read out into the correct outputs at the correct time; red elsewhere represents projection interference. Blue cells have negative projection onto the readout and, due to the ReLU, do not contribute to projection interference for their row's 's readout. (a) A 2-dimensional hidden state is only able represent features well and ignores the rest clear example of the "all-or-none" strategy discussed in Section 4.5) This result is equivalent to the panel of Figure (b) 5-dimensional hidden state performs much better. representing many more of the features, albeit with some interference. (c) A 10-dimensional hidden state performs very well, representing all the features with almost no interference.
33

---

We also measure the mean of the output feature direction projections onto the readout (i.e., mean(diag(W) W,=x))), which quantifies whether the appropriate features are projected onto the readout at the correct time to perform the task. We would expect this value to be positive for models matrix in Published as a conference paper at ICLR 2026
0.06
0.0036
Mean output projection onto read-outs
0.0034
0.04
0.0032
0.02
0.0030
Training Loss
0.0028
0.00
0.0026
0.0024
0.02
0.0022
-0.04
0.0020
0.05
-0.04
-0.03
-0.02
-0.01
0.00
Mean non-output projection onto read-outs
Figure 13: Mean projections output and non-output feature directions onto readouts colored by loss in nigher-dimennsonal hidden states. We train 1000 models with hidden size of 100 on the 2-delay task with 75 features. The x-axis indicates the mean of the non-output feature direction projections onto the readout (i.e., W W s.).) This is the analogous to the mean of the * 2 matrices in Figure 12 The y-axis indicates the mean of the output feature direction projections onto the readout (i.e., W W.-k), which is analogous to the diagonal of the 2 matrix in Figure 12 By looking at the plot, we see that all models learn to have non-output feature directions that have negative projections onto the readout (negative x-axis values), utilizing the interference-fre space. We also see that performance is correlated with the RNN's ability to positively project the output feature direction onto the readout, indicating that the optimal models are using the the interference- free space to minimize projection interference with this output feature.
mean(W TW y We**)) x)), which allows us to quantify the degree which non-output feature directions space. In particular, this value should be negative if the largest
group within the interference-fie non-output feature directions are grouping within the interference-fife space. This is analogous to the mean of the + 2 matrices in Figure T2
that successfully do this. This is analogous to the mean of the diagonal of the Figure 12
We train 1000 RNNs with a hidden size of 100 on 2-delay task with 75 features and plot the results in Figure 13 where each point is representative of single model and is colored by the final loss achieves. By looking the figure, we see that all models learn to group the largest non-output feature directions in the interference-fre space (indicated by the negative-valued x-axis). We also see that the berf-reng models (lowest loss) learn successfully project the output feature direction onto the readout (indicated by the positive-valued y-axis), indicating that these (optimal) models are outputting the correct features at the correct timestep and using the interference- free space to minimize projection interference with this output feature. Our results pertaining optimal model geometry are therefore corroborated by this experiment.
In our experiment, there models that do not learn the optimal solution (i.e., they have negative output feature direction projection onto the readout; the yellow/light green points in Figure 13). We suspect that this has to do with the gradient backpropagation through ReLU, as the negative projections can get clipped to zero by the ReLU nonlinearity, preventing learning signal from going through. Indeed, do not any additional methods assist with training in this experiment.
34

---

Published as a conference paper at ICLR 2026
Moreover, if we train instead with leaky ReLU activation (permitting gradients to backpropagate to negative projections), there are no models with a negative mean output projection (all models learn the optimal solution metric-wise). We note however that the extension of our work to other activation functions is non-trivial and beyond the scope of this work.
H TASKS WITH RANDOM DELAY
In this paper we primarily focus on tasks with a fixed k-delay. Here, we instead consider the effect of training task with random delay. The task consider is identical to the k-delay task in that the RNN must reproduce the input sequence after k timesteps, but now l is random for each training sample (k= ~ Uniform(0,10)) One dimension of the input corresponds to the cue, which remains O until the randomly selected k; after which it is set to 1 and the RNN is tasked with outputting the sequence, corresponding to the input from the t k Vt > k. We keep the input and output dimensions of the network the same- hence, with the addition of a cue input dimension, we also have an extra output dimension that is weighted with 0 importance in the loss. We mask the network output in the loss until after the cue is turned on, such that earlier outputs do not contribute to the loss. We train RNNs of each architecture (linear: Figure[4] SSM: Figure nonlinear: Figure [T)) and visualize how the feature geometry changes as the number of input features (e.g., via input dimensions) is varied (the rows) and sparsity varied (the columns). For each configuration, we train 500 models and plot the one achieving the lowest loss.
Although we are cautious about overinterpreting these plots, we provide a preliminary analysis. The results seem to suggest some intermediate geometry between spatial superposition and temporal superposition. Indeed, the notion of time-dependency here marks a departure from the rest of the paper in that the sequential ordering of features is important for the task, but time -ependenn output is not (in the sense of a fixed k). We see that many RNNs form solutions where feature directions lie shrinking line (instead of a spiral sink), with fixed point the origin (corresponding to forgettinn'). 'Age' (for sequential ordering) is still partially encoded by the magnitude of the feature direction on the line. RNNs also appear to be implementing some form of spatial superposition in some cases, partitioning the activation space for several different features; this behavior clearly contrasts from Figure which, for many input features and delays of up to 10, would only choose to represent one feature. However, we also often see a collapse of several feature directions onto the same line. In fact, although we study up to input features, the models typically converge to approximately 2-3 principle directions. We can also see how for 2 features, RNNs learn to either represent these features approximately orthogonally, or place the features in opposite directions.
Remarkably, in the SSM with high sparsity, we recover the pentagon of 5 features characteristic of spatial superposition. We suspect that the SSMs geometric strategies are a result of each feature direction placing itself in the interference-free space of the other feature directions' readouts (hence the spiraling, perpendicular, and pentagon shapes).
We note that these results are preliminary and that more work, both theoretical and empirical, is needed to characterize the effect of random delays on learned feature geometry. A limitation of this particular set of experiments is the lack of expressivity of the models we consider. Although they learn some interesting structure suggesting the use of the interference-free space/a superposition- like strategy, RNNs with 2 neurons do not have the expressivity to perform cued random-delay lask perfectly.
Previous work has shown that RNNs trained on tasks with random delays exhibit persistent activity in the form of fixed point attractors (Orhan & M| 2019 Liu et al. [02T]] Xie et al. 20225). which has been related to neural activity. In the results we show here, we do not find a clean example of (non- trivial) fixed point attractors forming. This difference may be due to the fact that the task we study requires the reproduction of a sequence relying on temporal ordering which may be challenging (if not impossible) to implement with fixed point solution (at least in the architectures we consider here). A critical component of temporal superposition, as we've introduced it here, is the importance of time-dependency in data (inputs and outputs): temporal superposition occurs because timing is important and inputs and outputs at different timesteps are treated as unique features (represented uniquely). Tasks based random delays mark slight departure, as the precise timing relationship of input-to-outee is modified (such that inputs and outputs may not always be 'unique features'), ) This is also true for other tasks where feature timing is unimportant. However, since the task
35

---

Published as a conference paper at ICLR 2026
consider here still requires the reproduction of sequence based on temporal ordering, such ordering still appears to affect representational geometry. Moreover, depending on the tasks an RNN is trained on and its architecture, it may implement combination of different solutions for different computations/suttasks based on temporal superposition, fixed point solutions, or otherwise (Driscol] al. 2024). Finally, stated earlier, sophisticated solutions and we use a specific version of a random-delay task based on masking the loss prior cue, followed by a sustained cue after the delay. It's possible that other implementations of random-delay task may produce other solutions.
emphasize that 2D models lack expressivity for more
1-p-0
1-p-0.7
1-p-0.9
1-p-0.97
1-p- 0.99
1 features
2 features
3 features
5 fealures
7 features
Solutions to randon-delay learned by linear RNNs. Rows correspond to different numbers of input features (1,2.3,5,7) and columns correspond to different sparsity levels (0, 0.7, 0.9, 0.97, 0.99). Each feature indicated by a separate marker, and 'age' in the network indicated by color (purple is new; yellow is old).
Figure 14:
36

---

Published as a conference paper at ICLR 2026
1-p-0
1-p-0.7
1-p-0.9
1-p-0.97
1-p-00
1 features
features
features
features
features
Figure 15: Solutions to random-delay learned by SSMs. Rows correspond to different numbers of input features and columns correspond to different sparsity levels. Each feature is indicated by separate marker, and 'age' in the network is indicated by color (purple is new; yellow is old).
37

---

Published as conference paper at ICLR 2026
1-p=0
1-p=07
1-p=0.9
1-p=0.97
1-p=0.99
1 features
2 features
features
features
7 features
Figure 16: Solutions to candom-dela learned by nonlinear RNNs. Rows correspond to different numbers of input features and columns correspond to different sparsity levels. Each feature is indi- cated by a separate marker, and 'age' in the network is indicated by color (purple is new; yellow is old).
T EXPERIMENTAL DETAILS
Code to reproduce all experiments and figures
available at https:://ithub. com/
1.1 NUMBER OF MODELS THAT DEVELOP THE FEATURE GEOMETRIES DISCUSSED
To verify that a reasonable number of models actually achieve the kinds of feature geometry dis- cussed in our work, we train 1000 models of each architecture for various delay and sparsity values (Table We use set of heuristics for each architecture to quantify whether models exhibit charac- teristics their expected feature geometries based on our observations. For linear RNNs, we expect the spectral radius to be reasonably large but below and the task-relevan features rotate through the plane less than 180s: 0.5 < p(W)) < 1,5" < k) arg(A)] < 180 where an eigenvalue of W₁, E R For SSMs,
expect the tesk-relevant feature direction vectors to instead rotate through more than 180%: 0.5 < p(W,,) < 1, ki arg( > 180º. For nonlinear RNNs, we quantify whether the output feature direction projects positively onto the readout and other feature directions
38

---

Published as a conference paper at ICLR 2026
project negatively (tolerating some small positive projection): wy wa=k O, o, w. 0.1 for 0 8 I k.
We find that substantial number of models satisfy heuristics. We note that optimizing with two-dimensional hidden space is very challenging for gradient descent. Indeed, in the original work toy models of superposition (Elhage al., 2022), the authors also study 2-dimensional hidden state and report that they fit each model multiple times and take the solution with the lowest loss due these optimization challenges.
In the case of the linear architecture, the number of models that achieve the expected "spiral sink" feature geometry decreases with increasing k this is simply because the task becomes too chal- lenging for such a simple architecture to learn well; in many cases, models resort to oscillatory behaviour that achieves suboptimal but wewer-than-baseling loss. contrast, over 40% of SSMs trained on larger & and under high sparsity learn the expected "spiral sink" solution. For the non- linear model, using a rudimentary heuristic, we find that reasonable proportion of models can be clearly said to have learned feature geometry in which they exploit the interference- prrere space as discussed in Section
Number models (%)
k
p
Linear
SSM
Nonlinear
2
0.9
26.5
31.7
31.4
2
0.97
30.8
20.6
26.8
2
0.99
27.0
28.5
27.6
3
0.9
21.0
41.3
18.8
3
0.97
18.2
32.8
15.6
3
0.99
19.0
33.8
13.8
5
0.9
19.4
53.5
8.20
5
0.97
18.2
48.0
10.3
5
0.99
16.0
45.3
9.40
7
0.9
14.3
47.1
12.3
7
0.97
11.4
42.5
15.6
7
0.99
11.4
41.5
13.0
Table 1: Percentage of models that exhibit characteristies of optimal geometry, as defined by the following heuristics. Linear: we expect reasonable spectral radius and for the feature direction vectors to rotate through less than 180 between input and output, so we count the number of models satisfying 0.5 < p(W,) < and 5 k| arg(A)] < 180%, where is eigenvalue of W,, RR 2x2 SSM: similar to the linear case, but here we expect the feature direction vectors to rotate through more than 180 between input and output, so we check for 0.5 < p (W,) < and k| arg( > 180". Nonlinear: the optimal geometry occurs when all but the output feature direction vector lie in the interference- space; only one feature direction vector should project positively onto the readout vector, check for w, wy > O and w. w. 0.1 (tolerating some small positive projections) for 0 < S k.
I.2 WEIGHT TYING
For visual clarity, in Figure I Figure 3 and Figure 5 we set the readout vector wy w,=k (or, in the case of vector inputs and outputs, W W W.=k). This means that, these cases, wy (or Wy) is not separate trainable parameter of the model and is instead entirely determined by the parameters W,, and w, (or W,). There are two reasons for doing this: first, eliminates the need for a separate readout vector to be shown, making the plots neater; second, it encourages features in spatial superposition to arrange into regular polygons (c.g. Figure Lab. In fact, this is merely an extension of the weight tying used by /Elhage et al. (2022). where the authors set W, = W. for the same reasons. Our weight tying is identical to theirs in the k = 0 case.
We note that this is just visualization trick and, to avoid doubt, Figure 17 plots spatial and tem- poral superposition with the readout weight untied (i.c., with w,, as a separate trainable parameter). Additionally, all figures in the appendix with weights untied.
39

---

Published as a conference paper at ICLR 2026
E
A
A
E
D
C
D
B
D
A
A
(a) Spatial superposition
(b) Spatial superposition (W, untied, shown)
(c) Temporal superposition (wy untied, shown)
(Wy untied, hidden)
Figure 17: Spatial and temporal superposition with untied readouts. Readout vectors are shown as dashed black arrows. (a) W,, untied from Wa= spatial superposition of features no longer forms regular pentagon. This is also true of the results in Elhage et al. (2022). (b) Untied results are more visually cluttered as both the W, and the Wy vectors need to be plotted, and these sets of vectors overlap be at different scales. (c) The difference between tied and untied results most significant when spatial superposition is involved; for purely temporal superposition, the difference minimal. Nevertheless, Appendix Figures [0 and [I each contain many examples of the feature geometry of recurrent models with untied weights.
1.3 DATA GENERATION
Unless otherwise specified, all experiments used Ug ~ Uniform 00, 1),
/ BlU 1 = E(U
When generating data to train a model that contains no bias terms, we only include sequences that contain non-zero input some timestep. This is an optimization that exploits the fact that bias- free models, by definition, cannot produce non-zero output for inputs that are all-zero; we can thus safely ignore all-zero sequences, the model will always produce the desired output in those cases, the loss will be zero and so no gradient will be backpropagated for such sequences. This makes training much quicker: in cases where sparsity is extremely high (e.g. p 0.999), many generated sequences are all-zero. Combined with the fact that we use constant learning rate, this optimization has impact the training of the model.
The specific shape and sparsity of data generated varies by experiment and is discussed below.
1.4 MODEL DEFINITION, INITIALIZATION AND TRAINING
As per Equation (T. there are two activation functions and oy that can be set as follows to achieve a linear RNN, SSM or nonlinear RNN:
Linear RNN: σn id, J. id
SSM: 0₁ id, Jy ReLU
Nonlinear RNN: σn ReLU, By ReLU
In all experiments, the model weights W, E NNN, and WA, E N,N were initialized using Xavier normal distributions. In cases where the readout weights W,, NN.N, not tied to W,W,, initialize W,, using Xavier normal distribution.
In practice, we often trained many models parallel to make efficient use GPU compute. Data was batched into 1000 batches; for each batch, computed each model's 's average training loss over an entire sequence (weighting the contribution of each feature to the loss by the feature's importance) and backpropagated from this value. We maintained exponential moving average (EMA) of this
40

---

Published as a conference paper at ICLR 2026
value for each model according to the update equation:
fEMA
fEMA + 0.01 (EMA)
After training on all 1000 batches, the final value of ÉEMA was used to compare models; unless otherwise stated, the model that achieved the lowest EMA training loss was selected for plotting,
In all experiments, we used the Adamä optimizer with a constant learning rate of 1r = 5 x 10-3,
Feature importance is attributed by weighting the loss according the importance value per feature, as in Elhage et (2022). In other words شi= [ ((ax=*)ll=xt>+ 3el?, where (x is scalar sum of the corresponding importance values of k.
1.5
FIGURE DETAILS
1.5.1 FIGURE
Panel (a): 100 nonlinear RNNs with 2-dimensional hidden states were trained on 10k non-zero sequences of length timesteps each. The task was O-delay, so Yt: = 2t, making this equivalent to the first task used in Elhage al. (2022). The input sequences had sparsity 0.99 (so p 0.01) and contained features, {A, BA,,CC,D,E with importances {1, 0.97, 0.972 0.973, 0.97* respectively.
Panel (b): Identical to panel (a), except trained the k task instead of k 0. To decrease visual clutter, only the most important feature (A) was plotted. Note that due to the random nature of data generation and training, it is not guaranteed that feature A will always be prioritized over the other, less important features (e.g. B might instead be prioritized), but a single feature is almost always prioritized over all others and A is the most common choice.
1.5.2 FIGURES AND
A linear RNN with 2- 2-dimensional hidden state was trained on 50k non-zero scalar input sequences, each of length 20 time steps and sparsity 0.9. The task was 3-delay. each step of training, we computed the values of w. w, for 0 < < 12 and used these to calculate the contribution of each of the four terms equation 5 5o the loss, as plotted in Figure 2 Figure 6 we additionally plot the cigenvalues of W,, and the corresponding w. geometry different points in training, well as the singular values.
1.5.3 FIGURE
For each architecture (linear, SSM, nonlinear), 100 models with 2-dimensional hidden states were trained on the 5-delay task using 10k non-zero sequences of length 25 timesteps each. We used sparsity level of 0.99, so p 0.01. We plotted illustrative examples of models achieving the lowest loss.
I.5.4 FIGURE 4
Here we trained models on the = 7 task. We swept through 200 uniformly spaced sparsity values in the interval 1); at each sparsity level, trained 1000 SSMs with 2-dimensional hidden states on 10k non-zero scalar input sequences of length 25 timesteps each. For each sparsity level, took the best 50 models (top 5%) terms of lowest EMA training loss. The mean value and standard deviation of and ko across these 50 best models for each sparsity level was plotted, thus indicating the "optimal" p and ke each value of sparsity.
I.5.5 FIGURE
This experiment is identical to that for Figure except that A = k and k 3 were also included. The heatmap was computed by taking 2000 x 2000 grid of points within the axes and, at each point, computing the sum of its non-negative projections onto the readout vectors (columns of Wr)). The heatmap therefore visualizes the region in activation space within which projection interference is zero the interference-free space.
41

---

We swept through 300 uniformly spaced values for [-2, and 300 uniformly spaced values Published as a conference paper at ICLR 2026
1.5.6 FIGURE AND FIGURE 9
for ó E |-2, 2]. For each pair (T, 6), we parameterized linear RNN with 2-dimensional hidden state as described in Appendix D.2 Each model was trained on 10k non-zero scalar input sequences of length 20 timesteps each. Results are plotted for each combination of delay and sparsity level (k: p) E (2. 4) x {0.7, 0.9, 0.999).
1.5.7 FIGURE 10
For each combination of delay and sparsity level, (k: 1k.1-p) E (2, {0, 0.7, 0.9, 0.97, 0.99), 100 models with 2-dimensional W,, linear recurrence and ReLU readout were trained on 10k input sequences, each of length 25 timesteps. Each plot shows the vectors of the best-performing model (as measured by lowest EMA training loss). The values and ko shown under the plots are calculated from the final W,, given its eigenvalues \1 A2 E C as follows:
P max (1 P22l)
k0 = k (A)
I.5.8 FIGURE
The process was the same as for Figure 10 but with a nonlinear RNN trained instead of an SSM. Due to the nonlinear recurrence, P and k0 were not meaningful values to compute and so omitted from the figure.
1.5.9 FIGURE 12
We train 100 nonlinear RNNs with hidden sizes of N₂, = 2,5, 10 on the 2-delay task with vector inputs (N₂ 10). For the best performing model of each hidden size, plot the values of W W.
for 4,
1.5.10 FIGURE 13
We train 1000 nonlinear RNNs with hidden size N₁ 100 on a 2-delay task with 75 input features. In the plot, for each model, we compute the mean the output projection onto the readouts (mean(diag(W,, Ws=k)) and the mean of the non-output projection onto the readouts (mean(W, (W s.))). We additionally color each model by the training loss achieved (purple being lowest, yellow being highest).
1.5.11 FIGURES
For each combination of number of features and sparsity level, (N₂ 1,1 e (1,2, x (0, 0.7, 0.9, 0.97, 0.99), 500 models of each architecture (linear, SSM, nonlinear) were trained on 10k input sequences, each of length 20 timesteps. Each model is tasked with reproducing the input sequence after & timesteps, but now One dimension of the input corresponds to the cue, which remains O until the randomly selected k. after which the input from t - k, Vt > k. We keep the input and output dimensions of the network the same, such that we include an extra output dimension that weighted with importance in the loss. We
random for each training sample (k Uniform(O, 10)).
and the RNN is tasked with outputting the sequence, corresponding to
mask the network output in the loss until after the cue is turned on, such that earlier outputs do not contribute to the loss.
Fach plot shows the trajectory of feature directions for each respective input feature through time (purple being new, yellow being old), indicated by a unique marker shape. Each plot visualizes the best-performing model.
42


Reviews:
[{"review_id": "gRJLamZpzn", "reviewer": "Reviewer_Jqdv", "summary": "The manuscript addresses the question of how artificial recurrent networks can store information, specifically in settings when the dimensionality of the state space is smaller than the total number of features that needs to be represented. Specifically, the authors investigate a phenomenon similar to that of spatial superposition, but in the temporal domain. They find that, when inputs are temporally sparse, even small recurrent neural networks can represent features over a comparatively long time by packing them into a subspace of the dynamics that is orthogonal to the network output, and by forgetting inputs once they become irrelevant. The manuscript focuses on a rather simple task in very simple RNN (linear and non-linear, low-d) to achieve a thorough understanding of the underlying mechanisms, combining analytical insights and simulations.", "questions": "The authors should discuss or explore the implication of their findings for other tasks, specifically also tasks that involve variable, randomized delays, whereby RNN outputs are triggered by the onset or offset of a dedicated go cue. \n\nThe authors should relate their work to past studies with RNN that have found dynamics that seem closely related to what is shown in this manuscript.\n\nI found the description of the phase transition hard to follow, only few results are shown to validate the point. The corresponding section is less clear and weaker than the remainder of the paper.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper is well motivated and well written. Relevant literature and the implications of the work are discussed appropriately. The paper is technically strong.\n\nThe focus on simple, small RNN, and on a single simple task, allows the authors to achieve a deep understanding of the underlying mechanisms. In particular, the authors combine simulations with insights from analytical derivations, which together make a very convincing case for their findings and conclusions.\n\nThe paper makes an interesting conceptual advance towards understanding the basic principles by which recurrent neural networks operate, by identifying a phenomenon akin to spatial summation in the temporal domain.", "weaknesses": "The focus of the paper on very simple, small RNNs and a single, comparatively simple task allows the authors to achieve a deep understanding of the underlying mechanisms, but also raises some questions about the broader relevance of the resulting insights. The main weakness may be the exclusive focus on the k-delay task. Many, if not all, findings about temporal superposition presented in the paper seem to be a direct consequence of using this task, which requires RNNs to remember an input feature for a fixed number of time-steps, then output it, and finally forget it. The authors make a strong case that that the type of dynamics they observe, which involves their mechanisms of temporal superposition, is “optimal” for this task. In fact, the mechanism they describe “makes a lot of sense” given what we know about RNN trained on simple tasks and is arguably not entirely surprising. It is less clear what are the implications of their findings to other tasks, which may require RNN to maintain information for a variable amount of time (variable k, whereby e.g. the RNN output is interrogated by providing a dedicated “go” signal) or to do more than just remember an input feature (e.g. by producing outputs that could control simple movements). \n\nIn particular, I would expect that for a variable k (i.e. a randomized time between input and output) the type of solutions implemented by the RNNs are very different. In simple neuroscience decision-making tasks, RNN trained with fixed delays are known to produce rotational dynamics that is timed just right to put their activity into the right location of state space when say an output is needed, which is exactly what the authors find. On the same tasks, training with random delays instead leads to stable representations during the delay period (based e.g. on stable fixed points). In such a setting, temporal summation does not seem to be relevant, and instead of spatial summation is at play.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xgAFrXFm2G", "reviewer": "Reviewer_DPAE", "summary": "Authors consider a set of linear and nonlinear RNNs trained to perform a K-back task and study the properties of learned solutions, primarily for RNNs with 2 units. For the linear case, authors are able to write down the loss function in terms of four distinct components and name the loss contributions in terms of what they call projection and composition interference. Essentially, this paper has the flavor of applying interpretability methods used in LLM world to the studies of computation performed by RNNs.", "questions": "I only have few questions/comments:\n\n- Eq. (1) is actually the more common RNN architecture in computational neuroscience. Hence, Appendix F.1. is making incorrect claims. Also, the connection of Eq. (1) to other form of vanilla RNN is well known. See [1]. \n\n- Please emphasize in the abstract that your main contributions hold exactly for linear RNNs, and then you show that some of the insights do generalize to nonlinear counterparts. However, generalization beyond the particular task is not shown, please state this explicitly as well.\n\n- What makes feature A different than others in Figure 4? \n\n- Figure 5 is very interesting, why is it in Appendix? I would argue it may be the most interesting figure in this paper.\n\n- Are you aware of [2]? If so, how does your work, especially Fig. 5, compare to their findings? In my reading, it seems you also find that geometric restructuring (as Haputhanthri et al. 2024 has defined it) can happen even without emergence of attractors, as your case simply uses a spiral. That seems like a noteworthy connection/extension of prior work as well.\n\nMy overall assessment is as follows: This work is limited in ambition and has some issues with rigor, both of which limit its contributions. However, it does introduce an interesting toy model and compared to how far we have come with RNN analyses, it does show how rudimentary the superposition/interpretability ideas in LLMs are. We need to do a lot better. With correct placement into the RNN literature, in which we can now study high-dimensional RNNs and their learned algorithms exactly (see recent low-rank RNN studies), I think this manuscript can be a fun read for the ICLR audience. \n\n[1] https://pubmed.ncbi.nlm.nih.gov/22023194/\n[2] https://openreview.net/forum?id=njmXdqzHJq", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The analytical derivation of the loss components is novel, and insightful for training and reverse-engineering RNNs to perform the K-back task. \n- The paper is packed with a lot of fun to read results/observations, e.g., the phase transition, the generalization to multi-input case and how all but one inputs get dropped out due to constrained representational capacity. \n- To me, this paper was instrumental in realizing how far ahead we need to go to in our LLM interpretability studies compared to what the field has achieved for RNNs. So, it is also a nice toy model and has pedagogical implications.", "weaknesses": "- As authors admit themselves, the work focuses primarily on a single task and a very small RNN. There is not necessarily a generalizable insight/evidence that I was able to take away, and that does diminish the contribution.\n- I find that the authors are less direct with how far RNN studies have come (specifically, the low-rank RNN studies). In many cases, we can now study the low-dimensional subspaces learned by RNNs and draw the exact flow maps they learn (not just the feature dimensions). In that sense, more modern discussion of low-rank RNNs is desirable.\n- It is unclear how many RNNs have developed these representations (especially for nonlinear ones) and how many failed to learn or learned completely different solutions. More rigor in reporting is desirable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GJ1rzBCQdR", "reviewer": "Reviewer_iPfa", "summary": "This paper investigates the representation geometry of RNNs, focusing on how time affects the so-called superposition hypothesis. \n\nThe authors introduce the concept of temporal superposition, analyze the resulting interferences and their interaction with spatial superposition, and derive analytical results on a simple task.", "questions": "The qualitative results shown in Figures 2 and 3 are a bit unclear: are these plots averaged over 100 runs? or are they representative examples? Could you include error bars to assess variability.\n\nDoes the analysis of spatial or temporal superposition provide any insight into how we should train or test our networks? Beyond being conceptually interesting, are there practical consequences of this phenomenon of temporal superposition?\n\nHave there been any empirical observations of temporal superposition in animal data?\n\nMinor: Figure 3 is not cited in the main text.\n\nI’m a bit unclear on why feature A competes with itself through time, since, in effect, the latent space at time t is different from the latent space at time t+1. The network doesn’t hold At and At+1 simultaneously, only ht worries about At and ht+1 worries about At+1. I’m viewing the RNN as unrolled.\n\nPut another way, could temporal superposition also occur in a feedforward neural network, with the layer index playing the role of time? If not, why not?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "Very well written, organized and easy to follow. The figures helps.\n\nThe initial analysis of a simpler model help intuition.\n\nGreat related works, very comprehensive.\n\nImportant generalization of an hypothesis that has garnered a lot of attention in the community. Significant.", "weaknesses": "Technically, the contribution could feel somewhat incremental compared to prior work on spatial superposition. However, the significance is still there.\n\nMany key results are presented only in the Appendix, which reduces the overall impact of the paper. This makes me wonder whether the work might be better suited for a journal that allows a longer format. For instance, Figure 9 is particularly helpful for understanding the core concept but is unfortunately buried in the Appendix. \n\nThe analysis with multiple dimensions should also be moved to the main text and discussed in more depth.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pYTFuT2EUq", "reviewer": "Reviewer_pihS", "summary": "In this paper, the authors introduce a novel concept of temporal superposition in RNNs, the idea that beyond spatial superposition of features, memory demand also poses a pressure for the network to pack features presented at different time steps into limited hidden dimensions. \n\nThe authors analyze RNNs on a k-delay task and identify two interference modes: projection interference and composition interference. They analytically decomposed the loss into four interpretable terms,  including the task benefits and the two interference modes, which can be used to explain the learned geometry of feature directions in the hidden state. The authors also identified a phase transition in RNN's representational geometry as a function of the sparsity of features. In their analysis of nonlinear RNNs, the authors provide a geometric perspective on how nonlinear RNNs can implement sharp forgetting while linear RNNs can only implement gradual forgetting. \n\nFinally, the authors also study the interaction between spatial and temporal superposition as a function of memory demand, and discovered that as memory demand increases, the network goes from representing all features equally to representing only the most important feature for all time steps.", "questions": "- The k-delay task effectively isolates memory demands, but as noted in Limitations, it focuses on reproduction rather than manipulation. What preliminary insights do you have on how temporal superposition manifests in tasks with variable delays or manipulation of information?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- This paper presents very interesting results regarding temporal superposition of features in RNNs under memory demand, which I believe can be of broad relevance for the mechanistic interpretability and computational neuroscience community. \n- The concept of temporal superposition is novel, and the authors use well-designed experiments to illustrate this effect on three types of RNNs while contrasting the role of nonlinearity in feature representation. \n- The authors provide clear, interpretable theory on RNN geometric strategy by decomposing the loss into four terms that correspond to the task performance, correction effect, and two types of interference. \n- The paper is exceptionally clear: key concepts are precisely defined, notation is consistent, and figures concretely illustrate the geometric claims, making it easy to follow.", "weaknesses": "- The core analysis revolves around the simple k-delay task. While this isolates temporal superposition effectively, it may not capture real-world memory demands involving manipulation, variable delays, or context-dependent processing.\n- Most results are in low-dimensional hidden states (Nh=2), with higher dimensions (up to Nh=10) only briefly explored in the appendix. Scaling to realistic model sizes could reveal different behaviors but those were not considered or discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)